New technology enhancing how EDAIC exam is developed, reviewed, delivered, and post-processed

New technology enhancing how EDAIC exam is developed, reviewed, delivered, and post-processed

  • Issue 73

Stefan Park
stefan.park@orzone.com

This year new digital technology was successfully introduced in how the EDAIC part I was delivered. This article discusses these changes, its history, and impact for the future. It will also outline some of the benefits achieved in recent years in how the ESA exams are developed and delivered.

Computers instead of paper in 2017

This year 4 centres running the EDAIC exam used only computers to deliver the exam, instead of paper. Some of the benefits of using computers for the exam are:

  • The candidate can choose language at time of exam,
  • No need to ship courier papers to over 100 places and less chance of leakage,
  • Questions and answers are presented in randomized order, which make it pointless to look at the person sitting next to you,
  • Lower cost and fewer errors,
  • Results are available for the exam committee immediately after the exam instead of weeks later(allowing the candidate to receive the results earlier),
  • This was the same technology that has been successfully used since 2011 for the OLA exam.

Behind the scenes of creating the exam

Although the biggest visible change this year was that part of the exam was delivered on computers instead of paper, the biggest change in recent years is actually in how the exams are being developed.

What may be less known is that there are almost 100 ESA staff and medical contributors working together with digital tools to write and review questions, accurately translate them to 15 languages, and to post-process the test with statistics and reports to candidates. The actual delivery using digital methods instead of paper is thus just the tip of the iceberg and part of a conscious journey that started almost a decade ago.

Short overview of the history of digital exam technologies in European context

About 10 years ago ESA was delivering the largest European medical specialist exam, EDAIC, with about 1,000 candidates. At the same time four seemingly unrelated activities were ongoing. UEMS/CESMA was founded to enhance European specialist exams. Its president in the first 10 years was Dr Zeev Goldik. UEMS had also launched an initiative to look at digital exams and portfolios and selected Orzone (Sweden, www.orzone.com) as its supplier. ESA was internally working to get the questions into a safe digital database. There were also efforts within ESA to translate the exams to gain wider acceptance.

ESA became the lead pilot location within Europe in 2009. In 2011 the first digital exam, OLA, was held for a little over 100 Anaesthesiology candidates in 15 different cities in Europe and Asia.

Development in recent years

Based on the initial pilot it was decided to:

  • Create a safe database for how to write and review questions in multiple languages.
  • Produce both paper exams and computer exams.
  • Automate many repetitive and time consuming tasks for what happens after an exam (e.g., generating statistics and reports).
  • Grow the OLA as a formative exam similar to EDAIC. The growth should be in geography and number of users and languages, and the exam should be low cost.

Since 2013 all written ESA exams are being developed in an integrated process based on the above. Data is stored in real-time 24/7.

Result and some short facts

  • In 2017 almost 4,000 candidates sat the different written exams developed by ESA (EDAIC, OLA, etc.); a growth of 300% in the last decade.
  • Exams are now being developed in 15 languages, on 5 continents, including Chinese characters.

Strengths and leadership of ESA

The anaesthesiologists were among the first to have specialist exams in Europe, in 1984. It has been a tremendous journey to get to today when thousands of anaesthesiologists take part every year. ESA was first to have exams recognized at national levels and by legislators. ESA has continued to lead the path in many different ways.

The exam creation and delivery is a group effort by close to 100 ESA administrators and medical professionals. With that said, there a few factors and people who deserve credit. Dr Zeev Goldik and Dr Sue Hill have for a long period of time worked tirelessly to innovate and lead. Dr Goldik has also presided over CESMA and had a key role in the establishment of European exams in general. They are now handing over the torch to Dr Andrey Varvinsky as Chair of the Examination Committee, Armen Varosyan as Chair of OLA, Wolfram Engelhard as Chair of EDAIC part I, and Dr Mario Zerafa as Chair of EDAIC part II, who will together with ESA Examination Subcommittees, examination centres, examiners, and ESA Exams Department staff and coordinators, continue to lead the way to the future. Credit should also be given to the exam team at ESA HQ with Hugues Scipioni as its team leader as well as Rodolphe Di Loreto, Odile Jacquet, Marc Gheeraert (Executive Director), and many others leading the way. Without a good working culture and a good collaboration between the staff and medical professionals the success would not have been achieved. Bias: Orzone is the supplier of tools for the ESA exams.