Table of Contents
Last year all spheres of our lives have changed once and for all. Many industries, businesses, and other organizations were forced to convert their workflows to online. Some of them succeeded, and some unfortunately not. These changes also touched the educational process of all educational facilities. While some industries had no problems with turning their learning processes digital, the healthcare industry continues to struggle with lots of limitations and difficulties.
Commonly, healthcare organizations of different types provide a learning plan for all their workers to constantly improve and build on their clinical skills. The pandemic made offline staff education unattractive, so medical facilities started to look for digital solutions that would meet all their demands.
The main thing that separates medical education from all other kinds of education is the level of responsibility that the medical profession carries. Hence, the development of professionals has to be treated extra carefully and extra diligence has to be applied when honing specialists’ clinical skills. Namely, their development process should be accompanied by a structured and thorough approach whereby progress is properly checked and assessed to define whether the student passed the course and increase the skill level. That is why healthcare organizations felt the need to implement clinical online assessment solutions that would give them the same level of assessments as they had available in pre-pandemic in-person settings.
In this article, we are going to discuss why clinical skills online assessments in medical education are so indispensable, now more than ever. And how to make hosting them online a reality, navigate all uncertainties and complications, as well as take into account all educational standards and organizational requirements.
Assessment of Clinical Skills (CSE)
Summative assessments are summary exams that are typically held at the end of medical courses and aim to check and evaluate the students` acquired knowledge. As mentioned and as you very well may know, medical education is slightly different from other types of educations, standing out in terms of its complexity, higher responsibility, and many would argue – value to society. Summative assessments are also meant to reflect on the effectiveness of their respective courses, find weaknesses in teaching strategies, and improve them to provide more qualified and impactful medical education.
Clinical e-assessments are divided into two sub-types – formative and summative.
Let’s get into detail about what each of these sub-types entails.
Formative assessments are progress and diagnostic tests that are held throughout the duration of a given learning course. They are meant to demonstrate students’ clinical skills and what they really know or can do, revealing their strengths as well as their weaknesses, while also exposing the shortcomings of the teaching material or style, and help pinpoint what the plan to improve the learning experience should include or focus on. This type of assessment influences and helps adjust the teaching curriculum. To highlight the key aims of the formative assessment, they are:
- motivate the students to learn as the whole learning path and their improvement and growth depend on and are up to the students, their work ethic and talents;
- provide feedback to the students about their progress to keep them even more engaged in the learning process;
- edit the teaching curriculum to ensure the provision of medical education of the highest quality.
The summative assessment is provided at the end of the module or the whole learning course. It considers all test results that a student was getting throughout the learning path, consolidates knowledge, and determines what grade this student will get.
Summative and formative assessments are correlated as if the summative assessment results are unsatisfactory then the formative assessment content is weak and needs to be upgraded.
Find out more interesting facts about formative and summative assessments from the video added below.
Best practices for summative assessments
The assessment results are incredibly important as generally, the summative outcomes define the student skill levels and take responsibility for their evidence. It has a huge impact on the further development of professionalism and future success as a medical worker. The main goal of clinical summative assessments is to show the efficiency of the educational process, tests and to check the level of knowledge students get when taking this course.
Skill summative assessment has to offer high-quality tests to provide high-class learning. We would like to discuss the most effective practices of structuring summative assessments to evaluate the final outcomes after the learning course.
Criteria for Excellent Assessment
Firstly, it is worth noting what criteria define that a particular summative assessment is efficient and shows the real result of flipped learning. These features were highlighted by Cees van der Vleuten, Professor of Education and chair of the Department of Educational Development and Research in the Faculty of Health, Medicine, and Life Sciences.
The excellent summative assessment should include the following features:
- reliability – means that the assessment should evaluate exactly what it aims to evaluate;
- feasibility and cost-effectiveness – means that if the summative test corresponds to the real situations specifically for local areas;
- educational impact – define if the assessment motivates students to give 100% to their learning process;
- equivalence – indicates that the tests of one organization will be valid for and are applicable to other medical organizations;
- catalytic – means that the assessment motivates students to continue learning and improving their clinical skills.
Learning Outcomes and Assessment Planning
Clinical skills include not only medical knowledge. They cover a bigger range of spheres that a student should be informed about. Along with qualified patient care, a learning course and therefore summative assessment should teach and check the ability to think critically, interact with other medical staff, meaning team skills, professionalism, and competence. All these features help to plan and form the upcoming assessment that will let tutors get the expected outcomes. Thus, the summative exam must contain not only simple medical tests but practical lessons and exams, peer assessments, presentations, portfolios, etc.
Meaningful Test Scores Explanation
Validity of test scores implies suitability of test evaluation to different groups of students as each person is unique and everyone learns differently. It is critical to creating a test score that would evaluate the knowledge correctly corresponding to student specifics. It also implies covering the results into test scores considering the student’s peculiarities. Among the criteria that should be assessed are:
- test content;
- response process;
- internal test structure;
- relations to other variables;
- assessment consequences.
Conformity of Learning and Test Content
The test content is extremely important as it has to correspond to the general assessment standards and be appropriate to all students. The content should be accurately related to the learning course and bring the expected outcomes. The summative assessment usually consists of the following parts:
- assessing cognitive skills like analyzing, remembering, understanding, applying information, and others;
- competency in medical (physician) knowledge like a physical exam, psychology, planning, and management, etc.
Double-check the Test Results
The validity of the response process is all about double-checking the final score of each student. There are so many cases when a student gets assigned the wrong scores in the spreadsheets because of human or system errors. It is vital to provide official instruction on how to work the process of entering the final grades before announcing them to students. It is important to ensure that the scores from learning management systems and manually entered scores correspond to reality. The scoring process involves grading online medical exams across various topics from different parts of the syllabus, and each of these subjects should be evaluated appropriately, as well as all should be brought together to demonstrate the big picture and bottom line.
Reliable Test Structure
The validity based on internal structure implies equal conditions for all students, meaning that each student gets the suitable approach of testing and scoring. The best way to check if the testing process is the same for all students is to provide a functional test. The test reliability is defined by the following factors:
- enough number of test items;
- high-quality discriminating items.
Relationships to Other Variables
The summative assessment outcomes show the student’s readiness to move further and continue the learning course. To provide the qualifying examinations, there is a practice of comparing the internal clinical skills tests and external standards tests from other medical organizations. Then the test outcomes should be compared to create the perfect formula for test content.
Specific Grading System
The grading system for medical students should likely be different than assigning letters like A or C. It would be better to adhere to a passfail grading system with a detailed description of the errors and how they can be fixed further. When a medical student gets a low grade with no explanations, the motivation for qualified education decreases.
Setting Standards
To monitor the progress of each student, it is advisable to provide a single database with all examination items like test questions, their results, grades, etc. This database allows tutors to analyze and compare the new and old items, and track the student`s performance. The availability of looking through the tests of the past lead to testing improvements and therefore more qualified knowledge evaluation. Students also can get annotation to their test results and discuss the fails with the tutor if needed.
The database lets monitoring the examination items efficacy and aims to change the syllabus if improvements are required. For example, it is not logical to put practice exams at the end of the learning course as it is important to consolidate knowledge directly after a certain part of the course.
Other Aspects of Assessment to Consider
As we have already mentioned above, the most critical thing about summative assessment is equality for all students. It means that each student should have the access to all learning items and information, practical data, etc. Additionally, it is important to hold the examination in equal conditions, meaning no student should feel stress or pressure from the examiners as it can influence the exam outcomes. It is also worth considering that usually, the clinical exams consist of several parts, especially practical assessments. Well, it is vital not to judge the student by one of the parts as the one failure does not determine the general outcomes.
We also would like to offer you to learn more detailed information about the best practices of holding medical formative and summative assessments from the report of professor of College of Medicine and University of Central Florida, Jonathan D. Kibble.
The impact of COVID-19
The pandemic has changed the educational process once and for all in almost all industries and healthcare is not an exception. Before the quarantine, there was no need for implementing software systems to apply a highly qualified learning process. However, this became impossible to visit classes, work in groups and hold the lectures offline. So most medical organizations who wanted to continue staff education started to look for effective solutions to the sudden problems.
The healthcare industry was also forced to go online, and we are talking not only about staff education. Many clinics started to adopt telemedicine that we will discuss further in this article.
Thus, the educational process and assessments have to turn online using custom healthcare app development. Of course, there are some readymade systems, like TeleOSCE, that could be applied for holding the learning and examination process. However, all of them have basic functionality and can not be customized according to certain clinics’ requirements. So most medical organizations faced the need of developing and integrating custom software to provide online education of the same quality as it was offline.
Online clinical skills learning proved not only efficient but also more profitable:
- each student can check the personal progress, look through the materials once again, and get extensive feedback about their clinical skills;
- as all student data is recorded in a database, tutors have access to all grades and outcomes to monitor the students learning progress;
- modern technologies provide an improved version of the practice skills.
Telemedicine
The development and adoption of telemedicine were caused by the rapid spread of Covid-19 all over the world. The clinics and hospitals were overcrowded by infected patients, so they were forced to eliminate the need for in-person visits inside medical establishments.
So what exactly does telemedicine or telehealth provide?
It allows to treat or assist people with all kinds of different diagnoses remotely, meaning patients visit hospitals only in cases of emergency, or clinics or laboratories if some kind of testing needs to be done. But clinics also create virtual care programs that are suitable for remote treatment and health monitoring. Telemedicine is possible only for those facilities where internal software systems like EHR and EMR are implemented. Each patient has personal health records that are accessible in real-time for doctors as well as for this patient. These records usually contain medical history, medication lists with detailed descriptions of what pills, in what dosages, and when the patient should take them. Doctors and patients also can add the latest information about the patients’ state of health or update the existing data. Moreover, patients get reminders about taking medications and can urgently alert doctors about the deterioration of their conditions. For example, patients can use the BrainCheck app to track the brain health and send this data to clinics.
Learn more about the benefits and future development of telemedicine from the video below.
Additionally, some hospitals supply patients with wearable devices that record and track vital signs such as the heart rate, pulse, blood pressure, blood sugar (i.e. glucose monitoring system), steps for activity, etc., which automatically transmits real-time information to the hospital system, which providers can then seamlessly analyze. Wearables provide improved remote monitoring and are effective at preventing crises.
TeleOSCE
TeleOSCE is considered to be one of the most effective readymade software systems that allow remote learning and knowledge gauging for medical students. The pandemic forced OSCE or Objective Structured Clinical Examination to turn their clinical skills assessments online and consequently adopt telemedicine to maintain the same high quality of the learning process and examination of the same high quality. Its task is to check the student’s clinical practice to determine whether it is worth it for a particular student to continue the educational process.
TeleOSCE offers simulations of real medical cases to teach students how to act in a given situation. After the learning lesson, students are to pass the examination that aims to prove and confirm their clinical skills.
Generally, passing this simulation implies that a student can:
- provide patient treatment via telemedicine;
- use CDS (Clinical Decision Support) systems to treat patients appropriately;
- determine the diagnosis and prescribe a suitable cure or treatment course;
- identify mental illnesses and determine the treatment plan;
- improved communication skills that are required to effectively communicate with patients.
Although this solution cannot be applied by and isn’t appropriate for all medical organizations that provide clinical staff education, it is and can be used widely. Along with its benefits, it can bring inconvenience like TeleOSCE, which is developed specifically for Oregon Health & Science University, taking into consideration their particular requirements, so the software cannot be modified or customized to reflect the features that your particular entity wants or needs.
Designing effective online assessment
As we have already figured out, all facilities that conduct education for medical workers were tasked with converting the learning process, including basic lectures, practice lessons, quizzes, and assessments completely to online. This process aimed to eliminate the need for in-person attendance.
If the classes could be held via video meeting apps like Zoom and Microsoft Teams for communication with tutors, sharing the learning information, providing some tests, showing presentations, videos, etc.
Talking about e-assessment, they could not simply be held online via video meetings. Medical education demands the highest quality and level of skills, hence everything needs to be held to the highest standard, including rigorous and accurate examination of participants. It should be a specifically implemented cloud-based software system that offers the option to hold the online formative and summative assessment of the expected level of quality, fairness, and credibility.
We would like to list the features that such a solution should provide:
- auto-scoring the different types of questions so there is no problem with making and accurately grading short, as well as elaborate questions;
- auto-mixing and pulling the questions from the database pool for each new examination;
- intuitive exam canvasses ( interface) that allow students to resize the screen during the exam;
- remote monitoring of the open-book examination process by supervisors (available only for cloud-based systems);
- eliminates the ability to abandon the test after already starting it, making it reliable and fair for all students, meaning when students start the examination, the only way to exit is to submit the test;
- ability to lock other programs and files on a student’s laptop for the time of the assessments, mostly required when tutors cannot be present during exams;
- cheating in paper written tests can take place when your organization has only started adopting the system;
- custom reports based on test analytics to show students their results, feedback, and studying effectiveness in general;
- real-time notification alert updates if the tests are checked manually and the scores are uploaded into the system or portal;
- delivery of test results via an email notification or integration with the existing digital tool with students` records.
It is worth noting that readymade solutions were not enough to cover all processes of the learning process.
E-assessment standards
Turning summative assessments online means providing the same level of learning and test quality. Just like any other online platform, e-assessments should enable data exchange between different educational systems, meaning students can use several different platforms to learn like some of them are for holding the lectures, some are for communication with professors and other students, and some are for summative tests. All of them have to be connected to each other in order to provide quick data exchange.
However, this data exchange is an important process that includes the range of standards as medical education a little bit differs from other professions. These standards should be considered before the actual planning, development, and implementation of the e-assessment systems to make them interoperable and accessible for all users.
And now we would like to highlight the main standards that should be taken into account when developing and integrating an educational digital solution.
Correlation of online assessment parameters with summative exam performance
The greatest concern about online education is that it does not provide the same quality and efficacy of medical education as in-person learning. The pandemic proved that any industry and sphere of our life can turn online, didn’t it? The key point of holding the online clinical skills learning and assessments is to provide efficient instructions for students as well as for tutors. As we have already mentioned above, e-assessment platforms should be developed and implemented considering the key standards and best practices of holding the assessments.
Online assessment platforms are tasked to improve the correlation of students and tutors, provide more expanded feedback about the conducted formative or summative assessments that help students get through the courses and graduate more successfully, possessing the highest levels of clinical skills. Students can also monitor their progress through an e-assessment platform that allows them to evaluate their results and keep motivated to learn better and further.
Development of the custom e-assessment system
Before the actual development process, it is important to research the existing solutions that provide the options of online clinical skills summative assessments. This step allows the Business Analyst to highlight the main functions this system has to perform and select the most appropriate one for your solution.
Additionally, it is no less important to analyze your medical organization, existing electronic systems that could be connected to the online assessment systems, and ways of staff learning that were conducted in offline versions.
At Altamira we normally call this the discovery stage, which is a must for any custom software development. This stage lets our Business Analysts reveal the objectives that should be reached with the custom software, the weak side of the previous solution that can be improved, and opportunities to update the staff educational process to make it more productive.
Further, our specialists, particularly Business Analysts, offer the accurate plan of the following actions that bring the implementation of custom online assessment solutions into your organization closer. Consequently, all these tasks are distributed among the developers by our project manager with particular deadlines and details.
Here at Altamira, we guarantee:
- keeping you informed about the latest changes in your project;
- constant communication with the development team, including video meetings;
- qualified project delivery considering all your requirements;
- maintenance services that keep your solution up to date with the latest technological trends.
Talking about the cost of this type of software solution, it all depends on the organizational goals, objectives, and demands that this solution should address. Initially, we recommend that you release an MVP to test it for your organization, reveal the weak side, and hopefully turn them into strengths.
FAQ
To conclude
The competition for online assessment solutions is kind of low and it is quite hard to find a readymade, reliable solution that is also suitable for meeting existing needs. The online assessment aims to deliver the same high quality of the learning process, the same expected outcomes, and clinical skill levels. Its popularity is only now starting to pick up and grow as many medical organizations have largely been skeptical about its effectiveness. However, the custom clinical skills e-assessment solution is something different as it is developed specifically according to your entity’s requirements and delivers the educational process of the highest level.