Dr. Jane Dacre, Consultant Physician and Rheumatologist, Professor of Medical Education and Director of the UCL Medical School, London, was in Chennai recently in connection with the examination of candidates seeking MRCP certification. In an interview to Education Plus that was moderated by Dr. Georgi Abraham, Professor of Medicine, Pondicherry Institute of Medical Sciences and Governing Council Member - International Society of Nephrology, she highlighted some aspects of medical education that need reform in India. Excerpts.
Q: What are the issues in medical education today, with particular reference to evaluation of students through examinations ?
A. Some of the issues are actually about using the best available evidence in medical education and assessment practise. There is a body of evidence about medical education and exams now - the scientific underpinning of medical education has become much more professional, and we now have statistical data that show how good or bad an individual exam question is, and how good or bad a whole exam is. We have taken this into account in the design of the MRCP examination.
There is strong evidence, for example that the viva examination is unreliable. The evidence shows that if the examiner likes the look of a candidate when they come in for a viva, they think that candidate is marvelous, however well he answers the questions, and if the examiner doesn’t like the candidate, whatever the quality of his answer, the examiner is less impressed.
The way to make things fairer would be to make viva questions more standardised, and to sample a bigger proportion of the candidate's knowledge. Also, we no longer use vivas in the MRCP clinical examination (PACES).
Q: The practice of medicine is slightly different even though we follow the British system in India. How will that influence the new system of examination, of teaching and impact of that on patient care?
A: The adoption of best policy in medical education has allowed us to provide an overall structure for the examination, and the specific medical content has to be provided locally. The content of the questions, and (decisions on) what questions are appropriate for your community, are very different from ours. We were looking today at snakebites and rabies in India, while in London, we don't get any snake bites or rabies. But the principles of asking candidates to describe what they would do in a situation, getting them to demonstrate it in a simulated situation, are common to both health systems. The scenarios we use in the examination can be very locality specific.
Q: We do not use such scenarios in the medical examination system in India. You are directly asked, for example, what do you hear, what murmur do you hear in a patient? How big is the murmur? Some may say it is a short murmur, others may say it is a long murmur. It depends on the fancy of the examiner to decide what the murmur is and judge the candidate on that. What you said is totally different. You need to understand the student, what he knows, and ask him questions on what he knows, not on what he does not know in the common practice of medicine. Is that right?
A: The examiner wants the students to demonstrate their skills. Not just to tell the examiner what they know. Candidates may be very knowledgeable, and I think the trainees here do know an enormous amount, but they cannot always demonstrate that they are able to put their knowledge into practice. In our MRCP clinical exam, we observe the candidates putting their clinical examination skills into practice. Knowing how to work out what the causes of the particular condition could be, is not enough. Candidates need to be able to examine a patient correctly, identify the physical signs and diagnose the conditions. They also need know the most common causes, and be able to discuss what to do about it. This includes how the condition should be investigated and treated, and how the doctor should communicate the diagnosis to the patient.
Q: But that is not the system that exists here...
A: Yes. It is not what some of the local candidates expect, it resembles what they do in their clinical practice rather than what they expect in an examination.
Q: As an examiner, while assessing, you should look into the local practices, conditions, and diseases that exist in that community, the investigations relevant to that, and the treatments?
A: Yes. Although there is less leeway for a good examiner in the MRCP PACES examination than there currently is in the traditional clinical examinations that you have in India. The good examiner, however, should have a clear idea about local practices, about what is appropriate for candidates to know, and what isn't because that information has been given beforehand.
Q: On admissions to medical school, what are your views?
A: In the UK now, we have highly standardised and fair methods of assessment to go into medical school and post-graduate medical courses. The old-fashioned patronage cannot happen. If my son wanted to be a doctor and had missed the grade, I would not be able to get him into medical school. It is absolutely based on merit.
Q: Why do you need 10 examiners to assess a candidate, why can't you have only two or three or four at the most, as in our country?
A: Because, if you have an examiner who either knows a candidate or likes a candidate for a particular reason, then he is more likely to favour that candidate. But if you have a number of different assessors looking at the candidate using a more objective marking schedule, then you make it fairer and minimise bias.
Q: In our examination system we don't have a timeline. The examiner can ask questions as long as he wants and the student must sit there according to the fancy and direction of the examiner. You have established a system, where there is a time limit and you cannot go beyond that. Which is the better system?
A: There is an intrinsic unfairness in allowing any variability in the time allowed per candidate. Depending on what you think of the candidate, you may take a longer or shorter time to assess them. This means the candidate will have more or less time to demonstrate what they know. The PACES exam (the MRCP Part 2 clinical examination) structure is standardised so that it gives every candidate the same opportunity to demonstrate their knowledge and skills. Certainly in the UK, and in US and lots of places internationally, that is thought to be a more appropriate system. The downside of it is that if you have a really fantastic candidate who is a just a pleasure to be with, the enjoyment for the examiner is less. We had to remind examiners when we introduced the PACES that this exam is for the candidate, and for our patients, and the enjoyment of the examiner takes a back seat.
Q: What would you think about when you see different countries in the world, different candidates, cultures and practices? Is your system of assessment uniform? How does one avoid discrimination based on ethnic and cultural aspects?
A: It is a difficult question. We do have evidence that candidates overseas who take the examination in the
UK do less well in our assessment, but we do not know why. What we do know is that the MRCP assessment system is robust. What perhaps would be best in the long term is to introduce more locally specific conditions for candidates to be tested on. The MRCP is a UK-based exam, and therefore our rules are currently based on the UK Health system. It is astounding to me how well overseas graduates do considering that they are taking an exam that wasn't designed for their country.
At the undergraduate level, I think it would be appropriate to introduce the examination rigour that we have in the UK, but also to make the examination locally specific to your own candidates, An exam needs to reflect the curriculum which has been taught. I assume you have your curriculum written down and therefore, you need to turn your curriculum into a blueprint which is a sampling grid of the areas where you expect your candidates to perform in. The examination could be based on that sampling grid.
Q: Candidates taking the MRCP examination in India examine the local patients, and candidates in Britain may be seeing patients from Britain who may not be Indians or Asians for that matter. Do you think that will influence the assessment of candidates and in the long run will not be a rational way?
A: I think that in the long-term our objective would be to have more locally appropriate centres in India, the expense and the cultural issues such as the need to learn English or Scottish law to be able to pass an assessment in the UK should not be necessary for international candidates.
Q: Do you think in two years you would have to modify your MRCP examination to suit new technologies and medications?
A: We have written components to the examination. We spend a lot of time ensuring that UK national guidelines are incorporated, as medicine is a constantly changing field. So we modify the questions all the time to make sure we are up to date.
