As call letters are being sent out by various IIMs to the lucky few who will enter the portals of the prestigious B-schools, the questions raised over the conduct of the computer-based test by students continue to be asked unabated.
Using online petitions, RTI petitions and even public interest litigation, students are trying to force a re-look into the test even as Prometric, the testing agency, says that the test was fair and achieved its aim of selecting the best students for the B-schools.
Vishnu has been scoring in the 99th percentile in the Common Aptitude Test (CAT) for the last few years but has failed to make it all these years. When the results for CAT 2009 were announced, Vishnu says he was shocked when he scored in the 72nd percentile in verbal though his overall percentile score was high. “Each year I have been scoring around the 90th percentile in the verbal section,” he says.
He has filed an RTI petition asking for details. Vishnu, Sowmya and others recently met up in Chennai to discuss what they called the non-transparency of the testing process.
“After all the glitches in the conducting of the CAT in November-December, we thought our problems would be resolved. But we cannot believe our scores are so low. The least that Prometric should do is explain the complete scoring system,” they say.
Prometric detailed the scoring process recently. A set of tests — slots — of roughly equal difficulty were prepared for the students. The raw score calculated based on their performance in the test was “equated” by using statistical methods to make the evaluation fair across slots and eliminate the possibility of one slot being “easier” than another.
These equated scores were then “scaled” using a “standard procedure” used in computer-based tests worldwide, and the scaled scores were provided for each of the three sections and for the overall score, from which the respective percentiles were provided.
The aggrieved students, however, point out a number of problems with the system. Vishnu says the system would be fair only if a slotwise percentile is calculated. “I must be judged based on how I performed with respect to the others who took the same test,” he says.
He says the absence of “watertight” sections makes the problem of “equating” difficult. “In GRE or similar tests, candidates are given a specific time for each section they attempt. But in CAT, we will allot time based on our perception of the difficulty of the paper. This makes it unfair if the authorities use statistical methods to assess difficulty of each particular question,” he says.
The students say they have been filing RTI petitions with the IIMs asking for more details about the system and also about their individual performances.
Soumitra Roy, managing director, Prometric India, says he welcomes the demand for transparency. “We have been providing all the details that the IIMs have been asking us because of the RTI petitions they receive. But until now we haven't had to revise a single score,” he claims.
On the request of the IIMs following a number of complaints specifically about the verbal section, Prometric India went through the verbal section details of each candidate to ensure there were no issues after a number of students complained of anomalies, he says.
“In 2008, there were 1,065 candidates who scored in the 95th percentile or above in Quant and DI but were in the 70th percentile or lower in the verbal section. In 2009, this number had reduced to 775, which shows that fewer students have the most common cause for complaint this year,” he says.
S. Balsubramaniam, director, T.I.M.E., Chennai, says there may be various reasons for adopting the system, but it would have been easier on students if the IIMs had allowed the option of a paper-based test for students after the initial glitches.
But Mr. Roy says the problem is that after 31 years of a particular testing mechanism, a “game-changing” pattern had been introduced causing confusion among students. The system will only be perfected over time, he adds, stating that Prometric India was willing to talk to all stakeholders to address their issues.
The students, though, are worried because they might be the individuals to suffer. “We will look at filing a PIL to restrain the IIMs from continuing with their admissions process. Otherwise we will lose a whole year,” they say.
