Last wishes before putting the IIT JEE on the electric chair.

Table of Contents Hide 40% to board exam?Difference in standardsThe Weaker SectionOn BITS PilaniOn ChinaOn Multiple Choice Questions and Aptitude TestsStatistical data from the past It is not very often…

It is not very often that one gets to hear about the goings on inside IITs and the opinions of the key stakeholders AT (deliberately capitalized) IITs. While we have heard about the HR ministry’s decision to merge the engineering exams that happen in this country (i.e. iseet enttance), we haven’t heard how the IITs themselves feel about this change. Nebraska's_electric_chair

Today, thanks to our former PI contributor Arvind, we came across this incendiary minutes of Senate meeting at IIT Kanpur. It is a reflection on how shallow the HR ministry’s unilateral decision was. We also get to know how the IITs themselves didn’t have adequate representation in the decision making process.

The IITs are one of the few brands this country knows and the contribution to India’s growth story cannot be gainsaid. Before tampering with a system that works (and works well), the government should be extra cautious. This is not an experiment where failure is acceptable – especially when we know the system works well today. The JEE also has been THE exam to study for and not without reason. Almost every IITian remembers his AIR (or hawa) even if he passed out a few decades ago. As the Senate discussion shows, including a relatively subjective metric like the 12th class marks, also will dilute the quality of students going into IITs. BITS Pilani, which had the same as a metric, has shifted totally to an entrance based test – the reasons for this change should be enlightening reading for the ministry.

Below we reproduce some sections from the Senate committee meeting. The full document (link/pdf) is also a good read.

There are larger questions however to be answered  – and we would love to know your opinions.

a) The ministry is empowered – but is it capable to decide on changing the mode of exams?

b) Shouldn’t the key stakeholders have more representation?

c) As autonomous bodies – should the IITs say digitus impedimentus to the ministry and do their own tests anyways?

d) Are we building a nation of sissies who can’t take the pressure of 10th exams, too many entrance tests etc? Isn’t our culture of tough exams and huge competition the best way to separate the wheat from the chaff?

e) Are the IIT exams such a big deal anyways?

Some candid points the pdf mentions –

1. It was disappointing that the report that has suggested such sweeping changes to the admission process of IITs and has been accepted by IIT Council, without seeking feedback from individual institutes (IITs).

2. Deciding on the admission process for IITs without any discussion with the IITs is an infringement of their autonomy. Till now, we could claim that ministry does not interfere in academic matters, and we had full academic autonomy. This claim will no longer be possible.

3. Ramasami Committee should have sought views directly from IIT faculty before preparing its report, and another round of feedback from all stake holders after the draft report was prepared.

4. Ramasami Committee claims to have interacted with “several faculty members” of IITs. No one in the Senate has admitted to have communicated with Ramasami Committee. It shows that the interaction was extremely limited.

5. It is strange that IIT Council, which should be primarily concerned with IIT system, has approved a report which proposes changes to admission process of all engineering institutes in the country, not just the IITs. How come IIT Council has such jurisdiction.

6. If coaching and stress are the major issues that Ramasami Committee is trying to solve, then a different approach can be considered. If we do not assign branch at the time of admission, but do so after a year, then the stress of each mark being important, will go away . We should not be discussing just the 6 options suggested by Ramasami Committee. We should look at all options to improve our admission process.

7. The real problem is lack of opportunities, which causes 5 lakh students to try for 10,000 seats. Government should do something to improve opportunities.

8. Using statistical procedures to determine rank of students from a very narrow percentile band is problematic, both because of extraneous factors coming into determining student performance, and the confidence one could place while applying to an individual a statistical procedure developed on a much broader set of scores.

9. The best part of JEE has been its process integrity, something that cannot be expected from the boards to the same degree.

10. The main point of the ISI study is to suggest that computing percentile score on the marks- distribution from previous years also serves the present year’s
performance. This claim however is not that well supported – e.g. around 90′ percentile level for the CBSE board, the same aggregate s_core (about 0.85, figure on page 2 of ISI report) we find a difference of about 3-4 percentile ranks between 2007 and 2010. Given the formula they suggest, this would make a difference of 12 to 16 points in the final -“score”. However, the gap is smaller above 95 percentile, and on the whole, this may serve as a practicable approach.

40% to board exam?

11. Keeping the 12th class marks as eligibility criteria, of the order of 90 percentile, will achieve the goal of students taking the board exams and hence school education seriously. It is not clear what extra will be achieved by giving 40‘percent weight to board marks.

12. Giving 40 percent weight to board marks can potentially increase corruption and use of unfair means in the board examinations. It may also lead to favoritism.

13. Giving 40 percent weight to board marks will result in increased coaching, as now the students will go for coaching for both the common entrance test and the board examinations.

14. Giving 40 percent weight to board marks will result in increased stress, as every mark in every subject counts for ranking in the admission process.

Difference in standards

Consider two populations each of 1000 students with a total ordering in merit. Let us say that Board A does manage to distinguish amongst the top 10, these students get the marks in a subject from 100, down to 91, consistent -with their individual merits. In the other board B, the top 10 students each scores 100 marks. In the standard way of computing percentile rank, the 10th student in merit from the top in Board B will have the rank 99.5. Whereas the similarly merited student in Board A would have scored 91, therefore, his percentile rank will be 99.00, assuming no bunching in scores in the range 91 to 100 in Board A. Thus two students, with same merit value, can get different percentile scores depending on the board, because of difference of standards in Boards. ‘

For normalization of marks across boards, one should not just look at the statistical methods, but there should be a group which should every year look at the functioning the boards, the quality of question papers, etc., and tweak the normalization process on a year to year basis.

An Engineering Entrance Examination should not and cannot be viewed as the principal instrument of fixing large scale problems. prevailing in the school education system-(lack of effective teachers, mod.e of delivery, fraudulent and farcical science practical, easily comprised processes etc.). It also cannot be viewed as a means of ending competition for premium opportunities in the face of huge demand, no matter how perfect is the examination system. On the converse the Entrance Examination should try and do its honest best to select in the face of these problems.

We should start with board marks only being used for eligibility. We can use board marks in future, if we can see that a greater focus on board exams have improved their functioning and reduced corrupt practices.

The Weaker Section

If corruption increases in boards due to 40 percent weight of class marks, it will become very difficult for students with financially weak background to come to IITs. We can use the proposed Main-cum-Advanced test for shortlisting about 50,000 students, and should have the old—style examination involving long answers and partial marking for correct steps. We should do away with negative marking in JEE.

On BITS Pilani

We should study why BITS Pilani, which had its own normalization process of 12th class marks, and used to admit students on the basis of normalized marks, has shifted completely to an entrance test. IIT system can help in ensuring integrity of the National Test, but it cannot be held responsible for ensuring this on the scale and scope of a National Test.

On China

We should study Chinese system, since they too have expanded in a big way in the last couple of decades, they too have some very good universities, and a large number of aspirants for those universities. How have they managed admission in such a scenario.

On Multiple Choice Questions and Aptitude Tests

Multiple Choice Questions (MCQ) are very limited in usefulness, and are necessary evil to deal with large number of candidates, along with necessity to machine grade their answers. Normal stepwise long answer questions are far more effective in assessing a learner’s abilities, and thinking orientation and degree of match with the desired thought process in a context.

Multiple Choice Aptitude Test is an untested bogey, or at its best an uncritical superstition, with no basis or proof that it works ( just as measures of intelligence such as IQ), de-contextualized pattern recognition or puzzle solving can fall easy prey to superficial training, or gaming the system, lacks the depth of a scholarly discipline or an intellectual tradition to test anything real, and at its best can become some surrogate version of intuitive Maths.

We should add a test of comprehension and other subjects that are normality taught at school, but are not part of JEE as of now.

Aptitude test will have a negative bearing on the rural—urban divide. Urban students will be able to perform better in an aptitude test. Hence it is unfair.

Statistical data from the past

Since we have JEE marks, board marks, and our performance data for the last several batches, we should study correlation between these quantities, and in case, there is lack of correlation between JEE marks and students‘ performance in IITs, then JEE should be fixed to become a better predictor of success, rather than removing JEE.

[Image credit: Wikipedia]

[Jointly written by Ashish Sinha and Pratyush Prasanna.]

Sign Up for NextBigWhat Newsletter