Statistically Authentic Z-core results for the GCE Advanced Level (A/L) Students sat for the year 2011 Examinations.

Posted on May 6th, 2012

*Dr. V. W. Jagath Vasanthathilaka*

### References related to discrepancies in the results of Advanced Level examinations and a collection of articles highlighting serious impact on Sri Lankan students facing examinations.

### Students have lost confidence on the entire examinations system. There is no use in conducting advanced level examination on this manner, z-score result needs a complete review without any delay. Continuation of a wrong methodology in next year and future examinations would be disastrous for the entire nation. Correction is not difficult, this can be done in few days.

Scientifically acceptable and accurate method of calculation of Z-scores would provide students with correct Z-score results and proper Island ranks. As far as statistical methodologies are concerned mathematical calculations provide a unique and a single figure as final calculation, it is impossible to withstand two or three different figures as the final result.

Calculation of average z scores for subjects Physics, Chemistry and Biology could exist as one and only accurate figure, any other figure given as average z-score could be considered inaccurate. There is a well-established universally accepted scientific method for calculation of z-score.

Statistical description of Z-score = x-µ/ ÃƒÃ†’ by definition

x is raw mark (x is the raw score for z-score calculation)

ÃƒÅ½Ã‚¼ is the mean of the student population.

ÃƒÃ†’ is the standard deviation of the same population.

His Excellency President has informed Department of Examinations to be transparent and release students with Mean values and standard deviations. Her Lordship Chief Justice Dr. Shirani Bandaranayake instructed that the students should be able to understand how the Z-score was calculated and how they got selected to University admission. According to several media discussions held on this subject and inline with the figures released to panels setting future examinations, mean values for new syllabus and old syllabus based examinations are not equal. These differences were found in all subject streams, including Arts, Business studies and Science subjects. All students now request raw marks standard deviations and Mean values of their results. Students have a right to know correct results.

First attempt. (New syllabus) Mean and Standard deviations for 2011 Advanced level examinations for biology subjects.

Physics (Mean) 37.1 & 15.17 (Standard Deviation)

Biology 43.7 & 15.2

Chemistry 35.17 & 15.7

Students obtain two A grades and one B grade could obtain following z-scores.

First Example.

Physics-raw marks 75 Z-score for Physics 2.49835201

Chemistry marks 65 Z-score for Chemistry 1.90000000

Biology marks 75 Z-score for Biology 2.05921053

Average Z=score 6.45756254/3 = 2.15252084

Z score result of the student= 2.1525

Second Example.

Physics raw marks 75 Z-score for Physics 2.49835201

Chemistry marks 75 Z-score for Chemistry 2.53694268

Biology marks 65 Z-score for Biology 1.40131579

Average Z=score for three subjects 6.43661048/3 = 2.145536826

Z score result of the student= 2.1455

Third Example

Physics raw marks 65 Z-score for Physics 1.83915623

Chemistry marks 75 Z-score for Chemistry 2.53694268

Biology marks 75 Z-score for Biology 2.05921053

Average Z=score 6.43530944/3 = 2.14510314

Z score result of the student= 2.1451

Scientific community does not recognize, and not aware of any other method for calculation of z-score. However, all students who obtained A and B grade results in the first attempt were given much lower z scores and lower all Island ranks based on wrong z-score results calculated by the department, according to the fraudulent method given by the UGC. Thereby, large numbers of students were deprived of opportunities for university education. Students actually obtained z score result above 2.0 in the 2011 examinations were given absolutely wrong figures much less than 2.0, figures around 1.5 and 1.96. Thereby those students who represent an Island Ranks between 200 to 600 were erroneously placed far below 1100 in all Island ranks. Displacement by more than 700 ranks from their actual position is a grave situation.

Best way to correct them is to recalculate according to the statement issued by Prof. R P Gunawardane, former secretary to the ministry of higher education. UGC may have to increase the uptake to accommodate more students after correction.

In the mean time figures for the purpose of calculation of other population of students are not equal. Obviously, these results need separate calculation.

Second and third attempts, (Old Syllabus) Mean and standard deviations.

Physics 41.1 & 16.2

Biology 52.8 & 15.3

Chemistry 39.5 & 15.6

Where there are more than twenty options in different subjects in Arts, students opted for difficult subjects which gives low mean values such as English, English literature receive very low z score values despite obtaining A grades. Those students also deprived of university education due to the erroneous method used in calculation of z scores. Second and third attempt students were severely affected in some arts subjects and Business studies (commerce) subjects, and deprived of higher education, entire result sheet is defective. As two syllabuses were not equal two examinations were held, if the syllabi were equal common examination would have been held. There are serious errors in results among the same examination as well, there are difficult papers and lower mean values in different subjects which are not equal. Students sat for difficult subjects receive much lower z-cores and a lower island rank in the same examination compared to their counterparts, in that way students were deprived of university education. Litigation issues can be propped up 5 to 10 years later, due to the fact that, students carry a wrong certificate throughout their life, in each and every future interview. Separate analysis would correct all these discrepancies that would be the biggest service done to the government and towards the younger generation to regain confidence on education system.

Students have lost confidence on the entire examinations system. There is no use in conducting advanced level examination on this manner, z-score result needs a complete review without any delay. Continuation of a wrong methodology in next year and future examinations would be disastrous for the entire nation. Correction is not difficult, this can be done in few days.

Thank you for your kind attention.

Important references.

http://english.srilankamirror.com/2012/03/president-too-finds-z-score-puzzling/

# President too, finds z-score puzzling

(Srilankamirror) “- President Mahinda Rajapaksa has ordered the education minister to submit a technical clarification to the cabinet within a fortnight about the formula for the z-score at the GEC advanced level examination”‚¦”‚¦”‚¦”‚¦.

http://www.island.lk/index.php?page_cat=article-details&page=article-details&code_title=43036

University admissions and the Z score

January 12, 2012, 12:00 pm

Prof. R.O. Thattil

The release of the GCE A/L results has led to many controversial issues regarding the method used to rank the students. The students and parents are in a dilemma whether to accept the results or not. Therefore, I decided to write this article to clarify matters since I was responsible for introducing the Z score as a basis for ranking a decade ago”‚¦”‚¦”‚¦”‚¦”‚¦..

GCE A/L-2011 results need urgent review

January 24, 2012, 7:09 pm

By Prof. R. P. Gunawardane

Indiana State University, USA

(Formerly Secretary, Ministry of Education and Higher Education, Secretary, Higher Education and IT Development, Chairman, National Education Commission, Senior Professor & Dean, Faculty of Science, University of Peradeniya)

I have watched with great pain of mind the events and developments that took place after the release of the results of GCE A/L Examination held in 2011. Many errors and omissions in the results have been reported and a presidential committee was appointed to look into this matter. The report of this committee has now been submitted to the President”‚¦”‚¦”‚¦”‚¦.

**Similar proportions of Advanced Level students according to the pattern observed in previous years could be selected for University education only on the basis of correct z-score results calculated using scientifically accepted and standard methodology, results cannot be altered by any means to get hold of arbitrary proportions. **

Sri Lankan Students sit for the advanced level examinations in the present day context are free to opt for different combinations of subjects. Curriculum and the different options available for combination of subjects for Advanced Level (A/L) examinations at present are wider, which is an essential feature of an education system that provides room for further expansion and improvement. There are more than 20 subjects in arts stream, as far as Biological sciences are concerned there are students opt for different combinations such as agriculture in place of biology, and in mathematics there are students opt for different combinations. In the stream of business studies (Commerce) there are students who opt for statistics instead of business studies. Combinations in arts subjects are complex and there are many different populations of students and when there are two examinations structure of the examinations and the mean values for each and every paper is different from the other. Therefore Z score cannot be calculated using any other figure other than the mean and standard deviation of the same population of students to obtain accurate results. Due to the fact that syllabuses were different two different examinations were held instead of a single examination in the year 2011.

Results for z-scores are absolutely wrong between two examinations and also within the same examination when the raw marks of different populations are pooled together for the purpose of calculation, z scores are inaccurate as there are several options in most subjects streams for advanced level students in present day curriculum. Each examination paper and the each subject have a different mean and a standard deviation. First of all average Z score for each student need to be calculated separately by taking the raw marks for each subject. For the correction of results, separate calculations are applicable within the same examination as well as for two different examinations as it is obvious each and every population of student is different.

Average numbers of students selected for Engineering faculties in the first attempt in the past three years were 64.66 percent according to figures given by the department. Unfortunately, figures were given only for three years despite a request made for the department to be transparent towards students. Average figure for 10 years would be a much better figure as it would be a more acceptable percentage for selection, figures for three years could be variable does not represent a long term pattern. These figures given by the department are misleading and the overall and actual figures when it comes to university admissions based on erroneous results would be different. Based on the limited information given by the examinations department and the University grant commission (UGC), if there are 1315 admissions for the engineering faculties 850 from the first attempt and 465 from the repeat examination could be absorbed on the basis of correct z scores calculated separately for two examinations. Instead of selecting a percentage or a proportion on the basis of correct Z score results, Z-scores cannot be distorted arbitrarily to obtain an illogical proportion preferable to a few on a very personal basis.

Generally over the period of last ten years more students were selected from first attempt in most of the subject streams. However, pattern for the last three years for admissions to medicine were 43% in the year 2008, 42% in 2009 and 41% in 2010 for the first attempt according to figures given by the department. Even if they consider average figures only for three years, at least 42 percent need to be absorbed from the first attempt for medicine after separate analysis. Ideally average figures for ten years could be taken as a more accurate figure.

Correct method of selecting students for medical schools would be to absorb 42 percent from the total number of admissions to all medical faculties from the first attempt, and the rest from repeat population of students. In order to select 42%, Z score needs to be calculated correctly in the first place therefore Z score need to be calculated separately before determining a percentage. Therefore, on the basis of average figure given for three years minimum of 494 students from the first attempt and 681 students from subsequent attempts need to be absorbed to medical faculties out of the given total figure of 1175 by the UGC. If possible total numbers need be increased.

As long as the calculation of Z score is wrong, due to the fact that erroneous pooled method is used by the department, students cannot be absorbed on the basis of their incorrect Island and district ranks. Whereas, when students are ranked separately on the basis of accurate z scores students can be absorbed on correct proportions by allocating 42 percent of total places to first attempt and 58 percent to the repeat population of students while maintaining the same percentages of admissions observed in previous years. Proportions of students allocated for each examination can be absorbed on the basis of merit (Forty percent) and on district basis separately. Student can be given the correct Island rank based on the examination which they have sat for in the year 2011. On this way, the proportions of students selected would be based on correct Island and district ranks, at the same time proportions of students selected would be comparable to average figures observed in previous ten years. Students receive a certificate which should be a legally acceptable document where the z-score is calculated accurately and their Island rank obtained in respective examination is given accurately.

It is also said by the department that, the department is satisfied and there is no departure from the pattern of results observed in previous years. This seems to be based only on data for last 3 years and overall and total admissions for2011 have not been analyzed.

year | bio stream(Medicine) | physical science stream(Engineering) | ||

1^{st} attempt |
repeat | 1^{st} attempt |
repeat | |

2008 | 43 | 57 | 70 | 30 |

2009 | 42 | 58 | 66 | 34 |

2010 | 41 | 59 | 58 | 42 |

pooled method | 37 | 63 | 63 | 37 |

Teachers and students collected data from the openly displayed result sheets to verify the accuracy of those figures. If the UGC try to absorb students on the basis of wrong z scores, actual overall percentage absorbed from first attempt would be lower than 29 percent. Further more, it is extremely difficult to accept the presumed figure of 37 percent given for the erroneous pooled method for first attempt students in medicine, it is not an overall analysis and so for UGC do not know the cut off marks for each district. As it is observed in most of the districts actual proportions are much less than the given figure of 37%. Overall analysis of all Island results would also show that actual proportions are much lower for first attempt when it comes to selections. On the other hand arbitrary figure of 37 percent is also based on wrong z scores.

Analyzing results in many districts show that only a small proportion, less than 29 percent of students would be selected to medical faculties from the first attempt more than 71 percent would be entered from the second attempt. Those figures are only for the first and second attempts, third attempt students cannot be included as it is not possible to collect result sheets of individual students who sat for the third time. Students may request the department to produce overall result sheets and overall proportions to verify correct figures, when the third attempt results are also added to the above mentioned proportions, figures for first attempt would be very much lower than 29 percent, and consequently an unacceptably large proportion of students would be deprived of further education. Less than thirty percent of students absorbed to medical faculties from the first attempt would be harmful for the system of university education as it is observed and proven by several research studies that first attempt undergraduates perform better than repeat students in the universities and in post graduate institutions.

Instead of giving correct z-score results and selecting a percentage of students on the basis of results, department has attempted to distort Z score results to obtain some form of irregular proportions. Unfortunately the arbitrary process of distortion used for a particular combination of subjects gives rise to much worse errors in a different subject stream, therefore results in other subject streams gives worse errors in a completely unexpected and irregular manner. Instead of selecting a percentage or giving a shrinkage factor considering the pattern observed in previous years for each subject steam entire Z score result has been messed up by the department. Entire result sheet is an irregularly irregular mess. Obviously it is not possible to satisfy to the pattern of results obtained in previous years by using this irregular method, results are not by any means reflected to the previous years by the so called pooled method given by the UGC.

Overall analysis of results would show discrepancies in all subject streams including Business studies, Arts, Medicine, Agriculture and Engineering subject streams. In those subjects streams where there are many options for example Arts subjects, statistics in commerce and in agriculture in biological science there are discrepancies within the second attempt and within the first attempts in the same examination, evidently the mean values for each and every subject is different and unequal. Accordingly, students within the same examination regardless of whether it is old or new syllabus large number of students who sat for difficult papers would be deprived of university education due to errors in Island ranks. There are students who obtained A and B grades ranked below eighteen thousand in Arts subjects. English medium students were badly affected due to the fact that their mean figures were lower as they had to face difficult question papers answering using a second language is also difficult.

Newly introduced method used by the UGC is previously unknown to the scientific community, such a method for calculation of z-scores is not recognized by the scientific researchers anywhere in the world, pooling raw marks to calculate z-score using a combined mean and standard deviation is scientifically inaccurate. Younger generations do not have to go on suffering due to these blunders. This has to be corrected for the sake of future generations in this Island.

It is impossible to proceed to the Advanced Level examination next year without correcting all these errors in A/L results, unresolved issues in results would be carried forward, that would definitely complicate future advanced level examinations, that would be extremely difficult and stressful for all categories of students.

When there are large numbers of students who were actually qualified and highly competent in their selected subjects and in the meantime attempting to compete with the next year batch of fresh students, it would be virtually impossible for the students who sit for the first attempt to score much higher marks for the same subjects and for the same examination as they are competing with an experienced and accomplished group of students left over from the previous year. Same phenomenon would be propagated for several years.

Worse scenario would be that, recognition of an erroneous pooled method would result in recommendation and application of an erroneous methodology for future examinations, which would go on for several years without understanding the danger and disastrous consequences for future generation, entire education system would be in serious trouble, hundreds of thousands of applications for re-corrections every year. Confidence on the examination system needs to be regained in the earliest possible.

There are suitable solutions, errors can easily be rectified by recalculation, and it is not a difficult task to correct them within a short period of time that would help regain confidence on the education system. Separate calculation, correction of Z score results on examination certificates, separate ranking and selection of students on the basis of previous patterns and proportions observed for the last ten years would help resolve errors in results and uncertainties in Advanced Level examinations. This process can be done openly and transparently. Government could certainly regain the trust and confidence of the younger generation.

http://newsradio.me/features/bidding-adieu-to-pythagoras%e2%80%a6 Further analysis on z-score results along with Mean and Standard deviations please se the third response to the article on News Radio.

Thank you.