|
(Peter Hallinan is a lecturer in educational psychology at the University College of Central Queensland.)
This paper concerns itself with a relatively informal and unstructured approach to the assessment of student ability,, through the subjective perception of the teacher. Such assessments are frequently made in the formative stages of learning, and the writer was interested to see what degree of agreement existed among the teachers concerned.
"Please rank these students from 1-10, on the basis of your perception of their academic progress, in the subject(s) you teach.The ten students' first names were placed in random order over the page above these instructions.
- Avoid tied ranks.
- Do not discuss this exercise with any other lecturer before the examiner's meeting.
- Please return this sheet to the researcher."
| STUDENTS (N=10) (rank order) | MEAN SCORE (X) | STAFF MEMBER* (N=7) | ||||||
| Keith | Brenda | Mary | Sam | Bill | Quentin | Sarah | ||
| Kerry | 2.29 | 2 | 2 | 3 | 1 | 1 | 3 | 5 |
| Don | 2.43 | 1 | 1 | 4 | 3 | 3 | 4 | 1 |
| Marian | 2.71 | 5 | 5 | 2 | 2 | 2 | 1 | 2 |
| Isabel | 4.29 | 4 | 4 | 1 | 7= | 8 | 3 | 3 |
| May | 5.71 | 3 | 3 | 7 | 10 | 4 | 6 | 7 |
| Hilda | 5.79 | 6 | 6 | 5 | 7= | 5 | 7.5= | 4 |
| Bobby | 6.64 | 7 | 7 | 6 | 7= | 6.5= | 5 | 8 |
| Neville | 7.57 | 8 | 8 | 10 | 7= | 6.5= | 7 5= | 6 |
| Raymond | 8.71 | 9 | 9 | 8 | 7= | 10 | 9 | 9 |
| Edith | 9.23 | 10 | 10 | 9 | 7= | 9 | 10 | 10 |
| * Names have been altered to preserve anonymity. | ||||||||
| Keith | Brenda | Mary | Sam | Bill | Quentin | Sarah | |
| Agreement with Mean | .94* | .94* | .82* | .64 | .85* | .89* | .81* |
| Keith | 1.00*+ | .58 | .39 | .78* | .75* | .73* | |
| Brenda | .58 | .39 | .78* | .75* | .73* | ||
| Mary | .54 | .51 | .86* | .75* | |||
| Sam | .60 | .64 | .54 | ||||
| Bill | .74* | .64 | |||||
| Quentin | .74 | ||||||
| Sarah | |||||||
| * p<.01 + spuriously high correlation | |||||||
| STUDENT SUBJECTIVE MEAN SCORE | RANK ORDER | STUDENT WEIGHTED GRADE POINT AVERAGE | RANK ORDER | |
| Kerry | 2.29 | 1 | 5.17 | 2 |
| Don | 2.43 | 2 | 5.35 | 1 |
| Marian | 2.71 | 3 | 4.88 | 4.5 |
| Isabel | 4.29 | 4 | 4.82 | 6 |
| May | 5.71 | 5 | 5.05 | 3 |
| Hilda | 5.79 | 6 | 4.70 | 7 |
| Bobby | 6.64 | 7 | 4.88 | 4.5 |
| Neville | 7.57 | 8 | 4.35 | 8 |
| Raymond | 8.71 | 9 | 4.20 | 9 |
| Edith | 9.23 | 10 | 3.68 | 10 |
In considering Table 1, Bill may have a point, as the staff generally seemed more in agreement about the top students (Kerry and Don) and the bottom students (Raymond and Edith) than about the bulk of the students in the middle ability range. Despite the exhortation to avoid discussion of results, there are grounds for skepticism in Keith and Brenda's case.
In considering Table 2, Spearman's rho was used due to the nature of the data and the small size of the sample (Van Dalen, 1979:491). Even discounting Keith and Brenda, the large number of significant correlations testifies to a fair degree of consensus among the staff members concerned. In Sam's case, his idiosyncratic approach, possibly linked to his particular academic discipline, led to his being the 'odd man out'. By contrast, despite her concern over elapsed time, Sarah reached a high degree of consensus with other staff. In terms of shared discipline areas, Keith with Brenda, Mary with Quentin, and Sam with Bill, could be said to share some common professional linkages.
Turning to Table 3, a grade point average over both semesters was also computed, using a five-point range from three points for a failing grade through to seven points for a grade of high distinction. With regard to Table 3 above, since the data were of rank-order nature, the SPSS NONPAR CORR procedure was used to compute the correlation between SS Mean and Mean GPA (SPSS, User's Guide, 1986, p.801). Since the direction of the relationship between variables was apparent in advance of the analysis, the significance levels of the coefficient was based on a one-tailed test (SPSSX User's Guide, p.803). Spearman and Kendall coefficients of correlation were computed. Both showed a high correlation between mean subjective assessment of rank order and mean GPA (p <.000 and p <.001 respectively).
In summary, while teacher-initiated chance meetings and conversations over the teaching year are clearly at the less formal and less structured end of the assessment continuum (Grace and Cassidy, 1988:78), a high degree of consensus among staff does exist, particularly for those students at the ends of the ability ranges, both high and low. This consensus is all the more noteworthy given the range of variability in attitudes and practices in assessment across the tertiary disciplines represented: language arts, mathematics, philosophy, psychology and sociology. Future studies might focus on the degree of consensus within and among specific disciplines, and the degree of consensus between judgments of tertiary staff at the preservice phase of the students' career, and those of educational administrators such as principals and inspectors, in subsequent years.
SPSSX Users Guide: Edition 2 (1986) Chicago: SPSS.
Van Dalen, D.B. (1979) Understanding Educational Research: An Introduction, New York: McGraw Hill.
| Please cite as: Hallinan, P. (1990). A summary of a small survey on subjective assessment of student ability. Queensland Researcher, 6(2), 16-20. http://www.iier.org.au/qjer/qr6/hallinan.html |