Wednesday, March 6, 2013

Illusions on testing illusions

About a month ago my eye was drawn to "Real accountability or an illusion of success?"
It's 14+ page call to action is worth a read regardless of your personal / professional views on large-scale assessments such as the standardized tests used by Ontario's Education Quality and Accountability Office.
The call to action is to review standardized testing. The authors chose Ontario, the first of some interesting choices they've been making since releasing the report. According to the very chart provided, only Manitoba and Prince Edward Island use standardized testing less than Ontario does. One might of thought they could have called for a review of the practice in provinces where it's used across more grade levels than in Ontario (B.C. and Saskatchewan) or a greater number of subjects (B.C., Quebec, Newfoundland and Labrador). The Ottawa-bylined but multi-based crew behind this chose to zero in on Ontario, perhaps because that's where they were exposed to a greater level of scrutiny over this form of testing.
All the main issues / critiques of testing are summarized in the call to action. I actually don't have any issues with the recommendations either— though the questions asked do presuppose this review of standardized testing in Ontario would conclude they should become sample-based or eliminated. It includes more questions of testing from its critics than from the agency tasked with administering the tests and holding all that data.
I've come to the conclusion however, reading the tweets sent out by the authors, that their anti-standardized testing bias is quite evident. Read 'em for yourself. Not one pro-testing or neutral tweet, they're all anti-testing or questioning the use/validity, etc.

As a result, I've come to the conclusion their goal is worthy, but this group should not be the ones leading the work that may lie ahead. Another Royal Commission may be the only way to ensure a balanced approach. The chances of that happening are small given other items before government right now, but if this group carries forward I'm not confident at this point the end result will be a fair examination of the questions.


Sebastien Despres said...

Thank you, Hugo, for your balanced blog post on our report.

While we used Ontario as a case study focus, our hope is that our report will inspire public policy across Canada. Ontario is a most challenging province for such an endeavour - often hailed as a educational leader (because of its students' performance on international measures such as the OECD's PISA), it is also the province whose education system is most often emulated. When our pan-Canadian Task Force conducted a preliminary analysis of Canada’s education systems, we identified three provinces as a potentially productive foci: Alberta, British Columbia, and Ontario. Our initial intent was to conduct a comparative analysis of these provinces' approaches. However, a number of considerations made such a comparison very difficult - not least of which was the uncertain situation in the two Westernmost provinces vis-à-vis standardized testing.

When looking at Ontario as a case study, two things are striking:

Firstly, the Royal Commission which established Ontario's testing regime was very clear that a five-year review was to be conducted following the province's adoption of the tests: “We would not want any structure we recommend to exist beyond its actual usefulness. It is not impossible that the EQAO might one day prove redundant, and it is entirely plausible that its responsibilities might need to be revised. We therefore recommend that the work and mandate of the EQAO be reviewed in five years” (Recommendation #166). Almost two decades later, this five-year review is yet to be conducted.

Secondly, Ontario's Education Quality and Assessment Office’s choice of census testing over sample testing is surprising given the province's use of test results. In its “Public Policy Statement regarding EQAO,” the Ministry of Education is very clear in its distinction: “Large-scale assessments such as those administered by the Education Quality and Accountability Office (EQAO) differ from classroom assessments in that they are designed to provide quite different information. Classroom assessments are developed by teachers to gather evidence about their students’ learning on an ongoing basis. This information is used to provide feedback to students about next steps in learning and to inform parents of their children’s progress. Classroom assessment data is also used to provide teachers with the information they need to differentiate instruction to meet the needs of all students. Large-scale assessments, on the other hand, are one-time measures with standardized content and administration procedures so that results for education systems can be compared over time. The results of large-scale assessments are used by governments and school boards to understand the strengths and weaknesses of education systems, to develop education policies, and to allocate resources.” In other words, the Ontario Ministry of Education uses standardized test results to AUDIT LEARNING – not as a LEARNING TOOL. In this light, the scale and frequency of testing may be inconsistent with the Ontario Education Ministry’s objectives, and its scope may not be aiding in the facilitation of the objectives set out for the education system.

We argue in our report that assessing accountability for Ontario’s public education system should begin with an understanding of its objectives. As set out in the Education Act, the system serves as the “the foundation of a prosperous, caring and civil society,” which “provide[s] students with the opportunity to realize their potential and develop into highly skilled, knowledgeable, caring citizens who contribute to their society.” Success in numeracy and literacy, as measured through standardized testing, provides a limited perspective on the successes and failures of the education system’s performance in relation to these broader objectives.

Doretta Wilson said...

We only have one part of the picture on testing in Ontario. There is (some) transparency, but NO accountability--no schools ever close, no staff ever gets fired, no trustee ever gets turfed, no educrats suffer, etc, if children fail to learn. Instead it's the children that are at fault--they come from poor backgrounds, they have poor parenting, they are at risk, not ready, ad nauseum.

Until this changes, the status quo remains. SQE says until parents can vote with their feet and choose ANY schooling for their children, the system has NO incentive to improve and NO accountability for under-performance.