Norm V Form
- I agree with a lot of what CB (Curmudgeonly Bugbear) has to say in his latest post. dk, it astonishes me how you have time to write such posts, teach, read (+ remember...hell...and *understand*) so much, paint and live your life. But I'm glad you do. You may be a C.B. (whom, for some strange reason, I had confused with Ug - perhaps because I've argued so much with both of you), but you're *our* C.B.!
As for the formative / normative thing, I'm not sure I agree. Isn't the whole thing about normative testing that it seeks to label people as passes or failures? It's not as innocent as being some sort of 'social' testing or a case of simply looking at people's work in the light of that of their colleagues. For example, I would hope and imagine that none of your students would be failed for writing a perfectly good composition simply because their other classmates wrote better.
I also fail to see how normative testing *can* be particularly formative. As an assessment tool, it seems to be pretty summative. Naturally, some students may respond rather well to it, but, I would suggest that that has more to do with the motivation of the students rather than the nature of the test. However, I agree with Rowntree who points out that it's not the nature of the test that decides whether it's formative or summative, but the intention underlying it.
As for your comment about tests being uninteresting and fairly useless, I couldn't agree more. And that includes gatekeeping tests too. If we accept that everybody should have the opportunity to study for as long as they want and to whatever level they want, it strkes me that the onus is upon society to make provision for them. One way might be to redirect the millions (billions?) spent on guns, bombs and razing Middle Eastern countries to the ground...but that's probably another list...
[Non-text portions of this message have been removed]
Well, thanks for the warm words, and even for the cold shower of
confusing me with...you know who. After all, you also confuse him
with some of the greatest painters who ever lived, the cavemen of
50,000 years ago.
Where does the time come from? Well, one thing the unlamented Ug got
dead right. Dogme is no skive, but academic jobs at Asian unis are a
doddle. Here at the University of Education, we teach ten hours a
week, and are considered grossly overworked by our colleagues at
Seoul National, who teach around six. We also get TWO holidays of
over two months each.
On my wall there is large painting (two metres by about one and a
half, Scott) which I am doing for the cover of my wife's next book. I
have been "painting" it for over a year now, and it's still not
finished, but then my wife has been writing the book for over six
years, and it's not finished either. She's at Seoul National, but
she's a PhD student, poor thing!
Today is Foundation Day, the day upon which Koreans celebrate the
foundation of the Korean universe, and one of the few days in the
nationalist calendar really worth celebrating. But not being Korean I
have come into my cave-office, painted a bit, and...damn, it's time
to write mid-term exams. So rather than painting rather than writing
mid-terms, I am writing to Diarmuid ABOUT mid-term exams.
Let me begin with two quotations. The first is Tim McNamara, writing
out the terrible problem of disambiguating the performances of two
kids taking an oral at the same time (as in the current version of
spoken section of FCE). Good Old van Lier points out that this is
just the latest version of a problem that has ALWAYS been there: how
do you disentangle, for example, the performance of the interviewee
from that of the (not always impartially) helpful interviewer?
McNamara worries about this problem, doesn't come to a conclusion,
and then asks rhetorically what course of action follows from reading
"Intellectual understanding can complicate, even paralyze action; but
action without understanding is blind and can be destructive. In a
cruel world, our dilemma in applied linguistics, poised uneasily
between thinking and acting, resembles that of Hamlet, contemplating
action, but the contemplation making action even more difficult.In
applied linguistics as a whole, and in language testing in
particular, in our efforts to become a scence we must remember that
our enterprise is irrevocably human." (Tim McNamara, "Interaction in
Second Language Performance Assessment: Whose Performance?", Applied
Linguistics 18/4: 460)
The second quote is less obviously relevant, except that it begins
with "Given the dark and bloody period in which I am writing--the
criminal ruling classes, the widespread doubt in the power of reason,
continually being misused--I think I can read the story thus:
"It is an age of warriors. Hamlet's father, king of Denmark, slew the
king of Norway in a successful war of spoliation. While the latter's
son Fortinbras is arming for a fresh war the Danish king is likewise
slain by his own brother. The slain kings' brothers, now themselves
kings, avert war by arranging that the Norwegian troops shall cross
Danish soil to launch a predatory war against Poland. But at this
point, the young Hamlet is summoned by his warrior father's ghost to
avenge the crime committed against him. After at first being
reluctant to answer one bloody deed by another, and even preparing to
go into exile, he meets young Fortinbras at the coast... Overcome by
this warrior-like example, he turns back and in a piece of barbaric
butchery slaughters his uncle, his mother, and himself, leaving
Denmark to the Norwegian. These events show the young man, already
somewhat stout, makng the most ineffective use of the new approach to
Reason which he has picked up at the university of Wittenberg."
Bertolt Brecht on Theatre, Methuen: 1964: p. 202.
Both Mac and Brecht are saying the same thing. From the hawkish point
of view, it doesn't really do to have a rational approach to
profoundly irrational decisions, such as the decision of who shall
pass and who shall fail, who shall live and who shall die. The
exercise of rationality in irrational decisions can only paralyze us.
The "normal distribution" is a wonderful example of this kind of
irrational application of rational technology. It really doesn't
matter where you put the cutoff line--there will be irrational
decisions, decisions based on nothing more than random variations, on
both sides of it. So of course they tend to put the cutoff at the
mean; that way they can make the absolute maximum number of totally
baseless decisions. Like Hamlet, they can proceed from ignorance to
It's only for the extremes of the bell curve that you can make
rational decisions based on non-random differences. You know that the
right tail is in some more or less important way different from the
left tail, even if you cannot say that the right side of the central
peak is in any significant way different from the left side.
Naturally, obvious decisions about the extremes are not the decisions
that the warrior gate-keepers are interested in.
But they ARE the decisions that teachers are interested in. We DO
want to know the range of things that the people in our class can do,
from the lowest level to the highest. Unlike coursebook writers and
publishers, we are NOT interested in teaching "median" or the "mean"
of the class; we want to teach every member.
Pauline Rea-Dickins is speaking here in Seoul on Saturday. She wrote
a pretty good article on the kind of informal formative testing that
all elementary school teachers engage in while planning syllabi. She
points out that it is usual to assume that formative testing is "low
stakes" and it's really okay if the teacher makes a mistake. She
argues that this is not always true--sometimes a decision made by a
teacher on wildly impressionistic data can, for example, force a
child to leave bilingual education prematurely. ("Snares or Silver
bullets: disentangling the construct of formative assessment",
Language Testing 17:2).
The beautfy of dogme is partly in the negotiability and the
reversability of the formative decisions that are going on in every
class. That negotiability and reversibility is perfectly possible
with any kind of testing: norm-based, criterion-based, direct or
indirect. It's just a matter of disentangling the irrational
decisions that the outside world wants tests for from the rational
purposes that teachers need testing for.
As the Chinese poet Cao Xueqin put it, "Blame not the crab for
walking sideways; it is the ways of the world which are crooked."
There is always the possibility that this paralysis of knowledge may
help teachers and learners sab the gate-keeping actions of society.
Sabbing the gate-keepers is the whole point of testing, for both
learners and, much more systematically and systemically, teachers.
We who know tests well know their limited scope. If we combine this
knowledge with our knowledge of the unlimited scope of our learners
needs, lives, and potential for learning, we are in a dreamy position
to take paralyzing action.
Our enemies know this too. It is partly knowledge of the paralyzing
nature of knowledge that explains the extreme anti-intellectualism of
the criminal American ruling classes. This is the source of their
ebullient combination of total gormlessness and total shamelessness
(yesterday I heard a senator explaining in complete deadpan, without
batting an eyelid, that the US had the right to defend itself because
Saddam Hussein was a threat to his own people), their complete lack
of embarrassment about their complete lack of coherence. They are at
peace with themselves, because they know they are at war with reason
and with humanity.