Designing Good Tests: Principles Into Practice

Keith Morrow is the editor of ELT Journal and has worked in language testing for many years. He was involved in developing some of the first ‘communicative’ language tests, and is currently working as a consultant to testing projects in Austria and Luxembourg. Keith hosted a Global Webinar ‘Designing Good Tests: Principles Into Practice’ on January 12th and will be repeating it on January 31st. You can find out more information and register to attend here.

Testing goes on in almost every educational institution in the world, and is familiar to both teachers and students. “On Thursday we’ll have a vocabulary test”.  “I want to get good marks in the end-of-year exams”.

Despite this, teacher training programmes often pay very little attention to the role, purpose, and nature of testing in the classroom. As a result many teachers feel insecure about the principles and practice of testing, and so they put together tests based on what they have always done – or just use tests from published sources.

Do you see a little bit of yourself in this description? Would like to find out more about some background ideas in testing?

For example, what is testing? Is it the same as assessment)? Why do we test? To help the students or to frighten them? Is it a carrot or a stick? How is a test made? What are the different forms a test might take? What are the different focuses a language test might have? And most importantly, of course, how can we design better tests in our own context and for our own purposes?

These are some of the areas we will be looking at in my webinar on 31st January. Please come and join me, to meet colleagues from all over the world online, and to have a chance to share ideas and insights about testing.

After the webinar on this topic that I gave earlier this month, there were a lot of questions that I didn’t have time to answer online. So here are some quick thoughts on some of them.

Can the selected response task test both elements of language and communicative skills?

A multiple-choice test can be a good way of finding out what students know. But finding out what students can do is rather more difficult. If you are thinking of communicative skills in terms of production (speaking and writing), I think you have to see how well they can actually speak or write. And you can’t do that with multiple choice.

How about testing reading skills in the students own language?

I think this is a really interesting idea. If you want to find out how well students have understood a reading text, ask them some questions on the text and let them answer the questions in their own language. After all, it is their reading you are interested in assessing, not (in this case) their writing in English. But there are problems. What do you do if you have a class with ten or twenty different “own languages” which you do not speak? And will you be able to persuade parents and the school authorities that this is a good idea?

Can we use a mix of both selected and created responses in the same reading, for example?

In principle there is no problem with this. But you need to ask yourself ‘why’? Why am I using a selected response here, why am I using a created response there? If you have good answers to these ‘why?’ questions, by all means mix the formats.

When a student has only two options – A or B, does that stop them analysing the question/ choices at all?

This is an interesting question about all multiple choice questions – not just A / B (True / False) questions. Some students find the answer to the question  in the way you would like them to – by reading the question carefully and thinking about the right choice; others try to “outsmart” the teacher (the answer to the last three questions was A so this one must be B); others just guess without bothering to think at all. There are some advantages to multiple choice questions (as we discussed in the webinar) but this is a big disadvantage.

What should we focus on in a speaking test?

That depends what you want to find out!! Do you want to test spoken production (how well the student can give a mini-presentation or a little talk about something – a monologue) or spoken interaction (how well the student can take part in a conversation). In judging what the student says, you might want to focus on areas like ‘communication’ (does the student get his / her message across?); ‘accuracy’ (is the language the student uses accurate and correct?); ‘range’ (is the student able to draw on a good range of vocabulary and grammar?). The important thing is not to turn a speaking test into a form of grammar test, penalising the student for each and every grammar mistake and not looking at other aspects of his/her performance.