There are many ways to assess learners, for example, mini-tests or observations, in order to evaluate and monitor their understanding and progress. As well as checking learners’ competencies in some specific language or skill, evaluation allows us to guide learners on how to improve. Part of this is noting any errors they make in completing the assessments, especially errors in the language they use. However, focusing on errors too much can be de-motivating for learners. They may struggle to improve because they are anxious about making mistakes, especially with productive tasks. So how can we correct English errors and at the same time keep learners motivated to improve? Continue reading
Tag Archives: Testing
Writing ELT tests for teenagers | ELTOC 2020
I don’t want to sound too stuffy as I firmly believe that 42 is the new 21, however, teenagers today live very different lives to those who came before. Starting this blog with a quick comparison of my teenage life and my niece’s teenage life seems a good way to start. I was 12 in 1988, my life revolved around school, family, youth club, and the 4 channels on UK television. I loved music and spent all my pocket money on tapes, spending my evenings memorising the lyrics from the tape inserts. Now, Millie, my niece is 12 in 2019 and her teenage years are radically different to mine. Still, her life revolves around family and schools but the impact of technology on her life is of fundamental importance and so creates the biggest difference between our teenage lives.
But what does all of this have to do with assessment? Well, as Director of Assessment at OUP responsible for English language tests, some of which are aimed at teenagers, it’s very much my concern that what we design is appropriate for the end-user. My ELTOC talk will be about designing assessments for teenagers. Let’s start by considering why…
Why do we design a test specifically for teenagers?
Our aim is to make the test an accurate reflection of the individual’s performance as possible, and that means removing any barriers that increase cognitive load. Tests can be stressful enough and so I see it as a fundamental part of my job to remove any extraneous stress. In terms of a test for teenagers, this means providing them with test items that have a familiar context. Imagine an 11-year-old doing an English language assessment and facing this writing task. It’s not a real task but it is indicative of lots of exam writing tasks.
The 11 year might have the linguistic competence to describe advantages and disadvantages, make comparisons and even offer their own opinion. However, the teenager is likely to struggle with the concepts in the task. The concepts of work and flexible working will not be familiar enough to enable them to answer this task to the best of their ability.
This is why we develop tests specifically aimed at teenagers. Tests that allow them to demonstrate linguistic competence that is set within domains and contexts that the teenager is familiar with. An alternative question that elicits the same level of language is given below. It might not be the perfect question for everybody but it’s a question that should be more accessible to most teenagers and that allows them to demonstrate linguistic competence within a familiar context.
We have a responsibility to get this right and to provide the best test experience for everybody to enable them to demonstrate their true abilities in the test scenario. For us, behind the scenes, there are lots of guidelines we provide our writers with to try to ensure that the test is appropriate for the target audience, in this case, teenagers. Let’s look at this in more detail.
Writing a test for teenagers
Let’s think about the vocabulary used by a teenager and vocabulary used by the adults writing our test content, the potential for anachronisms is huge. Let’s look at this issue through the evolution of phone technology.
As well as the item evolving, so has the language: phone / (mobile) phone / (smart) phone. The words in parentheses gradually become redundant as the evolved item becomes the norm so it’s only useful to say ‘mobile phone’ if you are differentiating between another type of phone. For those of us who have lived through this evolution, we may use all of the terms interchangeably and writers might choose to write tasks about the ‘smartphone’. However, teenagers have only ever known the ‘smart, mobile phone’- to them, it’s just a phone! It’s not a big deal unless you’re a teenager in an exam suddenly faced with a phrase that might cause confusion. Other examples of such anachronisms include:
- Video game, or is it just a game?
- Do we say desktop, laptop, or just computer?
- Would you talk about a digital camera or a camera, or would you just use your phone?
- Are good things: cool, wicked, rad, awesome, chill, lit or maybe you just stan?
Writing tests for teenagers that incorporate the kind of language they are used to needs to be considered but this should be balanced with maintaining and measuring a ‘standard English’ that is recognised by the majority of people doing the test in different countries around the world as we produce global tests. Another important consideration is creating tasks of sufficient complexity that we can be sure of the level we are measuring.
As a test provider, we have people whose job it is to solve some of these challenges. For teachers, who write assessments for their students, some of the same challenges exist but with less resource available to solve them. This is why you should join me for my ELTOC session!
Sarah spoke further on this topic at ELTOC 2020. Stay tuned to our Facebook and Twitter pages for more information about upcoming professional development events from Oxford University Press.
You can catch-up on past Professional Development events using our webinar library.
These resources are available via the Oxford Teacher’s Club.
Not a member? Registering is quick and easy to do, and it gives you access to a wealth of teaching resources.
Sarah Rogerson is Director of Assessment at Oxford University Press. She has worked in English language teaching and assessment for 20 years and is passionate about education for all and digital innovation in ELT. As a relative newcomer to OUP, Sarah is really excited about the Oxford Test of English and how well it caters to the 21st-century student.
What does Assessment for Learning look like in the classroom?
Assessment for learning (AfL) is a catchphrase with which many teachers may be familiar and yet may not feel confident that they know what it means in terms of classroom practice. Here I outline the basic ideas behind it and the kinds of classroom practices AfL may involve.
At heart, it’s what good teachers do every day:
- they gather information about where learners are in their learning, what they know and don’t know;
- they help their students understand what, and why, they are learning and what successful performance will look like;
- they give feedback which helps learners ‘close the gap’ between where they are in their learning and where they need to get to;
- they encourage learners to become more self-regulating and reflective.
The evidence is that, done well, these practices are among the most effective ways of improving learning and outcomes.
Assessment in this process is essentially informal, the information teachers gather comes in many forms, for example, through classroom dialogue, following up on unexpected answers, or recognising from puzzled looks that the students have not understood. Tests play a part, but only if they are used to feed directly into the teaching and learning process.
What would we expect to see in an AfL classroom?
Diagnostics. There would beevidence of teaching and learning that is active, with students involved in dialogue with their teachers and classmates. This goes beyond simple recall questions and will include seeking out students’ views (‘what do you think….) and giving them time to think about their answers – often with a classmate (‘pair and share’).
Clarity about learning intentions. This requires teachers to be clear about what is to be learned, how the lesson activities will encourage it, and where it fits in the learning progression. They then seek to make this clear to their students by linking it to what they have learned already and showing why it’s important. Expert teachers will use imaginative ways of introducing the learning intentions (‘why do you think we’re doing this?’) rather than routinely writing out the learning objectives.
Teachers will also clarify what a successful performance will look like, so that the learners can see the standard they need to achieve. Teachers may do this by negotiating with the class about what the learners think a good performance might involve (for example: ‘what would you look for in a good oral presentation?’). Another approach may be to exemplify the standard by using examples of work (best as anonymous work from other students). A teacher may give the class two pieces of work, she may then give the class the criteria for assessing the work (no more than two or three key criteria) and ask them, in groups, to make a judgement about their relative quality. This also provides a vital step in being able to evaluate the quality of their own work and become more self-regulated learners.
Giving effective feedback. Providing feedback that moves learning forward is a key, and complex, teaching skill. We know from research that feedback is hard to get right. Good feedback ‘closes the gap’ between a learner’s current performance and the standard that is to be achieved. Some of the key features in quality feedback are:
- It recognises what has been done well and then gives specific advice on what step the learner can take next. General comments such as ‘try harder’, ‘improve your handwriting’, or 7/10, do not provide the detail needed.
- It is clear and well-timed. The teacher gives feedback in language the learner understands and it is given when it is most useful.
- It relates to the success criteria and focuses on the key next steps. We may sometimes give too much feedback if we start to comment on presentational features (e.g. spelling) when these were not part of the learning intention.
- It involves action and is achievable.
In all this, the aim of assessment for learning is to encourage our students to increasingly think for themselves, and have the ability and desire to regulate their own learning.
Gordon Stobart is an assessment expert that has contributed to the latest Position Paper for Oxford University Press, ‘Assessment for Learning’. Download the paper today to learn about effective feedback, close the gap between where your learners are and where they need to be, and get access to exclusive professional development events!
Gordon Stobart is Emeritus Professor of Education at the Institute of Education, University College London, and an honorary research fellow at the University of Oxford. Having worked as a secondary school teacher and an educational psychologist, he spent twenty years as a senior policy researcher. He was a founder member of the Assessment Reform Group, which has promoted assessment for learning internationally. Gordon is the lead author of our Assessment for Learning Position Paper.
Don’t look now – the CEFR is in your classroom
Working in language education, it’s quite hard to escape from the CEFR, or Common European Framework of Reference for Languages. It crops up in courses at language schools and in publishers textbooks. International testing bodies label their products as suitable for levels called A2, B1+, or C1.
Ministries of education around the world are vying with each other to set the most demanding targets for the percentage of school children who will reach B2 in the languages they study by the time they graduate. People applying for a Tier 2 visa to do skilled work in the UK need a B1 level certificate in English in reading, writing, speaking and listening. If looking for work using their German language skills, applicants might be asked by their future employer to demonstrate at least an A1 level for unskilled work, B1 for a service role, or C1 for a professional level job involving meetings and negotiations.
Although it’s clearly important that people involved in language education should have a good understanding of such an influential object, there seems to be a lot of confusion around where the CEFR comes from and even about what exactly it is. Let’s start with the first of those points. The CEFR is not a product of the European Union, but was developed by the Council of Europe, an entirely different organisation which is both older (it was founded in 1949) and much bigger (it has 47 member states, many of which are not EU members, including Norway, Russia and Turkey). Its mission includes protecting human rights, democracy and the rule of law, promoting diversity, and combating discrimination against minorities. It has carried out successful campaigns among its members to end the death penalty and to support the rights of people with disabilities. Its work in language education involves promoting linguistic human rights and the teaching and learning of minority languages.
The Council of Europe and language education
As part of this work, the Council of Europe was pioneering in promoting one of the most revolutionary ideas in language education: the communicative approach. Instead of focussing (as teachers usually did before the 1970s) on what learners knew about a language – how many words or how much grammar – the Council of Europe focussed attention on what learners might actually want to do with the language they were learning – the activities they might need to carry out, and the ideas they might want to express. In 1975 the Council of Europe published Jan van Ek’s Threshold Level. This book defined a level (to become “B1” in the CEFR) that a language learner would need in order to be able to live independently for a while in a country where that language is spoken. In 2001 (the European year of languages) twenty-five years of further work involving extensive consultation with language teachers and academic experts culminated in the publication of the CEFR. This year, the Council of Europe has published a Companion Volume, available online that updates and expands on the original publication.
It is part of the Council of Europe’s educational philosophy that learners should be able to move easily between informal learning, schools, universities, and workplace training courses to pick up the practical skills that they need. Of course, doing this is much easier if everyone shares the same basic terms for talking about teaching and learning. If a ‘Beginner’ level class in school A is like an ‘Elementary’ level class in school B or a ‘Preliminary’ class in school C, and the ‘Starters’ book in textbook series X is like the ‘Grade 2’ book in series Y, life in the English classroom can soon get very confusing for the uninitiated. The CEFR provides a shared language to make it easier for teachers, learners, publishers, and testers to communicate across languages, educational sectors, and national boundaries.
| School A | School B | School C |
| Beginner | Elementary | Preliminary |
Table 1 shows the need for a shared ‘language’ for talking about levels.
Language learning levels, activities, and contexts
One contribution of the CEFR has been to provide terms for levels – running from Basic (pre-A1, A1 and A2), through Independent (B1 and B2) up to Proficient (C1 and C2) – that are defined in terms of what learners at each level can do with the language they are learning. For example, at the A1 level a learner, ‘can use simple phrases and sentences to describe where he/she lives and people he/she knows’, but at B2 ‘can present clear, detailed descriptions on a wide range of subjects related to his/her field of interest’.
| CEFR level A1 | CEFR level B2 |
| ‘can use simple phrases and sentences to describe where he/she lives and people he/she knows’ | ‘can present clear, detailed descriptions on a wide range of subjects related to his/her field of interest’ |
Table 2 gives examples of what students ‘can do’ at two CEFR levels.
Although levels are important, they are only a small part of what the CEFR offers. In fact, the Council of Europe suggests that levels are too reductive and that it is better to consider learners and learning in terms of profiles of abilities. For example, learners may be very effective speakers and listeners (B2 level), but struggle with the written language (A2 level). The CEFR does not follow the traditional “four skills model” of Reading, Writing, Listening and Speaking, but divides language use activities into reception, interaction, production and mediation. The framework also considers the contexts in which people use languages, recognising that learning a language to keep in touch with one’s grandparents is rather different (and suggests a different skills profile) from learning in order to pursue a career in Engineering.
Describing and explaining, not prescribing or imposing
The CEFR is not a test or a syllabus, it is not limited to the learning of indigenous “European” languages and it does not set out what learners should learn. There is no consensus view on what should be learned or what methods should be used and the CEFR is not a recipe book that recommends or requires its users to adopt a certain teaching method. Educational objectives and standards will inevitably differ according to the target language and the learning context; teaching methods will vary according to the local educational culture. What the CEFR does offer is sets of key questions that encourage educators to think about, describe and explain why they choose to learn, teach or test a language in the way that they do. As part of this process, they are encouraged to question their current aims and methods, but selectivity, flexibility and pluralism are seen to be essential. Users choose only those parts of the CEFR scheme that are seen to be relevant in their context. If the illustrative descriptions in the CEFR are not suitable for a particular group, it is clear that they are free to develop alternative descriptions that work better for them – and the CEFR suggests ways of doing just that. Indeed, the new Companion Volume brings together many of the Can-Do descriptors that have been developed since 2001 to fill gaps and expand the scope of the CEFR descriptive scheme.
If you think it’s time you found out more about the CEFR and Companion Volume and how they affect your work, visit the CEFR website to learn more.
Professor Anthony Green is Director of the Centre for Research in English Language Learning and Assessment at the University of Bedfordshire. He has published widely on language assessment and is a former President of the International Language Testing Association (ILTA). His most recent book Exploring Language Assessment and Testing (Routledge, 2014) provides trainee teachers and others with an introduction to this field. Professor Green’s main research interests concern relationships between language assessment, teaching and learning.
Further reading
Need further support, or just want to learn more about language assessment? We recommend that you take a look at these two titles: ‘Language Assessment for Classroom Teachers‘, and ‘Focus on Assessment‘.
Getting more out of your classroom tests Q&A
Thank you to everyone who attended the webinar ‘Get more out of classroom tests’! I talked about how teachers can make more effective use of tests in the classroom (as well as other forms of assessment) as ways of improving learning. I suggested that we need…
- to help learners to see the connections between testing and real language use inside and outside the classroom
- to explore why learners get some questions right and others wrong; how their current performance compares to successful performance; what they understand and what they need help with to improve
- to work with learners to improve their performance and make progress towards their goals
Some very interesting questions came up during the webinar and I’ll do my best to answer them here.
Caterina B asked about how we should pair or group students to work together to discuss questions: should they all be a similar level of ability?
This is a difficult question to answer without knowing the particular group of students. It can be good to group or pair students who are at a similar level. In this case, they may all tend to get the same number of questions right or wrong and it is unlikely that one student will know more answers than the others and do all the work for the group. If all the students feel they are at the same level, they may be more willing to discuss their answers. On the other hand, it can also be challenging for a more successful student to explain why a certain answer is correct – and motivating for the other students to learn from a friend. As with all the ideas I suggested, why not try different combinations and find the one that works best for you?
Maria A D O asked which age groups might be suitable for the teaching strategies I was presenting.
You may be surprised to know that many of these ideas come from primary school classrooms with children of five years old, or even younger. Of course, at that age, children may need more help with learning how to set goals and discuss answers. You might like to visit Shirley Clarke’s website to see some children from my part of the world getting involved in these kinds of activities. Of course, there is no upper limit. University students and other adult learners can also benefit greatly from setting goals, assessing themselves and each other, and monitoring their own progress.
Ruzanna V asked how we can teach classes where individual students all set different individual goals.
One possibility would be to ask the class to arrive at an agreed set of shared goals. This can be a good way of getting them to think about why they are learning a language and promoting discussion (especially if they can do this in the target language: maybe you could give them useful words and expressions to help). Another possibility, although more difficult for the teacher to manage, would be to allow different groups of students to work in class on different activities. One group chooses and reads one book; another group chooses a different book or does some writing instead; a third group practices giving presentations. It is a technique I’ve used often and it can work well with children of different ages, but the students usually need to be trained to do it in small steps.
I was also asked to recommend some further reading.
The British Council has a short article by Deborah Bullock on assessment for learning with links to more detailed resources: www.teachingenglish.org.uk/article/assessment-learning
Professor Anthony Green is Director of the Centre for Research in English Language Learning and Assessment at the University of Bedfordshire. He has published widely on language assessment and is a former President of the International Language Testing Association (ILTA). His most recent book Exploring Language Assessment and Testing (Routledge, 2014) provides trainee teachers and others with an introduction to this field. Professor Green’s main research interests concern relationships between language assessment, teaching, and learning.
Further reading
Need further support, or just want to learn more about language assessment? We recommend that you take a look at these two titles: ‘Language Assessment for Classroom Teachers‘, and ‘Focus on Assessment‘.