Oxford University Press

English Language Teaching Global Blog


1 Comment

Measuring Progress in Academic ESL Classes

Measuring Progress in Academic ESL ClassesLawrence J. Zwier, testing expert and series advisor for Q: Skills for Success, Second Edition, looks at some strategies for measuring student progress in language learning.

Language teachers often discuss the difficulty of measuring how well their students are doing.  A typical comment goes something like, “When you’re testing in a history class (or biology, or law, etc.) it’s easy. They either remember the material or they don’t.” This oversimplifies the situation in “content classes,” where analysis might be just as highly valued as memory, but the frustrated ESL/EFL teacher has a point. Teaching in a language class does not aim to convey a body of knowledge but to develop skills—and skill development is notoriously hard to assess. It’s even harder when the skills are meant for use outside the language classroom, but the only venue in which you can measure IS the language classroom.

However, all is not lost. There are many good, solid principles to apply in measuring how your students are doing. What’s more, they don’t require the assistance of test-construction experts or the statistical skills of a psychometrician. The average ESL/EFL teacher can do the measurement and interpret the results in ways that will have immediate benefits for their students.

The idea that measurement benefits students can get lost in discussions of measuring progress. So often, we think of measurement as serving the educational institution (which needs to promote people, issue grades, and so on) or the teacher (who needs to know how well a certain objective is being met). But it’s an established principle of memory science that frequent measurement (or, more familiarly, testing) is one of the best aids in learning. Researchers at Kent State University tested the recall of several pairs of English-Lithuanian word pairs—that is, they studied how well subjects remembered not just the Lithuanian or English words but also the pairing of those words across languages. The main variable was how often a given subject was tested on the associations of the pairs. The researchers found a clear correlation between the number of “retrievals”—the number of times a participant was required to recall the pairs on tests—and the long-term memory of the pairs.

You may be sensing a dichotomy you’ve noticed before, that of formative vs. summative evaluation. Summative evaluation comes after a period of learning and is meant to see how much learning took place. Think final exams, midterms, end-of-unit tests, and so on. Formative evaluation occurs during the period of learning and is a part of that learning. The test is a teaching tool. Each type of testing has its place. There’s nothing wrong with summative testing, and the educational system would lose structure without any of it. Many students would also lose motivation, because—love them or hate them—big tests have a way of making people work. But the Kent State research we mentioned clearly shows that formative testing is not just some touchy-feely distraction. Measuring your students often is instructive—both for you and for them. You can easily find examples of formative-assessment activities through a Web search; a good link to start out with is http://wvde.state.wv.us/teach21/ExamplesofFormativeAssessment.html.

Here is a brief look at some important principles in measuring the progress of ESL/EFL students.

Use many small measures, not just a few big ones. This is just common sense. If you rely on two or three measures during the course of a semester, your measurements are much more vulnerable to factors that skew the results—the students’ health, the students’ moods, problems with classroom technology, your own fallibility in writing test items, and so on. If your program requires some big tests, so be it. Make every effort to add other little tests/quizzes along the way as well—and have them influence the students’ grades in a significant way. Also, share the results of these measurements with your students.  An especially effective technique is to make these smaller tests and their grading echo what happens in the larger tests. That way, the frequent tests offer not only periodic retrieval of language points but also practice with the format of the larger test.

Don’t administer what you can’t evaluate. You can’t give frequent assessments if it takes you five hours to grade each one. Most of your questions in measurements should be discrete-point items. This means that the questions have clearly identifiable correct answers that are limited in scope.  Yes, I love seeing my students produce essays or get in front of class to give 5-minute presentations. However, I can’t assess—or give meaningful feedback on—more than two or three such long-form outputs in a semester. Especially when I’m teaching reading or listening, I have to depend on multiple-choice questions, true/false, fill-in, matching, and all those other limited-output formats. What you may have a harder time believing is that short-form questions are appropriate in writing and speaking classes as well. A writing student can demonstrate many skills in two or three sentences. A speaking student can demonstrate a lot by speaking for 45 or 60 seconds—as they do on the Internet-based TOEFL.

Avoid unnecessary interference from other skills. This dovetails with the previous point. If I am trying to measure reading comprehension—a very abstruse target, if you think about it—I don’t want the student’s weaknesses in writing, speaking, or even listening to get in the way. I want to ask a comprehension question that can tell me something about the student even if the student cannot compose a good sentence, articulate a spoken answer, or comprehend a long, spoken introduction. Give me something that requires minimal output to indicate the handling of input. Of course, there is no perfect question, nothing that can get me inside that student’s head and point out relevantly firing neurons, but a simply worded question that requires circling a letter, or writing T/F, or drawing a line is less likely to be muddied by other factors than one that requires complex answers. Gillian Brown and George Yule noted long ago how hard it is to assess actual listening comprehension. They pointed out that a listener’s “personal representation of the content of a text” is “inside the student’s head and not directly available for extraction and objective examination.” Simplify your attempts to examine it by avoiding obscurant factors.

Beware viral items. Digital technology makes test security harder every year. And don’t assume that student lore on the Internet concerns itself only with the big boys—the large, high-stakes tests. If you’ve handed out a piece of paper with a test question on it, there’s a decent chance that it now, somewhere, roams the pastures of the Web. If you were not terribly observant during the test, a student may have snapped a cell-phone picture of it.  Even if you were hawkishly watching, many students, by the time they reach 18 or so, have prodigious memories and a tradition of getting together beforehand to divvy up the memorization of a test: “You take questions 1 – 3, Sam will take 4 -7, and I’ll take 8 -10.” My colleagues and I have adapted by just not re-using any old material in important measures of progress. For quick practices with nothing on the line, I might not care. However, each truly important measurement instrument is a new one—though perhaps based on an old one, with answers re-jigged and re-ordered. (Such reshuffling reduces the amount of writing I have to do.)

Be your own grumpy editor. I work frequently with the best item writers in the ESL/EFL field. One mark of a good item writer is that he/she assumes there’s something wrong with the first draft of anything. After you write a measurement item, let it sit for a few hours or a day. Then go back to it carrying a nice, sharp boxcutter. You’ll be surprised how often you discover that the question doesn’t really address what you want to assess, or that there are actually two possible correct answers in your set of multiple choice options, or that the reading/listening passage doesn’t clearly say whether a measurement statement is true or false.  Good measurement is impossible without good items. It’s worth the effort to slash and rewrite.

 

References and Further Reading

Association for Psychological Science. “Testing improves memory: Study examines why memory is enhanced by repeated retrieval.” ScienceDaily. 16 June 2011. www.sciencedaily.com/releases/2011/06/110615171410.htm

Brown, Gillian, and George Yule. Teaching the Spoken Language: An Approach Based on the Analysis of Conversational English.  Cambridge, UK: Cambridge University Press. 1983

West Virginia Department of Education, “Examples of Formative Assessment.” Accessed 31 October 2014, at http://wvde.state.wv.us/teach21/ExamplesofFormativeAssessment.html.


2 Comments

Assessment in the multi-level classroom

Frustrated student at work in classroomIt can be tricky to test classes of students who come from very different learning backgrounds. Stacey Hughes, teacher trainer in the Professional Development team at Oxford University Press, offers some advice.

Testing and assessment are important in any classroom. In addition to the obvious goal of finding out if students have learned what is required for the end of term or year, assessment also gives teachers information about what students might need more work on. It can also motivate students to study, giving them a sense of achievement as they learn (Ur:1996).

A multilevel class poses additional challenges to the teacher. It could be argued that all classes to a certain extent are multi-level. However, for the purpose of this article, multi-level will be defined as those classrooms with students who come from very different learning backgrounds, or those in which students have very different levels of proficiency. Assessment in these situations needs to be fair for all students and needs to provide enough challenge or support so as not to bore or overstretch students. Here are some ideas for assessment:

1. Set individualised targets

You could consider setting individualised targets (or get your students to set their own). In order to assess students on their achievement of their target, you may need different assessment criteria and this difference needs to be made clear at the outset. As long as the assessment is not part of a final grade (and instead part of ongoing assessment for the purposes outlined above), students will be unlikely to opt for an easier option than they are capable of. Here are some examples:

a) Choose the 5 key words you think are absolutely necessary for all students to learn, several more that would be good for them to learn and a final few that would be great if they could learn. Assign the words to each student (or get them to choose their own level of challenge). Assess students on the words you have assigned or that they have chosen.
b) Set different word limits for paragraphs and essays. At the lowest level, ask students to write a 50-word paragraph. The next level might be a 100-word paragraph while the highest level might be two 100-word paragraphs. A similar design can be made for speaking tasks.
c) Set different criteria for writing or speaking. If a student’s work is hard to read because of spelling, set the target of improving spelling and assess only on that. Another student might not have problems spelling, but may have poor subject/verb agreement, so instead, make this the focus of the assessment.

2. Break your targets into manageable chunks

Create a master list of targets for yourself, and assign 2-3 targets at a time for students.
This has the effect of making learning manageable. Some students may already be quite good at word stress, for example, while others, possibly from L1 interference, might need to work a lot on their pronunciation.

Your master list should be comprehensive and cover all language areas. For pronunciation, it might include:

a) Correct word stress on vocabulary words
b) Clear distinction between /s/, /z/ and /Id/ in past tense
c) Rising intonation on yes/no questions

For speaking, it might look like this:

a. Can ask and respond to questions about likes and dislikes
b. Can speak about likes and dislikes for 1 minute
c. Can give reasons or examples for likes and dislikes

3. Differentiate between assessment questions and let students choose their level of challenge

Again, this will work best if the assessment is not marked or graded.

a) For a reading or listening assessment, provide many different questions, and ask students to answer more for higher levels of challenge. For example, the Level 1 challenge could be to answer questions 1-3, Level 2 could be questions 1-5 and Level 3 could be questions 1-7. If you set this kind of task, make sure each question increases in difficulty.
b) Allow for levelling in answers. Level 1 challenge answers could be 1-2 words or yes/no questions, while level 3 challenge answers could be whole sentences or open-ended questions.
c) Provide optional hints for those who need it. Students could choose to do the assessment with or without hints, for example. This works well in conjunction with digital or online assessments.

4. Provide a place for students to go next

At the end of the term or school year, it is customary to test whether or not students have reached the learning goals for the course. For those students who aren’t yet ready to progress, make sure they have a class to go into that isn’t just a repeat of the level they have just done. Some courses provide a middle level between levels that caters for those weaker students, for example, English File 3rd edition Intermediate Plus. In this way, weaker students don’t feel penalised, but feel a sense of achievement in having completed a level.

Assessing students in a multi-level class differently according to their level can benefit all students by providing the right amount of challenge. This can be encouraging and create a positive atmosphere of achievement in the classroom. I hope you enjoy trying out some of these ideas.

References & Further Reading

English club. (n.d.). Teaching multilevel classes. Found at: https://www.englishclub.com/teaching-tips/teaching-multi-level-classes.htm.
Accessed 30/04/14.

Ur, P. (1996). A course in language teaching: practice and theory. Cambridge: CUP.

This article first appeared in the May 2014 edition of the Teaching Adults Newsletter – a round-up of news, interviews and resources specifically for teachers of adults. If you teach adults, subscribe to the Teaching Adults Newsletter now.


Leave a comment

Discover the NEW International Express Tests and Teacher’s Guides

Students Sitting at Desks and WritingBruce Wade, Managing Editor of International Express, introduces his upcoming webinar on 11th September about the new International Express Tests and Teacher’s Guides.

The new International Express launched earlier this year, and now, plenty of extra resources to support the course are available for free on Oxford Teachers’ Club.  In this webinar, I’ll be exploring how the new Teacher’s Guides can help you quickly plan your lessons, and I’ll show you around the new tests and Student Progress Report, which help you to regularly and quickly check your students’ performance.

Plan your lessons in a flash

International Express comes with extensive extra resources including photocopiable activities, videos for every unit, and worksheets to support each video – so you won’t be short of material, but how do you make the best use of it all?  We’ve developed Teacher’s Guides for every level, which give a clear, one-page overview of the course, meaning that you can see all the syllabus items, target language and skills, and resources in one go. I’ll be exploring how you can use these to plan your lessons quickly and easily.

Regularly check students’ progress

Tests are an important part of every course, and International Express tests provide comprehensive coverage of all the language in the Student’s Book.  Most test items are written as A‒B exchanges to reflect the communicative nature of the course.   There is a separate test for each section so teachers can test their students after completing a section, or a unit.  I’ll explain the different ways you can use these with your class, and we’ll look at how you can analyse the results to make direct comparisons of your students’ performance across sections, and whole units.

Analyse students’ performance

We developed the unique Student Progress Report to help you measure students’ performance unit-by-unit, and across different skills.  I’ll explain how you can use this tool to see how a student is performing across the four sections of a unit, and we’ll look at how you can customise it, for example, by drawing different types of graphs, or by adding comments on your students’ performance.

I look forward to helping you making the most of all of these resources on 11th September.  In the meantime, you can take a look at them on Oxford Teachers’ Club – you just need to sign in with your usual log in details.


2 Comments

Child-friendly testing for young learners

Girl sat at computer smilingVerissimo Toste, an Oxford teacher trainer, looks at how you can make testing a child-friendly experience for your young learners, and useful for you.

“Testing young learners? Really? Seriously? Why?”

That’s usually my reaction when I hear teachers talking about testing young learners.

“So, how do you decide what to teach them? How do you know how to teach them? Testing young learners gives you important information.”

As a friend said this to me I realised my problem was with the word “testing”. For me, testing is judging and labelling, not teaching. Of course, I have always gathered information about my learners and used it to help me teach better. Testing is one way to gather information, but testing young learners needs to be a friendly, positive experience for them. You need to consider their age, use bright colours and fun images, and give them a sense of achievement for having gone through the experience.

Making testing a positive experience

In her book, Teaching Young Language Learners, Annamaria Pinter writes: “In order to understand what children have learnt, teachers may need to use a variety of assessment methods.” Along with observation, portfolios, and project work, testing can be a valuable tool, providing teachers with information quickly and easily. It is important, however, for teachers to take out any of the stress and tension usually associated with testing and work to make it a positive and motivating part of the learning experience.

Understanding the range of abilities in your class

The test also needs to be useful. After all, you are, in essence, gathering information about your learners to help you teach better. Firstly, information from a test can help a teacher place learners in groups of similar abilities, either as a class, or as groups within a class. Knowing the mix of levels in a class or a group, or the strengths and weaknesses of an individual student can help a teacher provide the right kind of support that motivates each student to learn.

Using the results to inform your teaching

This brings up the point of differentiated teaching. A test can provide teachers with important information about each of their students. Who is strong in their use of the language? Who is weak in listening? When listening, do they understand the gist of what they are listening to? Do they grasp the details? Who may have difficulty with vocabulary, or grammar? Having the answers to these questions can help a teacher target their teaching to the needs of the class.

To find out how to make placement testing a fun and positive experience for your young learners, whilst also giving you accurate and reliable results to help you target your teaching, watch our webinar entitled ‘An introduction to the Oxford Young Learners Placement Test.

 


10 Comments

Get the most out of group, peer, and self-assessment

Adult woman assessing another womanStacey Hughes, former teacher and current teacher trainer in the Professional Development team at Oxford University Press, shares some practical tips on getting your students to assess themselves, and each other.

We all need a fresh set of eyes sometimes. Successful writers or speakers often ask a peer for feedback. So, what is group, peer and self-assessment in the language classroom? For starters, it is not necessarily part of grading – we are not asking students to give each other a mark that will then count towards their grade. What it should involve is peer-to-peer communication about what is good about a piece of work and how it can be improved. It involves some learner training and it is important that assessment is not aimed at the individual, but at the work.

In this article, I’ll give examples of how group assessment might work for a presentation, how peer assessment might work for writing and how self-assessment might work in several contexts.

Preparing students to peer assess

Peer assessment may be new to students, so it is important to show them why it is a good idea, how it can benefit them and, most importantly, agree to some behavioural guidelines. Talk to your students about the benefits of peer assessment and get the class to draw up a list of guidelines. Some key points are below:

  • Peers can help us review our work so we can get a better grade.
  • Learning to help each other is a skill that will be used in a job.
  • Assessing a peer’s work can help us develop our own work by making us more aware of how a piece of writing or speaking affects the audience.

Example guidelines:

  • Comment on what is good.
  • Don’t make personal comments – just comment on the work.
  • Don’t judge. Be helpful.
  • Don’t just say something is bad or good. Say what is bad or good and why it’s bad or good.
  • Try to explain how it could be better.

Group assessment – group assesses group

As part of preparing for group presentations, set a date when all groups present to another group. You may need to send some groups out or if your classroom is suitable, just group them around the room. Give each member of the groups a sheet to fill in (see the example below) to help guide their comments. For a presentation, there are a number of different aspects that you might want groups to comment on. Create a feedback sheet that reflects what you have taught and what you will be grading on. The example below is fairly comprehensive and includes example student comments.

Please write a comment
Structure Is it clear?
Is there an introduction?
Is there a conclusion?
You need an introduction. I wasn’t sure what you were going to talk about. You could say what your conclusion is. The main body was clear.
Content Clear?
Engaging?
Did each person’s contribution connect with the others’?
Did each person contribute equally?
I really liked your topic. It was interesting. Maybe you could refer to each other’s section to make the presentation feel more connected. I think everyone spoke for the same amount of time.
Visuals Clear?
Did they add to the content?
The visuals were clear and made the presentation more interesting.
Body language & voice Confident?
Volume?
Eye contact?
Raul – very confident; good eye contact
Maria – I couldn’t hear you well; please speak more loudly Nida – maybe more eye contact; maybe use note cards instead of reading from a paper
Language & pronunciation Clear?
Key words pronounced clearly?
Mostly clear. Raul, I didn’t understand what you said about the water on the roads. Nida – you were reading, so it didn’t sound natural. Maybe practice in front of a mirror.
Answering questions How well did the group answer questions? You answered questions well. You have good knowledge of the topic.

Peer assesses peer

As part of the writing process, ask peers to assess each other’s written work. Sometimes students think that they can’t judge another’s work because they think their English isn’t good enough. To help students realise that their contribution can be valuable, make sure have clear guidelines for what to look for in each piece of writing. Tell them that their goal is not to find every grammar mistake, but to just comment from the reader’s perspective.

The example below (with example student comment) is based on a paragraph about a holiday.

Please write a comment
Structure Clear?
Topic sentence?
The structure is OK. You need a topic sentence. Your sentences are very short. You need to combine sentences 1 and 2 and 3 and 4. Maybe you can use and and but.
Content Does it answer the question?
Does it make sense?
Why is it interesting?
You wrote about your holiday, but not about your feelings. You need to include your feelings. It makes sense. It is interesting because I have never been to Thailand.
Language Vocabulary
Grammar
Punctuation
Spelling
You used good words like hotel reception and flew. Grammar is good, but past of take is took. You need to capitalise the city. Spelling is good.
Presentation Indented paragraph handwriting You should indent the first sentence. Your handwriting is very clear and neat.

Self-assessment

Self-assessment is not only useful as part of the writing process, but can also help students see the progress they are making.

The following simple checklist is an example of how to raise a student’s awareness of what they should be including in writing. It also gives them guidance on how to go about editing their work. It is based on writing a summary and review of a story.

  • Have you got two paragraphs?
  • Is the first paragraph a summary of the plot of the story?
  • Is the second paragraph about your views on the story?
  • Did you give reasons for your opinions?
  • Do you have topic sentences?
  • Look at your grammar: did you use present tense?
  • Look at your vocabulary: did you use some of the words you learned to describe plot? Did you use words like, because and for example?

Students can also assess their speaking performance in pair and group work. This could help motivate students to speak in English when it is often more natural to use the L1. Make a simple checklist which highlights the goals for speaking in pairs or groups:

  • I spoke in English
  • I asked another person a question
  • If I didn’t understand, I asked for clarification
  • If someone wasn’t speaking, I asked them for their opinion
  • When I didn’t know what to say, I said it another way

Can-do statements are a great way for students to assess whether or not they have achieved language aims. They should be very specific; for example, they can be directly linked to a unit. Collectively, these will help students see the progress they have made over time.

I have finished unit X and I can:

  • Use the past tense to talk about what I did yesterday
  • Use words like, last week, yesterday, a month ago in a sentence
  • Ask someone questions about what they did last week/ last month/ last year
  • Understand someone telling a story about their problems last week on the train

This article first appeared in the March 2014 edition of the Teaching Adults Newsletter – a round-up of news, interviews and resources specifically for teachers of adults. If you teach adults, subscribe to the Teaching Adults Newsletter now.

Follow

Get every new post delivered to your Inbox.

Join 2,544 other followers