Oxford University Press

English Language Teaching Global Blog


Leave a comment

Adaptive testing in ELT with Colin Finnerty | ELTOC 2020

OUP offers a suite of English language tests: the Oxford Online Placement test (for adults), the Oxford Young Learners Placement Test, The Oxford Test of English (a proficiency test for adults) and, from April 2020, the Oxford Test of English for Schools. What’s the one thing that unites all these tests (apart from them being brilliant!)? Well, they are all adaptive tests. In this blog, we’ll dip our toes into the topic of adaptive testing, which I’ll be exploring in more detail in my ELTOC session. If you like this blog, be sure to come along to the session.

The first standardized tests

Imagine the scene. A test taker walks nervously into the exam room, hands in any forbidden items to the invigilator (e.g. a bag, mobile phone, notepad, etc.) and is escorted to a randomly allocated desk, separated from other desks to prevent copying. The test taker completes a multiple-choice test, anonymised to protect against potential bias from the person marking the test, all under the watchful eyes of the invigilators. Sound familiar? But imagine this isn’t happening today, but over one-and-a-half thousand years ago.

The first recorded standardised tests date back to the year 606. A large-scale, high-stakes exam for the Chinese civil service, it pioneered many of the examination procedures that we take for granted today. And while the system had many features we would shy away from today (the tests were so long that people died while trying to finish them), this approach to standardised testing lasted a millennium until it came to an end in 1905. Coincidentally, that same year the next great innovation in testing was established by French polymath Alfred Binet.

A revolution in testing

Binet was an accomplished academic. His research included investigations into palmistry, the mnemonics of chess players, and experimental psychology. But perhaps his most well-known contribution is the IQ test. The test broke new ground, not only for being the first to attempt to measure intelligence, but also because it was the first ever adaptive test. Adaptive testing was an innovation well ahead of its time, and it was another 100 years before it became widely available. But why? To answer this, we first need to explore how traditional paper-based tests work.

The problem with paper-based tests

We’ve all done paper-based tests: everyone gets the same paper of, say, 100 questions. You then get a score out of 100 depending on how many questions you got right. These tests are known as ‘linear tests’ because everyone answers the same questions in the same order. It’s worth noting that many computer-based tests are actually linear, often being just paper-based tests which have been put onto a computer.

But how are these linear tests constructed? Well, they focus on “maximising internal consistency reliability by selecting items (questions) that are of average difficulty and high discrimination” (Weiss, 2011). Let’s unpack what that means with an illustration. Imagine a CEFR B1 paper-based English language test. Most of the items will be around the ‘middle’ of the B1 level, with fewer questions at either the lower or higher end of the B1 range. While this approach provides precise measurements for test takers in the middle of the B1 range, test takers at the extremes will be asked fewer questions at their level, and therefore receive a less precise score. That’s a very inefficient way to measure, and is a missed opportunity to offer a more accurate picture of the true ability of the test taker.

Standard Error of Measurement

Now we’ll develop this idea further. The concept of Standard Error of Measurement (SEM), from Classical Test Theory, is that whenever we measure a latent trait such as language ability or IQ, the measurement will always consist of some error. To illustrate, imagine giving the same test to the same test taker on two consecutive days (magically erasing their memory of the first test before the second to avoid practice effects). While their ‘True Score’ (i.e. underlying ability) would remain unchanged, the two measurements would almost certainly show some variation. SEM is a statistical measure of that variation. The smaller the variation, the more reliable the test score is likely to be. Now, applying this concept to the paper-based test example in the previous section, what we will see is that SEM will be higher for the test takers at both the lower and higher extremes of the B1 range.

Back to our B1 paper-based test example. In Figure 1, the horizontal axis of the graph shows B1 test scores going from low to high, and the vertical axis shows increasing SEM. The higher the SEM, the less precise the measurement. The dotted line illustrates the SEM. We can see that a test taker in the middle of the B1 range will have a low SEM, which means they are getting a precise score. However, the low and high level B1 test takers’ measurements are less precise.

Aren’t we supposed to treat all test takers the same?

                                                                                            Figure 1.

How computer-adaptive tests work

So how are computer-adaptive tests different? Well, unlike linear tests, computer-adaptive tests have a bank of hundreds of questions which have been calibrated with different difficulties. The questions are presented to the test taker based on a sophisticated algorithm, but in simple terms, if the test taker answers the question correctly, they are presented with a more difficult question; if they answer incorrectly, they are presented with a less difficult question. And so it goes until the end of the test when a ‘final ability estimate’ is produced and the test taker is given a final score.

Binet’s adaptive test was paper-based and must have been a nightmare to administer. It could only be administered to one test taker at a time, with an invigilator marking each question as the test taker completed it, then finding and administering each successive question. But the advent of the personal computer means that questions can be marked and administered in real-time, giving the test taker a seamless testing experience, and allowing a limitless number of people to take the test at the same time.

The advantages of adaptive testing

So why bother with adaptive testing? Well, there are lots of benefits compared with paper-based tests (or indeed linear tests on a computer). Firstly, because the questions are just the right level of challenge, the SEM is the same for each test taker, and scores are more precise than traditional linear tests (see Figure 2). This means that each test taker is treated fairly. Another benefit is that, because adaptive tests are more efficient, they can be shorter than traditional paper-based tests. That’s good news for test takers. The precision of measurement also means the questions presented to the test takers are at just the right level of challenge, so test takers won’t be stressed by being asked questions which are too difficult, or bored by being asked questions which are too easy.

This is all good news for test takers, who will benefit from an improved test experience and confidence in their results.

 

                                                                                            Figure 2.


ELTOC 2020

If you’re interested in hearing more about how we can make testing a better experience for test takers, come and join me at my ELTOC session. See you there!

 


Colin Finnerty is Head of Assessment Production at Oxford University Press. He has worked in language assessment at OUP for eight years, heading a team which created the Oxford Young Learner’s Placement Test and the Oxford Test of English. His interests include learner corpora, learning analytics, and adaptive technology.


References

Weiss, D. J. (2011). Better Data From Better Measurements Using Computerized Adaptive Testing. Testing Journal of Methods and Measurement in the Social Sciences Vol.2, no.1, 1-27.

Oxford Online Placement Test and Oxford Young Learners Placement Test: www.oxfordenglishtesting.com

The Oxford Test of English and Oxford Test of English for Schools: www.oxfordtestofenglish.com


2 Comments

5 Easy Classroom Activities Involving Movie Trailers

Teenagers watching a movieAlmost thirty years ago, when I first started teaching, the use of feature films in the classroom was considered a ‘Friday afternoon treat’ – something to give the students as a reward and, perhaps in some cases, to give teachers a chance to catch up on some marking! Some schools used movies randomly and at inappropriate levels, meaning students often got little to nothing in terms of language learning.

Having originally studied Film at university I was always keen to use movies in class, and some years later I ran a series of workshops for teachers on the use of video in the classroom (‘video’ gives an idea of exactly how long ago that was) and how to maximise learning opportunities. I offered a selection of lesson ideas I’d used to good effect in my own classes and now 25 years later, with increased online access to materials that are often not subject to copyright issues within an educational context, I’m sharing a few of those ideas here!

Why use movie trailers?

These ideas will concentrate on movie trailers specifically as promotional tools studios use to get audiences interested in films coming soon. The interest level in trailers is unquestionable – being among the top five forms of video content viewed by users (as an example, Marvel’s Avengers Endgame trailer had 129,527,344 views at the time of writing).

You can find trailers on YouTube or sites like iTunes Movie Trailers. A typical trailer will be between 2 and 3 minutes long, although teaser trailers – those released sometime before the movie’s planned release – will be shorter, and typically give less of the plot away, aiming to create a general mood instead (these can be useful in their own right).

You should ensure you are not breaking any copyright laws in your use of movies in class and be aware of the suitability of the subject matter for the students you are teaching. Note that the majority of trailers will be suitable for class use, but ‘red band trailers’ are those which feature violence and/ or abusive language.

1) Pick a Movie

If you do show movies in your school, as part of a ‘movie club’ or similar, trailers can provide an excellent opportunity to decide what movies are shown, while encouraging students’ analytical and presentation skills!

Method: Show three trailers. Students divide into three groups according to which movie they would prefer to see. Within each group, they decide what it is about their choice that appeals to them. A help guide could be provided with keywords to assist them: words like genre/ stars/ director/ themes etc. After half an hour, selected representatives present their choice reasons to the broader group. After presentations, the choice is put to a class vote.

2) Film Pitch

This is a more ‘drama’ based version of the ‘pick a movie’ idea and uses the concept of familiar social events such as the Cannes Film Festival etc. where filmmakers will try to sell their films to would-be studio buyers.

Method: Show a trailer and ask the students to consider it along the lines of stars/ genre/ look and feel/ what they saw. Invite students to write a ‘pitch’ for the movie, as if they were the maker. Their job is to pick out the most positive points about it and why people would want to see it. The second group of two/ three students will act as a ‘movie producer panel’ who can buy a movie. Their job is to decide whether they would buy the film in question, based on the quality of the presenters’ persuasive powers.

3) Red Light/ Green Light

This is a variation on Film Pitch, which doesn’t even need a trailer!

Method: Students in small groups come up with their own ideas for a film, and present a 2-minute ‘pitch’ of it to a panel of students who will decide whether their ‘studio’ will give it the green light (make it), or a red light (turn it down).

4) What Happens Next

The point of a trailer is to give a feel for what the movie will be about, without giving the whole plot away (some do this better than others). They often use ‘tropes’ – a movie language shorthand which allows an audience to see there is enough in the movie that reminds them of things they have previously liked without being ‘exactly the same’. If you have students who are interested and watch movies in their own time and depending on your class subject, this can work as a fun ‘warmer’ exercise.

Method: Show a trailer and ask students what they think will happen in the movie. Students can work in pairs or individually and either fill in a response or call out suggestions (from experience these can be humorous or serious, depending on your class…). If you are using an old movie then you can tell them who was closest (although they may have seen it), if it’s a new film then there will be a period of waiting before the answer is revealed…

5) My Favorite Genre:

… a fun self-study preparation/ classroom presentation project for classes who have an interest in movies.

Method: explain to students what a ‘genre’ means in film terms. This can be a fun classroom warmer to encourage students to take part: in the past I’ve put genre headings up on a board (e.g. Western, Sci-Fi, Horror etc.), and provided post-it notes of terms or words such as ‘Ghost’, ‘Horse’, ‘Time-travel’ etc. then asked students to put them under the appropriate genre. Students can then add their own elements under the genre they think is most fitting.

To extend this ask students to think about what their favourite genre is and tell them to find a trailer which they believe demonstrates this genre. Students can do this in their own time and present it to their classmates, pointing out what ‘genre’ elements it uses. This can lead to interesting discussions around cross-genres and storytelling techniques.

How do you use movies in the classroom? Let us know in the comments below!

 


 

Simon Bewick worked in ELT for 25 years and has watched movies for nearly 50. He is the author of several short story collections for both adults and young adults, available on Amazon. He writes about films, literature and culture on his website bewbob.com


3 Comments

The Complete Guide to Running a Blended Learning Course

Blended learning - students working together on laptopsWhat is blended learning?

Blended learning is both flexible and dynamic. By ‘flexible’, I mean it is not just one thing (a fixed combination of X and Y) but rather, it can be many things depending on your teaching context. By ‘dynamic’, I mean that the components which make up blended learning are constantly changing. A recent incarnation of blended learning, for example, involves students donning headsets and practising a talk in VR (Virtual Reality) in preparation for giving a presentation in real life.

The classic definition of blended learning combines teaching in a ‘bricks and mortar’ classroom with web-based learning. The latter is usually ‘online’ but could be ‘offline’ and might not even involve the Internet at all, such as doing exercises on a CD-ROM or using a ‘native’ app – an app which ‘lives’ in your mobile phone and does not require a Wi-Fi connection to function.

Another approach to blended learning involves blending the use of print and digital resources, effectively combining the traditional and the new, analogue and digital.

 

When should teachers use blended learning?

In a very narrow definition of blended learning (such as face-to-face plus online) the answer to this question is: when studying online is a realistic, feasible option. In a broader definition of blended learning, such as that described by Sharma and Barrett ‘face-to-face plus an appropriate use of technology’ (Pete Sharma & Barney Barrett, Blended Learning, Macmillan, 2007), the answer is: ‘All the time!’ In other words, teaching in this new digital age should use the technologies which students meet in their everyday lives, such as the Internet, laptop, smartphone and tablet.

 

Why blend?

There are many reasons why teachers decide to run a blended learning course, as opposed to (say) a 100% classroom course like those I ran when I first started teaching, or a 100% online course.

One is time. There’s simply not enough time in a course to cover everything. Moreover, some language areas are really suited to be studied outside the classroom. Extensive reading and practising difficult phonemes, for instance.

Combining the best of the classroom (live interaction with the teacher and classmates) and the best of technology (anytime, anywhere guided practice) in a principled way can produce a ‘better’ course for students. In other words, the best of both worlds.

 

What is the value of blended learning?

Flexibility is one advantage. Students taking a blended learning course are frequently offered choices. We all know a class of 12 comprises 12 individuals, displaying different learning preferences. Students can match their path through the material to suit their own learning style and approach.
Similarly, from the teacher’s point of view, blended learning enables the implementation of ‘differentiation’.

We are all familiar with the restrictions imposed by the teaching timetable. The English language lesson is at 16.00 on Thursday. Yet this is the age of u-learning, ubiquitous learning. The distant part of a blended learning course can be done anywhere, anytime – in a coffee shop with Wi-Fi, at the airport, in a hotel … , this ‘best of both worlds’ (the classroom and online) is a key feature and benefit of blended learning.

 

Different approaches to blended learning

The approaches taken to blended learning are as many and varied as the different types of teaching: YL (young learners), business English, CLIL (content and language integrated learning). One common approach would be to issue the students with a printed coursebook and have them use the code on the inside to access their online digital materials. I focus particularly on this approach in my series of articles on running a blended learning course.

 

Different types of digital activities

Here’s a snapshot of the vast range of tools available for blended learning:

 

  • a vocabulary memory game on an app to review new language
  • a podcast; students can listen as many times as they wish, using the pause and the slider to listen intensively to selected parts
  • a video, with on-demand sub-titles or a transcript
  • a discussion forum; students answer a question before their in-class lesson. The additional time helps develop critical thinking skills and contrasts the real-time pressure to reply in the classroom

 

How to run a blended learning course

Looking for some practical advice and tips on running a blended learning course? Read my complete guide to help you prepare, set-up and run a blended learning course:

 

Download the guide

 

References

Blended Learning, Pete Sharma & Barney Barrett (Macmillan, 2007)

 


 

Pete Sharma is a teacher trainer, consultant and ELT author. He works as a pre-sessional lecturer in EAP (English for Academic purposes) at Warwick University, UK. Pete worked for many years in business English as a teacher trainer and materials writer. He is a regular conference presenter at IATEFL (International Association of Teachers of English as a Foreign Language) and BESIG (Business English Special Interest Group) conferences and has given plenary talks and keynote speeches at conferences around the world. Pete is the co-author of several books on technology including Blended Learning (2007), 400 Ideas for Interactive Whiteboards (2011) in the Macmillan ‘Books for Teachers’ series, and How to Write for Digital Media (2014), and most recently Best Practices for Blended Learning. Pete was the Newsletter Editor of the IATEFL CALL Review (2008-2009) and has a Masters in Educational Technology and ELT from Manchester University.


2 Comments

5 Ways to Improve Feedback in your Classroom

Teacher and student high-fivingEffective feedback is the key to successful assessment for learning, and can greatly improve your students’ understanding. So how can you ensure that your feedback is as effective as possible? You need to understand what level your students are at and where they need to improve. Your students will also find your feedback more useful if they understand the purpose of what they are learning and know what success looks like.

 

Try these 5 tips to improve feedback in your classroom:

1. Ask questions to elicit deeper understanding

Most questions asked in the classroom are simple recall questions (‘What is a noun?’) or procedural questions (‘Where’s your book?’). Higher-order questions require student to make comparisons, speculate, and hypothesize. By asking more of these questions, you can learn more about the way your students understand and process language, and provide better feedback.

2. Increase wait time

Did you know that most teachers wait less than a second after asking a question before they say something else? Instead of waiting longer, they often re-phrase the question, continue talking, or select a student to answer it. This does not give students time to develop their answers or think deeply about the question. Try waiting just 3 seconds after a recall question and 10 seconds after a higher-order question to greatly improve your students’ answers.

3. Encourage feedback from your students

Asking questions should be a two-way process, where students are able to ask the teacher about issues they don’t understand. However, nervous or shy students often struggle to do so. Encourage students to ask more questions by asking them to come up with questions in groups, or write questions down and hand them in after class.

4. Help students understand what they are learning

Students perform better if they understand the purpose of what they are learning. Encourage students to think about why they are learning by linking each lesson back to what has been learned already, and regularly asking questions about learning intentions.

5. Help students understand the value of feedback

If students recognise the standard they are trying to achieve, they respond to feedback better and appreciate how it will help them progress. Try improving students’ understanding by explaining the criteria for success. You can also provide examples of successful work and work that could be improved for your students to compare.

 

Did you find this article useful? For more information and advice, read our position paper on Effective Feedback:

Download the position paper

 

Chris Robson graduated from the University of Oxford in 2016 with a degree in English Literature, before beginning an internship at Oxford University Press shortly afterwards. After joining ELT Marketing full time to work with our secondary products, including Project Explore, he is now focused on empowering the global ELT community through delivery of our position papers.


7 Comments

5 English Teaching apps for 21st Century ESL Teachers

Language learning no longer stops when students leave the classroom.

Smartphones allow language learners to carry the entire English language around with them in their pocket, soaking up new vocabulary through music, video, games, and social media.

A new wave of apps have launched designed specifically for those teachers and students keen to harness their mobile devices to create more structured and comprehensive learning experiences outside of the classroom. Make sure you have the latest! Here are 5 essential apps from Oxford University Press that you and your students need to download.

 

  1. Say It: English Pronunciation – Hear the Oxford English model, see the soundwave, then record and compare your pronunciation. Comes with 100 free British English words, 4 tests and 12 sounds, taken from the best-selling English File course and Oxford’s dictionaries. It’s quick, effective and fun to use.

Available on iOS

Available on Android

 

  1. LingoKids – A learning app for students from 2 to 8 years of age, for learning English in a fun, playful way. In Lingokids you’ll find the best English songs for children, the most fun videos with its characters, audiobooks, and printable worksheets for each topic, interactive exercises, and an endless supply of activities to learn over 3,000 words in English. Here are 10 ways you could use LingoKids with your students. If you’re using Mouse and Me, Jump in! or Show and Tell, you can access course content on the app using your coursebook!

Available on iOS

Available on Android

 

  1. Oxford Collocations Dictionary – Perfect for your learners that need to improve their accuracy and fluency, enabling them to express their ideas naturally and convincingly whether spoken or written. The Oxford Collocations Dictionary has over 250,000 word combinations, all based on analysis of the Oxford English Corpus.

Available on iOS

Available on Android

 

  1. Oxford Advanced Learner’s Dictionary – a digitised edition of the Oxford Advanced Learner’s dictionary that has already helped 100 million English language learners worldwide. This app shows learners what words mean, learn how to say them with high quality audio, and know how to use them in example sentences.

Available on iOS

Available on Android

 

  1. Practical English Usage – Practical English Usage is a world bestseller and a vital reference tool that helps teachers and higher-level learners with common language problems in English. Practical English Usage Fourth Edition is now available as an app, making it quicker and easier to look up the 600+ entries!

Coming soon for Android and iOS devices.

 

Extra apps that are worth exploring.

  • YouTube Kids – YouTube Kids is a safer and simpler way for kids to explore the world through online video – from their favourite shows and music to learning how to build a model volcano, and everything in between. There’s also a whole suite of parental controls, so you can tailor the experience to your family’s needs.
  • TinyTap – TinyTap offers the world’s largest collection of educational games, all handmade by teachers. If you can’t find what you’re looking for…create it yourself! On TinyTap, anyone can turn their ideas into educational games (without having to code) and share them with the world.
  • Google Expeditions – This is a virtual reality teaching tool that lets you lead or join immersive virtual trips all over the world — get up close with historical landmarks, dive underwater with sharks, even visit outer space! Built for the classroom and small group use, Google Expeditions allows a teacher acting as a “guide” to lead classroom-sized groups of “explorers” through collections of 360° and 3D images while pointing out interesting sights along the way. Instant, personalised audio-visual feedback will help your students identify precisely what they need to improve. They can even share the recording and the soundwave image of their pronunciation with you via email, directly from the app.
  • Flipgrid – Flipgrid helps learners of all ages find their voices, share their voices and respect the diverse voices of others. Educators spark discussions by posting Topics to a classroom, school, professional learning community, or public Grid. Students record, upload, view, react, and respond to each other through short videos. Flipgrid empowers student voice and builds global empathy through shared learning processes, stories and perspectives.

Interest in Mobile Apps for English Language Teaching?

Read Nik’s Focus Paper on Mobile Apps for English Language Teaching for more practical tips on mobile learning and useful apps for the ELT classroom!