Oxford University Press

English Language Teaching Global Blog


Leave a comment

Adaptive testing in ELT with Colin Finnerty | ELTOC 2020

OUP offers a suite of English language tests: the Oxford Online Placement test (for adults), the Oxford Young Learners Placement Test, The Oxford Test of English (a proficiency test for adults) and, from April 2020, the Oxford Test of English for Schools. What’s the one thing that unites all these tests (apart from them being brilliant!)? Well, they are all adaptive tests. In this blog, we’ll dip our toes into the topic of adaptive testing, which I’ll be exploring in more detail in my ELTOC session. If you like this blog, be sure to come along to the session.

The first standardized tests

Imagine the scene. A test taker walks nervously into the exam room, hands in any forbidden items to the invigilator (e.g. a bag, mobile phone, notepad, etc.) and is escorted to a randomly allocated desk, separated from other desks to prevent copying. The test taker completes a multiple-choice test, anonymised to protect against potential bias from the person marking the test, all under the watchful eyes of the invigilators. Sound familiar? But imagine this isn’t happening today, but over one-and-a-half thousand years ago.

The first recorded standardised tests date back to the year 606. A large-scale, high-stakes exam for the Chinese civil service, it pioneered many of the examination procedures that we take for granted today. And while the system had many features we would shy away from today (the tests were so long that people died while trying to finish them), this approach to standardised testing lasted a millennium until it came to an end in 1905. Coincidentally, that same year the next great innovation in testing was established by French polymath Alfred Binet.

A revolution in testing

Binet was an accomplished academic. His research included investigations into palmistry, the mnemonics of chess players, and experimental psychology. But perhaps his most well-known contribution is the IQ test. The test broke new ground, not only for being the first to attempt to measure intelligence, but also because it was the first ever adaptive test. Adaptive testing was an innovation well ahead of its time, and it was another 100 years before it became widely available. But why? To answer this, we first need to explore how traditional paper-based tests work.

The problem with paper-based tests

We’ve all done paper-based tests: everyone gets the same paper of, say, 100 questions. You then get a score out of 100 depending on how many questions you got right. These tests are known as ‘linear tests’ because everyone answers the same questions in the same order. It’s worth noting that many computer-based tests are actually linear, often being just paper-based tests which have been put onto a computer.

But how are these linear tests constructed? Well, they focus on “maximising internal consistency reliability by selecting items (questions) that are of average difficulty and high discrimination” (Weiss, 2011). Let’s unpack what that means with an illustration. Imagine a CEFR B1 paper-based English language test. Most of the items will be around the ‘middle’ of the B1 level, with fewer questions at either the lower or higher end of the B1 range. While this approach provides precise measurements for test takers in the middle of the B1 range, test takers at the extremes will be asked fewer questions at their level, and therefore receive a less precise score. That’s a very inefficient way to measure, and is a missed opportunity to offer a more accurate picture of the true ability of the test taker.

Standard Error of Measurement

Now we’ll develop this idea further. The concept of Standard Error of Measurement (SEM), from Classical Test Theory, is that whenever we measure a latent trait such as language ability or IQ, the measurement will always consist of some error. To illustrate, imagine giving the same test to the same test taker on two consecutive days (magically erasing their memory of the first test before the second to avoid practice effects). While their ‘True Score’ (i.e. underlying ability) would remain unchanged, the two measurements would almost certainly show some variation. SEM is a statistical measure of that variation. The smaller the variation, the more reliable the test score is likely to be. Now, applying this concept to the paper-based test example in the previous section, what we will see is that SEM will be higher for the test takers at both the lower and higher extremes of the B1 range.

Back to our B1 paper-based test example. In Figure 1, the horizontal axis of the graph shows B1 test scores going from low to high, and the vertical axis shows increasing SEM. The higher the SEM, the less precise the measurement. The dotted line illustrates the SEM. We can see that a test taker in the middle of the B1 range will have a low SEM, which means they are getting a precise score. However, the low and high level B1 test takers’ measurements are less precise.

Aren’t we supposed to treat all test takers the same?

                                                                                            Figure 1.

How computer-adaptive tests work

So how are computer-adaptive tests different? Well, unlike linear tests, computer-adaptive tests have a bank of hundreds of questions which have been calibrated with different difficulties. The questions are presented to the test taker based on a sophisticated algorithm, but in simple terms, if the test taker answers the question correctly, they are presented with a more difficult question; if they answer incorrectly, they are presented with a less difficult question. And so it goes until the end of the test when a ‘final ability estimate’ is produced and the test taker is given a final score.

Binet’s adaptive test was paper-based and must have been a nightmare to administer. It could only be administered to one test taker at a time, with an invigilator marking each question as the test taker completed it, then finding and administering each successive question. But the advent of the personal computer means that questions can be marked and administered in real-time, giving the test taker a seamless testing experience, and allowing a limitless number of people to take the test at the same time.

The advantages of adaptive testing

So why bother with adaptive testing? Well, there are lots of benefits compared with paper-based tests (or indeed linear tests on a computer). Firstly, because the questions are just the right level of challenge, the SEM is the same for each test taker, and scores are more precise than traditional linear tests (see Figure 2). This means that each test taker is treated fairly. Another benefit is that, because adaptive tests are more efficient, they can be shorter than traditional paper-based tests. That’s good news for test takers. The precision of measurement also means the questions presented to the test takers are at just the right level of challenge, so test takers won’t be stressed by being asked questions which are too difficult, or bored by being asked questions which are too easy.

This is all good news for test takers, who will benefit from an improved test experience and confidence in their results.

 

                                                                                            Figure 2.


ELTOC 2020

If you’re interested in hearing more about how we can make testing a better experience for test takers, come and join me at my ELTOC session. See you there!

 


Colin Finnerty is Head of Assessment Production at Oxford University Press. He has worked in language assessment at OUP for eight years, heading a team which created the Oxford Young Learner’s Placement Test and the Oxford Test of English. His interests include learner corpora, learning analytics, and adaptive technology.


References

Weiss, D. J. (2011). Better Data From Better Measurements Using Computerized Adaptive Testing. Testing Journal of Methods and Measurement in the Social Sciences Vol.2, no.1, 1-27.

Oxford Online Placement Test and Oxford Young Learners Placement Test: www.oxfordenglishtesting.com

The Oxford Test of English and Oxford Test of English for Schools: www.oxfordtestofenglish.com


Leave a comment

Combining the Four Cs | ELTOC 2020

Here we’ll discuss the role of creativity, critical thinking, communication and collaboration in the English language classroom, and suggest some practical ideas for giving students a challenging new take on these familiar concepts.

Global Skills

Life in the twenty-first century can be complex and stressful. Many of the interpersonal and interactive skills that we need in our everyday lives – things such as digital literacies, intercultural competence, and emotional self-regulation – have not always been formally taught in schools. The movement to embrace Global Skills in education is now looking to change that.

OUP’s Position Paper on Global Skills is a concise guide for empowering students inside the classroom – and beyond. It acts as a guide for teachers who would like to help equip their students with strategies for dealing with the challenges and opportunities of twenty-first century life.

From Four Cs to five skills clusters

Global Skills include Communication and Collaboration and Creativity and Critical Thinking as two of the key skills clusters, and these are concepts which will already be familiar to anyone who is acquainted with the Partnership for 21st Century Learning, in which they were grouped together as Learning Skills and referred to as The Four Cs.

OUP’s Global Skills are made up of five distinct skills clusters. If you would like to know more about the other three skills clusters of Global Skills – which are Intercultural Competence and Citizenship, Emotional Self-Regulation and Wellbeing, and Digital Literacies – see the OUP Position Paper.

Fresh perspectives

The skills of communication, collaboration, creativity and critical thinking are as important as ever, not only because they can have a positive impact on language proficiency, but also because they can be applied to the challenges of everyday life.

Communication and Collaboration

Why are they grouped together in Global Skills?

Learning to communicate involves being able to negotiate meaning – something which requires interacting with another person or people. And when we collaborate with someone else, both in the classroom and in real-life contexts, there is usually communication involved. The two skills therefore connect, and can very often be dependent on each other.

Pairwork and groupwork

The easiest way to generate conditions for collaboration is to get students to work together in pairs or in small groups. In order to ensure that there is communication as well, students need to share or exchange ideas in some way. Let’s look at a simple example.

Recalling an image

Show students the image below and ask them to pay attention to the small details. (Image source: Oxford Discover Futures, SB1, p86)

After about thirty seconds, remove the picture from view and get students into pairs. Thirty seconds is not very long, which means that students will probably have only a partial memory of the poster. Ask them to work together and recall as much as they can. Ask if they can remember:

  • the words on the poster – and the colour of each word
  • the shape of the figure – and what each ‘body part’ consists of

Some students are more observant than others – but the ones who remember the most do not always have the English with which to express all the information. For this reason, it is likely that students will use L1 to negotiate the answers to the prompts as they gather the English words that they need in order to complete their lists.

Communication and collaboration in action

This simple task mirrors real-life situations in which we need the help of someone else in order to piece together information and fill in the gaps with our own knowledge. The transfer of information is ‘communication’. The pair work is ‘collaboration’. Students help each other as they complete the task, while also checking each other, and correcting each other, as appropriate. Communication and collaboration go hand in hand.

Creativity and Critical Thinking

Why are they grouped together?

Creativity is the art of thinking – it is based on inspiration, intuition and subjective expression. Critical thinking is the science of thinking – it is based on reason, analysis, and evidence-informed judgements. As skills, they are complementary aspects of thinking outside the box, whether that involves coming up with something new, or seeing something that others have missed. Again, let’s take a look at a simple example.

Comparing posters

Show students all four of the posters related to diet shown below. (Image source: Oxford Discover Futures, SB1, p86)

Now give them the following statements to discuss. Ask them to express their ideas, listen to each other’s views, and then try to reach an agreement, by modifying the statements, if necessary.

  • the posters have nothing in common
  • the posters appeal to emotions, not intellect
  • the posters are intended for children
  • the most effective poster is poster ____
  • the least effective poster is poster ____

Finally, ask them to come up with a new poster of their own, designed to raise awareness of the importance of a healthy diet.

Creativity and Critical Thinking in action

The prompts above do more than check students’ comprehension of the posters; they engage their critical faculties, too. The statements are likely to be divisive, and students might well disagree with each other. Establishing the truth of what they can agree on will require negotiation and compromise, as well as creative recasting of some of the statements. Most interestingly, students will have to consider whether they want to change their initial beliefs in the light of information received from others. That is the kind of critical thinking that can be reached through communicative, collaborative classroom processes.

The final task – the design of a new poster – is an example of a creative task that extends naturally out of the tasks that have preceded it. The task combines language skills and non-language skills, so all students have a chance to make a meaningful contribution. Done collaboratively, it will generate further opportunities for communication, collaboration and critical thinking, too.

Double duty

We don’t need extra lessons to teach global skills, nor do we need to separate language skills from global skills. The activities above demonstrate that the learning tasks of the classroom can be asked to perform double duty: to generate opportunities to practise language and to develop students’ global skills.


ELTOC 2020

I hope this is useful. I’ll be expanding on this in my upcoming session at ELTOC 2020. I look forward to seeing you there!


Edmund Dudley is a teacher trainer, materials writer and teacher of English with more than 25 years of classroom experience. Based in Budapest, he has extensive experience of teaching EFL at both primary and secondary levels. He works with teachers from around the world as a freelance teacher trainer and as a tutor at the University of Oxford’s ELT Summer Seminar. He is the author of ETpedia Teenagers (2018, Pavilion Publishing) and co-author of Mixed-Ability Teaching (2015, Oxford University Press).

 


1 Comment

How can we assess global skills? | ELTOC 2020

Global Skills puzzle pieceWe all want our students to develop the global skills needed for modern life and work. We know that our teaching style, our classroom organisation, and what we expect of our students are critical in this. If we want our students to be collaborative and creative we have to provide opportunities for cooperation and problem-solving. However, any attempt to assess these skills raises ‘why?’ and ‘how?’ questions. During my session at ELTOC 2020, I will seek to answer some of these. In the meantime, here’s a brief summary of my approach:

Why do we need to assess this kind of learning?

  1. To signal its importance in the modern world. Language learning cannot be separated from functioning effectively in society. Global skills encourage sensitivity to the needs of others, problem-solving, and how to communicate effectively with those from different cultures. Assessing global skills shows we value them.
  2. To convince students, particularly those who are driven by external rewards, that these skills are important and should be attended to.
  3. Because their assessment helps students know how well they are doing in these skills. It becomes the basis of feedback to students on how well they are doing and what they need to do next to improve.

How do we assess global skills?

Global skills are broad and complex so we need to assess them in ways that does justice to this. If we want to capture performance in a global skill, some conventional assessment methods might not be fit-for-purpose. A multiple-choice test assessing creativity (and there have been some) won’t capture what is important. Nor would giving a mark for social responsibility and well-being be appropriate.

Instead, we will need to use more informal ways of gathering evidence in order to make more general holistic judgements about progress. These are the result of regular observations of student performance and continuous monitoring of their progress. This does not involve lots of extra record-keeping by teachers, it relies on their professional skills of both knowing what the skills involve and informally monitoring individuals’ performance.

As part of our students’ development of global skills we can put the responsibility for gathering evidence of performance on the students. What can they claim they have done to demonstrate a particular cluster of skills? Can they provide evidence of, for example, of creativity and communication? The very act of doing this may be evidence of emotional self-regulation and wellbeing.  

One of the best ways of capturing their achievements is for students to develop individual portfolios. These can be electronic, paper-based, or a blend of both. The aim is to demonstrate their development in each of the global skill clusters. The teacher’s role is to judge, along with their own observations, the student’s progress in skill development. This then provides an opportunity for feedback on where a student has reached and what steps could be taken to progress further.

How should we approach this more holistic approach to the assessment of global skills?

  1. Keep it simple

Our suggestion[i] is that we use just three classifications for each cluster of skills: working towards; achieved; exceeded. Each of these may generate specific feedback – what more is needed; where to go next; how to improve even further.

  1. Trust teacher judgement

The evidence for these holistic judgements comes from the teacher’s own informal evaluation of what is seen, heard and read in the classroom and outside. This is more dependable than narrow standardised skills because of the multiple and continuous opportunities for information gathering. These judgements require teachers to utilise and develop their skills of observation and continuous assessment.

  1. Continuously sample student performance

This does not mean informally assessing every student on every occasion, it involves focusing on different students on different occasions so that, over time, we will have monitored all our students’ performance.

  1. Use any assessments formatively

The purpose of the assessments is to inform students of their performance and to use our judgements to provide feedback on what to do next. The classifications should be used to encourage further development rather than as summative grades.


ELTOC 2020

I hope this is useful. I’ll be expanding on this in my upcoming session at ELTOC 2020. I look forward to seeing you there!


Gordon Stobart is Emeritus Professor of Education at the Institute of Education, University College London, and an honorary research fellow at the University of Oxford. Having worked as a secondary school teacher and an educational psychologist, he spent twenty years as a senior policy researcher. He was a founder member of the Assessment Reform Group, which has promoted assessment for learning internationally. Gordon is the lead author of our Assessment for Learning Position Paper.

[i] ELT Expert Panel (2019) Global Skills: Creating Empowered 21st Century Citizens Oxford University Press


Leave a comment

Teaching Phrasal Verbs – A New Approach | ELTOC 2020

The Curse: When Adam was expelled from Eden, it seems that God had an afterthought… ‘[Because of what thou hast done]… in sorrow shalt thou eat of [the ground], thorns also and thistles shall it bring forth to thee …in the sweat of thy face shalt thou eat bread… Oh – and thou shalt have to study phrasal verbs as well’ (Genesis 3:14 – 19). Yes, I do realise that the last bit is an addition to the original text, but I am sure that this is how many learners of English feel…

What is so special about phrasal verbs? Actually not much. Phrasal verbs are just like any other lexical items in English; that said, there are some good reasons why learners find them such a pain:

  • They are verbs and as such they are less concrete and ‘free-standing’ than most nouns (a hammer is a hammer, but what is ‘look up’?);
  • Many phrasal verbs tend to have more than one meaning (e.g. make up [an excuse] / make up [as in ‘kiss and make up’]);
  • The verb often has no direct connection with the meaning (e.g. ‘What have you been getting up to?’);
  • The fact that there are many, unrelated phrasal verbs with a similar form which mean totally different things (e.g. make up/make up for/make for etc.).

How to approach phrasal verbs: Here is the main idea: just because phrasal verbs can be difficult for our learners that does not mean that we have to come up with new ways of presenting or practising these lexical items. In fact, I would like to argue that in trying too hard to help our learners we often do more harm than good (yes, there is going to be a list of ‘dos’ and ‘don’ts’ in my upcoming webinar).

So – why should I attend this session? As it happens there are a number of new ways to help our learners. In my webinar, I am going to recommend three research-based strategies discovered by cognitive scientists which are certain to make your teaching far more effective. In fact, I would go so far as to say that by using these simple strategies, you are going to become at least 30% more efficient in teaching phrasal verbs. Here is an added bonus: these strategies also work for anything else you might want to teach your students – whether this is tenses, writing, functional language, history, biology, economics – anything! Here’s a taster of how it works.

Generation:  This strategy involves asking the students to do something which they lack the knowledge to do. This could be asking them to solve a type of maths problem they have never encountered before or to list the reasons for the French Revolution which they have not been taught. In our field, this could involve asking students to describe the picture of a living room when they do not yet have the vocabulary to do so, or to write a formal email when they have not been shown how to do it. The idea is that the frustration students experience actually prepares their mind to receive the new knowledge and heightens their level of alertness so they experience an ‘A-ha!’ moment when they see the right answer or the right model. ‘Unsuccessful attempts to solve a problem encourage deep processing of the answer when it is later supplied, creating fertile ground for its encoding, in a way that simply reading the answer cannot. It is better to solve a problem than to memorise a solution. It is better to attempt a solution and supply the incorrect answer than not to make the attempt’ (Brown, Roediger & McDaniel 2014 – p. 88).

Retrieval:  Retrieval is the single most effective strategy cognitive scientists have managed to discover. Here is the discovery in a nutshell: whereas most of us teachers focus on input, thinking that the best way to help our students is to structure the information in such a way that it ‘goes in’ more easily, it turns out that students learn best by trying to retrieve the information, that is when they try to get information out of their heads! ‘Retrieval practice occurs when learners recall and apply multiple examples of previously learned knowledge or skills after a period of forgetting’ (Agarwal & Bain 2019 – p. 37). Put another way, you just ask your students to remember things you have taught them. The simplest form of retrieval is when you just ask your class to write down everything they can remember at the end of the session. The more often we do this, the better. As cognitive scientists have shown, the problem with ‘forgetting’ is not so much that we forget – we just cannot access the information inside our head. ‘The more times we draw information from memory, the more deeply we carve out the pathway to it and the more we make that piece of information available for use in the future’ (Lang 2016 – p. 28).

Spacing:  The last strategy is the simplest thing you have ever heard: researchers have discovered that while massed practice (cramming) helps students remember things better in the short term (which is why they invariably do this before exams), in the long term this leads to very little learning. Instead, if one were to study exactly the same material but spread the study over a number of shorter sessions, allowing some time to elapse between them, the difference in the resulting long-term retention can be truly impressive. In Carey’s words ‘nothing in learning science comes close in terms of immediate, significant, and reliable improvements to learning’ (Carey 2014 – p. 76). It seems that when we stop studying and we engage in something else, our brain keeps working in the background, organising things and making connections without us realising it. ‘When we let time pass and space things out, students’ knowledge has the time to solidify and ‘simmer’ ‘ (Agarwal & Bain 2019 – p. 100). So, instead of asking students to engage in retrieval at the end of the lesson, you might ask them to do so at the beginning of the next lesson. Here is another idea: instead of giving students homework on what you did during the lesson, why not give them homework on something you did the previous week?

What about those phrasal verbs? So how can we apply all these ideas? How can we use them to facilitate the learning of phrasal verbs? Well, if I were to tell you everything here, you would not need to attend ELTOC 2020, would you?


ELTOC 2020

Suffice it to say that I intend to make the session a practical one and give you at least four practical, actionable ideas. So do please register, and feel free to bring a friend along!


Nick Michelioudakis (B. Econ., Dip. RSA, MSc [TEFL]) has been active in ELT for many years as a teacher, examiner, presenter and teacher trainer. He has travelled and given seminars and workshops in many countries all over the world.

He has written extensively on Methodology, though he is better known for his ‘Psychology and ELT’ articles in which he draws on insights from such disciplines as Marketing, Management and Social Psychology and which have appeared in numerous newsletters and magazines.

His areas of interest include Student Motivation, Learner Independence, Teaching one-to-one, and Humour.


References

Agarwal, P. & Bain, P. (2019) Powerful Teaching. San Francisco: Jossey-Bass

Brown, P., Roediger, H., McDaniel, M. (2014) Make it Stick: The Science of Successful Learning. Cambridge Massachusetts. Belknap Harvard

Carey, B. (2014) How We Learn. London: Macmillan

Lang, J. (2016) Small Teaching. San Francisco: Jossey-Bass

 


Leave a comment

Teaching English with Technology | ELTOC 2020

We know that with fast-paced change, it can be difficult to keep up with the latest trends. Jargon heavy instructions for unfamiliar technology can make getting started intimidating. And with so many Edtech innovations on offer, it can be difficult to know which solutions actually make a valuable difference to a student’s learning.

As a result, publishers have a duty to keep on top of trends and to experiment with technology. We challenge our own assumptions about technology and are always trying to learn how Edtech can benefit learning and help teachers to realise these benefits in their individual situations.

At Oxford University Press, some of that work is done by the Partnerships Team

The Partnerships team

The ELT Partnerships team.

We’re responsible for a lot of the English Language Teaching division’s partnerships with external companies and we work with a huge range of partners, from the start-up community all the way to the famous tech giants. In many of these relationships, we find innovative new ways to include Oxford content in the partner’s product.

This is a great way to help the quality educational material OUP produces to reach new audiences around the world, but it can also lead to deeper relationships, where we start offering a partner’s product in the packages that we offer to learners.  For example, our relationship with Lingokids has evolved from offering our content in their platform to their app being directly integrated into some of our primary courses.

This way of working helps us to learn more about technology companies can enhance our content and how their products are used by learners and teachers, before making a decision on whether we could include them directly in our courses. As a result, we can improve our technology solutions, for teachers with effective support on how to use them with their learners.

This also allows us to innovate with exciting new technologies.

We keep a close eye on the new technologies that have the potential to disrupt education, such as augmented reality, virtual reality and artificial intelligence. By working with partners, we can experiment with how new developments might be applied to English language learning.

For example, we worked with VictoryXR, an award-winning virtual reality developer, to create the first language learning VR experience using Oxford materials. Designed for learners in China, the product immerses the user in real-world scenarios, such as travelling through airport security or introducing themselves to a stranger, for unrivalled authentic English practice.

We were a content partner with Google for the launch of Expeditions AR tours, providing learning content to be used in augmented reality experiences that are freely available through the Expeditions app. Since then, we have worked on a variety of pilots to investigate the pedagogical benefit that using AR can provide. We have also developed multiple experiences for use with smart speakers at home, to test how artificial intelligence can help extend meaningful, independent language learning outside the classroom.

By working with partners, we interact with the world’s leading experts in Edtech, who are constantly innovating. As a result, OUP is constantly learning about new ways technology can benefit language learning and, just as importantly, the frustrations and issues it can cause when teachers try to implement it.

So, we’re here to answer your questions at ELTOC 2020

As part of OUP’s ELTOC conference in February 2020, we’ll be running a session designed to give you an introduction to all things Edtech. We’ll explore some of the most popular technology buzzwords, such as the difference between augmented and virtual reality or what artificial intelligence actually means, all easy to understand language and with real examples of free products you can use to get started and tips for how to implement them in your lessons.

We’ll also be answering your questions about Edtech and the future of digital products. This is your chance to clarify anything you don’t understand, ask for tips with a particular technology or start a discussion on something that interests you! Sign up to our ELTOC 2020 waitlist for more information.

For the chance to have your queries featured in the presentation, please submit your ideas or questions by commenting below.


Harry Cunningham is an Innovation Manager at Oxford University Press in the ELT division. He’s focused on enhancing and bringing OUP’s English Language Teaching content to life with the latest and best technological solutions.