Through the course of your time as a TEFL teacher or manager, you will more than likely find yourself asked to create, or at the very least vet, a test. I remember the first time that I was tasked with this and how daunting the whole process seemed. I’d given countless tests to students and had always complained about them; either because they looked unprofessional, contained mistakes or they didn’t seem to be testing what I’d been teaching in my classes. How could I do better than this?
The answer is more interesting than you might think. Come along with me as I explain how to create a valid, fair and, what’s more to the point, useful test. Also, by clicking on the button below, you can have a test making checklist emailed right to your inbox.
Is the Test Valid?
When sitting down to create a test, the first thing to consider is what it takes to create a valid test. There are three types of validity that you should check: face validity, construct validity and content validity. If you can nail these, you are off to a great start.
Face validity means that it looks like a good test. Your students shouldn’t sit down to take the test and immediately give you a knowing look as if you’d created the test on the back of a napkin. It should look professional and something worth their efforts to review. It’s common sense, but if the test looks professional, your students will take it more seriously.
To achieve face validity, use the materials that you have on hand. Namely, the student book and the workbook. You don’t have to spend hours either creating new images or trying to hunt down copyright free pictures, use the images that the students have been working with. If the image that you’re using is questionable, you’re actually testing two things: 1) their English ability (intended) 2) their ability to interpret an image (unintended). For example, lemons (or what are called lemons) in Vietnam are usually green. If a student has never before seen a picture that is used on the test and is just used to the cartoon image on a flashcard, they may not be able to make the quick leap on test day.
Construct validity is making sure that the knowledge, skill or ability (K.S.A.) is reliably being tested with the types of questions being asked. If you are trying to test reading skills at an intermediate level, you need to ensure that both the chosen text and the types of questions are reflective of testing at that level. In this case, trying to test a student’s ability to use context clues to determine the meaning of a word in the text is undermined if that student already knows all of the words. If you’re trying to test an intermediate student’s ability to express opinions fluently, using a Q&A interview format where short responses are appropriate won’t work.
A test creator should strive to use as many authentic materials in their test as possible without using types of materials that have never before been used in class. However, if your students commonly use newspapers or certain blogs for reference in class, it is absolutely valid to use one in the test as long as graded appropriately. The use of realia, or at least real pictures, is fine as long as the test creator can be assured that they have been used in all the classrooms the test will be given, prior to the test.
Content validity refers to whether the test contains tasks that the student is familiar with. To put it simply, you should create the test to reflect how you teach. Likewise, it should reflect the types of tasks that they have already come across either in class or in their workbook. This is essential when teaching young learners. A test creator should never assume that a student will be familiar with a crossword puzzle, for example, if they’ve never been asked to do one within the context of that class.
Things to Remember During Testing
There are also a number of practical considerations for anyone who is creating or administering, a test to consider that are often overlooked. The first is to consider the classroom setting in which the test will be given. If you are going to include an audio portion, you need to make sure that all of the classrooms where this test will be taken have reliable equipment that will allow the students a fair chance at answering the questions. If your test is going to include an interview portion, you need to make sure that it can be done in an environment that won’t be distracting. Again, you’re testing a student’s ability to listen or speak, not their hearing or their ability to cope with distractions.
In a similar vein, in order for it to be a real test of ability, it needs to be reliably the same every time it is taken. It may seem like a given when creating the test, that the same pieces of paper will be handedÂ to the students and the same audio files used, and will, therefore, result in a consistent test. Things that might be overlooked are what is done before the test is handed out. Did one group get a mini-review to ensure they would remember something while others did not? Are instructions given in the same way every time? Is the amount of time for students to finish different sections consistent?
Another matter of reliability comes down to the security of the test. One needs to be sure that those giving the test are taking the same measures to prevent cheating. Also, that the test is given in such a way that there is no chance of students getting a hold of the test before test day or gaining access to the answers. For example, if I have a class that takes the same test on Tuesday that I’ve given to a different class on Monday, how do I know that the answers weren’t leaked?
Is Testing Useful?
The above advice about validity and reliability may seem a bit overkill. However, if the test isn’t taken seriously, what is the point of giving it? If the test isn’t given in the same way every time, is it a real test?
Time and time again I hear teachers questioning the importance of testing. It’s a genuine concern, what is the purpose of a test? Is it useful to the learning process or does it get in the way? Through experience, I’ve come to believe that tests can be incredibly useful tools for students and teachers to measure progress. Also, that they can create a positive washback effect where students feel a sense of accomplishment, especially if the test is paired to clear course objectives.
In order for a test to be useful, it needs to be a real test that measures authentic skills. Likewise, it needs to be transparent enough that the students understand what is being tested. It needs to be a valid test that can be given in a consistent way in each classroom. If you as the test creator or administrator take it seriously, so will the students taking it.
Click on the link below for a useful test making a checklist that will help you on your journey.