Designing a test

Stage 1: Preparation

Prepare your test. It is a good idea to list in writing all the material that you want your test to cover: you can then refer back to the list during and after the test-writing to see if you have included all you intended.

Stage 2: Performance

If possible, administer your test to a class of learners; if not, ask colleagues to try doing it themselves.

Stage 3: Feedback

Look at how your test was done, and ask the testees how they felt about it.

GUIDELINES FOR TEST PREPARATION

Validity. Check that your items really do test what they are meant to!

Clarity. Make sure the instructions for each item are clear. They should usually include a sample item and solution.

'Do-ability'. The test should be quite do-able: not too difficult, with no trick questions. Ask a colleague to read through it and answer the questions before finalizing.

Marking. Decide exactly how you will assess each section of the test, and how much weighting (percentage of the total grade) you will give it. Make the marking system as simple as you can, and inform the testees what it is: write in the number of points allotted after the instructions for each question.

Interest. Try to go for interesting content and tasks, in order to make the test more motivating for the learners.

Heterogeneity. The test should be such that lower-level students can feel that they are able to do a substantial part of the test, while the higher-level ones have a chance to show what they know. So include both easy and difficult items, and make one or more of the difficult ones optional.

Test administration

The actual design of a test, whether oral or written, formal or informal is, of course, important, but it is not the end of the story. How the test is actually administered and returned can make a huge difference to motivation and performance; in particular, sensitive presentation of a test can significantly reduce learner anxiety.

QUESTIONS ON TEST ADMINISTRATION

Before the test

§          How far in advance do you announce the test?

§          How much do you tell the class about what is going to be in it, and about the criteria for marking?

§          How much information do you need to give them about the time, place, any limitations or rules?

§          Do you give them any 'tips' about how best to cope with the test format?

§          Do you expect them to prepare at home, or do you give them some class time for preparation?

Giving the test

§          How important is it for you yourself to administer the test?

§          Assuming that you do, what do you say before giving out the test papers?

§          Do you add anything when the papers have been distributed but students have not yet started work?

§          During the test, are you absolutely passive or are you interacting with the students in any way?

After the test

§          How long does it take you to mark and return the papers?

§          Do you then go through them in class?

§          Do you demand any follow-up work on the part of the students?

Before the test

The tests are announced at least a week in advance in order to give them plenty of time to prepare and details are given of when, where and how long the test will be. The class is also told as precisely as possible what material is to be tested, and what sort of items will be used.

Giving the test

It is quite important for me to administer the test myself, and more pleasant for my students. Thus, I will be able, if I wish to remind them about the test content, and format before giving out the papers; and sometimes run through the instructions with them after doing so in order to make sure that everything is clear - as well as wishing them good luck!

During the test, I may help students who still have difficulty with instructions; I do not normally help with the content itself.

After the test

The tests are marked and returned as quickly as possible (within a week) so that we can discussspecific points while the test is still fresh in the students’s minds. Usually I go through the answers in class. I tell them how I assessed this item and ask them if they agree with my scoring. (It sometimes happens that I counted the points the wrong way or just missed a correct answer).

ON TESTING

1. What is a 'valid' test?

2. What is a 'reliable' test?

3. What is 'backwash'?

4. What is the difference between an 'achievement' test and a 'proficiency' test?

5. What is the difference between a 'diagnostic' test and a 'prognostic' test?

6. Can you give an example of a 'discrete-point' test?

7. Can you give an example of an 'integrative' test?

8. Are Questions 1-7 above examples of 'objective' or 'subjective' test items? Why?

9. Give examples of:

    a) a multiple-choice item

      b) an extract from a cloze test.

10. Within the multiple-choice item you have given, can you identify:

       a) the stem?

       b) the options?

       c) the distractors?

Test on testing:

1.      A 'valid' test is one which actually tests what it is designed or intended to.

2.      A 'reliable' test is one that produces consistent results when administered on different occasions.

3.      'Backwash' is the effect, positive or negative, that a test has on the teaching and learning that precede it.

4.      An 'achievement' test measures how much of the material taught in a given course, or part of one, has in fact been learned; a 'proficiency' test measures the overall language proficiency of testees, without reference to a particular course.

5.      A 'diagnostic' test reveals the strong and weak points of a learner's knowledge; a 'prognostic' test predicts how well he or she is likely to do in a language course.

6.      'Discrete-point' tests consist of separate items. For example, the following set of items testing vocabulary:

Another word for 'sea' is _________.

The opposite of 'proud' is _____________.

Apples, pears and bananas are ____________________.

7.      An 'integrative' test involves whole pieces of discourse and tests a relatively broad command of the language: writing an essay for example, or doing a cloze test.

8.      Questions 1-7 above are, strictly speaking, 'subjective' test items: the exact way you express your answers may vary, and therefore there is an element of subjective judgement in the way the tester will assess their correctness.

9.   (a) An example of a multiple-choice item:

Many people these days _________ have enough to eat.

(1) doesn't    (2) isn't    (3) don't    (4) aren't

      (b) An extract from a cloze test:

A cold front is approaching from _________ west and we can therefore expect lower ________ tomorrow throughout the country. There is _____________________ to be some rain in the morning, ___________________ it will become brighter later in the ________________________.

Guidelines for test preparation

·        Validity. Check that your items really do test what they are meant to!

·        Clarity. Make sure the instructions for each item are clear. They should usually include a sample item and solution.

·        'Do-ability'. The test should be quite do-able: not too difficult, with no trick questions. Ask a colleague to read through it and answer the questions before finalizing.

·        Marking. Decide exactly how you will assess each section of the test, and how much weighting (percentage of the total grade) you will give it. Make the marking system as simple as you can, and inform the testees what it is: write in the number of points allotted after the instructions for each question.

·        Interest. Try to go for interesting content and tasks, in order to make the test more motivating for the learners.

·        Heterogeneity. The test should be such that lower-level students can feel that they are able to do a substantial part of the test, while the higher-level ones have a chance to show what they know. So include both easy and difficult items, and make one or more of the difficult ones optional.

The actual design of a test, whether oral or written, formal or informal is, of course, important, but it is not the end of the story.

How the test is actually administered can make a huge difference to motivation and performance.

QUESTIONS ON TEST ADMINISTRATION

Before a test

·        How far in advance do you announce the test?

·        How much do you tell the class about what is going to be in it, and about the criteria for marking?

·        How much information do you need to give them about the time, place, any limitations or rules?

·        Do you give them any “tips” about how best to cope with the test format?

Giving the test

·        How important is it for you yourself to administer the test?

·        What do you say before giving out the test papers?

·        Do you add anything when the papers have been distributed but students have not yet started work?

·        During the test, are you absolutely passive or are you interacting with the students in any way?

After the test

·        How long does it take you to mark and return the papers?

·        Do you then go through them in class?

·        Do you demand any follow-up work on the part of the students?

How can you assess the value of a test?

There are six factors that need to be taken into account when assessing the value of a test:

Match these factors with the questions:

1___   2___   3___   4___   5___   6___