online testing webclass

a quick how-to

 

The construction of test items

The following tutorial explains the construction of test items and shows how to create them using the "text-to-items" converter.

The converter can be run by clicking either the "Text Converter" link at the top of each set or "Import items" at the bottom (and then selecting "Text"). Please note that for these links to appear, we must (at least) create an empty set of items:

Here are the basic steps to follow when converting text into test items:

  1. Type in or paste some text into the editable area.
  2. Put square brackets around the words that you would like to replace with gaps (e.g. "The book is on [the] table"). The word in brackets is the key.
    TIP: You can simply double-click a word to convert it into a gap.
  3. There can be more words (or phrases) in the gap, separated by a forward slash:
    • Multiple Choice (up to six): The first one is the key, the others are the distractors. When the test item is created, the options are randomized. Example: "The book [is/be/are] on the table".
    • Multiple Response (no limit): If an option is incorrect, put an asterisk in front of it. Example: "The [book/cat/dog/lion/elephant/*clear] is on the table".
    • Fill Gaps (no limit): There are no distractors here, so all of the words must be correct. Example: "The book is [on/under] the table".
  4. When done, click "Create MC" or "Create FG/MT" at the bottom.

 

More details:

  1. Selected response items
    1. Multiple choice (MC)
    2. True/false (TF)
    3. Multiple-choice cloze (MC-CL)
    4. Right/wrong (RW)
    5. Multiple response (MR)
    6. Matching (MT)
    7. Other
  2. Constructed response items
    1. Fill gaps (FG)
    2. Cloze (CL)
    3. Transformations (word given) (TW)
    4. Transformations (complete) (TC)
    5. Error Correction (ER)
    6. Other (e.g. Short Answer (SA), Translation (TR), etc.)
    7. Phonetic transcription
  3. Extended answer

 

 

Selected response items

The first five types (MC, TF, MC-CL, RW, MR) are created by clicking "Add MC":

Matching (MT) items are created in a slightly different way (see below).

Also note that if we create classic fill-gaps items and provide all of the possible responses (e.g. in the rubric), we get a format that should be classified as selected rather than constructed response.

As the name suggests, none of these item formats requires test-takers to produce their own answers. Instead, answers are selected from the options given (by clicking the best answer, rearranging the options, or by means of drag-and-drop).

Back to the item-type menu

 

Multiple choice

Each MC item consists of:

  1. the stem, e.g.:

  2. the options/choices, e.g.:

The children ___ in the park.

was     were     been

The options include the key (selected) and distractors (incorrect choices).

 

  1. Number of options

    Although there is evidence that the optimal number of MC options is three (see, e.g., Trevisan et al, 1994 or Bruno and Dirkzwager, 1995), for the purpose of trial/pilot testing, it might be useful to have more, simply to find out which distractors perform best.

    Here, up to six options can be included in each item. The default number is three, but this can be changed by selecting "6" at the top of the editor:

    Number of MC options: (applies to the whole test)


    Only non-blank options are displayed in the test proper, so if we need 4, the last two should be left empty.

  2. Alignment

    MC items can be displayed either vertically or horizontally. The latter is the default setting, but it can be changed at the top of the editor:

    Format MC vertically (applies to the whole test)


    Additionally, the options can appear as a drop-down list:

    Drop-down MC (applies to the whole set)
    Note: Type in one or more underline characters (e.g. "__" or "___") in the stem to indicate the place where the options should appear.

 

Using the text converter to create MC items

The options should be enclosed in square brackets, separated by a forward slash. The first one is the key. The order of the key and distractors will be randomized when the items are created. If necessary, the target number of options for all items can be preset using the following select list:

Number of MC options:

If there are fewer options between any pair of brackets than the target number, the "missing" ones will be replaced with "(...)". If there are more options, the extra ones will be ignored.

The following text would be converted into 6 three-option MC items (view the output):

  1. I [needn’t/mustn’t/won’t] lock my front door when I go out; this is a very honest neighbourhood.
  2. I’m sure you [will/are going to/should] enjoy our new programme!
  3. ‘Ouch! At least you [might/can/should] have told me this handle was red-hot! I nearly dropped the soup all over the floor.’
  4. I’ll arrange for the carriers to come down as soon as I [get back/am getting back/will have got back].
  5. This time next week I [will be lying/will lie/am to lie] on the beach at Famagusta!
  6. Nona said, politely and carefully, ‘I think you [are being/are/are to be] silly. You haven't harmed us at all.’

If no gap is needed within the stem, there should be no text following the options (on the same line), for example:

  1. What is the capital of Portugal? [Lisbon/Vigo/Porto]
  2. What is the longest river in China? [Yangzi/Yellow River/Brahmaputra]
  3. What is saltiest sea in the world? [The Dead Sea/The Mediterranean/The Red Sea]
  4. Which of the following lakes is in Canada? [Winnipeg/Michigan/Victoria]

>> the output

 

Back to the item-type menu

 

True/false

TF items are created in exactly the same way as MC. The only difference is that they have only two options, and these are the same for each item.

For example:

  1. London is the capital of the UK. [true/false]
  2. Mars is larger than Venus. [t/f]
  3. Javascript is a markup language. [False/True]
  4. Dogs are more intelligent than cats. [F/T]

Again, the first option is the key. We can use 't' and 'f' for 'true' and 'false' respectively. The options can be in lower or upper case. Here is the output.

 

Back to the item-type menu

 

Multiple-choice cloze

Method 1

The cloze passage (with numbered gaps) can be given in the rubric. The items proper follow, but they should have no stems (leave them empty), just the options. If necessary, the text converter can be used to quickly destribute the options. Here is an example:

  1. [beat/blow/hit]
  2. [OF/FROM/AT]
  3. [sure/certain/obvious]
  4. [at/on/for]
  5. [far/long/closely]

The cloze as a whole can be seen here.

Method 2

The above seems to be OK, but here is a better alternative, especially with longer texts:

We can follow the rules of creating standard MC items and then select the following options:

Cloze/MR/RW      Drop-down MC

Note: The first option alone is used to create RW and MR items. If the second option is unchecked and
some MC items are detected in the respective set, a warning is displayed. See the next item type for more on this.

The second option converts MC choices into select lists and puts them into the corresponding stems. The position of the choices in the stem should be indicated by '__' or '___', etc. (two or more underlines). If we use the text converter, the underlines will be added automatically.

The first option makes sure that the items are displayed inline (on the same line) so that they look like a passage rather than a numbered list. If a line break is needed, it can be inserted by pressing ENTER where necessary.

 

For the converter, we need a passage such as the following:

A postal carrier is working on a new [beat/blow/hit]. He comes to a garden gate marked BEWARE [OF/FROM/AT] THE PARROT! He looks down the garden and, [sure/certain/obvious] enough, there's a parrot sitting on its perch. He has a little chuckle to himself [at/on/for] the sign and the parrot there on its perch. The mailman opens the gate and walks into the garden. He gets as [far/long/closely] as the parrot's perch, when suddenly, it calls out: "REX, ATTACK!"

This version of the cloze task can be seen here.

 

Back to the item-type menu

 

Right/wrong (RW)

These are single-option items (cf. MR below, which are multi-option items).

In general, they are created by selecting    Cloze/MR/RW    and clicking "Add MC" in the editor.

!!! For items of this (and of the following) type, the key is either 'a', meaning that the option is correct, or 'n', meaning that the option is incorrect. For the other MC-like types, the key is a letter corresponding to one of the choices (a-f). Because of that, if we convert regular MC items into RW or MR items, we must bear in mind that the original key will be overwritten in those cases where the correct answer is other than 'a'. The content of the options themselves, however, though invisible, is kept intact, just in case.

The construction of RW and MR items is probably somewhat less "intuitive" than that of the other types.

 

1. Passage

If it is a passage of text, the options should be between (rather than within) the stems. This does not apply to the last option, for which there is no stem to follow. Accordingly, the last option must also contain the ending of the passage.

If we are using the text converter, all we need to do is select the following option:

Right/wrong or Multiple response

NOTE: Put an asterisk (*) in front of an incorrect option.

Here is an example:

Going to bed the [other] night, I noticed people in my shed [*to steal] things. I phoned the police but was told there was no one in the [area] to help. The policeman said they would send someone [over] as soon as possible. I hung [*down.]

 

A minute later I rang again. 'Hello', I said, 'I called you a minute ago [*for] there were people in my shed. You don't have to hurry now, because I've shot them.' [Within] five minutes there were half a dozen police cars in the area, plus helicopters and an armed [*answer] unit. They caught the burglars red-handed. One of the officers said: 'I thought you said you [*would] shot them.' To which I replied: 'I thought you said there was no one [available.']

   Based on a story by Tony Gladstone

When the items are created, the incorrect options will be unchecked in the editor, like this:

Here is the final version (the colour of the options has been changed in the editor).

 

2. Statements

The stems from the True/false task above can be transformed into options in a Right/wrong task. In the text converter, each option as a whole should be enclosed in brackets:

  1. [London is the capital of the UK.]
  2. [Mars is larger than Venus.]
  3. [*Javascript is a markup language.]
  4. [*Dogs are more intelligent than cats.]

When the items are created, the stems are empty, and each option contains one of the statements:

When test takers click a statement, the question mark changes into a tick, which in turn changes into a cross.

 

Back to the item-type menu

 

Multiple response (MR)

MR items are exactly like RW (Right/wrong) items, but with multiple options. The difference between MR and MC is that in the case of the latter there is one (and only one) correct answer, whereas here no such restrictions apply.

What exactly counts as an item in a muti-response task might be somewhat problematic. Is it the stem with all of the options or should each option be regarded as a single item?

Whatever the right answer, there is one thing to remember: on this platform, one point is awarded for each correct decision on every single option. If a test consists of 5 stems, each followed by 4 options, the maximum number of points to score is 20.

Here is an example for the text converter (remember to check "Right/wrong or Multiple response"):

  1. He kept [*handing/giving/paying/*telling] me compliments on my cooking.
  2. I [swallowed/*devoured/*ate/*consumed] my pride and did as I was told.
  3. The festival is likely to [draw/*drag/*push/pull] huge crowds.
OR:
  1. He kept [*handing] me compliments on my cooking.
  2. [giving]
  3. [paying]
  4. [*telling]
  5. I [swallowed] my pride and did as I was told.
  6. [*devoured]
  7. [*ate]
  8. [*consumed]
  9. The festival is likely to [draw] huge crowds.
  10. [*drag]
  11. [*push]
  12. [pull]

With the following settings (in the editor):

we get three subsets, each consisting of a stem followed by four options (click to open).

Another example:

  1. Which of the following verbs are irregular? [break/tell/go/*walk]
  2. Indicate plosive consonants: [p/*s/t/*r]
  3. Which of the following are amphibians? [*crocodile/*whale/frog/*snake]

Here is the output.

 

Back to the item-type menu

 

Matching

Before matching items can be created, the following checkbox must be selected (when adding a new set):

Each item consists of a question (or premise) and a response. In the text converter, the pairs should be formatted in the following way:

  1. pay [] attention
  2. run [] a bookshop
  3. drop [] a hint
  4. do [] a favour
  5. make [] the headlines

In the case of matching items, there is no need to type any text between the brackets.

To make the task more difficult for the test takers, one or more extra responses can be added in the editor:

Also, the size of the container should be defined:

Here is the output.

Here is a matching task with pictures.

Note:
Responses are always shuffled. Premises can be, just like all other test items:

 

Back to the item-type menu

 

Other selected-response types

With standard fill-gaps items (see below), we expect test takers to produce their own answers. For example, the following sentences are to be completed with prepositions:

However, the words to be used in the gaps (responses) can be given in the instructions (rubric):

This makes the task look more like matching than gap-filling, and definitely selected rather than constructed response.

To make it even more "selected-like", the test takers can be instructed to drag and drop the responses into the gaps, instead of typing them in. For this to be possible, each response should be added as the value of a text input field:

1) Click the button indicated in the figure below.
2) Type in the value of the input field and check "Enable drag-and-drop".

  =>  

Here is a video which shows how to drag-and-drop into the gaps: open

The test itself: open

 

Back to the item-type menu

 

 

 

Constructed response items

Constructed-response items of every format are created by clicking "Add FG":

They are all, essentially, modifications of the gap-filling type.

They may differ from each other in the length of the input, the length of the expected response, the display style (list vs. passage), and also in whether the gap is empty or not (as for Error correction).

An additional setting that is available for constructed-response items is the number of spelling errors permitted for partial score (half point). There is one global setting (at the top of the editor):

and one for each individual item:

 

Back to the item-type menu

 

Fill gaps

The following text would be converted into a set of 5 gap-filling items. There is no limit on the number of correct responses enclosed in brackets (separated by a forward slash):

  1. I’ve lived in this town [since] I was a child.
  2. A female lion is called a [lioness].
  3. The train takes just [as] long as the bus.
  4. These questions seem to be getting harder and [harder].
  5. February is the [coldest/shortest/second] month of the year.

>> the output

Further options in the editor:

(1) Click to delete one of the keyed responses
(2) Click to add another keyed response
(3) Change the length of the gap
(4) Specify those responses which, though acceptable, you would not include in the key, for example "lioness." (with an unnecessary full stop).

 

Back to the item-type menu

 

Cloze

The term "cloze" is used loosely here to refer to items which are displayed as a passage, not as a numbered list.

For example, if the items created in the previous section were displayed inline, we would get the following:


Passage

The text-into-items converter can be used to quickly process longer texts. There is a button there which makes it possible to replace every instance of a given word (or words) with a gap.

In the text given below, all definite and indefinite articles ("the" & "a") have been enclosed in brackets. In addition, several "empty" gaps have been added (where there should be no article, represented by [x]).

Chimp recognises synthetic speech
By Matt Walker
Editor, BBC Nature

[A] talented chimpanzee called Panzee can recognise distorted and incomplete words spoken by [a] computer, scientists have discovered. That suggests that apes may be more capable of perceiving spoken sounds than previously thought, and that [the] common ancestor of humans and chimps may also have had this ability. It also refutes [the] idea that humans have brains uniquely adapted to process speech, say [the] scientists who have published their findings in [the] journal Current Biology.

Panzee was raised from 8 days old, by humans, and was spoken to and treated as if she were human. At [the] same time, she was taught to use symbols called lexigrams to communicate. "This has resulted in Panzee showing [x] proficiency in understanding approximately 130 English words," researcher Lisa Heimbauer told BBC Nature. That made her an ideal subject to test hypotheses about how well other species, rather than humans, might be able to understand speech. "There is [a] view about [the] human ability to produce and perceive speech that is called 'Speech is Special'," said Ms Heimbauer, who is studying for her PhD. "This argument proposes that, besides humans being [the] only species able to produce speech, due to their anatomy, they also have [a] specialised, cognitive module to process speech."

[x] evidence for that comes from studies showing that humans can understand speech even when it is incomplete or highly distorted. "However, an alternative view is that auditory processing is fundamentally similar across most mammals, and that animals therefore have latent abilities for [x] speech perception," said Ms Heimbauer. So she and her colleagues Michael Beran and Michael Owren, all from Georgia State University in Atlanta, US, tested Panzee to find out if she too could recognise incomplete or distorted spoken words. They played Panzee noise-vocoded speech, which alters [the] frequencies of [the] spoken words. This produces [a] sound similar to what people with cochlear implants hear. They also played Panzee so-called sine-wave speech, which is synthesised from just three pure tones.

Both types of [x] degraded speech have been shown to be understandable by people. "She is only one of [a] few animals who could be tested in this way, to reveal what [the] speech perception abilities of [a] common chimp/human ancestor may have been," said Ms Heimbauer. [The] researchers discovered that Panzee recognised these degraded spoken words far more often than she should by [x] chance. Her upbringing, say [the] scientists, appears to have given her enough experience of hearing and understanding spoken words to allow her to recognise them when they are distorted. That "highlights [the] importance of [x] early experience in shaping speech perception," said Ms Heimbauer. It also provides evidence that [a] common ancestor of humans and chimpanzees would have had [the] ability to perceive speech, she says. "If humans do possess [a] specialised, cognitive, speech-processing brain module, it would be something that evolved later in [x] humans, making us more efficient at what we do."

This is the result of the conversion (the only thing changed in the editor is the style of the title): open


Sentences with multiple gaps

The "Cloze/MR/RW" option can be used to create items such as:

Aspiration can be described as a kind of noise, puff of air, or voiceless ___ consisting of strongly expelled breath between the ___ of the plosive and the onset of the following ___.

This sentence will have to be divided into 3 items, one for every gap.

Here's how to do it using the text converter:

1) Type in or paste the input text into the editable area, and define the gaps:

  1. Aspiration can be described as a kind of noise, puff of air, or voiceless [interval] consisting of strongly expelled breath between the [release] of the plosive and the onset of the following [vowel].
  2. Only [fortis/voiceless] plosives can be strongly aspirated, and only when they occur at the beginning of a stressed [syllable].
  3. However, no aspiration occurs when they are [preceded] by "s".

2) Click "Create FG/MT".

3) When done, make sure the following options are selected:

>> the result

 

Back to the item-type menu

 

Transformations (word given)

For example:

NOTE! Each second line is preceded by SHIFT+ENTER, rather than ENTER itself.

  1. Perhaps they forgot your birthday. MIGHT
    [They might have forgotten] your birthday.
  2. Don’t tell anyone! MUST
    [You must not tell] anyone!
  3. Do you think we can generalize that the people of France are good mechanics? FRENCH
    Do you think we can generalize that [the French are] good mechanics?
  4. Ken’s English is better than Ben’s. WELL
    Ben doesn’t [speak English as well as] Ken does.
  5. I banged on the front door for ten minutes, getting increasingly cold. COLDER
    I banged on the front door for ten minutes, [getting colder and colder].

A look at the editor:

Here, in order to create a new line within a given item, either ENTER or SHIFT+ENTER can be pressed.

>> the final result (with some additional formatting applied to the words given)

 

Back to the item-type menu

 

Transformations (complete)

The same rules as above apply here.

Example:

  1. Mr Brown wasn’t at home but I managed to contact him at his office.
    Mr Brown wasn’t at home but I was [able to contact him] at his office.
  2. You have read the whole book but we were supposed to study only the second chapter.
    You needn’t [have read the whole book] because we were supposed to study only the second chapter.
  3. Have you got the newspaper that came out last Monday.
    Have you got [last Monday's] newspaper?
  4. Her questions embarrassed me.
    Her questions were [embarrassing/embarrassing for me].
  5. This policy is not logical.
    It’s an [illogical policy].

>> the final result

 

Back to the item-type menu

 

Error Correction

For the text converter, we need pairs of incorrect-correct sentences. The correct sentences are in brackets because they will be saved as the key.

For example:

  1. Why didn’t you stop? Are you aware that you could cause an accident?
    [Why didn’t you stop? Are you aware that you could have caused an accident?]
  2. On our way to Prague we stayed in an old little lovely village.
    [On our way to Prague we stayed in a lovely little old village.]
  3. Do you think that 45 minutes are enough time for this exam?
    [Do you think that 45 minutes is enough time for this exam?]
  4. It was a small beautiful silver Spanish harmonica.
    [It was a beautiful small Spanish silver harmonica.]
  5. The news were better than expected!
    [The news was better than expected!]

When the items are created, all we need to do is make sure the following option is selected:

>> the final result

 

Back to the item-type menu

 

Other constructed-response types

The item formats described here can be modified in many different ways.

Here are some suggestions.

 

Back to the item-type menu

 

Phonetic transcription

To make it possible for test takers to type in phonetic symbols, the following option should be selected:

IPA symbols will also appear in the editor:

Here is a sample test: open

 

Back to the item-type menu

 

 

Extended answer

An extended-answer task is created by clicking the following option:

Online marking of extended responses is possible. Moreover, all answers can be downloaded to MS Word:

 

Back to the item-type menu