About CATSS



Basic assumptions

The rationale for this test is based on three assumptions:

The most important component of word knowledge is the ability to establish the link between word form and word meaning.

Knowledge of meaning is not an ‘all or nothing’ phenomenon but can have different degrees of strength (here characterised as productive recall, receptive recall, productive recognition and receptive recognition) which, it is hypothesised, constitute an implicational scale.

Knowing many words (units of meaning) is more important than knowing few words in depth. Hence, a good vocabulary test should test how many words are known, or try to provide a picture of the learner's overall vocabulary.


Testing vocabulary size and strength

CATSS aims to assess vocabulary size, i.e. knowledge of word meaning. However, in an attempt to overcome the basic limitation of size tests by testing each word in more detail, four degrees of strength of knowledge are assessed. Each word is tested in four modalities
(i.e. productive recall, receptive recall, productive recognition, receptive recognition), as demonstrated using the word ‘melt’ below.



Productive recall

The task is to supply the L2 target word (melt). The first letter of the word is provided in order to avoid non-target words which have the same meaning.
Turn into water             m___________
(In the bilingual version, the prompt is the L1 translation of ‘melt’)


Receptive recall

The task is to demonstrate understanding of the meaning of the L2 word (melt) which is embedded in a phrase, or a short sentence to be completed by the test taker.
In this instance there are a range of acceptable responses (e.g. water, fluid, liquid).
When something melts, it turns into ______________.
(In the bilingual version, the word 'melt' is presented and learners are asked to translate it into L1.)


Productive recognition

The task is to choose the target word from four options (a b c d). The distractors, which are semantically unrelated (as we are not testing fine shades of meaning), are taken from the same frequency level as the target word. 
Turn into water
a. elect
b. blame
c. melt
d. threaten
(In the bilingual version, the L1 translation of 'melt' is provided as the prompt)


Receptive recognition

The task is to choose the meaning of the target word from the four options provided.
Most of these options are paraphrases of the distractors that appear in the productive recognition mode.
When something melts, it
a. chooses
b. accuses
c. makes threats
d. turns into water
(In the bilingual version, the distractors are translations into L1)



Our studies showed that the four modalities constitute a hierarchy of difficulty and are implicationally scaled. Therefore, if a test-taker answers an item correctly in one strength modality, it will not be necessary to test the same word in the subsequent strength modalities.
For example, if a test taker has correctly recalled the target word in productive recall mode, that word will not appear again in the other three modalities (i.e. receptive recall, productive recognition or receptive recognition).
Where no response, or an incorrect response is given, the computer keeps the word in its memory for presentation in the next modality, but only after all the other words at the same frequency level have been tested in that modality. Once it has been answered correctly,
the word is not presented again.


Scoring

The answers provided by the test-taker are matched against a pre-specified marking key and are scored as correct or incorrect.  A word answered correctly at productive recall receives 1 point, a word answered correctly at receptive recall receives 0.75 points,
at productive recognition - 0.5 points, at receptive recognition - 0.25 points. At the end of the test, test takers can see their results in a table.
The table provides the following information:

- the number of correct answers for each modality at each frequency level
- the total vocabulary size score per modality
- the total vocabulary size score in the test 
- the total vocabulary strength scores per modality 
- the total vocabulary strength score in the test
(The strength score is a weighted score which takes into account the modality at which each word was answered correctly and its corresponding score).


Sources