Here is a useful list of psychometric terminology.
Every psychometric test has at least one norm group.
The standalone score has little meaning, so a comparison with previous candidate performance is used. For example, percentile rank.
Pre-test (or Pilot) Item
A trial item as part of a larger-than-necessary set of pre-test items. The final, refined test is created by refining those items that work best. This is based on statistical analysis of the trial item set’s results.
Firstly, is the test a reliable measure?
Most commonly the internal consistency index coefficient alpha or its dichotomous formulation, KR-20. Under most conditions, these range from 0.0 to 1.0, with 1.0 being perfectly reliable measurement. A reliable test may still not be a valid test.
Subject Matter Expert (SME)
SMEs write items, review items, participate in standard-setting studies and job analyses.
Secondly, is the test valid? A measure of what it ‘says on the tin’?
In our opinion, both the initial content validation and later criterion validation analysis are vital for any bespoke psychometric test.
We recommend collecting additional recruitment data over time so that additional validation studies can be conducted. Such as assessment centre data.