bespoke psychometric test designs

#BespokePsychometrics

We have nearly 30 years experience of bespoke psychometric test designs. Rob Williams has published fivepsychometric test books, to date!

Our Practice Aptitude Test Books (listings on Amazon)

  • FREE Brilliant Passing Verbal Reasoning Tests ebook

  • Passing Numerical Reasoning Tests book

Our Bespoke Psychometric Test Designs

Bespoke personality test designs

Bespoke personality tests have become a key tool for development. I’ve recently produced motivational, customer-centric and sales-specific tools.

  • TPI’s sales personality design
  • Leadership assessment design for TPI
  • Alison’s  

Personality strengths research

Bespoke personality tests have become a key tool for development. I’ve recently produced motivational, customer-centric and sales-specific tools to name but a few.

Education sector specialisms

Admissions tests practice

Our aptitude test design projects

Citibank Aptitude Test Design Example

  • Designing blended assessments for retail and call centre operations across the USA.
  • Allowing for cultural differences in Mexican operations.
  • Managing item writing team for SJTs, ability tests, biodata questions etc.

Bespoke psychometric test designs

EPSO Aptitude Test Design Project

  • Development of project management test.
  • Design of IT skills-based aptitude tests.

Assessments Design

This psychometric test design is similar in nature to personality questionnaire design, explained here. Our bespoke strengths test design process…

  1. Has dimensions key to the role being assessed.
  2. Reflect the personality, attitudinal and motivational aspects of the role-specific dimensions.
  3. Have face valid questions.
  4. Be capable of completion in 20 minutes approx.
  5. Adopt a single-stimulus question format (Likert scale).
  6. Adopt a normative format of scoring utilising a sten look-up table (for each personality scale).

Situational Strengths Test

An excellent example is CAPP’s Situational Strengths Test. This builds on the many advantages that SJTs have, being:

–Easy to administer to a large audience.

–Relatively strong validity.

–Can assess a sample of the relevant KSA’s.

–Refreshed items possible.

–More difficult to fake responses.

Bespoke VBR Example

  • 3-4 scenarios will assess each value, giving approx. 15-16 questions.
  • Provides accurate and meaningful feedback to each respondent.
  • This practical job preview has the advantage of using the same SJT format.

Bespoke psychometric test designs

Situational interview(s) comprising:

  • Values.
  • Competency; and
  • Technical questions.

Situational Simulation Exercises

  • Scenarios from job analysis used to design simulation exercises.
  • Parallel version developed to maximise exercise integrity.
  • And to minimise the risk of applicants sharing details of tools.
  • Without compromising the validity of the assessment process.

Situational Values-Based Interview

  • We propose a structured interview comprising values, competency and technical pharmacy questions
  • Collaborate with job incumbents to design technical questions/scoring guide.

A situational judgement test, or SJT, presents realistic scenarios similar to those that would be encountered when doing a particular job. Typically, candidates must identify the Best and the Worst course of action from four multiple-choice options. In other words, you must make two “judgements” about what to do and what not to do next in the “situation” presented. An alternative SJT format asks the test-taker to rank multiple-choice options in terms of effectiveness.

Personality strengths in the news

On BBC Radio 2, in February 2014, Dr Almuth McDowell referred to the benefits of situational judgement tests and one of the SJT example items developed by Rob Williams Assessment Ltd.

The same interview on SJT benefits also featured in The Times on 4th Feb, Dr Almuth McDowell, a lecturer at the University of Surrey, says psychometric testing has an important role to play, but only in conjunction with other measures, not least because it is possible to cheat. 

Additional situational judgment test tips are available in Rob Williams Assessment Ltd’s latest Career Builder article entitled “How to succeed at management interview tests”.

Strengths Assessment Development

Psychometric test examples are useful to see the type of strengths development testing approaches.

In fact, there are many many types of psychometric test examples. Let’s consider the many different strengths that a person may possess. For example, their intelligence strengths and weaknesses from the many different types of intelligence.

The Intelligence Test demonstrates the range of possible intelligence strengths. There are many types of intelligence. For example, creative intelligence and musical aptitude which we have not included in our model.

Bespoke psychometric test designs

Strengths Assessment Design

An effective competency design framework is crucial to the operation of many HR practices.

Shown below is an example of a typical competency design framework used in an assessment centre. In this case, the group exercise’s competency component of the overall assessment centre competency matrix:

– Oral Communication

How clearly and confidently the individual communicates with the group.

– Planning and Organising

How much consideration is given to planning and systematically going through the issues outlined in the scenario.

– Judgement and Decision-Making

How logically the individual makes their decisions and judges other participants’ input/comments.

– Analysis and Problem-Solving

How effectively the individual analyses the scenario’s issues and the solutions proposed.

– Finding Solutions

The creative thinking that is shown. The number and effectiveness of the ideas generated.

– Teamworking

How well the individual works with and encourages the other group exercise participants.

Strengths framework design

Having a competency framework helps an organisation to:

– Know which knowledge, skills and abilities are important for success in each role

– Measure and benchmark organisational performance

– Select, develop and performance manage employees (against the competencies associated with each role).

– Is it more cost effective to adapt an already validated model?

– How best to get buy-in across the business? Firstly, for involvement in the development stage. Secondly, to get commitment to future use of the validated framework.

– What role analysis techniques are most suitable for analysing each of the job roles?

– How to benchmark employee performance against the trial competency framework?

– How to validate the effectiveness of the draft competency framework?

Strengths-based Telephone interview design

Previous projects that have encompassed telephone interview sift designs. These include:

  • Competency-based sift design for telephone and cv-based sifts (finance sector).
  • Telephone interview script design (consultancy firm).
  • Conducting telephone interviews for the armed services.
  • Telephone interview design (manufacturing sector).

Bespoke psychometric test designs

Strengths Development – Role Analysis

There follows a Role Analysis Case Study: a US bank had two specific requirements: ensuring a representative sample by samplying sufficiently high number of people per role. Interviewees were mainly high performers; sampled across both urban and rural branches; with their offices scattered right across the U.S.

1) Strengths Telephone-based interviews

Firstly, these formed a large part of the job analysis research. The advantages of using a telephone-based approach was that a standard template could be created with a script to adhere to whilst also allowing some follow-up question flexibility.

2) Assessment Designs – Visionary interviews

Secondly, these were run on a more select basis with a range of senior managers (within each job role). A scripted template ensured that all psychologists asked the standard set of questions. Face-to-face visionary interviews were also arranged with the most senior personnel at some of the focus group sites.

3) Strengths Focus groups

Thirdly, these were conducted in several American States, situated in Central, Eastern and Western time zones. A semi-structured focus group format represented the most suitable job analysis technique to use. Six participants, drawn from each job role, were invited to attend each role-specific focus group session.

4) Strengths Role survey

In addition, a role survey was created based on the bank’s competency framework, with two questions per competency. Survey results were also used to inform the item writing process.

The final 5-10 minute survey had two banks of questions with the stems:

  • How important do you do each of the following on a daily basis? and how important are the following to your job?

Example job analysis briefings

Candidate Experience Briefing – A separate briefing detailed the client style and language (US English) needs.

The main aims from a “Candidate Experience” perspectives included:

  • Reflecting the company brand
  • Engaging job applicants
  • Providing some elements of a realistic job preview
  • Overall look and feel to be consistent

Assessment Designs – Focus Group Briefing

The focus group aims were to confirm the role profile interview data.

Firstly, focuses on how best to structure the upcoming focus group sessions. Secondly, a consideration of the outputs required. Thirdly, how individual consultants could best run their focus group sessions.

Situational Strengths Test

One company that uses this test is Lloyds pharmacy, where the CAPP Situational Strengths Test is used to recruit pharmacists.

Situational strengths tests have these advantages (in any context):

– Easy to administer to large audience.

– Relatively strong validity.

– Can assess sample of the relevant KSA’s.

– Refreshed items possible.

– More difficult to fake responses.

There is a specialised form of situational judgement test called the Situational Strengths Test. One company that uses this test is Lloyds pharmacy, where the CAPP Situational Strengths Test is used to recruit pharmacists.

Situational strengths tests have these advantages (in any context):

– Easy to administer to large audience

– Relatively strong validity

– Can assess sample of the relevant KSA’s

– Refreshed items possible

– More difficult to fake responses

Bespoke psychometric test designs

Bespoke VBR Example

  • 3-4 scenarios will assess each value, giving approx. 15-16 questions
  • Provides accurate and meaningful feedback to each respondent
  • This practical job preview has the advantage of using the same SJT format

Assessment Designs – Situational Exercises

  • Scenarios from job analysis used to design simulation exercises
  • Parallel version developed to maximise exercise integrity and minimise risk of applicants sharing details of tools and compromising the validity of the assessment process

The following situational strengths scenarios are examples only. Actual client projects always start with the most suitable level of role analysis. Then again validate the content against client requirement by choosing the most predictive scenarios and answer options.

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Each scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Aptitude test design projects

Ability tests for graduates, including problem-solving aptitude tests and abstract reasoning tests.

UKCAT test practice questions for several clients. Each project encompas

  • Verbal reasoning practice tests;
  • Numerical reasoning practice tests,
  • Abstract reasoning practice tests and
  • Decision-making practice aptitude tests.

Firstly, verbal analogies test.
Secondly, ability tests at basic, graduate and at senior managerial levels.
Thirdly, numeracy tests.
Fourthly, practice critical reasoning ability tests (for the LNAT);

Bespoke psychometric test designs

Job Preview Design

Many application processes now start online with a realistic job preview design that potential applicants take on the company website to “test” if the role still appears suitable after finding out about the realities of the role. Hence it’s a job preview but a realistic one.

Realistic role preview design

Typically, the test format is similar to an SJT in that job scenarios are presented which give the test-taker / job applicant an idea of what the job entails. Whilst the realistic job preview (RJP) is not a testing phase often there will be a more sophisticated SJT in a subsequent phase. In more sophisticated examples still graphics or video technology is used to further enhance the Realistic job preview / SJT experience for candidates.

Virtual strengths design

This are an area that Rob Williams Assessment Ltd also specialise in. This builds on our situational judgement test design and we are often asked to design both a realistic job preview and a situational judgement test at the same time.

Our approach to the design of realistic job previews has the following phases.

Step-by-step role preview design 

(1) Planning a representative sample

(2) Conducting a highly detailed  set of telephone interviews and focus groups.

(3) Item writing

(4) SJT and RJP item reviewing

(5) compile a trial version to be reviewed by 10-12 subject matter experts (SME’s) in the role

(6) Trialling the situational judgement test

(7) Statistical analysis

(8) Presentation of trial results at standard-setting meeting with core SME’s

(9) Validation and norming the situational judgement test

Bespoke psychometric test designs

Situational design Exercises

  • Scenarios from job analysis used to design simulation exercises.
  • Parallel version developed to maximise exercise integrity.
  • To minimise risk of applicants sharing details of tools.

The following situational strengths scenarios are examples only. Actual client projects always start with the most suitable level of role analysis. Then again validate the content against client requirement by choosing the most predictive scenarios and answer options.

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Situational Strengths Research

Employers are increasingly using bespoke situational judgement tests (where the candidate is presented with scenarios and asked to select the best and the worst thing to do next) as a way to learn more about their character and attitudes to work.

The increase in the number of graduate courses and the career benefits of having a degree have driven a huge increase in the number of graduates. There remains a limited number of vacancies each year on graduate trainee schemes.

During the recent years of recession, the number of graduate entry roles became even more restricted; making graduate recruitment even more competitive. Microsoft, for example, received 15,000 job applications for each of its 150 graduate position in 2009. That year, the success ratio of applications to job offers was 1:100 at Microsoft.

Bespoke psychometric test designs

This has created a “bottleneck” between the high number of recent graduates and the considerably lower number who are successfully placed on graduate entry schemes. Clearly, there are severe implications of such a challenging job market for graduates.

For employers too, there is a “war on talent” to find and sign-up the best possible graduates in the marketplace. It is a recruiters’ market, however most recruiters want to recruit the top echelon of high-performing graduates who are showing the best leadership potential, the most effective ability to work in teams, the highest levels of motivation and drive etc. SJTs offer an effective means of measuring each of these abilities and attributes.