Psychometric development

Strengths Design

Assessment design projects involving virtual strengths design are one of our many psychometric test design specialities.

Virtual Strengths Design

Situational judgement tests (SJTs) have also become prevalent in graduate recruitment. These tests present scenarios to applicants and asks them to select the best and the worst thing to do next. The scenarios are set within the context of the recruiting organisation, so the questions are usually perceived favourably by candidates. SJTs are very popular in the United States due to their excellent record of fairness across different ethnic groups.

We have experience of designing situational strengths tests that are generic in nature. In addition to these SJT-based situational strengths tests, we offer these two alternative psychometric test designs.

Education and public sector aptitude test design projects

  • Numerical reasoning, verbal reasoning, abstract reasoning and decision analysis tests.
  • Critical reasoning aptitude test (legal sector).
  • Situational judgement test (health sector).

Virtual Strengths Design

Citibank Aptitude Test Design Example

  • Designing blended assessments for retail and call centre operations across the USA.
  • Allowing for cultural differences in Mexican operations.
  • Managing item writing team for SJTs, ability tests, biodata questions etc.

EPSO Aptitude Test Design Project

  • Development of project management test.
  • Design of IT skills-based aptitude tests.

Assessments Design

This psychometric test design is similar in nature to personality questionnaire design, explained here. Our bespoke strengths test design process…

  1. Has dimensions key to the role being assessed.
  2. Reflect the personality, attitudinal and motivational aspects of the role-specific dimensions.
  3. Have face valid questions.
  4. Be capable of completion in 20 minutes approx.
  5. Adopt a single-stimulus question format (Likert scale).
  6. Adopt a normative format of scoring utilising a sten look-up table (for each personality scale).

Virtual Strengths Design

Situational Strengths Test

An excellent example is CAPP’s Situational Strengths Test. This builds on the many advantages that SJTs have, being:

–Easy to administer to a large audience.

–Relatively strong validity.

–Can assess a sample of the relevant KSA’s.

–Refreshed items possible.

–More difficult to fake responses.

Bespoke VBR Example

  • 3-4 scenarios will assess each value, giving approx. 15-16 questions.
  • Provides accurate and meaningful feedback to each respondent.
  • This practical job preview has the advantage of using the same SJT format.

Situational interview(s) comprising:

  • Values.
  • Competency; and
  • Technical questions.

Situational Simulation Exercises

  • Scenarios from job analysis used to design simulation exercises.
  • Parallel version developed to maximise exercise integrity.
  • And to minimise the risk of applicants sharing details of tools.
  • Without compromising the validity of the assessment process

Situational Values-Based Interview

  • We propose a structured interview comprising values, competency and technical pharmacy questions
  • Collaborate with job incumbents to design technical questions/scoring guide.

A situational judgement test, or SJT, presents realistic scenarios similar to those that would be encountered when doing a particular job. Typically, candidates must identify the Best and the Worst course of action from four multiple-choice options. In other words, you must make two “judgements” about what to do and what not to do next in the “situation” presented. An alternative SJT format asks the test-taker to rank multiple-choice options in terms of effectiveness.

Situational strengths in the news

On BBC Radio 2, in February 2014, Dr Almuth McDowell referred to the benefits of situational judgement tests and one of the SJT example items developed by Rob Williams Assessment Ltd.

The same interview on SJT benefits also featured in The Times on 4th Feb, Dr Almuth McDowell, a lecturer at the University of Surrey, says psychometric testing has an important role to play, but only in conjunction with other measures, not least because it is possible to cheat. 

Additional situational judgment test tips are available in Rob Williams Assessment Ltd’s latest Career Builder article entitled “How to succeed at management interview tests”.

Strengths Assessment Development

Psychometric test examples are useful to see the type of strengths development testing approaches.

In fact, there are many many types of psychometric test examples. Let’s consider the many different strengths that a person may possess. For example, their intelligence strengths and weaknesses from the many different types of intelligence.

The Intelligence Test demonstrates the range of possible intelligence strengths. There are many types of intelligence. For example, creative intelligence and musical aptitude which we have not included in our model.

Strengths Assessment Design

An effective competency design framework is crucial to the operation of many HR practices.

Shown below is an example of a typical competency design framework used in an assessment centre. In this case, the group exercise’s competency component of the overall assessment centre competency matrix:

– Oral Communication

How clearly and confidently the individual communicates with the group.

– Planning and Organising

How much consideration is given to planning and systematically going through the issues outlined in the scenario.

– Judgement and Decision-Making

How logically the individual makes their decisions and judges other participants’ input/comments.

– Analysis and Problem-Solving

How effectively the individual analyses the scenario’s issues and the solutions proposed.

– Finding Solutions

The creative thinking that is shown. The number and effectiveness of the ideas generated.

– Teamworking

How well the individual works with and encourages the other group exercise participants.

Strengths framework design

Having a competency framework helps an organisation to:

– Know which knowledge, skills and abilities are important for success in each role

– Measure and benchmark organisational performance

– Select, develop and performance manage employees (against the competencies associated with each role).

– Is it more cost effective to adapt an already validated model?

– How best to get buy-in across the business? Firstly, for involvement in the development stage. Secondly, to get commitment to future use of the validated framework.

– What role analysis techniques are most suitable for analysing each of the job roles?

– How to benchmark employee performance against the trial competency framework?

– How to validate the effectiveness of the draft competency framework?

Strengths-based Telephone interview design

Previous projects that have encompassed telephone interview sift designs. These include:

  • Competency-based sift design for telephone and cv-based sifts (finance sector).
  • Telephone interview script design (consultancy firm).
  • Conducting telephone interviews for the armed services.
  • Telephone interview design (manufacturing sector).

Strengths Development – Role Analysis

There follows a Role Analysis Case Study: a US bank had two specific requirements: ensuring a representative sample by samplying sufficiently high number of people per role. Interviewees were mainly high performers; sampled across both urban and rural branches; with their offices scattered right across the U.S.

1) Strengths Telephone-based interviews

Firstly, these formed a large part of the job analysis research. The advantages of using a telephone-based approach was that a standard template could be created with a script to adhere to whilst also allowing some follow-up question flexibility.

2) Assessment Designs – Visionary interviews

Secondly, these were run on a more select basis with a range of senior managers (within each job role). A scripted template ensured that all psychologists asked the standard set of questions. Face-to-face visionary interviews were also arranged with the most senior personnel at some of the focus group sites.

3) Strengths Focus groups

Thirdly, these were conducted in several American States, situated in Central, Eastern and Western time zones. A semi-structured focus group format represented the most suitable job analysis technique to use. Six participants, drawn from each job role, were invited to attend each role-specific focus group session.

4) Strengths Role survey

In addition, a role survey was created based on the bank’s competency framework, with two questions per competency. Survey results were also used to inform the item writing process.

The final 5-10 minute survey had two banks of questions with the stems:

  • How important do you do each of the following on a daily basis? and how important are the following to your job?

10 personality tips to help your Study Skills

  1. Find your best time to study. 
  2. Keep to a routine. Work in the same place at the same time each day. Also, make sure you have everything you need before you start.
  3. Work to your strengths. Schedule challenging tasks for when you are most alert. Routine ones for when you’re tired.
  4. Don’t waste time. Skim and scan before reading something in-depth.
  5. Avoid distractions. Switch emails and social media off to prevent your mind wandering !
  6. Regularly review your notes. Edit out what you don’t need. Ask yourself, how does it relate to what I already know.
  7. Vary how you to take notes – For example, use Mind Maps and diagrams to generate ideas.
  8. Be critical. Maintain a critical and analytical approach at all times!
  9. Plan your work. So, produce a detailed plan first.
  10. Understand different styles.

Example job analysis briefings

Candidate Experience Briefing – A separate briefing detailed the client style and language (US English) needs.

The main aims from a “Candidate Experience” perspectives included:

  • Reflecting the company brand
  • Engaging job applicants
  • Providing some elements of a realistic job preview
  • Overall look and feel to be consistent

Assessment Designs – Focus Group Briefing

The focus group aims were to confirm the role profile interview data.

Firstly, focuses on how best to structure the upcoming focus group sessions. Secondly, a consideration of the outputs required. Thirdly, how individual consultants could best run their focus group sessions.

Situational Strengths Test

There is a specialised form of situational judgement test called the Situational Strengths Test. One company that uses this test is Lloyds pharmacy, where the CAPP Situational Strengths Test is used to recruit pharmacists.

Situational strengths tests have these advantages (in any context):

– Easy to administer to large audience

– Relatively strong validity

– Can assess sample of the relevant KSA’s

– Refreshed items possible

– More difficult to fake responses

Virtual strengths design

Bespoke VBR Example

  • 3-4 scenarios will assess each value, giving approx. 15-16 questions
  • Provides accurate and meaningful feedback to each respondent
  • This practical job preview has the advantage of using the same SJT format

Assessment Designs – Situational Exercises

  • Scenarios from job analysis used to design simulation exercises
  • Parallel version developed to maximise exercise integrity and minimise risk of applicants sharing details of tools and compromising the validity of the assessment process

The following situational strengths scenarios are examples only. Actual client projects always start with the most suitable level of role analysis. Then again validate the content against client requirement by choosing the most predictive scenarios and answer options.

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Each scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Aptitude test design projects

Ability tests for graduates, including problem-solving aptitude tests and abstract reasoning tests.

UKCAT test practice questions for several clients. Each project encompas

  • Verbal reasoning practice tests;
  • Numerical reasoning practice tests,
  • Abstract reasoning practice tests and
  • Decision-making practice aptitude tests.

Firstly, verbal analogies test.
Secondly, ability tests at basic, graduate and at senior managerial levels.
Thirdly, numeracy tests.
Fourthly, practice critical reasoning ability tests (for the LNAT);

Virtual strengths design

Job Preview Design

Many application processes now start online with a realistic job preview design that potential applicants take on the company website to “test” if the role still appears suitable after finding out about the realities of the role. Hence it’s a job preview but a realistic one.

Realistic role preview design

Typically, the test format is similar to an SJT in that job scenarios are presented which give the test-taker / job applicant an idea of what the job entails. Whilst the realistic job preview (RJP) is not a testing phase often there will be a more sophisticated SJT in a subsequent phase. In more sophisticated examples still graphics or video technology is used to further enhance the Realistic job preview / SJT experience for candidates.

Virtual strengths design

This are an area that Rob Williams Assessment Ltd also specialise in. This builds on our situational judgement test design and we are often asked to design both a realistic job preview and a situational judgement test at the same time.

Our approach to the design of realistic job previews has the following phases.

Step-by-step role preview design 

(1) Planning a representative sample

(2) Conducting a highly detailed  set of telephone interviews and focus groups.

(3) Item writing

(4) SJT and RJP item reviewing

(5) compile a trial version to be reviewed by 10-12 subject matter experts (SME’s) in the role

(6) Trialling the situational judgement test

(7) Statistical analysis

(8) Presentation of trial results at standard-setting meeting with core SME’s

(9) Validation and norming the situational judgement test

Aptitude test practice books

Rob Williams’s five practice aptitude tests books are all available on Amazon:

Firstly, in our opinion this is the best aptitude test practice book for Passing Verbal Reasoning Tests.

Secondly, in our opinion this is the best aptitude test practice book forPassing Numerical Reasoning Tests.

Assessment Design

Situational Strengths Test

One company that uses this test is Lloyds pharmacy, where the CAPP Situational Strengths Test is used to recruit pharmacists.

Situational strengths tests have these advantages (in any context):

– Easy to administer to large audience.

– Relatively strong validity.

– Can assess sample of the relevant KSA’s.

– Refreshed items possible.

– More difficult to fake responses.

Situational Exercises

  • Scenarios from job analysis used to design simulation exercises.
  • Parallel version developed to maximise exercise integrity.
  • To minimise risk of applicants sharing details of tools.

The following situational strengths scenarios are examples only. Actual client projects always start with the most suitable level of role analysis. Then again validate the content against client requirement by choosing the most predictive scenarios and answer options.

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

Every scenario used also gives candidates a realistic insight into the role. Thus your applicants’s understanding of their own fit with the role. This operates in addition to the assessment component (of the same situational strengths test).

strengths design

Situational Strengths Research

Employers are increasingly using bespoke situational judgement tests (where the candidate is presented with scenarios and asked to select the best and the worst thing to do next) as a way to learn more about their character and attitudes to work.

The increase in the number of graduate courses and the career benefits of having a degree have driven a huge increase in the number of graduates. There remains a limited number of vacancies each year on graduate trainee schemes.

During the recent years of recession, the number of graduate entry roles became even more restricted; making graduate recruitment even more competitive. Microsoft, for example, received 15,000 job applications for each of its 150 graduate position in 2009. That year, the success ratio of applications to job offers was 1:100 at Microsoft.

This has created a “bottleneck” between the high number of recent graduates and the considerably lower number who are successfully placed on graduate entry schemes. Clearly, there are severe implications of such a challenging job market for graduates.

For employers too, there is a “war on talent” to find and sign-up the best possible graduates in the marketplace. It is a recruiters’ market, however most recruiters want to recruit the top echelon of high-performing graduates who are showing the best leadership potential, the most effective ability to work in teams, the highest levels of motivation and drive etc. SJTs offer an effective means of measuring each of these abilities and attributes.

Situational Strengths Design


Situational judgment tests in the news

On BBC Radio 2, in February 2014, Dr Almuth McDowell referred to the benefits of situational judgement tests and one of the SJT example items developed by Rob Williams Assessment Ltd.

The same interview on SJT benefits also featured in The Times on 4th Feb, Dr Almuth McDowell, a lecturer at the University of Surrey, says psychometric testing has an important role to play, but only in conjunction with other measures, not least because it is possible to cheat. 

Additional situational judgment test tips are available in Rob Williams Assessment Ltd’s latest Career Builder article entitled “How to succeed at management interview tests”.

Example of situational judgement tests 

You mention in passing to a colleague that you believe there are some financial risks associated with a financial product that’s about to be launched to customers. In a team meeting later that week, your colleague shares this information with your manager – without mentioning your name. How do you react?

You are then asked to select your most preferred and least preferred responses

(a) Apologise on your colleague’s behalf for their poor explanation.

(b) Suggest that your colleague does their own research.

(c) Ask your colleague to also include you in future.

(d) Check that your manager understands the risk involved.

By using real life scenarios, the idea is that employers will get a better understanding of how you might operate in the work place.

Branching situational judgement tests 

As psychometric tests have become more commonplace, the bigger users have commissioned their own bespoke situational judgement tests. Rob Williams Assessment has worked on several such projects for High Street banks and for the European Union. Another recent innovation of test developers has been online adaptive tests. With these tests, if you are doing well, you will find that the questions get progressively harder. That can feel like a challenge since you are pushed until you reach the most challenging level you can. This is the level at which you – just like other candidates with your level of verbal reasoning – start to get questions wrong.

The innovative design of shorter and more efficient tests was driven by an increasingly aware of the immediacy of the Internet and our increasing use of emails and social media in short, sharp bursts.  This discourages test takers from spending 30-40 minutes online doing the same questionnaire. Its better for everyone to keep test takers engaged when being tested – not bored!

So what will adaptive tests mean for you as a prospective test taker?  The biggest difference is the shortness of the test. The second major difference is that you will find an adaptive test more challenging. Without getting into their highly technical make-up, the test adapts to your ability level. More specifically it adapts to find the most challenging question that you can answer correctly.

In the past you may have found questions on a test fluctuating in difficulty or generally becoming more and more difficult the further on you get in the test. Consider a test of twenty questions with the first the easiest and the twentieth the most difficult.

Knowledge-based situational judgement tests 

Some or all of the scenarios presented in an SJT can test specific job knowledge. For example, a retail marketing SJT may ask questions about the 3Ps (price, position, promotion) of product marketing. Alternatively both an SJT measuring generic decision-making skills may be used alongside a knowledge-based test.

Video based Situational judgment tests in 2018

Simulated situational judgement tests are increasingly common as recruitment sifts. Adding 2D or 3D workplace scenario graphics brings the situational judgment test scenarios to life. This can only promote the company brand and make employers using simulated situational judgment tests more desirable employers.

UK and US psychometric test publishers have produced both video-based and animated SJT scenarios. Animated SJTs are easier – and therefore cheaper – for global companies to develop.

situational strengths design

Situational judgement test research 2010 – 2018

Becker’s development and validation of a situational judgment test of employee integrity.

Secondly, Bergman’s scoring situational judgment tests: Once you get the data, your troubles begin.

Next, Bledlow’s situational judgment test of personal initiative and its relationship to performance.

Also, Campion’s state of research on situational judgment tests: A content analysis and directions for future research.

Then, Catano, V. M., Brochu, A. & Lamerson, C. D. (2012). Assessing the reliability of situational judgment tests used in high_stakes situations. International Journal of Selection and Assessment, 20(3), 333–346.

Also, Chan’s situational judgment and job performance.

Next, Guenole’s Are situational judgment tests precise enough for feedback in leadership development?

Houston’s development of the enlisted computer adaptive personality scales.

Plus, Krumm’s how “situational” is judgment in situational judgment tests?

Also, McDaniel;s towards an understanding of situational judgment item validity and group differences.

Then, McDaniel’s situational judgment tests.

Mumford’s development and validation of a team role knowledge situational judgment test.

Next, Peus’ situation-based measurement of the full range of leadership model. Development and validation of a situational judgment test.

Also, Putting judging situations into situational judgment tests: Evidence from intercultural multimedia situational judgement tests.

Finally, Sharma’s development and validation of a situational judgment test of emotional intelligence.

Situational judgement test research 2010 – 2019

Firstly, Becker’s development and validation of a situational judgment test of employee integrity.

Secondly, Bergman’s scoring situational judgment tests: Once you get the data, your troubles begin.

Next, Bledlow’s situational judgment test of personal initiative and its relationship to performance.

Also, Campion’s state of research on situational judgment tests: A content analysis and directions for future research.

Then, Catano, V. M., Brochu, A. & Lamerson, C. D. (2012). Assessing the reliability of situational judgment tests used in high_stakes situations. International Journal of Selection and Assessment, 20(3), 333–346.

Also, Chan’s situational judgment and job performance.

Next, Guenole’s Are situational judgment tests precise enough for feedback in leadership development?

Situational judgement test research 2010 – 2019 Part II

Also, Houston’s development of the enlisted computer adaptive personality scales.

Plus, Krumm’s how “situational” is judgment in situational judgment tests?

Also, McDaniel;s towards an understanding of situational judgment item validity and group differences.

Situational Strengths Design

Then, McDaniel’s situational judgment tests.

And finally, Mumford’s development and validation of a team role knowledge situational judgment test.

SJT research 2008 – 2019 Part III

Next, Peus’ situation-based measurement of the full range of leadership model. Development and validation of a situational judgment test.

Also, Putting judging situations into situational judgment tests: Evidence from intercultural multimedia situational judgement tests.

Finally, Sharma’s development and validation of a situational judgment test of emotional intelligence.

Secondly, Campion’s state of research on situational judgment tests: A content analysis and directions for future research.

Thirdly, Catano’s assessing the reliability of situational judgment tests used in high stakes situations.

Fourthly, Guenole’s are situational judgment tests precise enough for feedback in leadership development?

Fifthly, Krumm’s how “situational” is judgment in situational judgment tests?

Situational judgement test research 2008 – 2019 Part IV

Firstly, Lievens’ situational judgment tests: From measures of situational judgment to measures of general domain knowledge.

Secondly, McDaniel’s Toward an understanding of situational judgment item validity and group differences.

Thirdly, Mumford’s team role test. The development and validation of a team role knowledge situational judgment test.

Situational Strengths Design

Fourthly, Peus’ situation-based measurement of the full range of leadership model. The development and validation of a situational judgment test.

Then, next is Sharma’s development and validation of a situational judgment test of emotional intelligence.

And next is Rockstuhl’s putting judging situations into situational judgment tests: Evidence from intercultural multimedia situational judgment tests.

Plus, Weekley’s low-fidelity simulations.

Then finally, Westring’s estimating trait and situational variance in a situational judgment test.

SJT research 2008 – 2019 Part VI

Firstly, Allen, V., Rahman, N., Weissman, A., MacCann, C., Lewis, C., & Roberts, R. D. (2015). The Situational Test of Emotional Management–Brief (STEM-B): Development and validation using item response theory and latent class analysis. Personality and Individual Differences, 81, 195- 200.

Secondly, Ekman, P. (1992). An argument for basic emotions. Cognition & emotion6(3-4), 169-200.

Thirdly, Bledow, R., & Frese, M. (2009). A situational judgment test of personal initiative and its

Situational Strengths design

relationship to performance. Personnel Psychology62(2), 229-258.

Fourthly, Carver, C. S., Scheier, M. F., & Weintraub, J. K. (1989). Assessing coping strategies.

And next Chan, D., & Schmitt, N. (2017). Situational judgment tests. The Blackwell handbook of personnel selection.

Then finally, Ambady, N. (2003). When familiarity breeds accuracy: Cultural exposure and

facial emotion recognition. Journal of personality and social psychology85(2), 276-290.

Situational judgement test research 2008 – 2019 Part V

Firstly, Elfenbein, H. A., Der Foo, M., White, J., Tan, H. H., & Aik, V. C. (2007). Reading your

counterpart: The benefit of emotion recognition accuracy for effectiveness in negotiation. J

Secondly, Farh, C. I., Seo, M. G., & Tesluk, P. E. (2012). Emotional intelligence, teamwork effectiveness, and job performance: The moderating role of job context. Journal of Applied Psychology.

Thirdly, Heggestad, E. D., & Morrison, M. J. (2008). An inductive exploration of the social effectiveness construct space. Journal of personality.

Fourthly, Hogan, R., Chamorro-Premuzic, T., & Kaiser, R. B. (2013). Employability and career success:

Situational Strengths Tests

Bridging the gap between theory and reality. Industrial and Organizational Psychology.

And then, Kaplan, S., Cortina, J., Ruark, G., LaPort, K., & Nicolaides, V. (2014). The role of organizational leaders in employee emotion management: A theoretical model. The Leadership Quarterly25(3), 563-580.

Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology75(6), 640-647.

Ng, T. W., Eby, L. T., Sorensen, K. L., & Feldman, D. C. (2005). Predictors of objective and subjective career success: A meta‐ analysis. Personnel psychology58(2), 367-408.

Then finally, Schlegel, K., & Scherer, K. R. (2016). Introducing a short version of the Geneva Emotion Recognition Test (GERT-S): Psychometric properties and construct validation. Behavior research methods, 48(4), 1383-1392.

Strengths design

Situational strengths tests

Firstly, our situational judgement test designs.

Also finally, our Army situational judgement tests.

Our Psychometric Design Toolkit

Firstly, our strengths psychometric ~ Secondly, our psychometric test design ~ Then our personality test design ~ And finally our Realistic Job Preview Design.

Our psychometric test designs

Strengths Design  ~ Realistic Job Preview Design ~ Personality Test Design ~ Situational Judgment Test Design ~ Psychometric Test Design.

situational strengths design