Custom Assessment Design

Rob Williams Assessment Ltd specialise in psychometric test designs and custom assessment design services.

rrussellwilliams@hotmail.co.uk     I     +44 (0)77915 06395

Custom psychometric test designs

Rob Williams Assessment Ltd specialise in designing highly predictive psychometric solutions. In particular, situational judgement test design, realistic job preview design, aptitude test design and personality questionnaire design.

Headed up by Rob Williams – the highly experienced author of five books on psychometric testing. Our organisation prides itself on client satisfaction. We have many positive LinkedIn reviews from our big client projects.

Customised psychometric test design is our speciality. Another trend is offering company-specific tests. With a bespoke psychometric test, organisations can design the content of their test(s) to match their own industry sector.

Psychometric Fit Sifts

Many of the UK’s and US’s leading test publishers have used Rob Williams Assessment’s to calculate and prove the reliability of their newly developed psychometric tests.

Digital Assessments

Example of situational judgement tests 

You mention in passing to a colleague that you believe there are some financial risks associated with a financial product that’s about to be launched to customers. In a team meeting later that week, your colleague shares this information with your manager – without mentioning your name. How do you react?

You are then asked to select your most preferred and least preferred responses

(a) Apologise on your colleague’s behalf for their poor explanation.

(b) Suggest that your colleague does their own research.

(c) Ask your colleague to also include you in future.

(d) Check that your manager understands the risk involved.

By using real life scenarios, the idea is that employers will get a better understanding of how you might operate in the work place.

Branching situational judgement tests 

As psychometric tests have become more commonplace, the bigger users have commissioned their own bespoke situational judgement tests. Rob Williams Assessment has worked on several such projects for High Street banks and for the European Union. Another recent innovation of test developers has been online adaptive tests. With these tests, if you are doing well, you will find that the questions get progressively harder. That can feel like a challenge since you are pushed until you reach the most challenging level you can. This is the level at which you – just like other candidates with your level of verbal reasoning – start to get questions wrong.

The innovative design of shorter and more efficient tests was driven by an increasingly aware of the immediacy of the Internet and our increasing use of emails and social media in short, sharp bursts.  This discourages test takers from spending 30-40 minutes online doing the same questionnaire. Its better for everyone to keep test takers engaged when being tested – not bored!

So what will adaptive tests mean for you as a prospective test taker?  The biggest difference is the shortness of the test. The second major difference is that you will find an adaptive test more challenging. Without getting into their highly technical make-up, the test adapts to your ability level. More specifically it adapts to find the most challenging question that you can answer correctly.

In the past you may have found questions on a test fluctuating in difficulty or generally becoming more and more difficult the further on you get in the test. Consider a test of twenty questions with the first the easiest and the twentieth the most difficult.

Knowledge-based situational judgement tests 

Some or all of the scenarios presented in an SJT can test specific job knowledge. For example, a retail marketing SJT may ask questions about the 3Ps (price, position, promotion) of product marketing. Alternatively both an SJT measuring generic decision-making skills may be used alongside a knowledge-based test.

Video based Situational judgment tests 

Simulated situational judgement tests are increasingly common as recruitment sifts. Adding 2D or 3D workplace scenario graphics brings the situational judgment test scenarios to life. This can only promote the company brand and make employers using simulated situational judgment tests more desirable employers.

UK and US psychometric test publishers have produced both video-based and animated SJT scenarios. Animated SJTs are easier – and therefore cheaper – for global companies to develop.

Psychometric Design Criteria

There are two main types of reliability study: internal reliability and test re-retest reliability. Both represent best practice in psychometric design. Hence both reliability measures are key stages of a psychometric test development project.

Internal Reliability

An reliability value of 0.7 is the cutoff value for aptitude tests / ability tests. The measure used is called Cronbach’s Alpha.

A higher internal reliability value of 0.8 is considered acceptable for personality scales. There is academic debate about whether situational judgement tests can be expected to have internal reliability. Regardless of this Rob Williams Assessment ltd has developed SJTs with Cronbach’s Alpha values in excess of 0.8.

Test-Retest Reliability 

Test-retest reliability is a key final stage of test development. Typically, two administrations occur a few weeks apart. This tests the reliability of the two sets of results.

Bespoke situational judgement test design

Situational judgement test design clients include BUPA, Citibank, British Airways and Boots.

Bespoke aptitude test sifts

Aptitude test design clients include EPSO, Kenexa IBM and HireWindow.

What is a “Psychometric Test”?

A psychometric test is a series of written or practical tests which assess a clearly defined sample of human behaviour. There are many different types of psychometric tests, including measures of skill, ability, intelligence, personality, motivation and interests. Psychometric tests can play a useful role in the assessment and development of individuals. They can give objective evidence of the human attributes they measure, and have been shown to be one of the best predictors of job performance when used in selection. In practice, they are generally used in conjunction with other methods such as interviews to give the “whole picture”.

Psychometrically Sound Tests

To be psychometrically sound a test must be:

  • Objective – the results obtained are not influenced by the administrator’s personal characteristics or irrelevant factors such as the colour of a test-taker’s socks.
  • Standardised – the test is administered and scored according to standard procedures and people’s scores are compared to known standards.
  • Reliable – the test measures in a consistent way. The potential error is small and is quantifiable.
  • Valid – the test measures the characteristics which it set out to measure. A test used to select a job applicant should predict job performance. A test of verbal abilityy should predict this area and not some other skill.
  • Discriminating – the test should show clear differences between individuals on the behaviour being tested. It should not be discriminatory I.e. unfairly discriminating against minority groups on the basis of irrelevant characteristics

Example Psychometric Test Projects 

Rob Williams has over twenty years experience of bespoke psychometric test design. Plus ten years prior to this spent working for several of the UK’s leading test publishers. These include: IBM, OPP, SHL and HireVue.

British Airways 

Blended assessment project:

  • Situational judgment test design;
  • Numerical reasoning test design;
  • Verbal reasoning test design
  • Personality questionnaire design.

For these two British Airways roles:

– Cabin crew assessment; and

– Customer service representative assessment.

BUPA Psychometric Test Design

  • Best practice was followed throughout the design process in SJT design.
  • An SJT was produced which successfully incorporated a range of care home-specific scenarios.
  • The most suitable set of scenarios could be hand-picked at the SME panel meeting, as well as gaining buy-in and discussing implementation.
  • Providing some scenarios for telephone interview sifts.
  • Setting a suitable cut-off and validating the tool.

EPSO Psychometric Test Design

  • Development of project management test.
  • Design of IT skills-based aptitude tests.

Verbal Reasoning practice test book

41b7m573tbl
Firstly, Passing Verbal Reasoning Tests book by Rob Williams

Numerical Reasoning practice test book

Passing Numerical Reasoning Tests gif
Secondly, Passing Numerical Reasoning Tests book by Rob Williams

Assessment Design

Work Styles assessment

We will work with you to design the most suitable work styles tool to suit your needs. Our Bespoke Personality Questionnaire design process aims to:

  • Firstly, include key role dimensions.
  • Secondly, reflect the personality, attitudinal and motivational aspects of the role-specific dimensions.
  • Thirdly, have face valid questions.
  • Also, to be capable of completion in 20 minutes approx.
  • And to adopt a single-stimulus question format (Likert scale)
  • Plus, adopting a normative format of scoring utilising a sten look-up table (for each personality scale)
  • And finally, using a Social Desirability scale to deal with the issue of faking or extreme scoring patterns

Key work personality test design stages

Firstly, the ideal for personality questionnaire design is to have sets of items on each  scale measuring the same latent variable. Secondly, this is as described by the scale name and the scale descriptors.

Assessment Design

Our Brilliant Passing Verbal Reasoning Tests book

Free practice aptitude tests associated with this book are available on the publisher’s Pearson Prentice Hall website.

CONTENTS

part 1 Getting to grips with your test

  • Get started
  • Secondly, Practice makes perfect
  • Thirdly, Mastering reading comprehension
  • Fourthly, Sharpen your critical reasoning skills
  • Fifthly, Succeed on test day

part 2 Time to practice

  • Firstly, Warm up tests
  • Secondly, Reading comprehension tests
  • Thirdly, Verbal reasoning tests
  • Fourthly, Critical reasoning tests

Assessment Design

Numerical Reasoning Test Practice book

Firstly, our numerical reasoning test practice book contains hundreds of practise questions.

part 1 Getting to grips with your test.

  • Lets start.
  • Secondly, Practice makes perfect.
  • Thirdly, Brush up your maths skills.
  • Fourthly, Succeed on test day.

Assessment Design

part 2 Time to practice.

  • Firstly, Basic numerical reasoning tests.
  • Secondly, Numerical comprehension tests
  • Thirdly, Warm-up numerical reasoning tests
  • Fourthly, Numerical critical reasoning tests
  • Fifthly, Numerical data interpretation tests
  • Sixthly, Advanced numerical data interpretation

Assessment Interview Questions Example – for Data Scientist

  • Firstly, is there a trade-off between bias and variance?
  • Secondly, how can gradient descent be a problem?
  • Thirdly, when have you used Exploratory Data Analysis?
  • Fourthly, what steps do you take when choosing a Machine Learning model?
  • Next, are convolutions best for images?
  • Then, are Residual Networks important? 
  • Also, how does batch normalization work? 
  • And have you ever worked with an imbalanced dataset?
  • Next, what are the highlights of your MSc research? What didn’t work? What else could you have done? 
  • Then, how have you overcome over/ under fitting your data model?
  • What are dimensionality’s dangers?
  • Next, how would you run a Principal Component Analysis (PCA)?
  • What is data normalization?
  • Also, why is dimensionality reduction important?

London Private School Entrance Exam Practice

Firstly, Barnet.  Secondly, Brent. Thirdly, Camden private schools. Next, City of London. Then, Croydon. Also, Ealing. And Enfield. Plus, Greenwich. Then, Hackney. Also, Hammersmith & Fulham / Plus, Hampstead. Next, Haringey. Also, Harrow. And Havering. Next, Hillingdon. Also, Hounslow. Then, Islington. Plus, Kensington & Chelsea. And Kingston-upon-Thames / Also, LewishamMerton. And then Notting Hill. Plus also Redbridge. Then Richmond-upon-Thames. And next Streatham. Also Southwark. Plus Sutton. Next Wandsworth. And then Westminster. Finally Wimbledon.

Psychometric test design