Army Situational questions and Army situational judgement tests were designed.
To provide potential candidates with realistic (face valid) situations they are likely to encounter, and feedback on their responses. Such highly informative feedback aims to improve candidates’ understanding of the demands of Army life. Thus, better able to make informed judgements about their own suitability. As such, the SJTs are an important early selection filter, both for a candidate’s self-selection and as part of the advice given by Recruiting Officers. More specifically, the “formative” aims of the SJTs from a candidate’s perspective are to: firstly, identify likely suitability for undergoing and successfully completing Army training; and secondly, to broaden their understanding of the different demands of Army training (and future Army life).
Army situational judgement item writing design
From the start of the item-writing project phase, We emphasised the following points to its item writing teams:
- A training focus and the assessment of behaviours related to early stages of a person’s Army career; i.e. so-called ‘stick and fit’, or how likely a potential candidate is to successfully complete their training and how well they are likely to adapt to the very early stages of their Army career.
- In comparison with the optional ALP, the SJT is less transparent, reflecting its higher stakes design application, although the test results may be considered in conjunction when recruiting decisions are made.
Through feedback to both potential applicants and Recruiting Officers, SJTs will provide valuable information that will help Capita manage the risk associated primarily with training, but also with the very early stages of a person’s Army career. the purposes of the SJTs for Westbury’s recruiters are to improve Sandhurst’s training outcomes and the Army’s overall training selection efficiency. Specifically to:
Army Situational Judgement Test Design
Given the SJT’s military context, the test design’s aims were to:
After consideration of the target audience, it was decided that, Officer candidates would use a response format utilising ‘best’ and ‘worst’ response options. From the four options presented, candidates would be asked to indicate which option is the ‘best’ and which option is the ‘worst’ in relation to the scenario.
A significant advantage of this approach is that it offers a greater potential range of scores to be obtained from the same number of items than if only the ‘right’ or ‘best’ answer were used (Hauenstein et al, 2010). An increased range of scores will aid the statistical analysis of items during the development stage of the SJTs and the eventual reliability of the assessment, as this is linked to the spread of scores obtained from a test.
The use of an ‘effectiveness scale’ was also debated, as this has been shown to positively affect validity (Scott and Creighton, 2006). With this approach, each response option is individually rated in terms of its potential effectiveness for successfully addressing the situation.
Army Situational Judgement Test Validity
SJTs have higher validity if respondents are asked what they ‘would do’ as opposed to what they ‘should do’ (Ployhart and Ehrhart, 2003). ‘Would do’ prompts were therefore used because they are more likely to elicit responses that reflect respondents’ behavioural preferences, rather than their rating of what they consider would be the most appropriate thing to do (regardless of whether they would do this, or act otherwise).
Varying schools of thought exist with regard to response format. Whilst some authors argue that having ‘right’ as well as ‘wrong’ responses can make SJTs too obvious (Hauenstein et al, 2010), only scoring right options truncates the information available for scoring. Thus, we took the a priori decision to score both right and wrong options.
Army Situational Judgement Design
Our situational judgement design requirements were as follows:
Before you begin, remember:
- The answer options for each situation are all different, so read them through carefully.
- For each question, to indicate which action you consider would be best and worst.
- To be honest about what you think you would do in each situation.
- That you don’t need any Army experience to answer the questions, and:
- The SJT is not timed, but most people take about 30 minutes to complete it.
Drawing together previous research, with a particular emphasis on that of Riley and Walker-Smith (2006), the following areas were expected to emerge through the design process: Discipline; Military Awareness; Commitment; Sociability; and Resilience.
- Military awareness
See this page for more info on the Army Situational judgement test design and the Officer Situational questions design.