How to evaluate and effectively use various methods and tools for: selecting high potential global leadership talent and skill development planning and training.
Personnel assessment refers to any method of collecting information on individuals for the purpose of making some type of (selection) decision. Selection decisions include, hiring, placement, promotion, referral, retention, and entry into programs leading to advancement (such as training and career development). Selecting qualified applicants is a critical step in building a talented and committed workforce, supporting an effective organizational culture, and enhancing the overall performance of the company.
Many different methods of assessment may be used for a variety of employment testing purposes such as recruitment, selection of high potentials, succession planning, creation of skill development training programs and so on. The most popular of these methods include:
Not all assessment tools are appropriate for every job and organizational setting. Some methods correlate better with actual job performance than others; and employers will often use more than one to maximize their predictive power and utility.
On the other hand, misuse of tools by inexperienced and uneducated “amateur psychologists” has led to many multimillion-dollar lawsuits in recent years. Employers and HR professionals must bear in mind when using these tools that the legal liability is on the organization that uses the tool, not the vendor that sells the tool. So it is in the best interest of the company to make sure the tool is the correct one for the application.
For a test to be considered a valid, fair and legal assessment of future work performance, it must actually measure either the task performing competence or the knowledge, skills and abilities workers must possess in order to perform the job effectively, and be proven by evidence that it does so consistently and reliably.
On the other hand, assessment tools not considered valid (enough) for selection purposes can still be very useful in designing a skill development programs and also provide a quick, easy and inexpensive way to confirm the effectiveness of that training program. Assessment methods intended to be used for designing or supplementing training programs will still need to be base-lined to a pre-determined ideal skillset or attitude in order to accurately assess the starting state and the finishing state of the trainees. However, the skillsets can be much more general and universal. Skillsets such as those required for global leadership, successful negotiation, or effective communication can be used as the baseline to measure the participants against. And generic models of these preferred skillsets exist already, making the assessment tool development process somewhat less expensive and quicker.
The first step in developing or selecting an assessment method for a specific employment position or objective is to understand what the job requires employees to do and what knowledge, skills and abilities (KSAs) individuals must possess in order to perform the job effectively. This is done through a job analysis. Conducting a job analysis involves collecting information from job subject matter experts. The term subject matter expert (SME) is applied to anyone who has direct, up-to-date experience of a job and is familiar with all of its tasks. The person might currently hold the job or supervise the job.
Also, basing personnel assessment closely on job analysis results makes the connection between job requirements and personnel assessment tools more transparent, thereby improving the perceived fairness of the assessment process and aiding in avoiding costly lawsuits.
The part of a job analysis that focuses on what the job requires individuals to do is often referred to as a task-based job analysis, which involves eliciting a comprehensive list of work tasks that individuals are required to perform.
The part of a job analysis that focuses on the KSAs that workers must possess to be effective is often referred to as a worker-oriented or KSA-based job analysis. Typically, a job analyst first identifies the tasks that workers are required to perform on the job and then identifies the KSAs that are needed to effectively perform those tasks.
Figure 1: Job Analysis Process
Job analysis information is used as a basis for developing assessment tools. Specifically, assessments are developed to measure the most critical tasks or KSAs resulting from the job analysis for a given job. Some assessments involve work samples that simulate job tasks and require candidates to demonstrate that they can perform these tasks effectively. Task-based job analysis data are used as a basis for developing these types of assessments because they focus directly on assessing how well job candidates can perform critical work tasks.
Other assessment methods focus on measuring KSAs that are required to perform job tasks effectively, such as various mental abilities, physical abilities or personality traits, depending on the job’s requirements. If one were selecting a manager for an overseas assignment, for example, it would be important to assess whether candidates could solve complex business problems, handle cultural differences, deal with ambiguity and communicate clearly and effectively. KSA-based job analysis data are used as a basis for developing assessment methods that focus on a job candidate’s underlying abilities to perform important work tasks.
Figure 2: KSA vs. Task Based Assessment
Reliability expresses how consistent a tool is in measuring the target areas. Reliability coefficients range from -1 to +1. As the coefficient approaches zero the correlation decreases until there is no (zero) correlation between the variables. However, as the reliability coefficient gets closer to either -1 or +1, the correlation increases and so does the reliability. While there are no commonly accepted standards for reliability in training and development assessments, reliability coefficients near .60 are generally considered acceptable.
The validity of an assessment is the degree to which it effectively measures what it claims to measure. Keep in mind that an assessment’s validity is dependent upon how the assessment is used. For example, if an assessment is to be used in the context of a training experience, then the assessment can be considered valid at a lower level. If the same assessment were to be used as a predictive tool for selection purposes, then it might not be valid.
Face validity answers the question, “Does the assessment make sense to the average person and will it help him or her learn more effective behavior?”
Selecting and using effective assessment methods can greatly enhance the quality and productivity of an organization’s workforce. On the other hand, misuse of these tools can have serious negative impacts on individuals and subsequently on those who misused them.
Assessment testing is very technical and difficult to understand without formal education in industrial and organizational psychology, which has led to many misunderstandings and under-utilization of such tools. By providing this basic approach for evaluating assessment methods we hope to enable decision makers and HR professionals to introduce effective assessment methods into their organizations and to use them in a positive and productive way.
This post is also available in: Japanese