Types of Validity [Updated 2024]

Definition of Validity

The validity of a test is defined as the extent to which the test measures, what it design to measure. Aiken Psychological Testing and Assessment

In this article I will discuss major four types of validity and the question each addresses. I will also concluded regarding what are types of psychology.

 

Types of Validity

Content Validity in Psychology | Types of Validity Psychology

Types of Validity | Content Validity in Psychology is based on the extent to which a measurement reflects the specific intended domain of content. (Carmines and Zeller, 1991).

Content Validity in Psychology is a non-statistical type of validity that involves “the systematic examination of the test content to determine whether it covers a representative sample of the behavior domain to be measured” (Anastasi and Urbina, 1997). 

For example, does an IQ questionnaire have items covering all areas of intelligence discussed in the scientific literature?

Content validity in psychology evidence involves the degree to which the content of the test matches a content domain associated with the construct. For example, a test of the ability to add two numbers should include a range of combination of digits.

A test with only one-digit numbers, or only even numbers, would not have good coverage of the content domain. Content related evidence typically involves subject matter experts (SME's) evaluating test items against the test specifications.

A test has content validity in psychology built into it by careful selection of which items to include (Anastasi and Urbina, 1997). Items are chosen so that they comply with the test specifications which is drawn up through a detailed examination of the subject domain. 

Foxcraft et al. (2004) note that by using a panel of experts to review the test specifications and the selection of items the content validity of a test can be improved.

The experts will be able to review the items and comment on whether the items cover a representative sample of the behavior domain. Content validity in psychology is illustrated using the following examples.

Example of Content Validity in Psychology

Researchers aim to study mathematical learning and create a survey to test for mathematical skill.

If these researchers only tested for multiplication and then drew conclusions from that survey, their study would not show content validity in psychology because it excludes other mathematical functions.

Although the establishment of content validity in psychology for placement-type exams seems relatively straight-forward, the process becomes more complex as it moves into the more abstract domain of socio-cultural studies.

For example, a researcher needing to measure an attitude like self-esteem must decide what constitutes a relevant domain of content for that attitude. For socio-cultural studies, content validity forces the researchers to define the very domains they are attempting to study.

A test has content validity if it sufficiently covers the area that it is intended to cover. This is particularly important in ability or attainment tests that validate skills or knowledge in a particular domain.

In psychometric, content validity in psychology (also known as logical validity) refers to the extent to which a measure represents all facets of a given social construct.

For example, a depression scale may lack content validity if it only assesses the effective dimension of depression but fails to take into account the behavioral dimension. 

An element of subjectivity exists in relation to determining content validity, which requires a degree of agreement about what a particular personality trait such as extroversion represents. A disagreement about a personality trait will prevent the gain of a high content validity.

Content validity is related to face validity, but they should not be confused. The latter is not validity in the technical sense; it refers, not to what the test actually measures, but to what it appears superficially to measure. 

Face validity in psychology pertains to whether the test "looks valid" to the examinees who take it, the administrative personnel who decide on its use and other technically untrained observers.

Content validity in psychology requires more rigorous statistical tests than face validity, which only requires an intuitive judgment.

Content validity in psychology is most often addressed in academic and vocational testing, where test items need to reflect the knowledge actually required for a given topic area (e.g., history) or job skill (e.g., accounting). 

In clinical settings, content validity refers to the correspondence between test items and the symptom content of a syndrome. One widely used method of measuring content validity was developed by C. H. Lawshe. It is essentially a method for gauging agreement among raters or judges regarding how essential a particular item is.

Lawshe (1975) proposed that each of the subject matter expert raters (SMEs) on the judging panel respond to the following question for each item: "Is the skill or knowledge measured by this item 'essential,' 'useful, but not essential,' or 'not necessary' to the performance of the construct?"

According to Lawshe, if more than half the panelists indicate that an item is essential, that item has at least some content validity. Greater levels of content validity exist as larger numbers of panelists agree that a particular item is essential. Using these assumptions, Lawshe developed a formula termed the content validity ratio:

CVR = (ne - N/2)/ (N/2)

CVR= content validity ratio, ne= number of SME panelists indicating "essential", N= total number of SME panelists. This formula yields values which range from +1 to -1; positive values indicate that at least half the SMEs rated the item as essential. The mean CVR across items may be used as an indicator of overall test content validity.

Face Validity in Psychology | Types of Validity in Psychology

Types of Validity | What is Face Validity in Psychology - Face validity in Psychology is that the test appears to be valid. This is validated using common-sense rules, for example that a mathematical test should include some numerical elements.

A test can appear to be invalid but actually be perfectly valid, for example where correlations between unrelated items and the desired items have been found. For example, successful pilots in WW2 were found to very often have had an active childhood interest in flying model planes.

Types of Validity | A test that does not have face validity may be rejected by test-takers (if they have that option) and also people who are choosing the test to use from amongest a set of options.

Face validity in psychology is concerned with how a measure or procedure appears.

Does it seem like a reasonable way to gain the information the researchers are attempting to obtain?

Do it seems well designed?

Does it seem as though it will work reliably?

Types of Validity | Unlike content validity, face validity in psychology does not depend on established theories for support (Fink, 1995).

Face validity in psychology is an estimate of whether a test appears to measure a certain criterion; it does not guarantee that the test actually measures phenomena in that domain. Indeed, when a test is subject to faking (malingering), low face validity might make the test more valid.

Face validity in psychology is very closely related to content validity. While content validity depends on a theoretical basis for assuming if a test is assessing all domains of a certain criterion (e.g. does assessing addition skills yield in a good measure for mathematical skills?

To answer this you have to know, what different kinds of arithmetic skills, mathematical skills include) face validity relates to whether a test appears to be a good measure or not. This judgment is made on the "face" of the test, thus it can also be judged by the amateur.

Face validity in psychology is a starting point, but should NEVER be assumed to be provably valid for any given purpose. As the "experts have been wrong before the Malleus Malificarum (Hammer of Witches) had no support for its conclusions other than the self-imagined competence of two "experts" in "witchcraft detection. 

Yet it was used as a "test" to condemn and burn at the stake perhaps 100,000 women as "witches."

Criterion Related Validity in Psychology | Psychology Types of Validity

Types of Validity | What is Criterion related validity in psychology - It is like construct validity. But now relates the test to some external criterion. Such as particular aspects of the job.

There are dangers with the external criterion being selected based on its convenience. Rather than being a full representation of the job. For example an air traffic control test may use a limited set of scenarios.

Criterion validity in psychology evidence involves the correlation between the test and a criterion variable (or variables) taken as representative of the construct. In other words, it compares the test with other measures or outcomes (the criteria) already held to be valid.

Types of Validity | For example, employee selection tests are often validated against measures of job performance (the criterion). And IQ tests are often validated against measures of academic performance (the criterion).

If the test data and criterion data are collected at the same time, this is referred to as concurrent validity evidence. If the test data is collected first in order to predict criterion data collected at a later point in time. Then this is referred to as predictive validity evidence.

Criterion related validity in psychology, also referred to as instrumental validity. It is used to demonstrate the accuracy of a measure or procedure, by comparing it with another measure or procedure which has been demonstrated to be valid.

For example, imagine a hands-on driving test has been shown to be an accurate test of driving skills. By comparing the scores on the written driving test with the scores from the hands-on driving test. The written test can be validated by using a criterion related strategy in which the hands-on driving test is compared to the written test. Types of Validity.

In psychometric, criterion validity is a measure of how well one variable or set of variables predicts an outcome based on information from other variables. And will be achieved if a set of measures from a personality test relate to a behavioral criterion that psychologists agree on.

A typical way to achieve this is in relation to the extent to which a score on a personality test can predict future performance or behavior. Another way involves correlating test scores with another established test that also measures the same personality characteristic.

Criteria or concrete validity is the extent to which the measures are demonstrably related to concrete criteria in the "real" world. This type of validity is often divided into "concurrent" and "predictive" sub-types. 

Construct Validity in Psychology | Different Types of Validity

Types of Validity | Underlying many tests is a construct or theory that is being assessed. For example, there are a number of constructs for describing intelligence (spatial ability, verbal reasoning, etc.) which the test will individually assess.

What is construct validity in psychology - Constructs can be about causes, about effects and the cause effect relationship. Types of Validity.

If the construct is not valid then the test on which it is based will not be valid. For example, there have been historical constructs that intelligence is based on the size and shape of the skull.

Construct validity in psychology seeks agreement between a theoretical concept and a specific measuring device or procedure. For example, a researcher inventing a new IQ test might spend a great deal of time attempting to "define" intelligence in order to reach an acceptable level of construct validity.

Types of Validity - To understand whether a piece of research has construct validity, three steps should be followed.

1. First, the theoretical relationships must be specified. 
2. Second, the empirical relationships between the measures of the concepts must be examined. 
3. Third, the empirical evidence must be interpreted in terms of how it clarifies the construct validity of the particular measure being tested. (Carmines and Zeller)

Construct validity refers to the extent to which operationalization of a construct (e.g. practical tests developed from a theory) do actually measure what the theory says they do. For example, to what extent is an IQ questionnaire actually measuring "intelligence"?

Construct validity in psychology evidence involves the empirical and theoretical support for the interpretation of the construct. Such lines of evidence include statistical analysis of the internal structure of the test including the relationships between responses to different test items. They also include relationships between the test and measures of other constructs.

As currently understood, construct validity is not distinct from the support for the substantive theory of the construct that the test is designed to measure. As such, experiments designed to reveal aspects of the causal role of the construct also contribute to construct validity evidence.

Conclusion

Types of Validity in Psychology – Validity is about the accuracy of a test measure. Without calculating validity of any test we can’t collect accurate data for our research. It indicates that how a test or method measures something.

What is Validity in Psychology | Detailed Introduction to Validity

Other Different Types of Validity