LESSON   7

VALIDITY

7-1  Validity

Primary objective of most epidemiologic research - obtain valid estimate of an effect measure of interest.

In this Lesson

·       Three general types of validity problems

·       Distinguish validity from precision

·       Introduce term bias

·       How to adjust for bias.

Examples of Validity Problems

Potential problems with studies

·       Imperfections in study design

·       Imperfections in data collection

·       Imperfections in analysis

No imperfections    =    Valid

Imperfections         =    Bias

Bias results in distortion of results

Validity versus Precision

Validity & precision - influenced by 2 different errors

·       Systematic error affects validity

·       Random error affects precision.

Valid = No systematic error (i.e., unbiased)

The bull’s eye usually not known, therefore difficult to determine extent of bias.

Precision:  a lot of spread = poor precision “imprecise”

little spread     = good precision “precise”

Precision reflects sampling variability.

A Hierarchy of Populations

v  The sample - collection of individuals from which study data have been obtained.

v    The study population - the individuals that our sample actually represents (typically those we can feasibly study).

v    The source population - group of interest about which the investigator wishes to assess an exposure-disease relationship.

v  The external population - group to which the study has not been restricted but to which the investigator wishes to generalize.

We would often like to generalize our conclusions to a different external population.

Internal versus External Validity

Target shooting illustrates difference between internal and external validity.

·       Internal validity considers whether or not we are aiming at the center of the target.  If shooting off target, then study is not internally valid.

Internal validity - drawing conclusions about source population based on study population.

External validity – conceptually concerns a different target.  We might imagine this external target being screened from our vision.

External validity - applying conclusions to an external population beyond the study's restricted interest; subjective, less quantifiable than internal validity.

7-2  Validity (continued)

Quantitative Definition of Bias

A bias can be defined quantitatively in terms of the target parameter of interest and measure of effect actually being estimated in the study population.

A study that is not internally valid is said to have bias.

Target parameter - Greek letter θ (“theta”). We want to estimate the value of θ in the source population.

θ0 the measure of effect in the study population.

(“theta-hat”) denotes the estimate of our measure of effect obtained from the sample actually analyzed.

Differences between  and θ0 the result of random error.

A difference between θ0 and θ is due to systematic error.

v  Bias (,θ) = θ0 - θ

The not equal sign should be interpreted as a meaningful difference from zero.

Direction of the Bias

The precise magnitude of bias can never really be quantified, however, the direction of bias can often be determined.

·       The target parameter can be overestimated (bias away from the null)

·       The target parameter can be underestimated (bias towards the null).

Examples

Switchover bias

Positive vs. negative bias – do not worry about these terms

What Can be Done About Bias?

Three general approaches for addressing bias:

1.  Design stage - minimize or avoid bias. Avoid selection bias by including/excluding eligible subjects, by

·         Choice of source population

·         Choice of the comparison group

2.  Analysis stage - determine presence or direction of possible bias

Also, account for confounding in analysis.

3.  Publication stage - Potential biases typically described in "Discussion" section. For selection and information bias, is subjective, judgment expected given inherent difficulty in quantifying biases.