Primary objective of most epidemiologic research - obtain valid estimate of an effect measure of interest.
In this Lesson
· Three general types of validity problems
· Distinguish validity from precision
· Introduce term bias
· How to adjust for bias.
Examples of Validity Problems
Potential problems with studies
· Imperfections in study design
· Imperfections in data collection
· Imperfections in analysis
No imperfections = Valid
Imperfections = Bias
Bias results in distortion of results
Validity versus Precision
Validity & precision - influenced by 2 different errors
· Systematic error affects validity
· Random error affects precision.
Valid = No systematic error (i.e., unbiased)
The bull’s eye usually not known, therefore difficult to determine extent of bias.
Precision: a lot of spread = poor precision “imprecise”
little spread = good precision “precise”
Precision reflects sampling variability.
A Hierarchy of Populations
v The sample - collection of individuals from which study data have been obtained.
v The study population - the individuals that our sample actually represents (typically those we can feasibly study).
v The source population - group of interest about which the investigator wishes to assess an exposure-disease relationship.
v The external population - group to which the study has not been restricted but to which the investigator wishes to generalize.
We would often like to generalize our conclusions to a different external population.
Internal versus External Validity
Target shooting illustrates difference between internal and external validity.
· Internal validity considers whether or not we are aiming at the center of the target. If shooting off target, then study is not internally valid.
Internal validity - drawing conclusions about source population based on study population.
External validity – conceptually concerns a different target. We might imagine this external target being screened from our vision.
External validity - applying conclusions to an external population beyond the study's restricted interest; subjective, less quantifiable than internal validity.
7-2 Validity (continued)
Quantitative Definition of Bias
A bias can be defined quantitatively in terms of the target parameter of interest and measure of effect actually being estimated in the study population.
A study that is not internally valid is said to have bias.
Target parameter - Greek letter θ (“theta”). We want to estimate the value of θ in the source population.
θ0 the measure of effect in the study population.
(“theta-hat”) denotes the estimate of our measure of effect obtained from the sample actually analyzed.
Differences between and θ0 the result of random error.
A difference between θ0 and θ is due to systematic error.
v Bias (,θ) = θ0 - θ
The not equal sign should be interpreted as a meaningful difference from zero.
Direction of the Bias
The precise magnitude of bias can never really be quantified, however, the direction of bias can often be determined.
· The target parameter can be overestimated (bias away from the null)
· The target parameter can be underestimated (bias towards the null).
Positive vs. negative bias – do not worry about these terms
What Can be Done About Bias?
Three general approaches for addressing bias:
1. Design stage - minimize or avoid bias. Avoid selection bias by including/excluding eligible subjects, by
· Choice of source population
· Choice of the comparison group
2. Analysis stage - determine presence or direction of possible bias
Also, account for confounding in analysis.
3. Publication stage - Potential biases typically described in "Discussion" section. For selection and information bias, is subjective, judgment expected given inherent difficulty in quantifying biases.