Validation vs. Verification: What's the Difference?
Validation and verification are two important processes water labs need to understand in the context of their services. While these terms are often confused with one another, they actually fill distinct roles in the quality control process.
Knowing the difference is critical for ISO 17025 certification since both are required under the standard for testing and calibration laboratories.
So, what's the difference between these two terms? Let's explore how these processes fit into your lab's operations, who performs each process, what they evaluate, and how results are used.
What is Validation?
This is an independent analysis demonstrating that a method works as intended. It confirms the performance characteristics of a test method. A third party performs the process on behalf of the method developer. The results are compiled into a report that can be used by:
- Organizations that want to adopt the method.
- Regulatory bodies that decide to include the method in regulations.
- Standards organizations that want to include it in their standards.
Test Method Validation Characteristics
Test method validation shows that the procedure is capable of delivering results at the required performance level. It looks at parameters such as:
- Sensitivity: Proportion of positive cultures or colonies correctly assigned
- Specificity: Proportion of negative cultures or colonies correctly assigned
- False positives: Proportion of wells incorrectly identified as positive
- False negatives: Proportion of wells incorrectly identified as negative
- Repeatability: How well counts agree after repeat counting by one analyst
- Reproducibility: How well counts agree after repeat counting by two or more analysts
- Measurement of uncertainty: Uncertainty of results in terms of number of standard deviations
- Selectivity: How accurately a method measures an analyte despite expected interferences
- Efficiency: The fraction of wells correctly assigned
Calculating Validation Characteristics
Validation characteristics are easily calculated using a simple set of equations where:
- a = true positives (number of positive wells that contain the target)
- b = false negatives (number of negative wells that contain the target)
- c = false positives (number of positive wells that don't actually contain the target)
- d = true negatives (number of positive wells that don't contain the target)
Using these variables, characteristics can be calculated using the following equations:
- Sensitivity = a / (a+b)
- Specificity = d / (c+d)
- Selectivity = log10 [(a+c) / (a+b+c+d)]
- False positive rate = c / (a+c)
- False negative rate = b / (b+d)
- Efficiency = (ad) / (a+b+c+d)
ISO/TR 13843 for water quality microbiological methods includes guidance on how to measure repeatability and reproducibility based on relative standard deviations (RSDs) of repeat counts. The standard recommends that RSDs should be less than 0.02 when using pure cultures.
What is Verification?
Verification shows that a lab is capable of performing the test. It helps laboratories demonstrate the viability of a method in your lab. Labs should verify a method anytime they start using a new one.
For example, let's say your lab has been using a spread-plate method for Legionella testing, and you want to use a new liquid culture method. To compare the new method proposed with the previous one, you would verify the new method by:
- Evaluating results side-by-side from 10-20 split samples, either natural samples or ones spiked with quality control (QC) strains.
- Analyzing QC results of samples from the new method.
- Calculating repeatability.
- Calculating reproducibility.
In other words, you're choosing a few performance characteristics to verify, and you're ensuring that your lab can use the method to achieve accurate results. The results of this analysis should be compiled into a report that becomes part of your lab's quality assurance records.
Validation and verification fill two similar but distinctly different roles in microbiological testing. While the former tells you essential information about a test method's performance, the latter demonstrates that your lab's results are in line with how the test was designed to perform.
The result: more reliable data and a QA process that protects the integrity of your data overall.
Read These Next
