Table of contents
- Uncertainty basics: an introduction to expressing uncertainty in measurement
- Estimating uncertainty step 1
- Step 2: Decide on the Uncertainty Units
- Step 3: Estimate the Magnitude of the Uncertainty Contributors
- Step 4: Document, Document, Document
- Step 5: Combine the standard uncertainties into one number
- Step 6: Expand the combined the standard uncertainty
- Step 7: Reality check
- Step 8: Reporting the result
- Recap on all Steps
Source: Content taken from a Powerpoint presentation used for conferences
Author: Mr. Ouellette
Date: 2002-10-25
Uncertainty basics: an introduction to expressing uncertainty in measurement
Overview
- Why the big deal about uncertainty?
- Background stats: The basics — nothing fancy
- Minimal stats jargon
- Minimal algebraic notations
- No system modeling
- No calculus
- Promise!
- How to apply it (on the back of an envelope)
- A worked example
Why the big deal?
- 17025
- VIM (re. Traceability)
- No stated uncertainty = No traceability!
Don't get sucked in!
- ESTIMATE uncertainty only to the extent that you need, don't make a science of it.
- Accept that you'll never be certain about uncertainty.
- Focus mostly on the "biggies"; don't sweat the small stuff (much).
The background stats: the standard deviation
- Repeat measurements aren't identical.
- Standard deviation tells us how widely they are dispersed about the mean.
- Approx. 2/3 (i.e., 68%) of all readings fall within 1 standard deviation of the mean.
- Approx. 95.5% of all readings fall within 2 standard deviation of the mean.
Experimental standard deviation of the mean
- The standard deviation describes the spread of values in an individual set of measurements. What if we took several different sets of measurements?
- The mean of each set of measurements would vary.
- The spread of the means is given by the experimental standard deviation of the mean (stdm).
- To predict stdm from a set of n replicate measurements, divide the standard deviation by √(n).
Stdm = stdev /√(n)
Experimental standard deviation of the mean: an example
- If standard deviation, Stdev, is 5.93, and
- if number of replicate measurements, n, is 10, then
- experimental standard deviation of the mean, stdm, is
Stdm
= Stdev / square root(n)
= 5.93 / square root(10)
= 1.88
Experimental standard deviation of the mean: So what?
The stdm feeds directly into our uncertainty budget whenever we estimate uncertainty in the mean of a set of random repeated measurements.
Very useful! More on that later
Distributions of Measurements
The spread of a set of measurements can take on different frequency distributions.
Examples:
Distributions of measurements: normal distribution (Gaussian)
- Most values fall near the mean.
- Progressively fewer values falling further from the mean.
- Very common distribution in nature.
Distributions of measurements: uniform distribution (rectangular)
- Measurements distributed equally across the interval.
- Reasonably good choice when you don't have a clue!
Estimating uncertainty step 1
Identify the contributors to uncertainty
Make a list of contributors. Consider:
Reference standards & measurement equipment; e.g.
- Uncertainty in their calibration
- Long term drift
- Stability during measurement
- Resolution & quality of their scales
- Linearity
- Digitization
- Sensitivity to transportation & handling
- Design issues (such as unequal lengths of equal arm balances)
- Parallaxes
- Interpolation between calibration points
- Reading system
- Transporation
Not a complete list. Not all necessarily apply.
Environmental conditions; e.g.
- Absolute temperature
- Time variance in temperature
- Spatial gradiant in temperature
- Heat radiation from operator, etc.
- Humidity
- Noise, vibration
- Contamination
- Illumination
- Air pressure, composition, flow
- Gravity
- Electromagnetic interference
- Transients in power supply
Measurement setup; e.g.
- Warming up
- Cosine errors
- Optical aperture
- Parasitic voltages
- Current leakage
- Loading effects
- Cabling, shielding, filtering
- Stiffness/rigidity of mechanical systems
- Properties of measurement probes
Measurement object; e.g.
- Sensitivity to stresses of measurement
- Surface roughness
- Conductivity
- Weight, size, shape
- Magnetism
- Stability
- Cleanliness
- Internal strength
- Distortion during measurement
- Self heating (e.g., current measurement)
- Number of terminals
- Orientation
Measurement process; e.g.
- Repeatability, conditioning
- Number & order of measurements
- Duration of measurements
- Choice of principle of measurements
- Magnetism
- Alignment
- Choice of reference & apparatus
- Clamping, fixturing, probing
- Drift check
- Reversal measurements
- Multiple redundancy checks
- Strategy
Software & Calculations; e.g.
- Rounding
- Algorithms
- Number of significant digits in calculation
- Sampling
- Filtering
- Interpolation
- Extrapolation
- Outlier handling
Definition of the Measurement Characteristic; e.g.
E.g., definition of "Diameter" for dimensional measurements of objects that are not perfectly round.
D= 0.5 (Max + Min)?
D= Mean of many measurements?
D= Other? (e.g., Bullets —> Max diameter?)
Physical Constants and Conversion Factors; e.g.
- Uncertainty in knowledge of the physical values used
- Temperature coefficient
- Power coefficient
- Various properties of the working, measuring instrument
- Various properties of ambient air
- Local force of gravity
Metrologist Effects; e.g.
Differences in judgements of different operators.
How would you integrate shoulder peaks?
Note: Shouldn't need to include uncertainty due to "Operator Error." This is an aberration that should be corrected before measurement.
Summary of Some Contributors to Uncertainty
- Reference Standards & Measurement Equipment
- Environmental Conditions
- Measurement Setup
- Measurement Object
- Measurement Process
- Software & Calculations
- Definition of the Measurement Characteristic
- Physical Constant & Conversion Factors
- Metrologist Effects
Step 2: Decide on the Uncertainty Units
Units may be:
- ppm or % of result
- units of measurement
- other
Generally doesn't matter what you pick; just be consistent: Apples + Apples = Apples.
Everything gets boiled down to a standard deviation of the mean or equivalent (called standard uncertainty).
Step 3: Estimate the Magnitude of the Uncertainty Contributors
1) Type A, Normal
Type A contributors to uncertainty are those that you have statistical data for. Use this data if you have it.
E.g., For Standard Uncertainty in the mean of repeated measurements (preferably 10 or more) use stdm, the experimental standard deviation of the mean.
2) Type B
Type B contributors to uncertainty are those that you have no statistical data for; e.g.,
- Manufacturer's specification
- Professional judgement
- Uncertainty in cal certificate for your reference standard
GUM: There's "no substitute for critical thinking, intellectual honesty, and professional skill."
Gather information from wherever we can.
Boil down the uncertainty info to a standard uncertainty.
But info doesn't always come at level of confidence of 68.3%. It usually comes at a higher level of confidence. It's called an Expanded Uncertainty when coming from a normal distribution.
2) Type B, Normal
For a normal distribution, no problem!
Simply divide the Expanded Uncertainty by a Divisor, depending upon the level of confidence at which the Expanded Uncertainty is given.
| Divisor | Level of confidence |
|---|---|
| 0.676 | 50% |
| 1 | 68.27% |
| 1.645 | 90% |
| 1.960 | 95% |
| 2 | 95.45% |
| 2.576 | 99% |
| 3 | 99.73% |
E.g., Cal certificate for our mass standard states:
"The uncertainty in the reported mass is ±26 mg at a level of confidence of 95% assuming a normal distribution."
The standard uncertainty is … ±26 mg ÷ 1.960 = ±13 mg
| Divisor | Level of confidence |
|---|---|
| 1.960 | 95% |
| 2 | 95.45% |
| 2.576 | 99% |
| 3 | 99.73% |
E.g., Cal certificate for our resistor states:
"The expanded uncertainty in the reported resistance is ±30 Ω with a coverage factor, k, of 3."
The standard uncertainty is…
±30 Ω ÷ 3 = ±10 Ω
Here, the "coverage factor" is another word for "divisor".
2) Type B, Uniform
But not all distributions are normal.
Rectangular (I.e., Uniform) distributions also very common.
Equal probability that true value is anywhere between interval a- to a+.
For Rectangular (I.e., Uniform) distributions, the Standard Uncertainty, u, is the area shaded.
Standard Uncertainty, u, is the complete interval, a, divided by √3 or ˜1.73.
E.g., Gauge readability (resolution).
If smallest subdivision is 5 measurement units, and if we can "eyeball" the gauge to within 1/2 a subdivision, then the gauge's true value can be anywhere in the interval a = ±5 / 2 = ±2.5 with equal probability.
Standard Uncertainty,
u, = ±2.5 / √3 = ±1.4 meas. units
or, if you prefer,
u = ±5 / √12 = ±1.4 meas. units.
E.g., Ambient temperature
Approximately equal probability that temperature is anywhere between control limits a+ = 24 °C and a- = 22 °C.
Standard Uncertainty, u,±= ±1 °C / √3 = ±0.58 °C
But remember that we need to express all uncertainties in the same units!
If we're expressing uncertainty in ppm of reading of a resistor, and if temperature coefficient of resistor is 0.5 ppm/ °C, then std uncertainty becomes:
u = ±0.58 °C x (0.5 ppm per °C) = 0.29 ppm of Ω reading.
2) Type B, Unknown Distribution
Clueless about the distribution?
Usually safe to treat it as a Rectangular (Uniform) distribution
Standard Uncertainty, u, = ±(Max Interval width) / √3
E.g., An incomplete instrument specification:
"The anvils of the micrometer are certified to be parallel within ±0.02 mm."
No coverage factor or level of confidence,
No distribution is specification. based on a standard deviation?
Standard Uncertainty, u, = ±0.02 mm / √3 = ±0.012 mm
Step 4: Document, Document, Document
Your uncertainty budget is a living document that is revised as your measurement process changes and as your understanding of it improves. Therefore…
Write down how you arrived at each of the uncertainty estimates that you have listed.
File this info with the uncertainty budget.
Recap on Steps 1 to 4
1. Identify the contributors to measurement uncertainty.
2. Decide on consistent uncertainty units.
3. Estimate magnitude of each uncertainty contributor, and express each as a standard uncertainty.
4. Document the basis for your estimates.
The hardest part is done!
Step 5: Combine the standard uncertainties into one number
Use the RSS method:
- Take the square of each standard uncertainty, u
- Add up the squares
- Take the square root of the sum
This gives combined standard uncertainty, uc, of your measurement, with ˜68% confidence.
E.g., standard uncertainty, u, in ppm of reading:
- Cal of ref standard: 0.75 ppm
- Long term drift of ref standard: 1.16 ppm
- Repeatability, n=5: 0.07 ppm
Combined standard uncertainty, uc, = square root (0.752 + 1.162 + 0.072) = 1.4 ppm
i.e., ˜68% confident that true value is within ±1.4 ppm of the reported result.
Step 6: Expand the combined the standard uncertainty
What if we want to report uncertainty with a different level of confidence rather than ~68%?
No problem! Simply multiply combined standard uncertainty, uc, by an appropriate coverage factor, k. The product is called Expanded Combined Uncertainty, Uc.
| Coverage factor, k | Level of confidence |
|---|---|
| 0.676 | 50% |
| 1 | 68.27% |
| 1.645 | 90% |
| 1.960 | 95% |
| 2 | 95.45% |
| 2.576 | 99% |
| 3 | 99.73% |
Exercise: Combined standard uncertainty uc, is 1.4 ppm of rdg;
We want to report uncertainty with a level of confidence of "approximately 95%."
Expanded Combined Uncertainty,
Uc, = 1.4 ppm x 2 = 2.8 ppm
| Coverage factor, k | Level of confidence |
|---|---|
| 0.676 | 50% |
| 1 | 68.27% |
| 1.645 | 90% |
| 1.960 | 95% |
| 2 | 95.45% |
| 2.576 | 99% |
| 3 | 99.73% |
Step 7: Reality check
The preceding simplified approach is based upon several assumptions, including:
Dominant contributor(s) to uncertainty are known with reasonable certainty;
- For Type A: Have enough data (≥10 points)
- For Type B: No wild guessing
So identify the dominant contributor(s).
- Calibration of reference standard: 0.75 ppm
- Long term drift of reference standard: 1.16 ppm
- Repeatability, n=5: 0.07 ppm
Dominant contributor is drift. If wild guess or only few data points, then get a more reliable (otherwise a more conservative) estimate if possible. Otherwise follow detailed steps in the GUM on degrees of freedom.
- Calibration of reference standard: 0.75 ppm
- Long term drift of reference standard: 1.12 ppm
- Repeatability, n=5: 0.07 ppm
Small n (of 5) for repeatability is good enough because this contributor isn't dominant.
Focus on the biggies: Long term drift & reference standard calibration.
Step 8: Reporting the result
Not Recommended…
"The measured result is 10,000.051Ω ±2.8 ppm for a level of confidence of approximately 95%, k=2."
Better…
"The measured result is 10,000.051Ω ±0.028Ω for a level of confidence of approximately 95%, k=2."
Better yet…
"The measured result is 10,000.051Ω ±0.028Ω. The reported uncertainty is expanded using a coverage factor k=2 for a level of confidence of approximately 95%, assuming a normal distribution."
Recap on all Steps
- Identify the contributors to measurement uncertainty.
- Decide on consistent uncertainty units.
- Estimate magnitude of each uncertainty contributor, and express each as a standard uncertainty.
- Document the basis for your estimates.
- Combine the standard uncertainties.
- Expand combined std uncertainty to represent desired confidence.
- Reality check.
- Report the result.
A more Complete Example Uncertainty Budget
| Source of uncertainty | Value ±ppm |
Probability distribution |
Divisor | Standard uncertainty ± ppm |
|---|---|---|---|---|
| Repeatability | 0.071 | Normal | 1 | 0.071 |
| Calibration uncertainty of standard resistor | 1.5 | Normal | 2 | 0.75 |
| Uncorrected drift of standard resistor since last calibration | 2.0 | Rectangular | √3 | 1.155 |
| Effect of temperature of oil bath | 0.5 | Rectangular | √3 | 0.289 |
| Effect of voltmeter on measurement of standard resistor | 0.2 | Rectangular | √3 | 0.115 |
| Combined standard uncertainty, uc | empty space | Assumed normal (k=1) |
empty space | 1.418 |
| Expanded combined uncertainty, uc | empty space | Assumed normal (k=2) |
empty space | 2.9 |
Where the numbers came from
- Repeatability from 5 random repeat measurements in ppm from nominal: +10.4, +10.7, +10.6, +10.3, +10.5 —> stdm of 0.071 ppm.
- Cal of standard resistor from cal certificate saying uncertainty in reported result is ±1.5 ppm at a level of confidence of approximately 95% (k=2); 1.5 ppm/2 = 0.75 ppm.
- Drift of ref standard is based upon several years of calibration history and comparisions with a check standard. A trend line was fitted to the historical data and was used to correct the ref std's value at the date that it was used. The estimated uncertainty in the correction is ±2 ppm based upon an analysis of the regression residuals.
- Temperature effects were estimated from the control limits of the stirred oil bath (±0.2 °C) and the manufacturer's specs for the resistors' temperature coefficient (worst case 2.5 ppm/°C). Bath homogeneity was measured and found to be negligible (‹ 0.05 °C).
- Voltmeter effects are limited to linearity and resolution because it is used as a transfer device only. According to the manufacturer's specifications, the combined effects of linearity & resolution are ±0.2 ppm in the range of measurement. The uncertainty is considered twice: once for each resistor.