Calibration reports tell a lot about the instruments they cover as well as the labs they are performed in. Their reports can be confusing, but here’s how to use the information in them.
By: Howard Zion/Director of Technical Operations/Transcat/www.transcat.com
On November 1, 2011, the International Laboratory Accreditation Cooperation (ILAC) mandated that all reported measurements, represented by ISO 17025 accredited testing, or calibration laboratories, or both, include the lab’s measurement uncertainty for each reported value. This change in reporting requirements aligns industries around the globe with the definition of measurement traceability.
The U.S. Wind Energy Industry is not exempt from the new requirements. As it matures, the need for standards in all areas – safety, tool, and instrument performance – will continue rising. Accredited calibration is important to wind services and products, as well as the turbine life cycle, from manufacturing to maintenance.
The ultimate goal of a company is to profit by providing an effective and safe product or service. The use of test and measuring instruments indicates a need to base decisions on quantified values. These values must be reliable, which is why companies calibrate their instruments. But how confident can a company be that its test and measuring instruments are not impacting the product or service on which the instruments are used? Is it possible to tell when a reported calibration result – a reading – is valid or not?
If a calibration result is valid, then it is simply a matter of knowing when to perform an impact study. Different indicators can help decide when an impact analysis is necessary. But before demonstrating different “instrument wellness” indicators, consider an alternative: How do you tell that the calibration result is invalid?
Both concerns – validity of the calibration and instrument wellness – can be addressed together. To understand how, start with a look at each piece of information that is (or should be) included on a calibration report. The report should indicate, at a minimum, the value of the lab’s standard, the instrument’s value, the error of the instrument, and the uncertainty of the lab’s measurement. Additionally, including the instrument’s tolerances makes it easier for the owner to read the report and understand whether or not the instrument has maintained its expected tolerances. A statement on the front of the report identifies whether or not all readings maintained their tolerances. This is a Statement of Compliance.
Finally, if the instrument’s tolerances are along with the lab’s uncertainty for the measurement, then a ratio can be included as a general bias indicator (bias of the lab’s measuring uncertainty on the reported measurement).
Calibration and why it is necessary
A competent laboratory compares an instrument to a laboratory standard and quantifies the error in the instrument. This allows comparing the error to acceptable tolerances that indicate whether or not the instrument negatively influenced the application of its most recent usage.
Let’s break down the statement:
• A competent laboratory compares an instrument to a laboratory standard.
• This quantifies the error that existed in the instrument while it was being used.
• Technicians used the instrument to make decisions about their production or service processes during its calibration interval or usage cycle.
• If the error exceeded predetermined acceptable tolerances, then it indicates when an impact analysis is needed.
• An impact analysis determines if the instrument negatively influenced the application in which it was used during its most recent usage interval.
This lets a team or user conclude if current calibration processes (internal or outsourced) are helping or hindering a company in minimizing risk in its manufacturing or services. Here are five items for users of calibrated instrument to consider.
How does accreditation guarantee competence or consistency? How can one tell whether or not a laboratory is competent?
Such questions are the premise for ISO/IEC 17025: General Requirements for the Competence of Testing and Calibration Laboratories. The standard requires an unbiased accreditation body to assess the laboratory’s quality system (section 4, which mirrors ISO 9001:2000) and technical competence (covered under section 5). The accreditation audit identifies the laboratory’s capabilities and states them in a scope of accreditation.
The scope of accreditation lists each measurement parameter the lab can perform, indicating the lab’s Best Measurement Capability (BMC) for each parameter. Bottom line: the ISO 17025 accreditation process authenticates the laboratory’s measuring capabilities. It ensures that the lab knows how to indentify its measurement uncertainty when performing an instrument calibration.
This measurement uncertainty is the validation portion of a calibration that identifies whether or not the lab is capable of identifying the error in the instruments it tests. Generally, you can tell whether or not a lab is competent by looking at its ISO 17025 certificate and scope of accreditation. More importantly, you can tell whether a lab can identify the error in an instrument by comparing its tolerance to the lab’s measurement uncertainty for any reported reading. This comparison is usually calculated as a ratio, the Test Uncertainty Ratio, or TUR. Traditionally, a 4:1 TUR (or higher) is accepted as a good ratio. The 4:1 ratio indicates that the laboratory has a measurement capability four times better than the measurement on which it reports.
Instrument users should also be aware of an outdated ratio still in use in some companies. This Test Accuracy Ratio, or TAR, makes a comparison between the tolerance of an instrument and the accuracy of the standard used for the calibration. The ratio is outdated because it does not consider all factors that go into making a measurement. It only considers the estimate of error in the laboratory’s standard. It does not speak to other errors, which are an important part of determining whether or not a lab is competent in making a measurement. Therefore, the TAR indicator is incomplete and has been superseded by TUR.
Many calibration labs, however, still use TAR because they are not accredited or do not understand how to calculate their measurement uncertainty. But a laboratory without measurement uncertainty is one whose measurements cannot be validated. Without a means of authenticating a laboratory’s measurement capabilities, it cannot prove its competence to report calibration errors for the instruments it calibrates.
Confidence of a lab’s competence to report calibration result means more confidence in the accuracy of reported errors. This ensures that a company is neither wasting time performing unnecessary impact studies nor unaware of a situation in which an impact study was needed but not performed. Either situation can be costly. Wasting time means wasting resources (known as a “false reject” situation) and missing necessary studies can result in potentially bad consequences due to lack of impact analysis (a “false accept” situation).
It’s critical to take an interest in calibration results because those using the instrument are making decisions about the way in which the instrument is used. These decisions are based on values the instrument indicates. However, if these values are incorrect – either because the instrument is used incorrectly or has an error – then the decisions or results are also in error.
For example, if during an instrument’s normal use, a technician believes the process is out of control or exceeds a limit due to an instrument indication, and the indication is incorrect because the instrument has drifted beyond its allowable tolerances, then a bad decision has been made.
Furthermore, an unnecessary fix could follow, which can move an in-control process to out-of-control, due to the unknown error of the instrument. Upon return from calibration, this error would be identified on the calibration report and an impact study on the processes on which the instrument has been used since its last calibration would identify where the calibration error affected decisions made about these processes.
Without impact studies, a company is susceptible to negative consequences. Hence, if an engineering staff is not going to use calibration data, there is no point in calibrating instruments. But you also do not want the risks that go with measurement errors, so calibrate instruments and use the results to minimize that risk.
Many assumptions influence the expected performance of an instrument. A few include:
• Expectations for the instrument’s performance: Are the expectations the same for other users?
• Expectations of the instrument while in use: Who determines these? A quality or manufacturing engineer? A technician or production line worker using the instrument to perform daily work?
• The time the instrument will be in use before it is recalibrated.
• Tolerances that indicate how far the instrument can drift before it adversely affects decisions.
These illustrate a point about instrument error: If it is possible to interpret the acceptable tolerance of an instrument differently, then what interpretation is a calibration provider applying to your instruments?
Even if using a competent laboratory, no international standard governs the tolerances of an instrument. ISO 17025 states only that a laboratory must understand its client’s calibration need. Hence, it is a good idea to understand how a calibration provider applies tolerances to the instrument. This is important because, whether or not you look at each individual calibration result, if tolerances on the report do not match expected tolerance limits, the instrument will not be flagged as “Out of Tolerance” when it should. Alternately, it could be flagged “Out of Tolerance” when it should not.
Several reasons why company expectations and the lab’s reported tolerances might not match include:
• Multiple sources for accuracy specifications are reported by the manufacturer.
• Complex specifications are interpreted incorrectly.
• Manufacturer’s specifications expected by the instrument user different from specifications used by the lab.
• Varying ways for combining multi-part specifications.
Hence, it’s important to communicate expectations of instrument tolerances to a calibration provider so a user can rely upon the statement of compliance and associated tolerance conditions to correctly flag when you should perform an impact analysis (and when not to). This discussion may identify when the laboratory is calculating the spec differently than the company. One or both parties has made an incorrect assumption regarding accuracy or in converting the accuracy value to a tolerance limit.
The point here is not to place blame when a mismatch occurs, but rather to identify that there is a mismatch and correct it prior to calibration to preserve the intent of the process – that is, identify when the instrument exceeds the tolerances that indicate an impact study is needed. If it is not assured that this intent has been preserved, then the calibration may become useless.
Recall that someone is making decisions about important processes based on values taken from measuring instruments. You’ve predetermined that an instrument can drift some allowable amount before it potentially impacts the quality of the product or service. If the calibration report indicates an Out of Tolerance condition, it will be necessary to go through with an impact analysis to under-stand how much this error affected products or services, or decisions made about the product or service. This step closes the loop in using instrumentation to help control the quality of a product or service.
When to perform an impact study
It is necessary to make sense of calibration results with assistance from the calibration report. An area of risk to understand is that even competent laboratories can end up with situations in which they are not certain about the result of a measurement. This occurs when an instrument has drifted sufficiently close to one end of its tolerance limit, where the measurement uncertainty clouds a lab’s ability to state clearly whether the instrument is In Tolerance or Out of Tolerance – even if it appears that the instrument is In Tolerance to the average person.
When this situation occurs, it is important to understand how the instrument reading, the instrument tolerance, and the lab’s uncertainty come into play. Here, the TUR (or any ratio indicator) does not indicate that risk is present. Hence, the ratio is useless. Although not the norm, this situation can occur even if the lab has a capability that is 100 times better than the measurement on which they are reporting. WPE