Analyte Performance Map
An APG Exclusive

Article written by Michael Haller, Quality Manager, Analytical Products Group

Complete Article from Edition 31 APG eNewsletter

This article is the third in the series highlighting The Advantage Reporting Tool. The first article, published in the March Newsletter, discussed the foundation of The Advantage Reporting Tool. The second article, published in the May Newsletter, discussed the Certification Map.

Establishing a Minimum Threshold
I have found that the most effective way to convey the value of the Performance Map is to relate it to the grading scale that is used in University, high schools, and grade schools throughout the United States. Lets start with Accreditation as the basis for this analogy. Accreditation is all or nothing. Either a laboratory receives a satisfactory grade (acceptable) or an unsatisfactory grade (not acceptable). Satisfactory/unsatisfactory grades are normally given for grade school gym classes or driving tests. This tangent should not be perceived as belittling the efforts of grade school gym teachers or driving instructors but rather illustrating that satisfactory/unsatisfactory evaluations establish a minimum threshold of performance (also known as a minimum level of quality). This analogy is also not meant to diminish the efforts of laboratories or regulatory agencies but rather to illustrate that the accreditation process effectively eliminates unsatisfactory laboratories and gives satisfactory laboratories a level playing field.

APG Exclusive: The Performance Map
Let's extend the grading analogy. An A is excellent, a B is good and so on. The Performance Map delivers a visual interpretation of performance beyond acceptable/not acceptable. The Performance Map is an APG exclusive that uses inter-laboratory statistics to identify areas of exceptional performance and areas of concern. Here is how it works:

Using the Performance Map
To view the Performance Map, click the Reports Menu from the APGQA.com main menu. Select the Trend Reports from the Study Summary page. The Performance Map is located on the top right of the Trend Homepage.

You have the opportunity to tailor the Performance Map to your needs. You can select the analytes to include in the report and the timeframe to display.

Multiple Methods
Multiple results are displayed in the same month/year cell. You can place your mouse over an evaluation to determine the study, the method, and the performance evaluation.

Business Development = Points of Excellence
Very few data points are considered Points of Excellence – similar to A's in school. Laboratories should take pride in Points of Excellence, and business development managers should capitalize on Points of Excellence to differentiate their services. I am certain that every business manager is looking for a way to separate his or her laboratory from other accredited laboratories on something other than price. The reason they are searching for differentiation is the fact that over 90% of all data points are acceptable based on NELAC/EPA Criteria. Remember that NELAC/EPA Criteria serves as a minimum threshold of quality. Only 5% of data points are considered Points of Excellence.

Points of Opportunity/Points of Immediate Concern
In contrast to Points of Excellence, Points of Opportunity and Points of Immediate Concern are often considered acceptable by accreditation criteria but are likely not consistent when compared statistically to peer laboratories. Using the grading analogy again, Points of Opportunity are like Ds. They are passing but not by much. It means that future performance could easily slide to a not acceptable evaluation (also known as an F). Therefore, Points of Opportunity are an opportunity to correct issues before they become not acceptable evaluations. Points of Immediate Concern are definitely inconsistent evaluations; these are data points that require immediate corrective action to rectify a laboratory issue.

Performance Index (PI)
To this point, I have discussed the identification of individual points that are exceptionally good or exceptionally inconsistent. Let's once again look at the grading analogy. At the end of a semester, a student receives a grade point average (GPA). The GPA for an analyte is called the Performance Index (PI). The Performance Index assigns a point scale to each data point in the date range.

Just as an honor roll student who receives all A's and B's, a Performance Index above 3.5 means all data points were evaluated as excellent or good.

Here is an example using Aluminum for the first-half of 2005. The example laboratory participated in two programs (Drinking Water and Wastewater) and reported a total of six data points for two methods (EPA 200.7 and EPA 200.8).

Please note in the table above that the laboratory's May 2005 WS result was not statistically "close" to the inter-laboratory mean (even though the lab received an acceptable evaluation based on EPA/NELAC Criteria). The example laboratory reviewed the analytical procedures and found an issue with the preparation of the drinking water samples. They ran a quick turnaround PT STAT sample in June to be sure that the corrective action resolved the preparation issue. The average of the six data points made the Performance Index for Aluminum a very respectable 3.4.

Finding More Information - Drill Down
Click an evaluation on the Performance Map to view the inter-laboratory statistics, the EPA/NELAC criteria, an inter-laboratory participation report and participation graphs. More detailed trending information about percent recoveries, zscores, and EPA evaluations are available by clicking the analyte name. Future articles will discuss the Study Analyte Detail and the Trend Analyte pages. If you have any questions about any part of the Advantage Reporting System or the Performance Map, please send me an email or give me a call at 800.272.4442.