Assessing Results against Accuracy Limits

Overview

Accuracy assessment is a component of quality assessment applied to blank, standard and spike QC samples.

The accuracy of a measurement is how close a result comes to the true or accepted value. Accuracy is also known as repeatability, that is, how possible is it to repeat a reading on an instrument to get the same result

In order to make an assessment the accuracy of data over a batch of samples, it is necessary to include a proportion of a reference material which can be assigned with certified values. The uncertainty of these values is generally known and published. This uncertainty contributes to the acceptability limits for this particular reference material. Various other factors come into play when setting acceptability limits, and depend on the method of analysis, the analytical instrument used, reagent quality and understanding of sample matrix effects.

In CCLAS, the assessment of blank, standard or spike QC sample results against the standard lot's specification is known as accuracy assessment, and sets the standard status for the analysis.

QC managers require an accuracy assessment for results of blank, standard and spike QC samples to be evaluated in order to measure the repeatability of the readings against their standard specification limits. This is a process that occurs any time a result is entered or cleared for a blank, standard or spike QC sample. It can be triggered when a value is entered in for these samples in Workbook or through processing Instrument run files.

Blanks

Blanks are a specific type of reference material aimed at identifying contamination and unwanted inputs like instrument noise and other interferences. Generally, a number of blank QC types is created and used to identify where contamination is being introduced. These may include:

Method blanks
Equipment blanks
Field blanks
Trip blanks, and instrument blanks.

Information from blanks can be used to enhance the accuracy of other standards when placed in the same batch, and in certain cases, blank data is used to correct QC and real samples prior to assessment.

A blank QC sample is typically made from an inert material where the active analytes are expected to be zero (or very low, typically below the internal lower detection limit for the analyte).

Standards

Standards are prepared substances whose analysis give predictable results for the scheme sample analytes. These analytes are grouped into analyte specifications to define the available limits for QC samples.

A standard QC sample is typically made from a material that is similar to the unknown samples, where the active analytes are in the concentration range similar to the unknown samples.

Spikes

Spikes provide a variance on the information revealed by accuracy assessment. A reference material is added to a real sample. The timing of this addition effects the interpretation, as mentioned for the case of blank assessment. The use of spikes is primarily used to identify sample matrix and transport effects, but the overall intent, as with standards, is to provide an assessment of data accuracy.

A spike QC sample is a mixture of a portion of a client's sample and a material (like a standard) where the results are known. These are analysed parallel to the client's sample from whence the spike QC sample was made, as a test on the efficiency of the testing procedures.

Process

Creating the Scheme, Version and Analytes

Schemes are configured using the CCSCHM—Scheme application.

For a sample scheme analyte's result to be assessed for accuracy, the following conditions must hold:

The scheme version analyte must have a Data Type of Numeric, Numeric Only or Text.
The scheme version analyte must have a Unit Code.

Important: This also applies to text analytes, in which case, if there is no unit, create a unit called NO_UNIT and use that, for example.
The scheme version analyte's QC Active must be selected.
For the outcome of the assessment to be written to QC history, the scheme version analyte's Saved to QC History must be selected.

Configuring QC Types for Standard, Blanks and Spikes

Standard, blank and spike QC types are configured using the CCQCT)—QC Type application.

Maintaining QC Types and Assessments

Configuring Standards and Standard Lots

Standards and standard lots configured using the CCSTND—Standard application.

For a sample scheme analyte's result to be assessed for accuracy, the standard lot must link to a Specification Code.

Configuring a Standard Specification for Accuracy Assessment

Then link to or create a matching standard specification for the standard lot and update the standard specification to add the specification limits for the scheme version analyte units. Specifications are configured using the CCSPFN—Specification application.

A specification can contain multiple entries for scheme, version, analyte and unit.

For a sample scheme analyte's result to be assessed for accuracy, the specification must contain an entry for the related Scheme Code, Scheme Version, Analyte Code and analysis Unit Code.

Important: This also applies to text analytes, in which case, if there is no unit, create a unit called NO_UNIT and use that, for example.

Standard specifications are always treated as critical for validation, regardless of the Critical For Validation setting on the specification scheme version analyte, as the outcome of the assessment always contributes to the Composite QC Status. The Critical For Validation flag is only applicable to product specifications.

Numeric and Numeric Only Analytes

Where the scheme version analyte has an Analyte Data Type of Numeric or Numeric Only, accuracy assessment uses the specification scheme version analyte's internal absolute limits, where the warning and failure ranges are absolute.

There is the option to enter these limits as relative values, where a percentage around the target value for warning and failure limits is designated, however, the system determines the equivalent absolute values and uses them for the accuracy assessment.

A specification scheme version analyte unit's numeric limits are bound by any or all of the following:

< or = Internal Minimum Value—The lowest acceptable amount of substance for the specification to be satisfied, below which the material would fail the standard lot's specification.
< or = Internal Minimum Warning Value—The lowest acceptable amount of substance for the specification to be satisfied without warnings, below which the material would satisfy the standard lot's specification but with warnings.
Internal Target Value—The target value for the acceptable amount of substance in the material.
> or = Internal Maximum Warning Value—The highest acceptable amount of substance for the standard lot's specification to be satisfied without warnings, above which the material would satisfy the specification but with warnings.
> or = Internal Maximum Value—The highest acceptable amount of substance for the standard lot's specification to be satisfied, above which the material would fail the specification.

The method used to determine the percentage difference between an expected and observed value for accuracy assessment is set by preference. See how to Set the USE_STD_TARGET_FOR_PERCENT_DIFFERENCE application preference.

Text Analytes

Where the scheme version analyte has an Analyte Data Type of Text, the specification can be configured to assess strings with or without case-sensitivity.

Note: Leading and trailing spaces are not removed upon cell exit but are removed upon submit. Therefore, if a list of expected results in an applied specification contains "DEF" and not "DEF " for a text analyte, and "DEF " is entered into the cell, then upon cell exit, the cell is flagged red to show a failed specification assessment. However, upon submit, leading and trailing spaces are removed, so "DEF" is persisted, such that, upon workbook reload, if the cell undergoes Force Reassessment, then the cell shows a passed specification assessment.

Creating Blank, Standard or Spike QC Samples

When a QC sample is created, it must link to a Primary Analytical Type and optional Secondary Analytical Type that drives the type of QC assessments that take place to its results.

QC types are configured using the CCQCTP—QC Type application, which is also accessed using the Analysis Setup » QC Types menu option. Refer to Maintaining QC Types and Assessments.

For a sample scheme analyte's result to be assessed for accuracy, the sample must be either a:

Client blank sample—The sample's Secondary Analytical Type is Blank.
Client standard sample—The sample's Secondary Analytical Type is Standard.
Blank QC sample—The sample's Primary Analytical Type is Blank.
Blank QC duplicate sample—The sample's Primary Analytical Type is Blank.
Blank QC replicate sample—The sample's Primary Analytical Type is Blank.
Standard QC sample—The sample's Primary Analytical Type is Standard.
Standard duplicate QC sample—The sample's Primary Analytical Type is Standard.
Standard replicate QC sample—The sample's Primary Analytical Type is Standard.
Spike QC sample—The sample's Primary Analytical Type is Spike.
Spike duplicate QC sample—The sample's Primary Analytical Type is Spike.
Spike replicate QC sample—The sample's Primary Analytical Type is Spike.

For a sample scheme analyte's result to be assessed for accuracy, the sample must link to a Standard Code and Standard Lot Code. A sample's Standard Code is inherited from the QC mask member's Standard Code, or determined from the QC mask member's Standard Group Code, or updated on the sample. The sample's Standard Lot Code is set to the standard's Current Standard Lot.

Whether a spike QC sample can link to a standard material that is configured as with a QC Type of Standard instead of Spike, and whether the outcome of an accuracy assessment of a spike QC sample is written to the QC history table with an Analytical Type of Standard instead of Spike is set by preference. See how to Set the ALLOW_STANDARDS_AS_SPIKES application preference.

Refer to Creating QC Samples using QC Masks.

Where a sample scheme analyte is created or its result is cleared, then the sample scheme analyte's Standard Status is set to Not Tested.

When does Accuracy Assessment occur, and what does it set?

After a result is entered or cleared for a sample scheme analyte in the CCWKOP—Open Workbook application, accuracy assessment occurs after detection limit assessment, scheme limit assessment and precision assessment.

Refer to Entering Results into Workbook Cells and Forcing the Reassessment of Results in a Workbook Session.

When a sample scheme analyte is created or its result is cleared, then the sample scheme analyte's Standard Status is set to Not Tested.
Where conditions do not hold for accuracy assessment, then the sample scheme analyte's Standard Status is set to Not Required.
Otherwise, the outcome of an accuracy assessment sets the sample scheme analyte's Standard Status to either Passed, Lower Warning, Upper Warning, Lower Failure or Upper Failure, which contributes to set the sample scheme analyte's Composite QC Status.

The outcome of an accuracy assessment can be viewed in the lower panel. The outcomes of QC assessments can also be viewed in the CCVALN—Job and Sample Validation application.

The sample scheme analyte's result cannot be validated if there is any failure on the Composite QC Status.

Refer to: