Availability and quality of training and test data

AI-related objective: Training, validation and test data are available, representative of the intended use, of adequate quality and provenance, and managed so that their limitations are known. Used when establishing risk criteria (6.3.4), identifying consequences (6.4.2.6) and evaluating risk (6.4.4). Read availability and quality of training and test data against the objective the use threatens most.

What it asks

AI-related objective: Training, validation and test data are available, representative of the intended use, of adequate quality and provenance, and managed so that their limitations are known. Used when establishing risk criteria (6.3.4), identifying consequences (6.4.2.6) and evaluating risk (6.4.4).

Where generative AI changes the answer

Training and test data quality decides whether a generative system fails in a way the organisation can see, and retrieval data decides it too.

Reading it in practice

Training and test data quality decides whether a generative system fails visibly or silently. The evidence is a data quality record that covers duplication, staleness, coverage of the real input distribution and the presence of personal data. For a retrieval system, the same record has to cover the corpus the system reads at run time, not only the data it was trained on.

Who it binds

The AI risk owner, with the objective owner named in the risk treatment plan. For A.4 the named role is the owner of data quality and provenance records for training and test sets.

Evidence an auditor normally samples

• Data quality and provenance records for training and test sets

The test you can run

Take data quality and provenance records for training and test sets and trace it to the decision it supported; if the trail stops before a named person, the control is not evidenced.

Common failures

• Test data drawn from the training set

Before the draft publishes

For A.4, pick the two objectives the use most affects, write the test you would apply, and record the result before the draft is published. The first artefact to produce is data quality and provenance records for training and test sets.

Where this sits against the draft

A.4 carries the annex a: ai-related objectives part of the draft's published scope, which names risk sources, the stakeholders exposed to them, risk analysis, risk treatment and controls. When the draft publishes, compare its wording on A.4 (availability and quality of training and test data) with this record and note the change in the register.