Lack of transparency and explainability

AI-related risk source: Where the system's workings or the reasons for its outputs cannot be understood, errors go undetected, accountability weakens and stakeholders lose trust. The artefact behind the answer is assessment of the system's interpretability and the explanations available. Use lack of transparency and explainability as a prompt in the identification step, not as a finding.

What it asks

AI-related risk source: Where the system's workings or the reasons for its outputs cannot be understood, errors go undetected, accountability weakens and stakeholders lose trust.

Where generative AI changes the answer

Opacity is structural here: the organisation often cannot inspect the model at all, so the risk has to be managed through evidence about behaviour.

Reading it in practice

Opacity is structural when the model is a supplier's. The organisation often cannot inspect the model at all, so the risk has to be managed through evidence about behaviour: test results, output sampling, incident records and the supplier's disclosures. The evidence is that behavioural record, not a claim that the model is a black box.

Who it binds

The assessment team that runs identification, and the business owner who knows the use. For B.3 the named role is the owner of assessment of the system's interpretability and the explanations available.

Evidence an auditor normally samples

• Assessment of the system's interpretability and the explanations available

The test you can run

Take assessment of the system's interpretability and the explanations available and trace it to the decision it supported; if the trail stops before a named person, the control is not evidenced.

Common failures

• Opaque model used where explanation is required

Before the draft publishes

For B.3, add the sources that apply to your use to the identification checklist, with a short note on why each one is or is not present. The first artefact to produce is assessment of the system's interpretability and the explanations available.

Where this sits against the draft

B.3 carries the annex b: ai-related risk sources part of the draft's published scope, which names risk sources, the stakeholders exposed to them, risk analysis, risk treatment and controls. When the draft publishes, compare its wording on B.3 (lack of transparency and explainability) with this record and note the change in the register.