Monitoring and review

Monitoring and review exist to check and raise the quality and effectiveness of the process's design, implementation and results, and are planned into it (ISO 31000:2018, 6.6), covering planning, collecting and analysing information, recording what is found and feeding it back. The step before 6.6 supplies its input, so run both together.

What it asks

Monitoring and review exist to check and raise the quality and effectiveness of the process's design, implementation and results, and are planned into it (ISO 31000:2018, 6.6), covering planning, collecting and analysing information, recording what is found and feeding it back. For AI they compare the process with how the system actually behaves in use, pick up drift, new risks and changed context, and point to improvements.

Where generative AI changes the answer

Monitoring has to catch a silent model change, a new use of the same tool and a drop in output quality, and none of those arrives as an alert by default.

Who it binds

The AI risk owner, the assessment team and the business owner of each generative AI use. For 6.6 the named role is the owner of monitoring plan and review records for each AI system in operation.

Evidence an auditor normally samples

• Monitoring plan and review records for each AI system in operation

The test you can run

Take monitoring plan and review records for each AI system in operation and trace it to the decision it supported; if the trail stops before a named person, the control is not evidenced.

Common failures

• No monitoring of the deployed system's behaviour against the assessed risks

Before the draft publishes

For 6.6, run the step once, by hand, on one live generative AI use, and keep the working paper. A method that has never been run on a real use is a template. The first artefact to produce is monitoring plan and review records for each AI system in operation.

Where this sits against the draft

6.6 carries the risk management process part of the draft's published scope, which names risk sources, the stakeholders exposed to them, risk analysis, risk treatment and controls. When the draft publishes, compare its wording on 6.6 (monitoring and review) with this record and note the change in the register.