Week 23 self-check | AI GRCA Practice Lab

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

An AI triage tool helps decide which emergency department patients are seen first. What's the key practitioner question about its accuracy failures?

Whether false negatives (missing a genuinely urgent case) have been specifically tested and controlled, given the safety consequenceOnly whether average accuracy is above 90%Whether the tool is cheaper than adding staffWhether the interface looks modern

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

In public sector AI, why does 'equality impact' carry particular weight?

Public bodies typically have statutory duties to consider equality impacts on the communities they serveEquality impact assessments are optional paperwork with no legal basisThis concern is identical to general fairness testingEquality impact only matters in the private sector

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

A recruitment vendor's fairness testing only used labour-market data from one country. What's the practitioner concern if the tool is deployed elsewhere?

This only matters for public-sector recruitmentNo concern — fairness testing is universally generalisableThe testing evidence may not generalise to a different population or labour-market contextTesting location is irrelevant to fairness conclusions

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

Why is 'clinical decision support' distinguished from 'diagnosis' in healthcare AI governance?

Diagnosis tools are always safer than decision-support toolsThis distinction only matters for veterinary AIThere is no meaningful distinctionDecision-support tools inform a clinician's judgement; diagnostic tools may carry different regulatory and liability implications

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

Why does 'access control' matter specifically for enterprise RAG copilots?

Access control is irrelevant if the model is proprietaryWithout proper filtering, a copilot could surface documents to staff who shouldn't have access to themAccess control only matters for external-facing systemsRAG systems have no access to internal documents by design

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

A hospital AI tool recommends a treatment priority but a clinician always follows it without independent judgement. What sector-specific issue does this raise?

This is a purely IT infrastructure issueThis only matters if the hospital is publicly fundedNo issue — following AI recommendations is always appropriateA patient-safety and decision-support boundary issue, since the tool may be functioning as a de facto diagnostic decision-maker

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

Which sector is most associated with statutory 'equality impact' duties in this course's coverage?

Financial servicesInternal copilotsPublic sectorInsurance

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

In financial services AI, what does 'model risk management' specifically address?

Employee satisfaction with financial toolsMarketing effectiveness of financial productsThe risk that a model used in decision-making is flawed, misused or poorly understood, and how that's controlledOnly IT infrastructure uptime

WEEK 23 · Controls, operations & judgement · Sector Case Studies and Practitioner Judgement

Which two sectors in this week's coverage most explicitly require attention to how automated decisions affect vulnerable or differently-situated groups?

Internal copilots and recruitment onlyInsurance and public sectorOnly financial servicesOnly healthcare