Case 26
Safe Automation Design with Human Check
Scenario
You are designing an automation that drafts responses to routine customer service emails for an insurer. A team leader will check every draft before sending.
Why this framework
Uses full PINPOINT because the consequence is higher: adds a professional stance, examples, self-checks, your own review before use, and a plan to tweak and test.
Example context
Routine email types handled by the customer service team at Fernley Energy (domestic supply): 1) Acknowledgement of a billing query. 2) Confirmation of a direct debit date change. 3) Meter reading reminder. 4) Response to a request for a copy bill. 5) Reply to a complaint about a bill. 6) Notice of a planned supply interruption. Customer communication standards (extract): Plain English, reading age around 12; use the customer's preferred name; no promises of refunds or dates not confirmed by a system; always include a reference number and how to contact us; vulnerable customers (priority services register) are handled by a named adviser. Data protection policy (extract): Only the customer name, account number and the query text may be given to AI tools. Do not use payment details, bank details, health information, priority services register status, or full addresses. All AI drafts must be reviewed by a person before sending. No customer data is retained by the tool after the session.
Prompt
Purpose: Design how this email-drafting automation should work, including where a person must check and approve the output.
Context: Use the attached list of routine email types, our customer communication standards and the data protection policy. The data protection policy is the final word on what customer data may be used.
Narrow the ask: For each email type: is it suitable, what inputs the AI needs, what it must never do, what the reviewer checks, and when it should send the email to a person instead of drafting. Then a one-page reviewer checklist. Stop and ask if an email type is unclear.
Persona and role: Act as an operations designer who values customer trust over speed.
Check your work: For each material claim, label it as [Verified Source], [Inference], [Assumption], or [Unverified]. Cite the source where available. If an unverified or assumed point could materially change the conclusion, flag it for review rather than guessing. Flag any step where a mistake would reach a customer.
Before I use it: Operations, Compliance and I will review the design, and the reviewer checklist will be versioned.
Tweak and test: we will pilot two email types first and change one thing at a time based on what reviewers reject.
Materials (pasted here in place of the attachments):
Routine email types handled by the customer service team at Fernley Energy (domestic supply): 1) Acknowledgement of a billing query. 2) Confirmation of a direct debit date change. 3) Meter reading reminder. 4) Response to a request for a copy bill. 5) Reply to a complaint about a bill. 6) Notice of a planned supply interruption.
Customer communication standards (extract): Plain English, reading age around 12; use the customer's preferred name; no promises of refunds or dates not confirmed by a system; always include a reference number and how to contact us; vulnerable customers (priority services register) are handled by a named adviser.
Data protection policy (extract): Only the customer name, account number and the query text may be given to AI tools. Do not use payment details, bank details, health information, priority services register status, or full addresses. All AI drafts must be reviewed by a person before sending. No customer data is retained by the tool after the session.Example output from ChatGPT
Fernley Energy - AI Email-Drafting Automation Design
Design v0.1 | 2 November 2026 | For Operations and Compliance review
- Executive recommendation
Automate draft preparation, not customer communication decisions. The AI should prepare suitable routine emails using only permitted data. A human must review and approve every draft before sending.
The design uses three levels:
Suitable: AI prepares a draft; a person checks and sends.
Conditional: AI drafts only after required facts are verified; otherwise route to a person.
Human-led: Route directly to a named adviser or authorised team, without an AI-generated customer response.
[Verified Source] The data protection policy permits only customer name, account number and query text to be supplied to AI tools. It prohibits payment details, bank details, health information, priority services register status and full addresses. Every AI draft requires human review, and the tool must retain no customer data after the session.
- Email-type suitability matrix
| Email type | Suitability | Permitted AI inputs | What AI must never do | Reviewer checks | Route to person instead of drafting when |
|---|---|---|---|---|---|
| 1. Billing query acknowledgement | Suitable | Name, account number, sanitised query text | Invent a resolution, promise a refund or unconfirmed reply date | Correct query, neutral tone, reference, contact details | Query involves disputed liability, sensitive data, or a vulnerable customer |
| 2. Direct debit date change | Conditional | Name, account number, sanitised request text | Access bank details, claim a change is complete without verification, promise an unconfirmed date | Actual change confirmed in system; date and status correct | Change is pending, failed, disputed or confirmation is unavailable |
| 3. Meter reading reminder | Suitable | Name and account number; query text only if relevant | Invent meter readings, deadlines or consequences | Reminder is due; approved instructions and contact method are correct | Customer disputes the request, needs special assistance, or relevant facts cannot be verified |
| 4. Copy bill request | Conditional | Name, account number, sanitised request text | Include a bill or address in AI input; send a bill without required checks; claim dispatch before confirmation | Identity and recipient verified, correct bill selected, approved delivery method | Identity, bill selection or delivery authorisation is uncertain |
| 5. Bill complaint reply | Conditional | Name, account number, sanitised complaint text | Admit fault without evidence, invent explanations, promise refunds or outcomes | Complaint captured accurately, no unsupported conclusions, correct escalation | Complaint needs investigation, a remedy decision, or specialist handling |
| 6. Planned supply interruption | Conditional | Name and account number; approved generic wording only if it qualifies as permitted query text | Invent interruption dates, times, duration, affected addresses or restoration commitments | Notice details verified against operational source; customers and dates correct | Operational information is unconfirmed, notice is urgent, or customer-specific arrangements are needed |
Source labels: Email types, permitted data and prohibited actions - [Verified Source]. Suitability classifications, checks and routing rules - [Inference], proposed controls requiring approval. No external sources used.
Two important design boundaries
[Inference] Direct debit confirmations and interruption notices present special risks. Their accurate drafting may require information beyond the three permitted AI fields. An operational system or reviewer should supply those verified facts outside the AI session, using an approved template or human editing. Do not place extra information into the query-text field simply to bypass the data policy.
[Inference] Complaints and priority services cases require stronger routing. Complaint acknowledgements may be drafted from sanitised text, but substantive outcomes should remain human-led unless Operations and Compliance approve an evidence-based workflow. Priority services cases should go to the named adviser before any AI drafting.
-
Proposed end-to-end workflow
-
Receive and classify request Identify email type and required workflow
-
Human-routing and data screening Remove prohibited data; check whether named-adviser handling applies
-
Generate permitted draft Name, account number and sanitised query text only
-
Reviewer verifies and completes Checks facts, adds externally verified details, checks policy Approve Authorised person sends using approved customer system Reject / Escalate Correct, re-review or refer to appropriate adviser Mandatory operational controls
| Control | Proposed owner | Basis |
|---|---|---|
| Screen and sanitise input before AI processing | Operations / technical control | [Inference] Required to implement the data restrictions |
| Route priority services customers to a named adviser without passing their status to AI | Customer service routing system | [Verified Source] Named-adviser requirement; [Inference] routing design |
| Keep AI drafts separate from sending permissions | IT / Operations | [Inference] Strong control supporting mandatory human review |
| Require explicit reviewer approval | Customer service reviewer | [Verified Source] Human review required |
| Record approval, rejection and reason in an approved internal system | Operations | [Inference] Audit control; must not cause AI-session data retention |
| Clear AI session data after completion | IT / AI platform owner | [Verified Source] No customer data retained after session |
Customer-facing failure point: Steps 4 and 5 are the critical release controls. An unsupported claim, wrong recipient or unverified date must not pass to a customer.
- One-page reviewer checklist
FERNLEY ENERGY - DRAFT REVIEW
Checklist v0.1 | Complete for every AI-generated email 0/12 complete
DATA PROTECTION
Only name, account number and sanitised query text were supplied to AI. No prohibited sensitive information or full address was supplied.
CUSTOMER HANDLING
Priority services cases have been routed to a named adviser.
ACCURACY
Customer identity and intended recipient have been verified through the approved process. All factual statements match approved internal records. Any dates, changes or service commitments are confirmed. No unsupported refund, fault or resolution promises appear.
COMMUNICATION
Language is plain English, suitable for approximately age 12. Preferred name, reference number and contact method are correct. Tone is respectful, clear and appropriate to the situation.
RELEASE
Any escalation or specialist review has been completed. Final text and attachments have been checked before human-approved sending.
Reviewer decision
Approve for human-controlled sending Reject or return for correction Escalate to adviser / supervisor Pending review Copy checklist and decision This interactive checklist is a review aid, not a replacement for the organisation's approval or audit system. 5. Pilot design
Start with billing query acknowledgements and meter reading reminders. [Inference] Both are more suitable for controlled drafting because they can normally avoid confirmed financial changes and complaint decisions.
Pilot measurement and change control
| Measure | What to record |
|---|---|
| First-pass acceptance | Approved without changes / total drafts |
| Rejection rate | Rejected drafts / total drafts |
| Rejection reasons | Privacy, factual error, tone, missing information, policy |
| Escalation rate | Requests routed to human-led handling |
| Review effort | Reviewer time per draft |
| Customer-facing errors | Errors identified after sending, with severity |
Change only one item at a time - for example, the prompt instructions or a template phrase - and compare the same measures across pilot versions. Keep any proposed expansion on hold until Operations and Compliance approve the results.
- Open decisions before implementation
These points are not established by the supplied extracts and must be resolved before deployment.
| Issue | Status | Required decision |
|---|---|---|
| Whether the AI tool is approved to process customer data | [Unverified] | Compliance and IT approval |
| Whether query text can be reliably screened for prohibited information | [Unverified] | Validate the input-screening process |
| Whether preferred names, approved templates and operational facts can be inserted outside AI | [Assumption] | Confirm technical design |
| Identity verification and copy-bill delivery rules | [Unverified] | Confirm existing operational requirements |
| Whether complaints may receive substantive AI-generated replies | [Unverified] | Confirm complaint-handling authority |
| Interruption notice wording, recipients and release approvals | [Unverified] | Confirm service operations process |
| Whether the platform supports non-retention and evidence of that control | [Unverified] | IT verification and Compliance sign-off |
Overall recommendation: [Inference] Proceed only with a limited, human-approved drafting pilot after the data-handling controls have been validated. Do not enable automatic sending, substantive complaint decisions, or operational confirmations based solely on AI-generated text.
The key operating principle is simple: AI may help write the email. An authorised person remains responsible for what the customer receives.
