Human-in-the-Loop & Escalation SOP

Mercury Security | 2025

Introduction

Human oversight is a cornerstone of responsible AI deployment. No AI system should operate without defined escalation pathways that allow humans to intervene in real time. This Standard Operating Procedure (SOP) describes how human-in-the-loop (HITL) oversight is designed, tested, and maintained for AI agents. The EU AI Act, NIST AI RMF, and ISO/IEC 42001 all emphasize the requirement for clear oversight and accountability mechanisms (European Union, 2024; NIST, 2023; ISO, 2023).

Purpose

The purpose of this SOP is to ensure that:

• Human escalation points are clearly defined for all AI agents.
• Escalations are reliably triggered when certain thresholds are met.
• Logs capture both the trigger event and the outcome of escalation.
• Governance teams have a repeatable process for reviewing and improving oversight.

Scope

This SOP applies to all customer-facing and internal AI agents deployed by or audited through Mercury Security.

Roles and Responsibilities

• System Owner: Defines escalation criteria and ensures alignment with business context.
• Support/Ops Teams: Monitor escalation queues, resolve tickets, and document outcomes.
• Governance Lead: Reviews escalation performance and ensures compliance with frameworks.
• Auditor: Validates escalation functionality through sandbox testing.

Escalation Criteria

Escalation must be triggered under at least the following conditions:

• Confidence threshold not met. The AI system flags low confidence in its answer.
• Restricted topic encountered. Queries involving sensitive subjects such as medical, legal, or financial advice.
• Policy violation detected. AI response breaches guardrails (e.g., disallowed content).
• User request. A human explicitly asks to speak with a person.

Escalation Process

1. Trigger Event: AI system identifies a condition requiring escalation.
2. Routing: The query is forwarded to the appropriate escalation queue (support, compliance, legal).
3. Notification: Assigned personnel receive a real-time alert.
4. Resolution: Human agent responds, documents the action taken, and closes the loop.
5. Logging: The trigger, escalation route, resolution, and timeframe are captured in tamper-evident logs.

Testing HITL Functionality

Escalation processes must be tested at least quarterly. Sandbox testing should include:

• Simulated restricted prompts.
• Confidence-threshold triggers.
• Forced escalation by user request.

Auditors must validate that the escalation fires, reaches the correct queue, and is logged appropriately.

Review and Continuous Improvement

Governance teams must review escalation logs monthly. Any failures or delays should be documented, with remediation assigned and tracked. Escalation criteria must be updated as new risks or system capabilities emerge.

Conclusion

Human-in-the-loop oversight is not optional; it is the safeguard that makes AI systems auditable, compliant, and trustworthy. By following this SOP, organizations demonstrate defensible governance and accountability in line with regulatory expectations.

References

European Union. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (AI Act). Official Journal of the European Union. https://eur-lex.europa.eu

ISO. (2023). ISO/IEC 42001:2023 Information technology — Artificial intelligence — Management system. International Organization for Standardization.

National Institute of Standards and Technology. (2023). AI Risk Management Framework (NIST AI RMF 1.0). Gaithersburg, MD: NIST.