“Human in the loop” is not an operating policy.
Teams need explicit rules for when AI can act, when a person must review, and when the task should remain human-owned from the start.
The right boundary depends on confidence, sensitivity, account value, and how reversible the action is.
Score four types of risk
Targeting risk
How confident are you that the company and person belong in the ICP?
Context risk
How reliable and unambiguous is the signal? Could it have several reasonable explanations?
Message risk
Could the draft expose sensitive observation, make an unsupported claim, or create reputational damage?
Conversation risk
Is the next action easy to reverse, or does it involve pricing, legal, security, or a strategic relationship?
Use these dimensions to determine the review level.
Level 1: automatic with spot checks
Use automation when:
- ICP fit is strong
- the signal source is public and reliable
- the play has been tested
- the message structure is approved
- the next action is low risk
- stopping rules are clear
Spot-check samples for drift. Automatic does not mean unmonitored.
Level 2: review queue
Require review when:
- the signal is plausible but ambiguous
- the team is testing a new segment
- the message uses a new point of view
- several sources conflict
- the account has prior history
- the draft needed substantial research inference
The reviewer should see the evidence and the reason for prioritization beside the draft.
Level 3: human-owned
Keep a person responsible when:
- the account is strategic
- the context feels sensitive
- the prospect has replied negatively
- the conversation involves pricing, legal, security, or implementation
- the account has an active opportunity
- a public error could damage the brand
AI can still summarize context or suggest options, but it should not own the decision.
Define mandatory stop conditions
Automation should stop when:
- a reply arrives
- the buyer opts out
- the account is disqualified
- another owner starts a conversation
- source confidence drops
- the signal expires
- the message fails policy checks
A sequence that ignores new information is not intelligent automation.
Review the inputs, not only the output
A polished message can still be wrong because the account should never have entered the workflow.
Review:
- ICP criteria
- signal definitions
- attribution quality
- expiry windows
- scoring thresholds
- message exclusions
- ownership rules
The upstream decision often matters more than the final wording.
Track why humans intervene
Use standardized edit reasons:
- wrong account
- wrong contact
- stale signal
- weak hypothesis
- sensitive detail
- unsupported claim
- tone mismatch
- incorrect next action
If one reason appears frequently, change the system rule. Do not rely on reviewers to fix the same problem indefinitely.
Audit automation by play
Measure each play separately:
- automated versus reviewed volume
- approval rate
- edit rate and edit reasons
- replies and positive replies
- negative responses and opt-outs
- meetings and opportunities
- false positives
A play can earn more automation as evidence improves. It can also lose automation when quality drops.
Make accountability visible
Every action should have an owner, even when AI performs the work.
Document who owns:
- ICP definition
- source approval
- message policy
- exception review
- reply handling
- performance calibration
AI can execute a workflow. The team remains accountable for the workflow’s decisions.