An Agentic System is a combination of state, policies, tools, agents, and a loop that operates in an environment to achieve an outcome.
-
Trigger: An event that initiates a workflow, for example, a user message, a schedule, a webhook, a record change, or an alert.
-
Orchestrator: A coordinator that maintains state and lifecycle, routes tasks, and manages approvals, timeouts, retries, dependencies, and the final result.
-
Agent: Makes decisions at non-deterministic points.
-
Worker/Tool: Executes a specific, bounded piece of work.
Sample flow: Trigger → Validate/Deduplicate → Orchestrator → Agent/Worker Steps → Approval (if needed) → Result → Audit
An Orchestrator does not necessarily have to be an LLM. Often it’s better for the lifecycle and deterministic policies to be hardcoded, with only ambiguous decisions delegated to the model.
Trigger considerations: authentication, rate limiting, ordering, idempotency, deduplication, webhook verification, and the ability to replay.
Designing a Human-in-the-Loop with an Agentic System
Example: An Agent handling an Issue and proposing a patch
- Trigger: New issue or user command.
- Intake: Validate input, permissions, and scope.
- Discovery: Read relevant rules, repo, tests, and history.
- Planning: Build a hypothesis or plan with specific change points.
- Approval 1 (Human): A human reviews the scope and approves the riskier parts of the plan.
- Execution: Apply the change in an isolated branch or workspace.
- Verification: Run tests, lint, security checks, and review the diff.
- Evaluation: Assess acceptance criteria and confidence level.
- Approval 2 (Human): A human approves the merge, release, or irreversible action.
- Delivery/Report: Open a PR with evidence and an audit trail.
Human-in-the-Loop should not be just an “OK” button on something vague. At the approval point, the system must clearly show: the exact action, the reasoning, the data and system state, the effect, the cost, the rollback strategy, and the evidence.
Control pattern:
- Low risk + high confidence → auto-execute and log
- Medium risk or low confidence → ask, clarify, or review
- High risk or irreversible → explicit human approval
- Policy violation → block
Evaluation metrics: task success rate, need for manual correction, tool errors, safety violations, latency, cost, rate of escalation, and quality of evidence.