Build a support agent

This is a good first project because the failure modes are obvious. A support agent that invents a refund policy is worse than no support agent at all - so the interesting work is making it decline.

By the end you will have an agent that:

answers from your product documentation rather than the model's training data
follows your escalation procedure when it should not answer
remembers what a customer told it in an earlier conversation
lives in Slack, in threads, without cross-talk between people
asks a person before doing anything consequential

There are five steps, each one useful on its own.

The agent

skills_dir = "skills"

[[agents]]
config_dir = "agents/support"

Then axl-config/agents/support/agent.toml:

key = "support"
display_name = "Support"
description = "Answers product questions from the product documentation"
identity = """
You are a product support assistant. Answer from the product documentation.
If the documentation does not cover something, say so plainly and escalate.
Never guess at policy, pricing, or account-specific facts.
"""
toolsets = ["web", "memory"]

response_format = [
  "Lead with the answer",
  "Name the document you used",
  "Keep replies short enough to read on a phone",
]

The identity is doing real work here. "Never guess at policy, pricing, or account-specific facts" names the three categories where a confident wrong answer causes actual damage.

Give it your documentation

An agent that answers from documents needs documents. Declare a set in axl-config/docs.toml:

[[docs]]
name = "product-guide"
path = "docs/product-guide"
tags = ["product"]

Put your Markdown under axl-config/docs/product-guide/content/, then index it:

axl rag ingest -c /path/to/axl-config

Before connecting anything, check that retrieval actually works:

axl rag search "How do I reset a password?" -c /path/to/axl-config

If that returns nothing useful, stop here. No amount of prompt tuning on the agent will fix a corpus that does not answer the question - and you have just found the problem in the one place it is cheap to fix. See Ingest documents.

Now connect it. Create axl-config/agents/support/rag.toml:

sources = ["product-guide"]
trigger_mode = "always"

always suits a support agent: nearly every question is a product question, and the cost of a retrieval you did not need is far lower than the cost of an ungrounded answer.

Teach it to escalate

Escalation is a procedure, which makes it a skill rather than a document. Create axl-config/skills/escalation/SKILL.md:

---
name: escalation
description: Hand a support question to a human when it cannot be answered from documentation
tags: [support]
---

# Instructions

Escalate when any of these is true:

- The product documentation does not cover the question.
- The question concerns a specific customer's account, billing, or contract.
- The user is reporting a possible outage or data loss.
- The user has asked the same thing twice and is still unsatisfied.

To escalate: say plainly that you are handing this to a person, summarize the question
in one sentence, and state what you already checked.

Enable it:

skills = ["escalation"]

The description is what the agent sees at all times, so it is what decides whether the skill ever fires. Everything else loads on demand.

Put it in Slack

Configure the adapter:

SLACK_BOT_TOKEN=xoxb-...
SLACK_SIGNING_SECRET=...
SLACK_BOT_USER_ID=U...
SLACK_TEAM_ID=T...
SLACK_AGENT_KEY=support

Then set the reply mode:

[channel]
reply_mode = "thread_isolated"

In a busy support channel, thread_isolated gives each thread its own agent session. Without it everyone's questions share one history, and the agent starts answering one person using another person's context.

Bound what it can do

Add axl-config/agents/support/intent.toml:

[objective]
problem = "Answer product-support questions from the product documentation"
why = "Reduce time the team spends answering the same questions"
success_criteria = [
  "Answers name the source document",
  "Questions outside the documentation are escalated, not guessed at",
]

[constraints]
rules = [
  "Do not invent policy, pricing, or account-specific facts",
  "Do not promise remediation or timelines",
]

[autonomy]
level = "require_approval"

[stop_rules]
rules = ["Stop when the question is answered or has been escalated"]

require_approval is the right default while the agent is new. AXL enforces it in code, so it holds even if someone talks the model into being helpful.

What you now have

Roughly sixty lines of configuration, no application code, and:

Answers grounded in your documentation, with provenance on every ingested chunk
An escalation procedure the agent applies on its own
Per-thread isolation in Slack, and memory that follows a person across conversations
An explicit objective, constraints, and stop rules
A person in the loop for anything consequential
Every tool call in the audit log, and cost and latency per run in traces

Where it usually goes next

Answers are vague or wrong. The problem is almost always the corpus, not the prompt. Run axl rag search with the failing question and look at what comes back. See Configure search.

It escalates too much or too little. Edit the skill's rules - they are the criteria, and they are meant to be edited as you learn what the team actually wants handled.

You want to know whether it is any good. Build a dataset from real questions and score it before each change. See Evaluate agent quality.

Replies are too long. Tighten response_format. Models are verbose by default and stay that way until told otherwise.

Evaluate agent quality - measure it.
Configure the shield - what happens when a customer pastes something hostile.
Document pipeline - when the work is a process, not a chat.
Middleware - caching and approval gates as traffic grows.