RAG Support Agent with Real Tool-Calling
A retail support agent with real retrieval and real tool-calling,
not a chatbot demo. A cheap deterministic pre-router (an order-id regex, not an LLM call) picks
between two paths: structured order-status lookups via real Lambda-to-Lambda
lambda:Invoke calls, or grounding an answer in
unstructured FAQ text via real OpenSearch k-NN retrieval.
check_order_status,
create_support_ticket), and the agent Lambda that
routes between them. An order-id regex (O-\d{3,})
decides the path: a question naming an order calls
check_order_status (chaining into
create_support_ticket if the order isn't found);
anything else embeds the question, runs a real OpenSearch k-NN search over the FAQ index, and
generates a grounded answer from the top hit. Same dual-mode
LLMClient as
08-llm-gateway
(LLM_MODE=local by default;
LLM_MODE=bedrock can point at LocalEmu's own
Bedrock backend via BEDROCK_BACKEND=ollama, same code
either way).
Source: 23-rag-support-agent/ in the examples repo.
Deterministic routing
A regex, not an LLM call, decides the path. Reserve the model for what actually needs language understanding; keep structured lookups templated and fast.
Real tool-calling
Both tools are real lambda:Invoke calls to separate functions, the same mechanism a production tool-calling agent uses to run an action, not an in-process shortcut.
Real retrieval, real score
OpenSearch k-NN search over a real index, with a real similarity score, not a stubbed-out "closest match" function.
Step-by-Step Walkthrough
Step 1: Start LocalEmu
$ localemu start # no special flags, OpenSearch's Docker backend is on by default
# On Linux (not Docker Desktop), the agent Lambda container needs an
# explicit route back to the host for its OpenSearch calls:
$ LAMBDA_DOCKER_FLAGS="--add-host=host.docker.internal:host-gateway" localemu start On Linux, the agent Lambda's container can't resolve host.docker.internal to reach OpenSearch without this flag; Docker Desktop provides the name on its own.
Step 2: Deploy and seed
$ ./scripts/deploy.sh
→ creating OpenSearch domain le-rag-kb...
ready at <opensearch-endpoint>
$ cd terraform && terraform apply -auto-approve -var "opensearch_endpoint=<endpoint>"
→ deployed. outputs:
agent_function_name = "le-rag-agent"
check_order_status_function_name = "le-rag-check-order-status"
create_ticket_function_name = "le-rag-create-ticket"
opensearch_endpoint = "<opensearch-endpoint>"
orders_table = "le-rag-orders"
tickets_table = "le-rag-tickets"
→ seeding knowledge base + orders...
embedding dimension: 4
indexed doc 1: Our return policy allows returns within 30 days...
seeded order O-1001 -> SHIPPED The OpenSearch domain is created with the AWS CLI directly, not by Terraform: the Terraform AWS provider doesn't send its first DescribeDomain for 10 minutes on create (and again on delete), while LocalEmu reports the domain ready in seconds. deploy.sh creates the domain, waits for its endpoint, and passes it into Terraform. scripts/seed.py then builds the k-NN index and seeds the 4 FAQ docs and 2 known orders.
Step 3: The deterministic pre-router
# src/agent.py, the pre-router: a regex, not an LLM call.
ORDER_ID_RE = re.compile(r"\b(O-\d{3,})\b", re.IGNORECASE)
def handle(event, _ctx):
question = (event.get("question") or "").strip()
if not question:
return {"answer": None, "error": "missing_question"}
order_match = ORDER_ID_RE.search(question)
if order_match:
return _handle_order_question(order_match.group(1).upper(), question)
return _handle_kb_question(question) One regex, checked before anything else runs. No LLM call is made just to decide which path to take.
Step 4: Order-status path, with tool chaining
# Order-status path: a real lambda:Invoke to the check_order_status tool.
$ aws lambda invoke --function-name le-rag-agent \
--payload '{"question": "What\u0027s the status of order O-1001?"}' /tmp/out.json
$ cat /tmp/out.json
{
"answer": "Order O-1001 is currently SHIPPED.",
"route": "check_order_status",
"order_id": "O-1001",
"status": "SHIPPED"
}
# Unknown order: the agent chains a SECOND real lambda:Invoke, to
# create_support_ticket, when the first tool comes up empty.
$ aws lambda invoke --function-name le-rag-agent \
--payload '{"question": "Can you check on order O-9999 for me?"}' /tmp/out.json
{
"answer": "I couldn\u0027t find an order matching O-9999. I\u0027ve opened support ticket TKT-A1B2C3D4 so a human can look into it.",
"route": "check_order_status_then_create_ticket",
"order_id": "O-9999",
"ticket_id": "TKT-A1B2C3D4"
} A known order calls check_order_status and returns a templated answer. An unknown order chains into a second real tool call, create_support_ticket, so a human can pick it up.
Step 5: Knowledge-base path, with real retrieval
# Knowledge-base path: embed -> real OpenSearch k-NN search -> generate.
$ aws lambda invoke --function-name le-rag-agent \
--payload '{"question": "How long do I have to send back something I bought?"}' /tmp/out.json
$ cat /tmp/out.json
{
"answer": "You have 30 days from purchase to return an item with a valid receipt.",
"route": "knowledge_base",
"retrieved_doc": "Our return policy allows returns within 30 days of purchase with a valid receipt.",
"retrieval_score": 0.98
} The question is embedded, searched against the OpenSearch k-NN index, and the top hit's text is handed to the LLM as grounding context for a one-sentence answer. retrieval_score is a real similarity score from the index, not a stub.
What the run proves
Seven integration tests: three against the knowledge base, two known-order lookups, one two-tool chain, and one input-validation case.
| Test | Proves |
|---|---|
| test_kb_question_retrieves_correct_doc_and_grounds_the_answer | a return-policy question retrieves the return-policy doc and the answer contains "30 days" |
| test_kb_question_different_query_retrieves_different_doc | a shipping question retrieves the shipping doc, not the return-policy one |
| test_kb_question_retrieval_score_is_a_real_similarity_not_a_stub | the score is a real OpenSearch k-NN value, > 0.9 in local mode |
| test_known_order_status_uses_the_tool_lambda | order O-1001 routes to check_order_status, returns SHIPPED |
| test_another_known_order | order O-1002 returns PROCESSING |
| test_unknown_order_chains_into_a_support_ticket | an unknown order opens a real ticket row, status OPEN |
| test_missing_question_is_a_clean_error_not_a_crash | an empty question returns a structured error, not an exception |
Design note: reaching a sidecar container from inside a Lambda
OPENSEARCH_ENDPOINT (from Terraform) is a host-mapped address: reachable from your laptop, not from inside the agent Lambda's own Docker container, which is a peer container, not the host. agent.py swaps localhost/127.0.0.1 for host.docker.internal before calling OpenSearch.
No other example in this repo needed this: earlier ones talk to a sidecar container from a script running on your laptop, never from a Lambda calling into another sidecar container. On Linux, Docker Desktop's automatic name doesn't exist, so LocalEmu needs LAMBDA_DOCKER_FLAGS="--add-host=host.docker.internal:host-gateway" to add it.
Full source: src/agent.py
The router, retrieval, and tool-orchestration logic. Both tool calls are real Lambda-to-Lambda invokes.
"""Retail support agent - real OpenSearch k-NN retrieval + real tool-calling.
Routing is a cheap, deterministic pre-router (a regex for an order id),
not an LLM call - a common, pragmatic agent pattern: reserve the LLM for
what actually needs language understanding (grounding an answer in
unstructured FAQ text), and keep structured-data lookups templated and
fast. Two paths:
order-status question -> check_order_status tool Lambda
(not found -> create_support_ticket tool Lambda)
anything else -> embed -> OpenSearch k-NN search over the FAQ
knowledge base -> generate a grounded answer
Both tool calls are real Lambda-to-Lambda `lambda:Invoke` calls, not
in-process function calls - the same mechanism a production
tool-calling agent uses to run an action.
"""
from __future__ import annotations
import json
import logging
import os
import re
import boto3
from llm_client import get_client
LOG = logging.getLogger()
LOG.setLevel(logging.INFO)
OPENSEARCH_ENDPOINT = os.environ["OPENSEARCH_ENDPOINT"]
KB_INDEX = os.environ.get("KB_INDEX", "faq-kb")
CHECK_ORDER_STATUS_FUNCTION = os.environ["CHECK_ORDER_STATUS_FUNCTION"]
CREATE_TICKET_FUNCTION = os.environ["CREATE_TICKET_FUNCTION"]
_lambda = boto3.client("lambda")
_llm = get_client()
ORDER_ID_RE = re.compile(r"\b(O-\d{3,})\b", re.IGNORECASE)
def handle(event, _ctx):
question = (event.get("question") or "").strip()
if not question:
return {"answer": None, "error": "missing_question"}
order_match = ORDER_ID_RE.search(question)
if order_match:
return _handle_order_question(order_match.group(1).upper(), question)
return _handle_kb_question(question)
def _invoke(function_name: str, payload: dict) -> dict:
resp = _lambda.invoke(
FunctionName=function_name,
InvocationType="RequestResponse",
Payload=json.dumps(payload).encode(),
)
return json.loads(resp["Payload"].read())
def _handle_order_question(order_id: str, question: str) -> dict:
result = _invoke(CHECK_ORDER_STATUS_FUNCTION, {"order_id": order_id})
if result.get("found"):
status = result["status"]
return {
"answer": f"Order {order_id} is currently {status}.",
"route": "check_order_status",
"order_id": order_id,
"status": status,
}
ticket = _invoke(
CREATE_TICKET_FUNCTION,
{"subject": f"Order {order_id} not found", "body": question},
)
return {
"answer": (
f"I couldn't find an order matching {order_id}. I've opened support "
f"ticket {ticket['ticket_id']} so a human can look into it."
),
"route": "check_order_status_then_create_ticket",
"order_id": order_id,
"ticket_id": ticket["ticket_id"],
}
def _handle_kb_question(question: str) -> dict:
query_vector = _llm.embed(question)
hits = _knn_search(query_vector, k=1)
if not hits:
return {"answer": None, "route": "knowledge_base", "reason": "no_hits"}
top_doc = hits[0]["_source"]["text"]
prompt = (
f"Context: {top_doc}\n\n"
f"Question: {question}\n\n"
"Answer the question using only the context above, in one sentence."
)
answer = _llm.generate(prompt)
return {
"answer": answer,
"route": "knowledge_base",
"retrieved_doc": top_doc,
"retrieval_score": hits[0]["_score"],
}
def _container_reachable_endpoint(endpoint: str) -> str:
"""OPENSEARCH_ENDPOINT is a *host*-mapped address (``localhost:<port>``) -
reachable from your laptop, not from inside the agent Lambda's own
Docker container (a peer container, not the host). Docker Desktop's
``host.docker.internal`` resolves to the host from inside any
container, which routes back through the same port mapping - so this
swap is all that's needed, no raw container IP required.
"""
return endpoint.replace("localhost", "host.docker.internal").replace("127.0.0.1", "host.docker.internal")
def _knn_search(vector: list[float], *, k: int) -> list[dict]:
import urllib.request
endpoint = _container_reachable_endpoint(OPENSEARCH_ENDPOINT)
body = json.dumps({"size": k, "query": {"knn": {"embedding": {"vector": vector, "k": k}}}}).encode()
req = urllib.request.Request(
f"http://{endpoint}/{KB_INDEX}/_search",
data=body,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=10) as resp:
parsed = json.loads(resp.read())
return parsed.get("hits", {}).get("hits", []) Files
Repository layout.
23-rag-support-agent/
├── README.md
├── requirements.txt
├── src/
│ ├── agent.py (router + retrieval + tool orchestration)
│ ├── check_order_status.py (tool Lambda)
│ ├── create_support_ticket.py (tool Lambda)
│ ├── llm_client.py (same dual-mode client as 08-llm-gateway)
│ └── local_answers.jsonl (canned embed/generate answers for LLM_MODE=local)
├── terraform/ (11 resources, one provider)
├── tests/
│ └── test_agent.py (7 tests)
└── scripts/
├── deploy.sh
├── test.sh
├── teardown.sh
└── seed.py