Someone writes: “My order says delivered, but there’s nothing at my door.” A fluent answer is easy to generate. A useful answer needs more: relevant context, the right next step, and some honesty about what the system cannot check.
That is the problem behind my AmazonHelp Support Agent. It is a single-turn support prototype, not an integration with a live customer support team. Here is how its pieces fit together, without pretending a chatbot can magically look up an order.
Start with the message, not the model
I think of the system as a small assembly line. Each stage has one job. Intent classification identifies the kind of request. Retrieval finds similar historical customer/reply pairs. Generation uses those examples to draft a response. Escalation logic decides whether human follow-up is needed.
- Customer message“Where is my order?”
- Classify intentIdentify the kind of problem
- Retrieve examplesFind relevant past replies
- Draft a responseUse examples as context
- Choose a routeSelf-service or human follow-up
A retrieved reply is an example of how an issue was handled. It is not proof that the same policy still applies, or that this customer qualifies for a refund. That distinction matters more than making the answer sound confident.
Retrieval gives the draft something to work with
Instead of asking the model to answer from an empty page, the pipeline supplies related messages and replies. The model can use them to understand phrasing and likely next steps.
- Current messageThe problem to answer
- Relevant examplesSimilar questions and past replies
- InstructionsTone, boundaries, and response format
- DraftA proposed reply, not a completed action
The project prefers a sentence-transformer index and falls back to TF-IDF if building the embedding index fails. That fallback is about keeping retrieval available. It does not mean both methods find equally useful examples. I go into that tradeoff in TF-IDF vs Embeddings for Support Search.
This small sketch shows the separation of responsibilities. It is explanatory pseudocode, not a copy of the project's entry point:
examples = retriever.retrieve(message, k=3)
intent = classify_intent(message)
draft = generate_reply(message, examples)
route = decide_follow_up(message, intent, examples)
result = {"draft": draft, "route": route}Returning a draft and a route separately makes the output easier to inspect. “Here is a reply” and “this needs a person” can both be true.
A fallback should change the decision too
A common trap is swapping the search method while keeping every downstream assumption unchanged. The support project has retriever-specific similarity thresholds because the scores from its two retrieval methods are not interchangeable.
Even with a threshold, similarity is not a probability that an answer is correct. A familiar-looking message can still require account access or involve details missing from the corpus.
- Read the requestDoes it need account-specific action?
- Check the evidenceAre the retrieved examples useful?
- Check the next stepIs there a verified self-service route?
- Return a decisionOffer a route or request human follow-up
Keep links and actions boring
The project uses a curated support-link map rather than asking the model to invent destinations. Its reply handling replaces link placeholders and handles missing routes. Those links still need review before real use; a stored URL can become stale.
The agent does not actually refund an order, change an account, or remember a previous conversation. Keeping that boundary visible makes the output easier to trust and debug.
What I would check before real use
I would review examples where a confident draft is wrong, where retrieval finds nothing useful, and where a person is needed but not requested. I would also check the links independently. The repository contains labelled data and agreement tooling, but that is not evidence of production reliability.
For me, the useful part of this project is the plumbing around the model: context in, a bounded draft out, and a clear path when it cannot finish the job. For systems that choose their own tools, the next question is when to stop the loop.