The Core Problem
An AI agent based on a Large Language Model (LLM) alone is unreliable for business. Here’s why:
LLMs are trained on public internet data (cutoff dates in 2023-2024). They don’t know:
- Your customer information
- Your internal documentation
- Your product specifications
- Your company policies
- Your recent market data
So when your customer asks “How do I reset my password?”, the LLM doesn’t have access to your knowledge base. It makes a plausible-sounding guess. Sometimes it’s right. Sometimes it’s wrong—and you lose the customer.
This is hallucination. And it’s why most AI chatbots fail in production.
The solution: Retrieval-Augmented Generation (RAG).
What Is RAG?
RAG is deceptively simple: Before the LLM answers a question, retrieve the relevant documents from your business data. Then feed those documents to the LLM as context.
Without RAG:
- User asks: “What’s your refund policy?”
- LLM guesses based on general training data
- Agent gives a wrong answer
With RAG:
- User asks: “What’s your refund policy?”
- RAG searches your documentation for “refund policy”
- RAG retrieves your actual refund policy document
- LLM reads the document and answers based on your actual policy
- Agent gives a correct answer with a source citation
How RAG Works in Agentforce
Agentforce makes RAG work by connecting three components:
1. The Knowledge Source
Your data: knowledge base articles, help docs, product specs, CRM data, database records.
In Agentforce, knowledge sources include:
- Salesforce Knowledge Base
- Content management systems
- Custom databases
- External documents (PDFs, web pages)
2. The Retrieval Engine
When an agent needs to answer a question, the retrieval engine:
- Converts the question into a semantic search query
- Finds the most relevant documents from your knowledge source
- Ranks results by relevance
- Returns the top 3-5 documents
3. The LLM
The LLM reads the retrieved documents, then answers the question based on them.
LLM prompt (simplified): ``` You are a helpful customer support agent. Answer the following question using ONLY the provided documents. If the answer isn’t in the documents, say “I don’t know.”
Documents: [retrieved content here]
Question: What’s your refund policy?
Answer: [generates answer from documents] ```
Why This Matters for Your Business
RAG Improves Quality
- Accuracy: Agents reference your actual data, not guesses
- Currency: Agents answer based on your latest docs (updated in real time)
- Traceability: Every agent answer comes with a source citation
RAG Reduces Risk
- Hallucinations prevented: Agent can’t answer what’s not in your docs
- Brand safety: Agents can’t make promises your company didn’t make
- Compliance: Audit trail shows which documents informed each decision
RAG Improves ROI
- Faster resolution: Agent finds answers instantly (no human search)
- Higher resolution rate: Agents resolve 50-70% of conversations without escalation
- Lower support costs: One agent replacing 2-3 humans
The Hidden Complexity: Knowledge Quality
RAG only works if your knowledge source is good. Here’s the reality:
Most companies have terrible knowledge bases:
- Outdated documentation (2+ years old)
- Contradictory information (Policy A says X, Policy B says Y)
- Missing content (gaps in coverage)
- Poor organization (can’t find anything)
A RAG system is only as good as the data it retrieves.
Preparing Your Knowledge Base for RAG
Before deploying an agent with RAG:
-
Audit your docs
- What’s outdated? (Flag for refresh)
- What’s contradictory? (Resolve conflicts)
- What’s missing? (Fill gaps)
-
Organize by intent
- Group related content (not by department)
- Use consistent terminology
- Link related topics
-
Test retrieval
- Run sample questions
- Check if retrieval returns the right docs
- Refine search parameters
-
Monitor in production
- Track retrieval accuracy (is the right doc returned?)
- Track answer quality (is the agent answering well?)
- Adjust weights and rankings over time
RAG + Prompt Engineering = Reliable Agents
RAG prevents hallucination, but LLMs are still unpredictable. That’s where prompt engineering comes in.
Your agent’s prompt needs to tell the LLM:
- What to do: “You are a customer support agent”
- What to use: “Use ONLY the provided documents”
- What to do when stuck: “If you don’t know, escalate to a human”
- How to behave: “Be friendly but professional. Never make promises about timelines”
- What boundaries exist: “Don’t discuss pricing. Refer to the sales team”
Example (simplified): ``` You are a helpful customer support agent for Acme Corp.
Your job:
- Answer customer questions using the provided knowledge base documents
- If the answer isn’t in the docs, tell the customer you don’t know and escalate
Tone: Friendly, professional, concise Escalation triggers:
- Angry or threatening customer
- Technical issue you can’t resolve
- Billing dispute over $100
- Request for pricing or demos
You MUST:
- Always cite the source document
- Never make promises about refunds or returns
- Never discuss competitor products
- Admit when you don’t know something ```
With good prompts + RAG, you get reliable agents.
Real-World Example: Resolve Service Agent
In Funnelists’ Resolve product, we use RAG like this:
- Customer asks: “How do I cancel my subscription?”
- RAG retrieves: Your subscription cancellation policy doc
- Agent reads: The document (it says “3-day free trial, $50 cancellation fee after”)
- Agent answers: “You can cancel anytime. If you’re in the free trial, no fee. After that, there’s a $50 cancellation fee. Want me to process that now?”
- Result: Customer knows exactly what to expect. No hallucinations, no surprises.
This conversation happens 500+ times/month without human involvement. That’s 500 support tickets you don’t have to handle manually.
Common Mistakes with RAG
Mistake 1: Uploading Unstructured Data
RAG works best with organized, searchable data. Uploading a folder of PDFs and hoping for the best rarely works.
Fix: Structure your knowledge base. Use metadata tags (topic, audience, urgency). Make search easy.
Mistake 2: Not Monitoring Retrieval Quality
You deploy the agent and assume it’s working. But maybe it’s retrieving the wrong documents 30% of the time.
Fix: Track retrieval accuracy. Sample conversations weekly. Check if the right docs were retrieved.
Mistake 3: Ignoring Prompt Limitations
You think RAG solves everything. But a bad prompt can override good retrieval.
Fix: Treat prompts as seriously as retrieval. Refine them based on real conversations.
The Path Forward
If you’re building an agent in Agentforce:
Week 1: Audit your knowledge base. Identify gaps and outdated content.
Week 2: Organize and structure your docs. Tag by topic, audience, urgency.
Week 3: Test RAG retrieval. Sample 50 questions and check if you get the right docs.
Week 4: Engineer your prompts. Define behavior, boundaries, and escalation triggers.
Week 5: Deploy with human-in-the-loop. Monitor conversations and refine.
What’s Next?
RAG + LLMs + Agentforce = the foundation for trustworthy agent automation.
But there’s more. Add agent governance, multi-agent orchestration, and continuous monitoring—and you have a production-grade system.
Ready to build reliable AI agents for your business?
