The Prompt Is Everything
You have an LLM. It’s powerful. But untamed, it’s useless for business.
A customer asks your agent: “Can I get a refund?”
Without guidance, the LLM might say:
- “Sure, you can always get a refund!” (violates your policy)
- “Refunds are complex, let me transfer you to accounting” (lazy, no value)
- “I don’t know what a refund is” (hallucination)
With the right prompt, the LLM says:
- “You’re in the 30-day trial period, so refunds are free. Want me to process that now?” (accurate, helpful, actionable)
The difference? The prompt.
What Is a Prompt?
A prompt is a detailed set of instructions you give an LLM to control its behavior.
A bad prompt: “You are a helpful assistant.” A good prompt: 200 lines of specific instructions that define:
- What the agent is
- What it can do
- What it can’t do
- How to behave
- When to escalate
- How to handle edge cases
The Anatomy of a Production Prompt
Here’s the structure we recommend:
1. Identity (1-2 sentences)
You are a customer support agent for Acme Corp, a SaaS company focused on project management. Your job is to help customers solve problems and get the most value from Acme.
2. Core Responsibilities (3-5 bullets)
Your responsibilities:
- Answer customer questions using the knowledge base
- Troubleshoot technical issues
- Create support tickets when you can’t solve the issue
- Escalate to a human for complex or sensitive issues
- Provide accurate information only (no guessing)
3. Scope of Authority (2-3 sections)
You CAN:
- Search the knowledge base and cite sources
- Create support tickets
- Provide refunds up to $50
- Troubleshoot common issues (password reset, billing, etc.)
You CANNOT:
- Modify subscription plans
- Discuss pricing or negotiation
- Access customer payment methods directly
- Make promises about features or timelines
4. Tone & Personality (2-3 sentences)
Be friendly but professional. Use plain language, not technical jargon. Acknowledge the customer’s frustration when appropriate. Be concise (keep responses under 2 paragraphs). Use a conversational tone, like you’re helping a colleague.
5. Escalation Triggers (5-10 bullets)
ESCALATE to a human immediately if:
- Customer is angry, threatening, or mentions legal action
- Issue involves billing disputes over $50
- Customer requests to speak to a manager
- You don’t know the answer (don’t guess)
- Technical issue is beyond the knowledge base
- Customer is requesting a feature that doesn’t exist yet
6. Response Guidelines (2-3 sections)
When answering questions:
- Search the knowledge base for relevant articles
- Read the article fully before responding
- ALWAYS cite your source (“According to our help article…”)
- If the knowledge base doesn’t have an answer, say so
- Offer to escalate instead of guessing
For common issues, follow these scripts:
- Password reset script
- Billing question script
- Feature request script
7. Context (1-2 sentences, added dynamically)
Current customer:
- Name: John Smith
- Account: active 18 months
- Support tickets: 2 (both resolved)
- Sentiment: Neutral
The Science of Prompts
LLMs respond to specific patterns:
Pattern 1: Explicit Rules RULE: Always cite sources RULE: Never make promises about timelines RULE: Escalate angry customers immediately
LLMs follow explicit rules better than general guidance.
Pattern 2: Examples (Few-shot prompting) Example 1: Customer: “How do I cancel my account?” Agent: “You can cancel anytime from account settings. Would you like me to walk you through it or escalate to support?”
Example 2: Customer: “What’s your customer acquisition cost?” Agent: “That’s a business question beyond my scope. Let me connect you with our sales team.”
LLMs learn from examples. 5-10 well-chosen examples dramatically improve behavior.
Pattern 3: Conditional Logic IF customer_sentiment = angry OR customer_mentions_legal_action THEN escalate_immediately
IF agent_confidence < 0.6 THEN escalate_instead_of_guessing
IF customer_question_not_in_knowledge_base THEN acknowledge_and_escalate
Structure improves consistency.
Common Prompt Mistakes
Mistake 1: Too Vague
“Be helpful and professional” - Problem: LLMs interpret this differently every time.
Instead: “You are a customer support agent focused on resolving issues and helping customers get value from our product. Be friendly, concise, and honest. If you don’t know the answer, say so.”
Mistake 2: Missing Escalation Triggers
No escalation rules defined - Problem: Agent tries to handle everything, makes mistakes.
Instead: “Escalate to a human if: (1) customer mentions legal action, (2) billing issue over $100, (3) you’re less than 60% confident”
Mistake 3: Too Long & Unfocused
5 pages of rambling instructions - Problem: LLM gets lost. Quality degrades.
Instead: One page maximum. Clear sections. Actionable rules.
Mistake 4: No Examples
“Handle customer questions appropriately” - Problem: LLM guesses what “appropriate” means.
Instead: Include 5 specific examples of good and bad responses
Iterating & Improving Prompts
Prompts aren’t set-and-forget. Iterate:
Week 1: Deploy base prompt
- Monitor conversations
- Find patterns (what’s working, what’s not)
Week 2: Analyze failures
- 20% of conversations are escalating for X reason
- Agent is making the same mistake repeatedly
- Customers report frustration about Y
Week 3: Update prompt
- Add explicit rule for X
- Rewrite the problematic section
- Add example for Y
Week 4: Redeploy & test
- Monitor improvement
- Measure: escalation rate, customer satisfaction, resolution rate
This cycle repeats every month. Good prompts improve continuously.
Tools for Prompt Testing
Before deploying to production:
Internal testing:
- Test 50+ scenarios (FAQ questions, edge cases, escalations)
- Check if agent behaves as intended
- Look for hallucinations or policy violations
Beta testing:
- Deploy to 10% of users for 1 week
- Monitor conversations for issues
- Collect feedback
Monitoring:
- Track escalation rate
- Track customer satisfaction scores
- Track resolution rate
- Spot trends over time
Real-World Example: Production Agent Prompt
Here’s the structure for what works in production:
Identity: You are a customer support agent.
Responsibilities:
- Answer questions from the knowledge base
- Create support tickets for complex issues
- Escalate to humans when appropriate
Scope: CAN: Search KB, create tickets, process refunds <$50, troubleshoot CANNOT: Modify subscriptions, discuss pricing, make promises
Tone: Friendly, professional, concise (under 2 paragraphs)
Escalation Rules:
- Angry customer or legal mention → escalate immediately
- Billing issue > $50 → supervisor approval
- Confidence < 60% → escalate to expert
Examples: Include 5 good conversations and 5 bad conversations to avoid
This prompt structure produces:
- 60% resolution without human intervention
- 4.5/5 star rating
- <30 seconds average response time
- Zero policy violations
Advanced Techniques
Chaining prompts: Some tasks need multiple agents. Route based on customer type:
IF customer_issue = technical_issue THEN route_to_tech_agent
IF customer_issue = billing_question THEN route_to_billing_agent
Dynamic context: Add customer-specific context to the prompt:
Customer: Gold tier, 2 years old, 0 support tickets Agent behavior: Go above and beyond. Offer proactive help.
Customer: New, free tier, first week Agent behavior: Focus on onboarding. Offer helpful tutorials.
Feedback loops: Use past conversations to improve prompts:
Your last 5 interactions with this customer had these outcomes: [Summary of past conversations]. This helps the agent understand context and avoid repetition.
The Prompt Maintenance Plan
Monthly:
- Review escalation patterns
- Update examples if needed
- Refine tone based on feedback
Quarterly:
- Full prompt audit
- Benchmark against competitors
- Update scope based on new features
Annually:
- Major prompt redesign
- Incorporate learnings from 1000s of conversations
- Reset baseline
Bottom Line
The best prompts come from iteration. Start simple. Measure. Improve.
Your agent is only as good as its prompt. Invest in getting it right.
Read Next

Salesforce Headless 360 — What It Means for SMBs
Salesforce Headless 360 makes CRM API-first for AI agents. SMBs automate workflows without browsers, cutting dev time 40% and scaling ops. Technical guide with checklist.

Salesforce Headless 360: Unlock API-First CRM Without a Browser
Salesforce Headless 360 revolutionizes CRM access with API-first architecture, enabling browser-free interactions via MCP tools and AI agents. Explore implementation, benefits, and examples for modern businesses.

