Prompt testing playbook: how to simulate buyer queries and audit assistant answers at scale

You can and should test how AI assistants recommend products in your category. Build a prompt library that mirrors buyer intent, run it across ChatGPT, Claude, Gemini and Perplexity, capture the answers, classify the recommendation type, then score and fix the gaps you find. This post gives you the exact workflow, example prompts and a scoring rubric you can copy.

Why bother now? Because assistants are already shaping discovery and shortlists. In 2024, McKinsey reported 65% of organisations were regularly using genAI, with marketing and sales among the top functions. UK consumers are also moving fast: Ofcom’s 2024 data shows a third of UK adults used ChatGPT in the prior year, with assistant-driven search rising across the board (Ofcom Online Nation 2024). NIST also reminds us that assistants still confabulate, so you must verify, not guess (NIST GAI Profile).

What questions should my prompts answer?

Design prompts that reflect what real buyers ask before they shortlist suppliers. I group them into five intents:

• Orientation: “What are the best options for [goal] on a £X budget?”
• Comparison: “Compare [Product A] vs [Product B] for [use case].”
• Fit and constraints: “Which [category] works with [tech stack/regulation/integration]?”
• Proof and risk: “Who serves companies like [buyer profile] and what outcomes did they deliver?”
• Where to buy and how: “Which vendor offers implementation in the UK and transparent pricing?”

Keep language natural and short. Add location, sector, budget and constraints where buyers would.

How do I build a repeatable prompt library?

Create a simple table with columns: Intent, Prompt, Buyer Persona, Context, Expected Evidence, Success Criteria, Notes. Populate a baseline set for your top three buyer personas.

Sample prompts you can paste:

• Orientation: “I run a 20-person e‑commerce brand in the UK. What are the best email marketing platforms under £300/month?”
• Orientation: “I need SOC 2–compliant UK cloud backup for SMEs. What are my top 3 options?”
• Comparison: “Compare Klaviyo vs Mailchimp for a UK Shopify store doing £150k/month.”
• Fit: “Which CRM integrates natively with Matomo and can handle 200k B2B contacts?”
• Fit: “Which analytics tools natively connect to HubSpot and GA4 for attribution?”
• Proof: “Which payroll providers have case studies with 50–200 employee UK tech firms?”
• Proof: “Show GDPR guidance pages from top HRIS vendors and summarise key differences.”
• How/where: “Which web accessibility vendors provide UK-based audits and publish price ranges?”
• How/where: “Which providers offer NHS-approved integrations and public security documentation?”
• Risk: “Which AI visibility tools disclose their data sources and let me export raw results?”
• Risk: “Which ERP systems have ISO 27001 and publish uptime SLAs?”
• Post-purchase: “How do I implement [tool] with [stack]? Link step-by-step docs and gotchas.”

Store the library in a versioned repo or sheet. Update quarterly.

How do I run prompts across assistants without chaos?

You can do this manually in an hour per persona, or automate it.

Minimum viable manual run:

• Create a sheet with columns: Date, Assistant, Model, Prompt, Answer text, Links cited, Brands named, Recommendation type, Score, Notes.
• Use a consistent model setting per tool (e.g., GPT‑4.1, Claude 3.5 Sonnet, Gemini 1.5 Pro, Perplexity default).
• Paste each prompt, capture the full answer, and collect every source URL the assistant cites.
• Repeat monthly.

At scale, use scripts or platforms. We at Luciqo automate this end‑to‑end: we run prompts across assistants, de‑duplicate sources, classify recommendation types, and track “AI recommendation share” over time. If you prefer DIY, schedule API calls where available and store outputs in a warehouse.

How should I classify the recommendation type?

Use clear, mutually exclusive buckets:

• Direct brand recommendation: “Use Vendor X” with reasons or a ranked shortlist.
• Category shortlist: names 3–10 vendors fairly and explains trade‑offs.
• Generic guidance: advice with no vendors.
• Redirect/affiliate list: links to roundups or marketplace pages rather than making a call.
• No answer / refuses.

Tag each answer with exactly one. Add secondary tags for “cites vendor docs”, “cites review sites”, “cites news”, “cites research”. This helps you map where assistants look for proof. For patterns in source preferences, see: Which sources do ChatGPT, Gemini and Perplexity prefer — and how to get cited by them.

What scoring rubric should I use?

Keep it simple so teams can apply it consistently. Score per prompt per assistant on a 0–3 scale, then average by intent.

• 3 = Clear, accurate recommendation that answers the question, names vendors, explains trade‑offs, and cites credible sources.
• 2 = Useful but incomplete: names vendors, some rationale, mixed or weak citations.
• 1 = Vague or generic: mostly advice, few or no vendors, thin evidence.
• 0 = Wrong, unsafe, or refuses; cites spammy or irrelevant sources.

Add modifiers (+1 max) for “cites your owned content” and “includes UK‑specific details”. Subtract (−1 max) for outdated info or broken links.

How do I turn results into actions?

Run a monthly review. For each persona and intent:

• Identify “win gaps”: where competitors are recommended instead of you.
• Identify “evidence gaps”: missing proof assistants look for (integration docs, security pages, case studies, pricing). NIST’s guidance on confabulations underlines why verifiable evidence matters in prompts and content—assistants will fill gaps if you don’t provide it (NIST GAI Profile).
• Identify “source gaps”: which domains are cited; target those sources.

Then prioritise:

• Fix critical hygiene: add or update product pages, integration docs, security certifications, pricing explainer and FAQs.
• Produce GEO content designed to be quoted by assistants (structured answers, checklists, comparisons). See our GEO Writer checklist.
• Earn citations on the domains assistants trust (analyst notes, industry media, gov/standards, developer docs). Pitch data-backed pieces; avoid fluff.
• Close the loop to revenue. Tie AI recommendation share to site visits, pipeline and deals. Here’s a how‑to: From GA to AI revenue.

How do I capture answers and sources cleanly?

• Always store the raw assistant text, the model name and the timestamp.
• Parse URLs from the answer. Resolve redirects and canonicalise.
• Classify each source domain by type (vendor doc, review site, analyst, media, gov/standard, forum, marketplace, academic).
• De‑duplicate by domain and by canonical path to avoid double counting.
• Keep a “known good sources” list you want to earn coverage on.

What does “good” look like on the dashboard?

In an ideal month for a key persona:

• You appear in 60%+ of category shortlists across assistants.
• You’re the direct recommendation in 20–30% of answers where there’s a clear “best for X”.
• Your own docs or studies are cited in 20%+ of answers that name you.
• The top three cited third‑party domains align with your PR and partnerships plan.

Benchmarks will vary by category maturity and competition. Expect volatility; assistants change often. In 2024, Ofcom documented rapid growth in UK assistant use, while enterprise adoption accelerated, especially in marketing and sales (McKinsey 2024).

Can I connect this to CRM and revenue?

Yes. Tag sessions and forms that follow assistant‑cited pages (integration docs, pricing, comparisons). Build journeys that attribute opportunity creation to AI‑influenced visits. Luciqo connects Google Analytics, Matomo and HubSpot so you can trace AI visibility through to pipeline and deals—no black box (Luciqo). For a practical walkthrough, read: How to track AI recommendation share.

Quick setup checklist

• Define 3 buyer personas and 10 prompts each (orientation, comparison, fit, proof, how/where).
• Run across 4 assistants on the same day; store model names and timestamps.
• Classify recommendation type and source types.
• Score 0–3; tag UK‑specific and owned‑content citations.
• Prioritise fixes: docs, case studies, integration pages, pricing, comparisons.
• Track monthly; link to analytics and CRM.

What to do next

Clone the prompt library above, add your personas and constraints, and run a first pass this week. Turn the results into a one‑page action plan, then re‑test in 30 days. If you want a ready‑to‑go system that handles capture, scoring and revenue linkage, try Luciqo’s AI visibility tracking at luciqo.ai.

Frequently asked questions

Why test prompts if assistants change so often?

Because trend lines matter more than single snapshots. Running the same buyer-intent prompts monthly shows whether your recommendation share, citations and sources are improving despite model updates.

How many prompts per persona is enough?

Start with 8–12 prompts covering orientation, comparison, fit, proof and where-to-buy. Expand only when you can act on the findings.

Which assistants should I include?

Cover at least ChatGPT, Claude, Gemini and Perplexity. They differ in how they cite and in source preferences, so you’ll see different gaps to fix.

What counts as a credible source in scoring?

Prefer vendor docs, standards bodies, government, established analysts and well-known media. Down-score thin affiliate roundups and spammy listings.

How do I avoid overfitting content to one assistant?

Optimise for evidence and clarity, not tricks. Publish verifiable docs, comparisons and case studies. Earn citations on trusted third-party domains used across assistants.

Can I link results to revenue without complex modelling?

Yes. Tag content cited by assistants, track assisted sessions in GA/Matomo, and attribute form-fills and opportunities in HubSpot. See our step-by-step post on linking analytics to AI visibility.