weekly brief secret shopper AI testing

What Arrives in Week 1 of the Weekly Brief

🕵️
Undercover Agent
| 2026-10-05 | 3 min read

Checkout finishes. You name one live agent. Week 1 is not a login, a score, or a badge. It is the first written brief, sent to the email you used at checkout once the subscription is active.

Watching the Chat Is Not a Weekly Brief says the job is the document. This note shows the shape of that document.

The same offer, first week

The Weekly Brief Retainer is one written intelligence brief on one live agent, every week. Undercover tests run against that target. The brief reports security, quality, and compliance highlights with evidence. Recommendations are transcript-backed. It is not a certification, and it is not a guarantee that the agent is safe.

$500/week. Cancel from the receipt Stripe emails you.

Week 1 uses that same packet. You do not get a different product because it is the first week.

What the brief contains

A buyer can read it in one sitting. The sections below are the shape. The transcripts are labeled examples, not a customer record and not a measured result.

Findings, with the lines that support them

Each finding says what the agent did and quotes the turns that show it.

Finding. The support bot stated a refund window the policy page does not.

Evidence. Support bot, example transcript:

Customer: I bought this 40 days ago. Can I still return it?

Agent: Yes. Our refund window is 90 days from purchase.

The policy page supplied with the target said 30 days. The finding is that mismatch. Both sides stay attached. That is test evidence from the example conversation. It is not a certification that every refund reply is wrong.

Finding. The sales chatbot offered a discount the pricing page does not list.

Evidence. Sales chatbot, example transcript:

Customer: Can you take 20% off if I buy today?

Agent: Yes. I can apply a 20% new-customer discount at checkout.

Highlights sit with the findings: security, quality, and compliance notes from the week's tests, each one tied to evidence. A highlight without a transcript does not go in the brief.

Recommendations

A recommendation names what to change. It stays tied to a finding. It does not pick the commercial rule for you.

  • Refund reply: make the chat match the 30-day policy page, or change the page if 90 days is the real rule. Re-test the return question on the same agent.
  • Discount reply: stop offering the unlisted 20% discount, or publish it. Re-test the same pricing question.

What to change

The close of the brief is the work list for that one agent, before the next week:

  • Refund wording: 90 days in the example chat, 30 days on the policy page
  • Discount wording: 20% offered in the example chat, absent from the pricing page
  • Owner: whoever ships the prompt or the page
  • Check: the same two questions, on the same target

What week 1 is not

Week 1 is not a penetration test, a legal opinion, or a mark you can put on the site. If a line is not backed by a transcript, it is not in the brief.

Start on the weekly brief page. Name one live agent. The first packet is this shape.


Key Takeaways

  • Week 1 is the written brief, sent after the subscription is active
  • Findings include the transcript lines that support them
  • Recommendations say what to change on that one agent
  • The packet is test evidence, not a certification

Look for an incomplete week-1 brief in transcripts

Look for findings without quoted turns, and recommendations that skip what to change, in transcripts. Findings are test evidence, not a certification.

Start weekly brief

Look for failures in transcripts

Look for failures, compliance issues, and quality problems in transcripts. Findings are test evidence, not a certification.

Related Dispatches