AI Reply Suggestions & Confidence Scoring

Written By Chad McGuire (Sparrow Intel)

Overview

For most teams, this is the lesson that changes the math. Once you trust Chirp AI to write the first draft of every reply, the time it takes to clear an inbox drops dramatically.

This lesson is about how AI suggestions work, what the confidence assessment is telling you, and when to send a draft as-is, edit it, or override.

How a suggestion gets generated

When a guest message arrives, Chirp AI:

  1. Reads the new post and the conversation history
  2. Looks up the reservation, property, and any relevant knowledge snippets
  3. Detects the guest's language
  4. Drafts a reply in that language, in your team's tone
  5. Scores its own draft for confidence and risk
  6. Surfaces the draft in the conversation as a suggested reply

This happens automatically on incoming posts. You don't have to ask for it.

You can also generate a fresh suggestion on demand from the composer if you want a different angle.


Where to find the suggestion

Open any conversation that's awaiting a reply. The suggested reply appears just above the composer. You'll see:

  • The draft message text
  • A confidence score (0–100)
  • A risk level (Low / Medium / High)
  • A row of quality signals (more on these below)

You can:

  • Send as-is β€” click Send and it goes
  • Edit then send β€” click into the draft, modify it, then Send
  • Decline β€” dismiss the suggestion and write your own
  • Generate again β€” ask Chirp for a different draft

Reading the confidence assessment

The confidence assessment is Chirp's self-evaluation of its own draft. It's the most important thing to look at before sending an AI-generated reply.

Confidence score (0–100)

Higher means Chirp is more sure the draft is appropriate. As a rough guide:

85–100Routine inquiry, well-grounded in your data, low chance of mistakes β€” typically safe to send as-is
70–84Reasonable draft but worth a quick read
Below 70Edit before sending; something about the conversation is making Chirp uncertain

These bands are guidelines. The score interacts with risk and quality signals β€” don't treat any single number as the whole story.

Risk level

A separate axis from confidence. Risk reflects the consequence of getting it wrong, not the likelihood:

  • Low β€” routine question, factual answer, hard to make worse
  • Medium β€” answer involves policy, money, or expectations the guest will hold you to
  • High β€” complaint, refund discussion, emergency, or anything legally sensitive

A high-confidence reply to a high-risk message still deserves a human glance.

Quality signals

Five binary checks that tell you what specifically Chirp evaluated:

Addresses guest requestThe draft actually answers what was asked
Tone appropriateThe draft matches a hospitality voice β€” not robotic, not flippant
Factually groundedSpecific facts in the draft come from the reservation, property, or your knowledge snippets β€” not invented
Complete responseThe draft doesn't leave loose ends
Language matchedThe draft is in the same language as the guest's message

If any of these fail, the icon shows red and you should look at the draft critically before sending.


A workflow that scales

The pattern that works for most teams once they're comfortable:

  1. Open the next unassigned conversation
  2. Glance at the confidence and quality signals
  3. High confidence + low risk + all green signals β†’ read the draft once, send
  4. Anything else β†’ read more carefully, edit if needed, send
  5. Move to the next conversation

With a few weeks of tuning your knowledge snippets and templates (covered in later lessons), most routine inquiries will land in bucket 3, which means you can blow through them.


When you disagree with the AI

Editing a draft, declining it, or overriding tone is fine and expected. Chirp AI gets better the more your team uses it β€” your edits and patterns inform future drafts.

A few common cases for editing:

  • The draft is too formal/casual for your brand β€” edit the tone, or refine your team's tone preferences in settings
  • The draft missed a recent context change β€” check if the relevant info exists in a knowledge snippet or property notes; if not, add it
  • The draft is technically correct but you want to add a personal touch β€” go for it

If you find yourself rewriting most drafts on a particular topic, that's a signal to add a knowledge snippet or refine your templates.


Rating a suggestion

After you send a reply β€” whether you sent the AI draft as-is, edited it, or wrote your own β€” you can rate the AI's suggestion using the thumbs-up / thumbs-down controls that appear on the generated draft.

  • Thumbs up β€” the draft was a good response: accurate, on-tone, and worth sending
  • Thumbs down β€” the draft had a problem: wrong facts, wrong tone, missed the point, or something you had to significantly rework

Ratings are the fastest signal you can give Chirp AI. They're especially valuable when a draft looked confident but was off β€” that's the failure mode the system most needs to learn from.


What about Autopilot?

Autopilot is the next level: instead of suggesting and waiting for you, Chirp AI sends the reply automatically when the confidence and risk thresholds are met.

You don't have to enable Autopilot to use AI suggestions. Many teams use suggestions for months before they turn on Autopilot, and that's a perfectly fine place to live.

We cover Autopilot end-to-end in Autopilot: Auto-Reply Safely at Scale.


Up next

Message Templates β€” how to combine AI suggestions with your team's prewritten copy for the messages you send all the time.