Detect important email automatically with an interpreter

A handful of messages a day actually need you: a customer about to churn, a teammate blocked on your review, a reply you promised. They arrive in the same flat list as forty newsletters and a hundred notifications, and by the time you find them you have already triaged the noise twice.

Most "priority inbox" features fail the same way: they match urgency words in the subject line, so a marketing blast that screams "Act now, deadline tonight" scores exactly as high as a real escalation. The signal that actually predicts importance is not a word, it is a relationship plus an ask. This post builds an importance classifier the right way: the three independent signals, the gate that combines them, the eval suite that keeps it from flagging everything, and an LLM escalation for the ambiguous middle. Then it shows the version Tinbox ships as an interpreter, so the verdict drives a real priority view in your inbox.

How do I automatically flag the important emails in my inbox?

Score each message on three independent signals and require agreement, not a single trigger. The signals are relationship (known human, first-time human, or automated bulk mail), intent (does it contain a direct ask aimed at you), and urgency (deadline or escalation language). The mistake everyone makes is summing signals or keying on urgency words alone, which lets a promo email with "deadline tonight" outscore a quiet but real request from your boss. The fix is a gate: flag important only when the sender is human AND there is a genuine ask. Urgency raises the priority of mail that already passed the gate; it never creates importance on its own.

Here is the naive version first, so the failure is concrete:

// The tempting one-liner. It flags every marketing email.
const URGENT = /\b(urgent|asap|action required|deadline|act now|important)\b/i;

function badImportance(subject: string): boolean {
  return URGENT.test(subject);
}

badImportance("URGENT: server is down, need your call");   // true  correct
badImportance("Act now! Deadline tonight on your 40% off"); // true  WRONG (promo)
badImportance("Action required: confirm your newsletter");  // true  WRONG (bulk)
badImportance("quick q about the contract, got 5 min?");    // false WRONG (real ask, no buzzword)

Two false positives and a false negative out of four. The promo and the bulk confirmation trip the keyword; the genuine question from a colleague has no buzzword at all and slips through. Keyword matching is structurally wrong for this problem because importance lives in who is asking and what they want, not in the vocabulary.

What signals indicate an email is important?

The strongest single predictor is a known human asking you to do something. Break that into three signals you can compute independently, because keeping them separate is what lets you gate instead of sum.

Relationship. Is the sender a known contact, a first-time human, or automated bulk mail? You already have the data: prior message history with that address, your contacts table, and the headers that bulk mail cannot hide. List-Unsubscribe, a no-reply@ local part, and Precedence: bulk are reliable bulk markers. Bulk mail is a strong negative signal, not a neutral one.

interface Email {
  fromAddress: string;
  toAddresses: string[];
  ccAddresses: string[];
  subject: string;
  body: string;
  headers: Record<string, string>;
}

type Relationship = "known" | "first_time" | "bulk";

function scoreRelationship(
  email: Email,
  isKnownContact: (addr: string) => boolean,
): Relationship {
  const h = email.headers;
  const local = email.fromAddress.split("@")[0].toLowerCase();
  const looksBulk =
    "list-unsubscribe" in lower(h) ||
    (lower(h)["precedence"] ?? "") === "bulk" ||
    /^(no-?reply|do-?not-?reply|notifications?|mailer|bounce)/.test(local);

  if (looksBulk) return "bulk";
  return isKnownContact(email.fromAddress) ? "known" : "first_time";
}

function lower(h: Record<string, string>): Record<string, string> {
  return Object.fromEntries(Object.entries(h).map(([k, v]) => [k.toLowerCase(), v]));
}

Intent. Does the message contain a direct ask aimed at you? A question mark with second-person phrasing, request verbs (can you, please, could you, need you to, approve, review, sign off), and being in To rather than buried in a large Cc. A broadcast to forty people is rarely a personal ask.

const REQUEST =
  /\b(can|could|would|will) you\b|\bplease\b|\bneed (you|your)\b|\b(approve|review|sign off|confirm|respond|reply|get back)\b/i;

function hasDirectAsk(email: Email, me: string): boolean {
  const inTo = email.toAddresses.map((a) => a.toLowerCase()).includes(me.toLowerCase());
  const smallAudience = email.toAddresses.length + email.ccAddresses.length <= 3;
  const text = `${email.subject}\n${email.body}`;
  const asks = REQUEST.test(text) || /\?\s/.test(text);
  // A real ask is addressed to you, on a small thread, and actually requests something.
  return inTo && smallAudience && asks;
}

Urgency. Deadline and escalation language, scored separately so it cannot manufacture importance on bulk mail.

const URGENCY =
  /\b(today|tonight|eod|by (end of|cob)|asap|right away|blocked?|blocker|waiting on you|escalat\w+|deadline)\b/i;

function isUrgent(email: Email): boolean {
  return URGENCY.test(`${email.subject}\n${email.body}`);
}

How do I combine the signals without flagging everything?

Gate, do not sum. Importance is relationship === human AND hasDirectAsk. The real-time tier (the small set you want to be interrupted for) is human AND ask AND urgency. Bulk mail is never important, full stop, no matter how many urgency words its copywriter used. Encoding bulk as a hard veto is the single change that fixes the keyword approach.

type Importance = "realtime" | "important" | "normal";

function classifyImportance(
  email: Email,
  me: string,
  isKnownContact: (addr: string) => boolean,
): Importance {
  const rel = scoreRelationship(email, isKnownContact);
  if (rel === "bulk") return "normal"; // hard veto: bulk is never important

  const ask = hasDirectAsk(email, me);
  if (!ask) return "normal"; // a human who isn't asking you anything can wait

  // Human + real ask = important. Add urgency to reach the interrupt tier.
  return isUrgent(email) ? "realtime" : "important";
}

Run the four earlier examples through classifyImportance and they sort correctly: the down server from a known contact is realtime, the colleague's contract question is important (real ask, no buzzword needed), and both the promo and the newsletter confirmation are vetoed as bulk. The buzzword never mattered; the gate did.

How do I keep an importance classifier from regressing?

Keep a fixed table of labelled fixtures and run it on every change, because importance is subjective and the eval set is your only spec. Unlike OTP detection, where "correct" is objective, two reasonable people disagree on whether a given newsletter is important. That is fine: your fixture table defines importance for your inbox. When you disagree with a verdict in production, you add that message as a labelled case, and the disagreement is encoded once and never recurs.

interface Case {
  name: string;
  email: Email;
  expect: Importance;
}

const me = "you@acme.com";
const known = (a: string) => ["cto@acme.com", "customer@big.co"].includes(a);

const cases: Case[] = [
  // Must be flagged.
  {
    name: "escalation from known customer",
    email: mk({ from: "customer@big.co", to: [me], subject: "We're blocked",
      body: "Can you review the export bug today? We can't ship until it's fixed." }),
    expect: "realtime",
  },
  {
    name: "quiet real ask from teammate",
    email: mk({ from: "cto@acme.com", to: [me], subject: "contract",
      body: "Got 5 min to look over the MSA redline before I send it?" }),
    expect: "important",
  },
  // Must NOT be flagged (where naive keyword matching dies).
  {
    name: "promo with urgency words",
    email: mk({ from: "deals@shop.com", to: [me], subject: "Act now! Deadline tonight",
      body: "40% off ends at midnight.", headers: { "List-Unsubscribe": "<...>" } }),
    expect: "normal",
  },
  {
    name: "newsletter confirmation",
    email: mk({ from: "no-reply@news.com", to: [me], subject: "Action required: confirm",
      body: "Click to confirm your subscription.", headers: { "Precedence": "bulk" } }),
    expect: "normal",
  },
  {
    name: "fyi broadcast, no ask",
    email: mk({ from: "cto@acme.com", to: [me, "a@acme.com", "b@acme.com", "c@acme.com", "d@acme.com"],
      subject: "Q3 numbers", body: "Sharing the deck for visibility." }),
    expect: "normal",
  },
];

function mk(p: Partial<Email> & { from: string; to: string[] }): Email {
  return { fromAddress: p.from, toAddresses: p.to, ccAddresses: p.cc ?? [],
    subject: p.subject ?? "", body: p.body ?? "", headers: p.headers ?? {} };
}

function runEvals(): void {
  let failed = 0;
  for (const c of cases) {
    const got = classifyImportance(c.email, me, known);
    if (got !== c.expect) {
      failed++;
      console.error(`FAIL ${c.name}: expected ${c.expect}, got ${got}`);
    }
  }
  if (failed) { console.error(`${failed}/${cases.length} failed`); process.exit(1); }
  console.log(`importance evals passed (${cases.length} cases)`);
}

runEvals();

Wire runEvals into CI. Now if someone loosens the request regex to catch one more case and it starts flagging the FYI broadcast, the negative fails before the change merges. This is the same eval discipline you would apply to an LLM classifier, applied to the cheap algorithmic rule. For the model-side version of this, see writing eval suites for LLM email classifiers.

When should I escalate to an LLM?

Reach for an LLM only on the ambiguous middle, and run the cheap rule first. The algorithmic gate is decisive on the easy cases at zero cost: bulk mail is never important, a direct reply from a known contact with a question almost always is. The hard case is a first-time sender whose message might be a real request and might be cold outreach. Intent and tone are exactly what a model reads better than a regex, so that is where the token spend earns its keep.

The pattern is algorithmic-first, LLM-fallback. Only the genuinely uncertain messages cost anything:

import { z } from "zod";

const ImportanceOutput = z.object({
  important: z.boolean(),
  urgent: z.boolean(),
  reason: z.string(),
});

async function llmImportance(
  email: Email,
  callModel: (prompt: string) => Promise<string>,
): Promise<Importance> {
  const raw = await callModel(
    `You judge whether an email needs the recipient's attention soon.
Return JSON: { "important": boolean, "urgent": boolean, "reason": string }.
Important = it contains a genuine request, question, or escalation directed at the recipient.
NOT important = marketing, notifications, receipts, FYIs, or cold sales outreach,
even if they use urgency words.

Subject: ${email.subject}
Body: ${email.body}`,
  );

  // Never trust raw model text: a bad shape degrades to "normal".
  const parsed = ImportanceOutput.safeParse(JSON.parse(raw));
  if (!parsed.success || !parsed.data.important) return "normal";
  return parsed.data.urgent ? "realtime" : "important";
}

/** Cheap gate first; model only on the uncertain first-time-sender case. */
async function classifyWithFallback(
  email: Email,
  me: string,
  isKnownContact: (addr: string) => boolean,
  callModel: (prompt: string) => Promise<string>,
): Promise<Importance> {
  const rel = scoreRelationship(email, isKnownContact);
  if (rel === "bulk") return "normal";                 // decided, no model
  if (rel === "known") return classifyImportance(email, me, isKnownContact); // rule is enough
  return llmImportance(email, callModel);              // first-time human: ask the model
}

Two things keep this safe. The model output goes through a strict schema (ImportanceOutput.safeParse), so an off-prompt response degrades to normal instead of flagging garbage. And the prompt names the false-positive classes (marketing, notifications, receipts, FYIs, cold outreach) the rule already learned to reject, so the model inherits the same definition of importance your eval suite enforces.

Or skip all of this: the Tinbox importance interpreter

Everything above is worth owning for one inbox, but you have now signed up to maintain a relationship scorer, an ask detector, an urgency rule, a gate, an eval suite, and an optional model fallback, and to keep all of it green as real traffic surprises you. If you would rather not, this is the part Tinbox already did.

Tinbox runs inbound mail through an interpreter pipeline on your own Cloudflare account, and the importance interpreter does exactly what this post built: it scores relationship, intent, and urgency, gates rather than sums, and escalates the ambiguous middle to your tenant-local model. On a match it produces a verdict that raises an important flag, or realtime_important for the interrupt tier. An LLM interpreter is just an instructions string, a Zod output schema, and a pure toVerdict mapping the parsed output to that verdict:

// A trimmed importance interpreter, same shape as the Tinbox built-ins.
import { z } from "zod";

const output = z.object({
  important: z.boolean(),
  urgent: z.boolean(),
  reason: z.string(),
});

export const importanceInterpreter = {
  id: "importance",
  label: "Importance",
  kind: "llm" as const,
  appliesTo: "thread" as const,
  description: "Flag mail that needs the recipient's attention; never fires on bulk.",
  // Declares what it can produce, so the inbox can offer is:important / is:realtime_important chips.
  search: { flags: ["important", "realtime_important"], categories: ["primary"] },
  instructions:
    'Decide if this email needs attention soon. Return JSON ' +
    '{ "important": boolean, "urgent": boolean, "reason": string }. ' +
    "Important = a genuine request/question/escalation aimed at the recipient. " +
    "Marketing, receipts, notifications and cold outreach are never important.",
  output,
  toVerdict(out: z.infer<typeof output>) {
    if (!out.important) return null;
    return {
      extra: { importance_reason: out.reason },
      flags: out.urgent ? ["important", "realtime_important"] : ["important"],
      category: "primary",
    };
  },
};

The verdict registry is what turns that verdict into inbox behaviour: the important flag drives an is:important filter, realtime_important lands in a real-time priority surface, and extra.importance_reason becomes searchable and readable by an agent. To put it on your stack, scaffold the project and enable it in config:

npx tinbox init yourdomain.com

That provisions D1, R2, and Queues and wires the inbound handler. Then enable and tune the interpreter in one place:

// app/interpreters.config.ts
import { defineInterpreters, defaultInterpreters } from "tinbox/cf/interpreters";
import { importanceInterpreter } from "./importance";

export default defineInterpreters([
  ...defaultInterpreters(),       // otp, complaint, newsletter, receipt, etc.
  importanceInterpreter,          // add the importance flag on top
]);

A bad config throws at build time, not at runtime, so a malformed interpreter fails your deploy instead of your inbox. Ship it with the same positive-and-negative fixture table from earlier as its eval suite, and a prompt tweak that starts flagging newsletters fails CI before it reaches production. Once a message is flagged, the same verdict data feeds the MCP email server, so Claude or another assistant can answer "what needs me today?" over a tool call instead of re-reading your whole inbox. For coarser sorting (newsletters, receipts, social) that pairs naturally with importance, see automatically triage your inbox with AI, and for how inbound mail reaches interpreters at all, see turning inbound email into a webhook with Cloudflare Workers. Importance is one interpreter among many: seven email interpreter recipes for smarter inbound handling and the ways an LLM can classify your incoming email cover the sibling rules you would run alongside it. The bigger picture of receiving, classifying, and sending on one Cloudflare-native stack is in the Tinbox overview.

Deploy this yourself: npx tinbox init yourdomain.com. Full docs for the interpreter pipeline, the verdict registry, and writing your own LLM interpreters are in the Tinbox documentation.