What Three Months of AI Email Triage Taught Us

The Experiment

In November, I was getting about 140 emails a day. Not all of them needed responses, but each one needed a decision: respond now, respond later, delegate, archive, or ignore. That decision-making alone was eating 90 minutes of my morning.

So I set up an experiment. For three months, I'd let Billix's AI handle my initial email triage. It would categorize incoming mail, flag urgent items, draft responses for routine messages, and surface anything it wasn't sure about. I'd still have the final say on everything—but the AI would do the sorting and first-draft work.

Here's what actually happened.

Week One Surprises

The AI was immediately good at categorization. Marketing emails, automated notifications, meeting invitations—it sorted those with near-perfect accuracy from day one. The categories it struggled with were internal emails where context mattered. A message from a colleague saying "Can you look at this?" could mean anything depending on the sender and what "this" referred to.

The bigger surprise was the drafts. I expected them to be unusable—generic, overly formal, missing nuance. About 40% of them were actually sendable with minor edits. Another 30% needed significant revision but saved me from staring at a blank compose window. The remaining 30% were better off rewritten from scratch.

Here's how draft quality broke down in that first week:

• ~40% — Sendable with minor edits (a word here, a comma there)
• ~30% — Needed real revision, but the structure was right
• ~30% — Scrap and rewrite (tone-deaf, missed context, wrong approach)

Not bad for week one. Not great either.

The 80/20 Rule Applied

By week three, a pattern emerged. The AI excelled at high-volume, low-complexity emails: scheduling confirmations, status update requests, simple questions with factual answers. These made up about 60% of my inbox. For these, I was essentially just reviewing and hitting send.

It struggled with emails that required political awareness, relationship context, or creative thinking. A sensitive reply to a frustrated customer? The AI wrote something technically correct but emotionally tone-deaf. A pitch to a potential partner? Too formulaic. These still needed the human touch, and I stopped trying to make the AI handle them.

The sweet spot was accepting that AI email triage isn't about replacing you. It's about handling the bottom 60% of your inbox so you have energy for the top 40%.

False Positives and Trust

The hardest part wasn't the technology. It was trust. In week two, the AI filed a time-sensitive email from our CEO under "low priority" because it didn't contain any urgent keywords. It was a casual "thoughts on this?" about a strategy decision. The AI saw casual language and deprioritized it. I saw it three hours late.

After that, I set up explicit rules: emails from certain senders always get flagged as high priority, regardless of content. This hybrid approach—AI categorization with human-defined overrides—worked much better than either pure AI or pure rules.

Results After 90 Days

By the end of the experiment, my email processing time dropped from 90 minutes to about 35 minutes per day. That's almost an hour back, every single day.

Here's the breakdown of what the AI handled independently versus what I still touched:

• Fully automated (archive, no response needed): 45% of emails
• AI-drafted, I approved: 25% of emails
• AI-drafted, I heavily edited: 15% of emails
• I wrote from scratch: 15% of emails

The accuracy improved over time as the system learned from my corrections. By month three, the false positive rate on priority classification was under 3%.

Would I go back? Absolutely not. But I'd warn anyone trying this: start with low-stakes emails and build trust gradually. Don't hand over your entire inbox on day one.