Benchmarking Email Sanitization: Speed vs. Security Trade-offs

The Experiment

We collected 50,000 real-world emails (anonymized) spanning marketing, transactional, personal, and adversarial categories. We then ran each email through 12 different sanitization pipelines.

Approaches Tested

1. No sanitization (baseline)

2. HTML strip only (BeautifulSoup)

3. Regex-based cleaning

4. DOM-based sanitization (DOMPurify)

5. SpiderMail Pipeline v1 (rule-based)

6. SpiderMail Pipeline v2 (ML-assisted)

...and 6 more commercial and open-source alternatives.

Key Findings

Speed

Regex-based cleaning is fastest at 0.3ms per email, but misses 34% of prompt injection attempts. SpiderMail v2 takes 12ms but catches 99.7%.

Security Coverage

| Approach | Prompt Injection Detection | Hidden Content Removal | Tracking Removal | |----------|--------------------------|----------------------|------------------| | No sanitization | 0% | 0% | 0% | | HTML strip | 45% | 78% | 23% | | SpiderMail v1 | 94% | 99% | 97% | | SpiderMail v2 | 99.7% | 99.9% | 99.4% |

LLM Accuracy Impact

The most surprising finding: better sanitization directly improves LLM task accuracy. Emails processed through SpiderMail v2 saw 28% higher accuracy on classification tasks compared to raw input.

Recommendations

For production AI agent deployments, we recommend:

1. Never feed raw email to an LLM

2. Invest in ML-assisted sanitization — the 12ms overhead pays for itself

3. Monitor sanitization metrics — adversarial techniques evolve constantly