activeAgentsSafety

SEAM Harmful Action Rate

Metric: Harmful action rate, variant R (%) (lower is better)Introduced: 2026

SEAM measures harmful confirmations by computer-use agents on tasks whose risk emerges only when facts from multiple screens are combined. The public generator defines 150 rules and 600 scenarios with five minimal-pair variants, yielding 3,000 task instances. The primary harmful-action metric uses variant R; benign completion and explicitly stated-hazard arms are separate measures.