18 Feb 2026 · Android
Reading guardrail metrics on Android without drowning in ANR noise
Android stability graphs twitch. A new OEM skin, a Play services update, or a single viral device model can move ANR rates without your variant doing anything. If every twitch becomes a veto, you will never ship. If you ignore them, you will ship a paywall that janks on mid-range devices your conversion chart never showed.
Pick one stability veto, not five
In Power, Peeking, and Guardrails we ask for a single stability veto agreed before assignment starts — usually crash-free sessions on the binary that contains the experiment, sliced to the same assignment unit. ANR can sit as a watch metric. Watch metrics inform; they do not automatically stop the test.
Match the window to the binary
Comparing this week’s Play Console ANR to last month’s is not a guardrail. Compare variant versus control on the same binary, same window, same assignment. If you cannot get that slice out of your pipeline, you do not have a guardrail; you have a vibe.
Low-end devices are not an edge case here
Thailand’s Android mix is not a flagship-only story. A variant that is fine on a Pixel and heavy on a three-year-old Samsung will look “fine” in a blended crash-free number. Heterogeneity on device class belongs in the readout, even if it is not the primary metric.