SECOM Signal Triage

Of 590 signals, 464 are testable. The rest never vary, or are too sparsely observed in one outcome group to compare. Testing each against outcome at the usual 5% threshold flags 80. But 464 tests at that threshold are expected to flag around 23 by chance alone, with nothing real present. Each bar below removes a different way the count overstates itself: multiplicity, then redundancy between near-identical channels, then effects that do not reproduce across the record. The gap between 80 and 9 is the finding, not the 9.

Figure 1. Signal counts at five successive filters, from all sensors tested down to those that are significant, independent of near-duplicates, and stable across both halves of the record.

The 33 signals surviving correction, ordered by effect size. Colour marks whether the effect reproduces when the record is split in half. Ten do. Twenty-one appear only when the halves are pooled, and two cannot be assessed. Two patterns are worth noting: the six largest effects are all stable ones, and almost every asterisked signal, meaning one with a near-duplicate in the set, falls in the unstable group. The three filters are not independent objections. They agree about which signals to trust.

Figure 2. Cohen's d for each sensor passing Benjamini–Hochberg at FDR 0.10, coloured by whether the effect holds across both time halves. An asterisk marks a sensor sharing a redundancy group with at least one other.

Every signal lands in exactly one bucket. Two of these are recommendations to monitor less, and both rest only on structural facts: a signal taking one value across all 1,567 runs, or recorded on under 10% of them. Neither is built from a signal failing a significance test. That restraint is deliberate. A false entry on the shortlist costs an engineer an hour. A signal wrongly cut from monitoring stops producing the data that would reveal the mistake. The errors are not symmetric, so the irreversible one is never made on inferential grounds.

Figure 3. Every sensor assigned to one bucket. The two-sensor shortlist_low_coverage segment is drawn to scale, and is nearly invisible by design.

The failure rate is not stationary. It runs between 13% and 23% through mid-August, then between 1% and 5% from September. Because failures concentrate early, a signal that merely drifted over these 90 days will separate failing from passing runs without having anything to do with failure. That is why every effect is rechecked within each half of the record. The decline is also consistent with someone having found and fixed a process problem, in which case some early-only signals were real and are now historical.

Figure 4. Weekly failure rate over the 14-week window, against the overall rate. The shaded band is the early half of the median split used in the stability check.

Effect size is not separation. These are the strongest signals in the set, standardised to share an axis, and the passing and failing distributions still overlap heavily. A Cohen's d of 0.6 is a shift of half a standard deviation. Enough to rank a signal as worth checking first, nowhere near enough to classify a run from it. This analysis produces an ordering for human attention, not a detector.

Figure 5. Per-sensor pass and fail distributions for up to eight shortlist sensors with the largest effect sizes. Values are standardised within each sensor (z-score) so sensors on different scales can share an axis; this is a display transform only.