1,280
Engineer-hours per year, checking 80 signals per excursion
Which of 590 process sensors actually separate failing runs from passing ones?
A semiconductor plant monitors 590 process signals. When a lot fails, an engineer has to decide where to look. A conventional screen says 80 of those signals separate failing runs from passing ones. After correcting for the number of comparisons, grouping near-duplicates, and checking that each effect survives the process changing mid-record, nine do.
Of 590 signals, 464 are testable. The rest never vary, or are too sparsely observed in one outcome group to compare. Testing each against outcome at the usual 5% threshold flags 80. But 464 tests at that threshold are expected to flag around 23 by chance alone, with nothing real present. Each bar below removes a different way the count overstates itself: multiplicity, then redundancy between near-identical channels, then effects that do not reproduce across the record. The gap between 80 and 9 is the finding, not the 9.
Figure 1. Signal counts at five successive filters, from all sensors tested down to those that are significant, independent of near-duplicates, and stable across both halves of the record.
This dataset carries no cost information, so no saving is claimed here. The two counts above are ours; the rates below are yours. Set them to your fab and the arithmetic follows.
Read this as a scoping tool, not a business case. It assumes review time scales with the number of signals checked, that every signal on the list is actually reviewed, and that the nine-signal list finds root cause as often as the eighty-signal one. The third assumption is the load-bearing one and this dataset cannot test it. A fab holding both lists against its own excursion records for one quarter could.
The 33 signals surviving correction, ordered by effect size. Colour marks whether the effect reproduces when the record is split in half. Ten do. Twenty-one appear only when the halves are pooled, and two cannot be assessed. Two patterns are worth noting: the six largest effects are all stable ones, and almost every asterisked signal, meaning one with a near-duplicate in the set, falls in the unstable group. The three filters are not independent objections. They agree about which signals to trust.
Figure 2. Cohen's d for each sensor passing Benjamini–Hochberg at FDR 0.10, coloured by whether the effect holds across both time halves. An asterisk marks a sensor sharing a redundancy group with at least one other.
Every signal lands in exactly one bucket. Two of these are recommendations to monitor less, and both rest only on structural facts: a signal taking one value across all 1,567 runs, or recorded on under 10% of them. Neither is built from a signal failing a significance test. That restraint is deliberate. A false entry on the shortlist costs an engineer an hour. A signal wrongly cut from monitoring stops producing the data that would reveal the mistake. The errors are not symmetric, so the irreversible one is never made on inferential grounds.
Figure 3. Every sensor assigned to one bucket. The two-sensor shortlist_low_coverage segment is drawn to scale, and is nearly invisible by design.
The failure rate is not stationary. It runs between 13% and 23% through mid-August, then between 1% and 5% from September. Because failures concentrate early, a signal that merely drifted over these 90 days will separate failing from passing runs without having anything to do with failure. That is why every effect is rechecked within each half of the record. The decline is also consistent with someone having found and fixed a process problem, in which case some early-only signals were real and are now historical.
Figure 4. Weekly failure rate over the 14-week window, against the overall rate. The shaded band is the early half of the median split used in the stability check.
Effect size is not separation. These are the strongest signals in the set, standardised to share an axis, and the passing and failing distributions still overlap heavily. A Cohen's d of 0.6 is a shift of half a standard deviation. Enough to rank a signal as worth checking first, nowhere near enough to classify a run from it. This analysis produces an ordering for human attention, not a detector.
Figure 5. Per-sensor pass and fail distributions for up to eight shortlist sensors with the largest effect sizes. Values are standardised within each sensor (z-score) so sensors on different scales can share an axis; this is a display transform only.
| Action | Owner | Needs first | Reversible |
|---|---|---|---|
| Check the 33 shortlisted signals first on an excursion, leading with the ten that hold across both halves of the record | Process engineering | Nothing. Usable as a work instruction today | Yes |
| Investigate why sensors 112 and 113 are recorded on a minority of runs, before any change to their measurement families | Process engineering | ID-to-measurement mapping for two families | Yes |
| Review the 122 signals that have never varied, for sampling frequency or retirement | Metrology, equipment engineering | ID-to-measurement mapping; confirmation the constancy is not an acquisition fault | No |
| Review low-coverage signals at family level. The 50 fall into 8 groups recorded on identical runs, so this is 8 decisions and not 50 | Metrology, process engineering | Confirmation that partial coverage is by design rather than by fault | No |
| Change nothing for the remaining 385. They are measured, they vary, and nothing here connects them to outcome | — | — | n/a |
The last column is the reason the two reduction rows carry heavier prerequisites. A wrong entry on the shortlist costs an engineer an hour and corrects itself. A signal wrongly cut from monitoring stops producing the data that would reveal the error. Both reduction recommendations therefore rest only on structural facts about the record, never on a signal failing a significance test.
Source data: UCI SECOM, 1567 runs over 2008-07-19 to 2008-10-17, 924530 readings. Per-sensor Welch's t (fail vs pass), Benjamini–Hochberg at FDR 0.1 (with 0.05 also reported); near-duplicates grouped at |r| ≥ 0.95; stability checked by splitting at the median run timestamp. Reduced-monitoring buckets are assigned on structural grounds only, never on a significance result. Full pipeline and SQL: repository.
None of the top statistically-ranked signals held up under a real-world temporal check. Acting on any single one today would risk chasing a false fix — real investigation hours spent on a cause that may not hold. What this delivers is a validated screening method, not a signal to act on yet.
Multivariate SHAP ranking on the real SECOM record (XGBoost). These are the signals worth checking — not yet conclusions.
Each top-5 signal re-checked on the pinned 70/30 split (early vs late), split-then-check. Verdicts transcribed from the Layer 1 write-up.
| Signal | SHAP rank | Verdict | Why |
|---|---|---|---|
| sensor 60 | 1 | fails | Strong gradient early (1.4%→16.2%), flat late; sensor's own values drift down over time — a time-confound. |
| sensor 34 | 2 | fails | Non-monotonic within the early period itself (2.6→9.0→3.6→12.5%); minimal drift. |
| sensor 104 | 3 | fails (strict) | Monotonic early, but late dips then jumps; closest near-miss, no distribution drift. |
| sensor 22 | 4 | insufficient-data | Late period has only 2 quartiles with ≥5 fail events — direction not judgeable. |
| sensor 131 | 5 | fails | Non-monotonic within the early period (4.3→9.4→7.2→8.0%); no drift. |
Fail rate by pooled value quartile, computed separately in each period. ⚠ marks quartiles with fewer than 5 fail events (rate unstable).
| Quartile | Early n | Early fails | Early fail% | Late n | Late fails | Late fail% |
|---|---|---|---|---|---|---|
| Q1 | 210 | 3 | 1.43% ⚠ | 180 | 11 | 6.11% |
| Q2 | 253 | 3 | 1.19% ⚠ | 137 | 7 | 5.11% |
| Q3 | 270 | 14 | 5.19% | 120 | 7 | 5.83% |
| Q4 | 358 | 58 | 16.20% | 32 | 1 | 3.13% ⚠ |
Early shows a strong gradient (Q1 1.4% → Q4 16.2%); late flattens to ~5–6% and its high quartile nearly empties out. The gradient is not reproduced.
The sensor's own values drift downward across the record, so "high sensor 60" largely means "early run" — and early runs fail more. That is why the pooled pattern was time, not cause.
When a real, unconfounded cause exists, this screening approach finds it — and a designed follow-up experiment isolates and quantifies it. On this synthetic process the method recovered the injected cause, and the DOE attributed 75.9% of the explained variation to a single factor. The same pipeline is ready to run the moment a real signal is confirmed.
This tab uses a SYNTHETIC, illustrative plasma-etch process with a known, injected ground truth. It is not real SECOM data and its factors are not SECOM sensors (see the “How These Connect” tab).
Candidate drivers ranked by Random-Forest importance on the 400-run production log. Ground-truth labels shown alongside; the analysis never saw them.
Top-ranked factor = RF power = the injected dominant cause → match. The injected electrode-wear drift is also recovered (run-order significant, p ≈ 1e-57). Because factors here are independent of run order, both are identified at once — the case SECOM could not provide.
A rotatable central composite design (k=3: 8 factorial + 6 axial + 6 centre = 20 runs; α≈1.68), analysed with a full second-order model. Values match out/layer2/anova_table.csv exactly. Significance: *** p<0.001, ** p<0.01, * p<0.05, ns = not significant.
| Term | Sum Sq | df | F | p | % of SS | sig |
|---|---|---|---|---|---|---|
| x1 | 47553.8 | 1 | 317.99 | 6.57e-09 | 75.9% | *** |
| x2 | 9220.0 | 1 | 61.65 | 1.39e-05 | 14.7% | *** |
| x3 | 1078.0 | 1 | 7.21 | 0.023 | 1.7% | * |
| x1:x2 | 2272.3 | 1 | 15.20 | 0.003 | 3.6% | ** |
| x1:x3 | 12.8 | 1 | 0.09 | 0.776 | 0.0% | ns |
| x2:x3 | 0.6 | 1 | 0.00 | 0.950 | 0.0% | ns |
| I(x1 ** 2) | 460.0 | 1 | 3.08 | 0.110 | 0.7% | ns |
| I(x2 ** 2) | 462.1 | 1 | 3.09 | 0.109 | 0.7% | ns |
| I(x3 ** 2) | 127.4 | 1 | 0.85 | 0.378 | 0.2% | ns |
| Residual | 1495.4 | 10 | 2.4% |
RF power dominates (75.9% of SS), pressure is secondary (14.7%), and the RF×pressure interaction is real (3.6%). The quadratic curvature terms are not significant at this noise level — reported honestly, not hidden.
What's at stake is engineering time and yield risk. Chasing an unreplicated signal costs real investigation hours for a fix that might not hold; a validated method is ready to deploy the moment a real cause is confirmed. That trade-off — not the SHAP mechanics — is the takeaway.
Layer 1 asked a real question of real data and got an honest null: after a temporal replication check, none of the top-5 signals held, because the failure rate itself drifts over the record and the top signal's values track time. The method did the responsible thing — it refused to promote a signal it could not stand behind.
Layer 2 put the same ranking method on a synthetic process where the true cause is known and is not confounded with time. There, the method recovered the injected cause and a designed experiment quantified it. That proves the Layer 1 null was a property of the data — its non-stationarity — not a failure of the method.
Layer 2's factors — RF power, chamber pressure, gas flow — are a synthetic illustration. They are NOT a decoded version of any SECOM sensor. SECOM's sensor IDs are permanently anonymized: nothing in this app reveals which physical parameter sensor 60 (or 34, 104, 22, 131) represents. Layer 2 shows the method works; it does not resolve which real sensor matters.
A process engineer mapping SECOM's anonymized sensor IDs to physical process parameters. Once a real variable is identified, Layer 2's method — blind ranking plus a replication check — is exactly what you would run on it to confirm, and quantify, a genuine root cause.