SMS vs. Apps. Part 3: The Operating System
How to instrument an effective program. Recurring failure modes from commercial and research contexts. Will my compliance architecture survive scrutiny?
Part 3: The Operating System
3.1 How to instrument and measure and effective program.
3.2 Have I Designed Against the Recurring Failure Modes?
3.3 Will My Compliance Architecture Survive Scrutiny or Create Liability?
3.1. How to instrument and measure an effective program.
Builder TLDR: Instrumentation answers two questions: did the prescribed action happen, and does what you measure tie back to “effective.” Bottleneck pattern names the action layer; engagement metrics answer neither.
Effective as used in our field guides means something very specific. A program that survives its unit economics, hits criteria that enterprise healthcare buyers purchase and renew on, and most importantly, reduces human suffering.
A note on scale. What follows describes instrumentation for a program past first design. A team running its first 150 patients only needs to know which layer the mechanism sits in and instrument that layer; the full panel comes with scale.
Lens 1: Did the prescribed action happen?
The first instrumentation question is whether the prescribed action actually happens. Three bottleneck patterns recur, each pointing at a different action layer.
Titration-bottlenecked conditions (hypertension, polypharmacy, oncology dose adjustment, anticoagulation) make Layer 4 (clinical process and escalation) the operator’s primary leading indicator.
Track titration rate, escalations closed, time-to-human-review, false-positive burden. Bressman BP Pal demonstrates the pattern at the mechanism layer: program-driven data flow produced higher titration in the SMS arm, and titration is what moves the outcome the buyer pays on. (The trial was null on its primary six-month SBP endpoint with high attrition; the titration-rate signal is exploratory and operator-relevant, detailed in Section 2.1, Truth 4.)
Behavior-bottlenecked conditions (weight loss, glycemic management via diet and exercise, smoking cessation, physical activity) make Layer 3 (program adherence) the primary leading indicator. Track compliance ≥80% or the condition-specific equivalent: smoke-free days, food-log compliance rate, step counts above threshold. A patient who logs meals and consequently eats less is moving the outcome directly.
Reach-bottlenecked programs (Medicaid outreach, post-discharge, public-health campaigns where the binding constraint is delivery rather than action) make Layer 1 (reachability) the primary leading indicator. Track currently reachable, dormant-but-reachable, hard unreachable, re-engagement attempt success.
Figure 10. Engagement vs. program adherence as leading indicators for clinical outcomes.
Engagement metrics don’t answer the action-completion question.
