A Now / Next / Later product roadmap sequences the four earlier projects, with OKRs as goals and a value-versus-effort backlog of 17 issues (3 epics and 14 stories, #3 to #19), two sprint plans, and a pilot charter. It is part of an independent portfolio series on telecom customer analytics. The board is the click-through view of the same sheet.
Open the Now / Next / Later board
Projects in this series
- telco-churn-nba-engineNBA and churn score
- responsible-ai-packModel cards and CI gates
- omnichannel-care-analyticsJourney KPIs on a synthetic log
- care-automation-roin8n export and assumption ROI
- MULTI-HEAD-Three-head triage demo
Now, Next, Later
Three horizons across 3 columns on the sheet. Now is triage and routing. Next is NBA and churn. Later is self-healing journeys.
Now — Triage and routing
- Strictly below 0.6 is held for review
- emergency sets queue to senior_agent
- Export not run. No gold accuracy
MULTI-HEAD- at 809bccd61077898849c428f37521fa198f7c7bf6 is a three-head demo (issue, sentiment, urgency). Issue confidence strictly below 0.6 is held for review. The repo publishes no gold accuracy, F1, emergency recall, or false-emergency rate. Smoke floors minEmergencyRecall 0.01 and maxFalseEmergencyRate 0.99 are harness-health thresholds, not model quality. That reading is the one written in responsible-ai-pack MODEL_CARD_MULTIHEAD.md.
care-automation-roi ships workflows/n8n_emergency_escalation.json. An emergency label sets queue to senior_agent. The workflow README says the file has not been imported or executed. active is false. Precision 0.40, recall 0.75, and prevalence 0.08 in assumptions.yaml are illustrative placeholders.
Still needs. A labelled sample, and a workflow run against a real queue. Do not quote the smoke floors as accuracy.
Next — NBA and churn
- POST /score on 7,043 IBM rows
- Holdout ROC-AUC 0.846001, PR-AUC 0.656070
- Lift 2.806733. Floors 0.82, 0.63, and 2.60
telco-churn-nba-engine trains on the IBM Telco Customer Churn file, 7,043 rows, US sample. POST /score returns a churn probability, path contributions, a CLV proxy, add-on propensities, and a rule-based next action. On the published holdout the gradient-boosting churn model has ROC-AUC 0.846001, PR-AUC 0.656070, and top-decile lift 2.806733, against a dummy prior at ROC-AUC 0.500000. Those numbers are that holdout, not an operator result. Uptake scores are current add-on holding, not campaign response. The action table is a policy, not an uplift model.
responsible-ai-pack pins that repo at 21f6115931f4358ebc7cc87d9ba1f4d87fd015aa. It recomputes the same three metrics, draws TreeSHAP plots, runs Fairlearn on gender and SeniorCitizen, and fails CI under floors 0.82, 0.63, and 2.60.
Still needs. An operator extract, a refit, and a new pin. The IBM metrics stay IBM metrics.
Later — Self-healing journeys
- 4,000 synthetic journeys, seed 20260929
- Placeholder payback 8.47 months, year-1 ROI 0.4175
- The closed loop is not built
omnichannel-care-analytics computes journey KPIs on a seeded synthetic log (seed 20260929, 4,000 journeys). Charts are Plotly HTML labelled SYNTHETIC. Bitext supplies the intent taxonomy. The full Kaggle Twitter corpus was not downloaded: unauthenticated requests did not return the file, and no Kaggle credentials were used. pm4py is not used. The process view is a DuckDB directly-follows table.
care-automation-roi prices containment from assumptions.yaml. Every input is an illustrative placeholder. On those placeholders the base bot scenario shows payback 8.47 months and year-1 ROI 0.4175. There is no Streamlit app. The sensitivity chart is a committed PNG, reports/charts/roi_sensitivity_tornado.png, plus an HTML twin.
What is not built: a loop that changes a live route when a digital attempt fails. Later is that loop. The four repos do not contain it.
Still needs. Replace config/synthetic.yaml and assumptions.yaml with operator measurements. The loop that changes a live route is not built.
OKRs
Illustrative targets. None of these is a result this portfolio has achieved on operator data.
The gold note is the illustrative target. The green note is what the public demos already show.
Objective 1
A routing note a reviewer can defend without a fake accuracy number.
| Key result | Illustrative target | What the demos already show |
|---|---|---|
| KR1 | The routing note cites "not documented" for gold accuracy, and does not cite the smoke floors as quality. | The multi-head model card already says this. |
| KR2 | An import test of the n8n file, on a non-production n8n, shows emergency to senior_agent. | The file exists. It has not been run. |
| KR3 | Precision and recall leave assumptions.yaml only after a labelled sample is written down. | They are still placeholders. |
Objective 2
A next-best action with a gate in front of the score.
| Key result | Illustrative target | What the demos already show |
|---|---|---|
| KR1 | A pilot score returns churn probability, a CLV proxy, and one action from the published rule table. | POST /score does this for one IBM-sample customer. |
| KR2 | CI fails if holdout ROC-AUC, PR-AUC, or top-decile lift fall below 0.82, 0.63, and 2.60. | Those floors are in gates.yaml for the IBM artifact. |
| KR3 | A model card and the gender / SeniorCitizen table ship with the score before an agent sees it. | The pack has both, for the IBM sample only. |
Objective 3
Journey changes follow an operator log, not a synthetic clock.
| Key result | Illustrative target | What the demos already show |
|---|---|---|
| KR1 | Time-to-first-response, repeat contact, and digital-to-call are computed on an operator log. | They are computed on the synthetic log only. |
| KR2 | Cost and containment inputs are replaced from operator sources, and the placeholder label comes off only after that swap. | The label is still on every input. |
| KR3 | A failed digital attempt changes the next route in a named pilot path. | No repo implements that change. |
Backlog and prioritisation
Scores are illustrative planning scores for this artefact, on a 1 to 5 scale. They are not measured benefit and not hours. The quadrant is a reading aid for the backlog. The table is the score.
OKR targets and scores are illustrative planning values, not achieved results.
Position is the value and effort in the table. Colour is the quadrant column. Stories that share a score share one dot.
| Item | Issue | Horizon | Value | Effort | Quadrant | Status |
|---|---|---|---|---|---|---|
| Triage and routing | #3 | Now | 5 | 3 | Major bet | Demo partial |
| Triage quality gap | #4 | Now | 5 | 1 | Do first | Demo exists |
| Abstain at 0.6 | #5 | Now | 4 | 1 | Do first | Demo exists |
| n8n emergency branch | #6 | Now | 4 | 2 | Do first | Export, not run |
| Labelled triage sample | #7 | Now | 5 | 5 | Major bet | Not started |
| NBA, churn, and governance | #8 | Next | 5 | 3 | Major bet | Demo exists |
| POST /score | #9 | Next | 5 | 2 | Do first | Demo exists |
| CI gates | #10 | Next | 5 | 2 | Do first | Demo exists |
| Fairness table | #11 | Next | 4 | 2 | Do first | Demo exists |
| Operator churn refit | #12 | Next | 5 | 5 | Major bet | Not started |
| Self-healing journeys | #13 | Later | 4 | 5 | Major bet | Not built |
| Synthetic journey KPIs | #14 | Later | 3 | 2 | Do first | Demo exists |
| Assumption ROI | #15 | Later | 3 | 2 | Do first | Demo exists |
| Operator journey swap | #16 | Later | 5 | 5 | Major bet | Not started |
| Self-healing loop | #17 | Later | 4 | 5 | Major bet | Not started |
| pm4py | #18 | Later | 1 | 3 | Reconsider | Declined |
| Streamlit | #19 | No horizon | 2 | 3 | Reconsider | Declined |
Epics: #3 triage, #8 NBA and churn, #13 self-healing journeys. The full sheet is backlog.csv.
Pilot plan
Two sprint plans, with user stories and Given / When / Then criteria, live in docs/sprint-plans.md.
Sprint 1 and Sprint 2 as written in the sprint plan. These sprints were not run.
Sprint 1 — Triage routing contract
A reader can see what the triage demo does, and what it must not be quoted as.
- #4 State the triage quality gap before a route uses the label
- #5 Keep issue scores below 0.6 on the review path
- #6 Branch an emergency label in the n8n export
Out of this sprint: measuring precision and recall (#7), until a labelled sample exists.
Sprint 2 — NBA demo behind the published gates
The IBM-sample score, the rule, and the governance pack stay tied together.
- #9 Serve one-customer next-best action from POST /score
- #10 Gate held-out ROC-AUC, PR-AUC, and top-decile lift
- #11 Publish the gender and SeniorCitizen check with the score
Out of this sprint: the operator refit (#12).
Pilot / PoC charter template: docs/pilot-charter.md.
Lessons learned
- I keep the churn train/test split leak-free. The split is drawn before imputation, scaling, encoding, or the survival curve, and preprocessing stays inside a pipeline so held-out rows cannot change what was learned.
- I fail the build in CI when the saved churn model's holdout ROC-AUC, PR-AUC, or top-decile lift falls below the floors in gates.yaml (0.82, 0.63, and 2.60 on the IBM artifact).
- I label synthetic data everywhere it appears, including the journey charts, so a reader can see that the log is synthetic.
- I keep illustrative inputs explicit in config. Cost and containment figures stay in assumptions.yaml, and the placeholder label stays on each one.
- I test README numbers against committed outputs, so a figure in the write-up has to match the file the run produced.
Data and scope
Built on public and synthetic data as an independent portfolio project.
Limitations
- This roadmap sequences public demos. It is not a production operating plan.
- Illustrative OKR targets and the 1-5 scores are planning labels in this repo. They are not measurements.
- Lessons learned are engineering notes from building the four demos. They are not operator results, and they do not add a metric.
- The closed loop in Later is not implemented.
- Projects 1-4 use the IBM US sample, a synthetic journey log, Bitext intent names, a short Twitter preview, and labelled cost assumptions. None of that is operator customer data.
- The backlog is the GitHub Issues list. A Projects board is produced by scripts/create_board.sh when it is run with a token that has project scope.
- This repo does not add a new model, a new dataset, or a new metric.