# Correct HPT graph rolling cache refresh Workpackage

Status: complete
Type: workpackage
Updated: 2026-07-26
Next Action: none

Project Map: work/maps/map-smi-firmware-stabilization-and-qualification.md
Parent: work/wp/active/wp-sd-card-accelerated-endurance-evaluator-and-media-qualification.md
Canonical Truth: docs/sd-card-endurance-evaluator.md and docs/sd-card-operation-and-recovery.md

## Current Context

The firmware maintains a fixed `535,680`-record SDRAM chart array: 30 days of
real five-second positions plus one future day. At startup it blanks the array,
traverses the bounded retained HPT suffix, and enables the graph only after the
cache is complete.

The graph application then defeats that cache by requesting another complete
HPT traversal and blocking the HMI for up to 2,500 ticks. While the graph is
open it requests another complete traversal every 60 seconds. Controller A
finishes within the entry deadline; Controller B has already been evidenced as
requiring longer than the original 30-second observation window, even though
its card, HPT state, full-address scrub, and eventual `720/720` chart result
pass.

Current implementation surfaces:

- `src/Thread_SdCard_entry.c`: startup load, HPT traversal, queued append, RAM
  chart publication, midnight handling.
- `src/HMI_MainEventHandler.c`: graph entry and screen refresh behavior.
- `src/AppVariables.h`: fixed chart geometry and publication sequence.
- `script/check-hpt-commissioning.ps1`: current static HPT invariants.

## Recommendation

Keep the fixed 30-day cache and its five-second zoom resolution. Load it once
at startup, publish each successfully persisted production log into the cache,
roll the day window in RAM at midnight, and open the graph from a stable
published snapshot without waiting for media.

Do not merely increase the timeout. Do not weaken HPT validation, read directly
from the media in the HMI thread, expose a partially mutated cache, or change
the 30-day product graph range.

## Current State

Completed:

- Fixed, bounded chart storage and timestamp-index mapping exist.
- Startup full-cache reconstruction is serialized in the SD thread.
- Odd/even `gChartDataSequence` publication protects the HMI from a cache being
  rebuilt.
- Controller B's timeout is reproduced in evidence as a timing-bound traversal,
  not an SD read failure.
- Graph entry uses the completed stable RAM cache without a media request or
  blocking wait.
- Successfully persisted production records publish incrementally.
- A normal midnight boundary rolls the 30-day window in RAM; invalid states
  retain the bounded rebuild.
- Focused graph-cache, graph-policy, HPT commissioning, version-5 routing,
  customer-export, full-address-scrub, and review-mitigation checks pass.
- The clean managed Debug build and production-name packaging pass. Exact
  hashes and the operator handoff are recorded in
  `work/evidence/hpt-graph-rolling-cache-correction-20260725.md`.
- Per-lane recovered range metadata and direct sequential slot traversal are
  implemented; the exact split-lane model performs `767/767/852` slot reads
  with zero missing-copy header scans.
- The clean managed build/package gates pass for exact S-record
  `EAD175B1...81DA4B`, which is programmed, verified, reset, and running on both
  controls.
- Controller B was not formatted or reinitialized. It retains the expected
  CID and its 85-record lane split, then completes its chart cache in `3.12 s`
  (`2.30 s` traversal).
- Controller A completes its chart cache in `3.37 s` (`2.56 s` traversal).
- Both controls report a complete, stable graph cache containing 720 records
  and are running for operator graph-screen review.
- The follow-newest image `143F1941...C3124` is programmed on B then A. Live
  publication advances the active viewport with zero edge lag at 1, 3, 6, 12,
  and 24 hour zoom on both controls. Pan-away hold and exact Pan Right
  reattachment pass. A cache is `2.68 s`; B cache is `2.60 s` with unchanged
  CID and `86/86/1` lane starts. Both controls were reopened on current Graph
  for operator review.
- The format-compatible bulk image `A77ADA66...F7211` is now programmed on B
  and A. Both report graph-ready caches: A `1.98 s` total (`1.15 s`
  traversal), B `1.88 s` total (`1.04 s` traversal). B retained its CID and
  abnormal `86/86/1` lane starts without format or reinitialization.
- The succeeding retained-start image `91D05B35...0CCB5` is now programmed on
  B then A. Both remain graph ready: A `2.09 s` total (`1.24 s` traversal),
  B `1.98 s` total (`1.14 s` traversal). The larger captures contain 62 and
  68 additional records; normalized traversal shows no material regression.
  B remains unformatted with the same CID and `86/86/1` lane starts.
- The real-event timer image `D3D36155...A1EAF` passes A/B next-record
  advancement at 24, 12, 6, 3, and 1 hour zoom plus Pan Left hold and Pan
  Right reattachment. On 2026-07-26 the owner observed advancement while
  zooming in and accepted the live-refresh correction.

Incomplete:

- Complete the bounded operator graph-screen/export-entry visual review.

## Goal

The graph opens promptly from the already validated RAM cache on both
controllers, receives new persisted records without a media rescan, retains a
fixed rolling 30-day view across midnight, and preserves existing HPT
integrity, concurrency, and export behavior.

## Why It Matters

The current redundant scans create an operator-visible timeout on Controller B
and would contaminate subsequent endurance results with avoidable read load.
Correcting the lifecycle before further media qualification prevents validation
time from being invested in a known graph-path defect.

## Major Outcomes

- Graph entry performs no media request and no blocking wait.
- Each successful `STRT` or `EVNT` HPT append updates the current RAM window.
- Cache mutation remains guarded by the odd/even publication sequence.
- Midnight drops the expired day, preserves the remaining 29 days, and creates
  a blank current/future edge without rereading the card.
- Startup, new-media initialization, and exceptional cache invalidation retain
  a bounded full-rebuild path.
- Sequential HPT traversal resolves each lane's current segment range once and
  advances through physical segment order; an absent historical lane copy is
  rejected from range metadata rather than by a whole-card scan.
- Lane-fault qualification remains explicit evidence-producing work, but its
  persistent retirements must no longer create unbounded graph-cache latency.
- Cache-load timing and scan counters are target-visible so operator handoff
  reports measured behavior instead of a polling-window inference.
- Automated checks and a clean managed Debug build pass before operator review.

## Delivery Stages

Use stages when sequencing, parity gates, or handoff cost matter.

| Stage | Outcome | Entry Evidence | Exit Evidence |
| --- | --- | --- | --- |
| 1 | Lifecycle correction | Current timeout path and cache ownership traced | Source implements cached entry, incremental append, and RAM day roll |
| 2 | Split-lane traversal correction | B's sequence-85 retirement and `767/767/852` state preserved | Iterator resolves valid segment ranges without exhaustive missing-copy scans |
| 3 | Software verification | Source changes complete | Static/model checks and clean Debug build pass |
| 4 | Abnormal-media target proof | Exact image programmed without formatting B | B completes cache reconstruction with captured elapsed time and graph-ready state |
| 5 | Live follow-newest correction | Published chart records advance but open viewport remains grid-stuck | Explicit follow state, exact right edge, all-zoom checks, and A/B target proof pass |
| 6 | Real-event timer correction | Operator zoom reproduces stale render sequences while chart publication continues | Real GUI zoom/pan paths preserve the repeating Graph timer and A/B advance on subsequent minute records |
| 7 | Operator review handoff | A/B exact image and runtime evidence identified | Owner verifies A/B graph entry, visible data, refresh, and navigation |

## Related Artifacts

- Workpackages:
  `work/wp/active/wp-sd-card-accelerated-endurance-evaluator-and-media-qualification.md`,
  `work/wp/active/wp-triple-mirrored-circular-hpt-log-store.md`,
  `work/wp/completed/wp-optimize-hpt-triple-mirror-read-performance.md`
- Evidence: `work/evidence/post-freeze-integrated-qualification-20260725.md`
- Checklists:
  `work/checklists/checklist-hpt-media-2-autonomous-retail-card-qualification.md`
- Runbooks: `work/runbooks/runbook-autonomous-hpt-sd-qualification.md`
- Evidence:
  `work/evidence/hpt-graph-rolling-cache-correction-20260725.md`

## Rough Sequence

1. Preserve Controller B's split-lane card and capture its existing CID and
   `767/767/852` baseline.
2. Retain per-lane oldest/current segment metadata recovered during open.
3. Make sequential iteration maintain one cursor per lane and directly read
   the matching slot; skip absent lane ranges without scanning all segments.
4. Keep majority/degraded selection, CRC, commit, and payload validation
   unchanged.
5. Bound chart reconstruction to the newest graph-relevant suffix and expose
   start/completion/elapsed/scan counters.
6. Add focused invariants and an executable model for split-lane traversal.
7. Run all software gates and a clean managed Debug build.
8. Program the hash-locked image without formatting or replacing Controller
   B's card.
9. Capture B's cache-load elapsed time and stable `720/720` graph state, then
   leave the graph screen ready for operator review.

## Milestones

### Milestone 1: Source correction

Completion criteria:

- [x] Opening the graph performs no full-cache request and has no wait loop.
- [x] New HPT production records update the cache only after persistence
  succeeds.
- [x] Midnight rolls the fixed window in RAM.
- [x] Full rebuild remains available for startup and media initialization.

### Milestone 2: Software gate

Completion criteria:

- [x] Focused graph-cache invariants pass.
- [x] Relevant existing HPT checks pass.
- [x] Clean managed Debug build and artifact packaging pass.

### Milestone 3: Operator review

Completion criteria:

- [x] Controller B retains the pre-fix CID and intentional split-lane state;
  no format, card replacement, or HPT initialization occurs.
- [x] A split-lane iterator model proves missing copies do not cause
  per-sequence whole-media scans.
- [x] Target-visible counters report cache start, completion, elapsed ticks,
  scanned records, and selected chart records.
- [x] Controller B completes startup cache reconstruction on the preserved
  abnormal layout and its measured elapsed time is recorded.
- [x] Controller A is electronically ready to open the graph promptly with
  retained data (`3.37 s` startup cache).
- [x] Controller B is electronically ready to open the graph without the data
  timeout (`3.12 s` startup cache on preserved split-lane media).
- [x] A newly completed production record appears without reopening the graph
  or rescanning the card after every real GUI zoom level selection.
- [x] Legend, return, and graph export entry receive the final bounded operator
  visual review; automated zoom/pan signal handling is target-proven.

## Open Questions

- Exact touch-to-render and publication-delay instrumentation remains optional;
  the owner observed prompt Graph entry and live advancement and accepted the
  behavior.

## Completion Standard

This workpackage is complete when the source and automated/build gates pass,
the bulk-read successor is integrated, the exact optimized image is reviewed
on both controllers, the operator reports the four graph-review criteria pass,
and the result is linked into release qualification evidence.

## Closeout Routing

- Current truth promoted to: `docs/sd-card-operation-and-recovery.md`
- Historical context remains in:
  `work/evidence/post-freeze-integrated-qualification-20260725.md`
- Runtime/status evidence:
  `work/evidence/hpt-graph-rolling-cache-correction-20260725.md` and
  `work/evidence/candidate-identity-and-operator-graph-export-review-20260726.md`
- Cleanup deferred to: none
