# Optimize HPT triple-mirror read performance Workpackage

Status: complete
Type: workpackage
Updated: 2026-07-26
Next Action: none

Project Map: work/maps/map-smi-firmware-stabilization-and-qualification.md
Parent: work/wp/active/wp-triple-mirrored-circular-hpt-log-store.md
Depends On: work/wp/completed/wp-correct-hpt-graph-rolling-cache-refresh.md
Canonical Truth: docs/sd-card-operation-and-recovery.md

## Current Context

HPT media 2.0 stores every record as a data sector plus a durable commit sector
in each of three logical lane extents on one physical SD card. A healthy
sequential read currently issues six single-sector commands per record:
data/commit for lanes A, B, and C. The segment-aware cursor correction removed
Controller B's pathological repeated whole-lane searches, but it did not reduce
this healthy-path command or transfer cost.

The same iterator is used by:

- startup reconstruction of the 30-day/five-second RAM chart cache;
- customer USB export staging;
- lane/readback and wrap diagnostics.

USB export has two additional scaling defects. It scans the complete retained
snapshot regardless of the requested time range, processes one record per
slice, and normally returns to a 10-tick SD-thread sleep after each record.
At 100 ticks/second, this caps staging near 10 records/second before USB output.

Current exact-image target baselines use Debug S-record SHA-256
`EAD175B1F20D4E80858B69C4DCBC7A021125BDBE5EA6DD6B8E251ABCA681DA4B`:

| Metric | Controller A | Controller B |
| --- | ---: | ---: |
| Assembly / J-Link | `1751136 / 69664082` | `1751219 / 69660123` |
| CID | `035344535330384780976F0084018559` | `035344535330384780976F0085018507` |
| Records scanned | `927` | `885` |
| Cache traversal | `256 ticks / 2.56 s` | `230 ticks / 2.30 s` |
| Cache total | `337 ticks / 3.37 s` | `312 ticks / 3.12 s` |
| Lane records at timed load | `927/927/927` | `800/800/885` |

Controller B is the required first target because its intentional 85-record
lane split proves the optimized implementation handles non-aligned lane
ranges. Its card must not be formatted, reinitialized, removed while powered,
or replaced.

## Recommendation

First implement a format-compatible bulk read path that verifies all three
copies synchronously. Read aligned multi-block chunks from each lane, then
decode data/commit pairs, CRCs, payloads, lane masks, and quorum in RAM. Retry a
failed chunk and subdivide only the failing range; never degrade the healthy
path to single-sector reads.

Use one bulk iterator for chart reconstruction and USB export. Service queued
production logging between bounded chunks. Replace export's fixed
one-record/10-tick pacing with a time-budgeted batch while retaining snapshot,
gap, range-retention, capacity, and fail-closed publication rules.

Add a protected segment-level time/range index only after bulk traversal is
measured. It may be reconstructed from bounded segment evidence initially.
Any persistent media-format extension requires a separate compatibility and
power-interruption decision; it is not part of the first implementation.

Do not initially:

- skip the third lane or weaken synchronous three-copy health reporting;
- change HPT media 2.0, reformat either installed card, or manufacture a
  favorable Controller B layout;
- expose a partial chart or partial USB export;
- trade control/logging responsiveness for an unbounded SD transfer;
- infer month-scale acceptance from elapsed time alone without recording
  records, blocks, commands, bytes, retries, and normalized throughput.

## Current State

Completed:

- Per-lane oldest/current segment ranges and direct sequential slot traversal
  are implemented and model-tested.
- Controller B's preserved split-lane cache completes in `3.12 s`; Controller
  A completes in `3.37 s`.
- Cache start/end/traversal ticks and scanned/selected counts are visible on
  target and service 468.
- Graph entry and incremental graph refresh no longer rescan media.
- Existing CRC, durable commit, quorum, ambiguity, gap, and snapshot contracts
  have focused checks.

Dispositioned follow-on:

- Segment time/range indexing and deeper bulk-failure subdivision remain
  optional follow-on optimization; the current path retains exact bounded
  scalar fallback and fail-closed export rules.
- USB CSV/FileX throughput is now separately measured and is the dominant
  long-range export cost. No USB speed threshold was defined here; any later
  requirement to make long-range USB publication faster is separate product
  optimization work.

Implementation and target evidence complete:

- Format-compatible 16-record lane windows use up to 32-sector multi-block
  reads while all available copies remain synchronously validated.
- Chart reconstruction and customer export share the bulk iterator; export
  uses bounded 64-record slices and the active owner-loop cadence.
- Read command/block/max/refill/fallback and cache/export timing telemetry is
  target visible.
- All `script/check-*.ps1` suites, focused models, clean managed Debug rebuild,
  and production-name packaging pass.
- Exact candidate S-record `A77ADA66...F7211` is programmed on B then A.
- A: `996` records, `1.15 s` traversal, `1.98 s` total, `189` commands,
  graph ready.
- B: `944` records, `1.04 s` traversal, `1.88 s` total, `168` commands,
  graph ready; no format/reinitialization and `86/86/1` lane starts preserved.
- Evidence:
  `work/evidence/hpt-triple-mirror-read-performance-20260726.md`.
- Retained-range export UX is implemented and software-qualified. Exact
  candidate `91D05B35...0CCB5` is programmed on B then A. Both publish valid
  retained bounds and are graph-ready; B remains unformatted with `86/86/1`
  first sequences. Cache totals are `1.98 s` on B and `2.09 s` on A.
- Retained-start evidence:
  `work/evidence/hpt-export-start-adjustment-20260726.md`.
- B's real adjusted-start USB export published 1,024 records with zero gaps
  and a matching COMPLETE trailer. HPT scan/staging measured `2.84 s`;
  90-day staging projects to about `6.0 minutes`. USB CSV
  format/write/close/flush time is not separately instrumented.
- Final cumulative-candidate operator review passed on both controls. Fresh
  adjusted-start exports published 1,735 records on A in `2.18 s` HPT staging
  and 1,684 records on B in `2.08 s`, with COMPLETE trailers, zero
  controller-reported snapshot/range gaps, and no bulk fallbacks. Exact CSV
  hashes and source-event timestamp observations are recorded in
  `work/evidence/candidate-identity-and-operator-graph-export-review-20260726.md`.
- The warning-truth successor candidate
  `944BEA28BFD6DE5604A5BE1B03F4F5690CA84213B2BD923940ED27D193844501`
  was loaded on both controls and retained the same bulk-read implementation.
  With additional accumulated records, B scanned `1,445` in `1.64 s`
  traversal / `2.51 s` total while preserving `86/86/1`; A scanned `1,498`
  in `1.75 s` traversal / `2.60 s` total. Both caches are graph-ready with
  zero bulk fallbacks. See
  `work/evidence/operator-warning-truth-and-target-proof-20260726.md`.
- The cumulative graph-timer successor
  `D3D3615586FD522A9472504EAB55BC8A6F4955956AAE3229CCD397A0AD1A1EAF`
  is the only current A/B operator-review image. It includes the bulk-read,
  retained-start, warning-truth, and real-event timer corrections. The earlier
  `A77ADA66...`, `91D05B35...`, and `944BEA28...` hashes remain stage
  evidence and must not be treated as the current live candidate.
- The final USB-timing successor
  `A7055AC3F4314EEF440ADA77B76B107C387BDBC68095AD6E1EEBC15D04074FA4`
  is now loaded on B and A. B first published 1,711 records in `2.17 s` HPT
  staging, `31.71 s` USB publication, and `33.91 s` total. A then published
  1,762 records in `2.28 s`, `21.24 s`, and `23.53 s`. Both reported clean
  final rename/flush, zero gaps, zero fallbacks, zero dropped logs, and zero
  system/HPT/USB errors. Both controls are left on an active Graph following
  newest. See
  `work/evidence/hpt-usb-publication-timing-20260726.md`.
- The measured `2.36x` to `2.39x` normalized iterator gain does not meet the
  original `4x` stretch goal and is not presented as doing so. The result is
  accepted because commands fell more than `31x`, practical 30-day
  cache/staging projections are below `60 s`, bulk-8 to bulk-16 showed
  diminishing elapsed-time return, correctness remained unchanged, and USB
  publication now dominates end-to-end export time. The `4x` stretch is
  closed as unmet and non-blocking.

## Goal

Chart reconstruction and customer export read the current three-mirror media
at sustained multi-block speed, remain responsive to production logging, and
preserve all existing integrity and fail-closed behavior. Performance is
measured on both physical controls against the exact current-image baseline,
including Controller B's preserved abnormal lane layout.

## Why It Matters

The current layout reads about 133 MB and issues 259,200 SD commands for 30
days of healthy one-minute records. The byte volume is acceptable; the
single-sector command pattern is not. Export's additional pacing can make a
30-day snapshot take about 72 minutes to stage before USB output. That is
materially slower than the retired FAT32/FileX path and would become worse as
retained history grows.

## Major Outcomes

- Healthy reads use bounded multi-block commands and in-RAM validation.
- All three lane copies remain synchronously checked in the first production
  implementation.
- Cache and export share one correctness and recovery implementation.
- Narrow exports do not scan unrelated retained history.
- Foreground queued logging remains higher priority, with measured maximum
  service latency and zero new drops.
- A/B evidence reports absolute and normalized before/after performance.
- Controller B retains its CID, retirement state, and 85-record lane split.

## Delivery Stages

| Stage | Outcome | Entry Evidence | Exit Evidence |
| --- | --- | --- | --- |
| 1 | Baseline and observability | Current exact image and CID-linked cache evidence | Scalar cache/export/iterator command, byte, timing, and digest baselines |
| 2 | Format-compatible bulk core | Scalar iterator and split-lane model | Multi-block lane-chunk iterator produces identical results and bounded failure isolation |
| 3 | Consumer integration | Bulk core passes host tests | Chart reconstruction and export staging use shared bulk traversal with time-budgeted yielding |
| 4 | Software/build gate | Source integration complete | Focused models, all repository checks, clean managed Debug build, and packaging pass |
| 5 | Controller B abnormal-media proof | Exact candidate hash and rollback image recorded | B programmed without media initialization; CID/lane state preserved; cache and read benchmark pass |
| 6 | Controller A normal-media proof | B gate passes | A programmed; aligned-lane cache and read benchmark pass |
| 7 | Performance and operator disposition | A/B electronic gates pass | Before/after report, month-scale projection, graph review, and bounded USB export review complete |
| 8 | Retained-range export UX | Oldest recoverable customer record is target-visible | Start submit clamps to retained/90-day boundary, explains why, opens normal End Date, and adjusted filename/export metadata pass |

## Related Artifacts

- Workpackages:
  `work/wp/completed/wp-correct-hpt-graph-rolling-cache-refresh.md`,
  `work/wp/active/wp-triple-mirrored-circular-hpt-log-store.md`
- Ticket:
  `work/tickets/ticket-clamp-export-start-to-retained-data.md`
- Evidence:
  `work/evidence/hpt-graph-rolling-cache-correction-20260725.md`
  and `work/evidence/hpt-export-start-adjustment-20260726.md`
- Runbook:
  `work/runbooks/runbook-autonomous-hpt-sd-qualification.md`
- Inventories:
  `work/inventories/smi-current-setup-a.md`,
  `work/inventories/smi-current-setup-b.md`

## Rough Sequence

1. Preserve the exact current S-record, A/B CID, store sequence/ranges, lane
   state, and existing cache measurements as the rollback/baseline set.
2. Add non-mutating read-performance telemetry and a RAM-commanded benchmark
   over an immutable retained snapshot. Capture scalar throughput on both
   controls without formatting or appending test records.
3. Extend the block-device/iterator API for aligned multi-block chunks. Test
   candidate chunk sizes such as 16, 32, 64, and 128 sectors against transfer
   throughput, maximum call latency, DMA alignment, stack, and RAM limits.
4. Read lane chunks rather than alternating lanes per record. Decode and
   compare all data/commit records in RAM. On a failed bulk read, retry then
   bisect the range to retain exact error attribution.
5. Add executable fixtures for aligned lanes, B's sequence-85 split,
   retirements at chunk boundaries, wrap, truncated active segments, corrupt
   data, corrupt commit, mismatch, one-copy degradation, and I/O retry.
6. Use the bulk iterator for startup cache reconstruction. Preserve stable
   odd/even publication and the complete-before-visible graph contract.
7. Use the same iterator for customer export. Process bounded batches by time
   budget, service the production log queue between batches, and retain
   complete-snapshot-before-USB publication.
8. Add segment range/time selection so export reads only segments that can
   intersect the request while still proving retained bounds, sequence
   continuity, and requested-range gaps.
9. Run focused and repository-wide checks, the correct managed clean Debug
   build, packaging, hash capture, and source/static review.
10. Program Controller B first with hash lock and expected J-Link/CID. Do not
    format or initialize its card. Capture cache, scalar/bulk digest parity,
    throughput, lane ranges, and queue/drop health.
11. If B passes, program and measure Controller A identically.
12. Compare commands/record, blocks/second, records/second, cache traversal,
    export staging, total export, maximum SD-call latency, logging latency,
    retries, drops, resets, and system errors. Record projections separately
    from directly observed durations.
13. Leave both controls on the accepted image for graph and USB-export operator
    review. Restore the recorded `EAD175B1...81DA4B` image if correctness,
    responsiveness, or integrity gates fail.

## Milestones

### Milestone 1: Measurable reference behavior

Completion criteria:

- [x] Scalar and bulk modes can traverse the same immutable snapshot and
  publish comparable digests and health results.
- [x] Telemetry separates SD transfer, validation, cache processing, export
  staging, and USB publication time.
- [x] Baselines record exact image hash, assembly, J-Link, CID, first/end
  sequence, lane ranges, scanned/selected records, commands, blocks, bytes,
  retries, ticks, and records/second.

### Milestone 2: Format-compatible bulk traversal

Completion criteria:

- [x] Healthy lane reads use multi-block transfers.
- [x] All three lanes remain synchronously verified.
- [x] Scalar/bulk results match exactly across aligned, split-lane, retirement,
  wrap, corruption, degradation, and retry fixtures.
- [x] Failed or ambiguous bulk windows use bounded exact scalar resolution;
  healthy A/B traversals used zero fallback.
- [x] Chunk processing has an explicit maximum service-time budget.

### Milestone 3: Cache and export integration

Completion criteria:

- [x] Cache reconstruction uses the shared bulk iterator and remains hidden
  until a complete stable publication.
- [x] Export uses bounded batches rather than one record per 10-tick loop.
- [x] Export selects only the requested time range from the retained snapshot;
  segment-level pre-index skipping is explicitly deferred.
- [x] Snapshot mutation, non-retained range, gap, ambiguity, capacity, media
  I/O, and USB failures remain fail-closed.
- [x] Production records queued during long reads complete without new drops.

### Milestone 4: Software candidate

Completion criteria:

- [x] Focused models/static checks pass.
- [x] All repository `script/check-*.ps1` suites pass.
- [x] Correct managed clean Debug build and production-name packaging pass.
- [x] Exact ELF/S-record/map hashes and rollback hashes are recorded.

### Milestone 5: Controller performance proof

Completion criteria:

- [x] Controller B is programmed first without card format, initialization, or
  replacement; expected CID and 85-record split remain.
- [x] Controller B scalar/bulk parity, cache, export staging, logging, stack,
  watchdog, reset, and error gates pass.
- [x] Controller A passes the same gates on aligned lanes.
- [x] Healthy SD command count falls by at least one order of magnitude.
- [x] The `4x` stretch is measured and dispositioned: the actual `2.36x` to
  `2.39x` gain is accepted as an explicit stretch miss, not claimed as `4x`.
- [x] Measured month-scale normal-periodic projection is below `60 s` for
  cache traversal and below `90 s` for export staging on both controls.
- [x] Qualifying runs have no new dropped logs, system errors, CID changes,
  record gaps, mismatches, or integrity regressions. A discarded pre-run
  harness reset is documented and followed by a fresh B CID/cache baseline.
- [x] Operator confirms graph entry/data/navigation and a bounded USB export.

## Open Questions

- What multi-block count gives the best throughput without an unacceptable
  single-call latency on the installed cards and SSP driver?
- Can current DMA read directly into external SDRAM, or should internal aligned
  ping-pong buffers be used?
- Can a trustworthy segment time index be reconstructed without changing media
  2.0, or is a separately approved backward-compatible metadata extension
  required?
- After synchronous three-lane bulk performance is measured, is a two-lane
  foreground quorum plus rotating background third-lane scrub worth a separate
  policy decision?

## Completion Standard

This workpackage is complete because one exact optimized image passed software
gates, was programmed and measured on B then A, preserved B's abnormal media
and all three-mirror correctness contracts, met the accepted practical
read/staging thresholds, characterized USB publication through final flush and
rename, introduced no control/logging regression, and passed bounded graph and
USB-export review. The `4x` stretch miss is explicitly accepted rather than
rewritten as a pass.

## Closeout Routing

- Current truth promoted to: `docs/sd-card-operation-and-recovery.md`
- Historical context remains in:
  `work/evidence/hpt-graph-rolling-cache-correction-20260725.md`
- Runtime/status evidence:
  `work/evidence/hpt-triple-mirror-read-performance-20260726.md` and
  `work/evidence/hpt-usb-publication-timing-20260726.md`
- Cleanup deferred to: optional quorum-first policy and any media-format/index
  evolution require separate decisions after the format-compatible result
