# Idea Brief: HPT2 qualification

Status: completed
Type: idea-brief
Updated: 2026-08-27
Next Action: execute the approved three-stage retained-history performance campaign, stopping on the first failed stage
Produces: work/maps/map-hpt2-qualification.md
Source: work/spikes/spike-hpt2-service-life-retrieval-performance-requirements.md
PRM Plan: work/maps/map-hpt2-qualification.md
PRM Roadmap: work/roadmaps/roadmap-hpt2-qualification-r5.md

## Current Synthesis

### Idea

Explore how to qualify HPT2 as a service-life logging and retrieval system,
not merely whether an SD-card specimen works. HPT2 must remain correct and
responsive while retained media grows through maximum geometry, wrap,
reclaim, degradation, reboot, and concurrent customer access.

The central concern is bounded performance. A function that eventually
returns the right result but spends hours or days scanning unrelated retained
history is not acceptable. Charts, exports, open, and recovery must have
explicit budgets and measurable work bounds.

### Why It Matters

Customers can view only the newest 30 days in charts and export at most the
newest 90 days, but the circular HPT2 lanes may contain years-equivalent
internal history before reuse. That deeper retained state must not make normal
customer retrieval, logging, startup, or recovery progressively slower.

Earlier work improved bounded export and graph behavior, but maximum-depth
physical performance, repeated wrap/reclaim, recovery scaling, dense events,
and concurrency remain unproven.

The immediate remaining question is deliberately smaller: does unrelated
retained database size slow the exact same customer graph and export work?
Answer it with clean Controller A datasets at 100 days, 13 calendar months,
and two years. At each stage, restore the exact final Release candidate,
reconstruct the newest 30-day graph cache, and have the operator run the same
full newest-90-day export through the customer UI. Stop on the first failure;
do not continue loading larger fixtures until that stage is satisfactory.

### Desired Outcome

HPT2 has a small, quantitative, reproducible qualification contract showing
that:

- production logging remains correct and does not starve control, alarms, HMI,
  communications, or watchdog progress;
- charts for the newest 30 days and exports for no more than the newest 90 days
  complete within approved budgets;
- normal open, recovery, cancellation, and append interference remain bounded
  at supported minimum and maximum geometries;
- first and repeated wrap, reclaim, degradation, reboot, and approved
  interruption cases preserve correctness; and
- formal evidence distinguishes the HPT2 result from USB evidence state,
  operator interruption, and test-method or automation failure.

## Constraints And Non-Goals

- Apply KISS: only a safety hazard, proven data corruption or loss, or failure
  of the explicit core HPT2 qualification claim may block or expand the
  current future milestone. Other findings narrow the claim or become later
  work.
- SD-card removal, insertion, or replacement requires controller power off;
  powered hot swapping is prohibited.
- Formal SD/controller evidence includes the complete required CID identity
  block at baseline and after relevant replacement, power-off reinsertion,
  initialization, or reboot.
- USB absence may be operator-overridden without making the technical HPT2
  test fail. A run without required durable evidence receives no formal
  qualification credit unless an approved equivalent evidence path exists.
- Years-equivalent retention is an internal aging, wrap, and scaling stress
  condition. It does not extend customer access beyond 30-day charts and a
  maximum 90-day export.
- Firmware can characterize logical block writes and visible write
  amplification, but it cannot prove physical NAND wear distribution or
  remaining flash life inside an SD card.
- This idea is separate from qualification of a specific SD-card family or
  specimen and from supply-chain, ECO, rollout, or release execution.
- This brief is the Discovery/Plan source for the existing HPT2 project map
  and proposed roadmap. Association does not approve either artifact.
- The retained-history proof uses only Controller A's installed expendable
  development card. Each stage starts from an empty HPT2 store so its history
  is chronological and ends at the same effective present.
- The newest 90 days are identical at all three stages. Database size is the
  intentional variable; the exported row count, candidate, card, UI path, and
  acceptance budgets are held constant.
- This proof does not reopen wrap/reclaim, recovery breadth, Controller B,
  retail-card qualification, ECO, release, or rollout work.

## Possibilities And Tradeoffs

| Possibility | Benefits | Costs / Risks | Evidence / Status |
| --- | --- | --- | --- |
| Measure the current implementation before selecting changes | Finds the actual limiting paths and avoids speculative architecture | Requires instrumentation and reproducible maximum-geometry fixtures | Preferred starting direction |
| Direct iterator positioning from the range index | Can avoid walking segment headers from the oldest retained sequence for bounded requests | Must preserve envelope proofs, mirror voting, snapshots, and fallback correctness | Candidate; not approved |
| Redundant checksummed recovery checkpoint or directory | Could make normal open independent of a full segment-header scan | Adds durable metadata, compatibility, update-order, and corruption-recovery complexity | Candidate only if baseline justifies it |
| Retain a full scan as explicit fail-closed recovery | Preserves recovery when fast state is missing, stale, or corrupt | May be slow and must have progress, cancellation, and an approved exceptional budget | Required fallback concept; budget undecided |
| Separate fast regression from long physical qualification profiles | Keeps ordinary development feedback fast while preserving service-life evidence | Requires clear impact rules so needed long tests are not skipped | Recommended KISS structure |

## Open Questions

- What are the owner-approved budgets for normal open, fail-closed recovery,
  graph reconstruction and interaction, ten-minute/30-day/90-day export,
  cancellation, and append jitter?
- What is the smallest geometry, workload, controller, and physical-media
  matrix that can support the intended claim?
- How many physical wraps and interruption cases are necessary beyond
  accelerated deterministic models?
- Do measured results justify direct indexed seek, a recovery checkpoint,
  both, or neither?
- Which counters belong in production firmware and which are
  qualification-only?
- What evidence, if any, is equivalent to required USB output when the
  operator uses the USB-absence override?
- Is the project-map direction sufficiently settled to approve and promote
  this brief, or is more discovery needed first?

## Decisions

- The effort is called the **HPT2 Qualification Project**.
- HPT2 service-life qualification is separate from SD-card specimen or family
  qualification.
- Fast retrieval for charts and exports is part of success, not an optional
  optimization.
- Measure algorithmic work as well as elapsed time so avoidable linear scans
  cannot hide behind eventual functional success.
- Require a performance-plausibility adjudication that reconciles the
  hardware/media capability, the code path's predicted I/O and scaling, and
  observed target behavior. A path does not pass merely because it falls under
  a ceiling; it must be reasonable for the parts and data structures involved.
- Treat traversal from sector 1, the oldest retained segment, or another media
  origin toward a bounded request as presumptively unacceptable when validated
  indexed or checkpoint state can position the work directly.
- Keep technical outcome, evidence completeness, operator interruption, and
  test-method or automation outcome separate.
- No campaign is created or started during brainstorming.
- Use this Idea Brief as the explicit discovery source for the existing HPT2
  PRM.
- Use three roadmap milestones by merging physical qualification and truthful
  claim closeout into one evidence-gated milestone.
- Add one final KISS milestone with three clean fail-fast stages: 100 days
  (144,000 minute records), 13 calendar months (570,240 minute records for the
  2026-08-27 anchor), and two years (1,051,200 minute records).
- Rebuild the store from empty at each stage instead of appending older
  timestamps. This keeps all datasets chronological and gives each one the
  same newest-90-day export content.
- At every stage, restore and verify the exact final Release candidate before
  measurements, reconstruct the newest 30-day graph cache, and run the full
  newest-90-day customer export through the normal UI.
- Use the existing contract budgets: graph reconstruction at most 60 seconds;
  full 90-day HPT staging at most 240 seconds; full export at most 300 seconds;
  selected rows plus at most 8,192 proof neighbors and 438 index entries. Credit
  the exact target-selected rolling-window count when the fixture-to-candidate
  reflash edge is measured and independently verified; the owner accepted the
  100-day rung's 11-minute edge and 129,595 selected rows.
- Do not start the next larger fixture after a failed stage. Diagnose and
  remediate that observed failure, repeat the failed stage, then run one
  smaller-stage regression before continuing.

## Don't Forget

- Exercise realistic minute records, dense events, duplicate timestamps, clock
  changes, segment transitions, wrap, reclaim, retired segments, degradation,
  reboot, and concurrent logging/retrieval.
- Preserve mirror quorum, ordering, protected retention, fail-closed export,
  consistent snapshots, publication completeness, and essential-system
  isolation while improving speed.
- Record segment headers visited, records scanned/selected/emitted, SD commands
  and blocks, retries, fallbacks, writes/flushes, append delay, progress, and
  cancellation checkpoints where practical.
- Reuse applicable prior evidence by impact instead of restarting all
  qualification, but do not treat host-model speed as target/media proof.
- Keep one future milestone outcome and no more than five pass conditions at a
  time.
- Keep the latest 90 days deterministic and identical across the 100-day,
  13-month, and two-year stores so database size is the only planned variable.

## Exploration Log

### 2026-08-27

- Replaced the one-shot two-year/ten-minute-boundary proposal with the owner's
  simpler three-stage proof: clean 100-day, 13-calendar-month, and two-year
  stores, each followed by the same newest-30-day graph reconstruction and
  full newest-90-day customer export.
- Each stage is a gate. A failure stops the ladder for focused remediation;
  the next larger store is not built until the current stage passes.
- Clean rebuilds preserve chronological timestamps and identical newest-90-day
  content. Incrementally appending older data was rejected as unrealistic.
- The comparison separates fixed lookup/index/header work from the unavoidable
  processing of the same 129,600 selected minute rows at every stage.

- The owner approved the project-map direction by authorizing the recommended
  approval sequence.
- Promoted this brief after the approved project map captured its direction.
- Roadmap approval remains a separate exact-token gate; no campaign was
  created, queued, or started.
- During M3 execution, the owner clarified that qualification development must
  actively judge whether measured performance and algorithmic work make sense
  for the hardware, media, and code path rather than only checking thresholds.
  The approved contract and active M3 completion gate now require an explicit
  hardware/code/observation plausibility review and `ACCEPT`, `MITIGATE`, or
  `NARROW` disposition.
- S6 coverage now distinguishes paths measured by independent immutable
  counters from the path that happened to be largest. Qualification must not
  manufacture stochastic dominance; structural transition/reclaim credit still
  requires positive target identity proof.
- Final exact-candidate target measurement validated the performance gate:
  checkpointed open passed on both variants, while the same 56,160-record
  bounded graph reconstruction took 61.82 and 62.59 seconds and about 338,000
  SD blocks. The full claim was narrowed instead of rerunning, loosening the
  budget, or continuing destructive breadth that could not change that result.

### 2026-08-26

- Converted the completed HPT2 service-life retrieval-performance spike into
  the durable `ts: bs` discovery format while preserving the spike as source
  provenance.
- Carried forward the owner's KISS blocker rule, retrieval-performance concern,
  service-life states, safety boundaries, evidence separation, candidate
  mitigations, and open quantitative decisions.
- An HPT2 project map and proposed Program Roadmap already exist from earlier
  planning. This brainstorming operation did not approve, revise, delete, or
  materialize them, and no campaign was created.
- Associated this brief with the existing HPT2 PRM and moved it to
  `ready-for-prm` for the Plan Cycle.
- Reviewed the earlier proposal against Tool Shed 0.29.14. Retained its core
  claim and safety boundaries, added explicit Idea Brief provenance, and chose
  a three-milestone KISS structure for the replacement proposal.

Keep the synthesis current and append only useful dated exploration notes. Use `Status: ready-for-prm`
when the owner chooses to promote it, `promoted` after approved project-map direction captures it,
or `parked` when it is intentionally set aside. A promoted brief names its output in `Produces:`.
