# Characterize USB FileX write-size durability Workpackage

Status: complete
Type: workpackage
Updated: 2026-07-29
Next Action: none

Project Map: work/maps/map-smi-firmware-stabilization-and-qualification.md
Canonical Truth: docs/sd-card-operation-and-recovery.md

## Current Context

The fastest weekend candidate combined a 128 KiB SDRAM CSV window, 128 KiB
USBX BOT phase, 8 KiB FileX media buffer, 2 KiB USBHS FIFO, and single
approximately 95 KiB FileX writes. Controller A later exposed removed-media
false success, while bounded 16 KiB writes have since passed. Real HPT exports
confound USB write-size characterization with SD traversal and record state.

## Recommendation

Use a conservative, capability-based USB mass-storage profile instead of
brand/model exceptions. R4 is rejected and must not reach Controller B.
Sequence the remaining work as four isolated changes:

1. R5 lifecycle correctness at the existing transfer/write settings.
2. A conservative 32 KiB universal transfer/application baseline.
3. Session-scoped promotion to the owner-selected 64 KiB HPT cap only after a
   drive passes a physical reconnect and independent persistence proof.
4. A separate SSP 2.7.0/USBX 6.4.0 and Renesas HCD investigation; do not fold
   middleware or HCD changes into R5.

The 96 KiB diagnostic result remains headroom evidence, not a production
target. Do not use VID/PID, manufacturer, model, capacity, or USB serial as the
primary compatibility decision. Optional SCSI capability state must be scoped
to one completed enumeration session and discarded on removal, failed
enumeration, VBUS cycle, or controller reboot.

## Current State

Completed:

- Exact B-profile comparison and conservative rollback artifacts exist.
- Controller A is the authorized first specimen.
- Guarded implementation, managed build, 23/23 checks, rollback capture,
  Controller A deployment, and post-reboot identity verification are complete.
- A-2 passed the 1 MiB screen and 7 MiB sustained tier at 16, 32, and 64 KiB.
- Independent Pi sizes, row counts, identical-content hashes, and unmounted
  FAT checks passed after both tiers.
- A separately built four-size diagnostic established that 96 KiB also passed
  the 1 MiB and 7 MiB controller and independent Pi gates on A-2.
- A Windows-formatted 8 GB General UDisk accepted all four 1 MiB payloads but
  failed every first media flush with `FX_IO_ERROR`; Pi inspection isolated
  an incompatible synchronize-cache durability barrier and post-failure FAT
  metadata discrepancies.
- Display-board schematic review plus an illuminated-stick target test proved
  that Controller A physically removes connector VBUS through PB00 and the
  TPS2051 switch for a commanded five-second interval, then remounts.
- The exact-sense compatibility implementation passed the managed build and
  all 23 repository check suites, then was deployed and verified on Controller A.
- The original General UDisk passed the 1 MiB and 7 MiB four-size tiers. The
  exact compatibility path ran 16 times, every post-cycle remount and complete
  file verification passed, and independent Pi hashes, byte comparisons, and
  unmounted FAT checks were clean.
- The owner selected 64 KiB as the intended production HPT FileX write cap.
  End-to-end execution is authorized, but implementation remains correctly
  sequenced after VBUS-only A/B legacy-media parity.
- The VBUS-only candidate passed Controller B with USB01_2G (then labeled
  B-CTRL) at every 1 MiB and
  7 MiB tier, complete USB feature parity, and independent Pi verification.
- The same candidate exposed a Controller-B/USB02_8G (then labeled A-CTRL)
  legacy interaction:
  FileX writes and exact directory sizes succeeded, but immediate VBUS removal
  after STOP left partly written or erased `0xFF` payloads. This is a
  cache-quiescence defect, not evidence against the selected 64 KiB cap, and
  must be mitigated before the 64 KiB candidate is built.
- The legacy-only five-second post-STOP cache-quiescence mitigation passed
  Controller A and B 1 MiB/7 MiB 16/32/64/96 KiB tiers and independent Pi
  verification. The VBUS-only sequencing gate for the 64 KiB candidate is
  therefore complete.
- The isolated 64 KiB real-HPT candidate was built and deployed to Controller
  A. USB02_8G (then labeled A-CTRL) failed closed during file close after 10
  seconds. USB04_32G (then labeled A-CTRL-2)
  reported controller-side success for a 131,027-byte, two-write export, but
  independent unmounted Pi inspection disproved durability: the two FATs
  differed, `HPTE.TMP` remained at zero directory length with an allocated
  cluster chain, the free-cluster summary was wrong, and the requested final
  CSV was absent. The 64 KiB candidate is rejected in its present lifecycle.
- Source review identified a remaining lifecycle hole: a compliant drive
  received only the short power-off interval, fresh verification could still
  observe drive-retained cache, and no final STOP/quiescence barrier preceded
  the Success screen and operator removal.
- R2 added universal five-second VBUS loss and a final STOP/quiescence barrier,
  but independent Pi inspection again found differing FATs, `HPTE.TMP`, and no
  final CSV. This disproved insufficient power-off/removal timing as the full
  cause. The remaining false-success path is FileX fault-tolerant rename
  recovery after remount: firmware verified the recovered FileX cache without
  first flushing that recovery transaction to the physical device.
- R3 added that post-remount flush, but the Pi again found the unchanged
  pre-power state. Callback review then exposed the actual fail-open
  mechanism: the application ignored `ux_system_host_storage_fx_media_get`
  status, retained the prior global media pointer, and unconditionally set
  `gIsUsbStickMounted` from the device event. Thus a failed/unavailable new
  FileX session was reported as mounted and all "fresh" checks could read the
  prior RAM session.
- R4 made USB mounted state conditional on a successful, instance-matched
  FileX media lookup, invalidated application media pointers on removal, and
  required a changed nonzero media-session generation after VBUS reconnection.
  The managed Debug build and all 23 repository checks passed. Controller A
  was programmed with SREC SHA-256
  `7021EA4CFEFBD9B67EA130C225A10426492F76382D2C7C130E341321FACC6B4F`.
  Its real-HPT run reported Success for 4,654 healthy records, zero gaps or
  degraded records, 131,027 requested bytes, two FileX writes with a 65,536
  byte maximum, and a 13.08-second total duration. Independent unmounted Pi
  inspection nevertheless found differing FAT copies, only zero-length
  `HPTE.TMP` with an allocated cluster chain, and no `HPT64_A_R4.csv`.
  Therefore R4 remains a false positive: a changed callback generation and
  instance-matched media pointer do not prove that USBX/FileX established a
  physically fresh media state.
- R5 lifecycle correction passed Controller A with `USB04_32G` and Controller
  B with `USB01_2G`, including independent Pi verification.
- The isolated 32 KiB production HPT write-cap candidate passed focused checks,
  all 23 repository checks, and managed Debug and Release builds.
- Controller A passed production exports with `USB01_2G`, `USB02_8G`,
  `USB03_32G`, and `USB04_32G`; Controller B passed representative
  cross-controller exports with `USB01_2G` and `USB04_32G`.
- Every 32 KiB run reached successful terminal publication with zero USB/HPT
  errors and passed independent Pi persistence verification. The pre-existing
  FAT summary mismatches on `USB02_8G`, `USB03_32G`, and `USB04_32G` were
  preserved without repair.
- The 32 KiB profile is the accepted current production compatibility baseline.
  No brand/model allowlist is required for the tested population.

Incomplete:

- On 2026-07-29 the owner accepted the qualified 32 KiB performance and chose
  not to pursue 64 KiB or greater production write caps at this time.
- Electrical and protocol qualification still requires VBUS voltage/current,
  overcurrent, discharge, inrush/droop, and USBHS signal-integrity evidence.
- The isolated SSP 2.7.0/USBX 6.4.0 and Renesas HCD investigation remains a
  research spike and authorizes neither migration nor patching.

## Session Handoff — 2026-07-28

- Stop condition: R4 failed independent physical-media verification even
  though Controller A reported Success.
- Historical USB location: `USB04_32G`, then labeled `A-CTRL-2`, was attached
  to `srv-rpi00.local` as `/dev/sda1`; disk serial `071C49DE246A1F97`. It was
  unmounted and was
  left unchanged after read-only `fsck.vfat -n -v`.
- USB observed state: FAT copies differ; `/HPTE.TMP` has directory size zero
  with a nonempty allocated cluster chain; final `HPT64_A_R4.csv` is absent.
- Controller location/state: Controller A remains programmed with R4; do not
  deploy this candidate to Controller B.
- R4 frozen artifacts:
  `Debug/7520553_SMI_FW_HPT_64K_DURABILITY_R4_20260728.{elf,srec,map}`.
- R4 hashes: ELF
  `14A694FC42BF1D4312600F833242F712CADA8A7C394C7A195F3CE680DD4814B8`;
  SREC
  `7021EA4CFEFBD9B67EA130C225A10426492F76382D2C7C130E341321FACC6B4F`;
  MAP
  `F0AEED5FEB60DE6CB8DF2237F86F62DC6C19ADF44501B7785FF8C5F00279E11F`.
- R4 run evidence:
  `work/evidence/hpt-log-store-circular-mirror/controller-a/hpt-64k-20260728/real-export-compliant-a-ctrl-2-r4`.
- Resume by tracing USBX removal/insertion and FileX media-open/close ownership,
  including the generated host-storage removal path and mount return status.
  Establish what state survives after application globals are cleared. Do not
  reformat or reuse `USB04_32G` (then labeled `A-CTRL-2`) until the current
  failed filesystem state has
  been captured sufficiently for that diagnosis.
- Planning-only update on 2026-07-29: no firmware edit, build, deployment, media
  mutation, or hardware execution is authorized by this plan update.

## Goal

Establish a broadly compatible Renesas USB-host/FileX lifecycle whose Success
result is proven from a genuinely new physical media session, then qualify a
conservative 32 KiB baseline and optional session-scoped 64 KiB promotion.

## Why It Matters

Production should not retain 1 KiB writes if a larger bounded request is
demonstrably durable, but a controller-side Success screen is insufficient.

## Major Outcomes

- Deterministic identical-content files make host hashes directly comparable.
- Per-file timing, call-count, maximum-write, close, flush, rename, and verify
  status remain available through stable symbols.
- Test execution cannot begin accidentally: exact RAM magic is required.

## Delivery Stages

Use stages when sequencing, parity gates, or handoff cost matter.

| Stage | Outcome | Entry Evidence | Exit Evidence |
| --- | --- | --- | --- |
| 1 | Preserve and diagnose R4 | failed USB04_32G (`A-CTRL-2` at the time) state remains unchanged | lifecycle ownership and retained-state mechanism documented |
| 2 | Plan R5 lifecycle correction | diagnosed teardown/activation ordering | accepted fail-closed teardown and fresh-session contract |
| 3 | Implement/build R5 | separate execution authorization | complete; lifecycle-only exact candidate and software gates passed |
| 4 | Qualify R5 | Controller A rollback and preserved specimen policy | complete; A/B new-session, durability, and independent host proof passed |
| 5 | Conservative compatibility | R5 accepted | complete; 32 KiB production baseline passed the available four-drive matrix with representative A/B coverage |
| 6 | Adaptive 64 KiB disposition | conservative population passes | no action by owner decision; reopen only through a new explicit decision and qualification scope |
| 7 | Modernization spike | lifecycle work stable | disposition of SSP 2.7.0 and Renesas HCD PING behavior |

## Related Artifacts

- Tickets:
- Checklists: work/checklists/checklist-controller-a-usb-write-size-specimen-qualification.md
- Spikes: work/spikes/spike-evaluate-ssp-2-7-0-usbx-modernization-and-renesas-hcd-interoperability.md
- ADRs:
- Runbooks:
- Inventories:
- Decision matrices:

## Planned Sequence

Planning only; every implementation, build, deployment, media mutation, and
physical run requires separate authorization.

Execution was authorized on 2026-07-29. R5 now waits for the old storage
instance to disappear from the USBX class registry, requires every published
application media pointer to be invalidated before VBUS restoration, and
requires the remounted storage instance to be registered/live before FileX
verification. The managed Debug and Release builds passed, as did all 23
repository check suites. Controller A `1751136` was programmed and verified
through J-Link `69664082` with Debug SREC SHA-256
`DD91B7D3F615A09EEC166D94ECEDF6C3A9ADADEB95D488731ABCAA1EB2C3EF98`.
The post-deployment baseline reconfirmed SD CID
`035344535330384780976F0084018559`, but reported `usb_mounted=0` because
preserved specimen USB04_32G (then labeled A-CTRL-2) remains physically
attached to the Pi. No R5
diagnostic was triggered, no USB filesystem was mutated, and Controller B was
not touched. Baseline evidence is under
`work/evidence/hpt-log-store-circular-mirror/controller-a/usb-r5-lifecycle-20260729/`.

The corrected qualification harness was then made lifecycle-unconditional so a
drive that supports `SYNCHRONIZE CACHE` could not pass by write/readback alone.
Controller A passed the 1 MiB and 7 MiB 16/32/64/96 KiB matrices, including
observed unmounted VBUS-off intervals and fresh remount verification after
every file. Independent unmounted Pi inspection found exact file sizes and
matching hashes within each tier; FAT1 and FAT2 were identical. The historical
R4 zero-length `/HPTE.TMP` allocated-chain finding remains, and `fsck.vfat -n`
left it unchanged. See
`work/evidence/controller-a-usb-r5-lifecycle-qualification-20260729.md`.

Controller B `1751219` / J-Link `69660123` then received the identical frozen
R5 image and passed both forced lifecycle matrices with USB01_2G (then labeled
B-CTRL). Independent
unmounted Pi inspection found a clean read-only FAT check, exact sizes and
expected hashes for all eight files, identical FAT copies, and a final
unmounted device state. See
`work/evidence/controller-b-usb-r5-lifecycle-qualification-20260729.md`.

1. Preserve R4 artifacts and the unchanged failed `USB04_32G` filesystem
   (historical label `A-CTRL-2`).
2. Trace the complete USBX/FileX teardown and activation path.
3. Define the R5 lifecycle-only contract:
   - serialize one USB owner and stop new work;
   - close every file, flush FileX, and attempt cache synchronization;
   - close FileX media and invalidate application references;
   - treat START STOP UNIT as an optional capability, not sole durability proof;
   - wait for endpoint abort, media-ID clear, media-memory release, and old
     storage-instance destruction before restoring VBUS;
   - require a new activation cookie after a completed removal epoch;
   - reopen and verify exact name, length, trailer, record count, and content
     hash from the new session;
   - use direct-media or otherwise demonstrably uncached checks for critical
     filesystem state and require an independent non-correcting host check.
4. Re-enable FileX error checking for the USB production path unless measured
   evidence demonstrates an unacceptable cost; retain `FX_SINGLE_THREAD` only
   with proven single-owner access.
5. Recognize both Illegal Request `20h/00h` and `24h/00h` as unsupported
   SYNCHRONIZE CACHE capability results. All timeouts, phase errors,
   medium/hardware errors, aborted commands, and unrelated sense values remain
   terminal. Never blindly retry an ambiguously completed write.
6. After R5 passes, create a separate 32 KiB universal compatibility candidate.
7. Only after the conservative candidate passes, add optional 64 KiB
   session-scoped promotion; reset capability state on every lifecycle change.
8. Qualify FAT12/16/32 and whole-device exFAT, MBR and superfloppy layouts,
   multiple capacities, old/new controller families, unsupported optional
   commands, cold/warm enumeration, reboot, clean idle removal/reinsertion, and
   BOT reset recovery. Safely reject unsupported filesystems, native 4 KiB
   logical sectors, malformed layouts, and unsupported multi-LUN cases.
9. Run the linked SSP/HCD spike only as a separate investigation.

## Milestones

### Milestone 1: Deployable diagnostic

Completion criteria:

- [x] Six deterministic file outputs are implemented.
- [x] A wrong/duplicate/busy command fails closed.
- [x] Correct managed build and static/model gates pass.
- [x] Controller A rollback and deployed hashes are recorded.

### Milestone 2: Physical characterization

- [x] 1 MiB 16/32/64 files pass controller and Pi gates.
- [x] Passing candidates run the 7 MiB sustained tier.
- [x] The owner selected 64 KiB as the intended production successor, retaining
  32 KiB below the passing 96 KiB diagnostic candidate and 64 KiB below the
  fixed 128 KiB transport/application bounds.
- [x] The candidate passes removal/reinsertion, reboot, and additional-specimen gates before production adoption.
- [x] The 64 KiB change is isolated from and follows successful A/B
  qualification of the VBUS-only candidate.

### Milestone 3: Legacy USB compatibility

- [x] The tested legacy population passes the accepted compatibility path;
  any additional `20h/00h` or `24h/00h` capability expansion remains with
  `work/wp/active/wp-migrate-usb-vbus-control-out-of-ssp.md`.
- [x] Detection is based on the actual SCSI response, not a manufacturer,
  model, capacity, or USB-ID allowlist.
- [x] Timeouts, transport failures, medium/hardware errors, and all other
  sense codes remain terminal errors.
- [x] Compatibility publication requires physical VBUS removal, fresh
  enumeration/remount, and complete file verification.
- [x] General UDisk passes the 1 MiB and 7 MiB four-size tiers.
- [x] Independent Pi hashes, exact sizes, and unmounted FAT checks pass.
- [x] Any remaining per-enumeration capability-state expansion is transferred
  to `work/wp/active/wp-migrate-usb-vbus-control-out-of-ssp.md`; R5 lifecycle
  truth and the accepted 32 KiB population are complete.

### Milestone 4: Lifecycle truth

- [x] FileX media is closed by USBX deactivation and application references are
  invalidated during deliberate VBUS removal.
- [x] VBUS is not restored until the old storage instance is absent from the
  USBX class registry and published media/storage references are invalidated.
- [x] A completed removal epoch plus new activation cookie proves a new session;
  callback count and reusable pointers cannot satisfy the gate.
- [x] Post-remount verification follows a five-second physical VBUS-off interval,
  new registered storage activation, FileX reopen, exact-size/content checks,
  and an independent unmounted Pi hash check.
- [x] Any teardown, remount, verification, BOT recovery, or filesystem-check
  uncertainty reports failure and never Success.

### Milestone 5: Broad compatibility baseline

- [x] A 32 KiB universal transfer/application profile passes the representative
  drive/filesystem/capacity matrix.
- [x] Optional 64 KiB-or-greater promotion is dispositioned as no action; any
  future reopening requires its own physical reconnect and persistence proof.
- [x] No brand/model allowlist is required for the accepted population.
- [x] Remaining VBUS electrical and USBHS signal-integrity work is retained in
  `work/wp/active/wp-migrate-usb-vbus-control-out-of-ssp.md`; it does not block
  acceptance of the qualified 32 KiB cap.

## Open Questions

- Does the selected 64 KiB target repeat across lifecycle gates and an
  additional specimen?
- Which USBX/FileX objects or caches survive the R4 VBUS cycle, and what exact
  teardown event proves the old storage instance is destroyed?
- Can critical post-remount verification bypass both the pre-cycle FileX cache
  and any retained device-controller cache?
- Does the additional approximately 23% sustained write-time reduction at
  96 KiB justify using 75% of the fixed 128 KiB transport/buffer bounds?
- Can an unsupported-SYNCHRONIZE-CACHE fallback use the proven
  physical VBUS cycle and complete post-remount verification to preserve
  honest durability across compliant and legacy USB media?

## Completion Standard

This workpackage is complete when the lifecycle cannot report Success from
retained state, the conservative profile passes a representative USB-drive
population, larger-cap promotion is either physically proven or explicitly
dispositioned, and the accepted behavior and limitations are promoted to
current documentation.

## Closeout Routing

- Current truth promoted to: docs/sd-card-operation-and-recovery.md
- Historical context remains in: this workpackage
- Runtime/status evidence: work/evidence/
- Cleanup deferred to:
