# Onda 2.A Follow-up #2 — Ranged Resume Mid-Download

> **For Claude:** REQUIRED SUB-SKILL: Use `superpowers:executing-plans` to implement this plan task-by-task. Mid-stream resume is a Manual-aligned use case (§6.2 endorses it explicitly).

**Branch:** `sta/onda2a-followup-2-ranged-resume` (this worktree, cut from `origin/main` post Onda 2.A core merge — origin/main HEAD `92e0be2e`)
**PR target:** `main`
**Depends on:** Onda 2.A core (PR #30, merged at `3af17964`) — `Client.download_part/2` already exists with §6.4 wire format pinned.
**Estimated effort:** 2-3h actual · LOC delta estimate: +200-250 (mostly tests + state machine).
**Parent plan:** `sta/docs/plans/2026-04-26-sta-conformance-rollout.md` (Pattern: Manual Verification mandatory)
**Related:** Onda 2.A core plan (`2026-04-26-sta-onda2a-resilient-transfer.md`) Task 6 noted this as "follow-up task; this function fetches the full file in one transaction and verifies hash. The data captured in the result is sufficient for a future caller to drive that path."

---

## Index

1. [Task 0 — Manual Verification](#task-0)
2. [Architecture](#architecture)
3. [Pre-flight](#pre-flight)
4. [Tasks](#tasks)
5. [Acceptance](#acceptance)
6. [Out of scope](#out-of-scope)
7. [References](#references)

---

<a id="task-0"></a>
## Task 0 — Manual Verification

Per the **Pattern: Manual Verification** mandated for every Ondas 2-4 sub-PR (`sta/docs/plans/2026-04-26-sta-conformance-rollout.md#pattern-manual-verification`). Performed before any code work for this follow-up.

**Source:** `sta/Manual_STA_Web_Services.pdf` (Manual de utilização dos Web Services do STA, v1.5, jul/2022). Sections re-read in this session: §6.1, §6.2, §6.3, §6.4.

### Verified `[VERIFIED §X.Y]`

| Claim | Manual section | Notes |
|---|---|---|
| Resume mid-download is documented use case | §6.2 | "Caso a conexão seja interrompida durante o recebimento (download), é possível retomá-lo da parte do arquivo que ainda falta baixar. Para isso, utilize a requisição descrita na seção 6.4 - Recebimento de parte do arquivo, informando o intervalo de bytes do arquivo que ainda não foi baixado." Manual explicitly endorses; the Onda 2.A core handoff was overcautious in suggesting it might be ambiguous. |
| `Range: bytes={inicio-fim}` syntax requires both bounds explicit | §6.4 | "inicio-fim: byte inicial e byte final da parte" — no documented support for open-ended `bytes=N-`. |
| `If-Match` and `If-Unmodified-Since` are optional | §6.4 | "os cabeçalhos If-Match e If-Unmodified-Since são opcionais" |
| 206 Partial Content response carries ETag + Last-Modified + X-Content-Hash | §6.4 | Same response headers as full GET (§6.1). |
| `X-Content-Hash` on 206 = SHA-256 of the **complete** file | §6.1+§6.4 | Same wording: "hash do arquivo" — not "hash do range". Allows caller to verify after stitching all parts. |
| HTTP 412 on Range request = If-Match/If-Unmodified-Since failed | §6.4 | Already mapped in `Client.download_part/2` as `:precondition_failed` (PR #30). |
| HTTP 416 on Range request = bad range | §6.4 | Already mapped as `:invalid_range`. |
| HTTP 501 on Range request = multipart range unsupported | §6.4 | Already mapped as `:range_multipart_unsupported`. |
| HTTP 410 on download = file unavailable | §6.1+§6.4 | Already mapped as `:file_unavailable`. |
| `Content-Type` MUST NOT be in request | §6.1+§6.4 | Both endpoints' "Atenção" notes. Already enforced in `download_file/2` and `download_part/2` from Onda 2.A. |

### Items `[STUDY-CONJECTURE]` (Manual silent — conservative read applied)

| # | Topic | Manual silence | Decision |
|---|---|---|---|
| 1 | Open-ended `Range: bytes=N-` form | §6.4 only documents `bytes={inicio-fim}` (both bounds). RFC 7233 standard allows `bytes=N-` but Manual doesn't mention it. | **Do not use**. Require caller to supply `:expected_size` (we already get this from `FileInfo.file_size` in `list_available_files`). Conservative read per Pattern. |
| 2 | Stitching strategy when partial response's hash header ≠ full file's hash header from prior attempt | Manual silent on whether ETag/Last-Modified/X-Content-Hash can change between requests. | Treat any 412 (If-Match validation) as "file changed; restart from byte 0". On any X-Content-Hash mismatch after stitching, return `:hash_mismatch_after_resume`. No silent acceptance of inconsistent results. |
| 3 | Behavior when caller-supplied `:expected_size` is wrong | Manual silent on what happens if caller's size is too small/large. | If too small: caller asks for `bytes=N-{wrong_end}`, BCB returns 416 → restart from 0. If too large: BCB might 416 the initial Range request (start ≥ size) or return body shorter than expected → assembled body smaller than expected → hash mismatch → `:hash_mismatch_after_resume`. Both error paths covered. |
| 4 | Maximum number of resume attempts before giving up | Manual silent. | Reuse `:max_retries` from existing `download_with_retry/2` (default 3). Each attempt-to-attempt transition counts as one retry. |

### Items `[NEEDS-OPERATOR-CONFIRM]`

None for this follow-up. (The Onda 2.A core had 2 such items — concurrency budget and chunk size — both inherited unchanged here.)

### Misdiagnoses caught

None. The Onda 2.A core handoff suggested `bytes=N-` open-ended form was an open question; this is not a misdiagnosis but a **clarification**: Manual §6.4 is silent on it, and per Pattern silence requires conservative read (don't use).

---

<a id="architecture"></a>
## Architecture

### Component 1: New `Client.download_file_streamed/2`

Existing `Client.download_file/2` uses `Finch.request/3` which buffers the full body. To capture partial bytes from a mid-stream connection drop, we need streaming.

**Public function:**

```elixir
@spec download_file_streamed(String.t(), keyword()) ::
        {:ok, Types.DownloadResult.t()}
        | {:partial, %{body: binary(), headers: map(), bytes_received: non_neg_integer()}}
        | {:error, Error.t()}
def download_file_streamed(protocol_number, opts \\ [])
```

**Returns:**
- `{:ok, %DownloadResult{}}` — stream completed cleanly with HTTP 200; ETag/Last-Modified/X-Content-Hash populated (existing extraction logic reused).
- `{:partial, %{body, headers, bytes_received}}` — stream started (status + at least one `:data` chunk received) but failed mid-stream. Caller can use `bytes_received` + headers (ETag) to drive `download_part/2` resume.
- `{:error, %Error{}}` — pre-stream failure (DNS, TCP, timeout before any byte; HTTP 4xx/5xx with no body received). Existing retry-loop semantics apply.

**Implementation:** new `:download_file_streamed` operation atom routed through existing `handle_call({:execute, ...})`. New private `execute_streaming_request/3` parallel to `execute_request/3` uses `Finch.async_request/3` + receive loop to track accumulator state across `:status`/`:headers`/`:data`/`:done`/`:error` messages. On `:error` with `bytes_received > 0` → return `{:partial, ...}`. On `:error` with `bytes_received == 0` → return `{:error, Error}` (retry-loop will retry pre-stream errors).

The retry-loop in `do_execute_with_retry/6` treats `{:partial, ...}` as a non-retryable success-ish result — it's the Operations layer's job to drive resume, not the Client's retry-loop. This preserves the existing single-attempt semantics of `download_file/2` while opting streaming callers into a richer return shape.

### Component 2: `Operations.download_with_retry/2` resume branch

**Signature change (additive):**

```elixir
@type download_opts :: [
        {:max_retries, non_neg_integer()},
        {:retry_delay_ms, non_neg_integer()},
        {:circuit_breaker, atom() | nil},
        {:rate_limiter, atom() | nil},
        {:client_opts, keyword()},
        {:verify_hash, boolean()},
        {:expected_size, pos_integer()}  # NEW — opt-in resume capability
      ]
```

**Behavior matrix:**

| `:expected_size` | Behavior |
|---|---|
| Not provided (nil) | Existing single-shot path via `Client.download_file/2`. Zero behavior change. |
| Provided | New `do_download_with_resume/4` state machine (private). |

**State machine `do_download_with_resume/4`:**

```
state := %{etag, last_modified, bytes_received, parts: iolist, attempt}
expected_size := caller-supplied
context := same as do_download_with_retry/3

ATTEMPT(state, attempt) when attempt >= max_retries:
  return {:error, :max_retries_exceeded_during_resume}

ATTEMPT(%{bytes_received: 0}, attempt):    # fresh start
  case Client.download_file_streamed(protocol):
    {:ok, %DownloadResult{}}                → verify_hash + return result
    {:partial, %{body, headers, bytes_received}}
                                             → state' := %{etag: headers[:etag],
                                                            last_modified: headers[:last_modified],
                                                            bytes_received: bytes_received,
                                                            parts: [body]}
                                             → ATTEMPT(state', attempt + 1)
    {:error, _} = err                        → existing retry-loop semantics: retry up to max_retries with backoff (zero behavior change)

ATTEMPT(%{bytes_received: n} when n > 0, attempt):     # resume
  case Client.download_part(protocol_number, start_byte: n, end_byte: expected_size - 1, if_match: state.etag):
    {:ok, %DownloadResult{content: body, x_content_hash: hash}}
                                             → assembled := IO.iodata_to_binary([state.parts, body])
                                             → verify(SHA-256(assembled) == hash)
                                             → return result
    {:error, :precondition_failed}           → reset state to fresh; ATTEMPT(fresh, attempt + 1)
    {:error, :invalid_range}                 → reset state to fresh; ATTEMPT(fresh, attempt + 1)
    {:error, :range_multipart_unsupported}   → reset state to fresh; ATTEMPT(fresh, attempt + 1)
    {:error, _} = err                        → return err
```

Note: `download_part/2`'s response body in `DownloadResult.content` is the bytes from `start_byte` to `end_byte` only (not the whole file). Full-file hash verification happens AFTER stitching. The X-Content-Hash header on 206 is the full-file hash per Manual §6.4.

### Component 3: Types unchanged

`DownloadResult` already has `:etag`, `:last_modified`, `:x_content_hash`, `:content` from Onda 2.A core. No new struct needed for happy path. The `:partial` return is a plain map (not a struct) intentionally — it's a transient intermediate state, not a public-facing type.

---

<a id="pre-flight"></a>
## Pre-flight

```bash
cd /Users/luizpenha/monetarie/.worktrees/sta-paridade

# 1. Branch state
git status              # On sta/onda2a-followup-2-ranged-resume, clean
git log --oneline -1    # Should show this Task 0 + plan commit when landed

# 2. Compile baseline (5 warnings expected)
cd sta/backend
mix deps.get
mix compile             # 5 baseline warnings (Logger.info macro, unused alias, ungrouped clauses)

# 3. Touchstone tests
mix test test/sta_connector/sta/client_test.exs         # 55 tests, 6 baseline failures
mix test test/sta_connector/sta/operations_test.exs     # 45 tests (post-#30+#34 in main, pre-#36), 0 failures
mix test test/sta_connector/files/outbound_file_test.exs # 7/7 PASS
mix test test/sta_connector/application_test.exs        # 3/3 PASS

# 4. Postgres dev (only needed if any DB-touching code added; this follow-up does NOT touch DB)
docker ps --filter name=sta-connector-db-dev --format '{{.Names}}: {{.Status}}'
```

**Note:** This branch was cut from `origin/main` post Onda 2.A core (PR #30) + Follow-up #1 (PR #34) merges. PR #36 (Follow-up #3, `resume_upload`) is OPEN at time of writing — orthogonal (upload-side) and lands independently.

---

<a id="tasks"></a>
## Tasks

### Task 1 — `Client.download_file_streamed/2` with mid-stream failure capture

**Files:**
- Modify: `sta/backend/lib/sta_connector/sta/client.ex` (new public function + new operation atom + new `execute_streaming_request/3`)
- Modify: `sta/backend/lib/sta_connector/sta/client_behaviour.ex` (new callback)
- Test: `sta/backend/test/sta_connector/sta/client_test.exs` (new `describe "download_file_streamed/2 wire format"` block)

**Step 1 (TDD red):** 4 Bypass tests:
1. **Successful full stream** — Bypass plug sends 200 + ETag + Last-Modified + X-Content-Hash + binary body in one chunk. Asserts `{:ok, %DownloadResult{content, etag, last_modified, x_content_hash}}` with all fields populated. Asserts NO `Content-Type` request header (§6.1 reuse).
2. **Mid-stream connection drop** — Bypass plug sends `Plug.Conn.send_chunked(200)` + headers + `Plug.Conn.chunk(partial_body)` + raises (or `Process.exit(self(), :kill)` inside the plug callback) → asserts `{:partial, %{body: partial_body, headers, bytes_received: byte_size(partial_body)}}` with `headers[:etag]` populated.
3. **Pre-stream HTTP 410** — Bypass plug returns 410 + XML error body → asserts `{:error, %Error{type: :file_unavailable | ...}}`.
4. **Pre-stream timeout** — Bypass plug calls `Process.sleep(:infinity)` → asserts `{:error, %Error{type: :timeout, retryable: true}}` with timeout less than test default.

**Step 2 (TDD green):** Implement using `Finch.async_request/3` + `receive` loop:

```elixir
defp execute_streaming_request(:download_file_streamed, params, state) do
  # build request (reuse build_rest_request(:download_file, params) — same path/method)
  # call Finch.async_request, receive loop accumulating %{status, headers, body_iolist, bytes_received}
  # on :done: parse_streamed_response(...) -> {:ok, DownloadResult} or error if status was 4xx/5xx
  # on :error and bytes_received > 0: build {:partial, ...}
  # on :error and bytes_received == 0: build {:error, Error}
  # on receive timeout: cancel async + treat as error or partial
end
```

**Step 3:** Add `client_behaviour.ex` callback with Manual §6.1 + §6.2 citations.

**Step 4:** `mix format` on touched files. Commit `feat(sta): Client.download_file_streamed/2 with mid-stream failure capture`.

**Estimate:** 60-90 min.

---

### Task 2 — `Operations.download_with_retry/2` `:expected_size` resume state machine

**Files:**
- Modify: `sta/backend/lib/sta_connector/sta/operations.ex` (new private `do_download_with_resume/4` + dispatch in `download_with_retry/2`)
- Test: `sta/backend/test/sta_connector/sta/operations_test.exs` (new `describe "download_with_retry/2 with :expected_size resume"` block)

**Step 1 (TDD red):** 2 Bypass tests:
1. **Happy path full GET first attempt** — Bypass plug returns 200 + headers + full body. Caller passes `:expected_size`. Result: `{:ok, %DownloadResult{}}` with hash verified. Identical externally to non-resume path.
2. **Mid-stream failure → range-resume on attempt 2** — First request: Bypass sends partial chunked body + drops. Second request: Bypass asserts `Range: bytes={N}-{expected_size-1}` + `If-Match: <etag>` headers, returns 206 + remaining bytes. Operations stitches and verifies full-file X-Content-Hash. Result: `{:ok, %DownloadResult{content: full_body}}` with bytes from both attempts assembled.

**Step 2 (TDD green):** Implement per Architecture state machine. Branch in `download_with_retry/2`: if `:expected_size` opt is set, call `do_download_with_resume/4` with initial fresh state; else fall through to existing `do_download_with_retry/3`.

**Step 3:** `mix format` on touched files. Commit `feat(sta): Operations.download_with_retry/2 :expected_size resume state machine`.

**Estimate:** 60-90 min.

---

### Task 3 — Coverage for 412/416/501 + max_retries + multi-cycle resume

**Files:**
- Modify: `sta/backend/test/sta_connector/sta/operations_test.exs` (extend the resume `describe` block)

**Step 1 (TDD red+green):** 5 tests:
1. **Resume hits 412** — First attempt partial. Second attempt's `download_part` returns 412 (file changed). State resets. Third attempt full GET succeeds. Total assembled correctly.
2. **Resume hits 416** — Same shape with 416 instead of 412.
3. **Resume hits 501** — Same shape with 501 instead.
4. **max_retries exhausted with all attempts partial** — Bypass plug always sends partial+drop. After `max_retries` attempts, result is `{:error, :max_retries_exceeded_during_resume}`.
5. **Multiple resume cycles** — Bypass plug returns 30 bytes partial → 30 bytes partial → 40 bytes complete (3 attempts to assemble 100-byte file). Result: `{:ok, %DownloadResult{content: 100-byte body}}` with hash verified.

**Step 2:** Commit `test(sta): coverage for 412/416/501 resume restart + max_retries exhaustion + multi-cycle`.

**Estimate:** 30-45 min.

---

### Task 4 — Touchstone validation + handoff doc + final commit

**Steps:**
1. Run all 5 touchstone test files. Confirm:
   - 5 baseline warnings on `mix compile`
   - `client_test.exs`: 55→59 (4 new), 6 baseline failures preserved
   - `operations_test.exs`: 45→52 (7 new — 2 from Task 2 + 5 from Task 3), 0 failures
   - `outbound_file_test.exs`: 7/7 unchanged
   - `application_test.exs`: 3/3 unchanged
2. Guard rail: `git diff $(git merge-base origin/main HEAD)..HEAD --name-only | grep -v "^sta/"` returns empty.
3. Write handoff doc `sta/docs/handoff/2026-04-26-sta-onda2a-followup-2-ranged-resume.md` summarizing:
   - What landed (Task 1 + Task 2 + Task 3)
   - Touchstone state at PR creation
   - Caller-wiring still pending (out-of-scope) — `Outbound.Worker` will need to start passing `:expected_size` from `FileInfo.file_size` to enable resume in production. Future sub-PR.
   - Onda 2.A truly complete after this merges.
4. Commit `docs(sta): handoff for Onda 2.A follow-up #2 PR`.
5. Stop. **Ask user explicit auth before push + PR.**

**Estimate:** 15-20 min.

---

<a id="acceptance"></a>
## Acceptance

- 4 commits land on branch (1 docs+plan, 1 feat client, 1 feat ops, 1 test, 1 docs handoff = 5 commits actually)
- Touchstones at PR creation:
  - 5 baseline warnings (unchanged)
  - `client_test.exs`: 55→59
  - `operations_test.exs`: 45→52 (or 51→58 if we re-base after PR #36 merges, but likely won't)
  - All other touchstones unchanged
- `mix format --check-formatted` clean for touched files only
- Guard rail: zero touches outside `sta/*`
- PR body: cite this plan, `[VERIFIED §X.Y]` claims from Task 0, `[STUDY-CONJECTURE]` items from Task 0, list of new tests
- Real BCB smoke: deferred to operator post-merge (no creds in homologation testbed for streaming-failure scenarios)

---

<a id="out-of-scope"></a>
## Out of scope

Explicit cuts (called out in PR body so reviewers know):

1. **Caller wiring** — `Outbound.Worker` continues to call `download_with_retry/2` without `:expected_size`. Resume is opt-in; production callers must pass `:expected_size` (data already comes from `FileInfo.file_size` returned by `list_available_files`). Wiring becomes a separate sub-PR (could bundle with caller-wiring of `resume_upload/3` from PR #36).
2. **Parallel downloads (Manual §6.3)** — orthogonal feature. Belongs in Onda 2.B/C scope.
3. **Open-ended Range form `bytes=N-`** — Manual silent; conservative read applied (require `:expected_size`).
4. **Changes to existing `Client.download_file/2` or `Client.download_part/2`** — both stable since PR #30. New function `download_file_streamed/2` is additive.
5. **Hash mismatch recovery beyond restart** — if SHA-256(assembled) ≠ X-Content-Hash, return `{:error, :hash_mismatch_after_resume}`. No hash-mismatch retry logic — caller can decide.
6. **DB schema changes** — migration `20260426150000_add_resilient_transfer_columns` from PR #30 already covers needed columns (`etag`, `last_modified`, `bytes_uploaded`). Persistence between resume attempts is in-memory only (within a single `download_with_retry/2` call). Cross-process resume (worker crashes mid-download) would need new DB columns + caller-wiring; explicitly out of scope.

---

<a id="references"></a>
## References

- **Manual canônico:** `sta/Manual_STA_Web_Services.pdf` v1.5 jul/2022
  - §6.1 Recebimento completo
  - §6.2 Retomada do recebimento (resume) — **explicit endorsement of resume use case**
  - §6.3 Recebimento em paralelo (out of scope here)
  - §6.4 Recebimento de parte do arquivo
- **Parent plan:** `sta/docs/plans/2026-04-26-sta-conformance-rollout.md` (Pattern: Manual Verification)
- **Onda 2.A core plan:** `sta/docs/plans/2026-04-26-sta-onda2a-resilient-transfer.md` (Task 6 anticipated this follow-up in its docstring)
- **PR #30 (Onda 2.A core, MERGED):** https://github.com/FluxiqBR/monetarie/pull/30
- **PR #34 (Follow-up #1, MERGED):** https://github.com/FluxiqBR/monetarie/pull/34
- **PR #36 (Follow-up #3, OPEN):** https://github.com/FluxiqBR/monetarie/pull/36
- **RFCs referenced by Manual:**
  - RFC 7233 (HTTP Range Requests) — `Range`, `Content-Range`
  - RFC 7232 (HTTP Conditional Requests) — `If-Match`, `If-Unmodified-Since`, `Last-Modified`, `ETag`
- **Finch streaming API:** `Finch.async_request/3` + receive-loop pattern (alternative: `Finch.stream/4` doesn't expose accumulator state on `:error`, hence `async_request`)

---

**Plan version:** 1.0 (Task 0 + Tasks 1-4) · **Author:** Claude (under direction of Luiz Marcelo) · **Date:** 2026-04-26