# 100% BACEN SPB Protocol Coverage — R1/R2/R3 + COA/COD + Timeline

## Context

The SPB system has 985 message modules across 30 categories, but **ZERO R1/R2/R3 response modules** exist. The legacy partner system (EvolutionPro) shows that every sent message follows the lifecycle:

```
SENT → COA → COD → R1 (confirmed/error/rejected) → R2 (settlement) → CONFIRMED
```

Real data from partner SQL Server (50,320 COA messages, 50,320 COD, 26,724 R1, 18,390 R2) confirms this is the CORE protocol flow. Currently:

- 417 modules declare `response_types` (297 R1-only, 111 R1+R2, 9 R1+R2+R3)
- Zero R1/R2/R3 module files exist
- `@valid_states` only has 5 states (pending, processing, sent, confirmed, failed) — missing R0/R1/R2/COA/COD
- `legacy_parity.ex` has R0/R1/R2 stubs calling unimplemented private functions
- Simulator generates generic XML responses with no COA/COD or typed R1/R2
- Frontend has no message timeline/lifecycle view
- No COA/COD orchestration (GEN0014/0015/0017/0018)

**Goal**: 100% BACEN protocol compliance with full message lifecycle, all R1/R2/R3 response modules, COA/COD handling, timeline tracking, and simulator support.

---

## Phase 1: Database Schema Enhancement

### 1A. Migration: Extend `bacen_messages` table

**File**: `services/bacen_gateway/priv/repo/migrations/TIMESTAMP_add_lifecycle_columns.exs`

Add columns:
```sql
ALTER TABLE bacen_messages ADD COLUMN
  correlation_id VARCHAR(50),        -- links request/response chain
  control_number VARCHAR(23),        -- nr_ctrl_if equivalent
  nuop VARCHAR(23),                  -- BACEN operation number
  domain VARCHAR(5),                 -- SPB domain (STR, PAG, etc.)
  r0_received_at TIMESTAMP,          -- R0 ACK timestamp
  coa_received_at TIMESTAMP,         -- COA timestamp
  cod_received_at TIMESTAMP,         -- COD timestamp
  r1_received_at TIMESTAMP,          -- R1 transport ACK timestamp
  r2_confirmed_at TIMESTAMP,         -- R2 business confirmation timestamp
  r3_settled_at TIMESTAMP,           -- R3 settlement timestamp
  response_code VARCHAR(10),         -- R1/R2 response code (ACCP/RJCT)
  operation_id UUID                  -- groups related messages (request + COA + COD + R1 + R2)
```

Update `@valid_states` in `bacen_message.ex`:
```elixir
@valid_states ~w(
  created awaiting_approval sent_to_mq
  coa_received cod_received
  r1_confirmed r1_error r1_rejected
  r2_confirmed r2_rejected
  r3_settled
  cancelled expired dead_letter
  pending processing sent confirmed failed
)
```

Keep backward-compatible: old 5 states still valid.

### 1B. New table: `message_tracking`

**File**: `services/bacen_gateway/priv/repo/migrations/TIMESTAMP_create_message_tracking.exs`

Modern equivalent of legacy `spb_tb_msg_ctrl_rastreamento` (NEW names):
```sql
CREATE TABLE message_tracking (
  id BIGSERIAL PRIMARY KEY,
  operation_id UUID NOT NULL,          -- groups the full lifecycle
  message_code VARCHAR(10) NOT NULL,   -- e.g. STR0008, GEN0001R1
  integrated_at TIMESTAMP,             -- message created/received
  sent_at TIMESTAMP,                   -- sent to BACEN/MQ
  r1_response_at TIMESTAMP,            -- R1 received
  error_response_at TIMESTAMP,         -- error response received
  cancelled_at TIMESTAMP,              -- cancellation processed
  not_processed_at TIMESTAMP,          -- couldn't process
  inserted_at TIMESTAMP DEFAULT NOW()
);
CREATE INDEX idx_tracking_operation ON message_tracking(operation_id);
CREATE INDEX idx_tracking_code ON message_tracking(message_code);
```

### 1C. New table: `operation_events`

Modern equivalent of legacy `spb_tb_msg_ctrl_operacoes_status` (NEW names):
```sql
CREATE TABLE operation_events (
  id BIGSERIAL PRIMARY KEY,
  operation_id UUID NOT NULL,
  status_code INTEGER NOT NULL,
  event_tag VARCHAR(100),              -- ENV, RSPR1, RSPER, CNC, etc.
  occurred_at TIMESTAMP NOT NULL DEFAULT NOW(),
  metadata JSONB DEFAULT '{}'
);
CREATE INDEX idx_events_operation ON operation_events(operation_id);
```

### 1D. New table: `operation_status_catalog`

Modern equivalent of legacy `spb_tb_gen_status_oper` (NEW names):
```sql
CREATE TABLE operation_status_catalog (
  status_code INTEGER PRIMARY KEY,
  description VARCHAR(100) NOT NULL,
  is_r1_response BOOLEAN DEFAULT FALSE,
  requires_coa_cod BOOLEAN DEFAULT FALSE,
  tracking_action VARCHAR(10),         -- EMSG, EMQS, RSPR1, RSPER, CNC, ERR
  allows_cancellation BOOLEAN DEFAULT FALSE,
  is_final BOOLEAN DEFAULT FALSE
);
```

Seed with all 58 status codes from legacy system.

**Files to modify**:
- `services/bacen_gateway/lib/bacen_gateway/messages/bacen_message.ex` — expand `@valid_states`
- `services/bacen_gateway/lib/bacen_gateway/messages/message_state_history.ex` — expand `@valid_states`

---

## Phase 2: R1/R2/R3 Response Module Generator

### 2A. Create `ResponseHandler` behaviour

**File**: `services/bacen_gateway/lib/bacen_gateway/messages/response_handler.ex`

```elixir
defmodule BacenGateway.Messages.ResponseHandler do
  @callback message_code() :: String.t()
  @callback category() :: String.t()
  @callback namespace() :: String.t()
  @callback parent_message_code() :: String.t()
  @callback response_level() :: :r1 | :r2 | :r3
  @callback parse(String.t()) :: {:ok, map()} | {:error, term()}
  @callback validate(map()) :: {:ok, map()} | {:error, list()}
end
```

### 2B. Create Mix task to generate ALL R1/R2/R3 modules

**File**: `services/bacen_gateway/lib/mix/tasks/gen_response_modules.ex`

This Mix task:
1. Reads all 417 base modules that declare `response_types`
2. For each response type (R1, R2, R3), generates a module file
3. Each generated module implements `ResponseHandler` behaviour
4. Uses the parent module's XSD namespace pattern

**Generated module count**:
- 417 R1 modules (all that declare responses)
- 120 R2 modules (111 R1+R2 + 9 R1+R2+R3)
- 9 R3 modules
- **Total: 546 new files**

**Generated module template** (e.g. `str/str0008r1.ex`):
```elixir
defmodule BacenGateway.Messages.STR.STR0008R1 do
  @behaviour BacenGateway.Messages.ResponseHandler
  alias BacenGateway.ISO20022.Parser

  def message_code, do: "STR0008R1"
  def category, do: "STR"
  def namespace, do: "http://www.bcb.gov.br/SPB/STR0008R1.xsd"
  def parent_message_code, do: "STR0008"
  def response_level, do: :r1
  def is_response?, do: true

  def parse(xml) when is_binary(xml) do
    try do
      {:ok, doc} = Parser.parse_document(xml)
      bcmsg = Parser.get_element(doc, "//BCMSG")
      sismsg = Parser.get_element(doc, "//SISMSG/STR0008R1")
      data = %{
        identd_emissor: Parser.get_text(bcmsg, "IdentdEmissor"),
        identd_destinatario: Parser.get_text(bcmsg, "IdentdDestinatario"),
        dom_sist: Parser.get_text(bcmsg, "DomSist"),
        nu_op: Parser.get_text(bcmsg, "NUOp"),
        cod_msg: Parser.get_text(sismsg, "CodMsg"),
        sit_msg: Parser.get_text(sismsg, "SitMsg"),       -- ACCP/RJCT
        cod_erro: Parser.get_text(sismsg, "CodErro"),
        desc_sit_msg: Parser.get_text(sismsg, "DescSitMsg"),
        dt_hr_bc: Parser.get_text(sismsg, "DtHrBC"),
      }
      {:ok, data |> Enum.reject(fn {_k, v} -> is_nil(v) end) |> Map.new()}
    rescue
      e -> {:error, {:parse_error, Exception.message(e)}}
    end
  end

  def validate(data) when is_map(data) do
    errors = []
    errors = if data[:sit_msg] in ["ACCP", "RJCT", nil], do: errors,
             else: ["SitMsg must be ACCP or RJCT" | errors]
    case errors do
      [] -> {:ok, data}
      _ -> {:error, errors}
    end
  end
end
```

### 2C. Update Registry to include R1/R2/R3 modules

**File**: `services/bacen_gateway/lib/bacen_gateway/messages/registry.ex`

Add `get_response_module/1` function:
```elixir
def get_response_module(response_code) do
  # e.g. "STR0008R1" -> BacenGateway.Messages.STR.STR0008R1
  {category, _} = parse_message_code(response_code)
  module = Module.concat([BacenGateway.Messages, String.upcase(category), String.upcase(response_code)])
  if Code.ensure_loaded?(module), do: {:ok, module}, else: {:error, :not_found}
end
```

---

## Phase 3: Message Lifecycle Engine

### 3A. Complete `legacy_parity.ex` implementation

**File**: `services/bacen_gateway/lib/bacen_gateway/legacy_parity.ex`

Replace stubs with real implementations:

1. `parse_r0_response/1` — Parse R0 XML using ISO20022.Parser
2. `parse_r1_response/1` — Dispatch to specific R1 module via Registry
3. `parse_r2_response/1` — Dispatch to specific R2 module via Registry
4. `validate_correlation/2` — Match by correlation_id/nuop
5. `update_message_status/3` — Update bacen_messages + insert operation_events
6. `log_message_tracking/3` — Insert into message_tracking table

### 3B. Create `LifecycleEngine` module

**File**: `services/bacen_gateway/lib/bacen_gateway/lifecycle_engine.ex`

Central state machine that processes the full lifecycle:
```
process_outgoing(message) → CREATED → SENT_TO_MQ → (await COA) → (await COD) → (await R1) → [await R2] → CONFIRMED
process_incoming(xml) → identify type → dispatch to handler → update tracking → emit events
```

Key functions:
- `handle_coa(operation_id, coa_xml)` — mark COA received, update tracking
- `handle_cod(operation_id, cod_xml)` — mark COD received, update tracking
- `handle_r1(operation_id, r1_xml)` — parse via R1 module, check ACCP/RJCT, update state
- `handle_r2(operation_id, r2_xml)` — parse via R2 module, trigger settlement
- `get_timeline(operation_id)` — return full lifecycle events for display

### 3C. COA/COD Orchestration

**File**: `services/bacen_gateway/lib/bacen_gateway/coa_cod_handler.ex`

Handles day opening/closing protocol:
- `process_day_opening()` — Send GEN0014, await GEN0015 response
- `process_day_closing()` — Send GEN0017, await GEN0018 response
- `handle_coa_message(xml)` — Process COA for individual messages
- `handle_cod_message(xml)` — Process COD for individual messages

Based on legacy pattern: every SENT message receives COA + COD before R1.

---

## Phase 4: Backend API Endpoints

### 4A. New endpoints for timeline/lifecycle

**File**: `services/bacen_gateway/lib/bacen_gateway_web/router.ex`

Add routes:
```elixir
get "/messages/:id/timeline", MessagesController, :timeline
get "/operations/:id/lifecycle", OperationsController, :lifecycle
get "/operations/:id/events", OperationsController, :events
```

### 4B. Timeline controller actions

**File**: `services/bacen_gateway/lib/bacen_gateway_web/controllers/messages_controller.ex`

New `timeline` action:
```elixir
def timeline(conn, %{"id" => id}) do
  # Query message_tracking + operation_events for this message's operation_id
  # Return ordered list of events with timestamps
end
```

### 4C. Operation lifecycle controller

**File**: `services/bacen_gateway/lib/bacen_gateway_web/controllers/operations_controller.ex`

New `lifecycle` action returns:
```json
{
  "operation_id": "uuid",
  "message_code": "STR0008",
  "events": [
    {"event": "created", "at": "2026-02-08T08:00:00", "detail": "Message integrated"},
    {"event": "sent_to_mq", "at": "2026-02-08T08:00:01", "detail": "Sent to BACEN MQ"},
    {"event": "coa_received", "at": "2026-02-08T08:00:03", "detail": "COA from BACEN"},
    {"event": "cod_received", "at": "2026-02-08T08:00:03", "detail": "COD from BACEN"},
    {"event": "r1_confirmed", "at": "2026-02-08T08:00:05", "detail": "R1 ACCP - Accepted"}
  ],
  "current_state": "r1_confirmed",
  "elapsed_ms": 5000
}
```

---

## Phase 5: Simulator Enhancement

### 5A. Update ResponseGenerator for typed R1/R2

**File**: `simulator/lib/simulator/core/response_generator.ex`

Current behavior: generic XML for all. Change to:
1. Accept message type and generate the CORRECT R1/R2 type
2. e.g. STR0008 → generates STR0008R1.xsd XML (not generic)
3. Include proper SitMsg (ACCP/RJCT) and DtHrBC fields

### 5B. Add COA/COD response generation

**File**: `simulator/lib/simulator/core/response_generator.ex`

New functions:
- `generate_coa(request_data)` — COA XML with proper BCMSG structure
- `generate_cod(request_data)` — COD XML with proper BCMSG structure

### 5C. Update message handlers to emit COA → COD → R1 → R2 sequence

**File**: `simulator/lib/simulator/messages/str_handler.ex` (and all handlers)

Instead of single response, generate the full sequence:
```elixir
def handle(message_type, data, xml) do
  coa = ResponseGenerator.generate_coa(data)
  cod = ResponseGenerator.generate_cod(data)
  r1 = ResponseGenerator.generate_typed_r1(message_type, data)
  # If message has R2, also generate R2
  r2 = if has_r2?(message_type), do: ResponseGenerator.generate_typed_r2(message_type, data)
  {:ok, %{coa: coa, cod: cod, r1: r1, r2: r2}}
end
```

### 5D. Add scenario support

**File**: `simulator/lib/simulator/core/scenario_engine.ex`

Enhance existing scenario engine:
- `success` — Full lifecycle (COA → COD → R1 ACCP → R2 ACCP)
- `r1_reject` — COA → COD → R1 RJCT
- `r1_error` — COA → COD → R1 with error code
- `timeout` — COA → COD → no R1 (timeout scenario)
- `partial` — COA only, no COD (transport failure)

---

## Phase 6: Frontend — Message Timeline View

### 6A. New MessageTimeline component

**File**: `frontend-vue/src/components/messages/MessageTimeline.vue`

Visual vertical timeline showing:
- Each lifecycle event as a node (created → sent → COA → COD → R1 → R2)
- Color-coded: green=success, red=error, yellow=pending, grey=not-yet
- Timestamps and elapsed time between events
- Error details when R1/R2 is RJCT

### 6B. Enhance MessageDetailView

**File**: `frontend-vue/src/apps/admin/views/MessageDetailView.vue` (or create new)

Add timeline tab/section that:
1. Calls `GET /api/messages/:id/timeline`
2. Renders `MessageTimeline` component
3. Shows related messages (parent + all responses in the operation)

### 6C. Update MessagesView table

**File**: `frontend-vue/src/apps/admin/views/MessagesView.vue`

Add new columns:
- `r1_status` — ACCP/RJCT/pending chip
- `r2_status` — ACCP/RJCT/pending/N/A chip
- Clickable row → opens detail with timeline

### 6D. API service updates

**File**: `frontend-vue/src/services/api.ts`

Add to `messagesAPI`:
```typescript
getTimeline(id: number): Promise<ApiResponse>
```

Add to `operationsAPI`:
```typescript
getLifecycle(id: string): Promise<ApiResponse>
getEvents(id: string): Promise<ApiResponse>
```

---

## Execution Order

### Step 1: Database migrations + schema updates (Phase 1)
1. Create migration for bacen_messages new columns
2. Create message_tracking table
3. Create operation_events table
4. Create + seed operation_status_catalog
5. Update BacenMessage schema (`@valid_states`, new fields)
6. Update MessageStateHistory schema
7. Run `mix ecto.migrate`

### Step 2: ResponseHandler behaviour + Mix task (Phase 2)
1. Create ResponseHandler behaviour
2. Create Mix task `gen_response_modules`
3. Run Mix task to generate 546 R1/R2/R3 module files
4. Update Registry with `get_response_module/1`
5. Verify: `mix compile` — all 546 new modules compile

### Step 3: Lifecycle Engine + COA/COD (Phase 3)
1. Create LifecycleEngine module
2. Create CoaCodHandler module
3. Complete legacy_parity.ex stubs
4. Verify: `mix compile` — zero warnings

### Step 4: Backend API endpoints (Phase 4)
1. Add routes to router.ex
2. Add timeline/lifecycle/events controller actions
3. Verify: `mix compile` — zero warnings

### Step 5: Simulator enhancement (Phase 5)
1. Update ResponseGenerator with typed R1/R2 + COA/COD
2. Update all message handlers for full sequence
3. Enhance scenario engine
4. Verify: `cd simulator && mix compile` — zero warnings

### Step 6: Frontend timeline (Phase 6)
1. Create MessageTimeline.vue component
2. Add timeline endpoints to api.ts
3. Enhance MessagesView with R1/R2 status columns
4. Create/enhance message detail view with timeline tab
5. Verify: `npm run build` — zero TypeScript errors

---

## Files to Create

| File | Purpose |
|------|---------|
| `priv/repo/migrations/*_add_lifecycle_columns.exs` | Phase 1A |
| `priv/repo/migrations/*_create_message_tracking.exs` | Phase 1B |
| `priv/repo/migrations/*_create_operation_events.exs` | Phase 1C |
| `priv/repo/migrations/*_create_operation_status_catalog.exs` | Phase 1D |
| `lib/bacen_gateway/messages/response_handler.ex` | Phase 2A behaviour |
| `lib/mix/tasks/gen_response_modules.ex` | Phase 2B generator |
| **546 R1/R2/R3 modules** across 30 category dirs | Phase 2B output |
| `lib/bacen_gateway/lifecycle_engine.ex` | Phase 3B |
| `lib/bacen_gateway/coa_cod_handler.ex` | Phase 3C |
| `frontend-vue/src/components/messages/MessageTimeline.vue` | Phase 6A |

## Files to Modify

| File | Changes |
|------|---------|
| `lib/bacen_gateway/messages/bacen_message.ex` | Expand @valid_states, add new fields |
| `lib/bacen_gateway/messages/message_state_history.ex` | Expand @valid_states |
| `lib/bacen_gateway/messages/registry.ex` | Add get_response_module/1 |
| `lib/bacen_gateway/legacy_parity.ex` | Complete R0/R1/R2 stubs |
| `lib/bacen_gateway_web/router.ex` | Add timeline/lifecycle routes |
| `lib/bacen_gateway_web/controllers/messages_controller.ex` | Add timeline action |
| `lib/bacen_gateway_web/controllers/operations_controller.ex` | Add lifecycle/events actions |
| `simulator/lib/simulator/core/response_generator.ex` | Typed R1/R2 + COA/COD |
| `simulator/lib/simulator/messages/str_handler.ex` | Full sequence responses |
| `simulator/lib/simulator/messages/*.ex` | All handlers updated |
| `simulator/lib/simulator/core/scenario_engine.ex` | R1 reject/error/timeout scenarios |
| `frontend-vue/src/services/api.ts` | Timeline/lifecycle API methods |
| `frontend-vue/src/apps/admin/views/MessagesView.vue` | R1/R2 status columns |

---

## Legacy Reference (SQL Server Docker)

Connection: `docker exec spbcabin-mssql /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P 'StrongP@ssw0rd123' -d EvolutionPro -C`

| Legacy Table | Modern Equivalent | Purpose |
|---|---|---|
| `spb_tb_msg_ctrl_rastreamento` | `message_tracking` | Lifecycle timestamps |
| `spb_tb_msg_ctrl_operacoes` | `bacen_messages` (enhanced) | Message operations |
| `spb_tb_msg_ctrl_operacoes_status` | `operation_events` | Status history |
| `spb_tb_gen_status_oper` | `operation_status_catalog` | Status definitions |
| `spb_tb_gen_grade_horario` | `bacen_operating_schedule` (exists) | Operating hours |

---

## Verification

1. **Backend compiles**: `cd services/bacen_gateway && mix compile` — zero warnings
2. **Migrations run**: `mix ecto.migrate` — all tables created
3. **546 R1/R2/R3 modules exist**: `find lib/bacen_gateway/messages -name "*r1.ex" -o -name "*r2.ex" -o -name "*r3.ex" | wc -l` → 546
4. **Registry resolves responses**: `BacenGateway.Messages.Registry.get_response_module("STR0008R1")` → `{:ok, STR0008R1}`
5. **Simulator compiles**: `cd simulator && mix compile` — zero warnings
6. **Frontend builds**: `cd frontend-vue && npm run build` — zero TypeScript errors
7. **Timeline API works**: `curl /api/messages/1/timeline` → returns lifecycle events
8. **COA/COD flow**: Simulator returns COA+COD+R1 sequence for any sent message