# Manutencao

Tarefas de manutencao rotineira para a plataforma Monetarie PIX, incluindo banco de dados, aplicacao, NATS, Redis e rotacao de certificados.

## Pre-requisitos

- Acesso kubectl ao cluster Kubernetes (namespace `pix`)
- Acesso psql ao PostgreSQL com privilegios de superusuario
- Acesso redis-cli ao Redis
- Acesso nats CLI ao cluster NATS
- Helm instalado para gerenciamento de releases (se aplicavel)
- Janela de manutencao aprovada para operacoes que impactam disponibilidade

## Calendario de Manutencao

| Tarefa | Frequencia | Janela | Impacto |
|--------|-----------|--------|---------|
| VACUUM ANALYZE | Semanal (domingo 03:00 UTC) | 30 min | Nenhum (nao-bloqueante) |
| REINDEX CONCURRENTLY | Mensal (1o domingo 04:00 UTC) | 60 min | Nenhum (concorrente) |
| Verificacao de particoes | Diario (automatico — PartitionManager) | 5 min | Nenhum |
| Purga de auditoria | Diario (automatico — PartitionManager) | 10 min | Nenhum |
| Atualizacao de imagem Docker | Sob demanda | 15 min | Rolling update |
| Rotacao de certificados | 90 dias antes do vencimento | 30 min | Nenhum (zero-downtime) |
| Compactacao de streams NATS | Mensal | 15 min | Nenhum |
| Monitoramento de memoria Redis | Semanal | 5 min | Nenhum |

## Manutencao do Banco de Dados

### Migrations

Executar migrations pendentes apos deploy:

```bash
# Via Kubernetes (producao)
kubectl exec deployment/pix-backend -n pix -- \
  bin/monetarie_pix eval "Shared.Release.migrate()"

# Saida esperada:
# 14:30:00.123 [info] == Running 20260210000001 Shared.Repo.Migrations.FixPartitionedTables.change/0
# 14:30:00.456 [info] == Migrated 20260210000001 in 0.3s

# Via mix (desenvolvimento local)
cd /monetarie/pix/backend && mix ecto.migrate

# Verificar status das migrations
kubectl exec deployment/pix-backend -n pix -- \
  bin/monetarie_pix eval "Shared.Release.migrations()"
```

::: warning Repos em modo eval
O modo `eval` inicia uma instancia BEAM separada. Se o Repo nao auto-iniciar, use `Shared.Release.migrate()` que cuida da inicializacao. Nunca use `mix ecto.migrate` em producao — o mix nao esta disponivel no container release.
:::

### VACUUM e ANALYZE

Executar VACUUM ANALYZE nas tabelas grandes para manter estatisticas atualizadas:

```bash
# VACUUM ANALYZE nas tabelas mais acessadas
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  VACUUM ANALYZE monetarie_spi.messages;
  VACUUM ANALYZE monetarie_spi.payments;
  VACUUM ANALYZE monetarie_spi.message_history;
  VACUUM ANALYZE monetarie_dict.keys;
  VACUUM ANALYZE monetarie_dict.operations;
  VACUUM ANALYZE monetarie_auth.activity_log;
  VACUUM ANALYZE monetarie_auth.login_history;
"

# Saida esperada:
# VACUUM
# VACUUM
# ...

# Verificar ultima execucao do autovacuum
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  SELECT schemaname, relname, last_vacuum, last_autovacuum, last_analyze, n_dead_tup
  FROM pg_stat_user_tables
  WHERE schemaname LIKE 'monetarie_%'
  ORDER BY n_dead_tup DESC
  LIMIT 10;
"
```

Script automatizado:

```bash
#!/bin/bash
# /opt/monetarie/scripts/vacuum-analyze.sh
# Executar: 0 3 * * 0 (domingo 03:00 UTC)

set -euo pipefail

TABLES=(
  "monetarie_spi.messages"
  "monetarie_spi.payments"
  "monetarie_spi.message_history"
  "monetarie_spi.balance_history"
  "monetarie_dict.keys"
  "monetarie_dict.operations"
  "monetarie_auth.activity_log"
  "monetarie_auth.login_history"
  "monetarie_audit.xml_audit_logs"
  "monetarie_audit.bacen_api_validations"
)

for TABLE in "${TABLES[@]}"; do
  echo "[$(date)] VACUUM ANALYZE ${TABLE}..."
  psql -h 10.140.241.2 -U postgres -d monetarie -c "VACUUM ANALYZE ${TABLE};"
done

echo "[$(date)] VACUUM ANALYZE concluido"
```

### Gerenciamento de Particoes

O `Shared.PartitionManager` GenServer executa diariamente e:

1. Cria particoes para os proximos 60 dias
2. Purga registros de auditoria expirados
3. Loga erros com `try/rescue` para resiliencia

Tabelas particionadas:

| Tabela | Tipo de Particao | Coluna | Frequencia |
|--------|-----------------|--------|-----------|
| `monetarie_auth.activity_log` | Range (mensal) | `created_at` | Mensal |
| `monetarie_auth.login_history` | Range (mensal) | `login_at` | Mensal |

Verificar particoes existentes:

```bash
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  SELECT parent.relname AS parent, child.relname AS partition,
         pg_get_expr(child.relpartbound, child.oid) AS bounds
  FROM pg_inherits
  JOIN pg_class parent ON parent.oid = pg_inherits.inhparent
  JOIN pg_class child ON child.oid = pg_inherits.inhrelid
  WHERE parent.relname IN ('activity_log', 'login_history')
  ORDER BY parent.relname, child.relname;
"

# Saida esperada:
#     parent       |          partition          |                     bounds
# -----------------+----------------------------+------------------------------------------------
#  activity_log    | activity_log_2026_01       | FOR VALUES FROM ('2026-01-01') TO ('2026-02-01')
#  activity_log    | activity_log_2026_02       | FOR VALUES FROM ('2026-02-01') TO ('2026-03-01')
#  activity_log    | activity_log_2026_03       | FOR VALUES FROM ('2026-03-01') TO ('2026-04-01')
#  activity_log    | activity_log_2026_04       | FOR VALUES FROM ('2026-04-01') TO ('2026-05-01')
#  activity_log    | activity_log_default       | DEFAULT
#  login_history   | login_history_2026_01      | FOR VALUES FROM ('2026-01-01') TO ('2026-02-01')
#  ...
```

Criar particao manualmente (se o PartitionManager falhar):

```bash
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  CREATE TABLE IF NOT EXISTS monetarie_auth.activity_log_2026_05
  PARTITION OF monetarie_auth.activity_log
  FOR VALUES FROM ('2026-05-01') TO ('2026-06-01');
"
```

### Manutencao de Indexes

Reindexar indexes fragmentados (operacao concorrente, sem bloqueio):

```bash
# Verificar tamanho e fragmentacao dos indexes
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  SELECT
    schemaname,
    indexrelname,
    pg_size_pretty(pg_relation_size(indexrelid)) AS index_size,
    idx_scan,
    idx_tup_read
  FROM pg_stat_user_indexes
  WHERE schemaname LIKE 'monetarie_%'
  ORDER BY pg_relation_size(indexrelid) DESC
  LIMIT 15;
"

# Reindexar concorrentemente (sem lock)
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  REINDEX INDEX CONCURRENTLY monetarie_spi.idx_messages_status_created;
  REINDEX INDEX CONCURRENTLY monetarie_spi.idx_messages_e2e_id_trgm;
  REINDEX INDEX CONCURRENTLY monetarie_spi.idx_messages_branch_id;
  REINDEX INDEX CONCURRENTLY monetarie_spi.idx_messages_cpf_cnpj;
"
```

Indexes TPS criticos (8 indexes da migration 23):

| Index | Tabela | Colunas | Tipo |
|-------|--------|---------|------|
| idx_messages_branch_id | messages | branch_id | B-tree |
| idx_messages_branch_status | messages | branch_id, status_id | B-tree composto |
| idx_messages_cpf_cnpj | messages | debtor_cpf_cnpj, creditor_cpf_cnpj | B-tree |
| idx_messages_e2e_id_trgm | messages | (end_to_end_id::text) | GIN trigram |
| idx_message_history_msg_id | message_history | message_id | B-tree |
| idx_balance_history_account | balance_history | account_id, created_at | B-tree composto |
| idx_dict_keys_value | keys | key_value | B-tree |
| idx_dict_keys_owner | keys | owner_ispb, owner_cpf_cnpj | B-tree composto |

### Purga de Dados de Auditoria

O PartitionManager executa purga automatica, mas para purga manual:

```bash
# Verificar volume de dados de auditoria
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  SELECT
    'xml_audit_logs' AS table,
    COUNT(*) AS rows,
    pg_size_pretty(pg_total_relation_size('monetarie_audit.xml_audit_logs')) AS size
  UNION ALL
  SELECT
    'bacen_api_validations',
    COUNT(*),
    pg_size_pretty(pg_total_relation_size('monetarie_audit.bacen_api_validations'))
  FROM monetarie_audit.bacen_api_validations;
"

# Purga de registros ICOM antigos (>10 anos, conforme regulamentacao BACEN)
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  DELETE FROM monetarie_audit.xml_audit_logs
  WHERE created_at < NOW() - INTERVAL '10 years'
  AND message_type LIKE 'ICOM%';
"

# Purga de registros DICT reads (>2 anos)
psql -h 10.140.241.2 -U postgres -d monetarie -c "
  DELETE FROM monetarie_audit.xml_audit_logs
  WHERE created_at < NOW() - INTERVAL '2 years'
  AND message_type LIKE 'DICT%'
  AND operation = 'READ';
"
```

## Manutencao da Aplicacao

### Rolling Update (Zero-Downtime)

Deploy de nova versao do backend:

```bash
# 1. Build da imagem
SHORT_SHA=$(git rev-parse --short HEAD)
gcloud builds submit --config=backend/cloudbuild.yaml \
  --substitutions=SHORT_SHA=${SHORT_SHA} \
  --machine-type=E2_HIGHCPU_8 --region=southamerica-east1 backend/

# 2. Atualizar imagem (se cloudbuild.yaml nao faz auto-deploy)
kubectl set image deployment/pix-backend -n pix \
  pix-backend=southamerica-east1-docker.pkg.dev/fluxiqbr/monetarie/pix-backend:${SHORT_SHA}

# 3. Acompanhar rollout
kubectl rollout status deployment/pix-backend -n pix --timeout=300s
# Saida esperada:
# Waiting for deployment "pix-backend" rollout to finish: 1 old replicas are pending termination...
# deployment "pix-backend" successfully rolled out

# 4. Verificar pods
kubectl get pods -n pix -l app=pix-backend
```

Estrategia de rollout configurada:

```yaml
# deploy/backend.yaml
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0     # Nunca ter pod indisponivel
      maxSurge: 1           # Criar 1 pod novo antes de remover antigo
  # Graceful shutdown
  terminationGracePeriodSeconds: 60
```

### Rollback

```bash
# Verificar historico de rollouts
kubectl rollout history deployment/pix-backend -n pix
# Saida esperada:
# REVISION  CHANGE-CAUSE
# 1         <none>
# 2         <none>
# 3         <none>

# Rollback para revisao anterior
kubectl rollout undo deployment/pix-backend -n pix

# Rollback para revisao especifica
kubectl rollout undo deployment/pix-backend -n pix --to-revision=2

# Acompanhar rollback
kubectl rollout status deployment/pix-backend -n pix
```

### Atualizacao do Frontend

```bash
# Admin frontend
SHORT_SHA=$(git rev-parse --short HEAD)
gcloud builds submit --config=frontend/admin/cloudbuild.yaml \
  --substitutions=SHORT_SHA=${SHORT_SHA} \
  --machine-type=E2_HIGHCPU_8 --region=southamerica-east1 frontend/admin/

# User frontend
gcloud builds submit --config=frontend/user/cloudbuild.yaml \
  --substitutions=SHORT_SHA=${SHORT_SHA} \
  --machine-type=E2_HIGHCPU_8 --region=southamerica-east1 frontend/user/

# Verificar deploy
kubectl get pods -n pix -l app=pix-admin-frontend
kubectl get pods -n pix -l app=pix-user-frontend
```

### Migrations Pos-Deploy

Apos deploy de nova versao com migrations:

```bash
# Executar migrations
kubectl exec deployment/pix-backend -n pix -- \
  bin/monetarie_pix eval "Shared.Release.migrate()"

# Verificar que todas as migrations foram aplicadas
kubectl exec deployment/pix-backend -n pix -- \
  bin/monetarie_pix eval "
    Application.ensure_all_started(:ecto_sql)
    Shared.Repo.start_link()
    Ecto.Migrator.migrations(Shared.Repo, Shared.Repo.config()[:priv] <> \"/migrations\")
    |> Enum.each(fn {status, version, name} ->
      IO.puts(\"#{status} #{version} #{name}\")
    end)
  "
```

## Manutencao NATS

### Compactacao de Streams

Forcar compactacao para liberar espaco em disco:

```bash
# Verificar uso atual de cada stream
nats stream list --server=nats://10.10.40.5:4222

# Purgar mensagens antigas de uma stream (respeitando retencao)
nats stream purge MONETARIE_SPI \
  --keep=100000 \
  --server=nats://10.10.40.5:4222
# Saida esperada:
# Purged 32,450 messages from stream MONETARIE_SPI

# Verificar uso de disco NATS
nats server report jetstream --server=nats://10.10.40.5:4222
```

### Monitoramento de Consumer Lag

```bash
# Script para verificar lag de todos os consumers
for STREAM in MONETARIE_SPI MONETARIE_DICT MONETARIE_SETTLEMENT MONETARIE_CORE; do
  echo "=== ${STREAM} ==="
  nats consumer list ${STREAM} --server=nats://10.10.40.5:4222 2>/dev/null | while read CONSUMER; do
    PENDING=$(nats consumer info ${STREAM} ${CONSUMER} --server=nats://10.10.40.5:4222 2>/dev/null | grep "Pending" | awk '{print $NF}')
    echo "  ${CONSUMER}: ${PENDING:-N/A} pendentes"
  done
done
```

### Substituicao de No NATS

```bash
# 1. Verificar cluster atual
nats server list --server=nats://10.10.40.5:4222

# 2. Adicionar novo no ao cluster
# Configurar nats-server.conf no novo no com routes para os outros 2

# 3. Verificar que o cluster tem 3+ nos
nats server list --server=nats://10.10.40.5:4222
# Deve mostrar 3+ servidores

# 4. Remover no antigo (graceful shutdown)
ssh nats-dev-old "systemctl stop nats-server"

# 5. Verificar integridade das streams
nats stream list --server=nats://10.10.40.5:4222
```

## Manutencao Redis

### Monitoramento de Memoria

```bash
# Verificar uso de memoria
redis-cli -h 10.140.240.4 INFO memory
# Procurar por:
# used_memory_human: 245.67M
# used_memory_peak_human: 312.45M
# maxmemory_human: 1.00G
# maxmemory_policy: allkeys-lru

# Verificar distribuicao de chaves
redis-cli -h 10.140.240.4 INFO keyspace
# Saida esperada:
# db0:keys=12450,expires=8900,avg_ttl=3600000

# Verificar chaves com maior consumo (scan, nao-bloqueante)
redis-cli -h 10.140.240.4 --bigkeys
```

### Verificacao de Expiracao de Chaves

```bash
# Verificar chaves de blacklist (devem ter TTL)
redis-cli -h 10.140.240.4 KEYS "blacklist:*" | head -5
redis-cli -h 10.140.240.4 TTL "blacklist:<primeiro_resultado>"
# Saida: (numero positivo em segundos)

# Verificar chaves de rate limit
redis-cli -h 10.140.240.4 KEYS "rate_limit:*" | wc -l
# Deve ser proporcional ao numero de clientes ativos

# Verificar chaves de idempotency (TTL = 24h = 86400s)
redis-cli -h 10.140.240.4 KEYS "idempotency:*" | wc -l
```

### Substituicao de No Redis

Para modo standalone:

```bash
# 1. Criar snapshot
redis-cli -h 10.140.240.4 BGSAVE

# 2. Copiar RDB para novo servidor
scp old-redis:/var/lib/redis/dump.rdb new-redis:/var/lib/redis/dump.rdb

# 3. Iniciar Redis no novo servidor
ssh new-redis "systemctl start redis"

# 4. Atualizar REDIS_HOST no Kubernetes
kubectl set env deployment/pix-backend -n pix REDIS_HOST=<novo_ip>

# 5. Reiniciar pods
kubectl rollout restart deployment/pix-backend -n pix
```

## Rotacao de Certificados (Zero-Downtime)

### Verificar Validade dos Certificados

```bash
# Script de verificacao
#!/bin/bash
echo "=== Certificados ICP-Brasil ==="

for CERT in /opt/monetarie/certs/*.pem; do
  EXPIRY=$(openssl x509 -in "$CERT" -noout -enddate | cut -d= -f2)
  DAYS_LEFT=$(( ($(date -d "$EXPIRY" +%s) - $(date +%s)) / 86400 ))

  if [ $DAYS_LEFT -lt 90 ]; then
    echo "AVISO: ${CERT} expira em ${DAYS_LEFT} dias (${EXPIRY})"
  else
    echo "OK: ${CERT} expira em ${DAYS_LEFT} dias (${EXPIRY})"
  fi
done
```

### Procedimento de Rotacao

```bash
# 1. Obter novo certificado ICP-Brasil (CPIC/CPIA)

# 2. Atualizar secret Kubernetes com novos certificados
kubectl create secret generic bacen-certs -n pix \
  --from-file=cpic.pem=/opt/monetarie/certs/new/cpic.pem \
  --from-file=cpic.key=/opt/monetarie/certs/new/cpic.key \
  --from-file=cpia.pem=/opt/monetarie/certs/new/cpia.pem \
  --from-file=cpia.key=/opt/monetarie/certs/new/cpia.key \
  --from-file=ca-chain.pem=/opt/monetarie/certs/new/ca-chain.pem \
  --dry-run=client -o yaml | kubectl apply -f -

# 3. O CertificatePool GenServer recarrega automaticamente a cada 5 minutos
# Ou forcar recarga reiniciando pods:
kubectl rollout restart deployment/pix-backend -n pix

# 4. Verificar que o novo certificado esta em uso
kubectl logs deployment/pix-backend -n pix | grep "CertificatePool"
# Saida esperada:
# [CertificatePool] Loaded 3 certificates, next refresh in 300s

# 5. Testar comunicacao BACEN
curl -s https://pixapi-dev.fluxiq.com.br/api/v1/simulator/status | jq .
```

### TLS Ingress

Renovar certificado TLS do ingress:

```bash
# Atualizar secret TLS
kubectl create secret tls monetarie-dev-tls -n pix \
  --cert=/opt/monetarie/certs/tls/fullchain.pem \
  --key=/opt/monetarie/certs/tls/privkey.pem \
  --dry-run=client -o yaml | kubectl apply -f -

# Traefik detecta automaticamente a mudanca no secret
# Verificar
curl -v https://pixapi-dev.fluxiq.com.br/health 2>&1 | grep "expire date"
```

## Checklist de Manutencao Semanal

```markdown
- [ ] Verificar health check: `curl https://pixapi-dev.fluxiq.com.br/health`
- [ ] Verificar logs de erro: `kubectl logs deployment/pix-backend -n pix | grep error | tail -20`
- [ ] Verificar consumer lag NATS: `nats consumer info MONETARIE_SPI inbound-processor`
- [ ] Verificar memoria Redis: `redis-cli INFO memory | grep used_memory_human`
- [ ] Verificar uso de disco DB: `SELECT pg_size_pretty(pg_database_size('monetarie'));`
- [ ] Verificar validade dos certificados
- [ ] Verificar backups: existencia e integridade do ultimo backup
- [ ] Verificar metricas no Grafana: error rate, latencia P95, worker throughput
```

## Resultado Esperado

Ao seguir este guia de manutencao, voce mantera:

- Banco de dados otimizado com VACUUM semanal e REINDEX mensal
- Particoes criadas automaticamente para os proximos 60 dias
- Aplicacao atualizada via rolling updates sem downtime
- Streams NATS compactados e consumers sem lag excessivo
- Redis com memoria monitorada e chaves com TTL correto
- Certificados ICP-Brasil renovados antes do vencimento
- Procedimentos de rollback testados e documentados