# Deployment Notes

## Docker Image Requirements

### Messenger Worker
- Symfony Messenger (`symfony/messenger` + `symfony/doctrine-messenger`) handles async OCR
- Worker command: `php bin/console messenger:consume async -vv`
- Needs a separate container/process for the worker (supervisor or entrypoint script)
- Transport: Doctrine (MariaDB) — table `messenger_messages` must exist
- Create table: `php bin/console messenger:setup-transports` (fails on MariaDB 10.4 — use manual SQL, see below)

### Manual messenger table (MariaDB 10.4 workaround)
```sql
CREATE TABLE IF NOT EXISTS messenger_messages (
    id BIGINT AUTO_INCREMENT NOT NULL,
    body LONGTEXT NOT NULL,
    headers LONGTEXT NOT NULL,
    queue_name VARCHAR(190) NOT NULL,
    created_at DATETIME NOT NULL COMMENT '(DC2Type:datetime_immutable)',
    available_at DATETIME NOT NULL COMMENT '(DC2Type:datetime_immutable)',
    delivered_at DATETIME DEFAULT NULL COMMENT '(DC2Type:datetime_immutable)',
    PRIMARY KEY (id),
    INDEX IDX_75EA56E0FB7336F0 (queue_name),
    INDEX IDX_75EA56E0E3BD61CE (available_at),
    INDEX IDX_75EA56E016BA31DB (delivered_at)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE = InnoDB;
```

### Environment Variables (new)
```
MISTRAL_OCR_ENDPOINT=
MISTRAL_OCR_API_KEY=
MISTRAL_OCR_MODEL=mistral-document-ai-2505
```

### File Uploads
- Path: `public/uploads/references/{username}/`
- Must be a persistent volume, not ephemeral container storage
- Permissions: writable by www-data

### MariaDB 10.4.27 Caveats
- `doctrine:migrations:diff` / `make:migration` broken — use `dbal:run-sql` for schema changes
- `messenger:setup-transports` fails — use manual SQL above

### Elasticsearch: Reference Chunks Index
- Index name: `user_reference_chunks`
- Mapping file: `shamra_integration/reference_chunks_mapping.json`
- Create index: `php bin/console app:process-reference-ocr` (or run the PHP script in `shamra_integration/create_index.php`)
- Or manually via curl:
```bash
curl -X PUT "localhost:9200/user_reference_chunks" \
  -H "Content-Type: application/json" \
  -d @shamra_integration/reference_chunks_mapping.json
```
- Uses Arabic text analyzer with stemming (`arabic_normalization` + `arabic_stemmer`)
- Chunks are created automatically after OCR completes (chained via Messenger)
- Pipeline: Upload → OCR (async) → Chunking + ES Indexing (async) → `is_indexed = 1`

### Pending Migrations (if not applied)
- `deleted_at` column: `ALTER TABLE user_reference ADD deleted_at DATETIME DEFAULT NULL COMMENT '(DC2Type:datetime_immutable)';`
- `is_indexed` column: `ALTER TABLE user_reference ADD is_indexed TINYINT(1) NOT NULL DEFAULT 0;`

### Console Commands
- `app:process-reference-ocr --pending` — batch OCR for any missed references (cron fallback)
- `app:reset-playground-credits` — monthly credit reset (already exists)

### Packages Added This Session
- `symfony/messenger ^8.0`
- `symfony/doctrine-messenger ^8.0`

### Messenger Worker — Keeping It Running

The Messenger worker **must** be running at all times for async jobs (OCR, indexing, summarization, translation) to be processed. Without it, messages pile up in `messenger_messages` and the UI shows "Processing..." / "Translating..." indefinitely.

#### Quick test (foreground)
```bash
php bin/console messenger:consume async -vv
```

#### Background (temporary, dies on reboot)
```bash
nohup php bin/console messenger:consume async -vv --time-limit=3600 > var/log/messenger_worker.log 2>&1 &
```

#### Recommended: systemd service (persistent)

Create `/etc/systemd/system/shamra-messenger-worker.service`:

```ini
[Unit]
Description=Shamra Academia Messenger Worker
After=network.target mariadb.service

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/academia_v2
ExecStart=/usr/bin/php bin/console messenger:consume async --time-limit=3600 --memory-limit=256M -vv
Restart=always
RestartSec=5
# Gracefully stop worker before restart
ExecReload=/bin/kill -s TERM $MAINPID

# Environment (adjust if needed)
Environment=APP_ENV=prod

# Logging
StandardOutput=append:/var/www/academia_v2/var/log/messenger_worker.log
StandardError=append:/var/www/academia_v2/var/log/messenger_worker.log

[Install]
WantedBy=multi-user.target
```

Then enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable shamra-messenger-worker
sudo systemctl start shamra-messenger-worker

# Check status
sudo systemctl status shamra-messenger-worker

# View logs
sudo journalctl -u shamra-messenger-worker -f
# or
tail -f /var/www/academia_v2/var/log/messenger_worker.log
```

**Key points:**
- `--time-limit=3600` makes the worker restart every hour (picks up code changes, prevents memory leaks)
- `Restart=always` + `RestartSec=5` ensures it comes back automatically after timeout or crash
- For higher throughput, run multiple workers by duplicating the service file or using a template (`shamra-messenger-worker@.service`)
- After deploying new code, restart the worker: `sudo systemctl restart shamra-messenger-worker`

#### Debugging stuck messages
If a worker dies mid-processing, messages get stuck with `delivered_at` set:
```sql
-- Check for stuck messages
SELECT id, queue_name, created_at, delivered_at FROM messenger_messages WHERE delivered_at IS NOT NULL;

-- Unlock stuck messages (worker died before finishing)
UPDATE messenger_messages SET delivered_at = NULL WHERE delivered_at IS NOT NULL;
```