# Email System Documentation

## Overview

Shamra Academia sends emails through two separate channels:

| Channel | Provider | Purpose |
|---------|----------|---------|
| **Transactional Emails** | **Mailgun** (via Symfony Mailer) | Password reset, email verification, partner notifications, contact replies |
| **Marketing / Bulk Emails** | **Ecomail** (via `MailSystem` interface) | Newsletter campaigns, mailing list management, subscriber segmentation |

### Historical Note
- **SendGrid** was previously used for marketing emails — the `SendGrid.php` service and related entities (`SendGridList`, `SendGridRecipients`) still exist in the codebase but are **not active**. The `MailSystem` interface is currently aliased to `Ecomail` in `config/services.yaml`.
- **Gmail SMTP** is used in the dev environment (`.env.dev`).

---

## Configuration

### Mailer DSN (Transactional)

| Environment | DSN | File |
|-------------|-----|------|
| **Production** | `mailgun+https://${MAILGUN_API_KEY}:shamra-academia.com@default` | `.env.local` |
| **Dev/Test** | `gmail+smtp://...@default` | `.env.dev` |

Configured in `config/packages/mailer.yaml`:
```yaml
framework:
    mailer:
        dsn: '%env(MAILER_DSN)%'
```

### Ecomail (Marketing)

| Parameter | Source |
|-----------|--------|
| `ECOMAIL_API_KEY` | `.env.local` |

Configured in `config/packages/framework.yaml` and bound in `config/services.yaml`:
```yaml
# config/services.yaml
App\Service\MailSystem\MailSystem:
    class: App\Service\MailSystem\Ecomail
```

### Sender Address

All transactional emails are sent from:
```
From: info@shamra-academia.com
Name: شمرا - أكاديميا
```

---

## Email Events / Cases

### 1. Account Email Verification (Registration)

| Field | Value |
|-------|-------|
| **When** | User registers a new account |
| **Recipient** | The new user |
| **Subject** | `الرجاء تاكيد البريد الالكتروني` (Please confirm your email) |
| **Template** | `templates/registration/confirmation_email.html.twig` |
| **Controller** | `RegistrationController::register()` |
| **Service** | `EmailVerifier::sendEmailConfirmation()` |
| **Context** | `signedUrl`, `expiresAtMessageKey`, `expiresAtMessageData`, `username`, `firstName`, `lastName`, `country` |
| **Route** | `POST /register` |

The email contains a signed verification URL that the user clicks to confirm their email. The `VerifyEmailHelperDecorater` generates the signed URL with a time-limited token.

### 2. Resend Email Verification

| Field | Value |
|-------|-------|
| **When** | Authenticated user requests re-verification |
| **Recipient** | The logged-in user |
| **Subject** | `الرجاء تاكيد البريد الالكتروني` |
| **Template** | `templates/registration/confirmation_email.html.twig` |
| **Controller** | `RegistrationController::reSendVerifyUserEmail()` |
| **Service** | `EmailVerifier::sendEmailConfirmation()` |
| **Route** | `GET /resend/verify/email/{username}` |

Same template as case #1, triggered from the user profile page.

### 3. Facebook Registration — Welcome Email (No Confirmation)

| Field | Value |
|-------|-------|
| **When** | New user registers via Facebook OAuth |
| **Recipient** | The new Facebook-linked user |
| **Subject** | `مرحباً بك في شمرا أكاديميا! اكتشف أدوات الذكاء الاصطناعي` |
| **Template** | `templates/emails/welcome.html.twig` (body from DB `email_template` table) |
| **Controller** | `FacebookController::forceForm()` → `sendWelcomeEmail()` |
| **Tracking** | `EmailTrackingService` — open pixel + click tracking |
| **Placeholders** | `{{firstName}}`, `{{lastName}}`, `{{playgroundUrl}}` |

Facebook already verifies the user's email, so **no confirmation email is sent** — only the welcome/playground promotion email. The `sendWelcomeEmail()` private method checks if the `welcome_email` template is active before sending.

### 3b. Google Registration — Welcome Email (No Confirmation)

| Field | Value |
|-------|-------|
| **When** | New user registers via Google OAuth |
| **Recipient** | The new Google-linked user |
| **Subject** | `مرحباً بك في شمرا أكاديميا! اكتشف أدوات الذكاء الاصطناعي` |
| **Template** | `templates/emails/welcome.html.twig` (body from DB `email_template` table) |
| **Controller** | `GoogleController::forceForm()` → `sendWelcomeEmail()` |
| **Tracking** | `EmailTrackingService` — open pixel + click tracking |
| **Placeholders** | `{{firstName}}`, `{{lastName}}`, `{{playgroundUrl}}` |

Same as Facebook — Google pre-verifies emails, so only the welcome email is sent.

### 4. Password Reset

| Field | Value |
|-------|-------|
| **When** | User requests a password reset |
| **Recipient** | The user who forgot their password |
| **Subject** | `تغير كلمة السر` (Change Password) |
| **Template** | `templates/reset_password/email.html.twig` |
| **Controller** | `ResetPasswordController::processSendingPasswordResetEmail()` |
| **Context** | `resetToken`, `firstName`, `lastName`, `country` |
| **Route** | `POST /reset-password` |

Uses the `SymfonyCasts\ResetPassword` bundle. The email contains a signed reset link (`/reset-password/reset/{token}`). The token is validated with `ResetPasswordHelperInterface`. For security, the endpoint does not reveal whether the email exists.

### 5. Partner Program — Join Request (Internal Notification)

| Field | Value |
|-------|-------|
| **When** | User applies to join the partner program |
| **Recipient** | `info@shamra-academia.com` (Shamra team) |
| **Subject** | `طلب انضمام الى اتفاقية الشركاء` (Partner join request) |
| **Template** | `templates/Profile/joinEmail.html.twig` |
| **Controller** | `ProfileController::joinPartner()` |
| **Context** | `name`, `userEmail`, `country` |
| **Route** | `POST /joinPartner` |

This is an **internal notification** — the email goes to the Shamra team, not to the user.

### 6. Partner Program — Join Response (Admin → User)

| Field | Value |
|-------|-------|
| **When** | Admin approves/rejects a partner application |
| **Recipient** | The applicant user |
| **Subject** | `الرد على طلب الانضمام الى اتفاقية الشركاء` (Partner join response) |
| **Template** | `templates/Profile/responeJoinEmail.html.twig` |
| **Controller** | `ControlPanelController::changePartnerStatus()` |
| **Context** | `status`, `message`, `name` |

Sent from the admin control panel when a partner application is approved or rejected.

### 7. Partner Content — Submission Notification (Internal)

| Field | Value |
|-------|-------|
| **When** | Partner submits new content for review |
| **Recipient** | `info@shamra-academia.com` (Shamra team) |
| **Subject** | `طلب إضافة محتوى` (Content submission request) |
| **Template** | `templates/partner_content/addContentEmail.html.twig` |
| **Controller** | `PartnerContentController::newAction()` |
| **Context** | `name`, `userEmail` |
| **Route** | `POST /partner-content/add` |

### 8. Partner Content — Submission Confirmation (Author)

| Field | Value |
|-------|-------|
| **When** | Partner submits new content (same event as #7) |
| **Recipient** | The content author |
| **Subject** | `إرسال ملف بنجاح` (File sent successfully) |
| **Template** | `templates/partner_content/responeAddContentEmail.html.twig` |
| **Controller** | `PartnerContentController::newAction()` |

Both emails (#7 and #8) are sent together when a partner submits content.

### 9. Partner Content — Review Result (Admin → Author)

| Field | Value |
|-------|-------|
| **When** | Admin reviews (accepts/rejects) partner content |
| **Recipient** | The content author |
| **Subject** | `مراجعة المحتوى` (Content review) |
| **Template** | `templates/partner_content/acceptContentEmail.html.twig` |
| **Controller** | `ControlPanelController::contentReview()` |
| **Context** | `status`, `title`, `feedback` |

### 10. Contact Us — Reply (Admin → User)

| Field | Value |
|-------|-------|
| **When** | Admin replies to a contact form enquiry |
| **Recipient** | The user who submitted the enquiry |
| **Subject** | `شكراً لك على تواصلك مع شمرا أكاديميا` (Thank you for contacting us) |
| **Template** | `src/syndex/ContactUsBundle/Resources/views/Page/email.html.twig` |
| **Controller** | `ContactController::replyAction()` |
| **Context** | `title`, `reply` |

### 11. Welcome Email (Playground Promotion) — with Open/Click Tracking

| Field | Value |
|-------|-------|
| **When** | User registers a new account via any method (if template is active) |
| **Recipient** | The new user |
| **Subject** | `مرحباً بك في شمرا أكاديميا! اكتشف أدوات الذكاء الاصطناعي` |
| **Template** | `templates/emails/welcome.html.twig` (ZURB Ink wrapper) |
| **Body** | Stored in DB (`email_template` table, `name='welcome_email'`) — admin-editable via dashboard |
| **Controllers** | `RegistrationController::register()`, `FacebookController::sendWelcomeEmail()`, `GoogleController::sendWelcomeEmail()` |
| **Tracking** | `EmailTrackingService` creates a tracker record; links are wrapped for click tracking; a 1×1 pixel is embedded for open tracking |
| **Placeholders** | `{{firstName}}`, `{{lastName}}`, `{{playgroundUrl}}` |
| **Admin UI** | Dashboard → Welcome Email tab: toggle active, edit subject/body (TinyMCE WYSIWYG), preview, test send, engagement analytics |

**Trigger behavior by registration path:**

| Registration Type | Confirmation Email | Welcome Email |
|---|---|---|
| **Regular** (`/register`) | ✅ Yes | ✅ Yes |
| **Facebook** (OAuth) | ❌ No (email pre-verified by Facebook) | ✅ Yes |
| **Google** (OAuth) | ❌ No (email pre-verified by Google) | ✅ Yes |

**Template features (in order):**
1. ✨ الكتابة بالذكاء الاصطناعي — Write with AI (access to 100k+ researches, guaranteed citation correctness)
2. 📚 المكتبة الذكية — Smart Library
3. 🌐 الترجمة الأكاديمية — Academic Translation
4. 🔍 البحث في المراجع — Reference Search
5. 📝 التوثيق التلقائي — Auto Citation
6. ✏️ التدقيق اللغوي — Proofreading
7. 📁 المشاريع البحثية — Research Projects
8. 📤 التصدير المتعدد — Multi-format Export

Ends with a "100 free credits" banner and CTA button linking to Playground.

**Admin test send:** `PlaygroundAdminController::apiTestWelcomeEmail()` — sends a `[TEST]` prefixed email with tracking to any address.

**Admin editor:** The Welcome Email tab uses a **TinyMCE WYSIWYG** editor (loaded from cdnjs) to let admins edit the email body visually. The editor supports the `{{firstName}}`, `{{lastName}}`, and `{{playgroundUrl}}` placeholder tokens.

---

## Recommendation Newsletter

### Overview

A personalized daily newsletter that recommends research papers to users based on their reading history. Uses Elasticsearch **More Like This (MLT)** queries to find similar documents to what each user has recently read.

### How It Works

```
User reads research → ReadHistoryService tracks slug+index → (daily cron)
→ SendRecommendationsCommand → RecommendationService.getRecommendations()
→ Resolve slugs → ES _ids → MLT query → top 5 similar papers
→ Render HTML+text templates → Send via Mailgun (MIME transport)
```

### Components

| Component | Path | Purpose |
|-----------|------|--------|
| **Entity** | `src/Entity/UserReadHistory.php` | Stores last 50 reads per user (slug, ES index, title) |
| **Repository** | `src/Repository/UserReadHistoryRepository.php` | Queries: recent slugs by index, prune old entries, count |
| **ReadHistoryService** | `src/Service/ReadHistoryService.php` | Track reads, get recent reads grouped by index, find eligible users |
| **RecommendationService** | `src/Service/RecommendationService.php` | Slug→ES _id resolution, MLT query, abstract extraction |
| **Command** | `src/Command/SendRecommendationsCommand.php` | CLI entry point for sending emails |
| **HTML Template** | `templates/emails/newsletter_recommendations.html.twig` | Responsive HTML email (RTL/LTR, purple gradient design) |
| **Text Template** | `templates/emails/newsletter_recommendations.txt.twig` | Plain text alternative (improves deliverability) |
| **Controller** | `src/Controller/NewsletterController.php` | Unsubscribe endpoint with HMAC token validation |
| **Tracking (read)** | `src/syndex/AcademicBundle/Controller/ResearchController.php` (`showAction`) | Calls `trackRead()` when authenticated user views research |

### Database

**Table: `user_read_history`**

| Column | Type | Description |
|--------|------|-------------|
| `id` | int (PK, auto) | Primary key |
| `user_id` | int (FK → `fos_user.id`, CASCADE) | The user |
| `es_doc_id` | varchar(64) | ES document _id |
| `es_index` | varchar(32) | `arabic_research` or `english_research` |
| `slug` | varchar(500), nullable | Research slug for URL building |
| `title` | varchar(1000), nullable | Cached title for display |
| `read_at` | datetime_immutable | When the user last read this research |

Indexes: `idx_user_read_history_user_date` (user_id, read_at), `idx_user_read_history_date` (read_at).

Max 50 entries per user — older entries are pruned automatically.

**User entity fields** (in `fos_user`):

| Column | Type | Default | Description |
|--------|------|---------|-------------|
| `preferred_locale` | varchar | `'ar'` | Used to select Arabic vs English MLT fields and email locale |
| `recommendation_newsletter_enabled` | boolean | `true` | User can opt out via profile settings |

### Read History Tracking

Triggered in `ResearchController::showAction()` at two points:

1. **MySQL-backed research** (~line 1017): After the research entity is loaded from DB, calls `trackRead($user, $esDocId, $esIndex, $slug, $title)`.
2. **ES-only research** (~line 802): If the research exists only in Elasticsearch (no MySQL row), calls `trackRead()` using the ES document data.

Both paths are wrapped in `try/catch` so tracking failures never break the page.

### Recommendation Algorithm (MLT)

**Pipeline:**

1. `ReadHistoryService::getRecentReadsByIndex()` — Get slugs grouped by `{arabic: [...], english: [...]}`
2. `RecommendationService::resolveSlugToEsIds()` — Bulk `terms` query on `slug` field to get real ES `_id` values
3. `RecommendationService::queryMLT()` — Run More Like This query

**MLT Query Parameters:**

| Parameter | Value | Notes |
|-----------|-------|-------|
| `fields` | `['arabic_full_title', 'content']` (AR) / `['english_full_title', 'content']` (EN) | **CRITICAL**: Only text-analyzed fields work. Including `tag` (keyword) or `arabic_abstract` causes 0 results. |
| `min_term_freq` | 1 | Minimum term frequency in source doc |
| `max_query_terms` | 40 | Max terms extracted from source docs |
| `min_doc_freq` | 1 | Minimum document frequency for terms |
| `like` | Array of `{_index, _id}` objects | Up to 20 source documents from read history |

**Exclusions** (in `must_not`):
- `ids.values` — Exclude already-read documents by ES `_id`
- `terms.slug` — Exclude by slug as a safety net

**Locale priority**: If user locale is `ar`, Arabic index is tried first; if `en`, English first. Falls back to whichever index has data.

### Email Details

| Field | Value |
|-------|-------|
| **Command** | `app:send-recommendations` |
| **Cron** | `0 9 * * *` (daily at 9:00 AM) |
| **From** | `info@shamra-academia.com` / `شمرا أكاديميا` (AR) or `Shamra Academia` (EN) |
| **Subject (AR)** | `أبحاث مقترحة لك — شمرا أكاديميا` |
| **Subject (EN)** | `Recommended Research For You — Shamra Academia` |
| **Transport** | `mailgun+https://` (MIME-based, preserves UTF-8 charset) |
| **Min recommendations** | 2 (skipped if fewer) |
| **Max recommendations** | 5 |
| **Abstracts** | Truncated to first 2 sentences via `extractSentences()` |
| **Headers** | `List-Unsubscribe` + `List-Unsubscribe-Post` (RFC 8058 one-click) |
| **Unsubscribe URL** | `https://shamra-academia.com/newsletter/unsubscribe?email=...&token=...` (HMAC-SHA256) |

### Command Options

```bash
# Send to all eligible users (default: last 6 months login, limit 500)
sudo -u www-data php bin/console app:send-recommendations --env=prod

# Send to a specific user (testing)
sudo -u www-data php bin/console app:send-recommendations --user=5 --env=prod

# Dry run — see what would be sent without sending
sudo -u www-data php bin/console app:send-recommendations --dry-run --env=prod

# Limit to first N users
sudo -u www-data php bin/console app:send-recommendations --limit=10 --env=prod

# Exclude users who already have a newsletter_log entry (critical for batched sends)
sudo -u www-data php bin/console app:send-recommendations --exclude-sent --env=prod

# Change login eligibility window (default 6 months)
sudo -u www-data php bin/console app:send-recommendations --login-months=24 --env=prod

# Skip login filter entirely — include ALL verified/enabled users regardless of login date
sudo -u www-data php bin/console app:send-recommendations --no-login-filter --env=prod

# Use a different Listmonk template ID (default: 6)
sudo -u www-data php bin/console app:send-recommendations --template-id=6 --env=prod
```

### Sending to New Users (Batched Sends)

To continue sending to users who were never sent to, combine `--exclude-sent` with `--no-login-filter`:

```bash
# Dry run first — preview what would be sent
sudo -u www-data php bin/console app:send-recommendations --env=prod \
  --dry-run --exclude-sent --limit=1500 --no-login-filter

# Actual send (background on server)
nohup sudo -u www-data php bin/console app:send-recommendations --env=prod \
  --exclude-sent --limit=1500 --no-login-filter > /tmp/newsletter-send.log 2>&1 &

# Monitor progress
grep -c 'Sent to' /tmp/newsletter-send.log
tail -5 /tmp/newsletter-send.log
```

Each send is logged in the `newsletter_log` table, so `--exclude-sent` will automatically skip already-sent users in subsequent runs. Just re-run the same command to send the next batch.

### Send History

| Date | Batch | Users Sent | Skipped | Errors | Options Used |
|------|-------|-----------|---------|--------|--------------|
| 2026-02-28 → 2026-03-02 | Initial sends | ~1,809 | varies | varies | `--login-months=6` (default login filter, recent active users) |
| 2026-03-03 | Backfill batch 1 | 1,500 | 0 | 0 | `--exclude-sent --limit=1500 --no-login-filter` (all verified users never sent to, ordered by most recent registration) |

**Total sent as of 2026-03-03:** ~3,309 unique users
**Remaining unsent (verified + newsletter enabled):** ~28,471

### Eligible Users Criteria

The `ReadHistoryService::getEligibleUsers()` query selects users who meet ALL of:
- `recommendationNewsletterEnabled = true`
- `isVerified = true` (maps to `enabled` column in `fos_user`)
- `email IS NOT NULL`
- `lastLogin >= N months ago` (default 6, configurable via `--login-months`, skippable via `--no-login-filter`)
- Optionally excludes users already in `newsletter_log` (`--exclude-sent`)
- Users with read history are prioritized first (LEFT JOIN on `user_read_history`)
- Without `--no-login-filter`: ordered by `lastLogin DESC`
- With `--no-login-filter`: ordered by `id DESC` (most recently registered first)

### Unsubscribe Flow

1. Email footer contains unsubscribe link with HMAC-SHA256 token (generated from `APP_SECRET` + email)
2. `GET /newsletter/unsubscribe?email=...&token=...` → `NewsletterController::unsubscribe()`
3. Token validated → sets `User.recommendationNewsletterEnabled = false`
4. Shows confirmation page (`templates/newsletter/unsubscribe.html.twig`)
5. `POST` variant supports RFC 8058 one-click unsubscribe (Gmail/Outlook)

### Template Design

- **Layout**: Centered 580px container, purple gradient header (`#8b5cf6` → `#6d28d9`)
- **Research cards**: Light purple background (`#f8f5ff`) with border, title as link, metadata row (publisher, date, downloads), abstract snippet, tag pills, "Read more" link
- **Responsive**: Cards stack vertically on mobile (<620px)
- **RTL**: Full `dir="rtl"` support based on user locale
- **Footer**: Unsubscribe link, settings link, copyright

### Known Constraints

- **MLT fields**: Only text-analyzed fields work (`arabic_full_title`, `english_full_title`, `content`). `tag` is `keyword` type and `arabic_abstract` silently breaks MLT — returns 0 results.
- **ES `_id` format**: ES uses auto-generated hash IDs (e.g., `u55_oZwBfjW5ckYaMRtf`), not MySQL IDs. Read history stores slugs; `resolveSlugToEsIds()` does the lookup.
- **Transport**: Must use `mailgun+https://` (not `mailgun+api://`). The API transport strips `Content-Type` charset, causing Arabic text to appear as mojibake.
- **Twig arrays**: ES results are arrays, not objects. Use `research.field|default` instead of `research.field is defined` (Twig limitation).

---

## Bulk / Marketing Email System

The admin control panel provides a bulk email system through the `MailSystem` interface (currently backed by **Ecomail**).

### Admin Panel Routes

| Action | Controller Method | Description |
|--------|-------------------|-------------|
| Email Dashboard | `ControlPanelController::mailIndex()` | View all mailing lists with subscriber counts |
| Send to List | `ControlPanelController::sendMailList()` | Compose and send a campaign to a mailing list |
| Build Recent Members List | `ControlPanelController::makeRecentMembersList()` | Auto-populate a list with recently registered users |
| Build Last Active List | `ControlPanelController::makeLastActiveList()` | Auto-populate a list with recently active users |

### Registration → Mailing List

When a user registers, they are **automatically added** to an Ecomail mailing list based on their study field:
```
RegistrationController::register()
    → MailSystem::makeAndAddRecipient()
    → Ecomail API (add subscriber to list)
```
The system maps the user's `studyField` to a `MailList` entity, which holds the Ecomail `listId`.

### MailSystem Interface

Located at `src/Service/MailSystem/MailSystem.php`:

```php
interface MailSystem {
    public function send();                                          // Send a campaign
    public function handelMailData($data);                           // Prepare campaign data
    public function makeList($data);                                 // Create a mailing list
    public function makeAndAddRecipient($emails, $listId);           // Add subscriber to list
    public function removeRecipient(string $recipientId, string $listId); // Remove subscriber
    public function countRecipientAtList(string $listId);            // Count subscribers
    public function getSubscribers($listId);                         // Get all subscribers
}
```

**Implementations:**
| Class | File | Status |
|-------|------|--------|
| `Ecomail` | `src/Service/MailSystem/Ecomail.php` | **Active** (aliased in services.yaml) |
| `SendGrid` | `src/Service/MailSystem/SendGrid.php` | Legacy, not used |

---

## Email Template Locations

All email templates use the **ZURB Ink v1.0.5** responsive email framework with RTL support (`dir="rtl"`). They are standalone HTML files (no Twig inheritance) with inline CSS for maximum email client compatibility.

### Transactional Email Templates

| Template | Path | Used By |
|----------|------|---------|
| Email Verification | `templates/registration/confirmation_email.html.twig` | Registration, Resend verification |
| Password Reset | `templates/reset_password/email.html.twig` | Password reset flow |
| Partner Join Request | `templates/Profile/joinEmail.html.twig` | Partner program application (→ Shamra team) |
| Partner Join Response | `templates/Profile/responeJoinEmail.html.twig` | Admin partner approval/rejection (→ user) |
| Content Submission (internal) | `templates/partner_content/addContentEmail.html.twig` | Partner content submission (→ Shamra team) |
| Content Submission (author) | `templates/partner_content/responeAddContentEmail.html.twig` | Partner content submission (→ author) |
| Content Review Result | `templates/partner_content/acceptContentEmail.html.twig` | Admin content review (→ author) |
| Contact Us Reply | `src/syndex/ContactUsBundle/Resources/views/Page/email.html.twig` | Admin reply to contact enquiry |
| Welcome Email (Playground) | `templates/emails/welcome.html.twig` | Welcome email with tracking, body from DB `email_template` table (Registration, Facebook OAuth, Google OAuth) |
| Recommendation Newsletter (HTML) | `templates/emails/newsletter_recommendations.html.twig` | Personalized MLT-based research recommendations (daily cron) |
| Recommendation Newsletter (Text) | `templates/emails/newsletter_recommendations.txt.twig` | Plain text alternative for deliverability |
| Newsletter Unsubscribe | `templates/newsletter/unsubscribe.html.twig` | Unsubscribe confirmation page |

### Non-Email Page Templates (Related)

| Template | Path | Purpose |
|----------|------|---------|
| Password Reset Form | `templates/reset_password/request.html.twig` | Request password reset page |
| Password Reset Confirm | `templates/reset_password/check_email.html.twig` | "Check your email" confirmation page |
| Password Reset New Password | `templates/reset_password/reset.html.twig` | Set new password page |
| Registration Form | `templates/registration/register.html.twig` | User registration page |

### Admin Email Panel Templates

| Template | Path | Purpose |
|----------|------|---------|
| Email Dashboard | `templates/ControlPanel/emailPanel.html.twig` | Compose campaign UI |
| Mailing Lists | `templates/ControlPanel/mail-index.html.twig` | Mailing list management |

---

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────┐
│                    TRANSACTIONAL EMAILS                  │
│                                                         │
│  Controller/Service                                     │
│       │                                                 │
│       ▼                                                 │
│  TemplatedEmail (Symfony Mime)                           │
│       │                                                 │
│       ▼                                                 │
│  MailerInterface (Symfony Mailer)                        │
│       │                                                 │
│       ▼                                                 │
│  Mailgun API (MAILER_DSN)                               │
│       │                                                 │
│       ▼                                                 │
│  info@shamra-academia.com → User                        │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│                    MARKETING / BULK EMAILS               │
│                                                         │
│  ControlPanelController                                 │
│       │                                                 │
│       ▼                                                 │
│  MailSystem Interface                                   │
│       │                                                 │
│       ▼                                                 │
│  Ecomail (active) / SendGrid (legacy)                   │
│       │                                                 │
│       ▼                                                 │
│  Ecomail API (campaigns, subscriber lists)              │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│              EMAIL OPEN / CLICK TRACKING                 │
│                                                         │
│  RegistrationController / FacebookController /             │
│  GoogleController / PlaygroundAdminController                │
│       │                                                 │
│       ▼                                                 │
│  EmailTrackingService                                   │
│       ├─ createTracker() → email_tracker table          │
│       ├─ wrapLinksForTracking() → rewrite <a> hrefs     │
│       └─ getTrackingPixelTag() → 1×1 <img> tag         │
│                                                         │
│  User opens email:                                      │
│       GET /email/t/o/{trackingId}.gif                   │
│       → EmailTrackingController::trackOpen()            │
│       → returns 1×1 transparent GIF                     │
│                                                         │
│  User clicks link:                                      │
│       GET /email/t/c/{trackingId}?url=...               │
│       → EmailTrackingController::trackClick()           │
│       → records click, 302 redirect to original URL     │
│                                                         │
│  Admin dashboard:                                       │
│       GET /jim19ud83/playground/api/email-stats          │
│       → stats (open rate, click rate, CTO rate)         │
│       → daily trend (sent/opened/clicked per day)       │
│       → recent sends (last 50 with status)              │
└─────────────────────────────────────────────────────────┘
```

---

## Key Services

| Service | File | Role |
|---------|------|------|
| `EmailVerifier` | `src/Security/EmailVerifier.php` | Generates signed verification URLs and sends email confirmation |
| `VerifyEmailHelperDecorater` | `src/Security/VerifyEmailHelperDecorater.php` | Custom decorator for `VerifyEmailHelperInterface` — generates/validates signed URLs |
| `ResetPasswordHelper` | SymfonyCasts bundle | Generates/validates password reset tokens |
| `Ecomail` | `src/Service/MailSystem/Ecomail.php` | Ecomail API wrapper for mailing lists and campaigns |
| `SendGrid` | `src/Service/MailSystem/SendGrid.php` | Legacy SendGrid API wrapper (inactive) |
| `EmailTrackingService` | `src/Service/EmailTrackingService.php` | Creates tracker records, wraps links for click tracking, generates open-tracking pixel |
| `EmailTrackingController` | `src/Controller/EmailTrackingController.php` | Public routes for open pixel (`/email/t/o/{id}.gif`) and click redirect (`/email/t/c/{id}`) |

---

## Summary Table — All Email Events

| # | Event | Recipient | Subject (Arabic) | Template | Tracking |
|---|-------|-----------|-------------------|----------|----------|
| 1 | New registration | User | الرجاء تاكيد البريد الالكتروني | `registration/confirmation_email.html.twig` | — |
| 2 | Resend verification | User | الرجاء تاكيد البريد الالكتروني | `registration/confirmation_email.html.twig` | — |
| 3 | Facebook registration | User | مرحباً بك في شمرا أكاديميا! | `emails/welcome.html.twig` | ✅ Open + Click |
| 3b | Google registration | User | مرحباً بك في شمرا أكاديميا! | `emails/welcome.html.twig` | ✅ Open + Click |
| 4 | Password reset | User | تغير كلمة السر | `reset_password/email.html.twig` | — |
| 5 | Partner join request | Shamra team | طلب انضمام الى اتفاقية الشركاء | `Profile/joinEmail.html.twig` | — |
| 6 | Partner join response | User | الرد على طلب الانضمام | `Profile/responeJoinEmail.html.twig` | — |
| 7 | Content submission (notify) | Shamra team | طلب إضافة محتوى | `partner_content/addContentEmail.html.twig` | — |
| 8 | Content submission (confirm) | Author | إرسال ملف بنجاح | `partner_content/responeAddContentEmail.html.twig` | — |
| 9 | Content review result | Author | مراجعة المحتوى | `partner_content/acceptContentEmail.html.twig` | — |
| 10 | Contact us reply | Enquirer | شكراً لك على تواصلك | `ContactUsBundle/.../email.html.twig` | — |
| 11 | Welcome email (Playground) | User | مرحباً بك في شمرا أكاديميا! | `emails/welcome.html.twig` | ✅ Open + Click |
| 12 | Recommendation newsletter | User | أبحاث مقترحة لك — شمرا أكاديميا | `emails/newsletter_recommendations.html.twig` + `.txt.twig` | — |

---

## Email Tracking System

### Overview

The welcome email (#11) includes **open and click tracking** via `EmailTrackingService`. Every email sent creates a record in the `email_tracker` table with a unique 32-character hex `trackingId`.

### How It Works

| Event | Mechanism | Endpoint |
|-------|-----------|----------|
| **Open** | 1×1 transparent GIF embedded as `<img>` at bottom of email | `GET /email/t/o/{trackingId}.gif` |
| **Click** | All `<a href>` links rewritten to pass through redirect | `GET /email/t/c/{trackingId}?url={originalUrl}` |

### Database: `email_tracker` table

| Column | Type | Description |
|--------|------|-------------|
| `id` | INT PK | Auto-increment |
| `tracking_id` | VARCHAR(64) UNIQUE | 32-char hex identifier |
| `template_name` | VARCHAR(100) | e.g. `welcome_email` |
| `recipient_email` | VARCHAR(255) | Recipient address |
| `user_id` | INT NULL | FK to `fos_user.id` (if known) |
| `sent_at` | DATETIME | When the email was sent |
| `opened_at` | DATETIME NULL | First open timestamp |
| `open_count` | INT DEFAULT 0 | Total open events |
| `clicked_at` | DATETIME NULL | First click timestamp |
| `click_count` | INT DEFAULT 0 | Total click events |
| `last_click_url` | VARCHAR(500) NULL | Last clicked URL |
| `user_agent` | VARCHAR(100) NULL | Client user-agent string |

### Admin Dashboard Analytics

The **Welcome Email** tab is one of 8 tabs in the Playground admin dashboard (`/jim19ud83/playground/dashboard`):

| # | Tab | Purpose |
|---|-----|----------|
| 1 | **Overview** | Real-time stat cards (requests today, tokens, cost, active users, avg latency, success rate), budget alerts (daily/weekly/monthly with progress bars), 30-day trend chart, hourly activity chart |
| 2 | **Usage Details** | Operations breakdown table, model usage table with token I/O and cost |
| 3 | **Top Users** | Top 50 users by cost in the last 30 days (username, email, tier, request count, tokens, cost) |
| 4 | **Trial Users** | All trial-tier subscribers with usage data (credits remaining, usage %, status, last activity), operation breakdown for trial users |
| 5 | **Engagement** | Subscriber engagement & retention risk segmentation (dormant / at-risk / engaged / power users), health score, per-tier breakdown |
| 6 | **Welcome Email** | Template editor (TinyMCE WYSIWYG), preview, test send, engagement analytics with Chart.js charts |
| 7 | **Pricing & Tiers** | Tier configuration cards with pricing, credits, and features from `playground_tier_config` table |
| 8 | **Errors** | Last 100 failed requests with user, operation type, model, error message, and timestamp |

The Welcome Email tab specifically includes:

- **Stat cards**: Total Sent, Open Rate %, Click Rate %, Click-to-Open Rate %
- **Daily trend chart**: Line chart showing sent/opened/clicked per day (Chart.js)
- **Recent sends table**: Last 50 emails with recipient, username, timestamps, open/click counts, and status badge
- **Period selector**: 7 / 30 / 90 / 365 day windows

API endpoint: `GET /jim19ud83/playground/api/email-stats?days=30&template=welcome_email`

### Key Files

| File | Purpose |
|------|---------|
| `src/Entity/EmailTracker.php` | Doctrine entity — `email_tracker` table |
| `src/Repository/EmailTrackerRepository.php` | Stats queries (getStats, getDailyTrend, getRecentSends) |
| `src/Service/EmailTrackingService.php` | Creates trackers, wraps links, generates pixel tags |
| `src/Controller/EmailTrackingController.php` | Public open/click endpoints (no auth) |
| `migrations/Version20260220100000.php` | Creates `email_tracker` table, updates welcome template body |



ListMonk settings:


# Listmonk API — Add Subscriber on Registration

## Authentication

Listmonk v6 uses **username:token** basic auth. Use the API user you created:

```
Username: ponkyapi
API key: 4gB8wpPYizVWZ1ytVnI7k7TQDJ0hPN7e
```

## Add a Subscriber

**Endpoint:** `POST https://newsletter.shamra-academia.com/api/subscribers`

**Headers:**
```
Content-Type: application/json
Authorization: Basic <base64(ponkyapi:TOKEN)>
```

**Body:**
```json
{
  "email": "user@example.com",
  "name": "First Last",
  "status": "enabled",
  "lists": [6],
  "preconfirm_subscriptions": true,
  "attribs": {
    "firstName": "First",
    "lastName": "Last",
    "gender": "m",
    "source": "shamra-academia",
    "registered": "2026-02-20"
  }
}
```

| Field | Required | Description |
|-------|----------|-------------|
| `email` | Yes | Subscriber email |
| `name` | Yes | Display name |
| `status` | Yes | `enabled` (active) or `blocklisted` |
| `lists` | Yes | Array of list IDs. ShamraAcademia = `6` |
| `preconfirm_subscriptions` | No | `true` to skip double opt-in |
| `attribs` | No | JSON object with custom attributes |

## Examples

### cURL

```bash
curl -X POST https://newsletter.shamra-academia.com/api/subscribers \
  -u "ponkyapi:YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "newuser@example.com",
    "name": "Ahmad Ali",
    "status": "enabled",
    "lists": [6],
    "preconfirm_subscriptions": true,
    "attribs": {
      "firstName": "Ahmad",
      "lastName": "Ali",
      "source": "shamra-academia",
      "registered": "2026-02-20"
    }
  }'
```

### Python

```python
import requests
from datetime import date

LISTMONK_API = "https://newsletter.shamra-academia.com/api"
API_USER = "ponkyapi"
API_TOKEN = "YOUR_API_TOKEN"  # from Listmonk dashboard
SHAMRA_LIST_ID = 6

def add_subscriber(email, first_name, last_name, gender="m"):
    resp = requests.post(
        f"{LISTMONK_API}/subscribers",
        auth=(API_USER, API_TOKEN),
        json={
            "email": email,
            "name": f"{first_name} {last_name}",
            "status": "enabled",
            "lists": [SHAMRA_LIST_ID],
            "preconfirm_subscriptions": True,
            "attribs": {
                "firstName": first_name,
                "lastName": last_name,
                "gender": gender,
                "source": "shamra-academia",
                "registered": str(date.today()),
            },
        },
    )
    resp.raise_for_status()
    return resp.json()
```

### JavaScript / Node.js

```javascript
const LISTMONK_API = "https://newsletter.shamra-academia.com/api";
const API_USER = "ponkyapi";
const API_TOKEN = "YOUR_API_TOKEN";
const SHAMRA_LIST_ID = 6;

async function addSubscriber(email, firstName, lastName, gender = "m") {
  const res = await fetch(`${LISTMONK_API}/subscribers`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: "Basic " + btoa(`${API_USER}:${API_TOKEN}`),
    },
    body: JSON.stringify({
      email,
      name: `${firstName} ${lastName}`,
      status: "enabled",
      lists: [SHAMRA_LIST_ID],
      preconfirm_subscriptions: true,
      attribs: {
        firstName,
        lastName,
        gender,
        source: "shamra-academia",
        registered: new Date().toISOString().split("T")[0],
      },
    }),
  });
  if (!res.ok) throw new Error(await res.text());
  return res.json();
}
```

## Response

**Success (200):**
```json
{
  "data": {
    "id": 12345,
    "uuid": "...",
    "email": "newuser@example.com",
    "name": "Ahmad Ali",
    "status": "enabled",
    "lists": [{ "id": 6, "name": "ShamraAcademia" }]
  }
}
```

**Duplicate email (409):**
```json
{
  "message": "subscriber already exists"
}
```

## Handle Existing Subscribers

If the user already exists and you want to add them to the list:

```bash
# 1. Find subscriber by email
curl -s -u "ponkyapi:TOKEN" \
  "https://newsletter.shamra-academia.com/api/subscribers?query=subscribers.email='user@example.com'"

# 2. Add to list by subscriber ID
curl -X PUT -u "ponkyapi:TOKEN" \
  -H "Content-Type: application/json" \
  "https://newsletter.shamra-academia.com/api/subscribers/lists" \
  -d '{"ids": [SUBSCRIBER_ID], "action": "add", "target_list_ids": [6], "status": "confirmed"}'
```