# Plan: Research Progress Tracker for Graduate Students

> **Status**: ✅ Implemented  
> **Last Updated**: March 19, 2026  
> **Priority**: High  
> **Dependencies**: Reading Lists, Ask AI, PlaygroundSubscription

---

## Implementation Summary

**Deployed**: March 19, 2026

### Files Created
- `src/Entity/ResearchTracker.php` - Main tracker entity
- `src/Entity/ResearchTrackerMilestone.php` - Milestones with subtasks
- `src/Entity/ResearchDeadline.php` - Deadline reminders
- `src/Entity/ExperimentLog.php` - Experiment tracking
- `src/Repository/ResearchTracker*.php` - Repositories (4 files)
- `src/Service/ResearchTrackerService.php` - Business logic
- `src/Controller/ResearchTrackerController.php` - 14 routes
- `templates/tracker/index.html.twig` - Dashboard
- `templates/tracker/create.html.twig` - Creation wizard
- `translations/Tracker.ar.yml` / `Tracker.en.yml` - Translations
- `public/js/tracker.js` - Frontend JavaScript
- `migrations/Version20260319100000.php` - Database migration

### Bug Fixes (Same Day)
1. **YAML colons** - Arabic translation values containing `:` caused parse errors → quoted all affected values
2. **FieldRepository autowiring** - Old-style Doctrine repository not autowirable → use `EntityManager->getRepository(Field::class)`
3. **Translation key prefix** - Template used `tracker.key` but YAML keys were just `key` → removed `tracker.` prefix from templates
4. **Rating UI** - Numbers not centered, no scale indicator → added flexbox centering + "(5 = best)" label

### Features Implemented
1. Three tracker types: Master's Thesis, PhD Dissertation, Research Paper
2. Auto-populated milestones by type (8-12 defaults)
3. Viability assessment wizard (3 criteria, 1-5 scale)
4. Progress tracking with visual bar
5. Custom milestone support
6. Deadline management with reminder days
7. Experiment logging for data milestones
8. Linked resources stats (papers, references, word count, AI sessions)
9. Subtask checklists for writing milestones
10. BibTeX export for linked papers
11. Navigation link added to header

### Routes
- `GET /tracker` - Dashboard or creation wizard
- `POST /tracker/create` - Create tracker
- `PATCH /tracker/update` - Update settings
- `DELETE /tracker/delete` - Delete tracker
- `PATCH /api/tracker/milestone/{id}/status` - Toggle status
- `PATCH /api/tracker/milestone/{id}/notes` - Update notes
- `PATCH /api/tracker/milestone/{id}/subtask/{index}` - Toggle subtask
- `POST /api/tracker/milestone/add` - Add custom milestone
- `DELETE /api/tracker/milestone/{id}` - Remove milestone
- `POST /api/tracker/deadline/add` - Add deadline
- `DELETE /api/tracker/deadline/{id}` - Remove deadline
- `POST /api/tracker/milestone/{id}/experiment` - Add experiment
- `DELETE /api/tracker/experiment/{id}` - Remove experiment
- `GET /api/tracker/milestone/{id}/bibtex` - Export BibTeX

### To Deploy
```bash
php bin/console doctrine:migrations:migrate --no-interaction
```

---

## TL;DR
Build a thesis/research progress tracker targeted at master's and PhD students (the core audience per GSC data). A guided checklist that links to Playground projects, Reference Library, Reading Lists, and Ask AI — turning Shamra from "a PDF I downloaded once" into "the tool I use every day for my thesis."

---

## Inspiration Sources

Enhancements informed by:
- **[ResearchClaw](https://github.com/ymx10086/ResearchClaw)** — AI research assistant with experiment tracking, deadline reminders, citation alerts, paper digests
- **[R2T2](https://github.com/ImperialCollegeLondon/R2T2)** — Reference tracking tool linking papers to specific code/methodology sections
- **[Doing Applied Research](https://github.com/Mixtape-Sessions/Doing-Applied-Research)** — Academic workflow course covering project viability, writing structure, journal targeting

---

## Current State

- **Playground Projects**: `src/Entity/PlaygroundProject.php` — users can create writing projects with title, keywords, content, language
- **Reference Library**: `src/Controller/UserReferenceController.php` — upload, organize PDFs in folders
- **Read History**: `src/Service/ReadHistoryService.php` — tracks papers read per user (last 50)
- **No progress tracker exists**: Zero code for milestones, checklists, or thesis tracking

---

## Steps

### Step 1: Create Entities

**`src/Entity/ResearchTracker.php`** — table `research_tracker`:
- `id` (int, PK)
- `user` (ManyToOne → User, NOT NULL, unique per type)
- `title` (string 255) — thesis/research title
- `type` (string 50) — `masters_thesis`, `phd_dissertation`, `research_paper`
- `field` (ManyToOne → Field, nullable)
- `targetDate` (date, nullable) — expected completion date
- `playgroundProject` (ManyToOne → PlaygroundProject, nullable) — linked writing project
- `linkedReadingList` (ManyToOne → ReadingList, nullable) — auto-created "📚 {Title} - Sources"
- `targetJournal` (string 255, nullable) — target publication journal
- `viabilityInteresting` (int 1-5, nullable) — Is it interesting/important?
- `viabilityGap` (int 1-5, nullable) — Gap in literature?
- `viabilityData` (int 1-5, nullable) — Data available?
- `createdAt` (datetime_immutable)
- `updatedAt` (datetime_immutable)

**`src/Entity/ResearchTrackerMilestone.php`** — table `research_tracker_milestone`:
- `id` (int, PK)
- `tracker` (ManyToOne → ResearchTracker, NOT NULL)
- `phase` (string 100) — milestone identifier
- `titleAr` (string 255)
- `titleEn` (string 255)
- `status` (string 20) — `not_started`, `in_progress`, `completed`
- `completedAt` (datetime_immutable, nullable)
- `notes` (text, nullable) — user's notes (markdown)
- `sortOrder` (integer)
- `linkedPapers` (ManyToMany → ReadingListItem, nullable) — papers influencing this milestone
- `hasSubtasks` (boolean) — for writing milestones with checklists
- `subtasksJson` (json, nullable) — `[{"title": "...", "done": false}, ...]`

**`src/Entity/ResearchDeadline.php`** — table `research_deadline` (NEW):
- `id` (int, PK)
- `tracker` (ManyToOne → ResearchTracker, NOT NULL)
- `title` (string 255) — e.g., "Conference submission", "Defense date"
- `deadlineDate` (date)
- `reminderDays` (json) — `[30, 14, 7, 1]` days before
- `notifiedDates` (json) — track which reminders sent
- `createdAt` (datetime_immutable)

**`src/Entity/ExperimentLog.php`** — table `experiment_log` (NEW):
- `id` (int, PK)
- `milestone` (ManyToOne → ResearchTrackerMilestone, NOT NULL)
- `title` (string 255) — experiment name
- `parameters` (json) — `{"sample_size": 100, "method": "..."}`
- `results` (json) — `{"p_value": 0.03, "effect_size": 0.4}`
- `notes` (text, nullable)
- `createdAt` (datetime_immutable)

### Step 2: Default Milestone Templates

When a user creates a tracker, auto-populate milestones based on type:

**Master's Thesis Template:**
1. تحديد موضوع البحث / Topic Selection
2. مراجعة الأدبيات / Literature Review — `linkedPapers` enabled, progress from Reading List
3. تصميم المنهجية / Methodology Design — `experimentLogs` enabled
4. جمع البيانات / Data Collection — `experimentLogs` enabled
5. تحليل البيانات / Data Analysis — `experimentLogs` enabled
6. كتابة الفصول / Writing Chapters — `hasSubtasks: true` with writing checklist
7. المراجعة والتدقيق / Review & Proofreading
8. التقديم للمناقشة / Submission for Defense

**PhD Dissertation Template:** Same + "Research Proposal", "Publications", "Comprehensive Exam", "Ethics/IRB Approval"

**Research Paper Template:** Topic → Literature → Method → Results → Writing → Peer Review → Submission

**Writing Milestone Subtasks** (auto-populated for "Writing Chapters"):
- [ ] Introduction with contributions clearly stated
- [ ] Background/institutional details section
- [ ] Literature review positioned
- [ ] Data description with summary stats
- [ ] Empirical methods explained
- [ ] Results with tables/figures
- [ ] Conclusion with policy implications

### Step 3: Create TrackerController

Create `src/Controller/ResearchTrackerController.php`:
- `GET /tracker` → `index()` — show user's tracker (or creation wizard if none)
- `POST /tracker/create` → `create()` — create tracker with type selection
- `PATCH /api/tracker/milestone/{id}` → `updateMilestone()` — update status, add notes (AJAX)
- `POST /api/tracker/milestone/add` → `addCustomMilestone()` — user adds custom milestones
- `DELETE /api/tracker/milestone/{id}` → `removeMilestone()` — remove custom milestone
- `PATCH /api/tracker` → `updateTracker()` — update title, target date
- `GET /api/tracker/stats` → `getStats()` — progress stats (JSON)

### Step 4: Tracker Dashboard Template

Create `templates/tracker/index.html.twig`:

```
┌──────────────────────────────────────────────────┐
│ 📖 My Thesis: {title}                            │
│ Target: {targetDate} · {daysRemaining} days left  │
│ Progress: ████████░░░░░ 62%                       │
├──────────────────────────────────────────────────┤
│ ✅ Topic Selection — completed Mar 1              │
│ ✅ Literature Review — completed Mar 15           │
│ 🔄 Methodology Design — in progress              │
│    Notes: "Need to finalize survey design"        │
│ ⬜ Data Collection                                │
│ ⬜ Data Analysis                                  │
│ ⬜ Writing Chapters                               │
│ ⬜ Review & Proofreading                          │
│ ⬜ Submission for Defense                         │
├──────────────────────────────────────────────────┤
│ Linked Resources:                                 │
│ 📂 Reference Library: 24 papers saved            │
│ 📖 Papers Read: 47 total                          │
│ ✍️  Writing Project: "Chapter 2 Draft"            │
│ 🤖 AI Sessions: 12 conversations                 │
└──────────────────────────────────────────────────┘
```

### Step 5: Auto-Link Activity

Pull stats from existing services:
- **Papers read**: `ReadHistoryService` count
- **References saved**: `UserReference` count
- **Playground project**: Link the selected project, show word count
- **AI sessions**: `PlaygroundChat` count for linked project
- **Days since start**: `tracker.createdAt` → now

Show a weekly mini-chart: "This week you read 5 papers and wrote 1,200 words"

### Step 6: Motivational Nudges

If a user hasn't made progress in 7 days, include a nudge in the weekly digest email (Plan 01):
- "You haven't updated your thesis tracker in 7 days. Your next milestone is '{milestone}' — keep going!"

If the target date is approaching:
- "Only {days} days until your target date. You have {remaining} milestones left."

### Step 7: Add to Navigation

- Add "My Research" or "متابعة بحثي" to header nav for logged-in users (between Profile and References)
- On homepage dashboard (Plan 03): show compact tracker progress bar

### Step 8: Translations

- `translations/Tracker.ar.yml` — milestone names, UI labels, nudge messages
- `translations/Tracker.en.yml`

### Step 9: Viability Check Wizard (NEW)

When creating a tracker, show a "Project Viability" wizard (inspired by Doing Applied Research):

```
┌──────────────────────────────────────────────────┐
│ 📋 Evaluate Your Research Project                │
├──────────────────────────────────────────────────┤
│ 1. Is it interesting/important/policy relevant? │
│    ○ 1  ○ 2  ○ 3  ○ 4  ○ 5                      │
│    Tip: Consider impact on field + real-world   │
├──────────────────────────────────────────────────┤
│ 2. Are you filling a gap in the literature?     │
│    ○ 1  ○ 2  ○ 3  ○ 4  ○ 5                      │
│    Tip: Search Shamra for similar papers first  │
├──────────────────────────────────────────────────┤
│ 3. Are the data available?                       │
│    ○ 1  ○ 2  ○ 3  ○ 4  ○ 5                      │
│    Tip: Identify data sources now               │
├──────────────────────────────────────────────────┤
│ Average Score: 4.0/5 — Good viability! ✅        │
└──────────────────────────────────────────────────┘
```

If average < 3, show warning: "Consider refining your research question."

### Step 10: Ask AI Integration (NEW)

Add "🤖 Ask AI" button on each milestone that pre-populates context:

- **Endpoint**: `POST /api/research-chat` with pre-filled question
- **Context injection**: thesis title, field, current milestone, linked papers
- **Example prompt**: "I'm working on [Literature Review] for my thesis titled '[User's Title]' in [Field]. Help me identify key papers I should read."

UI: Small AI button on each milestone card → opens Ask AI modal with context.

### Step 11: Reading List Auto-Link (NEW)

When user creates a tracker:
1. Auto-create a Reading List: "📚 {Thesis Title} - Sources"
2. Link it to `tracker.linkedReadingList`
3. For "Literature Review" milestone, show progress from this list:
   - "32/50 papers read" (count of items with status=`read`)
   - Papers marked "read" in linked list auto-contribute to milestone progress

### Step 12: Deadline Reminders (NEW)

Cron job (`shamra-tracker-reminders`) runs daily:
1. Query `research_deadline` where `deadlineDate - today` matches any value in `reminderDays`
2. If not already in `notifiedDates`, send email:
   - Subject: "⏰ {days} days until {deadline title}"
   - Body: Milestone status summary + urgency nudge
3. Add date to `notifiedDates` JSON

### Step 13: Reference Export per Milestone (NEW)

For each milestone with `linkedPapers`:
- "Export References" button → generates BibTeX file
- Format: `@article{shamra_123, title={...}, ...}`
- Pulls citation data from `ReadingListItem` → `elasticResearchId` → ES document

---

## Verification

1. Create a tracker as "Master's Thesis" → verify 8 default milestones appear
2. Toggle milestones to "in_progress" / "completed" → verify progress bar updates
3. Link a Playground project → verify word count shows
4. Check reference count and read history count display correctly
5. Set target date to 7 days from now → verify urgency indicator
6. **NEW**: Verify viability wizard scores are saved and displayed
7. **NEW**: Verify auto-created Reading List is linked to tracker
8. **NEW**: Verify "Ask AI" button opens chat with pre-populated context
9. **NEW**: Verify "Literature Review" progress syncs with linked Reading List read count
10. **NEW**: Add a deadline → verify reminder email sent at configured days
11. **NEW**: Link papers to milestone → verify BibTeX export works
12. **NEW**: Add experiment log to "Data Analysis" milestone → verify parameters/results saved

---

## Decisions

- **One tracker per user** (initially): Most users are working on one thesis. Multi-tracker can come later.
- **Milestones are editable**: Users can add/remove/rename milestones to match their actual process.
- **No supervisor features**: This is a personal tool, not a student-supervisor management system.
- **Lightweight data model**: No complex dependencies. The tracker is just a checklist with smart links to existing data.
- **NEW: Viability scores are advisory**: We don't block tracker creation on low scores, just warn.
- **NEW: Reading List integration is optional**: Users can unlink or use a different list.
- **NEW: Experiment logs are JSON-based**: Flexible schema for different research types (quantitative, qualitative, mixed methods).
- **NEW: Deadlines are separate entity**: Allows multiple deadlines per tracker (defense, conferences, journal submissions).

---

## Implementation Plan

### Phase 1: Data Layer (Day 1-2)
| # | Task | Files |
|---|------|-------|
| 1.1 | Create `ResearchTracker` entity (with viability + journal fields) | `src/Entity/ResearchTracker.php` |
| 1.2 | Create `ResearchTrackerMilestone` entity (with subtasks + linkedPapers) | `src/Entity/ResearchTrackerMilestone.php` |
| 1.3 | Create `ResearchDeadline` entity | `src/Entity/ResearchDeadline.php` |
| 1.4 | Create `ExperimentLog` entity | `src/Entity/ExperimentLog.php` |
| 1.5 | Create repositories | `src/Repository/ResearchTracker*.php`, `ResearchDeadlineRepository.php`, `ExperimentLogRepository.php` |
| 1.6 | Generate migration | `migrations/VersionYYYYMMDDHHMMSS.php` |
| 1.7 | Run migration locally & verify tables |

### Phase 2: Service Layer (Day 2-3)
| # | Task | Files |
|---|------|-------|
| 2.1 | Create `ResearchTrackerService` | `src/Service/ResearchTrackerService.php` |
| 2.2 | Implement milestone template definitions (array of default milestones per type) |
| 2.3 | Implement `createTracker()` with auto-populated milestones + auto-create Reading List |
| 2.4 | Implement `updateMilestoneStatus()` |
| 2.5 | Implement `getStats()` - progress %, linked resources, days remaining |
| 2.6 | Implement `calculateViabilityScore()` - average of 3 scores |
| 2.7 | Implement `syncLitReviewProgress()` - count read papers from linked Reading List |
| 2.8 | Implement `exportMilestoneBibTeX()` - generate BibTeX from linked papers |

### Phase 3: Controller & Routes (Day 3-4)
| # | Task | Files |
|---|------|-------|
| 3.1 | Create `ResearchTrackerController` | `src/Controller/ResearchTrackerController.php` |
| 3.2 | `GET /tracker` → dashboard or creation wizard |
| 3.3 | `POST /tracker/create` → create tracker (with viability wizard data) |
| 3.4 | `PATCH /api/tracker/milestone/{id}` → update milestone (AJAX) |
| 3.5 | `POST /api/tracker/milestone/add` → add custom milestone |
| 3.6 | `DELETE /api/tracker/milestone/{id}` → remove milestone |
| 3.7 | `PATCH /api/tracker` → update tracker settings |
| 3.8 | `POST /api/tracker/deadline` → add deadline |
| 3.9 | `DELETE /api/tracker/deadline/{id}` → remove deadline |
| 3.10 | `POST /api/tracker/milestone/{id}/experiment` → add experiment log |
| 3.11 | `GET /api/tracker/milestone/{id}/bibtex` → export BibTeX |
| 3.12 | `POST /api/tracker/milestone/{id}/ask-ai` → Ask AI with context |

### Phase 4: Templates & UI (Day 2-3)
| # | Task | Files |
|---|------|-------|
| 4.1 | Create tracker dashboard template | `templates/tracker/index.html.twig` |
| 4.2 | Create creation wizard partial | `templates/tracker/_create_wizard.html.twig` |
| 4.3 | Create milestone card partial | `templates/tracker/_milestone.html.twig` |
| 4.4 | Add progress bar component (reusable) |
| 4.5 | Create tracker JS for AJAX interactions | `assets/js/tracker.js` |
| 4.6 | Add CSS/styles | `assets/styles/tracker.scss` |

### Phase 5: Translations (Day 3)
| # | Task | Files |
|---|------|-------|
| 5.1 | Create Arabic translations | `translations/Tracker.ar.yml` |
| 5.2 | Create English translations | `translations/Tracker.en.yml` |
| 5.3 | Add all milestone names, UI labels, statuses, button text |

### Phase 6: Navigation & Integration (Day 5)
| # | Task | Files |
|---|------|-------|
| 6.1 | Add "My Research" link to header nav | `templates/header.html.twig` |
| 6.2 | Add translations for nav item | `translations/UserBundle.ar.yml`, `UserBundle.en.yml` |
| 6.3 | Add mobile menu entry |
| 6.4 | Integrate Ask AI button with ResearchChatController | `src/Controller/ResearchChatController.php` |
| 6.5 | Auto-create Reading List on tracker creation | `src/Service/ReadingListService.php` |

### Phase 7: Deadline Reminders (Day 5-6)
| # | Task | Files |
|---|------|-------|
| 7.1 | Create `CheckDeadlineRemindersCommand` | `src/Command/CheckDeadlineRemindersCommand.php` |
| 7.2 | Create deadline reminder email template | `templates/emails/deadline_reminder.html.twig` |
| 7.3 | Add translations for email | `translations/Tracker.ar.yml`, `Tracker.en.yml` |
| 7.4 | Configure cron in systemd or scheduler |

### Phase 8: Testing & Polish (Day 6-7)
| # | Task |
|---|------|
| 8.1 | Manual test: create tracker → verify 8 milestones |
| 8.2 | Test milestone status toggle via AJAX |
| 8.3 | Test custom milestone add/remove |
| 8.4 | Test Playground project linking |
| 8.5 | Test reference/read history counts |
| 8.6 | Test RTL layout (Arabic) |
| 8.7 | Test mobile responsiveness |
| 8.8 | Test viability wizard scores saved correctly |
| 8.9 | Test auto-created Reading List linking |
| 8.10 | Test Ask AI with pre-populated context |
| 8.11 | Test Lit Review progress sync from Reading List |
| 8.12 | Test deadline reminder emails |
| 8.13 | Test BibTeX export per milestone |
| 8.14 | Test experiment log CRUD |

---

## File Inventory (New Files)

```
# Entities
src/Entity/ResearchTracker.php
src/Entity/ResearchTrackerMilestone.php
src/Entity/ResearchDeadline.php           # NEW
src/Entity/ExperimentLog.php              # NEW

# Repositories
src/Repository/ResearchTrackerRepository.php
src/Repository/ResearchTrackerMilestoneRepository.php
src/Repository/ResearchDeadlineRepository.php    # NEW
src/Repository/ExperimentLogRepository.php       # NEW

# Service & Controller
src/Service/ResearchTrackerService.php
src/Controller/ResearchTrackerController.php

# Commands
src/Command/CheckDeadlineRemindersCommand.php    # NEW

# Templates
templates/tracker/index.html.twig
templates/tracker/_create_wizard.html.twig
templates/tracker/_viability_wizard.html.twig    # NEW
templates/tracker/_milestone.html.twig
templates/tracker/_deadline_card.html.twig       # NEW
templates/tracker/_experiment_log.html.twig      # NEW
templates/tracker/_ask_ai_modal.html.twig        # NEW
templates/emails/deadline_reminder.html.twig     # NEW

# Assets
assets/js/tracker.js
assets/styles/tracker.scss

# Translations
translations/Tracker.ar.yml
translations/Tracker.en.yml

# Migration
migrations/VersionYYYYMMDDHHMMSS.php
```

## Dependencies (Existing Code to Use)

| Dependency | How Used |
|------------|----------|
| `PlaygroundProject` entity | ManyToOne link for writing project |
| `ReadHistoryService` | Get count of papers read |
| `UserReferenceRepository` | Get count of saved references |
| `PlaygroundChat` entity | Count AI sessions (via project->getChats()) |
| `Field` entity | Optional field selection for thesis |
| `ReadingList` entity | Auto-create & link "Sources" list (NEW) |
| `ReadingListItem` entity | Sync Lit Review progress + BibTeX export (NEW) |
| `ReadingListService` | Create list, count read items (NEW) |
| `ResearchChatController` | Ask AI with context injection (NEW) |
| `UsageMonitorService` | Deduct credits for Ask AI calls (NEW) |

---

## Estimation

| Phase | Effort |
|-------|--------|
| Phase 1: Data Layer (entities, repos, migration) | 3-4 hours |
| Phase 2: Service Layer (incl. viability, BibTeX, sync) | 4-5 hours |
| Phase 3: Controller (all routes incl. deadlines, experiments) | 3-4 hours |
| Phase 4: Templates (incl. viability wizard, Ask AI modal) | 5-6 hours |
| Phase 5: Translations | 1-2 hours |
| Phase 6: Navigation & Integration | 1-2 hours |
| Phase 7: Deadline Reminders (command, email, cron) | 2-3 hours |
| Phase 8: Testing & Polish | 3-4 hours |
| **Total** | **~23-30 hours (5-7 days)** |

---

## Future Enhancements (Phase 2+)

Ideas deferred to a later iteration:

| Feature | Description | Source |
|---------|-------------|--------|
| Citation Alerts | Notify when linked papers get new citations (Semantic Scholar API) | ResearchClaw |
| Paper Digest | Weekly email with new papers matching thesis keywords/field | ResearchClaw |
| Reference Graph | Visualize which papers influence which thesis sections | R2T2 |
| Smart Milestone Suggestions | Use Ask AI to suggest field-specific milestones (IRB, Ethics, etc.) | ResearchClaw |
| Stale Milestone Nudge | If milestone stays in_progress >60 days → "Review viability scores" | Doing Applied Research |
| Journal Requirements | Show word limits, formatting rules for target journal | Doing Applied Research |
| Pivot Guidance | If viability <3, show prompts for refining research question | Doing Applied Research |