# User Reference Library Feature ## Status: ✅ IMPLEMENTED ## Implementation Summary ### Files Created: - `src/Entity/UserReference.php` - Doctrine entity with 30+ fields - `src/Repository/UserReferenceRepository.php` - Query methods for references - `src/Service/Playground/UserReferenceService.php` - Business logic service - `src/Controller/UserReferenceController.php` - API endpoints and page controller - `templates/references/index.html.twig` - Vue.js 3 frontend - `migrations/Version20260130180000.php` - Database migration - Translations in `UserBundle.en.yml` and `UserBundle.ar.yml` ### Access URL: `/myreferences` ## Overview This feature allows users to upload and manage their personal reference library (PDF files) with full citation metadata. These references can then be used as grounding data in the "Write with AI" feature. ## Architecture ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ User Reference Library │ └─────────────────────────────────────────────────────────────────────────────┘ │ ┌─────────────────┼─────────────────┐ ▼ ▼ ▼ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ Upload PDF │ │ Manage Refs │ │ Use in AI │ │ + Metadata │ │ (CRUD) │ │ (Grounding) │ └───────────────┘ └───────────────┘ └───────────────┘ │ │ │ ▼ ▼ ▼ ┌───────────────────────────────────────────────────┐ │ UserReference Entity │ │ - PDF Storage │ │ - Citation Metadata │ │ - Extracted Text (for search) │ └───────────────────────────────────────────────────┘ ``` ## Database Schema ### UserReference Entity ```php class UserReference { // Primary private int $id; private User $user; // File Storage private string $originalFilename; // Original uploaded filename private string $storedFilename; // UUID-based stored filename private string $filePath; // Relative path to file private int $fileSize; // Size in bytes private string $mimeType; // application/pdf // Core Citation Metadata (Required) private string $title; // Paper/Book title private string $authors; // Comma-separated authors private ?int $year; // Publication year private string $publicationType; // journal, book, conference, thesis, report, website, other // Publication Details (Optional) private ?string $journalName; // Journal/Book/Conference name private ?string $volume; // Volume number private ?string $issue; // Issue number private ?string $pages; // Page range (e.g., "123-145") private ?string $publisher; // Publisher name private ?string $edition; // For books // Identifiers (Optional) private ?string $doi; // Digital Object Identifier private ?string $isbn; // For books private ?string $issn; // For journals private ?string $arxivId; // ArXiv identifier private ?string $pmid; // PubMed ID // Content private ?string $abstract; // Abstract text private ?string $keywords; // Comma-separated keywords private ?string $extractedText; // Full text extracted from PDF (for search) private ?string $notes; // User's personal notes // URLs private ?string $url; // Online URL private ?string $accessDate; // When URL was accessed // Categorization private string $language; // 'ar' or 'en' private ?string $tags; // User-defined tags private ?string $folder; // Virtual folder/category // Status private bool $isProcessed; // PDF text extraction complete private bool $isPublic; // Share with other users (future) private \DateTime $createdAt; private \DateTime $updatedAt; } ``` ## API Endpoints | Method | Endpoint | Description | |--------|----------|-------------| | GET | `/myreferences` | Library management page | | GET | `/api/references` | List all user references | | GET | `/api/references/{id}` | Get single reference details | | POST | `/api/references/upload` | Upload PDF with metadata | | PUT | `/api/references/{id}` | Update reference metadata | | DELETE | `/api/references/{id}` | Delete reference | | GET | `/api/references/{id}/download` | Download PDF file | | POST | `/api/references/{id}/extract` | Trigger text extraction | | GET | `/api/references/search` | Search references | | GET | `/api/references/folders` | Get user's folder list | ## File Storage Structure ``` var/ └── uploads/ └── references/ └── {user_id}/ ├── {uuid1}.pdf ├── {uuid2}.pdf └── ... ``` ## Citation Metadata by Publication Type ### Journal Article - Title, Authors, Year (required) - Journal Name, Volume, Issue, Pages - DOI, ISSN ### Book - Title, Authors, Year (required) - Publisher, Edition - ISBN, Pages (total) ### Book Chapter - Chapter Title, Authors, Year (required) - Book Title, Editors - Publisher, Pages, ISBN ### Conference Paper - Title, Authors, Year (required) - Conference Name, Location - Pages, DOI ### Thesis/Dissertation - Title, Author, Year (required) - Institution, Degree Type (PhD, Master's, etc.) ### Report - Title, Authors, Year (required) - Institution, Report Number ### Website - Title, Authors (optional), Year/Access Date - URL (required) ## PDF Text Extraction ### Strategy 1. **On Upload**: Quick metadata extraction (if available in PDF) 2. **Background Job**: Full text extraction using: - `pdftotext` (poppler-utils) for standard PDFs - OCR fallback for scanned documents (future) ### Extracted Content Uses 1. Full-text search within user's library 2. Grounding data for AI generation 3. Auto-suggest keywords ## Integration with Write with AI ### Modified Search Sources Panel ``` Search Sources: ☑ Shamra Academia Library ☐ ArXiv Repository ☐ My References (12 papers) <-- NEW ``` ### When "My References" is selected: 1. Search user's reference library using extracted text 2. Return matching references with metadata 3. Include in grounding context for generation 4. Properly cite in References section ## UI Design ### /myreferences Page Layout ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ 📚 My Reference Library [+ Upload Reference] │ ├─────────────────────────────────────────────────────────────────────────────┤ │ 🔍 Search... [Filter ▼] [Sort ▼] [View: Grid/List] │ ├───────────────┬─────────────────────────────────────────────────────────────┤ │ │ │ │ Folders │ Reference Cards / List │ │ ─────────── │ ┌─────────────────────────────────────────────────────┐ │ │ 📁 All (24) │ │ 📄 Machine Learning Approaches to NLP │ │ │ 📁 AI (8) │ │ Smith, J., Johnson, M. (2024) │ │ │ 📁 NLP (12) │ │ Journal of AI Research, Vol. 15, pp. 123-145 │ │ │ 📁 Other (4) │ │ [Edit] [Delete] [Download] [View] │ │ │ │ └─────────────────────────────────────────────────────┘ │ │ [+ Folder] │ ┌─────────────────────────────────────────────────────┐ │ │ │ │ 📄 Deep Learning for Translation │ │ │ │ │ ... │ │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ │ └───────────────┴─────────────────────────────────────────────────────────────┘ ``` ### Upload Modal ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ 📤 Upload Reference [X] │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ │ │ │ │ │ 📄 Drag & drop PDF here or click to browse │ │ │ │ (Max 50MB) │ │ │ │ │ │ │ └──────────────────────────────────────────────────────────────────────┘ │ │ │ │ Publication Type: [Journal Article ▼] │ │ │ │ ═══════════════════════ Required ═══════════════════════ │ │ │ │ Title*: [_______________________________________________] │ │ Authors*: [_______________________________________________] │ │ (Separate multiple authors with commas) │ │ Year*: [____] │ │ │ │ ═══════════════════════ Publication Details ═══════════════════════ │ │ │ │ Journal/Book: [_______________________________________________] │ │ Volume: [______] Issue: [______] Pages: [__________] │ │ Publisher: [_______________________________________________] │ │ │ │ ═══════════════════════ Identifiers ═══════════════════════ │ │ │ │ DOI: [_______________________________________________] │ │ ISBN/ISSN: [_______________________________________________] │ │ URL: [_______________________________________________] │ │ │ │ ═══════════════════════ Content ═══════════════════════ │ │ │ │ Abstract: [_______________________________________________] │ │ [_______________________________________________] │ │ Keywords: [_______________________________________________] │ │ Language: [English ▼] │ │ Folder: [Select or create... ▼] │ │ Notes: [_______________________________________________] │ │ │ │ [Cancel] [Upload & Save] │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` ## Citation Formats Supported ### APA 7th Edition ``` Smith, J., & Johnson, M. (2024). Machine learning approaches to NLP. Journal of AI Research, 15(3), 123-145. https://doi.org/10.1234/example ``` ### MLA 9th Edition ``` Smith, John, and Mary Johnson. "Machine Learning Approaches to NLP." Journal of AI Research, vol. 15, no. 3, 2024, pp. 123-145. ``` ### Chicago ``` Smith, John, and Mary Johnson. "Machine Learning Approaches to NLP." Journal of AI Research 15, no. 3 (2024): 123-145. ``` ### IEEE ``` J. Smith and M. Johnson, "Machine learning approaches to NLP," J. AI Res., vol. 15, no. 3, pp. 123-145, 2024. ``` ## Security Considerations 1. **File Validation** - Check MIME type (must be application/pdf) - Verify file magic bytes - Scan for malware (optional) - Max file size: 50MB 2. **Access Control** - Users can only access their own references - File storage uses UUID names (not guessable) - Download requires authentication 3. **Storage Quotas** - Per-user storage limit (e.g., 500MB or 50 references) - Show usage in UI ## Implementation Phases ### Phase 1: Core Functionality - [ ] Entity and migrations - [ ] File upload service - [ ] CRUD API endpoints - [ ] Basic /myreferences page - [ ] Upload modal with metadata form ### Phase 2: Text Extraction - [ ] PDF text extraction service - [ ] Background job for processing - [ ] Full-text search ### Phase 3: Write with AI Integration - [ ] Add "My References" checkbox - [ ] Search user references - [ ] Include in grounding context - [ ] Proper citation generation ### Phase 4: Enhancements - [ ] Folder organization - [ ] Multiple citation formats - [ ] Bulk upload - [ ] DOI auto-lookup - [ ] Duplicate detection