optimizations

docs/recommendation-ai-production.md

@@ -0,0 +1,216 @@
+# Recommendation AI Production Readiness
+
+This runbook covers the production prerequisites for the Skinbase recommendation AI stack:
+
+- vision analysis via `VISION_GATEWAY_URL` or CLIP/YOLO endpoints
+- embedding generation
+- vector upsert/search via the vector gateway
+- hybrid discovery feed v2/v3
+- upload-triggered AI processing
+
+## 1. Required runtime services
+
+You need these services available before enabling the stack:
+
+- Laravel queue workers
+- a configured public files/CDN base so artwork derivatives resolve to public URLs
+- a vision gateway for `/analyze/all`, or CLIP and YOLO services individually
+- a vector gateway exposing `/vectors/upsert` and `/vectors/search`
+- a queue backend suitable for async jobs, preferably Redis in production
+
+## 2. Required environment variables
+
+### Core queue and upload
+
+```dotenv
+QUEUE_CONNECTION=redis
+UPLOAD_QUEUE_DERIVATIVES=true
+VISION_QUEUE=vision
+RECOMMENDATIONS_QUEUE=recommendations
+DISCOVERY_QUEUE=discovery
+```
+
+If you do not want dedicated queues, leave the queue names unset or set them to `default`.
+
+### Vision analysis
+
+```dotenv
+VISION_ENABLED=true
+VISION_IMAGE_VARIANT=md
+
+VISION_GATEWAY_URL=https://vision.example.com
+VISION_GATEWAY_TIMEOUT=10
+VISION_GATEWAY_CONNECT_TIMEOUT=3
+
+CLIP_BASE_URL=https://clip.example.com
+CLIP_ANALYZE_ENDPOINT=/analyze
+CLIP_EMBED_ENDPOINT=/embed
+
+YOLO_ENABLED=true
+YOLO_BASE_URL=https://yolo.example.com
+YOLO_ANALYZE_ENDPOINT=/analyze
+YOLO_PHOTOGRAPHY_ONLY=true
+```
+
+Notes:
+
+- `VISION_GATEWAY_URL` is the preferred unified path because the app can ingest CLIP tags, BLIP caption, and YOLO objects from `/analyze/all`
+- CLIP embedding generation still uses the CLIP embed endpoint configured in `config/recommendations.php`
+
+### Vector search and indexing
+
+```dotenv
+VISION_VECTOR_GATEWAY_ENABLED=true
+VISION_VECTOR_GATEWAY_URL=https://vector.example.com
+VISION_VECTOR_GATEWAY_API_KEY=replace-me
+VISION_VECTOR_GATEWAY_COLLECTION=images
+VISION_VECTOR_GATEWAY_UPSERT_ENDPOINT=/vectors/upsert
+VISION_VECTOR_GATEWAY_SEARCH_ENDPOINT=/vectors/search
+VISION_VECTOR_GATEWAY_DELETE_ENDPOINT=/vectors/delete
+```
+
+### Discovery and rollout
+
+```dotenv
+DISCOVERY_V2_ENABLED=true
+DISCOVERY_V2_ALGO_VERSION=clip-cosine-v2-adaptive
+DISCOVERY_V2_ROLLOUT_PERCENTAGE=100
+
+DISCOVERY_V3_ENABLED=true
+DISCOVERY_V3_CACHE_TTL_MINUTES=5
+DISCOVERY_V3_VECTOR_SIMILARITY_WEIGHT=0.8
+DISCOVERY_V3_VECTOR_BASE_SCORE=0.75
+DISCOVERY_V3_MAX_SEED_ARTWORKS=3
+DISCOVERY_V3_VECTOR_CANDIDATE_POOL=60
+```
+
+### Embeddings
+
+```dotenv
+RECOMMENDATIONS_EMBEDDING_ENABLED=true
+RECOMMENDATIONS_EMBEDDING_MODEL=clip
+RECOMMENDATIONS_EMBEDDING_MODEL_VERSION=v1
+RECOMMENDATIONS_ALGO_VERSION=clip-cosine-v1
+RECOMMENDATIONS_MIN_DIM=64
+RECOMMENDATIONS_MAX_DIM=4096
+```
+
+## 3. Queue worker coverage
+
+The AI stack dispatches jobs onto these queue families:
+
+- `vision` via `AutoTagArtworkJob`
+- `recommendations` via embedding generation, vector backfill, and similarity jobs
+- `discovery` via feed cache regeneration
+- `default` if dedicated queue env vars are not set
+
+The example worker configs in `deploy/` now listen on:
+
+```text
+forum-security,forum-moderation,vision,recommendations,discovery,mail,default
+```
+
+If you run separate workers per queue, ensure all configured queue names are consumed somewhere.
+
+## 4. Upload pipeline behavior
+
+Upload finish dispatches these async jobs after derivatives exist:
+
+- `AutoTagArtworkJob`
+- `GenerateArtworkEmbeddingJob`
+
+If `UPLOAD_QUEUE_DERIVATIVES=true`, derivative generation is also queued before those jobs can run. That means a broken worker setup can block the entire AI post-processing chain even if uploads themselves succeed.
+
+## 5. Database and schema expectations
+
+Confirm the database includes the current AI/discovery schema:
+
+- artwork AI metadata columns on `artworks`
+- `last_vector_indexed_at` on `artworks`
+- artwork embeddings table
+- discovery cache and event tables used by feed generation
+
+Recommended checks:
+
+```bash
+php artisan migrate --force
+php artisan schema:audit-migrations
+```
+
+## 6. Deployment checklist
+
+Run these after deploy:
+
+```bash
+php artisan config:cache
+php artisan route:cache
+php artisan queue:restart
+php artisan optimize
+```
+
+If you changed frontend assets, also ensure the Vite manifest exists before smoke-testing browser flows.
+
+## 7. Smoke tests
+
+Recommended application-level checks:
+
+```bash
+php artisan artworks:vectors-index --limit=1
+php artisan artworks:vectors-search {artwork_id} --limit=5
+vendor/bin/pest tests/Feature/Vision/GenerateArtworkEmbeddingJobTest.php
+vendor/bin/pest tests/Feature/Vision/AiArtworkSearchApiTest.php
+vendor/bin/pest tests/Feature/Discovery/FeedEndpointV2Test.php
+```
+
+Recommended HTTP checks:
+
+- `GET /api/art/{id}/similar-ai`
+- `POST /api/search/image`
+- `GET /api/v1/feed?algo_version=clip-cosine-v2-adaptive&limit=12`
+
+## 8. Failure modes to watch
+
+### Uploads succeed but AI fields stay empty
+
+Likely causes:
+
+- workers are not running
+- workers do not consume `vision` or `recommendations`
+- `VISION_ENABLED=false`
+- derivative URLs are not publicly reachable by the vision services
+
+### Similar AI endpoints return 503
+
+Likely causes:
+
+- `VISION_VECTOR_GATEWAY_ENABLED=false`
+- missing `VISION_VECTOR_GATEWAY_URL`
+- missing `VISION_VECTOR_GATEWAY_API_KEY`
+
+### Feed works but has no vector influence
+
+Likely causes:
+
+- `DISCOVERY_V3_ENABLED=false`
+- vector gateway search unavailable
+- no recent user seed artworks
+- artworks have local embeddings but were never upserted
+
+### Vector repair/backfill stalls
+
+Likely causes:
+
+- `RECOMMENDATIONS_QUEUE` is set but workers do not listen on it
+- queue backend is unhealthy
+
+## 9. Operational recommendation
+
+For first rollout, keep the queue names explicit but simple:
+
+```dotenv
+VISION_QUEUE=default
+RECOMMENDATIONS_QUEUE=default
+DISCOVERY_QUEUE=default
+```
+
+Once the stack is stable, split them into dedicated workers only if queue volume justifies it.