Commit Graph

19 Commits

Author SHA1 Message Date
baf497b015 feat(maturity): add dedicated NSFW/maturity analysis service
- Add maturity/ service (FastAPI + Falconsai/nsfw_image_detection ViT classifier)
  - /analyze (URL) and /analyze/file (multipart upload) endpoints
  - Normalized response: maturity_label, confidence, score, labels,
    action_hint (safe/review/flag_high), advisory, threshold_used,
    analysis_time_ms, model, source
  - Configurable thresholds via MATURITY_THRESHOLD_MATURE / MATURITY_THRESHOLD_REVIEW
  - Reuses common/image_io for URL validation and file-size enforcement
  - Explicit 502/503 errors on failure — no silent safe fallback
  - Per-request structured logging (score, label, threshold path, elapsed ms)

- Update gateway/main.py
  - MATURITY_URL + MATURITY_ENABLED env vars
  - POST /analyze/maturity and POST /analyze/maturity/file endpoints
  - /health includes maturity service status
  - _assert_maturity_enabled() guard for clean 503 when disabled
  - All existing endpoints untouched (additive change)

- Update docker-compose.yml
  - Add maturity service with healthcheck (start_period: 90s)
  - Gateway environment: MATURITY_URL, MATURITY_ENABLED
  - Gateway depends_on: maturity (service_healthy)

- Update README.md and USAGE.md
  - Document maturity service, env vars, curl examples,
    full response schema table, action_hint logic, failure guidance
2026-04-11 17:29:26 +02:00
f681ab980d fix: coverage_pct and ram_estimate use points_count (vectors_count deprecated in Qdrant v1.x) 2026-03-31 20:36:23 +02:00
3f925e17d5 docs: update README and USAGE for card-renderer, Qdrant optimization endpoints, and search params 2026-03-31 20:16:55 +02:00
6ea91c3452 fix: quantization in /configure, HNSW defaults in POST /collections, filter_metadata in search/file 2026-03-31 20:08:58 +02:00
609485a0f0 fix(qdrant): complete optimization gaps from v1
- qdrant/main.py: search/file now accepts hnsw_ef, exact, indexed_only form fields
  (was silently ignoring them, using server defaults only)
- qdrant/main.py: add GET /inspect endpoint — full diagnostic summary for all
  collections: HNSW, optimizer, quantization, segment count, payload index coverage,
  raw RAM estimate (vectors * dim * 4B * 1.5)
- gateway/main.py: vectors/search/file now forwards hnsw_ef, exact, indexed_only
- gateway/main.py: add GET /vectors/inspect proxy
2026-03-31 20:01:52 +02:00
c7ea347e2b feat(qdrant): optimization — payload indexes, HNSW tuning, search params (v1)
Inspection findings:
- _ensure_collection() created collections with bare VectorParams (no HNSW/optimizer config)
- _do_search() had no SearchParams — used Qdrant defaults (ef often ~100, no indexed_only)
- No payload index management at all — filtered searches scanned unindexed fields every time
- collection_info() returned minimal data — impossible to inspect production state
- No way to create/ensure payload indexes via the API

Changes — qdrant/main.py:
- Add SEARCH_HNSW_EF env var (default 128, above Qdrant default for better recall)
- _ensure_collection(): configure HnswConfigDiff(m=16, ef_construct=200, on_disk=False)
  and OptimizersConfigDiff(indexing_threshold=20000, default_segment_number=4) on creation
- _do_search(): use SearchParams(hnsw_ef, exact, indexed_only) on every query
- SearchUrlRequest + SearchVectorRequest: expose hnsw_ef, exact, indexed_only per request
- collection_info(): expand to full HNSW/optimizer/quantization/segment/payload_schema detail
- GET  /collections/{name}/indexes     — list all payload indexes
- POST /collections/{name}/indexes     — create a single payload index
- POST /collections/{name}/ensure-indexes — idempotent bulk index creation (skip existing)
- POST /collections/{name}/configure   — apply HNSW/optimizer changes to existing collections

Changes — gateway/main.py:
- Expose the 4 new qdrant-svc endpoints under /vectors/collections/{name}/...

Changes — docker-compose.yml:
- Add SEARCH_HNSW_EF=128 to qdrant-svc environment

Critical usage note for existing collections:
  After deploying, call POST /vectors/collections/images/ensure-indexes with the
  payload fields actually used in filter_metadata (is_public, category_id, etc.)
  to add missing indexes. This is the highest-impact single action for filtered search.
2026-03-31 19:58:47 +02:00
58ee1b3bdd feat: add card-renderer internal service (v1)
- New card-renderer FastAPI service (Python 3.11 + Pillow)
  - GET /health, GET /templates
  - POST /render (URL input)
  - POST /render/file (multipart upload)
  - POST /render/meta (dry-run layout metadata)
  - nova-artwork-v1 template: cover crop, gradient overlay, text, logo
  - SSRF-safe async image fetch with redirect validation
  - Smart center cover crop isolated for future YOLO focal-point support
  - Graceful font/logo fallback when assets are absent

- docker-compose.yml: add card-renderer service + healthcheck;
  extend gateway with CARD_RENDERER_URL and depends_on

- gateway/main.py: proxy endpoints under /cards/*
  - GET  /cards/templates
  - POST /cards/render
  - POST /cards/render/file
  - POST /cards/render/meta
  All protected by existing APIKeyMiddleware
2026-03-31 10:39:29 +02:00
root
613023de86 Update 2026-03-23 20:37:44 +01:00
b8c44bd1b2 Persist Qdrant to host: bind-mount ./data/qdrant; add data dir ignore; update docs 2026-03-23 19:52:43 +01:00
8f758cf3b5 qdrant: enforce int/UUID point ids; preserve original id in payload; add error handling 2026-03-21 10:12:34 +01:00
4218034d70 Add HUGGINGFACE_TOKEN passthrough for clip/blip and update .env.example 2026-03-21 10:09:04 +01:00
ecf8c9a401 Fix: use pydantic pattern; handle httpx RequestError in gateway and qdrant 2026-03-21 10:07:20 +01:00
e6da9924ff gateway: handle non-JSON upstream responses with 502 and upstream text 2026-03-21 09:43:37 +01:00
7208c45768 Fix docker-compose YAML: env_file and service block indentations 2026-03-21 09:40:40 +01:00
3a015eb3b9 docker-compose: load .env and use for gateway 2026-03-21 09:39:32 +01:00
7a795e7fd3 Add .env.example 2026-03-21 09:31:03 +01:00
788e193c8a Docs: add X-API-Key header to curl examples 2026-03-21 09:22:29 +01:00
3b300ab3b4 Add Qdrant vectorstore service, gateway API-key middleware, docs and .gitignore 2026-03-21 09:16:11 +01:00
8da669c0e1 first commit 2026-03-21 09:09:28 +01:00