vision/README.md

# Skinbase Vision Stack (CLIP + BLIP + YOLO + Qdrant + Card Renderer + Maturity) – Dockerized FastAPI

This repository provides **six standalone vision services** (CLIP / BLIP / YOLO / Qdrant / Card Renderer / Maturity)
and a **Gateway API** that can call them individually or together.

## Services & Ports

- `gateway` (exposed): `https://vision.klevze.net`
- `clip`: internal only
- `blip`: internal only
- `yolo`: internal only
- `qdrant`: vector DB (port `6333` exposed for direct access)
- `qdrant-svc`: internal Qdrant API wrapper
- `card-renderer`: internal card rendering service
- `maturity`: internal NSFW/maturity classifier service

## Run

```bash
docker compose up -d --build
```

If you use BLIP, create a `.env` file first.

Required variables:

```bash
API_KEY=your_api_key_here
HUGGINGFACE_TOKEN=your_huggingface_token_here
```

`HUGGINGFACE_TOKEN` is required when the configured BLIP model is private, gated, or otherwise requires Hugging Face authentication.

Optional maturity configuration (override in `.env` if needed):

```bash
MATURITY_MODEL=Falconsai/nsfw_image_detection
MATURITY_THRESHOLD_MATURE=0.80
MATURITY_THRESHOLD_REVIEW=0.60
MATURITY_ENABLED=true
```

Service startup now waits on container healthchecks, so first boot may take longer while models finish loading.

## Health

```bash
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/health
```

## Universal analyze (ALL)

### With URL
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/all \
  -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","limit":5}'
```

### With file upload (multipart)
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/all/file \
  -F "file=@/path/to/image.webp" \
  -F "limit=5"
```

## Individual services (via gateway)

### CLIP tags
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/clip -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","limit":5}'
```

### CLIP tags (file)
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/clip/file \
  -F "file=@/path/to/image.webp" \
  -F "limit=5"
```

### BLIP caption
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/blip -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","variants":3}'
```

### BLIP caption (file)
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/blip/file \
  -F "file=@/path/to/image.webp" \
  -F "variants=3" \
  -F "max_length=60"
```

### YOLO detect
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/yolo -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","conf":0.25}'
```

### YOLO detect (file)
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/yolo/file \
  -F "file=@/path/to/image.webp" \
  -F "conf=0.25"
```

## Maturity / NSFW analysis

Analyzes an image and returns a normalized maturity signal for Nova moderation workflows.

### Analyze by URL
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/maturity \
  -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp"}'
```

### Analyze from file upload
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/maturity/file \
  -F "file=@/path/to/image.webp"
```

Example response:
```json
{
  "maturity_label": "mature",
  "confidence": 0.94,
  "score": 0.94,
  "labels": ["nsfw"],
  "model": "Falconsai/nsfw_image_detection",
  "threshold_used": 0.80,
  "analysis_time_ms": 183.0,
  "source": "maturity-service",
  "action_hint": "flag_high",
  "advisory": "High-confidence mature content detected"
}
```

`action_hint` values: `safe`, `review`, `flag_high`. Nova should use these to decide blur/queue/flag behaviour.

## Vector DB (Qdrant) via gateway

Qdrant point IDs must be either:

- an unsigned integer
- a UUID string

If you send another string value, the wrapper may replace it with a generated UUID. In that case the original value is stored in the payload as `_original_id`.

You can fetch a stored point by its preserved original application ID:

```bash
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/points/by-original-id/img-001
```

### Store image embedding by URL
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/upsert \
  -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","id":"550e8400-e29b-41d4-a716-446655440000","metadata":{"category":"wallpaper"}}'
```

### Store image embedding by file upload
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/upsert/file \
  -F "file=@/path/to/image.webp" \
  -F 'id=550e8400-e29b-41d4-a716-446655440001' \
  -F 'metadata_json={"category":"photo"}'
```

### Search similar images by URL
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/search \
  -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","limit":5,"filter_metadata":{"is_public":true}}'
```

Optional search parameters: `hnsw_ef` (int), `exact` (bool), `indexed_only` (bool), `score_threshold` (float), `filter_metadata` (object).

### Search similar images by file upload
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/search/file \
  -F "file=@/path/to/image.webp" \
  -F "limit=5" \
  -F 'filter_metadata_json={"is_public":true}'
```

### List collections
```bash
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/collections
```

### Get collection info
```bash
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/collections/images
```

### Full diagnostic inspect
```bash
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/inspect
```

Returns HNSW config, optimizer config, quantization, segment count, payload index coverage, and RAM estimate for every collection.

### Payload index management
```bash
# List indexes
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/collections/images/indexes

# Create a single index
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/collections/images/indexes \
  -H "Content-Type: application/json" \
  -d '{"field":"is_public","type":"bool"}'

# Ensure multiple indexes (idempotent)
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/collections/images/ensure-indexes \
  -H "Content-Type: application/json" \
  -d '{"fields":[{"field":"is_public","type":"bool"},{"field":"category_id","type":"integer"}]}'
```

Supported index types: `keyword`, `integer`, `float`, `bool`, `geo`, `datetime`, `text`, `uuid`.

### Collection configuration (HNSW / optimizer / quantization)
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/collections/images/configure \
  -H "Content-Type: application/json" \
  -d '{"hnsw_m":16,"hnsw_ef_construct":200,"indexing_threshold":20000,"quantization_type":"int8"}'
```

### Delete points
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/delete \
  -H "Content-Type: application/json" \
  -d '{"ids":["550e8400-e29b-41d4-a716-446655440000","550e8400-e29b-41d4-a716-446655440001"]}'
```

If you let the wrapper generate a UUID, use the returned `id` value for later `get`, `search`, or `delete` operations.

## Card Renderer

### List available templates
```bash
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/cards/templates
```

### Render a card from a URL
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/cards/render \
  -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","title":"Artwork Title","username":"@artist","template":"nova-artwork-v1"}'
```

Returns binary image bytes (WebP by default).

### Render a card from a file upload
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/cards/render/file \
  -F "file=@/path/to/image.webp" \
  -F "title=Artwork Title" \
  -F "username=@artist" \
  -F "template=nova-artwork-v1"
```

### Get card layout metadata (no image rendered)
```bash
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/cards/render/meta \
  -H "Content-Type: application/json" \
  -d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","title":"Artwork Title"}'
```

## Notes

- Models are loaded at service startup; initial container start can take 1–2 minutes as model weights are downloaded.
- Qdrant data is persisted in the project folder at `./data/qdrant`, so it survives container restarts and recreates.
- Remote image URLs are restricted to public `http`/`https` hosts. Localhost, private IP ranges, and non-image content types are rejected.
- The maturity service uses `Falconsai/nsfw_image_detection` (ViT-based). Thresholds are configurable via `.env`. The model handles photos and stylized digital art but should be calibrated against real Skinbase content before production use.
- For production: add auth, rate limits, and restrict gateway exposure (private network).
- GPU: you can add NVIDIA runtime later (compose profiles) if needed.