236 lines
7.8 KiB
Markdown
236 lines
7.8 KiB
Markdown
# Skinbase Vision Stack (CLIP + BLIP + YOLO + Qdrant + Card Renderer) – Dockerized FastAPI
|
||
|
||
This repository provides **five standalone vision services** (CLIP / BLIP / YOLO / Qdrant / Card Renderer)
|
||
and a **Gateway API** that can call them individually or together.
|
||
|
||
## Services & Ports
|
||
|
||
- `gateway` (exposed): `https://vision.klevze.net`
|
||
- `clip`: internal only
|
||
- `blip`: internal only
|
||
- `yolo`: internal only
|
||
- `qdrant`: vector DB (port `6333` exposed for direct access)
|
||
- `qdrant-svc`: internal Qdrant API wrapper
|
||
- `card-renderer`: internal card rendering service
|
||
|
||
## Run
|
||
|
||
```bash
|
||
docker compose up -d --build
|
||
```
|
||
|
||
If you use BLIP, create a `.env` file first.
|
||
|
||
Required variables:
|
||
|
||
```bash
|
||
API_KEY=your_api_key_here
|
||
HUGGINGFACE_TOKEN=your_huggingface_token_here
|
||
```
|
||
|
||
`HUGGINGFACE_TOKEN` is required when the configured BLIP model is private, gated, or otherwise requires Hugging Face authentication.
|
||
|
||
Service startup now waits on container healthchecks, so first boot may take longer while models finish loading.
|
||
|
||
## Health
|
||
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/health
|
||
```
|
||
|
||
## Universal analyze (ALL)
|
||
|
||
### With URL
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/all \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","limit":5}'
|
||
```
|
||
|
||
### With file upload (multipart)
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/all/file \
|
||
-F "file=@/path/to/image.webp" \
|
||
-F "limit=5"
|
||
```
|
||
|
||
## Individual services (via gateway)
|
||
|
||
### CLIP tags
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/clip -H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","limit":5}'
|
||
```
|
||
|
||
### CLIP tags (file)
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/clip/file \
|
||
-F "file=@/path/to/image.webp" \
|
||
-F "limit=5"
|
||
```
|
||
|
||
### BLIP caption
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/blip -H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","variants":3}'
|
||
```
|
||
|
||
### BLIP caption (file)
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/blip/file \
|
||
-F "file=@/path/to/image.webp" \
|
||
-F "variants=3" \
|
||
-F "max_length=60"
|
||
```
|
||
|
||
### YOLO detect
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/yolo -H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","conf":0.25}'
|
||
```
|
||
|
||
### YOLO detect (file)
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/analyze/yolo/file \
|
||
-F "file=@/path/to/image.webp" \
|
||
-F "conf=0.25"
|
||
```
|
||
|
||
## Vector DB (Qdrant) via gateway
|
||
|
||
Qdrant point IDs must be either:
|
||
|
||
- an unsigned integer
|
||
- a UUID string
|
||
|
||
If you send another string value, the wrapper may replace it with a generated UUID. In that case the original value is stored in the payload as `_original_id`.
|
||
|
||
You can fetch a stored point by its preserved original application ID:
|
||
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/points/by-original-id/img-001
|
||
```
|
||
|
||
### Store image embedding by URL
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/upsert \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","id":"550e8400-e29b-41d4-a716-446655440000","metadata":{"category":"wallpaper"}}'
|
||
```
|
||
|
||
### Store image embedding by file upload
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/upsert/file \
|
||
-F "file=@/path/to/image.webp" \
|
||
-F 'id=550e8400-e29b-41d4-a716-446655440001' \
|
||
-F 'metadata_json={"category":"photo"}'
|
||
```
|
||
|
||
### Search similar images by URL
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/search \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","limit":5,"filter_metadata":{"is_public":true}}'
|
||
```
|
||
|
||
Optional search parameters: `hnsw_ef` (int), `exact` (bool), `indexed_only` (bool), `score_threshold` (float), `filter_metadata` (object).
|
||
|
||
### Search similar images by file upload
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/search/file \
|
||
-F "file=@/path/to/image.webp" \
|
||
-F "limit=5" \
|
||
-F 'filter_metadata_json={"is_public":true}'
|
||
```
|
||
|
||
### List collections
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/collections
|
||
```
|
||
|
||
### Get collection info
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/collections/images
|
||
```
|
||
|
||
### Full diagnostic inspect
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/inspect
|
||
```
|
||
|
||
Returns HNSW config, optimizer config, quantization, segment count, payload index coverage, and RAM estimate for every collection.
|
||
|
||
### Payload index management
|
||
```bash
|
||
# List indexes
|
||
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/vectors/collections/images/indexes
|
||
|
||
# Create a single index
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/collections/images/indexes \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"field":"is_public","type":"bool"}'
|
||
|
||
# Ensure multiple indexes (idempotent)
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/collections/images/ensure-indexes \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"fields":[{"field":"is_public","type":"bool"},{"field":"category_id","type":"integer"}]}'
|
||
```
|
||
|
||
Supported index types: `keyword`, `integer`, `float`, `bool`, `geo`, `datetime`, `text`, `uuid`.
|
||
|
||
### Collection configuration (HNSW / optimizer / quantization)
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/collections/images/configure \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"hnsw_m":16,"hnsw_ef_construct":200,"indexing_threshold":20000,"quantization_type":"int8"}'
|
||
```
|
||
|
||
### Delete points
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/vectors/delete \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"ids":["550e8400-e29b-41d4-a716-446655440000","550e8400-e29b-41d4-a716-446655440001"]}'
|
||
```
|
||
|
||
If you let the wrapper generate a UUID, use the returned `id` value for later `get`, `search`, or `delete` operations.
|
||
|
||
## Card Renderer
|
||
|
||
### List available templates
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" https://vision.klevze.net/cards/templates
|
||
```
|
||
|
||
### Render a card from a URL
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/cards/render \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","title":"Artwork Title","username":"@artist","template":"nova-artwork-v1"}'
|
||
```
|
||
|
||
Returns binary image bytes (WebP by default).
|
||
|
||
### Render a card from a file upload
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/cards/render/file \
|
||
-F "file=@/path/to/image.webp" \
|
||
-F "title=Artwork Title" \
|
||
-F "username=@artist" \
|
||
-F "template=nova-artwork-v1"
|
||
```
|
||
|
||
### Get card layout metadata (no image rendered)
|
||
```bash
|
||
curl -H "X-API-Key: <your-api-key>" -X POST https://vision.klevze.net/cards/render/meta \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"url":"https://files.skinbase.org/img/aa/bb/cc/md.webp","title":"Artwork Title"}'
|
||
```
|
||
|
||
## Notes
|
||
|
||
- This is a **starter scaffold**. Models are loaded at service startup.
|
||
- Qdrant data is persisted in the project folder at `./data/qdrant`, so it survives container restarts and recreates.
|
||
- Remote image URLs are restricted to public `http`/`https` hosts. Localhost, private IP ranges, and non-image content types are rejected.
|
||
- For production: add auth, rate limits, and restrict gateway exposure (private network).
|
||
- GPU: you can add NVIDIA runtime later (compose profiles) if needed.
|