Allow heading tags (h1-h6) in ContentSanitizer so news editor headings render
This commit is contained in:
209
services/enhance-worker/README.md
Normal file
209
services/enhance-worker/README.md
Normal file
@@ -0,0 +1,209 @@
|
||||
# Skinbase Enhance Worker
|
||||
|
||||
This service is the private HTTP worker for the Skinbase Enhance `external_worker` engine. Laravel remains responsible for job state, permissions, queueing, moderation, storage, and cleanup. The worker only downloads the copied Enhance source image, upscales it, exposes a temporary result, and deletes that temporary result when Laravel is done.
|
||||
|
||||
## Local build and run
|
||||
|
||||
```bash
|
||||
cd services/enhance-worker
|
||||
docker compose -f docker-compose.example.yml up --build
|
||||
curl http://127.0.0.1:8095/health
|
||||
```
|
||||
|
||||
The example compose file uses `WORKER_ENGINE=pillow` so local development works without Real-ESRGAN weights.
|
||||
|
||||
## Real-ESRGAN ncnn runtime
|
||||
|
||||
The worker supports these engine values:
|
||||
|
||||
- `WORKER_ENGINE=pillow`
|
||||
- `WORKER_ENGINE=realesrgan-ncnn`
|
||||
- `WORKER_ENGINE=realesrgan`
|
||||
|
||||
`realesrgan` currently aliases to the ncnn-vulkan CLI path. Pillow mode remains the safe local and CI fallback.
|
||||
|
||||
The worker expects the Real-ESRGAN ncnn runtime at `/app/bin/realesrgan-ncnn-vulkan` and model files under `/app/models`. These files are not committed. Keep `bin/` and `models/` private to your deployment or local environment.
|
||||
|
||||
## Endpoints
|
||||
|
||||
- `GET /health`
|
||||
- `POST /v1/upscale` with `Authorization: Bearer <WORKER_TOKEN>`
|
||||
- `GET /v1/results/{filename}` for temporary internal result download
|
||||
- `DELETE /v1/results/{filename}` with bearer token for Laravel cleanup
|
||||
|
||||
## Worker env
|
||||
|
||||
```env
|
||||
WORKER_HOST=0.0.0.0
|
||||
WORKER_PORT=8095
|
||||
WORKER_TOKEN=change-this-token
|
||||
|
||||
WORKER_ENGINE=pillow
|
||||
WORKER_DEVICE=cpu
|
||||
|
||||
WORKER_MAX_UPLOAD_MB=20
|
||||
WORKER_MAX_INPUT_WIDTH=4096
|
||||
WORKER_MAX_INPUT_HEIGHT=4096
|
||||
WORKER_MAX_OUTPUT_WIDTH=8192
|
||||
WORKER_MAX_OUTPUT_HEIGHT=8192
|
||||
|
||||
WORKER_TMP_DIR=/app/storage/tmp
|
||||
WORKER_OUTPUT_DIR=/app/storage/output
|
||||
WORKER_RESULT_TTL_MINUTES=60
|
||||
|
||||
WORKER_MODEL_DIR=/app/app/models
|
||||
WORKER_DEFAULT_MODEL=realesrgan-x4plus
|
||||
|
||||
WORKER_REALESRGAN_BIN=/app/bin/realesrgan-ncnn-vulkan
|
||||
WORKER_REALESRGAN_MODEL_DIR=/app/models
|
||||
WORKER_REALESRGAN_DEFAULT_MODEL=realesrgan-x4plus
|
||||
WORKER_REALESRGAN_ANIME_MODEL=realesrgan-x4plus-anime
|
||||
WORKER_REALESRGAN_TILE=0
|
||||
WORKER_REALESRGAN_TTA=false
|
||||
WORKER_REALESRGAN_VERBOSE=false
|
||||
WORKER_REALESRGAN_TIMEOUT_SECONDS=900
|
||||
WORKER_REALESRGAN_PREPROCESS_MAX_PIXELS=16777216
|
||||
WORKER_REALESRGAN_OUTPUT_EXT=webp
|
||||
WORKER_REALESRGAN_ALLOW_MODEL_FALLBACK=true
|
||||
```
|
||||
|
||||
## Laravel env
|
||||
|
||||
```env
|
||||
ENHANCE_ENGINE=external_worker
|
||||
ENHANCE_WORKER_URL=http://127.0.0.1:8095
|
||||
ENHANCE_WORKER_TIMEOUT=600
|
||||
ENHANCE_WORKER_TOKEN=change-this-token
|
||||
ENHANCE_WORKER_MAX_DOWNLOAD_MB=60
|
||||
ENHANCE_QUEUE=enhance
|
||||
```
|
||||
|
||||
After updating Laravel env:
|
||||
|
||||
```bash
|
||||
php artisan config:clear
|
||||
php artisan enhance:health
|
||||
php artisan queue:work --queue=enhance,default
|
||||
php artisan test --filter=EnhanceExternalWorker
|
||||
```
|
||||
|
||||
## Example request
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:8095/v1/upscale \
|
||||
-H "Authorization: Bearer change-this-token" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"job_id": 1,
|
||||
"source_url": "https://example.com/test.webp",
|
||||
"scale": 2,
|
||||
"mode": "artwork",
|
||||
"output_format": "webp"
|
||||
}'
|
||||
```
|
||||
|
||||
## Installing runtime files
|
||||
|
||||
```bash
|
||||
cd services/enhance-worker
|
||||
bash scripts/download-realesrgan-ncnn.sh
|
||||
bash scripts/verify-realesrgan.sh
|
||||
```
|
||||
|
||||
You can override the upstream release tag with:
|
||||
|
||||
```bash
|
||||
bash scripts/download-realesrgan-ncnn.sh --version v0.2.0
|
||||
```
|
||||
|
||||
If you already manage the binary and models yourself, place them in `bin/` and `models/` instead of using the helper script.
|
||||
|
||||
## Model files
|
||||
|
||||
Expected examples:
|
||||
|
||||
- `realesrgan-x4plus.param`
|
||||
- `realesrgan-x4plus.bin`
|
||||
- `realesrgan-x4plus-anime.param`
|
||||
- `realesrgan-x4plus-anime.bin`
|
||||
|
||||
Illustration mode prefers the anime model. If that model is missing and `WORKER_REALESRGAN_ALLOW_MODEL_FALLBACK=true`, the worker falls back to the default model and records the fallback in response metadata.
|
||||
|
||||
## Running in Pillow mode
|
||||
|
||||
- `docker compose -f docker-compose.example.yml up --build`
|
||||
- health should report `engine: pillow`
|
||||
- metadata reports `real_ai_upscale: false`
|
||||
|
||||
## Running in Real-ESRGAN mode
|
||||
|
||||
```bash
|
||||
cd services/enhance-worker
|
||||
docker compose -f docker-compose.realesrgan.example.yml up --build
|
||||
curl http://127.0.0.1:8095/health
|
||||
```
|
||||
|
||||
If the binary or models are missing, health will report `status: degraded` and the worker will return a safe unavailable error for upscale requests.
|
||||
|
||||
## Health checks
|
||||
|
||||
`GET /health` returns general worker status for every engine.
|
||||
|
||||
When using `realesrgan-ncnn`, health also reports:
|
||||
|
||||
- whether the binary is configured
|
||||
- whether the binary exists
|
||||
- whether it is executable
|
||||
- whether the model directory exists
|
||||
- the available model list
|
||||
- the configured default model
|
||||
|
||||
## Engines
|
||||
|
||||
- `WORKER_ENGINE=pillow`: local development and tests. Metadata reports `real_ai_upscale: false`.
|
||||
- `WORKER_ENGINE=realesrgan-ncnn`: runs the ncnn-vulkan CLI through a safe `subprocess.run(..., shell=False)` path.
|
||||
- `WORKER_ENGINE=realesrgan`: alias to `realesrgan-ncnn`.
|
||||
|
||||
2x handling uses the 4x model output and a deterministic high-quality downsample. 4x requests return the native model scale.
|
||||
|
||||
## CPU and GPU
|
||||
|
||||
CPU mode is supported by default.
|
||||
|
||||
Optional GPU deployment later:
|
||||
|
||||
- install NVIDIA Container Toolkit
|
||||
- use a CUDA-compatible image and runtime
|
||||
- set `WORKER_DEVICE=cuda`
|
||||
- install Real-ESRGAN and its GPU-capable dependencies
|
||||
|
||||
Do not remove CPU fallback.
|
||||
|
||||
For `realesrgan-ncnn`, `WORKER_DEVICE=vulkan` is the intended production setting when the host has compatible drivers and GPU access.
|
||||
|
||||
## Performance notes
|
||||
|
||||
- 2x requests still pay much of the 4x cost because the worker currently runs the 4x model then downsamples.
|
||||
- Very large images may need tiling. `WORKER_REALESRGAN_TILE=0` keeps the runtime default behavior.
|
||||
- `WORKER_REALESRGAN_TTA=true` may improve quality but is slower.
|
||||
- CPU-only Real-ESRGAN is suitable for verification and low-volume use, but usually too slow for production.
|
||||
- Start production with conservative input limits and a dedicated `enhance` queue.
|
||||
|
||||
## Security notes
|
||||
|
||||
- Keep the worker on `127.0.0.1` or a private container network.
|
||||
- Do not expose the worker publicly.
|
||||
- Use a strong `WORKER_TOKEN`.
|
||||
- The worker only accepts `http` and `https` source URLs.
|
||||
- The result endpoint only serves sanitized filenames from the worker output directory.
|
||||
- Temporary inputs are deleted after processing, and stale temporary outputs are pruned by TTL.
|
||||
- Real-ESRGAN model names are selected from an internal allowlist only. Users cannot inject arbitrary CLI flags.
|
||||
- The worker uses `subprocess.run` without `shell=True`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- `401 Unauthorized`: check `WORKER_TOKEN` and Laravel `ENHANCE_WORKER_TOKEN`.
|
||||
- `Upscale engine is not available...`: the worker is in `realesrgan-ncnn` mode but the binary or models are missing, not executable, or the CLI run failed.
|
||||
- `Worker rejected the image.`: invalid mode, scale, output format, MIME type, dimensions, or download size.
|
||||
- `status: degraded` on `/health`: use `bash scripts/verify-realesrgan.sh` and either fix the runtime files or switch back to `WORKER_ENGINE=pillow`.
|
||||
- Jobs stuck in `queued`: if `ENHANCE_QUEUE=enhance`, make sure workers or Horizon consume the `enhance` queue.
|
||||
Reference in New Issue
Block a user