klevze/UploadShied
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add CONFIG_REFERENCE.md and CONTRIBUTING.md; update INTEGRATION and README

Browse Source
This commit is contained in:
Gregor Klevže
2026-02-12 10:18:32 +01:00
parent 1768f61da1
commit d6a19929bf
5 changed files with 404 additions and 528 deletions
Download Patch File Download Diff File Expand all files Collapse all files

72
CONFIG_REFERENCE.md Normal file
View File

@@ -0,0 +1,72 @@
# Configuration Reference
This file maps the top-level configuration keys used by `upload-logger.json` to their effect and defaults. Use absolute paths in production where possible.
## Top-level sections
- `modules` (object): enable or disable features by name. Keys used in code:
- `flood` (bool) — per-IP upload counting and flood alerts. Default: `true`.
- `filename` (bool) — run `FilenameDetector`. Default: `true`.
- `mime_sniff` (bool) — run `MimeDetector` & content sniffing. Default: `true`.
- `hashing` (bool) — compute hashes for forensic records. Default: `true`.
- `base64_detection` (bool) — detect JSON/base64 embedded payloads in raw bodies. Default: `true`.
- `raw_peek` (bool) — allow guarded reads of `php://input`. Disabled by default (`false`) because it may consume request bodies.
- `archive_inspect` (bool) — inspect archives moved to quarantine. Default: `true`.
- `quarantine` (bool) — enable quarantine moves. Default: `true`.
- `paths` (object): filesystem locations used by the script. Common keys:
- `log_file` (string) — path to JSON log file. Default: `logs/uploads.log` (relative to script). Recommended: absolute path under a per-site `.security` folder.
- `quarantine_dir` (string) — path to quarantine directory. Default: `quarantine`.
- `state_dir` (string) — path to store flood counters/state. Default: `state`.
- `allowlist_file` (string) — optional allowlist of URIs/content-types. Default: `allowlist.json`.
- `limits` (object): thresholds controlling scanning and resource limits.
- `max_size` (int) — bytes threshold for `big_upload` warning. Default: `52428800` (50 MB).
- `raw_body_min` (int) — min bytes for raw body events. Default: `512000` (500 KB).
- `sniff_max_bytes` (int) — bytes to read from a file head for content sniffing. Default: `8192` (8 KB).
- `sniff_max_filesize` (int) — only sniff files up to this size. Default: `2097152` (2 MB).
- `hash_max_filesize` (int) — max file size to compute hashes for. Default: `10485760` (10 MB).
- `archive_max_inspect_size` (int) — skip inspecting archives larger than this. Default: `52428800` (50 MB).
- `archive_max_entries` (int) — max entries to inspect inside an archive. Default: `200`.
- `ops` (object): operator-facing options.
- `quarantine_owner` / `quarantine_group` (string) — desired owner:group for quarantine. Default: `root`:`www-data`.
- `quarantine_dir_perms` (string) — octal string for dir perms (recommended `0700`). Default: `0700`.
- `block_suspicious` (bool) — when `true` the script returns 403 for suspicious uploads. Default: `false` (observe mode).
- `log_rotate` (object) — hints for log rotation (size, keep, enabled) used in examples.
- `trusted_proxy_ips` (array) — proxies allowed to signal buffered bodies or peek permission.
- `allowlists` (object): reduce false positives for known safe flows.
- `base64_uris` (array of strings) — substrings or PCRE (when wrapped with `#`) matching URIs to ignore for base64/raw detections.
- `ctypes` (array of strings) — content-types treated as trusted for encoded payloads (e.g., `image/svg+xml`).
## Detector-specific keys
- `detectors.content` (object)
- `sniff_max_bytes` (int) — override `limits.sniff_max_bytes` for content detector.
- `sniff_max_filesize` (int) — override `limits.sniff_max_filesize`.
- `allow_xml_eval` (bool) — relax `eval()` detection for XML/SVG content when true.
- `custom_patterns` (array) — array of PCRE patterns (as strings) applied to the file head. Invalid patterns are ignored.
## Example `upload-logger.json`
```json
{
"modules": { "flood": true, "filename": true, "mime_sniff": true, "hashing": true, "base64_detection": true, "raw_peek": false, "archive_inspect": true, "quarantine": true },
"paths": { "log_file": "/var/www/site/.security/logs/uploads.log", "quarantine_dir": "/var/www/site/quarantine", "state_dir": "/var/www/site/state", "allowlist_file": "/var/www/site/.security/allowlist.json" },
"limits": { "max_size": 52428800, "raw_body_min": 512000, "sniff_max_bytes": 8192, "sniff_max_filesize": 2097152 },
"ops": { "quarantine_owner": "root", "quarantine_group": "www-data", "quarantine_dir_perms": "0700", "block_suspicious": false },
"allowlists": { "base64_uris": ["/api/uploads/avatars"], "ctypes": ["image/svg+xml"] }
}
```
## Operational tips
- Keep `block_suspicious` disabled while tuning; use `allowlists.base64_uris` and `ctypes` to reduce false positives.
- Avoid enabling `raw_peek` unless your front end buffers request bodies or you accept the risk of consuming `php://input`.
- Use absolute paths in `paths.*` when deploying under systemd/Ansible to avoid cwd surprises.
- Ensure `quarantine_dir` is inaccessible from the web and set to owner `root` and mode `0700`; files inside should be `0600`.
If you want, I can generate a per-site `upload-logger.json` filled with your preferred absolute paths and ownership values.
--

75
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,75 @@
# Contributing
Thanks for contributing! This document explains how to run tests, linting, and the suggested workflow for changes.
## Quick dev setup
1. Install dependencies (developer machine):
```bash
git clone <repo-url> /path/to/upload-logger
cd /path/to/upload-logger
composer install --no-interaction --prefer-dist
```
2. Run unit tests and static analysis:
```bash
vendor/bin/phpunit --configuration phpunit.xml
vendor/bin/phpstan analyse -c phpstan.neon
```
3. Run PHP lint across the project (example):
```bash
find . -name '*.php' -not -path './vendor/*' -print0 | xargs -0 -n1 php -l
```
## Branching & PR workflow
- Create a feature branch from `main` (or `master` if your repo uses it):
```bash
git checkout -b feature/short-description
```
- Make small, focused commits with clear messages. Example:
```
Add CONFIG_REFERENCE.md mapping configuration options
```
- Push and open a pull request to `main`. Provide a short description of the change and mention testing steps.
## Tests and CI
- The repository uses GitHub Actions to run PHPUnit and PHPStan on supported PHP versions. Ensure tests pass locally before opening a PR.
- If you add new functionality, provide unit tests in `tests/` and update `phpunit.xml` if needed.
## Smoke tests
- A basic smoke harness exists under `tests/smoke/`. To run locally:
```bash
php -S 127.0.0.1:8000 -t tests/smoke/public -d auto_prepend_file=$(pwd)/upload-logger.php
# then POST files with curl or a test client
```
## Coding style
- Keep changes minimal and consistent with existing code. Avoid reformatting unrelated files.
- Follow PSR-12 style where practical for new PHP code.
## Adding docs
- For user-facing changes, update `README.md`, `docs/INSTALLATION.md` and `INTEGRATION.md` accordingly. Prefer short, copy-paste examples for operators.
## Security disclosures
- If you find a security vulnerability, do not open a public issue. Contact maintainers privately and include reproduction steps.
## Contact
- Open an issue or a PR on GitHub; maintainers will review and respond.
Thank you for helping improve Upload Logger.

595
INTEGRATION.md
View File

@@ -1,464 +1,89 @@
## Integration ## Integration & Tuning
Example `upload-logger.json` (commented for easy copy/paste into your environment): This document complements the installation steps in [docs/INSTALLATION.md](docs/INSTALLATION.md) by focusing on detector tuning, allowlists, and advanced integrations (log forwarding, Fail2Ban, etc.).
Example `upload-logger.json` (simplified):
```json ```json
// { {
// "modules": { "modules": {
// "flood": true, "flood": true,
// "filename": true, "filename": true,
// "mime_sniff": true, "mime_sniff": true,
// "hashing": true, "hashing": true,
// "base64_detection": true, "base64_detection": true,
// "raw_peek": false, "raw_peek": false,
// "archive_inspect": true, "archive_inspect": true,
// "quarantine": true "quarantine": true
// }, },
// "paths": { "paths": {
// "log_file": "logs/uploads.log", "log_file": "logs/uploads.log",
// "quarantine_dir": "quarantine", "quarantine_dir": "quarantine",
// "state_dir": "state", "state_dir": "state",
// "allowlist_file": "allowlist.json" "allowlist_file": "allowlist.json"
// }, },
// "limits": { "limits": {
// "max_size": 52428800, "max_size": 52428800,
// "raw_body_min": 512000, "raw_body_min": 512000,
// "sniff_max_bytes": 8192, "sniff_max_bytes": 8192,
// "sniff_max_filesize": 2097152, "sniff_max_filesize": 2097152
// "hash_max_filesize": 10485760, },
// "archive_max_inspect_size": 52428800, "ops": {
// "archive_max_entries": 200 "quarantine_owner": "root",
// }, "quarantine_group": "www-data",
// "ops": { "quarantine_dir_perms": "0700",
// "quarantine_owner": "root", "block_suspicious": false
// "quarantine_group": "www-data", },
// "quarantine_dir_perms": "0700", "allowlists": {
// "log_rotate": { "base64_uris": [
// "enabled": true, "/api/uploads/avatars",
// "size": 10485760, "#/hooks/(github|gitlab|stripe|slack)#"
// "keep": 7 ],
// } "ctypes": ["image/svg+xml","application/xml","text/xml"]
// }, }
// "allowlists": { }
// "base64_uris": [
// "/api/uploads/avatars",
// "/api/v1/avatars",
// "/user/avatar",
// "/media/upload",
// "/api/media",
// "/api/uploads",
// "/api/v1/uploads",
// "/attachments/upload",
// "/upload",
// "#^/internal/webhook#",
// "#/hooks/(github|gitlab|stripe|slack)#",
// "/services/avatars",
// "/api/profile/photo"
// ],
// "ctypes": ["image/svg+xml","application/xml","text/xml"]
// }
// }
``` ```
Notes: Notes:
- Remove the `//` comments if copying from examples. Use absolute paths in production where possible.
- Remove the leading `// ` when copying this into a real `upload-logger.json` file. ### Content detector tuning
- Adjust paths, owners, and limits to match your environment and PHP-FPM worker permissions.
ContentDetector tuning and false-positive guidance - The `ContentDetector` performs a fast head-scan to detect PHP open-tags and common webshell indicators (e.g., `passthru`, `system`, `exec`, `base64_decode`, `eval`, `assert`).
- Tuning options (in `upload-logger.json`):
- `limits.sniff_max_bytes` (default 8192) — how many bytes to scan from the file head.
- `limits.sniff_max_filesize` (default 2097152) — only scan files up to this size.
- `detectors.content.allow_xml_eval` — relax `eval()` detection for XML/SVG when appropriate.
- The repository includes a `ContentDetector` that performs a fast head-scan of uploaded files to detect PHP open-tags and common webshell indicators (for example `passthru()`, `system()`, `exec()`, `shell_exec()`, `proc_open()`, `popen()`, `base64_decode()`, `eval()`, `assert()`). It intentionally limits the scan to a small number of bytes to reduce CPU/IO overhead. False positives
- `eval(` appears in benign contexts (SVG/JS). To reduce false positives:
- Add trusted URIs to `allowlists.base64_uris`.
- Add trusted content-types to `allowlists.ctypes`.
- Tune scan size limits.
- Tuning options (place these in `upload-logger.json`): ### Allowlists
- `limits.sniff_max_bytes` (integer): number of bytes to read from the file head for scanning. Default: `8192`.
- `limits.sniff_max_filesize` (integer): only perform head-scan on files with size <= this value. Default: `2097152` (2 MB).
- `allowlists.ctypes` (array): content-types that should be considered trusted for base64/raw payloads (for example `image/svg+xml`, `application/xml`, `text/xml`) and may relax some detections.
- `allowlists.base64_uris` (array): URI patterns that should be ignored for large base64 payloads (webhooks, avatar uploads, etc.).
- False positives: `eval(` and other tokens commonly appear in client-side JS inside SVG files or in benign templating contexts. If you observe false positives: - `allowlists.base64_uris`: URI patterns (substring or PCRE when wrapped with `#`) that should bypass base64/raw detection.
- Add trusted URIs to `allowlists.base64_uris` for endpoints that legitimately accept encoded content. - `allowlists.ctypes`: content-types to treat as permitted for encoded payloads (e.g., `image/svg+xml`).
- Add trusted content-types to `allowlists.ctypes` to relax detection for XML/SVG uploads.
- Tune `limits.sniff_max_bytes` and `limits.sniff_max_filesize` to increase or decrease sensitivity.
- Suggested (example) detector tuning block (commented): ### Archive inspection & quarantine
```json - Archives uploaded are flagged and—if quarantine is enabled—moved to the quarantine directory for inspection.
// "detectors": { - Quarantine should be owner `root`, group `www-data`, mode `0700`. Files inside should be `0600`.
// "content": {
// "enabled": true,
// "sniff_max_bytes": 8192,
// "sniff_max_filesize": 2097152,
// "allow_xml_eval": false
// }
// }
```
Remove the leading `// ` when copying these example snippets into a real `upload-logger.json` file. ## Fail2Ban integration (example)
# 🔐 Per-Site PHP Upload Guard Integration Guide
This guide explains how to integrate a global PHP upload monitoring script Create a Fail2Ban filter that matches suspicious JSON log lines and captures the IP as `<HOST>`.
using `auto_prepend_file`, on a **per-site basis**, with isolated security
folders.
--- Filter (`/etc/fail2ban/filter.d/php-upload.conf`):
## 📁 1. Recommended Folder Structure
Each website should contain its own hidden security directory:
```
/var/www/sites/example-site/
├── public/
├── app/
├── uploads/
├── .security/
│ ├── upload_guard.php
│ └── logs/
│ └── uploads.log
````
Benefits:
- Per-site isolation
- Easier debugging
- Independent log files
- Reduced attack surface
---
## 🔧 2. Create the Security Directory
From the site root:
```bash
cd /var/www/sites/example-site
mkdir .security
mkdir .security/logs
````
Set secure permissions:
- Set secure permissions:
```bash
chown -R root:www-data .security
chmod 750 .security
chmod 750 .security/logs
```
Quarantine hardening (important):
- Ensure the quarantine directory is owner `root`, group `www-data`, and mode `0700` so quarantined files are not accessible to other system users. Example provisioning script `scripts/provision_dirs.sh` now enforces these permissions and tightens existing files to `0600`.
- If using Ansible, the playbook `scripts/ansible/upload-logger-provision.yml` includes a task that sets any existing files in the quarantine directory to `0600` and enforces owner/group.
- Verify SELinux/AppArmor contexts after provisioning; the script attempts to register fcontext entries and calls `restorecon` when available.
---
## 📄 3. Install the Upload Guard Script
Create the script file:
```bash
nano .security/upload_guard.php
```
Paste your hardened upload monitoring script.
Inside the script, configure logging:
```php
$logFile = __DIR__ . '/logs/uploads.log';
```
Lock the script:
```bash
chown root:root .security/upload_guard.php
chmod 644 .security/upload_guard.php
```
---
## ⚙️ 4. Enable auto_prepend_file (Per Site)
### Option A — PHP-FPM Pool (Recommended)
Edit the site’s PHP-FPM pool configuration:
```bash
nano /etc/php/8.x/fpm/pool.d/example-site.conf
```
Add:
```ini
php_admin_value[auto_prepend_file] = /var/www/sites/example-site/.security/upload_guard.php
```
Reload PHP-FPM:
```bash
systemctl reload php8.x-fpm
```
# 🔐 Per-Site PHP Upload Guard Integration Guide
This guide explains how to integrate a global PHP upload monitoring script
using `auto_prepend_file`, on a **per-site basis**, with isolated security
folders.
---
## 📁 1. Recommended Folder Structure
Each website should contain its own hidden security directory:
```text
/var/www/sites/example-site/
├── public/
├── app/
├── uploads/
├── .security/
│ ├── upload-logger.php
│ └── logs/
│ └── uploads.log
```
Benefits:
- Per-site isolation
- Easier debugging
- Independent log files
- Reduced attack surface
---
## 🔧 2. Create the Security Directory
From the site root:
```bash
cd /var/www/sites/example-site
mkdir .security
mkdir .security/logs
```
Set secure permissions:
```bash
chown -R root:www-data .security
chmod 750 .security
chmod 750 .security/logs
```
---
## 📄 3. Install the Upload Guard Script
Create the script file:
```bash
nano .security/upload-logger.php
```
Paste your hardened upload monitoring script.
Inside the script, configure logging:
```php
$logFile = __DIR__ . '/logs/uploads.log';
```
Lock the script:
```bash
chown root:root .security/upload-logger.php
chmod 644 .security/upload-logger.php
```
---
## ⚙️ 4. Enable auto_prepend_file (Per Site)
### Option A — PHP-FPM Pool (Recommended)
Edit the site’s PHP-FPM pool configuration:
```bash
nano /etc/php/8.x/fpm/pool.d/example-site.conf
```
Add:
```ini
php_admin_value[auto_prepend_file] = /var/www/sites/example-site/.security/upload-logger.php
```
Reload PHP-FPM (adjust service name to match your PHP version):
```bash
systemctl reload php8.x-fpm
```
---
### Option B — Apache Virtual Host
If using a shared PHP-FPM pool, configure in the vHost:
```apache
<Directory /var/www/sites/example-site>
php_admin_value auto_prepend_file /var/www/sites/example-site/.security/upload-logger.php
</Directory>
```
Reload Apache:
```bash
systemctl reload apache2
```
---
## 🚫 5. Block Web Access to `.security`
Prevent direct HTTP access to the security folder.
In the vHost:
```apache
<Directory /var/www/sites/example-site/.security>
Require all denied
</Directory>
```
Or in `.htaccess` (if allowed):
```apache
Require all denied
```
---
## ✅ 6. Verify Installation
Create a temporary file:
```php
<?php phpinfo();
```
Open it in browser and search for:
```text
auto_prepend_file
```
Expected output:
```text
/var/www/sites/example-site/.security/upload_guard.php
```
Remove the test file after verification.
---
## 🧪 7. Test Upload Logging
Create a simple upload test:
```php
<form method="post" enctype="multipart/form-data">
<input type="file" name="testfile">
<button>Upload</button>
</form>
```
Upload any file and check logs:
```bash
cat .security/logs/uploads.log
```
You should see a new entry.
---
## 🔒 8. Disable PHP Execution in Uploads
Always block PHP execution in upload directories.
Example (Apache):
```apache
<Directory /var/www/sites/example-site/uploads>
php_admin_flag engine off
AllowOverride None
</Directory>
```
Reload Apache after changes.
---
## 🛡️ 9. Enable Blocking Mode (Optional)
After monitoring for some time, enable blocking.
Edit:
```php
$BLOCK_SUSPICIOUS = true;
```
Then reload PHP-FPM.
---
## 📊 10. (Optional) Fail2Ban Integration (JSON logs)
Create a JSON-aware filter that matches `event: "suspicious"` and extracts the IP address.
```bash
nano /etc/fail2ban/filter.d/php-upload.conf
```
```ini ```ini
[Definition] [Definition]
# Match JSON lines where event == "suspicious" and capture the IPv4 address as <HOST>
failregex = ^.*"event"\s*:\s*"suspicious".*"ip"\s*:\s*"(?P<host>\d{1,3}(?:\.\d{1,3}){3})".*$ failregex = ^.*"event"\s*:\s*"suspicious".*"ip"\s*:\s*"(?P<host>\d{1,3}(?:\.\d{1,3}){3})".*$
ignoreregex = ignoreregex =
``` ```
Create a jail that points to the per-site logs (or a central aggregated log): Jail example (adjust `logpath`):
```ini
[php-upload]
enabled = true
filter = php-upload
logpath = /var/www/sites/*/.security/logs/uploads.log
maxretry = 3
findtime = 600
bantime = 86400
action = iptables-multiport[name=php-upload, port="http,https", protocol=tcp]
```
Restart Fail2Ban:
```bash
systemctl restart fail2ban
```
### Fail2Ban action: nftables example
If your host uses nftables, prefer the `nftables` action so bans use the system firewall:
```ini ```ini
[php-upload] [php-upload]
@@ -471,13 +96,11 @@ bantime = 86400
action = nftables[name=php-upload, port="http,https", protocol=tcp] action = nftables[name=php-upload, port="http,https", protocol=tcp]
``` ```
This uses Fail2Ban's `nftables` action (available on modern distributions). Adjust `port`/`protocol` to match your services. Test with `fail2ban-regex` using a representative JSON log line.
### Central log aggregation (Filebeat / rsyslog) ## Central log aggregation (Filebeat / rsyslog)
Forwarding per-site JSON logs to a central collector simplifies alerts and Fail2Ban at scale. Two lightweight options: Forward JSON logs to your aggregator to centralize alerts and analysis. Example Filebeat input:
- Filebeat prospector (send to Logstash/Elasticsearch):
```yaml ```yaml
filebeat.inputs: filebeat.inputs:
@@ -492,86 +115,16 @@ output.logstash:
hosts: ["logserver:5044"] hosts: ["logserver:5044"]
``` ```
- rsyslog `imfile` forwarding to remote syslog (central rsyslog/logstash): ## Logrotate & SELinux notes
Add to `/etc/rsyslog.d/10-upload-logger.conf`: Per-site `logrotate` snippets are included in `examples/logrotate.d/upload-logger`. Use `copytruncate` or reload PHP-FPM after rotation.
```text If SELinux is enabled, the provisioning script attempts to register fcontexts and run `restorecon`. Verify contexts manually as needed.
module(load="imfile" PollingInterval="10")
input(type="imfile" File="/var/www/sites/*/.security/logs/uploads.log" Tag="uploadlogger" Severity="info" Facility="local7")
*.* @@logserver:514
```
Both options keep JSON intact for downstream parsing and reduce per-host Fail2Ban complexity. ## Final notes
### Testing your Fail2Ban filter - Use observe mode (`ops.block_suspicious: false`) while tuning.
- After tuning, enable blocking in a controlled rollout (canary hosts first).
- Keep `upload-logger.php` and `.security` owned by `root` and ensure logs and quarantine are not web-accessible.
Create a temporary file containing a representative JSON log line emitted by `upload-logger.php` and run `fail2ban-regex` against your filter to validate detection. For installation steps and per-site configuration, see `docs/INSTALLATION.md`.
```bash
# create test file with a suspicious event
cat > /tmp/test_upload.log <<'JSON'
{"ts":"$(date -u +%Y-%m-%dT%H:%M:%SZ)","event":"suspicious","ip":"1.2.3.4","user":"guest","name":"evil.php.jpg","real_mime":"application/x-php","reasons":["bad_name","php_payload"]}
JSON
# test the filter (adjust path to filter if different)
fail2ban-regex /tmp/test_upload.log /etc/fail2ban/filter.d/php-upload.conf
```
`fail2ban-regex` will report how many matches were found and display sample matched groups (including the captured `<HOST>`). Use this to iterate on the `failregex` if it doesn't extract the IP as expected.
---
## 🏁 Final Architecture
```text
Client → Web Server → PHP (auto_prepend) → Application → Disk
↓
Log / Alert / Ban
```
This provides multi-layer upload monitoring and protection.
---
## 🗂️ Log rotation & SELinux/AppArmor notes
- Example `logrotate` snippet to rotate per-site logs weekly and keep 8 rotations:
```text
/var/www/sites/*/.security/logs/uploads.log {
weekly
rotate 8
compress
missingok
notifempty
create 0640 root adm
}
```
- If your host enforces SELinux or AppArmor, ensure the `.security` directory and log files have the correct context so PHP-FPM can read the script and write logs. For SELinux (RHEL/CentOS) you may need:
```bash
chcon -R -t httpd_sys_rw_content_t /var/www/sites/example-site/.security/logs
restorecon -R /var/www/sites/example-site/.security
```
Adjust commands to match your platform and policy. AppArmor profiles may require adding paths to the PHP-FPM profile.
## ⚠️ Security Notes
- Never use `777` permissions
- Keep `.security` owned by `root`
- Regularly review logs
- Update PHP and extensions
- Combine with OS-level auditing for best results
---
## 📌 Recommended Maintenance
Weekly:
```bash
grep ALERT .security/logs/uploads.log
```

3
README.md
View File

@@ -45,7 +45,8 @@ Content detector & tuning
- Behavior note: `eval()` and similar tokens commonly appear inside SVG/JS contexts. The detector uses the detected MIME to be more permissive for XML/SVG-like content, but you should test and tune for your application's upload patterns to avoid false positives (see `INTEGRATION.md`). - Behavior note: `eval()` and similar tokens commonly appear inside SVG/JS contexts. The detector uses the detected MIME to be more permissive for XML/SVG-like content, but you should test and tune for your application's upload patterns to avoid false positives (see `INTEGRATION.md`).
- If your application legitimately accepts encoded or templated payloads, add application-specific allowlist rules (URI or content-type) in `allowlist.json` or extend `upload-logger.json` with detector-specific tuning before enabling blocking mode. - If your application legitimately accepts encoded or templated payloads, add application-specific allowlist rules (URI or content-type) in `allowlist.json` or extend `upload-logger.json` with detector-specific tuning before enabling blocking mode.
Further integration Further integration
- Read the `INTEGRATION.md` for a commented example `upload-logger.json`, logrotate hints, and deployment caveats. - Read the `INTEGRATION.md` for detector tuning, allowlists, and examples for log forwarding and Fail2Ban.
- See `docs/INSTALLATION.md` for a step-by-step per-site install and `auto_prepend_file` examples.
- Provision the required directories (`quarantine`, `state`) and set ownership/SELinux via the included provisioning script: `scripts/provision_dirs.sh`. - Provision the required directories (`quarantine`, `state`) and set ownership/SELinux via the included provisioning script: `scripts/provision_dirs.sh`.
- Example automation: `scripts/ansible/upload-logger-provision.yml` and `scripts/systemd/upload-logger-provision.service` are included as examples to run provisioning at deploy-time or boot. - Example automation: `scripts/ansible/upload-logger-provision.yml` and `scripts/systemd/upload-logger-provision.service` are included as examples to run provisioning at deploy-time or boot.

175
docs/INSTALLATION.md Normal file
View File

@@ -0,0 +1,175 @@
# Installation & Production Deployment Guide
This guide shows a minimal, secure installation and rollout path for `upload-logger.php`.
Follow these steps in a staging environment first; do not enable blocking until detectors are tuned.
**Prerequisites**
- A Linux host running PHP-FPM (PHP 8.0+ recommended).
- `composer` available for dev tasks.
- SSH/privileged access to configure the site pool and run provisioning scripts.
**Quick overview**
1. Place `upload-logger.php` in a secure per-site folder (recommended `.security`).
2. Create `logs/`, `quarantine/`, `state/` and set strict ownership and permissions.
3. Configure `upload-logger.json` for your environment; keep `ops.block_suspicious` off for initial tuning.
4. Enable `auto_prepend_file` in the site PHP-FPM pool to run the logger before application code.
5. Verify logging, tune detectors, deploy log rotation, and enable alerting.
**1. Clone & dependencies (developer workstation)**
- Clone the repository and install dev deps (for tests/static analysis):
```bash
git clone <repo-url> /srv/upload-logger
cd /srv/upload-logger
composer install --no-interaction --prefer-dist
```
Run tests locally:
```bash
vendor/bin/phpunit --configuration phpunit.xml
vendor/bin/phpstan analyse -c phpstan.neon
```
**2. Recommended file layout (per-site)**
Use a hidden per-site security folder so `upload-logger.php` is not web-accessible.
Example layout:
```
/var/www/sites/example-site/
├── public/
├── app/
├── uploads/
├── .security/
│ ├── upload-logger.php
│ └── logs/
│ └── uploads.log
├── quarantine/
└── state/
```
**3. Copy files & configure**
- Place `upload-logger.php` into `.security/upload-logger.php`.
- Copy `upload-logger.json` from the repository to the same directory and edit paths to absolute values, e.g.:
- `paths.log_file` → `/var/www/sites/example-site/.security/logs/uploads.log`
- `paths.quarantine_dir` → `/var/www/sites/example-site/quarantine`
- `paths.state_dir` → `/var/www/sites/example-site/state`
- Ensure `ops.block_suspicious` is `false` initially (observe mode).
**4. Create directories & set permissions (run as root)**
Adjust user/group to your site environment (`www-data` used in examples):
```bash
# example: run on target host as root
mkdir -p /var/www/sites/example-site/.security/logs
mkdir -p /var/www/sites/example-site/quarantine
mkdir -p /var/www/sites/example-site/state
chown -R root:www-data /var/www/sites/example-site/.security
chmod 750 /var/www/sites/example-site/.security
chmod 750 /var/www/sites/example-site/.security/logs
# quarantine must be restrictive
chown -R root:www-data /var/www/sites/example-site/quarantine
chmod 0700 /var/www/sites/example-site/quarantine
# state directory writable by PHP-FPM if required (group-write)
chown -R root:www-data /var/www/sites/example-site/state
chmod 0750 /var/www/sites/example-site/state
# ensure log file exists with safe perms
touch /var/www/sites/example-site/.security/logs/uploads.log
chown root:www-data /var/www/sites/example-site/.security/logs/uploads.log
chmod 0640 /var/www/sites/example-site/.security/logs/uploads.log
```
Alternatively use the included provisioning scripts on the host:
- `scripts/provision_dirs.sh` — run as root; idempotent and attempts to set SELinux fcontext.
- `scripts/ansible/upload-logger-provision.yml` — Ansible playbook for bulk provisioning.
**5. PHP‑FPM configuration (per-site pool)**
Edit the site's FPM pool (example: `/etc/php/8.1/fpm/pool.d/example-site.conf`) and add:
```ini
; Ensure upload-logger runs before application code
php_admin_value[auto_prepend_file] = /var/www/sites/example-site/.security/upload-logger.php
```
Then reload PHP-FPM:
```bash
sudo systemctl reload php8.1-fpm
```
Notes:
- Use the absolute path to `upload-logger.php`.
- If your host uses a shared PHP-FPM pool, consider enabling per-vhost `auto_prepend_file` via nginx/apache or create a dedicated pool for the site.
**6. Log rotation**
Create `/etc/logrotate.d/upload-logger` with content adapted to your paths:
```
/var/www/sites/example-site/.security/logs/uploads.log {
rotate 7
size 10M
compress
missingok
notifempty
copytruncate
create 0640 root www-data
sharedscripts
postrotate
if systemctl is-active --quiet php8.1-fpm; then
systemctl reload php8.1-fpm >/dev/null 2>&1 || true
fi
endscript
}
```
`copytruncate` is safe and avoids needing to stop PHP; alternatively use `postrotate` to reload FPM.
**7. Verify installation (smoke tests)**
- Upload a benign file via your app or via a simple test form and confirm a JSON `upload` line appears in the log.
- Upload a file containing `<?php` to confirm detectors log `suspicious_upload` entries.
- Check `logs/uploads.log` for events.
Example quick test (on host):
```bash
# from site root
php -S 127.0.0.1:8000 -t . -d auto_prepend_file=/var/www/sites/example-site/.security/upload-logger.php
# then curl a file POST to your test endpoint
curl -F "file=@/path/to/sample.txt" http://127.0.0.1:8000/upload_test.php
```
**8. Tuning & staging**
- Keep `ops.block_suspicious` set to `false` while monitoring logs and tuning:
- `allowlists.base64_uris` — add URIs for trusted base64 endpoints.
- `allowlists.ctypes` — add trusted content-types such as `image/svg+xml` if needed.
- `limits.sniff_max_bytes` / `limits.sniff_max_filesize` — tune scanning cost vs coverage.
- Run the system under representative traffic and check for false positives.
**9. Gradual enable blocking**
- After tuning, enable blocking in a controlled manner:
1. Set `ops.block_suspicious` → `true` in `upload-logger.json` on a small subset of sites or a canary host.
2. Monitor errors, rollback quickly if issues appear.
3. Gradually roll out to remaining hosts.
**10. Monitoring & alerting**
- Forward `logs/uploads.log` to your SIEM or log aggregator (Filebeat/Fluentd).
- Create alerts for `event == suspicious_upload` or `event == raw_body` and for rapid flood counts.
- Monitor disk usage for `logs/` and ensure `logrotate` is active.
**11. Security & operational checklist**
- Ensure `quarantine/` and `.security/logs` are not accessible from the web server.
- Verify SELinux/AppArmor contexts after running provisioning.
- Ensure owner/group are set to root and web group (e.g., `root:www-data`) and modes match the guide.
- Keep `upload-logger.php` readable by root (644) and the logs readable only by the intended group (640).
**Rollback**
- Disable `php_admin_value[auto_prepend_file]` in the pool and reload PHP-FPM.
- Remove or rotate the `upload-logger` files if needed.
**Further reading & files**
- Integration notes: [INTEGRATION.md](INTEGRATION.md)
- Provisioning script: `scripts/provision_dirs.sh`
- Ansible playbook: `scripts/ansible/upload-logger-provision.yml`
- Example configuration: `upload-logger.json`
---
If you want, I can: (a) generate a site-specific copy of these snippets for your exact paths/PHP version, (b) open a PR with the updated documentation, or (c) produce a one-command installer playbook that runs the provisioning and copies files to a remote host. Tell me which option you prefer.