Rust (axum + sqlx) API and worker sharing a Postgres-backed job queue (SKIP LOCKED, heartbeat, reaper, typed statuses), S3 storage with derived keys and a fully private bucket, OIDC photographer login with per-request allowlist checks, client share links with argon2 passwords and lockout, cookie-based image authorization with sliding expiry, hand-rolled spec-compliant streaming ZIP downloads with exact Content-Length, React + Vite gallery frontend, single Docker image, Helm chart for external S3 + Postgres, and Gitea CI. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude
@@ -0,0 +1,163 @@
|
||||
# Photos
|
||||
|
||||
Self-hosted client gallery for photographers. Upload RAWs/JPGs into albums,
|
||||
share them with clients via private (optionally password-protected) links,
|
||||
collect ratings and tags, and let clients download originals.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
browser ── ingress ──► │ api (Rust) │ ──► Postgres (data + job queue)
|
||||
│ axum │ ──► S3 (originals, previews, thumbs)
|
||||
└─────────────┘
|
||||
┌─────────────┐
|
||||
│ worker(s) │ ◄── polls jobs table (SKIP LOCKED)
|
||||
│ exiftool + │ ──► renders preview (2048px) + thumb
|
||||
│ image crate │ (512px) JPEGs into S3
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
- **Backend**: Rust (axum, sqlx). Two binaries from one crate: `server` (API +
|
||||
serves the built frontend) and `worker` (job processor).
|
||||
- **Job queue**: plain Postgres table claimed with `FOR UPDATE SKIP LOCKED`,
|
||||
with retries and exponential backoff. Job kinds: `process_photo`,
|
||||
`delete_s3_prefix`.
|
||||
- **RAW handling**: the worker extracts the camera's embedded JPEG preview via
|
||||
`exiftool` (fast, matches in-camera rendering), applies EXIF orientation,
|
||||
and resizes. Originals are always stored and downloadable untouched.
|
||||
- **Storage layout**: `photos/<photo_id>/original/<filename>`,
|
||||
`photos/<photo_id>/preview.jpg`, `photos/<photo_id>/thumb.jpg`. The bucket
|
||||
stays fully private; all image traffic is streamed through the API with
|
||||
auth checks (no bucket CORS or public access needed).
|
||||
- **Auth**: photographer signs in via any OIDC provider (authorization-code
|
||||
flow + userinfo); only emails in `ALLOWED_EMAILS` may sign in, and sessions
|
||||
are re-checked against the allowlist on every request, so removing an email
|
||||
revokes access immediately. Clients use unguessable share tokens, optionally
|
||||
gated by an argon2-hashed password (10 wrong guesses lock the link for
|
||||
15 minutes).
|
||||
- **Frontend**: React + Vite SPA — justified gallery, lightbox with rating
|
||||
stars and tag chips, drag-and-drop multi-file upload with progress.
|
||||
|
||||
## Local development
|
||||
|
||||
Requirements: Rust (rustup — the pinned toolchain in `rust-toolchain.toml`
|
||||
installs automatically), Node 20+, `exiftool`, Docker.
|
||||
|
||||
```sh
|
||||
docker compose up -d # Postgres + MinIO (bucket auto-created)
|
||||
cp .env.example .env # then edit OIDC_* and ALLOWED_EMAILS
|
||||
set -a; source .env; set +a
|
||||
|
||||
cargo run --bin server # API on :8080 (runs migrations on start)
|
||||
cargo run --bin worker # job worker (separate terminal, same env)
|
||||
|
||||
cd frontend && npm install && npm run dev # UI on :5173, proxies /api
|
||||
```
|
||||
|
||||
Register the OIDC client with redirect URI `<PUBLIC_URL>/api/auth/callback`
|
||||
(locally: `http://localhost:5173/api/auth/callback`). Any standard OIDC
|
||||
provider works (Authentik, Keycloak, Zitadel, Dex, ...); the app uses
|
||||
discovery, so only the issuer URL is configured.
|
||||
|
||||
## Configuration
|
||||
|
||||
All configuration is via environment variables:
|
||||
|
||||
| Variable | Required | Description |
|
||||
| --- | --- | --- |
|
||||
| `DATABASE_URL` | yes | Postgres connection string |
|
||||
| `PUBLIC_URL` | yes | External base URL, e.g. `https://photos.example.com` |
|
||||
| `SESSION_SECRET` | yes | ≥32 chars; signs session/share cookies |
|
||||
| `S3_BUCKET` | yes | Bucket name |
|
||||
| `S3_ACCESS_KEY` / `S3_SECRET_KEY` | yes | S3 credentials |
|
||||
| `S3_ENDPOINT` | no | Set for MinIO/Ceph/etc.; empty = AWS S3 |
|
||||
| `S3_REGION` | no | Default `us-east-1` |
|
||||
| `S3_FORCE_PATH_STYLE` | no | `true` for MinIO |
|
||||
| `OIDC_ISSUER` | yes | Issuer URL (discovery is fetched from it) |
|
||||
| `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` | yes | OIDC client credentials |
|
||||
| `ALLOWED_EMAILS` | yes | Comma-separated photographer emails |
|
||||
| `BIND_ADDR` | no | Default `0.0.0.0:8080` |
|
||||
| `STATIC_DIR` | no | Built frontend dir (default `frontend/dist`) |
|
||||
| `WORKER_CONCURRENCY` | no | Parallel jobs per worker pod (default 2) |
|
||||
| `RUST_LOG` | no | e.g. `info,sqlx=warn` |
|
||||
| `DEV_AUTOLOGIN_EMAIL` | no | **Dev only**: skip OIDC and sign in as this email (must also be in `ALLOWED_EMAILS`). The server refuses to start with this set when `PUBLIC_URL` is https. |
|
||||
|
||||
## Deploying to Kubernetes
|
||||
|
||||
Build and push the image (single image contains `server`, `worker`, and the
|
||||
built frontend):
|
||||
|
||||
```sh
|
||||
docker build -t ghcr.io/YOU/photos:0.1.0 .
|
||||
docker push ghcr.io/YOU/photos:0.1.0
|
||||
```
|
||||
|
||||
Install the chart, pointing it at your existing Postgres and S3:
|
||||
|
||||
```sh
|
||||
helm install photos deploy/chart \
|
||||
--set image.repository=ghcr.io/YOU/photos \
|
||||
--set image.tag=0.1.0 \
|
||||
--set publicUrl=https://photos.example.com \
|
||||
--set ingress.host=photos.example.com \
|
||||
--set config.oidcIssuer=https://auth.example.com \
|
||||
--set config.allowedEmails=you@example.com \
|
||||
--set config.s3.bucket=photos \
|
||||
--set config.s3.endpoint=https://s3.example.com \
|
||||
--set config.s3.forcePathStyle=true \
|
||||
--set secrets.databaseUrl=postgres://... \
|
||||
--set secrets.s3AccessKey=... \
|
||||
--set secrets.s3SecretKey=... \
|
||||
--set secrets.oidcClientId=photos \
|
||||
--set secrets.oidcClientSecret=... \
|
||||
--set secrets.sessionSecret=$(openssl rand -hex 32)
|
||||
```
|
||||
|
||||
For production prefer a values file, or create the Secret yourself and set
|
||||
`secrets.existingSecret` (keys: `DATABASE_URL`, `S3_ACCESS_KEY`,
|
||||
`S3_SECRET_KEY`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, `SESSION_SECRET`).
|
||||
|
||||
Notes:
|
||||
|
||||
- The default ingress annotations disable nginx's request body limit and
|
||||
request buffering so multi-GB RAW uploads stream through. Adapt for other
|
||||
ingress controllers.
|
||||
- Migrations run automatically on startup of either binary (they take a
|
||||
Postgres advisory lock, so concurrent starts are safe).
|
||||
- Scale `worker.replicas` (or `worker.concurrency`) if imports queue up;
|
||||
the queue is safe for any number of workers.
|
||||
|
||||
## How sharing works
|
||||
|
||||
- Each album can have any number of share links (`/s/<24-char-token>`), each
|
||||
with its own label (e.g. the client's name), optional password, optional
|
||||
expiry, and a per-link download toggle.
|
||||
- Ratings (1–5 stars) and free-form tags are stored **per link**, so create
|
||||
one link per client to keep feedback separate. The album view shows all
|
||||
feedback grouped by link label.
|
||||
- Clients (and you) can multi-select photos and download them — or the whole
|
||||
album — as a ZIP. Archives are streamed (each file spools briefly through a
|
||||
temp file for its checksum, then pipelines while the next one prefetches),
|
||||
stored uncompressed with real capture-date timestamps, and are fully
|
||||
spec-compliant — they extract with strict streaming readers (Java
|
||||
`ZipInputStream`, piped `bsdtar`) as well as Finder/Explorer/unzip/7-Zip.
|
||||
Responses carry an exact `Content-Length`, so browsers show progress and
|
||||
flag interrupted downloads as failed. Concurrent zip streams are capped
|
||||
at 4.
|
||||
- 10 wrong passwords lock a link for 15 minutes (fresh attempts after the
|
||||
window). A locked link shows in the album's share list with an Unlock
|
||||
button.
|
||||
- Deleting a link removes its ratings/tags; deleting photos or albums cleans
|
||||
up S3 objects via background jobs.
|
||||
|
||||
## Known limitations / deliberate v1 cuts
|
||||
|
||||
- Full RAW develop fallback for files whose embedded preview is tiny
|
||||
(exceedingly rare on modern cameras; `darktable-cli` in the worker image
|
||||
would cover it).
|
||||
- Multiple photographer accounts with separate libraries (any allowed email
|
||||
sees everything).
|
||||
- No S3 orphan sweeper: a crash in the narrow window between an upload's S3
|
||||
put and its DB commit can leave an unreferenced original in the bucket
|
||||
(never data loss — just unclaimed storage).
|
||||
Reference in New Issue
Block a user