Initial commit
ci / docker (backend, backend/Dockerfile, backend) (push) Failing after 13s
ci / docker (frontend, frontend/Dockerfile, frontend) (push) Failing after 12s

This commit is contained in:
2025-12-17 13:37:31 +01:00
commit 3c9a9fc060
399 changed files with 80777 additions and 0 deletions
+71
View File
@@ -0,0 +1,71 @@
Papercrate REST API
===================
Unless noted otherwise, endpoints below require a valid `Authorization: Bearer <token>` header.
Authentication
--------------
- POST /api/auth/login - Exchange username/password for an access token and refresh cookie (public). Returns the active tenant as `{ tenant: { id, name } }`. When multiple tenants are available, the response contains an `access_token` (tenant-selector token) and tenant list instead.
- POST /api/auth/refresh - Rotate the refresh cookie and return a new access token (public, requires refresh cookie). Response also includes the current tenant `{ tenant: { id, name } }`.
- POST /api/auth/logout - Revoke the caller's refresh tokens and clear the cookie.
- GET /api/auth/me - Return the authenticated principal payload.
Health
------
- GET /api/health - Lightweight liveness probe (no authentication required).
Documents
---------
- GET /api/documents - List or search documents. Optional filters: `folder_id` (defaults to root when omitted), `include_descendants` (defaults to true unless explicitly set to `false` without filters), `status` (`active`, `deleted`, or `all`; defaults to `active`), `query` (Quickwit full-text), `tags` (comma-separated tag UUIDs), and `correspondents` (comma-separated correspondent UUIDs). Each entry includes tags, correspondent assignments, and current version info.
- GET /api/documents/check?checksum=<sha256> - Lightweight checksum preflight. Returns `exists=false` when no document with the supplied SHA-256 checksum is present; otherwise returns `exists=true` plus the current document metadata.
- POST /api/documents - Upload a document via multipart form-data. Required field: `file`. Optional fields: `title`, `folder_id`, JSON `metadata`, JSON array `tag_ids`, JSON array `correspondents` (each with `correspondent_id`), and `issued_at` (RFC3339). When `title` is supplied, the stored filename becomes `<title><original_extension>`. Include `skip_existing=true` to receive `204 No Content` instead of reusing a matching document.
- POST /api/documents/bulk/move - Move multiple documents to a target folder.
- POST /api/documents/bulk/tags - Add or remove tags across multiple documents.
- POST /api/documents/bulk/correspondents - Bulk correspondent actions. Use `action=add` (default) to attach correspondents or `action=remove` to detach the provided correspondents.
- POST /api/documents/bulk/reanalyze - Queue re-analysis jobs for selected documents.
- GET /api/documents/:id - Retrieve metadata and current version details for a document.
- PATCH /api/documents/:id - Update document metadata (currently title).
- POST /api/documents/:id/trash - Move a document to trash (soft delete, reversible).
- DELETE /api/documents/:id - Permanently erase a trashed document. Returns 202 Accepted, queues a purge job, and fails with 409 if the document is still active.
- PATCH /api/documents/:id/folder - Move a document to another folder.
- POST /api/documents/:id/restore - Restore a soft-deleted document. Optional body `{ "folder_id": <uuid> }` to send it to a specific folder; defaults to the original folder or root if missing.
- GET /api/documents/:id/versions - List version history for a document.
- GET /api/documents/:id/versions/:version_id - Fetch metadata and assets for a specific version.
- POST /api/documents/:id/tags - Assign one or more tags to a document.
- DELETE /api/documents/:id/tags/:tag_id - Remove a single tag from a document.
- POST /api/documents/:id/correspondents - Assign correspondents (`assignments[]` with `correspondent_id`; optional `replace=true` overwrites existing assignments).
- DELETE /api/documents/:id/correspondents/:correspondent_id - Remove a correspondent assignment.
Document Assets
---------------
- GET /api/documents/:id/assets - List generated assets for the current version.
- POST /api/documents/:id/assets - Request (re)generation of document assets; accepts optional `force` query flag.
- GET /api/assets/:asset_id - Fetch asset metadata plus a presigned URL for a range of objects (query params: `start` and `limit`, defaulting to the first object).
Downloads
---------
- GET /api/download/:token - Follow a one-time download token; redirects to a pre-signed URL (public token required).
Folders
-------
- POST /api/folders - Create a folder (optionally under a parent).
- POST /api/folders/path - Ensure a nested folder path exists, creating missing segments.
- GET /api/folders/:id - Fetch folder metadata.
- GET /api/folders/:id/contents - List subfolders and documents inside a folder; use `root` for the workspace root.
- DELETE /api/folders/:id - Soft-delete a folder.
- PATCH /api/folders/:id - Update a folder's parent (`parent_id`) and/or rename it (`name`).
- GET /api/docs/openapi.json - Generated OpenAPI specification (JSON).
Tags
----
- GET /api/tags - List all tags with usage counts.
- POST /api/tags - Create a new tag.
- PATCH /api/tags/:id - Update a tag's label or color.
- DELETE /api/tags/:id - Remove a tag; fails with 400 if still assigned to any document.
Correspondents
--------------
- GET /api/correspondents - List correspondents with usage totals.
- POST /api/correspondents - Create a correspondent (name + optional metadata JSON).
- PATCH /api/correspondents/:id - Update name and/or metadata.
- DELETE /api/correspondents/:id - Remove a correspondent; fails with 400 if referenced by any document.
+17
View File
@@ -0,0 +1,17 @@
# API response helpers
The backend now exposes `crate::http::responders`, which wraps common success and
error patterns for routes:
- `ok_json`, `created_json`, `accepted_json` return `JsonResponse<T>` with the
respective status codes.
- `no_content`/`empty` provide shared empty responses.
- `JsonResponse<T>` implements `IntoResponse`, so any handler can return
`AppResult<JsonResponse<T>>` without pairing tuples manually.
- `IntoAppResult`, `RowsAffectedExt`, and friends convert Diesel results into
`AppResult<T>` with consistent `AppError` handling.
When adding new routes, import from `crate::http::responders` instead of
constructing `(StatusCode, Json<T>)` tuples directly. The folders, documents,
auth, capability-set, correspondent, tag, and profile routers now all share
these helpers; WebDAV keeps its bespoke streaming responses for now.
+100
View File
@@ -0,0 +1,100 @@
# Capability Sets
Capability sets are the tenant-scoped bundles of REST and WebDAV permissions. Every user membership and API token now references one of these sets, and the capability guard middleware enforces the scopes on every route.
## Enumerated Capabilities
All capabilities live in the `ApiCapability` enum. The current list is:
- `documents:read`
- `documents:edit`
- `documents:write`
- `documents:upload`
- `folders:read`
- `folders:edit`
- `folders:write`
- `tags:read`
- `tags:edit`
- `tags:write`
- `correspondents:read`
- `correspondents:edit`
- `correspondents:write`
- `profile:read`
- `profile:write`
- `webdav:read`
- `webdav:write`
- `capability_sets:read`
- `capability_sets:write`
## Default Sets
Provisioning (and the test harness) seed four system capability sets per tenant:
- `owner` — contains the full set above. Tenant owners, admin users, and freshly minted API tokens effectively get unrestricted access.
- `user` — the default interactive role: full document/tag/correspondent/profile access, but no capability-set or WebDAV write privileges.
- `readonly` — interactive but read-only: document/folder/tag/correspondent reads plus WebDAV downloads, but no modifying routes.
- `webdav` — contains only `webdav:read`. WebDAV backup scripts can bind to this set for read-only access.
System sets are flagged with `is_system = true` and cannot be modified or deleted via the API.
## REST API
The capability-set endpoints live at `/api/capability-sets` and require the new admin capabilities:
| Method & Path | Capability | Description |
|----------------------------------------|-------------------------|-----------------------------------------|
| `GET /api/capability-sets` | `capability_sets:read` | List all sets for the tenant |
| `POST /api/capability-sets` | `capability_sets:write` | Create a new set |
| `GET /api/capability-sets/{id}` | `capability_sets:read` | Fetch details of a specific set |
| `PATCH /api/capability-sets/{id}` | `capability_sets:write` | Replace capabilities / rename the set |
| `DELETE /api/capability-sets/{id}` | `capability_sets:write` | Remove a custom set (must be unused) |
### Examples
Create a read-only set:
```bash
curl -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
https://app.papercrate.org/api/capability-sets \
-d '{
"slug": "api_readonly",
"capabilities": [
"documents:read",
"folders:read",
"tags:read"
]
}'
```
Update an existing set:
```bash
curl -X PATCH \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
https://app.papercrate.org/api/capability-sets/$SET_ID \
-d '{
"capabilities": ["documents:read", "documents:edit"]
}'
```
Delete (fails if still referenced by memberships or tokens):
```bash
curl -X DELETE \
-H "Authorization: Bearer $TOKEN" \
https://app.papercrate.org/api/capability-sets/$SET_ID
```
## Assigning Sets
- **User memberships**: change the `capability_set_id` column (via future admin APIs or direct SQL) to reassign a user. The authentication pipeline will enforce the new capabilities automatically.
- **API tokens**: `POST /api/profile/api-tokens` requires a `capability_set_id`. Tokens are bound to the selected set; raw capability arrays are no longer accepted.
## Guard Coverage
The `RequireCapabilitiesLayer` middleware wraps all protected routers (documents, folders, tags, correspondents, profile, capability sets, assets). Requests missing the necessary capability now terminate with a 403 containing `missing_capability` details.
Integration tests in `backend/tests/capability_guards_flow.rs` ensure read-only users cannot upload or manage capability sets, and WebDAV tokens without `webdav:read` are rejected (`backend/tests/api_tokens_flow.rs`).
+15
View File
@@ -0,0 +1,15 @@
# Desktop Workspace Interaction Spec
The desktop workspace should apply the following selection and drag behaviours:
- **Click on a non-selected card**: clear any existing selection, then select the clicked card only.
- **Click on a selected card**: keep the selection and open the detail panel for that card (no selection change).
- **Drag on a non-selected card**: clear the selection, select the dragged card, then drag that single card.
- **Drag on a selected card**: drag the entire current selection without altering which cards are selected.
- **Cmd/Ctrl + click on a non-selected card**: add that card to the existing selection.
- **Cmd/Ctrl + click on a selected card**: expand the selection by adding the stack of cards beneath the clicked card.
- **Cmd/Ctrl + drag on a non-selected card**: replace the current selection with the entire stack beneath the pointer, then drag that stack.
- **Cmd/Ctrl + drag on a selected card**: replace the current selection with the stack beneath the pointer, then drag that stack.
- **Touch long-press**: behaves like a stack-select gesture, expanding the selection to the stack under the pressed card without requiring modifier keys.
These rules ensure the selection model remains predictable while supporting stack-aware gestures unique to the desktop workspace.
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

+92
View File
@@ -0,0 +1,92 @@
# Document Data Model
This note describes the core persistence model for documents: the metadata held in
`documents`, how versions are tracked, and the way auxiliary assets are stored.
## documents
Each row represents the logical document a user interacts with in the UI. Key
fields:
- `id (uuid)` Stable identifier used in API paths.
- `tenant_id (uuid)` Multi-tenancy boundary; all joins filter by this.
- `title (varchar)` Display name editable via PATCH.
- `filename / original_name (varchar)` Current storage filename vs. the name
captured during upload.
- `folder_id (uuid, nullable)` Parent folder, `NULL` means root.
- `metadata (jsonb)` Arbitrary structured metadata (source import details,
custom fields, etc.).
- `issued_at (timestamptz, nullable)` User-provided timestamp for when the
document was issued (invoice date, etc.).
- `current_version_id (uuid)` FK pointing at the active `document_versions`
row; updated whenever a new version is promoted.
- `deleted_at (timestamptz, nullable)` Soft-delete marker; non-NULL rows are
treated as living in the trash.
- `created_at / updated_at (timestamptz)` Audit stamps; `updated_at` reflects
metadata or version changes.
Other indexes enforce per-tenant uniqueness for `(folder, filename)` and support
common queries (folder listing, trash filtering).
## document_versions
Every binary revision lives here. Fields of interest:
- `document_id (uuid)` Back-reference to the logical document.
- `version_number (int)` Monotonic per document (1, 2, …); enforced via
`UNIQUE(document_id, version_number)`.
- `s3_key (varchar)` Object storage path for the binary (used for download).
- `size_bytes`, `checksum` Stored metadata about the binary; checksum is a
hex-encoded SHA-256 hash used for dedupe/conflicts.
- `metadata (jsonb)` Small metadata blob specific to the version (extracted
text summary, processing hints, etc.).
- `tenant_id (uuid)` Mirrors the owning documents tenant.
The row referenced by `documents.current_version_id` is treated as the latest
revision. Older versions remain queryable for download or audit.
## Assets
A document version can have zero or more derived artifacts (thumbnails, OCR
output, previews). These are modelled via:
- `document_assets`
- `document_version_id` FK to the owning version.
- `asset_type (text)` Logical type identifier (e.g. `thumbnail`, `ocr_text`).
- `mime_type (text)` Media type for consumers.
- `metadata (jsonb)` Asset-specific metadata (dimensions, page count, etc.).
- `cardinality (int, nullable)` Optional hint for multi-object assets.
- `tenant_id (uuid)` Tenant scoping.
- Uniqueness on `(document_version_id, asset_type)` ensures one logical asset
per type; multi-object cases are stored in `document_asset_objects`.
- `document_asset_objects`
- `asset_id` FK to `document_assets`.
- `ordinal (int)` 1-based position for multi-part assets.
- `s3_key (text)` Object storage key for the binary blob.
- `metadata (jsonb)` Per-object metadata if needed (e.g. page number).
Simple assets (single thumbnail) live solely in `document_assets`. Complex ones
(e.g. per-page previews) use `document_asset_objects` to point at multiple S3
objects under a single logical asset.
## Related tables
- `document_tags` and `document_correspondents` provide many-to-many
relationships for categorisation.
- `jobs` records background work (OCR, thumbnails, indexing) keyed by tenant.
- `api_tokens`, `user_sessions`, and `user_passkeys` live alongside but do
not alter the document schema directly.
## Lifecycle summary
1. Upload creates a `documents` row and an initial `document_versions` entry.
2. Workers generate derived assets, inserting rows into `document_assets`
(and possibly `document_asset_objects`).
3. When a new version is promoted, a fresh `document_versions` row is written
and `documents.current_version_id` is updated atomically.
4. Soft-deleting the document sets `deleted_at`; restore clears it and the
document reappears in listings.
This schema allows arbitrary metadata expansion while maintaining a clear
separation between logical documents, their version history, and derived assets.
Binary file not shown.

After

Width:  |  Height:  |  Size: 683 KiB

+40
View File
@@ -0,0 +1,40 @@
# Job Catalogue
Papercrate stores asynchronous work in the shared `jobs` table. Each job carries a
`tenant_id`, a small JSON payload, and one of the statuses defined in
`backend/src/jobs.rs` (`queued`, `processing`, `succeeded`, `failed`). Workers
continuously reserve jobs by type and execute the appropriate handler. This
document lists every job type that is currently recognized by the backend and
briefly describes what it does.
| Job type | Payload shape | When it is enqueued | Work performed |
| --- | --- | --- | --- |
| `analyze-document` | `{ "document_id": Uuid, "document_version_id": Uuid, "force": bool }` | Uploading a document, calling the re-analyze bulk action, or after a metadata edit (e.g. title change) | Runs the taskflow pipeline (`GenerateThumbnailsTask`, `GenerateOcrTask`, `DetermineIssuedAtTask`, `IndexDocumentTask`) for the specified document version. The handler refuses to run if the tenant is not `Active`. |
| `purge-document` | `{ "document_id": Uuid }` | `DELETE /api/documents/{id}` after the document has been trashed | Removes every version and asset object from tenant storage, deletes database rows (`documents`, `document_versions`, associated assets/tags/correspondents), and leaves the system ready for GC. |
| `provision-tenant` | `{ "members": [Uuid, ...] }` | When a tenant is created with status `creating` | Creates/ensures the tenants Quickwit index, materializes the system capability sets (`owner`, `user`, `readonly`, `webdav`), attaches the initial member list, and flips the tenant status to `active`. |
| `delete-tenant` | `{ "remove_tenant": bool, "tenant_name": string, "action": "delete"\|"reset", "nonce": string, "issued_at": RFC3339 datetime, "signature": hex(HMAC-SHA256), "final_status"?: "active"\|"suspended" }` | Administrative action after a tenant has been marked `deleting` | Deletes all tenant-scoped storage objects, wipes the tenants Quickwit index (and optionally deletes it entirely), truncates the tenant schemas/tables, removes queued jobs for that tenant, and either deletes the tenant row or leaves it in the requested final status (defaults to `suspended`) while recreating an empty Quickwit index. |
## Retired job types
`generate-thumbnails` and `generate-ocr-text` once existed as standalone jobs.
Those behaviors now run as tasks inside `analyze-document`. No worker is
registered for the legacy types; keep them out of new payloads.
### Tenant delete/reset safety checks
The `delete-tenant` job refuses to run without a signed payload. The admin CLI
derives a message of the form `v1|tenant_id|tenant_name|action|nonce|issued_at|final_status`
and signs it with an HMAC-SHA256 key based on the servers JWT secret.
Workers verify the signature, ensure the payload matches the job flags, and
require the `issued_at` timestamp to be no more than five minutes old. This
protects against accidental wipes triggered by stale requests or insufficiently
scoped API calls.
## Operational notes
* Every job handler calls `ensure_active_tenant` (or an equivalent guard) before
touching tenant data. If a tenant is suspended or deleting, the job will fail
immediately.
* Jobs are only enqueued for the tenant they operate on. Consequently, wiping a
tenant with `delete-tenant` also removes any remaining queued jobs for that
tenant so workers do not waste effort on work that can no longer succeed.
+56
View File
@@ -0,0 +1,56 @@
# Bucket CORS for Presigned Asset Fetches
The frontend loads certain assets (e.g. OCR text) with `fetch()` against their presigned URLs
(see `frontend/src/preview/DocumentViewerPanel.jsx`). Browsers will block that request unless
the storage bucket sends CORS headers that allow the frontend origin. Configure a rule that
includes:
* the list of allowed origins (your production, staging, or local domains)
* `GET` (and optionally other methods you expose)
* permissive request headers (usually `"*"` is fine for presigned URLs)
* exposed response headers if the frontend needs them (`etag`, `content-length`, etc.)
## Example CORS document
```json
{
"CORSRules": [
{
"AllowedOrigins": ["https://app.example"],
"AllowedMethods": ["GET"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["etag", "content-length", "content-type"],
"MaxAgeSeconds": 300
}
]
}
```
Replace `https://app.example` with each domain that must fetch presigned assets. Add additional
rules if different origins require different methods.
## Applying the rule
### AWS S3 CLI
```bash
aws s3api put-bucket-cors \
--bucket <bucket-name> \
--cors-configuration file://cors.json \
[--endpoint-url <custom-endpoint>]
```
Save the JSON payload as `cors.json`. When targeting S3-compatible providers (e.g. Hetzner, Ceph RGW),
pass their endpoint via `--endpoint-url`.
### s3cmd (Ceph RGW / generic S3)
```bash
s3cmd setcors cors.json s3://<bucket-name>
```
### MinIO Client (`mc`)
```bash
mc alias set storage <endpoint> <access-key> <secret-key>
mc anonymous set-json storage/<bucket-name> cors.json
```
Most dashboards expose a similar form—paste the JSON rule into the CORS section for the bucket.
Once the rule is active, browsers will allow the frontend to read presigned assets with fetch().
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

+27
View File
@@ -0,0 +1,27 @@
# Integration Test Setup
The backend integration tests talk to a real Postgres database. To spin up an ephemeral instance locally, use the dedicated compose file:
```bash
docker compose -f docker-compose.test.yml up -d
```
This starts Postgres on port `5433` with the database/user both named `papercrate` and password `papercrate_test`. Point the test harness at it:
```bash
export TEST_DATABASE_URL=postgres://papercrate:papercrate_test@localhost:5433/papercrate_test
```
Run the tests as usual:
```bash
cargo test
```
When you are done, stop the container:
```bash
docker compose -f docker-compose.test.yml down
```
The compose file uses a tmpfs volume, so each run starts with a clean database.