Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
be451bda1e | ||
|
|
b8c2426079 |
@@ -5,15 +5,40 @@ integration testing, and infrastructure automation.
|
|||||||
|
|
||||||
## Local Development
|
## Local Development
|
||||||
|
|
||||||
Use the provided `papercrate.tmux` to spin up the full stack in one tmux session:
|
The entire application stack (frontend, backend, worker, database, minio, quickwit) runs fully containerized via Docker Compose.
|
||||||
|
|
||||||
|
### Start Development Environment
|
||||||
|
|
||||||
|
To start the stack (builds are handled automatically):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tmux -f papercrate.tmux attach
|
docker compose -f docker-compose.dev.yml up --build
|
||||||
```
|
```
|
||||||
|
|
||||||
This creates windows for the compose stack, frontend dev server, backend API, and
|
### Apply Code Changes
|
||||||
background worker using the repository-relative paths defined in the tmux file.
|
|
||||||
Detach with `Ctrl+b d` and reattach later with the same command.
|
Hot-reloading is handled automatically by `cargo watch` inside the container.
|
||||||
|
When you save files in `backend/src`, the watcher will:
|
||||||
|
|
||||||
|
1. Rebuild the modified binaries.
|
||||||
|
2. Restart the `backend`, `worker`, and `webdav` services via `supervisord`.
|
||||||
|
|
||||||
|
No manual restart is required.
|
||||||
|
|
||||||
|
### Running Migrations
|
||||||
|
|
||||||
|
Since `diesel-cli` runs inside the container:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run pending migrations
|
||||||
|
docker compose -f docker-compose.dev.yml exec server diesel migration run
|
||||||
|
|
||||||
|
# Revert last migration
|
||||||
|
docker compose -f docker-compose.dev.yml exec server diesel migration revert
|
||||||
|
|
||||||
|
# Create new migration
|
||||||
|
docker compose -f docker-compose.dev.yml exec server diesel migration generate name_of_migration
|
||||||
|
```
|
||||||
|
|
||||||
The development Postgres container now seeds two database roles:
|
The development Postgres container now seeds two database roles:
|
||||||
|
|
||||||
@@ -27,21 +52,21 @@ role with `SET ROLE papercrate_app_login;` before querying tenant tables.
|
|||||||
|
|
||||||
## Backend Integration Tests
|
## Backend Integration Tests
|
||||||
|
|
||||||
Integration tests require a running Postgres instance (and, optionally, Quickwit
|
Integration tests run in a dedicated, ephemeral container stack. The repository includes a lightweight compose file that provisions a fresh Postgres instance (using tmpfs) and Quickwit for every run.
|
||||||
for OCR indexing). The repository includes a lightweight compose file for local
|
|
||||||
runs:
|
To run the tests:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose -f docker-compose.test.yml up -d
|
docker compose -f docker-compose.test.yml run --rm test-runner
|
||||||
export TEST_DATABASE_URL=postgres://papercrate:papercrate_test@localhost:5433/papercrate_test
|
|
||||||
# optional, enables Quickwit indexing jobs
|
|
||||||
export QUICKWIT_ENDPOINT=http://localhost:7280
|
|
||||||
export QUICKWIT_INDEX=documents
|
|
||||||
cargo test
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Stop the database when you are done:
|
This will:
|
||||||
|
1. Spin up `postgres-test` and `quickwit-test` (in background if not running).
|
||||||
|
2. Start the `test-runner` container.
|
||||||
|
3. Wait for DB, run migrations, and execute `cargo test`.
|
||||||
|
4. Remove the runner container after exit.
|
||||||
|
|
||||||
|
To clean up the infrastructure afterwards:
|
||||||
```bash
|
```bash
|
||||||
docker compose -f docker-compose.test.yml down
|
docker compose -f docker-compose.test.yml down
|
||||||
```
|
```
|
||||||
@@ -50,14 +75,8 @@ The compose service uses tmpfs storage, giving each test run a clean database.
|
|||||||
|
|
||||||
## Runtime Dependencies
|
## Runtime Dependencies
|
||||||
|
|
||||||
- `ocrmypdf` (optional but recommended): Used by the OCR worker to extract text
|
- `ocrmypdf`: Used by the worker to extract text from images. If missing, the worker logs a warning and skips text extraction for that document.
|
||||||
from PDFs when no embedded text layer is available. Ensure it is installed and
|
- `Quickwit`: Used for full-text search. If configured (via `QUICKWIT_ENDPOINT`), the worker pushes extracted text to the index. If missing, search features will simply be unavailable.
|
||||||
available on the worker hosts if OCR is desired.
|
|
||||||
- Quickwit (optional): The Quickwit indexer is used to ingest extracted text for
|
|
||||||
search. Set `QUICKWIT_ENDPOINT` and `QUICKWIT_INDEX` in the environment when
|
|
||||||
running workers if you want indexing jobs to run. The local compose file starts
|
|
||||||
a Quickwit instance on `http://localhost:7280` and seeds the `documents` index
|
|
||||||
automatically.
|
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@@ -76,9 +95,8 @@ runtime settings in staging without exposing credentials.
|
|||||||
|
|
||||||
## Running Migrations in Kubernetes
|
## Running Migrations in Kubernetes
|
||||||
|
|
||||||
The backend container image ships the `diesel` CLI, so schema migrations can be
|
The backend container image ships with the `papercrate-admin` binary, which can execute schema migrations
|
||||||
executed as a short-lived Job (or Helm hook) before rolling out new pods. Example
|
as a short-lived Job (or Helm hook) before rolling out new pods. Example manifest:
|
||||||
manifest:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
apiVersion: batch/v1
|
apiVersion: batch/v1
|
||||||
@@ -92,16 +110,19 @@ spec:
|
|||||||
containers:
|
containers:
|
||||||
- name: migrate
|
- name: migrate
|
||||||
image: ghcr.io/example/papercrate-backend:<TAG>
|
image: ghcr.io/example/papercrate-backend:<TAG>
|
||||||
command: ["/usr/local/bin/diesel", "migration", "run"]
|
command: ["/usr/local/bin/papercrate-admin", "migrate-database"]
|
||||||
env:
|
env:
|
||||||
- name: DATABASE_URL
|
- name: DATABASE_URL
|
||||||
valueFrom:
|
valueFrom:
|
||||||
secretKeyRef:
|
secretKeyRef:
|
||||||
name: papercrate-db
|
name: papercrate-db
|
||||||
key: DATABASE_URL
|
key: DATABASE_URL
|
||||||
|
|
||||||
|
- name: MIGRATIONS_DATABASE_URL
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: papercrate-db
|
||||||
|
key: DATABASE_URL
|
||||||
```
|
```
|
||||||
|
|
||||||
Run the Job manually (`kubectl apply -f migrate-job.yaml`) or configure it as a
|
Run the Job manually or use the Helm hooks configured in `k8s/papercrate/templates/migrate-job.yaml`. The `papercrate-admin` binary is built specifically for administrative tasks.
|
||||||
Helm pre-install/pre-upgrade hook so migrations run automatically on each
|
|
||||||
deployment. Once the Job succeeds, deploy/update the backend `Deployment` as
|
|
||||||
usual.
|
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
target/
|
||||||
@@ -1,82 +1,192 @@
|
|||||||
# syntax=docker/dockerfile:1.7
|
# ------------------------------------------------------------------------------
|
||||||
|
# Global Arguments
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
ARG RUST_VERSION=1
|
ARG RUST_VERSION=1
|
||||||
FROM --platform=$BUILDPLATFORM rust:${RUST_VERSION}-slim AS builder
|
ARG RUNTIME_DEPS="ocrmypdf tesseract-ocr ghostscript qpdf ffmpeg"
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Base Stage: Shared Logic (PDFium)
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM --platform=$BUILDPLATFORM rust:${RUST_VERSION}-slim AS base
|
||||||
|
ARG RUNTIME_DEPS
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install PDFium
|
||||||
|
ARG TARGETARCH
|
||||||
|
RUN set -eux; \
|
||||||
|
case "${TARGETARCH}" in \
|
||||||
|
amd64|x86_64) pdfium_package=pdfium-linux-x64.tgz ;; \
|
||||||
|
arm64|aarch64) pdfium_package=pdfium-linux-arm64.tgz ;; \
|
||||||
|
*) echo "Unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
|
||||||
|
esac; \
|
||||||
|
apt-get update && apt-get install -y --no-install-recommends curl ca-certificates; \
|
||||||
|
curl -fsSL "https://github.com/bblanchon/pdfium-binaries/releases/latest/download/${pdfium_package}" -o /tmp/pdfium.tgz; \
|
||||||
|
mkdir -p /tmp/pdfium; \
|
||||||
|
tar -xzf /tmp/pdfium.tgz -C /tmp/pdfium --strip-components=1; \
|
||||||
|
pdfium_so="$(find /tmp/pdfium -name libpdfium.so -type f | head -n1)"; \
|
||||||
|
[ -n "${pdfium_so}" ]; \
|
||||||
|
mkdir -p /usr/local/lib; \
|
||||||
|
cp "${pdfium_so}" /usr/local/lib/libpdfium.so; \
|
||||||
|
rm -rf /tmp/pdfium.tgz /tmp/pdfium
|
||||||
|
|
||||||
|
# Install common build dependencies AND runtime deps (for dev/testing)
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
build-essential \
|
||||||
|
pkg-config \
|
||||||
|
libssl-dev \
|
||||||
|
libpq-dev \
|
||||||
|
libjpeg-dev \
|
||||||
|
libpng-dev \
|
||||||
|
zlib1g-dev \
|
||||||
|
ca-certificates \
|
||||||
|
curl \
|
||||||
|
git \
|
||||||
|
${RUNTIME_DEPS} \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Development Stage
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Chef Stage: Install cargo-chef (used for caching)
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM base AS chef
|
||||||
|
RUN cargo install cargo-chef
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Planner Stage: Compute lockfile recipe
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM chef AS planner
|
||||||
|
COPY . .
|
||||||
|
RUN cargo chef prepare --recipe-path recipe.json
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Cacher Stage: Build dependencies only
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM chef AS cacher
|
||||||
|
ENV CARGO_TARGET_DIR=/cargo-target
|
||||||
|
COPY --from=planner /app/recipe.json recipe.json
|
||||||
|
# Build dependencies (including test deps) based on the recipe
|
||||||
|
RUN cargo chef cook --tests --recipe-path recipe.json
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Development Stage
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM base AS development
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install additional development tools
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
procps \
|
||||||
|
postgresql-client \
|
||||||
|
supervisor \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Install diesel-cli for migrations
|
||||||
|
RUN cargo install diesel_cli --no-default-features --features postgres
|
||||||
|
|
||||||
|
# Install cargo-watch for hot reloading
|
||||||
|
RUN cargo install cargo-watch
|
||||||
|
|
||||||
|
# Setup Cache
|
||||||
|
ENV CARGO_TARGET_DIR=/cargo-target
|
||||||
|
COPY --from=cacher /cargo-target /cargo-target
|
||||||
|
COPY --from=cacher /usr/local/cargo /usr/local/cargo
|
||||||
|
|
||||||
|
# Copy PDFium from base
|
||||||
|
COPY --from=base /usr/local/lib/libpdfium.so /usr/local/lib/libpdfium.so
|
||||||
|
ENV LD_LIBRARY_PATH=/usr/local/lib
|
||||||
|
RUN ldconfig
|
||||||
|
|
||||||
|
ENV RUST_LOG=info
|
||||||
|
CMD ["./run-dev.sh"]
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Production Builder Stage
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# We restart from base to keep the image clean, but copy PDFium if needed for build/tests
|
||||||
|
FROM base AS builder
|
||||||
ARG TARGETARCH
|
ARG TARGETARCH
|
||||||
ENV TARGETARCH=${TARGETARCH}
|
ENV TARGETARCH=${TARGETARCH}
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Resolve cross-compilation target script
|
||||||
RUN cat <<'SCRIPT' >/usr/local/bin/resolve-target.sh
|
RUN cat <<'SCRIPT' >/usr/local/bin/resolve-target.sh
|
||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -e
|
set -e
|
||||||
case "$1" in
|
case "$1" in
|
||||||
amd64)
|
amd64) echo x86_64-unknown-linux-gnu ;;
|
||||||
echo x86_64-unknown-linux-gnu
|
arm64) echo aarch64-unknown-linux-gnu ;;
|
||||||
;;
|
*) echo "Unsupported TARGETARCH: $1" >&2; exit 1 ;;
|
||||||
arm64)
|
|
||||||
echo aarch64-unknown-linux-gnu
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo "Unsupported TARGETARCH: $1" >&2
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
esac
|
||||||
SCRIPT
|
SCRIPT
|
||||||
RUN chmod +x /usr/local/bin/resolve-target.sh
|
RUN chmod +x /usr/local/bin/resolve-target.sh
|
||||||
|
|
||||||
ENV PKG_CONFIG_ALLOW_CROSS=1 \
|
# Install cross-compilation deps
|
||||||
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
|
ARG BUILDPLATFORM
|
||||||
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
|
RUN if [ "${TARGETARCH}" = "amd64" ]; then \
|
||||||
PKG_CONFIG_LIBDIR_aarch64_unknown_linux_gnu=/usr/lib/aarch64-linux-gnu/pkgconfig \
|
echo "x86_64-linux-gnu" > /tmp/target_deb_arch; \
|
||||||
PKG_CONFIG_PATH_aarch64_unknown_linux_gnu=/usr/lib/aarch64-linux-gnu/pkgconfig \
|
elif [ "${TARGETARCH}" = "arm64" ]; then \
|
||||||
PKG_CONFIG_SYSROOT_DIR_aarch64_unknown_linux_gnu=/usr/aarch64-linux-gnu \
|
echo "aarch64-linux-gnu" > /tmp/target_deb_arch; \
|
||||||
OPENSSL_DIR_aarch64_unknown_linux_gnu=/usr/lib/aarch64-linux-gnu \
|
else \
|
||||||
OPENSSL_LIB_DIR_aarch64_unknown_linux_gnu=/usr/lib/aarch64-linux-gnu \
|
echo "Unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1; \
|
||||||
OPENSSL_INCLUDE_DIR_aarch64_unknown_linux_gnu=/usr/include/aarch64-linux-gnu \
|
fi
|
||||||
CC_x86_64_unknown_linux_gnu=x86_64-linux-gnu-gcc \
|
|
||||||
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=x86_64-linux-gnu-gcc \
|
|
||||||
PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu=/usr/lib/x86_64-linux-gnu/pkgconfig \
|
|
||||||
PKG_CONFIG_PATH_x86_64_unknown_linux_gnu=/usr/lib/x86_64-linux-gnu/pkgconfig \
|
|
||||||
OPENSSL_DIR_x86_64_unknown_linux_gnu=/usr/lib/x86_64-linux-gnu \
|
|
||||||
OPENSSL_LIB_DIR_x86_64_unknown_linux_gnu=/usr/lib/x86_64-linux-gnu \
|
|
||||||
OPENSSL_INCLUDE_DIR_x86_64_unknown_linux_gnu=/usr/include/x86_64-linux-gnu
|
|
||||||
|
|
||||||
RUN set -eux; \
|
RUN set -eux; \
|
||||||
case "${TARGETARCH}" in \
|
# Detect build arch (assuming debian-like names compatible with apt)
|
||||||
amd64) TARGET_DEB_ARCH=amd64 ;; \
|
dpkg_arch="$(dpkg --print-architecture)"; \
|
||||||
arm64) TARGET_DEB_ARCH=arm64 ;; \
|
target_deb_arch="$(cat /tmp/target_deb_arch)"; \
|
||||||
*) echo "Unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
|
\
|
||||||
esac; \
|
# If we are cross-compiling
|
||||||
BUILD_DEB_ARCH="$(dpkg --print-architecture)"; \
|
if [ "${dpkg_arch}" != "${TARGETARCH}" ]; then \
|
||||||
if [ "${TARGET_DEB_ARCH}" != "${BUILD_DEB_ARCH}" ]; then \
|
# Map target arch to debian package arch suffix if needed, but usually apt handles :arch
|
||||||
dpkg --add-architecture "${TARGET_DEB_ARCH}"; \
|
# For cross-compiling, we need to add the architecture
|
||||||
fi; \
|
dpkg --add-architecture "${TARGETARCH}"; \
|
||||||
apt-get update; \
|
apt-get update; \
|
||||||
apt-get install -y --no-install-recommends \
|
\
|
||||||
build-essential \
|
|
||||||
pkg-config \
|
|
||||||
curl \
|
|
||||||
libssl-dev \
|
|
||||||
libpq-dev \
|
|
||||||
libjpeg-dev \
|
|
||||||
libpng-dev \
|
|
||||||
zlib1g-dev; \
|
|
||||||
if [ "${TARGET_DEB_ARCH}" != "${BUILD_DEB_ARCH}" ]; then \
|
|
||||||
case "${TARGETARCH}" in \
|
case "${TARGETARCH}" in \
|
||||||
arm64) CROSS_GCC=gcc-aarch64-linux-gnu ;; \
|
arm64) CROSS_GCC=gcc-aarch64-linux-gnu ;; \
|
||||||
amd64) CROSS_GCC=gcc-x86-64-linux-gnu ;; \
|
amd64) CROSS_GCC=gcc-x86-64-linux-gnu ;; \
|
||||||
esac; \
|
esac; \
|
||||||
|
\
|
||||||
apt-get install -y --no-install-recommends \
|
apt-get install -y --no-install-recommends \
|
||||||
"${CROSS_GCC}" \
|
"${CROSS_GCC}" \
|
||||||
"libc6-dev:${TARGET_DEB_ARCH}" \
|
"libc6-dev:${TARGETARCH}" \
|
||||||
"libssl-dev:${TARGET_DEB_ARCH}" \
|
"libssl-dev:${TARGETARCH}" \
|
||||||
"libpq-dev:${TARGET_DEB_ARCH}" \
|
"libpq-dev:${TARGETARCH}" \
|
||||||
"libjpeg-dev:${TARGET_DEB_ARCH}" \
|
"libjpeg-dev:${TARGETARCH}" \
|
||||||
"libpng-dev:${TARGET_DEB_ARCH}" \
|
"libpng-dev:${TARGETARCH}" \
|
||||||
"zlib1g-dev:${TARGET_DEB_ARCH}"; \
|
"zlib1g-dev:${TARGETARCH}"; \
|
||||||
|
\
|
||||||
|
# Configure PKG_CONFIG and LINKER to find foreign libraries
|
||||||
|
case "${TARGETARCH}" in \
|
||||||
|
"amd64") \
|
||||||
|
GNU_ARCH="x86_64-linux-gnu" \
|
||||||
|
RUST_ARCH="x86_64_unknown_linux_gnu" \
|
||||||
|
RUST_ARCH_UPPER="X86_64_UNKNOWN_LINUX_GNU" \
|
||||||
|
;; \
|
||||||
|
"arm64") \
|
||||||
|
GNU_ARCH="aarch64-linux-gnu" \
|
||||||
|
RUST_ARCH="aarch64_unknown_linux_gnu" \
|
||||||
|
RUST_ARCH_UPPER="AARCH64_UNKNOWN_LINUX_GNU" \
|
||||||
|
;; \
|
||||||
|
esac; \
|
||||||
|
\
|
||||||
|
{ \
|
||||||
|
echo "export PKG_CONFIG_ALLOW_CROSS=1"; \
|
||||||
|
echo "export PKG_CONFIG_PATH=/usr/lib/${GNU_ARCH}/pkgconfig"; \
|
||||||
|
echo "export OPENSSL_DIR=/usr/lib/${GNU_ARCH}"; \
|
||||||
|
echo "export OPENSSL_LIB_DIR=/usr/lib/${GNU_ARCH}"; \
|
||||||
|
echo "export OPENSSL_INCLUDE_DIR=/usr/include/${GNU_ARCH}"; \
|
||||||
|
echo "export CARGO_TARGET_${RUST_ARCH_UPPER}_LINKER=${GNU_ARCH}-gcc"; \
|
||||||
|
echo "export CC_${RUST_ARCH}=${GNU_ARCH}-gcc"; \
|
||||||
|
echo "export CXX_${RUST_ARCH}=${GNU_ARCH}-g++"; \
|
||||||
|
} >> /etc/profile; \
|
||||||
fi; \
|
fi; \
|
||||||
rm -rf /var/lib/apt/lists/*
|
rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
ENV PKG_CONFIG_ALLOW_CROSS=1
|
||||||
|
|
||||||
COPY Cargo.toml Cargo.lock build.rs ./
|
COPY Cargo.toml Cargo.lock build.rs ./
|
||||||
COPY src ./src
|
COPY src ./src
|
||||||
COPY migrations ./migrations
|
COPY migrations ./migrations
|
||||||
@@ -85,6 +195,9 @@ COPY resources ./resources
|
|||||||
COPY diesel.toml ./
|
COPY diesel.toml ./
|
||||||
|
|
||||||
RUN set -eux; \
|
RUN set -eux; \
|
||||||
|
echo "Loading cross-compilation environment..."; \
|
||||||
|
. /etc/profile; \
|
||||||
|
export PATH="$PATH:/usr/local/cargo/bin"; \
|
||||||
TARGET="$(/usr/local/bin/resolve-target.sh "${TARGETARCH}")"; \
|
TARGET="$(/usr/local/bin/resolve-target.sh "${TARGETARCH}")"; \
|
||||||
rustup target add "${TARGET}"; \
|
rustup target add "${TARGET}"; \
|
||||||
cargo build --release --target "${TARGET}" --bin backend --bin worker --bin webdav --bin admin; \
|
cargo build --release --target "${TARGET}" --bin backend --bin worker --bin webdav --bin admin; \
|
||||||
@@ -93,37 +206,18 @@ RUN set -eux; \
|
|||||||
cp "target/${TARGET}/release/${bin}" "/artifacts/${bin}"; \
|
cp "target/${TARGET}/release/${bin}" "/artifacts/${bin}"; \
|
||||||
done
|
done
|
||||||
|
|
||||||
RUN set -eux; \
|
# ------------------------------------------------------------------------------
|
||||||
case "${TARGETARCH}" in \
|
# Production Runtime Stage
|
||||||
amd64) pdfium_package=pdfium-linux-x64.tgz ;; \
|
# ------------------------------------------------------------------------------
|
||||||
arm64) pdfium_package=pdfium-linux-arm64.tgz ;; \
|
FROM debian:trixie-slim AS runtime
|
||||||
*) echo "Unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
|
|
||||||
esac; \
|
|
||||||
curl -fsSL "https://github.com/bblanchon/pdfium-binaries/releases/latest/download/${pdfium_package}" -o /tmp/pdfium.tgz; \
|
|
||||||
mkdir -p /tmp/pdfium; \
|
|
||||||
tar -xzf /tmp/pdfium.tgz -C /tmp/pdfium --strip-components=1; \
|
|
||||||
pdfium_so="$(find /tmp/pdfium -name libpdfium.so -type f | head -n1)"; \
|
|
||||||
[ -n "${pdfium_so}" ]; \
|
|
||||||
cp "${pdfium_so}" /artifacts/libpdfium.so; \
|
|
||||||
rm -rf /tmp/pdfium.tgz /tmp/pdfium
|
|
||||||
|
|
||||||
FROM --platform=$TARGETPLATFORM debian:trixie-slim AS runtime
|
|
||||||
ARG TARGETARCH
|
ARG TARGETARCH
|
||||||
|
ARG RUNTIME_DEPS
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends \
|
&& apt-get install -y --no-install-recommends \
|
||||||
ca-certificates \
|
ca-certificates curl libssl3 libpq5 libjpeg62-turbo libpng16-16 \
|
||||||
curl \
|
${RUNTIME_DEPS} \
|
||||||
libssl3 \
|
|
||||||
libpq5 \
|
|
||||||
libjpeg62-turbo \
|
|
||||||
libpng16-16 \
|
|
||||||
ocrmypdf \
|
|
||||||
tesseract-ocr \
|
|
||||||
ghostscript \
|
|
||||||
qpdf \
|
|
||||||
ffmpeg \
|
|
||||||
&& rm -rf /var/lib/apt/lists/* \
|
&& rm -rf /var/lib/apt/lists/* \
|
||||||
&& mkdir -p /usr/local/lib \
|
&& mkdir -p /usr/local/lib \
|
||||||
&& useradd --system --create-home --uid 10001 appuser
|
&& useradd --system --create-home --uid 10001 appuser
|
||||||
@@ -132,7 +226,8 @@ COPY --from=builder /artifacts/backend /usr/local/bin/papercrate-backend
|
|||||||
COPY --from=builder /artifacts/worker /usr/local/bin/papercrate-worker
|
COPY --from=builder /artifacts/worker /usr/local/bin/papercrate-worker
|
||||||
COPY --from=builder /artifacts/webdav /usr/local/bin/papercrate-webdav
|
COPY --from=builder /artifacts/webdav /usr/local/bin/papercrate-webdav
|
||||||
COPY --from=builder /artifacts/admin /usr/local/bin/papercrate-admin
|
COPY --from=builder /artifacts/admin /usr/local/bin/papercrate-admin
|
||||||
COPY --from=builder /artifacts/libpdfium.so /usr/local/lib/libpdfium.so
|
COPY --from=base /usr/local/lib/libpdfium.so /usr/local/lib/libpdfium.so
|
||||||
|
|
||||||
RUN ldconfig
|
RUN ldconfig
|
||||||
COPY migrations ./migrations
|
COPY migrations ./migrations
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,2 @@
|
|||||||
[print_schema]
|
|
||||||
file = "src/schema.rs"
|
|
||||||
custom_type_derives = ["diesel::query_builder::QueryId", "Clone"]
|
|
||||||
|
|
||||||
[migrations_directory]
|
[migrations_directory]
|
||||||
dir = "migrations"
|
dir = "migrations"
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "Waiting for Postgres..."
|
||||||
|
until pg_isready -h postgres -U papercrate; do
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Running migrations..."
|
||||||
|
diesel migration run
|
||||||
|
|
||||||
|
echo "Building binaries for first run..."
|
||||||
|
cargo build --bin backend --bin worker --bin webdav
|
||||||
|
|
||||||
|
echo "Starting supervisord..."
|
||||||
|
exec supervisord -c supervisord.conf
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
[supervisord]
|
||||||
|
nodaemon=true
|
||||||
|
user=root
|
||||||
|
logfile=/dev/null
|
||||||
|
logfile_maxbytes=0
|
||||||
|
pidfile=/var/run/supervisord.pid
|
||||||
|
|
||||||
|
[unix_http_server]
|
||||||
|
file=/var/run/supervisor.sock
|
||||||
|
chmod=0700
|
||||||
|
|
||||||
|
[rpcinterface:supervisor]
|
||||||
|
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
|
||||||
|
|
||||||
|
[supervisorctl]
|
||||||
|
serverurl=unix:///var/run/supervisor.sock
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Applications
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
[program:backend]
|
||||||
|
command=/cargo-target/debug/backend
|
||||||
|
directory=/app
|
||||||
|
autostart=true
|
||||||
|
autorestart=true
|
||||||
|
stdout_logfile=/dev/stdout
|
||||||
|
stdout_logfile_maxbytes=0
|
||||||
|
stderr_logfile=/dev/stderr
|
||||||
|
stderr_logfile_maxbytes=0
|
||||||
|
|
||||||
|
[program:worker]
|
||||||
|
command=/cargo-target/debug/worker
|
||||||
|
directory=/app
|
||||||
|
autostart=true
|
||||||
|
autorestart=true
|
||||||
|
stdout_logfile=/dev/stdout
|
||||||
|
stdout_logfile_maxbytes=0
|
||||||
|
stderr_logfile=/dev/stderr
|
||||||
|
stderr_logfile_maxbytes=0
|
||||||
|
|
||||||
|
[program:webdav]
|
||||||
|
command=/cargo-target/debug/webdav
|
||||||
|
directory=/app
|
||||||
|
autostart=true
|
||||||
|
autorestart=true
|
||||||
|
stdout_logfile=/dev/stdout
|
||||||
|
stdout_logfile_maxbytes=0
|
||||||
|
stderr_logfile=/dev/stderr
|
||||||
|
stderr_logfile_maxbytes=0
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Watcher
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
[program:watcher]
|
||||||
|
# 1. Check syntax
|
||||||
|
# 2. Build binaries
|
||||||
|
# 3. Restart services via supervisorctl
|
||||||
|
command=cargo watch -w src -x check -x "build --bin backend --bin worker --bin webdav" -s "supervisorctl restart backend worker webdav"
|
||||||
|
directory=/app
|
||||||
|
autostart=true
|
||||||
|
autorestart=true
|
||||||
|
stdout_logfile=/dev/stdout
|
||||||
|
stdout_logfile_maxbytes=0
|
||||||
|
stderr_logfile=/dev/stderr
|
||||||
|
stderr_logfile_maxbytes=0
|
||||||
@@ -1,3 +1,17 @@
|
|||||||
|
x-backend-env: &backend-env
|
||||||
|
DATABASE_URL: postgres://papercrate:papercrate_dev@postgres:5432/papercrate
|
||||||
|
DATABASE_MAX_POOL_SIZE: 2
|
||||||
|
AWS_ENDPOINT_URL: http://minio:9000
|
||||||
|
AWS_ACCESS_KEY_ID: minioadmin
|
||||||
|
AWS_SECRET_ACCESS_KEY: minioadmin
|
||||||
|
AWS_REGION: us-east-1
|
||||||
|
S3_BUCKET: documents
|
||||||
|
PROXY_DOWNLOADS: "true"
|
||||||
|
JWT_SECRET: change-me-super-secret
|
||||||
|
QUICKWIT_ENDPOINT: http://quickwit:7280
|
||||||
|
WEBAUTHN_RP_ID: localhost
|
||||||
|
WEBAUTHN_ORIGIN: http://localhost:5173
|
||||||
|
|
||||||
services:
|
services:
|
||||||
postgres:
|
postgres:
|
||||||
image: postgres:16-alpine
|
image: postgres:16-alpine
|
||||||
@@ -62,32 +76,49 @@ services:
|
|||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 5
|
retries: 5
|
||||||
|
|
||||||
admin-bootstrap:
|
server:
|
||||||
build:
|
build:
|
||||||
context: ./backend
|
context: ./backend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
target: development
|
||||||
|
network: host
|
||||||
|
volumes:
|
||||||
|
- ./backend:/app
|
||||||
|
environment:
|
||||||
|
<<: *backend-env
|
||||||
|
RUST_LOG: info
|
||||||
|
SERVER_HOST: 0.0.0.0
|
||||||
|
SERVER_PORT: 3000
|
||||||
|
ports:
|
||||||
|
- "3000:3000"
|
||||||
|
- "3001:3001"
|
||||||
depends_on:
|
depends_on:
|
||||||
postgres:
|
postgres:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
|
minio:
|
||||||
|
condition: service_healthy
|
||||||
quickwit:
|
quickwit:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
|
createbuckets:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
|
||||||
|
frontend:
|
||||||
|
build:
|
||||||
|
context: ./frontend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
target: development
|
||||||
|
network: host
|
||||||
|
volumes:
|
||||||
|
- ./frontend:/app
|
||||||
|
- /app/node_modules
|
||||||
|
- /app/dist
|
||||||
|
ports:
|
||||||
|
- "5173:5173"
|
||||||
environment:
|
environment:
|
||||||
DATABASE_URL: postgres://papercrate:papercrate_dev@postgres:5432/papercrate
|
API_PROXY_PASS: http://server:3000
|
||||||
DATABASE_MAX_POOL_SIZE: 2
|
depends_on:
|
||||||
AWS_ENDPOINT_URL: http://minio:9000
|
server:
|
||||||
AWS_ACCESS_KEY_ID: minioadmin
|
condition: service_started
|
||||||
AWS_SECRET_ACCESS_KEY: minioadmin
|
|
||||||
AWS_REGION: us-east-1
|
|
||||||
S3_BUCKET: documents
|
|
||||||
JWT_SECRET: change-me-super-secret
|
|
||||||
QUICKWIT_ENDPOINT: http://quickwit:7280
|
|
||||||
entrypoint: []
|
|
||||||
command: >
|
|
||||||
/bin/sh -c "
|
|
||||||
echo 'Running database migrations' &&
|
|
||||||
diesel migration run
|
|
||||||
"
|
|
||||||
user: root
|
|
||||||
restart: "no"
|
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
postgres_data:
|
postgres_data:
|
||||||
|
|||||||
@@ -31,28 +31,33 @@ services:
|
|||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 5
|
retries: 5
|
||||||
|
|
||||||
admin-bootstrap:
|
test-runner:
|
||||||
build:
|
build:
|
||||||
context: ./backend
|
context: ./backend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
target: development
|
||||||
depends_on:
|
depends_on:
|
||||||
postgres-test:
|
postgres-test:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
quickwit-test:
|
quickwit-test:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
|
volumes:
|
||||||
|
- ./backend:/app
|
||||||
|
- /app/target
|
||||||
environment:
|
environment:
|
||||||
DATABASE_URL: postgres://papercrate:papercrate_test@postgres-test:5432/papercrate_test
|
TEST_DATABASE_URL: postgres://papercrate:papercrate_test@postgres-test:5432/papercrate_test
|
||||||
DATABASE_MAX_POOL_SIZE: 1
|
DATABASE_MAX_POOL_SIZE: 1
|
||||||
S3_BUCKET: documents
|
S3_BUCKET: documents
|
||||||
JWT_SECRET: change-me-super-secret
|
JWT_SECRET: change-me-super-secret
|
||||||
QUICKWIT_ENDPOINT: http://quickwit-test:7280
|
QUICKWIT_ENDPOINT: http://quickwit-test:7280
|
||||||
entrypoint: []
|
RUST_LOG: info
|
||||||
command: >
|
command: >
|
||||||
/bin/sh -c "
|
/bin/bash -c "
|
||||||
echo 'Running database migrations' &&
|
echo 'Waiting for Postgres...' &&
|
||||||
diesel migration run
|
until pg_isready -h postgres-test -U papercrate; do sleep 1; done &&
|
||||||
|
echo 'Running tests...' &&
|
||||||
|
cargo test
|
||||||
"
|
"
|
||||||
user: root
|
|
||||||
restart: "no"
|
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
quickwit_test_data:
|
quickwit_test_data:
|
||||||
|
|||||||
|
Before Width: | Height: | Size: 2.7 MiB After Width: | Height: | Size: 2.5 MiB |
|
Before Width: | Height: | Size: 1.4 MiB After Width: | Height: | Size: 2.0 MiB |
|
Before Width: | Height: | Size: 683 KiB After Width: | Height: | Size: 1.4 MiB |
|
Before Width: | Height: | Size: 1.1 MiB After Width: | Height: | Size: 557 KiB |
@@ -3,16 +3,32 @@
|
|||||||
ARG NODE_IMAGE=node:20-alpine
|
ARG NODE_IMAGE=node:20-alpine
|
||||||
ARG NGINX_IMAGE=nginx:alpine
|
ARG NGINX_IMAGE=nginx:alpine
|
||||||
|
|
||||||
FROM --platform=$BUILDPLATFORM ${NODE_IMAGE} AS build
|
# ------------------------------------------------------------------------------
|
||||||
|
# Base Stage: Install Dependencies
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM --platform=$BUILDPLATFORM ${NODE_IMAGE} AS base
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
COPY package.json package-lock.json ./
|
COPY package.json package-lock.json ./
|
||||||
RUN npm ci --no-audit --no-fund
|
RUN npm install
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Development Stage
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM base AS development
|
||||||
|
EXPOSE 5173
|
||||||
|
CMD ["npm", "run", "dev", "--", "--host", "0.0.0.0"]
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Build Stage
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM base AS build
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN npm run build
|
RUN npm run build
|
||||||
|
|
||||||
FROM ${NGINX_IMAGE}
|
# ------------------------------------------------------------------------------
|
||||||
|
# Runtime Stage
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
FROM ${NGINX_IMAGE} AS runtime
|
||||||
WORKDIR /usr/share/nginx/html
|
WORKDIR /usr/share/nginx/html
|
||||||
|
|
||||||
COPY --from=build /app/dist ./
|
COPY --from=build /app/dist ./
|
||||||
|
|||||||
@@ -65,7 +65,7 @@ module.exports = {
|
|||||||
proxy: [
|
proxy: [
|
||||||
{
|
{
|
||||||
context: ['/api', '/download'],
|
context: ['/api', '/download'],
|
||||||
target: 'http://127.0.0.1:3000',
|
target: process.env.API_PROXY_PASS,
|
||||||
changeOrigin: true,
|
changeOrigin: true,
|
||||||
secure: false,
|
secure: false,
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -1,11 +0,0 @@
|
|||||||
# tmux session for Papercrate dev stack
|
|
||||||
new-session -d -s papercrate -n docker -c . 'docker compose -f docker-composer.dev.yml up'
|
|
||||||
|
|
||||||
new-window -t papercrate:1 -n frontend -c ./frontend 'npm run dev'
|
|
||||||
|
|
||||||
new-window -t papercrate:2 -n backend -c ./backend 'cargo run --bin backend'
|
|
||||||
|
|
||||||
new-window -t papercrate:3 -n worker -c ./backend 'cargo run --bin worker'
|
|
||||||
|
|
||||||
select-window -t papercrate:0
|
|
||||||
attach-session -t papercrate
|
|
||||||