Local product development

For the current route/menu map, see the desktop guide.

Purpose and Prerequisites

Run the locally delivered Recollect platform with Rust/Cargo, Node/npm, Python 3 and Docker Compose. Verified with Rust 1.94, Node 26 and Docker 29 on macOS ARM. The runtime ADR records the user's speed-first development posture. The owner credential is environment-backed and invited account passwords are stored as Argon2id hashes (ADR 0024); no artifact digests or strict release/version gates are introduced. Developer supply-chain pins (container image digests, downloaded-tool SHA-256 values and Actions commit SHAs) pin what the repository pulls in and add no product hash, handshake or release gate.

Procedure

From the repository root:

./scripts/dev.sh

Open http://127.0.0.1:8787. Sign in with RECOLLECT_OWNER_USERNAME and RECOLLECT_OWNER_PASSWORD from the generated, ignored .env file. Do not paste that file into logs or issues. Initial setup creates random credentials and sets file permissions to 0600. Re-running setup preserves the existing file.

The command starts the repository's PostgreSQL/pgvector and Neo4j/GDS services, applies missing migrations, builds the generated API client and browser, and runs the native Rust API and worker. Ctrl-C stops both child processes; data and databases remain. Stop only this project's databases with:

./scripts/docker.sh compose stop

PostgreSQL data belongs to the Compose-labelled recollect_postgres_data Docker volume. Graph data/logs/plugins live under ignored .data/neo4j/. The abandoned .data/postgres/ directory, if present from the initial failed macOS bind-mount attempt, is not used by the service. Do not remove volumes as a restart procedure.

Host database ports are loopback-only: PostgreSQL 55432, Neo4j HTTP 57474. Other Docker projects and reference checkouts are untouched. The Docker wrapper uses a repository-local anonymous registry configuration for public development images, avoiding the blocked desktop credential helper without editing personal config.

For API-only iteration after initial setup, load the environment and run:

set -a
source .env
set +a
cargo run -p recollect-server -- serve

After server interface changes run ./scripts/generate-api.sh, then npm --prefix web run build. Generation checks for duplicate operation IDs before building the typed client. The committed web/src/api-schema.d.ts is the output of the locked generator (openapi-typescript 7.13.0), so regenerating without an API change leaves it byte-identical. The browser is served from web/dist at the same origin. The Brain Sources route supports imports, source history and overlapping views; see source evidence. Artifacts live under RECOLLECT_ARTIFACT_DIR, default .data/artifacts, and must be available to both the API and worker. The workspace panel and plugin provide checkout registration and task/subagent scope without uploading ordinary source files. The publication workflow adds isolated committed extraction, shared snapshots and environment revision histories. The packaged plugin includes Enola; source-checkout developers install the adapter with python3 scripts/setup-enola.py (pinned SHA-256, checked before extraction) before publishing. An optional OpenAI connection probe uses the user's environment key with fixed synthetic text. The delivered Brain model policy and autonomous maintenance use the selected text/embedding pair through the shared OpenAI/OpenRouter gateway. API-created Brains default to transmission disabled unless managed setup is requested; the browser includes the managed autonomous preset. Existing Brains retain their policies. Once configured, the native worker learns permitted sources, revises machine-maintained memory and refreshes handovers without individual review. The graph panel shows automatically built repository/knowledge projections, exact inputs, evidence and bounded paths. Use combined repository graphs for exact manifest-selected repositories and parsed pinned dependency links.

Verification

The platform suite needs PostgreSQL/Neo4j. Two encrypted-recovery cases (remote_erasure_failure_denies_locally_and_retries_one_encrypted_entry and recovery_analytics_ownership_is_mirrored_before_native_work_and_retained_for_restore) also need the owned SFTP fixture. Without it the platform script skips them and prints NOT RUN: <names> (no SFTP fixture; ...) before and after the run, so a passing summary does not claim they ran. To run them, prepare the fixture once:

python3 scripts/setup-recovery-tools.py
./scripts/docker.sh build -t recollect-recovery-sftp:local infra/recovery
python3 scripts/start-recovery-fixture.py
export RECOLLECT_TEST_RECOVERY_FIXTURE=/absolute/path/printed/fixture.json

RECOLLECT_TEST_REQUIRE_RECOVERY=1 restores the strict early refusal when the fixture is not selected, for a complete pre-release run. The fixture command prints its exact private JSON path and owned container name. After the tests, remove that named fixture container and its generated directory; do not use a blanket database/container cleanup. Failed fixture databases and independent journals must remain until their owned graph work is absent. The platform script builds the required native MCP example and passes extra arguments (name filters, --exact) to the test binary. Native coding-host tests use their separate ./scripts/test-mcp-hosts.sh fixture. Run the graph/interruption tests without concurrent builds or other full suites so their short controlled pauses remain meaningful.

Test helper executables (mcp-fixture, recollect-agent, recollect-plugin) are found next to the running test binary, so CARGO_TARGET_DIR or build.target-dir work; ./scripts/test-ui.sh reads the directory from cargo metadata and, for an .env written before setup-local.sh recorded COMPOSE_FILE, defaults it to the same two Compose files. Per-test state (credentials, artifacts, erasure journals) lives under <target>/debug/test-state unless RECOLLECT_TEST_STATE_DIR names another directory, and is removed with the database on success. The scripts and the repository's .cargo/config.toml set RUST_MIN_STACK to 8 MiB because the long native-worker tests nest large debug-build futures; export your own value to override it (the config never replaces an existing variable). Failed runs keep their recollect_test_* / recollect_ui_* databases for diagnosis; list them with python3 scripts/clean-test-databases.py and, once no test run is active, drop the idle ones with --drop. It matches only those two exact name patterns and never touches the installed Brain's database.

Two platform tests bound real time and are sensitive to machine load. organization_weighted_5000_inputs_publish_without_a_long_brain_lock prepares 5,000 sources under the contract's five-second read-only bound. On a shared developer machine (unoptimized server library, PostgreSQL shared with other work) one preparation took 3.7 to 6.9 seconds, so the scheduling pass and the job can each defer and retry as in production. The test repeats those retries up to fixed ceilings, records preparation_deferrals in its .cache/automatic-mapping receipts and still fails if every attempt overruns. The semantic execution admission tests drive the production worker, whose five-second lease heartbeat can land between the staged cancellation or reclaim and the worker's completion; such an attempt is inconclusive and is repeated on a fresh fixture (at most three times), never counted as gateway-boundary proof.

curl -fsS http://127.0.0.1:8787/health/ready
cargo run -p recollect-agent -- health
./scripts/test-platform.sh
./scripts/test-ui.sh
./scripts/validate.sh

Readiness requires actual PostgreSQL, pgvector distance and Neo4j/GDS queries. The authenticated installation health view shows individual connection state and observed versions. A running container alone is insufficient.

The core Rust integration scenarios create uniquely named disposable databases and exercise production handlers, authorization/RLS, atomic commands, concurrent replay, lease recovery, retries, backpressure and dependency health. They also cover invited identity, recovery and effective role/ownership changes. Device scenarios cover browser approval, bearer authority and queued revocation. Evidence scenarios cover real retained files, immutable versions, shared views, capture policy, source spans and missing/storage-failure recovery. Workspace scenarios cover private checkout paths, shared repository identities, concurrent child scopes, immutable operations and effective revocation. Run cargo test -p recollect-protocol -p recollect-agent --lib for the real Git/worktree discovery and origin-normalization proof. Publication adds immutable artifacts, contributor deduplication, scoped processing, missing-artifact recovery and manifest history. The real native extraction fixture is cargo test -p recollect-agent --test publication after Enola setup. Claims/review scenarios cover temporal evidence, human review, rejection and revalidation, conflict dispositions, replay and independent applicability. See claims and time and review/corrections. Retention scenarios cover deadlines, independent excerpts, dependency erasure, cleanup retry, actual older-database replay and native offline-copy isolation. The retention runbook describes the durable erasure journal, which must survive separately from database backups. Model scenarios cover actual HTTP responses, policy/role/budget admission, explicit retry, source learning, rejected/conflicting values and in-flight erasure/revocation. Procedure/handover scenarios cover typed observations, combined scope, exact contributors, synthesis and restoration. Autonomous scenarios cover catch-up, same-identity revision, handover refresh, retirement, uncertainty, bounded retries, durable human overrides and physical expiry without individual acceptance calls. The opt-in real OpenAI browser workflow is documented in the model runbook. The browser tests run desktop/mobile workflows with a fresh disposable database per test file and a separate API on 8788, using an isolated Chrome session. Google Chrome is required for that check; neither personal browser state nor real workspace Brains are changed. Screenshots are in .cache/ui/; traces/video are disabled to avoid recording credentials. It also starts a fresh native worker to drain persisted queued work. Test browser assets are built under .cache/ui/web-dist, preserving the normal application's served bundle during focused browser checks. Native companion tests create a unique OS-store profile and delete only that test entry; see device pairing. Optional OIDC signed exchange and its full browser workflow are checked with ./scripts/test-oidc.sh and ./scripts/test-oidc.sh ui; see team access.

Failure and Recovery

• Database unavailable: inspect ./scripts/docker.sh compose ps. PostgreSQL's health check executes a database query, detecting incomplete initialization. could not resize shared memory segment (SQLSTATE 53100) followed by pool timeouts means /dev/shm was too small for parallel queries; the Compose service sets shm_size: 256m, which takes effect when the container is next recreated.
• Graph degraded: confirm the GDS plugin loaded and the configured graph target is reachable. Recollect does not label a failed Query API call connected.
• Login failed: verify local environment names without printing passwords. Ten attempts per minute for one username from one client address cause temporary throttling for that address only. Anonymous pairing and organization sign-in starts allow 30 per address per minute. Behind the bundled proxy the client address comes from its X-Forwarded-For header.
• Owner recovery: stop the API, edit the owner password in the ignored .env, load the environment as above, run cargo run -p recollect-server -- recover-owner, and restart. This re-enables the persisted owner and revokes its sessions; it does not change the owner identity or grant unrelated Brain access.
• Port conflict: stop only a known Recollect process, or adjust its bind and public origin together. Never kill an unrelated listener to free a port.
• Processing delayed: open the Brain's Processing panel for actual queue state, attempts and safe failure reasons. Admins can cancel queued/running work and retry failed/cancelled work. The worker log is .cache/runtime/worker.log. With the environment loaded, run cargo run -p recollect-server -- worker to resume the durable queue independently of the API. Interrupted leases recover after 20 seconds; three attempts exhaust automatic retries. Current canonical Brain metadata stays available while its projection catches up.

This runbook covers local bootstrap, durable work, team access, devices and source evidence, workspace/task scope, committed repository publication, claims/review and retention/erasure plus governed model learning. Delivered recall and graph procedures have linked runbooks. Analytics, graph exploration, MCP/Vault and shared/recovery operation retain their roadmap slices.