Flip the 4 business ORM models (ThreadMetaRow, RunRow, FeedbackRow,
RunEventRow) to ``workspace_id: Mapped[str]`` with ``nullable=False``.
PR5's alembic 0003 already enforces NOT NULL at the DB layer; this
aligns the ORM-driven ``create_all()`` path (dev / tests) with the same
invariant so a new install ends up at the post-0003 schema without
running alembic.
Test fallout absorbed:
- `tests/conftest.py` autouse seed now produces a fully consistent
pair: user row (default_workspace_id = test-workspace-autouse) plus
the workspace itself. The PR5 backfill script's "users without
default_workspace_id" query no longer picks the fixture up. Insert
order is user → workspace → UPDATE user, walking around the chicken-
and-egg FK between `workspaces.owner_id` and `users.default_workspace_id`.
- `tests/test_backfill_workspace_id.py` adds a file-scoped autouse
fixture that temporarily flips `column.nullable = True` for the four
business tables (production correctness comes from alembic 0003;
the script's own job is exactly to fill rows between 0002 and 0003
so its tests need that transient state to be representable). Its
`_init_engine` also deletes the autouse seed rows to match the
"fresh DB" model the tests assume.
- `test_thread_meta_workspace_filter::test_create_workspace_none_bypasses`
renamed to `test_create_workspace_none_rejected_by_orm` and asserts
the new IntegrityError on explicit None — write paths can no longer
bypass workspace scope.
- 5 `test_workspace_context` tests + the auth-middleware reset test
get `@pytest.mark.no_auto_workspace` so they keep testing the
unset-contextvar path.
- `test_workspace_repo::test_list_by_user_bypass_returns_all` switches
to membership assertions instead of strict equality since the
autouse fixture surfaces under `user_id=None`.
3214 passed, 30 skipped; the remaining 17 are the documented
pre-existing caplog ordering flakes (all pass in isolation).
`app/gateway/app.py:_check_path_migration_pending` runs at lifespan
startup and emits a single WARNING log when `{base_dir}/users/` still
has content — the operator's cue to run the new migration script. The
warning is best-effort (silent on permission errors) so it can never
escalate into a gateway boot failure; reads from the legacy tree keep
working via the `user_id` branch of `Paths.thread_dir` until the
migration runs.
The root Makefile gains a `migrate-paths` target that wraps
`scripts/migrate_paths_to_workspace.py`. Two opt-in env knobs:
- `DRY_RUN=1` — pass `--dry-run` (preview only, no writes)
- `DEFAULT_WORKSPACE=…` — claim un-assigned users under this workspace id
(defaults to `legacy_workspace` to align with PR5's orphan-row bucket).
`help` documents the new target.
3 tests cover the warning's three states (legacy dir with content, dir
missing, dir empty).
`scripts/migrate_paths_to_workspace.py` walks the PR4 per-user filesystem
layout and rewrites it under the PR6 per-workspace dimension:
- `{base}/users/{uid}/threads/{tid}/` → `{base}/workspaces/{wid}/threads/{tid}/`
- `{base}/users/{uid}/memory.json` → `{base}/workspaces/{wid}/users/{uid}/memory.json`
- `{base}/users/{uid}/agents/{name}/` → `{base}/workspaces/{wid}/users/{uid}/agents/{name}/`
`{wid}` is read from `users.default_workspace_id` in the local sqlite
DB; users without one fall back to `--default-workspace`, defaulting to
`legacy_workspace` (the same bucket PR5's row backfill uses for orphan
rows so the file-system and database alignments stay consistent).
Pre-existing destinations route the legacy copy to
`{base}/migration-conflicts/workspace-migration/...` for manual review;
empty source dirs are cleaned up. The script is idempotent and supports
`--dry-run` (writes nothing, still logs the full report).
9 tests cover: thread / memory / agent rewrite, dry-run no-write, the
fallback workspace path, conflict diversion, post-migration cleanup of
empty `users/` dirs, and the missing-DB / missing-table degraded
modes.
`ThreadDataMiddleware.before_agent` now reads
`get_effective_workspace_id()` and routes the per-thread directory tree
through `Paths.sandbox_*_dir(thread_id, workspace_id=...)`, producing
`{base_dir}/workspaces/{wid}/threads/{tid}/user-data/...`. The legacy
user-id-only layout is no longer written by this middleware; the
migration script in T6.12 will lift any pre-existing `users/{uid}/...`
trees into the new shape.
In no-auth dev mode the contextvar is empty so
`get_effective_workspace_id` returns `"default"` and writes land at
`workspaces/default/...` — the layout invariant ("threads always live
inside a workspace") holds without a real auth setup. `thread_data` now
also exposes `user_id` and `workspace_id` so downstream middlewares
(sandbox, memory, etc.) can read them without re-resolving the
contextvar themselves.
4 new tests cover: contextvar workspace → expected path,
`no_auto_workspace` falls back to `default`, eager mode creates the
right dirs, and the `get_config` fallback still routes through
workspace. Existing 4 thread_data middleware tests stay green.
`Paths` learns a new top-level filesystem dimension. Precedence in
descending order:
- workspace_id given (PR6+):
`{base_dir}/workspaces/{wid}/threads/{tid}/...`
`{base_dir}/workspaces/{wid}/users/{uid}/memory.json`
`{base_dir}/workspaces/{wid}/users/{uid}/agents/{name}/...`
- user_id only (legacy user-isolation): unchanged
- neither (very legacy): unchanged
API: every thread/sandbox/host/ACP helper now takes
`workspace_id: str | None = None` alongside the existing `user_id`,
plus `Paths.workspace_dir()` and `_validate_workspace_id()` for safe
id substitution. `resolve_virtual_path` likewise routes through the
new precedence. The migration script in T6.12 will walk the old
`users/{uid}/threads/...` tree and rewrite to the new shape.
15 new tests (workspace shape, validation, sandbox dirs, ACP,
user-memory/agents under workspace, traversal defence) pass; 46
existing path tests stay green.
End-to-end coverage for the workspace boundary:
- GET /api/threads/{tid} from workspace B → 404
- DELETE /api/threads/{tid} from workspace B → 404
- PATCH /api/threads/{tid} from workspace B → 404
- Positive control: same workspace → 200
Wires `MemoryThreadMetaStore` (real impl, not a mock) behind a
stub-authed FastAPI app. The check_access call inside
`@require_permission` returns False on cross-workspace and the
decorator converts to 404 — proving the boundary holds at the HTTP
boundary, not just the unit level. Run stream/wait are skipped here
(they spin a background worker); their guard goes through the same
decorator path so unit coverage in `test_require_permission_workspace`
is sufficient.
`_StubAuthMiddleware` gains `override_user_contextvar=True` so
cross-user / cross-workspace tests can drive both contextvars from the
stub. Default stays off — the autouse user fixture continues to own
the contextvar for legacy tests whose routes resolve filesystem paths
via `get_effective_user_id()`.
86 router / boundary tests stay green.
`create_thread` in `app/gateway/routers/threads.py` already relies on
the AUTO sentinel default for ``ThreadMetaStore.create``, so once T6.1
landed the workspace_id was implicitly carried from the contextvar.
This commit adds the integration coverage: TestClient + stub auth
middleware + ``MemoryThreadMetaStore`` round-trip showing the persisted
row carries the right ``workspace_id`` under two distinct workspaces.
No production code change — the route wiring was correct, but the
guarantee was previously only proven at the unit level.
The decorator path was already updated in T6.4 to read
`get_effective_workspace_id()` and forward it as the third positional
to `check_access`. T6.6 closes the loop with dedicated coverage:
- `_StubAuthMiddleware` and `make_authed_test_app` now accept an
optional `workspace_factory` so router tests can drive the active
workspace contextvar end-to-end through the FastAPI middleware stack.
- 5 new probe tests assert: cross-workspace → 404 (not 403), same
workspace → 200, the third positional reaching `check_access`
carries the contextvar id, no-context tests (marked
`no_auto_workspace`) fall back to "default", and read-style routes
(`require_existing=False`) thread workspace_id too.
75 existing router tests (artifacts / runs / threads / uploads /
suggestions) stay green.
`ThreadMetaStore.check_access(thread_id, user_id, workspace_id, *,
require_existing)` is now a three-positional method. Cross-workspace is
denied unconditionally — even when the row exists and `user_id` matches
— so the decorator layer can convert the False into a **404** and never
leak the existence of a thread across tenants. Inside the workspace, the
existing legacy semantics still hold (NULL `row.user_id` stays
"shared in workspace", `require_existing` still gates the missing-row
path against ghost-row re-targeting).
`@require_permission(owner_check=True)` in `app/gateway/authz.py` now
reads the active workspace from `get_effective_workspace_id()` (set by
PR4 AuthMiddleware; falls back to "default" in no-auth dev) and passes
it through. The existing 404-not-403 mapping is unchanged.
Existing positional callers in `test_thread_meta_repo.py` and the
permissive mock in `test_threads_router.py` were updated for the new
arity. 58 thread_meta / router / memory tests stay green; 90 auth /
uploads / suggestions tests stay green.
Every remaining ThreadMetaStore method gains a `workspace_id` keyword
mirroring `user_id`:
- `search()` adds WHERE workspace_id and (for the memory impl) folds it
into the BaseStore filter dict.
- `update_display_name` / `update_status` / `update_metadata` / `delete`
no-op if the row lives in a different workspace. The SQL helper
`_check_ownership()` was widened to do both checks in one pass.
- `MemoryThreadMetaStore._get_owned_record()` likewise takes both ids.
5 new isolation tests prove writes from workspace B against workspace A's
thread are silently dropped (no row mutation when the caller re-reads
from workspace A). 38 existing thread_meta / owner / memory-store tests
still pass.
`get()` accepts `workspace_id: str | None | _AutoSentinel = AUTO` and
moves the workspace check into the SQL WHERE so cross-workspace lookups
short-circuit without loading the row. `MemoryThreadMetaStore._get_owned_record`
gets the same filter for parity.
The user_id check stays as a post-load comparison (preserves the existing
shared-row semantics where row.user_id IS NULL means "everyone in this
workspace"). Cross-workspace always returns None, never the row.
3 new tests cover the three states: in-workspace get returns the row,
out-of-workspace get returns None even when user_id matches, explicit
workspace_id=None bypasses (migration / CLI). Existing 22 thread_meta
tests stay green.
`create()` now accepts `workspace_id: str | None | _AutoSentinel = AUTO`
on both the SQL and in-memory implementations (and the abstract base).
AUTO resolves via `resolve_workspace_id()` from the workspace contextvar
that PR4 AuthMiddleware sets; explicit None bypasses for migration paths;
explicit str overrides the contextvar.
Test infrastructure:
- conftest gains an autouse `_auto_workspace_context` fixture mirroring
the existing user fixture. Opt-out via `@pytest.mark.no_auto_workspace`.
- A SQLAlchemy `after_create` listener on `Base.metadata` seeds the
matching `test-workspace-autouse` + `test-user-autouse` rows whenever
`init_engine` runs `create_all()`, so the FK from threads_meta to
workspaces resolves. Alembic migration tests bypass create_all and are
unaffected, keeping real FK constraints under test.
- `test_thread_meta_workspace_filter.py` covers the three AUTO / explicit
/ None paths.
3 new tests pass; 115 existing thread_meta/run/feedback/run_event/owner
tests stay green.
11 PR5 commits: alembic 0002 (nullable workspace_id + FK + composite
index on 4 business tables) -> backfill_workspace_id.py (3-step
idempotent) -> alembic 0003 (pre-flight refuse + NOT NULL + UNIQUE).
18 new tests (9 alembic 0002+0003 + 9 backfill, including dry-run and
no-users error). 3150 passed + 30 skipped + 16 pre-existing caplog
flake (unchanged from PR4).
T5.11 (ORM nullable=False) deferred to PR6 — flipping the ORM-side
NOT NULL would break 6 INSERT sites whose repository signatures get
workspace_id wiring in PR6 anyway. DB-level invariant is already
enforced by alembic 0003 along the production path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three SQLite cases + Postgres twins: happy-path upgrade after
populating workspace_id, refuse-with-NULL-leftover raises RuntimeError,
and a UNIQUE(workspace_id, thread_id) IntegrityError reproducer. The
UNIQUE test uses a scratch table because the production schema's
PRIMARY KEY on thread_id would shadow the new UNIQUE index. Index
truthiness asserted instead of bool-equality so SQLite (int 1) and
Postgres (True) both pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refuses to upgrade if any of the 4 business tables still has NULL
workspace_id rows (pre-flight count then RuntimeError pointing the
operator at scripts/backfill_workspace_id.py). Then op.batch_alter_table
flips each column to NOT NULL and adds the threads_meta
(workspace_id, thread_id) UNIQUE index that PR6 routers will rely on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Snapshots workspace / membership / users.default_workspace_id row
counts before backfill(dry_run=True), runs the orchestrator, asserts
no counts changed. Pins the dry-run report's per-step semantics:
Step 1 counts candidates without populating defaults, so Step 2's
JOIN reports 0; Step 3 picks up all NULL business rows untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_ensure_legacy_workspace creates the nil-UUID anchor (slug=legacy)
owned by the platform admin (or oldest user as fallback). Raises a
clear error if the DB has no users at all so we never silently create
an orphaned workspace. Step 3 UPDATEs each table's remaining
workspace_id IS NULL rows to LEGACY_WORKSPACE_ID. Orchestrator wires
ensure-then-loop between Step 2 and Step 3. 3 new tests: orphan
fan-out, no-users error, end-to-end orchestrator with mixed owned +
orphan rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-table correlated subquery UPDATE (portable across SQLite + Postgres).
Filters workspace_id IS NULL AND user_id IS NOT NULL so already-tagged
rows and orphan rows are skipped. Dry-run mode counts via a JOIN, never
writes. Returns rows-updated for the orchestrator report. 2 new tests:
single-user 4-table fan-out + multi-user isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For each user with NULL default_workspace_id, generate a base slug
from the email, walk past collisions/blacklist via next_available_slug,
create the workspace + owner membership, and set default_workspace_id.
Idempotent: candidates list is filtered by IS NULL, so re-running on a
populated DB is a no-op. 3 new unit tests (creation, idempotence,
blacklist-walker behaviour).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
argparse with --dry-run, async orchestrator that loops the 4 business
tables, and three step stubs (Step 1: users -> workspace creation;
Step 2: UPDATE tables from users.default_workspace_id; Step 3: orphan
rows -> legacy_workspace anchor at the LOCK'd nil UUID). Step bodies
are filled in T5.5-T5.7 alongside their tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ORM-side mirror of alembic 0002. threads_meta gets the matching
(workspace_id, user_id, updated_at) composite index so fresh
create_all() dev databases land on the same shape Alembic produces.
Pre-existing test_alembic_default_workspace_id pins to revision
0001 explicitly since 'head' now includes 0002.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors test_alembic_default_workspace_id pattern: synchronous bodies
(env.py uses asyncio.run internally), pre-PR5 schema bootstrap with the
minimal columns the migration touches, upgrade/downgrade assertions for
the workspace_id column + FK + composite index on all 4 business tables.
Postgres cases auto-skip when Docker daemon is unreachable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ALTER threads_meta / runs / feedback / run_events to add nullable
workspace_id String(36) + FK to workspaces (ON DELETE CASCADE) and a
composite (workspace_id, user_id, updated_at) index on threads_meta for
the workspace-scoped thread-list query. Downgrade reverses all four FKs,
columns, and the index.
Stage 0 PR5 step 1/2; revision 0003 flips NOT NULL after the backfill.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Records the 14-task PR4 delivery (T4.1-T4.14), final test baseline
(3136 passed / 26 skipped), key design decisions that diverged from
the plan draft (TokenPayload optional fields, ensure_default_workspace
helper shape, slug walker treating blacklist as taken), follow-up
items, and a copy-paste live smoke command list for the user to
verify make-dev end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-PR4 admins have users.default_workspace_id NULL. Without this
backfill they would log in successfully but immediately bounce off the
workspace gate (T4.7) because their token cannot encode a wid. The
lifespan hook now resolves the admin user and calls
ensure_default_workspace (idempotent — no-op if the user already has
a workspace), so the post-upgrade boot makes the admin usable.
Renamed routers.auth._ensure_default_workspace → ensure_default_workspace
to make it importable across modules.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /auth/me now responds with the new UserMeResponse: the existing
user fields plus default_workspace_id and a workspaces list that
joins WorkspaceRepository.list_by_user with the caller's role from
WorkspaceMembershipRepository.list_by_user. Stage 0 every user has
exactly one entry there, but the shape is forward-compatible for
Stage 2 multi-workspace memberships.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After PR4 every protected request demands wid. Three JWT-issue paths
must encode it; T4.8/T4.9 covered /initialize and /register, and this
commit closes the remaining two:
- login_local calls _ensure_default_workspace (idempotent — returns
the existing default_workspace_id when set) so users who pre-date
PR4 are backfilled at login time, and signs the new cookie with
wid + role='owner'.
- change_password does the same, ensuring a password change does not
strip the workspace claim and lock the user out on their next
request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors T4.8: the regular registration path now invokes the same
_ensure_default_workspace helper so every new user lands with a
1-person workspace (owner) and a JWT carrying wid + role='owner'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
initialize_admin now seeds a 1-person workspace immediately after the
admin user is created: WorkspaceRepository.create(name, slug, owner_id)
+ WorkspaceMembershipRepository.add(role='owner') + writes the new
workspace id back to users.default_workspace_id. The session JWT is
re-issued with wid + role='owner' so subsequent requests pass the
T4.7 workspace gate.
Mechanical pieces:
- SQLiteUserRepository row<->user mapping now includes
default_workspace_id (sql update_user too) so the column persists.
- workspace.sql.SLUG_BLACKLIST is now public (was _SLUG_BLACKLIST) and
the registration helper treats blacklisted slugs as "taken" so the
walker steps past reserved names like "admin" instead of crashing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
auto_slug_from_email maps an email's local-part to a schema-valid base
slug (lowercase, [+_.]→'-', alphanumeric-and-hyphen, clamp 32 chars,
fallback to user-{token_hex(4)} for pathological inputs).
next_available_slug walks the {base, base-2, base-3, ...} sequence
against a caller-supplied async exists_check until it lands on a free
slot, truncating base when the suffix would push past 32 chars.
Pulled forward of T4.8/T4.9 because both /auth/initialize and
/auth/register need it. Lives in app.gateway.auth (not persistence)
since "email → slug" is a registration-time concern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
deps.get_current_user_from_request now stashes the decoded payload on
request.state.auth_payload so AuthMiddleware can populate the
workspace_context ContextVar without a second decode. Reset is paired
in the same try/finally as user_context to keep teardown atomic.
Also adds:
- auth.models.ActiveWorkspace — minimal proxy that satisfies the
CurrentWorkspace protocol (id + role only).
- auth.models.User.default_workspace_id — surfaces the DB column added
in T4.4 so the eventual /auth/me payload can reference it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After PR4 every JWT must carry wid (workspace_id). decode_token now
returns the new TokenError.WORKSPACE_MISSING when the signature is
valid but the payload lacks wid; expired tokens still report EXPIRED
first so /auth/refresh logic stays correct. AuthErrorCode gains a
matching WORKSPACE_REQUIRED for middleware to surface to clients.
Updates 13 existing test sites that issued tokens without wid to pass
workspace_id="ws-test" + role="owner", reflecting the new contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TokenPayload gains optional wid (workspace_id) and role claims;
create_access_token accepts them as keyword-only args and only
encodes them when provided. Existing tokens and existing callers
keep working unchanged — the contract that protected requests
must carry wid is enforced by middleware (PR4 T4.6/T4.7), not by
the JWT type system.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a nullable FK column to mirror alembic 0001. On fresh deployments
metadata.create_all() will create users with this column; existing
deployments rely on the alembic migration to add it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies alembic upgrade 0001 adds users.default_workspace_id (with FK
to workspaces) and that downgrade cleanly removes it. Runs on both
dialects because the migration uses op.batch_alter_table for SQLite
ALTER compatibility.
Tests are intentionally sync — alembic's command layer is sync and
env.py calls asyncio.run(); running under pytest-anyio would deadlock
on the inner event loop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First Alembic revision for the DeerFlow application schema. Adds a
nullable users.default_workspace_id column with an ON DELETE SET NULL
FK to workspaces(id), so newly-registered users can be routed back to
their default workspace on next login without consulting memberships.
Uses op.batch_alter_table for SQLite ALTER compatibility (sqlite is
still a supported dev fallback).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
alembic heads/history both empty, versions/ dir empty: 0001 can be the
first revision with down_revision=None. alembic_version table will be
auto-created on first upgrade head. doctor.py needs no new check.
Also records T4 preparation grep findings on _ensure_admin_user current
behavior (open issue #3 resolved).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two unblocking events on 2026-05-12: team signed off all 7 irreversible
schema decisions, and docs branch was finally pushed to origin (38 commits,
via SSH-over-443 to bypass local proxy). PR4 is now unblocked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1445043649