`app/gateway/app.py:_check_path_migration_pending` runs at lifespan
startup and emits a single WARNING log when `{base_dir}/users/` still
has content — the operator's cue to run the new migration script. The
warning is best-effort (silent on permission errors) so it can never
escalate into a gateway boot failure; reads from the legacy tree keep
working via the `user_id` branch of `Paths.thread_dir` until the
migration runs.
The root Makefile gains a `migrate-paths` target that wraps
`scripts/migrate_paths_to_workspace.py`. Two opt-in env knobs:
- `DRY_RUN=1` — pass `--dry-run` (preview only, no writes)
- `DEFAULT_WORKSPACE=…` — claim un-assigned users under this workspace id
(defaults to `legacy_workspace` to align with PR5's orphan-row bucket).
`help` documents the new target.
3 tests cover the warning's three states (legacy dir with content, dir
missing, dir empty).
`scripts/migrate_paths_to_workspace.py` walks the PR4 per-user filesystem
layout and rewrites it under the PR6 per-workspace dimension:
- `{base}/users/{uid}/threads/{tid}/` → `{base}/workspaces/{wid}/threads/{tid}/`
- `{base}/users/{uid}/memory.json` → `{base}/workspaces/{wid}/users/{uid}/memory.json`
- `{base}/users/{uid}/agents/{name}/` → `{base}/workspaces/{wid}/users/{uid}/agents/{name}/`
`{wid}` is read from `users.default_workspace_id` in the local sqlite
DB; users without one fall back to `--default-workspace`, defaulting to
`legacy_workspace` (the same bucket PR5's row backfill uses for orphan
rows so the file-system and database alignments stay consistent).
Pre-existing destinations route the legacy copy to
`{base}/migration-conflicts/workspace-migration/...` for manual review;
empty source dirs are cleaned up. The script is idempotent and supports
`--dry-run` (writes nothing, still logs the full report).
9 tests cover: thread / memory / agent rewrite, dry-run no-write, the
fallback workspace path, conflict diversion, post-migration cleanup of
empty `users/` dirs, and the missing-DB / missing-table degraded
modes.
`ThreadDataMiddleware.before_agent` now reads
`get_effective_workspace_id()` and routes the per-thread directory tree
through `Paths.sandbox_*_dir(thread_id, workspace_id=...)`, producing
`{base_dir}/workspaces/{wid}/threads/{tid}/user-data/...`. The legacy
user-id-only layout is no longer written by this middleware; the
migration script in T6.12 will lift any pre-existing `users/{uid}/...`
trees into the new shape.
In no-auth dev mode the contextvar is empty so
`get_effective_workspace_id` returns `"default"` and writes land at
`workspaces/default/...` — the layout invariant ("threads always live
inside a workspace") holds without a real auth setup. `thread_data` now
also exposes `user_id` and `workspace_id` so downstream middlewares
(sandbox, memory, etc.) can read them without re-resolving the
contextvar themselves.
4 new tests cover: contextvar workspace → expected path,
`no_auto_workspace` falls back to `default`, eager mode creates the
right dirs, and the `get_config` fallback still routes through
workspace. Existing 4 thread_data middleware tests stay green.
`Paths` learns a new top-level filesystem dimension. Precedence in
descending order:
- workspace_id given (PR6+):
`{base_dir}/workspaces/{wid}/threads/{tid}/...`
`{base_dir}/workspaces/{wid}/users/{uid}/memory.json`
`{base_dir}/workspaces/{wid}/users/{uid}/agents/{name}/...`
- user_id only (legacy user-isolation): unchanged
- neither (very legacy): unchanged
API: every thread/sandbox/host/ACP helper now takes
`workspace_id: str | None = None` alongside the existing `user_id`,
plus `Paths.workspace_dir()` and `_validate_workspace_id()` for safe
id substitution. `resolve_virtual_path` likewise routes through the
new precedence. The migration script in T6.12 will walk the old
`users/{uid}/threads/...` tree and rewrite to the new shape.
15 new tests (workspace shape, validation, sandbox dirs, ACP,
user-memory/agents under workspace, traversal defence) pass; 46
existing path tests stay green.
End-to-end coverage for the workspace boundary:
- GET /api/threads/{tid} from workspace B → 404
- DELETE /api/threads/{tid} from workspace B → 404
- PATCH /api/threads/{tid} from workspace B → 404
- Positive control: same workspace → 200
Wires `MemoryThreadMetaStore` (real impl, not a mock) behind a
stub-authed FastAPI app. The check_access call inside
`@require_permission` returns False on cross-workspace and the
decorator converts to 404 — proving the boundary holds at the HTTP
boundary, not just the unit level. Run stream/wait are skipped here
(they spin a background worker); their guard goes through the same
decorator path so unit coverage in `test_require_permission_workspace`
is sufficient.
`_StubAuthMiddleware` gains `override_user_contextvar=True` so
cross-user / cross-workspace tests can drive both contextvars from the
stub. Default stays off — the autouse user fixture continues to own
the contextvar for legacy tests whose routes resolve filesystem paths
via `get_effective_user_id()`.
86 router / boundary tests stay green.
`create_thread` in `app/gateway/routers/threads.py` already relies on
the AUTO sentinel default for ``ThreadMetaStore.create``, so once T6.1
landed the workspace_id was implicitly carried from the contextvar.
This commit adds the integration coverage: TestClient + stub auth
middleware + ``MemoryThreadMetaStore`` round-trip showing the persisted
row carries the right ``workspace_id`` under two distinct workspaces.
No production code change — the route wiring was correct, but the
guarantee was previously only proven at the unit level.
The decorator path was already updated in T6.4 to read
`get_effective_workspace_id()` and forward it as the third positional
to `check_access`. T6.6 closes the loop with dedicated coverage:
- `_StubAuthMiddleware` and `make_authed_test_app` now accept an
optional `workspace_factory` so router tests can drive the active
workspace contextvar end-to-end through the FastAPI middleware stack.
- 5 new probe tests assert: cross-workspace → 404 (not 403), same
workspace → 200, the third positional reaching `check_access`
carries the contextvar id, no-context tests (marked
`no_auto_workspace`) fall back to "default", and read-style routes
(`require_existing=False`) thread workspace_id too.
75 existing router tests (artifacts / runs / threads / uploads /
suggestions) stay green.
`ThreadMetaStore.check_access(thread_id, user_id, workspace_id, *,
require_existing)` is now a three-positional method. Cross-workspace is
denied unconditionally — even when the row exists and `user_id` matches
— so the decorator layer can convert the False into a **404** and never
leak the existence of a thread across tenants. Inside the workspace, the
existing legacy semantics still hold (NULL `row.user_id` stays
"shared in workspace", `require_existing` still gates the missing-row
path against ghost-row re-targeting).
`@require_permission(owner_check=True)` in `app/gateway/authz.py` now
reads the active workspace from `get_effective_workspace_id()` (set by
PR4 AuthMiddleware; falls back to "default" in no-auth dev) and passes
it through. The existing 404-not-403 mapping is unchanged.
Existing positional callers in `test_thread_meta_repo.py` and the
permissive mock in `test_threads_router.py` were updated for the new
arity. 58 thread_meta / router / memory tests stay green; 90 auth /
uploads / suggestions tests stay green.
Every remaining ThreadMetaStore method gains a `workspace_id` keyword
mirroring `user_id`:
- `search()` adds WHERE workspace_id and (for the memory impl) folds it
into the BaseStore filter dict.
- `update_display_name` / `update_status` / `update_metadata` / `delete`
no-op if the row lives in a different workspace. The SQL helper
`_check_ownership()` was widened to do both checks in one pass.
- `MemoryThreadMetaStore._get_owned_record()` likewise takes both ids.
5 new isolation tests prove writes from workspace B against workspace A's
thread are silently dropped (no row mutation when the caller re-reads
from workspace A). 38 existing thread_meta / owner / memory-store tests
still pass.
`get()` accepts `workspace_id: str | None | _AutoSentinel = AUTO` and
moves the workspace check into the SQL WHERE so cross-workspace lookups
short-circuit without loading the row. `MemoryThreadMetaStore._get_owned_record`
gets the same filter for parity.
The user_id check stays as a post-load comparison (preserves the existing
shared-row semantics where row.user_id IS NULL means "everyone in this
workspace"). Cross-workspace always returns None, never the row.
3 new tests cover the three states: in-workspace get returns the row,
out-of-workspace get returns None even when user_id matches, explicit
workspace_id=None bypasses (migration / CLI). Existing 22 thread_meta
tests stay green.
`create()` now accepts `workspace_id: str | None | _AutoSentinel = AUTO`
on both the SQL and in-memory implementations (and the abstract base).
AUTO resolves via `resolve_workspace_id()` from the workspace contextvar
that PR4 AuthMiddleware sets; explicit None bypasses for migration paths;
explicit str overrides the contextvar.
Test infrastructure:
- conftest gains an autouse `_auto_workspace_context` fixture mirroring
the existing user fixture. Opt-out via `@pytest.mark.no_auto_workspace`.
- A SQLAlchemy `after_create` listener on `Base.metadata` seeds the
matching `test-workspace-autouse` + `test-user-autouse` rows whenever
`init_engine` runs `create_all()`, so the FK from threads_meta to
workspaces resolves. Alembic migration tests bypass create_all and are
unaffected, keeping real FK constraints under test.
- `test_thread_meta_workspace_filter.py` covers the three AUTO / explicit
/ None paths.
3 new tests pass; 115 existing thread_meta/run/feedback/run_event/owner
tests stay green.
Three SQLite cases + Postgres twins: happy-path upgrade after
populating workspace_id, refuse-with-NULL-leftover raises RuntimeError,
and a UNIQUE(workspace_id, thread_id) IntegrityError reproducer. The
UNIQUE test uses a scratch table because the production schema's
PRIMARY KEY on thread_id would shadow the new UNIQUE index. Index
truthiness asserted instead of bool-equality so SQLite (int 1) and
Postgres (True) both pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refuses to upgrade if any of the 4 business tables still has NULL
workspace_id rows (pre-flight count then RuntimeError pointing the
operator at scripts/backfill_workspace_id.py). Then op.batch_alter_table
flips each column to NOT NULL and adds the threads_meta
(workspace_id, thread_id) UNIQUE index that PR6 routers will rely on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Snapshots workspace / membership / users.default_workspace_id row
counts before backfill(dry_run=True), runs the orchestrator, asserts
no counts changed. Pins the dry-run report's per-step semantics:
Step 1 counts candidates without populating defaults, so Step 2's
JOIN reports 0; Step 3 picks up all NULL business rows untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_ensure_legacy_workspace creates the nil-UUID anchor (slug=legacy)
owned by the platform admin (or oldest user as fallback). Raises a
clear error if the DB has no users at all so we never silently create
an orphaned workspace. Step 3 UPDATEs each table's remaining
workspace_id IS NULL rows to LEGACY_WORKSPACE_ID. Orchestrator wires
ensure-then-loop between Step 2 and Step 3. 3 new tests: orphan
fan-out, no-users error, end-to-end orchestrator with mixed owned +
orphan rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-table correlated subquery UPDATE (portable across SQLite + Postgres).
Filters workspace_id IS NULL AND user_id IS NOT NULL so already-tagged
rows and orphan rows are skipped. Dry-run mode counts via a JOIN, never
writes. Returns rows-updated for the orchestrator report. 2 new tests:
single-user 4-table fan-out + multi-user isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For each user with NULL default_workspace_id, generate a base slug
from the email, walk past collisions/blacklist via next_available_slug,
create the workspace + owner membership, and set default_workspace_id.
Idempotent: candidates list is filtered by IS NULL, so re-running on a
populated DB is a no-op. 3 new unit tests (creation, idempotence,
blacklist-walker behaviour).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
argparse with --dry-run, async orchestrator that loops the 4 business
tables, and three step stubs (Step 1: users -> workspace creation;
Step 2: UPDATE tables from users.default_workspace_id; Step 3: orphan
rows -> legacy_workspace anchor at the LOCK'd nil UUID). Step bodies
are filled in T5.5-T5.7 alongside their tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ORM-side mirror of alembic 0002. threads_meta gets the matching
(workspace_id, user_id, updated_at) composite index so fresh
create_all() dev databases land on the same shape Alembic produces.
Pre-existing test_alembic_default_workspace_id pins to revision
0001 explicitly since 'head' now includes 0002.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors test_alembic_default_workspace_id pattern: synchronous bodies
(env.py uses asyncio.run internally), pre-PR5 schema bootstrap with the
minimal columns the migration touches, upgrade/downgrade assertions for
the workspace_id column + FK + composite index on all 4 business tables.
Postgres cases auto-skip when Docker daemon is unreachable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ALTER threads_meta / runs / feedback / run_events to add nullable
workspace_id String(36) + FK to workspaces (ON DELETE CASCADE) and a
composite (workspace_id, user_id, updated_at) index on threads_meta for
the workspace-scoped thread-list query. Downgrade reverses all four FKs,
columns, and the index.
Stage 0 PR5 step 1/2; revision 0003 flips NOT NULL after the backfill.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-PR4 admins have users.default_workspace_id NULL. Without this
backfill they would log in successfully but immediately bounce off the
workspace gate (T4.7) because their token cannot encode a wid. The
lifespan hook now resolves the admin user and calls
ensure_default_workspace (idempotent — no-op if the user already has
a workspace), so the post-upgrade boot makes the admin usable.
Renamed routers.auth._ensure_default_workspace → ensure_default_workspace
to make it importable across modules.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /auth/me now responds with the new UserMeResponse: the existing
user fields plus default_workspace_id and a workspaces list that
joins WorkspaceRepository.list_by_user with the caller's role from
WorkspaceMembershipRepository.list_by_user. Stage 0 every user has
exactly one entry there, but the shape is forward-compatible for
Stage 2 multi-workspace memberships.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After PR4 every protected request demands wid. Three JWT-issue paths
must encode it; T4.8/T4.9 covered /initialize and /register, and this
commit closes the remaining two:
- login_local calls _ensure_default_workspace (idempotent — returns
the existing default_workspace_id when set) so users who pre-date
PR4 are backfilled at login time, and signs the new cookie with
wid + role='owner'.
- change_password does the same, ensuring a password change does not
strip the workspace claim and lock the user out on their next
request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors T4.8: the regular registration path now invokes the same
_ensure_default_workspace helper so every new user lands with a
1-person workspace (owner) and a JWT carrying wid + role='owner'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
initialize_admin now seeds a 1-person workspace immediately after the
admin user is created: WorkspaceRepository.create(name, slug, owner_id)
+ WorkspaceMembershipRepository.add(role='owner') + writes the new
workspace id back to users.default_workspace_id. The session JWT is
re-issued with wid + role='owner' so subsequent requests pass the
T4.7 workspace gate.
Mechanical pieces:
- SQLiteUserRepository row<->user mapping now includes
default_workspace_id (sql update_user too) so the column persists.
- workspace.sql.SLUG_BLACKLIST is now public (was _SLUG_BLACKLIST) and
the registration helper treats blacklisted slugs as "taken" so the
walker steps past reserved names like "admin" instead of crashing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
auto_slug_from_email maps an email's local-part to a schema-valid base
slug (lowercase, [+_.]→'-', alphanumeric-and-hyphen, clamp 32 chars,
fallback to user-{token_hex(4)} for pathological inputs).
next_available_slug walks the {base, base-2, base-3, ...} sequence
against a caller-supplied async exists_check until it lands on a free
slot, truncating base when the suffix would push past 32 chars.
Pulled forward of T4.8/T4.9 because both /auth/initialize and
/auth/register need it. Lives in app.gateway.auth (not persistence)
since "email → slug" is a registration-time concern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
deps.get_current_user_from_request now stashes the decoded payload on
request.state.auth_payload so AuthMiddleware can populate the
workspace_context ContextVar without a second decode. Reset is paired
in the same try/finally as user_context to keep teardown atomic.
Also adds:
- auth.models.ActiveWorkspace — minimal proxy that satisfies the
CurrentWorkspace protocol (id + role only).
- auth.models.User.default_workspace_id — surfaces the DB column added
in T4.4 so the eventual /auth/me payload can reference it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After PR4 every JWT must carry wid (workspace_id). decode_token now
returns the new TokenError.WORKSPACE_MISSING when the signature is
valid but the payload lacks wid; expired tokens still report EXPIRED
first so /auth/refresh logic stays correct. AuthErrorCode gains a
matching WORKSPACE_REQUIRED for middleware to surface to clients.
Updates 13 existing test sites that issued tokens without wid to pass
workspace_id="ws-test" + role="owner", reflecting the new contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TokenPayload gains optional wid (workspace_id) and role claims;
create_access_token accepts them as keyword-only args and only
encodes them when provided. Existing tokens and existing callers
keep working unchanged — the contract that protected requests
must carry wid is enforced by middleware (PR4 T4.6/T4.7), not by
the JWT type system.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a nullable FK column to mirror alembic 0001. On fresh deployments
metadata.create_all() will create users with this column; existing
deployments rely on the alembic migration to add it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies alembic upgrade 0001 adds users.default_workspace_id (with FK
to workspaces) and that downgrade cleanly removes it. Runs on both
dialects because the migration uses op.batch_alter_table for SQLite
ALTER compatibility.
Tests are intentionally sync — alembic's command layer is sync and
env.py calls asyncio.run(); running under pytest-anyio would deadlock
on the inner event loop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First Alembic revision for the DeerFlow application schema. Adds a
nullable users.default_workspace_id column with an ON DELETE SET NULL
FK to workspaces(id), so newly-registered users can be routed back to
their default workspace on next login without consulting memberships.
Uses op.batch_alter_table for SQLite ALTER compatibility (sqlite is
still a supported dev fallback).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(task): remove max_turns parameter from task tool interface
Subagents should always use their configured max_turns value. Exposing
this parameter allowed callers to override the admin-configured limit,
which is undesirable. The value is now exclusively driven by subagent
config (per-agent overrides and global defaults in config.yaml).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(tools): introduce Runtime type alias to eliminate Pydantic serialization warning
Add deerflow/tools/types.py with:
Runtime = ToolRuntime[dict[str, Any], ThreadState]
Replace every runtime: ToolRuntime[ContextT, ThreadState] and
runtime: ToolRuntime[dict[str, Any], ThreadState] annotation in
sandbox/tools.py, present_file_tool.py, task_tool.py, view_image_tool.py,
and skill_manage_tool.py with the new Runtime alias.
The unbound ContextT TypeVar (default None) caused
PydanticSerializationUnexpectedValue warnings on every tool call because
LangChain's BaseTool._parse_input calls model_dump() on the auto-generated
args_schema while DeerFlow passes a dict as runtime context.
Binding the context to dict[str, Any] aligns Pydantic's serialization
expectations with reality and removes the noise from all run modes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(tools): extend Runtime alias to setup_agent and update_agent tools
Replace bare ToolRuntime annotations in setup_agent_tool.py and
update_agent_tool.py with the shared Runtime alias introduced in the
previous commit, and add both tools to the Pydantic serialization
warning regression test (13 cases total).
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(tools): loosen Pydantic warning filter to avoid version-specific format
Replace the brittle "field_name='context'" substring check with a looser
"context" match so the assertion stays valid if Pydantic changes its
internal warning format across versions.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(tools): simplify warning filter and clean up docstring
Remove the "context" substring condition from the Pydantic warning
filter — asserting that no PydanticSerializationUnexpectedValue fires
at all is both simpler and more comprehensive, since the test payload
contains only the tool's own args plus runtime.
Also update the module docstring to remove the version-specific warning
format example that was inconsistent with the looser filter.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
1445043649