Commit Graph

543 Commits

Author SHA1 Message Date
1445043649 c77ee0dc06 docs(readme): 中文 README 改以多租户改造为主线 + 忽略 IDE/agent 产物
- README_zh.md:全面重写,把 workspace 租户模型 / Postgres 默认后端 /
  行级隔离 / 扩展 JWT / per-workspace 路径 / Headless API schema / apps
  脚手架作为主线,新增「多租户改造」「数据库后端」「多租户架构详解」
  「apps/」章节,并补 Stage 0–4 路线图与文档入口
- .gitignore:忽略 .qoder/(Qoder repowiki 产物)与 AGENTS.md(agent 指令)
- verify_stage0.sh:e2e 校验改用 verify-stage0.com 邮箱域、先 initialize
  admin、切到 /api/v1/ auth 路径

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 23:15:48 +08:00
1445043649 7dfc9968fe feat(apps): 新增 DeerFlow 应用脚手架,并修复 store 的 database 回退
apps/:新增「基于 DeerFlow 的应用」目录,与 backend/、frontend/ 平级,
位于「app 消费 deerflow、不反向依赖」边界的正确侧。含两种集成示例:
  - examples/http-chat   —— HTTP Gateway (REST+SSE),含登录/CSRF/建线程/流式对话
  - examples/embedded-chat —— 进程内直接调 DeerFlowClient
README 说明边界规则、两种模式、鉴权流程及新建应用约定。

runtime/store:修复 make_store 缺失的 database 段回退。原先 store 工厂只读
legacy 的 checkpointer 段,导致仅配 database:postgres 时,checkpointer 走了
Postgres、但 store 仍回退 InMemoryStore(并打出误导性的「线程列表会丢失」告警,
实际线程在 threads_meta 表里、本就持久)。现对齐 checkpointer 工厂的优先级:
checkpointer 段 → database 段 → InMemoryStore;postgres 分支同样剥掉 +asyncpg
方言前缀,使一个 DATABASE_URL 同时满足 SQLAlchemy 与 LangGraph 的 psycopg store。
告警文案也修正为「跨线程 store 数据会丢失」。

tests:新增 test_store_provider.py(3 例,TDD)覆盖 database→postgres 回退、
无配置时的内存回退、以及 checkpointer 段优先级。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 22:02:44 +08:00
1445043649 84de632b19 chore(scripts): add verify_stage0.sh — systematic Stage 0 verification
Adds a single script that walks the 5 verification layers in order
and emits pass/fail counts at the end:

  static   — boundary scans + full pytest (baseline ≥ 3250 passed,
             ≤ 18 fails matching known caplog flake set) + ruff lint
  paths    — .deer-flow/users/ should be empty (or absent);
             workspaces/{wid}/threads/{tid}/... layout in place;
             exercises `make migrate-paths DRY_RUN=1`
  rds      — PG reachable; alembic at 0003; service_accounts +
             api_keys + external_users tables present;
             idx_api_keys_active partial index has
             "WHERE revoked_at IS NULL" predicate; workspace_id is
             NOT NULL on thread_meta / runs / feedback / run_events;
             UNIQUE(workspace_id, thread_id) on thread_meta
             (requires DATABASE_URL; skipped if unset)
  runtime  — curl /health on the Gateway (requires `make dev`;
             skips downstream e2e if unreachable)
  e2e      — register two users via /api/auth/register, capture each
             session's csrf_token, create one thread per user,
             cross-access GET + DELETE both return 404 (per PR6
             "404 not 403" contract), same-workspace GET returns 200

Each layer is independently runnable: `./verify_stage0.sh paths rds`.
With no args runs all five.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:54:02 +08:00
1445043649 f803f393d3 test(persistence): PR8 T8.6 — verify all 3 PR8 tables auto-created
Adds tests/test_pr8_metadata_registration.py: opens a fresh SQLite
engine via init_engine() and asserts inspect(conn).get_table_names()
contains service_accounts, api_keys, and external_users. Guards
against an ORM row class being added under deerflow/persistence/* but
accidentally left out of deerflow/persistence/models/__init__.py —
which would leave the table un-provisioned at startup and surface as
a confusing "no such table" later in Stage 1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:14:14 +08:00
1445043649 6f806ff4a4 feat(persistence): PR8 T8.5 — ExternalUserRow ORM + 2 tests
Adds deerflow/persistence/external_user/{__init__,model}.py with
ExternalUserRow:
  - id: UUID36 PK
  - workspace_id: FK workspaces ON DELETE CASCADE (redundant with SA's
    workspace_id but stored directly to speed workspace-scoped queries
    that span multiple SAs)
  - service_account_id: FK service_accounts ON DELETE CASCADE
  - external_id: String(128) — caller-supplied X-External-User-Id
  - display_name: String(128) nullable (admin UI only, not auth-relevant)
  - metadata_json: JSON nullable=False default {} — plan tier / region /
    custom tags
  - created_at / last_seen_at (UTC)
  - UniqueConstraint (service_account_id, external_id)
    name=uq_external_users_sa_external — the same external_id may be
    reused under a different SA, but is upsert-unique under a single SA

T8.5 tests:
  - test_unique_service_account_id_plus_external_id: second row with
    same (SA, external_id) raises IntegrityError
  - test_cascade_on_service_account_delete: deleting parent SA removes
    all external_users rows under it

Registered in deerflow/persistence/models/__init__.py — all three PR8
tables (service_accounts / api_keys / external_users) are now wired
into Base.metadata.create_all().

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:12:43 +08:00
1445043649 52e9999a61 feat(persistence): PR8 T8.4 — ApiKeyRow ORM + 3 tests
Adds deerflow/persistence/api_key/{__init__,model}.py with ApiKeyRow:
  - id: UUID36 PK
  - service_account_id: FK service_accounts ON DELETE CASCADE
  - key_prefix: String(16), UNIQUE — global uniqueness preserved even
    after revoke so audit logs never reference an ambiguous prefix
  - key_hash: String(128) sha-256 hex (plaintext returned only once at
    create time)
  - name / scopes / rate_limit_rpm / expires_at / last_used_at /
    revoked_at — all nullable or default-providing
  - created_at: UTC
  - Index idx_api_keys_sa (service_account_id) — list keys for an SA
  - Index idx_api_keys_active (key_prefix) WHERE revoked_at IS NULL —
    partial index, dual-dialect via sqlite_where + postgresql_where,
    shrinks the hot-path lookup index by excluding revoked keys

T8.4 tests:
  - test_unique_key_prefix_enforced: column-level UNIQUE blocks two
    rows from sharing key_prefix (active or revoked alike)
  - test_active_index_declares_both_dialect_where_clauses: schema
    introspection confirms idx_api_keys_active has both
    `dialect_options.sqlite.where` and `dialect_options.postgresql.where`
    set to `revoked_at IS NULL` — guards against accidental loss of the
    dual-driver hint when the Index is edited later
  - test_cascade_on_service_account_delete: deleting the parent SA
    removes all api_keys rows

Registered in deerflow/persistence/models/__init__.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:10:32 +08:00
1445043649 bb7289781e test(persistence): PR8 T8.2 + T8.3 — ServiceAccount cascade / restrict
T8.2 test_cascade_on_workspace_delete: seed user → workspace → SA,
delete workspace, assert SA row is gone (ondelete CASCADE on
workspace_id FK).

T8.3 test_restrict_on_created_by_user_delete: seed user → workspace
→ SA, then attempt DELETE FROM users WHERE id = created_by, assert
IntegrityError raises and the SA row survives (ondelete RESTRICT on
created_by FK).

SQLite enforces FKs because the engine's connect-listener turns on
`PRAGMA foreign_keys = ON` for every new connection — see engine.py
init_engine().

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:07:16 +08:00
1445043649 1fb07e48e6 feat(persistence): PR8 T8.1 — ServiceAccountRow ORM + insert smoke
Adds deerflow/persistence/service_account/{__init__,model}.py with
ServiceAccountRow:
  - id: UUID36 PK
  - workspace_id: FK workspaces ON DELETE CASCADE
  - name: String(64)
  - role: String(16) default "member" (Stage 0 lone value; Stage 2 RBAC)
  - identity_mode: String(16) default "collapsed" — three states
    "collapsed" / "external_passthrough" / "both"; Stage 1 API key
    auth layer branches on this to decide whether each call writes
    an external_users row
  - status: String(16) default "active"
  - created_by: FK users ON DELETE RESTRICT (must hand off / delete
    SAs before removing their creator)
  - created_at / updated_at (UTC, onupdate)
  - Index idx_service_accounts_workspace (workspace_id, status)

Registers in deerflow/persistence/models/__init__.py so the engine's
side-effect import path picks it up for Base.metadata.create_all().

T8.1 test: insert_smoke writes a row through a session, reads it back,
asserts each column round-trips. SQLite ephemeral DB per test via
tmp_path, no Postgres required at this layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:06:21 +08:00
1445043649 d8b13afc59 docs(backend): PR7 T7.5 — document workspace boundary scan in CLAUDE.md
Extends the existing "Boundary check" section with two new entries:

- tests/test_workspace_boundary.py — AST static scan for direct imports of
  langgraph.checkpoint.* (and the third-party postgres/sqlite packages)
  outside tests/boundary_allowlist.toml. Notes the TYPE_CHECKING exemption
  and points readers to `app.gateway.deps.get_checkpointer` as the
  intended path; documents the "append to allowlist in the same PR"
  contract for new legitimate importers.
- tests/test_workspace_boundary_self.py — self-tests guarding the scanner
  from silent-empty regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:54:31 +08:00
1445043649 d0f1877070 test(boundary): PR7 T7.3 — self-tests for the scanner
Adds backend/tests/test_workspace_boundary_self.py (9 cases) to guard
against silent-empty regressions in the boundary scanner: every claim
the scanner makes is reproduced against a synthetic .py snippet written
to tmp_path and fed to `collect_runtime_checkpoint_imports`.

Coverage:
- direct `from langgraph.checkpoint.* import X` -> flagged
- bare `import langgraph.checkpoint.memory` -> flagged
- third-party `langgraph_checkpoint_postgres` / `..._sqlite` -> flagged
- `if TYPE_CHECKING:` block -> skipped (Name form)
- `if typing.TYPE_CHECKING:` block -> skipped (Attribute form)
- nested-under-`if True:` TYPE_CHECKING block -> skipped (parent walk)
- unrelated imports (`os`, `langgraph.graph.state`, `app.gateway.deps`) -> ignored
- string literal containing the target module path -> ignored (not an import node)
- syntax-broken file -> returns [] (parser exception swallowed)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:52:36 +08:00
1445043649 ba30d14041 feat(tests): PR7 T7.2 — AST boundary scanner for langgraph.checkpoint
Adds backend/tests/test_workspace_boundary.py: walks every .py under
backend/ (excluding tests/, docs/, build artefacts), parses with ast,
flags any ImportFrom/Import that targets langgraph.checkpoint.*,
langgraph_checkpoint_postgres or langgraph_checkpoint_sqlite outside the
allowlist loaded from tests/boundary_allowlist.toml.

Skip rules:
- Imports inside `if TYPE_CHECKING:` (or `if typing.TYPE_CHECKING:`)
  blocks are exempt: they do not enter runtime so cannot bypass the
  boundary. Verified against deerflow/agents/factory.py which only uses
  BaseCheckpointSaver as a parameter annotation.

The scanner walks parent pointers up from each import node to detect
the enclosing TYPE_CHECKING guard, rather than checking only direct
parents — handles nested guards correctly.

RED proof (empty allowlist): 14 violations across the 4 ground-truth
runtime importers (threads / async_provider / provider / worker).
GREEN (with this PR's allowlist): single test passes in ~2s.

The collector function `collect_runtime_checkpoint_imports` and
`scan_violations` are kept at module scope so T7.3's self-test can
exercise them on a synthetic temp file in the next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:50:24 +08:00
1445043649 1a6ccc9aaa feat(tests): PR7 T7.1 — boundary scan allowlist toml
Adds backend/tests/boundary_allowlist.toml that names the four current
legitimate runtime importers of langgraph.checkpoint.*:
  - app/gateway/routers/threads.py (empty_checkpoint for thread init)
  - packages/harness/deerflow/runtime/checkpointer/async_provider.py
  - packages/harness/deerflow/runtime/checkpointer/provider.py
  - packages/harness/deerflow/runtime/runs/worker.py (empty_checkpoint)

The allowlist replaces plan's draft (thread_runs.py / gateway/app.py)
with the ground-truth grep: both of those reach the checkpointer through
`app.gateway.deps.get_checkpointer` DI, so they need not be listed.
factory.py imports BaseCheckpointSaver only under `if TYPE_CHECKING:` —
the scanner (next commit) exempts type-only imports automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:50:12 +08:00
1445043649 87ea715c2a feat(persistence): PR6 T5.11 — ORM workspace_id nullable=False
Flip the 4 business ORM models (ThreadMetaRow, RunRow, FeedbackRow,
RunEventRow) to ``workspace_id: Mapped[str]`` with ``nullable=False``.
PR5's alembic 0003 already enforces NOT NULL at the DB layer; this
aligns the ORM-driven ``create_all()`` path (dev / tests) with the same
invariant so a new install ends up at the post-0003 schema without
running alembic.

Test fallout absorbed:

- `tests/conftest.py` autouse seed now produces a fully consistent
  pair: user row (default_workspace_id = test-workspace-autouse) plus
  the workspace itself. The PR5 backfill script's "users without
  default_workspace_id" query no longer picks the fixture up. Insert
  order is user → workspace → UPDATE user, walking around the chicken-
  and-egg FK between `workspaces.owner_id` and `users.default_workspace_id`.
- `tests/test_backfill_workspace_id.py` adds a file-scoped autouse
  fixture that temporarily flips `column.nullable = True` for the four
  business tables (production correctness comes from alembic 0003;
  the script's own job is exactly to fill rows between 0002 and 0003
  so its tests need that transient state to be representable). Its
  `_init_engine` also deletes the autouse seed rows to match the
  "fresh DB" model the tests assume.
- `test_thread_meta_workspace_filter::test_create_workspace_none_bypasses`
  renamed to `test_create_workspace_none_rejected_by_orm` and asserts
  the new IntegrityError on explicit None — write paths can no longer
  bypass workspace scope.
- 5 `test_workspace_context` tests + the auth-middleware reset test
  get `@pytest.mark.no_auto_workspace` so they keep testing the
  unset-contextvar path.
- `test_workspace_repo::test_list_by_user_bypass_returns_all` switches
  to membership assertions instead of strict equality since the
  autouse fixture surfaces under `user_id=None`.

3214 passed, 30 skipped; the remaining 17 are the documented
pre-existing caplog ordering flakes (all pass in isolation).
2026-05-13 18:06:53 +08:00
1445043649 c5c66ccbfc feat(gateway/make): PR6 T6.14 — migrate-paths target + lifespan warning
`app/gateway/app.py:_check_path_migration_pending` runs at lifespan
startup and emits a single WARNING log when `{base_dir}/users/` still
has content — the operator's cue to run the new migration script. The
warning is best-effort (silent on permission errors) so it can never
escalate into a gateway boot failure; reads from the legacy tree keep
working via the `user_id` branch of `Paths.thread_dir` until the
migration runs.

The root Makefile gains a `migrate-paths` target that wraps
`scripts/migrate_paths_to_workspace.py`. Two opt-in env knobs:
- `DRY_RUN=1`           — pass `--dry-run` (preview only, no writes)
- `DEFAULT_WORKSPACE=…`  — claim un-assigned users under this workspace id
   (defaults to `legacy_workspace` to align with PR5's orphan-row bucket).
`help` documents the new target.

3 tests cover the warning's three states (legacy dir with content, dir
missing, dir empty).
2026-05-13 17:50:47 +08:00
1445043649 56f2c8d873 feat(scripts): PR6 T6.12 + T6.13 — migrate_paths_to_workspace.py + tests
`scripts/migrate_paths_to_workspace.py` walks the PR4 per-user filesystem
layout and rewrites it under the PR6 per-workspace dimension:

- `{base}/users/{uid}/threads/{tid}/` → `{base}/workspaces/{wid}/threads/{tid}/`
- `{base}/users/{uid}/memory.json`    → `{base}/workspaces/{wid}/users/{uid}/memory.json`
- `{base}/users/{uid}/agents/{name}/`  → `{base}/workspaces/{wid}/users/{uid}/agents/{name}/`

`{wid}` is read from `users.default_workspace_id` in the local sqlite
DB; users without one fall back to `--default-workspace`, defaulting to
`legacy_workspace` (the same bucket PR5's row backfill uses for orphan
rows so the file-system and database alignments stay consistent).

Pre-existing destinations route the legacy copy to
`{base}/migration-conflicts/workspace-migration/...` for manual review;
empty source dirs are cleaned up. The script is idempotent and supports
`--dry-run` (writes nothing, still logs the full report).

9 tests cover: thread / memory / agent rewrite, dry-run no-write, the
fallback workspace path, conflict diversion, post-migration cleanup of
empty `users/` dirs, and the missing-DB / missing-table degraded
modes.
2026-05-13 17:48:49 +08:00
1445043649 2d3b546bf9 feat(agents): PR6 T6.11 — ThreadDataMiddleware switches to workspace layout
`ThreadDataMiddleware.before_agent` now reads
`get_effective_workspace_id()` and routes the per-thread directory tree
through `Paths.sandbox_*_dir(thread_id, workspace_id=...)`, producing
`{base_dir}/workspaces/{wid}/threads/{tid}/user-data/...`. The legacy
user-id-only layout is no longer written by this middleware; the
migration script in T6.12 will lift any pre-existing `users/{uid}/...`
trees into the new shape.

In no-auth dev mode the contextvar is empty so
`get_effective_workspace_id` returns `"default"` and writes land at
`workspaces/default/...` — the layout invariant ("threads always live
inside a workspace") holds without a real auth setup. `thread_data` now
also exposes `user_id` and `workspace_id` so downstream middlewares
(sandbox, memory, etc.) can read them without re-resolving the
contextvar themselves.

4 new tests cover: contextvar workspace → expected path,
`no_auto_workspace` falls back to `default`, eager mode creates the
right dirs, and the `get_config` fallback still routes through
workspace. Existing 4 thread_data middleware tests stay green.
2026-05-13 17:45:06 +08:00
1445043649 f013fc1a65 feat(paths): PR6 T6.9 + T6.10 — workspace-scoped Paths
`Paths` learns a new top-level filesystem dimension. Precedence in
descending order:

- workspace_id given (PR6+):
  `{base_dir}/workspaces/{wid}/threads/{tid}/...`
  `{base_dir}/workspaces/{wid}/users/{uid}/memory.json`
  `{base_dir}/workspaces/{wid}/users/{uid}/agents/{name}/...`
- user_id only (legacy user-isolation): unchanged
- neither (very legacy): unchanged

API: every thread/sandbox/host/ACP helper now takes
`workspace_id: str | None = None` alongside the existing `user_id`,
plus `Paths.workspace_dir()` and `_validate_workspace_id()` for safe
id substitution. `resolve_virtual_path` likewise routes through the
new precedence. The migration script in T6.12 will walk the old
`users/{uid}/threads/...` tree and rewrite to the new shape.

15 new tests (workspace shape, validation, sandbox dirs, ACP,
user-memory/agents under workspace, traversal defence) pass; 46
existing path tests stay green.
2026-05-13 17:43:51 +08:00
1445043649 a7ecd76e0a test(boundary): PR6 T6.8 — cross-workspace isolation e2e (4 cases)
End-to-end coverage for the workspace boundary:

- GET /api/threads/{tid}    from workspace B → 404
- DELETE /api/threads/{tid} from workspace B → 404
- PATCH /api/threads/{tid}  from workspace B → 404
- Positive control: same workspace → 200

Wires `MemoryThreadMetaStore` (real impl, not a mock) behind a
stub-authed FastAPI app. The check_access call inside
`@require_permission` returns False on cross-workspace and the
decorator converts to 404 — proving the boundary holds at the HTTP
boundary, not just the unit level. Run stream/wait are skipped here
(they spin a background worker); their guard goes through the same
decorator path so unit coverage in `test_require_permission_workspace`
is sufficient.

`_StubAuthMiddleware` gains `override_user_contextvar=True` so
cross-user / cross-workspace tests can drive both contextvars from the
stub. Default stays off — the autouse user fixture continues to own
the contextvar for legacy tests whose routes resolve filesystem paths
via `get_effective_user_id()`.

86 router / boundary tests stay green.
2026-05-13 17:40:56 +08:00
1445043649 f6a922921b test(routers): PR6 T6.7 — POST /api/threads workspace_id integration
`create_thread` in `app/gateway/routers/threads.py` already relies on
the AUTO sentinel default for ``ThreadMetaStore.create``, so once T6.1
landed the workspace_id was implicitly carried from the contextvar.
This commit adds the integration coverage: TestClient + stub auth
middleware + ``MemoryThreadMetaStore`` round-trip showing the persisted
row carries the right ``workspace_id`` under two distinct workspaces.

No production code change — the route wiring was correct, but the
guarantee was previously only proven at the unit level.
2026-05-13 17:35:27 +08:00
1445043649 0456606dc1 test(authz): PR6 T6.6 — @require_permission workspace_id propagation tests
The decorator path was already updated in T6.4 to read
`get_effective_workspace_id()` and forward it as the third positional
to `check_access`. T6.6 closes the loop with dedicated coverage:

- `_StubAuthMiddleware` and `make_authed_test_app` now accept an
  optional `workspace_factory` so router tests can drive the active
  workspace contextvar end-to-end through the FastAPI middleware stack.
- 5 new probe tests assert: cross-workspace → 404 (not 403), same
  workspace → 200, the third positional reaching `check_access`
  carries the contextvar id, no-context tests (marked
  `no_auto_workspace`) fall back to "default", and read-style routes
  (`require_existing=False`) thread workspace_id too.

75 existing router tests (artifacts / runs / threads / uploads /
suggestions) stay green.
2026-05-13 17:33:02 +08:00
1445043649 b4fa3bf12a feat(persistence): PR6 T6.5 — Run/Feedback/RunEvent repos workspace_id
`RunRepository` (put / get / list_by_thread / delete), `FeedbackRepository`
(create / get / list_by_run / list_by_thread / list_by_thread_grouped /
upsert / delete / delete_by_run), and `DbRunEventStore` (put / put_batch /
list_messages / list_events / list_messages_by_run / count_messages /
delete_by_thread / delete_by_run) all accept
`workspace_id: str | None | _AutoSentinel = AUTO`.

- Write paths stamp `workspace_id` from the contextvar (same shape as
  the existing `user_id` stamping). For event writes the soft-read
  `_workspace_id_from_context()` mirrors `_user_id_from_context()` so
  background worker writes without a contextvar leave the column NULL —
  consistent with PR5's nullable-during-backfill stance.
- Read paths get an extra `WHERE workspace_id = :wid` clause when the
  resolved value is not None.

7 new tests (3 RunRepo + 2 Feedback + 2 RunEvent) prove cross-workspace
reads see zero rows. 92 existing run / feedback / event tests stay green.
2026-05-13 17:29:03 +08:00
1445043649 05be7f9ad0 feat(persistence/authz): PR6 T6.4 — check_access takes workspace_id
`ThreadMetaStore.check_access(thread_id, user_id, workspace_id, *,
require_existing)` is now a three-positional method. Cross-workspace is
denied unconditionally — even when the row exists and `user_id` matches
— so the decorator layer can convert the False into a **404** and never
leak the existence of a thread across tenants. Inside the workspace, the
existing legacy semantics still hold (NULL `row.user_id` stays
"shared in workspace", `require_existing` still gates the missing-row
path against ghost-row re-targeting).

`@require_permission(owner_check=True)` in `app/gateway/authz.py` now
reads the active workspace from `get_effective_workspace_id()` (set by
PR4 AuthMiddleware; falls back to "default" in no-auth dev) and passes
it through. The existing 404-not-403 mapping is unchanged.

Existing positional callers in `test_thread_meta_repo.py` and the
permissive mock in `test_threads_router.py` were updated for the new
arity. 58 thread_meta / router / memory tests stay green; 90 auth /
uploads / suggestions tests stay green.
2026-05-13 17:23:38 +08:00
1445043649 28ad6c2b0b feat(persistence): PR6 T6.3 — search/update_*/delete workspace_id sentinel
Every remaining ThreadMetaStore method gains a `workspace_id` keyword
mirroring `user_id`:

- `search()` adds WHERE workspace_id and (for the memory impl) folds it
  into the BaseStore filter dict.
- `update_display_name` / `update_status` / `update_metadata` / `delete`
  no-op if the row lives in a different workspace. The SQL helper
  `_check_ownership()` was widened to do both checks in one pass.
- `MemoryThreadMetaStore._get_owned_record()` likewise takes both ids.

5 new isolation tests prove writes from workspace B against workspace A's
thread are silently dropped (no row mutation when the caller re-reads
from workspace A). 38 existing thread_meta / owner / memory-store tests
still pass.
2026-05-13 17:20:40 +08:00
1445043649 296a4f1950 feat(persistence): PR6 T6.2 — ThreadMetaRepository.get filters by workspace_id
`get()` accepts `workspace_id: str | None | _AutoSentinel = AUTO` and
moves the workspace check into the SQL WHERE so cross-workspace lookups
short-circuit without loading the row. `MemoryThreadMetaStore._get_owned_record`
gets the same filter for parity.

The user_id check stays as a post-load comparison (preserves the existing
shared-row semantics where row.user_id IS NULL means "everyone in this
workspace"). Cross-workspace always returns None, never the row.

3 new tests cover the three states: in-workspace get returns the row,
out-of-workspace get returns None even when user_id matches, explicit
workspace_id=None bypasses (migration / CLI). Existing 22 thread_meta
tests stay green.
2026-05-13 17:19:00 +08:00
1445043649 361e653d37 feat(persistence): PR6 T6.1 — ThreadMetaRepository.create workspace_id sentinel
`create()` now accepts `workspace_id: str | None | _AutoSentinel = AUTO`
on both the SQL and in-memory implementations (and the abstract base).
AUTO resolves via `resolve_workspace_id()` from the workspace contextvar
that PR4 AuthMiddleware sets; explicit None bypasses for migration paths;
explicit str overrides the contextvar.

Test infrastructure:
- conftest gains an autouse `_auto_workspace_context` fixture mirroring
  the existing user fixture. Opt-out via `@pytest.mark.no_auto_workspace`.
- A SQLAlchemy `after_create` listener on `Base.metadata` seeds the
  matching `test-workspace-autouse` + `test-user-autouse` rows whenever
  `init_engine` runs `create_all()`, so the FK from threads_meta to
  workspaces resolves. Alembic migration tests bypass create_all and are
  unaffected, keeping real FK constraints under test.
- `test_thread_meta_workspace_filter.py` covers the three AUTO / explicit
  / None paths.

3 new tests pass; 115 existing thread_meta/run/feedback/run_event/owner
tests stay green.
2026-05-13 17:17:56 +08:00
1445043649 30f2bd0084 test(persistence): alembic 0003 — NOT NULL + UNIQUE coverage
Three SQLite cases + Postgres twins: happy-path upgrade after
populating workspace_id, refuse-with-NULL-leftover raises RuntimeError,
and a UNIQUE(workspace_id, thread_id) IntegrityError reproducer. The
UNIQUE test uses a scratch table because the production schema's
PRIMARY KEY on thread_id would shadow the new UNIQUE index. Index
truthiness asserted instead of bool-equality so SQLite (int 1) and
Postgres (True) both pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:16:24 +08:00
1445043649 73d0b7017b feat(persistence): alembic 0003 — workspace_id NOT NULL + UNIQUE(wid, tid)
Refuses to upgrade if any of the 4 business tables still has NULL
workspace_id rows (pre-flight count then RuntimeError pointing the
operator at scripts/backfill_workspace_id.py). Then op.batch_alter_table
flips each column to NOT NULL and adds the threads_meta
(workspace_id, thread_id) UNIQUE index that PR6 routers will rely on.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:15:07 +08:00
1445043649 8a03abac75 test(persistence): backfill --dry-run end-to-end no-write proof
Snapshots workspace / membership / users.default_workspace_id row
counts before backfill(dry_run=True), runs the orchestrator, asserts
no counts changed. Pins the dry-run report's per-step semantics:
Step 1 counts candidates without populating defaults, so Step 2's
JOIN reports 0; Step 3 picks up all NULL business rows untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:13:16 +08:00
1445043649 def45dd0c6 feat(persistence): backfill Step 3 — orphan rows -> legacy_workspace
_ensure_legacy_workspace creates the nil-UUID anchor (slug=legacy)
owned by the platform admin (or oldest user as fallback). Raises a
clear error if the DB has no users at all so we never silently create
an orphaned workspace. Step 3 UPDATEs each table's remaining
workspace_id IS NULL rows to LEGACY_WORKSPACE_ID. Orchestrator wires
ensure-then-loop between Step 2 and Step 3. 3 new tests: orphan
fan-out, no-users error, end-to-end orchestrator with mixed owned +
orphan rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:12:26 +08:00
1445043649 56f6572086 feat(persistence): backfill Step 2 — UPDATE 4 tables from users.default_workspace_id
Per-table correlated subquery UPDATE (portable across SQLite + Postgres).
Filters workspace_id IS NULL AND user_id IS NOT NULL so already-tagged
rows and orphan rows are skipped. Dry-run mode counts via a JOIN, never
writes. Returns rows-updated for the orchestrator report. 2 new tests:
single-user 4-table fan-out + multi-user isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:10:49 +08:00
1445043649 e6bb220979 feat(persistence): backfill Step 1 — workspace per user without default
For each user with NULL default_workspace_id, generate a base slug
from the email, walk past collisions/blacklist via next_available_slug,
create the workspace + owner membership, and set default_workspace_id.
Idempotent: candidates list is filtered by IS NULL, so re-running on a
populated DB is a no-op. 3 new unit tests (creation, idempotence,
blacklist-walker behaviour).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:09:16 +08:00
1445043649 ad322543d0 feat(persistence): backfill_workspace_id.py — script skeleton
argparse with --dry-run, async orchestrator that loops the 4 business
tables, and three step stubs (Step 1: users -> workspace creation;
Step 2: UPDATE tables from users.default_workspace_id; Step 3: orphan
rows -> legacy_workspace anchor at the LOCK'd nil UUID). Step bodies
are filled in T5.5-T5.7 alongside their tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:07:44 +08:00
1445043649 d3361dba59 feat(persistence): 4 business ORM models — nullable workspace_id
ORM-side mirror of alembic 0002. threads_meta gets the matching
(workspace_id, user_id, updated_at) composite index so fresh
create_all() dev databases land on the same shape Alembic produces.
Pre-existing test_alembic_default_workspace_id pins to revision
0001 explicitly since 'head' now includes 0002.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:06:08 +08:00
1445043649 4e26ec9884 test(persistence): alembic 0002 round-trip on SQLite + Postgres
Mirrors test_alembic_default_workspace_id pattern: synchronous bodies
(env.py uses asyncio.run internally), pre-PR5 schema bootstrap with the
minimal columns the migration touches, upgrade/downgrade assertions for
the workspace_id column + FK + composite index on all 4 business tables.
Postgres cases auto-skip when Docker daemon is unreachable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:04:21 +08:00
1445043649 a732697855 feat(persistence): alembic 0002 — business tables nullable workspace_id
ALTER threads_meta / runs / feedback / run_events to add nullable
workspace_id String(36) + FK to workspaces (ON DELETE CASCADE) and a
composite (workspace_id, user_id, updated_at) index on threads_meta for
the workspace-scoped thread-list query. Downgrade reverses all four FKs,
columns, and the index.

Stage 0 PR5 step 1/2; revision 0003 flips NOT NULL after the backfill.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:04:13 +08:00
1445043649 5c7753c0b8 feat(auth): lifespan _ensure_admin_user backfills missing workspace
Pre-PR4 admins have users.default_workspace_id NULL. Without this
backfill they would log in successfully but immediately bounce off the
workspace gate (T4.7) because their token cannot encode a wid. The
lifespan hook now resolves the admin user and calls
ensure_default_workspace (idempotent — no-op if the user already has
a workspace), so the post-upgrade boot makes the admin usable.

Renamed routers.auth._ensure_default_workspace → ensure_default_workspace
to make it importable across modules.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:39:09 +08:00
1445043649 91846a201e feat(auth): /auth/me returns workspaces[] with id/name/slug/role
GET /auth/me now responds with the new UserMeResponse: the existing
user fields plus default_workspace_id and a workspaces list that
joins WorkspaceRepository.list_by_user with the caller's role from
WorkspaceMembershipRepository.list_by_user. Stage 0 every user has
exactly one entry there, but the shape is forward-compatible for
Stage 2 multi-workspace memberships.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:37:29 +08:00
1445043649 634e5119e1 feat(auth): login + change_password re-issue JWTs with wid + role
After PR4 every protected request demands wid. Three JWT-issue paths
must encode it; T4.8/T4.9 covered /initialize and /register, and this
commit closes the remaining two:

- login_local calls _ensure_default_workspace (idempotent — returns
  the existing default_workspace_id when set) so users who pre-date
  PR4 are backfilled at login time, and signs the new cookie with
  wid + role='owner'.
- change_password does the same, ensuring a password change does not
  strip the workspace claim and lock the user out on their next
  request.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:35:06 +08:00
1445043649 84701730da feat(auth): /register auto-creates default workspace + owner membership
Mirrors T4.8: the regular registration path now invokes the same
_ensure_default_workspace helper so every new user lands with a
1-person workspace (owner) and a JWT carrying wid + role='owner'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:32:39 +08:00
1445043649 a657d17995 feat(auth): /initialize auto-creates default workspace + owner membership
initialize_admin now seeds a 1-person workspace immediately after the
admin user is created: WorkspaceRepository.create(name, slug, owner_id)
+ WorkspaceMembershipRepository.add(role='owner') + writes the new
workspace id back to users.default_workspace_id. The session JWT is
re-issued with wid + role='owner' so subsequent requests pass the
T4.7 workspace gate.

Mechanical pieces:
- SQLiteUserRepository row<->user mapping now includes
  default_workspace_id (sql update_user too) so the column persists.
- workspace.sql.SLUG_BLACKLIST is now public (was _SLUG_BLACKLIST) and
  the registration helper treats blacklisted slugs as "taken" so the
  walker steps past reserved names like "admin" instead of crashing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:31:53 +08:00
1445043649 a06e88d58b feat(auth): workspace slug helpers — auto_slug_from_email + next_available_slug
auto_slug_from_email maps an email's local-part to a schema-valid base
slug (lowercase, [+_.]→'-', alphanumeric-and-hyphen, clamp 32 chars,
fallback to user-{token_hex(4)} for pathological inputs).
next_available_slug walks the {base, base-2, base-3, ...} sequence
against a caller-supplied async exists_check until it lands on a free
slot, truncating base when the suffix would push past 32 chars.

Pulled forward of T4.8/T4.9 because both /auth/initialize and
/auth/register need it. Lives in app.gateway.auth (not persistence)
since "email → slug" is a registration-time concern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:27:42 +08:00
1445043649 2145d36744 feat(auth): AuthMiddleware injects workspace ContextVar from JWT wid/role
deps.get_current_user_from_request now stashes the decoded payload on
request.state.auth_payload so AuthMiddleware can populate the
workspace_context ContextVar without a second decode. Reset is paired
in the same try/finally as user_context to keep teardown atomic.

Also adds:
- auth.models.ActiveWorkspace — minimal proxy that satisfies the
  CurrentWorkspace protocol (id + role only).
- auth.models.User.default_workspace_id — surfaces the DB column added
  in T4.4 so the eventual /auth/me payload can reference it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:25:43 +08:00
1445043649 b4bef65079 feat(auth): decode_token rejects legacy 4-field JWTs as WORKSPACE_MISSING
After PR4 every JWT must carry wid (workspace_id). decode_token now
returns the new TokenError.WORKSPACE_MISSING when the signature is
valid but the payload lacks wid; expired tokens still report EXPIRED
first so /auth/refresh logic stays correct. AuthErrorCode gains a
matching WORKSPACE_REQUIRED for middleware to surface to clients.

Updates 13 existing test sites that issued tokens without wid to pass
workspace_id="ws-test" + role="owner", reflecting the new contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:22:10 +08:00
1445043649 54cb94c30f feat(auth): JWT TokenPayload accepts wid + role
TokenPayload gains optional wid (workspace_id) and role claims;
create_access_token accepts them as keyword-only args and only
encodes them when provided. Existing tokens and existing callers
keep working unchanged — the contract that protected requests
must carry wid is enforced by middleware (PR4 T4.6/T4.7), not by
the JWT type system.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:17:55 +08:00
1445043649 54762f491c feat(persistence): UserRow.default_workspace_id
Adds a nullable FK column to mirror alembic 0001. On fresh deployments
metadata.create_all() will create users with this column; existing
deployments rely on the alembic migration to add it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:15:05 +08:00
1445043649 917d8fbeaf test(persistence): alembic 0001 round-trip on SQLite + Postgres
Verifies alembic upgrade 0001 adds users.default_workspace_id (with FK
to workspaces) and that downgrade cleanly removes it. Runs on both
dialects because the migration uses op.batch_alter_table for SQLite
ALTER compatibility.

Tests are intentionally sync — alembic's command layer is sync and
env.py calls asyncio.run(); running under pytest-anyio would deadlock
on the inner event loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:11:11 +08:00
1445043649 8efbb2f9e5 feat(persistence): alembic 0001 — users.default_workspace_id
First Alembic revision for the DeerFlow application schema. Adds a
nullable users.default_workspace_id column with an ON DELETE SET NULL
FK to workspaces(id), so newly-registered users can be routed back to
their default workspace on next login without consulting memberships.

Uses op.batch_alter_table for SQLite ALTER compatibility (sqlite is
still a supported dev fallback).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:08:44 +08:00
1445043649 3313047ff1 test(workspace): partial unique on owner — Postgres twin test
新建 test_workspace_partial_unique_postgres.py,2 个 @pytest.mark.postgres
test,用 PR1 的 testcontainers postgres_url fixture 验:
  - test_partial_unique_on_owner_enforced_on_postgres:第 2 个 owner
    IntegrityError(SQLite twin 在 test_workspace_membership_repo 已覆盖)
  - test_multiple_admins_allowed_on_postgres:多个 admin/member 不触发约束

为什么 PG 单独写:sqlite_where vs postgresql_where 是两份 DDL;SQLAlchemy
能 emit 不代表 PG 真的执行。本测试 pin 两边语义一致。

本地无 docker daemon → SKIPPED;CI workflow backend-postgres-tests 会实跑。

Stage 0 PR3 T3.8。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:15:25 +08:00
1445043649 36ffe2713f feat(persistence): WorkspaceMembershipRepository + 8 unit tests
新建 backend/packages/harness/deerflow/persistence/workspace_membership/sql.py,方法:
  - add(workspace_id, user_id, role, invited_by=None) 校验 role ∈
    {owner, admin, member}(Stage 0 仅写 owner,schema 已支持其余)
  - remove(workspace_id, user_id) → bool(rowcount > 0)
  - list_by_user(user_id) 按 joined_at desc
  - list_by_workspace(workspace_id) 按 joined_at asc
  - get_role(workspace_id, user_id) → str | None
  - change_role(workspace_id, user_id, new_role) → bool

MembershipValidationError 自定义异常(role 非法)。

8 test 覆盖:
  - add → get_role round-trip
  - remove 命中/未命中返回 True/False
  - 同 workspace 第 2 个 owner 触发 IntegrityError(partial unique)
  - 同 workspace 多个 admin 不触发(Stage 2 forward compat)
  - CASCADE: 删 user 自动清理 memberships
  - list_by_user 按 joined_at desc 排序(多 workspace)
  - role 校验拒绝 'viewer'
  - change_role 命中改值 + 未命中返 False

注:owner 转让需要事务内两行原子 swap(先把现 owner 改 admin,再把新 owner
插 owner),放 PR4+ auth router 内做带权限检查的版本;本仓储不暴露
transfer_ownership 方法以保持单一职责。

Stage 0 PR3 T3.7。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:13:52 +08:00
1445043649 a40df03521 feat(persistence): WorkspaceRepository + 23 unit tests
新建 backend/packages/harness/deerflow/persistence/workspace/sql.py,方法:
  - create(name, slug, owner_id, *, workspace_id=None, status='active')
    UUID v4 自动生成;强校验 slug 格式(regex ^[a-z0-9](-?[a-z0-9])*\$ + 3-32
    长度)+ slug 黑名单(25 个保留字)+ status 枚举
  - get(workspace_id, *, user_id=AUTO) JOIN workspace_memberships 做成员校验;
    user_id=None 显式 bypass(迁移/admin)
  - get_by_slug(slug) 不带成员校验(path-based routing 用:先 slug→workspace_id
    再到 route handler 里查成员)
  - list_by_user(*, user_id=AUTO) 列 user 所属所有 workspace
  - update_status / delete platform-admin 操作,不带成员校验

WorkspaceValidationError 自定义异常(slug 格式 / 黑名单 / status)。

23 test 覆盖:
  - CRUD smoke + get_by_slug missing
  - 重复 slug → IntegrityError
  - 8 个 invalid slug pattern(短/长/大写/空格/破折号位置/连续破折号/下划线)
  - 6 个 blacklisted slug
  - status 状态机 + 非法值拒绝
  - delete CASCADE 到 memberships(SQLite FK PRAGMA 已开启)
  - get/list 成员过滤(user-A 看不见 user-B 的 workspace)
  - list user_id=None 显式 bypass

全部在 SQLite ephemeral DB 上跑(< 1s)。partial-unique 双驱动验证留给 T3.8。

Stage 0 PR3 T3.4 + T3.5。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:11:28 +08:00