Compare commits

138 Commits

Author SHA1 Message Date
1445043649 0b866a0bee docs(stage-1): mark control-plane scope limitation resolved (Stage 1 收口)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 23:07:06 +08:00
1445043649 428d4e9eb5 docs(stage-1): spec — use path-boundary matching for data-plane allowlist
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:43:27 +08:00
1445043649 b26c5e4dd0 harden(gateway): API keys default-deny on control-plane routes (Stage 1 收口)
Service principals (Bearer dfk_...) may only reach the data plane
(threads / runs / assistants). All control-plane routes — mcp, models,
skills, channels, agents, memory, auth, management — now return 403
insufficient_scope. The check sits after the NULL/401 guard so an
invalid key on a control-plane path still returns 401, not 403. Human
cookie requests bypass the bearer branch entirely and are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:42:41 +08:00
1445043649 02628d4e08 test(auth): move bearer probe routes under /api/v1/threads (Stage 1 收口)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:27:11 +08:00
1445043649 2d982c3f0b feat(authz): data-plane allowlist helper + INSUFFICIENT_SCOPE code (Stage 1 收口)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:25:46 +08:00
1445043649 1b7d8217dd docs(stage-1): implementation plan for control-plane default-deny
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:09:31 +08:00
1445043649 a173d4f93c docs(stage-1): API key control-plane default-deny design spec
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:05:14 +08:00
1445043649 cdd9d6701f docs(stage-1): tick all plan checkboxes to match landed commits (Stage 1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:49:34 +08:00
1445043649 4cd2acc192 feat(apps): 新增 headless(API Key) 多租户冒烟测试,并登记进 apps/README
multi_tenant.py 的「server-to-server / 无人值守」版:每个租户先由人类 owner
(cookie 会话)建 service account 并 mint 一把 workspace-scoped key
(POST /api/v1/service-accounts → POST /api/v1/api-keys,plaintext 仅返回一次),
之后所有对话只用 Authorization: Bearer dfk_live_...(独立 Session,免 cookie/CSRF)。

除并发 / 多轮上下文 / 租户隔离(同 cookie 版)外,额外校验两条 headless 专属性质:
- scope 强制:缺 runs:create 的 key 发起 stream 返回 403
- 撤销即失效:DELETE /api/v1/api-keys/{id} 后该 key 立即 401

对 :8001 实跑(DF_TENANTS=4 DF_TURNS=10)全绿。同时更正 apps/README 中
「API Key 鉴权尚未接入」的过时说明。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:45:37 +08:00
1445043649 37c3417bfb docs(stage-1): record known scope/least-privilege limitation from final review (Stage 1 PR5)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:20:36 +08:00
1445043649 938c2eb0be harden(gateway): uniform 404 on API key revoke to hide cross-tenant existence (Stage 1 PR4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:19:20 +08:00
1445043649 dee1eb2374 docs(stage-1): backfill plan link in spec; sync plan with review-driven changes (Stage 1 PR5)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:12:07 +08:00
1445043649 190c1dc3c8 feat(frontend): migrate API calls to /api/v1 (Stage 1 PR5)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:07:20 +08:00
1445043649 e6a12b9a7c test(gateway): strengthen dual-mount coverage to a v1-twin invariant (Stage 1 PR5)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:01:29 +08:00
1445043649 2461a923be feat(gateway): dual-mount legacy routers on /api and /api/v1 (Stage 1 PR5)
Strip /api prefix from 13 legacy router APIRouter() declarations and dual-mount
each on prefix="/api" (backward compat) and prefix="/api/v1" (versioned surface)
in app.py. Auth, service-accounts, api-keys, assistants-compat remain single-mount.
Update 10 test files to pass prefix="/api" when directly including stripped routers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:49:05 +08:00
1445043649 3f0d5c8c96 feat(gateway): X-API-Deprecated header for legacy /api/* paths (Stage 1 PR5)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:37:36 +08:00
1445043649 9d6decd91b test(gateway): end-to-end headless API mint/use/isolation smoke (Stage 1 PR4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:34:56 +08:00
1445043649 c3c4e57416 harden(gateway): block API key mint on inactive SA; cover cross-workspace IDOR (Stage 1 PR4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:25:17 +08:00
1445043649 950964e0ef feat(gateway): api-keys management endpoints with one-time plaintext (Stage 1 PR4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:19:32 +08:00
1445043649 84e06ca396 harden(gateway): constrain service-account role/identity_mode at API boundary (Stage 1 PR4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:15:40 +08:00
1445043649 d16e294185 feat(gateway): service-accounts management endpoints (Stage 1 PR4)
Owner/admin self-service CRUD for service accounts: POST create,
GET list, PATCH status; workspace-scoped with 404 existence hiding
for cross-workspace targets. Gated by require_workspace_admin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:11:13 +08:00
1445043649 e92fe0d7fb feat(authz): require_workspace_admin dependency (Stage 1 PR4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:07:55 +08:00
1445043649 9eb6103a4d feat(csrf): skip CSRF for bearer-header requests (Stage 1 PR3)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:06:23 +08:00
1445043649 78359c3fd8 harden(auth): guard bearer auth errors as 503; widen AuthContext to ServicePrincipal (Stage 1 PR2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:04:19 +08:00
1445043649 3ec4fb8537 feat(auth): AuthMiddleware bearer dfk_ path (Stage 1 PR2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:58:14 +08:00
1445043649 ac8b37bd27 test(auth): cover expired-key rejection in APIKeyAuthBackend (Stage 1 PR2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:44:41 +08:00
1445043649 d9a86878f4 feat(auth): APIKeyAuthBackend resolves token to SA principal (Stage 1 PR2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:40:27 +08:00
1445043649 4f9116e3fe fix(auth): keep CurrentUser protocol id-only; is_service_account is opt-in (Stage 1 PR2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:37:06 +08:00
1445043649 ec4769a33f feat(auth): ServicePrincipal + is_service_account discriminator (Stage 1 PR2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:31:49 +08:00
1445043649 978b0cf24d test(persistence): cover ExternalUserRepository.get + document upsert semantics (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:29:44 +08:00
1445043649 5093d3d123 feat(persistence): ExternalUserRepository scaffold (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:26:34 +08:00
1445043649 ac2ab26a7e refactor(persistence): ApiKeyRepository hot path via indexed prefix + constant-time hash verify (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:24:06 +08:00
1445043649 ab40a4a17e feat(persistence): ApiKeyRepository with active-key hot path (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:17:07 +08:00
1445043649 04170205dd fix(persistence): validate status in ServiceAccountRepository.create (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:13:14 +08:00
1445043649 9166ab205d feat(persistence): ServiceAccountRepository (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:08:20 +08:00
1445043649 427709e0a8 test(auth): polish token utility tests + docstring (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:05:20 +08:00
1445043649 e5ff6e74f9 feat(auth): API key token generation/hashing utilities (Stage 1 PR1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:01:03 +08:00
1445043649 ef32a6de0f docs(stage-1): headless API Pattern A auth foundation implementation plan
5-PR TDD plan derived from the Stage 1 spec: token utils + 3-table repos
(deerflow layer), APIKeyAuthBackend + AuthMiddleware bearer path, CSRF
bearer skip, owner/admin mint endpoints with cross-workspace 404, and
/api/v1 dual-mount migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 10:57:24 +08:00
1445043649 240c6bd0e2 docs(mt): 统一 multi-tenant-redesign 命名约定 + README 补执行层索引
命名统一为 .zh-CN.md 后缀(与既有 17 个文件 + README 一致):
- 03-impl/{pr1-8,STATUS}.md → *.zh-CN.md
- Stage 1 spec 去日期前缀、加 .zh-CN,对齐 01-redesign 语义命名

README.zh-CN.md 修复 4 处不统一:
- 顶部加进度指引(现状只信 STATUS,本文是设计/路线导航)
- §0 文档总图补 03-impl 层 + Stage 1 spec + 命名约定注
- §1 表加 Stage 1 spec 行;新增 §1.1 执行记录层(STATUS + 8 impl note 索引)
- §7 阅读路径首次进项目/Stage 1 均加 STATUS + spec 入口

同步更新所有交叉链接(STATUS/pr/spec 自引用、database-schema-as-built、
根 README_zh.md、Stage 0 master plan);全树相对链接校验可达。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 10:40:42 +08:00
1445043649 8ea5507d27 docs(spec): 将 Stage 1 鉴权地基 spec 移入 multi-tenant-redesign/01-redesign
与 Stage 0 设计(ADR / schema-design)并排,统一策展主线;
相对链接从 ../../multi-tenant-redesign/ 改为 ../。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 10:30:25 +08:00
1445043649 45b019dae8 docs(spec): Stage 1 headless API Pattern A 鉴权地基设计
承接 Stage 0(PR1-8 全 merge)与 headless-api-track 轨道二。
锁定 5 个 PR:三表仓储+token 工具 / APIKeyAuthBackend 双路径 /
CSRF skip on bearer / 管理 endpoint mint 闭环 / api/v1 全量迁移。
含 5 项 brainstorm 决策(D1 SA→CurrentUser 映射 / D2 管理 endpoint /
D3 全量迁 v1 / D4 scope 白捡 / D5 dfk_ key 格式)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 10:19:26 +08:00
1445043649 3a16da6449 docs(stage-0): 记录 multi_tenant.py live smoke PASS,关闭退出门工程项
2026-06-27 用户运行 apps/examples/http-chat/multi_tenant.py 打到运行中
Gateway,确认 verdict=PASS。该 smoke 覆盖退出门「注册→workspace 自建→
创建 thread→跨 workspace 互调 404」,且更强:N 租户真并发 + 多轮链式上
下文 + 双向隔离(search 不泄漏 + 跨租户 GET 404)。

- STATUS.md: 新增「Live smoke 结果」段(只记代码层核实的不变量,未编造
  租户数/轮数;标注 JWT wid claim 未显式断言的缺口);PR4 T4.14 标 、
  PR6 T6.15 标部分 done(文件迁移仍 );更新一句话状态 + 分支改名
- plan 退出门:勾上 5 条工程项(PR 合入/测试/CI/smoke/LOCK),仅剩
  2 条时间门 + 1 条业务门未关

纯文档,无代码改动。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 09:59:18 +08:00
1445043649 30fa16ba7a feat(apps): 新增 http-chat 多租户并发示例并登记进 apps/README
multi_tenant.py:app.py 的并发/多租户版,用 /api/v1/auth/register
并发创建多个租户(各自独立 workspace + Session),同时跑 N 轮链式
对话,校验真并发、多轮上下文按 thread 各自保持、租户隔离
(search 仅见己有线程 + 跨租户 GET 404)。轮数/租户数可配
(DF_TURNS / DF_TENANTS),唯一邮箱可重复跑且不触发限流。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 23:42:24 +08:00
1445043649 c77ee0dc06 docs(readme): 中文 README 改以多租户改造为主线 + 忽略 IDE/agent 产物
- README_zh.md:全面重写,把 workspace 租户模型 / Postgres 默认后端 /
  行级隔离 / 扩展 JWT / per-workspace 路径 / Headless API schema / apps
  脚手架作为主线,新增「多租户改造」「数据库后端」「多租户架构详解」
  「apps/」章节,并补 Stage 0–4 路线图与文档入口
- .gitignore:忽略 .qoder/(Qoder repowiki 产物)与 AGENTS.md(agent 指令)
- verify_stage0.sh:e2e 校验改用 verify-stage0.com 邮箱域、先 initialize
  admin、切到 /api/v1/ auth 路径

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 23:15:48 +08:00
1445043649 45efba50e4 docs(db): 补充数据库设计落地版(as-built)并登记进 README
新增 docs/multi-tenant-redesign/01-redesign/database-schema-as-built.zh-CN.md:
对照实现代码(persistence/*/model.py + base.py/engine.py + alembic 0001-0003)
生成的事实参考,补齐之前只有「锁定版」(动手前决策)而缺失的落地 schema 文档。

涵盖:
- 持久化层总览:memory/sqlite/postgres 三后端、create_all vs Alembic、
  Postgres 库自愈、SQLite WAL、JSON ensure_ascii=False、partial index 双 where 兼容
- LangGraph checkpointer/store 表不归 ORM 管的说明
- ER 图(mermaid)+ 10 张表全字段参考(users/workspaces/workspace_memberships/
  threads_meta/runs/run_events/feedback/service_accounts/api_keys/external_users)
- 外键与删除策略矩阵(CASCADE/RESTRICT/SET NULL)
- 迁移历史 0001-0003(含 0002→0003 两段式上线:可空列→回填→锁 NOT NULL)
- 与锁定版的差异(run_events 已确认为 DB 表、runs token 分项列、status 状态机等)

README.zh-CN.md 的「文档总图」与「状态表」登记该文档,并注明
与锁定版冲突时以落地版为准。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 22:55:01 +08:00
1445043649 dd1d40368d feat(scripts): 新增本地调试与示例启动脚本,并登记进 apps/README
新增 4 个脚本,统一子命令风格(start/stop/restart/status/logs/run):

- scripts/dev-gateway.sh:方式 B,只起 Gateway(:8001),用 backend/.venv 虚拟环境,
  自动加载 .env、释放端口、等待就绪;带 PID/日志文件,可查状态与跟随日志。
- scripts/dev-full.sh:方式 A,复用 serve.sh 守护模式起全量栈(Gateway+前端+nginx),
  补齐 serve.sh 缺失的 status 与 logs;restart 默认跳过依赖安装,统一入口 :2026。
- apps/examples/http-chat/run.sh:自动探测网关(:2026 优先,回退 :8001),
  优先 uv 临时环境带 requests(--no-project,不污染系统),无 uv 时回退 venv+pip。
- apps/examples/embedded-chat/run.sh:自动定位 backend、加载 .env 后用 uv run 运行,
  内嵌 SDK 模式无需起服务。

apps/README.md 增加「本地调试脚本」与「运行示例」两节,说明上述脚本用法。

全部脚本均已本地实跑验证:全量栈三服务 HTTP 200,两个示例多轮对话正常。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 22:47:30 +08:00
1445043649 7dfc9968fe feat(apps): 新增 DeerFlow 应用脚手架,并修复 store 的 database 回退
apps/:新增「基于 DeerFlow 的应用」目录,与 backend/、frontend/ 平级,
位于「app 消费 deerflow、不反向依赖」边界的正确侧。含两种集成示例:
  - examples/http-chat   —— HTTP Gateway (REST+SSE),含登录/CSRF/建线程/流式对话
  - examples/embedded-chat —— 进程内直接调 DeerFlowClient
README 说明边界规则、两种模式、鉴权流程及新建应用约定。

runtime/store:修复 make_store 缺失的 database 段回退。原先 store 工厂只读
legacy 的 checkpointer 段,导致仅配 database:postgres 时,checkpointer 走了
Postgres、但 store 仍回退 InMemoryStore(并打出误导性的「线程列表会丢失」告警,
实际线程在 threads_meta 表里、本就持久)。现对齐 checkpointer 工厂的优先级:
checkpointer 段 → database 段 → InMemoryStore;postgres 分支同样剥掉 +asyncpg
方言前缀,使一个 DATABASE_URL 同时满足 SQLAlchemy 与 LangGraph 的 psycopg store。
告警文案也修正为「跨线程 store 数据会丢失」。

tests:新增 test_store_provider.py(3 例,TDD)覆盖 database→postgres 回退、
无配置时的内存回退、以及 checkpointer 段优先级。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 22:02:44 +08:00
1445043649 84de632b19 chore(scripts): add verify_stage0.sh — systematic Stage 0 verification
Adds a single script that walks the 5 verification layers in order
and emits pass/fail counts at the end:

  static   — boundary scans + full pytest (baseline ≥ 3250 passed,
             ≤ 18 fails matching known caplog flake set) + ruff lint
  paths    — .deer-flow/users/ should be empty (or absent);
             workspaces/{wid}/threads/{tid}/... layout in place;
             exercises `make migrate-paths DRY_RUN=1`
  rds      — PG reachable; alembic at 0003; service_accounts +
             api_keys + external_users tables present;
             idx_api_keys_active partial index has
             "WHERE revoked_at IS NULL" predicate; workspace_id is
             NOT NULL on thread_meta / runs / feedback / run_events;
             UNIQUE(workspace_id, thread_id) on thread_meta
             (requires DATABASE_URL; skipped if unset)
  runtime  — curl /health on the Gateway (requires `make dev`;
             skips downstream e2e if unreachable)
  e2e      — register two users via /api/auth/register, capture each
             session's csrf_token, create one thread per user,
             cross-access GET + DELETE both return 404 (per PR6
             "404 not 403" contract), same-workspace GET returns 200

Each layer is independently runnable: `./verify_stage0.sh paths rds`.
With no args runs all five.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:54:02 +08:00
1445043649 6090b9e5b1 docs(impl): PR8 implementation note + STATUS update (Stage 0 收尾)
Adds `pr8-headless-api-schema.md`:
- Scope summary (3 schema-only tables for Stage 1 headless API base)
- Per-task commit table (T8.1-T8.6)
- Acceptance: 9 new tests + 3 tables auto-created + FK behaviour + partial index DDL pinned
- Architecture decisions (key_prefix global UNIQUE rationale, String scopes not PG text[], external_users workspace_id redundancy, no Repository class until Stage 1, T8.6 reverse invariant)
- File structure index (new vs modified)
- Live smoke commands (RDS \dt + \d+ api_keys for partial index)

Updates STATUS.md:
- One-line status: Stage 0 工程层面收尾 (PR1-PR8 全合)
- 8-PR status table: PR8 row marked merged with commit range and impl note link
- Test baseline: PR8 末 3250 passed + 31 skipped + 18 flake; +163 new tests over Stage 0
- Skipped/deferred 行 加 PR8 RDS live smoke 项
- Next-step suggestion 翻新:6 个 live verification 用户跟进项 + Stage 0 退出 Go/No-Go 工程门已满足 + Stage 1 可启动方向
- PR8 经验回顾段:Inline + 严格 TDD,3 张表互相独立,T8.6 反向 invariant 锁定历史坑

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:18:18 +08:00
1445043649 f803f393d3 test(persistence): PR8 T8.6 — verify all 3 PR8 tables auto-created
Adds tests/test_pr8_metadata_registration.py: opens a fresh SQLite
engine via init_engine() and asserts inspect(conn).get_table_names()
contains service_accounts, api_keys, and external_users. Guards
against an ORM row class being added under deerflow/persistence/* but
accidentally left out of deerflow/persistence/models/__init__.py —
which would leave the table un-provisioned at startup and surface as
a confusing "no such table" later in Stage 1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:14:14 +08:00
1445043649 6f806ff4a4 feat(persistence): PR8 T8.5 — ExternalUserRow ORM + 2 tests
Adds deerflow/persistence/external_user/{__init__,model}.py with
ExternalUserRow:
  - id: UUID36 PK
  - workspace_id: FK workspaces ON DELETE CASCADE (redundant with SA's
    workspace_id but stored directly to speed workspace-scoped queries
    that span multiple SAs)
  - service_account_id: FK service_accounts ON DELETE CASCADE
  - external_id: String(128) — caller-supplied X-External-User-Id
  - display_name: String(128) nullable (admin UI only, not auth-relevant)
  - metadata_json: JSON nullable=False default {} — plan tier / region /
    custom tags
  - created_at / last_seen_at (UTC)
  - UniqueConstraint (service_account_id, external_id)
    name=uq_external_users_sa_external — the same external_id may be
    reused under a different SA, but is upsert-unique under a single SA

T8.5 tests:
  - test_unique_service_account_id_plus_external_id: second row with
    same (SA, external_id) raises IntegrityError
  - test_cascade_on_service_account_delete: deleting parent SA removes
    all external_users rows under it

Registered in deerflow/persistence/models/__init__.py — all three PR8
tables (service_accounts / api_keys / external_users) are now wired
into Base.metadata.create_all().

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:12:43 +08:00
1445043649 52e9999a61 feat(persistence): PR8 T8.4 — ApiKeyRow ORM + 3 tests
Adds deerflow/persistence/api_key/{__init__,model}.py with ApiKeyRow:
  - id: UUID36 PK
  - service_account_id: FK service_accounts ON DELETE CASCADE
  - key_prefix: String(16), UNIQUE — global uniqueness preserved even
    after revoke so audit logs never reference an ambiguous prefix
  - key_hash: String(128) sha-256 hex (plaintext returned only once at
    create time)
  - name / scopes / rate_limit_rpm / expires_at / last_used_at /
    revoked_at — all nullable or default-providing
  - created_at: UTC
  - Index idx_api_keys_sa (service_account_id) — list keys for an SA
  - Index idx_api_keys_active (key_prefix) WHERE revoked_at IS NULL —
    partial index, dual-dialect via sqlite_where + postgresql_where,
    shrinks the hot-path lookup index by excluding revoked keys

T8.4 tests:
  - test_unique_key_prefix_enforced: column-level UNIQUE blocks two
    rows from sharing key_prefix (active or revoked alike)
  - test_active_index_declares_both_dialect_where_clauses: schema
    introspection confirms idx_api_keys_active has both
    `dialect_options.sqlite.where` and `dialect_options.postgresql.where`
    set to `revoked_at IS NULL` — guards against accidental loss of the
    dual-driver hint when the Index is edited later
  - test_cascade_on_service_account_delete: deleting the parent SA
    removes all api_keys rows

Registered in deerflow/persistence/models/__init__.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:10:32 +08:00
1445043649 bb7289781e test(persistence): PR8 T8.2 + T8.3 — ServiceAccount cascade / restrict
T8.2 test_cascade_on_workspace_delete: seed user → workspace → SA,
delete workspace, assert SA row is gone (ondelete CASCADE on
workspace_id FK).

T8.3 test_restrict_on_created_by_user_delete: seed user → workspace
→ SA, then attempt DELETE FROM users WHERE id = created_by, assert
IntegrityError raises and the SA row survives (ondelete RESTRICT on
created_by FK).

SQLite enforces FKs because the engine's connect-listener turns on
`PRAGMA foreign_keys = ON` for every new connection — see engine.py
init_engine().

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:07:16 +08:00
1445043649 1fb07e48e6 feat(persistence): PR8 T8.1 — ServiceAccountRow ORM + insert smoke
Adds deerflow/persistence/service_account/{__init__,model}.py with
ServiceAccountRow:
  - id: UUID36 PK
  - workspace_id: FK workspaces ON DELETE CASCADE
  - name: String(64)
  - role: String(16) default "member" (Stage 0 lone value; Stage 2 RBAC)
  - identity_mode: String(16) default "collapsed" — three states
    "collapsed" / "external_passthrough" / "both"; Stage 1 API key
    auth layer branches on this to decide whether each call writes
    an external_users row
  - status: String(16) default "active"
  - created_by: FK users ON DELETE RESTRICT (must hand off / delete
    SAs before removing their creator)
  - created_at / updated_at (UTC, onupdate)
  - Index idx_service_accounts_workspace (workspace_id, status)

Registers in deerflow/persistence/models/__init__.py so the engine's
side-effect import path picks it up for Base.metadata.create_all().

T8.1 test: insert_smoke writes a row through a session, reads it back,
asserts each column round-trips. SQLite ephemeral DB per test via
tmp_path, no Postgres required at this layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 14:06:21 +08:00
1445043649 b0bf033f15 docs(impl): PR7 implementation note + STATUS update
Adds `pr7-ci-boundary-scan.md`:
- Scope summary (allowlist + AST scanner + TYPE_CHECKING exemption + self-tests + CLAUDE.md docs)
- Per-task commit table (T7.1-T7.5; T7.4 is the revert-after-prove injection drill — no commit by design)
- Acceptance: RED->GREEN cycle, 9 self-tests, T7.4 inject/revert, 3241 passed + 31 skipped + 18 caplog flake
- Architecture decisions (TYPE_CHECKING exemption vs allowlist inclusion, toml path-list shape, pytest-not-separate-CI-step, scope kept narrow, plan-draft vs ground-truth allowlist)
- File structure index (new vs modified)

Updates STATUS.md:
- One-line status moves to PR7 merged; Stage 0 now only PR8 left
- 8-PR status table: PR7 row marked merged with commit range and impl note link; commit count notes T7.4 has no commit by design
- Test baseline line: PR7 末 3241 + 31 + 18 (PR6 末 3214 + 30 + 17, +27/+1/+1 explained)
- Next-step suggestion switches to PR8 (parallel-eligible since PR4); PR7 inline-mode retro added

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:59:22 +08:00
1445043649 d8b13afc59 docs(backend): PR7 T7.5 — document workspace boundary scan in CLAUDE.md
Extends the existing "Boundary check" section with two new entries:

- tests/test_workspace_boundary.py — AST static scan for direct imports of
  langgraph.checkpoint.* (and the third-party postgres/sqlite packages)
  outside tests/boundary_allowlist.toml. Notes the TYPE_CHECKING exemption
  and points readers to `app.gateway.deps.get_checkpointer` as the
  intended path; documents the "append to allowlist in the same PR"
  contract for new legitimate importers.
- tests/test_workspace_boundary_self.py — self-tests guarding the scanner
  from silent-empty regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:54:31 +08:00
1445043649 d0f1877070 test(boundary): PR7 T7.3 — self-tests for the scanner
Adds backend/tests/test_workspace_boundary_self.py (9 cases) to guard
against silent-empty regressions in the boundary scanner: every claim
the scanner makes is reproduced against a synthetic .py snippet written
to tmp_path and fed to `collect_runtime_checkpoint_imports`.

Coverage:
- direct `from langgraph.checkpoint.* import X` -> flagged
- bare `import langgraph.checkpoint.memory` -> flagged
- third-party `langgraph_checkpoint_postgres` / `..._sqlite` -> flagged
- `if TYPE_CHECKING:` block -> skipped (Name form)
- `if typing.TYPE_CHECKING:` block -> skipped (Attribute form)
- nested-under-`if True:` TYPE_CHECKING block -> skipped (parent walk)
- unrelated imports (`os`, `langgraph.graph.state`, `app.gateway.deps`) -> ignored
- string literal containing the target module path -> ignored (not an import node)
- syntax-broken file -> returns [] (parser exception swallowed)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:52:36 +08:00
1445043649 ba30d14041 feat(tests): PR7 T7.2 — AST boundary scanner for langgraph.checkpoint
Adds backend/tests/test_workspace_boundary.py: walks every .py under
backend/ (excluding tests/, docs/, build artefacts), parses with ast,
flags any ImportFrom/Import that targets langgraph.checkpoint.*,
langgraph_checkpoint_postgres or langgraph_checkpoint_sqlite outside the
allowlist loaded from tests/boundary_allowlist.toml.

Skip rules:
- Imports inside `if TYPE_CHECKING:` (or `if typing.TYPE_CHECKING:`)
  blocks are exempt: they do not enter runtime so cannot bypass the
  boundary. Verified against deerflow/agents/factory.py which only uses
  BaseCheckpointSaver as a parameter annotation.

The scanner walks parent pointers up from each import node to detect
the enclosing TYPE_CHECKING guard, rather than checking only direct
parents — handles nested guards correctly.

RED proof (empty allowlist): 14 violations across the 4 ground-truth
runtime importers (threads / async_provider / provider / worker).
GREEN (with this PR's allowlist): single test passes in ~2s.

The collector function `collect_runtime_checkpoint_imports` and
`scan_violations` are kept at module scope so T7.3's self-test can
exercise them on a synthetic temp file in the next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:50:24 +08:00
1445043649 1a6ccc9aaa feat(tests): PR7 T7.1 — boundary scan allowlist toml
Adds backend/tests/boundary_allowlist.toml that names the four current
legitimate runtime importers of langgraph.checkpoint.*:
  - app/gateway/routers/threads.py (empty_checkpoint for thread init)
  - packages/harness/deerflow/runtime/checkpointer/async_provider.py
  - packages/harness/deerflow/runtime/checkpointer/provider.py
  - packages/harness/deerflow/runtime/runs/worker.py (empty_checkpoint)

The allowlist replaces plan's draft (thread_runs.py / gateway/app.py)
with the ground-truth grep: both of those reach the checkpointer through
`app.gateway.deps.get_checkpointer` DI, so they need not be listed.
factory.py imports BaseCheckpointSaver only under `if TYPE_CHECKING:` —
the scanner (next commit) exempts type-only imports automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:50:12 +08:00
1445043649 4c0b4fab48 docs(impl): PR6 implementation note + STATUS update
Adds `pr6-routes-paths-workspace.md`:
- Scope summary (routes + repos + Paths + middleware + migration script + lifespan + T5.11)
- Per-task commit table (T5.11 + T6.1..T6.15)
- Acceptance checklist with the 64 net-new test count and 17 flake breakdown
- Architecture decisions (404 not 403, check_access signature, three-state semantics, path shape, fallback workspace)
- Live smoke command list for the migration follow-up
- File structure index (new vs modified)

Updates STATUS.md:
- One-line status moves to PR6 merged, T5.11 cleared as PR6 deliverable
- 8-PR status table: PR6 row marked merged with commit range and impl note link
- Skipped/deferred rows: T5.11 struck as completed, new PR6 T6.15 live-smoke row added
- Next-step suggestion switches to PR7 (CI boundary static scan); PR8 noted parallelisable
2026-05-13 18:09:42 +08:00
1445043649 87ea715c2a feat(persistence): PR6 T5.11 — ORM workspace_id nullable=False
Flip the 4 business ORM models (ThreadMetaRow, RunRow, FeedbackRow,
RunEventRow) to ``workspace_id: Mapped[str]`` with ``nullable=False``.
PR5's alembic 0003 already enforces NOT NULL at the DB layer; this
aligns the ORM-driven ``create_all()`` path (dev / tests) with the same
invariant so a new install ends up at the post-0003 schema without
running alembic.

Test fallout absorbed:

- `tests/conftest.py` autouse seed now produces a fully consistent
  pair: user row (default_workspace_id = test-workspace-autouse) plus
  the workspace itself. The PR5 backfill script's "users without
  default_workspace_id" query no longer picks the fixture up. Insert
  order is user → workspace → UPDATE user, walking around the chicken-
  and-egg FK between `workspaces.owner_id` and `users.default_workspace_id`.
- `tests/test_backfill_workspace_id.py` adds a file-scoped autouse
  fixture that temporarily flips `column.nullable = True` for the four
  business tables (production correctness comes from alembic 0003;
  the script's own job is exactly to fill rows between 0002 and 0003
  so its tests need that transient state to be representable). Its
  `_init_engine` also deletes the autouse seed rows to match the
  "fresh DB" model the tests assume.
- `test_thread_meta_workspace_filter::test_create_workspace_none_bypasses`
  renamed to `test_create_workspace_none_rejected_by_orm` and asserts
  the new IntegrityError on explicit None — write paths can no longer
  bypass workspace scope.
- 5 `test_workspace_context` tests + the auth-middleware reset test
  get `@pytest.mark.no_auto_workspace` so they keep testing the
  unset-contextvar path.
- `test_workspace_repo::test_list_by_user_bypass_returns_all` switches
  to membership assertions instead of strict equality since the
  autouse fixture surfaces under `user_id=None`.

3214 passed, 30 skipped; the remaining 17 are the documented
pre-existing caplog ordering flakes (all pass in isolation).
2026-05-13 18:06:53 +08:00
1445043649 c5c66ccbfc feat(gateway/make): PR6 T6.14 — migrate-paths target + lifespan warning
`app/gateway/app.py:_check_path_migration_pending` runs at lifespan
startup and emits a single WARNING log when `{base_dir}/users/` still
has content — the operator's cue to run the new migration script. The
warning is best-effort (silent on permission errors) so it can never
escalate into a gateway boot failure; reads from the legacy tree keep
working via the `user_id` branch of `Paths.thread_dir` until the
migration runs.

The root Makefile gains a `migrate-paths` target that wraps
`scripts/migrate_paths_to_workspace.py`. Two opt-in env knobs:
- `DRY_RUN=1`           — pass `--dry-run` (preview only, no writes)
- `DEFAULT_WORKSPACE=…`  — claim un-assigned users under this workspace id
   (defaults to `legacy_workspace` to align with PR5's orphan-row bucket).
`help` documents the new target.

3 tests cover the warning's three states (legacy dir with content, dir
missing, dir empty).
2026-05-13 17:50:47 +08:00
1445043649 56f2c8d873 feat(scripts): PR6 T6.12 + T6.13 — migrate_paths_to_workspace.py + tests
`scripts/migrate_paths_to_workspace.py` walks the PR4 per-user filesystem
layout and rewrites it under the PR6 per-workspace dimension:

- `{base}/users/{uid}/threads/{tid}/` → `{base}/workspaces/{wid}/threads/{tid}/`
- `{base}/users/{uid}/memory.json`    → `{base}/workspaces/{wid}/users/{uid}/memory.json`
- `{base}/users/{uid}/agents/{name}/`  → `{base}/workspaces/{wid}/users/{uid}/agents/{name}/`

`{wid}` is read from `users.default_workspace_id` in the local sqlite
DB; users without one fall back to `--default-workspace`, defaulting to
`legacy_workspace` (the same bucket PR5's row backfill uses for orphan
rows so the file-system and database alignments stay consistent).

Pre-existing destinations route the legacy copy to
`{base}/migration-conflicts/workspace-migration/...` for manual review;
empty source dirs are cleaned up. The script is idempotent and supports
`--dry-run` (writes nothing, still logs the full report).

9 tests cover: thread / memory / agent rewrite, dry-run no-write, the
fallback workspace path, conflict diversion, post-migration cleanup of
empty `users/` dirs, and the missing-DB / missing-table degraded
modes.
2026-05-13 17:48:49 +08:00
1445043649 2d3b546bf9 feat(agents): PR6 T6.11 — ThreadDataMiddleware switches to workspace layout
`ThreadDataMiddleware.before_agent` now reads
`get_effective_workspace_id()` and routes the per-thread directory tree
through `Paths.sandbox_*_dir(thread_id, workspace_id=...)`, producing
`{base_dir}/workspaces/{wid}/threads/{tid}/user-data/...`. The legacy
user-id-only layout is no longer written by this middleware; the
migration script in T6.12 will lift any pre-existing `users/{uid}/...`
trees into the new shape.

In no-auth dev mode the contextvar is empty so
`get_effective_workspace_id` returns `"default"` and writes land at
`workspaces/default/...` — the layout invariant ("threads always live
inside a workspace") holds without a real auth setup. `thread_data` now
also exposes `user_id` and `workspace_id` so downstream middlewares
(sandbox, memory, etc.) can read them without re-resolving the
contextvar themselves.

4 new tests cover: contextvar workspace → expected path,
`no_auto_workspace` falls back to `default`, eager mode creates the
right dirs, and the `get_config` fallback still routes through
workspace. Existing 4 thread_data middleware tests stay green.
2026-05-13 17:45:06 +08:00
1445043649 f013fc1a65 feat(paths): PR6 T6.9 + T6.10 — workspace-scoped Paths
`Paths` learns a new top-level filesystem dimension. Precedence in
descending order:

- workspace_id given (PR6+):
  `{base_dir}/workspaces/{wid}/threads/{tid}/...`
  `{base_dir}/workspaces/{wid}/users/{uid}/memory.json`
  `{base_dir}/workspaces/{wid}/users/{uid}/agents/{name}/...`
- user_id only (legacy user-isolation): unchanged
- neither (very legacy): unchanged

API: every thread/sandbox/host/ACP helper now takes
`workspace_id: str | None = None` alongside the existing `user_id`,
plus `Paths.workspace_dir()` and `_validate_workspace_id()` for safe
id substitution. `resolve_virtual_path` likewise routes through the
new precedence. The migration script in T6.12 will walk the old
`users/{uid}/threads/...` tree and rewrite to the new shape.

15 new tests (workspace shape, validation, sandbox dirs, ACP,
user-memory/agents under workspace, traversal defence) pass; 46
existing path tests stay green.
2026-05-13 17:43:51 +08:00
1445043649 a7ecd76e0a test(boundary): PR6 T6.8 — cross-workspace isolation e2e (4 cases)
End-to-end coverage for the workspace boundary:

- GET /api/threads/{tid}    from workspace B → 404
- DELETE /api/threads/{tid} from workspace B → 404
- PATCH /api/threads/{tid}  from workspace B → 404
- Positive control: same workspace → 200

Wires `MemoryThreadMetaStore` (real impl, not a mock) behind a
stub-authed FastAPI app. The check_access call inside
`@require_permission` returns False on cross-workspace and the
decorator converts to 404 — proving the boundary holds at the HTTP
boundary, not just the unit level. Run stream/wait are skipped here
(they spin a background worker); their guard goes through the same
decorator path so unit coverage in `test_require_permission_workspace`
is sufficient.

`_StubAuthMiddleware` gains `override_user_contextvar=True` so
cross-user / cross-workspace tests can drive both contextvars from the
stub. Default stays off — the autouse user fixture continues to own
the contextvar for legacy tests whose routes resolve filesystem paths
via `get_effective_user_id()`.

86 router / boundary tests stay green.
2026-05-13 17:40:56 +08:00
1445043649 f6a922921b test(routers): PR6 T6.7 — POST /api/threads workspace_id integration
`create_thread` in `app/gateway/routers/threads.py` already relies on
the AUTO sentinel default for ``ThreadMetaStore.create``, so once T6.1
landed the workspace_id was implicitly carried from the contextvar.
This commit adds the integration coverage: TestClient + stub auth
middleware + ``MemoryThreadMetaStore`` round-trip showing the persisted
row carries the right ``workspace_id`` under two distinct workspaces.

No production code change — the route wiring was correct, but the
guarantee was previously only proven at the unit level.
2026-05-13 17:35:27 +08:00
1445043649 0456606dc1 test(authz): PR6 T6.6 — @require_permission workspace_id propagation tests
The decorator path was already updated in T6.4 to read
`get_effective_workspace_id()` and forward it as the third positional
to `check_access`. T6.6 closes the loop with dedicated coverage:

- `_StubAuthMiddleware` and `make_authed_test_app` now accept an
  optional `workspace_factory` so router tests can drive the active
  workspace contextvar end-to-end through the FastAPI middleware stack.
- 5 new probe tests assert: cross-workspace → 404 (not 403), same
  workspace → 200, the third positional reaching `check_access`
  carries the contextvar id, no-context tests (marked
  `no_auto_workspace`) fall back to "default", and read-style routes
  (`require_existing=False`) thread workspace_id too.

75 existing router tests (artifacts / runs / threads / uploads /
suggestions) stay green.
2026-05-13 17:33:02 +08:00
1445043649 b4fa3bf12a feat(persistence): PR6 T6.5 — Run/Feedback/RunEvent repos workspace_id
`RunRepository` (put / get / list_by_thread / delete), `FeedbackRepository`
(create / get / list_by_run / list_by_thread / list_by_thread_grouped /
upsert / delete / delete_by_run), and `DbRunEventStore` (put / put_batch /
list_messages / list_events / list_messages_by_run / count_messages /
delete_by_thread / delete_by_run) all accept
`workspace_id: str | None | _AutoSentinel = AUTO`.

- Write paths stamp `workspace_id` from the contextvar (same shape as
  the existing `user_id` stamping). For event writes the soft-read
  `_workspace_id_from_context()` mirrors `_user_id_from_context()` so
  background worker writes without a contextvar leave the column NULL —
  consistent with PR5's nullable-during-backfill stance.
- Read paths get an extra `WHERE workspace_id = :wid` clause when the
  resolved value is not None.

7 new tests (3 RunRepo + 2 Feedback + 2 RunEvent) prove cross-workspace
reads see zero rows. 92 existing run / feedback / event tests stay green.
2026-05-13 17:29:03 +08:00
1445043649 05be7f9ad0 feat(persistence/authz): PR6 T6.4 — check_access takes workspace_id
`ThreadMetaStore.check_access(thread_id, user_id, workspace_id, *,
require_existing)` is now a three-positional method. Cross-workspace is
denied unconditionally — even when the row exists and `user_id` matches
— so the decorator layer can convert the False into a **404** and never
leak the existence of a thread across tenants. Inside the workspace, the
existing legacy semantics still hold (NULL `row.user_id` stays
"shared in workspace", `require_existing` still gates the missing-row
path against ghost-row re-targeting).

`@require_permission(owner_check=True)` in `app/gateway/authz.py` now
reads the active workspace from `get_effective_workspace_id()` (set by
PR4 AuthMiddleware; falls back to "default" in no-auth dev) and passes
it through. The existing 404-not-403 mapping is unchanged.

Existing positional callers in `test_thread_meta_repo.py` and the
permissive mock in `test_threads_router.py` were updated for the new
arity. 58 thread_meta / router / memory tests stay green; 90 auth /
uploads / suggestions tests stay green.
2026-05-13 17:23:38 +08:00
1445043649 28ad6c2b0b feat(persistence): PR6 T6.3 — search/update_*/delete workspace_id sentinel
Every remaining ThreadMetaStore method gains a `workspace_id` keyword
mirroring `user_id`:

- `search()` adds WHERE workspace_id and (for the memory impl) folds it
  into the BaseStore filter dict.
- `update_display_name` / `update_status` / `update_metadata` / `delete`
  no-op if the row lives in a different workspace. The SQL helper
  `_check_ownership()` was widened to do both checks in one pass.
- `MemoryThreadMetaStore._get_owned_record()` likewise takes both ids.

5 new isolation tests prove writes from workspace B against workspace A's
thread are silently dropped (no row mutation when the caller re-reads
from workspace A). 38 existing thread_meta / owner / memory-store tests
still pass.
2026-05-13 17:20:40 +08:00
1445043649 296a4f1950 feat(persistence): PR6 T6.2 — ThreadMetaRepository.get filters by workspace_id
`get()` accepts `workspace_id: str | None | _AutoSentinel = AUTO` and
moves the workspace check into the SQL WHERE so cross-workspace lookups
short-circuit without loading the row. `MemoryThreadMetaStore._get_owned_record`
gets the same filter for parity.

The user_id check stays as a post-load comparison (preserves the existing
shared-row semantics where row.user_id IS NULL means "everyone in this
workspace"). Cross-workspace always returns None, never the row.

3 new tests cover the three states: in-workspace get returns the row,
out-of-workspace get returns None even when user_id matches, explicit
workspace_id=None bypasses (migration / CLI). Existing 22 thread_meta
tests stay green.
2026-05-13 17:19:00 +08:00
1445043649 361e653d37 feat(persistence): PR6 T6.1 — ThreadMetaRepository.create workspace_id sentinel
`create()` now accepts `workspace_id: str | None | _AutoSentinel = AUTO`
on both the SQL and in-memory implementations (and the abstract base).
AUTO resolves via `resolve_workspace_id()` from the workspace contextvar
that PR4 AuthMiddleware sets; explicit None bypasses for migration paths;
explicit str overrides the contextvar.

Test infrastructure:
- conftest gains an autouse `_auto_workspace_context` fixture mirroring
  the existing user fixture. Opt-out via `@pytest.mark.no_auto_workspace`.
- A SQLAlchemy `after_create` listener on `Base.metadata` seeds the
  matching `test-workspace-autouse` + `test-user-autouse` rows whenever
  `init_engine` runs `create_all()`, so the FK from threads_meta to
  workspaces resolves. Alembic migration tests bypass create_all and are
  unaffected, keeping real FK constraints under test.
- `test_thread_meta_workspace_filter.py` covers the three AUTO / explicit
  / None paths.

3 new tests pass; 115 existing thread_meta/run/feedback/run_event/owner
tests stay green.
2026-05-13 17:17:56 +08:00
1445043649 430f4a1132 docs(impl): PR5 implementation note + STATUS update
11 PR5 commits: alembic 0002 (nullable workspace_id + FK + composite
index on 4 business tables) -> backfill_workspace_id.py (3-step
idempotent) -> alembic 0003 (pre-flight refuse + NOT NULL + UNIQUE).
18 new tests (9 alembic 0002+0003 + 9 backfill, including dry-run and
no-users error). 3150 passed + 30 skipped + 16 pre-existing caplog
flake (unchanged from PR4).

T5.11 (ORM nullable=False) deferred to PR6 — flipping the ORM-side
NOT NULL would break 6 INSERT sites whose repository signatures get
workspace_id wiring in PR6 anyway. DB-level invariant is already
enforced by alembic 0003 along the production path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:22:19 +08:00
1445043649 30f2bd0084 test(persistence): alembic 0003 — NOT NULL + UNIQUE coverage
Three SQLite cases + Postgres twins: happy-path upgrade after
populating workspace_id, refuse-with-NULL-leftover raises RuntimeError,
and a UNIQUE(workspace_id, thread_id) IntegrityError reproducer. The
UNIQUE test uses a scratch table because the production schema's
PRIMARY KEY on thread_id would shadow the new UNIQUE index. Index
truthiness asserted instead of bool-equality so SQLite (int 1) and
Postgres (True) both pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:16:24 +08:00
1445043649 73d0b7017b feat(persistence): alembic 0003 — workspace_id NOT NULL + UNIQUE(wid, tid)
Refuses to upgrade if any of the 4 business tables still has NULL
workspace_id rows (pre-flight count then RuntimeError pointing the
operator at scripts/backfill_workspace_id.py). Then op.batch_alter_table
flips each column to NOT NULL and adds the threads_meta
(workspace_id, thread_id) UNIQUE index that PR6 routers will rely on.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:15:07 +08:00
1445043649 8a03abac75 test(persistence): backfill --dry-run end-to-end no-write proof
Snapshots workspace / membership / users.default_workspace_id row
counts before backfill(dry_run=True), runs the orchestrator, asserts
no counts changed. Pins the dry-run report's per-step semantics:
Step 1 counts candidates without populating defaults, so Step 2's
JOIN reports 0; Step 3 picks up all NULL business rows untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:13:16 +08:00
1445043649 def45dd0c6 feat(persistence): backfill Step 3 — orphan rows -> legacy_workspace
_ensure_legacy_workspace creates the nil-UUID anchor (slug=legacy)
owned by the platform admin (or oldest user as fallback). Raises a
clear error if the DB has no users at all so we never silently create
an orphaned workspace. Step 3 UPDATEs each table's remaining
workspace_id IS NULL rows to LEGACY_WORKSPACE_ID. Orchestrator wires
ensure-then-loop between Step 2 and Step 3. 3 new tests: orphan
fan-out, no-users error, end-to-end orchestrator with mixed owned +
orphan rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:12:26 +08:00
1445043649 56f6572086 feat(persistence): backfill Step 2 — UPDATE 4 tables from users.default_workspace_id
Per-table correlated subquery UPDATE (portable across SQLite + Postgres).
Filters workspace_id IS NULL AND user_id IS NOT NULL so already-tagged
rows and orphan rows are skipped. Dry-run mode counts via a JOIN, never
writes. Returns rows-updated for the orchestrator report. 2 new tests:
single-user 4-table fan-out + multi-user isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:10:49 +08:00
1445043649 e6bb220979 feat(persistence): backfill Step 1 — workspace per user without default
For each user with NULL default_workspace_id, generate a base slug
from the email, walk past collisions/blacklist via next_available_slug,
create the workspace + owner membership, and set default_workspace_id.
Idempotent: candidates list is filtered by IS NULL, so re-running on a
populated DB is a no-op. 3 new unit tests (creation, idempotence,
blacklist-walker behaviour).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:09:16 +08:00
1445043649 ad322543d0 feat(persistence): backfill_workspace_id.py — script skeleton
argparse with --dry-run, async orchestrator that loops the 4 business
tables, and three step stubs (Step 1: users -> workspace creation;
Step 2: UPDATE tables from users.default_workspace_id; Step 3: orphan
rows -> legacy_workspace anchor at the LOCK'd nil UUID). Step bodies
are filled in T5.5-T5.7 alongside their tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:07:44 +08:00
1445043649 d3361dba59 feat(persistence): 4 business ORM models — nullable workspace_id
ORM-side mirror of alembic 0002. threads_meta gets the matching
(workspace_id, user_id, updated_at) composite index so fresh
create_all() dev databases land on the same shape Alembic produces.
Pre-existing test_alembic_default_workspace_id pins to revision
0001 explicitly since 'head' now includes 0002.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:06:08 +08:00
1445043649 4e26ec9884 test(persistence): alembic 0002 round-trip on SQLite + Postgres
Mirrors test_alembic_default_workspace_id pattern: synchronous bodies
(env.py uses asyncio.run internally), pre-PR5 schema bootstrap with the
minimal columns the migration touches, upgrade/downgrade assertions for
the workspace_id column + FK + composite index on all 4 business tables.
Postgres cases auto-skip when Docker daemon is unreachable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:04:21 +08:00
1445043649 a732697855 feat(persistence): alembic 0002 — business tables nullable workspace_id
ALTER threads_meta / runs / feedback / run_events to add nullable
workspace_id String(36) + FK to workspaces (ON DELETE CASCADE) and a
composite (workspace_id, user_id, updated_at) index on threads_meta for
the workspace-scoped thread-list query. Downgrade reverses all four FKs,
columns, and the index.

Stage 0 PR5 step 1/2; revision 0003 flips NOT NULL after the backfill.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 09:04:13 +08:00
1445043649 44f84800e9 docs(impl): PR4 implementation note + STATUS update
Records the 14-task PR4 delivery (T4.1-T4.14), final test baseline
(3136 passed / 26 skipped), key design decisions that diverged from
the plan draft (TokenPayload optional fields, ensure_default_workspace
helper shape, slug walker treating blacklist as taken), follow-up
items, and a copy-paste live smoke command list for the user to
verify make-dev end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:43:04 +08:00
1445043649 5c7753c0b8 feat(auth): lifespan _ensure_admin_user backfills missing workspace
Pre-PR4 admins have users.default_workspace_id NULL. Without this
backfill they would log in successfully but immediately bounce off the
workspace gate (T4.7) because their token cannot encode a wid. The
lifespan hook now resolves the admin user and calls
ensure_default_workspace (idempotent — no-op if the user already has
a workspace), so the post-upgrade boot makes the admin usable.

Renamed routers.auth._ensure_default_workspace → ensure_default_workspace
to make it importable across modules.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:39:09 +08:00
1445043649 91846a201e feat(auth): /auth/me returns workspaces[] with id/name/slug/role
GET /auth/me now responds with the new UserMeResponse: the existing
user fields plus default_workspace_id and a workspaces list that
joins WorkspaceRepository.list_by_user with the caller's role from
WorkspaceMembershipRepository.list_by_user. Stage 0 every user has
exactly one entry there, but the shape is forward-compatible for
Stage 2 multi-workspace memberships.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:37:29 +08:00
1445043649 634e5119e1 feat(auth): login + change_password re-issue JWTs with wid + role
After PR4 every protected request demands wid. Three JWT-issue paths
must encode it; T4.8/T4.9 covered /initialize and /register, and this
commit closes the remaining two:

- login_local calls _ensure_default_workspace (idempotent — returns
  the existing default_workspace_id when set) so users who pre-date
  PR4 are backfilled at login time, and signs the new cookie with
  wid + role='owner'.
- change_password does the same, ensuring a password change does not
  strip the workspace claim and lock the user out on their next
  request.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:35:06 +08:00
1445043649 84701730da feat(auth): /register auto-creates default workspace + owner membership
Mirrors T4.8: the regular registration path now invokes the same
_ensure_default_workspace helper so every new user lands with a
1-person workspace (owner) and a JWT carrying wid + role='owner'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:32:39 +08:00
1445043649 a657d17995 feat(auth): /initialize auto-creates default workspace + owner membership
initialize_admin now seeds a 1-person workspace immediately after the
admin user is created: WorkspaceRepository.create(name, slug, owner_id)
+ WorkspaceMembershipRepository.add(role='owner') + writes the new
workspace id back to users.default_workspace_id. The session JWT is
re-issued with wid + role='owner' so subsequent requests pass the
T4.7 workspace gate.

Mechanical pieces:
- SQLiteUserRepository row<->user mapping now includes
  default_workspace_id (sql update_user too) so the column persists.
- workspace.sql.SLUG_BLACKLIST is now public (was _SLUG_BLACKLIST) and
  the registration helper treats blacklisted slugs as "taken" so the
  walker steps past reserved names like "admin" instead of crashing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:31:53 +08:00
1445043649 a06e88d58b feat(auth): workspace slug helpers — auto_slug_from_email + next_available_slug
auto_slug_from_email maps an email's local-part to a schema-valid base
slug (lowercase, [+_.]→'-', alphanumeric-and-hyphen, clamp 32 chars,
fallback to user-{token_hex(4)} for pathological inputs).
next_available_slug walks the {base, base-2, base-3, ...} sequence
against a caller-supplied async exists_check until it lands on a free
slot, truncating base when the suffix would push past 32 chars.

Pulled forward of T4.8/T4.9 because both /auth/initialize and
/auth/register need it. Lives in app.gateway.auth (not persistence)
since "email → slug" is a registration-time concern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:27:42 +08:00
1445043649 2145d36744 feat(auth): AuthMiddleware injects workspace ContextVar from JWT wid/role
deps.get_current_user_from_request now stashes the decoded payload on
request.state.auth_payload so AuthMiddleware can populate the
workspace_context ContextVar without a second decode. Reset is paired
in the same try/finally as user_context to keep teardown atomic.

Also adds:
- auth.models.ActiveWorkspace — minimal proxy that satisfies the
  CurrentWorkspace protocol (id + role only).
- auth.models.User.default_workspace_id — surfaces the DB column added
  in T4.4 so the eventual /auth/me payload can reference it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:25:43 +08:00
1445043649 b4bef65079 feat(auth): decode_token rejects legacy 4-field JWTs as WORKSPACE_MISSING
After PR4 every JWT must carry wid (workspace_id). decode_token now
returns the new TokenError.WORKSPACE_MISSING when the signature is
valid but the payload lacks wid; expired tokens still report EXPIRED
first so /auth/refresh logic stays correct. AuthErrorCode gains a
matching WORKSPACE_REQUIRED for middleware to surface to clients.

Updates 13 existing test sites that issued tokens without wid to pass
workspace_id="ws-test" + role="owner", reflecting the new contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:22:10 +08:00
1445043649 54cb94c30f feat(auth): JWT TokenPayload accepts wid + role
TokenPayload gains optional wid (workspace_id) and role claims;
create_access_token accepts them as keyword-only args and only
encodes them when provided. Existing tokens and existing callers
keep working unchanged — the contract that protected requests
must carry wid is enforced by middleware (PR4 T4.6/T4.7), not by
the JWT type system.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:17:55 +08:00
1445043649 54762f491c feat(persistence): UserRow.default_workspace_id
Adds a nullable FK column to mirror alembic 0001. On fresh deployments
metadata.create_all() will create users with this column; existing
deployments rely on the alembic migration to add it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:15:05 +08:00
1445043649 917d8fbeaf test(persistence): alembic 0001 round-trip on SQLite + Postgres
Verifies alembic upgrade 0001 adds users.default_workspace_id (with FK
to workspaces) and that downgrade cleanly removes it. Runs on both
dialects because the migration uses op.batch_alter_table for SQLite
ALTER compatibility.

Tests are intentionally sync — alembic's command layer is sync and
env.py calls asyncio.run(); running under pytest-anyio would deadlock
on the inner event loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:11:11 +08:00
1445043649 8efbb2f9e5 feat(persistence): alembic 0001 — users.default_workspace_id
First Alembic revision for the DeerFlow application schema. Adds a
nullable users.default_workspace_id column with an ON DELETE SET NULL
FK to workspaces(id), so newly-registered users can be routed back to
their default workspace on next login without consulting memberships.

Uses op.batch_alter_table for SQLite ALTER compatibility (sqlite is
still a supported dev fallback).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:08:44 +08:00
1445043649 d98498b705 docs(impl): T4.1 — confirm alembic baseline is not needed
alembic heads/history both empty, versions/ dir empty: 0001 can be the
first revision with down_revision=None. alembic_version table will be
auto-created on first upgrade head. doctor.py needs no new check.

Also records T4 preparation grep findings on _ensure_admin_user current
behavior (open issue #3 resolved).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:07:34 +08:00
1445043649 c70c6594de docs(impl): STATUS — record LOCK sign-off and origin push
Two unblocking events on 2026-05-12: team signed off all 7 irreversible
schema decisions, and docs branch was finally pushed to origin (38 commits,
via SSH-over-443 to bypass local proxy). PR4 is now unblocked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:02:33 +08:00
1445043649 a592319e3c docs(impl): STATUS update after PR3 merge
PR3  merged 进 docs branch(7 commits, T3.1-T3.10)+ RDS live 验证通过
(11 张表含 workspaces/workspace_memberships,partial unique on owner 索引
建出来)。测试基线 3134 passed + 25 skipped + 0 failed。

下一个 PR4(auth 改造)plan 推荐 Inline 模式(跨 jwt.py/auth_middleware.py/
routers/auth.py 多文件耦合紧)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:34:05 +08:00
1445043649 dda8264057 docs(impl): PR3 implementation note + acceptance checklist
记录 Stage 0 PR3 的实施落点、LOCK 决策、跟进项与 6 commit 列表。

验收:3134 passed + 25 skipped + 0 failed(PR2 末 3087 + 47 新 ws_context/
ws_repo/membership_repo 测试 + 2 PG-only skipped)。RDS live 验证通过:
workspaces / workspace_memberships 表 + partial unique on owner 都建出来。

T3.6 (membership ORM) 提前到 T3.4-T3.5 (workspace repo) 之前,因为
WorkspaceRepository.get/list_by_user 会 JOIN memberships。

Stage 0 PR3 T3.10. PR3 完成。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:18:19 +08:00
1445043649 3313047ff1 test(workspace): partial unique on owner — Postgres twin test
新建 test_workspace_partial_unique_postgres.py,2 个 @pytest.mark.postgres
test,用 PR1 的 testcontainers postgres_url fixture 验:
  - test_partial_unique_on_owner_enforced_on_postgres:第 2 个 owner
    IntegrityError(SQLite twin 在 test_workspace_membership_repo 已覆盖)
  - test_multiple_admins_allowed_on_postgres:多个 admin/member 不触发约束

为什么 PG 单独写:sqlite_where vs postgresql_where 是两份 DDL;SQLAlchemy
能 emit 不代表 PG 真的执行。本测试 pin 两边语义一致。

本地无 docker daemon → SKIPPED;CI workflow backend-postgres-tests 会实跑。

Stage 0 PR3 T3.8。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:15:25 +08:00
1445043649 36ffe2713f feat(persistence): WorkspaceMembershipRepository + 8 unit tests
新建 backend/packages/harness/deerflow/persistence/workspace_membership/sql.py,方法:
  - add(workspace_id, user_id, role, invited_by=None) 校验 role ∈
    {owner, admin, member}(Stage 0 仅写 owner,schema 已支持其余)
  - remove(workspace_id, user_id) → bool(rowcount > 0)
  - list_by_user(user_id) 按 joined_at desc
  - list_by_workspace(workspace_id) 按 joined_at asc
  - get_role(workspace_id, user_id) → str | None
  - change_role(workspace_id, user_id, new_role) → bool

MembershipValidationError 自定义异常(role 非法)。

8 test 覆盖:
  - add → get_role round-trip
  - remove 命中/未命中返回 True/False
  - 同 workspace 第 2 个 owner 触发 IntegrityError(partial unique)
  - 同 workspace 多个 admin 不触发(Stage 2 forward compat)
  - CASCADE: 删 user 自动清理 memberships
  - list_by_user 按 joined_at desc 排序(多 workspace)
  - role 校验拒绝 'viewer'
  - change_role 命中改值 + 未命中返 False

注:owner 转让需要事务内两行原子 swap(先把现 owner 改 admin,再把新 owner
插 owner),放 PR4+ auth router 内做带权限检查的版本;本仓储不暴露
transfer_ownership 方法以保持单一职责。

Stage 0 PR3 T3.7。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:13:52 +08:00
1445043649 a40df03521 feat(persistence): WorkspaceRepository + 23 unit tests
新建 backend/packages/harness/deerflow/persistence/workspace/sql.py,方法:
  - create(name, slug, owner_id, *, workspace_id=None, status='active')
    UUID v4 自动生成;强校验 slug 格式(regex ^[a-z0-9](-?[a-z0-9])*\$ + 3-32
    长度)+ slug 黑名单(25 个保留字)+ status 枚举
  - get(workspace_id, *, user_id=AUTO) JOIN workspace_memberships 做成员校验;
    user_id=None 显式 bypass(迁移/admin)
  - get_by_slug(slug) 不带成员校验(path-based routing 用:先 slug→workspace_id
    再到 route handler 里查成员)
  - list_by_user(*, user_id=AUTO) 列 user 所属所有 workspace
  - update_status / delete platform-admin 操作,不带成员校验

WorkspaceValidationError 自定义异常(slug 格式 / 黑名单 / status)。

23 test 覆盖:
  - CRUD smoke + get_by_slug missing
  - 重复 slug → IntegrityError
  - 8 个 invalid slug pattern(短/长/大写/空格/破折号位置/连续破折号/下划线)
  - 6 个 blacklisted slug
  - status 状态机 + 非法值拒绝
  - delete CASCADE 到 memberships(SQLite FK PRAGMA 已开启)
  - get/list 成员过滤(user-A 看不见 user-B 的 workspace)
  - list user_id=None 显式 bypass

全部在 SQLite ephemeral DB 上跑(< 1s)。partial-unique 双驱动验证留给 T3.8。

Stage 0 PR3 T3.4 + T3.5。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:11:28 +08:00
1445043649 d2d2d29c34 feat(persistence): add WorkspaceMembershipRow ORM model
新建 backend/packages/harness/deerflow/persistence/workspace_membership/
{__init__.py, model.py},注册到 Base.metadata。

WorkspaceMembershipRow schema(来自 workspace-schema-design §2.2 锁定版):
  - workspace_id String(36) FK workspaces.id ON DELETE CASCADE — PK part
  - user_id String(36) FK users.id ON DELETE CASCADE — PK part
  - role String(16) NOT NULL(Stage 0 仅写 'owner',schema 允许 'admin'/'member'
    为 Stage 2 RBAC 准备)
  - invited_by String(36) FK users.id ON DELETE SET NULL nullable(Stage 2
    invitation 流程用)
  - joined_at DateTime(tz=True)

索引:
  - idx_workspace_memberships_user (user_id, workspace_id) — 倒查索引让
    /auth/me 列 user 所属 workspaces 走索引
  - idx_one_owner_per_workspace partial UNIQUE on workspace_id WHERE role='owner'
    —— 一 workspace 严格 1 个 owner;sqlite_where + postgresql_where 双驱动
    并存(实测两边都识别)

Stage 0 PR3 T3.6(提前到 T3.4-T3.5 之前;WorkspaceRepository.list_by_user
等会 JOIN 这张表)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:09:19 +08:00
1445043649 8323bf68d2 feat(persistence): add WorkspaceRow ORM model
新建 backend/packages/harness/deerflow/persistence/workspace/{__init__.py, model.py}
+ 在 persistence.models.__init__ 注册 WorkspaceRow 让 Base.metadata.create_all
能自动建表。

WorkspaceRow schema(来自 workspace-schema-design §2.1 锁定版):
  - id String(36) PK(UUID v4 字符串,与 users.id 对齐)
  - name String(64) NOT NULL(显示名)
  - slug String(32) NOT NULL UNIQUE(URL 标识,正则 ^[a-z0-9](-?[a-z0-9])*$)
  - status String(16) default 'active'(active/suspended/deleted)
  - owner_id String(36) FK users.id ON DELETE RESTRICT(删 owner 时阻拦,
    必须先转让所有权)
  - created_at / updated_at DateTime(tz=True)

每列 + 表都带中文 comment(沿用 commit 9ff79055 的 convention)。

Stage 0 PR3 T3.3。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:07:30 +08:00
1445043649 f63089aea8 feat(runtime): add workspace_context module + tests
新建 backend/packages/harness/deerflow/runtime/workspace_context.py,仿
user_context.py 的 API 形态:

  - CurrentWorkspace Protocol(要求 .id: str + .role: str)
  - _current_workspace ContextVar + set/reset/get/require
  - DEFAULT_WORKSPACE_ID = "default" + get_effective_workspace_id(fallback
    友好,不抛错;用于文件系统路径)
  - AUTO 哨兵 + resolve_workspace_id 三态(AUTO/str/None)

与 user_context 的区别:CurrentWorkspace 额外要求 .role 字段,让 Protocol 同时
约束"workspace 是哪个"和"caller 在该 workspace 内的角色"(Stage 0 只见 'owner',
Stage 2 RBAC 打开 admin/member)。

backend/tests/test_workspace_context.py 16 个 test,覆盖:
  - 4 个 set/reset/require 行为
  - 3 个 Protocol structural check(接受 .id+.role / 拒少 .role / 拒少 .id)
  - 4 个 get_effective_workspace_id(含 UUID → str 强转)
  - 5 个 resolve_workspace_id 三态(AUTO/AUTO 无 ctx 抛错/explicit str/explicit
    None/AUTO 强转 str)

Stage 0 PR3 T3.1 + T3.2。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:04:49 +08:00
1445043649 cbbb83a706 docs(impl): STATUS update after PR2 live verification on RDS
PR1 + PR2 已 live 验证:远程 Aliyun RDS (PostgreSQL 17.9) 上 9 张表
(DeerFlow 5 + LangGraph 4) 全部 create_all 成功。

记 3 个 PR2 follow-up commits(不在原 plan task list 但 live 必需):
  - testcontainers image 16→17 对齐 RDS 大版本
  - serve.sh auto-add --extra postgres 防 uv sync 卸 asyncpg
  - async_provider 剥 +asyncpg dialect 前缀让 LangGraph saver 能解析 URL

更新"一句话状态" + 标 PG live 验证(区分 testcontainers 路径仍未实跑)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 18:01:22 +08:00
1445043649 39f8e117e8 fix(checkpointer): strip SQLAlchemy dialect prefix before passing to LangGraph
DeerFlow 用同一个 \`postgres_url\` 喂两条路径:
  - SQLAlchemy 引擎(需 \`postgresql+asyncpg://\`)
  - LangGraph AsyncPostgresSaver(psycopg 直连,需 libpq 风格 \`postgresql://\`)

PR2 把 .env / config.example.yaml 默认 URL 改成 \`+asyncpg\` 形态后,LangGraph
saver from_conn_string 会被 psycopg parser 直接抛 ProgrammingError
'missing "=" after "postgresql+asyncpg://..."'。

修:在 async_provider.py 传给 AsyncPostgresSaver 之前用正则剥掉
\`postgresql+\\w+://\` → \`postgresql://\`。一行修复,让同一个 URL 同时满足
SQLAlchemy 和 LangGraph 两条路径。

实测:make dev-daemon 起 gateway 成功,远程 RDS 上 9 张表(DeerFlow 5 +
LangGraph 4)全部 create_all 出来。

Stage 0 PR2 follow-up(不在原 plan task list 内,是 live make dev 触发的)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 17:54:57 +08:00
1445043649 bdef6c6b9f fix(serve): auto-install postgres extras when config.yaml selects postgres
scripts/serve.sh 在 \`uv sync --quiet\` 前 grep config.yaml 探测
\`database.backend: postgres\`;若是则追加 \`--extra postgres\`,否则保持原状。

修一个 PR1+PR2 没覆盖到的隐患:serve.sh 每次 dev 启动都跑 uv sync,会把
之前手动 \`uv sync --extra postgres\` 装的 asyncpg 卸掉,导致 gateway
启动时 ImportError。

Stage 0 PR2 follow-up(不在原 plan task list 内,是 live 验证时发现)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 17:35:23 +08:00
1445043649 7f17cb8fee chore(test): bump testcontainers Postgres to 17-alpine
\`make doctor\` 实测远程 Aliyun RDS = PostgreSQL 17.9。把
backend/tests/fixtures/postgres.py 的 testcontainers 镜像从
postgres:16-alpine 调到 postgres:17-alpine,与生产对齐。

同步把 STATUS.md "RDS 大版本对齐" 项标 done。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 17:29:16 +08:00
1445043649 ad22242ecc docs(impl): add Stage 0 progress dashboard (STATUS.md)
新增 docs/multi-tenant-redesign/03-impl/STATUS.md 作为 Stage 0 唯一的"现在
到哪了"权威来源:

  - 8 PR 状态表(PR0-PR2  merged,PR3-PR8 🟡 pending)
  - 用户必须跟进的事(live PG 实跑 / RDS 大版本对齐 / push origin / LOCK
    review / 远程 RDS 密码轮换)
  - 跳过/推迟的子任务(T1.10 docker-pending、T2.7 acked、T2.8 optional
    skipped、T2.9 backend/CLAUDE.md 待补)
  - 即将遇到的开放问题(PR4 alembic baseline / _ensure_admin_user 现状 /
    PG 大版本)
  - PR3 模式选择 trade-off(Inline vs Subagent-Driven)
  - 维护规则:完成 PR 后必更新;新 session 首先读本文件

填补 plan/impl notes 之外的空白:plan 是静态的,impl notes 是 per-PR
快照;STATUS 是跨 PR 的执行状态视图。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 16:44:09 +08:00
1445043649 1112a1971b docs(impl): PR2 implementation note + acceptance checklist
记录 Stage 0 PR2 的实施落点、关键 LOCK 决策、跟进项与 7 commit 列表。
验收:3087 passed + 23 skipped + 0 failed(vs PR1 基线 3085 + 2 新
default-backend 测试)。

Stage 0 PR2 T2.10. PR2 完成。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:23:16 +08:00
1445043649 c53295dfee docs(readme): add Database backend section in Quick Start
Quick Start 加 Step 3 "Database backend (Stage 0+ defaults to Postgres)":
  - 说明 config.example.yaml 默认 postgres + DATABASE_URL 写 .env
  - 给本地 dev 起 docker compose postgres 一行命令
  - 提示 make doctor / make dev preflight 行为
  - <details> 折叠 SQLite fallback 说明(offline dev 用)

Stage 0 PR2 T2.9(README 部分;backend/CLAUDE.md follow-up)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:20:26 +08:00
1445043649 745a33e05d chore: T2.7 acknowledgement (covered by T1.8)
Plan T2.7 = "setup_wizard.py 推荐 postgres,引导填 DATABASE_URL"。
T1.8 commit eae01901 已经实现:

  - 加 "Use Postgres? (y = postgres, n = sqlite)" 问答,默认 y
  - 选 y 时引导填 DATABASE_URL,可留空稍后填 .env
  - writer.build_minimal_config(database_backend='postgres') 输出
    database.backend=postgres + postgres_url=\$DATABASE_URL

无新代码改动;本 commit 仅为 task tracking 完整性。

Stage 0 PR2 T2.7(acknowledged-only)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:17:51 +08:00
1445043649 83b680b2ea feat(check): postgres preflight in scripts/check.py
新增 check_postgres_preflight() —— make dev 链 (check.py → serve.sh) 的最后一道
preflight:
  - config.yaml 不存在 → silent skip(让 setup_wizard 引导)
  - backend != postgres → silent skip
  - DATABASE_URL 未设 → FAIL with hint
  - postgres 设了但 host:port 3s socket 不通 → FAIL with 启 docker 提示
  - 通则 OK + 显示 host:port

不在 serve.sh 里加:Makefile 已经把 check.py 串在 serve.sh 之前,FAIL 会
自然阻断启动;避免 bash + python 两处实现 PG 探测。

doctor.py 的 check_database 是事后诊断(make doctor);本 check 是事前
preflight(make dev/start)—— 互补。

Stage 0 PR2 T2.6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:17:19 +08:00
1445043649 3e62a0f6ef feat(docker): gateway depends_on postgres healthcheck (dev only)
dev compose gateway 段加 depends_on: { postgres: { condition: service_healthy } },
确保 postgres 通过 pg_isready 后 gateway 才启 uvicorn。

prod compose 不动:production 用远程 RDS(不在 compose 内),无 postgres
service 可 depend;docker-compose.yaml 不加此 depends_on。

YAML 通过 docker compose config 校验。

Stage 0 PR2 T2.5.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:13:44 +08:00
1445043649 d312bdf968 test(config): pin default postgres backend + sqlite regression
新建 backend/tests/test_default_database_backend.py 两个测试:

  - test_explicit_sqlite_backend_still_works (T2.3)
    显式 database.backend=sqlite 仍生效;防 PR2 改默认后 SQLite 用户
    悄无声息 regression
  - test_config_example_default_backend_is_postgres (T2.4)
    直接读 on-disk config.example.yaml,断言 database.backend == 'postgres'
    且 postgres_url == '\$DATABASE_URL'(不允许硬编码凭据)

Stage 0 PR2 T2.3 + T2.4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:12:28 +08:00
1445043649 7d3d3560ad feat(env): activate DATABASE_URL with local-dev default
.env.example DATABASE_URL 行从注释改为活值(指向 docker compose 起的本地
postgres)。同时加 RDS 示例注释作为参考。psycopg2 风格 URL 升级到
postgresql+asyncpg:// 与 SQLAlchemy 异步 dialect 对齐。

Stage 0 PR2 T2.2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 04:24:32 +08:00
1445043649 404135a16a feat(config): default database backend to postgres
config.example.yaml database 段:postgres 成为活跃默认(postgres_url=\$DATABASE_URL),
SQLite 改为注释掉的 fallback 块(offline dev 用)。bump config_version 9→10。

为什么默认改 postgres:Stage 0+ 已要求 ALTER 4 张表加 workspace_id;
SQLite 加列后再迁 PG 是返工。Stage 0 没有生产数据,迁移阻力最小。

Stage 0 PR2 T2.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 23:03:52 +08:00
1445043649 85a14f4c05 docs(impl): PR1 implementation note + acceptance checklist
记录 Stage 0 PR1 的实施落点、关键 LOCK 决策、fixture 用法示例、跟进项与
10 commit 列表。验收:3085 passed + 23 skipped + 0 failed(无 regression),
PG smoke 在 docker daemon 起来后实跑(CI workflow 已配)。

Stage 0 PR1 T1.10. PR1 完成。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:58:21 +08:00
1445043649 a33b46b4af ci(postgres): add Postgres workflow running @pytest.mark.postgres tests
新增 .github/workflows/backend-postgres-tests.yml,独立于现有
backend-unit-tests workflow:
  - GitHub runner (ubuntu-latest) 自带 docker daemon
  - testcontainers 在 runner 上起 postgres:16-alpine 容器
  - uv sync --extra postgres-test 拉 testcontainers + asyncpg + psycopg
  - pytest -m postgres 仅跑 PG 标签的 4 个 smoke test (+ 后续 PR 增量)

为什么独立 workflow 而不是合到 backend-unit-tests:
  - 现有 fast 测试不受 PG container 启动延迟影响
  - 可独立 fail-soft(早期 Stage 0 rollout 期间需要时加 continue-on-error)
  - fork 用户不强制承担 docker infra 成本

Stage 0 PR1 T1.9.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:55:53 +08:00
1445043649 eae0190184 feat(wizard): add Postgres backend question to setup wizard
setup_wizard.py 在 Step 3 (Execution) 之后新增一个 Database 问答:
"Use Postgres? (y = postgres, n = sqlite)",默认 y(Stage 0+ 推荐 PG)。
选 y 时引导填 DATABASE_URL(可留空稍后写 .env);DATABASE_URL 进 .env,
config.yaml 写入 database.backend=postgres + postgres_url=\$DATABASE_URL。

writer.py build_minimal_config 加 database_backend 参数,postgres 时
覆盖 base_config 的 database 段;默认 sqlite 时沿用 base_config 行为
(继承 config.example.yaml 的 sqlite_dir 等)。

minimal pattern:不新建 wizard/steps/database.py,inline 在 main 里加 1
问答 + writer 加 1 参数。后续如果需要更复杂数据库选项再升为完整 step 模块。

Stage 0 PR1 T1.8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:55:19 +08:00
1445043649 e6f5ba53bd feat(doctor): add Database section probing configured backend
scripts/doctor.py 新增 check_database():

  - sqlite → OK + 显示 sqlite_dir
  - memory → WARN(数据非持久化)
  - postgres → 解析 database.postgres_url 中的 \$DATABASE_URL,asyncpg
    实际连接 + SELECT version() → OK with server version;连接失败时
    FAIL 给出可执行 fix(启 docker compose postgres / 检查 DATABASE_URL)
  - 未知 backend → WARN

Database section 插在 LLM Provider 与 Sandbox 之间。SQLite 部署不会触发
PG 探测(不破坏现有 dev 体验)。

Stage 0 PR1 T1.7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:52:26 +08:00
1445043649 8f480cd76f test(persistence): postgres smoke tests for init_engine + ThreadMetaRepo
新建 backend/tests/test_postgres_smoke.py,4 个 test 用 postgres_url fixture:

  - test_postgres_url_creates_isolated_database (T1.4)
    asyncpg 直连验证 fixture 落到 test_<hex> DB
  - test_postgres_url_isolates_between_tests (T1.4)
    手动起第 2 个 DB 比对,证明每 test 隔离
  - test_init_engine_postgres_creates_tables (T1.5)
    init_engine('postgres', postgres_url) 跑完后 information_schema
    出现 5 张现有表(users/threads_meta/runs/feedback/run_events)
  - test_thread_meta_repo_postgres_round_trip (T1.6)
    ThreadMetaRepository.create + get 完整 round-trip,验证 PG 上仓储行为
    与 SQLite 对齐

全部 @pytest.mark.postgres 门控;Docker 不可用时由 fixture 跳过。本次本地
docker daemon 未起,4 测试 SKIP;CI workflow(T1.9)将带 docker service
触发实跑。

Regression:跑全套 `pytest tests/` 3085 passed + 23 skipped + 0 failed
(基线 3086 passed + 18 skipped;diff = 4 新 postgres skip + 1 env 相关
live test 偶发 skip)。

Stage 0 PR1 T1.4 + T1.5 + T1.6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:50:10 +08:00
1445043649 31361e2dcf test(fixtures): add testcontainers Postgres fixture for Stage 0
新建 backend/tests/fixtures/postgres.py,提供两层 fixture:
  - postgres_container (session 级):用 PostgresContainer 起 postgres:16-alpine
  - postgres_url (function 级):每 test 一个 ephemeral DB,teardown 时
    pg_terminate_backend 清掉残连后 DROP DATABASE

为什么 per-DB 而不是 per-schema:asyncpg + SQLAlchemy 不通过 URL 传 search_path,
per-DB 一次 ~50ms 开销可接受,让 test 代码不感知 schema。

Docker 不可用时 fixture 自动 pytest.skip 而非 error,dev 环境无 Docker
仍能跑其余 3086 个测试。

注册 pytest mark `postgres`、把 tests/ 加 sys.path 让 pytest_plugins
按 `fixtures.postgres` 路径解析(不加 tests/__init__.py 避免干扰
现有 pytest 发现行为)。

Stage 0 PR1 T1.3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:44:02 +08:00
1445043649 ae4ea46be2 feat(docker): add postgres service to dev compose
新增 postgres:16-alpine service + named volume + healthcheck,作为 Stage 0+
默认 DB backend。生产可通过 DATABASE_URL 指向远程 RDS 时跳过此 service。
端口 / 用户 / 密码 / 库名都走 env var 覆盖(POSTGRES_USER/PORT/...
默认 deerflow/5432/deerflow_dev)。

YAML 通过 `docker compose config` 校验;live healthcheck 待 docker daemon 起后由
T1.3 testcontainers 路径覆盖。

Stage 0 PR1 T1.2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:40:12 +08:00
1445043649 fab85b14a6 build(deps): add postgres-test optional extra for testcontainers
新增 postgres-test extra(harness + backend),引入 testcontainers[postgres]
4.14.2 用于 Stage 0 PR1 的 PG fixture。复用现有 postgres extra 的
asyncpg/psycopg/langgraph-checkpoint-postgres,pytest-asyncio 沿用 dev group。

Stage 0 PR1 T1.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:38:50 +08:00
1445043649 a74b88a45c docs(superpowers): add Stage 0 multi-tenant foundation master plan
8 PR 串成的 stage-level 拆解(Postgres 切换 + workspaces schema + auth
扩字段 + 入口路由强校验 + CI boundary + headless schema 预留)。每个 PR
含 commit-sized TDD task list 和验收清单。执行时各 PR 独立 review
checkpoint。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:33:25 +08:00
1445043649 89fe54cc07 docs(multi-tenant): 加入汇总索引、跨文档一致性修订、Postgres 切换前移到 Stage 0
* 新增 README.zh-CN.md 汇总索引:ADR 状态表 + Stage 0-4 业务目标 / 技术路径 /
  验证方式 + Stage↔ADR 对照矩阵 + 不可逆决策一览 + 用语映射 + FAQ + 阅读路径
* 新增 workspace-schema-design.zh-CN.md(Stage 0 schema 锁定版)一并入库
* 7 份 ADR 顶部加"代码命名"映射行(tenant_id ↔ workspace_id)
* ADR-002 §1 加分期落地提示,明确 K8s 推迟到 Stage 3
* ADR-005 §5"第 1/2 阶段"补出与 rollout Stage 2/3 的映射
* ADR-007 §4 加 /api/v1/ 反向链接;§8 加 tid → wid 字段名映射
* headless-api §0/§7 把"SaaS + on-prem 双主线"改为"SaaS 主线、schema 兼容 on-prem"
* phased-rollout 去除重复的"Go/No-Go 进入 Stage 2"段

Postgres 切换从 Stage 1 提前到 Stage 0:Stage 0 已要 ALTER 4 张表加
workspace_id,先 SQLite 再 PG 是纯返工;Stage 0 没有生产数据,迁移阻力最小。
同步调整 phased-rollout / headless-api / phase-0-plan / workspace-schema-design /
README 中的时间盒(Stage 0: 3-4→4-5 周;Stage 1: 10-15→8-13 周)、不可逆决策
清单、PR 顺序、轨道前置依赖。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:39:58 +08:00
1445043649 9ff790554d docs(persistence): 给 ORM 表/字段补充中文注释
通过 SQLAlchemy 的 comment= 给 5 张持久化表(users / threads_meta / runs /
run_events / feedback)的所有字段以及表本身加上中文注释,便于读代码、
生成文档与未来切到 Postgres 时直接落库为 COMMENT ON。

SQLite 引擎本身不支持 COMMENT ON,运行时不会改变 .schema 输出。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 18:50:17 +08:00
1445043649 ecc9339ede docs(multi-tenant): 加入 Headless API 改造轨道(Pattern A/B)+ Stage 1 双轨修订
新增 headless-api-track 文档,把 DeerFlow 作为后端服务的两种集成
pattern 全部纳入设计:

- Pattern A(业务 backend 代理):业务系统 backend 用 API key 调,
  浏览器走他们的 backend。覆盖 IM channels + server-to-server 集成。
- Pattern B(自研 web 浏览器直连):业务 backend 调
  /api/v1/auth/exchange-token 换 5-15 min 短期 JWT → 浏览器拿 JWT 直连
  含 SSE。核心 4 件事:exchange-token endpoint、ServiceTokenAuthBackend
  (AuthMiddleware 第三条路径)、workspaces.allowed_origins + CORS
  中间件、SSE 跨域验证。
- 不做 widget / iframe(纯 API)。

数据模型:service_accounts / api_keys / external_users 表(schema 在
Stage 0 末加上不阻塞);身份模式三态(collapsed / external_passthrough
/ both)按 endpoint 分支。

Stage 1 改为三轨并行:付费 SaaS / Pattern A / Pattern B;时间盒从
6-10 周延到 10-15 周。Pattern B 依赖 Pattern A 完成,建议 Stage 1
末 1-2 周做。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 23:29:46 +08:00
1445043649 fd8d0d637d docs(multi-tenant): 加入 Stage 0 代码地图
把 Stage 0 必做项落到具体文件 + 行号,覆盖 7 个子系统:
auth、user 数据模型、thread 入口路由、ThreadDataMiddleware + 路径系统、
仓储访问模式、setup / 注册流程、CSRF + ContextVar 注入。

文档结构:每节有【关键文件 + 行号】、【关键函数 / 类】、【当前数据流】、
【Stage 0 改动锚点】;外加改动影响面总览(每条 Stage 0 必做项映射到
具体代码位置)、不可逆决策落点、推荐阅读顺序(11 个文件,1.5-2.5h
粗读)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 23:29:27 +08:00
1445043649 35ae97d0a6 docs(multi-tenant): Stage 2 加入 per-user skill 覆盖与 config 设计
补充团队 workspace 内不同成员的 skill 区分模型:

- 启用状态:workspace 级默认 + per-user 覆盖
  (final_enabled = user_override ?? workspace_default)
- skill 私有配置(API key 等):per-user 强制隔离 + KMS 加密
- 上传权限:仅 owner / admin,成员只能 enable/disable + 填自己 config

PR 顺序新增第 8 步,依赖 RBAC + KMS 已就位。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 22:47:05 +08:00
1445043649 e78ed687a6 docs(multi-tenant): 加入按规模分期的落地方案(02-rollout)
基于已收敛的目标客户画像(个人用户为主 + 小团队 / 中心化 SaaS /
freemium)把 7 份 ADR 拆成 5 期落地:

- Stage 0:workspace 模型立起来 + auth 收紧(不可逆决策集中在此)
- Stage 1:Postgres + Quota 必落(freemium 不上 quota = 信用卡递给攻击者)
- Stage 2:DeerFlow 表 RLS、KMS、ObjectStorage S3、付费分层
- Stage 3:K8s sandbox + BYO key + audit DB 拆分
- Stage 4:SSO / custom domain / per-tenant DB(按 enterprise 客户合同驱动)

关键取舍:① Stage 0 末就把 workspace_id 列加到 SQLite,避免 Stage 1
切 Postgres 时再补;② Quota 比 ADR-003 原稿提前一档到 Stage 1;③ K8s
sandbox 推到 Stage 3,AioSandbox + 出网白名单 + 资源限额撑到几千用户。

时间盒:最小可付费 4 个月(Stage 0+1)/ 风险可控增长 8 个月(+Stage 2)
/ 全功能 14 个月(+Stage 3)。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 22:28:55 +08:00
1445043649 8bdd308ea0 docs(multi-tenant): 据审计对齐 ADR-002/003/004/005 与 phase-0 计划
非 spike 驱动的对齐改动:
- ADR-002/004/005:头部加现状提示 + 链接审计报告(沙箱出网/资源缺位、
  token_version 已存在 MembershipCache 全新建、ObjectStorage/7 表/KMS
  全部从 0 起)
- ADR-003 LLM 计费:修正 TokenUsageMiddleware 当前只 log 不持久化的描述;
  补充 create_chat_model sync→async 改造的连带影响说明
- phase-0 计划:新增 §3.5 底座先行(Postgres 测试夹具 / ObjectStorage
  Protocol / KMS 抽象 3 件并行做),时间盒 2 → 3 周;ADR-006/007 摘要
  对齐;DoD 加底座骨架检查项

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 21:03:32 +08:00
1445043649 27c4f14233 docs(multi-tenant): 加入 ADR 审计 + spike,并据其修订 ADR-001/006/007
新增两份评审产出物:
- adr-vs-code-audit:7 份 ADR 与现状代码的差异核对,标注每条假设是否成立
- adr-spike-langgraph-postgres:实测 langgraph-checkpoint-postgres==3.0.5
  注入能力,确认不存在 connection_factory 参数,且 psycopg_pool 自带的
  configure callback 不是 per-acquire hook

据 spike 与审计修订三份 ADR:
- ADR-001 数据隔离:LangGraph 表改为应用层强校验 + threads_meta unique
  约束兜底(不再挂 RLS、不 ALTER 表);hook 点从 AssistantsCompat 修正
  为 threads.py + thread_runs.py
- ADR-006 运行时与渠道:§2.1 完全重写为应用层强校验;MCP OAuth token
  从"无持久化进程内存"直接做加密 DB;channel store binding 改为新建
  channel_bindings 表
- ADR-007 路由与前端:删除 Better Auth 假设(前端实际无此依赖),改为
  扩展现有 auth/jwt.py TokenPayload 加 tid/role 字段

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 21:03:16 +08:00
1445043649 dce5e9598b docs(multi-tenant): add ADRs and phase-0 plan for multi-tenant redesign
Add 7 ADRs and a phase-0 plan covering the multi-tenant redesign of
DeerFlow, plus an architecture-overview snapshot of the current state.

ADRs:
- 001 data isolation: row-level tenant_id + Postgres RLS, including
  LangGraph-owned checkpoint tables (subquery RLS or column upgrade path).
- 002 sandbox isolation: K8s namespace + gVisor + NetworkPolicy default-
  deny, threat model and pod spec defaults.
- 003 LLM key & billing: hybrid platform/BYO with pessimistic reservation
  to handle the "ghost token" overflow on the last call, plus a usage
  category split for memory/title/summarization charges.
- 004 RBAC: two-level (owner/admin/member), JWT-with-role + 30s LRU
  cache for reads, strict DB lookup for sensitive writes, token_version
  bump as the single revocation path.
- 005 storage topology: Postgres (structured) + S3-compatible object
  store (large objects) + emptyDir (ephemeral); explicit treatment of
  the extensions_config.json migration's downstream effects.
- 006 runtime & channel tenancy: per-tenant MCP cache, dual-track skills
  loader, sandbox provider routing by namespace, internal LLM call
  billing, IM channel-to-tenant binding model.
- 007 routing & frontend: path-slug URL form, JWT-only API auth,
  TenantProvider, hard-reload tenant switch, Better Auth integration.

These docs are decision records; no code changes are included.
2026-05-08 23:45:33 +08:00
204 changed files with 24060 additions and 524 deletions
+5 -2
View File
@@ -38,8 +38,11 @@ INFOQUEST_API_KEY=your-infoquest-api-key
# GitHub API Token # GitHub API Token
# GITHUB_TOKEN=your-github-token # GITHUB_TOKEN=your-github-token
# Database (only needed when config.yaml has database.backend: postgres) # Database (Stage 0+ default; required when config.yaml has database.backend: postgres)
# DATABASE_URL=postgresql://deerflow:password@localhost:5432/deerflow # Local dev — start with: docker compose -f docker/docker-compose-dev.yaml up -d postgres
DATABASE_URL=postgresql+asyncpg://deerflow:deerflow_dev@localhost:5432/deerflow
# Remote RDS example:
# DATABASE_URL=postgresql+asyncpg://USER:PASSWORD@HOST:5432/DBNAME
# #
# WECOM_BOT_ID=your-wecom-bot-id # WECOM_BOT_ID=your-wecom-bot-id
# WECOM_BOT_SECRET=your-wecom-bot-secret # WECOM_BOT_SECRET=your-wecom-bot-secret
@@ -0,0 +1,56 @@
name: Postgres Tests
# Stage 0 PR1 · runs the @pytest.mark.postgres subset against a real
# Postgres backend (testcontainers spawns postgres:16-alpine on the
# GitHub runner's docker daemon).
#
# Kept as a separate workflow from `Unit Tests` so:
# - the existing fast unit test loop is unchanged
# - PG tests can fail-soft during early Stage 0 rollout if needed
# (set continue-on-error: true on the run step)
# - infra cost is opt-in for forks
on:
push:
branches: [ 'main' ]
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
concurrency:
group: postgres-tests-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
backend-postgres-tests:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v7
- name: Verify Docker daemon (testcontainers needs it)
run: docker version
- name: Install backend dependencies (with postgres-test extra)
working-directory: backend
run: uv sync --group dev --extra postgres-test
- name: Run @pytest.mark.postgres tests
working-directory: backend
# -m postgres selects only postgres-tagged tests; testcontainers
# spins up postgres:16-alpine as the fixture's session-scoped
# container. ~5-10s container startup + per-test DB CREATE/DROP.
run: PYTHONPATH=. uv run pytest -m postgres -v
+5
View File
@@ -31,6 +31,7 @@ extensions_config.json
# IDE # IDE
.idea/ .idea/
.vscode/ .vscode/
.qoder/
# Coverage report # Coverage report
coverage.xml coverage.xml
@@ -60,3 +61,7 @@ config.yaml.bak
/frontend/playwright-report/ /frontend/playwright-report/
.gstack/ .gstack/
.worktrees .worktrees
skills/gstack
skills/superpowers
CLAUDE.md
AGENTS.md
+9
View File
@@ -31,6 +31,7 @@ help:
@echo " make start-daemon - Start prod services in background (daemon mode)" @echo " make start-daemon - Start prod services in background (daemon mode)"
@echo " make stop - Stop all running services" @echo " make stop - Stop all running services"
@echo " make clean - Clean up processes and temporary files" @echo " make clean - Clean up processes and temporary files"
@echo " make migrate-paths - Migrate legacy users/ tree into workspaces/ layout (DRY_RUN=1 to preview)"
@echo "" @echo ""
@echo "Docker Production Commands:" @echo "Docker Production Commands:"
@echo " make up - Build and start production Docker services (localhost:2026)" @echo " make up - Build and start production Docker services (localhost:2026)"
@@ -144,6 +145,14 @@ clean: stop
@-rm -rf logs/*.log 2>/dev/null || true @-rm -rf logs/*.log 2>/dev/null || true
@echo "✓ Cleanup complete" @echo "✓ Cleanup complete"
# Lift legacy per-user paths into the per-workspace layout (PR6).
# Pass DRY_RUN=1 to log the migration plan without writing.
# DEFAULT_WORKSPACE=<wid> claims un-assigned users (defaults to legacy_workspace).
migrate-paths:
@cd backend && PYTHONPATH=. uv run python scripts/migrate_paths_to_workspace.py \
$(if $(filter 1 true,$(DRY_RUN)),--dry-run) \
$(if $(DEFAULT_WORKSPACE),--default-workspace $(DEFAULT_WORKSPACE))
# ========================================== # ==========================================
# Docker Development Commands # Docker Development Commands
# ========================================== # ==========================================
+32
View File
@@ -202,6 +202,38 @@ That prompt is intended for coding agents. It tells the agent to clone the repo
</details> </details>
3. **Database backend (Stage 0+ defaults to Postgres)**
`config.example.yaml` ships with `database.backend: postgres` and `postgres_url: $DATABASE_URL`. Set `DATABASE_URL` in `.env`:
```bash
DATABASE_URL=postgresql+asyncpg://deerflow:deerflow_dev@localhost:5432/deerflow
```
Start the local Postgres dev container:
```bash
docker compose -f docker/docker-compose-dev.yaml up -d postgres
```
Or point `DATABASE_URL` at a remote RDS / Cloud SQL instance.
`make doctor` will report the configured backend, attempt an asyncpg connection, and surface actionable fix hints. `make dev` preflights Postgres reachability before starting services and aborts if `DATABASE_URL` is unreachable.
<details>
<summary>Offline dev (SQLite fallback)</summary>
If you prefer no Postgres, edit `config.yaml`:
```yaml
database:
backend: sqlite
sqlite_dir: .deer-flow/data
```
SQLite is preserved as a valid backend for offline development. RLS / multi-node features (Stage 2+) require Postgres.
</details>
### Running the Application ### Running the Application
#### Deployment Sizing #### Deployment Sizing
+186 -74
View File
@@ -1,20 +1,59 @@
# 🦌 DeerFlow - 2.0 # 🦌 DeerFlow - 2.0 · 多租户改造
[English](./README.md) | 中文 | [日本語](./README_ja.md) | [Français](./README_fr.md) | [Русский](./README_ru.md) [English](./README.md) | 中文 | [日本語](./README_ja.md) | [Français](./README_fr.md) | [Русский](./README_ru.md)
[![Python](https://img.shields.io/badge/Python-3.12%2B-3776AB?logo=python&logoColor=white)](./backend/pyproject.toml) [![Python](https://img.shields.io/badge/Python-3.12%2B-3776AB?logo=python&logoColor=white)](./backend/pyproject.toml)
[![Node.js](https://img.shields.io/badge/Node.js-22%2B-339933?logo=node.js&logoColor=white)](./Makefile) [![Node.js](https://img.shields.io/badge/Node.js-22%2B-339933?logo=node.js&logoColor=white)](./Makefile)
[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-default-4169E1?logo=postgresql&logoColor=white)](./config.example.yaml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
<a href="https://trendshift.io/repositories/14699" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14699" alt="bytedance%2Fdeer-flow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> DeerFlow**D**eep **E**xploration and **E**fficient **R**esearch **Flow**)是一个开源的 **super agent harness**:它把 **sub-agents**、**memory**、**sandbox** 组织在一起,再配合可扩展的 **skills**,让 agent 可以完成几乎任何事情。
> 2026 年 2 月 28 日,DeerFlow 2 发布后登上 GitHub Trending 第 1 名。非常感谢社区的支持,这是大家一起做到的。
DeerFlow**D**eep **E**xploration and **E**fficient **R**esearch **Flow**)是一个开源的 **super agent harness**。它把 **sub-agents**、**memory** 和 **sandbox** 组织在一起,再配合可扩展的 **skills**,让 agent 可以完成几乎任何事情。 > [!IMPORTANT]
> **本分支(`docs/multi-tenant-redesign`)是 DeerFlow 的多租户改造主线。** 在保留原有 super agent harness 全部能力的基础上,引入了 **workspace 租户模型、Postgres 为默认后端、带 `workspace_id` 的行级数据隔离、扩展后的 JWT(带 `wid`/`role`)、per-workspace 的文件系统布局、Headless API schema 底座,以及 `apps/` 上层应用脚手架**。下面的 [多租户改造](#多租户改造本分支主线) 一节是阅读本仓库的入口。
> [!NOTE]
> **DeerFlow 2.0 是一次彻底重写。** 它和 v1 没有共用代码。如果你要找的是最初的 Deep Research 框架,可以前往 [`1.x` 分支](https://github.com/bytedance/deer-flow/tree/main-1.x)。
https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18 https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18
> [!NOTE] ## 多租户改造(本分支主线)
> **DeerFlow 2.0 是一次彻底重写。** 它和 v1 没有共用代码。如果你要找的是最初的 Deep Research 框架,可以前往 [`1.x` 分支](https://github.com/bytedance/deer-flow/tree/main-1.x)。那里仍然欢迎贡献;当前的主要开发已经转向 2.0。
DeerFlow 原本面向"单机可信环境、单用户"。本分支按"以个人用户为主、少量小团队,统一只有 **workspace** 概念(个人 = 1 人 workspace),中心化 SaaS 为主线"的目标,把租户能力分 **Stage 04** 渐进落地。**目前 Stage 0(底座)工程层面已全部合入。**
### Stage 0 已落地的能力
| 能力 | 说明 | 落点 |
|---|---|---|
| **workspace 租户模型** | 新增 `workspaces` + `workspace_memberships` 两张表与仓储;每个用户注册时自动建 1 人 workspaceowner=自己);slug 唯一、黑名单校验。 | `persistence/workspace*` |
| **Postgres 成为默认后端** | `config.example.yaml` / `.env.example` / `make dev` / `make doctor` 默认走 Postgres,与生产对齐;SQLite 保留为离线开发兜底。 | 见 [数据库后端](#数据库后端) |
| **行级数据隔离** | 4 张业务表(`threads_meta` / `runs` / `run_events` / `feedback`)加 `workspace_id` 列(`NOT NULL` + `UNIQUE(workspace_id, thread_id)` 兜底),入口路由按 `(workspace_id, thread_id)` 强校验,跨 workspace 访问必 404。 | `persistence/*`、Gateway routers |
| **扩展后的 JWT** | TokenPayload 一次到位为 `{sub, wid, role, exp, iat, ver}`;登录 / 改密 / `/auth/me` 全部带上 workspace 与角色;旧版 4 字段 JWT 被识别为 `WORKSPACE_MISSING` 并要求重登。 | `app/gateway/auth/` |
| **per-workspace 文件系统** | 运行期状态从 `users/{uid}/...` 迁移到 `workspaces/{wid}/threads/{tid}/...`,提供 `make migrate-paths` 迁移脚本(支持 `DRY_RUN=1` 预览)。 | `config/paths.py``thread_data_middleware.py` |
| **边界扫描围栏** | CI 静态扫描禁止任何路径绕过入口直连 LangGraph checkpoint/store,确保隔离不被旁路。 | `tests/boundary_allowlist.toml``test_workspace_boundary*.py` |
| **Headless API schema 底座** | 预建 `service_accounts` / `api_keys``dfk_live_*` / `dfk_test_*`/ `external_users` 三张表(schema-only),为 Stage 1 的无人值守接入做准备。 | `persistence/{service_account,api_key,external_user}` |
| **Alembic 迁移 + 回填** | `0001``0003` 迁移链 + `backfill_workspace_id.py` 回填脚本,dev 用 `create_all()` 自愈、生产用迁移。 | `persistence/migrations/` |
> 数据库的事实参考(10 张表全字段 / 外键 / 索引)见 [`database-schema-as-built.zh-CN.md`](docs/multi-tenant-redesign/01-redesign/database-schema-as-built.zh-CN.md)。
### 路线图:Stage 04
| Stage | 目标 | 关键内容 | 状态 |
|---|---|---|---|
| **0** | workspace 模型立起来 + Postgres 切换 + auth 收紧 | 上表全部 | ✅ 工程层面已合入(业务门 / live 验证待跟进)|
| **1** | 第一批付费客户 + 业务系统集成(双轨并行)| quota + 计费 + AioSandbox 轻量加固;Headless APIPattern A/B)接通 PR8 三张表 | 🔜 已具备底座 |
| **2** | 增长期,安全与隔离深化 | DeerFlow 自有表启用 RLS、KMS、ObjectStorage、完整 RBAC + invitation | 📋 规划中 |
| **3** | 成熟期,K8s 隔离 + BYO | K8s namespace + NetworkPolicy、BYO LLM key、audit DB 拆分 | 📋 规划中 |
| **4** | 企业化,按需开启 | SSO、自定义域名、per-tenant DB、gVisor/Kata、合规审计 | 📋 按合同 |
### 多租户文档入口
- **汇总索引(先读这个)**[`docs/multi-tenant-redesign/README.zh-CN.md`](docs/multi-tenant-redesign/README.zh-CN.md)
- **现状架构鸟瞰**[`00-current-state/architecture-overview.zh-CN.md`](docs/multi-tenant-redesign/00-current-state/architecture-overview.zh-CN.md)
- **决策(7 份 ADR + spike + 审计)**[`01-redesign/`](docs/multi-tenant-redesign/01-redesign/)
- **Stage 0 schema 锁定版 / 落地版**[`workspace-schema-design`](docs/multi-tenant-redesign/01-redesign/workspace-schema-design.zh-CN.md) · [`database-schema-as-built`](docs/multi-tenant-redesign/01-redesign/database-schema-as-built.zh-CN.md)
- **落地路线 + 集成轨道**[`02-rollout/`](docs/multi-tenant-redesign/02-rollout/)
- **Stage 0 进度面板(权威"现在到哪了"**[`03-impl/STATUS.zh-CN.md`](docs/multi-tenant-redesign/03-impl/STATUS.zh-CN.md)
## 官网 ## 官网
@@ -32,13 +71,13 @@ https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18
## 目录 ## 目录
- [🦌 DeerFlow - 2.0](#-deerflow---20) - [🦌 DeerFlow - 2.0 · 多租户改造](#-deerflow---20--多租户改造)
- [多租户改造(本分支主线)](#多租户改造本分支主线)
- [官网](#官网) - [官网](#官网)
- [InfoQuest](#infoquest) - [字节跳动火山引擎方舟 Coding Plan](#字节跳动火山引擎方舟-coding-plan)
- [目录](#目录)
- [一句话交给 Coding Agent 安装](#一句话交给-coding-agent-安装)
- [快速开始](#快速开始) - [快速开始](#快速开始)
- [配置](#配置) - [配置](#配置)
- [数据库后端](#数据库后端)
- [运行应用](#运行应用) - [运行应用](#运行应用)
- [部署建议与资源规划](#部署建议与资源规划) - [部署建议与资源规划](#部署建议与资源规划)
- [方式一:Docker(推荐)](#方式一docker推荐) - [方式一:Docker(推荐)](#方式一docker推荐)
@@ -48,7 +87,7 @@ https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18
- [MCP Server](#mcp-server) - [MCP Server](#mcp-server)
- [IM 渠道](#im-渠道) - [IM 渠道](#im-渠道)
- [LangSmith 链路追踪](#langsmith-链路追踪) - [LangSmith 链路追踪](#langsmith-链路追踪)
- [从 Deep Research 到 Super Agent Harness](#从-deep-research-到-super-agent-harness) - [多租户架构详解](#多租户架构详解)
- [核心特性](#核心特性) - [核心特性](#核心特性)
- [Skills 与 Tools](#skills-与-tools) - [Skills 与 Tools](#skills-与-tools)
- [Claude Code 集成](#claude-code-集成) - [Claude Code 集成](#claude-code-集成)
@@ -56,25 +95,14 @@ https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18
- [Sandbox 与文件系统](#sandbox-与文件系统) - [Sandbox 与文件系统](#sandbox-与文件系统)
- [Context Engineering](#context-engineering) - [Context Engineering](#context-engineering)
- [长期记忆](#长期记忆) - [长期记忆](#长期记忆)
- [推荐模型](#推荐模型) - [在 DeerFlow 之上构建应用(apps/](#在-deerflow-之上构建应用apps)
- [内嵌 Python Client](#内嵌-python-client) - [内嵌 Python Client](#内嵌-python-client)
- [推荐模型](#推荐模型)
- [文档](#文档) - [文档](#文档)
- [⚠️ 安全使用](#-安全使用) - [⚠️ 安全使用](#-安全使用)
- [参与贡献](#参与贡献) - [参与贡献](#参与贡献)
- [许可证](#许可证) - [许可证](#许可证)
- [致谢](#致谢) - [致谢](#致谢)
- [核心贡献者](#核心贡献者)
- [Star History](#star-history)
## 一句话交给 Coding Agent 安装
如果你在用 Claude Code、Codex、Cursor、Windsurf 或其他 coding agent,可以直接把下面这句话发给它:
```text
如果还没 clone DeerFlow,就先 clone,然后按照 https://raw.githubusercontent.com/bytedance/deer-flow/main/Install.md 把它的本地开发环境初始化好
```
这条提示词是给 coding agent 用的。它会在需要时先 clone 仓库,优先选择 Docker,完成初始化,并在结束时告诉你下一条启动命令,以及还缺哪些配置需要你补充。
## 快速开始 ## 快速开始
@@ -149,6 +177,48 @@ https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18
api_key: your-actual-api-key-here # 替换为真实 key api_key: your-actual-api-key-here # 替换为真实 key
``` ```
### 数据库后端
多租户改造后,**Stage 0+ 默认后端是 Postgres**(与生产对齐,并为后续 RLS 留好空间)。`config.example.yaml` 默认带:
```yaml
database:
backend: postgres
postgres_url: $DATABASE_URL
```
在 `.env` 中设置 `DATABASE_URL`
```bash
DATABASE_URL=postgresql+asyncpg://deerflow:deerflow_dev@localhost:5432/deerflow
# 远程 RDS / Cloud SQL 示例:
# DATABASE_URL=postgresql+asyncpg://USER:PASSWORD@HOST:5432/DBNAME
```
启动本地 Postgres 开发容器:
```bash
docker compose -f docker/docker-compose-dev.yaml up -d postgres
```
- `make doctor` 会报告当前配置的后端、尝试 asyncpg 连接,并给出可执行的修复建议。
- `make dev` 在启动各服务前会先 preflight Postgres 可达性;`DATABASE_URL` 不可达时直接中止。
- **dev** 启动时用 `Base.metadata.create_all()` 自动建缺失的表(不改已存在的表);**生产**用 Alembic 迁移(`backend/packages/harness/deerflow/persistence/migrations/`)。目标库不存在时会自动 `CREATE DATABASE` 后重试。
<details>
<summary>离线开发(SQLite 兜底)</summary>
如果你不想起 Postgres,把 `config.yaml` 改成:
```yaml
database:
backend: sqlite
sqlite_dir: .deer-flow/data
```
SQLite 仍是合法的离线开发后端;但 RLS / 多节点等 Stage 2+ 能力需要 Postgres。
</details>
### 运行应用 ### 运行应用
#### 部署建议与资源规划 #### 部署建议与资源规划
@@ -194,7 +264,7 @@ make down # 停止并移除容器
如果你更希望直接在本地启动各个服务: 如果你更希望直接在本地启动各个服务:
前提:先完成上面的配置步骤(`make config`模型 API key 配置)。`make dev` 需要有效配置文件,默认读取项目根目录下的 `config.yaml`。可以用 `DEER_FLOW_PROJECT_ROOT` 显式指定项目根目录,也可以用 `DEER_FLOW_CONFIG_PATH` 指向某个具体配置文件。运行期状态默认写到项目根目录下的 `.deer-flow`,可用 `DEER_FLOW_HOME` 覆盖;skills 默认读取项目根目录下的 `skills/`,可用 `DEER_FLOW_SKILLS_PATH` 覆盖。 前提:先完成上面的"配置"步骤(`make config`模型 API key、`DATABASE_URL`)。`make dev` 需要有效配置文件,默认读取项目根目录下的 `config.yaml`。可以用 `DEER_FLOW_PROJECT_ROOT` 显式指定项目根目录,也可以用 `DEER_FLOW_CONFIG_PATH` 指向某个具体配置文件。运行期状态默认写到项目根目录下的 `.deer-flow`,可用 `DEER_FLOW_HOME` 覆盖;skills 默认读取项目根目录下的 `skills/`,可用 `DEER_FLOW_SKILLS_PATH` 覆盖。
在 Windows 上,请使用 Git Bash 运行本地开发流程。基于 bash 的服务脚本不支持直接在原生 `cmd.exe` 或 PowerShell 中执行,且 WSL 也不保证可用,因为部分脚本依赖 Git for Windows 的 `cygpath` 等工具。 在 Windows 上,请使用 Git Bash 运行本地开发流程。基于 bash 的服务脚本不支持直接在原生 `cmd.exe` 或 PowerShell 中执行,且 WSL 也不保证可用,因为部分脚本依赖 Git for Windows 的 `cygpath` 等工具。
1. **检查依赖环境** 1. **检查依赖环境**
@@ -220,6 +290,16 @@ make down # 停止并移除容器
5. **访问地址**http://localhost:2026 5. **访问地址**http://localhost:2026
> [!TIP]
> `make dev` 是前台阻塞运行。日常调试更顺手的是仓库根 `scripts/` 下两个生命周期脚本(子命令统一为 `start / stop / restart / status / logs / run`):
> - `scripts/dev-gateway.sh` — 只起 Gateway`http://localhost:8001`),起得快,适合调后端 API / 接入示例。
> - `scripts/dev-full.sh` — Gateway + 前端 + nginx`http://localhost:2026`),连前端一起调。
>
> 例如 `./scripts/dev-gateway.sh start`、`./scripts/dev-gateway.sh logs`、`SKIP_INSTALL=1 ./scripts/dev-full.sh start`。详见 [apps/README.md](apps/README.md)。
> [!NOTE]
> 把历史的 `users/` 目录树迁移到新的 per-workspace 布局:`make migrate-paths`(加 `DRY_RUN=1` 仅预览,`DEFAULT_WORKSPACE=<wid>` 指定未分配用户的归属 workspace)。
### 进阶配置 ### 进阶配置
#### Sandbox 模式 #### Sandbox 模式
@@ -405,23 +485,48 @@ LANGSMITH_PROJECT=xxx
Docker 部署时,追踪默认关闭。在 `.env` 中设置 `LANGSMITH_TRACING=true` 和 `LANGSMITH_API_KEY` 即可启用。 Docker 部署时,追踪默认关闭。在 `.env` 中设置 `LANGSMITH_TRACING=true` 和 `LANGSMITH_API_KEY` 即可启用。
## 从 Deep Research 到 Super Agent Harness ## 多租户架构详解
DeerFlow 最初是一个 Deep Research 框架,后来社区把它一路推到了更远的地方。上线之后,开发者拿它去做的事情早就不止研究:搭数据流水线、生成演示文稿、快速起 dashboard、自动化内容流程,很多方向一开始连我们自己都没想到 > 这一节展开 [多租户改造](#多租户改造本分支主线) 里 Stage 0 已落地的实现细节。完整决策与路线见 [`docs/multi-tenant-redesign/`](docs/multi-tenant-redesign/)
这让我们意识到一件事:DeerFlow 不只是一个研究工具。它更像一个 **harness**,一个真正让 agents 把事情做完的运行时基础设施。 **workspace 是唯一的隔离粒度。** 个人用户 = 1 人 workspace,小团队 = 多人 workspace。骨架是两条主线:
所以我们把它从头重做了一遍 1. **租户骨架**`users` ↔ `workspaces`(多对多经 `workspace_memberships`)。每个用户注册时自动建 1 人 workspaceowner=自己),slug 唯一且过黑名单校验
2. **业务数据**`threads_meta` → `runs` → `run_events` / `feedback`,全部挂 `workspace_id`(行级隔离),workspace 删除时级联清空。
DeerFlow 2.0 不再是一个需要你自己拼装的 framework。它是一个开箱即用、同时又足够可扩展的 super agent harness。基于 LangGraph 和 LangChain 构建,默认就带上了 agent 真正会用到的关键能力:文件系统、memory、skills、sandbox 执行环境,以及为复杂多步骤任务做规划、拉起 sub-agents 的能力 **数据隔离怎么做的。** DeerFlow 自有表走行级 `workspace_id` + `UNIQUE(workspace_id, thread_id)` 兜底;入口路由(`threads.py` / `thread_runs.py`)按 `(workspace_id, thread_id)` 强校验,跨 workspace 访问必返 404。LangGraph 自己的 checkpointer / store 表(`langgraph-checkpoint-postgres==3.0.5` 无 `connection_factory`,无法注入 RLS)则走**应用层强校验**,并由 CI 静态扫描(`tests/boundary_allowlist.toml`)禁止任何路径绕过入口直连这些表
你可以直接拿来用,也可以拆开重组,改成你自己的样子。 **身份与会话。** JWT TokenPayload 一次到位为 `{sub, wid, role, exp, iat, ver}`
- `wid` — 当前 workspace`role` — workspace 内角色(Stage 0 简化为 owner-onlyStage 2 扩到 owner/admin/member)。
- `ver` — `token_version`bump 后旧 token 全失效。
- 旧版 4 字段 JWT 会被识别为 `WORKSPACE_MISSING` 并要求重登;登录 / 改密 / `/auth/me` 都会带上 workspace 与角色(`/auth/me` 返回 `workspaces[]`,含 id/name/slug/role)。
**文件系统布局。** 运行期状态按 workspace 分目录:
```text
${DEER_FLOW_HOME:-./.deer-flow}/
└── workspaces/
└── {workspace_id}/
├── threads/{thread_id}/... ← 每个 thread 的 sandbox / 产物
└── users/{user_id}/... ← 用户级状态
```
历史的 `users/{uid}/...` 布局用 `make migrate-paths``DRY_RUN=1` 预览)迁移过来。
**Headless API schema 底座(Stage 0 末预建,schema-only)。** 为 Stage 1 的无人值守 / 业务系统接入准备:
- `service_accounts` — workspace 内的非人身份,带 `identity_mode` 三态(`collapsed` / `external_passthrough` / `both`)。
- `api_keys` — service account 的凭证,格式 `dfk_live_*` / `dfk_test_*``key_prefix` 全局唯一 + 部分索引 `WHERE revoked_at IS NULL`。
- `external_users` — passthrough 终端身份,`(service_account_id, external_id)` 复合唯一。
> ⚠️ Stage 0 只建表,**API Key 鉴权中间件尚未接入**。外部系统当前只能走会话 cookie(见 [apps/README.md](apps/README.md) 的鉴权说明);等 `Authorization: Bearer dfk_live_...` 在 Stage 1 落地后再补无人值守接入。
## 核心特性 ## 核心特性
### Skills 与 Tools ### Skills 与 Tools
Skills 是 DeerFlow 能做几乎任何事的关键。 Skills 是 DeerFlow 能做"几乎任何事"的关键。
标准的 Agent Skill 是一种结构化能力模块,通常就是一个 Markdown 文件,里面定义了工作流、最佳实践,以及相关的参考资源。DeerFlow 自带一批内置 skills,覆盖研究、报告生成、演示文稿制作、网页生成、图像和视频生成等场景。真正有意思的地方在于它的扩展性:你可以加自己的 skills,替换内置 skills,或者把多个 skills 组合成复合工作流。 标准的 Agent Skill 是一种结构化能力模块,通常就是一个 Markdown 文件,里面定义了工作流、最佳实践,以及相关的参考资源。DeerFlow 自带一批内置 skills,覆盖研究、报告生成、演示文稿制作、网页生成、图像和视频生成等场景。真正有意思的地方在于它的扩展性:你可以加自己的 skills,替换内置 skills,或者把多个 skills 组合成复合工作流。
@@ -431,8 +536,6 @@ Skills 采用按需渐进加载,不会一次性把所有内容都塞进上下
Tools 也是同样的思路。DeerFlow 自带一组核心工具:网页搜索、网页抓取、文件操作、bash 执行;同时也支持通过 MCP Server 和 Python 函数扩展自定义工具。你可以替换任何一项,也可以继续往里加。 Tools 也是同样的思路。DeerFlow 自带一组核心工具:网页搜索、网页抓取、文件操作、bash 执行;同时也支持通过 MCP Server 和 Python 函数扩展自定义工具。你可以替换任何一项,也可以继续往里加。
Gateway 生成后续建议时,现在会先把普通字符串输出和 block/list 风格的富文本内容统一归一化,再去解析 JSON 数组响应,因此不同 provider 的内容包装方式不会再悄悄把建议吞掉。
```text ```text
# sandbox 容器内的路径 # sandbox 容器内的路径
/mnt/skills/public /mnt/skills/public
@@ -456,24 +559,7 @@ Gateway 生成后续建议时,现在会先把普通字符串输出和 block/li
npx skills add https://github.com/bytedance/deer-flow --skill claude-to-deerflow npx skills add https://github.com/bytedance/deer-flow --skill claude-to-deerflow
``` ```
然后确认 DeerFlow 已经启动(默认地址是 `http://localhost:2026`),在 Claude Code 里使用 `/claude-to-deerflow` 命令即可。 然后确认 DeerFlow 已经启动(默认地址是 `http://localhost:2026`),在 Claude Code 里使用 `/claude-to-deerflow` 命令即可。完整 API 说明见 [`skills/public/claude-to-deerflow/SKILL.md`](skills/public/claude-to-deerflow/SKILL.md)。
**你可以做的事情包括:**
- 给 DeerFlow 发送消息,并接收流式响应
- 选择执行模式:flash(更快)、standard、pro(规划模式)、ultrasub-agents 模式)
- 检查 DeerFlow 健康状态,列出 models / skills / agents
- 管理 threads 和会话历史
- 上传文件做分析
**环境变量**(可选,用于自定义端点):
```bash
DEERFLOW_URL=http://localhost:2026 # 统一代理基地址
DEERFLOW_GATEWAY_URL=http://localhost:2026 # Gateway API
DEERFLOW_LANGGRAPH_URL=http://localhost:2026/api/langgraph # LangGraph API
```
完整 API 说明见 [`skills/public/claude-to-deerflow/SKILL.md`](skills/public/claude-to-deerflow/SKILL.md)。
### Sub-Agents ### Sub-Agents
@@ -485,12 +571,10 @@ lead agent 可以按需动态拉起 sub-agents。每个 sub-agent 都有自己
### Sandbox 与文件系统 ### Sandbox 与文件系统
DeerFlow 不只是会说它能做,它是真的有一台自己的电脑 DeerFlow 不只是"会说它能做",它是真的有一台自己的"电脑"
每个任务都运行在隔离的 Docker 容器里,里面有完整的文件系统,包括 skills、workspace、uploads、outputs。agent 可以读写和编辑文件,可以执行 bash 命令和代码,也可以查看图片。整个过程都在 sandbox 内完成,可审计、会隔离,不会在不同 session 之间互相污染。 每个任务都运行在隔离的 Docker 容器里,里面有完整的文件系统,包括 skills、workspace、uploads、outputs。agent 可以读写和编辑文件,可以执行 bash 命令和代码,也可以查看图片。整个过程都在 sandbox 内完成,可审计、会隔离,不会在不同 session 之间互相污染。
这就是“带工具的聊天机器人”和“真正有执行环境的 agent”之间的差别。
```text ```text
# sandbox 容器内的路径 # sandbox 容器内的路径
/mnt/user-data/ /mnt/user-data/
@@ -509,16 +593,30 @@ DeerFlow 不只是“会说它能做”,它是真的有一台自己的“电
大多数 agents 会在对话结束后把一切都忘掉,DeerFlow 不一样。 大多数 agents 会在对话结束后把一切都忘掉,DeerFlow 不一样。
跨 session 使用时,DeerFlow 会逐步积累关于你的持久 memory,包括你的个人偏好、知识背景,以及长期沉淀下来的工作习惯。你用得越多,它越了解你的写作风格、技术栈和重复出现的工作流。memory 保存在本地,控制权也始终在你手里。 跨 session 使用时,DeerFlow 会逐步积累关于你的持久 memory,包括你的个人偏好、知识背景,以及长期沉淀下来的工作习惯。你用得越多,它越了解你的写作风格、技术栈和重复出现的工作流。
## 推荐模型 ## 在 DeerFlow 之上构建应用(apps/
DeerFlow 对模型没有强绑定,只要实现了 OpenAI 兼容 API 的 LLM,理论上都可以接入。不过在下面这些能力上表现更强的模型,通常会更适合 DeerFlow `apps/`(仓库根目录、与 `backend/` / `frontend/` 平级)用于存放**消费 DeerFlow 能力的上层应用**,遵循严格的依赖方向:**app 可以依赖 deerflowdeerflow 不能依赖 app / apps**。
- **长上下文窗口**100k+ tokens),适合深度研究和多步骤任务 两种集成模式:
- **推理能力**,适合自适应规划和复杂拆解
- **多模态输入**,适合理解图片和视频 | 模式 | 适用场景 | 怎么连 | 示例 |
- **稳定的 tool use 能力**,适合可靠的函数调用和结构化输出 |---|---|---|---|
| **HTTP Gateway**REST+SSE | 上层是别的服务 / 多语言 | 调 `http://localhost:2026/api/*` | [`apps/examples/http-chat/`](apps/examples/http-chat/) |
| **内嵌 DeerFlowClient** | 上层本身是 Python,进程内直接当 SDK 调 | `from deerflow.client import DeerFlowClient` | [`apps/examples/embedded-chat/`](apps/examples/embedded-chat/) |
每个示例自带 `run.sh`
```bash
# ① HTTP 模式:需要先起 Gatewaydev-gateway 或 dev-full 都行)
./apps/examples/http-chat/run.sh
# ② 内嵌模式:不需要起任何服务,run.sh 自动进 backend uv 环境运行
./apps/examples/embedded-chat/run.sh
```
完整说明、鉴权流程与新建应用约定见 [apps/README.md](apps/README.md)。
## 内嵌 Python Client ## 内嵌 Python Client
@@ -546,12 +644,25 @@ client.upload_files("thread-1", ["./report.pdf"]) # {"success": True, "files":
所有返回 dict 的方法都会在 CI 中通过 Gateway 的 Pydantic 响应模型校验(`TestGatewayConformance`),以确保内嵌 client 始终和 HTTP API schema 保持同步。完整 API 说明见 `backend/packages/harness/deerflow/client.py`。 所有返回 dict 的方法都会在 CI 中通过 Gateway 的 Pydantic 响应模型校验(`TestGatewayConformance`),以确保内嵌 client 始终和 HTTP API schema 保持同步。完整 API 说明见 `backend/packages/harness/deerflow/client.py`。
## 推荐模型
DeerFlow 对模型没有强绑定,只要实现了 OpenAI 兼容 API 的 LLM,理论上都可以接入。不过在下面这些能力上表现更强的模型,通常会更适合 DeerFlow:
- **长上下文窗口**100k+ tokens),适合深度研究和多步骤任务
- **推理能力**,适合自适应规划和复杂拆解
- **多模态输入**,适合理解图片和视频
- **稳定的 tool use 能力**,适合可靠的函数调用和结构化输出
## 文档 ## 文档
- [多租户改造汇总索引](docs/multi-tenant-redesign/README.zh-CN.md) - workspace / Postgres / RLS / Headless API 的决策与路线
- [Stage 0 进度面板](docs/multi-tenant-redesign/03-impl/STATUS.zh-CN.md) - "现在到哪了"的权威来源
- [数据库设计落地版](docs/multi-tenant-redesign/01-redesign/database-schema-as-built.zh-CN.md) - 10 张表全字段 / 外键 / 索引参考
- [贡献指南](CONTRIBUTING.md) - 开发环境搭建与协作流程 - [贡献指南](CONTRIBUTING.md) - 开发环境搭建与协作流程
- [配置指南](backend/docs/CONFIGURATION.md) - 安装与配置说明 - [配置指南](backend/docs/CONFIGURATION.md) - 安装与配置说明
- [架构概览](backend/CLAUDE.md) - 技术架构说明 - [架构概览](backend/CLAUDE.md) - 技术架构说明
- [后端架构](backend/README.md) - 后端架构与 API 参考 - [后端架构](backend/README.md) - 后端架构与 API 参考
- [apps/ 上层应用](apps/README.md) - 在 DeerFlow 之上构建应用
## ⚠️ 安全使用 ## ⚠️ 安全使用
@@ -562,6 +673,9 @@ DeerFlow 具备**系统指令执行、资源操作、业务逻辑调用**等关
- **未授权的非法调用**:agent 功能被未授权的第三方、公网恶意扫描程序探测到,进而发起批量非法调用请求,执行系统命令、文件读写等高危操作,可能导致安全后果。 - **未授权的非法调用**:agent 功能被未授权的第三方、公网恶意扫描程序探测到,进而发起批量非法调用请求,执行系统命令、文件读写等高危操作,可能导致安全后果。
- **合规与法律风险**:若 agent 被非法调用用于实施网络攻击、信息窃取等违法违规行为,可能产生法律责任与合规风险。 - **合规与法律风险**:若 agent 被非法调用用于实施网络攻击、信息窃取等违法违规行为,可能产生法律责任与合规风险。
> [!NOTE]
> 多租户改造引入的 workspace 行级隔离 / 入口强校验 / 边界扫描,目标是**应用内**的租户隔离;它不替代上面的网络层 / 部署层防护。把 DeerFlow 曝光到不可信网络仍需配合下面的安全措施。多租户更强的 DB 层兜底(RLS / KMS / K8s)规划在 Stage 23。
### 安全使用建议 ### 安全使用建议
**注意:建议您将 DeerFlow 部署在本地可信的网络环境下。** 若您有跨设备、跨网络的部署需求,必须加入严格的安全措施。例如,采取如下手段: **注意:建议您将 DeerFlow 部署在本地可信的网络环境下。** 若您有跨设备、跨网络的部署需求,必须加入严格的安全措施。例如,采取如下手段:
@@ -575,7 +689,12 @@ DeerFlow 具备**系统指令执行、资源操作、业务逻辑调用**等关
欢迎参与贡献。开发环境、工作流和相关规范见 [CONTRIBUTING.md](CONTRIBUTING.md)。 欢迎参与贡献。开发环境、工作流和相关规范见 [CONTRIBUTING.md](CONTRIBUTING.md)。
目前回归测试已经覆盖 Docker sandbox 模式识别,以及 `backend/tests/` 中 provisioner kubeconfig-path 处理相关测试。 提 PR 前请先在本地跑通校验(CI 会在每个 PR 上执行 backend lint + 测试,含 Postgres matrix):
```bash
cd backend && make lint && make test # ruff + pytest
cd frontend && pnpm lint && pnpm typecheck
```
## 许可证 ## 许可证
@@ -583,22 +702,15 @@ DeerFlow 具备**系统指令执行、资源操作、业务逻辑调用**等关
## 致谢 ## 致谢
DeerFlow 建立在开源社区大量优秀工作的基础上。所有让 DeerFlow 成为可能的项目和贡献者,我们都心怀感谢。毫不夸张地说,我们是站在巨人的肩膀上继续往前走。 DeerFlow 建立在开源社区大量优秀工作的基础上。所有让 DeerFlow 成为可能的项目和贡献者,我们都心怀感谢。
特别感谢以下项目带来的关键支持: 特别感谢以下项目带来的关键支持:
- **[LangChain](https://github.com/langchain-ai/langchain)**:它们提供的优秀框架支撑了我们的 LLM 交互与 chains,让整体集成和能力编排顺畅可用 - **[LangChain](https://github.com/langchain-ai/langchain)**:它们提供的优秀框架支撑了我们的 LLM 交互与 chains。
- **[LangGraph](https://github.com/langchain-ai/langgraph)**:它们在多 agent 编排上的创新方式,是 DeerFlow 复杂工作流得以成立的重要基础。 - **[LangGraph](https://github.com/langchain-ai/langgraph)**:它们在多 agent 编排上的创新方式,是 DeerFlow 复杂工作流得以成立的重要基础。
这些项目体现了开源协作真正的力量,我们也很高兴能继续建立在这些基础之上。
### 核心贡献者
感谢 `DeerFlow` 的核心作者,是他们的判断、投入和持续推进,才让这个项目真正落地:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
## Star History ## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date) [![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
</content>
</invoke>
+9
View File
@@ -0,0 +1,9 @@
# 本地依赖 / 虚拟环境
.venv/
__pycache__/
*.pyc
# 鉴权 cookie / 本地产物
jar.txt
*.local
.env
+113
View File
@@ -0,0 +1,113 @@
# apps/ — 基于 DeerFlow 的应用
这个目录用于存放**消费 DeerFlow 智能体能力的上层应用**。每个应用一个子文件夹。
## 为什么放在这里
DeerFlow 的代码有一条严格的依赖方向(见根 `CLAUDE.md`):
```
backend/packages/harness/deerflow/ ← 可发布的 Agent 框架(deerflow.*
backend/app/ ← Gateway / IM 通道(app.*
apps/ ← 你的应用(消费 deerflow,不反向依赖) ← 本目录
```
规则:**app 可以依赖 deerflowdeerflow 不能依赖 app / apps**。本目录放在 `backend/` 之外、与 `frontend/` 平级,天然符合这条边界。
## 两种集成模式
| 模式 | 适用场景 | 怎么连 | 示例 |
|---|---|---|---|
| **HTTP Gateway**(REST+SSE) | 上层是别的服务 / 多语言 | 调 `http://localhost:2026/api/*` | [`examples/http-chat/`](examples/http-chat/) |
| **内嵌 DeerFlowClient** | 上层本身是 Python,进程内直接当 SDK 调 | `from deerflow.client import DeerFlowClient` | [`examples/embedded-chat/`](examples/embedded-chat/) |
> 还有第三种:LangGraph SDK`langgraph_sdk.get_client(url=".../api")`graph id `lead_agent`),用于接入 LangGraph 生态工具链。需要的话照 HTTP 示例的鉴权流程拿 cookie 即可。
### 运行示例(每个示例自带 run.sh)
```bash
# ① HTTP 模式:需要先起 Gatewaydev-gateway 或 dev-full 都行)
# run.sh 会自动探测网关地址:优先 :2026,回退 :8001
./apps/examples/http-chat/run.sh
DF_BASE=http://localhost:8001 ./apps/examples/http-chat/run.sh # 也可手动指定
# ② 内嵌模式:不需要起任何服务,run.sh 自动进 backend uv 环境运行
./apps/examples/embedded-chat/run.sh
```
http-chat 的 `run.sh` 优先用 `uv run --no-project --with requests`(临时环境,不污染系统),没有 uv 才回退到本地 `.venv` + pip;可用 `DF_BASE` / `DF_EMAIL` / `DF_PASSWORD` 覆盖。embedded-chat 的 `run.sh` 自动定位 `backend/`、加载 `.env` 后用 `uv run` 启动,依赖 `config.yaml` 里有可用模型。
### 多租户并发验证(HTTP 模式)
[`examples/http-chat/multi_tenant.py`](examples/http-chat/multi_tenant.py) 是 `app.py` 的并发 / 多租户版:用 `/api/v1/auth/register` 并发创建多个租户(每个注册用户自带独立 workspace),各自一个 `requests.Session`(独立 cookie)同时跑 N 轮链式对话,并校验:① 真并发(对话时间窗重叠);② 多轮上下文按 `thread_id` 各自保持(第 2 轮起每轮都依赖上一轮结果);③ 租户隔离(`POST /api/threads/search` 仅见己有线程,跨租户 `GET /api/threads/{id}` 返回 404)。
```bash
# 前提:已起 Gatewaydev-gateway 或 dev-full
DF_BASE=http://localhost:8001 DF_TENANTS=4 DF_TURNS=10 \
uv run --no-project --with requests python apps/examples/http-chat/multi_tenant.py
```
环境变量:`DF_BASE`(网关地址,默认 :8001)、`DF_TENANTS`(并发租户数,默认 3)、`DF_TURNS`(每租户轮数,默认 10)。每次运行用唯一邮箱新建租户,可重复跑,不撞 email、也不触发登录限流(`/register` 不限流;`setup-status` 全程只调一次以避开 60s/IP 限流)。
### 多租户验证(Headless / API Key 模式)
[`examples/http-chat/multi_tenant_headless.py`](examples/http-chat/multi_tenant_headless.py) 是上面那个测试的 **server-to-server(无人值守)** 版,验证 Stage 1 的 API Key 鉴权:每个租户先由一个人类 owner(cookie 会话)建 service account 并 mint 一把 workspace-scoped key`POST /api/v1/service-accounts``POST /api/v1/api-keys`,plaintext 仅返回一次),之后所有对话只用 `Authorization: Bearer dfk_live_...`(独立 Session、**不带 cookie / CSRF**)。除并发 / 上下文 / 隔离(同 cookie 版)外,额外校验两条 headless 专属性质:④ **scope 强制**——一把缺 `runs:create` 的 key 发起对话返回 403;⑤ **撤销即失效**——`DELETE /api/v1/api-keys/{id}` 后该 key 立即 401。
```bash
# 前提:已起 Gatewaydev-gateway 或 dev-full
DF_BASE=http://localhost:8001 DF_TENANTS=4 DF_TURNS=10 \
uv run --no-project --with requests python apps/examples/http-chat/multi_tenant_headless.py
```
环境变量同上,外加 `DF_EXTRA=0` 可跳过 scope / 撤销专项检查。实测(`:8001``DF_TENANTS=2 DF_TURNS=3`)全绿:真并发、多轮上下文保持、跨租户 `GET` 均 404、只读 key stream 403、撤销后 401。
## 前置:先把 DeerFlow 跑起来
在**仓库根目录**
```bash
make dev # 起 Gateway(8001) + 前端(3000) + nginx(2026),统一入口 http://localhost:2026
```
确保 `config.yaml` 里至少配了一个可用模型 + API key。
### 本地调试脚本(推荐)
`make dev` 是前台阻塞运行。日常调试更顺手的是仓库根 `scripts/` 下两个生命周期脚本,子命令统一为 `start / stop / restart / status / logs / run`
| 脚本 | 起什么 | 入口 | 适合 |
|---|---|---|---|
| `scripts/dev-gateway.sh` | 只起 Gateway | `http://localhost:8001` | 调后端 API / 接入示例,起得快 |
| `scripts/dev-full.sh` | Gateway + 前端 + nginx | `http://localhost:2026` | 连前端一起调,完整体验 |
```bash
./scripts/dev-gateway.sh start # 后台启动,等就绪后返回
./scripts/dev-gateway.sh status # PID / 端口 / HTTP 健康检查
./scripts/dev-gateway.sh logs # tail -f 跟随日志(不影响服务)
./scripts/dev-gateway.sh stop
./scripts/dev-full.sh start # 全量栈后台启动(首次装依赖)
SKIP_INSTALL=1 ./scripts/dev-full.sh start # 跳过依赖安装,重启更快
./scripts/dev-full.sh status # 三服务一览
./scripts/dev-full.sh run # 前台运行(= make devgateway 带热重载)
```
环境变量:`PORT=`(换端口)、`NO_RELOAD=1`(关热重载,断点更稳)、`SKIP_INSTALL=1`(全量栈跳过装依赖)。
## 鉴权(HTTP 模式必读)
Gateway 是 **fail-closed** 的——除少数公开路径外所有请求都要带会话 cookie:
1. `GET /api/v1/auth/setup-status` → 是否还没管理员
2. 首次 `POST /api/v1/auth/initialize`JSON `{email,password}`)建第一个管理员;之后 `POST /api/v1/auth/login/local`**表单** `username`=邮箱 + `password`
3. 成功后 Session 里有 `access_token`(HttpOnly) + `csrf_token` 两个 cookie
4. **所有写请求**POST/PUT/DELETE/PATCH)必须带 `X-CSRF-Token` 头 = `csrf_token`
> 多租户:浏览器式接入走会话 cookie(每个注册用户即一个独立租户,自带 workspace),并发与隔离可用 [`multi_tenant.py`](examples/http-chat/multi_tenant.py) 验证。**无人值守 / 业务后端**接入已支持 API KeyStage 1):owner 经 `POST /api/v1/service-accounts` + `POST /api/v1/api-keys` mint 一把 key,业务侧用 `Authorization: Bearer dfk_live_...` 直连(免 cookie / CSRF),端到端示例见 [`multi_tenant_headless.py`](examples/http-chat/multi_tenant_headless.py)。
## 新建一个应用
```bash
mkdir apps/my-app
# 放你的代码;HTTP 模式参照 examples/http-chat,内嵌模式参照 examples/embedded-chat
```
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""
内嵌模式示例:进程内直接把 DeerFlow 当 SDK 调,不起 HTTP。
必须在 backend 的 uv 环境里跑(这样才能 import deerflow.*):
cd backend
uv run python ../apps/examples/embedded-chat/app.py
依赖 config.yaml 里配好至少一个可用模型 + API key(路径解析见根 CLAUDE.md)。
API 对照:backend/packages/harness/deerflow/client.py
"""
from deerflow.client import DeerFlowClient
from deerflow.runtime.checkpointer.provider import get_checkpointer
def main() -> None:
# checkpointer 提供跨轮状态持久化(sqlite/postgres 由 config.yaml 决定)
client = DeerFlowClient(
checkpointer=get_checkpointer(),
thinking_enabled=True,
)
thread_id = "embedded-demo-1"
# ① 流式:stream() 产出 StreamEvent
print("👤 用一句话介绍你自己,然后心算 17 * 23。\n🤖 ", end="", flush=True)
for ev in client.stream("用一句话介绍你自己,然后心算 17 * 23。", thread_id=thread_id):
if ev.type == "messages-tuple" and ev.data.get("type") == "ai":
print(ev.data.get("content", ""), end="", flush=True) # AI 文本增量
elif ev.type == "end":
print(f"\n[usage] {ev.data.get('usage')}")
# ② 阻塞式:chat() 直接返回完整 AI 文本(复用 thread_id 即多轮)
print("\n👤 刚才结果再乘以 2 是多少?")
answer = client.chat("刚才结果再乘以 2 是多少?", thread_id=thread_id)
print(f"🤖 {answer}")
# 其它能力:list_models() / list_skills() / get_memory() / upload_files() ...
models = client.list_models().get("models", [])
print(f"\n[已配置模型] {[m.get('name') for m in models]}")
if __name__ == "__main__":
main()
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
#
# embedded-chat 示例启动脚本
# ------------------------------------------------------------------
# 内嵌 SDK 模式:进程内直接 import deerflow.*,不需要起任何服务。
# 必须在 backend 的 uv 虚拟环境里跑(才能解析 deerflow-harness / app 包),
# 本脚本自动 cd 到 backend 并用 uv run 启动。
#
# 用法:
# ./run.sh
#
# 前提:
# - 已 `cd backend && uv sync`(或跑过任意一个 dev 脚本,venv 已建好)
# - config.yaml 里配好至少一个可用模型 + API key
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
APP="$SCRIPT_DIR/app.py"
BACKEND="$REPO_ROOT/backend"
# ── 前置检查 ──────────────────────────────────────────────────────
command -v uv >/dev/null 2>&1 || { echo "✗ 未找到 uv。安装:curl -LsSf https://astral.sh/uv/install.sh | sh" >&2; exit 1; }
[ -f "$REPO_ROOT/config.yaml" ] || echo "⚠ 未找到 $REPO_ROOT/config.yaml —— 没有可用模型会启动失败" >&2
if [ ! -d "$BACKEND/.venv" ]; then
echo "→ 未发现 backend/.venv,执行 uv sync"
(cd "$BACKEND" && uv sync)
fi
# ── 加载 .env(模型 key / 数据库等)──────────────────────────────
if [ -f "$REPO_ROOT/.env" ]; then
set -a
# shellcheck disable=SC1091
source "$REPO_ROOT/.env"
set +a
fi
# ── 在 backend uv 环境里运行(config.yaml 解析依赖运行目录为 backend/)──
echo "→ 在 backend uv 环境中运行 embedded-chat"
cd "$BACKEND"
exec env PYTHONPATH=. uv run python "$APP"
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""
HTTP 模式示例:把 DeerFlow 当底层服务,通过 Gateway (REST + SSE) 对话。
运行:
pip install -r requirements.txt
python app.py # 前提:仓库根目录已 `make dev`
字段 / 事件名均已对照后端源码核对:
鉴权 backend/app/gateway/routers/auth.py + auth_middleware.py + csrf_middleware.py
线程/运行 backend/app/gateway/routers/threads.py + thread_runs.py
SSE 事件名 backend/packages/harness/deerflow/runtime/runs/worker.py
"""
import os
import json
import requests
# 默认走 nginx(:2026);只起了 Gateway 时用 BASE=http://localhost:8001 覆盖
BASE = os.environ.get("DF_BASE", "http://localhost:2026")
EMAIL = os.environ.get("DF_EMAIL", "admin@example.com")
PASSWORD = os.environ.get("DF_PASSWORD", "change-me-please-123") # 至少 8 位,避免弱口令
def authenticate(s: requests.Session) -> None:
"""首启则初始化管理员,否则登录。成功后 cookie 落在 session。"""
status = s.get(f"{BASE}/api/v1/auth/setup-status").json()
if status.get("needs_setup"):
print("→ 首次启动,创建管理员账号")
r = s.post(f"{BASE}/api/v1/auth/initialize",
json={"email": EMAIL, "password": PASSWORD})
else:
print("→ 已有账号,登录")
# login/local 是 OAuth2 表单:字段名 username(填邮箱)+ password
r = s.post(f"{BASE}/api/v1/auth/login/local",
data={"username": EMAIL, "password": PASSWORD})
r.raise_for_status()
print(" cookies:", list(s.cookies.keys()))
def _csrf(s: requests.Session) -> dict:
"""双提交 cookie 模式:csrf_token cookie 的值放进 X-CSRF-Token 头。"""
token = s.cookies.get("csrf_token")
if not token:
raise RuntimeError("缺少 csrf_token cookie —— 鉴权可能失败")
return {"X-CSRF-Token": token}
def create_thread(s: requests.Session) -> str:
r = s.post(f"{BASE}/api/threads", json={}, headers=_csrf(s))
r.raise_for_status()
tid = r.json()["thread_id"]
print(f"→ 线程已创建: {tid}")
return tid
_seen_text = "" # 简单状态,按需扩展为按 message-id 维护
def stream_chat(s: requests.Session, thread_id: str, message: str) -> None:
body = {
"assistant_id": "lead_agent", # 见 backend/langgraph.json
"input": {"messages": [{"role": "user", "content": message}]},
"stream_mode": ["messages-tuple", "values"], # 增量文本 + 全量状态
}
headers = {**_csrf(s), "Accept": "text/event-stream"}
with s.post(f"{BASE}/api/threads/{thread_id}/runs/stream",
json=body, headers=headers, stream=True) as resp:
resp.raise_for_status()
print(f"\n👤 {message}\n🤖 ", end="", flush=True)
event, buf = None, []
for raw in resp.iter_lines(decode_unicode=True):
if raw is None:
continue
line = raw.strip()
if line == "": # 一帧结束
if event:
_handle(event, "\n".join(buf))
event, buf = None, []
elif line.startswith(":"): # 心跳注释
continue
elif line.startswith("event:"):
event = line[6:].strip()
elif line.startswith("data:"):
buf.append(line[5:].strip())
print()
def _handle(event: str, data: str) -> None:
if event == "end" or not data:
return
try:
payload = json.loads(data)
except json.JSONDecodeError:
return
if event == "messages":
# 形如 [chunk_dict, metadata_dict]AI 文本是增量
chunk = payload[0] if isinstance(payload, list) and payload else {}
if chunk.get("type") in ("ai", "AIMessageChunk"):
content = chunk.get("content")
text = content if isinstance(content, str) else _flatten(content)
if text:
print(text, end="", flush=True)
# event == "metadata" → {run_id, thread_id}
# event == "values" → 全量状态快照(title / messages / artifacts ...
def _flatten(content) -> str:
if isinstance(content, list):
return "".join(b.get("text", "") for b in content if isinstance(b, dict))
return ""
def main() -> None:
s = requests.Session()
authenticate(s)
tid = create_thread(s)
stream_chat(s, tid, "用一句话介绍你自己,然后心算 17 * 23。")
stream_chat(s, tid, "刚才结果再乘以 2 是多少?") # 复用 thread_id 即多轮
if __name__ == "__main__":
main()
+297
View File
@@ -0,0 +1,297 @@
#!/usr/bin/env python3
"""
多租户并发测试:以 http-chat 的方式(Gateway REST + SSE),多个租户同时对话,
并验证租户隔离。app.py 的并发 / 多租户版。
运行(前提:仓库根已起 Gateway,如 `./scripts/dev-gateway.sh start`):
DF_BASE=http://localhost:8001 DF_TENANTS=4 \
uv run --no-project --with requests python multi_tenant.py
# 没有 uv 时:pip install -r requirements.txt && python multi_tenant.py
环境变量:DF_BASE(网关地址,默认 :8001)、DF_TENANTS(并发租户数,默认 3)。
每个注册用户 = 一个独立租户(自带独立 workspace)。并发 = 每租户一个
requests.Session(独立 cookie),放进线程池同时跑。
字段/事件名沿用 apps/examples/http-chat/app.py(已对照后端源码):
鉴权 /api/v1/auth/{setup-status,register,me}
线程 POST /api/threads ;列举 POST /api/threads/search ;单查 GET /api/threads/{id}
SSE POST /api/threads/{id}/runs/stream
约束(来自后端源码):
- GET /auth/setup-status 限流 1 次/60s/IP → 整个测试只调用一次
- POST /auth/login/local 限流 5 次/5min/IP → 本测试用 /register 建新租户,不走 login
- 跨租户访问线程返回 404(不是 403)→ 即隔离信号
"""
import os
import re
import json
import time
import uuid
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
import requests
BASE = os.environ.get("DF_BASE", "http://localhost:8001")
N_TENANTS = int(os.environ.get("DF_TENANTS", "3"))
N_TURNS = int(os.environ.get("DF_TURNS", "10")) # 每租户的对话轮数
# 同一批次唯一后缀,避免重复运行时 email 冲突
RUN_ID = uuid.uuid4().hex[:8]
PASSWORD = "DeerTenant-7x9q!" # ≥8 位且不在弱口令黑名单
_print_lock = threading.Lock()
def log(msg: str) -> None:
with _print_lock:
print(msg, flush=True)
def _csrf(s: requests.Session) -> dict:
token = s.cookies.get("csrf_token")
if not token:
raise RuntimeError("缺少 csrf_token cookie —— 鉴权可能失败")
return {"X-CSRF-Token": token}
def register(s: requests.Session, email: str) -> dict:
"""注册并自动登录(register 会同时下发 access_token + csrf_token cookie)。"""
r = s.post(f"{BASE}/api/v1/auth/register", json={"email": email, "password": PASSWORD})
r.raise_for_status()
return r.json() # {id, email, system_role}
def whoami(s: requests.Session) -> dict:
r = s.get(f"{BASE}/api/v1/auth/me")
r.raise_for_status()
return r.json() # {id, email, default_workspace_id, workspaces:[...]}
def create_thread(s: requests.Session) -> str:
r = s.post(f"{BASE}/api/threads", json={}, headers=_csrf(s))
r.raise_for_status()
return r.json()["thread_id"]
def stream_answer(s: requests.Session, thread_id: str, message: str) -> dict:
"""发一条消息,按 message-id 分组收集 AI 增量文本。
注意:TitleMiddleware 会另起一条 AI 消息生成线程标题,它和正文答复
是不同的 message-id。必须按 id 分组,否则正文数字会和标题数字粘连
(如 180 + "12乘15...""18012"),导致校验误判。
"""
body = {
"assistant_id": "lead_agent",
"input": {"messages": [{"role": "user", "content": message}]},
"stream_mode": ["messages-tuple", "values"],
}
headers = {**_csrf(s), "Accept": "text/event-stream"}
by_id: dict[str, str] = {}
with s.post(f"{BASE}/api/threads/{thread_id}/runs/stream", json=body, headers=headers, stream=True) as resp:
resp.raise_for_status()
event, buf = None, []
for raw in resp.iter_lines(decode_unicode=True):
if raw is None:
continue
line = raw.strip()
if line == "":
if event == "messages" and buf:
_collect(by_id, "\n".join(buf))
event, buf = None, []
elif line.startswith(":"):
continue
elif line.startswith("event:"):
event = line[6:].strip()
elif line.startswith("data:"):
buf.append(line[5:].strip())
return by_id # {message_id: text}
def _collect(by_id: dict, data: str) -> None:
try:
payload = json.loads(data)
except json.JSONDecodeError:
return
chunk = payload[0] if isinstance(payload, list) and payload else {}
if chunk.get("type") in ("ai", "AIMessageChunk"):
content = chunk.get("content")
if isinstance(content, str):
text = content
elif isinstance(content, list):
text = "".join(b.get("text", "") for b in content if isinstance(b, dict))
else:
text = ""
mid = chunk.get("id") or "_"
by_id[mid] = by_id.get(mid, "") + text
def search_threads(s: requests.Session) -> list:
r = s.post(f"{BASE}/api/threads/search", json={"limit": 100, "offset": 0}, headers=_csrf(s))
r.raise_for_status()
return r.json() # bare array of ThreadResponse
def get_thread_status(s: requests.Session, thread_id: str) -> int:
return s.get(f"{BASE}/api/threads/{thread_id}").status_code
def _contains(by_id: dict, n: int) -> bool:
"""某条 AI 消息里是否独立出现数字 n(按 message-id 分组比对,避免与标题数字粘连)。"""
return any(str(n) in re.findall(r"\d+", text.replace(",", "")) for text in by_id.values())
def _main_text(by_id: dict) -> str:
"""取最长的一条 AI 消息当正文(标题通常更短)。"""
return (max(by_id.values(), key=len) if by_id else "").strip()
# ── 一个租户的完整链路(在独立线程里跑)────────────────────────────────
def run_tenant(idx: int) -> dict:
"""N_TURNS 轮链式对话,复用同一 thread:
T1 = a×b;之后每轮「把上一条数字再加 d」(d 每租户不同)。
每轮都必须记得上一轮结果,逐轮校验,验证多轮上下文在并发下各自保持。
"""
email = f"tenant-{RUN_ID}-{idx}@example.com"
a, b = 11 + idx, 13 + idx * 2 # 每租户不同算式
d = 2 + idx # 每租户不同步长,进一步坐实无串扰
# 预先算出每轮期望值
expected = [a * b]
for _ in range(1, N_TURNS):
expected.append(expected[-1] + d)
s = requests.Session()
rec = {"idx": idx, "email": email, "d": d, "expected": expected, "turns": [], "ok": False}
t0 = time.time()
try:
user = register(s, email)
me = whoami(s)
rec["user_id"] = user["id"]
rec["workspace_id"] = me.get("default_workspace_id")
rec["t_start"] = t0
log(f"[租户{idx}] 注册完成 user={user['id'][:8]} ws={str(rec['workspace_id'])[:8]} d={d} email={email}")
tid = create_thread(s)
rec["thread_id"] = tid
for k in range(N_TURNS):
if k == 0:
q = f"只回答最终数字:{a} 乘以 {b} 等于多少?"
else:
q = f"把你上一条回答的那个数字再加 {d},只回答最终数字。"
by_id = stream_answer(s, tid, q)
hit = _contains(by_id, expected[k])
rec["turns"].append({"k": k + 1, "expected": expected[k], "text": _main_text(by_id), "ok": hit})
mark = "" if hit else ""
log(f"[租户{idx}] T{k + 1:>2}/{N_TURNS} 期望 {expected[k]:>5}{mark} {rec['turns'][-1]['text'][:24]!r}")
rec["turns_passed"] = sum(t["ok"] for t in rec["turns"])
rec["all_turns_ok"] = rec["turns_passed"] == N_TURNS
rec["context_ok"] = all(t["ok"] for t in rec["turns"][1:]) # 第 2 轮起依赖上下文
rec["t_end"] = time.time()
rec["session"] = s
rec["ok"] = True
log(f"[租户{idx}] ✓ 完成 {rec['turns_passed']}/{N_TURNS}")
except Exception as e: # noqa: BLE001
rec["error"] = f"{type(e).__name__}: {e}"
log(f"[租户{idx}] ✗ 失败:{rec['error']}")
return rec
def main() -> None:
log(f"=== 多租户并发测试 BASE={BASE} 租户数={N_TENANTS} 轮数={N_TURNS} 批次={RUN_ID} ===\n")
# setup-status 只调用一次(60s 限流)
try:
st = requests.get(f"{BASE}/api/v1/auth/setup-status", timeout=5)
if st.status_code == 200:
log(f"setup-status: {st.json()}")
if st.json().get("needs_setup"):
log("⚠ 系统尚未初始化管理员。请先创建管理员(apps/examples/http-chat/app.py 首启会建),再跑本测试。")
return
else:
log(f"setup-status: HTTP {st.status_code}(限流则忽略,按已初始化处理)")
except Exception as e: # noqa: BLE001
log(f"setup-status 请求失败:{e}")
# 并发跑所有租户
log(f"\n── 并发启动 {N_TENANTS} 个租户 ──")
results = []
with ThreadPoolExecutor(max_workers=N_TENANTS) as ex:
futs = [ex.submit(run_tenant, i) for i in range(N_TENANTS)]
for f in as_completed(futs):
results.append(f.result())
results.sort(key=lambda r: r["idx"])
ok = [r for r in results if r.get("ok")]
# 并发证据:对话时间窗是否重叠
log("\n── 并发证据(对话时间窗,相对秒)──")
if ok:
base_t = min(r["t_start"] for r in ok)
for r in ok:
s_off = r["t_start"] - base_t
e_off = r["t_end"] - base_t
bar = " " * int(s_off * 4) + "" * max(1, int((e_off - s_off) * 4))
log(f" 租户{r['idx']}: [{s_off:5.1f}s → {e_off:5.1f}s] {bar}")
spans = [(r["t_start"], r["t_end"]) for r in ok]
overlapped = any(
a[0] < b[1] and b[0] < a[1] for i, a in enumerate(spans) for b in spans[i + 1 :]
)
log(f" → 存在时间窗重叠(真并发):{overlapped}")
# 隔离校验
log("\n── 隔离校验 ──")
iso_pass = True
own_thread = {r["idx"]: r["thread_id"] for r in ok}
for r in ok:
s = r["session"]
mine = {t["thread_id"] for t in search_threads(s)}
# 1) search 只含自己的线程
only_own = mine == {r["thread_id"]} if mine else False
leaked = {own_thread[j] for j in own_thread if j != r["idx"]} & mine
# 2) 直接 GET 别人的线程 → 期望 404
cross_ok = True
for j, tid in own_thread.items():
if j == r["idx"]:
continue
code = get_thread_status(s, tid)
if code != 404:
cross_ok = False
log(f" ✗ 租户{r['idx']} 访问 租户{j} 的线程返回 {code}(期望 404")
if leaked:
iso_pass = False
log(f" ✗ 租户{r['idx']} 的 search 里出现了别人的线程:{leaked}")
if not cross_ok:
iso_pass = False
if only_own and cross_ok and not leaked:
log(f" ✓ 租户{r['idx']}:search 仅见己有线程,跨租户 GET 均 404")
# 汇总
log(f"\n── 汇总(每租户 {N_TURNS} 轮链式对话:T1=a×b,之后每轮 +d)──")
log(f"{'租户':<6}{'user_id':<12}{'thread':<14}{'步长d':<8}{'通过轮数':<12}{'逐轮':<14}{'状态'}")
for r in results:
if r.get("ok"):
seq = "".join("" if t["ok"] else "" for t in r["turns"])
passed = f"{r['turns_passed']}/{N_TURNS}"
log(f"{r['idx']:<6}{r['user_id'][:8]:<12}{r['thread_id'][:10]:<14}{r['d']:<8}{passed:<12}{seq:<14}OK")
else:
log(f"{r['idx']:<6}{'-':<12}{'-':<14}{'-':<8}{'-':<12}{'-':<14}FAIL: {r.get('error')}")
all_ok = len(ok) == N_TENANTS
turn1_ok = all(r["turns"][0]["ok"] for r in ok)
context_ok = all(r.get("context_ok") for r in ok) # 第 2 轮起全对
all_turns_ok = all(r.get("all_turns_ok") for r in ok) # N 轮全对
log("\n=== 结果 ===")
log(f" 租户全部成功: {all_ok} ({len(ok)}/{N_TENANTS})")
log(f" 首轮答复无串扰: {turn1_ok}")
log(f" 多轮上下文保持: {context_ok} ← 第 2~{N_TURNS} 轮每轮都依赖上一轮结果")
log(f" 全程 {N_TURNS} 轮全对: {all_turns_ok}")
log(f" 租户隔离: {iso_pass}")
verdict = all_ok and turn1_ok and context_ok and iso_pass
log(f" >>> {'PASS ✅' if verdict else 'FAIL ❌'}")
if __name__ == "__main__":
main()
@@ -0,0 +1,398 @@
#!/usr/bin/env python3
"""
Headless 多租户测试:以 **API KeyAuthorization: Bearer dfk_...** 的 server-to-server
方式跑多租户并发对话并验证隔离 —— 即 [multi_tenant.py] 的「无人值守 / 业务后端」版。
与 multi_tenant.py(浏览器式 cookie + CSRF)的区别:
- 每个租户先由一个 **人类 owner**cookie 会话)创建 service account 并 mint 一把
workspace-scoped API keyplaintext 仅返回一次);
- 之后所有对话只用 **Bearer key**(独立 Session、不带任何 cookie / CSRF),
模拟业务系统 backend 直连 Gateway。
运行(前提:仓库根已起 Gateway,如 `./scripts/dev-gateway.sh start`):
DF_BASE=http://localhost:8001 DF_TENANTS=4 \
uv run --no-project --with requests python multi_tenant_headless.py
# 没有 uv 时:pip install -r requirements.txt && python multi_tenant_headless.py
环境变量:DF_BASE(网关地址,默认 :8001)、DF_TENANTS(并发租户数,默认 3)、
DF_TURNS(每租户链式对话轮数,默认 10)、DF_EXTRA=0 可跳过 scope/撤销专项检查。
字段 / 端点(已对照 Stage 1 headless-api 实现):
人类鉴权 POST /api/v1/auth/{register,me} cookie + CSRF
建 SA POST /api/v1/service-accounts owner cookie + CSRF
mint key POST /api/v1/api-keys → {plaintext, key_prefix, id, ...}(仅此一次返 plaintext
撤销 key DELETE /api/v1/api-keys/{id} → 204
对话 POST /api/v1/threads /threads/search /threads/{id}/runs/streamBearer,无 CSRF
校验信号(来自实现):
- Bearer 路径下 thread 归属 user_id = service_account.id + workspace_id,与真人同构 → 隔离一致
- 跨 workspace 访问线程返回 404(藏存在性,非 403)
- key.scopes 经 AuthContext.permissions 灌入 @require_permission:缺 runs:create → stream 403
- 撤销后的 key → 401
"""
import json
import os
import re
import threading
import time
import uuid
from concurrent.futures import ThreadPoolExecutor, as_completed
import requests
BASE = os.environ.get("DF_BASE", "http://localhost:8001")
N_TENANTS = int(os.environ.get("DF_TENANTS", "3"))
N_TURNS = int(os.environ.get("DF_TURNS", "10")) # 每租户的对话轮数
EXTRA_CHECKS = os.environ.get("DF_EXTRA", "1") != "0" # scope / 撤销专项检查
RUN_ID = uuid.uuid4().hex[:8]
PASSWORD = "DeerTenant-7x9q!" # ≥8 位且不在弱口令黑名单
# 一把「全权」key 的 scope —— 覆盖 chat 链路需要的 runs:create / threads:read 等。
FULL_SCOPES = "threads:read,threads:write,threads:delete,runs:create,runs:read,runs:cancel"
_print_lock = threading.Lock()
def log(msg: str) -> None:
with _print_lock:
print(msg, flush=True)
# ── 人类 owner 侧(cookie + CSRF):注册 → 建 SA → mint key ─────────────
def _csrf(s: requests.Session) -> dict:
token = s.cookies.get("csrf_token")
if not token:
raise RuntimeError("缺少 csrf_token cookie —— 鉴权可能失败")
return {"X-CSRF-Token": token}
def register(s: requests.Session, email: str) -> dict:
r = s.post(f"{BASE}/api/v1/auth/register", json={"email": email, "password": PASSWORD})
r.raise_for_status()
return r.json() # {id, email, system_role}
def whoami(s: requests.Session) -> dict:
r = s.get(f"{BASE}/api/v1/auth/me")
r.raise_for_status()
return r.json() # {id, email, default_workspace_id, workspaces:[...]}
def create_service_account(s: requests.Session, name: str) -> dict:
r = s.post(f"{BASE}/api/v1/service-accounts", json={"name": name}, headers=_csrf(s))
r.raise_for_status()
return r.json() # {id, workspace_id, name, role, status, ...}
def mint_key(s: requests.Session, sa_id: str, name: str, scopes: str) -> dict:
r = s.post(
f"{BASE}/api/v1/api-keys",
json={"service_account_id": sa_id, "name": name, "scopes": scopes, "env": "live"},
headers=_csrf(s),
)
r.raise_for_status()
return r.json() # {id, key_prefix, plaintext, scopes, ...} ← plaintext 仅此一次
def revoke_key(s: requests.Session, key_id: str) -> int:
r = s.delete(f"{BASE}/api/v1/api-keys/{key_id}", headers=_csrf(s))
return r.status_code # 204 = 成功
# ── 业务后端侧(Bearer key,无 cookie / 无 CSRF)──────────────────────
def bearer_session(plaintext: str) -> requests.Session:
s = requests.Session()
s.headers.update({"Authorization": f"Bearer {plaintext}"})
return s
def create_thread(s: requests.Session) -> str:
r = s.post(f"{BASE}/api/v1/threads", json={})
r.raise_for_status()
return r.json()["thread_id"]
def stream_answer(s: requests.Session, thread_id: str, message: str) -> dict:
"""发一条消息,按 message-id 分组收集 AI 增量文本(TitleMiddleware 会另起一条
AI 消息生成标题,必须按 id 分组,否则正文数字会和标题数字粘连导致误判)。"""
body = {
"assistant_id": "lead_agent",
"input": {"messages": [{"role": "user", "content": message}]},
"stream_mode": ["messages-tuple", "values"],
}
headers = {"Accept": "text/event-stream"} # Bearer 已在 session.headers
by_id: dict[str, str] = {}
with s.post(f"{BASE}/api/v1/threads/{thread_id}/runs/stream", json=body, headers=headers, stream=True) as resp:
resp.raise_for_status()
event, buf = None, []
for raw in resp.iter_lines(decode_unicode=True):
if raw is None:
continue
line = raw.strip()
if line == "":
if event == "messages" and buf:
_collect(by_id, "\n".join(buf))
event, buf = None, []
elif line.startswith(":"):
continue
elif line.startswith("event:"):
event = line[6:].strip()
elif line.startswith("data:"):
buf.append(line[5:].strip())
return by_id # {message_id: text}
def _collect(by_id: dict, data: str) -> None:
try:
payload = json.loads(data)
except json.JSONDecodeError:
return
chunk = payload[0] if isinstance(payload, list) and payload else {}
if chunk.get("type") in ("ai", "AIMessageChunk"):
content = chunk.get("content")
if isinstance(content, str):
text = content
elif isinstance(content, list):
text = "".join(b.get("text", "") for b in content if isinstance(b, dict))
else:
text = ""
mid = chunk.get("id") or "_"
by_id[mid] = by_id.get(mid, "") + text
def search_threads(s: requests.Session) -> list:
r = s.post(f"{BASE}/api/v1/threads/search", json={"limit": 100, "offset": 0})
r.raise_for_status()
return r.json() # bare array of ThreadResponse
def get_thread_status(s: requests.Session, thread_id: str) -> int:
return s.get(f"{BASE}/api/v1/threads/{thread_id}").status_code
def _contains(by_id: dict, n: int) -> bool:
return any(str(n) in re.findall(r"\d+", text.replace(",", "")) for text in by_id.values())
def _main_text(by_id: dict) -> str:
return (max(by_id.values(), key=len) if by_id else "").strip()
# ── provisioning:人类 owner 建租户 + SA + key(顺序,cookie 侧)──────────
def provision_tenant(idx: int) -> dict:
email = f"htenant-{RUN_ID}-{idx}@example.com"
owner = requests.Session()
register(owner, email)
me = whoami(owner)
sa = create_service_account(owner, name=f"ci-bot-{idx}")
key = mint_key(owner, sa["id"], name="prod", scopes=FULL_SCOPES)
rec = {
"idx": idx,
"email": email,
"owner": owner,
"workspace_id": me.get("default_workspace_id"),
"sa_id": sa["id"],
"sa_role": sa.get("role"),
"key_id": key["id"],
"key_prefix": key["key_prefix"],
"plaintext": key["plaintext"],
"bearer": bearer_session(key["plaintext"]),
}
log(
f"[租户{idx}] 已开通 ws={str(rec['workspace_id'])[:8]} sa={sa['id'][:8]} "
f"key_prefix={key['key_prefix']} (sa_role={rec['sa_role']})"
)
return rec
# ── 一个租户的并发链路(Bearer key,独立线程)──────────────────────────
def run_tenant(prov: dict) -> dict:
idx = prov["idx"]
s = prov["bearer"]
a, b = 11 + idx, 13 + idx * 2 # 每租户不同算式
d = 2 + idx # 每租户不同步长,坐实无串扰
expected = [a * b]
for _ in range(1, N_TURNS):
expected.append(expected[-1] + d)
rec = {**prov, "d": d, "expected": expected, "turns": [], "ok": False}
t0 = time.time()
try:
rec["t_start"] = t0
tid = create_thread(s)
rec["thread_id"] = tid
for k in range(N_TURNS):
if k == 0:
q = f"只回答最终数字:{a} 乘以 {b} 等于多少?"
else:
q = f"把你上一条回答的那个数字再加 {d},只回答最终数字。"
by_id = stream_answer(s, tid, q)
hit = _contains(by_id, expected[k])
rec["turns"].append({"k": k + 1, "expected": expected[k], "text": _main_text(by_id), "ok": hit})
mark = "" if hit else ""
log(f"[租户{idx}] T{k + 1:>2}/{N_TURNS} 期望 {expected[k]:>5}{mark} {rec['turns'][-1]['text'][:24]!r}")
rec["turns_passed"] = sum(t["ok"] for t in rec["turns"])
rec["all_turns_ok"] = rec["turns_passed"] == N_TURNS
rec["context_ok"] = all(t["ok"] for t in rec["turns"][1:])
rec["t_end"] = time.time()
rec["ok"] = True
log(f"[租户{idx}] ✓ 完成 {rec['turns_passed']}/{N_TURNS}")
except Exception as e: # noqa: BLE001
rec["error"] = f"{type(e).__name__}: {e}"
log(f"[租户{idx}] ✗ 失败:{rec['error']}")
return rec
# ── headless 专项:scope 强制 + 撤销(在租户 0 的 owner 上做)──────────
def extra_checks(prov0: dict) -> dict:
owner = prov0["owner"]
sa_id = prov0["sa_id"]
out = {"scope_enforced": None, "revocation_401": None}
# 1) scope 强制:mint 一把只有 threads:read(无 runs:create)的 key → stream 应 403
try:
limited = mint_key(owner, sa_id, name="readonly", scopes="threads:read")
ls = bearer_session(limited["plaintext"])
tid = create_thread(ls) # 建线程不需要 scope(仅鉴权),应成功
body = {
"assistant_id": "lead_agent",
"input": {"messages": [{"role": "user", "content": "hi"}]},
"stream_mode": ["messages-tuple", "values"],
}
r = ls.post(f"{BASE}/api/v1/threads/{tid}/runs/stream", json=body, headers={"Accept": "text/event-stream"})
out["scope_enforced"] = r.status_code == 403
log(f" scope 强制:只读 key 发起 stream → HTTP {r.status_code}(期望 403{'' if out['scope_enforced'] else ''}")
revoke_key(owner, limited["id"])
except Exception as e: # noqa: BLE001
out["scope_error"] = f"{type(e).__name__}: {e}"
log(f" scope 强制:检查异常 {out['scope_error']}")
# 2) 撤销:mint 一把临时 key,验证可用 → 撤销 → 再用应 401
try:
tmp = mint_key(owner, sa_id, name="throwaway", scopes=FULL_SCOPES)
ts = bearer_session(tmp["plaintext"])
before = ts.post(f"{BASE}/api/v1/threads", json={}).status_code # 撤销前可建线程
code = revoke_key(owner, tmp["id"])
after = ts.post(f"{BASE}/api/v1/threads", json={}).status_code # 撤销后应 401
out["revocation_401"] = before in (200, 201) and code == 204 and after == 401
log(
f" 撤销:撤销前建线程 HTTP {before} → DELETE {code} → 撤销后 HTTP {after}"
f"(期望 2xx→204→401{'' if out['revocation_401'] else ''}"
)
except Exception as e: # noqa: BLE001
out["revocation_error"] = f"{type(e).__name__}: {e}"
log(f" 撤销:检查异常 {out['revocation_error']}")
return out
def main() -> None:
log(f"=== Headless 多租户测试 BASE={BASE} 租户数={N_TENANTS} 轮数={N_TURNS} 批次={RUN_ID} ===\n")
# setup-status 只调用一次(60s 限流)
try:
st = requests.get(f"{BASE}/api/v1/auth/setup-status", timeout=5)
if st.status_code == 200:
log(f"setup-status: {st.json()}")
if st.json().get("needs_setup"):
log("⚠ 系统尚未初始化管理员。请先创建管理员(app.py 首启会建),再跑本测试。")
return
else:
log(f"setup-status: HTTP {st.status_code}(限流则忽略,按已初始化处理)")
except Exception as e: # noqa: BLE001
log(f"setup-status 请求失败:{e}")
# 1) 顺序开通每个租户(人类 owner 建 SA + mint key
log(f"\n── 开通 {N_TENANTS} 个租户(owner cookie → SA → API key)──")
provs = []
for i in range(N_TENANTS):
try:
provs.append(provision_tenant(i))
except Exception as e: # noqa: BLE001
log(f"[租户{i}] ✗ 开通失败:{type(e).__name__}: {e}")
if not provs:
log("没有成功开通的租户,终止。")
return
# 2) 并发跑所有租户(只用 Bearer key)
log(f"\n── 并发启动 {len(provs)} 个租户的 Bearer 对话 ──")
results = []
with ThreadPoolExecutor(max_workers=len(provs)) as ex:
futs = [ex.submit(run_tenant, p) for p in provs]
for f in as_completed(futs):
results.append(f.result())
results.sort(key=lambda r: r["idx"])
ok = [r for r in results if r.get("ok")]
# 并发证据:对话时间窗是否重叠
log("\n── 并发证据(对话时间窗,相对秒)──")
if ok:
base_t = min(r["t_start"] for r in ok)
for r in ok:
s_off, e_off = r["t_start"] - base_t, r["t_end"] - base_t
bar = " " * int(s_off * 4) + "" * max(1, int((e_off - s_off) * 4))
log(f" 租户{r['idx']}: [{s_off:5.1f}s → {e_off:5.1f}s] {bar}")
spans = [(r["t_start"], r["t_end"]) for r in ok]
overlapped = any(a[0] < b[1] and b[0] < a[1] for i, a in enumerate(spans) for b in spans[i + 1 :])
log(f" → 存在时间窗重叠(真并发):{overlapped}")
# 3) 隔离校验(Bearer key 之间)
log("\n── 隔离校验(跨租户 Bearer)──")
iso_pass = True
own_thread = {r["idx"]: r["thread_id"] for r in ok}
for r in ok:
s = r["bearer"]
mine = {t["thread_id"] for t in search_threads(s)}
only_own = mine == {r["thread_id"]} if mine else False
leaked = {own_thread[j] for j in own_thread if j != r["idx"]} & mine
cross_ok = True
for j, tid in own_thread.items():
if j == r["idx"]:
continue
code = get_thread_status(s, tid)
if code != 404:
cross_ok = False
log(f" ✗ 租户{r['idx']} 的 key 访问 租户{j} 的线程返回 {code}(期望 404")
if leaked:
iso_pass = False
log(f" ✗ 租户{r['idx']} 的 search 里出现了别人的线程:{leaked}")
if not cross_ok:
iso_pass = False
if only_own and cross_ok and not leaked:
log(f" ✓ 租户{r['idx']}:search 仅见己有线程,跨租户 GET 均 404")
# 4) headless 专项(scope 强制 + 撤销)
extra = {}
if EXTRA_CHECKS and ok:
log("\n── Headless 专项检查(scope 强制 + 撤销)──")
extra = extra_checks(provs[0])
# 汇总
log(f"\n── 汇总(每租户 {N_TURNS} 轮链式对话:T1=a×b,之后每轮 +d)──")
log(f"{'租户':<6}{'sa_id':<12}{'key_prefix':<20}{'thread':<14}{'步长d':<8}{'通过轮数':<12}{'逐轮':<14}{'状态'}")
for r in results:
if r.get("ok"):
seq = "".join("" if t["ok"] else "" for t in r["turns"])
passed = f"{r['turns_passed']}/{N_TURNS}"
log(f"{r['idx']:<6}{r['sa_id'][:8]:<12}{r['key_prefix']:<20}{r['thread_id'][:10]:<14}{r['d']:<8}{passed:<12}{seq:<14}OK")
else:
log(f"{r['idx']:<6}{'-':<12}{'-':<20}{'-':<14}{'-':<8}{'-':<12}{'-':<14}FAIL: {r.get('error')}")
all_ok = len(ok) == len(provs) and len(provs) == N_TENANTS
turn1_ok = all(r["turns"][0]["ok"] for r in ok) if ok else False
context_ok = all(r.get("context_ok") for r in ok) if ok else False
log("\n=== 结果 ===")
log(f" 租户全部开通+成功: {all_ok} ({len(ok)}/{N_TENANTS})")
log(f" 首轮答复无串扰: {turn1_ok}")
log(f" 多轮上下文保持: {context_ok} ← 第 2~{N_TURNS} 轮每轮都依赖上一轮结果")
log(f" 租户隔离(Bearer): {iso_pass}")
if EXTRA_CHECKS:
log(f" scope 强制(403): {extra.get('scope_enforced')} ← 缺 runs:create 的 key 不能 stream")
log(f" 撤销即失效(401): {extra.get('revocation_401')}")
verdict = all_ok and turn1_ok and context_ok and iso_pass
if EXTRA_CHECKS:
verdict = verdict and extra.get("scope_enforced") and extra.get("revocation_401")
log(f" >>> {'PASS ✅' if verdict else 'FAIL ❌'}")
if __name__ == "__main__":
main()
+1
View File
@@ -0,0 +1 @@
requests>=2.31
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
#
# http-chat 示例启动脚本
# ------------------------------------------------------------------
# - 自动探测网关地址:优先 :2026(nginx 全量栈),回退 :8001(只起 Gateway)
# - 优先用 uv 临时虚拟环境带上 requests--no-project,不污染系统/项目)
# 没有 uv 时回退到本地 .venv + pip
#
# 用法:
# ./run.sh # 自动探测网关并运行
# DF_BASE=http://localhost:8001 ./run.sh # 手动指定网关
# DF_EMAIL=a@b.com DF_PASSWORD=xxxx ./run.sh
#
# 前提:先起好 Gateway
# ../../../scripts/dev-gateway.sh start # → :8001
# ../../../scripts/dev-full.sh start # → :2026
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
# ── 探测可用网关 ──────────────────────────────────────────────────
_alive() { curl -s -o /dev/null -w "%{http_code}" "$1/api/v1/auth/setup-status" 2>/dev/null | grep -qE "200|429"; }
if [ -z "${DF_BASE:-}" ]; then
if _alive "http://localhost:2026"; then DF_BASE="http://localhost:2026"
elif _alive "http://localhost:8001"; then DF_BASE="http://localhost:8001"
else
echo "✗ 没探测到运行中的网关(:2026 / :8001 都不通)。" >&2
echo " 先启动:scripts/dev-gateway.sh start 或 scripts/dev-full.sh start" >&2
echo " 或手动指定:DF_BASE=http://your-host:port ./run.sh" >&2
exit 1
fi
fi
export DF_BASE
echo "→ 使用网关: $DF_BASE"
# ── 运行:优先 uv,回退 venv+pip ─────────────────────────────────
if command -v uv >/dev/null 2>&1; then
echo "→ uv 临时环境运行(--with requests"
exec uv run --no-project --with "requests>=2.31" python app.py
else
echo "→ 未找到 uv,使用本地 .venv + pip"
if [ ! -d .venv ]; then
python3 -m venv .venv
./.venv/bin/pip install -q -r requirements.txt
fi
exec ./.venv/bin/python app.py
fi
+2
View File
@@ -102,6 +102,8 @@ Regression tests related to Docker/provisioner behavior:
Boundary check (harness → app import firewall): Boundary check (harness → app import firewall):
- `tests/test_harness_boundary.py` — ensures `packages/harness/deerflow/` never imports from `app.*` - `tests/test_harness_boundary.py` — ensures `packages/harness/deerflow/` never imports from `app.*`
- `tests/test_workspace_boundary.py` — AST static scan that forbids direct imports of `langgraph.checkpoint.*` (and the third-party `langgraph_checkpoint_postgres` / `langgraph_checkpoint_sqlite` packages) outside the allowlist in `tests/boundary_allowlist.toml`. Everywhere else must obtain a checkpointer via `app.gateway.deps.get_checkpointer` or the harness `deerflow.runtime.checkpointer` factory. Imports inside `if TYPE_CHECKING:` blocks are exempt automatically (they do not enter runtime). When a legitimate new importer is genuinely needed, append its path to `boundary_allowlist.toml` in the same PR
- `tests/test_workspace_boundary_self.py` — self-tests for the scanner above (9 cases over synthetic `.py` files) guarding against silent-empty regressions
CI runs these regression tests for every pull request via [.github/workflows/backend-unit-tests.yml](../.github/workflows/backend-unit-tests.yml). CI runs these regression tests for every pull request via [.github/workflows/backend-unit-tests.yml](../.github/workflows/backend-unit-tests.yml).
+103 -28
View File
@@ -10,9 +10,11 @@ from fastapi.middleware.cors import CORSMiddleware
from app.gateway.auth_middleware import AuthMiddleware from app.gateway.auth_middleware import AuthMiddleware
from app.gateway.config import get_gateway_config from app.gateway.config import get_gateway_config
from app.gateway.csrf_middleware import CSRFMiddleware from app.gateway.csrf_middleware import CSRFMiddleware
from app.gateway.deprecation_middleware import ApiDeprecationMiddleware
from app.gateway.deps import langgraph_runtime from app.gateway.deps import langgraph_runtime
from app.gateway.routers import ( from app.gateway.routers import (
agents, agents,
api_keys,
artifacts, artifacts,
assistants_compat, assistants_compat,
auth, auth,
@@ -22,6 +24,7 @@ from app.gateway.routers import (
memory, memory,
models, models,
runs, runs,
service_accounts,
skills, skills,
suggestions, suggestions,
thread_runs, thread_runs,
@@ -108,6 +111,23 @@ async def _ensure_admin_user(app: FastAPI) -> None:
admin_id = str(row.id) admin_id = str(row.id)
# Stage 0 PR4 backfill: pre-PR4 admins have no default_workspace_id.
# Create their personal workspace + owner membership on next boot so
# they can log in and pass the workspace gate without hand-rolling
# SQL. Idempotent — ensure_default_workspace short-circuits when the
# column is already set.
try:
admin_user = await provider.get_user(admin_id)
if admin_user is not None and not admin_user.default_workspace_id:
from app.gateway.routers.auth import ensure_default_workspace
ws_id = await ensure_default_workspace(admin_user)
logger.info("Backfilled default workspace %s for admin %s", ws_id, admin_id)
except Exception:
# Don't fail startup if backfill stumbles — the user can still
# log in (login_local calls the same helper on its hot path).
logger.exception("Admin workspace backfill failed (non-fatal)")
# LangGraph store orphan migration — non-fatal. # LangGraph store orphan migration — non-fatal.
# This covers the "no-auth → with-auth" upgrade path for users # This covers the "no-auth → with-auth" upgrade path for users
# whose existing LangGraph thread metadata has no user_id set. # whose existing LangGraph thread metadata has no user_id set.
@@ -158,6 +178,34 @@ async def _migrate_orphaned_threads(store, admin_user_id: str) -> int:
return migrated return migrated
def _check_path_migration_pending(app: FastAPI) -> None:
"""Warn the operator if the PR4 legacy user-isolation layout still has content.
PR6 routes every new write into ``{base_dir}/workspaces/{wid}/...`` via
``Paths``. Pre-PR6 installations have data at
``{base_dir}/users/{uid}/...`` that needs ``make migrate-paths`` to lift
it under a workspace. We emit a warning at boot rather than crashing so
the gateway keeps serving (reads from the legacy tree still work via the
user_id branch of ``Paths.thread_dir``), but with a loud signal to run
the migration script.
"""
from deerflow.config.paths import get_paths
legacy_users = get_paths().base_dir / "users"
if not legacy_users.exists():
return
try:
has_content = any(legacy_users.iterdir())
except OSError:
# Permission or transient FS issue — don't escalate; lifespan must succeed.
return
if has_content:
logger.warning(
"Legacy per-user layout detected at %s. Run `make migrate-paths` to lift it under the per-workspace layout (PR6).",
legacy_users,
)
@asynccontextmanager @asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]: async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
"""Application lifespan handler.""" """Application lifespan handler."""
@@ -174,6 +222,8 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
config = get_gateway_config() config = get_gateway_config()
logger.info(f"Starting API Gateway on {config.host}:{config.port}") logger.info(f"Starting API Gateway on {config.host}:{config.port}")
_check_path_migration_pending(app)
# Initialize LangGraph runtime components (StreamBridge, RunManager, checkpointer, store) # Initialize LangGraph runtime components (StreamBridge, RunManager, checkpointer, store)
async with langgraph_runtime(app): async with langgraph_runtime(app):
logger.info("LangGraph runtime initialised") logger.info("LangGraph runtime initialised")
@@ -307,6 +357,9 @@ This gateway provides custom endpoints for models, MCP configuration, skills, an
# CSRF: Double Submit Cookie pattern for state-changing requests # CSRF: Double Submit Cookie pattern for state-changing requests
app.add_middleware(CSRFMiddleware) app.add_middleware(CSRFMiddleware)
# Deprecation: stamp X-API-Deprecated on unversioned /api/* responses
app.add_middleware(ApiDeprecationMiddleware)
# CORS: when GATEWAY_CORS_ORIGINS is set (dev without nginx), add CORS middleware. # CORS: when GATEWAY_CORS_ORIGINS is set (dev without nginx), add CORS middleware.
# In production, nginx handles CORS and no middleware is needed. # In production, nginx handles CORS and no middleware is needed.
cors_origins_env = os.environ.get("GATEWAY_CORS_ORIGINS", "") cors_origins_env = os.environ.get("GATEWAY_CORS_ORIGINS", "")
@@ -328,50 +381,72 @@ This gateway provides custom endpoints for models, MCP configuration, skills, an
) )
# Include routers # Include routers
# Models API is mounted at /api/models # Legacy routers are dual-mounted on /api (backward compat) and /api/v1 (versioned).
app.include_router(models.router) # The deprecation middleware (Task 5.1) stamps X-API-Deprecated on /api responses.
# MCP API is mounted at /api/mcp # Models API — /api/models and /api/v1/models
app.include_router(mcp.router) app.include_router(models.router, prefix="/api")
app.include_router(models.router, prefix="/api/v1")
# Memory API is mounted at /api/memory # MCP API — /api/mcp and /api/v1/mcp
app.include_router(memory.router) app.include_router(mcp.router, prefix="/api")
app.include_router(mcp.router, prefix="/api/v1")
# Skills API is mounted at /api/skills # Memory API — /api/memory and /api/v1/memory
app.include_router(skills.router) app.include_router(memory.router, prefix="/api")
app.include_router(memory.router, prefix="/api/v1")
# Artifacts API is mounted at /api/threads/{thread_id}/artifacts # Skills API — /api/skills and /api/v1/skills
app.include_router(artifacts.router) app.include_router(skills.router, prefix="/api")
app.include_router(skills.router, prefix="/api/v1")
# Uploads API is mounted at /api/threads/{thread_id}/uploads # Artifacts API — /api/threads/{thread_id}/artifacts and /api/v1/threads/{thread_id}/artifacts
app.include_router(uploads.router) app.include_router(artifacts.router, prefix="/api")
app.include_router(artifacts.router, prefix="/api/v1")
# Thread cleanup API is mounted at /api/threads/{thread_id} # Uploads API — /api/threads/{thread_id}/uploads and /api/v1/threads/{thread_id}/uploads
app.include_router(threads.router) app.include_router(uploads.router, prefix="/api")
app.include_router(uploads.router, prefix="/api/v1")
# Agents API is mounted at /api/agents # Threads API — /api/threads/{thread_id} and /api/v1/threads/{thread_id}
app.include_router(agents.router) app.include_router(threads.router, prefix="/api")
app.include_router(threads.router, prefix="/api/v1")
# Suggestions API is mounted at /api/threads/{thread_id}/suggestions # Agents API — /api/agents and /api/v1/agents
app.include_router(suggestions.router) app.include_router(agents.router, prefix="/api")
app.include_router(agents.router, prefix="/api/v1")
# Channels API is mounted at /api/channels # Suggestions API — /api/threads/{thread_id}/suggestions and /api/v1/threads/{thread_id}/suggestions
app.include_router(channels.router) app.include_router(suggestions.router, prefix="/api")
app.include_router(suggestions.router, prefix="/api/v1")
# Assistants compatibility API (LangGraph Platform stub) # Channels API — /api/channels and /api/v1/channels
app.include_router(channels.router, prefix="/api")
app.include_router(channels.router, prefix="/api/v1")
# Assistants compatibility API (LangGraph Platform stub) — intentionally NOT dual-mounted
app.include_router(assistants_compat.router) app.include_router(assistants_compat.router)
# Auth API is mounted at /api/v1/auth # Auth API — /api/v1/auth only (already versioned; must NOT get an /api/auth twin)
app.include_router(auth.router) app.include_router(auth.router)
# Feedback API is mounted at /api/threads/{thread_id}/runs/{run_id}/feedback # Service Accounts API — /api/v1/service-accounts only (already versioned)
app.include_router(feedback.router) app.include_router(service_accounts.router)
# Thread Runs API (LangGraph Platform-compatible runs lifecycle) # API Keys API — /api/v1/api-keys only (already versioned)
app.include_router(thread_runs.router) app.include_router(api_keys.router)
# Stateless Runs API (stream/wait without a pre-existing thread) # Feedback API — /api/threads/{thread_id}/runs/{run_id}/feedback and /api/v1/... twin
app.include_router(runs.router) app.include_router(feedback.router, prefix="/api")
app.include_router(feedback.router, prefix="/api/v1")
# Thread Runs API — /api/threads/{thread_id}/runs and /api/v1/... twin
app.include_router(thread_runs.router, prefix="/api")
app.include_router(thread_runs.router, prefix="/api/v1")
# Stateless Runs API — /api/runs and /api/v1/runs
app.include_router(runs.router, prefix="/api")
app.include_router(runs.router, prefix="/api/v1")
@app.get("/health", tags=["health"]) @app.get("/health", tags=["health"])
async def health_check() -> dict: async def health_check() -> dict:
@@ -0,0 +1,97 @@
"""API key authentication backend (Stage 1 PR2).
Resolves an ``Authorization: Bearer dfk_...`` token into a
``ServicePrincipal`` + workspace + scopes, so ``AuthMiddleware`` can
stamp the same contextvars a cookie-authenticated human would set
(spec D1: user_id = SA.id).
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from deerflow.auth.tokens import hash_api_key, split_prefix
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class ServicePrincipal:
"""Non-human principal backing an API key. Satisfies the
``deerflow.runtime.user_context.CurrentUser`` protocol."""
id: str
is_service_account: bool = True
def parse_scopes(scopes: str) -> list[str]:
"""Parse a comma-separated scope string into a permission list.
``"threads:read, threads:write"`` -> ``["threads:read", "threads:write"]``.
Empty / whitespace-only segments are dropped.
"""
return [s.strip() for s in scopes.split(",") if s.strip()]
@dataclass(frozen=True)
class ApiKeyAuthResult:
"""Everything ``AuthMiddleware`` needs to stamp request state +
contextvars from a verified API key."""
principal: ServicePrincipal
workspace_id: str
role: str
permissions: list[str]
class APIKeyAuthBackend:
def __init__(self, *, api_key_repo, service_account_repo, workspace_repo) -> None:
self._api_key_repo = api_key_repo
self._service_account_repo = service_account_repo
self._workspace_repo = workspace_repo
async def authenticate(self, token: str) -> ApiKeyAuthResult | None:
"""Resolve a plaintext token to an auth result, or None (→ 401)."""
# Look up by the indexed public prefix; the repo constant-time
# verifies the full hash.
key = await self._api_key_repo.get_active_by_hash(hash_api_key(token), key_prefix=split_prefix(token))
if key is None:
return None
sa = await self._service_account_repo.get_active(key["service_account_id"])
if sa is None:
return None
# SA is not a workspace *member* — bypass the membership filter
# with the documented user_id=None admin/migration path.
workspace = await self._workspace_repo.get(sa["workspace_id"], user_id=None)
if workspace is None or workspace["status"] != "active":
return None
# Best-effort: never block the request if the timestamp write fails.
try:
await self._api_key_repo.touch_last_used(key["id"])
except Exception: # noqa: BLE001 — best-effort, log and continue
logger.warning("touch_last_used failed for api_key %s", key["id"], exc_info=True)
return ApiKeyAuthResult(
principal=ServicePrincipal(id=sa["id"]),
workspace_id=sa["workspace_id"],
role=sa["role"],
permissions=parse_scopes(key["scopes"]),
)
def build_api_key_backend() -> APIKeyAuthBackend | None:
"""Construct a backend from the global session factory, or None when
persistence is the in-memory backend (no DB → no API keys)."""
from deerflow.persistence.api_key import ApiKeyRepository
from deerflow.persistence.engine import get_session_factory
from deerflow.persistence.service_account import ServiceAccountRepository
from deerflow.persistence.workspace import WorkspaceRepository
sf = get_session_factory()
if sf is None:
return None
return APIKeyAuthBackend(api_key_repo=ApiKeyRepository(sf), service_account_repo=ServiceAccountRepository(sf), workspace_repo=WorkspaceRepository(sf))
+5
View File
@@ -21,6 +21,8 @@ class AuthErrorCode(StrEnum):
PROVIDER_NOT_FOUND = "provider_not_found" PROVIDER_NOT_FOUND = "provider_not_found"
NOT_AUTHENTICATED = "not_authenticated" NOT_AUTHENTICATED = "not_authenticated"
SYSTEM_ALREADY_INITIALIZED = "system_already_initialized" SYSTEM_ALREADY_INITIALIZED = "system_already_initialized"
WORKSPACE_REQUIRED = "workspace_required"
INSUFFICIENT_SCOPE = "insufficient_scope"
class TokenError(StrEnum): class TokenError(StrEnum):
@@ -29,6 +31,7 @@ class TokenError(StrEnum):
EXPIRED = "expired" EXPIRED = "expired"
INVALID_SIGNATURE = "invalid_signature" INVALID_SIGNATURE = "invalid_signature"
MALFORMED = "malformed" MALFORMED = "malformed"
WORKSPACE_MISSING = "workspace_missing"
class AuthErrorResponse(BaseModel): class AuthErrorResponse(BaseModel):
@@ -42,4 +45,6 @@ def token_error_to_code(err: TokenError) -> AuthErrorCode:
"""Map TokenError to AuthErrorCode — single source of truth.""" """Map TokenError to AuthErrorCode — single source of truth."""
if err == TokenError.EXPIRED: if err == TokenError.EXPIRED:
return AuthErrorCode.TOKEN_EXPIRED return AuthErrorCode.TOKEN_EXPIRED
if err == TokenError.WORKSPACE_MISSING:
return AuthErrorCode.WORKSPACE_REQUIRED
return AuthErrorCode.TOKEN_INVALID return AuthErrorCode.TOKEN_INVALID
+40 -8
View File
@@ -1,6 +1,7 @@
"""JWT token creation and verification.""" """JWT token creation and verification."""
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
from typing import Any
import jwt import jwt
from pydantic import BaseModel from pydantic import BaseModel
@@ -10,30 +11,51 @@ from app.gateway.auth.errors import TokenError
class TokenPayload(BaseModel): class TokenPayload(BaseModel):
"""JWT token payload.""" """JWT token payload.
`wid` / `role` were added in Stage 0 PR4 and are optional at the
model level so legacy 4-field tokens still parse into a value —
callers (middleware, decode_token) decide what "missing wid" means.
Post-PR4 production tokens always carry both fields.
"""
sub: str # user_id sub: str # user_id
wid: str | None = None # workspace_id (Stage 0 PR4)
role: str | None = None # owner / admin / member (Stage 0 PR4)
exp: datetime exp: datetime
iat: datetime | None = None iat: datetime | None = None
ver: int = 0 # token_version — must match User.token_version ver: int = 0 # token_version — must match User.token_version
def create_access_token(user_id: str, expires_delta: timedelta | None = None, token_version: int = 0) -> str: def create_access_token(
user_id: str,
expires_delta: timedelta | None = None,
token_version: int = 0,
*,
workspace_id: str | None = None,
role: str | None = None,
) -> str:
"""Create a JWT access token. """Create a JWT access token.
Args: Args:
user_id: The user's UUID as string user_id: The user's UUID as string.
expires_delta: Optional custom expiry, defaults to 7 days expires_delta: Optional custom expiry, defaults to 7 days.
token_version: User's current token_version for invalidation token_version: User's current token_version for invalidation.
workspace_id: Optional active workspace id; encoded as the ``wid`` claim.
role: Optional workspace role; encoded as the ``role`` claim.
Returns: Returns:
Encoded JWT string Encoded JWT string.
""" """
config = get_auth_config() config = get_auth_config()
expiry = expires_delta or timedelta(days=config.token_expiry_days) expiry = expires_delta or timedelta(days=config.token_expiry_days)
now = datetime.now(UTC) now = datetime.now(UTC)
payload = {"sub": user_id, "exp": now + expiry, "iat": now, "ver": token_version} payload: dict[str, Any] = {"sub": user_id, "exp": now + expiry, "iat": now, "ver": token_version}
if workspace_id is not None:
payload["wid"] = workspace_id
if role is not None:
payload["role"] = role
return jwt.encode(payload, config.jwt_secret, algorithm="HS256") return jwt.encode(payload, config.jwt_secret, algorithm="HS256")
@@ -46,10 +68,20 @@ def decode_token(token: str) -> TokenPayload | TokenError:
config = get_auth_config() config = get_auth_config()
try: try:
payload = jwt.decode(token, config.jwt_secret, algorithms=["HS256"]) payload = jwt.decode(token, config.jwt_secret, algorithms=["HS256"])
return TokenPayload(**payload)
except jwt.ExpiredSignatureError: except jwt.ExpiredSignatureError:
return TokenError.EXPIRED return TokenError.EXPIRED
except jwt.InvalidSignatureError: except jwt.InvalidSignatureError:
return TokenError.INVALID_SIGNATURE return TokenError.INVALID_SIGNATURE
except jwt.PyJWTError: except jwt.PyJWTError:
return TokenError.MALFORMED return TokenError.MALFORMED
# Reject legacy pre-PR4 tokens that lack the wid claim. Reported as
# WORKSPACE_MISSING (not MALFORMED) so middleware can surface a
# specific 401 telling the frontend to re-issue via /select-workspace.
if "wid" not in payload or payload.get("wid") is None:
return TokenError.WORKSPACE_MISSING
try:
return TokenPayload(**payload)
except Exception:
return TokenError.MALFORMED
+46
View File
@@ -31,6 +31,17 @@ class User(BaseModel):
needs_setup: bool = Field(default=False, description="True for auto-created admin until setup completes") needs_setup: bool = Field(default=False, description="True for auto-created admin until setup completes")
token_version: int = Field(default=0, description="Incremented on password change to invalidate old JWTs") token_version: int = Field(default=0, description="Incremented on password change to invalidate old JWTs")
# Headless API discriminator (Stage 1 PR2). Always False for human
# users; ServicePrincipal sets it True. Lets downstream code branch
# on principal kind without isinstance gymnastics.
is_service_account: bool = Field(default=False, description="True only for API-key service accounts, never for human users")
# Workspace linkage (Stage 0 PR4)
default_workspace_id: str | None = Field(
default=None,
description="The workspace the user lands in by default after login. NULL → /select-workspace.",
)
class UserResponse(BaseModel): class UserResponse(BaseModel):
"""Response model for user info endpoint.""" """Response model for user info endpoint."""
@@ -39,3 +50,38 @@ class UserResponse(BaseModel):
email: str email: str
system_role: Literal["admin", "user"] system_role: Literal["admin", "user"]
needs_setup: bool = False needs_setup: bool = False
class UserMeWorkspace(BaseModel):
"""One workspace entry in ``GET /auth/me`` (Stage 0 PR4)."""
id: str
name: str
slug: str
role: str
class UserMeResponse(BaseModel):
"""Response model for ``GET /auth/me`` — extends UserResponse with workspaces."""
id: str
email: str
system_role: Literal["admin", "user"]
needs_setup: bool = False
default_workspace_id: str | None = None
workspaces: list[UserMeWorkspace] = []
class ActiveWorkspace(BaseModel):
"""Lightweight workspace proxy injected into the request-scoped contextvar.
Implements the structural ``CurrentWorkspace`` protocol expected by
``deerflow.runtime.workspace_context``: only ``.id`` (str) and
``.role`` (str) are required. We intentionally do *not* embed the
full ``WorkspaceRow`` here — the middleware needs to set the
contextvar on every request and an extra DB lookup just to populate
a name/slug we don't use yet would be wasted work.
"""
id: str
role: str
@@ -46,6 +46,7 @@ class SQLiteUserRepository(UserRepository):
oauth_id=row.oauth_id, oauth_id=row.oauth_id,
needs_setup=row.needs_setup, needs_setup=row.needs_setup,
token_version=row.token_version, token_version=row.token_version,
default_workspace_id=row.default_workspace_id,
) )
@staticmethod @staticmethod
@@ -60,6 +61,7 @@ class SQLiteUserRepository(UserRepository):
oauth_id=user.oauth_id, oauth_id=user.oauth_id,
needs_setup=user.needs_setup, needs_setup=user.needs_setup,
token_version=user.token_version, token_version=user.token_version,
default_workspace_id=user.default_workspace_id,
) )
# ── CRUD ────────────────────────────────────────────────────────── # ── CRUD ──────────────────────────────────────────────────────────
@@ -106,6 +108,7 @@ class SQLiteUserRepository(UserRepository):
row.oauth_id = user.oauth_id row.oauth_id = user.oauth_id
row.needs_setup = user.needs_setup row.needs_setup = user.needs_setup
row.token_version = user.token_version row.token_version = user.token_version
row.default_workspace_id = user.default_workspace_id
await session.commit() await session.commit()
return user return user
@@ -0,0 +1,83 @@
"""Workspace slug helpers for the registration / initialize flow.
Stage 0 PR4 T4.10. Two responsibilities:
1. ``auto_slug_from_email(email)`` — pure transform from an email's
local part to a base slug that matches the schema's
``^[a-z0-9](-?[a-z0-9])*$`` pattern.
2. ``next_available_slug(base, exists_check=...)`` — collision walker
that appends ``-2``, ``-3``, … until ``exists_check`` reports the
candidate is free. Kept separate from ``auto_slug_from_email`` so
the pure function can be tested without a database.
Lives in the auth package (not in ``persistence``) because the input
is the user's email — a registration-time concept that doesn't belong
in a generic ``WorkspaceRepository``.
"""
from __future__ import annotations
import re
import secrets
from collections.abc import Awaitable, Callable
# Mirror the schema's slug rules from
# ``deerflow.persistence.workspace.sql`` so callers of this module
# never need to import private constants from persistence.
_SLUG_MIN_LEN = 3
_SLUG_MAX_LEN = 32
def auto_slug_from_email(email: str) -> str:
"""Map an email to a deterministic, schema-valid base slug.
Algorithm (from workspace-schema-design §3.1):
1. Take the local part (before ``@``).
2. Replace ``+``, ``_``, ``.`` with ``-`` and lowercase.
3. Strip everything that isn't ``[a-z0-9-]``.
4. Collapse repeated ``-``; strip leading/trailing ``-``.
5. Clamp to 32 chars.
6. If the result is shorter than the schema minimum (3 chars) or
empty, fall back to ``user-{token_hex(4)}`` so we always emit
a valid slug.
The returned slug is the *base* — callers must run it through
:func:`next_available_slug` before persisting to handle collisions.
"""
local = email.split("@", 1)[0]
local = re.sub(r"[+_.]", "-", local).lower()
local = re.sub(r"[^a-z0-9-]", "", local)
local = re.sub(r"-+", "-", local).strip("-")
slug = local[:_SLUG_MAX_LEN]
if len(slug) < _SLUG_MIN_LEN:
return f"user-{secrets.token_hex(4)}"
return slug
async def next_available_slug(
base: str,
*,
exists_check: Callable[[str], Awaitable[bool]],
) -> str:
"""Return the first of ``base``, ``base-2``, ``base-3``, … that ``exists_check`` reports free.
Caller-supplied ``exists_check`` is awaited once per candidate so
we can swap in a repository's ``get_by_slug`` without coupling
this module to persistence imports.
When ``base + '-N'`` would exceed the 32-char schema limit, the
base is truncated before the suffix is appended. The walker never
returns an over-long slug.
"""
if not await exists_check(base):
return base
n = 2
while True:
suffix = f"-{n}"
max_base_len = _SLUG_MAX_LEN - len(suffix)
candidate = f"{base[:max_base_len]}{suffix}"
if not await exists_check(candidate):
return candidate
n += 1
+89 -2
View File
@@ -9,6 +9,7 @@ owner filtering works automatically via the sentinel pattern.
Fine-grained permission checks remain in authz.py decorators. Fine-grained permission checks remain in authz.py decorators.
""" """
import logging
from collections.abc import Callable from collections.abc import Callable
from fastapi import HTTPException, Request, Response from fastapi import HTTPException, Request, Response
@@ -16,10 +17,15 @@ from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import JSONResponse from starlette.responses import JSONResponse
from starlette.types import ASGIApp from starlette.types import ASGIApp
from app.gateway.auth.api_key_backend import build_api_key_backend
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse
from app.gateway.auth.models import ActiveWorkspace
from app.gateway.authz import _ALL_PERMISSIONS, AuthContext from app.gateway.authz import _ALL_PERMISSIONS, AuthContext
from app.gateway.internal_auth import INTERNAL_AUTH_HEADER_NAME, get_internal_user, is_valid_internal_auth_token from app.gateway.internal_auth import INTERNAL_AUTH_HEADER_NAME, get_internal_user, is_valid_internal_auth_token
from deerflow.runtime.user_context import reset_current_user, set_current_user from deerflow.runtime.user_context import reset_current_user, set_current_user
from deerflow.runtime.workspace_context import reset_current_workspace, set_current_workspace
logger = logging.getLogger(__name__)
# Paths that never require authentication. # Paths that never require authentication.
_PUBLIC_PATH_PREFIXES: tuple[str, ...] = ( _PUBLIC_PATH_PREFIXES: tuple[str, ...] = (
@@ -49,6 +55,34 @@ def _is_public(path: str) -> bool:
return any(path.startswith(prefix) for prefix in _PUBLIC_PATH_PREFIXES) return any(path.startswith(prefix) for prefix in _PUBLIC_PATH_PREFIXES)
# Data-plane / SDK route prefixes a service principal (API key) may reach.
# Everything else (global control plane: models/mcp/memory/skills/channels/
# agents, plus management/auth endpoints) is denied by default for API keys.
# NOTE: nginx rewrites /api/langgraph/(.*) -> /api/$1 before the gateway, so
# AuthMiddleware never sees /api/langgraph; the SDK surface arrives as
# /api/threads, /api/runs, /api/assistants. assistants.search()/get() is
# required for langgraph-sdk client init, so /api/assistants is allowed.
_DATAPLANE_PREFIXES: tuple[str, ...] = (
"/api/threads",
"/api/v1/threads",
"/api/runs",
"/api/v1/runs",
"/api/assistants",
)
def _is_dataplane_path(path: str) -> bool:
"""True if an API key request may reach this path. Reusable by a future
Pattern B service-token branch.
Matches a prefix only at a path-segment boundary (exact match, or the
prefix immediately followed by ``/``), so the allowlist can't be silently
widened by a similarly-named route — e.g. ``/api/threads-export`` shares
the ``/api/threads`` prefix but crosses no segment boundary, so it stays
denied."""
return any(path == prefix or path.startswith(prefix + "/") for prefix in _DATAPLANE_PREFIXES)
class AuthMiddleware(BaseHTTPMiddleware): class AuthMiddleware(BaseHTTPMiddleware):
"""Strict auth gate: reject requests without a valid session. """Strict auth gate: reject requests without a valid session.
@@ -76,6 +110,45 @@ class AuthMiddleware(BaseHTTPMiddleware):
if _is_public(request.url.path): if _is_public(request.url.path):
return await call_next(request) return await call_next(request)
# API key path: "Authorization: Bearer dfk_..." authenticates a
# service account. Resolved principal is mapped to the same
# (user_id, workspace_id) contextvars a human would set (spec D1),
# so all downstream isolation works unchanged.
auth_header = request.headers.get("authorization", "")
if auth_header.startswith("Bearer dfk_"):
token = auth_header[len("Bearer ") :]
backend = build_api_key_backend()
try:
result = await backend.authenticate(token) if backend is not None else None
except Exception:
logger.exception("API key authentication failed unexpectedly")
return JSONResponse(status_code=503, content={"detail": "Authentication service unavailable"})
if result is None:
return JSONResponse(
status_code=401,
content={"detail": AuthErrorResponse(code=AuthErrorCode.TOKEN_INVALID, message="Invalid API key").model_dump()},
)
# Default-deny: a service principal may only reach the data plane
# (threads/runs/assistants). Control-plane routes (mcp/skills/
# channels/models/agents/memory + management/auth) are global,
# un-partitioned config — never reachable by an API key. New
# control-plane routes are denied automatically (allowlist, not
# blocklist). Humans (cookie path) never enter this branch.
if not _is_dataplane_path(request.url.path):
return JSONResponse(
status_code=403,
content={"detail": AuthErrorResponse(code=AuthErrorCode.INSUFFICIENT_SCOPE, message="API keys cannot access this endpoint").model_dump()},
)
request.state.user = result.principal
request.state.auth = AuthContext(user=result.principal, permissions=result.permissions)
user_token = set_current_user(result.principal)
ws_token = set_current_workspace(ActiveWorkspace(id=result.workspace_id, role=result.role))
try:
return await call_next(request)
finally:
reset_current_workspace(ws_token)
reset_current_user(user_token)
internal_user = None internal_user = None
if is_valid_internal_auth_token(request.headers.get(INTERNAL_AUTH_HEADER_NAME)): if is_valid_internal_auth_token(request.headers.get(INTERNAL_AUTH_HEADER_NAME)):
internal_user = get_internal_user() internal_user = get_internal_user()
@@ -119,8 +192,22 @@ class AuthMiddleware(BaseHTTPMiddleware):
# JWT-decode + DB-lookup pipeline a second time per request). # JWT-decode + DB-lookup pipeline a second time per request).
request.state.user = user request.state.user = user
request.state.auth = AuthContext(user=user, permissions=_ALL_PERMISSIONS) request.state.auth = AuthContext(user=user, permissions=_ALL_PERMISSIONS)
token = set_current_user(user) user_token = set_current_user(user)
# Inject workspace contextvar from the JWT's wid/role claims.
# decode_token has already rejected legacy no-wid tokens upstream,
# so by the time we get here payload.wid is guaranteed non-None
# for cookie-authenticated requests. Internal-auth requests skip
# the workspace contextvar (they don't have a workspace scope —
# the internal user is a system actor).
ws_token = None
payload = getattr(request.state, "auth_payload", None)
if payload is not None and payload.wid is not None:
ws_token = set_current_workspace(ActiveWorkspace(id=payload.wid, role=payload.role or "owner"))
try: try:
return await call_next(request) return await call_next(request)
finally: finally:
reset_current_user(token) if ws_token is not None:
reset_current_workspace(ws_token)
reset_current_user(user_token)
+34 -11
View File
@@ -38,6 +38,7 @@ from typing import TYPE_CHECKING, Any, ParamSpec, TypeVar
from fastapi import HTTPException, Request from fastapi import HTTPException, Request
if TYPE_CHECKING: if TYPE_CHECKING:
from app.gateway.auth.api_key_backend import ServicePrincipal
from app.gateway.auth.models import User from app.gateway.auth.models import User
P = ParamSpec("P") P = ParamSpec("P")
@@ -65,13 +66,14 @@ class AuthContext:
Stored in request.state.auth after require_auth decoration. Stored in request.state.auth after require_auth decoration.
Attributes: Attributes:
user: The authenticated user, or None if anonymous user: The authenticated principal (human ``User`` or
``ServicePrincipal`` for API-key requests), or None if anonymous
permissions: List of permission strings (e.g., "threads:read") permissions: List of permission strings (e.g., "threads:read")
""" """
__slots__ = ("user", "permissions") __slots__ = ("user", "permissions")
def __init__(self, user: User | None = None, permissions: list[str] | None = None): def __init__(self, user: User | ServicePrincipal | None = None, permissions: list[str] | None = None):
self.user = user self.user = user
self.permissions = permissions or [] self.permissions = permissions or []
@@ -93,8 +95,11 @@ class AuthContext:
permission = f"{resource}:{action}" permission = f"{resource}:{action}"
return permission in self.permissions return permission in self.permissions
def require_user(self) -> User: def require_user(self) -> User | ServicePrincipal:
"""Get user or raise 401. """Get the authenticated principal or raise 401.
Returns the human ``User`` or the ``ServicePrincipal`` backing an
API key, depending on how the request authenticated.
Raises: Raises:
HTTPException 401 if not authenticated HTTPException 401 if not authenticated
@@ -268,24 +273,27 @@ def require_permission(
# Owner check for thread-specific resources. # Owner check for thread-specific resources.
# #
# 2.0-rc moved thread metadata into the SQL persistence layer # PR6: ``check_access`` now takes ``workspace_id`` as the third
# (``threads_meta`` table). We verify ownership via # positional argument; cross-workspace always denies regardless
# ``ThreadMetaStore.check_access``: it returns True for # of user_id match. We pull workspace_id from the contextvar
# missing rows (untracked legacy thread) and for rows whose # AuthMiddleware sets per request (and fall back to "default"
# ``user_id`` is NULL (shared / pre-auth data), so this is # in no-auth dev mode so smoke flows keep working). Failures
# strict-deny rather than strict-allow — only an *existing* # convert to **404**, not 403, so the response never leaks the
# row with a *different* user_id triggers 404. # existence of a thread that belongs to a different tenant.
if owner_check: if owner_check:
thread_id = kwargs.get("thread_id") thread_id = kwargs.get("thread_id")
if thread_id is None: if thread_id is None:
raise ValueError("require_permission with owner_check=True requires 'thread_id' parameter") raise ValueError("require_permission with owner_check=True requires 'thread_id' parameter")
from app.gateway.deps import get_thread_store from app.gateway.deps import get_thread_store
from deerflow.runtime.workspace_context import get_effective_workspace_id
workspace_id = get_effective_workspace_id()
thread_store = get_thread_store(request) thread_store = get_thread_store(request)
allowed = await thread_store.check_access( allowed = await thread_store.check_access(
thread_id, thread_id,
str(auth.user.id), str(auth.user.id),
workspace_id,
require_existing=require_existing, require_existing=require_existing,
) )
if not allowed: if not allowed:
@@ -299,3 +307,18 @@ def require_permission(
return wrapper return wrapper
return decorator return decorator
def require_workspace_admin() -> None:
"""FastAPI dependency: require the caller's workspace role to be
owner or admin. Reads the role from the workspace contextvar that
AuthMiddleware stamps per request.
Raises HTTPException 403 if no workspace is in context or the role is
below admin. Use on management endpoints (service accounts, API keys).
"""
from deerflow.runtime.workspace_context import get_current_workspace
workspace = get_current_workspace()
if workspace is None or getattr(workspace, "role", None) not in ("owner", "admin"):
raise HTTPException(status_code=403, detail="workspace owner/admin role required")
+14 -1
View File
@@ -29,15 +29,28 @@ def generate_csrf_token() -> str:
return secrets.token_urlsafe(CSRF_TOKEN_LENGTH) return secrets.token_urlsafe(CSRF_TOKEN_LENGTH)
def has_bearer_header(request: Request) -> bool:
"""True if the request carries an ``Authorization: Bearer ...`` header.
Bearer requests authenticate via header, not cookie, so they are not
vulnerable to CSRF (the browser never auto-attaches a bearer header).
"""
return request.headers.get("authorization", "").startswith("Bearer ")
def should_check_csrf(request: Request) -> bool: def should_check_csrf(request: Request) -> bool:
"""Determine if a request needs CSRF validation. """Determine if a request needs CSRF validation.
CSRF is checked for state-changing methods (POST, PUT, DELETE, PATCH). CSRF is checked for state-changing methods (POST, PUT, DELETE, PATCH).
GET, HEAD, OPTIONS, and TRACE are exempt per RFC 7231. GET, HEAD, OPTIONS, and TRACE are exempt per RFC 7231. Bearer-header
(API key / token) requests are exempt — they don't ride on cookies.
""" """
if request.method not in ("POST", "PUT", "DELETE", "PATCH"): if request.method not in ("POST", "PUT", "DELETE", "PATCH"):
return False return False
if has_bearer_header(request):
return False
path = request.url.path.rstrip("/") path = request.url.path.rstrip("/")
# Exempt /api/v1/auth/me endpoint # Exempt /api/v1/auth/me endpoint
if path == "/api/v1/auth/me": if path == "/api/v1/auth/me":
@@ -0,0 +1,31 @@
"""Marks responses to legacy unversioned /api/* paths as deprecated.
Stamps ``X-API-Deprecated: <sunset-date>`` on any /api/* response that is
neither versioned (/api/v1/*) nor the LangGraph SDK surface
(/api/langgraph/*). Sunset date is the track-2 contract (2027-01-01).
"""
from __future__ import annotations
from collections.abc import Callable
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.types import ASGIApp
API_SUNSET_DATE = "2027-01-01"
def _is_deprecated_path(path: str) -> bool:
return path.startswith("/api/") and not path.startswith("/api/v1/") and not path.startswith("/api/langgraph/")
class ApiDeprecationMiddleware(BaseHTTPMiddleware):
def __init__(self, app: ASGIApp) -> None:
super().__init__(app)
async def dispatch(self, request: Request, call_next: Callable) -> Response:
response = await call_next(request)
if _is_deprecated_path(request.url.path):
response.headers["X-API-Deprecated"] = API_SUNSET_DATE
return response
+4
View File
@@ -220,6 +220,10 @@ async def get_current_user_from_request(request: Request):
detail=AuthErrorResponse(code=AuthErrorCode.TOKEN_INVALID, message="Token revoked (password changed)").model_dump(), detail=AuthErrorResponse(code=AuthErrorCode.TOKEN_INVALID, message="Token revoked (password changed)").model_dump(),
) )
# Stash decoded payload on request.state so AuthMiddleware can read
# wid/role for the workspace contextvar without a second decode.
request.state.auth_payload = payload
return user return user
+1 -1
View File
@@ -14,7 +14,7 @@ from deerflow.config.paths import get_paths
from deerflow.runtime.user_context import get_effective_user_id from deerflow.runtime.user_context import get_effective_user_id
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["agents"]) router = APIRouter(tags=["agents"])
AGENT_NAME_PATTERN = re.compile(r"^[A-Za-z0-9-]+$") AGENT_NAME_PATTERN = re.compile(r"^[A-Za-z0-9-]+$")
+111
View File
@@ -0,0 +1,111 @@
"""API key management endpoints (Stage 1 PR4).
Owner/admin mint / list / revoke API keys for a service account in the
caller's workspace. The plaintext token is returned exactly once, at
create time; list responses never include plaintext or the hash. The
target service account must belong to the caller's workspace, else 404.
"""
from __future__ import annotations
from datetime import datetime
from typing import Literal
from fastapi import APIRouter, Depends, HTTPException, Response
from pydantic import BaseModel, Field
from app.gateway.authz import require_workspace_admin
from deerflow.auth.tokens import generate_api_key
from deerflow.persistence.api_key import ApiKeyRepository
from deerflow.persistence.service_account import ServiceAccountRepository
from deerflow.runtime.workspace_context import get_current_workspace
router = APIRouter(prefix="/api/v1/api-keys", tags=["api-keys"])
class CreateApiKeyRequest(BaseModel):
service_account_id: str
name: str = Field(..., min_length=1, max_length=64)
scopes: str = Field(default="")
env: Literal["live", "test"] = "live"
expires_at: datetime | None = None
def get_api_key_repo() -> ApiKeyRepository:
from deerflow.persistence.engine import get_session_factory
sf = get_session_factory()
if sf is None:
raise HTTPException(status_code=503, detail="persistence backend not available")
return ApiKeyRepository(sf)
def get_service_account_repo() -> ServiceAccountRepository:
from deerflow.persistence.engine import get_session_factory
sf = get_session_factory()
if sf is None:
raise HTTPException(status_code=503, detail="persistence backend not available")
return ServiceAccountRepository(sf)
def _current_workspace_id() -> str:
ws = get_current_workspace()
if ws is None:
raise HTTPException(status_code=403, detail="no workspace in context")
return str(ws.id)
async def _require_sa_in_workspace(sa_id: str, sa_repo: ServiceAccountRepository) -> dict:
sa = await sa_repo.get(sa_id)
if sa is None or sa["workspace_id"] != _current_workspace_id():
raise HTTPException(status_code=404, detail="service account not found")
return sa
@router.post("", status_code=201, dependencies=[Depends(require_workspace_admin)])
async def create_api_key(
body: CreateApiKeyRequest,
key_repo: ApiKeyRepository = Depends(get_api_key_repo),
sa_repo: ServiceAccountRepository = Depends(get_service_account_repo),
):
sa = await _require_sa_in_workspace(body.service_account_id, sa_repo)
if sa["status"] != "active":
raise HTTPException(status_code=409, detail="service account is not active")
gen = generate_api_key(body.env)
created = await key_repo.create(
service_account_id=body.service_account_id,
key_prefix=gen.prefix,
key_hash=gen.key_hash,
name=body.name,
scopes=body.scopes,
expires_at=body.expires_at,
)
# plaintext returned exactly once; never persisted, never re-served.
return {**created, "plaintext": gen.plaintext}
@router.get("", dependencies=[Depends(require_workspace_admin)])
async def list_api_keys(
service_account_id: str,
key_repo: ApiKeyRepository = Depends(get_api_key_repo),
sa_repo: ServiceAccountRepository = Depends(get_service_account_repo),
):
await _require_sa_in_workspace(service_account_id, sa_repo)
return await key_repo.list_by_service_account(service_account_id)
@router.delete("/{key_id}", status_code=204, dependencies=[Depends(require_workspace_admin)])
async def revoke_api_key(
key_id: str,
key_repo: ApiKeyRepository = Depends(get_api_key_repo),
sa_repo: ServiceAccountRepository = Depends(get_service_account_repo),
):
key = await key_repo.get(key_id)
if key is None:
raise HTTPException(status_code=404, detail="api key not found")
sa = await sa_repo.get(key["service_account_id"])
if sa is None or sa["workspace_id"] != _current_workspace_id():
raise HTTPException(status_code=404, detail="api key not found")
await key_repo.revoke(key_id)
return Response(status_code=204)
+1 -1
View File
@@ -12,7 +12,7 @@ from app.gateway.path_utils import resolve_thread_virtual_path
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["artifacts"]) router = APIRouter(tags=["artifacts"])
ACTIVE_CONTENT_MIME_TYPES = { ACTIVE_CONTENT_MIME_TYPES = {
"text/html", "text/html",
+118 -8
View File
@@ -15,11 +15,60 @@ from app.gateway.auth import (
) )
from app.gateway.auth.config import get_auth_config from app.gateway.auth.config import get_auth_config
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse
from app.gateway.auth.models import UserMeResponse, UserMeWorkspace
from app.gateway.auth.workspace_slug import auto_slug_from_email, next_available_slug
from app.gateway.csrf_middleware import is_secure_request from app.gateway.csrf_middleware import is_secure_request
from app.gateway.deps import get_current_user_from_request, get_local_provider from app.gateway.deps import get_current_user_from_request, get_local_provider
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
async def ensure_default_workspace(user) -> str:
"""Create the user's personal workspace + owner membership, set default_workspace_id.
Returns the new workspace id. Idempotent for users who already
have a default_workspace_id (used by both the registration flow
and the lifespan backfill in app.py).
"""
from deerflow.persistence.engine import get_session_factory
from deerflow.persistence.workspace import WorkspaceRepository
from deerflow.persistence.workspace.sql import SLUG_BLACKLIST
from deerflow.persistence.workspace_membership import WorkspaceMembershipRepository
if user.default_workspace_id:
return user.default_workspace_id
sf = get_session_factory()
ws_repo = WorkspaceRepository(sf)
m_repo = WorkspaceMembershipRepository(sf)
base_slug = auto_slug_from_email(user.email)
async def slug_exists(s: str) -> bool:
# Treat blacklisted slugs as "taken" so the walker skips them
# instead of letting WorkspaceRepository.create raise after a
# successful slug computation (the user picked a reserved name
# like "admin@example.com" → base slug "admin").
if s in SLUG_BLACKLIST:
return True
return (await ws_repo.get_by_slug(s)) is not None
unique_slug = await next_available_slug(base_slug, exists_check=slug_exists)
display_local = user.email.split("@", 1)[0]
workspace = await ws_repo.create(
name=f"{display_local}'s Workspace"[:64],
slug=unique_slug,
owner_id=str(user.id),
)
await m_repo.add(workspace_id=workspace["id"], user_id=str(user.id), role="owner")
user.default_workspace_id = workspace["id"]
await get_local_provider().update_user(user)
return workspace["id"]
router = APIRouter(prefix="/api/v1/auth", tags=["auth"]) router = APIRouter(prefix="/api/v1/auth", tags=["auth"])
@@ -292,7 +341,15 @@ async def login_local(
) )
_record_login_success(client_ip) _record_login_success(client_ip)
token = create_access_token(str(user.id), token_version=user.token_version) # Ensure the user has a workspace (covers pre-PR4 users still in DB
# whose default_workspace_id was never backfilled by the lifespan hook).
workspace_id = await ensure_default_workspace(user)
token = create_access_token(
str(user.id),
token_version=user.token_version,
workspace_id=workspace_id,
role="owner",
)
_set_session_cookie(response, token, request) _set_session_cookie(response, token, request)
return LoginResponse( return LoginResponse(
@@ -316,7 +373,14 @@ async def register(request: Request, response: Response, body: RegisterRequest):
detail=AuthErrorResponse(code=AuthErrorCode.EMAIL_ALREADY_EXISTS, message="Email already registered").model_dump(), detail=AuthErrorResponse(code=AuthErrorCode.EMAIL_ALREADY_EXISTS, message="Email already registered").model_dump(),
) )
token = create_access_token(str(user.id), token_version=user.token_version) workspace_id = await ensure_default_workspace(user)
token = create_access_token(
str(user.id),
token_version=user.token_version,
workspace_id=workspace_id,
role="owner",
)
_set_session_cookie(response, token, request) _set_session_cookie(response, token, request)
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role) return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role)
@@ -368,18 +432,57 @@ async def change_password(request: Request, response: Response, body: ChangePass
await provider.update_user(user) await provider.update_user(user)
# Re-issue cookie with new token_version # Re-issue cookie with new token_version. wid + role must be carried
token = create_access_token(str(user.id), token_version=user.token_version) # forward so the re-signed JWT still passes the AuthMiddleware
# workspace gate; ensure_default_workspace fills in for the (rare)
# case where the user predates PR4 and has not been backfilled.
workspace_id = await ensure_default_workspace(user)
token = create_access_token(
str(user.id),
token_version=user.token_version,
workspace_id=workspace_id,
role="owner",
)
_set_session_cookie(response, token, request) _set_session_cookie(response, token, request)
return MessageResponse(message="Password changed successfully") return MessageResponse(message="Password changed successfully")
@router.get("/me", response_model=UserResponse) @router.get("/me", response_model=UserMeResponse)
async def get_me(request: Request): async def get_me(request: Request):
"""Get current authenticated user info.""" """Get current authenticated user info, including the workspaces they belong to."""
from deerflow.persistence.engine import get_session_factory
from deerflow.persistence.workspace import WorkspaceRepository
from deerflow.persistence.workspace_membership import WorkspaceMembershipRepository
user = await get_current_user_from_request(request) user = await get_current_user_from_request(request)
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role, needs_setup=user.needs_setup)
sf = get_session_factory()
workspaces: list[UserMeWorkspace] = []
if sf is not None:
ws_repo = WorkspaceRepository(sf)
m_repo = WorkspaceMembershipRepository(sf)
ws_rows = await ws_repo.list_by_user(user_id=str(user.id))
memberships = await m_repo.list_by_user(user_id=str(user.id))
role_by_ws = {m["workspace_id"]: m["role"] for m in memberships}
workspaces = [
UserMeWorkspace(
id=w["id"],
name=w["name"],
slug=w["slug"],
role=role_by_ws.get(w["id"], "member"),
)
for w in ws_rows
]
return UserMeResponse(
id=str(user.id),
email=user.email,
system_role=user.system_role,
needs_setup=user.needs_setup,
default_workspace_id=user.default_workspace_id,
workspaces=workspaces,
)
_SETUP_STATUS_COOLDOWN: dict[str, float] = {} _SETUP_STATUS_COOLDOWN: dict[str, float] = {}
@@ -452,7 +555,14 @@ async def initialize_admin(request: Request, response: Response, body: Initializ
detail=AuthErrorResponse(code=AuthErrorCode.SYSTEM_ALREADY_INITIALIZED, message="System already initialized").model_dump(), detail=AuthErrorResponse(code=AuthErrorCode.SYSTEM_ALREADY_INITIALIZED, message="System already initialized").model_dump(),
) )
token = create_access_token(str(user.id), token_version=user.token_version) workspace_id = await ensure_default_workspace(user)
token = create_access_token(
str(user.id),
token_version=user.token_version,
workspace_id=workspace_id,
role="owner",
)
_set_session_cookie(response, token, request) _set_session_cookie(response, token, request)
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role) return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role)
+1 -1
View File
@@ -9,7 +9,7 @@ from pydantic import BaseModel
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/channels", tags=["channels"]) router = APIRouter(prefix="/channels", tags=["channels"])
class ChannelStatusResponse(BaseModel): class ChannelStatusResponse(BaseModel):
+1 -1
View File
@@ -16,7 +16,7 @@ from app.gateway.authz import require_permission
from app.gateway.deps import get_current_user, get_feedback_repo, get_run_store from app.gateway.deps import get_current_user, get_feedback_repo, get_run_store
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads", tags=["feedback"]) router = APIRouter(prefix="/threads", tags=["feedback"])
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+1 -1
View File
@@ -9,7 +9,7 @@ from pydantic import BaseModel, Field
from deerflow.config.extensions_config import ExtensionsConfig, get_extensions_config, reload_extensions_config from deerflow.config.extensions_config import ExtensionsConfig, get_extensions_config, reload_extensions_config
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["mcp"]) router = APIRouter(tags=["mcp"])
class McpOAuthConfigResponse(BaseModel): class McpOAuthConfigResponse(BaseModel):
+1 -1
View File
@@ -15,7 +15,7 @@ from deerflow.agents.memory.updater import (
from deerflow.config.memory_config import get_memory_config from deerflow.config.memory_config import get_memory_config
from deerflow.runtime.user_context import get_effective_user_id from deerflow.runtime.user_context import get_effective_user_id
router = APIRouter(prefix="/api", tags=["memory"]) router = APIRouter(tags=["memory"])
class ContextSection(BaseModel): class ContextSection(BaseModel):
+1 -1
View File
@@ -4,7 +4,7 @@ from pydantic import BaseModel, Field
from app.gateway.deps import get_config from app.gateway.deps import get_config
from deerflow.config.app_config import AppConfig from deerflow.config.app_config import AppConfig
router = APIRouter(prefix="/api", tags=["models"]) router = APIRouter(tags=["models"])
class ModelResponse(BaseModel): class ModelResponse(BaseModel):
+1 -1
View File
@@ -21,7 +21,7 @@ from app.gateway.services import sse_consumer, start_run
from deerflow.runtime import serialize_channel_values from deerflow.runtime import serialize_channel_values
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/runs", tags=["runs"]) router = APIRouter(prefix="/runs", tags=["runs"])
def _resolve_thread_id(body: RunCreateRequest) -> str: def _resolve_thread_id(body: RunCreateRequest) -> str:
@@ -0,0 +1,81 @@
"""Service account management endpoints (Stage 1 PR4).
Owner/admin self-service: create / list / suspend service accounts in
the caller's current workspace. All operations are workspace-scoped;
cross-workspace targets return 404 (existence hidden).
"""
from __future__ import annotations
from typing import Literal
from fastapi import APIRouter, Depends, HTTPException, Request
from pydantic import BaseModel, Field
from app.gateway.authz import require_workspace_admin
from deerflow.persistence.service_account import ServiceAccountRepository
from deerflow.runtime.workspace_context import get_current_workspace
router = APIRouter(prefix="/api/v1/service-accounts", tags=["service-accounts"])
class CreateServiceAccountRequest(BaseModel):
name: str = Field(..., min_length=1, max_length=64)
# Constrained to Stage-1-supported values; widen these Literals in
# future stages alongside the behavior (Stage 2 RBAC opens role
# admin/viewer; the passthrough PR opens identity_mode).
role: Literal["member"] = "member"
identity_mode: Literal["collapsed"] = "collapsed"
class UpdateServiceAccountRequest(BaseModel):
status: str = Field(..., pattern="^(active|suspended|deleted)$")
def get_service_account_repo() -> ServiceAccountRepository:
from deerflow.persistence.engine import get_session_factory
sf = get_session_factory()
if sf is None:
raise HTTPException(status_code=503, detail="persistence backend not available")
return ServiceAccountRepository(sf)
def _current_workspace_id() -> str:
ws = get_current_workspace()
if ws is None:
raise HTTPException(status_code=403, detail="no workspace in context")
return str(ws.id)
@router.post("", status_code=201, dependencies=[Depends(require_workspace_admin)])
async def create_service_account(
body: CreateServiceAccountRequest,
request: Request,
repo: ServiceAccountRepository = Depends(get_service_account_repo),
):
return await repo.create(
workspace_id=_current_workspace_id(),
name=body.name,
created_by=str(request.state.user.id),
role=body.role,
identity_mode=body.identity_mode,
)
@router.get("", dependencies=[Depends(require_workspace_admin)])
async def list_service_accounts(repo: ServiceAccountRepository = Depends(get_service_account_repo)):
return await repo.list_by_workspace(_current_workspace_id())
@router.patch("/{sa_id}", dependencies=[Depends(require_workspace_admin)])
async def update_service_account(
sa_id: str,
body: UpdateServiceAccountRequest,
repo: ServiceAccountRepository = Depends(get_service_account_repo),
):
sa = await repo.get(sa_id)
if sa is None or sa["workspace_id"] != _current_workspace_id():
raise HTTPException(status_code=404, detail="service account not found")
await repo.update_status(sa_id, body.status)
return await repo.get(sa_id)
+1 -1
View File
@@ -18,7 +18,7 @@ from deerflow.skills.types import SKILL_MD_FILE, SkillCategory
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["skills"]) router = APIRouter(tags=["skills"])
class SkillResponse(BaseModel): class SkillResponse(BaseModel):
+1 -1
View File
@@ -12,7 +12,7 @@ from deerflow.models import create_chat_model
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["suggestions"]) router = APIRouter(tags=["suggestions"])
class SuggestionMessage(BaseModel): class SuggestionMessage(BaseModel):
+1 -1
View File
@@ -25,7 +25,7 @@ from app.gateway.services import sse_consumer, start_run
from deerflow.runtime import RunRecord, serialize_channel_values from deerflow.runtime import RunRecord, serialize_channel_values
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads", tags=["runs"]) router = APIRouter(prefix="/threads", tags=["runs"])
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+1 -1
View File
@@ -29,7 +29,7 @@ from deerflow.runtime.user_context import get_effective_user_id
from deerflow.utils.time import coerce_iso, now_iso from deerflow.utils.time import coerce_iso, now_iso
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads", tags=["threads"]) router = APIRouter(prefix="/threads", tags=["threads"])
# Metadata keys that the server controls; clients are not allowed to set # Metadata keys that the server controls; clients are not allowed to set
+1 -1
View File
@@ -30,7 +30,7 @@ from deerflow.utils.file_conversion import CONVERTIBLE_EXTENSIONS, convert_file_
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads/{thread_id}/uploads", tags=["uploads"]) router = APIRouter(prefix="/threads/{thread_id}/uploads", tags=["uploads"])
UPLOAD_CHUNK_SIZE = 8192 UPLOAD_CHUNK_SIZE = 8192
DEFAULT_MAX_FILES = 10 DEFAULT_MAX_FILES = 10
@@ -11,6 +11,7 @@ from langgraph.runtime import Runtime
from deerflow.agents.thread_state import ThreadDataState from deerflow.agents.thread_state import ThreadDataState
from deerflow.config.paths import Paths, get_paths from deerflow.config.paths import Paths, get_paths
from deerflow.runtime.user_context import get_effective_user_id from deerflow.runtime.user_context import get_effective_user_id
from deerflow.runtime.workspace_context import get_effective_workspace_id
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -24,10 +25,15 @@ class ThreadDataMiddlewareState(AgentState):
class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]): class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]):
"""Create thread data directories for each thread execution. """Create thread data directories for each thread execution.
Creates the following directory structure: PR6 routes thread storage through the workspace dimension. When a
- {base_dir}/threads/{thread_id}/user-data/workspace workspace contextvar is set (production via AuthMiddleware; tests via
- {base_dir}/threads/{thread_id}/user-data/uploads the autouse fixture), directories live at
- {base_dir}/threads/{thread_id}/user-data/outputs ``{base_dir}/workspaces/{wid}/threads/{thread_id}/user-data/{workspace,uploads,outputs}``.
In no-auth dev mode ``get_effective_workspace_id()`` returns
``"default"`` so the layout stays valid; the ``user_id`` falls through
to the same default constant. Either way thread state lives below a
workspace bucket, never directly under ``{base_dir}/threads`` (legacy)
or ``{base_dir}/users`` (PR4 layout).
Lifecycle Management: Lifecycle Management:
- With lazy_init=True (default): Only compute paths, directories created on-demand - With lazy_init=True (default): Only compute paths, directories created on-demand
@@ -49,34 +55,18 @@ class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]):
self._paths = Paths(base_dir) if base_dir else get_paths() self._paths = Paths(base_dir) if base_dir else get_paths()
self._lazy_init = lazy_init self._lazy_init = lazy_init
def _get_thread_paths(self, thread_id: str, user_id: str | None = None) -> dict[str, str]: def _get_thread_paths(self, thread_id: str, *, workspace_id: str, user_id: str) -> dict[str, str]:
"""Get the paths for a thread's data directories.
Args:
thread_id: The thread ID.
user_id: Optional user ID for per-user path isolation.
Returns:
Dictionary with workspace_path, uploads_path, and outputs_path.
"""
return { return {
"workspace_path": str(self._paths.sandbox_work_dir(thread_id, user_id=user_id)), "workspace_path": str(self._paths.sandbox_work_dir(thread_id, workspace_id=workspace_id)),
"uploads_path": str(self._paths.sandbox_uploads_dir(thread_id, user_id=user_id)), "uploads_path": str(self._paths.sandbox_uploads_dir(thread_id, workspace_id=workspace_id)),
"outputs_path": str(self._paths.sandbox_outputs_dir(thread_id, user_id=user_id)), "outputs_path": str(self._paths.sandbox_outputs_dir(thread_id, workspace_id=workspace_id)),
"user_id": user_id,
"workspace_id": workspace_id,
} }
def _create_thread_directories(self, thread_id: str, user_id: str | None = None) -> dict[str, str]: def _create_thread_directories(self, thread_id: str, *, workspace_id: str, user_id: str) -> dict[str, str]:
"""Create the thread data directories. self._paths.ensure_thread_dirs(thread_id, workspace_id=workspace_id)
return self._get_thread_paths(thread_id, workspace_id=workspace_id, user_id=user_id)
Args:
thread_id: The thread ID.
user_id: Optional user ID for per-user path isolation.
Returns:
Dictionary with the created directory paths.
"""
self._paths.ensure_thread_dirs(thread_id, user_id=user_id)
return self._get_thread_paths(thread_id, user_id=user_id)
@override @override
def before_agent(self, state: ThreadDataMiddlewareState, runtime: Runtime) -> dict | None: def before_agent(self, state: ThreadDataMiddlewareState, runtime: Runtime) -> dict | None:
@@ -90,14 +80,15 @@ class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]):
raise ValueError("Thread ID is required in runtime context or config.configurable") raise ValueError("Thread ID is required in runtime context or config.configurable")
user_id = get_effective_user_id() user_id = get_effective_user_id()
workspace_id = get_effective_workspace_id()
if self._lazy_init: if self._lazy_init:
# Lazy initialization: only compute paths, don't create directories # Lazy initialization: only compute paths, don't create directories
paths = self._get_thread_paths(thread_id, user_id=user_id) paths = self._get_thread_paths(thread_id, workspace_id=workspace_id, user_id=user_id)
else: else:
# Eager initialization: create directories immediately # Eager initialization: create directories immediately
paths = self._create_thread_directories(thread_id, user_id=user_id) paths = self._create_thread_directories(thread_id, workspace_id=workspace_id, user_id=user_id)
logger.debug("Created thread data directories for thread %s", thread_id) logger.debug("Created thread data directories for thread %s under workspace %s", thread_id, workspace_id)
messages = list(state.get("messages", [])) messages = list(state.get("messages", []))
last_message = messages[-1] if messages else None last_message = messages[-1] if messages else None
@@ -0,0 +1,13 @@
"""Auth primitives shared by the headless API (Stage 1).
Lives in the ``deerflow`` (harness) layer because both the persistence
hot path (``ApiKeyRepository.get_active_by_hash``) and the app-layer
mint endpoint need token generation/hashing, and the harness boundary
forbids ``deerflow`` importing ``app``.
"""
from __future__ import annotations
from deerflow.auth.tokens import GeneratedKey, generate_api_key, hash_api_key, split_prefix
__all__ = ["GeneratedKey", "generate_api_key", "hash_api_key", "split_prefix"]
@@ -0,0 +1,51 @@
"""API key generation, hashing, and prefix extraction (Stage 1 PR1).
Format is irreversible once business systems integrate (spec D5):
``dfk_live_<24>`` / ``dfk_test_<24>``. The public ``key_prefix`` is the
leading slice of the plaintext (``dfk_live_`` plus a few random chars,
length ``_PREFIX_LEN``) and is stored UNIQUE for audit logging; the DB
only ever stores ``sha256(plaintext)`` hex, never the plaintext.
"""
from __future__ import annotations
import hashlib
import secrets
from dataclasses import dataclass
from typing import Literal
_PREFIX_LEN = 16
# token_urlsafe(18) yields ceil(18 * 4 / 3) = 24 url-safe chars.
_RANDOM_BYTES = 18
@dataclass(frozen=True)
class GeneratedKey:
"""A freshly minted key. ``plaintext`` is returned to the caller
exactly once; only ``prefix`` + ``key_hash`` are persisted."""
plaintext: str
prefix: str
key_hash: str
def hash_api_key(plaintext: str) -> str:
"""Return the sha-256 hex digest of a plaintext token."""
return hashlib.sha256(plaintext.encode("utf-8")).hexdigest()
def split_prefix(plaintext: str) -> str:
"""Return the public, loggable prefix (first 16 chars) of a token."""
return plaintext[:_PREFIX_LEN]
def generate_api_key(env: Literal["live", "test"]) -> GeneratedKey:
"""Generate a new API key for the given environment.
Raises ``ValueError`` for any env other than ``"live"`` / ``"test"``.
"""
if env not in ("live", "test"):
raise ValueError(f"env must be 'live' or 'test', got {env!r}")
random_part = secrets.token_urlsafe(_RANDOM_BYTES)
plaintext = f"dfk_{env}_{random_part}"
return GeneratedKey(plaintext=plaintext, prefix=split_prefix(plaintext), key_hash=hash_api_key(plaintext))
+105 -79
View File
@@ -10,6 +10,7 @@ VIRTUAL_PATH_PREFIX = "/mnt/user-data"
_SAFE_THREAD_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$") _SAFE_THREAD_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
_SAFE_USER_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$") _SAFE_USER_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
_SAFE_WORKSPACE_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
def _default_local_base_dir() -> Path: def _default_local_base_dir() -> Path:
@@ -31,6 +32,13 @@ def _validate_user_id(user_id: str) -> str:
return user_id return user_id
def _validate_workspace_id(workspace_id: str) -> str:
"""Validate a workspace ID before using it in filesystem paths."""
if not _SAFE_WORKSPACE_ID_RE.match(workspace_id):
raise ValueError(f"Invalid workspace_id {workspace_id!r}: only alphanumeric characters, hyphens, and underscores are allowed.")
return workspace_id
def _join_host_path(base: str, *parts: str) -> str: def _join_host_path(base: str, *parts: str) -> str:
"""Join host filesystem path segments while preserving native style. """Join host filesystem path segments while preserving native style.
@@ -148,116 +156,129 @@ class Paths:
"""Legacy per-agent memory file: `{base_dir}/agents/{name}/memory.json`.""" """Legacy per-agent memory file: `{base_dir}/agents/{name}/memory.json`."""
return self.agent_dir(name) / "memory.json" return self.agent_dir(name) / "memory.json"
def user_dir(self, user_id: str) -> Path: def workspace_dir(self, workspace_id: str) -> Path:
"""Directory for a specific user: `{base_dir}/users/{user_id}/`.""" """Directory for a specific workspace: `{base_dir}/workspaces/{workspace_id}/`.
PR6 introduces this as the top-level isolation dimension. Per-user
state and per-thread state both live underneath their workspace so
a user with access to two workspaces never sees state bleed between
them on the filesystem.
"""
return self.base_dir / "workspaces" / _validate_workspace_id(workspace_id)
def user_dir(self, user_id: str, *, workspace_id: str | None = None) -> Path:
"""Directory for a specific user.
When ``workspace_id`` is provided (PR6+):
``{base_dir}/workspaces/{wid}/users/{user_id}/``
Otherwise (legacy layout):
``{base_dir}/users/{user_id}/``
"""
if workspace_id is not None:
return self.workspace_dir(workspace_id) / "users" / _validate_user_id(user_id)
return self.base_dir / "users" / _validate_user_id(user_id) return self.base_dir / "users" / _validate_user_id(user_id)
def user_memory_file(self, user_id: str) -> Path: def user_memory_file(self, user_id: str, *, workspace_id: str | None = None) -> Path:
"""Per-user memory file: `{base_dir}/users/{user_id}/memory.json`.""" """Per-user memory file under the active workspace (legacy without)."""
return self.user_dir(user_id) / "memory.json" return self.user_dir(user_id, workspace_id=workspace_id) / "memory.json"
def user_agents_dir(self, user_id: str) -> Path: def user_agents_dir(self, user_id: str, *, workspace_id: str | None = None) -> Path:
"""Per-user root for that user's custom agents: `{base_dir}/users/{user_id}/agents/`.""" """Per-user root for custom agents under the active workspace."""
return self.user_dir(user_id) / "agents" return self.user_dir(user_id, workspace_id=workspace_id) / "agents"
def user_agent_dir(self, user_id: str, agent_name: str) -> Path: def user_agent_dir(self, user_id: str, agent_name: str, *, workspace_id: str | None = None) -> Path:
"""Per-user per-agent directory: `{base_dir}/users/{user_id}/agents/{name}/`.""" """Per-user per-agent directory under the active workspace."""
return self.user_agents_dir(user_id) / agent_name.lower() return self.user_agents_dir(user_id, workspace_id=workspace_id) / agent_name.lower()
def user_agent_memory_file(self, user_id: str, agent_name: str) -> Path: def user_agent_memory_file(self, user_id: str, agent_name: str, *, workspace_id: str | None = None) -> Path:
"""Per-user per-agent memory: `{base_dir}/users/{user_id}/agents/{name}/memory.json`.""" """Per-user per-agent memory file under the active workspace."""
return self.user_agent_dir(user_id, agent_name) / "memory.json" return self.user_agent_dir(user_id, agent_name, workspace_id=workspace_id) / "memory.json"
def thread_dir(self, thread_id: str, *, user_id: str | None = None) -> Path: def thread_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
""" """
Host path for a thread's data. Host path for a thread's data.
When *user_id* is provided: Precedence workspace beats user, both beat legacy:
`{base_dir}/users/{user_id}/threads/{thread_id}/`
Otherwise (legacy layout):
`{base_dir}/threads/{thread_id}/`
This directory contains a `user-data/` subdirectory that is mounted * ``workspace_id`` given (PR6+):
as `/mnt/user-data/` inside the sandbox. ``{base_dir}/workspaces/{wid}/threads/{thread_id}/``
* ``user_id`` only (legacy after user-isolation migration):
``{base_dir}/users/{user_id}/threads/{thread_id}/``
* neither (very legacy, pre-isolation):
``{base_dir}/threads/{thread_id}/``
The contained ``user-data/`` subdirectory is mounted as
``/mnt/user-data/`` inside the sandbox regardless of which form is
chosen only the host-side parent differs.
Raises: Raises:
ValueError: If `thread_id` or `user_id` contains unsafe characters (path ValueError: If any of the supplied ids contains unsafe characters.
separators or `..`) that could cause directory traversal.
""" """
if workspace_id is not None:
return self.workspace_dir(workspace_id) / "threads" / _validate_thread_id(thread_id)
if user_id is not None: if user_id is not None:
return self.user_dir(user_id) / "threads" / _validate_thread_id(thread_id) return self.base_dir / "users" / _validate_user_id(user_id) / "threads" / _validate_thread_id(thread_id)
return self.base_dir / "threads" / _validate_thread_id(thread_id) return self.base_dir / "threads" / _validate_thread_id(thread_id)
def sandbox_work_dir(self, thread_id: str, *, user_id: str | None = None) -> Path: def sandbox_work_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
""" """
Host path for the agent's workspace directory. Host path for the agent's workspace directory.
Host: `{base_dir}/threads/{thread_id}/user-data/workspace/` Host: ``{thread_dir}/user-data/workspace/``
Sandbox: `/mnt/user-data/workspace/` Sandbox: ``/mnt/user-data/workspace/``
""" """
return self.thread_dir(thread_id, user_id=user_id) / "user-data" / "workspace" return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data" / "workspace"
def sandbox_uploads_dir(self, thread_id: str, *, user_id: str | None = None) -> Path: def sandbox_uploads_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
""" """Host path for user-uploaded files; sandbox: ``/mnt/user-data/uploads/``."""
Host path for user-uploaded files. return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data" / "uploads"
Host: `{base_dir}/threads/{thread_id}/user-data/uploads/`
Sandbox: `/mnt/user-data/uploads/`
"""
return self.thread_dir(thread_id, user_id=user_id) / "user-data" / "uploads"
def sandbox_outputs_dir(self, thread_id: str, *, user_id: str | None = None) -> Path: def sandbox_outputs_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
""" """Host path for agent-generated artifacts; sandbox: ``/mnt/user-data/outputs/``."""
Host path for agent-generated artifacts. return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data" / "outputs"
Host: `{base_dir}/threads/{thread_id}/user-data/outputs/`
Sandbox: `/mnt/user-data/outputs/`
"""
return self.thread_dir(thread_id, user_id=user_id) / "user-data" / "outputs"
def acp_workspace_dir(self, thread_id: str, *, user_id: str | None = None) -> Path: def acp_workspace_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
""" """
Host path for the ACP workspace of a specific thread. Host path for the ACP workspace of a specific thread; sandbox: ``/mnt/acp-workspace/``.
Host: `{base_dir}/threads/{thread_id}/acp-workspace/`
Sandbox: `/mnt/acp-workspace/`
Each thread gets its own isolated ACP workspace so that concurrent Each thread gets its own isolated ACP workspace so that concurrent
sessions cannot read each other's ACP agent outputs. sessions cannot read each other's ACP agent outputs.
""" """
return self.thread_dir(thread_id, user_id=user_id) / "acp-workspace" return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "acp-workspace"
def sandbox_user_data_dir(self, thread_id: str, *, user_id: str | None = None) -> Path: def sandbox_user_data_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
""" """Host path for the user-data root; sandbox: ``/mnt/user-data/``."""
Host path for the user-data root. return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data"
Host: `{base_dir}/threads/{thread_id}/user-data/`
Sandbox: `/mnt/user-data/`
"""
return self.thread_dir(thread_id, user_id=user_id) / "user-data"
def host_thread_dir(self, thread_id: str, *, user_id: str | None = None) -> str: def host_thread_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
"""Host path for a thread directory, preserving Windows path syntax.""" """Host path for a thread directory, preserving Windows path syntax."""
if workspace_id is not None:
return _join_host_path(self._host_base_dir_str(), "workspaces", _validate_workspace_id(workspace_id), "threads", _validate_thread_id(thread_id))
if user_id is not None: if user_id is not None:
return _join_host_path(self._host_base_dir_str(), "users", _validate_user_id(user_id), "threads", _validate_thread_id(thread_id)) return _join_host_path(self._host_base_dir_str(), "users", _validate_user_id(user_id), "threads", _validate_thread_id(thread_id))
return _join_host_path(self._host_base_dir_str(), "threads", _validate_thread_id(thread_id)) return _join_host_path(self._host_base_dir_str(), "threads", _validate_thread_id(thread_id))
def host_sandbox_user_data_dir(self, thread_id: str, *, user_id: str | None = None) -> str: def host_sandbox_user_data_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
"""Host path for a thread's user-data root.""" """Host path for a thread's user-data root."""
return _join_host_path(self.host_thread_dir(thread_id, user_id=user_id), "user-data") return _join_host_path(self.host_thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "user-data")
def host_sandbox_work_dir(self, thread_id: str, *, user_id: str | None = None) -> str: def host_sandbox_work_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
"""Host path for the workspace mount source.""" """Host path for the workspace mount source."""
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, user_id=user_id), "workspace") return _join_host_path(self.host_sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "workspace")
def host_sandbox_uploads_dir(self, thread_id: str, *, user_id: str | None = None) -> str: def host_sandbox_uploads_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
"""Host path for the uploads mount source.""" """Host path for the uploads mount source."""
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, user_id=user_id), "uploads") return _join_host_path(self.host_sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "uploads")
def host_sandbox_outputs_dir(self, thread_id: str, *, user_id: str | None = None) -> str: def host_sandbox_outputs_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
"""Host path for the outputs mount source.""" """Host path for the outputs mount source."""
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, user_id=user_id), "outputs") return _join_host_path(self.host_sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "outputs")
def host_acp_workspace_dir(self, thread_id: str, *, user_id: str | None = None) -> str: def host_acp_workspace_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
"""Host path for the ACP workspace mount source.""" """Host path for the ACP workspace mount source."""
return _join_host_path(self.host_thread_dir(thread_id, user_id=user_id), "acp-workspace") return _join_host_path(self.host_thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "acp-workspace")
def ensure_thread_dirs(self, thread_id: str, *, user_id: str | None = None) -> None: def ensure_thread_dirs(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> None:
"""Create all standard sandbox directories for a thread. """Create all standard sandbox directories for a thread.
Directories are created with mode 0o777 so that sandbox containers Directories are created with mode 0o777 so that sandbox containers
@@ -271,24 +292,28 @@ class Paths:
ACP agent invocation. ACP agent invocation.
""" """
for d in [ for d in [
self.sandbox_work_dir(thread_id, user_id=user_id), self.sandbox_work_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
self.sandbox_uploads_dir(thread_id, user_id=user_id), self.sandbox_uploads_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
self.sandbox_outputs_dir(thread_id, user_id=user_id), self.sandbox_outputs_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
self.acp_workspace_dir(thread_id, user_id=user_id), self.acp_workspace_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
]: ]:
d.mkdir(parents=True, exist_ok=True) d.mkdir(parents=True, exist_ok=True)
d.chmod(0o777) d.chmod(0o777)
def delete_thread_dir(self, thread_id: str, *, user_id: str | None = None) -> None: def delete_thread_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> None:
"""Delete all persisted data for a thread. """Delete all persisted data for a thread. Idempotent."""
thread_dir = self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id)
The operation is idempotent: missing thread directories are ignored.
"""
thread_dir = self.thread_dir(thread_id, user_id=user_id)
if thread_dir.exists(): if thread_dir.exists():
shutil.rmtree(thread_dir) shutil.rmtree(thread_dir)
def resolve_virtual_path(self, thread_id: str, virtual_path: str, *, user_id: str | None = None) -> Path: def resolve_virtual_path(
self,
thread_id: str,
virtual_path: str,
*,
workspace_id: str | None = None,
user_id: str | None = None,
) -> Path:
"""Resolve a sandbox virtual path to the actual host filesystem path. """Resolve a sandbox virtual path to the actual host filesystem path.
Args: Args:
@@ -296,7 +321,8 @@ class Paths:
virtual_path: Virtual path as seen inside the sandbox, e.g. virtual_path: Virtual path as seen inside the sandbox, e.g.
``/mnt/user-data/outputs/report.pdf``. ``/mnt/user-data/outputs/report.pdf``.
Leading slashes are stripped before matching. Leading slashes are stripped before matching.
user_id: Optional user ID for user-scoped path resolution. workspace_id: Optional workspace ID for workspace-scoped resolution.
user_id: Optional user ID for legacy user-scoped resolution.
Returns: Returns:
The resolved absolute host filesystem path. The resolved absolute host filesystem path.
@@ -314,7 +340,7 @@ class Paths:
raise ValueError(f"Path must start with /{prefix}") raise ValueError(f"Path must start with /{prefix}")
relative = stripped[len(prefix) :].lstrip("/") relative = stripped[len(prefix) :].lstrip("/")
base = self.sandbox_user_data_dir(thread_id, user_id=user_id).resolve() base = self.sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id).resolve()
actual = (base / relative).resolve() actual = (base / relative).resolve()
try: try:
@@ -0,0 +1,19 @@
"""API key persistence — ORM model only (Stage 0 PR8).
An API key is the credential a service_account uses to call the
headless API. Each key has a public ``key_prefix`` (printed in audit
logs and used for quick lookup) and a ``key_hash`` (sha-256 of the
plaintext token, never reversed). Plaintext tokens are only ever
returned to the caller at create time.
PR8 introduces only the schema + ORM row class. Token generation,
hashing, scope parsing, rate limiting, and the API-key auth
middleware live in Stage 1 alongside the headless API surface.
"""
from __future__ import annotations
from deerflow.persistence.api_key.model import ApiKeyRow
from deerflow.persistence.api_key.sql import ApiKeyRepository
__all__ = ["ApiKeyRepository", "ApiKeyRow"]
@@ -0,0 +1,94 @@
"""ORM model for API keys (credentials owned by a service account)."""
from __future__ import annotations
from datetime import UTC, datetime
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, text
from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base
class ApiKeyRow(Base):
__tablename__ = "api_keys"
id: Mapped[str] = mapped_column(
String(36),
primary_key=True,
comment="API key 主键,UUID 字符串(36 字符)",
)
service_account_id: Mapped[str] = mapped_column(
String(36),
ForeignKey("service_accounts.id", ondelete="CASCADE"),
nullable=False,
comment="所属 service_accountservice_account 删除时级联清掉所有 api_key",
)
key_prefix: Mapped[str] = mapped_column(
String(16),
nullable=False,
unique=True,
comment="公开 prefix(如 'dfk_live_abc12345'),可在审计日志 / UI 中打印;全局唯一,撤销后亦不复用以避免审计混淆",
)
key_hash: Mapped[str] = mapped_column(
String(128),
nullable=False,
comment="完整 token 的 sha-256 hex64 字符;预留 128 以兼容未来更长哈希),plaintext token 仅在创建时返回给调用方",
)
name: Mapped[str] = mapped_column(
String(64),
nullable=False,
comment="key 的人类可读标签(如 'ci pipeline' / 'frontend prod'),同 service_account 内不强制唯一",
)
scopes: Mapped[str] = mapped_column(
String(1024),
nullable=False,
default="",
comment="scope 列表,逗号分隔字符串(如 'threads:read,threads:write');用 String 而非 PG text[] 以保 SQLite dev 兼容,Stage 2 切纯 PG 后可平滑迁",
)
rate_limit_rpm: Mapped[int | None] = mapped_column(
Integer,
nullable=True,
comment="每分钟请求数限制;NULL 表示走该 service_account 的默认限速(Stage 1 起生效)",
)
expires_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="过期时间(UTC);NULL = 不过期",
)
last_used_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="最近一次成功鉴权时间(UTC);Stage 1 鉴权中间件每次更新",
)
revoked_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="撤销时间(UTC);NULL = 仍然有效。被撤销的 key 不删行(保留审计),但鉴权层据此拒绝",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.now(UTC),
comment="创建时间(UTC",
)
__table_args__ = (
Index("idx_api_keys_sa", "service_account_id"),
# 部分索引:只索引活跃 keyrevoked_at IS NULL),鉴权热路径走 prefix lookup
# 撤销后的 key 不进活跃索引以减小热索引大小。SQLite + Postgres 均支持
# WHERE 子句的部分索引;双驱动维护两套等价 where 表达式。
Index(
"idx_api_keys_active",
"key_prefix",
sqlite_where=text("revoked_at IS NULL"),
postgresql_where=text("revoked_at IS NULL"),
),
{
"comment": (
"API key 表(headless API 凭证)。每行属于唯一 service_accountkey_prefix 全局唯一可在日志中打印,"
"key_hash 是完整 token 的 sha-256plaintext token 只在创建时返给调用方。撤销保留行(revoked_at 非空),"
"活跃 key 走部分索引 idx_api_keys_active 加速鉴权热路径。Stage 0 仅落 schemaStage 1 起接鉴权 + 限速。"
)
},
)
@@ -0,0 +1,116 @@
"""SQLAlchemy-backed API key repository (Stage 1 PR1).
``get_active_by_hash`` is the auth hot path: it looks the key up by its
public ``key_prefix`` UNIQUE and covered by the partial index
``idx_api_keys_active`` (WHERE revoked_at IS NULL) then verifies the
full ``key_hash`` with a constant-time compare. Expiry is filtered in
Python so the behaviour is identical across sqlite/postgres drivers.
``_row_to_dict`` deliberately omits ``key_hash`` no dict this
repository returns ever carries the secret material.
"""
from __future__ import annotations
import secrets
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.api_key.model import ApiKeyRow
class ApiKeyRepository:
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
@staticmethod
def _row_to_dict(row: ApiKeyRow) -> dict[str, Any]:
return {
"id": row.id,
"service_account_id": row.service_account_id,
"key_prefix": row.key_prefix,
"name": row.name,
"scopes": row.scopes,
"rate_limit_rpm": row.rate_limit_rpm,
"expires_at": row.expires_at.isoformat() if row.expires_at else None,
"last_used_at": row.last_used_at.isoformat() if row.last_used_at else None,
"revoked_at": row.revoked_at.isoformat() if row.revoked_at else None,
"created_at": row.created_at.isoformat() if row.created_at else None,
}
async def create(
self,
*,
service_account_id: str,
key_prefix: str,
key_hash: str,
name: str,
scopes: str,
expires_at: datetime | None = None,
) -> dict[str, Any]:
row = ApiKeyRow(
id=str(uuid.uuid4()),
service_account_id=service_account_id,
key_prefix=key_prefix,
key_hash=key_hash,
name=name,
scopes=scopes,
expires_at=expires_at,
created_at=datetime.now(UTC),
)
async with self._sf() as session:
session.add(row)
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
async def get(self, key_id: str) -> dict[str, Any] | None:
async with self._sf() as session:
row = await session.get(ApiKeyRow, key_id)
return self._row_to_dict(row) if row else None
async def get_active_by_hash(self, key_hash: str, *, key_prefix: str) -> dict[str, Any] | None:
"""Auth hot path: resolve an active, unexpired key.
Looks the key up by its public ``key_prefix`` UNIQUE and covered
by the partial index ``idx_api_keys_active`` (WHERE revoked_at IS
NULL) then verifies the full ``key_hash`` with a constant-time
compare. Returns None on miss / hash mismatch / revoked / expired.
Expiry is filtered in Python so behaviour is driver-agnostic
(sqlite returns naive datetimes; postgres returns aware).
"""
async with self._sf() as session:
result = await session.execute(select(ApiKeyRow).where(ApiKeyRow.key_prefix == key_prefix, ApiKeyRow.revoked_at.is_(None)))
row = result.scalar_one_or_none()
if row is None:
return None
if not secrets.compare_digest(row.key_hash, key_hash):
return None
expires_at = row.expires_at
if expires_at is not None:
if expires_at.tzinfo is None:
expires_at = expires_at.replace(tzinfo=UTC)
if expires_at <= datetime.now(UTC):
return None
return self._row_to_dict(row)
async def list_by_service_account(self, service_account_id: str) -> list[dict[str, Any]]:
async with self._sf() as session:
result = await session.execute(select(ApiKeyRow).where(ApiKeyRow.service_account_id == service_account_id).order_by(ApiKeyRow.created_at.desc()))
return [self._row_to_dict(r) for r in result.scalars()]
async def revoke(self, key_id: str) -> None:
"""Soft-revoke: set ``revoked_at`` (row is kept for audit)."""
async with self._sf() as session:
await session.execute(update(ApiKeyRow).where(ApiKeyRow.id == key_id, ApiKeyRow.revoked_at.is_(None)).values(revoked_at=datetime.now(UTC)))
await session.commit()
async def touch_last_used(self, key_id: str) -> None:
"""Best-effort: stamp ``last_used_at`` after a successful auth."""
async with self._sf() as session:
await session.execute(update(ApiKeyRow).where(ApiKeyRow.id == key_id).values(last_used_at=datetime.now(UTC)))
await session.commit()
@@ -0,0 +1,17 @@
"""External user persistence — ORM model only (Stage 0 PR8).
An external user represents the end-user identity that a
service_account passes through on each call (typically via an
``X-External-User-Id`` header). The row is upserted each time a
new ``external_id`` is seen under a given service_account.
PR8 introduces only the schema + ORM row class. The upsert logic,
header parsing, and quota attribution all live in Stage 1.
"""
from __future__ import annotations
from deerflow.persistence.external_user.model import ExternalUserRow
from deerflow.persistence.external_user.sql import ExternalUserRepository
__all__ = ["ExternalUserRepository", "ExternalUserRow"]
@@ -0,0 +1,71 @@
"""ORM model for external users (end-user identities passed through a service account)."""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base
class ExternalUserRow(Base):
__tablename__ = "external_users"
id: Mapped[str] = mapped_column(
String(36),
primary_key=True,
comment="external_user 主键,UUID 字符串(36 字符)",
)
workspace_id: Mapped[str] = mapped_column(
String(36),
ForeignKey("workspaces.id", ondelete="CASCADE"),
nullable=False,
comment="所属 workspace(冗余存储——可经 service_account 间接得到,但直接存以加速 workspace-scope 查询);workspace 删除时级联",
)
service_account_id: Mapped[str] = mapped_column(
String(36),
ForeignKey("service_accounts.id", ondelete="CASCADE"),
nullable=False,
comment="passthrough 的 service_accountservice_account 删除时级联",
)
external_id: Mapped[str] = mapped_column(
String(128),
nullable=False,
comment="终端调用方传入的 X-External-User-Id(最多 128 字符;推荐 UUID / opaque token,不要塞 PII",
)
display_name: Mapped[str | None] = mapped_column(
String(128),
nullable=True,
comment="可选显示名(如 'alice@customer.com');仅用于 admin UI 展示,不参与鉴权",
)
metadata_json: Mapped[dict[str, Any]] = mapped_column(
JSON,
nullable=False,
default=dict,
comment="任意 JSON 附属信息(plan tier / region / 自定义 tag);Stage 1 由 upsert 调用方写入",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.now(UTC),
comment="首次见到该 external_id 的时间(UTC",
)
last_seen_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="最近一次该 external_id 触发请求的时间(UTC);Stage 1 鉴权层每次更新",
)
__table_args__ = (
UniqueConstraint("service_account_id", "external_id", name="uq_external_users_sa_external"),
{
"comment": (
"终端用户身份表(passthrough 模式下的 end-user)。每行由 service_account 的鉴权中间件 upsert——同一 "
"(service_account_id, external_id) 组合只存一行。workspace_id 冗余存储以加速跨 SA 的 workspace-scope 聚合查询。"
"Stage 0 仅落 schemaStage 1 起接 upsert / 配额聚合。"
)
},
)
@@ -0,0 +1,106 @@
"""SQLAlchemy-backed external user repository (Stage 1 PR1).
Built but NOT yet wired to any auth path the X-External-User-Id
passthrough that calls ``upsert`` lands in a later track-2 PR. ``upsert``
is idempotent on (service_account_id, external_id).
"""
from __future__ import annotations
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.external_user.model import ExternalUserRow
class ExternalUserRepository:
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
@staticmethod
def _row_to_dict(row: ExternalUserRow) -> dict[str, Any]:
return {
"id": row.id,
"workspace_id": row.workspace_id,
"service_account_id": row.service_account_id,
"external_id": row.external_id,
"display_name": row.display_name,
"metadata": dict(row.metadata_json or {}),
"created_at": row.created_at.isoformat() if row.created_at else None,
"last_seen_at": row.last_seen_at.isoformat() if row.last_seen_at else None,
}
async def get(self, external_user_id: str) -> dict[str, Any] | None:
async with self._sf() as session:
row = await session.get(ExternalUserRow, external_user_id)
return self._row_to_dict(row) if row else None
async def get_by_external_id(self, *, service_account_id: str, external_id: str) -> dict[str, Any] | None:
async with self._sf() as session:
result = await session.execute(
select(ExternalUserRow).where(
ExternalUserRow.service_account_id == service_account_id,
ExternalUserRow.external_id == external_id,
)
)
row = result.scalar_one_or_none()
return self._row_to_dict(row) if row else None
async def list_by_workspace(self, workspace_id: str) -> list[dict[str, Any]]:
async with self._sf() as session:
result = await session.execute(select(ExternalUserRow).where(ExternalUserRow.workspace_id == workspace_id).order_by(ExternalUserRow.created_at.desc()))
return [self._row_to_dict(r) for r in result.scalars()]
async def upsert(
self,
*,
workspace_id: str,
service_account_id: str,
external_id: str,
display_name: str | None = None,
metadata: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Insert a new external user or refresh ``last_seen_at`` on an
existing (service_account_id, external_id) row.
``display_name`` and ``metadata`` are only written when explicitly
passed (non-None); ``None`` means "leave unchanged" you cannot
clear ``display_name`` back to None via this method. ``workspace_id``
is only used on insert; it is ignored on update."""
now = datetime.now(UTC)
async with self._sf() as session:
result = await session.execute(
select(ExternalUserRow).where(
ExternalUserRow.service_account_id == service_account_id,
ExternalUserRow.external_id == external_id,
)
)
row = result.scalar_one_or_none()
if row is None:
row = ExternalUserRow(
id=str(uuid.uuid4()),
workspace_id=workspace_id,
service_account_id=service_account_id,
external_id=external_id,
display_name=display_name,
metadata_json=metadata or {},
created_at=now,
last_seen_at=now,
)
# NOTE: concurrent inserts of the same pair will raise IntegrityError
# from uq_external_users_sa_external — the future auth caller should
# catch it and re-read rather than treat it as fatal.
session.add(row)
else:
row.last_seen_at = now
if display_name is not None:
row.display_name = display_name
if metadata is not None:
row.metadata_json = metadata
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
@@ -4,7 +4,7 @@ from __future__ import annotations
from datetime import UTC, datetime from datetime import UTC, datetime
from sqlalchemy import DateTime, String, Text, UniqueConstraint from sqlalchemy import DateTime, ForeignKey, String, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base from deerflow.persistence.base import Base
@@ -13,20 +13,22 @@ from deerflow.persistence.base import Base
class FeedbackRow(Base): class FeedbackRow(Base):
__tablename__ = "feedback" __tablename__ = "feedback"
__table_args__ = (UniqueConstraint("thread_id", "run_id", "user_id", name="uq_feedback_thread_run_user"),) __table_args__ = (
UniqueConstraint("thread_id", "run_id", "user_id", name="uq_feedback_thread_run_user"),
{"comment": "用户对运行结果的反馈(点赞/点踩 + 文字评论),(thread, run, user) 唯一"},
)
feedback_id: Mapped[str] = mapped_column(String(64), primary_key=True) feedback_id: Mapped[str] = mapped_column(String(64), primary_key=True, comment="反馈主键")
run_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True) run_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True, comment="关联的运行 IDruns.run_id")
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True) thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True, comment="关联的会话 IDthreads_meta.thread_id")
user_id: Mapped[str | None] = mapped_column(String(64), index=True) user_id: Mapped[str | None] = mapped_column(String(64), index=True, comment="反馈作者;为 NULL 表示历史无主数据")
message_id: Mapped[str | None] = mapped_column(String(64)) workspace_id: Mapped[str] = mapped_column(
# message_id is an optional RunEventStore event identifier — String(36),
# allows feedback to target a specific message or the entire run ForeignKey("workspaces.id", ondelete="CASCADE"),
nullable=False,
rating: Mapped[int] = mapped_column(nullable=False) comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
# +1 (thumbs-up) or -1 (thumbs-down) )
message_id: Mapped[str | None] = mapped_column(String(64), comment="可选的 RunEventStore 事件 ID;为 NULL 表示针对整次运行而非单条消息")
comment: Mapped[str | None] = mapped_column(Text) rating: Mapped[int] = mapped_column(nullable=False, comment="评分:+1 点赞,-1 点踩")
# Optional text feedback from the user comment: Mapped[str | None] = mapped_column(Text, comment="可选的文字评论")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
@@ -13,6 +13,13 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.feedback.model import FeedbackRow from deerflow.persistence.feedback.model import FeedbackRow
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
from deerflow.runtime.workspace_context import (
_AutoSentinel as _WorkspaceAutoSentinel,
)
from deerflow.runtime.workspace_context import (
resolve_workspace_id,
)
class FeedbackRepository: class FeedbackRepository:
@@ -34,6 +41,7 @@ class FeedbackRepository:
thread_id: str, thread_id: str,
rating: int, rating: int,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
message_id: str | None = None, message_id: str | None = None,
comment: str | None = None, comment: str | None = None,
) -> dict: ) -> dict:
@@ -41,11 +49,13 @@ class FeedbackRepository:
if rating not in (1, -1): if rating not in (1, -1):
raise ValueError(f"rating must be +1 or -1, got {rating}") raise ValueError(f"rating must be +1 or -1, got {rating}")
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.create") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.create")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.create")
row = FeedbackRow( row = FeedbackRow(
feedback_id=str(uuid.uuid4()), feedback_id=str(uuid.uuid4()),
run_id=run_id, run_id=run_id,
thread_id=thread_id, thread_id=thread_id,
user_id=resolved_user_id, user_id=resolved_user_id,
workspace_id=resolved_workspace_id,
message_id=message_id, message_id=message_id,
rating=rating, rating=rating,
comment=comment, comment=comment,
@@ -62,12 +72,16 @@ class FeedbackRepository:
feedback_id: str, feedback_id: str,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> dict | None: ) -> dict | None:
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.get") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.get")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.get")
async with self._sf() as session: async with self._sf() as session:
row = await session.get(FeedbackRow, feedback_id) row = await session.get(FeedbackRow, feedback_id)
if row is None: if row is None:
return None return None
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
return None
if resolved_user_id is not None and row.user_id != resolved_user_id: if resolved_user_id is not None and row.user_id != resolved_user_id:
return None return None
return self._row_to_dict(row) return self._row_to_dict(row)
@@ -79,9 +93,13 @@ class FeedbackRepository:
*, *,
limit: int = 100, limit: int = 100,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> list[dict]: ) -> list[dict]:
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_run") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_run")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.list_by_run")
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id, FeedbackRow.run_id == run_id) stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id, FeedbackRow.run_id == run_id)
if resolved_workspace_id is not None:
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id) stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit) stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit)
@@ -95,9 +113,13 @@ class FeedbackRepository:
*, *,
limit: int = 100, limit: int = 100,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> list[dict]: ) -> list[dict]:
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.list_by_thread")
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id) stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id)
if resolved_workspace_id is not None:
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id) stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit) stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit)
@@ -110,12 +132,16 @@ class FeedbackRepository:
feedback_id: str, feedback_id: str,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> bool: ) -> bool:
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.delete")
async with self._sf() as session: async with self._sf() as session:
row = await session.get(FeedbackRow, feedback_id) row = await session.get(FeedbackRow, feedback_id)
if row is None: if row is None:
return False return False
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
return False
if resolved_user_id is not None and row.user_id != resolved_user_id: if resolved_user_id is not None and row.user_id != resolved_user_id:
return False return False
await session.delete(row) await session.delete(row)
@@ -129,18 +155,22 @@ class FeedbackRepository:
thread_id: str, thread_id: str,
rating: int, rating: int,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
comment: str | None = None, comment: str | None = None,
) -> dict: ) -> dict:
"""Create or update feedback for (thread_id, run_id, user_id). rating must be +1 or -1.""" """Create or update feedback for (thread_id, run_id, user_id). rating must be +1 or -1."""
if rating not in (1, -1): if rating not in (1, -1):
raise ValueError(f"rating must be +1 or -1, got {rating}") raise ValueError(f"rating must be +1 or -1, got {rating}")
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.upsert") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.upsert")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.upsert")
async with self._sf() as session: async with self._sf() as session:
stmt = select(FeedbackRow).where( stmt = select(FeedbackRow).where(
FeedbackRow.thread_id == thread_id, FeedbackRow.thread_id == thread_id,
FeedbackRow.run_id == run_id, FeedbackRow.run_id == run_id,
FeedbackRow.user_id == resolved_user_id, FeedbackRow.user_id == resolved_user_id,
) )
if resolved_workspace_id is not None:
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
result = await session.execute(stmt) result = await session.execute(stmt)
row = result.scalar_one_or_none() row = result.scalar_one_or_none()
if row is not None: if row is not None:
@@ -153,6 +183,7 @@ class FeedbackRepository:
run_id=run_id, run_id=run_id,
thread_id=thread_id, thread_id=thread_id,
user_id=resolved_user_id, user_id=resolved_user_id,
workspace_id=resolved_workspace_id,
rating=rating, rating=rating,
comment=comment, comment=comment,
created_at=datetime.now(UTC), created_at=datetime.now(UTC),
@@ -168,15 +199,19 @@ class FeedbackRepository:
thread_id: str, thread_id: str,
run_id: str, run_id: str,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> bool: ) -> bool:
"""Delete the current user's feedback for a run. Returns True if a record was deleted.""" """Delete the current user's feedback for a run. Returns True if a record was deleted."""
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete_by_run") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete_by_run")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.delete_by_run")
async with self._sf() as session: async with self._sf() as session:
stmt = select(FeedbackRow).where( stmt = select(FeedbackRow).where(
FeedbackRow.thread_id == thread_id, FeedbackRow.thread_id == thread_id,
FeedbackRow.run_id == run_id, FeedbackRow.run_id == run_id,
FeedbackRow.user_id == resolved_user_id, FeedbackRow.user_id == resolved_user_id,
) )
if resolved_workspace_id is not None:
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
result = await session.execute(stmt) result = await session.execute(stmt)
row = result.scalar_one_or_none() row = result.scalar_one_or_none()
if row is None: if row is None:
@@ -190,10 +225,14 @@ class FeedbackRepository:
thread_id: str, thread_id: str,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> dict[str, dict]: ) -> dict[str, dict]:
"""Return feedback grouped by run_id for a thread: {run_id: feedback_dict}.""" """Return feedback grouped by run_id for a thread: {run_id: feedback_dict}."""
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread_grouped") resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread_grouped")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.list_by_thread_grouped")
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id) stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id)
if resolved_workspace_id is not None:
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id) stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
async with self._sf() as session: async with self._sf() as session:
@@ -0,0 +1,44 @@
"""users.default_workspace_id column + FK to workspaces
Revision ID: 0001_users_default_workspace
Revises: None
Create Date: 2026-05-12
First Alembic revision for the DeerFlow application schema. Adds
`users.default_workspace_id` so a newly-registered user can be sent
back to their default workspace on next login without consulting the
memberships table.
Existing deployments created `users` via `metadata.create_all()` without
this column; running `alembic upgrade head` on those DBs will simply add
the column (ON DELETE SET NULL FK), no data backfill needed.
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# Alembic identifiers.
revision: str = "0001_users_default_workspace"
down_revision: str | None = None
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
with op.batch_alter_table("users") as batch:
batch.add_column(sa.Column("default_workspace_id", sa.String(36), nullable=True))
batch.create_foreign_key(
"fk_users_default_workspace",
"workspaces",
["default_workspace_id"],
["id"],
ondelete="SET NULL",
)
def downgrade() -> None:
with op.batch_alter_table("users") as batch:
batch.drop_constraint("fk_users_default_workspace", type_="foreignkey")
batch.drop_column("default_workspace_id")
@@ -0,0 +1,70 @@
"""Business tables: nullable workspace_id + FK to workspaces
Revision ID: 0002_business_tables_workspace
Revises: 0001_users_default_workspace
Create Date: 2026-05-13
Stage 0 PR5 step 1/2 (the second step lives in revision 0003).
This revision adds a *nullable* ``workspace_id`` column to the four
business tables that need tenancy scoping:
* ``threads_meta``
* ``runs``
* ``feedback``
* ``run_events``
The column is nullable here on purpose running ``upgrade`` on a
database with existing rows leaves those rows with ``workspace_id = NULL``
until ``scripts/backfill_workspace_id.py`` populates them. Once the
backfill finishes, revision 0003 flips the column to ``NOT NULL`` and
adds the threads_meta ``(workspace_id, thread_id)`` UNIQUE index.
A composite index ``idx_threads_meta_workspace_user_updated`` is added
on ``threads_meta`` to support the common "list a workspace's threads
for a user, newest first" access pattern that PR6 routers will rely on.
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision: str = "0002_business_tables_workspace"
down_revision: str | None = "0001_users_default_workspace"
branch_labels: str | None = None
depends_on: str | None = None
# Tables that receive the new column. The order matters only for human
# readability in migration logs — there are no inter-table data deps
# during ALTER, and the FK references workspaces (introduced in PR3) which
# is already present at this point.
_BUSINESS_TABLES = ("threads_meta", "runs", "feedback", "run_events")
def upgrade() -> None:
for table in _BUSINESS_TABLES:
with op.batch_alter_table(table) as batch:
batch.add_column(sa.Column("workspace_id", sa.String(36), nullable=True))
batch.create_foreign_key(
f"fk_{table}_workspace_id",
"workspaces",
["workspace_id"],
["id"],
ondelete="CASCADE",
)
op.create_index(
"idx_threads_meta_workspace_user_updated",
"threads_meta",
["workspace_id", "user_id", "updated_at"],
)
def downgrade() -> None:
op.drop_index("idx_threads_meta_workspace_user_updated", table_name="threads_meta")
for table in reversed(_BUSINESS_TABLES):
with op.batch_alter_table(table) as batch:
batch.drop_constraint(f"fk_{table}_workspace_id", type_="foreignkey")
batch.drop_column("workspace_id")
@@ -0,0 +1,62 @@
"""Business tables: workspace_id NOT NULL + UNIQUE(workspace_id, thread_id)
Revision ID: 0003_business_tables_workspace_not_null
Revises: 0002_business_tables_workspace
Create Date: 2026-05-13
Stage 0 PR5 step 2/2. Flips ``workspace_id`` on the four business
tables to ``NOT NULL`` and adds the threads_meta ``(workspace_id,
thread_id)`` UNIQUE index promised in workspace-schema-design §4.
**Refuses to upgrade** if any of the four tables still has rows with
``workspace_id IS NULL`` the operator must run
``scripts/backfill_workspace_id.py`` first. Doing the NOT NULL ALTER
with stragglers in place would either fail (Postgres) or silently
corrupt SQLite tables via ``batch_alter_table`` rebuilds.
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision: str = "0003_business_tables_workspace_not_null"
down_revision: str | None = "0002_business_tables_workspace"
branch_labels: str | None = None
depends_on: str | None = None
_BUSINESS_TABLES = ("threads_meta", "runs", "feedback", "run_events")
class _BackfillRequiredError(RuntimeError):
"""Raised when null workspace_id rows remain at the start of upgrade."""
def upgrade() -> None:
conn = op.get_bind()
for table in _BUSINESS_TABLES:
# Quoted identifier is fine here — table names are constants in this
# module, no operator input reaches the SQL.
count = conn.execute(sa.text(f"SELECT count(*) FROM {table} WHERE workspace_id IS NULL")).scalar() or 0
if count > 0:
raise _BackfillRequiredError(
f"Cannot ALTER {table}.workspace_id to NOT NULL: {count} row(s) still have workspace_id=NULL. Run `python scripts/backfill_workspace_id.py` first.",
)
for table in _BUSINESS_TABLES:
with op.batch_alter_table(table) as batch:
batch.alter_column("workspace_id", existing_type=sa.String(36), nullable=False)
op.create_index(
"idx_threads_meta_workspace_thread",
"threads_meta",
["workspace_id", "thread_id"],
unique=True,
)
def downgrade() -> None:
op.drop_index("idx_threads_meta_workspace_thread", table_name="threads_meta")
for table in reversed(_BUSINESS_TABLES):
with op.batch_alter_table(table) as batch:
batch.alter_column("workspace_id", existing_type=sa.String(36), nullable=True)
@@ -8,16 +8,37 @@ The actual ORM classes have moved to entity-specific subpackages:
- ``deerflow.persistence.run`` - ``deerflow.persistence.run``
- ``deerflow.persistence.feedback`` - ``deerflow.persistence.feedback``
- ``deerflow.persistence.user`` - ``deerflow.persistence.user``
- ``deerflow.persistence.workspace`` (Stage 0 PR3)
- ``deerflow.persistence.workspace_membership`` (Stage 0 PR3)
- ``deerflow.persistence.service_account`` (Stage 0 PR8)
- ``deerflow.persistence.api_key`` (Stage 0 PR8)
- ``deerflow.persistence.external_user`` (Stage 0 PR8)
``RunEventRow`` remains in ``deerflow.persistence.models.run_event`` because ``RunEventRow`` remains in ``deerflow.persistence.models.run_event`` because
its storage implementation lives in ``deerflow.runtime.events.store.db`` and its storage implementation lives in ``deerflow.runtime.events.store.db`` and
there is no matching entity directory. there is no matching entity directory.
""" """
from deerflow.persistence.api_key.model import ApiKeyRow
from deerflow.persistence.external_user.model import ExternalUserRow
from deerflow.persistence.feedback.model import FeedbackRow from deerflow.persistence.feedback.model import FeedbackRow
from deerflow.persistence.models.run_event import RunEventRow from deerflow.persistence.models.run_event import RunEventRow
from deerflow.persistence.run.model import RunRow from deerflow.persistence.run.model import RunRow
from deerflow.persistence.service_account.model import ServiceAccountRow
from deerflow.persistence.thread_meta.model import ThreadMetaRow from deerflow.persistence.thread_meta.model import ThreadMetaRow
from deerflow.persistence.user.model import UserRow from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace.model import WorkspaceRow
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
__all__ = ["FeedbackRow", "RunEventRow", "RunRow", "ThreadMetaRow", "UserRow"] __all__ = [
"ApiKeyRow",
"ExternalUserRow",
"FeedbackRow",
"RunEventRow",
"RunRow",
"ServiceAccountRow",
"ThreadMetaRow",
"UserRow",
"WorkspaceMembershipRow",
"WorkspaceRow",
]
@@ -4,7 +4,7 @@ from __future__ import annotations
from datetime import UTC, datetime from datetime import UTC, datetime
from sqlalchemy import JSON, DateTime, Index, String, Text, UniqueConstraint from sqlalchemy import JSON, DateTime, ForeignKey, Index, String, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base from deerflow.persistence.base import Base
@@ -13,23 +13,31 @@ from deerflow.persistence.base import Base
class RunEventRow(Base): class RunEventRow(Base):
__tablename__ = "run_events" __tablename__ = "run_events"
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True) id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True, comment="自增主键")
thread_id: Mapped[str] = mapped_column(String(64), nullable=False) thread_id: Mapped[str] = mapped_column(String(64), nullable=False, comment="所属会话 IDthreads_meta.thread_id")
run_id: Mapped[str] = mapped_column(String(64), nullable=False) run_id: Mapped[str] = mapped_column(String(64), nullable=False, comment="所属运行 IDruns.run_id")
# Owner of the conversation this event belongs to. Nullable for data user_id: Mapped[str | None] = mapped_column(
# created before auth was introduced; populated by auth middleware on String(64),
# new writes and by the boot-time orphan migration on existing rows. nullable=True,
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True) index=True,
event_type: Mapped[str] = mapped_column(String(32), nullable=False) comment="会话所有者;为 NULL 表示鉴权引入之前的历史数据,新写入由 auth 中间件填充,启动期 orphan 迁移会回填存量",
category: Mapped[str] = mapped_column(String(16), nullable=False) )
# "message" | "trace" | "lifecycle" workspace_id: Mapped[str] = mapped_column(
content: Mapped[str] = mapped_column(Text, default="") String(36),
event_metadata: Mapped[dict] = mapped_column(JSON, default=dict) ForeignKey("workspaces.id", ondelete="CASCADE"),
seq: Mapped[int] = mapped_column(nullable=False) nullable=False,
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC)) comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
)
event_type: Mapped[str] = mapped_column(String(32), nullable=False, comment="事件子类型(具体含义由 category 决定,如 ai_message_chunk、tool_call、run_started")
category: Mapped[str] = mapped_column(String(16), nullable=False, comment='事件大类:"message" 消息 / "trace" 追踪 / "lifecycle" 生命周期')
content: Mapped[str] = mapped_column(Text, default="", comment="事件文本内容(消息体、错误、状态字符串等)")
event_metadata: Mapped[dict] = mapped_column(JSON, default=dict, comment="事件结构化元数据(JSON),随 event_type 而异")
seq: Mapped[int] = mapped_column(nullable=False, comment="在 thread 内的全局递增序号;与 thread_id 组合唯一")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC")
__table_args__ = ( __table_args__ = (
UniqueConstraint("thread_id", "seq", name="uq_events_thread_seq"), UniqueConstraint("thread_id", "seq", name="uq_events_thread_seq"),
Index("ix_events_thread_cat_seq", "thread_id", "category", "seq"), Index("ix_events_thread_cat_seq", "thread_id", "category", "seq"),
Index("ix_events_run", "thread_id", "run_id", "seq"), Index("ix_events_run", "thread_id", "run_id", "seq"),
{"comment": "运行事件流(消息/追踪/生命周期事件按 seq 顺序追加,是消息回放与审计的真源)"},
) )
@@ -4,7 +4,7 @@ from __future__ import annotations
from datetime import UTC, datetime from datetime import UTC, datetime
from sqlalchemy import JSON, DateTime, Index, String, Text from sqlalchemy import JSON, DateTime, ForeignKey, Index, String, Text
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base from deerflow.persistence.base import Base
@@ -13,37 +13,49 @@ from deerflow.persistence.base import Base
class RunRow(Base): class RunRow(Base):
__tablename__ = "runs" __tablename__ = "runs"
run_id: Mapped[str] = mapped_column(String(64), primary_key=True) run_id: Mapped[str] = mapped_column(String(64), primary_key=True, comment="运行主键")
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True) thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True, comment="所属会话 IDthreads_meta.thread_id")
assistant_id: Mapped[str | None] = mapped_column(String(128)) assistant_id: Mapped[str | None] = mapped_column(String(128), comment="使用的 Assistant ID(自定义智能体名);为 NULL 表示默认 lead agent")
user_id: Mapped[str | None] = mapped_column(String(64), index=True) user_id: Mapped[str | None] = mapped_column(String(64), index=True, comment="发起本次运行的用户 ID")
status: Mapped[str] = mapped_column(String(20), default="pending") workspace_id: Mapped[str] = mapped_column(
# "pending" | "running" | "success" | "error" | "timeout" | "interrupted" String(36),
ForeignKey("workspaces.id", ondelete="CASCADE"),
nullable=False,
comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
)
status: Mapped[str] = mapped_column(
String(20),
default="pending",
comment='运行状态:"pending" / "running" / "success" / "error" / "timeout" / "interrupted"',
)
model_name: Mapped[str | None] = mapped_column(String(128), comment="本次运行的主模型名(来自 config.yaml.models[*].name")
multitask_strategy: Mapped[str] = mapped_column(
String(20),
default="reject",
comment='并发策略:同一 thread 已有运行时怎么处理("reject" / "interrupt" / "rollback" / "enqueue"',
)
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict, comment="运行级元数据(JSON),如 channel/source 等")
kwargs_json: Mapped[dict] = mapped_column(JSON, default=dict, comment="提交运行时的额外参数(JSON),如 thinking_enabled、tool 配置等")
error: Mapped[str | None] = mapped_column(Text, comment="运行失败时的错误文本;成功时为 NULL")
model_name: Mapped[str | None] = mapped_column(String(128)) message_count: Mapped[int] = mapped_column(default=0, comment="本次运行产生的消息总数(便利字段,避免列表页查 RunEventStore")
multitask_strategy: Mapped[str] = mapped_column(String(20), default="reject") first_human_message: Mapped[str | None] = mapped_column(Text, comment="首条用户消息文本预览(用于列表展示)")
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict) last_ai_message: Mapped[str | None] = mapped_column(Text, comment="末条 AI 消息文本预览(用于列表展示)")
kwargs_json: Mapped[dict] = mapped_column(JSON, default=dict)
error: Mapped[str | None] = mapped_column(Text)
# Convenience fields (for listing pages without querying RunEventStore) total_input_tokens: Mapped[int] = mapped_column(default=0, comment="累计输入 token 数(运行结束时由 RunJournal 落盘)")
message_count: Mapped[int] = mapped_column(default=0) total_output_tokens: Mapped[int] = mapped_column(default=0, comment="累计输出 token 数")
first_human_message: Mapped[str | None] = mapped_column(Text) total_tokens: Mapped[int] = mapped_column(default=0, comment="累计 token 总数 = input + output")
last_ai_message: Mapped[str | None] = mapped_column(Text) llm_call_count: Mapped[int] = mapped_column(default=0, comment="累计 LLM 调用次数")
lead_agent_tokens: Mapped[int] = mapped_column(default=0, comment="主 agent 自身消耗的 token 数")
subagent_tokens: Mapped[int] = mapped_column(default=0, comment="子 agenttask 工具委派)消耗的 token 数")
middleware_tokens: Mapped[int] = mapped_column(default=0, comment="中间件(如 summarization、title)消耗的 token 数")
# Token usage (accumulated in-memory by RunJournal, written on run completion) follow_up_to_run_id: Mapped[str | None] = mapped_column(String(64), comment="续接的上一次运行 ID(用于'重新生成'/'继续'等链式调用)")
total_input_tokens: Mapped[int] = mapped_column(default=0)
total_output_tokens: Mapped[int] = mapped_column(default=0)
total_tokens: Mapped[int] = mapped_column(default=0)
llm_call_count: Mapped[int] = mapped_column(default=0)
lead_agent_tokens: Mapped[int] = mapped_column(default=0)
subagent_tokens: Mapped[int] = mapped_column(default=0)
middleware_tokens: Mapped[int] = mapped_column(default=0)
# Follow-up association created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC")
follow_up_to_run_id: Mapped[str | None] = mapped_column(String(64)) updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC), comment="最近更新时间(UTC,写入时自动更新)")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC)) __table_args__ = (
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC)) Index("ix_runs_thread_status", "thread_id", "status"),
{"comment": "运行(一次完整 agent 执行)的元数据 + 累计 token 指标"},
__table_args__ = (Index("ix_runs_thread_status", "thread_id", "status"),) )
@@ -17,6 +17,13 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.run.model import RunRow from deerflow.persistence.run.model import RunRow
from deerflow.runtime.runs.store.base import RunStore from deerflow.runtime.runs.store.base import RunStore
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
from deerflow.runtime.workspace_context import (
_AutoSentinel as _WorkspaceAutoSentinel,
)
from deerflow.runtime.workspace_context import (
resolve_workspace_id,
)
class RunRepository(RunStore): class RunRepository(RunStore):
@@ -70,6 +77,7 @@ class RunRepository(RunStore):
thread_id, thread_id,
assistant_id=None, assistant_id=None,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
status="pending", status="pending",
multitask_strategy="reject", multitask_strategy="reject",
metadata=None, metadata=None,
@@ -79,12 +87,14 @@ class RunRepository(RunStore):
follow_up_to_run_id=None, follow_up_to_run_id=None,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.put") resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.put")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.put")
now = datetime.now(UTC) now = datetime.now(UTC)
row = RunRow( row = RunRow(
run_id=run_id, run_id=run_id,
thread_id=thread_id, thread_id=thread_id,
assistant_id=assistant_id, assistant_id=assistant_id,
user_id=resolved_user_id, user_id=resolved_user_id,
workspace_id=resolved_workspace_id,
status=status, status=status,
multitask_strategy=multitask_strategy, multitask_strategy=multitask_strategy,
metadata_json=self._safe_json(metadata) or {}, metadata_json=self._safe_json(metadata) or {},
@@ -103,12 +113,16 @@ class RunRepository(RunStore):
run_id, run_id,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.get") resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.get")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.get")
async with self._sf() as session: async with self._sf() as session:
row = await session.get(RunRow, run_id) row = await session.get(RunRow, run_id)
if row is None: if row is None:
return None return None
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
return None
if resolved_user_id is not None and row.user_id != resolved_user_id: if resolved_user_id is not None and row.user_id != resolved_user_id:
return None return None
return self._row_to_dict(row) return self._row_to_dict(row)
@@ -118,10 +132,14 @@ class RunRepository(RunStore):
thread_id, thread_id,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
limit=100, limit=100,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.list_by_thread") resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.list_by_thread")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.list_by_thread")
stmt = select(RunRow).where(RunRow.thread_id == thread_id) stmt = select(RunRow).where(RunRow.thread_id == thread_id)
if resolved_workspace_id is not None:
stmt = stmt.where(RunRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(RunRow.user_id == resolved_user_id) stmt = stmt.where(RunRow.user_id == resolved_user_id)
stmt = stmt.order_by(RunRow.created_at.desc()).limit(limit) stmt = stmt.order_by(RunRow.created_at.desc()).limit(limit)
@@ -142,12 +160,16 @@ class RunRepository(RunStore):
run_id, run_id,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.delete") resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.delete")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.delete")
async with self._sf() as session: async with self._sf() as session:
row = await session.get(RunRow, run_id) row = await session.get(RunRow, run_id)
if row is None: if row is None:
return return
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
return
if resolved_user_id is not None and row.user_id != resolved_user_id: if resolved_user_id is not None and row.user_id != resolved_user_id:
return return
await session.delete(row) await session.delete(row)
@@ -0,0 +1,18 @@
"""Service account persistence — ORM model only (Stage 0 PR8).
A service account is a non-human principal that lives inside a workspace
and authenticates via API keys rather than email + password. Each
service account belongs to exactly one workspace and is created by a
human user (``created_by``).
PR8 introduces only the schema + ORM row class. Repository, API-key
authentication middleware, and the ``@require_permission`` scope
upgrade live in Stage 1 alongside the headless API surface.
"""
from __future__ import annotations
from deerflow.persistence.service_account.model import ServiceAccountRow
from deerflow.persistence.service_account.sql import ServiceAccountRepository, ServiceAccountValidationError
__all__ = ["ServiceAccountRepository", "ServiceAccountRow", "ServiceAccountValidationError"]
@@ -0,0 +1,83 @@
"""ORM model for service accounts (non-human principals inside a workspace)."""
from __future__ import annotations
from datetime import UTC, datetime
from sqlalchemy import DateTime, ForeignKey, Index, String
from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base
class ServiceAccountRow(Base):
__tablename__ = "service_accounts"
id: Mapped[str] = mapped_column(
String(36),
primary_key=True,
comment="服务账号主键,UUID 字符串(36 字符),与 users.id 类型对齐",
)
workspace_id: Mapped[str] = mapped_column(
String(36),
ForeignKey("workspaces.id", ondelete="CASCADE"),
nullable=False,
comment="所属 workspaceworkspace 删除时级联清掉所有 service_account(连同其 api_keys / external_users",
)
name: Mapped[str] = mapped_column(
String(64),
nullable=False,
comment="服务账号显示名(同 workspace 内不强制唯一;Stage 1 可由 admin UI 重复使用同名 + 不同 key)",
)
role: Mapped[str] = mapped_column(
String(16),
nullable=False,
default="member",
comment='服务账号在 workspace 内的角色字符串:Stage 0 仅支持 "member"Stage 2 RBAC 打开 "admin"/"viewer"。用 String(16) 而非 enum 以便未来扩枚举值不动 schema',
)
identity_mode: Mapped[str] = mapped_column(
String(16),
nullable=False,
default="collapsed",
comment=(
'身份模式三态:"collapsed"(所有调用 collapse 到该 service_account;不记录 external_user/'
' "external_passthrough"(每次调用必带 X-External-User-Id,写入 external_users 表)/'
' "both"(带就写、不带就 collapse)。Stage 1 API key 鉴权层据此分流'
),
)
status: Mapped[str] = mapped_column(
String(16),
nullable=False,
default="active",
comment='状态:"active"(正常)/ "suspended"admin 暂停)/ "deleted"(软删;保留审计)',
)
created_by: Mapped[str] = mapped_column(
String(36),
ForeignKey("users.id", ondelete="RESTRICT"),
nullable=False,
comment="创建者 user_id;删除该 user 时 RESTRICT 阻拦(必须先转移或删除该 user 名下所有 service_account",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.now(UTC),
comment="创建时间(UTC",
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.now(UTC),
onupdate=lambda: datetime.now(UTC),
comment="最近更新时间(UTC,写入时自动更新)",
)
__table_args__ = (
Index("idx_service_accounts_workspace", "workspace_id", "status"),
{
"comment": (
"服务账号表(headless API 的非人身份)。每个 service_account 属于唯一 workspace"
"通过 api_keys 表的 API key 鉴权调用 Gatewayidentity_mode 控制是否在 external_users 表"
"记录终端用户身份。Stage 0 仅落 schemaStage 1 起接 API key 鉴权 + 路由 scope 升级。"
)
},
)
@@ -0,0 +1,98 @@
"""SQLAlchemy-backed service account repository (Stage 1 PR1).
Mirrors :class:`WorkspaceRepository`: fresh session per method,
``_row_to_dict`` static helper. Workspace scoping is enforced by the
caller (route layer reads the workspace contextvar); the repository
takes ``workspace_id`` explicitly.
"""
from __future__ import annotations
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.service_account.model import ServiceAccountRow
_VALID_STATUSES = frozenset({"active", "suspended", "deleted"})
class ServiceAccountValidationError(ValueError):
"""Raised when service account input fails application-layer validation."""
class ServiceAccountRepository:
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
@staticmethod
def _row_to_dict(row: ServiceAccountRow) -> dict[str, Any]:
return {
"id": row.id,
"workspace_id": row.workspace_id,
"name": row.name,
"role": row.role,
"identity_mode": row.identity_mode,
"status": row.status,
"created_by": row.created_by,
"created_at": row.created_at.isoformat() if row.created_at else None,
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
}
async def create(
self,
*,
workspace_id: str,
name: str,
created_by: str,
role: str = "member",
identity_mode: str = "collapsed",
status: str = "active",
) -> dict[str, Any]:
if status not in _VALID_STATUSES:
raise ServiceAccountValidationError(f"status {status!r} not in {_VALID_STATUSES!r}")
now = datetime.now(UTC)
row = ServiceAccountRow(
id=str(uuid.uuid4()),
workspace_id=workspace_id,
name=name,
role=role,
identity_mode=identity_mode,
status=status,
created_by=created_by,
created_at=now,
updated_at=now,
)
async with self._sf() as session:
session.add(row)
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
async def get(self, sa_id: str) -> dict[str, Any] | None:
async with self._sf() as session:
row = await session.get(ServiceAccountRow, sa_id)
return self._row_to_dict(row) if row else None
async def get_active(self, sa_id: str) -> dict[str, Any] | None:
"""Return the row only when ``status == 'active'`` (auth hot path)."""
async with self._sf() as session:
row = await session.get(ServiceAccountRow, sa_id)
if row is None or row.status != "active":
return None
return self._row_to_dict(row)
async def list_by_workspace(self, workspace_id: str) -> list[dict[str, Any]]:
async with self._sf() as session:
result = await session.execute(select(ServiceAccountRow).where(ServiceAccountRow.workspace_id == workspace_id).order_by(ServiceAccountRow.created_at.desc()))
return [self._row_to_dict(r) for r in result.scalars()]
async def update_status(self, sa_id: str, status: str) -> None:
if status not in _VALID_STATUSES:
raise ServiceAccountValidationError(f"status {status!r} not in {_VALID_STATUSES!r}")
async with self._sf() as session:
await session.execute(update(ServiceAccountRow).where(ServiceAccountRow.id == sa_id).values(status=status, updated_at=datetime.now(UTC)))
await session.commit()
@@ -4,12 +4,18 @@ Implementations:
- ThreadMetaRepository: SQL-backed (sqlite / postgres via SQLAlchemy) - ThreadMetaRepository: SQL-backed (sqlite / postgres via SQLAlchemy)
- MemoryThreadMetaStore: wraps LangGraph BaseStore (memory mode) - MemoryThreadMetaStore: wraps LangGraph BaseStore (memory mode)
All mutating and querying methods accept a ``user_id`` parameter with All mutating and querying methods accept both a ``user_id`` parameter
three-state semantics (see :mod:`deerflow.runtime.user_context`): (member-scoped owner check) and a ``workspace_id`` parameter (tenant
scope). Both follow three-state semantics:
- ``AUTO`` (default): resolve from the request-scoped contextvar. - ``AUTO`` (default): resolve from the request-scoped contextvar.
- Explicit ``str``: use the provided value verbatim. - Explicit ``str``: use the provided value verbatim.
- Explicit ``None``: bypass owner filtering (migration/CLI only). - Explicit ``None``: bypass that filter (migration / CLI only).
The workspace scope is the **outer** boundary: a row in workspace A is
unreachable from any user_id under workspace B. ``check_access`` returns
False on cross-workspace mismatch so the route layer can convert it into
a 404 instead of leaking thread existence across tenants.
""" """
from __future__ import annotations from __future__ import annotations
@@ -17,6 +23,8 @@ from __future__ import annotations
import abc import abc
from deerflow.runtime.user_context import AUTO, _AutoSentinel from deerflow.runtime.user_context import AUTO, _AutoSentinel
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
from deerflow.runtime.workspace_context import _AutoSentinel as _WorkspaceAutoSentinel
class ThreadMetaStore(abc.ABC): class ThreadMetaStore(abc.ABC):
@@ -27,13 +35,20 @@ class ThreadMetaStore(abc.ABC):
*, *,
assistant_id: str | None = None, assistant_id: str | None = None,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
display_name: str | None = None, display_name: str | None = None,
metadata: dict | None = None, metadata: dict | None = None,
) -> dict: ) -> dict:
pass pass
@abc.abstractmethod @abc.abstractmethod
async def get(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> dict | None: async def get(
self,
thread_id: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> dict | None:
pass pass
@abc.abstractmethod @abc.abstractmethod
@@ -45,32 +60,72 @@ class ThreadMetaStore(abc.ABC):
limit: int = 100, limit: int = 100,
offset: int = 0, offset: int = 0,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> list[dict]: ) -> list[dict]:
pass pass
@abc.abstractmethod @abc.abstractmethod
async def update_display_name(self, thread_id: str, display_name: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def update_display_name(
self,
thread_id: str,
display_name: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
pass pass
@abc.abstractmethod @abc.abstractmethod
async def update_status(self, thread_id: str, status: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def update_status(
self,
thread_id: str,
status: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
pass pass
@abc.abstractmethod @abc.abstractmethod
async def update_metadata(self, thread_id: str, metadata: dict, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def update_metadata(
self,
thread_id: str,
metadata: dict,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
"""Merge ``metadata`` into the thread's metadata field. """Merge ``metadata`` into the thread's metadata field.
Existing keys are overwritten by the new values; keys absent from Existing keys are overwritten by the new values; keys absent from
``metadata`` are preserved. No-op if the thread does not exist ``metadata`` are preserved. No-op if the thread does not exist
or the owner check fails. or the user/workspace check fails.
""" """
pass pass
@abc.abstractmethod @abc.abstractmethod
async def check_access(self, thread_id: str, user_id: str, *, require_existing: bool = False) -> bool: async def check_access(
"""Check if ``user_id`` has access to ``thread_id``.""" self,
thread_id: str,
user_id: str,
workspace_id: str,
*,
require_existing: bool = False,
) -> bool:
"""Check whether ``user_id`` (in ``workspace_id``) can access ``thread_id``.
Cross-workspace access returns ``False`` unconditionally so the
decorator layer can convert it into a 404 never leak the
existence of a thread that belongs to a different tenant.
"""
pass pass
@abc.abstractmethod @abc.abstractmethod
async def delete(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def delete(
self,
thread_id: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
pass pass
@@ -13,6 +13,13 @@ from langgraph.store.base import BaseStore
from deerflow.persistence.thread_meta.base import ThreadMetaStore from deerflow.persistence.thread_meta.base import ThreadMetaStore
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
from deerflow.runtime.workspace_context import (
_AutoSentinel as _WorkspaceAutoSentinel,
)
from deerflow.runtime.workspace_context import (
resolve_workspace_id,
)
from deerflow.utils.time import coerce_iso, now_iso from deerflow.utils.time import coerce_iso, now_iso
THREADS_NS: tuple[str, ...] = ("threads",) THREADS_NS: tuple[str, ...] = ("threads",)
@@ -26,15 +33,19 @@ class MemoryThreadMetaStore(ThreadMetaStore):
self, self,
thread_id: str, thread_id: str,
user_id: str | None | _AutoSentinel, user_id: str | None | _AutoSentinel,
workspace_id: str | None | _WorkspaceAutoSentinel,
method_name: str, method_name: str,
) -> dict | None: ) -> dict | None:
"""Fetch a record and verify ownership. Returns a mutable copy, or None.""" """Fetch a record and verify workspace + ownership. Returns a mutable copy, or None."""
resolved = resolve_user_id(user_id, method_name=method_name) resolved_user = resolve_user_id(user_id, method_name=method_name)
resolved_workspace = resolve_workspace_id(workspace_id, method_name=method_name)
item = await self._store.aget(THREADS_NS, thread_id) item = await self._store.aget(THREADS_NS, thread_id)
if item is None: if item is None:
return None return None
record = dict(item.value) record = dict(item.value)
if resolved is not None and record.get("user_id") != resolved: if resolved_workspace is not None and record.get("workspace_id") != resolved_workspace:
return None
if resolved_user is not None and record.get("user_id") != resolved_user:
return None return None
return record return record
@@ -44,15 +55,18 @@ class MemoryThreadMetaStore(ThreadMetaStore):
*, *,
assistant_id: str | None = None, assistant_id: str | None = None,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
display_name: str | None = None, display_name: str | None = None,
metadata: dict | None = None, metadata: dict | None = None,
) -> dict: ) -> dict:
resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.create") resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.create")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="MemoryThreadMetaStore.create")
now = now_iso() now = now_iso()
record: dict[str, Any] = { record: dict[str, Any] = {
"thread_id": thread_id, "thread_id": thread_id,
"assistant_id": assistant_id, "assistant_id": assistant_id,
"user_id": resolved_user_id, "user_id": resolved_user_id,
"workspace_id": resolved_workspace_id,
"display_name": display_name, "display_name": display_name,
"status": "idle", "status": "idle",
"metadata": metadata or {}, "metadata": metadata or {},
@@ -63,8 +77,14 @@ class MemoryThreadMetaStore(ThreadMetaStore):
await self._store.aput(THREADS_NS, thread_id, record) await self._store.aput(THREADS_NS, thread_id, record)
return record return record
async def get(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> dict | None: async def get(
return await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.get") self,
thread_id: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> dict | None:
return await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.get")
async def search( async def search(
self, self,
@@ -74,13 +94,17 @@ class MemoryThreadMetaStore(ThreadMetaStore):
limit: int = 100, limit: int = 100,
offset: int = 0, offset: int = 0,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> list[dict]: ) -> list[dict]:
resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.search") resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.search")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="MemoryThreadMetaStore.search")
filter_dict: dict[str, Any] = {} filter_dict: dict[str, Any] = {}
if metadata: if metadata:
filter_dict.update(metadata) filter_dict.update(metadata)
if status: if status:
filter_dict["status"] = status filter_dict["status"] = status
if resolved_workspace_id is not None:
filter_dict["workspace_id"] = resolved_workspace_id
if resolved_user_id is not None: if resolved_user_id is not None:
filter_dict["user_id"] = resolved_user_id filter_dict["user_id"] = resolved_user_id
@@ -92,33 +116,64 @@ class MemoryThreadMetaStore(ThreadMetaStore):
) )
return [self._item_to_dict(item) for item in items] return [self._item_to_dict(item) for item in items]
async def check_access(self, thread_id: str, user_id: str, *, require_existing: bool = False) -> bool: async def check_access(
self,
thread_id: str,
user_id: str,
workspace_id: str,
*,
require_existing: bool = False,
) -> bool:
item = await self._store.aget(THREADS_NS, thread_id) item = await self._store.aget(THREADS_NS, thread_id)
if item is None: if item is None:
return not require_existing return not require_existing
record_workspace_id = item.value.get("workspace_id")
if record_workspace_id is not None and record_workspace_id != workspace_id:
return False
record_user_id = item.value.get("user_id") record_user_id = item.value.get("user_id")
if record_user_id is None: if record_user_id is None:
return True return True
return record_user_id == user_id return record_user_id == user_id
async def update_display_name(self, thread_id: str, display_name: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def update_display_name(
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.update_display_name") self,
thread_id: str,
display_name: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.update_display_name")
if record is None: if record is None:
return return
record["display_name"] = display_name record["display_name"] = display_name
record["updated_at"] = now_iso() record["updated_at"] = now_iso()
await self._store.aput(THREADS_NS, thread_id, record) await self._store.aput(THREADS_NS, thread_id, record)
async def update_status(self, thread_id: str, status: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def update_status(
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.update_status") self,
thread_id: str,
status: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.update_status")
if record is None: if record is None:
return return
record["status"] = status record["status"] = status
record["updated_at"] = now_iso() record["updated_at"] = now_iso()
await self._store.aput(THREADS_NS, thread_id, record) await self._store.aput(THREADS_NS, thread_id, record)
async def update_metadata(self, thread_id: str, metadata: dict, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def update_metadata(
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.update_metadata") self,
thread_id: str,
metadata: dict,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.update_metadata")
if record is None: if record is None:
return return
merged = dict(record.get("metadata") or {}) merged = dict(record.get("metadata") or {})
@@ -127,8 +182,14 @@ class MemoryThreadMetaStore(ThreadMetaStore):
record["updated_at"] = now_iso() record["updated_at"] = now_iso()
await self._store.aput(THREADS_NS, thread_id, record) await self._store.aput(THREADS_NS, thread_id, record)
async def delete(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None: async def delete(
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.delete") self,
thread_id: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None:
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.delete")
if record is None: if record is None:
return return
await self._store.adelete(THREADS_NS, thread_id) await self._store.adelete(THREADS_NS, thread_id)
@@ -4,7 +4,7 @@ from __future__ import annotations
from datetime import UTC, datetime from datetime import UTC, datetime
from sqlalchemy import JSON, DateTime, String from sqlalchemy import JSON, DateTime, ForeignKey, Index, String
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base from deerflow.persistence.base import Base
@@ -13,11 +13,25 @@ from deerflow.persistence.base import Base
class ThreadMetaRow(Base): class ThreadMetaRow(Base):
__tablename__ = "threads_meta" __tablename__ = "threads_meta"
thread_id: Mapped[str] = mapped_column(String(64), primary_key=True) thread_id: Mapped[str] = mapped_column(String(64), primary_key=True, comment="会话主键(LangGraph thread_id")
assistant_id: Mapped[str | None] = mapped_column(String(128), index=True) assistant_id: Mapped[str | None] = mapped_column(String(128), index=True, comment="关联的 Assistant ID(自定义智能体名);为 NULL 表示默认 lead agent")
user_id: Mapped[str | None] = mapped_column(String(64), index=True) user_id: Mapped[str | None] = mapped_column(String(64), index=True, comment="会话所有者;为 NULL 表示历史无主数据")
display_name: Mapped[str | None] = mapped_column(String(256)) workspace_id: Mapped[str] = mapped_column(
status: Mapped[str] = mapped_column(String(20), default="idle") String(36),
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict) ForeignKey("workspaces.id", ondelete="CASCADE"),
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC)) nullable=False,
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC)) comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
)
display_name: Mapped[str | None] = mapped_column(String(256), comment="会话显示名(自动生成的标题或用户手改)")
status: Mapped[str] = mapped_column(String(20), default="idle", comment='会话状态:"idle" 空闲 / "busy" 正在产出')
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict, comment="任意扩展元数据(JSON")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC), comment="最近更新时间(UTC,写入时自动更新)")
__table_args__ = (
# Workspace-scoped "list threads of this user, newest first" index;
# added by alembic 0002. Mirrored on the ORM side so create_all()
# produces the same shape on fresh dev databases.
Index("idx_threads_meta_workspace_user_updated", "workspace_id", "user_id", "updated_at"),
{"comment": "会话元数据(每个 LangGraph thread 的概要信息)"},
)
@@ -11,6 +11,13 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.thread_meta.base import ThreadMetaStore from deerflow.persistence.thread_meta.base import ThreadMetaStore
from deerflow.persistence.thread_meta.model import ThreadMetaRow from deerflow.persistence.thread_meta.model import ThreadMetaRow
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
from deerflow.runtime.workspace_context import (
_AutoSentinel as _WorkspaceAutoSentinel,
)
from deerflow.runtime.workspace_context import (
resolve_workspace_id,
)
class ThreadMetaRepository(ThreadMetaStore): class ThreadMetaRepository(ThreadMetaStore):
@@ -33,17 +40,21 @@ class ThreadMetaRepository(ThreadMetaStore):
*, *,
assistant_id: str | None = None, assistant_id: str | None = None,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
display_name: str | None = None, display_name: str | None = None,
metadata: dict | None = None, metadata: dict | None = None,
) -> dict: ) -> dict:
# Auto-resolve user_id from contextvar when AUTO; explicit None # Auto-resolve both user_id and workspace_id from contextvars when
# creates an orphan row (used by migration scripts). # AUTO; explicit None creates an orphan row (used by migration
# scripts that intentionally bypass scope).
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.create") resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.create")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.create")
now = datetime.now(UTC) now = datetime.now(UTC)
row = ThreadMetaRow( row = ThreadMetaRow(
thread_id=thread_id, thread_id=thread_id,
assistant_id=assistant_id, assistant_id=assistant_id,
user_id=resolved_user_id, user_id=resolved_user_id,
workspace_id=resolved_workspace_id,
display_name=display_name, display_name=display_name,
metadata_json=metadata or {}, metadata_json=metadata or {},
created_at=now, created_at=now,
@@ -60,43 +71,52 @@ class ThreadMetaRepository(ThreadMetaStore):
thread_id: str, thread_id: str,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> dict | None: ) -> dict | None:
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.get") resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.get")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.get")
stmt = select(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id)
if resolved_workspace_id is not None:
stmt = stmt.where(ThreadMetaRow.workspace_id == resolved_workspace_id)
async with self._sf() as session: async with self._sf() as session:
row = await session.get(ThreadMetaRow, thread_id) row = (await session.execute(stmt)).scalar_one_or_none()
if row is None: if row is None:
return None return None
# Enforce owner filter unless explicitly bypassed (user_id=None). # Owner filter still applies inside the workspace scope.
if resolved_user_id is not None and row.user_id != resolved_user_id: if resolved_user_id is not None and row.user_id != resolved_user_id:
return None return None
return self._row_to_dict(row) return self._row_to_dict(row)
async def check_access(self, thread_id: str, user_id: str, *, require_existing: bool = False) -> bool: async def check_access(
"""Check if ``user_id`` has access to ``thread_id``. self,
thread_id: str,
user_id: str,
workspace_id: str,
*,
require_existing: bool = False,
) -> bool:
"""Check if ``user_id`` in ``workspace_id`` has access to ``thread_id``.
Two modes one row, two distinct semantics depending on what Three filters layered, from outside in:
the caller is about to do:
- ``require_existing=False`` (default, permissive): - Cross-workspace is **always** denied (returns False), even when
Returns True for: row missing (untracked legacy thread), the row exists and ``user_id`` matches. The decorator layer
``row.user_id`` is None (shared / pre-auth data), converts a False into a 404 so cross-tenant access never leaks
or ``row.user_id == user_id``. Use for **read-style** the existence of a thread.
decorators where treating an untracked thread as accessible - Missing row honours ``require_existing``: False by default
preserves backward-compat. (permissive untracked legacy threads still readable), True
for destructive routes (DELETE / PATCH) so a re-targeted ghost
- ``require_existing=True`` (strict): row cannot be claimed.
Returns True **only** when the row exists AND - Within the workspace, ``row.user_id IS NULL`` keeps the legacy
(``row.user_id == user_id`` OR ``row.user_id is None``). "shared / pre-auth" semantics readable by anyone in the
Use for **destructive / mutating** decorators (DELETE, PATCH, workspace. ``row.user_id == user_id`` is the normal case.
state-update) so a thread that has *already been deleted*
cannot be re-targeted by any caller closing the
delete-idempotence cross-user gap where the row vanishing
made every other user appear to "own" it.
""" """
async with self._sf() as session: async with self._sf() as session:
row = await session.get(ThreadMetaRow, thread_id) row = await session.get(ThreadMetaRow, thread_id)
if row is None: if row is None:
return not require_existing return not require_existing
if row.workspace_id is not None and row.workspace_id != workspace_id:
return False
if row.user_id is None: if row.user_id is None:
return True return True
return row.user_id == user_id return row.user_id == user_id
@@ -109,14 +129,19 @@ class ThreadMetaRepository(ThreadMetaStore):
limit: int = 100, limit: int = 100,
offset: int = 0, offset: int = 0,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> list[dict]: ) -> list[dict]:
"""Search threads with optional metadata and status filters. """Search threads with optional metadata and status filters.
Owner filter is enforced by default: caller must be in a user Both workspace and owner filters are enforced by default. Pass
context. Pass ``user_id=None`` to bypass (migration/CLI). ``workspace_id=None`` and / or ``user_id=None`` to bypass for
migration / CLI paths.
""" """
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.search") resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.search")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.search")
stmt = select(ThreadMetaRow).order_by(ThreadMetaRow.updated_at.desc()) stmt = select(ThreadMetaRow).order_by(ThreadMetaRow.updated_at.desc())
if resolved_workspace_id is not None:
stmt = stmt.where(ThreadMetaRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(ThreadMetaRow.user_id == resolved_user_id) stmt = stmt.where(ThreadMetaRow.user_id == resolved_user_id)
if status: if status:
@@ -138,12 +163,22 @@ class ThreadMetaRepository(ThreadMetaStore):
result = await session.execute(stmt) result = await session.execute(stmt)
return [self._row_to_dict(r) for r in result.scalars()] return [self._row_to_dict(r) for r in result.scalars()]
async def _check_ownership(self, session: AsyncSession, thread_id: str, resolved_user_id: str | None) -> bool: async def _check_ownership(
"""Return True if the row exists and is owned (or filter bypassed).""" self,
if resolved_user_id is None: session: AsyncSession,
return True # explicit bypass thread_id: str,
resolved_user_id: str | None,
resolved_workspace_id: str | None,
) -> bool:
"""Return True if the row exists, is in scope, and is owned (or filter bypassed)."""
row = await session.get(ThreadMetaRow, thread_id) row = await session.get(ThreadMetaRow, thread_id)
return row is not None and row.user_id == resolved_user_id if row is None:
return False
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
return False
if resolved_user_id is not None and row.user_id != resolved_user_id:
return False
return True
async def update_display_name( async def update_display_name(
self, self,
@@ -151,11 +186,13 @@ class ThreadMetaRepository(ThreadMetaStore):
display_name: str, display_name: str,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None: ) -> None:
"""Update the display_name (title) for a thread.""" """Update the display_name (title) for a thread."""
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_display_name") resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_display_name")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.update_display_name")
async with self._sf() as session: async with self._sf() as session:
if not await self._check_ownership(session, thread_id, resolved_user_id): if not await self._check_ownership(session, thread_id, resolved_user_id, resolved_workspace_id):
return return
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(display_name=display_name, updated_at=datetime.now(UTC))) await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(display_name=display_name, updated_at=datetime.now(UTC)))
await session.commit() await session.commit()
@@ -166,10 +203,12 @@ class ThreadMetaRepository(ThreadMetaStore):
status: str, status: str,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None: ) -> None:
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_status") resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_status")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.update_status")
async with self._sf() as session: async with self._sf() as session:
if not await self._check_ownership(session, thread_id, resolved_user_id): if not await self._check_ownership(session, thread_id, resolved_user_id, resolved_workspace_id):
return return
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(status=status, updated_at=datetime.now(UTC))) await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(status=status, updated_at=datetime.now(UTC)))
await session.commit() await session.commit()
@@ -180,18 +219,22 @@ class ThreadMetaRepository(ThreadMetaStore):
metadata: dict, metadata: dict,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None: ) -> None:
"""Merge ``metadata`` into ``metadata_json``. """Merge ``metadata`` into ``metadata_json``.
Read-modify-write inside a single session/transaction so concurrent Read-modify-write inside a single session/transaction so concurrent
callers see consistent state. No-op if the row does not exist or callers see consistent state. No-op if the row does not exist or
the user_id check fails. the workspace / user check fails.
""" """
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_metadata") resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_metadata")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.update_metadata")
async with self._sf() as session: async with self._sf() as session:
row = await session.get(ThreadMetaRow, thread_id) row = await session.get(ThreadMetaRow, thread_id)
if row is None: if row is None:
return return
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
return
if resolved_user_id is not None and row.user_id != resolved_user_id: if resolved_user_id is not None and row.user_id != resolved_user_id:
return return
merged = dict(row.metadata_json or {}) merged = dict(row.metadata_json or {})
@@ -205,12 +248,16 @@ class ThreadMetaRepository(ThreadMetaStore):
thread_id: str, thread_id: str,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
) -> None: ) -> None:
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.delete") resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.delete")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.delete")
async with self._sf() as session: async with self._sf() as session:
row = await session.get(ThreadMetaRow, thread_id) row = await session.get(ThreadMetaRow, thread_id)
if row is None: if row is None:
return return
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
return
if resolved_user_id is not None and row.user_id != resolved_user_id: if resolved_user_id is not None and row.user_id != resolved_user_id:
return return
await session.delete(row) await session.delete(row)
@@ -13,7 +13,7 @@ from __future__ import annotations
from datetime import UTC, datetime from datetime import UTC, datetime
from sqlalchemy import Boolean, DateTime, Index, String, text from sqlalchemy import Boolean, DateTime, ForeignKey, Index, String, text
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base from deerflow.persistence.base import Base
@@ -22,31 +22,26 @@ from deerflow.persistence.base import Base
class UserRow(Base): class UserRow(Base):
__tablename__ = "users" __tablename__ = "users"
# UUIDs are stored as 36-char strings for cross-backend portability. id: Mapped[str] = mapped_column(String(36), primary_key=True, comment="用户主键,UUID 字符串(36 字符),跨数据库可移植")
id: Mapped[str] = mapped_column(String(36), primary_key=True) email: Mapped[str] = mapped_column(String(320), unique=True, nullable=False, index=True, comment="登录邮箱,全局唯一")
password_hash: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="本地账户的密码哈希;OAuth-only 用户为 NULL")
email: Mapped[str] = mapped_column(String(320), unique=True, nullable=False, index=True) system_role: Mapped[str] = mapped_column(String(16), nullable=False, default="user", comment='系统角色:"admin""user";用字符串以便未来扩展角色而不必 ALTER TABLE')
password_hash: Mapped[str | None] = mapped_column(String(128), nullable=True)
# "admin" | "user" — kept as plain string to avoid ALTER TABLE pain
# when new roles are introduced.
system_role: Mapped[str] = mapped_column(String(16), nullable=False, default="user")
created_at: Mapped[datetime] = mapped_column( created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), DateTime(timezone=True),
nullable=False, nullable=False,
default=lambda: datetime.now(UTC), default=lambda: datetime.now(UTC),
comment="账户创建时间(UTC",
)
oauth_provider: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="OAuth 提供商名(如 google/github);本地账户为 NULL")
oauth_id: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="OAuth 提供商内的用户 ID;与 oauth_provider 组合需唯一")
needs_setup: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, comment="是否需要完成首次设置(admin 自动创建后改密码/邮箱)")
token_version: Mapped[int] = mapped_column(nullable=False, default=0, comment="JWT 令牌版本号;自增即吊销该用户所有旧令牌")
default_workspace_id: Mapped[str | None] = mapped_column(
String(36),
ForeignKey("workspaces.id", ondelete="SET NULL"),
nullable=True,
comment="登录后默认进入的 workspaceNULL 时强制走 pickeruser 多 workspace 场景)",
) )
# OAuth linkage (optional). A partial unique index enforces one
# account per (provider, oauth_id) pair, leaving NULL/NULL rows
# unconstrained so plain password accounts can coexist.
oauth_provider: Mapped[str | None] = mapped_column(String(32), nullable=True)
oauth_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
# Auth lifecycle flags
needs_setup: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
token_version: Mapped[int] = mapped_column(nullable=False, default=0)
__table_args__ = ( __table_args__ = (
Index( Index(
@@ -56,4 +51,5 @@ class UserRow(Base):
unique=True, unique=True,
sqlite_where=text("oauth_provider IS NOT NULL AND oauth_id IS NOT NULL"), sqlite_where=text("oauth_provider IS NOT NULL AND oauth_id IS NOT NULL"),
), ),
{"comment": "用户账户表(本地密码登录 + OAuth 联合登录)"},
) )
@@ -0,0 +1,17 @@
"""Workspace persistence — ORM model + repository.
A workspace is the multi-tenant scope unit. Every user has at least
one (auto-created on registration; their personal workspace where
they are sole owner). Team plans get multi-member workspaces.
Stage 0 PR3 introduces the schema + repository; PR4 wires it into
the registration / login flow; PR5+ ALTER existing business tables
to FK back to ``workspaces.id``.
"""
from __future__ import annotations
from deerflow.persistence.workspace.model import WorkspaceRow
from deerflow.persistence.workspace.sql import WorkspaceRepository, WorkspaceValidationError
__all__ = ["WorkspaceRepository", "WorkspaceRow", "WorkspaceValidationError"]
@@ -0,0 +1,58 @@
"""ORM model for workspaces (multi-tenant scope)."""
from __future__ import annotations
from datetime import UTC, datetime
from sqlalchemy import DateTime, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base
class WorkspaceRow(Base):
__tablename__ = "workspaces"
id: Mapped[str] = mapped_column(
String(36),
primary_key=True,
comment="工作空间主键,UUID 字符串(36 字符),与 users.id 类型对齐",
)
name: Mapped[str] = mapped_column(
String(64),
nullable=False,
comment="工作空间显示名(用户注册时默认 <email 前缀>'s Workspace",
)
slug: Mapped[str] = mapped_column(
String(32),
nullable=False,
unique=True,
comment="URL 标识(^[a-z0-9](-?[a-z0-9])*$3-32 字符);全局唯一,DB 存小写",
)
status: Mapped[str] = mapped_column(
String(16),
nullable=False,
default="active",
comment='状态:"active"(正常)/ "suspended"(平台 admin 暂停)/ "deleted"(软删)',
)
owner_id: Mapped[str] = mapped_column(
String(36),
ForeignKey("users.id", ondelete="RESTRICT"),
nullable=False,
comment="所有者 user_id;与 workspace_memberships 中 role='owner' 行严格一致(事务保证);删除 owner 时 RESTRICT 阻拦(必须先转让所有权)",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.now(UTC),
comment="创建时间(UTC",
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.now(UTC),
onupdate=lambda: datetime.now(UTC),
comment="最近更新时间(UTC,写入时自动更新)",
)
__table_args__ = ({"comment": ("工作空间表(多租户隔离粒度单位)。每个用户注册时自动建一个 1 人 workspace,owner 即注册者。团队订阅时 workspace 可有多个 member。")},)
@@ -0,0 +1,230 @@
"""SQLAlchemy-backed workspace repository.
CRUD + slug lookup for ``workspaces``. Membership-aware methods
(``get``, ``list_by_user``) JOIN against ``workspace_memberships`` so
callers cannot read workspaces they don't belong to.
Three-state ``user_id`` parameter (same convention as
:class:`ThreadMetaRepository`):
- ``AUTO`` read from contextvar; raise if unset
- explicit ``str`` override contextvar (admin / tests)
- explicit ``None`` no filter (migration / CLI only)
"""
from __future__ import annotations
import re
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.workspace.model import WorkspaceRow
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
# slug 字符集 / 长度(与 workspace-schema-design §2.1 锁定一致)。
_SLUG_PATTERN = re.compile(r"^[a-z0-9](-?[a-z0-9])*$")
_SLUG_MIN_LEN = 3
_SLUG_MAX_LEN = 32
# slug 黑名单(应用层校验,不写 DB constraint)。包含 ADR-007 §4 保留 slug
# + 路径 + Next.js 保留 + 业务保留词。
SLUG_BLACKLIST = frozenset(
{
"admin",
"api",
"auth",
"login",
"signup",
"accept-invite",
"pricing",
"docs",
"status",
"platform",
"system",
"health",
"static",
"public",
"favicon.ico",
"robots.txt",
"sitemap.xml",
"_next",
".well-known",
"settings",
"billing",
"onboarding",
"select-workspace",
}
)
# 允许的 status 集合。
_VALID_STATUSES = frozenset({"active", "suspended", "deleted"})
class WorkspaceValidationError(ValueError):
"""Raised when workspace input fails application-layer validation
(slug format / blacklist / status enum)."""
def _validate_slug(slug: str) -> None:
"""Raise :class:`WorkspaceValidationError` if slug is invalid."""
if not isinstance(slug, str):
raise WorkspaceValidationError(f"slug must be a string, got {type(slug).__name__}")
if not (_SLUG_MIN_LEN <= len(slug) <= _SLUG_MAX_LEN):
raise WorkspaceValidationError(f"slug length must be between {_SLUG_MIN_LEN} and {_SLUG_MAX_LEN}, got {len(slug)}")
if not _SLUG_PATTERN.fullmatch(slug):
raise WorkspaceValidationError(f"slug {slug!r} does not match required pattern ^[a-z0-9](-?[a-z0-9])*$")
if slug in SLUG_BLACKLIST:
raise WorkspaceValidationError(f"slug {slug!r} is reserved")
def _validate_status(status: str) -> None:
if status not in _VALID_STATUSES:
raise WorkspaceValidationError(f"status {status!r} is not in allowed set {_VALID_STATUSES!r}")
class WorkspaceRepository:
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
@staticmethod
def _row_to_dict(row: WorkspaceRow) -> dict[str, Any]:
return {
"id": row.id,
"name": row.name,
"slug": row.slug,
"status": row.status,
"owner_id": row.owner_id,
"created_at": row.created_at.isoformat() if row.created_at else None,
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
}
async def create(
self,
*,
name: str,
slug: str,
owner_id: str,
workspace_id: str | None = None,
status: str = "active",
) -> dict[str, Any]:
"""Create a new workspace.
``workspace_id`` is optional UUID v4 is generated if omitted.
Caller is responsible for creating the matching ``owner`` row
in ``workspace_memberships`` (this is typically done in the same
transaction by the registration flow; we deliberately don't bundle
it here to keep the repository single-responsibility).
Raises :class:`WorkspaceValidationError` for invalid slug / status.
Raises :class:`sqlalchemy.exc.IntegrityError` for slug collision
or invalid owner_id FK.
"""
_validate_slug(slug)
_validate_status(status)
wid = workspace_id or str(uuid.uuid4())
now = datetime.now(UTC)
row = WorkspaceRow(
id=wid,
name=name,
slug=slug,
status=status,
owner_id=owner_id,
created_at=now,
updated_at=now,
)
async with self._sf() as session:
session.add(row)
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
async def get(
self,
workspace_id: str,
*,
user_id: str | None | _AutoSentinel = AUTO,
) -> dict[str, Any] | None:
"""Return workspace row IFF caller is a member.
``user_id=None`` bypasses the membership filter (migration / CLI).
Returns ``None`` if the workspace does not exist or the caller is
not a member.
"""
resolved_user_id = resolve_user_id(user_id, method_name="WorkspaceRepository.get")
async with self._sf() as session:
row = await session.get(WorkspaceRow, workspace_id)
if row is None:
return None
if resolved_user_id is None:
# Explicit bypass (migration / admin path).
return self._row_to_dict(row)
# Membership check via separate SELECT (cheap; index covers it).
membership = await session.execute(
select(WorkspaceMembershipRow).where(
WorkspaceMembershipRow.workspace_id == workspace_id,
WorkspaceMembershipRow.user_id == resolved_user_id,
)
)
if membership.scalar_one_or_none() is None:
return None
return self._row_to_dict(row)
async def get_by_slug(self, slug: str) -> dict[str, Any] | None:
"""Public slug lookup — does NOT check membership.
Used by path-based routing (``/{slug}/...``) where we need to
resolve slug workspace_id BEFORE we know if caller belongs.
Membership check happens downstream in the route handler.
"""
async with self._sf() as session:
result = await session.execute(select(WorkspaceRow).where(WorkspaceRow.slug == slug))
row = result.scalar_one_or_none()
return self._row_to_dict(row) if row else None
async def list_by_user(
self,
*,
user_id: str | None | _AutoSentinel = AUTO,
) -> list[dict[str, Any]]:
"""Return all workspaces caller is a member of, ordered by joined_at desc.
``user_id=None`` lists ALL workspaces (migration / admin path).
"""
resolved_user_id = resolve_user_id(user_id, method_name="WorkspaceRepository.list_by_user")
async with self._sf() as session:
stmt = select(WorkspaceRow).order_by(WorkspaceRow.created_at.desc())
if resolved_user_id is not None:
stmt = stmt.join(
WorkspaceMembershipRow,
WorkspaceMembershipRow.workspace_id == WorkspaceRow.id,
).where(WorkspaceMembershipRow.user_id == resolved_user_id)
result = await session.execute(stmt)
return [self._row_to_dict(r) for r in result.scalars()]
async def update_status(self, workspace_id: str, status: str) -> None:
"""Platform-admin operation: change workspace status (active/suspended/deleted).
No membership check this is for platform-level operations. Audit
logging belongs at the route layer.
"""
_validate_status(status)
async with self._sf() as session:
await session.execute(update(WorkspaceRow).where(WorkspaceRow.id == workspace_id).values(status=status, updated_at=datetime.now(UTC)))
await session.commit()
async def delete(self, workspace_id: str) -> None:
"""Hard-delete a workspace. CASCADE drops all memberships.
Intentionally no membership check caller (platform admin route)
must enforce authorization. Stage 0 doesn't expose this to end
users; Stage 2+ adds it behind owner_only permission.
"""
async with self._sf() as session:
row = await session.get(WorkspaceRow, workspace_id)
if row is not None:
await session.delete(row)
await session.commit()
@@ -0,0 +1,24 @@
"""Workspace membership persistence — ORM model + repository.
Tracks who is a member of which workspace and what their role is
within that workspace. Composite primary key ``(workspace_id, user_id)``;
``role`` follows the three-state ``owner`` / ``admin`` / ``member`` model
(Stage 0 only writes ``'owner'``; Stage 2 RBAC rollout opens admin/member).
A workspace MUST have exactly one ``owner`` enforced by a partial
unique index. Owner transfer is a two-row transactional swap.
"""
from __future__ import annotations
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
from deerflow.persistence.workspace_membership.sql import (
MembershipValidationError,
WorkspaceMembershipRepository,
)
__all__ = [
"MembershipValidationError",
"WorkspaceMembershipRepository",
"WorkspaceMembershipRow",
]
@@ -0,0 +1,62 @@
"""ORM model for workspace memberships (which user is in which workspace, with what role)."""
from __future__ import annotations
from datetime import UTC, datetime
from sqlalchemy import DateTime, ForeignKey, Index, String, text
from sqlalchemy.orm import Mapped, mapped_column
from deerflow.persistence.base import Base
class WorkspaceMembershipRow(Base):
__tablename__ = "workspace_memberships"
workspace_id: Mapped[str] = mapped_column(
String(36),
ForeignKey("workspaces.id", ondelete="CASCADE"),
primary_key=True,
comment="所属 workspaceworkspace 删除时级联清掉成员记录",
)
user_id: Mapped[str] = mapped_column(
String(36),
ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
comment="成员 user_id;用户删除时级联清掉成员记录",
)
role: Mapped[str] = mapped_column(
String(16),
nullable=False,
comment='角色字符串:Stage 0 仅写 "owner"Stage 2 RBAC 打开 "admin"/"member"。用 String(16) 而非 enum 以便未来加 "viewer"/"auditor" 不动 schema',
)
invited_by: Mapped[str | None] = mapped_column(
String(36),
ForeignKey("users.id", ondelete="SET NULL"),
nullable=True,
comment="邀请人 user_idStage 2 invitation 流程才写);邀请人被删时此字段清空(不影响成员记录本身)",
)
joined_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
nullable=False,
default=lambda: datetime.now(UTC),
comment="加入 workspace 的时间(UTC",
)
__table_args__ = (
# 倒查索引:列出 user 所属的所有 workspace/auth/me 用)。
# 复合主键 (workspace_id, user_id) 的前导列是 workspace_id
# 所以"按 user_id 查"需要单独的索引。
Index("idx_workspace_memberships_user", "user_id", "workspace_id"),
# 一个 workspace 严格 1 个 owner —— partial unique on role='owner'。
# SQLite + Postgres 都支持 WHERE 子句的 partial unique;双驱动
# 维护两套等价 where 表达式。
Index(
"idx_one_owner_per_workspace",
"workspace_id",
unique=True,
sqlite_where=text("role = 'owner'"),
postgresql_where=text("role = 'owner'"),
),
{"comment": ("工作空间成员表(multi-tenant RBAC)。复合 PK (workspace_id, user_id);每个 workspace 必有恰好 1 个 ownerpartial unique 约束保证)。Stage 0 仅写 ownerStage 2 RBAC 打开 admin/member。")},
)
@@ -0,0 +1,149 @@
"""SQLAlchemy-backed workspace membership repository.
Manages the ``workspace_memberships`` join table. Stage 0 only writes
``role='owner'`` (single-user workspaces); the repository accepts the
full Stage 2 RBAC role enum so the schema is forward-compatible.
Owner transfer is intentionally NOT modelled here as a single method
it requires a two-row transactional swap with careful retry semantics,
and belongs in the auth router (PR4+) where it can be wrapped in a
permission check.
"""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import delete, select, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
# 允许的 role 字面值。Stage 0 实际仅写 owneradmin/member 留给 Stage 2 RBAC。
_VALID_ROLES = frozenset({"owner", "admin", "member"})
class MembershipValidationError(ValueError):
"""Raised when role is not in the allowed enum."""
def _validate_role(role: str) -> None:
if role not in _VALID_ROLES:
raise MembershipValidationError(f"role {role!r} is not in allowed set {_VALID_ROLES!r}")
class WorkspaceMembershipRepository:
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
self._sf = session_factory
@staticmethod
def _row_to_dict(row: WorkspaceMembershipRow) -> dict[str, Any]:
return {
"workspace_id": row.workspace_id,
"user_id": row.user_id,
"role": row.role,
"invited_by": row.invited_by,
"joined_at": row.joined_at.isoformat() if row.joined_at else None,
}
async def add(
self,
*,
workspace_id: str,
user_id: str,
role: str,
invited_by: str | None = None,
) -> dict[str, Any]:
"""Insert a new membership row.
Raises:
- :class:`MembershipValidationError` for unknown role values
- :class:`sqlalchemy.exc.IntegrityError` for:
- duplicate (workspace_id, user_id) composite PK collision
- second ``owner`` in the same workspace partial unique index
- invalid workspace_id / user_id / invited_by FK
"""
_validate_role(role)
row = WorkspaceMembershipRow(
workspace_id=workspace_id,
user_id=user_id,
role=role,
invited_by=invited_by,
joined_at=datetime.now(UTC),
)
async with self._sf() as session:
session.add(row)
await session.commit()
await session.refresh(row)
return self._row_to_dict(row)
async def remove(self, *, workspace_id: str, user_id: str) -> bool:
"""Delete a membership row; returns True if a row was deleted.
Owner removal is allowed at the repository layer auth router
(PR4) layers the "cannot remove last owner" rule on top.
"""
async with self._sf() as session:
result = await session.execute(
delete(WorkspaceMembershipRow).where(
WorkspaceMembershipRow.workspace_id == workspace_id,
WorkspaceMembershipRow.user_id == user_id,
)
)
await session.commit()
return (result.rowcount or 0) > 0
async def list_by_user(self, *, user_id: str) -> list[dict[str, Any]]:
"""Return all memberships for ``user_id``, ordered by joined_at desc.
No contextvar resolution here caller is responsible for passing
the correct user_id. Used by ``/auth/me`` to list workspaces a
user belongs to.
"""
async with self._sf() as session:
result = await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.user_id == user_id).order_by(WorkspaceMembershipRow.joined_at.desc()))
return [self._row_to_dict(r) for r in result.scalars()]
async def list_by_workspace(self, *, workspace_id: str) -> list[dict[str, Any]]:
"""Return all members of a workspace, ordered by joined_at asc."""
async with self._sf() as session:
result = await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.workspace_id == workspace_id).order_by(WorkspaceMembershipRow.joined_at.asc()))
return [self._row_to_dict(r) for r in result.scalars()]
async def get_role(self, *, workspace_id: str, user_id: str) -> str | None:
"""Return the role string, or None if user is not a member."""
async with self._sf() as session:
result = await session.execute(
select(WorkspaceMembershipRow.role).where(
WorkspaceMembershipRow.workspace_id == workspace_id,
WorkspaceMembershipRow.user_id == user_id,
)
)
return result.scalar_one_or_none()
async def change_role(
self,
*,
workspace_id: str,
user_id: str,
new_role: str,
) -> bool:
"""Update a member's role; returns True iff a row was updated.
Validates ``new_role`` against the allowed enum. Owner-transfer
flow needs to swap two rows atomically do that with a manual
transaction in the caller; this method is for non-owner changes.
"""
_validate_role(new_role)
async with self._sf() as session:
result = await session.execute(
update(WorkspaceMembershipRow)
.where(
WorkspaceMembershipRow.workspace_id == workspace_id,
WorkspaceMembershipRow.user_id == user_id,
)
.values(role=new_role)
)
await session.commit()
return (result.rowcount or 0) > 0
@@ -20,6 +20,7 @@ from __future__ import annotations
import asyncio import asyncio
import contextlib import contextlib
import logging import logging
import re
from collections.abc import AsyncIterator from collections.abc import AsyncIterator
from langgraph.types import Checkpointer from langgraph.types import Checkpointer
@@ -114,7 +115,14 @@ async def _async_checkpointer_from_database(db_config) -> AsyncIterator[Checkpoi
if not db_config.postgres_url: if not db_config.postgres_url:
raise ValueError("database.postgres_url is required for the postgres backend") raise ValueError("database.postgres_url is required for the postgres backend")
async with AsyncPostgresSaver.from_conn_string(db_config.postgres_url) as saver: # LangGraph's AsyncPostgresSaver wraps psycopg directly and expects a
# libpq-style conninfo (`postgresql://...`). DeerFlow's own SQLAlchemy
# engine uses the same `postgres_url` but needs the `+asyncpg` dialect
# prefix. Strip the dialect prefix here so the same env var/config
# value satisfies both paths.
lg_conn_str = re.sub(r"^postgresql\+\w+://", "postgresql://", db_config.postgres_url)
async with AsyncPostgresSaver.from_conn_string(lg_conn_str) as saver:
await saver.setup() await saver.setup()
yield saver yield saver
return return
@@ -17,6 +17,14 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from deerflow.persistence.models.run_event import RunEventRow from deerflow.persistence.models.run_event import RunEventRow
from deerflow.runtime.events.store.base import RunEventStore from deerflow.runtime.events.store.base import RunEventStore
from deerflow.runtime.user_context import AUTO, _AutoSentinel, get_current_user, resolve_user_id from deerflow.runtime.user_context import AUTO, _AutoSentinel, get_current_user, resolve_user_id
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
from deerflow.runtime.workspace_context import (
_AutoSentinel as _WorkspaceAutoSentinel,
)
from deerflow.runtime.workspace_context import (
get_current_workspace,
resolve_workspace_id,
)
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -86,6 +94,19 @@ class DbRunEventStore(RunEventStore):
user = get_current_user() user = get_current_user()
return str(user.id) if user is not None else None return str(user.id) if user is not None else None
@staticmethod
def _workspace_id_from_context() -> str | None:
"""Soft read of workspace_id from contextvar for write paths.
Mirrors :meth:`_user_id_from_context`. Returns ``None`` (no stamp)
when no workspace is in context typical for background worker
writes that fire outside an HTTP request. The DB column is
nullable through PR5 and becomes NOT NULL only after the alembic
0003 migration runs (verified by the backfill path).
"""
workspace = get_current_workspace()
return str(workspace.id) if workspace is not None else None
async def put(self, *, thread_id, run_id, event_type, category, content="", metadata=None, created_at=None): # noqa: D401 async def put(self, *, thread_id, run_id, event_type, category, content="", metadata=None, created_at=None): # noqa: D401
"""Write a single event — low-frequency path only. """Write a single event — low-frequency path only.
@@ -98,6 +119,7 @@ class DbRunEventStore(RunEventStore):
content, metadata = self._truncate_trace(category, content, metadata) content, metadata = self._truncate_trace(category, content, metadata)
db_content, metadata = self._content_to_db(content, metadata) db_content, metadata = self._content_to_db(content, metadata)
user_id = self._user_id_from_context() user_id = self._user_id_from_context()
workspace_id = self._workspace_id_from_context()
async with self._sf() as session: async with self._sf() as session:
async with session.begin(): async with session.begin():
# Use FOR UPDATE to serialize seq assignment within a thread. # Use FOR UPDATE to serialize seq assignment within a thread.
@@ -109,6 +131,7 @@ class DbRunEventStore(RunEventStore):
thread_id=thread_id, thread_id=thread_id,
run_id=run_id, run_id=run_id,
user_id=user_id, user_id=user_id,
workspace_id=workspace_id,
event_type=event_type, event_type=event_type,
category=category, category=category,
content=db_content, content=db_content,
@@ -123,6 +146,7 @@ class DbRunEventStore(RunEventStore):
if not events: if not events:
return [] return []
user_id = self._user_id_from_context() user_id = self._user_id_from_context()
workspace_id = self._workspace_id_from_context()
async with self._sf() as session: async with self._sf() as session:
async with session.begin(): async with session.begin():
# Get max seq for the thread (assume all events in batch belong to same thread). # Get max seq for the thread (assume all events in batch belong to same thread).
@@ -143,6 +167,7 @@ class DbRunEventStore(RunEventStore):
thread_id=e["thread_id"], thread_id=e["thread_id"],
run_id=e["run_id"], run_id=e["run_id"],
user_id=e.get("user_id", user_id), user_id=e.get("user_id", user_id),
workspace_id=e.get("workspace_id", workspace_id),
event_type=e["event_type"], event_type=e["event_type"],
category=category, category=category,
content=db_content, content=db_content,
@@ -162,9 +187,13 @@ class DbRunEventStore(RunEventStore):
before_seq=None, before_seq=None,
after_seq=None, after_seq=None,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages") resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.list_messages")
stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message") stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message")
if resolved_workspace_id is not None:
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(RunEventRow.user_id == resolved_user_id) stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
if before_seq is not None: if before_seq is not None:
@@ -194,9 +223,13 @@ class DbRunEventStore(RunEventStore):
event_types=None, event_types=None,
limit=500, limit=500,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_events") resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_events")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.list_events")
stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id) stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id)
if resolved_workspace_id is not None:
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(RunEventRow.user_id == resolved_user_id) stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
if event_types: if event_types:
@@ -215,13 +248,17 @@ class DbRunEventStore(RunEventStore):
before_seq=None, before_seq=None,
after_seq=None, after_seq=None,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages_by_run") resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages_by_run")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.list_messages_by_run")
stmt = select(RunEventRow).where( stmt = select(RunEventRow).where(
RunEventRow.thread_id == thread_id, RunEventRow.thread_id == thread_id,
RunEventRow.run_id == run_id, RunEventRow.run_id == run_id,
RunEventRow.category == "message", RunEventRow.category == "message",
) )
if resolved_workspace_id is not None:
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(RunEventRow.user_id == resolved_user_id) stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
if before_seq is not None: if before_seq is not None:
@@ -246,9 +283,13 @@ class DbRunEventStore(RunEventStore):
thread_id, thread_id,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.count_messages") resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.count_messages")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.count_messages")
stmt = select(func.count()).select_from(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message") stmt = select(func.count()).select_from(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message")
if resolved_workspace_id is not None:
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
stmt = stmt.where(RunEventRow.user_id == resolved_user_id) stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
async with self._sf() as session: async with self._sf() as session:
@@ -259,10 +300,14 @@ class DbRunEventStore(RunEventStore):
thread_id, thread_id,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_thread") resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_thread")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.delete_by_thread")
async with self._sf() as session: async with self._sf() as session:
count_conditions = [RunEventRow.thread_id == thread_id] count_conditions = [RunEventRow.thread_id == thread_id]
if resolved_workspace_id is not None:
count_conditions.append(RunEventRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
count_conditions.append(RunEventRow.user_id == resolved_user_id) count_conditions.append(RunEventRow.user_id == resolved_user_id)
count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions) count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions)
@@ -278,10 +323,14 @@ class DbRunEventStore(RunEventStore):
run_id, run_id,
*, *,
user_id: str | None | _AutoSentinel = AUTO, user_id: str | None | _AutoSentinel = AUTO,
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
): ):
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_run") resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_run")
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.delete_by_run")
async with self._sf() as session: async with self._sf() as session:
count_conditions = [RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id] count_conditions = [RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id]
if resolved_workspace_id is not None:
count_conditions.append(RunEventRow.workspace_id == resolved_workspace_id)
if resolved_user_id is not None: if resolved_user_id is not None:
count_conditions.append(RunEventRow.user_id == resolved_user_id) count_conditions.append(RunEventRow.user_id == resolved_user_id)
count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions) count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions)
@@ -19,6 +19,7 @@ from __future__ import annotations
import contextlib import contextlib
import logging import logging
import re
from collections.abc import AsyncIterator from collections.abc import AsyncIterator
from langgraph.store.base import BaseStore from langgraph.store.base import BaseStore
@@ -80,6 +81,57 @@ async def _async_store(config) -> AsyncIterator[BaseStore]:
raise ValueError(f"Unknown store backend type: {config.type!r}") raise ValueError(f"Unknown store backend type: {config.type!r}")
@contextlib.asynccontextmanager
async def _async_store_from_database(db_config) -> AsyncIterator[BaseStore]:
"""Async context manager that constructs a Store from a unified DatabaseConfig.
Mirrors :func:`deerflow.runtime.checkpointer.async_provider._async_checkpointer_from_database`
so the store and checkpointer share one ``database`` section.
"""
if db_config.backend == "memory":
from langgraph.store.memory import InMemoryStore
logger.info("Store: using InMemoryStore (in-process, not persistent)")
yield InMemoryStore()
return
if db_config.backend == "sqlite":
try:
from langgraph.store.sqlite.aio import AsyncSqliteStore
except ImportError as exc:
raise ImportError(SQLITE_STORE_INSTALL) from exc
conn_str = db_config.sqlite_path
ensure_sqlite_parent_dir(conn_str)
async with AsyncSqliteStore.from_conn_string(conn_str) as store:
await store.setup()
logger.info("Store: using AsyncSqliteStore (%s)", conn_str)
yield store
return
if db_config.backend == "postgres":
try:
from langgraph.store.postgres.aio import AsyncPostgresStore # type: ignore[import]
except ImportError as exc:
raise ImportError(POSTGRES_STORE_INSTALL) from exc
if not db_config.postgres_url:
raise ValueError(POSTGRES_CONN_REQUIRED)
# LangGraph's AsyncPostgresStore wraps psycopg and expects a libpq-style
# conninfo (`postgresql://...`). DeerFlow's SQLAlchemy engine uses the
# same URL with the `+asyncpg` dialect prefix — strip it so one
# DATABASE_URL satisfies both paths (same as the checkpointer factory).
lg_conn_str = re.sub(r"^postgresql\+\w+://", "postgresql://", db_config.postgres_url)
async with AsyncPostgresStore.from_conn_string(lg_conn_str) as store:
await store.setup()
logger.info("Store: using AsyncPostgresStore")
yield store
return
raise ValueError(f"Unknown database backend: {db_config.backend!r}")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Public async context manager # Public async context manager
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -97,18 +149,29 @@ async def make_store(app_config: AppConfig | None = None) -> AsyncIterator[BaseS
async with make_store(app_config) as store: async with make_store(app_config) as store:
app.state.store = store app.state.store = store
Yields an :class:`~langgraph.store.memory.InMemoryStore` when no Priority (mirrors the checkpointer factory):
``checkpointer`` section is configured (emits a WARNING in that case). 1. Legacy ``checkpointer:`` config section (backward compatible)
2. Unified ``database:`` config section
3. Default InMemoryStore (emits a WARNING)
""" """
if app_config is None: if app_config is None:
app_config = get_app_config() app_config = get_app_config()
if app_config.checkpointer is None: # Legacy: standalone checkpointer config takes precedence
from langgraph.store.memory import InMemoryStore if app_config.checkpointer is not None:
async with _async_store(app_config.checkpointer) as store:
logger.warning("No 'checkpointer' section in config.yaml — using InMemoryStore for the store. Thread list will be lost on server restart. Configure a sqlite or postgres backend for persistence.") yield store
yield InMemoryStore()
return return
async with _async_store(app_config.checkpointer) as store: # Unified database config
yield store db_config = getattr(app_config, "database", None)
if db_config is not None and db_config.backend != "memory":
async with _async_store_from_database(db_config) as store:
yield store
return
# Default: in-memory
from langgraph.store.memory import InMemoryStore
logger.warning("No persistent store backend configured (no 'checkpointer' or 'database' section) — using InMemoryStore. Cross-thread store data will be lost on server restart. Configure a sqlite or postgres backend for persistence.")
yield InMemoryStore()
@@ -42,8 +42,15 @@ from typing import Final, Protocol, runtime_checkable
class CurrentUser(Protocol): class CurrentUser(Protocol):
"""Structural type for the current authenticated user. """Structural type for the current authenticated user.
Any object with an ``.id: str`` attribute satisfies this protocol. Requires only ``.id: str`` the persistence layer reads nothing else,
Concrete implementations live in ``app.gateway.auth.models.User``. and keeping the contract minimal lets any ``.id``-bearing object (incl.
test fixtures) satisfy it. A principal MAY additionally carry
``.is_service_account: bool`` to distinguish a headless service account
(API key) from a human; concrete carriers are
``app.gateway.auth.models.User`` (False) and
``app.gateway.auth.api_key_backend.ServicePrincipal`` (True). Since that
attribute is NOT part of this structural contract, app-layer readers
must access it defensively: ``getattr(user, "is_service_account", False)``.
""" """
id: str id: str
@@ -0,0 +1,182 @@
"""Request-scoped workspace context for multi-tenant authorization.
Sibling of :mod:`deerflow.runtime.user_context`. Holds a
:class:`~contextvars.ContextVar` that the gateway's auth middleware sets
after JWT verification (PR4 will wire this up). Repository methods read
the contextvar via a sentinel default parameter, letting routers stay
free of ``workspace_id`` boilerplate.
Three-state semantics for the repository ``workspace_id`` parameter:
- ``AUTO`` (sentinel, default): read from contextvar; raise
:class:`RuntimeError` if unset.
- Explicit ``str``: use the provided value, overriding contextvar.
- Explicit ``None``: no WHERE clause used only by migration scripts
and admin CLIs that intentionally bypass workspace isolation.
Concept boundary
----------------
A workspace is the multi-tenant scope: a single-user free account is
its own 1-person workspace; a team subscription is a multi-member
workspace. The user_id contextvar narrows further to "which member of
the workspace", letting some operations be member-scoped while others
(skill install, billing, etc.) are workspace-scoped.
Dependency direction
--------------------
``persistence`` (lower layer) reads from this module; ``gateway.auth``
(higher layer) writes to it. ``CurrentWorkspace`` is defined here as a
:class:`typing.Protocol` so that ``persistence`` never needs to import
the concrete ``Workspace`` row class from ``deerflow.persistence.workspace``.
Any object with ``.id: str`` and ``.role: str`` attributes structurally
satisfies the protocol.
Asyncio semantics
-----------------
Identical to ``user_context``: ``ContextVar`` is task-local under asyncio.
``asyncio.create_task`` inherits the parent task's workspace context;
threading.Timer does **not** (callers spawning timers must capture
``get_effective_workspace_id()`` at enqueue time, the same way
:mod:`deerflow.agents.memory.queue` captures ``user_id``).
"""
from __future__ import annotations
from contextvars import ContextVar, Token
from typing import Final, Protocol, runtime_checkable
@runtime_checkable
class CurrentWorkspace(Protocol):
"""Structural type for the current active workspace.
Any object with ``.id: str`` and ``.role: str`` attributes satisfies
this protocol. Concrete implementations live in
``app.gateway.auth.models`` (PR4 will add them).
``role`` is the *caller's* role within this workspace
(``'owner'`` / ``'admin'`` / ``'member'``), not the workspace's
own metadata. Stage 0 sees ``'owner'`` only Stage 2 RBAC rollout
opens up the other values.
"""
id: str
role: str
_current_workspace: Final[ContextVar[CurrentWorkspace | None]] = ContextVar("deerflow_current_workspace", default=None)
def set_current_workspace(workspace: CurrentWorkspace) -> Token[CurrentWorkspace | None]:
"""Set the current workspace for this async task.
Returns a reset token that should be passed to
:func:`reset_current_workspace` in a ``finally`` block to restore
the previous context.
"""
return _current_workspace.set(workspace)
def reset_current_workspace(token: Token[CurrentWorkspace | None]) -> None:
"""Restore the context to the state captured by ``token``."""
_current_workspace.reset(token)
def get_current_workspace() -> CurrentWorkspace | None:
"""Return the current workspace, or ``None`` if unset.
Safe to call in any context. Used by code paths that can proceed
without a workspace (migration scripts, public endpoints).
"""
return _current_workspace.get()
def require_current_workspace() -> CurrentWorkspace:
"""Return the current workspace, or raise :class:`RuntimeError`.
Used by repository code that must not be called outside a
request-authenticated context. The error message is phrased so
that a caller debugging a stack trace can locate the offending
code path.
"""
workspace = _current_workspace.get()
if workspace is None:
raise RuntimeError("repository accessed without workspace context")
return workspace
# ---------------------------------------------------------------------------
# Effective workspace_id helpers (filesystem isolation)
# ---------------------------------------------------------------------------
DEFAULT_WORKSPACE_ID: Final[str] = "default"
def get_effective_workspace_id() -> str:
"""Return the current workspace id as a string, or DEFAULT_WORKSPACE_ID if unset.
Unlike :func:`require_current_workspace` this never raises it is
designed for filesystem-path resolution where a valid workspace
bucket is always needed (PR6 will switch
``Paths.thread_dir(workspace_id=...)`` to read from here).
"""
workspace = _current_workspace.get()
if workspace is None:
return DEFAULT_WORKSPACE_ID
return str(workspace.id)
# ---------------------------------------------------------------------------
# Sentinel-based workspace_id resolution
# ---------------------------------------------------------------------------
#
# Repository methods accept a ``workspace_id`` keyword-only argument that
# defaults to ``AUTO``. The three possible values drive distinct
# behaviours; see the docstring on :func:`resolve_workspace_id`.
class _AutoSentinel:
"""Singleton marker meaning 'resolve workspace_id from contextvar'."""
_instance: _AutoSentinel | None = None
def __new__(cls) -> _AutoSentinel:
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance
def __repr__(self) -> str:
return "<AUTO>"
AUTO: Final[_AutoSentinel] = _AutoSentinel()
def resolve_workspace_id(
value: str | None | _AutoSentinel,
*,
method_name: str = "repository method",
) -> str | None:
"""Resolve the workspace_id parameter passed to a repository method.
Three-state semantics:
- :data:`AUTO` (default): read from contextvar; raise
:class:`RuntimeError` if no workspace is in context. This is the
common case for request-scoped calls.
- Explicit ``str``: use the provided id verbatim, overriding any
contextvar value. Useful for tests and admin-override flows.
- Explicit ``None``: no filter the repository should skip the
workspace_id WHERE clause entirely. Reserved for migration scripts
and CLI tools that intentionally bypass workspace isolation.
"""
if isinstance(value, _AutoSentinel):
workspace = _current_workspace.get()
if workspace is None:
raise RuntimeError(
f"{method_name} called with workspace_id=AUTO but no workspace context is set; pass an explicit workspace_id, set the contextvar via auth middleware, or opt out with workspace_id=None for migration/CLI paths."
)
# Coerce to ``str`` at the boundary; persistence stores
# ``workspace_id`` as ``String(36)`` (UUID v4 text).
return str(workspace.id)
return value
+4
View File
@@ -46,6 +46,10 @@ postgres = [
"psycopg[binary]>=3.3.3", "psycopg[binary]>=3.3.3",
"psycopg-pool>=3.3.0", "psycopg-pool>=3.3.0",
] ]
postgres-test = [
"deerflow-harness[postgres]",
"testcontainers[postgres]>=4.0",
]
pymupdf = ["pymupdf4llm>=0.0.17"] pymupdf = ["pymupdf4llm>=0.0.17"]
[build-system] [build-system]
+4 -1
View File
@@ -25,6 +25,7 @@ dependencies = [
[project.optional-dependencies] [project.optional-dependencies]
postgres = ["deerflow-harness[postgres]"] postgres = ["deerflow-harness[postgres]"]
postgres-test = ["deerflow-harness[postgres-test]"]
[dependency-groups] [dependency-groups]
dev = [ dev = [
@@ -36,7 +37,9 @@ dev = [
[tool.pytest.ini_options] [tool.pytest.ini_options]
markers = [ markers = [
"no_auto_user: disable the conftest autouse contextvar fixture for this test", "no_auto_user: disable the conftest autouse user contextvar fixture for this test",
"no_auto_workspace: disable the conftest autouse workspace contextvar fixture for this test",
"postgres: requires a Postgres testcontainer (Docker daemon); skipped if unavailable",
] ]
[tool.uv] [tool.uv]
+306
View File
@@ -0,0 +1,306 @@
"""Backfill ``workspace_id`` on PR5 business tables.
Three-step backfill (each idempotent re-running picks up where a crash
left off because every step's WHERE clause filters already-processed rows):
1. For each user without ``default_workspace_id``: create a personal
workspace + ``owner`` membership + write back the user's
``default_workspace_id``.
2. ``UPDATE`` each of ``threads_meta`` / ``runs`` / ``feedback`` /
``run_events`` setting ``workspace_id`` from the row's owner's
``users.default_workspace_id``. Only touches rows where
``workspace_id IS NULL`` and ``user_id IS NOT NULL``.
3. Any rows still with ``workspace_id IS NULL`` (truly orphan they had
``user_id = NULL`` to begin with) are assigned the *legacy* workspace
UUID ``00000000-0000-0000-0000-000000000000``. The script creates
that workspace on demand, owned by the platform admin.
Usage::
PYTHONPATH=. python scripts/backfill_workspace_id.py [--dry-run]
T5.4 only ships the skeleton the three step bodies are filled in by
T5.5 / T5.6 / T5.7 along with their per-step tests.
"""
from __future__ import annotations
import argparse
import asyncio
import logging
from typing import Any
from sqlalchemy import func, select, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from app.gateway.auth.workspace_slug import auto_slug_from_email, next_available_slug
from deerflow.persistence.base import Base
from deerflow.persistence.feedback.model import FeedbackRow
from deerflow.persistence.models.run_event import RunEventRow
from deerflow.persistence.run.model import RunRow
from deerflow.persistence.thread_meta.model import ThreadMetaRow
from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace import WorkspaceRepository
from deerflow.persistence.workspace.sql import SLUG_BLACKLIST
from deerflow.persistence.workspace_membership import WorkspaceMembershipRepository
logger = logging.getLogger(__name__)
# The legacy workspace anchor. Stage 0 LOCK'd UUID — chosen as the standard
# nil UUID so SQL log scans can spot it instantly.
LEGACY_WORKSPACE_ID = "00000000-0000-0000-0000-000000000000"
LEGACY_WORKSPACE_SLUG = "legacy"
LEGACY_WORKSPACE_NAME = "Legacy Workspace"
# The four business tables that gained ``workspace_id`` in alembic 0002.
_BUSINESS_TABLES: tuple[str, ...] = ("threads_meta", "runs", "feedback", "run_events")
# Map table-name to ORM class so we can build a portable correlated UPDATE
# using SQLAlchemy expression language (SQLite < 3.33 lacks UPDATE-FROM
# but supports correlated subqueries on every version we ship).
_TABLE_MODELS: dict[str, type[Base]] = {
"threads_meta": ThreadMetaRow,
"runs": RunRow,
"feedback": FeedbackRow,
"run_events": RunEventRow,
}
async def _step1_create_workspaces_for_users(
session_factory: async_sessionmaker[AsyncSession],
*,
dry_run: bool,
) -> int:
"""Create one workspace + owner membership for each user missing default_workspace_id.
Returns the count of users a workspace was created for. Idempotent
users that already have ``default_workspace_id`` are skipped so a
crashed run can resume safely.
"""
async with session_factory() as session:
result = await session.execute(select(UserRow.id, UserRow.email).where(UserRow.default_workspace_id.is_(None)))
candidates = [(row.id, row.email) for row in result]
if not candidates:
return 0
ws_repo = WorkspaceRepository(session_factory)
m_repo = WorkspaceMembershipRepository(session_factory)
created = 0
for user_id, email in candidates:
base_slug = auto_slug_from_email(email)
async def slug_exists(s: str) -> bool:
if s in SLUG_BLACKLIST:
return True
return (await ws_repo.get_by_slug(s)) is not None
unique_slug = await next_available_slug(base_slug, exists_check=slug_exists)
if dry_run:
logger.info("WOULD create workspace for user=%s email=%s slug=%s", user_id, email, unique_slug)
created += 1
continue
display_local = email.split("@", 1)[0]
workspace = await ws_repo.create(
name=f"{display_local}'s Workspace"[:64],
slug=unique_slug,
owner_id=user_id,
)
await m_repo.add(workspace_id=workspace["id"], user_id=user_id, role="owner")
async with session_factory() as session:
await session.execute(update(UserRow).where(UserRow.id == user_id).values(default_workspace_id=workspace["id"]))
await session.commit()
created += 1
logger.info("Created workspace %s (slug=%s) for user=%s", workspace["id"], unique_slug, user_id)
return created
async def _step2_update_table_from_users(
session_factory: async_sessionmaker[AsyncSession],
table: str,
*,
dry_run: bool,
) -> int:
"""UPDATE *table* setting workspace_id from owner's users.default_workspace_id.
Uses a correlated subquery (portable across SQLite + Postgres). Filters
``workspace_id IS NULL AND user_id IS NOT NULL`` so already-set rows
and truly orphan rows are skipped (Step 3 handles the latter).
Returns the number of rows updated (or that *would* be updated under
``dry_run``).
"""
model = _TABLE_MODELS[table]
workspace_col = model.workspace_id
user_col = model.user_id
# Subquery: pull the user's default_workspace_id for each row.
correlated_default = select(UserRow.default_workspace_id).where(UserRow.id == user_col).scalar_subquery()
if dry_run:
# Count rows whose owner has a default_workspace_id assigned — only
# those would get touched by the actual UPDATE.
count_stmt = select(func.count()).select_from(model).join(UserRow, UserRow.id == user_col).where(workspace_col.is_(None), user_col.is_not(None), UserRow.default_workspace_id.is_not(None))
async with session_factory() as session:
count = (await session.execute(count_stmt)).scalar_one() or 0
logger.info("WOULD update %d rows in %s from users.default_workspace_id", count, table)
return int(count)
stmt = update(model).where(workspace_col.is_(None), user_col.is_not(None)).values(workspace_id=correlated_default)
async with session_factory() as session:
result = await session.execute(stmt)
await session.commit()
rowcount = result.rowcount or 0
logger.info("Updated %d rows in %s from users.default_workspace_id", rowcount, table)
return int(rowcount)
async def _ensure_legacy_workspace(
session_factory: async_sessionmaker[AsyncSession],
*,
dry_run: bool,
) -> bool:
"""Create the ``legacy_workspace`` anchor row idempotently.
The anchor is needed before Step 3 can point orphan rows at it. We
pick the platform admin (``system_role='admin'``) as owner; if no
admin exists yet we fall back to the oldest user. If the database
has no users at all we refuse to continue running this script on
an unbootstrapped DB would create a workspace with no owner and the
FK to ``users`` would fail anyway.
Returns True if the workspace was just created (or would be, under
``dry_run``). False if it already existed.
"""
from deerflow.persistence.workspace.model import WorkspaceRow
async with session_factory() as session:
existing = await session.get(WorkspaceRow, LEGACY_WORKSPACE_ID)
if existing is not None:
return False
async with session_factory() as session:
admin_id = (await session.execute(select(UserRow.id).where(UserRow.system_role == "admin").order_by(UserRow.created_at).limit(1))).scalar_one_or_none()
if admin_id is None:
admin_id = (await session.execute(select(UserRow.id).order_by(UserRow.created_at).limit(1))).scalar_one_or_none()
if admin_id is None:
raise RuntimeError(
"Cannot create legacy_workspace: no users exist. Bootstrap an admin via /auth/initialize before running backfill.",
)
if dry_run:
logger.info("WOULD create legacy_workspace (id=%s) owned by user=%s", LEGACY_WORKSPACE_ID, admin_id)
return True
ws_repo = WorkspaceRepository(session_factory)
await ws_repo.create(
workspace_id=LEGACY_WORKSPACE_ID,
name=LEGACY_WORKSPACE_NAME,
slug=LEGACY_WORKSPACE_SLUG,
owner_id=admin_id,
)
m_repo = WorkspaceMembershipRepository(session_factory)
await m_repo.add(workspace_id=LEGACY_WORKSPACE_ID, user_id=admin_id, role="owner")
logger.info("Created legacy_workspace (id=%s) owned by user=%s", LEGACY_WORKSPACE_ID, admin_id)
return True
async def _step3_assign_legacy_workspace(
session_factory: async_sessionmaker[AsyncSession],
table: str,
*,
dry_run: bool,
) -> int:
"""Assign LEGACY_WORKSPACE_ID to *table* rows still missing workspace_id.
Callers should ensure :func:`_ensure_legacy_workspace` has run first;
the orchestrator does this between Step 2 and Step 3. Orphan rows are
rows whose ``user_id`` was already NULL (or pointed at a deleted user)
so Step 2's correlated subquery left them untouched.
"""
model = _TABLE_MODELS[table]
workspace_col = model.workspace_id
if dry_run:
count_stmt = select(func.count()).select_from(model).where(workspace_col.is_(None))
async with session_factory() as session:
count = (await session.execute(count_stmt)).scalar_one() or 0
logger.info("WOULD assign %d orphan row(s) in %s to legacy_workspace", count, table)
return int(count)
stmt = update(model).where(workspace_col.is_(None)).values(workspace_id=LEGACY_WORKSPACE_ID)
async with session_factory() as session:
result = await session.execute(stmt)
await session.commit()
rowcount = result.rowcount or 0
logger.info("Assigned %d orphan row(s) in %s to legacy_workspace", rowcount, table)
return int(rowcount)
async def backfill(
session_factory: async_sessionmaker[AsyncSession],
*,
dry_run: bool = False,
) -> dict[str, Any]:
"""Run all three backfill steps; return a per-step row-count report.
Order matters: Step 1 must populate ``users.default_workspace_id``
before Step 2 can correlate business rows back through ``users``.
"""
report: dict[str, Any] = {"dry_run": dry_run}
report["users_workspaces_created"] = await _step1_create_workspaces_for_users(session_factory, dry_run=dry_run)
for table in _BUSINESS_TABLES:
report[f"{table}_from_users"] = await _step2_update_table_from_users(session_factory, table, dry_run=dry_run)
report["legacy_workspace_created"] = await _ensure_legacy_workspace(session_factory, dry_run=dry_run)
for table in _BUSINESS_TABLES:
report[f"{table}_legacy"] = await _step3_assign_legacy_workspace(session_factory, table, dry_run=dry_run)
return report
def _build_session_factory_from_config() -> async_sessionmaker[AsyncSession]:
"""Build an async session factory from the active config.yaml.
Avoids importing on module load so unit tests can stub
``session_factory`` directly without booting the full config pipeline.
"""
from deerflow.config import get_app_config
from deerflow.persistence.engine import get_session_factory, init_engine_from_config
asyncio.run(init_engine_from_config(get_app_config().database))
sf = get_session_factory()
if sf is None:
raise RuntimeError(
"database.backend=memory: nothing to backfill. Switch config.yaml to sqlite/postgres first.",
)
return sf
def main() -> None:
parser = argparse.ArgumentParser(description="Backfill workspace_id on Stage 0 business tables (idempotent).")
parser.add_argument(
"--dry-run",
action="store_true",
help="Print the rows each step would touch without writing.",
)
args = parser.parse_args()
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
sf = _build_session_factory_from_config()
report = asyncio.run(backfill(sf, dry_run=args.dry_run))
logger.info("Backfill report (dry_run=%s):", args.dry_run)
for key, value in report.items():
if key == "dry_run":
continue
logger.info(" %s: %s", key, value)
if __name__ == "__main__":
main()
@@ -0,0 +1,212 @@
"""One-time migration: lift per-user thread / memory / agent dirs into per-workspace layout.
PR4 introduced ``{base_dir}/users/{user_id}/...`` for per-user isolation. PR6
adds the multi-tenant top dimension: ``{base_dir}/workspaces/{wid}/...``,
with per-user state nested under each workspace
(``{base_dir}/workspaces/{wid}/users/{uid}/memory.json`` etc.). This script
walks the old PR4 layout, looks up each user's ``default_workspace_id``
from the ``users`` table, and rewrites the path.
Usage:
PYTHONPATH=. python scripts/migrate_paths_to_workspace.py [--dry-run] [--default-workspace WID]
The script is idempotent re-running it after a successful migration is a no-op.
A ``--dry-run`` invocation must not write anything; it only logs what would happen.
Mapping rules:
- ``{base_dir}/users/{uid}/threads/{tid}/`` -> ``{base_dir}/workspaces/{wid}/threads/{tid}/``
- ``{base_dir}/users/{uid}/memory.json`` -> ``{base_dir}/workspaces/{wid}/users/{uid}/memory.json``
- ``{base_dir}/users/{uid}/agents/{name}/`` -> ``{base_dir}/workspaces/{wid}/users/{uid}/agents/{name}/``
``{wid}`` is read from ``users.default_workspace_id``. Users without one
fall through to ``--default-workspace`` (defaults to the special bucket
``"legacy_workspace"`` so they can be triaged manually). Pre-existing
destinations are preserved; legacy copies are moved to
``{base_dir}/migration-conflicts/<...>`` for human review.
"""
from __future__ import annotations
import argparse
import logging
import shutil
import sqlite3
from pathlib import Path
from deerflow.config.paths import Paths, get_paths
logger = logging.getLogger(__name__)
LEGACY_WORKSPACE_FALLBACK = "legacy_workspace"
def _load_user_workspaces(paths: Paths) -> dict[str, str | None]:
"""Read ``user_id -> default_workspace_id`` from the local sqlite DB.
Returns an empty dict when the database does not exist (fresh install
/ Postgres-only deployments). The Postgres path is out of scope for
this script the operator should run it with ``--default-workspace``
set to the target workspace and skip the DB lookup.
"""
db_path = paths.base_dir / "deer-flow.db"
if not db_path.exists():
logger.info("No sqlite database at %s — every user will use the fallback workspace.", db_path)
return {}
conn = sqlite3.connect(str(db_path))
try:
try:
cursor = conn.execute("SELECT id, default_workspace_id FROM users")
except sqlite3.OperationalError as e:
logger.warning("Failed to query users.default_workspace_id: %s", e)
return {}
return {row[0]: row[1] for row in cursor.fetchall()}
finally:
conn.close()
def _resolve_workspace(user_id: str, user_workspaces: dict[str, str | None], fallback: str) -> str:
wid = user_workspaces.get(user_id)
return wid or fallback
def _move(src: Path, dest: Path, conflict_root: Path, *, dry_run: bool, label: str) -> str:
"""Move ``src`` to ``dest``; on conflict, divert legacy under ``conflict_root``.
Returns a short string describing what happened (for the report).
"""
if dest.exists():
conflict_dest = conflict_root / label / src.name
if not dry_run:
conflict_dest.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(src), str(conflict_dest))
logger.warning("Conflict for %s/%s: legacy copy diverted to %s", label, src.name, conflict_dest)
return f"conflict -> {conflict_dest}"
if not dry_run:
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(src), str(dest))
return f"moved -> {dest}"
def migrate_user_tree(
paths: Paths,
user_id: str,
workspace_id: str,
*,
dry_run: bool,
) -> list[dict]:
"""Lift one user's tree under a workspace. Returns per-asset report rows."""
report: list[dict] = []
legacy_user_dir = paths.base_dir / "users" / user_id
if not legacy_user_dir.exists():
return report
conflict_root = paths.base_dir / "migration-conflicts" / "workspace-migration"
# 1. threads/{tid}/ -> workspaces/{wid}/threads/{tid}/
legacy_threads = legacy_user_dir / "threads"
if legacy_threads.exists():
for thread_dir in sorted(legacy_threads.iterdir()):
if not thread_dir.is_dir():
continue
dest = paths.thread_dir(thread_dir.name, workspace_id=workspace_id)
action = _move(thread_dir, dest, conflict_root, dry_run=dry_run, label=f"threads/{workspace_id}")
report.append({"asset": "thread", "user_id": user_id, "workspace_id": workspace_id, "name": thread_dir.name, "action": action})
if not dry_run and legacy_threads.exists() and not any(legacy_threads.iterdir()):
legacy_threads.rmdir()
# 2. memory.json -> workspaces/{wid}/users/{uid}/memory.json
legacy_mem = legacy_user_dir / "memory.json"
if legacy_mem.exists():
dest = paths.user_memory_file(user_id, workspace_id=workspace_id)
action = _move(legacy_mem, dest, conflict_root, dry_run=dry_run, label=f"users/{user_id}/memory")
report.append({"asset": "memory", "user_id": user_id, "workspace_id": workspace_id, "name": "memory.json", "action": action})
# 3. agents/{name}/ -> workspaces/{wid}/users/{uid}/agents/{name}/
legacy_agents = legacy_user_dir / "agents"
if legacy_agents.exists():
for agent_dir in sorted(legacy_agents.iterdir()):
if not agent_dir.is_dir():
continue
dest = paths.user_agent_dir(user_id, agent_dir.name, workspace_id=workspace_id)
action = _move(agent_dir, dest, conflict_root, dry_run=dry_run, label=f"users/{user_id}/agents")
report.append({"asset": "agent", "user_id": user_id, "workspace_id": workspace_id, "name": agent_dir.name, "action": action})
if not dry_run and legacy_agents.exists() and not any(legacy_agents.iterdir()):
legacy_agents.rmdir()
# 4. If the user dir is now empty, remove it so lifespan warnings clear.
if not dry_run and legacy_user_dir.exists() and not any(legacy_user_dir.iterdir()):
legacy_user_dir.rmdir()
return report
def migrate(
paths: Paths,
*,
user_workspaces: dict[str, str | None],
fallback_workspace: str,
dry_run: bool,
) -> list[dict]:
"""Top-level entry point: iterate every user under ``{base_dir}/users``."""
legacy_users = paths.base_dir / "users"
if not legacy_users.exists():
logger.info("No legacy ``users/`` directory under %s — nothing to migrate.", paths.base_dir)
return []
report: list[dict] = []
for user_dir in sorted(legacy_users.iterdir()):
if not user_dir.is_dir():
continue
user_id = user_dir.name
workspace_id = _resolve_workspace(user_id, user_workspaces, fallback_workspace)
logger.info("Migrating user %s -> workspace %s", user_id, workspace_id)
report.extend(migrate_user_tree(paths, user_id, workspace_id, dry_run=dry_run))
if not dry_run and legacy_users.exists() and not any(legacy_users.iterdir()):
legacy_users.rmdir()
return report
def main() -> None:
parser = argparse.ArgumentParser(description="Lift legacy per-user paths into per-workspace layout (PR6).")
parser.add_argument("--dry-run", action="store_true", help="Log actions without making changes.")
parser.add_argument(
"--default-workspace",
default=LEGACY_WORKSPACE_FALLBACK,
metavar="WID",
help=(f"Workspace id to use for users without a ``default_workspace_id`` in the DB. Defaults to ``{LEGACY_WORKSPACE_FALLBACK}`` (matches the orphan-row bucket used by PR5 backfill)."),
)
args = parser.parse_args()
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
paths = get_paths()
logger.info("Base directory: %s", paths.base_dir)
logger.info("Dry run: %s", args.dry_run)
logger.info("Fallback workspace: %s", args.default_workspace)
user_workspaces = _load_user_workspaces(paths)
logger.info("Loaded %d user->workspace mappings from DB", len(user_workspaces))
report = migrate(
paths,
user_workspaces=user_workspaces,
fallback_workspace=args.default_workspace,
dry_run=args.dry_run,
)
if not report:
logger.info("Nothing to migrate.")
return
logger.info("Migration report (%d entries):", len(report))
for entry in report:
logger.info(" asset=%s user=%s workspace=%s name=%s action=%s", entry["asset"], entry["user_id"], entry["workspace_id"], entry["name"], entry["action"])
if __name__ == "__main__":
main()
+436
View File
@@ -0,0 +1,436 @@
#!/usr/bin/env bash
# verify_stage0.sh — systematic verification of the Stage 0 multi-tenant rollout.
#
# Layers (each can be run individually; defaults run all that are applicable):
# static — full pytest suite + lint + boundary scan (no external deps)
# paths — local filesystem layout: legacy users/ should be empty after migration
# rds — schema + index + alembic state on the remote Postgres (needs DATABASE_URL)
# runtime — gateway health probes (needs `make dev` running)
# e2e — register 2 users via curl, each creates a thread, cross-access must 404
#
# Usage:
# ./scripts/verify_stage0.sh # run everything that has prerequisites
# ./scripts/verify_stage0.sh static rds # only those two
# DATABASE_URL=postgres://... ./scripts/verify_stage0.sh
# GATEWAY_URL=http://localhost:8001 ./scripts/verify_stage0.sh runtime e2e
#
# Exit code: 0 if every executed assertion passes, 1 if any fail.
set -uo pipefail
# ── colours ──────────────────────────────────────────────────────────────
if [ -t 1 ]; then
C_RED=$'\033[0;31m'; C_GREEN=$'\033[0;32m'; C_YELLOW=$'\033[0;33m'
C_BLUE=$'\033[0;34m'; C_BOLD=$'\033[1m'; C_DIM=$'\033[2m'; C_RESET=$'\033[0m'
else
C_RED=''; C_GREEN=''; C_YELLOW=''; C_BLUE=''; C_BOLD=''; C_DIM=''; C_RESET=''
fi
# ── counters ─────────────────────────────────────────────────────────────
PASS_COUNT=0
FAIL_COUNT=0
WARN_COUNT=0
FAILED_STEPS=()
SKIPPED_PHASES=()
ok() { printf ' %s✓%s %s\n' "$C_GREEN" "$C_RESET" "$1"; PASS_COUNT=$((PASS_COUNT+1)); }
fail() { printf ' %s✗%s %s\n' "$C_RED" "$C_RESET" "$1"; FAIL_COUNT=$((FAIL_COUNT+1)); FAILED_STEPS+=("$1"); }
warn() { printf ' %s!%s %s\n' "$C_YELLOW" "$C_RESET" "$1"; WARN_COUNT=$((WARN_COUNT+1)); }
info() { printf ' %s·%s %s\n' "$C_DIM" "$C_RESET" "$1"; }
phase() { printf '\n%s== %s ==%s\n' "$C_BLUE$C_BOLD" "$1" "$C_RESET"; }
# Run-cmd helpers — capture both streams for grep but return original exit code.
run() {
local label="$1"; shift
local out
out=$("$@" 2>&1)
local rc=$?
if [ "$rc" -eq 0 ]; then
ok "$label"
printf '%s' "$out"
return 0
fi
fail "$label (exit $rc)"
printf '%s%s%s\n' "$C_DIM" "$out" "$C_RESET" >&2
return "$rc"
}
# ── env ──────────────────────────────────────────────────────────────────
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
BACKEND_DIR="$REPO_ROOT/backend"
GATEWAY_URL="${GATEWAY_URL:-http://localhost:8001}"
DEER_FLOW_HOME="${DEER_FLOW_HOME:-$REPO_ROOT/.deer-flow}"
# ── phase: static ────────────────────────────────────────────────────────
phase_static() {
phase "STATIC — unit + boundary + full pytest"
cd "$BACKEND_DIR"
# boundary scans first — fast and high-signal
info "running boundary scans (harness + workspace)"
if PYTHONPATH=. uv run pytest -q \
tests/test_harness_boundary.py \
tests/test_workspace_boundary.py \
tests/test_workspace_boundary_self.py >/tmp/verify_boundary.log 2>&1; then
local boundary_passed
boundary_passed=$(grep -Eo '[0-9]+ passed' /tmp/verify_boundary.log | head -1)
ok "boundary scans green ($boundary_passed)"
else
fail "boundary scans red — see /tmp/verify_boundary.log"
fi
info "running full pytest (this takes ~90-130s)"
local pytest_log=/tmp/verify_full_pytest.log
PYTHONPATH=. uv run pytest -q >"$pytest_log" 2>&1
local rc=$?
local last_line
last_line=$(tail -1 "$pytest_log")
info "result: $last_line"
# Parse "<X> passed, <Y> failed, <Z> skipped"
local passed_n failed_n
passed_n=$(printf '%s' "$last_line" | grep -Eo '[0-9]+ passed' | grep -Eo '[0-9]+' | head -1 || echo 0)
failed_n=$(printf '%s' "$last_line" | grep -Eo '[0-9]+ failed' | grep -Eo '[0-9]+' | head -1 || echo 0)
if [ "${passed_n:-0}" -ge 3250 ]; then
ok "pytest pass count $passed_n ≥ 3250 (PR8 baseline)"
else
fail "pytest pass count $passed_n < 3250 — regression suspected"
fi
if [ "${failed_n:-0}" -le 18 ]; then
ok "pytest fail count $failed_n ≤ 18 (Stage 0 known-flake ceiling)"
[ "${failed_n:-0}" -gt 0 ] && warn "non-zero failures expected to be in the 18 caplog flake set; cross-check tail of /tmp/verify_full_pytest.log"
else
fail "pytest fail count $failed_n > 18 — new failures introduced beyond known caplog flake set"
fi
info "running ruff lint"
if make lint >/tmp/verify_lint.log 2>&1; then
ok "ruff lint clean"
else
fail "ruff lint dirty — see /tmp/verify_lint.log"
fi
}
# ── phase: paths ─────────────────────────────────────────────────────────
phase_paths() {
phase "PATHS — legacy users/ migration state"
if [ ! -d "$DEER_FLOW_HOME" ]; then
warn "DEER_FLOW_HOME ($DEER_FLOW_HOME) does not exist — fresh install, nothing to migrate"
return
fi
info "DEER_FLOW_HOME = $DEER_FLOW_HOME"
local legacy_dir="$DEER_FLOW_HOME/users"
if [ ! -d "$legacy_dir" ]; then
ok "no legacy users/ directory present (PR6 migration not needed or already done)"
else
local count
count=$(find "$legacy_dir" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | wc -l | tr -d ' ')
if [ "$count" -eq 0 ]; then
ok "legacy users/ is empty (migration complete or no prior data)"
else
warn "legacy users/ still has $count user dir(s) — run \`make migrate-paths\` (after \`make migrate-paths DRY_RUN=1\` to preview)"
fi
fi
local workspace_dir="$DEER_FLOW_HOME/workspaces"
if [ -d "$workspace_dir" ]; then
local wcount
wcount=$(find "$workspace_dir" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | wc -l | tr -d ' ')
ok "workspaces/ layout in place ($wcount workspace dir(s))"
else
warn "workspaces/ dir does not exist yet — will be created when the first thread is opened"
fi
# Stage 0 PR6 dry-run check: just exercise the script flag, do not perform writes.
info "exercising migrate-paths dry-run (no writes)"
cd "$REPO_ROOT"
if make migrate-paths DRY_RUN=1 >/tmp/verify_migrate_dry.log 2>&1; then
ok "make migrate-paths DRY_RUN=1 ran without error"
else
fail "make migrate-paths DRY_RUN=1 failed — see /tmp/verify_migrate_dry.log"
fi
}
# ── phase: rds ───────────────────────────────────────────────────────────
phase_rds() {
phase "RDS — schema + alembic + indexes"
if [ -z "${DATABASE_URL:-}" ]; then
warn "DATABASE_URL not set — skipping RDS phase"
SKIPPED_PHASES+=("rds (DATABASE_URL unset)")
return
fi
if ! command -v psql >/dev/null 2>&1; then
fail "psql not installed — cannot run RDS phase"
return
fi
info "DATABASE_URL host: $(printf '%s' "$DATABASE_URL" | sed -E 's#.*@([^/]+)/.*#\1#')"
# PG version sanity
local pg_version
pg_version=$(psql "$DATABASE_URL" -tA -c "SELECT version()" 2>/dev/null | head -1)
if [ -z "$pg_version" ]; then
fail "cannot connect to RDS — check DATABASE_URL"
return
fi
ok "PG reachable: $pg_version"
# Alembic head
info "checking alembic head against expected 0003"
cd "$BACKEND_DIR"
if PYTHONPATH=. uv run alembic current 2>/tmp/verify_alembic.err | tee /tmp/verify_alembic.log | grep -q "^0003"; then
ok "alembic current is at 0003 (workspace_id NOT NULL + UNIQUE(wid, tid))"
else
fail "alembic current is not at 0003 — see /tmp/verify_alembic.log"
fi
cd "$REPO_ROOT"
# PR8 tables present
info "checking PR8 tables exist"
local pr8_count
pr8_count=$(psql "$DATABASE_URL" -tA -c "
SELECT count(*) FROM information_schema.tables
WHERE table_schema = current_schema()
AND table_name IN ('service_accounts','api_keys','external_users');
" 2>/dev/null | tr -d ' ')
if [ "${pr8_count:-0}" -eq 3 ]; then
ok "service_accounts + api_keys + external_users all present"
else
fail "PR8 tables missing — expected 3, got ${pr8_count:-0}"
fi
# Partial index on api_keys: WHERE revoked_at IS NULL
info "checking idx_api_keys_active partial-index predicate"
local idx_def
idx_def=$(psql "$DATABASE_URL" -tA -c "
SELECT indexdef FROM pg_indexes
WHERE schemaname = current_schema()
AND indexname = 'idx_api_keys_active';
" 2>/dev/null | head -1)
if printf '%s' "$idx_def" | grep -qi 'WHERE.*revoked_at IS NULL'; then
ok "idx_api_keys_active has 'WHERE revoked_at IS NULL' predicate"
else
fail "idx_api_keys_active missing or wrong predicate — got: ${idx_def:-<none>}"
fi
# 4 business tables workspace_id NOT NULL
info "checking workspace_id NOT NULL on 4 business tables"
local nullable_rows
nullable_rows=$(psql "$DATABASE_URL" -tA -c "
SELECT table_name FROM information_schema.columns
WHERE table_schema = current_schema()
AND column_name = 'workspace_id'
AND table_name IN ('thread_meta','runs','feedback','run_events')
AND is_nullable = 'YES';
" 2>/dev/null)
if [ -z "$nullable_rows" ]; then
ok "thread_meta + runs + feedback + run_events all have workspace_id NOT NULL"
else
fail "workspace_id is nullable in: $(printf '%s' "$nullable_rows" | tr '\n' ' ')"
fi
# Unique (workspace_id, thread_id) on thread_meta
info "checking UNIQUE(workspace_id, thread_id) on thread_meta"
local uq_count
uq_count=$(psql "$DATABASE_URL" -tA -c "
SELECT count(*) FROM pg_indexes
WHERE schemaname = current_schema()
AND tablename = 'thread_meta'
AND indexdef ILIKE '%UNIQUE%workspace_id%thread_id%';
" 2>/dev/null | tr -d ' ')
if [ "${uq_count:-0}" -ge 1 ]; then
ok "UNIQUE(workspace_id, thread_id) constraint/index present"
else
fail "no UNIQUE(workspace_id, thread_id) on thread_meta"
fi
}
# ── phase: runtime ───────────────────────────────────────────────────────
phase_runtime() {
phase "RUNTIME — gateway health"
if ! command -v curl >/dev/null 2>&1; then
fail "curl missing — cannot run runtime phase"
return
fi
info "probing $GATEWAY_URL/health"
local health
health=$(curl -sS -o /dev/null -w '%{http_code}' --max-time 5 "$GATEWAY_URL/health" 2>/dev/null || true)
if [ "$health" = "200" ]; then
ok "gateway /health returns 200"
else
fail "gateway /health returned '$health' — is \`make dev\` running?"
SKIPPED_PHASES+=("e2e (gateway not reachable)")
SKIP_E2E=1
fi
}
# ── phase: e2e ───────────────────────────────────────────────────────────
phase_e2e() {
phase "E2E — register 2 users + cross-workspace 404"
if [ "${SKIP_E2E:-0}" = "1" ]; then
warn "skipped because gateway probe failed"
return
fi
if ! command -v curl >/dev/null 2>&1; then
fail "curl missing"
return
fi
local tag
tag=$(date +%s)
local alice="alice-${tag}@verify-stage0.com"
local bob="bob-${tag}@verify-stage0.com"
local pw="VerifyStage0_${tag}!"
local jar_a=/tmp/verify_alice_${tag}.cookies
local jar_b=/tmp/verify_bob_${tag}.cookies
rm -f "$jar_a" "$jar_b"
# Ensure system is initialized (admin account exists) before registering users.
info "ensuring admin account exists (POST /api/v1/auth/initialize)"
local init_code
init_code=$(curl -sS -o /dev/null -w '%{http_code}' \
-H 'Content-Type: application/json' \
-d "{\"email\":\"admin-${tag}@verify-stage0.com\",\"password\":\"$pw\"}" \
"$GATEWAY_URL/api/v1/auth/initialize")
if [ "$init_code" = "201" ]; then
ok "admin initialized (first boot)"
elif [ "$init_code" = "409" ]; then
ok "admin already exists (system previously initialized)"
else
fail "admin initialization returned $init_code"
return
fi
register_user() {
local jar="$1"; local email="$2"
curl -sS -c "$jar" -o /tmp/verify_register_$$.json -w '%{http_code}' \
-H 'Content-Type: application/json' \
-d "{\"email\":\"$email\",\"password\":\"$pw\"}" \
"$GATEWAY_URL/api/v1/auth/register"
}
info "registering Alice + Bob"
local code_a code_b
code_a=$(register_user "$jar_a" "$alice")
code_b=$(register_user "$jar_b" "$bob")
if [ "$code_a" = "201" ] && [ "$code_b" = "201" ]; then
ok "both users registered (201 / 201)"
else
fail "registration failed: alice=$code_a, bob=$code_b"
return
fi
# Pull CSRF tokens from cookie jars (csrf_token cookie value, 4th-from-last field).
csrf_from_jar() {
awk '$6 == "csrf_token" { print $7 }' "$1" | tail -1
}
local csrf_a csrf_b
csrf_a=$(csrf_from_jar "$jar_a")
csrf_b=$(csrf_from_jar "$jar_b")
if [ -n "$csrf_a" ] && [ -n "$csrf_b" ]; then
ok "CSRF token captured for both sessions"
else
fail "CSRF cookie missing (alice='${csrf_a:0:8}...', bob='${csrf_b:0:8}...')"
return
fi
create_thread() {
local jar="$1"; local csrf="$2"; local tid="$3"
curl -sS -b "$jar" -o /tmp/verify_thread_$$.json -w '%{http_code}' \
-H 'Content-Type: application/json' \
-H "X-CSRF-Token: $csrf" \
-d "{\"thread_id\":\"$tid\"}" \
"$GATEWAY_URL/api/threads"
}
local tid_a="verify-alice-${tag}"
local tid_b="verify-bob-${tag}"
info "Alice creates thread $tid_a; Bob creates thread $tid_b"
local cta ctb
cta=$(create_thread "$jar_a" "$csrf_a" "$tid_a")
ctb=$(create_thread "$jar_b" "$csrf_b" "$tid_b")
if [ "$cta" = "200" ] && [ "$ctb" = "200" ]; then
ok "both threads created (200 / 200)"
else
fail "thread creation failed: alice=$cta, bob=$ctb"
return
fi
# Cross access: Alice tries to GET Bob's thread → must be 404 (per PR6 contract:
# cross-workspace returns 404, not 403, to avoid leaking existence).
info "Alice → Bob's thread (GET)"
local cross_get
cross_get=$(curl -sS -b "$jar_a" -o /dev/null -w '%{http_code}' "$GATEWAY_URL/api/threads/$tid_b")
if [ "$cross_get" = "404" ]; then
ok "cross-workspace GET returns 404 (no existence leak)"
else
fail "cross-workspace GET returned '$cross_get', expected 404 — PR6 isolation broken"
fi
info "Alice → Bob's thread (DELETE)"
local cross_del
cross_del=$(curl -sS -b "$jar_a" -X DELETE -H "X-CSRF-Token: $csrf_a" \
-o /dev/null -w '%{http_code}' "$GATEWAY_URL/api/threads/$tid_b")
if [ "$cross_del" = "404" ]; then
ok "cross-workspace DELETE returns 404"
else
fail "cross-workspace DELETE returned '$cross_del', expected 404"
fi
# Same-workspace GET — sanity check Alice can still reach her own thread.
info "Alice → Alice's thread (sanity)"
local same_get
same_get=$(curl -sS -b "$jar_a" -o /dev/null -w '%{http_code}' "$GATEWAY_URL/api/threads/$tid_a")
if [ "$same_get" = "200" ]; then
ok "same-workspace GET returns 200 (isolation is not over-blocking)"
else
fail "same-workspace GET returned '$same_get', expected 200"
fi
info "cookie jars left in /tmp for debugging: $jar_a $jar_b"
}
# ── main ─────────────────────────────────────────────────────────────────
main() {
local args=("$@")
if [ ${#args[@]} -eq 0 ]; then
args=(static paths rds runtime e2e)
fi
printf '%s%sStage 0 verification — %s%s\n' "$C_BOLD" "$C_BLUE" "$(date)" "$C_RESET"
printf 'Repo root: %s\n' "$REPO_ROOT"
printf 'Phases: %s\n' "${args[*]}"
for phase_name in "${args[@]}"; do
case "$phase_name" in
static) phase_static ;;
paths) phase_paths ;;
rds) phase_rds ;;
runtime) phase_runtime ;;
e2e) phase_e2e ;;
all)
phase_static; phase_paths; phase_rds; phase_runtime; phase_e2e
;;
*)
warn "unknown phase: $phase_name"
;;
esac
done
printf '\n%s%s──── summary ────%s\n' "$C_BOLD" "$C_BLUE" "$C_RESET"
printf ' %s%d passed%s %s%d failed%s %s%d warn%s\n' \
"$C_GREEN" "$PASS_COUNT" "$C_RESET" \
"$C_RED" "$FAIL_COUNT" "$C_RESET" \
"$C_YELLOW" "$WARN_COUNT" "$C_RESET"
if [ ${#SKIPPED_PHASES[@]} -gt 0 ]; then
printf ' skipped: %s\n' "${SKIPPED_PHASES[*]}"
fi
if [ ${#FAILED_STEPS[@]} -gt 0 ]; then
printf '\n%sfailing steps:%s\n' "$C_RED" "$C_RESET"
for step in "${FAILED_STEPS[@]}"; do
printf ' ✗ %s\n' "$step"
done
fi
[ "$FAIL_COUNT" -eq 0 ]
}
main "$@"
+50 -4
View File
@@ -36,8 +36,16 @@ from fastapi import FastAPI, Request, Response
from starlette.middleware.base import BaseHTTPMiddleware from starlette.middleware.base import BaseHTTPMiddleware
from starlette.types import ASGIApp from starlette.types import ASGIApp
from app.gateway.auth.models import User from app.gateway.auth.models import ActiveWorkspace, User
from app.gateway.authz import AuthContext, Permissions from app.gateway.authz import AuthContext, Permissions
from deerflow.runtime.user_context import (
reset_current_user,
set_current_user,
)
from deerflow.runtime.workspace_context import (
reset_current_workspace,
set_current_workspace,
)
# Default permission set granted to the stub user. Mirrors `_ALL_PERMISSIONS` # Default permission set granted to the stub user. Mirrors `_ALL_PERMISSIONS`
# in authz.py — kept inline so the tests don't import a private symbol. # in authz.py — kept inline so the tests don't import a private symbol.
@@ -67,22 +75,55 @@ class _StubAuthMiddleware(BaseHTTPMiddleware):
Mirrors what production ``AuthMiddleware`` does after the JWT decode Mirrors what production ``AuthMiddleware`` does after the JWT decode
+ DB lookup short-circuit, so ``@require_permission`` finds an + DB lookup short-circuit, so ``@require_permission`` finds an
authenticated context and skips its own re-authentication path. authenticated context and skips its own re-authentication path.
Optionally stamps the workspace contextvar too needed by the PR6
decorator path that calls ``get_effective_workspace_id()`` before
delegating to ``check_access``.
""" """
def __init__(self, app: ASGIApp, user_factory: Callable[[], User]) -> None: def __init__(
self,
app: ASGIApp,
user_factory: Callable[[], User],
workspace_factory: Callable[[], ActiveWorkspace | None] | None = None,
override_user_contextvar: bool = False,
) -> None:
super().__init__(app) super().__init__(app)
self._user_factory = user_factory self._user_factory = user_factory
self._workspace_factory = workspace_factory
# Tests that only need ``request.state.auth`` (the @require_permission
# path) keep the autouse user contextvar — flipping it to a per-call
# UUID would break legacy tests whose routes resolve paths via
# ``get_effective_user_id()``. Cross-user / cross-workspace tests opt
# in by setting this flag so the contextvar matches the request user.
self._override_user_contextvar = override_user_contextvar
async def dispatch(self, request: Request, call_next: Callable) -> Response: async def dispatch(self, request: Request, call_next: Callable) -> Response:
user = self._user_factory() user = self._user_factory()
request.state.user = user request.state.user = user
request.state.auth = AuthContext(user=user, permissions=list(_STUB_PERMISSIONS)) request.state.auth = AuthContext(user=user, permissions=list(_STUB_PERMISSIONS))
return await call_next(request)
user_token = set_current_user(user) if self._override_user_contextvar else None
ws_token = None
if self._workspace_factory is not None:
workspace = self._workspace_factory()
if workspace is not None:
request.state.workspace = workspace
ws_token = set_current_workspace(workspace)
try:
return await call_next(request)
finally:
if ws_token is not None:
reset_current_workspace(ws_token)
if user_token is not None:
reset_current_user(user_token)
def make_authed_test_app( def make_authed_test_app(
*, *,
user_factory: Callable[[], User] | None = None, user_factory: Callable[[], User] | None = None,
workspace_factory: Callable[[], ActiveWorkspace | None] | None = None,
override_user_contextvar: bool = False,
owner_check_passes: bool = True, owner_check_passes: bool = True,
) -> FastAPI: ) -> FastAPI:
"""Build a FastAPI test app with stub auth + permissive thread_store. """Build a FastAPI test app with stub auth + permissive thread_store.
@@ -103,7 +144,12 @@ def make_authed_test_app(
""" """
factory = user_factory or _make_stub_user factory = user_factory or _make_stub_user
app = FastAPI() app = FastAPI()
app.add_middleware(_StubAuthMiddleware, user_factory=factory) app.add_middleware(
_StubAuthMiddleware,
user_factory=factory,
workspace_factory=workspace_factory,
override_user_contextvar=override_user_contextvar,
)
repo = MagicMock() repo = MagicMock()
repo.check_access = AsyncMock(return_value=owner_check_passes) repo.check_access = AsyncMock(return_value=owner_check_passes)
+28
View File
@@ -0,0 +1,28 @@
# PR7 — boundary scan allowlist for direct LangGraph checkpoint/saver imports.
#
# Paths are relative to backend/. A path here means: "this file is permitted to
# import langgraph.checkpoint.* at runtime". Adding to this list requires a
# matching justification in the PR that introduces the new importer.
#
# Consumed by tests/test_workspace_boundary.py.
#
# Imports inside `if TYPE_CHECKING:` blocks are exempt automatically — they do
# not pull the symbol into runtime — so type-only references (e.g. annotations
# on `BaseCheckpointSaver` parameters) do NOT need to be listed here.
langgraph_checkpoint_importers = [
# Gateway thread plumbing: constructs an empty checkpoint when initialising
# a new thread's state via the LangGraph runtime.
"app/gateway/routers/threads.py",
# Harness checkpointer factories: the single authorised place to construct
# InMemorySaver / SqliteSaver / PostgresSaver implementations. Everywhere
# else must obtain a checkpointer via `app.gateway.deps.get_checkpointer`
# or `deerflow.runtime.checkpointer` helpers.
"packages/harness/deerflow/runtime/checkpointer/async_provider.py",
"packages/harness/deerflow/runtime/checkpointer/provider.py",
# Background run worker: uses `empty_checkpoint` to seed state for runs
# resumed from a missing/expired checkpoint id.
"packages/harness/deerflow/runtime/runs/worker.py",
]
+127
View File
@@ -15,6 +15,13 @@ import pytest
# Make 'app' and 'deerflow' importable from any working directory # Make 'app' and 'deerflow' importable from any working directory
sys.path.insert(0, str(Path(__file__).parent.parent)) sys.path.insert(0, str(Path(__file__).parent.parent))
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "scripts")) sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "scripts"))
# Make 'fixtures.*' importable as plugin modules from this conftest.
sys.path.insert(0, str(Path(__file__).parent))
# Register fixture plugin modules so tests can request fixtures by name
# without ad-hoc imports. ``fixtures.postgres`` provides
# ``postgres_container`` (session-scoped) and ``postgres_url`` (per-test).
pytest_plugins = ["fixtures.postgres"]
# Break the circular import chain that exists in production code: # Break the circular import chain that exists in production code:
# deerflow.subagents.__init__ # deerflow.subagents.__init__
@@ -38,6 +45,95 @@ _executor_mock.get_background_task_result = MagicMock()
sys.modules["deerflow.subagents.executor"] = _executor_mock sys.modules["deerflow.subagents.executor"] = _executor_mock
# ---------------------------------------------------------------------------
# Auto-seed test workspace + user when Base.metadata.create_all() runs
# ---------------------------------------------------------------------------
#
# PR6 makes every business-row INSERT carry ``workspace_id`` (resolved
# from the autouse workspace contextvar = "test-workspace-autouse"). The
# Stage 0 schema has a NOT NULL FK from those rows to ``workspaces`` and
# from ``workspaces.owner_id`` to ``users``. Without the seed below,
# every legacy repo test would fail with a FOREIGN KEY error the moment
# it tries to insert a thread.
#
# We register an ``after_create`` hook on ``Base.metadata`` so that
# whenever ``init_engine`` finishes ``create_all()`` (the auto-create
# path used by tests and dev), the two anchor rows are present. Alembic
# migration tests don't trigger create_all so they are unaffected and
# keep exercising real FK constraints in isolation.
def _register_test_seed_listener() -> None:
"""Attach an after_create hook that seeds the autouse user + workspace."""
try:
from sqlalchemy import event, update
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
from deerflow.persistence.base import Base
from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace.model import WorkspaceRow
except ImportError:
return
from datetime import UTC, datetime
def _seed(_target, connection, **kw): # noqa: ARG001
tables = {t.name for t in kw.get("tables", []) or []}
if "users" not in tables or "workspaces" not in tables:
return
dialect = connection.dialect.name
now = datetime.now(UTC)
# Seed both rows with a consistent ``default_workspace_id`` so the
# PR5 backfill script (which scans ``users.default_workspace_id IS
# NULL``) does not pick up the test fixtures as candidates.
# Insert user first with NULL default_workspace_id (chicken-and-egg
# with workspaces.owner_id FK), then workspace, then UPDATE the user
# row to point at the workspace so the PR5 backfill script does not
# pick up the autouse user as a candidate.
user_values = {
"id": "test-user-autouse",
"email": "test-user-autouse@local",
"password_hash": None,
"system_role": "user",
"created_at": now,
"oauth_provider": None,
"oauth_id": None,
"needs_setup": False,
"token_version": 0,
"default_workspace_id": None,
}
workspace_values = {
"id": "test-workspace-autouse",
"name": "Autouse Test Workspace",
"slug": "autouse-test",
"status": "active",
"owner_id": "test-user-autouse",
"created_at": now,
"updated_at": now,
}
if dialect == "sqlite":
user_stmt = sqlite_insert(UserRow.__table__).values(**user_values).on_conflict_do_nothing(index_elements=["id"])
ws_stmt = sqlite_insert(WorkspaceRow.__table__).values(**workspace_values).on_conflict_do_nothing(index_elements=["id"])
elif dialect == "postgresql":
user_stmt = pg_insert(UserRow.__table__).values(**user_values).on_conflict_do_nothing(index_elements=["id"])
ws_stmt = pg_insert(WorkspaceRow.__table__).values(**workspace_values).on_conflict_do_nothing(index_elements=["id"])
else:
return
connection.execute(user_stmt)
connection.execute(ws_stmt)
connection.execute(update(UserRow.__table__).where(UserRow.__table__.c.id == "test-user-autouse").where(UserRow.__table__.c.default_workspace_id.is_(None)).values(default_workspace_id="test-workspace-autouse"))
event.listen(Base.metadata, "after_create", _seed)
_register_test_seed_listener()
@pytest.fixture() @pytest.fixture()
def provisioner_module(): def provisioner_module():
"""Load docker/provisioner/app.py as an importable test module. """Load docker/provisioner/app.py as an importable test module.
@@ -110,3 +206,34 @@ def _auto_user_context(request):
yield yield
finally: finally:
reset_current_user(token) reset_current_user(token)
@pytest.fixture(autouse=True)
def _auto_workspace_context(request):
"""Inject a default ``test-workspace-autouse`` into the workspace contextvar.
Mirror of :func:`_auto_user_context`. PR6 adds ``workspace_id=AUTO``
sentinels to every repository method; without an autouse workspace
fixture every legacy persistence test would raise RuntimeError.
Opt-out via ``@pytest.mark.no_auto_workspace``.
"""
if request.node.get_closest_marker("no_auto_workspace"):
yield
return
try:
from deerflow.runtime.workspace_context import (
reset_current_workspace,
set_current_workspace,
)
except ImportError:
yield
return
workspace = SimpleNamespace(id="test-workspace-autouse", role="owner")
token = set_current_workspace(workspace)
try:
yield
finally:
reset_current_workspace(token)
View File
+82
View File
@@ -0,0 +1,82 @@
"""Postgres testcontainer fixtures for Stage 0 PR1.
Provides per-test ephemeral database isolation atop a single session-scoped
container. Tests marked ``@pytest.mark.postgres`` request the ``postgres_url``
fixture, which yields an asyncpg connection URL pointing at a freshly-created
database. The database is force-dropped after the test (any leaked connections
get pg_terminate_backend'd first).
Why per-database rather than per-schema:
asyncpg (the SQLAlchemy async driver we use) doesn't honor URL-embedded
search_path the way psycopg does. Per-database isolation is one extra
CREATE/DROP per test (~50ms), but lets test app code use its full schema
unchanged.
"""
from __future__ import annotations
import secrets
from collections.abc import Iterator
import pytest
@pytest.fixture(scope="session")
def postgres_container():
"""Session-scoped Postgres 16 container shared across all postgres tests.
Started once per pytest session. Subsequent tests piggy-back on the same
container; each gets its own database via the ``postgres_url`` fixture.
Skipped (and the test marked skip) if Docker is unavailable on the host
testcontainers raises ``DockerException`` when it can't reach the daemon.
"""
try:
from testcontainers.postgres import PostgresContainer
except ImportError as exc: # pragma: no cover - install boundary
pytest.skip(f"testcontainers[postgres] not installed: {exc}")
try:
# Image tag aligned with the production Aliyun RDS (PostgreSQL 17.9
# confirmed by `make doctor` 2026-05-11). Bump together with RDS upgrades.
with PostgresContainer("postgres:17-alpine") as pg:
yield pg
except Exception as exc: # pragma: no cover - environment-dependent
# DockerException, ConnectionError, etc. — surface a skip rather than
# an error so devs without Docker can still run the rest of the suite.
pytest.skip(f"could not start Postgres container ({exc})")
@pytest.fixture
def postgres_url(postgres_container) -> Iterator[str]:
"""Per-test ephemeral database URL (asyncpg dialect).
Each invocation creates a unique database on the shared container and
yields its URL. Teardown force-drops the database, terminating any
backend connections the test forgot to close.
"""
import psycopg
from psycopg import sql
db_name = f"test_{secrets.token_hex(8)}"
raw = postgres_container.get_connection_url() # postgresql+psycopg2://...
# Strip the SQLAlchemy dialect prefix so plain psycopg can connect.
base = raw.replace("postgresql+psycopg2://", "postgresql://")
parent_url = base.rsplit("/", 1)[0] + "/postgres"
# CREATE DATABASE must run outside a transaction; psycopg autocommit=True.
with psycopg.connect(parent_url, autocommit=True) as conn:
conn.execute(sql.SQL("CREATE DATABASE {}").format(sql.Identifier(db_name)))
asyncpg_url = raw.replace("postgresql+psycopg2://", "postgresql+asyncpg://").rsplit("/", 1)[0] + f"/{db_name}"
try:
yield asyncpg_url
finally:
with psycopg.connect(parent_url, autocommit=True) as conn:
# Kick any leaked connections so DROP DATABASE doesn't block.
conn.execute(
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = %s AND pid <> pg_backend_pid()",
(db_name,),
)
conn.execute(sql.SQL("DROP DATABASE IF EXISTS {}").format(sql.Identifier(db_name)))
@@ -0,0 +1,357 @@
"""Alembic 0002 / 0003 round-trips on both Postgres and SQLite.
PR5 has two migrations:
* ``0002_business_tables_workspace`` adds *nullable* ``workspace_id``
+ FK to ``workspaces`` on ``threads_meta`` / ``runs`` / ``feedback`` /
``run_events``, plus a composite index on threads_meta for the common
"list a workspace's threads for a user, newest first" query.
* ``0003_business_tables_workspace_not_null`` flips the column to
``NOT NULL`` (refusing to upgrade if NULL rows remain) and adds the
UNIQUE (workspace_id, thread_id) index on ``threads_meta``.
Pattern mirrors ``test_alembic_default_workspace_id.py``: synchronous
test bodies (alembic's command layer is sync, ``env.py`` calls
``asyncio.run`` internally running under pytest-anyio would explode
that nested event loop). The pre-migration schema is bootstrapped with
just the columns those migrations touch, so we don't depend on the
production ORM staying frozen.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, inspect, text
from sqlalchemy.exc import IntegrityError
_ALEMBIC_INI = Path(__file__).resolve().parents[1] / "packages" / "harness" / "deerflow" / "persistence" / "migrations" / "alembic.ini"
# Tables the PR5 migrations touch.
_BUSINESS_TABLES = ("threads_meta", "runs", "feedback", "run_events")
def _make_alembic_config(url: str) -> Config:
cfg = Config(str(_ALEMBIC_INI))
cfg.set_main_option("sqlalchemy.url", url)
return cfg
def _bootstrap_pre_pr5_schema(sync_url: str) -> None:
"""Create the post-PR4 schema PR5 migrations expect.
Includes ``workspaces`` (FK target), ``users`` (already has
``default_workspace_id`` from 0001 but we skip 0001 here and bootstrap
the columns directly so the migration's behaviour can be tested in
isolation), and the four business tables.
"""
engine = create_engine(sync_url)
with engine.begin() as conn:
conn.execute(
text(
"""
CREATE TABLE workspaces (
id VARCHAR(36) PRIMARY KEY,
name VARCHAR(64) NOT NULL
)
"""
)
)
conn.execute(
text(
"""
CREATE TABLE users (
id VARCHAR(36) PRIMARY KEY,
email VARCHAR(255) NOT NULL,
default_workspace_id VARCHAR(36)
)
"""
)
)
conn.execute(
text(
"""
CREATE TABLE threads_meta (
thread_id VARCHAR(64) PRIMARY KEY,
user_id VARCHAR(64),
updated_at TIMESTAMP
)
"""
)
)
conn.execute(
text(
"""
CREATE TABLE runs (
run_id VARCHAR(64) PRIMARY KEY,
thread_id VARCHAR(64) NOT NULL,
user_id VARCHAR(64)
)
"""
)
)
conn.execute(
text(
"""
CREATE TABLE feedback (
feedback_id VARCHAR(64) PRIMARY KEY,
thread_id VARCHAR(64) NOT NULL,
run_id VARCHAR(64) NOT NULL,
user_id VARCHAR(64),
rating INTEGER NOT NULL
)
"""
)
)
conn.execute(
text(
"""
CREATE TABLE run_events (
id INTEGER PRIMARY KEY,
thread_id VARCHAR(64) NOT NULL,
run_id VARCHAR(64) NOT NULL,
user_id VARCHAR(64),
event_type VARCHAR(32) NOT NULL,
category VARCHAR(16) NOT NULL,
content TEXT,
seq INTEGER NOT NULL
)
"""
)
)
# Pretend 0001 has already run so 0002 is the next revision applied.
conn.execute(
text(
"""
CREATE TABLE alembic_version (version_num VARCHAR(32) NOT NULL PRIMARY KEY)
"""
)
)
conn.execute(text("INSERT INTO alembic_version (version_num) VALUES ('0001_users_default_workspace')"))
engine.dispose()
def _assert_workspace_id_present(sync_url: str, *, nullable: bool) -> None:
engine = create_engine(sync_url)
insp = inspect(engine)
for table in _BUSINESS_TABLES:
cols = {c["name"]: c for c in insp.get_columns(table)}
assert "workspace_id" in cols, f"{table}: workspace_id missing; got {list(cols)}"
col = cols["workspace_id"]
assert col["nullable"] is nullable, f"{table}.workspace_id nullable expected {nullable}, got {col['nullable']}"
fks = insp.get_foreign_keys(table)
ws_fks = [fk for fk in fks if fk.get("referred_table") == "workspaces" and fk.get("constrained_columns") == ["workspace_id"]]
assert ws_fks, f"{table}: FK to workspaces missing; got {fks}"
idxs = {i["name"] for i in insp.get_indexes("threads_meta")}
assert "idx_threads_meta_workspace_user_updated" in idxs, f"threads_meta composite index missing; got {idxs}"
engine.dispose()
def _assert_workspace_id_absent(sync_url: str) -> None:
engine = create_engine(sync_url)
insp = inspect(engine)
for table in _BUSINESS_TABLES:
cols = {c["name"] for c in insp.get_columns(table)}
assert "workspace_id" not in cols, f"{table}: workspace_id still present; got {cols}"
idxs = {i["name"] for i in insp.get_indexes("threads_meta")}
assert "idx_threads_meta_workspace_user_updated" not in idxs, f"threads_meta composite index still present; got {idxs}"
engine.dispose()
# ---------- SQLite tests ----------------------------------------------------
def test_sqlite_upgrade_0002_adds_workspace_id_column() -> None:
"""0002 adds nullable workspace_id + FK + composite index on SQLite."""
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
sync_url = f"sqlite:///{db_path}"
async_url = f"sqlite+aiosqlite:///{db_path}"
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(async_url)
command.upgrade(cfg, "0002_business_tables_workspace")
_assert_workspace_id_present(sync_url, nullable=True)
def test_sqlite_downgrade_0002_removes_workspace_id_column() -> None:
"""0002 downgrade drops the column + FK + composite index cleanly."""
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
sync_url = f"sqlite:///{db_path}"
async_url = f"sqlite+aiosqlite:///{db_path}"
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(async_url)
command.upgrade(cfg, "0002_business_tables_workspace")
_assert_workspace_id_present(sync_url, nullable=True)
command.downgrade(cfg, "-1")
_assert_workspace_id_absent(sync_url)
# ---------- Postgres tests --------------------------------------------------
@pytest.mark.postgres
def test_postgres_upgrade_0002_adds_workspace_id_column(postgres_url: str) -> None:
"""Postgres: 0002 adds nullable workspace_id + FK + composite index."""
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(postgres_url)
command.upgrade(cfg, "0002_business_tables_workspace")
_assert_workspace_id_present(sync_url, nullable=True)
@pytest.mark.postgres
def test_postgres_downgrade_0002_removes_workspace_id_column(postgres_url: str) -> None:
"""Postgres: 0002 downgrade drops column + FK + index cleanly."""
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(postgres_url)
command.upgrade(cfg, "0002_business_tables_workspace")
_assert_workspace_id_present(sync_url, nullable=True)
command.downgrade(cfg, "-1")
_assert_workspace_id_absent(sync_url)
# ---------- 0003 helpers ----------------------------------------------------
def _populate_business_rows_for_0003(sync_url: str, workspace_id: str = "w1", thread_id: str = "t1", run_id: str = "r1") -> None:
"""Insert one row per business table with workspace_id populated.
Used as the pre-condition for the 0003 happy-path test: every row
has a non-NULL workspace_id, so the pre-flight count is zero and
the NOT NULL ALTER succeeds.
"""
engine = create_engine(sync_url)
with engine.begin() as conn:
conn.execute(text("INSERT INTO workspaces (id, name) VALUES (:id, :name)"), {"id": workspace_id, "name": "W"})
conn.execute(text("INSERT INTO threads_meta (thread_id, workspace_id) VALUES (:t, :w)"), {"t": thread_id, "w": workspace_id})
conn.execute(text("INSERT INTO runs (run_id, thread_id, workspace_id) VALUES (:r, :t, :w)"), {"r": run_id, "t": thread_id, "w": workspace_id})
conn.execute(text("INSERT INTO feedback (feedback_id, thread_id, run_id, rating, workspace_id) VALUES ('f1', :t, :r, 1, :w)"), {"t": thread_id, "r": run_id, "w": workspace_id})
conn.execute(text("INSERT INTO run_events (thread_id, run_id, event_type, category, seq, workspace_id) VALUES (:t, :r, 'x', 'lifecycle', 1, :w)"), {"t": thread_id, "r": run_id, "w": workspace_id})
engine.dispose()
def _assert_workspace_id_not_null_and_unique_index(sync_url: str) -> None:
engine = create_engine(sync_url)
insp = inspect(engine)
for table in _BUSINESS_TABLES:
cols = {c["name"]: c for c in insp.get_columns(table)}
assert cols["workspace_id"]["nullable"] is False, f"{table}.workspace_id should be NOT NULL after 0003; got {cols['workspace_id']}"
idxs = {i["name"]: i for i in insp.get_indexes("threads_meta")}
assert "idx_threads_meta_workspace_thread" in idxs, f"UNIQUE index missing; got {idxs}"
# SQLAlchemy reflection returns ``unique`` as int(1) on SQLite and
# bool(True) on Postgres — assert truthiness so both backends pass.
assert idxs["idx_threads_meta_workspace_thread"]["unique"], f"index should be UNIQUE; got {idxs['idx_threads_meta_workspace_thread']}"
engine.dispose()
# ---------- 0003 SQLite tests -----------------------------------------------
def test_sqlite_upgrade_0003_succeeds_with_populated_workspace_id() -> None:
"""0003 NOT NULL ALTER + UNIQUE index lands when no NULL rows remain."""
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
sync_url = f"sqlite:///{db_path}"
async_url = f"sqlite+aiosqlite:///{db_path}"
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(async_url)
command.upgrade(cfg, "0002_business_tables_workspace")
_populate_business_rows_for_0003(sync_url)
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
_assert_workspace_id_not_null_and_unique_index(sync_url)
def test_sqlite_upgrade_0003_requires_no_null_workspace_id() -> None:
"""0003 refuses to upgrade if any business row still has workspace_id=NULL."""
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
sync_url = f"sqlite:///{db_path}"
async_url = f"sqlite+aiosqlite:///{db_path}"
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(async_url)
command.upgrade(cfg, "0002_business_tables_workspace")
# Leave one threads_meta row with workspace_id NULL.
engine = create_engine(sync_url)
with engine.begin() as conn:
conn.execute(text("INSERT INTO threads_meta (thread_id) VALUES ('t-orphan')"))
engine.dispose()
with pytest.raises(RuntimeError, match="Cannot ALTER"):
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
def test_sqlite_threads_meta_unique_workspace_thread() -> None:
"""After 0003, duplicate (workspace_id, thread_id) raises IntegrityError on insert."""
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
sync_url = f"sqlite:///{db_path}"
async_url = f"sqlite+aiosqlite:///{db_path}"
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(async_url)
command.upgrade(cfg, "0002_business_tables_workspace")
_populate_business_rows_for_0003(sync_url)
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
engine = create_engine(sync_url)
# threads_meta.thread_id is the table's PRIMARY KEY in our bootstrap
# schema, so a second row with the same thread_id would always fail.
# Use a *different* thread_id with the same (workspace_id, thread_id)
# pair would imply changing thread_id — that's not possible. Instead
# we drop the PK constraint via a fresh table that omits it, so the
# UNIQUE index is the only barrier.
with engine.begin() as conn:
conn.execute(text("CREATE TABLE threads_meta_test_unique (id INTEGER PRIMARY KEY, thread_id VARCHAR(64), workspace_id VARCHAR(36))"))
conn.execute(text("CREATE UNIQUE INDEX idx_test_unique ON threads_meta_test_unique (workspace_id, thread_id)"))
conn.execute(text("INSERT INTO threads_meta_test_unique (thread_id, workspace_id) VALUES ('t', 'w')"))
with pytest.raises(IntegrityError):
with engine.begin() as conn:
conn.execute(text("INSERT INTO threads_meta_test_unique (thread_id, workspace_id) VALUES ('t', 'w')"))
engine.dispose()
# ---------- 0003 Postgres tests ---------------------------------------------
@pytest.mark.postgres
def test_postgres_upgrade_0003_succeeds_with_populated_workspace_id(postgres_url: str) -> None:
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(postgres_url)
command.upgrade(cfg, "0002_business_tables_workspace")
_populate_business_rows_for_0003(sync_url)
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
_assert_workspace_id_not_null_and_unique_index(sync_url)
@pytest.mark.postgres
def test_postgres_upgrade_0003_requires_no_null_workspace_id(postgres_url: str) -> None:
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
_bootstrap_pre_pr5_schema(sync_url)
cfg = _make_alembic_config(postgres_url)
command.upgrade(cfg, "0002_business_tables_workspace")
engine = create_engine(sync_url)
with engine.begin() as conn:
conn.execute(text("INSERT INTO threads_meta (thread_id) VALUES ('t-orphan')"))
engine.dispose()
with pytest.raises(RuntimeError, match="Cannot ALTER"):
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
@@ -0,0 +1,148 @@
"""Alembic revision 0001 round-trips on both Postgres and SQLite.
Verifies the first DeerFlow migration adds `users.default_workspace_id`
(with FK to `workspaces`) on upgrade and removes it on downgrade. Both
backends are exercised because the migration relies on
`op.batch_alter_table` for SQLite ALTER compatibility we want to know
if either dialect regresses.
These tests are synchronous: alembic's command layer is sync, and our
`env.py` calls `asyncio.run(...)` internally. Running under
pytest-anyio would put us inside an event loop and crash that
`asyncio.run` call, so we keep the test bodies plain `def`.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, inspect, text
_ALEMBIC_INI = Path(__file__).resolve().parents[1] / "packages" / "harness" / "deerflow" / "persistence" / "migrations" / "alembic.ini"
def _make_alembic_config(url: str) -> Config:
cfg = Config(str(_ALEMBIC_INI))
cfg.set_main_option("sqlalchemy.url", url)
return cfg
def _bootstrap_pre_pr4_schema(sync_url: str) -> None:
"""Create the minimal pre-PR4 schema the migration needs to ALTER.
Only `users` (without `default_workspace_id`) and `workspaces` (just `id`)
are required so the FK target resolves. The rest of the production
schema is irrelevant to this migration.
"""
engine = create_engine(sync_url)
with engine.begin() as conn:
conn.execute(
text(
"""
CREATE TABLE workspaces (
id VARCHAR(36) PRIMARY KEY,
name VARCHAR(64) NOT NULL
)
"""
)
)
conn.execute(
text(
"""
CREATE TABLE users (
id VARCHAR(36) PRIMARY KEY,
email VARCHAR(255) NOT NULL
)
"""
)
)
engine.dispose()
def _assert_column_present(sync_url: str) -> None:
engine = create_engine(sync_url)
insp = inspect(engine)
cols = {c["name"] for c in insp.get_columns("users")}
assert "default_workspace_id" in cols, f"column missing; got {cols}"
fks = insp.get_foreign_keys("users")
fk_to_ws = [fk for fk in fks if fk.get("referred_table") == "workspaces"]
assert fk_to_ws, f"FK to workspaces missing; got {fks}"
assert fk_to_ws[0]["constrained_columns"] == ["default_workspace_id"]
assert fk_to_ws[0]["referred_columns"] == ["id"]
engine.dispose()
def _assert_column_absent(sync_url: str) -> None:
engine = create_engine(sync_url)
insp = inspect(engine)
cols = {c["name"] for c in insp.get_columns("users")}
assert "default_workspace_id" not in cols, f"column still present; got {cols}"
engine.dispose()
# ---------- SQLite tests ----------------------------------------------------
def test_sqlite_upgrade_adds_default_workspace_id_with_fk() -> None:
"""SQLite: upgrade 0001 adds column + FK; batch_alter_table works."""
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
sync_url = f"sqlite:///{db_path}"
async_url = f"sqlite+aiosqlite:///{db_path}"
_bootstrap_pre_pr4_schema(sync_url)
cfg = _make_alembic_config(async_url)
command.upgrade(cfg, "0001_users_default_workspace")
_assert_column_present(sync_url)
def test_sqlite_downgrade_removes_default_workspace_id() -> None:
"""SQLite: downgrade 0001 removes the column it added."""
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "test.db"
sync_url = f"sqlite:///{db_path}"
async_url = f"sqlite+aiosqlite:///{db_path}"
_bootstrap_pre_pr4_schema(sync_url)
cfg = _make_alembic_config(async_url)
command.upgrade(cfg, "0001_users_default_workspace")
_assert_column_present(sync_url)
command.downgrade(cfg, "-1")
_assert_column_absent(sync_url)
# ---------- Postgres tests --------------------------------------------------
@pytest.mark.postgres
def test_postgres_upgrade_adds_default_workspace_id_with_fk(postgres_url: str) -> None:
"""Postgres: upgrade 0001 adds column + FK pointing at workspaces(id)."""
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
_bootstrap_pre_pr4_schema(sync_url)
cfg = _make_alembic_config(postgres_url)
command.upgrade(cfg, "head")
_assert_column_present(sync_url)
@pytest.mark.postgres
def test_postgres_downgrade_removes_default_workspace_id(postgres_url: str) -> None:
"""Postgres: downgrade 0001 cleanly drops the FK and column."""
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
_bootstrap_pre_pr4_schema(sync_url)
cfg = _make_alembic_config(postgres_url)
command.upgrade(cfg, "head")
_assert_column_present(sync_url)
command.downgrade(cfg, "-1")
_assert_column_absent(sync_url)
@@ -0,0 +1,59 @@
"""Deprecation header tests (Stage 1 PR5)."""
from __future__ import annotations
from starlette.testclient import TestClient
def _make_app():
from fastapi import FastAPI
from app.gateway.deprecation_middleware import ApiDeprecationMiddleware
app = FastAPI()
app.add_middleware(ApiDeprecationMiddleware)
@app.get("/api/threads")
async def legacy():
return {"ok": True}
@app.get("/api/v1/threads")
async def versioned():
return {"ok": True}
@app.get("/api/langgraph/info")
async def lg():
return {"ok": True}
@app.get("/api/assistants/info")
async def assistants():
return {"ok": True}
return app
def test_legacy_path_gets_deprecation_header():
client = TestClient(_make_app())
r = client.get("/api/threads")
assert r.headers.get("X-API-Deprecated") == "2027-01-01"
def test_versioned_path_no_header():
client = TestClient(_make_app())
r = client.get("/api/v1/threads")
assert "X-API-Deprecated" not in r.headers
def test_langgraph_path_no_header():
client = TestClient(_make_app())
r = client.get("/api/langgraph/info")
assert "X-API-Deprecated" not in r.headers
def test_assistants_compat_path_gets_deprecation_header():
# assistants_compat is an un-versioned LangGraph-platform stub; it
# intentionally carries the deprecation header (it is /api/, not
# /api/v1 or /api/langgraph). Documented here to prevent confusion.
client = TestClient(_make_app())
r = client.get("/api/assistants/info")
assert r.headers.get("X-API-Deprecated") == "2027-01-01"
+137
View File
@@ -0,0 +1,137 @@
"""Tests for the API key auth backend (Stage 1 PR2)."""
from __future__ import annotations
import dataclasses
import pytest
from app.gateway.auth.api_key_backend import ServicePrincipal, parse_scopes
from deerflow.auth.tokens import generate_api_key
pytestmark = pytest.mark.anyio
def test_parse_scopes_splits_and_strips():
assert parse_scopes("threads:read, threads:write") == ["threads:read", "threads:write"]
def test_parse_scopes_empty_string_is_empty_list():
assert parse_scopes("") == []
assert parse_scopes(" ") == []
def test_parse_scopes_drops_empty_segments():
assert parse_scopes("threads:read,,runs:create,") == ["threads:read", "runs:create"]
def test_service_principal_is_service_account_true_by_default():
p = ServicePrincipal(id="sa-1")
assert p.id == "sa-1"
assert p.is_service_account is True
def test_service_principal_is_frozen():
p = ServicePrincipal(id="sa-1")
with pytest.raises(dataclasses.FrozenInstanceError):
p.id = "other" # type: ignore[misc]
@pytest.fixture
def anyio_backend() -> str:
return "asyncio"
async def _setup_backend(tmp_path, *, sa_status="active", ws_status="active", scopes="threads:read", expires_at=None, revoke=False):
from app.gateway.auth.api_key_backend import APIKeyAuthBackend
from deerflow.persistence.api_key import ApiKeyRepository
from deerflow.persistence.engine import get_session_factory, init_engine
from deerflow.persistence.service_account import ServiceAccountRepository
from deerflow.persistence.service_account.model import ServiceAccountRow
from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace import WorkspaceRepository
from deerflow.persistence.workspace.model import WorkspaceRow
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
sf = get_session_factory()
async with sf() as session:
session.add(UserRow(id="u-alice", email="alice@example.com"))
await session.commit()
async with sf() as session:
session.add(WorkspaceRow(id="w-1", name="WS", slug="ws", owner_id="u-alice", status=ws_status))
await session.commit()
async with sf() as session:
session.add(ServiceAccountRow(id="sa-1", workspace_id="w-1", name="bot", role="member", identity_mode="collapsed", status=sa_status, created_by="u-alice"))
await session.commit()
api_key_repo = ApiKeyRepository(sf)
gen = generate_api_key("live")
created = await api_key_repo.create(service_account_id="sa-1", key_prefix=gen.prefix, key_hash=gen.key_hash, name="k", scopes=scopes, expires_at=expires_at)
if revoke:
await api_key_repo.revoke(created["id"])
backend = APIKeyAuthBackend(api_key_repo=api_key_repo, service_account_repo=ServiceAccountRepository(sf), workspace_repo=WorkspaceRepository(sf))
return backend, gen
async def _cleanup():
from deerflow.persistence.engine import close_engine
await close_engine()
async def test_authenticate_valid_key(tmp_path):
backend, gen = await _setup_backend(tmp_path, scopes="threads:read,threads:write")
try:
result = await backend.authenticate(gen.plaintext)
assert result is not None
assert result.principal.id == "sa-1"
assert result.principal.is_service_account is True
assert result.workspace_id == "w-1"
assert result.role == "member"
assert result.permissions == ["threads:read", "threads:write"]
finally:
await _cleanup()
async def test_authenticate_unknown_token_returns_none(tmp_path):
backend, _ = await _setup_backend(tmp_path)
try:
assert await backend.authenticate("dfk_live_doesnotexist000000000000") is None
finally:
await _cleanup()
async def test_authenticate_revoked_key_returns_none(tmp_path):
backend, gen = await _setup_backend(tmp_path, revoke=True)
try:
assert await backend.authenticate(gen.plaintext) is None
finally:
await _cleanup()
async def test_authenticate_suspended_sa_returns_none(tmp_path):
backend, gen = await _setup_backend(tmp_path, sa_status="suspended")
try:
assert await backend.authenticate(gen.plaintext) is None
finally:
await _cleanup()
async def test_authenticate_suspended_workspace_returns_none(tmp_path):
backend, gen = await _setup_backend(tmp_path, ws_status="suspended")
try:
assert await backend.authenticate(gen.plaintext) is None
finally:
await _cleanup()
async def test_authenticate_expired_key_returns_none(tmp_path):
from datetime import UTC, datetime
backend, gen = await _setup_backend(tmp_path, expires_at=datetime(2000, 1, 1, tzinfo=UTC))
try:
assert await backend.authenticate(gen.plaintext) is None
finally:
await _cleanup()
+195
View File
@@ -0,0 +1,195 @@
"""API key control-plane default-deny tests (Stage 1 收口).
service principal (API key) 只能访问数据平面 (threads/runs/assistants);
控制平面 (models/mcp/memory/skills/channels/agents 与管理/auth) 一律 403
真人 cookie 路径不受影响设计见 spec
docs/superpowers/specs/2026-06-28-api-key-control-plane-default-deny-design.md
"""
from __future__ import annotations
import pytest
from fastapi import Request
from starlette.testclient import TestClient
from app.gateway.auth_middleware import _is_dataplane_path
from deerflow.auth.tokens import generate_api_key
pytestmark = pytest.mark.anyio
@pytest.mark.parametrize(
"path",
[
"/api/threads",
"/api/threads/abc",
"/api/v1/threads",
"/api/v1/threads/abc/runs/xyz/feedback",
"/api/runs",
"/api/runs/stream",
"/api/v1/runs/stream",
"/api/assistants",
"/api/assistants/search",
],
)
def test_dataplane_paths_allowed(path):
assert _is_dataplane_path(path) is True
@pytest.mark.parametrize(
"path",
[
"/api/models",
"/api/v1/models",
"/api/mcp/config",
"/api/v1/mcp/config",
"/api/v1/memory",
"/api/v1/skills/install",
"/api/v1/channels/restart",
"/api/v1/agents",
"/api/v1/service-accounts",
"/api/v1/api-keys",
"/api/v1/auth/me",
"/api/v1/assistants", # assistants 是 LangGraph 兼容 shim,无 /api/v1 孪生:只放行 /api/assistants,缺 v1 变体是有意为之
"/api/langgraph/threads", # nginx 死代码:中间件本看不到,真混进来也应 deny
"/api/threads-export", # boundary guard — prefix must end at a path segment
"/api/runsX", # boundary guard — prefix must end at a path segment
],
)
def test_control_plane_paths_denied(path):
assert _is_dataplane_path(path) is False
# ---------------------------------------------------------------------------
# Integration tests: AuthMiddleware bearer default-deny (Task 3)
# ---------------------------------------------------------------------------
@pytest.fixture
def anyio_backend() -> str:
return "asyncio"
async def _seed_key(tmp_path, *, scopes="threads:read"):
from deerflow.persistence.api_key import ApiKeyRepository
from deerflow.persistence.engine import get_session_factory, init_engine
from deerflow.persistence.service_account.model import ServiceAccountRow
from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace.model import WorkspaceRow
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
sf = get_session_factory()
async with sf() as session:
session.add(UserRow(id="u-alice", email="alice@example.com"))
await session.commit()
async with sf() as session:
session.add(WorkspaceRow(id="w-1", name="WS", slug="ws", owner_id="u-alice"))
await session.commit()
async with sf() as session:
session.add(ServiceAccountRow(id="sa-1", workspace_id="w-1", name="bot", role="member", identity_mode="collapsed", status="active", created_by="u-alice"))
await session.commit()
repo = ApiKeyRepository(sf)
gen = generate_api_key("live")
await repo.create(service_account_id="sa-1", key_prefix=gen.prefix, key_hash=gen.key_hash, name="k", scopes=scopes)
return gen
async def _cleanup():
from deerflow.persistence.engine import close_engine
await close_engine()
def _make_app():
from fastapi import FastAPI
from app.gateway.auth_middleware import AuthMiddleware
from deerflow.runtime.user_context import get_effective_user_id
app = FastAPI()
app.add_middleware(AuthMiddleware)
# NOTE: Request must NOT be imported locally here. With `from __future__ import
# annotations` active, local imports are invisible to get_type_hints, causing
# FastAPI to treat `request: Request` as a query param → 422. Module-level
# import (above) makes it resolvable. See test_auth_middleware_api_key.py docstring.
@app.get("/api/v1/threads/_probe")
async def threads_probe(request: Request):
return {"user_id": get_effective_user_id()}
@app.get("/api/assistants/search")
async def assistants_probe():
return {"ok": True}
return app
async def test_sa_allowed_on_dataplane(tmp_path):
gen = await _seed_key(tmp_path)
try:
client = TestClient(_make_app())
r = client.get("/api/v1/threads/_probe", headers={"Authorization": f"Bearer {gen.plaintext}"})
assert r.status_code == 200
assert r.json() == {"user_id": "sa-1"}
finally:
await _cleanup()
async def test_sa_allowed_on_assistants_init(tmp_path):
gen = await _seed_key(tmp_path)
try:
client = TestClient(_make_app())
r = client.get("/api/assistants/search", headers={"Authorization": f"Bearer {gen.plaintext}"})
assert r.status_code == 200
finally:
await _cleanup()
@pytest.mark.parametrize(
"path",
[
"/api/v1/mcp/config",
"/api/mcp/config",
"/api/v1/models",
"/api/v1/skills/install",
"/api/v1/channels/restart",
"/api/v1/agents",
"/api/v1/memory",
"/api/v1/service-accounts",
],
)
async def test_sa_denied_on_control_plane(tmp_path, path):
gen = await _seed_key(tmp_path)
try:
client = TestClient(_make_app())
r = client.get(path, headers={"Authorization": f"Bearer {gen.plaintext}"})
assert r.status_code == 403
assert r.json()["detail"]["code"] == "insufficient_scope"
finally:
await _cleanup()
async def test_invalid_key_still_401_not_403(tmp_path):
# 无效 key 命中控制平面路径,应是 401 (TOKEN_INVALID),不是 403 ——
# deny 检查在 None 校验之后。
await _seed_key(tmp_path)
try:
client = TestClient(_make_app())
r = client.get("/api/v1/mcp/config", headers={"Authorization": "Bearer dfk_live_bogus00000000000000000"})
assert r.status_code == 401
finally:
await _cleanup()
async def test_cookie_path_unaffected_by_deny(tmp_path):
# 非 bearer-dfk 请求不进 bearer 分支:控制平面路径走 cookie 路径,
# 无 cookie → 401 not_authenticated,绝不会拿到 403 insufficient_scope。
await _seed_key(tmp_path)
try:
client = TestClient(_make_app())
r = client.get("/api/v1/mcp/config")
assert r.status_code == 401
assert r.json()["detail"]["code"] != "insufficient_scope"
finally:
await _cleanup()
+168
View File
@@ -0,0 +1,168 @@
"""Tests for ApiKeyRepository (Stage 1 PR1).
get_active_by_hash is the auth hot path: must return None for revoked
and expired keys. Expiry is filtered in Python (driver-agnostic) while
revoked_at IS NULL rides the partial index idx_api_keys_active.
"""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
import pytest
from deerflow.auth.tokens import generate_api_key
from deerflow.persistence.api_key import ApiKeyRepository
from deerflow.persistence.service_account.model import ServiceAccountRow
from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace.model import WorkspaceRow
pytestmark = pytest.mark.anyio
@pytest.fixture
def anyio_backend() -> str:
return "asyncio"
async def _make_repo(tmp_path):
from deerflow.persistence.engine import get_session_factory, init_engine
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
return ApiKeyRepository(get_session_factory())
async def _cleanup():
from deerflow.persistence.engine import close_engine
await close_engine()
async def _seed_sa(repo, *, sa_id="sa-1") -> None:
async with repo._sf() as session:
session.add(UserRow(id="u-alice", email="alice@example.com"))
await session.commit()
async with repo._sf() as session:
session.add(WorkspaceRow(id="w-1", name="WS", slug="ws", owner_id="u-alice"))
await session.commit()
async with repo._sf() as session:
session.add(ServiceAccountRow(id=sa_id, workspace_id="w-1", name="bot", role="member", identity_mode="collapsed", status="active", created_by="u-alice"))
await session.commit()
async def _mint(repo, *, expires_at=None, scopes="threads:read"):
gen = generate_api_key("live")
created = await repo.create(service_account_id="sa-1", key_prefix=gen.prefix, key_hash=gen.key_hash, name="k", scopes=scopes, expires_at=expires_at)
return gen, created
async def test_create_then_get_active_by_hash(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
gen, created = await _mint(repo)
assert created["key_prefix"] == gen.prefix
assert "key_hash" not in created # never expose the hash in dicts
found = await repo.get_active_by_hash(gen.key_hash, key_prefix=gen.prefix)
assert found is not None
assert found["id"] == created["id"]
assert found["scopes"] == "threads:read"
finally:
await _cleanup()
async def test_get_active_by_hash_miss_returns_none(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
assert await repo.get_active_by_hash("deadbeef", key_prefix="dfk_live_nomatch0") is None
finally:
await _cleanup()
async def test_wrong_hash_for_valid_prefix_returns_none(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
gen, _ = await _mint(repo)
assert await repo.get_active_by_hash("0" * 64, key_prefix=gen.prefix) is None
finally:
await _cleanup()
async def test_revoked_key_not_active(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
gen, created = await _mint(repo)
await repo.revoke(created["id"])
assert await repo.get_active_by_hash(gen.key_hash, key_prefix=gen.prefix) is None
finally:
await _cleanup()
async def test_expired_key_not_active(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
past = datetime.now(UTC) - timedelta(hours=1)
gen, _ = await _mint(repo, expires_at=past)
assert await repo.get_active_by_hash(gen.key_hash, key_prefix=gen.prefix) is None
finally:
await _cleanup()
async def test_future_expiry_still_active(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
future = datetime.now(UTC) + timedelta(hours=1)
gen, _ = await _mint(repo, expires_at=future)
assert await repo.get_active_by_hash(gen.key_hash, key_prefix=gen.prefix) is not None
finally:
await _cleanup()
async def test_touch_last_used_sets_timestamp(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
gen, created = await _mint(repo)
assert created["last_used_at"] is None
await repo.touch_last_used(created["id"])
refetched = await repo.get(created["id"])
assert refetched["last_used_at"] is not None
finally:
await _cleanup()
async def test_list_by_service_account(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
await _mint(repo)
await _mint(repo)
rows = await repo.list_by_service_account("sa-1")
assert len(rows) == 2
assert all("key_hash" not in r for r in rows)
finally:
await _cleanup()
async def test_list_by_service_account_excludes_other_sa(tmp_path):
repo = await _make_repo(tmp_path)
try:
await _seed_sa(repo)
async with repo._sf() as session:
from deerflow.persistence.service_account.model import ServiceAccountRow
session.add(ServiceAccountRow(id="sa-2", workspace_id="w-1", name="bot2", role="member", identity_mode="collapsed", status="active", created_by="u-alice"))
await session.commit()
await _mint(repo) # belongs to sa-1
g2 = generate_api_key("live")
await repo.create(service_account_id="sa-2", key_prefix=g2.prefix, key_hash=g2.key_hash, name="k2", scopes="")
rows = await repo.list_by_service_account("sa-1")
assert len(rows) == 1
finally:
await _cleanup()
+156
View File
@@ -0,0 +1,156 @@
"""Schema tests for ``ApiKeyRow`` (Stage 0 PR8).
PR8 is schema-only no repository class, no API. Tests exercise raw
ORM behaviour: column-level UNIQUE on key_prefix, the dual-dialect
partial index DDL (sqlite_where + postgresql_where), and CASCADE on
service_account delete.
"""
from __future__ import annotations
from datetime import UTC, datetime
import pytest
from sqlalchemy import delete
from sqlalchemy.exc import IntegrityError
from deerflow.persistence.api_key import ApiKeyRow
from deerflow.persistence.service_account import ServiceAccountRow
from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace.model import WorkspaceRow
pytestmark = pytest.mark.anyio
@pytest.fixture
def anyio_backend() -> str:
return "asyncio"
async def _setup(tmp_path):
from deerflow.persistence.engine import get_session_factory, init_engine
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
return get_session_factory()
async def _cleanup():
from deerflow.persistence.engine import close_engine
await close_engine()
async def _seed_parents(sf) -> None:
now = datetime.now(UTC)
async with sf() as session:
session.add(UserRow(id="u-alice", email="alice@example.com"))
await session.commit()
async with sf() as session:
session.add(WorkspaceRow(id="w-1", name="Alice WS", slug="alice", owner_id="u-alice"))
await session.commit()
async with sf() as session:
session.add(
ServiceAccountRow(
id="sa-1",
workspace_id="w-1",
name="bot",
role="member",
identity_mode="collapsed",
status="active",
created_by="u-alice",
created_at=now,
updated_at=now,
)
)
await session.commit()
def _make_key(*, key_id: str, prefix: str, revoked_at: datetime | None = None) -> ApiKeyRow:
now = datetime.now(UTC)
return ApiKeyRow(
id=key_id,
service_account_id="sa-1",
key_prefix=prefix,
key_hash="0" * 64,
name="default",
scopes="",
rate_limit_rpm=None,
expires_at=None,
last_used_at=None,
revoked_at=revoked_at,
created_at=now,
)
# ---------------------------------------------------------------------------
# T8.4-1 — column-level UNIQUE on key_prefix
# ---------------------------------------------------------------------------
async def test_unique_key_prefix_enforced(tmp_path):
"""Two api_key rows sharing the same key_prefix raise IntegrityError."""
sf = await _setup(tmp_path)
try:
await _seed_parents(sf)
async with sf() as session:
session.add(_make_key(key_id="ak-1", prefix="dfk_live_abc12345"))
await session.commit()
with pytest.raises(IntegrityError):
async with sf() as session:
session.add(_make_key(key_id="ak-2", prefix="dfk_live_abc12345"))
await session.commit()
finally:
await _cleanup()
# ---------------------------------------------------------------------------
# T8.4-2 — partial index DDL covers both SQLite and Postgres
# ---------------------------------------------------------------------------
def test_active_index_declares_both_dialect_where_clauses():
"""idx_api_keys_active must compile to a partial index on both drivers.
The plan locks ``sqlite_where`` *and* ``postgresql_where`` so the same
Index emits a partial index regardless of backend (Stage 0 dev still
runs SQLite locally; production is Postgres). Both ``dialect_options``
entries must be present.
"""
active_idx = next((idx for idx in ApiKeyRow.__table__.indexes if idx.name == "idx_api_keys_active"), None)
assert active_idx is not None, "idx_api_keys_active not declared"
sqlite_where = active_idx.dialect_options.get("sqlite", {}).get("where")
postgres_where = active_idx.dialect_options.get("postgresql", {}).get("where")
assert sqlite_where is not None, "sqlite_where missing on idx_api_keys_active"
assert postgres_where is not None, "postgresql_where missing on idx_api_keys_active"
assert "revoked_at IS NULL" in str(sqlite_where)
assert "revoked_at IS NULL" in str(postgres_where)
# ---------------------------------------------------------------------------
# T8.4-3 — CASCADE on service_account delete
# ---------------------------------------------------------------------------
async def test_cascade_on_service_account_delete(tmp_path):
"""Deleting the parent service_account removes all child api_keys."""
sf = await _setup(tmp_path)
try:
await _seed_parents(sf)
async with sf() as session:
session.add(_make_key(key_id="ak-cascade-1", prefix="dfk_live_cascade1"))
session.add(_make_key(key_id="ak-cascade-2", prefix="dfk_live_cascade2"))
await session.commit()
async with sf() as session:
await session.execute(delete(ServiceAccountRow).where(ServiceAccountRow.id == "sa-1"))
await session.commit()
async with sf() as session:
row1 = await session.get(ApiKeyRow, "ak-cascade-1")
row2 = await session.get(ApiKeyRow, "ak-cascade-2")
assert row1 is None
assert row2 is None
finally:
await _cleanup()
+175
View File
@@ -0,0 +1,175 @@
"""api-keys router tests (Stage 1 PR4).
plaintext is returned exactly once at create time; never on list.
"""
from __future__ import annotations
import pytest
from starlette.testclient import TestClient
pytestmark = pytest.mark.anyio
@pytest.fixture
def anyio_backend() -> str:
return "asyncio"
async def _init_db_with_sa(tmp_path, *, sa_id="sa-1", workspace_id="w-1"):
from deerflow.persistence.engine import get_session_factory, init_engine
from deerflow.persistence.service_account.model import ServiceAccountRow
from deerflow.persistence.user.model import UserRow
from deerflow.persistence.workspace.model import WorkspaceRow
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
sf = get_session_factory()
async with sf() as session:
session.add(UserRow(id="u-alice", email="alice@example.com"))
await session.commit()
async with sf() as session:
session.add(WorkspaceRow(id=workspace_id, name="WS", slug=f"ws-{workspace_id}", owner_id="u-alice"))
await session.commit()
async with sf() as session:
session.add(ServiceAccountRow(id=sa_id, workspace_id=workspace_id, name="bot", role="member", identity_mode="collapsed", status="active", created_by="u-alice"))
await session.commit()
async def _cleanup():
from deerflow.persistence.engine import close_engine
await close_engine()
def _make_app(*, role="owner", workspace_id="w-1"):
from fastapi import FastAPI, Request
from starlette.middleware.base import BaseHTTPMiddleware
from app.gateway.authz import _ALL_PERMISSIONS, AuthContext
from app.gateway.routers import api_keys
from deerflow.runtime.user_context import reset_current_user, set_current_user
from deerflow.runtime.workspace_context import reset_current_workspace, set_current_workspace
class _Stamp(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
user = type("U", (), {"id": "u-alice", "is_service_account": False})()
ws = type("W", (), {"id": workspace_id, "role": role})()
request.state.user = user
request.state.auth = AuthContext(user=user, permissions=_ALL_PERMISSIONS)
ut = set_current_user(user)
wt = set_current_workspace(ws)
try:
return await call_next(request)
finally:
reset_current_workspace(wt)
reset_current_user(ut)
app = FastAPI()
app.add_middleware(_Stamp)
app.include_router(api_keys.router)
return app
async def test_create_returns_plaintext_once(tmp_path):
await _init_db_with_sa(tmp_path)
try:
client = TestClient(_make_app())
r = client.post("/api/v1/api-keys", json={"service_account_id": "sa-1", "name": "ci", "scopes": "threads:read"})
assert r.status_code == 201, r.text
body = r.json()
assert body["plaintext"].startswith("dfk_live_")
assert body["key_prefix"] == body["plaintext"][:16]
lst = client.get("/api/v1/api-keys", params={"service_account_id": "sa-1"})
assert lst.status_code == 200
rows = lst.json()
assert len(rows) == 1
assert "plaintext" not in rows[0]
assert "key_hash" not in rows[0]
finally:
await _cleanup()
async def test_create_for_other_workspace_sa_404(tmp_path):
await _init_db_with_sa(tmp_path, sa_id="sa-1", workspace_id="w-1")
try:
# Caller is in w-2 but targets sa-1 which lives in w-1 → 404.
client = TestClient(_make_app(workspace_id="w-2"))
r = client.post("/api/v1/api-keys", json={"service_account_id": "sa-1", "name": "x", "scopes": ""})
assert r.status_code == 404
finally:
await _cleanup()
async def test_revoke_key(tmp_path):
await _init_db_with_sa(tmp_path)
try:
client = TestClient(_make_app())
created = client.post("/api/v1/api-keys", json={"service_account_id": "sa-1", "name": "k", "scopes": ""}).json()
r = client.delete(f"/api/v1/api-keys/{created['id']}")
assert r.status_code == 204
rows = client.get("/api/v1/api-keys", params={"service_account_id": "sa-1"}).json()
assert rows[0]["revoked_at"] is not None
finally:
await _cleanup()
async def test_member_cannot_create_key(tmp_path):
await _init_db_with_sa(tmp_path)
try:
client = TestClient(_make_app(role="member"))
r = client.post("/api/v1/api-keys", json={"service_account_id": "sa-1", "name": "x", "scopes": ""})
assert r.status_code == 403
finally:
await _cleanup()
async def test_revoke_other_workspace_key_404(tmp_path):
await _init_db_with_sa(tmp_path, sa_id="sa-1", workspace_id="w-1")
try:
client_a = TestClient(_make_app(workspace_id="w-1"))
created = client_a.post("/api/v1/api-keys", json={"service_account_id": "sa-1", "name": "k", "scopes": ""}).json()
client_b = TestClient(_make_app(workspace_id="w-2"))
assert client_b.delete(f"/api/v1/api-keys/{created['id']}").status_code == 404
finally:
await _cleanup()
async def test_list_other_workspace_sa_404(tmp_path):
await _init_db_with_sa(tmp_path, sa_id="sa-1", workspace_id="w-1")
try:
client_b = TestClient(_make_app(workspace_id="w-2"))
assert client_b.get("/api/v1/api-keys", params={"service_account_id": "sa-1"}).status_code == 404
finally:
await _cleanup()
async def test_create_for_suspended_sa_409(tmp_path):
await _init_db_with_sa(tmp_path)
try:
from deerflow.persistence.engine import get_session_factory
from deerflow.persistence.service_account import ServiceAccountRepository
await ServiceAccountRepository(get_session_factory()).update_status("sa-1", "suspended")
client = TestClient(_make_app())
r = client.post("/api/v1/api-keys", json={"service_account_id": "sa-1", "name": "x", "scopes": ""})
assert r.status_code == 409
finally:
await _cleanup()
async def test_revoke_404_bodies_are_indistinguishable(tmp_path):
await _init_db_with_sa(tmp_path, sa_id="sa-1", workspace_id="w-1")
try:
client_a = TestClient(_make_app(workspace_id="w-1"))
created = client_a.post("/api/v1/api-keys", json={"service_account_id": "sa-1", "name": "k", "scopes": ""}).json()
client_b = TestClient(_make_app(workspace_id="w-2"))
# cross-workspace existing key, and a non-existent key, must return identical 404 bodies
cross = client_b.delete(f"/api/v1/api-keys/{created['id']}")
missing = client_b.delete("/api/v1/api-keys/does-not-exist")
assert cross.status_code == 404
assert missing.status_code == 404
assert cross.json() == missing.json()
finally:
await _cleanup()

Some files were not shown because too many files have changed in this diff Show More