Compare commits
91 Commits
main
..
84de632b19
| Author | SHA1 | Date | |
|---|---|---|---|
| 84de632b19 | |||
| 6090b9e5b1 | |||
| f803f393d3 | |||
| 6f806ff4a4 | |||
| 52e9999a61 | |||
| bb7289781e | |||
| 1fb07e48e6 | |||
| b0bf033f15 | |||
| d8b13afc59 | |||
| d0f1877070 | |||
| ba30d14041 | |||
| 1a6ccc9aaa | |||
| 4c0b4fab48 | |||
| 87ea715c2a | |||
| c5c66ccbfc | |||
| 56f2c8d873 | |||
| 2d3b546bf9 | |||
| f013fc1a65 | |||
| a7ecd76e0a | |||
| f6a922921b | |||
| 0456606dc1 | |||
| b4fa3bf12a | |||
| 05be7f9ad0 | |||
| 28ad6c2b0b | |||
| 296a4f1950 | |||
| 361e653d37 | |||
| 430f4a1132 | |||
| 30f2bd0084 | |||
| 73d0b7017b | |||
| 8a03abac75 | |||
| def45dd0c6 | |||
| 56f6572086 | |||
| e6bb220979 | |||
| ad322543d0 | |||
| d3361dba59 | |||
| 4e26ec9884 | |||
| a732697855 | |||
| 44f84800e9 | |||
| 5c7753c0b8 | |||
| 91846a201e | |||
| 634e5119e1 | |||
| 84701730da | |||
| a657d17995 | |||
| a06e88d58b | |||
| 2145d36744 | |||
| b4bef65079 | |||
| 54cb94c30f | |||
| 54762f491c | |||
| 917d8fbeaf | |||
| 8efbb2f9e5 | |||
| d98498b705 | |||
| c70c6594de | |||
| a592319e3c | |||
| dda8264057 | |||
| 3313047ff1 | |||
| 36ffe2713f | |||
| a40df03521 | |||
| d2d2d29c34 | |||
| 8323bf68d2 | |||
| f63089aea8 | |||
| cbbb83a706 | |||
| 39f8e117e8 | |||
| bdef6c6b9f | |||
| 7f17cb8fee | |||
| ad22242ecc | |||
| 1112a1971b | |||
| c53295dfee | |||
| 745a33e05d | |||
| 83b680b2ea | |||
| 3e62a0f6ef | |||
| d312bdf968 | |||
| 7d3d3560ad | |||
| 404135a16a | |||
| 85a14f4c05 | |||
| a33b46b4af | |||
| eae0190184 | |||
| e6f5ba53bd | |||
| 8f480cd76f | |||
| 31361e2dcf | |||
| ae4ea46be2 | |||
| fab85b14a6 | |||
| a74b88a45c | |||
| 89fe54cc07 | |||
| 9ff790554d | |||
| ecc9339ede | |||
| fd8d0d637d | |||
| 35ae97d0a6 | |||
| e78ed687a6 | |||
| 8bdd308ea0 | |||
| 27c4f14233 | |||
| dce5e9598b |
+5
-2
@@ -38,8 +38,11 @@ INFOQUEST_API_KEY=your-infoquest-api-key
|
|||||||
# GitHub API Token
|
# GitHub API Token
|
||||||
# GITHUB_TOKEN=your-github-token
|
# GITHUB_TOKEN=your-github-token
|
||||||
|
|
||||||
# Database (only needed when config.yaml has database.backend: postgres)
|
# Database (Stage 0+ default; required when config.yaml has database.backend: postgres)
|
||||||
# DATABASE_URL=postgresql://deerflow:password@localhost:5432/deerflow
|
# Local dev — start with: docker compose -f docker/docker-compose-dev.yaml up -d postgres
|
||||||
|
DATABASE_URL=postgresql+asyncpg://deerflow:deerflow_dev@localhost:5432/deerflow
|
||||||
|
# Remote RDS example:
|
||||||
|
# DATABASE_URL=postgresql+asyncpg://USER:PASSWORD@HOST:5432/DBNAME
|
||||||
#
|
#
|
||||||
# WECOM_BOT_ID=your-wecom-bot-id
|
# WECOM_BOT_ID=your-wecom-bot-id
|
||||||
# WECOM_BOT_SECRET=your-wecom-bot-secret
|
# WECOM_BOT_SECRET=your-wecom-bot-secret
|
||||||
|
|||||||
@@ -0,0 +1,56 @@
|
|||||||
|
name: Postgres Tests
|
||||||
|
|
||||||
|
# Stage 0 PR1 · runs the @pytest.mark.postgres subset against a real
|
||||||
|
# Postgres backend (testcontainers spawns postgres:16-alpine on the
|
||||||
|
# GitHub runner's docker daemon).
|
||||||
|
#
|
||||||
|
# Kept as a separate workflow from `Unit Tests` so:
|
||||||
|
# - the existing fast unit test loop is unchanged
|
||||||
|
# - PG tests can fail-soft during early Stage 0 rollout if needed
|
||||||
|
# (set continue-on-error: true on the run step)
|
||||||
|
# - infra cost is opt-in for forks
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [ 'main' ]
|
||||||
|
pull_request:
|
||||||
|
types: [opened, synchronize, reopened, ready_for_review]
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: postgres-tests-${{ github.event.pull_request.number || github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
backend-postgres-tests:
|
||||||
|
if: github.event.pull_request.draft == false
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 20
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
uses: actions/checkout@v6
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v6
|
||||||
|
with:
|
||||||
|
python-version: '3.12'
|
||||||
|
|
||||||
|
- name: Install uv
|
||||||
|
uses: astral-sh/setup-uv@v7
|
||||||
|
|
||||||
|
- name: Verify Docker daemon (testcontainers needs it)
|
||||||
|
run: docker version
|
||||||
|
|
||||||
|
- name: Install backend dependencies (with postgres-test extra)
|
||||||
|
working-directory: backend
|
||||||
|
run: uv sync --group dev --extra postgres-test
|
||||||
|
|
||||||
|
- name: Run @pytest.mark.postgres tests
|
||||||
|
working-directory: backend
|
||||||
|
# -m postgres selects only postgres-tagged tests; testcontainers
|
||||||
|
# spins up postgres:16-alpine as the fixture's session-scoped
|
||||||
|
# container. ~5-10s container startup + per-test DB CREATE/DROP.
|
||||||
|
run: PYTHONPATH=. uv run pytest -m postgres -v
|
||||||
@@ -60,3 +60,6 @@ config.yaml.bak
|
|||||||
/frontend/playwright-report/
|
/frontend/playwright-report/
|
||||||
.gstack/
|
.gstack/
|
||||||
.worktrees
|
.worktrees
|
||||||
|
skills/gstack
|
||||||
|
skills/superpowers
|
||||||
|
CLAUDE.md
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ help:
|
|||||||
@echo " make start-daemon - Start prod services in background (daemon mode)"
|
@echo " make start-daemon - Start prod services in background (daemon mode)"
|
||||||
@echo " make stop - Stop all running services"
|
@echo " make stop - Stop all running services"
|
||||||
@echo " make clean - Clean up processes and temporary files"
|
@echo " make clean - Clean up processes and temporary files"
|
||||||
|
@echo " make migrate-paths - Migrate legacy users/ tree into workspaces/ layout (DRY_RUN=1 to preview)"
|
||||||
@echo ""
|
@echo ""
|
||||||
@echo "Docker Production Commands:"
|
@echo "Docker Production Commands:"
|
||||||
@echo " make up - Build and start production Docker services (localhost:2026)"
|
@echo " make up - Build and start production Docker services (localhost:2026)"
|
||||||
@@ -144,6 +145,14 @@ clean: stop
|
|||||||
@-rm -rf logs/*.log 2>/dev/null || true
|
@-rm -rf logs/*.log 2>/dev/null || true
|
||||||
@echo "✓ Cleanup complete"
|
@echo "✓ Cleanup complete"
|
||||||
|
|
||||||
|
# Lift legacy per-user paths into the per-workspace layout (PR6).
|
||||||
|
# Pass DRY_RUN=1 to log the migration plan without writing.
|
||||||
|
# DEFAULT_WORKSPACE=<wid> claims un-assigned users (defaults to legacy_workspace).
|
||||||
|
migrate-paths:
|
||||||
|
@cd backend && PYTHONPATH=. uv run python scripts/migrate_paths_to_workspace.py \
|
||||||
|
$(if $(filter 1 true,$(DRY_RUN)),--dry-run) \
|
||||||
|
$(if $(DEFAULT_WORKSPACE),--default-workspace $(DEFAULT_WORKSPACE))
|
||||||
|
|
||||||
# ==========================================
|
# ==========================================
|
||||||
# Docker Development Commands
|
# Docker Development Commands
|
||||||
# ==========================================
|
# ==========================================
|
||||||
|
|||||||
@@ -202,6 +202,38 @@ That prompt is intended for coding agents. It tells the agent to clone the repo
|
|||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
|
3. **Database backend (Stage 0+ defaults to Postgres)**
|
||||||
|
|
||||||
|
`config.example.yaml` ships with `database.backend: postgres` and `postgres_url: $DATABASE_URL`. Set `DATABASE_URL` in `.env`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
DATABASE_URL=postgresql+asyncpg://deerflow:deerflow_dev@localhost:5432/deerflow
|
||||||
|
```
|
||||||
|
|
||||||
|
Start the local Postgres dev container:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker/docker-compose-dev.yaml up -d postgres
|
||||||
|
```
|
||||||
|
|
||||||
|
Or point `DATABASE_URL` at a remote RDS / Cloud SQL instance.
|
||||||
|
|
||||||
|
`make doctor` will report the configured backend, attempt an asyncpg connection, and surface actionable fix hints. `make dev` preflights Postgres reachability before starting services and aborts if `DATABASE_URL` is unreachable.
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>Offline dev (SQLite fallback)</summary>
|
||||||
|
|
||||||
|
If you prefer no Postgres, edit `config.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
database:
|
||||||
|
backend: sqlite
|
||||||
|
sqlite_dir: .deer-flow/data
|
||||||
|
```
|
||||||
|
|
||||||
|
SQLite is preserved as a valid backend for offline development. RLS / multi-node features (Stage 2+) require Postgres.
|
||||||
|
</details>
|
||||||
|
|
||||||
### Running the Application
|
### Running the Application
|
||||||
|
|
||||||
#### Deployment Sizing
|
#### Deployment Sizing
|
||||||
|
|||||||
@@ -102,6 +102,8 @@ Regression tests related to Docker/provisioner behavior:
|
|||||||
|
|
||||||
Boundary check (harness → app import firewall):
|
Boundary check (harness → app import firewall):
|
||||||
- `tests/test_harness_boundary.py` — ensures `packages/harness/deerflow/` never imports from `app.*`
|
- `tests/test_harness_boundary.py` — ensures `packages/harness/deerflow/` never imports from `app.*`
|
||||||
|
- `tests/test_workspace_boundary.py` — AST static scan that forbids direct imports of `langgraph.checkpoint.*` (and the third-party `langgraph_checkpoint_postgres` / `langgraph_checkpoint_sqlite` packages) outside the allowlist in `tests/boundary_allowlist.toml`. Everywhere else must obtain a checkpointer via `app.gateway.deps.get_checkpointer` or the harness `deerflow.runtime.checkpointer` factory. Imports inside `if TYPE_CHECKING:` blocks are exempt automatically (they do not enter runtime). When a legitimate new importer is genuinely needed, append its path to `boundary_allowlist.toml` in the same PR
|
||||||
|
- `tests/test_workspace_boundary_self.py` — self-tests for the scanner above (9 cases over synthetic `.py` files) guarding against silent-empty regressions
|
||||||
|
|
||||||
CI runs these regression tests for every pull request via [.github/workflows/backend-unit-tests.yml](../.github/workflows/backend-unit-tests.yml).
|
CI runs these regression tests for every pull request via [.github/workflows/backend-unit-tests.yml](../.github/workflows/backend-unit-tests.yml).
|
||||||
|
|
||||||
|
|||||||
@@ -108,6 +108,23 @@ async def _ensure_admin_user(app: FastAPI) -> None:
|
|||||||
|
|
||||||
admin_id = str(row.id)
|
admin_id = str(row.id)
|
||||||
|
|
||||||
|
# Stage 0 PR4 backfill: pre-PR4 admins have no default_workspace_id.
|
||||||
|
# Create their personal workspace + owner membership on next boot so
|
||||||
|
# they can log in and pass the workspace gate without hand-rolling
|
||||||
|
# SQL. Idempotent — ensure_default_workspace short-circuits when the
|
||||||
|
# column is already set.
|
||||||
|
try:
|
||||||
|
admin_user = await provider.get_user(admin_id)
|
||||||
|
if admin_user is not None and not admin_user.default_workspace_id:
|
||||||
|
from app.gateway.routers.auth import ensure_default_workspace
|
||||||
|
|
||||||
|
ws_id = await ensure_default_workspace(admin_user)
|
||||||
|
logger.info("Backfilled default workspace %s for admin %s", ws_id, admin_id)
|
||||||
|
except Exception:
|
||||||
|
# Don't fail startup if backfill stumbles — the user can still
|
||||||
|
# log in (login_local calls the same helper on its hot path).
|
||||||
|
logger.exception("Admin workspace backfill failed (non-fatal)")
|
||||||
|
|
||||||
# LangGraph store orphan migration — non-fatal.
|
# LangGraph store orphan migration — non-fatal.
|
||||||
# This covers the "no-auth → with-auth" upgrade path for users
|
# This covers the "no-auth → with-auth" upgrade path for users
|
||||||
# whose existing LangGraph thread metadata has no user_id set.
|
# whose existing LangGraph thread metadata has no user_id set.
|
||||||
@@ -158,6 +175,34 @@ async def _migrate_orphaned_threads(store, admin_user_id: str) -> int:
|
|||||||
return migrated
|
return migrated
|
||||||
|
|
||||||
|
|
||||||
|
def _check_path_migration_pending(app: FastAPI) -> None:
|
||||||
|
"""Warn the operator if the PR4 legacy user-isolation layout still has content.
|
||||||
|
|
||||||
|
PR6 routes every new write into ``{base_dir}/workspaces/{wid}/...`` via
|
||||||
|
``Paths``. Pre-PR6 installations have data at
|
||||||
|
``{base_dir}/users/{uid}/...`` that needs ``make migrate-paths`` to lift
|
||||||
|
it under a workspace. We emit a warning at boot rather than crashing so
|
||||||
|
the gateway keeps serving (reads from the legacy tree still work via the
|
||||||
|
user_id branch of ``Paths.thread_dir``), but with a loud signal to run
|
||||||
|
the migration script.
|
||||||
|
"""
|
||||||
|
from deerflow.config.paths import get_paths
|
||||||
|
|
||||||
|
legacy_users = get_paths().base_dir / "users"
|
||||||
|
if not legacy_users.exists():
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
has_content = any(legacy_users.iterdir())
|
||||||
|
except OSError:
|
||||||
|
# Permission or transient FS issue — don't escalate; lifespan must succeed.
|
||||||
|
return
|
||||||
|
if has_content:
|
||||||
|
logger.warning(
|
||||||
|
"Legacy per-user layout detected at %s. Run `make migrate-paths` to lift it under the per-workspace layout (PR6).",
|
||||||
|
legacy_users,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
@asynccontextmanager
|
||||||
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
|
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
|
||||||
"""Application lifespan handler."""
|
"""Application lifespan handler."""
|
||||||
@@ -174,6 +219,8 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
|
|||||||
config = get_gateway_config()
|
config = get_gateway_config()
|
||||||
logger.info(f"Starting API Gateway on {config.host}:{config.port}")
|
logger.info(f"Starting API Gateway on {config.host}:{config.port}")
|
||||||
|
|
||||||
|
_check_path_migration_pending(app)
|
||||||
|
|
||||||
# Initialize LangGraph runtime components (StreamBridge, RunManager, checkpointer, store)
|
# Initialize LangGraph runtime components (StreamBridge, RunManager, checkpointer, store)
|
||||||
async with langgraph_runtime(app):
|
async with langgraph_runtime(app):
|
||||||
logger.info("LangGraph runtime initialised")
|
logger.info("LangGraph runtime initialised")
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ class AuthErrorCode(StrEnum):
|
|||||||
PROVIDER_NOT_FOUND = "provider_not_found"
|
PROVIDER_NOT_FOUND = "provider_not_found"
|
||||||
NOT_AUTHENTICATED = "not_authenticated"
|
NOT_AUTHENTICATED = "not_authenticated"
|
||||||
SYSTEM_ALREADY_INITIALIZED = "system_already_initialized"
|
SYSTEM_ALREADY_INITIALIZED = "system_already_initialized"
|
||||||
|
WORKSPACE_REQUIRED = "workspace_required"
|
||||||
|
|
||||||
|
|
||||||
class TokenError(StrEnum):
|
class TokenError(StrEnum):
|
||||||
@@ -29,6 +30,7 @@ class TokenError(StrEnum):
|
|||||||
EXPIRED = "expired"
|
EXPIRED = "expired"
|
||||||
INVALID_SIGNATURE = "invalid_signature"
|
INVALID_SIGNATURE = "invalid_signature"
|
||||||
MALFORMED = "malformed"
|
MALFORMED = "malformed"
|
||||||
|
WORKSPACE_MISSING = "workspace_missing"
|
||||||
|
|
||||||
|
|
||||||
class AuthErrorResponse(BaseModel):
|
class AuthErrorResponse(BaseModel):
|
||||||
@@ -42,4 +44,6 @@ def token_error_to_code(err: TokenError) -> AuthErrorCode:
|
|||||||
"""Map TokenError to AuthErrorCode — single source of truth."""
|
"""Map TokenError to AuthErrorCode — single source of truth."""
|
||||||
if err == TokenError.EXPIRED:
|
if err == TokenError.EXPIRED:
|
||||||
return AuthErrorCode.TOKEN_EXPIRED
|
return AuthErrorCode.TOKEN_EXPIRED
|
||||||
|
if err == TokenError.WORKSPACE_MISSING:
|
||||||
|
return AuthErrorCode.WORKSPACE_REQUIRED
|
||||||
return AuthErrorCode.TOKEN_INVALID
|
return AuthErrorCode.TOKEN_INVALID
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
"""JWT token creation and verification."""
|
"""JWT token creation and verification."""
|
||||||
|
|
||||||
from datetime import UTC, datetime, timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
import jwt
|
import jwt
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
@@ -10,30 +11,51 @@ from app.gateway.auth.errors import TokenError
|
|||||||
|
|
||||||
|
|
||||||
class TokenPayload(BaseModel):
|
class TokenPayload(BaseModel):
|
||||||
"""JWT token payload."""
|
"""JWT token payload.
|
||||||
|
|
||||||
|
`wid` / `role` were added in Stage 0 PR4 and are optional at the
|
||||||
|
model level so legacy 4-field tokens still parse into a value —
|
||||||
|
callers (middleware, decode_token) decide what "missing wid" means.
|
||||||
|
Post-PR4 production tokens always carry both fields.
|
||||||
|
"""
|
||||||
|
|
||||||
sub: str # user_id
|
sub: str # user_id
|
||||||
|
wid: str | None = None # workspace_id (Stage 0 PR4)
|
||||||
|
role: str | None = None # owner / admin / member (Stage 0 PR4)
|
||||||
exp: datetime
|
exp: datetime
|
||||||
iat: datetime | None = None
|
iat: datetime | None = None
|
||||||
ver: int = 0 # token_version — must match User.token_version
|
ver: int = 0 # token_version — must match User.token_version
|
||||||
|
|
||||||
|
|
||||||
def create_access_token(user_id: str, expires_delta: timedelta | None = None, token_version: int = 0) -> str:
|
def create_access_token(
|
||||||
|
user_id: str,
|
||||||
|
expires_delta: timedelta | None = None,
|
||||||
|
token_version: int = 0,
|
||||||
|
*,
|
||||||
|
workspace_id: str | None = None,
|
||||||
|
role: str | None = None,
|
||||||
|
) -> str:
|
||||||
"""Create a JWT access token.
|
"""Create a JWT access token.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
user_id: The user's UUID as string
|
user_id: The user's UUID as string.
|
||||||
expires_delta: Optional custom expiry, defaults to 7 days
|
expires_delta: Optional custom expiry, defaults to 7 days.
|
||||||
token_version: User's current token_version for invalidation
|
token_version: User's current token_version for invalidation.
|
||||||
|
workspace_id: Optional active workspace id; encoded as the ``wid`` claim.
|
||||||
|
role: Optional workspace role; encoded as the ``role`` claim.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Encoded JWT string
|
Encoded JWT string.
|
||||||
"""
|
"""
|
||||||
config = get_auth_config()
|
config = get_auth_config()
|
||||||
expiry = expires_delta or timedelta(days=config.token_expiry_days)
|
expiry = expires_delta or timedelta(days=config.token_expiry_days)
|
||||||
|
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
payload = {"sub": user_id, "exp": now + expiry, "iat": now, "ver": token_version}
|
payload: dict[str, Any] = {"sub": user_id, "exp": now + expiry, "iat": now, "ver": token_version}
|
||||||
|
if workspace_id is not None:
|
||||||
|
payload["wid"] = workspace_id
|
||||||
|
if role is not None:
|
||||||
|
payload["role"] = role
|
||||||
return jwt.encode(payload, config.jwt_secret, algorithm="HS256")
|
return jwt.encode(payload, config.jwt_secret, algorithm="HS256")
|
||||||
|
|
||||||
|
|
||||||
@@ -46,10 +68,20 @@ def decode_token(token: str) -> TokenPayload | TokenError:
|
|||||||
config = get_auth_config()
|
config = get_auth_config()
|
||||||
try:
|
try:
|
||||||
payload = jwt.decode(token, config.jwt_secret, algorithms=["HS256"])
|
payload = jwt.decode(token, config.jwt_secret, algorithms=["HS256"])
|
||||||
return TokenPayload(**payload)
|
|
||||||
except jwt.ExpiredSignatureError:
|
except jwt.ExpiredSignatureError:
|
||||||
return TokenError.EXPIRED
|
return TokenError.EXPIRED
|
||||||
except jwt.InvalidSignatureError:
|
except jwt.InvalidSignatureError:
|
||||||
return TokenError.INVALID_SIGNATURE
|
return TokenError.INVALID_SIGNATURE
|
||||||
except jwt.PyJWTError:
|
except jwt.PyJWTError:
|
||||||
return TokenError.MALFORMED
|
return TokenError.MALFORMED
|
||||||
|
|
||||||
|
# Reject legacy pre-PR4 tokens that lack the wid claim. Reported as
|
||||||
|
# WORKSPACE_MISSING (not MALFORMED) so middleware can surface a
|
||||||
|
# specific 401 telling the frontend to re-issue via /select-workspace.
|
||||||
|
if "wid" not in payload or payload.get("wid") is None:
|
||||||
|
return TokenError.WORKSPACE_MISSING
|
||||||
|
|
||||||
|
try:
|
||||||
|
return TokenPayload(**payload)
|
||||||
|
except Exception:
|
||||||
|
return TokenError.MALFORMED
|
||||||
|
|||||||
@@ -31,6 +31,12 @@ class User(BaseModel):
|
|||||||
needs_setup: bool = Field(default=False, description="True for auto-created admin until setup completes")
|
needs_setup: bool = Field(default=False, description="True for auto-created admin until setup completes")
|
||||||
token_version: int = Field(default=0, description="Incremented on password change to invalidate old JWTs")
|
token_version: int = Field(default=0, description="Incremented on password change to invalidate old JWTs")
|
||||||
|
|
||||||
|
# Workspace linkage (Stage 0 PR4)
|
||||||
|
default_workspace_id: str | None = Field(
|
||||||
|
default=None,
|
||||||
|
description="The workspace the user lands in by default after login. NULL → /select-workspace.",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class UserResponse(BaseModel):
|
class UserResponse(BaseModel):
|
||||||
"""Response model for user info endpoint."""
|
"""Response model for user info endpoint."""
|
||||||
@@ -39,3 +45,38 @@ class UserResponse(BaseModel):
|
|||||||
email: str
|
email: str
|
||||||
system_role: Literal["admin", "user"]
|
system_role: Literal["admin", "user"]
|
||||||
needs_setup: bool = False
|
needs_setup: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class UserMeWorkspace(BaseModel):
|
||||||
|
"""One workspace entry in ``GET /auth/me`` (Stage 0 PR4)."""
|
||||||
|
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
slug: str
|
||||||
|
role: str
|
||||||
|
|
||||||
|
|
||||||
|
class UserMeResponse(BaseModel):
|
||||||
|
"""Response model for ``GET /auth/me`` — extends UserResponse with workspaces."""
|
||||||
|
|
||||||
|
id: str
|
||||||
|
email: str
|
||||||
|
system_role: Literal["admin", "user"]
|
||||||
|
needs_setup: bool = False
|
||||||
|
default_workspace_id: str | None = None
|
||||||
|
workspaces: list[UserMeWorkspace] = []
|
||||||
|
|
||||||
|
|
||||||
|
class ActiveWorkspace(BaseModel):
|
||||||
|
"""Lightweight workspace proxy injected into the request-scoped contextvar.
|
||||||
|
|
||||||
|
Implements the structural ``CurrentWorkspace`` protocol expected by
|
||||||
|
``deerflow.runtime.workspace_context``: only ``.id`` (str) and
|
||||||
|
``.role`` (str) are required. We intentionally do *not* embed the
|
||||||
|
full ``WorkspaceRow`` here — the middleware needs to set the
|
||||||
|
contextvar on every request and an extra DB lookup just to populate
|
||||||
|
a name/slug we don't use yet would be wasted work.
|
||||||
|
"""
|
||||||
|
|
||||||
|
id: str
|
||||||
|
role: str
|
||||||
|
|||||||
@@ -46,6 +46,7 @@ class SQLiteUserRepository(UserRepository):
|
|||||||
oauth_id=row.oauth_id,
|
oauth_id=row.oauth_id,
|
||||||
needs_setup=row.needs_setup,
|
needs_setup=row.needs_setup,
|
||||||
token_version=row.token_version,
|
token_version=row.token_version,
|
||||||
|
default_workspace_id=row.default_workspace_id,
|
||||||
)
|
)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@@ -60,6 +61,7 @@ class SQLiteUserRepository(UserRepository):
|
|||||||
oauth_id=user.oauth_id,
|
oauth_id=user.oauth_id,
|
||||||
needs_setup=user.needs_setup,
|
needs_setup=user.needs_setup,
|
||||||
token_version=user.token_version,
|
token_version=user.token_version,
|
||||||
|
default_workspace_id=user.default_workspace_id,
|
||||||
)
|
)
|
||||||
|
|
||||||
# ── CRUD ──────────────────────────────────────────────────────────
|
# ── CRUD ──────────────────────────────────────────────────────────
|
||||||
@@ -106,6 +108,7 @@ class SQLiteUserRepository(UserRepository):
|
|||||||
row.oauth_id = user.oauth_id
|
row.oauth_id = user.oauth_id
|
||||||
row.needs_setup = user.needs_setup
|
row.needs_setup = user.needs_setup
|
||||||
row.token_version = user.token_version
|
row.token_version = user.token_version
|
||||||
|
row.default_workspace_id = user.default_workspace_id
|
||||||
await session.commit()
|
await session.commit()
|
||||||
return user
|
return user
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,83 @@
|
|||||||
|
"""Workspace slug helpers for the registration / initialize flow.
|
||||||
|
|
||||||
|
Stage 0 PR4 T4.10. Two responsibilities:
|
||||||
|
|
||||||
|
1. ``auto_slug_from_email(email)`` — pure transform from an email's
|
||||||
|
local part to a base slug that matches the schema's
|
||||||
|
``^[a-z0-9](-?[a-z0-9])*$`` pattern.
|
||||||
|
2. ``next_available_slug(base, exists_check=...)`` — collision walker
|
||||||
|
that appends ``-2``, ``-3``, … until ``exists_check`` reports the
|
||||||
|
candidate is free. Kept separate from ``auto_slug_from_email`` so
|
||||||
|
the pure function can be tested without a database.
|
||||||
|
|
||||||
|
Lives in the auth package (not in ``persistence``) because the input
|
||||||
|
is the user's email — a registration-time concept that doesn't belong
|
||||||
|
in a generic ``WorkspaceRepository``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
import secrets
|
||||||
|
from collections.abc import Awaitable, Callable
|
||||||
|
|
||||||
|
# Mirror the schema's slug rules from
|
||||||
|
# ``deerflow.persistence.workspace.sql`` so callers of this module
|
||||||
|
# never need to import private constants from persistence.
|
||||||
|
_SLUG_MIN_LEN = 3
|
||||||
|
_SLUG_MAX_LEN = 32
|
||||||
|
|
||||||
|
|
||||||
|
def auto_slug_from_email(email: str) -> str:
|
||||||
|
"""Map an email to a deterministic, schema-valid base slug.
|
||||||
|
|
||||||
|
Algorithm (from workspace-schema-design §3.1):
|
||||||
|
|
||||||
|
1. Take the local part (before ``@``).
|
||||||
|
2. Replace ``+``, ``_``, ``.`` with ``-`` and lowercase.
|
||||||
|
3. Strip everything that isn't ``[a-z0-9-]``.
|
||||||
|
4. Collapse repeated ``-``; strip leading/trailing ``-``.
|
||||||
|
5. Clamp to 32 chars.
|
||||||
|
6. If the result is shorter than the schema minimum (3 chars) or
|
||||||
|
empty, fall back to ``user-{token_hex(4)}`` so we always emit
|
||||||
|
a valid slug.
|
||||||
|
|
||||||
|
The returned slug is the *base* — callers must run it through
|
||||||
|
:func:`next_available_slug` before persisting to handle collisions.
|
||||||
|
"""
|
||||||
|
local = email.split("@", 1)[0]
|
||||||
|
local = re.sub(r"[+_.]", "-", local).lower()
|
||||||
|
local = re.sub(r"[^a-z0-9-]", "", local)
|
||||||
|
local = re.sub(r"-+", "-", local).strip("-")
|
||||||
|
slug = local[:_SLUG_MAX_LEN]
|
||||||
|
if len(slug) < _SLUG_MIN_LEN:
|
||||||
|
return f"user-{secrets.token_hex(4)}"
|
||||||
|
return slug
|
||||||
|
|
||||||
|
|
||||||
|
async def next_available_slug(
|
||||||
|
base: str,
|
||||||
|
*,
|
||||||
|
exists_check: Callable[[str], Awaitable[bool]],
|
||||||
|
) -> str:
|
||||||
|
"""Return the first of ``base``, ``base-2``, ``base-3``, … that ``exists_check`` reports free.
|
||||||
|
|
||||||
|
Caller-supplied ``exists_check`` is awaited once per candidate so
|
||||||
|
we can swap in a repository's ``get_by_slug`` without coupling
|
||||||
|
this module to persistence imports.
|
||||||
|
|
||||||
|
When ``base + '-N'`` would exceed the 32-char schema limit, the
|
||||||
|
base is truncated before the suffix is appended. The walker never
|
||||||
|
returns an over-long slug.
|
||||||
|
"""
|
||||||
|
if not await exists_check(base):
|
||||||
|
return base
|
||||||
|
|
||||||
|
n = 2
|
||||||
|
while True:
|
||||||
|
suffix = f"-{n}"
|
||||||
|
max_base_len = _SLUG_MAX_LEN - len(suffix)
|
||||||
|
candidate = f"{base[:max_base_len]}{suffix}"
|
||||||
|
if not await exists_check(candidate):
|
||||||
|
return candidate
|
||||||
|
n += 1
|
||||||
@@ -17,9 +17,11 @@ from starlette.responses import JSONResponse
|
|||||||
from starlette.types import ASGIApp
|
from starlette.types import ASGIApp
|
||||||
|
|
||||||
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse
|
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse
|
||||||
|
from app.gateway.auth.models import ActiveWorkspace
|
||||||
from app.gateway.authz import _ALL_PERMISSIONS, AuthContext
|
from app.gateway.authz import _ALL_PERMISSIONS, AuthContext
|
||||||
from app.gateway.internal_auth import INTERNAL_AUTH_HEADER_NAME, get_internal_user, is_valid_internal_auth_token
|
from app.gateway.internal_auth import INTERNAL_AUTH_HEADER_NAME, get_internal_user, is_valid_internal_auth_token
|
||||||
from deerflow.runtime.user_context import reset_current_user, set_current_user
|
from deerflow.runtime.user_context import reset_current_user, set_current_user
|
||||||
|
from deerflow.runtime.workspace_context import reset_current_workspace, set_current_workspace
|
||||||
|
|
||||||
# Paths that never require authentication.
|
# Paths that never require authentication.
|
||||||
_PUBLIC_PATH_PREFIXES: tuple[str, ...] = (
|
_PUBLIC_PATH_PREFIXES: tuple[str, ...] = (
|
||||||
@@ -119,8 +121,22 @@ class AuthMiddleware(BaseHTTPMiddleware):
|
|||||||
# JWT-decode + DB-lookup pipeline a second time per request).
|
# JWT-decode + DB-lookup pipeline a second time per request).
|
||||||
request.state.user = user
|
request.state.user = user
|
||||||
request.state.auth = AuthContext(user=user, permissions=_ALL_PERMISSIONS)
|
request.state.auth = AuthContext(user=user, permissions=_ALL_PERMISSIONS)
|
||||||
token = set_current_user(user)
|
user_token = set_current_user(user)
|
||||||
|
|
||||||
|
# Inject workspace contextvar from the JWT's wid/role claims.
|
||||||
|
# decode_token has already rejected legacy no-wid tokens upstream,
|
||||||
|
# so by the time we get here payload.wid is guaranteed non-None
|
||||||
|
# for cookie-authenticated requests. Internal-auth requests skip
|
||||||
|
# the workspace contextvar (they don't have a workspace scope —
|
||||||
|
# the internal user is a system actor).
|
||||||
|
ws_token = None
|
||||||
|
payload = getattr(request.state, "auth_payload", None)
|
||||||
|
if payload is not None and payload.wid is not None:
|
||||||
|
ws_token = set_current_workspace(ActiveWorkspace(id=payload.wid, role=payload.role or "owner"))
|
||||||
|
|
||||||
try:
|
try:
|
||||||
return await call_next(request)
|
return await call_next(request)
|
||||||
finally:
|
finally:
|
||||||
reset_current_user(token)
|
if ws_token is not None:
|
||||||
|
reset_current_workspace(ws_token)
|
||||||
|
reset_current_user(user_token)
|
||||||
|
|||||||
@@ -268,24 +268,27 @@ def require_permission(
|
|||||||
|
|
||||||
# Owner check for thread-specific resources.
|
# Owner check for thread-specific resources.
|
||||||
#
|
#
|
||||||
# 2.0-rc moved thread metadata into the SQL persistence layer
|
# PR6: ``check_access`` now takes ``workspace_id`` as the third
|
||||||
# (``threads_meta`` table). We verify ownership via
|
# positional argument; cross-workspace always denies regardless
|
||||||
# ``ThreadMetaStore.check_access``: it returns True for
|
# of user_id match. We pull workspace_id from the contextvar
|
||||||
# missing rows (untracked legacy thread) and for rows whose
|
# AuthMiddleware sets per request (and fall back to "default"
|
||||||
# ``user_id`` is NULL (shared / pre-auth data), so this is
|
# in no-auth dev mode so smoke flows keep working). Failures
|
||||||
# strict-deny rather than strict-allow — only an *existing*
|
# convert to **404**, not 403, so the response never leaks the
|
||||||
# row with a *different* user_id triggers 404.
|
# existence of a thread that belongs to a different tenant.
|
||||||
if owner_check:
|
if owner_check:
|
||||||
thread_id = kwargs.get("thread_id")
|
thread_id = kwargs.get("thread_id")
|
||||||
if thread_id is None:
|
if thread_id is None:
|
||||||
raise ValueError("require_permission with owner_check=True requires 'thread_id' parameter")
|
raise ValueError("require_permission with owner_check=True requires 'thread_id' parameter")
|
||||||
|
|
||||||
from app.gateway.deps import get_thread_store
|
from app.gateway.deps import get_thread_store
|
||||||
|
from deerflow.runtime.workspace_context import get_effective_workspace_id
|
||||||
|
|
||||||
|
workspace_id = get_effective_workspace_id()
|
||||||
thread_store = get_thread_store(request)
|
thread_store = get_thread_store(request)
|
||||||
allowed = await thread_store.check_access(
|
allowed = await thread_store.check_access(
|
||||||
thread_id,
|
thread_id,
|
||||||
str(auth.user.id),
|
str(auth.user.id),
|
||||||
|
workspace_id,
|
||||||
require_existing=require_existing,
|
require_existing=require_existing,
|
||||||
)
|
)
|
||||||
if not allowed:
|
if not allowed:
|
||||||
|
|||||||
@@ -220,6 +220,10 @@ async def get_current_user_from_request(request: Request):
|
|||||||
detail=AuthErrorResponse(code=AuthErrorCode.TOKEN_INVALID, message="Token revoked (password changed)").model_dump(),
|
detail=AuthErrorResponse(code=AuthErrorCode.TOKEN_INVALID, message="Token revoked (password changed)").model_dump(),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Stash decoded payload on request.state so AuthMiddleware can read
|
||||||
|
# wid/role for the workspace contextvar without a second decode.
|
||||||
|
request.state.auth_payload = payload
|
||||||
|
|
||||||
return user
|
return user
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -15,11 +15,60 @@ from app.gateway.auth import (
|
|||||||
)
|
)
|
||||||
from app.gateway.auth.config import get_auth_config
|
from app.gateway.auth.config import get_auth_config
|
||||||
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse
|
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse
|
||||||
|
from app.gateway.auth.models import UserMeResponse, UserMeWorkspace
|
||||||
|
from app.gateway.auth.workspace_slug import auto_slug_from_email, next_available_slug
|
||||||
from app.gateway.csrf_middleware import is_secure_request
|
from app.gateway.csrf_middleware import is_secure_request
|
||||||
from app.gateway.deps import get_current_user_from_request, get_local_provider
|
from app.gateway.deps import get_current_user_from_request, get_local_provider
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def ensure_default_workspace(user) -> str:
|
||||||
|
"""Create the user's personal workspace + owner membership, set default_workspace_id.
|
||||||
|
|
||||||
|
Returns the new workspace id. Idempotent for users who already
|
||||||
|
have a default_workspace_id (used by both the registration flow
|
||||||
|
and the lifespan backfill in app.py).
|
||||||
|
"""
|
||||||
|
from deerflow.persistence.engine import get_session_factory
|
||||||
|
from deerflow.persistence.workspace import WorkspaceRepository
|
||||||
|
from deerflow.persistence.workspace.sql import SLUG_BLACKLIST
|
||||||
|
from deerflow.persistence.workspace_membership import WorkspaceMembershipRepository
|
||||||
|
|
||||||
|
if user.default_workspace_id:
|
||||||
|
return user.default_workspace_id
|
||||||
|
|
||||||
|
sf = get_session_factory()
|
||||||
|
ws_repo = WorkspaceRepository(sf)
|
||||||
|
m_repo = WorkspaceMembershipRepository(sf)
|
||||||
|
|
||||||
|
base_slug = auto_slug_from_email(user.email)
|
||||||
|
|
||||||
|
async def slug_exists(s: str) -> bool:
|
||||||
|
# Treat blacklisted slugs as "taken" so the walker skips them
|
||||||
|
# instead of letting WorkspaceRepository.create raise after a
|
||||||
|
# successful slug computation (the user picked a reserved name
|
||||||
|
# like "admin@example.com" → base slug "admin").
|
||||||
|
if s in SLUG_BLACKLIST:
|
||||||
|
return True
|
||||||
|
return (await ws_repo.get_by_slug(s)) is not None
|
||||||
|
|
||||||
|
unique_slug = await next_available_slug(base_slug, exists_check=slug_exists)
|
||||||
|
|
||||||
|
display_local = user.email.split("@", 1)[0]
|
||||||
|
workspace = await ws_repo.create(
|
||||||
|
name=f"{display_local}'s Workspace"[:64],
|
||||||
|
slug=unique_slug,
|
||||||
|
owner_id=str(user.id),
|
||||||
|
)
|
||||||
|
await m_repo.add(workspace_id=workspace["id"], user_id=str(user.id), role="owner")
|
||||||
|
|
||||||
|
user.default_workspace_id = workspace["id"]
|
||||||
|
await get_local_provider().update_user(user)
|
||||||
|
|
||||||
|
return workspace["id"]
|
||||||
|
|
||||||
|
|
||||||
router = APIRouter(prefix="/api/v1/auth", tags=["auth"])
|
router = APIRouter(prefix="/api/v1/auth", tags=["auth"])
|
||||||
|
|
||||||
|
|
||||||
@@ -292,7 +341,15 @@ async def login_local(
|
|||||||
)
|
)
|
||||||
|
|
||||||
_record_login_success(client_ip)
|
_record_login_success(client_ip)
|
||||||
token = create_access_token(str(user.id), token_version=user.token_version)
|
# Ensure the user has a workspace (covers pre-PR4 users still in DB
|
||||||
|
# whose default_workspace_id was never backfilled by the lifespan hook).
|
||||||
|
workspace_id = await ensure_default_workspace(user)
|
||||||
|
token = create_access_token(
|
||||||
|
str(user.id),
|
||||||
|
token_version=user.token_version,
|
||||||
|
workspace_id=workspace_id,
|
||||||
|
role="owner",
|
||||||
|
)
|
||||||
_set_session_cookie(response, token, request)
|
_set_session_cookie(response, token, request)
|
||||||
|
|
||||||
return LoginResponse(
|
return LoginResponse(
|
||||||
@@ -316,7 +373,14 @@ async def register(request: Request, response: Response, body: RegisterRequest):
|
|||||||
detail=AuthErrorResponse(code=AuthErrorCode.EMAIL_ALREADY_EXISTS, message="Email already registered").model_dump(),
|
detail=AuthErrorResponse(code=AuthErrorCode.EMAIL_ALREADY_EXISTS, message="Email already registered").model_dump(),
|
||||||
)
|
)
|
||||||
|
|
||||||
token = create_access_token(str(user.id), token_version=user.token_version)
|
workspace_id = await ensure_default_workspace(user)
|
||||||
|
|
||||||
|
token = create_access_token(
|
||||||
|
str(user.id),
|
||||||
|
token_version=user.token_version,
|
||||||
|
workspace_id=workspace_id,
|
||||||
|
role="owner",
|
||||||
|
)
|
||||||
_set_session_cookie(response, token, request)
|
_set_session_cookie(response, token, request)
|
||||||
|
|
||||||
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role)
|
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role)
|
||||||
@@ -368,18 +432,57 @@ async def change_password(request: Request, response: Response, body: ChangePass
|
|||||||
|
|
||||||
await provider.update_user(user)
|
await provider.update_user(user)
|
||||||
|
|
||||||
# Re-issue cookie with new token_version
|
# Re-issue cookie with new token_version. wid + role must be carried
|
||||||
token = create_access_token(str(user.id), token_version=user.token_version)
|
# forward so the re-signed JWT still passes the AuthMiddleware
|
||||||
|
# workspace gate; ensure_default_workspace fills in for the (rare)
|
||||||
|
# case where the user predates PR4 and has not been backfilled.
|
||||||
|
workspace_id = await ensure_default_workspace(user)
|
||||||
|
token = create_access_token(
|
||||||
|
str(user.id),
|
||||||
|
token_version=user.token_version,
|
||||||
|
workspace_id=workspace_id,
|
||||||
|
role="owner",
|
||||||
|
)
|
||||||
_set_session_cookie(response, token, request)
|
_set_session_cookie(response, token, request)
|
||||||
|
|
||||||
return MessageResponse(message="Password changed successfully")
|
return MessageResponse(message="Password changed successfully")
|
||||||
|
|
||||||
|
|
||||||
@router.get("/me", response_model=UserResponse)
|
@router.get("/me", response_model=UserMeResponse)
|
||||||
async def get_me(request: Request):
|
async def get_me(request: Request):
|
||||||
"""Get current authenticated user info."""
|
"""Get current authenticated user info, including the workspaces they belong to."""
|
||||||
|
from deerflow.persistence.engine import get_session_factory
|
||||||
|
from deerflow.persistence.workspace import WorkspaceRepository
|
||||||
|
from deerflow.persistence.workspace_membership import WorkspaceMembershipRepository
|
||||||
|
|
||||||
user = await get_current_user_from_request(request)
|
user = await get_current_user_from_request(request)
|
||||||
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role, needs_setup=user.needs_setup)
|
|
||||||
|
sf = get_session_factory()
|
||||||
|
workspaces: list[UserMeWorkspace] = []
|
||||||
|
if sf is not None:
|
||||||
|
ws_repo = WorkspaceRepository(sf)
|
||||||
|
m_repo = WorkspaceMembershipRepository(sf)
|
||||||
|
ws_rows = await ws_repo.list_by_user(user_id=str(user.id))
|
||||||
|
memberships = await m_repo.list_by_user(user_id=str(user.id))
|
||||||
|
role_by_ws = {m["workspace_id"]: m["role"] for m in memberships}
|
||||||
|
workspaces = [
|
||||||
|
UserMeWorkspace(
|
||||||
|
id=w["id"],
|
||||||
|
name=w["name"],
|
||||||
|
slug=w["slug"],
|
||||||
|
role=role_by_ws.get(w["id"], "member"),
|
||||||
|
)
|
||||||
|
for w in ws_rows
|
||||||
|
]
|
||||||
|
|
||||||
|
return UserMeResponse(
|
||||||
|
id=str(user.id),
|
||||||
|
email=user.email,
|
||||||
|
system_role=user.system_role,
|
||||||
|
needs_setup=user.needs_setup,
|
||||||
|
default_workspace_id=user.default_workspace_id,
|
||||||
|
workspaces=workspaces,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
_SETUP_STATUS_COOLDOWN: dict[str, float] = {}
|
_SETUP_STATUS_COOLDOWN: dict[str, float] = {}
|
||||||
@@ -452,7 +555,14 @@ async def initialize_admin(request: Request, response: Response, body: Initializ
|
|||||||
detail=AuthErrorResponse(code=AuthErrorCode.SYSTEM_ALREADY_INITIALIZED, message="System already initialized").model_dump(),
|
detail=AuthErrorResponse(code=AuthErrorCode.SYSTEM_ALREADY_INITIALIZED, message="System already initialized").model_dump(),
|
||||||
)
|
)
|
||||||
|
|
||||||
token = create_access_token(str(user.id), token_version=user.token_version)
|
workspace_id = await ensure_default_workspace(user)
|
||||||
|
|
||||||
|
token = create_access_token(
|
||||||
|
str(user.id),
|
||||||
|
token_version=user.token_version,
|
||||||
|
workspace_id=workspace_id,
|
||||||
|
role="owner",
|
||||||
|
)
|
||||||
_set_session_cookie(response, token, request)
|
_set_session_cookie(response, token, request)
|
||||||
|
|
||||||
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role)
|
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role)
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ from langgraph.runtime import Runtime
|
|||||||
from deerflow.agents.thread_state import ThreadDataState
|
from deerflow.agents.thread_state import ThreadDataState
|
||||||
from deerflow.config.paths import Paths, get_paths
|
from deerflow.config.paths import Paths, get_paths
|
||||||
from deerflow.runtime.user_context import get_effective_user_id
|
from deerflow.runtime.user_context import get_effective_user_id
|
||||||
|
from deerflow.runtime.workspace_context import get_effective_workspace_id
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -24,10 +25,15 @@ class ThreadDataMiddlewareState(AgentState):
|
|||||||
class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]):
|
class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]):
|
||||||
"""Create thread data directories for each thread execution.
|
"""Create thread data directories for each thread execution.
|
||||||
|
|
||||||
Creates the following directory structure:
|
PR6 routes thread storage through the workspace dimension. When a
|
||||||
- {base_dir}/threads/{thread_id}/user-data/workspace
|
workspace contextvar is set (production via AuthMiddleware; tests via
|
||||||
- {base_dir}/threads/{thread_id}/user-data/uploads
|
the autouse fixture), directories live at
|
||||||
- {base_dir}/threads/{thread_id}/user-data/outputs
|
``{base_dir}/workspaces/{wid}/threads/{thread_id}/user-data/{workspace,uploads,outputs}``.
|
||||||
|
In no-auth dev mode ``get_effective_workspace_id()`` returns
|
||||||
|
``"default"`` so the layout stays valid; the ``user_id`` falls through
|
||||||
|
to the same default constant. Either way thread state lives below a
|
||||||
|
workspace bucket, never directly under ``{base_dir}/threads`` (legacy)
|
||||||
|
or ``{base_dir}/users`` (PR4 layout).
|
||||||
|
|
||||||
Lifecycle Management:
|
Lifecycle Management:
|
||||||
- With lazy_init=True (default): Only compute paths, directories created on-demand
|
- With lazy_init=True (default): Only compute paths, directories created on-demand
|
||||||
@@ -49,34 +55,18 @@ class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]):
|
|||||||
self._paths = Paths(base_dir) if base_dir else get_paths()
|
self._paths = Paths(base_dir) if base_dir else get_paths()
|
||||||
self._lazy_init = lazy_init
|
self._lazy_init = lazy_init
|
||||||
|
|
||||||
def _get_thread_paths(self, thread_id: str, user_id: str | None = None) -> dict[str, str]:
|
def _get_thread_paths(self, thread_id: str, *, workspace_id: str, user_id: str) -> dict[str, str]:
|
||||||
"""Get the paths for a thread's data directories.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
thread_id: The thread ID.
|
|
||||||
user_id: Optional user ID for per-user path isolation.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dictionary with workspace_path, uploads_path, and outputs_path.
|
|
||||||
"""
|
|
||||||
return {
|
return {
|
||||||
"workspace_path": str(self._paths.sandbox_work_dir(thread_id, user_id=user_id)),
|
"workspace_path": str(self._paths.sandbox_work_dir(thread_id, workspace_id=workspace_id)),
|
||||||
"uploads_path": str(self._paths.sandbox_uploads_dir(thread_id, user_id=user_id)),
|
"uploads_path": str(self._paths.sandbox_uploads_dir(thread_id, workspace_id=workspace_id)),
|
||||||
"outputs_path": str(self._paths.sandbox_outputs_dir(thread_id, user_id=user_id)),
|
"outputs_path": str(self._paths.sandbox_outputs_dir(thread_id, workspace_id=workspace_id)),
|
||||||
|
"user_id": user_id,
|
||||||
|
"workspace_id": workspace_id,
|
||||||
}
|
}
|
||||||
|
|
||||||
def _create_thread_directories(self, thread_id: str, user_id: str | None = None) -> dict[str, str]:
|
def _create_thread_directories(self, thread_id: str, *, workspace_id: str, user_id: str) -> dict[str, str]:
|
||||||
"""Create the thread data directories.
|
self._paths.ensure_thread_dirs(thread_id, workspace_id=workspace_id)
|
||||||
|
return self._get_thread_paths(thread_id, workspace_id=workspace_id, user_id=user_id)
|
||||||
Args:
|
|
||||||
thread_id: The thread ID.
|
|
||||||
user_id: Optional user ID for per-user path isolation.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dictionary with the created directory paths.
|
|
||||||
"""
|
|
||||||
self._paths.ensure_thread_dirs(thread_id, user_id=user_id)
|
|
||||||
return self._get_thread_paths(thread_id, user_id=user_id)
|
|
||||||
|
|
||||||
@override
|
@override
|
||||||
def before_agent(self, state: ThreadDataMiddlewareState, runtime: Runtime) -> dict | None:
|
def before_agent(self, state: ThreadDataMiddlewareState, runtime: Runtime) -> dict | None:
|
||||||
@@ -90,14 +80,15 @@ class ThreadDataMiddleware(AgentMiddleware[ThreadDataMiddlewareState]):
|
|||||||
raise ValueError("Thread ID is required in runtime context or config.configurable")
|
raise ValueError("Thread ID is required in runtime context or config.configurable")
|
||||||
|
|
||||||
user_id = get_effective_user_id()
|
user_id = get_effective_user_id()
|
||||||
|
workspace_id = get_effective_workspace_id()
|
||||||
|
|
||||||
if self._lazy_init:
|
if self._lazy_init:
|
||||||
# Lazy initialization: only compute paths, don't create directories
|
# Lazy initialization: only compute paths, don't create directories
|
||||||
paths = self._get_thread_paths(thread_id, user_id=user_id)
|
paths = self._get_thread_paths(thread_id, workspace_id=workspace_id, user_id=user_id)
|
||||||
else:
|
else:
|
||||||
# Eager initialization: create directories immediately
|
# Eager initialization: create directories immediately
|
||||||
paths = self._create_thread_directories(thread_id, user_id=user_id)
|
paths = self._create_thread_directories(thread_id, workspace_id=workspace_id, user_id=user_id)
|
||||||
logger.debug("Created thread data directories for thread %s", thread_id)
|
logger.debug("Created thread data directories for thread %s under workspace %s", thread_id, workspace_id)
|
||||||
|
|
||||||
messages = list(state.get("messages", []))
|
messages = list(state.get("messages", []))
|
||||||
last_message = messages[-1] if messages else None
|
last_message = messages[-1] if messages else None
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ VIRTUAL_PATH_PREFIX = "/mnt/user-data"
|
|||||||
|
|
||||||
_SAFE_THREAD_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
|
_SAFE_THREAD_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
|
||||||
_SAFE_USER_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
|
_SAFE_USER_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
|
||||||
|
_SAFE_WORKSPACE_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$")
|
||||||
|
|
||||||
|
|
||||||
def _default_local_base_dir() -> Path:
|
def _default_local_base_dir() -> Path:
|
||||||
@@ -31,6 +32,13 @@ def _validate_user_id(user_id: str) -> str:
|
|||||||
return user_id
|
return user_id
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_workspace_id(workspace_id: str) -> str:
|
||||||
|
"""Validate a workspace ID before using it in filesystem paths."""
|
||||||
|
if not _SAFE_WORKSPACE_ID_RE.match(workspace_id):
|
||||||
|
raise ValueError(f"Invalid workspace_id {workspace_id!r}: only alphanumeric characters, hyphens, and underscores are allowed.")
|
||||||
|
return workspace_id
|
||||||
|
|
||||||
|
|
||||||
def _join_host_path(base: str, *parts: str) -> str:
|
def _join_host_path(base: str, *parts: str) -> str:
|
||||||
"""Join host filesystem path segments while preserving native style.
|
"""Join host filesystem path segments while preserving native style.
|
||||||
|
|
||||||
@@ -148,116 +156,129 @@ class Paths:
|
|||||||
"""Legacy per-agent memory file: `{base_dir}/agents/{name}/memory.json`."""
|
"""Legacy per-agent memory file: `{base_dir}/agents/{name}/memory.json`."""
|
||||||
return self.agent_dir(name) / "memory.json"
|
return self.agent_dir(name) / "memory.json"
|
||||||
|
|
||||||
def user_dir(self, user_id: str) -> Path:
|
def workspace_dir(self, workspace_id: str) -> Path:
|
||||||
"""Directory for a specific user: `{base_dir}/users/{user_id}/`."""
|
"""Directory for a specific workspace: `{base_dir}/workspaces/{workspace_id}/`.
|
||||||
|
|
||||||
|
PR6 introduces this as the top-level isolation dimension. Per-user
|
||||||
|
state and per-thread state both live underneath their workspace so
|
||||||
|
a user with access to two workspaces never sees state bleed between
|
||||||
|
them on the filesystem.
|
||||||
|
"""
|
||||||
|
return self.base_dir / "workspaces" / _validate_workspace_id(workspace_id)
|
||||||
|
|
||||||
|
def user_dir(self, user_id: str, *, workspace_id: str | None = None) -> Path:
|
||||||
|
"""Directory for a specific user.
|
||||||
|
|
||||||
|
When ``workspace_id`` is provided (PR6+):
|
||||||
|
``{base_dir}/workspaces/{wid}/users/{user_id}/``
|
||||||
|
|
||||||
|
Otherwise (legacy layout):
|
||||||
|
``{base_dir}/users/{user_id}/``
|
||||||
|
"""
|
||||||
|
if workspace_id is not None:
|
||||||
|
return self.workspace_dir(workspace_id) / "users" / _validate_user_id(user_id)
|
||||||
return self.base_dir / "users" / _validate_user_id(user_id)
|
return self.base_dir / "users" / _validate_user_id(user_id)
|
||||||
|
|
||||||
def user_memory_file(self, user_id: str) -> Path:
|
def user_memory_file(self, user_id: str, *, workspace_id: str | None = None) -> Path:
|
||||||
"""Per-user memory file: `{base_dir}/users/{user_id}/memory.json`."""
|
"""Per-user memory file under the active workspace (legacy without)."""
|
||||||
return self.user_dir(user_id) / "memory.json"
|
return self.user_dir(user_id, workspace_id=workspace_id) / "memory.json"
|
||||||
|
|
||||||
def user_agents_dir(self, user_id: str) -> Path:
|
def user_agents_dir(self, user_id: str, *, workspace_id: str | None = None) -> Path:
|
||||||
"""Per-user root for that user's custom agents: `{base_dir}/users/{user_id}/agents/`."""
|
"""Per-user root for custom agents under the active workspace."""
|
||||||
return self.user_dir(user_id) / "agents"
|
return self.user_dir(user_id, workspace_id=workspace_id) / "agents"
|
||||||
|
|
||||||
def user_agent_dir(self, user_id: str, agent_name: str) -> Path:
|
def user_agent_dir(self, user_id: str, agent_name: str, *, workspace_id: str | None = None) -> Path:
|
||||||
"""Per-user per-agent directory: `{base_dir}/users/{user_id}/agents/{name}/`."""
|
"""Per-user per-agent directory under the active workspace."""
|
||||||
return self.user_agents_dir(user_id) / agent_name.lower()
|
return self.user_agents_dir(user_id, workspace_id=workspace_id) / agent_name.lower()
|
||||||
|
|
||||||
def user_agent_memory_file(self, user_id: str, agent_name: str) -> Path:
|
def user_agent_memory_file(self, user_id: str, agent_name: str, *, workspace_id: str | None = None) -> Path:
|
||||||
"""Per-user per-agent memory: `{base_dir}/users/{user_id}/agents/{name}/memory.json`."""
|
"""Per-user per-agent memory file under the active workspace."""
|
||||||
return self.user_agent_dir(user_id, agent_name) / "memory.json"
|
return self.user_agent_dir(user_id, agent_name, workspace_id=workspace_id) / "memory.json"
|
||||||
|
|
||||||
def thread_dir(self, thread_id: str, *, user_id: str | None = None) -> Path:
|
def thread_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
|
||||||
"""
|
"""
|
||||||
Host path for a thread's data.
|
Host path for a thread's data.
|
||||||
|
|
||||||
When *user_id* is provided:
|
Precedence — workspace beats user, both beat legacy:
|
||||||
`{base_dir}/users/{user_id}/threads/{thread_id}/`
|
|
||||||
Otherwise (legacy layout):
|
|
||||||
`{base_dir}/threads/{thread_id}/`
|
|
||||||
|
|
||||||
This directory contains a `user-data/` subdirectory that is mounted
|
* ``workspace_id`` given (PR6+):
|
||||||
as `/mnt/user-data/` inside the sandbox.
|
``{base_dir}/workspaces/{wid}/threads/{thread_id}/``
|
||||||
|
* ``user_id`` only (legacy after user-isolation migration):
|
||||||
|
``{base_dir}/users/{user_id}/threads/{thread_id}/``
|
||||||
|
* neither (very legacy, pre-isolation):
|
||||||
|
``{base_dir}/threads/{thread_id}/``
|
||||||
|
|
||||||
|
The contained ``user-data/`` subdirectory is mounted as
|
||||||
|
``/mnt/user-data/`` inside the sandbox regardless of which form is
|
||||||
|
chosen — only the host-side parent differs.
|
||||||
|
|
||||||
Raises:
|
Raises:
|
||||||
ValueError: If `thread_id` or `user_id` contains unsafe characters (path
|
ValueError: If any of the supplied ids contains unsafe characters.
|
||||||
separators or `..`) that could cause directory traversal.
|
|
||||||
"""
|
"""
|
||||||
|
if workspace_id is not None:
|
||||||
|
return self.workspace_dir(workspace_id) / "threads" / _validate_thread_id(thread_id)
|
||||||
if user_id is not None:
|
if user_id is not None:
|
||||||
return self.user_dir(user_id) / "threads" / _validate_thread_id(thread_id)
|
return self.base_dir / "users" / _validate_user_id(user_id) / "threads" / _validate_thread_id(thread_id)
|
||||||
return self.base_dir / "threads" / _validate_thread_id(thread_id)
|
return self.base_dir / "threads" / _validate_thread_id(thread_id)
|
||||||
|
|
||||||
def sandbox_work_dir(self, thread_id: str, *, user_id: str | None = None) -> Path:
|
def sandbox_work_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
|
||||||
"""
|
"""
|
||||||
Host path for the agent's workspace directory.
|
Host path for the agent's workspace directory.
|
||||||
Host: `{base_dir}/threads/{thread_id}/user-data/workspace/`
|
Host: ``{thread_dir}/user-data/workspace/``
|
||||||
Sandbox: `/mnt/user-data/workspace/`
|
Sandbox: ``/mnt/user-data/workspace/``
|
||||||
"""
|
"""
|
||||||
return self.thread_dir(thread_id, user_id=user_id) / "user-data" / "workspace"
|
return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data" / "workspace"
|
||||||
|
|
||||||
def sandbox_uploads_dir(self, thread_id: str, *, user_id: str | None = None) -> Path:
|
def sandbox_uploads_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
|
||||||
"""
|
"""Host path for user-uploaded files; sandbox: ``/mnt/user-data/uploads/``."""
|
||||||
Host path for user-uploaded files.
|
return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data" / "uploads"
|
||||||
Host: `{base_dir}/threads/{thread_id}/user-data/uploads/`
|
|
||||||
Sandbox: `/mnt/user-data/uploads/`
|
|
||||||
"""
|
|
||||||
return self.thread_dir(thread_id, user_id=user_id) / "user-data" / "uploads"
|
|
||||||
|
|
||||||
def sandbox_outputs_dir(self, thread_id: str, *, user_id: str | None = None) -> Path:
|
def sandbox_outputs_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
|
||||||
"""
|
"""Host path for agent-generated artifacts; sandbox: ``/mnt/user-data/outputs/``."""
|
||||||
Host path for agent-generated artifacts.
|
return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data" / "outputs"
|
||||||
Host: `{base_dir}/threads/{thread_id}/user-data/outputs/`
|
|
||||||
Sandbox: `/mnt/user-data/outputs/`
|
|
||||||
"""
|
|
||||||
return self.thread_dir(thread_id, user_id=user_id) / "user-data" / "outputs"
|
|
||||||
|
|
||||||
def acp_workspace_dir(self, thread_id: str, *, user_id: str | None = None) -> Path:
|
def acp_workspace_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
|
||||||
"""
|
"""
|
||||||
Host path for the ACP workspace of a specific thread.
|
Host path for the ACP workspace of a specific thread; sandbox: ``/mnt/acp-workspace/``.
|
||||||
Host: `{base_dir}/threads/{thread_id}/acp-workspace/`
|
|
||||||
Sandbox: `/mnt/acp-workspace/`
|
|
||||||
|
|
||||||
Each thread gets its own isolated ACP workspace so that concurrent
|
Each thread gets its own isolated ACP workspace so that concurrent
|
||||||
sessions cannot read each other's ACP agent outputs.
|
sessions cannot read each other's ACP agent outputs.
|
||||||
"""
|
"""
|
||||||
return self.thread_dir(thread_id, user_id=user_id) / "acp-workspace"
|
return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "acp-workspace"
|
||||||
|
|
||||||
def sandbox_user_data_dir(self, thread_id: str, *, user_id: str | None = None) -> Path:
|
def sandbox_user_data_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> Path:
|
||||||
"""
|
"""Host path for the user-data root; sandbox: ``/mnt/user-data/``."""
|
||||||
Host path for the user-data root.
|
return self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id) / "user-data"
|
||||||
Host: `{base_dir}/threads/{thread_id}/user-data/`
|
|
||||||
Sandbox: `/mnt/user-data/`
|
|
||||||
"""
|
|
||||||
return self.thread_dir(thread_id, user_id=user_id) / "user-data"
|
|
||||||
|
|
||||||
def host_thread_dir(self, thread_id: str, *, user_id: str | None = None) -> str:
|
def host_thread_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
|
||||||
"""Host path for a thread directory, preserving Windows path syntax."""
|
"""Host path for a thread directory, preserving Windows path syntax."""
|
||||||
|
if workspace_id is not None:
|
||||||
|
return _join_host_path(self._host_base_dir_str(), "workspaces", _validate_workspace_id(workspace_id), "threads", _validate_thread_id(thread_id))
|
||||||
if user_id is not None:
|
if user_id is not None:
|
||||||
return _join_host_path(self._host_base_dir_str(), "users", _validate_user_id(user_id), "threads", _validate_thread_id(thread_id))
|
return _join_host_path(self._host_base_dir_str(), "users", _validate_user_id(user_id), "threads", _validate_thread_id(thread_id))
|
||||||
return _join_host_path(self._host_base_dir_str(), "threads", _validate_thread_id(thread_id))
|
return _join_host_path(self._host_base_dir_str(), "threads", _validate_thread_id(thread_id))
|
||||||
|
|
||||||
def host_sandbox_user_data_dir(self, thread_id: str, *, user_id: str | None = None) -> str:
|
def host_sandbox_user_data_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
|
||||||
"""Host path for a thread's user-data root."""
|
"""Host path for a thread's user-data root."""
|
||||||
return _join_host_path(self.host_thread_dir(thread_id, user_id=user_id), "user-data")
|
return _join_host_path(self.host_thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "user-data")
|
||||||
|
|
||||||
def host_sandbox_work_dir(self, thread_id: str, *, user_id: str | None = None) -> str:
|
def host_sandbox_work_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
|
||||||
"""Host path for the workspace mount source."""
|
"""Host path for the workspace mount source."""
|
||||||
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, user_id=user_id), "workspace")
|
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "workspace")
|
||||||
|
|
||||||
def host_sandbox_uploads_dir(self, thread_id: str, *, user_id: str | None = None) -> str:
|
def host_sandbox_uploads_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
|
||||||
"""Host path for the uploads mount source."""
|
"""Host path for the uploads mount source."""
|
||||||
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, user_id=user_id), "uploads")
|
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "uploads")
|
||||||
|
|
||||||
def host_sandbox_outputs_dir(self, thread_id: str, *, user_id: str | None = None) -> str:
|
def host_sandbox_outputs_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
|
||||||
"""Host path for the outputs mount source."""
|
"""Host path for the outputs mount source."""
|
||||||
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, user_id=user_id), "outputs")
|
return _join_host_path(self.host_sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "outputs")
|
||||||
|
|
||||||
def host_acp_workspace_dir(self, thread_id: str, *, user_id: str | None = None) -> str:
|
def host_acp_workspace_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> str:
|
||||||
"""Host path for the ACP workspace mount source."""
|
"""Host path for the ACP workspace mount source."""
|
||||||
return _join_host_path(self.host_thread_dir(thread_id, user_id=user_id), "acp-workspace")
|
return _join_host_path(self.host_thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id), "acp-workspace")
|
||||||
|
|
||||||
def ensure_thread_dirs(self, thread_id: str, *, user_id: str | None = None) -> None:
|
def ensure_thread_dirs(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> None:
|
||||||
"""Create all standard sandbox directories for a thread.
|
"""Create all standard sandbox directories for a thread.
|
||||||
|
|
||||||
Directories are created with mode 0o777 so that sandbox containers
|
Directories are created with mode 0o777 so that sandbox containers
|
||||||
@@ -271,24 +292,28 @@ class Paths:
|
|||||||
ACP agent invocation.
|
ACP agent invocation.
|
||||||
"""
|
"""
|
||||||
for d in [
|
for d in [
|
||||||
self.sandbox_work_dir(thread_id, user_id=user_id),
|
self.sandbox_work_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
|
||||||
self.sandbox_uploads_dir(thread_id, user_id=user_id),
|
self.sandbox_uploads_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
|
||||||
self.sandbox_outputs_dir(thread_id, user_id=user_id),
|
self.sandbox_outputs_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
|
||||||
self.acp_workspace_dir(thread_id, user_id=user_id),
|
self.acp_workspace_dir(thread_id, workspace_id=workspace_id, user_id=user_id),
|
||||||
]:
|
]:
|
||||||
d.mkdir(parents=True, exist_ok=True)
|
d.mkdir(parents=True, exist_ok=True)
|
||||||
d.chmod(0o777)
|
d.chmod(0o777)
|
||||||
|
|
||||||
def delete_thread_dir(self, thread_id: str, *, user_id: str | None = None) -> None:
|
def delete_thread_dir(self, thread_id: str, *, workspace_id: str | None = None, user_id: str | None = None) -> None:
|
||||||
"""Delete all persisted data for a thread.
|
"""Delete all persisted data for a thread. Idempotent."""
|
||||||
|
thread_dir = self.thread_dir(thread_id, workspace_id=workspace_id, user_id=user_id)
|
||||||
The operation is idempotent: missing thread directories are ignored.
|
|
||||||
"""
|
|
||||||
thread_dir = self.thread_dir(thread_id, user_id=user_id)
|
|
||||||
if thread_dir.exists():
|
if thread_dir.exists():
|
||||||
shutil.rmtree(thread_dir)
|
shutil.rmtree(thread_dir)
|
||||||
|
|
||||||
def resolve_virtual_path(self, thread_id: str, virtual_path: str, *, user_id: str | None = None) -> Path:
|
def resolve_virtual_path(
|
||||||
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
virtual_path: str,
|
||||||
|
*,
|
||||||
|
workspace_id: str | None = None,
|
||||||
|
user_id: str | None = None,
|
||||||
|
) -> Path:
|
||||||
"""Resolve a sandbox virtual path to the actual host filesystem path.
|
"""Resolve a sandbox virtual path to the actual host filesystem path.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
@@ -296,7 +321,8 @@ class Paths:
|
|||||||
virtual_path: Virtual path as seen inside the sandbox, e.g.
|
virtual_path: Virtual path as seen inside the sandbox, e.g.
|
||||||
``/mnt/user-data/outputs/report.pdf``.
|
``/mnt/user-data/outputs/report.pdf``.
|
||||||
Leading slashes are stripped before matching.
|
Leading slashes are stripped before matching.
|
||||||
user_id: Optional user ID for user-scoped path resolution.
|
workspace_id: Optional workspace ID for workspace-scoped resolution.
|
||||||
|
user_id: Optional user ID for legacy user-scoped resolution.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The resolved absolute host filesystem path.
|
The resolved absolute host filesystem path.
|
||||||
@@ -314,7 +340,7 @@ class Paths:
|
|||||||
raise ValueError(f"Path must start with /{prefix}")
|
raise ValueError(f"Path must start with /{prefix}")
|
||||||
|
|
||||||
relative = stripped[len(prefix) :].lstrip("/")
|
relative = stripped[len(prefix) :].lstrip("/")
|
||||||
base = self.sandbox_user_data_dir(thread_id, user_id=user_id).resolve()
|
base = self.sandbox_user_data_dir(thread_id, workspace_id=workspace_id, user_id=user_id).resolve()
|
||||||
actual = (base / relative).resolve()
|
actual = (base / relative).resolve()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -0,0 +1,18 @@
|
|||||||
|
"""API key persistence — ORM model only (Stage 0 PR8).
|
||||||
|
|
||||||
|
An API key is the credential a service_account uses to call the
|
||||||
|
headless API. Each key has a public ``key_prefix`` (printed in audit
|
||||||
|
logs and used for quick lookup) and a ``key_hash`` (sha-256 of the
|
||||||
|
plaintext token, never reversed). Plaintext tokens are only ever
|
||||||
|
returned to the caller at create time.
|
||||||
|
|
||||||
|
PR8 introduces only the schema + ORM row class. Token generation,
|
||||||
|
hashing, scope parsing, rate limiting, and the API-key auth
|
||||||
|
middleware live in Stage 1 alongside the headless API surface.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from deerflow.persistence.api_key.model import ApiKeyRow
|
||||||
|
|
||||||
|
__all__ = ["ApiKeyRow"]
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
"""ORM model for API keys (credentials owned by a service account)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, text
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class ApiKeyRow(Base):
|
||||||
|
__tablename__ = "api_keys"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
primary_key=True,
|
||||||
|
comment="API key 主键,UUID 字符串(36 字符)",
|
||||||
|
)
|
||||||
|
service_account_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("service_accounts.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
comment="所属 service_account;service_account 删除时级联清掉所有 api_key",
|
||||||
|
)
|
||||||
|
key_prefix: Mapped[str] = mapped_column(
|
||||||
|
String(16),
|
||||||
|
nullable=False,
|
||||||
|
unique=True,
|
||||||
|
comment="公开 prefix(如 'dfk_live_abc12345'),可在审计日志 / UI 中打印;全局唯一,撤销后亦不复用以避免审计混淆",
|
||||||
|
)
|
||||||
|
key_hash: Mapped[str] = mapped_column(
|
||||||
|
String(128),
|
||||||
|
nullable=False,
|
||||||
|
comment="完整 token 的 sha-256 hex(64 字符;预留 128 以兼容未来更长哈希),plaintext token 仅在创建时返回给调用方",
|
||||||
|
)
|
||||||
|
name: Mapped[str] = mapped_column(
|
||||||
|
String(64),
|
||||||
|
nullable=False,
|
||||||
|
comment="key 的人类可读标签(如 'ci pipeline' / 'frontend prod'),同 service_account 内不强制唯一",
|
||||||
|
)
|
||||||
|
scopes: Mapped[str] = mapped_column(
|
||||||
|
String(1024),
|
||||||
|
nullable=False,
|
||||||
|
default="",
|
||||||
|
comment="scope 列表,逗号分隔字符串(如 'threads:read,threads:write');用 String 而非 PG text[] 以保 SQLite dev 兼容,Stage 2 切纯 PG 后可平滑迁",
|
||||||
|
)
|
||||||
|
rate_limit_rpm: Mapped[int | None] = mapped_column(
|
||||||
|
Integer,
|
||||||
|
nullable=True,
|
||||||
|
comment="每分钟请求数限制;NULL 表示走该 service_account 的默认限速(Stage 1 起生效)",
|
||||||
|
)
|
||||||
|
expires_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
comment="过期时间(UTC);NULL = 不过期",
|
||||||
|
)
|
||||||
|
last_used_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
comment="最近一次成功鉴权时间(UTC);Stage 1 鉴权中间件每次更新",
|
||||||
|
)
|
||||||
|
revoked_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
comment="撤销时间(UTC);NULL = 仍然有效。被撤销的 key 不删行(保留审计),但鉴权层据此拒绝",
|
||||||
|
)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
default=lambda: datetime.now(UTC),
|
||||||
|
comment="创建时间(UTC)",
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("idx_api_keys_sa", "service_account_id"),
|
||||||
|
# 部分索引:只索引活跃 key(revoked_at IS NULL),鉴权热路径走 prefix lookup,
|
||||||
|
# 撤销后的 key 不进活跃索引以减小热索引大小。SQLite + Postgres 均支持
|
||||||
|
# WHERE 子句的部分索引;双驱动维护两套等价 where 表达式。
|
||||||
|
Index(
|
||||||
|
"idx_api_keys_active",
|
||||||
|
"key_prefix",
|
||||||
|
sqlite_where=text("revoked_at IS NULL"),
|
||||||
|
postgresql_where=text("revoked_at IS NULL"),
|
||||||
|
),
|
||||||
|
{
|
||||||
|
"comment": (
|
||||||
|
"API key 表(headless API 凭证)。每行属于唯一 service_account;key_prefix 全局唯一可在日志中打印,"
|
||||||
|
"key_hash 是完整 token 的 sha-256,plaintext token 只在创建时返给调用方。撤销保留行(revoked_at 非空),"
|
||||||
|
"活跃 key 走部分索引 idx_api_keys_active 加速鉴权热路径。Stage 0 仅落 schema;Stage 1 起接鉴权 + 限速。"
|
||||||
|
)
|
||||||
|
},
|
||||||
|
)
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
"""External user persistence — ORM model only (Stage 0 PR8).
|
||||||
|
|
||||||
|
An external user represents the end-user identity that a
|
||||||
|
service_account passes through on each call (typically via an
|
||||||
|
``X-External-User-Id`` header). The row is upserted each time a
|
||||||
|
new ``external_id`` is seen under a given service_account.
|
||||||
|
|
||||||
|
PR8 introduces only the schema + ORM row class. The upsert logic,
|
||||||
|
header parsing, and quota attribution all live in Stage 1.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from deerflow.persistence.external_user.model import ExternalUserRow
|
||||||
|
|
||||||
|
__all__ = ["ExternalUserRow"]
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
"""ORM model for external users (end-user identities passed through a service account)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalUserRow(Base):
|
||||||
|
__tablename__ = "external_users"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
primary_key=True,
|
||||||
|
comment="external_user 主键,UUID 字符串(36 字符)",
|
||||||
|
)
|
||||||
|
workspace_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("workspaces.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
comment="所属 workspace(冗余存储——可经 service_account 间接得到,但直接存以加速 workspace-scope 查询);workspace 删除时级联",
|
||||||
|
)
|
||||||
|
service_account_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("service_accounts.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
comment="passthrough 的 service_account;service_account 删除时级联",
|
||||||
|
)
|
||||||
|
external_id: Mapped[str] = mapped_column(
|
||||||
|
String(128),
|
||||||
|
nullable=False,
|
||||||
|
comment="终端调用方传入的 X-External-User-Id(最多 128 字符;推荐 UUID / opaque token,不要塞 PII)",
|
||||||
|
)
|
||||||
|
display_name: Mapped[str | None] = mapped_column(
|
||||||
|
String(128),
|
||||||
|
nullable=True,
|
||||||
|
comment="可选显示名(如 'alice@customer.com');仅用于 admin UI 展示,不参与鉴权",
|
||||||
|
)
|
||||||
|
metadata_json: Mapped[dict[str, Any]] = mapped_column(
|
||||||
|
JSON,
|
||||||
|
nullable=False,
|
||||||
|
default=dict,
|
||||||
|
comment="任意 JSON 附属信息(plan tier / region / 自定义 tag);Stage 1 由 upsert 调用方写入",
|
||||||
|
)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
default=lambda: datetime.now(UTC),
|
||||||
|
comment="首次见到该 external_id 的时间(UTC)",
|
||||||
|
)
|
||||||
|
last_seen_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=True,
|
||||||
|
comment="最近一次该 external_id 触发请求的时间(UTC);Stage 1 鉴权层每次更新",
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("service_account_id", "external_id", name="uq_external_users_sa_external"),
|
||||||
|
{
|
||||||
|
"comment": (
|
||||||
|
"终端用户身份表(passthrough 模式下的 end-user)。每行由 service_account 的鉴权中间件 upsert——同一 "
|
||||||
|
"(service_account_id, external_id) 组合只存一行。workspace_id 冗余存储以加速跨 SA 的 workspace-scope 聚合查询。"
|
||||||
|
"Stage 0 仅落 schema;Stage 1 起接 upsert / 配额聚合。"
|
||||||
|
)
|
||||||
|
},
|
||||||
|
)
|
||||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, String, Text, UniqueConstraint
|
from sqlalchemy import DateTime, ForeignKey, String, Text, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from deerflow.persistence.base import Base
|
from deerflow.persistence.base import Base
|
||||||
@@ -13,20 +13,22 @@ from deerflow.persistence.base import Base
|
|||||||
class FeedbackRow(Base):
|
class FeedbackRow(Base):
|
||||||
__tablename__ = "feedback"
|
__tablename__ = "feedback"
|
||||||
|
|
||||||
__table_args__ = (UniqueConstraint("thread_id", "run_id", "user_id", name="uq_feedback_thread_run_user"),)
|
__table_args__ = (
|
||||||
|
UniqueConstraint("thread_id", "run_id", "user_id", name="uq_feedback_thread_run_user"),
|
||||||
|
{"comment": "用户对运行结果的反馈(点赞/点踩 + 文字评论),(thread, run, user) 唯一"},
|
||||||
|
)
|
||||||
|
|
||||||
feedback_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
feedback_id: Mapped[str] = mapped_column(String(64), primary_key=True, comment="反馈主键")
|
||||||
run_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
run_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True, comment="关联的运行 ID(runs.run_id)")
|
||||||
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True, comment="关联的会话 ID(threads_meta.thread_id)")
|
||||||
user_id: Mapped[str | None] = mapped_column(String(64), index=True)
|
user_id: Mapped[str | None] = mapped_column(String(64), index=True, comment="反馈作者;为 NULL 表示历史无主数据")
|
||||||
message_id: Mapped[str | None] = mapped_column(String(64))
|
workspace_id: Mapped[str] = mapped_column(
|
||||||
# message_id is an optional RunEventStore event identifier —
|
String(36),
|
||||||
# allows feedback to target a specific message or the entire run
|
ForeignKey("workspaces.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
rating: Mapped[int] = mapped_column(nullable=False)
|
comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
|
||||||
# +1 (thumbs-up) or -1 (thumbs-down)
|
)
|
||||||
|
message_id: Mapped[str | None] = mapped_column(String(64), comment="可选的 RunEventStore 事件 ID;为 NULL 表示针对整次运行而非单条消息")
|
||||||
comment: Mapped[str | None] = mapped_column(Text)
|
rating: Mapped[int] = mapped_column(nullable=False, comment="评分:+1 点赞,-1 点踩")
|
||||||
# Optional text feedback from the user
|
comment: Mapped[str | None] = mapped_column(Text, comment="可选的文字评论")
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC)")
|
||||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
|
|
||||||
|
|||||||
@@ -13,6 +13,13 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|||||||
|
|
||||||
from deerflow.persistence.feedback.model import FeedbackRow
|
from deerflow.persistence.feedback.model import FeedbackRow
|
||||||
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
||||||
|
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
_AutoSentinel as _WorkspaceAutoSentinel,
|
||||||
|
)
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
resolve_workspace_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class FeedbackRepository:
|
class FeedbackRepository:
|
||||||
@@ -34,6 +41,7 @@ class FeedbackRepository:
|
|||||||
thread_id: str,
|
thread_id: str,
|
||||||
rating: int,
|
rating: int,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
message_id: str | None = None,
|
message_id: str | None = None,
|
||||||
comment: str | None = None,
|
comment: str | None = None,
|
||||||
) -> dict:
|
) -> dict:
|
||||||
@@ -41,11 +49,13 @@ class FeedbackRepository:
|
|||||||
if rating not in (1, -1):
|
if rating not in (1, -1):
|
||||||
raise ValueError(f"rating must be +1 or -1, got {rating}")
|
raise ValueError(f"rating must be +1 or -1, got {rating}")
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.create")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.create")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.create")
|
||||||
row = FeedbackRow(
|
row = FeedbackRow(
|
||||||
feedback_id=str(uuid.uuid4()),
|
feedback_id=str(uuid.uuid4()),
|
||||||
run_id=run_id,
|
run_id=run_id,
|
||||||
thread_id=thread_id,
|
thread_id=thread_id,
|
||||||
user_id=resolved_user_id,
|
user_id=resolved_user_id,
|
||||||
|
workspace_id=resolved_workspace_id,
|
||||||
message_id=message_id,
|
message_id=message_id,
|
||||||
rating=rating,
|
rating=rating,
|
||||||
comment=comment,
|
comment=comment,
|
||||||
@@ -62,12 +72,16 @@ class FeedbackRepository:
|
|||||||
feedback_id: str,
|
feedback_id: str,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> dict | None:
|
) -> dict | None:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.get")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.get")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.get")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(FeedbackRow, feedback_id)
|
row = await session.get(FeedbackRow, feedback_id)
|
||||||
if row is None:
|
if row is None:
|
||||||
return None
|
return None
|
||||||
|
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
|
||||||
|
return None
|
||||||
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
return None
|
return None
|
||||||
return self._row_to_dict(row)
|
return self._row_to_dict(row)
|
||||||
@@ -79,9 +93,13 @@ class FeedbackRepository:
|
|||||||
*,
|
*,
|
||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> list[dict]:
|
) -> list[dict]:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_run")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_run")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.list_by_run")
|
||||||
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id, FeedbackRow.run_id == run_id)
|
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id, FeedbackRow.run_id == run_id)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
|
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
|
||||||
stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit)
|
stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit)
|
||||||
@@ -95,9 +113,13 @@ class FeedbackRepository:
|
|||||||
*,
|
*,
|
||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> list[dict]:
|
) -> list[dict]:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.list_by_thread")
|
||||||
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id)
|
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
|
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
|
||||||
stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit)
|
stmt = stmt.order_by(FeedbackRow.created_at.asc()).limit(limit)
|
||||||
@@ -110,12 +132,16 @@ class FeedbackRepository:
|
|||||||
feedback_id: str,
|
feedback_id: str,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.delete")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(FeedbackRow, feedback_id)
|
row = await session.get(FeedbackRow, feedback_id)
|
||||||
if row is None:
|
if row is None:
|
||||||
return False
|
return False
|
||||||
|
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
|
||||||
|
return False
|
||||||
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
return False
|
return False
|
||||||
await session.delete(row)
|
await session.delete(row)
|
||||||
@@ -129,18 +155,22 @@ class FeedbackRepository:
|
|||||||
thread_id: str,
|
thread_id: str,
|
||||||
rating: int,
|
rating: int,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
comment: str | None = None,
|
comment: str | None = None,
|
||||||
) -> dict:
|
) -> dict:
|
||||||
"""Create or update feedback for (thread_id, run_id, user_id). rating must be +1 or -1."""
|
"""Create or update feedback for (thread_id, run_id, user_id). rating must be +1 or -1."""
|
||||||
if rating not in (1, -1):
|
if rating not in (1, -1):
|
||||||
raise ValueError(f"rating must be +1 or -1, got {rating}")
|
raise ValueError(f"rating must be +1 or -1, got {rating}")
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.upsert")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.upsert")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.upsert")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
stmt = select(FeedbackRow).where(
|
stmt = select(FeedbackRow).where(
|
||||||
FeedbackRow.thread_id == thread_id,
|
FeedbackRow.thread_id == thread_id,
|
||||||
FeedbackRow.run_id == run_id,
|
FeedbackRow.run_id == run_id,
|
||||||
FeedbackRow.user_id == resolved_user_id,
|
FeedbackRow.user_id == resolved_user_id,
|
||||||
)
|
)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
|
||||||
result = await session.execute(stmt)
|
result = await session.execute(stmt)
|
||||||
row = result.scalar_one_or_none()
|
row = result.scalar_one_or_none()
|
||||||
if row is not None:
|
if row is not None:
|
||||||
@@ -153,6 +183,7 @@ class FeedbackRepository:
|
|||||||
run_id=run_id,
|
run_id=run_id,
|
||||||
thread_id=thread_id,
|
thread_id=thread_id,
|
||||||
user_id=resolved_user_id,
|
user_id=resolved_user_id,
|
||||||
|
workspace_id=resolved_workspace_id,
|
||||||
rating=rating,
|
rating=rating,
|
||||||
comment=comment,
|
comment=comment,
|
||||||
created_at=datetime.now(UTC),
|
created_at=datetime.now(UTC),
|
||||||
@@ -168,15 +199,19 @@ class FeedbackRepository:
|
|||||||
thread_id: str,
|
thread_id: str,
|
||||||
run_id: str,
|
run_id: str,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""Delete the current user's feedback for a run. Returns True if a record was deleted."""
|
"""Delete the current user's feedback for a run. Returns True if a record was deleted."""
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete_by_run")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.delete_by_run")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.delete_by_run")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
stmt = select(FeedbackRow).where(
|
stmt = select(FeedbackRow).where(
|
||||||
FeedbackRow.thread_id == thread_id,
|
FeedbackRow.thread_id == thread_id,
|
||||||
FeedbackRow.run_id == run_id,
|
FeedbackRow.run_id == run_id,
|
||||||
FeedbackRow.user_id == resolved_user_id,
|
FeedbackRow.user_id == resolved_user_id,
|
||||||
)
|
)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
|
||||||
result = await session.execute(stmt)
|
result = await session.execute(stmt)
|
||||||
row = result.scalar_one_or_none()
|
row = result.scalar_one_or_none()
|
||||||
if row is None:
|
if row is None:
|
||||||
@@ -190,10 +225,14 @@ class FeedbackRepository:
|
|||||||
thread_id: str,
|
thread_id: str,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> dict[str, dict]:
|
) -> dict[str, dict]:
|
||||||
"""Return feedback grouped by run_id for a thread: {run_id: feedback_dict}."""
|
"""Return feedback grouped by run_id for a thread: {run_id: feedback_dict}."""
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread_grouped")
|
resolved_user_id = resolve_user_id(user_id, method_name="FeedbackRepository.list_by_thread_grouped")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="FeedbackRepository.list_by_thread_grouped")
|
||||||
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id)
|
stmt = select(FeedbackRow).where(FeedbackRow.thread_id == thread_id)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(FeedbackRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
|
stmt = stmt.where(FeedbackRow.user_id == resolved_user_id)
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
|
|||||||
+44
@@ -0,0 +1,44 @@
|
|||||||
|
"""users.default_workspace_id column + FK to workspaces
|
||||||
|
|
||||||
|
Revision ID: 0001_users_default_workspace
|
||||||
|
Revises: None
|
||||||
|
Create Date: 2026-05-12
|
||||||
|
|
||||||
|
First Alembic revision for the DeerFlow application schema. Adds
|
||||||
|
`users.default_workspace_id` so a newly-registered user can be sent
|
||||||
|
back to their default workspace on next login without consulting the
|
||||||
|
memberships table.
|
||||||
|
|
||||||
|
Existing deployments created `users` via `metadata.create_all()` without
|
||||||
|
this column; running `alembic upgrade head` on those DBs will simply add
|
||||||
|
the column (ON DELETE SET NULL FK), no data backfill needed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
# Alembic identifiers.
|
||||||
|
revision: str = "0001_users_default_workspace"
|
||||||
|
down_revision: str | None = None
|
||||||
|
branch_labels: str | None = None
|
||||||
|
depends_on: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table("users") as batch:
|
||||||
|
batch.add_column(sa.Column("default_workspace_id", sa.String(36), nullable=True))
|
||||||
|
batch.create_foreign_key(
|
||||||
|
"fk_users_default_workspace",
|
||||||
|
"workspaces",
|
||||||
|
["default_workspace_id"],
|
||||||
|
["id"],
|
||||||
|
ondelete="SET NULL",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("users") as batch:
|
||||||
|
batch.drop_constraint("fk_users_default_workspace", type_="foreignkey")
|
||||||
|
batch.drop_column("default_workspace_id")
|
||||||
+70
@@ -0,0 +1,70 @@
|
|||||||
|
"""Business tables: nullable workspace_id + FK to workspaces
|
||||||
|
|
||||||
|
Revision ID: 0002_business_tables_workspace
|
||||||
|
Revises: 0001_users_default_workspace
|
||||||
|
Create Date: 2026-05-13
|
||||||
|
|
||||||
|
Stage 0 PR5 step 1/2 (the second step lives in revision 0003).
|
||||||
|
|
||||||
|
This revision adds a *nullable* ``workspace_id`` column to the four
|
||||||
|
business tables that need tenancy scoping:
|
||||||
|
|
||||||
|
* ``threads_meta``
|
||||||
|
* ``runs``
|
||||||
|
* ``feedback``
|
||||||
|
* ``run_events``
|
||||||
|
|
||||||
|
The column is nullable here on purpose — running ``upgrade`` on a
|
||||||
|
database with existing rows leaves those rows with ``workspace_id = NULL``
|
||||||
|
until ``scripts/backfill_workspace_id.py`` populates them. Once the
|
||||||
|
backfill finishes, revision 0003 flips the column to ``NOT NULL`` and
|
||||||
|
adds the threads_meta ``(workspace_id, thread_id)`` UNIQUE index.
|
||||||
|
|
||||||
|
A composite index ``idx_threads_meta_workspace_user_updated`` is added
|
||||||
|
on ``threads_meta`` to support the common "list a workspace's threads
|
||||||
|
for a user, newest first" access pattern that PR6 routers will rely on.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
revision: str = "0002_business_tables_workspace"
|
||||||
|
down_revision: str | None = "0001_users_default_workspace"
|
||||||
|
branch_labels: str | None = None
|
||||||
|
depends_on: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
# Tables that receive the new column. The order matters only for human
|
||||||
|
# readability in migration logs — there are no inter-table data deps
|
||||||
|
# during ALTER, and the FK references workspaces (introduced in PR3) which
|
||||||
|
# is already present at this point.
|
||||||
|
_BUSINESS_TABLES = ("threads_meta", "runs", "feedback", "run_events")
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
with op.batch_alter_table(table) as batch:
|
||||||
|
batch.add_column(sa.Column("workspace_id", sa.String(36), nullable=True))
|
||||||
|
batch.create_foreign_key(
|
||||||
|
f"fk_{table}_workspace_id",
|
||||||
|
"workspaces",
|
||||||
|
["workspace_id"],
|
||||||
|
["id"],
|
||||||
|
ondelete="CASCADE",
|
||||||
|
)
|
||||||
|
|
||||||
|
op.create_index(
|
||||||
|
"idx_threads_meta_workspace_user_updated",
|
||||||
|
"threads_meta",
|
||||||
|
["workspace_id", "user_id", "updated_at"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_index("idx_threads_meta_workspace_user_updated", table_name="threads_meta")
|
||||||
|
for table in reversed(_BUSINESS_TABLES):
|
||||||
|
with op.batch_alter_table(table) as batch:
|
||||||
|
batch.drop_constraint(f"fk_{table}_workspace_id", type_="foreignkey")
|
||||||
|
batch.drop_column("workspace_id")
|
||||||
+62
@@ -0,0 +1,62 @@
|
|||||||
|
"""Business tables: workspace_id NOT NULL + UNIQUE(workspace_id, thread_id)
|
||||||
|
|
||||||
|
Revision ID: 0003_business_tables_workspace_not_null
|
||||||
|
Revises: 0002_business_tables_workspace
|
||||||
|
Create Date: 2026-05-13
|
||||||
|
|
||||||
|
Stage 0 PR5 step 2/2. Flips ``workspace_id`` on the four business
|
||||||
|
tables to ``NOT NULL`` and adds the threads_meta ``(workspace_id,
|
||||||
|
thread_id)`` UNIQUE index promised in workspace-schema-design §4.
|
||||||
|
|
||||||
|
**Refuses to upgrade** if any of the four tables still has rows with
|
||||||
|
``workspace_id IS NULL`` — the operator must run
|
||||||
|
``scripts/backfill_workspace_id.py`` first. Doing the NOT NULL ALTER
|
||||||
|
with stragglers in place would either fail (Postgres) or silently
|
||||||
|
corrupt SQLite tables via ``batch_alter_table`` rebuilds.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
revision: str = "0003_business_tables_workspace_not_null"
|
||||||
|
down_revision: str | None = "0002_business_tables_workspace"
|
||||||
|
branch_labels: str | None = None
|
||||||
|
depends_on: str | None = None
|
||||||
|
|
||||||
|
_BUSINESS_TABLES = ("threads_meta", "runs", "feedback", "run_events")
|
||||||
|
|
||||||
|
|
||||||
|
class _BackfillRequiredError(RuntimeError):
|
||||||
|
"""Raised when null workspace_id rows remain at the start of upgrade."""
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
# Quoted identifier is fine here — table names are constants in this
|
||||||
|
# module, no operator input reaches the SQL.
|
||||||
|
count = conn.execute(sa.text(f"SELECT count(*) FROM {table} WHERE workspace_id IS NULL")).scalar() or 0
|
||||||
|
if count > 0:
|
||||||
|
raise _BackfillRequiredError(
|
||||||
|
f"Cannot ALTER {table}.workspace_id to NOT NULL: {count} row(s) still have workspace_id=NULL. Run `python scripts/backfill_workspace_id.py` first.",
|
||||||
|
)
|
||||||
|
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
with op.batch_alter_table(table) as batch:
|
||||||
|
batch.alter_column("workspace_id", existing_type=sa.String(36), nullable=False)
|
||||||
|
|
||||||
|
op.create_index(
|
||||||
|
"idx_threads_meta_workspace_thread",
|
||||||
|
"threads_meta",
|
||||||
|
["workspace_id", "thread_id"],
|
||||||
|
unique=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_index("idx_threads_meta_workspace_thread", table_name="threads_meta")
|
||||||
|
for table in reversed(_BUSINESS_TABLES):
|
||||||
|
with op.batch_alter_table(table) as batch:
|
||||||
|
batch.alter_column("workspace_id", existing_type=sa.String(36), nullable=True)
|
||||||
@@ -8,16 +8,37 @@ The actual ORM classes have moved to entity-specific subpackages:
|
|||||||
- ``deerflow.persistence.run``
|
- ``deerflow.persistence.run``
|
||||||
- ``deerflow.persistence.feedback``
|
- ``deerflow.persistence.feedback``
|
||||||
- ``deerflow.persistence.user``
|
- ``deerflow.persistence.user``
|
||||||
|
- ``deerflow.persistence.workspace`` (Stage 0 PR3)
|
||||||
|
- ``deerflow.persistence.workspace_membership`` (Stage 0 PR3)
|
||||||
|
- ``deerflow.persistence.service_account`` (Stage 0 PR8)
|
||||||
|
- ``deerflow.persistence.api_key`` (Stage 0 PR8)
|
||||||
|
- ``deerflow.persistence.external_user`` (Stage 0 PR8)
|
||||||
|
|
||||||
``RunEventRow`` remains in ``deerflow.persistence.models.run_event`` because
|
``RunEventRow`` remains in ``deerflow.persistence.models.run_event`` because
|
||||||
its storage implementation lives in ``deerflow.runtime.events.store.db`` and
|
its storage implementation lives in ``deerflow.runtime.events.store.db`` and
|
||||||
there is no matching entity directory.
|
there is no matching entity directory.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
from deerflow.persistence.api_key.model import ApiKeyRow
|
||||||
|
from deerflow.persistence.external_user.model import ExternalUserRow
|
||||||
from deerflow.persistence.feedback.model import FeedbackRow
|
from deerflow.persistence.feedback.model import FeedbackRow
|
||||||
from deerflow.persistence.models.run_event import RunEventRow
|
from deerflow.persistence.models.run_event import RunEventRow
|
||||||
from deerflow.persistence.run.model import RunRow
|
from deerflow.persistence.run.model import RunRow
|
||||||
|
from deerflow.persistence.service_account.model import ServiceAccountRow
|
||||||
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
||||||
from deerflow.persistence.user.model import UserRow
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
|
||||||
__all__ = ["FeedbackRow", "RunEventRow", "RunRow", "ThreadMetaRow", "UserRow"]
|
__all__ = [
|
||||||
|
"ApiKeyRow",
|
||||||
|
"ExternalUserRow",
|
||||||
|
"FeedbackRow",
|
||||||
|
"RunEventRow",
|
||||||
|
"RunRow",
|
||||||
|
"ServiceAccountRow",
|
||||||
|
"ThreadMetaRow",
|
||||||
|
"UserRow",
|
||||||
|
"WorkspaceMembershipRow",
|
||||||
|
"WorkspaceRow",
|
||||||
|
]
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
from sqlalchemy import JSON, DateTime, Index, String, Text, UniqueConstraint
|
from sqlalchemy import JSON, DateTime, ForeignKey, Index, String, Text, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from deerflow.persistence.base import Base
|
from deerflow.persistence.base import Base
|
||||||
@@ -13,23 +13,31 @@ from deerflow.persistence.base import Base
|
|||||||
class RunEventRow(Base):
|
class RunEventRow(Base):
|
||||||
__tablename__ = "run_events"
|
__tablename__ = "run_events"
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
|
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True, comment="自增主键")
|
||||||
thread_id: Mapped[str] = mapped_column(String(64), nullable=False)
|
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, comment="所属会话 ID(threads_meta.thread_id)")
|
||||||
run_id: Mapped[str] = mapped_column(String(64), nullable=False)
|
run_id: Mapped[str] = mapped_column(String(64), nullable=False, comment="所属运行 ID(runs.run_id)")
|
||||||
# Owner of the conversation this event belongs to. Nullable for data
|
user_id: Mapped[str | None] = mapped_column(
|
||||||
# created before auth was introduced; populated by auth middleware on
|
String(64),
|
||||||
# new writes and by the boot-time orphan migration on existing rows.
|
nullable=True,
|
||||||
user_id: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
|
index=True,
|
||||||
event_type: Mapped[str] = mapped_column(String(32), nullable=False)
|
comment="会话所有者;为 NULL 表示鉴权引入之前的历史数据,新写入由 auth 中间件填充,启动期 orphan 迁移会回填存量",
|
||||||
category: Mapped[str] = mapped_column(String(16), nullable=False)
|
)
|
||||||
# "message" | "trace" | "lifecycle"
|
workspace_id: Mapped[str] = mapped_column(
|
||||||
content: Mapped[str] = mapped_column(Text, default="")
|
String(36),
|
||||||
event_metadata: Mapped[dict] = mapped_column(JSON, default=dict)
|
ForeignKey("workspaces.id", ondelete="CASCADE"),
|
||||||
seq: Mapped[int] = mapped_column(nullable=False)
|
nullable=False,
|
||||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
|
comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
|
||||||
|
)
|
||||||
|
event_type: Mapped[str] = mapped_column(String(32), nullable=False, comment="事件子类型(具体含义由 category 决定,如 ai_message_chunk、tool_call、run_started)")
|
||||||
|
category: Mapped[str] = mapped_column(String(16), nullable=False, comment='事件大类:"message" 消息 / "trace" 追踪 / "lifecycle" 生命周期')
|
||||||
|
content: Mapped[str] = mapped_column(Text, default="", comment="事件文本内容(消息体、错误、状态字符串等)")
|
||||||
|
event_metadata: Mapped[dict] = mapped_column(JSON, default=dict, comment="事件结构化元数据(JSON),随 event_type 而异")
|
||||||
|
seq: Mapped[int] = mapped_column(nullable=False, comment="在 thread 内的全局递增序号;与 thread_id 组合唯一")
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC)")
|
||||||
|
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
UniqueConstraint("thread_id", "seq", name="uq_events_thread_seq"),
|
UniqueConstraint("thread_id", "seq", name="uq_events_thread_seq"),
|
||||||
Index("ix_events_thread_cat_seq", "thread_id", "category", "seq"),
|
Index("ix_events_thread_cat_seq", "thread_id", "category", "seq"),
|
||||||
Index("ix_events_run", "thread_id", "run_id", "seq"),
|
Index("ix_events_run", "thread_id", "run_id", "seq"),
|
||||||
|
{"comment": "运行事件流(消息/追踪/生命周期事件按 seq 顺序追加,是消息回放与审计的真源)"},
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
from sqlalchemy import JSON, DateTime, Index, String, Text
|
from sqlalchemy import JSON, DateTime, ForeignKey, Index, String, Text
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from deerflow.persistence.base import Base
|
from deerflow.persistence.base import Base
|
||||||
@@ -13,37 +13,49 @@ from deerflow.persistence.base import Base
|
|||||||
class RunRow(Base):
|
class RunRow(Base):
|
||||||
__tablename__ = "runs"
|
__tablename__ = "runs"
|
||||||
|
|
||||||
run_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
run_id: Mapped[str] = mapped_column(String(64), primary_key=True, comment="运行主键")
|
||||||
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True, comment="所属会话 ID(threads_meta.thread_id)")
|
||||||
assistant_id: Mapped[str | None] = mapped_column(String(128))
|
assistant_id: Mapped[str | None] = mapped_column(String(128), comment="使用的 Assistant ID(自定义智能体名);为 NULL 表示默认 lead agent")
|
||||||
user_id: Mapped[str | None] = mapped_column(String(64), index=True)
|
user_id: Mapped[str | None] = mapped_column(String(64), index=True, comment="发起本次运行的用户 ID")
|
||||||
status: Mapped[str] = mapped_column(String(20), default="pending")
|
workspace_id: Mapped[str] = mapped_column(
|
||||||
# "pending" | "running" | "success" | "error" | "timeout" | "interrupted"
|
String(36),
|
||||||
|
ForeignKey("workspaces.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
|
||||||
|
)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(20),
|
||||||
|
default="pending",
|
||||||
|
comment='运行状态:"pending" / "running" / "success" / "error" / "timeout" / "interrupted"',
|
||||||
|
)
|
||||||
|
model_name: Mapped[str | None] = mapped_column(String(128), comment="本次运行的主模型名(来自 config.yaml.models[*].name)")
|
||||||
|
multitask_strategy: Mapped[str] = mapped_column(
|
||||||
|
String(20),
|
||||||
|
default="reject",
|
||||||
|
comment='并发策略:同一 thread 已有运行时怎么处理("reject" / "interrupt" / "rollback" / "enqueue")',
|
||||||
|
)
|
||||||
|
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict, comment="运行级元数据(JSON),如 channel/source 等")
|
||||||
|
kwargs_json: Mapped[dict] = mapped_column(JSON, default=dict, comment="提交运行时的额外参数(JSON),如 thinking_enabled、tool 配置等")
|
||||||
|
error: Mapped[str | None] = mapped_column(Text, comment="运行失败时的错误文本;成功时为 NULL")
|
||||||
|
|
||||||
model_name: Mapped[str | None] = mapped_column(String(128))
|
message_count: Mapped[int] = mapped_column(default=0, comment="本次运行产生的消息总数(便利字段,避免列表页查 RunEventStore)")
|
||||||
multitask_strategy: Mapped[str] = mapped_column(String(20), default="reject")
|
first_human_message: Mapped[str | None] = mapped_column(Text, comment="首条用户消息文本预览(用于列表展示)")
|
||||||
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict)
|
last_ai_message: Mapped[str | None] = mapped_column(Text, comment="末条 AI 消息文本预览(用于列表展示)")
|
||||||
kwargs_json: Mapped[dict] = mapped_column(JSON, default=dict)
|
|
||||||
error: Mapped[str | None] = mapped_column(Text)
|
|
||||||
|
|
||||||
# Convenience fields (for listing pages without querying RunEventStore)
|
total_input_tokens: Mapped[int] = mapped_column(default=0, comment="累计输入 token 数(运行结束时由 RunJournal 落盘)")
|
||||||
message_count: Mapped[int] = mapped_column(default=0)
|
total_output_tokens: Mapped[int] = mapped_column(default=0, comment="累计输出 token 数")
|
||||||
first_human_message: Mapped[str | None] = mapped_column(Text)
|
total_tokens: Mapped[int] = mapped_column(default=0, comment="累计 token 总数 = input + output")
|
||||||
last_ai_message: Mapped[str | None] = mapped_column(Text)
|
llm_call_count: Mapped[int] = mapped_column(default=0, comment="累计 LLM 调用次数")
|
||||||
|
lead_agent_tokens: Mapped[int] = mapped_column(default=0, comment="主 agent 自身消耗的 token 数")
|
||||||
|
subagent_tokens: Mapped[int] = mapped_column(default=0, comment="子 agent(task 工具委派)消耗的 token 数")
|
||||||
|
middleware_tokens: Mapped[int] = mapped_column(default=0, comment="中间件(如 summarization、title)消耗的 token 数")
|
||||||
|
|
||||||
# Token usage (accumulated in-memory by RunJournal, written on run completion)
|
follow_up_to_run_id: Mapped[str | None] = mapped_column(String(64), comment="续接的上一次运行 ID(用于'重新生成'/'继续'等链式调用)")
|
||||||
total_input_tokens: Mapped[int] = mapped_column(default=0)
|
|
||||||
total_output_tokens: Mapped[int] = mapped_column(default=0)
|
|
||||||
total_tokens: Mapped[int] = mapped_column(default=0)
|
|
||||||
llm_call_count: Mapped[int] = mapped_column(default=0)
|
|
||||||
lead_agent_tokens: Mapped[int] = mapped_column(default=0)
|
|
||||||
subagent_tokens: Mapped[int] = mapped_column(default=0)
|
|
||||||
middleware_tokens: Mapped[int] = mapped_column(default=0)
|
|
||||||
|
|
||||||
# Follow-up association
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC)")
|
||||||
follow_up_to_run_id: Mapped[str | None] = mapped_column(String(64))
|
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC), comment="最近更新时间(UTC,写入时自动更新)")
|
||||||
|
|
||||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
|
__table_args__ = (
|
||||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC))
|
Index("ix_runs_thread_status", "thread_id", "status"),
|
||||||
|
{"comment": "运行(一次完整 agent 执行)的元数据 + 累计 token 指标"},
|
||||||
__table_args__ = (Index("ix_runs_thread_status", "thread_id", "status"),)
|
)
|
||||||
|
|||||||
@@ -17,6 +17,13 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|||||||
from deerflow.persistence.run.model import RunRow
|
from deerflow.persistence.run.model import RunRow
|
||||||
from deerflow.runtime.runs.store.base import RunStore
|
from deerflow.runtime.runs.store.base import RunStore
|
||||||
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
||||||
|
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
_AutoSentinel as _WorkspaceAutoSentinel,
|
||||||
|
)
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
resolve_workspace_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class RunRepository(RunStore):
|
class RunRepository(RunStore):
|
||||||
@@ -70,6 +77,7 @@ class RunRepository(RunStore):
|
|||||||
thread_id,
|
thread_id,
|
||||||
assistant_id=None,
|
assistant_id=None,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
status="pending",
|
status="pending",
|
||||||
multitask_strategy="reject",
|
multitask_strategy="reject",
|
||||||
metadata=None,
|
metadata=None,
|
||||||
@@ -79,12 +87,14 @@ class RunRepository(RunStore):
|
|||||||
follow_up_to_run_id=None,
|
follow_up_to_run_id=None,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.put")
|
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.put")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.put")
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
row = RunRow(
|
row = RunRow(
|
||||||
run_id=run_id,
|
run_id=run_id,
|
||||||
thread_id=thread_id,
|
thread_id=thread_id,
|
||||||
assistant_id=assistant_id,
|
assistant_id=assistant_id,
|
||||||
user_id=resolved_user_id,
|
user_id=resolved_user_id,
|
||||||
|
workspace_id=resolved_workspace_id,
|
||||||
status=status,
|
status=status,
|
||||||
multitask_strategy=multitask_strategy,
|
multitask_strategy=multitask_strategy,
|
||||||
metadata_json=self._safe_json(metadata) or {},
|
metadata_json=self._safe_json(metadata) or {},
|
||||||
@@ -103,12 +113,16 @@ class RunRepository(RunStore):
|
|||||||
run_id,
|
run_id,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.get")
|
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.get")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.get")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(RunRow, run_id)
|
row = await session.get(RunRow, run_id)
|
||||||
if row is None:
|
if row is None:
|
||||||
return None
|
return None
|
||||||
|
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
|
||||||
|
return None
|
||||||
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
return None
|
return None
|
||||||
return self._row_to_dict(row)
|
return self._row_to_dict(row)
|
||||||
@@ -118,10 +132,14 @@ class RunRepository(RunStore):
|
|||||||
thread_id,
|
thread_id,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
limit=100,
|
limit=100,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.list_by_thread")
|
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.list_by_thread")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.list_by_thread")
|
||||||
stmt = select(RunRow).where(RunRow.thread_id == thread_id)
|
stmt = select(RunRow).where(RunRow.thread_id == thread_id)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(RunRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(RunRow.user_id == resolved_user_id)
|
stmt = stmt.where(RunRow.user_id == resolved_user_id)
|
||||||
stmt = stmt.order_by(RunRow.created_at.desc()).limit(limit)
|
stmt = stmt.order_by(RunRow.created_at.desc()).limit(limit)
|
||||||
@@ -142,12 +160,16 @@ class RunRepository(RunStore):
|
|||||||
run_id,
|
run_id,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.delete")
|
resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.delete")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="RunRepository.delete")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(RunRow, run_id)
|
row = await session.get(RunRow, run_id)
|
||||||
if row is None:
|
if row is None:
|
||||||
return
|
return
|
||||||
|
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
|
||||||
|
return
|
||||||
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
return
|
return
|
||||||
await session.delete(row)
|
await session.delete(row)
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
"""Service account persistence — ORM model only (Stage 0 PR8).
|
||||||
|
|
||||||
|
A service account is a non-human principal that lives inside a workspace
|
||||||
|
and authenticates via API keys rather than email + password. Each
|
||||||
|
service account belongs to exactly one workspace and is created by a
|
||||||
|
human user (``created_by``).
|
||||||
|
|
||||||
|
PR8 introduces only the schema + ORM row class. Repository, API-key
|
||||||
|
authentication middleware, and the ``@require_permission`` scope
|
||||||
|
upgrade live in Stage 1 alongside the headless API surface.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from deerflow.persistence.service_account.model import ServiceAccountRow
|
||||||
|
|
||||||
|
__all__ = ["ServiceAccountRow"]
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
"""ORM model for service accounts (non-human principals inside a workspace)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from sqlalchemy import DateTime, ForeignKey, Index, String
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class ServiceAccountRow(Base):
|
||||||
|
__tablename__ = "service_accounts"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
primary_key=True,
|
||||||
|
comment="服务账号主键,UUID 字符串(36 字符),与 users.id 类型对齐",
|
||||||
|
)
|
||||||
|
workspace_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("workspaces.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
comment="所属 workspace;workspace 删除时级联清掉所有 service_account(连同其 api_keys / external_users)",
|
||||||
|
)
|
||||||
|
name: Mapped[str] = mapped_column(
|
||||||
|
String(64),
|
||||||
|
nullable=False,
|
||||||
|
comment="服务账号显示名(同 workspace 内不强制唯一;Stage 1 可由 admin UI 重复使用同名 + 不同 key)",
|
||||||
|
)
|
||||||
|
role: Mapped[str] = mapped_column(
|
||||||
|
String(16),
|
||||||
|
nullable=False,
|
||||||
|
default="member",
|
||||||
|
comment='服务账号在 workspace 内的角色字符串:Stage 0 仅支持 "member";Stage 2 RBAC 打开 "admin"/"viewer"。用 String(16) 而非 enum 以便未来扩枚举值不动 schema',
|
||||||
|
)
|
||||||
|
identity_mode: Mapped[str] = mapped_column(
|
||||||
|
String(16),
|
||||||
|
nullable=False,
|
||||||
|
default="collapsed",
|
||||||
|
comment=(
|
||||||
|
'身份模式三态:"collapsed"(所有调用 collapse 到该 service_account;不记录 external_user)/'
|
||||||
|
' "external_passthrough"(每次调用必带 X-External-User-Id,写入 external_users 表)/'
|
||||||
|
' "both"(带就写、不带就 collapse)。Stage 1 API key 鉴权层据此分流'
|
||||||
|
),
|
||||||
|
)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(16),
|
||||||
|
nullable=False,
|
||||||
|
default="active",
|
||||||
|
comment='状态:"active"(正常)/ "suspended"(admin 暂停)/ "deleted"(软删;保留审计)',
|
||||||
|
)
|
||||||
|
created_by: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("users.id", ondelete="RESTRICT"),
|
||||||
|
nullable=False,
|
||||||
|
comment="创建者 user_id;删除该 user 时 RESTRICT 阻拦(必须先转移或删除该 user 名下所有 service_account)",
|
||||||
|
)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
default=lambda: datetime.now(UTC),
|
||||||
|
comment="创建时间(UTC)",
|
||||||
|
)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
default=lambda: datetime.now(UTC),
|
||||||
|
onupdate=lambda: datetime.now(UTC),
|
||||||
|
comment="最近更新时间(UTC,写入时自动更新)",
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("idx_service_accounts_workspace", "workspace_id", "status"),
|
||||||
|
{
|
||||||
|
"comment": (
|
||||||
|
"服务账号表(headless API 的非人身份)。每个 service_account 属于唯一 workspace;"
|
||||||
|
"通过 api_keys 表的 API key 鉴权调用 Gateway;identity_mode 控制是否在 external_users 表"
|
||||||
|
"记录终端用户身份。Stage 0 仅落 schema;Stage 1 起接 API key 鉴权 + 路由 scope 升级。"
|
||||||
|
)
|
||||||
|
},
|
||||||
|
)
|
||||||
@@ -4,12 +4,18 @@ Implementations:
|
|||||||
- ThreadMetaRepository: SQL-backed (sqlite / postgres via SQLAlchemy)
|
- ThreadMetaRepository: SQL-backed (sqlite / postgres via SQLAlchemy)
|
||||||
- MemoryThreadMetaStore: wraps LangGraph BaseStore (memory mode)
|
- MemoryThreadMetaStore: wraps LangGraph BaseStore (memory mode)
|
||||||
|
|
||||||
All mutating and querying methods accept a ``user_id`` parameter with
|
All mutating and querying methods accept both a ``user_id`` parameter
|
||||||
three-state semantics (see :mod:`deerflow.runtime.user_context`):
|
(member-scoped owner check) and a ``workspace_id`` parameter (tenant
|
||||||
|
scope). Both follow three-state semantics:
|
||||||
|
|
||||||
- ``AUTO`` (default): resolve from the request-scoped contextvar.
|
- ``AUTO`` (default): resolve from the request-scoped contextvar.
|
||||||
- Explicit ``str``: use the provided value verbatim.
|
- Explicit ``str``: use the provided value verbatim.
|
||||||
- Explicit ``None``: bypass owner filtering (migration/CLI only).
|
- Explicit ``None``: bypass that filter (migration / CLI only).
|
||||||
|
|
||||||
|
The workspace scope is the **outer** boundary: a row in workspace A is
|
||||||
|
unreachable from any user_id under workspace B. ``check_access`` returns
|
||||||
|
False on cross-workspace mismatch so the route layer can convert it into
|
||||||
|
a 404 instead of leaking thread existence across tenants.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -17,6 +23,8 @@ from __future__ import annotations
|
|||||||
import abc
|
import abc
|
||||||
|
|
||||||
from deerflow.runtime.user_context import AUTO, _AutoSentinel
|
from deerflow.runtime.user_context import AUTO, _AutoSentinel
|
||||||
|
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
|
||||||
|
from deerflow.runtime.workspace_context import _AutoSentinel as _WorkspaceAutoSentinel
|
||||||
|
|
||||||
|
|
||||||
class ThreadMetaStore(abc.ABC):
|
class ThreadMetaStore(abc.ABC):
|
||||||
@@ -27,13 +35,20 @@ class ThreadMetaStore(abc.ABC):
|
|||||||
*,
|
*,
|
||||||
assistant_id: str | None = None,
|
assistant_id: str | None = None,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
display_name: str | None = None,
|
display_name: str | None = None,
|
||||||
metadata: dict | None = None,
|
metadata: dict | None = None,
|
||||||
) -> dict:
|
) -> dict:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
async def get(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> dict | None:
|
async def get(
|
||||||
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> dict | None:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
@@ -45,32 +60,72 @@ class ThreadMetaStore(abc.ABC):
|
|||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
offset: int = 0,
|
offset: int = 0,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> list[dict]:
|
) -> list[dict]:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
async def update_display_name(self, thread_id: str, display_name: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def update_display_name(
|
||||||
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
display_name: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
async def update_status(self, thread_id: str, status: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def update_status(
|
||||||
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
status: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
async def update_metadata(self, thread_id: str, metadata: dict, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def update_metadata(
|
||||||
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
metadata: dict,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
"""Merge ``metadata`` into the thread's metadata field.
|
"""Merge ``metadata`` into the thread's metadata field.
|
||||||
|
|
||||||
Existing keys are overwritten by the new values; keys absent from
|
Existing keys are overwritten by the new values; keys absent from
|
||||||
``metadata`` are preserved. No-op if the thread does not exist
|
``metadata`` are preserved. No-op if the thread does not exist
|
||||||
or the owner check fails.
|
or the user/workspace check fails.
|
||||||
"""
|
"""
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
async def check_access(self, thread_id: str, user_id: str, *, require_existing: bool = False) -> bool:
|
async def check_access(
|
||||||
"""Check if ``user_id`` has access to ``thread_id``."""
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
user_id: str,
|
||||||
|
workspace_id: str,
|
||||||
|
*,
|
||||||
|
require_existing: bool = False,
|
||||||
|
) -> bool:
|
||||||
|
"""Check whether ``user_id`` (in ``workspace_id``) can access ``thread_id``.
|
||||||
|
|
||||||
|
Cross-workspace access returns ``False`` unconditionally so the
|
||||||
|
decorator layer can convert it into a 404 — never leak the
|
||||||
|
existence of a thread that belongs to a different tenant.
|
||||||
|
"""
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
async def delete(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def delete(
|
||||||
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
pass
|
pass
|
||||||
|
|||||||
@@ -13,6 +13,13 @@ from langgraph.store.base import BaseStore
|
|||||||
|
|
||||||
from deerflow.persistence.thread_meta.base import ThreadMetaStore
|
from deerflow.persistence.thread_meta.base import ThreadMetaStore
|
||||||
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
||||||
|
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
_AutoSentinel as _WorkspaceAutoSentinel,
|
||||||
|
)
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
resolve_workspace_id,
|
||||||
|
)
|
||||||
from deerflow.utils.time import coerce_iso, now_iso
|
from deerflow.utils.time import coerce_iso, now_iso
|
||||||
|
|
||||||
THREADS_NS: tuple[str, ...] = ("threads",)
|
THREADS_NS: tuple[str, ...] = ("threads",)
|
||||||
@@ -26,15 +33,19 @@ class MemoryThreadMetaStore(ThreadMetaStore):
|
|||||||
self,
|
self,
|
||||||
thread_id: str,
|
thread_id: str,
|
||||||
user_id: str | None | _AutoSentinel,
|
user_id: str | None | _AutoSentinel,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel,
|
||||||
method_name: str,
|
method_name: str,
|
||||||
) -> dict | None:
|
) -> dict | None:
|
||||||
"""Fetch a record and verify ownership. Returns a mutable copy, or None."""
|
"""Fetch a record and verify workspace + ownership. Returns a mutable copy, or None."""
|
||||||
resolved = resolve_user_id(user_id, method_name=method_name)
|
resolved_user = resolve_user_id(user_id, method_name=method_name)
|
||||||
|
resolved_workspace = resolve_workspace_id(workspace_id, method_name=method_name)
|
||||||
item = await self._store.aget(THREADS_NS, thread_id)
|
item = await self._store.aget(THREADS_NS, thread_id)
|
||||||
if item is None:
|
if item is None:
|
||||||
return None
|
return None
|
||||||
record = dict(item.value)
|
record = dict(item.value)
|
||||||
if resolved is not None and record.get("user_id") != resolved:
|
if resolved_workspace is not None and record.get("workspace_id") != resolved_workspace:
|
||||||
|
return None
|
||||||
|
if resolved_user is not None and record.get("user_id") != resolved_user:
|
||||||
return None
|
return None
|
||||||
return record
|
return record
|
||||||
|
|
||||||
@@ -44,15 +55,18 @@ class MemoryThreadMetaStore(ThreadMetaStore):
|
|||||||
*,
|
*,
|
||||||
assistant_id: str | None = None,
|
assistant_id: str | None = None,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
display_name: str | None = None,
|
display_name: str | None = None,
|
||||||
metadata: dict | None = None,
|
metadata: dict | None = None,
|
||||||
) -> dict:
|
) -> dict:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.create")
|
resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.create")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="MemoryThreadMetaStore.create")
|
||||||
now = now_iso()
|
now = now_iso()
|
||||||
record: dict[str, Any] = {
|
record: dict[str, Any] = {
|
||||||
"thread_id": thread_id,
|
"thread_id": thread_id,
|
||||||
"assistant_id": assistant_id,
|
"assistant_id": assistant_id,
|
||||||
"user_id": resolved_user_id,
|
"user_id": resolved_user_id,
|
||||||
|
"workspace_id": resolved_workspace_id,
|
||||||
"display_name": display_name,
|
"display_name": display_name,
|
||||||
"status": "idle",
|
"status": "idle",
|
||||||
"metadata": metadata or {},
|
"metadata": metadata or {},
|
||||||
@@ -63,8 +77,14 @@ class MemoryThreadMetaStore(ThreadMetaStore):
|
|||||||
await self._store.aput(THREADS_NS, thread_id, record)
|
await self._store.aput(THREADS_NS, thread_id, record)
|
||||||
return record
|
return record
|
||||||
|
|
||||||
async def get(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> dict | None:
|
async def get(
|
||||||
return await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.get")
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> dict | None:
|
||||||
|
return await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.get")
|
||||||
|
|
||||||
async def search(
|
async def search(
|
||||||
self,
|
self,
|
||||||
@@ -74,13 +94,17 @@ class MemoryThreadMetaStore(ThreadMetaStore):
|
|||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
offset: int = 0,
|
offset: int = 0,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> list[dict]:
|
) -> list[dict]:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.search")
|
resolved_user_id = resolve_user_id(user_id, method_name="MemoryThreadMetaStore.search")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="MemoryThreadMetaStore.search")
|
||||||
filter_dict: dict[str, Any] = {}
|
filter_dict: dict[str, Any] = {}
|
||||||
if metadata:
|
if metadata:
|
||||||
filter_dict.update(metadata)
|
filter_dict.update(metadata)
|
||||||
if status:
|
if status:
|
||||||
filter_dict["status"] = status
|
filter_dict["status"] = status
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
filter_dict["workspace_id"] = resolved_workspace_id
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
filter_dict["user_id"] = resolved_user_id
|
filter_dict["user_id"] = resolved_user_id
|
||||||
|
|
||||||
@@ -92,33 +116,64 @@ class MemoryThreadMetaStore(ThreadMetaStore):
|
|||||||
)
|
)
|
||||||
return [self._item_to_dict(item) for item in items]
|
return [self._item_to_dict(item) for item in items]
|
||||||
|
|
||||||
async def check_access(self, thread_id: str, user_id: str, *, require_existing: bool = False) -> bool:
|
async def check_access(
|
||||||
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
user_id: str,
|
||||||
|
workspace_id: str,
|
||||||
|
*,
|
||||||
|
require_existing: bool = False,
|
||||||
|
) -> bool:
|
||||||
item = await self._store.aget(THREADS_NS, thread_id)
|
item = await self._store.aget(THREADS_NS, thread_id)
|
||||||
if item is None:
|
if item is None:
|
||||||
return not require_existing
|
return not require_existing
|
||||||
|
record_workspace_id = item.value.get("workspace_id")
|
||||||
|
if record_workspace_id is not None and record_workspace_id != workspace_id:
|
||||||
|
return False
|
||||||
record_user_id = item.value.get("user_id")
|
record_user_id = item.value.get("user_id")
|
||||||
if record_user_id is None:
|
if record_user_id is None:
|
||||||
return True
|
return True
|
||||||
return record_user_id == user_id
|
return record_user_id == user_id
|
||||||
|
|
||||||
async def update_display_name(self, thread_id: str, display_name: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def update_display_name(
|
||||||
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.update_display_name")
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
display_name: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
|
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.update_display_name")
|
||||||
if record is None:
|
if record is None:
|
||||||
return
|
return
|
||||||
record["display_name"] = display_name
|
record["display_name"] = display_name
|
||||||
record["updated_at"] = now_iso()
|
record["updated_at"] = now_iso()
|
||||||
await self._store.aput(THREADS_NS, thread_id, record)
|
await self._store.aput(THREADS_NS, thread_id, record)
|
||||||
|
|
||||||
async def update_status(self, thread_id: str, status: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def update_status(
|
||||||
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.update_status")
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
status: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
|
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.update_status")
|
||||||
if record is None:
|
if record is None:
|
||||||
return
|
return
|
||||||
record["status"] = status
|
record["status"] = status
|
||||||
record["updated_at"] = now_iso()
|
record["updated_at"] = now_iso()
|
||||||
await self._store.aput(THREADS_NS, thread_id, record)
|
await self._store.aput(THREADS_NS, thread_id, record)
|
||||||
|
|
||||||
async def update_metadata(self, thread_id: str, metadata: dict, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def update_metadata(
|
||||||
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.update_metadata")
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
metadata: dict,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
|
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.update_metadata")
|
||||||
if record is None:
|
if record is None:
|
||||||
return
|
return
|
||||||
merged = dict(record.get("metadata") or {})
|
merged = dict(record.get("metadata") or {})
|
||||||
@@ -127,8 +182,14 @@ class MemoryThreadMetaStore(ThreadMetaStore):
|
|||||||
record["updated_at"] = now_iso()
|
record["updated_at"] = now_iso()
|
||||||
await self._store.aput(THREADS_NS, thread_id, record)
|
await self._store.aput(THREADS_NS, thread_id, record)
|
||||||
|
|
||||||
async def delete(self, thread_id: str, *, user_id: str | None | _AutoSentinel = AUTO) -> None:
|
async def delete(
|
||||||
record = await self._get_owned_record(thread_id, user_id, "MemoryThreadMetaStore.delete")
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
|
) -> None:
|
||||||
|
record = await self._get_owned_record(thread_id, user_id, workspace_id, "MemoryThreadMetaStore.delete")
|
||||||
if record is None:
|
if record is None:
|
||||||
return
|
return
|
||||||
await self._store.adelete(THREADS_NS, thread_id)
|
await self._store.adelete(THREADS_NS, thread_id)
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
from sqlalchemy import JSON, DateTime, String
|
from sqlalchemy import JSON, DateTime, ForeignKey, Index, String
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from deerflow.persistence.base import Base
|
from deerflow.persistence.base import Base
|
||||||
@@ -13,11 +13,25 @@ from deerflow.persistence.base import Base
|
|||||||
class ThreadMetaRow(Base):
|
class ThreadMetaRow(Base):
|
||||||
__tablename__ = "threads_meta"
|
__tablename__ = "threads_meta"
|
||||||
|
|
||||||
thread_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
thread_id: Mapped[str] = mapped_column(String(64), primary_key=True, comment="会话主键(LangGraph thread_id)")
|
||||||
assistant_id: Mapped[str | None] = mapped_column(String(128), index=True)
|
assistant_id: Mapped[str | None] = mapped_column(String(128), index=True, comment="关联的 Assistant ID(自定义智能体名);为 NULL 表示默认 lead agent")
|
||||||
user_id: Mapped[str | None] = mapped_column(String(64), index=True)
|
user_id: Mapped[str | None] = mapped_column(String(64), index=True, comment="会话所有者;为 NULL 表示历史无主数据")
|
||||||
display_name: Mapped[str | None] = mapped_column(String(256))
|
workspace_id: Mapped[str] = mapped_column(
|
||||||
status: Mapped[str] = mapped_column(String(20), default="idle")
|
String(36),
|
||||||
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict)
|
ForeignKey("workspaces.id", ondelete="CASCADE"),
|
||||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC))
|
nullable=False,
|
||||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC))
|
comment="所属 workspace。PR5 引入时 nullable 用于回填;alembic 0003 + PR6 仓储接入完成后 NOT NULL",
|
||||||
|
)
|
||||||
|
display_name: Mapped[str | None] = mapped_column(String(256), comment="会话显示名(自动生成的标题或用户手改)")
|
||||||
|
status: Mapped[str] = mapped_column(String(20), default="idle", comment='会话状态:"idle" 空闲 / "busy" 正在产出')
|
||||||
|
metadata_json: Mapped[dict] = mapped_column(JSON, default=dict, comment="任意扩展元数据(JSON)")
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), comment="创建时间(UTC)")
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC), comment="最近更新时间(UTC,写入时自动更新)")
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# Workspace-scoped "list threads of this user, newest first" index;
|
||||||
|
# added by alembic 0002. Mirrored on the ORM side so create_all()
|
||||||
|
# produces the same shape on fresh dev databases.
|
||||||
|
Index("idx_threads_meta_workspace_user_updated", "workspace_id", "user_id", "updated_at"),
|
||||||
|
{"comment": "会话元数据(每个 LangGraph thread 的概要信息)"},
|
||||||
|
)
|
||||||
|
|||||||
@@ -11,6 +11,13 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|||||||
from deerflow.persistence.thread_meta.base import ThreadMetaStore
|
from deerflow.persistence.thread_meta.base import ThreadMetaStore
|
||||||
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
||||||
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
||||||
|
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
_AutoSentinel as _WorkspaceAutoSentinel,
|
||||||
|
)
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
resolve_workspace_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class ThreadMetaRepository(ThreadMetaStore):
|
class ThreadMetaRepository(ThreadMetaStore):
|
||||||
@@ -33,17 +40,21 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
*,
|
*,
|
||||||
assistant_id: str | None = None,
|
assistant_id: str | None = None,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
display_name: str | None = None,
|
display_name: str | None = None,
|
||||||
metadata: dict | None = None,
|
metadata: dict | None = None,
|
||||||
) -> dict:
|
) -> dict:
|
||||||
# Auto-resolve user_id from contextvar when AUTO; explicit None
|
# Auto-resolve both user_id and workspace_id from contextvars when
|
||||||
# creates an orphan row (used by migration scripts).
|
# AUTO; explicit None creates an orphan row (used by migration
|
||||||
|
# scripts that intentionally bypass scope).
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.create")
|
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.create")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.create")
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
row = ThreadMetaRow(
|
row = ThreadMetaRow(
|
||||||
thread_id=thread_id,
|
thread_id=thread_id,
|
||||||
assistant_id=assistant_id,
|
assistant_id=assistant_id,
|
||||||
user_id=resolved_user_id,
|
user_id=resolved_user_id,
|
||||||
|
workspace_id=resolved_workspace_id,
|
||||||
display_name=display_name,
|
display_name=display_name,
|
||||||
metadata_json=metadata or {},
|
metadata_json=metadata or {},
|
||||||
created_at=now,
|
created_at=now,
|
||||||
@@ -60,43 +71,52 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
thread_id: str,
|
thread_id: str,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> dict | None:
|
) -> dict | None:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.get")
|
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.get")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.get")
|
||||||
|
stmt = select(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(ThreadMetaRow.workspace_id == resolved_workspace_id)
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(ThreadMetaRow, thread_id)
|
row = (await session.execute(stmt)).scalar_one_or_none()
|
||||||
if row is None:
|
if row is None:
|
||||||
return None
|
return None
|
||||||
# Enforce owner filter unless explicitly bypassed (user_id=None).
|
# Owner filter still applies inside the workspace scope.
|
||||||
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
return None
|
return None
|
||||||
return self._row_to_dict(row)
|
return self._row_to_dict(row)
|
||||||
|
|
||||||
async def check_access(self, thread_id: str, user_id: str, *, require_existing: bool = False) -> bool:
|
async def check_access(
|
||||||
"""Check if ``user_id`` has access to ``thread_id``.
|
self,
|
||||||
|
thread_id: str,
|
||||||
|
user_id: str,
|
||||||
|
workspace_id: str,
|
||||||
|
*,
|
||||||
|
require_existing: bool = False,
|
||||||
|
) -> bool:
|
||||||
|
"""Check if ``user_id`` in ``workspace_id`` has access to ``thread_id``.
|
||||||
|
|
||||||
Two modes — one row, two distinct semantics depending on what
|
Three filters layered, from outside in:
|
||||||
the caller is about to do:
|
|
||||||
|
|
||||||
- ``require_existing=False`` (default, permissive):
|
- Cross-workspace is **always** denied (returns False), even when
|
||||||
Returns True for: row missing (untracked legacy thread),
|
the row exists and ``user_id`` matches. The decorator layer
|
||||||
``row.user_id`` is None (shared / pre-auth data),
|
converts a False into a 404 so cross-tenant access never leaks
|
||||||
or ``row.user_id == user_id``. Use for **read-style**
|
the existence of a thread.
|
||||||
decorators where treating an untracked thread as accessible
|
- Missing row honours ``require_existing``: False by default
|
||||||
preserves backward-compat.
|
(permissive — untracked legacy threads still readable), True
|
||||||
|
for destructive routes (DELETE / PATCH) so a re-targeted ghost
|
||||||
- ``require_existing=True`` (strict):
|
row cannot be claimed.
|
||||||
Returns True **only** when the row exists AND
|
- Within the workspace, ``row.user_id IS NULL`` keeps the legacy
|
||||||
(``row.user_id == user_id`` OR ``row.user_id is None``).
|
"shared / pre-auth" semantics — readable by anyone in the
|
||||||
Use for **destructive / mutating** decorators (DELETE, PATCH,
|
workspace. ``row.user_id == user_id`` is the normal case.
|
||||||
state-update) so a thread that has *already been deleted*
|
|
||||||
cannot be re-targeted by any caller — closing the
|
|
||||||
delete-idempotence cross-user gap where the row vanishing
|
|
||||||
made every other user appear to "own" it.
|
|
||||||
"""
|
"""
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(ThreadMetaRow, thread_id)
|
row = await session.get(ThreadMetaRow, thread_id)
|
||||||
if row is None:
|
if row is None:
|
||||||
return not require_existing
|
return not require_existing
|
||||||
|
if row.workspace_id is not None and row.workspace_id != workspace_id:
|
||||||
|
return False
|
||||||
if row.user_id is None:
|
if row.user_id is None:
|
||||||
return True
|
return True
|
||||||
return row.user_id == user_id
|
return row.user_id == user_id
|
||||||
@@ -109,14 +129,19 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
offset: int = 0,
|
offset: int = 0,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> list[dict]:
|
) -> list[dict]:
|
||||||
"""Search threads with optional metadata and status filters.
|
"""Search threads with optional metadata and status filters.
|
||||||
|
|
||||||
Owner filter is enforced by default: caller must be in a user
|
Both workspace and owner filters are enforced by default. Pass
|
||||||
context. Pass ``user_id=None`` to bypass (migration/CLI).
|
``workspace_id=None`` and / or ``user_id=None`` to bypass for
|
||||||
|
migration / CLI paths.
|
||||||
"""
|
"""
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.search")
|
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.search")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.search")
|
||||||
stmt = select(ThreadMetaRow).order_by(ThreadMetaRow.updated_at.desc())
|
stmt = select(ThreadMetaRow).order_by(ThreadMetaRow.updated_at.desc())
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(ThreadMetaRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(ThreadMetaRow.user_id == resolved_user_id)
|
stmt = stmt.where(ThreadMetaRow.user_id == resolved_user_id)
|
||||||
if status:
|
if status:
|
||||||
@@ -138,12 +163,22 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
result = await session.execute(stmt)
|
result = await session.execute(stmt)
|
||||||
return [self._row_to_dict(r) for r in result.scalars()]
|
return [self._row_to_dict(r) for r in result.scalars()]
|
||||||
|
|
||||||
async def _check_ownership(self, session: AsyncSession, thread_id: str, resolved_user_id: str | None) -> bool:
|
async def _check_ownership(
|
||||||
"""Return True if the row exists and is owned (or filter bypassed)."""
|
self,
|
||||||
if resolved_user_id is None:
|
session: AsyncSession,
|
||||||
return True # explicit bypass
|
thread_id: str,
|
||||||
|
resolved_user_id: str | None,
|
||||||
|
resolved_workspace_id: str | None,
|
||||||
|
) -> bool:
|
||||||
|
"""Return True if the row exists, is in scope, and is owned (or filter bypassed)."""
|
||||||
row = await session.get(ThreadMetaRow, thread_id)
|
row = await session.get(ThreadMetaRow, thread_id)
|
||||||
return row is not None and row.user_id == resolved_user_id
|
if row is None:
|
||||||
|
return False
|
||||||
|
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
|
||||||
|
return False
|
||||||
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
async def update_display_name(
|
async def update_display_name(
|
||||||
self,
|
self,
|
||||||
@@ -151,11 +186,13 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
display_name: str,
|
display_name: str,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Update the display_name (title) for a thread."""
|
"""Update the display_name (title) for a thread."""
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_display_name")
|
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_display_name")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.update_display_name")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
if not await self._check_ownership(session, thread_id, resolved_user_id):
|
if not await self._check_ownership(session, thread_id, resolved_user_id, resolved_workspace_id):
|
||||||
return
|
return
|
||||||
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(display_name=display_name, updated_at=datetime.now(UTC)))
|
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(display_name=display_name, updated_at=datetime.now(UTC)))
|
||||||
await session.commit()
|
await session.commit()
|
||||||
@@ -166,10 +203,12 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
status: str,
|
status: str,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> None:
|
) -> None:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_status")
|
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_status")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.update_status")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
if not await self._check_ownership(session, thread_id, resolved_user_id):
|
if not await self._check_ownership(session, thread_id, resolved_user_id, resolved_workspace_id):
|
||||||
return
|
return
|
||||||
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(status=status, updated_at=datetime.now(UTC)))
|
await session.execute(update(ThreadMetaRow).where(ThreadMetaRow.thread_id == thread_id).values(status=status, updated_at=datetime.now(UTC)))
|
||||||
await session.commit()
|
await session.commit()
|
||||||
@@ -180,18 +219,22 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
metadata: dict,
|
metadata: dict,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Merge ``metadata`` into ``metadata_json``.
|
"""Merge ``metadata`` into ``metadata_json``.
|
||||||
|
|
||||||
Read-modify-write inside a single session/transaction so concurrent
|
Read-modify-write inside a single session/transaction so concurrent
|
||||||
callers see consistent state. No-op if the row does not exist or
|
callers see consistent state. No-op if the row does not exist or
|
||||||
the user_id check fails.
|
the workspace / user check fails.
|
||||||
"""
|
"""
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_metadata")
|
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.update_metadata")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.update_metadata")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(ThreadMetaRow, thread_id)
|
row = await session.get(ThreadMetaRow, thread_id)
|
||||||
if row is None:
|
if row is None:
|
||||||
return
|
return
|
||||||
|
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
|
||||||
|
return
|
||||||
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
return
|
return
|
||||||
merged = dict(row.metadata_json or {})
|
merged = dict(row.metadata_json or {})
|
||||||
@@ -205,12 +248,16 @@ class ThreadMetaRepository(ThreadMetaStore):
|
|||||||
thread_id: str,
|
thread_id: str,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
) -> None:
|
) -> None:
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.delete")
|
resolved_user_id = resolve_user_id(user_id, method_name="ThreadMetaRepository.delete")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="ThreadMetaRepository.delete")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
row = await session.get(ThreadMetaRow, thread_id)
|
row = await session.get(ThreadMetaRow, thread_id)
|
||||||
if row is None:
|
if row is None:
|
||||||
return
|
return
|
||||||
|
if resolved_workspace_id is not None and row.workspace_id != resolved_workspace_id:
|
||||||
|
return
|
||||||
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
if resolved_user_id is not None and row.user_id != resolved_user_id:
|
||||||
return
|
return
|
||||||
await session.delete(row)
|
await session.delete(row)
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
from sqlalchemy import Boolean, DateTime, Index, String, text
|
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, String, text
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from deerflow.persistence.base import Base
|
from deerflow.persistence.base import Base
|
||||||
@@ -22,31 +22,26 @@ from deerflow.persistence.base import Base
|
|||||||
class UserRow(Base):
|
class UserRow(Base):
|
||||||
__tablename__ = "users"
|
__tablename__ = "users"
|
||||||
|
|
||||||
# UUIDs are stored as 36-char strings for cross-backend portability.
|
id: Mapped[str] = mapped_column(String(36), primary_key=True, comment="用户主键,UUID 字符串(36 字符),跨数据库可移植")
|
||||||
id: Mapped[str] = mapped_column(String(36), primary_key=True)
|
email: Mapped[str] = mapped_column(String(320), unique=True, nullable=False, index=True, comment="登录邮箱,全局唯一")
|
||||||
|
password_hash: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="本地账户的密码哈希;OAuth-only 用户为 NULL")
|
||||||
email: Mapped[str] = mapped_column(String(320), unique=True, nullable=False, index=True)
|
system_role: Mapped[str] = mapped_column(String(16), nullable=False, default="user", comment='系统角色:"admin" 或 "user";用字符串以便未来扩展角色而不必 ALTER TABLE')
|
||||||
password_hash: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
||||||
|
|
||||||
# "admin" | "user" — kept as plain string to avoid ALTER TABLE pain
|
|
||||||
# when new roles are introduced.
|
|
||||||
system_role: Mapped[str] = mapped_column(String(16), nullable=False, default="user")
|
|
||||||
|
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True),
|
DateTime(timezone=True),
|
||||||
nullable=False,
|
nullable=False,
|
||||||
default=lambda: datetime.now(UTC),
|
default=lambda: datetime.now(UTC),
|
||||||
|
comment="账户创建时间(UTC)",
|
||||||
|
)
|
||||||
|
oauth_provider: Mapped[str | None] = mapped_column(String(32), nullable=True, comment="OAuth 提供商名(如 google/github);本地账户为 NULL")
|
||||||
|
oauth_id: Mapped[str | None] = mapped_column(String(128), nullable=True, comment="OAuth 提供商内的用户 ID;与 oauth_provider 组合需唯一")
|
||||||
|
needs_setup: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, comment="是否需要完成首次设置(admin 自动创建后改密码/邮箱)")
|
||||||
|
token_version: Mapped[int] = mapped_column(nullable=False, default=0, comment="JWT 令牌版本号;自增即吊销该用户所有旧令牌")
|
||||||
|
default_workspace_id: Mapped[str | None] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("workspaces.id", ondelete="SET NULL"),
|
||||||
|
nullable=True,
|
||||||
|
comment="登录后默认进入的 workspace;NULL 时强制走 picker(user 多 workspace 场景)",
|
||||||
)
|
)
|
||||||
|
|
||||||
# OAuth linkage (optional). A partial unique index enforces one
|
|
||||||
# account per (provider, oauth_id) pair, leaving NULL/NULL rows
|
|
||||||
# unconstrained so plain password accounts can coexist.
|
|
||||||
oauth_provider: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
|
||||||
oauth_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
|
||||||
|
|
||||||
# Auth lifecycle flags
|
|
||||||
needs_setup: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
|
||||||
token_version: Mapped[int] = mapped_column(nullable=False, default=0)
|
|
||||||
|
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
Index(
|
Index(
|
||||||
@@ -56,4 +51,5 @@ class UserRow(Base):
|
|||||||
unique=True,
|
unique=True,
|
||||||
sqlite_where=text("oauth_provider IS NOT NULL AND oauth_id IS NOT NULL"),
|
sqlite_where=text("oauth_provider IS NOT NULL AND oauth_id IS NOT NULL"),
|
||||||
),
|
),
|
||||||
|
{"comment": "用户账户表(本地密码登录 + OAuth 联合登录)"},
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
"""Workspace persistence — ORM model + repository.
|
||||||
|
|
||||||
|
A workspace is the multi-tenant scope unit. Every user has at least
|
||||||
|
one (auto-created on registration; their personal workspace where
|
||||||
|
they are sole owner). Team plans get multi-member workspaces.
|
||||||
|
|
||||||
|
Stage 0 PR3 introduces the schema + repository; PR4 wires it into
|
||||||
|
the registration / login flow; PR5+ ALTER existing business tables
|
||||||
|
to FK back to ``workspaces.id``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
from deerflow.persistence.workspace.sql import WorkspaceRepository, WorkspaceValidationError
|
||||||
|
|
||||||
|
__all__ = ["WorkspaceRepository", "WorkspaceRow", "WorkspaceValidationError"]
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
"""ORM model for workspaces (multi-tenant scope)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from sqlalchemy import DateTime, ForeignKey, String
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class WorkspaceRow(Base):
|
||||||
|
__tablename__ = "workspaces"
|
||||||
|
|
||||||
|
id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
primary_key=True,
|
||||||
|
comment="工作空间主键,UUID 字符串(36 字符),与 users.id 类型对齐",
|
||||||
|
)
|
||||||
|
name: Mapped[str] = mapped_column(
|
||||||
|
String(64),
|
||||||
|
nullable=False,
|
||||||
|
comment="工作空间显示名(用户注册时默认 <email 前缀>'s Workspace)",
|
||||||
|
)
|
||||||
|
slug: Mapped[str] = mapped_column(
|
||||||
|
String(32),
|
||||||
|
nullable=False,
|
||||||
|
unique=True,
|
||||||
|
comment="URL 标识(^[a-z0-9](-?[a-z0-9])*$,3-32 字符);全局唯一,DB 存小写",
|
||||||
|
)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(16),
|
||||||
|
nullable=False,
|
||||||
|
default="active",
|
||||||
|
comment='状态:"active"(正常)/ "suspended"(平台 admin 暂停)/ "deleted"(软删)',
|
||||||
|
)
|
||||||
|
owner_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("users.id", ondelete="RESTRICT"),
|
||||||
|
nullable=False,
|
||||||
|
comment="所有者 user_id;与 workspace_memberships 中 role='owner' 行严格一致(事务保证);删除 owner 时 RESTRICT 阻拦(必须先转让所有权)",
|
||||||
|
)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
default=lambda: datetime.now(UTC),
|
||||||
|
comment="创建时间(UTC)",
|
||||||
|
)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
default=lambda: datetime.now(UTC),
|
||||||
|
onupdate=lambda: datetime.now(UTC),
|
||||||
|
comment="最近更新时间(UTC,写入时自动更新)",
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = ({"comment": ("工作空间表(多租户隔离粒度单位)。每个用户注册时自动建一个 1 人 workspace,owner 即注册者。团队订阅时 workspace 可有多个 member。")},)
|
||||||
@@ -0,0 +1,230 @@
|
|||||||
|
"""SQLAlchemy-backed workspace repository.
|
||||||
|
|
||||||
|
CRUD + slug lookup for ``workspaces``. Membership-aware methods
|
||||||
|
(``get``, ``list_by_user``) JOIN against ``workspace_memberships`` so
|
||||||
|
callers cannot read workspaces they don't belong to.
|
||||||
|
|
||||||
|
Three-state ``user_id`` parameter (same convention as
|
||||||
|
:class:`ThreadMetaRepository`):
|
||||||
|
- ``AUTO`` → read from contextvar; raise if unset
|
||||||
|
- explicit ``str`` → override contextvar (admin / tests)
|
||||||
|
- explicit ``None`` → no filter (migration / CLI only)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
import uuid
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import select, update
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||||
|
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id
|
||||||
|
|
||||||
|
# slug 字符集 / 长度(与 workspace-schema-design §2.1 锁定一致)。
|
||||||
|
_SLUG_PATTERN = re.compile(r"^[a-z0-9](-?[a-z0-9])*$")
|
||||||
|
_SLUG_MIN_LEN = 3
|
||||||
|
_SLUG_MAX_LEN = 32
|
||||||
|
|
||||||
|
# slug 黑名单(应用层校验,不写 DB constraint)。包含 ADR-007 §4 保留 slug
|
||||||
|
# + 路径 + Next.js 保留 + 业务保留词。
|
||||||
|
SLUG_BLACKLIST = frozenset(
|
||||||
|
{
|
||||||
|
"admin",
|
||||||
|
"api",
|
||||||
|
"auth",
|
||||||
|
"login",
|
||||||
|
"signup",
|
||||||
|
"accept-invite",
|
||||||
|
"pricing",
|
||||||
|
"docs",
|
||||||
|
"status",
|
||||||
|
"platform",
|
||||||
|
"system",
|
||||||
|
"health",
|
||||||
|
"static",
|
||||||
|
"public",
|
||||||
|
"favicon.ico",
|
||||||
|
"robots.txt",
|
||||||
|
"sitemap.xml",
|
||||||
|
"_next",
|
||||||
|
".well-known",
|
||||||
|
"settings",
|
||||||
|
"billing",
|
||||||
|
"onboarding",
|
||||||
|
"select-workspace",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
# 允许的 status 集合。
|
||||||
|
_VALID_STATUSES = frozenset({"active", "suspended", "deleted"})
|
||||||
|
|
||||||
|
|
||||||
|
class WorkspaceValidationError(ValueError):
|
||||||
|
"""Raised when workspace input fails application-layer validation
|
||||||
|
(slug format / blacklist / status enum)."""
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_slug(slug: str) -> None:
|
||||||
|
"""Raise :class:`WorkspaceValidationError` if slug is invalid."""
|
||||||
|
if not isinstance(slug, str):
|
||||||
|
raise WorkspaceValidationError(f"slug must be a string, got {type(slug).__name__}")
|
||||||
|
if not (_SLUG_MIN_LEN <= len(slug) <= _SLUG_MAX_LEN):
|
||||||
|
raise WorkspaceValidationError(f"slug length must be between {_SLUG_MIN_LEN} and {_SLUG_MAX_LEN}, got {len(slug)}")
|
||||||
|
if not _SLUG_PATTERN.fullmatch(slug):
|
||||||
|
raise WorkspaceValidationError(f"slug {slug!r} does not match required pattern ^[a-z0-9](-?[a-z0-9])*$")
|
||||||
|
if slug in SLUG_BLACKLIST:
|
||||||
|
raise WorkspaceValidationError(f"slug {slug!r} is reserved")
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_status(status: str) -> None:
|
||||||
|
if status not in _VALID_STATUSES:
|
||||||
|
raise WorkspaceValidationError(f"status {status!r} is not in allowed set {_VALID_STATUSES!r}")
|
||||||
|
|
||||||
|
|
||||||
|
class WorkspaceRepository:
|
||||||
|
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
||||||
|
self._sf = session_factory
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _row_to_dict(row: WorkspaceRow) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": row.id,
|
||||||
|
"name": row.name,
|
||||||
|
"slug": row.slug,
|
||||||
|
"status": row.status,
|
||||||
|
"owner_id": row.owner_id,
|
||||||
|
"created_at": row.created_at.isoformat() if row.created_at else None,
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
async def create(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
name: str,
|
||||||
|
slug: str,
|
||||||
|
owner_id: str,
|
||||||
|
workspace_id: str | None = None,
|
||||||
|
status: str = "active",
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Create a new workspace.
|
||||||
|
|
||||||
|
``workspace_id`` is optional — UUID v4 is generated if omitted.
|
||||||
|
Caller is responsible for creating the matching ``owner`` row
|
||||||
|
in ``workspace_memberships`` (this is typically done in the same
|
||||||
|
transaction by the registration flow; we deliberately don't bundle
|
||||||
|
it here to keep the repository single-responsibility).
|
||||||
|
|
||||||
|
Raises :class:`WorkspaceValidationError` for invalid slug / status.
|
||||||
|
Raises :class:`sqlalchemy.exc.IntegrityError` for slug collision
|
||||||
|
or invalid owner_id FK.
|
||||||
|
"""
|
||||||
|
_validate_slug(slug)
|
||||||
|
_validate_status(status)
|
||||||
|
wid = workspace_id or str(uuid.uuid4())
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
row = WorkspaceRow(
|
||||||
|
id=wid,
|
||||||
|
name=name,
|
||||||
|
slug=slug,
|
||||||
|
status=status,
|
||||||
|
owner_id=owner_id,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
async with self._sf() as session:
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
|
await session.refresh(row)
|
||||||
|
return self._row_to_dict(row)
|
||||||
|
|
||||||
|
async def get(
|
||||||
|
self,
|
||||||
|
workspace_id: str,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
"""Return workspace row IFF caller is a member.
|
||||||
|
|
||||||
|
``user_id=None`` bypasses the membership filter (migration / CLI).
|
||||||
|
Returns ``None`` if the workspace does not exist or the caller is
|
||||||
|
not a member.
|
||||||
|
"""
|
||||||
|
resolved_user_id = resolve_user_id(user_id, method_name="WorkspaceRepository.get")
|
||||||
|
async with self._sf() as session:
|
||||||
|
row = await session.get(WorkspaceRow, workspace_id)
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
if resolved_user_id is None:
|
||||||
|
# Explicit bypass (migration / admin path).
|
||||||
|
return self._row_to_dict(row)
|
||||||
|
# Membership check via separate SELECT (cheap; index covers it).
|
||||||
|
membership = await session.execute(
|
||||||
|
select(WorkspaceMembershipRow).where(
|
||||||
|
WorkspaceMembershipRow.workspace_id == workspace_id,
|
||||||
|
WorkspaceMembershipRow.user_id == resolved_user_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if membership.scalar_one_or_none() is None:
|
||||||
|
return None
|
||||||
|
return self._row_to_dict(row)
|
||||||
|
|
||||||
|
async def get_by_slug(self, slug: str) -> dict[str, Any] | None:
|
||||||
|
"""Public slug lookup — does NOT check membership.
|
||||||
|
|
||||||
|
Used by path-based routing (``/{slug}/...``) where we need to
|
||||||
|
resolve slug → workspace_id BEFORE we know if caller belongs.
|
||||||
|
Membership check happens downstream in the route handler.
|
||||||
|
"""
|
||||||
|
async with self._sf() as session:
|
||||||
|
result = await session.execute(select(WorkspaceRow).where(WorkspaceRow.slug == slug))
|
||||||
|
row = result.scalar_one_or_none()
|
||||||
|
return self._row_to_dict(row) if row else None
|
||||||
|
|
||||||
|
async def list_by_user(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Return all workspaces caller is a member of, ordered by joined_at desc.
|
||||||
|
|
||||||
|
``user_id=None`` lists ALL workspaces (migration / admin path).
|
||||||
|
"""
|
||||||
|
resolved_user_id = resolve_user_id(user_id, method_name="WorkspaceRepository.list_by_user")
|
||||||
|
async with self._sf() as session:
|
||||||
|
stmt = select(WorkspaceRow).order_by(WorkspaceRow.created_at.desc())
|
||||||
|
if resolved_user_id is not None:
|
||||||
|
stmt = stmt.join(
|
||||||
|
WorkspaceMembershipRow,
|
||||||
|
WorkspaceMembershipRow.workspace_id == WorkspaceRow.id,
|
||||||
|
).where(WorkspaceMembershipRow.user_id == resolved_user_id)
|
||||||
|
result = await session.execute(stmt)
|
||||||
|
return [self._row_to_dict(r) for r in result.scalars()]
|
||||||
|
|
||||||
|
async def update_status(self, workspace_id: str, status: str) -> None:
|
||||||
|
"""Platform-admin operation: change workspace status (active/suspended/deleted).
|
||||||
|
|
||||||
|
No membership check — this is for platform-level operations. Audit
|
||||||
|
logging belongs at the route layer.
|
||||||
|
"""
|
||||||
|
_validate_status(status)
|
||||||
|
async with self._sf() as session:
|
||||||
|
await session.execute(update(WorkspaceRow).where(WorkspaceRow.id == workspace_id).values(status=status, updated_at=datetime.now(UTC)))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async def delete(self, workspace_id: str) -> None:
|
||||||
|
"""Hard-delete a workspace. CASCADE drops all memberships.
|
||||||
|
|
||||||
|
Intentionally no membership check — caller (platform admin route)
|
||||||
|
must enforce authorization. Stage 0 doesn't expose this to end
|
||||||
|
users; Stage 2+ adds it behind owner_only permission.
|
||||||
|
"""
|
||||||
|
async with self._sf() as session:
|
||||||
|
row = await session.get(WorkspaceRow, workspace_id)
|
||||||
|
if row is not None:
|
||||||
|
await session.delete(row)
|
||||||
|
await session.commit()
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
"""Workspace membership persistence — ORM model + repository.
|
||||||
|
|
||||||
|
Tracks who is a member of which workspace and what their role is
|
||||||
|
within that workspace. Composite primary key ``(workspace_id, user_id)``;
|
||||||
|
``role`` follows the three-state ``owner`` / ``admin`` / ``member`` model
|
||||||
|
(Stage 0 only writes ``'owner'``; Stage 2 RBAC rollout opens admin/member).
|
||||||
|
|
||||||
|
A workspace MUST have exactly one ``owner`` — enforced by a partial
|
||||||
|
unique index. Owner transfer is a two-row transactional swap.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
from deerflow.persistence.workspace_membership.sql import (
|
||||||
|
MembershipValidationError,
|
||||||
|
WorkspaceMembershipRepository,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"MembershipValidationError",
|
||||||
|
"WorkspaceMembershipRepository",
|
||||||
|
"WorkspaceMembershipRow",
|
||||||
|
]
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
"""ORM model for workspace memberships (which user is in which workspace, with what role)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from sqlalchemy import DateTime, ForeignKey, Index, String, text
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class WorkspaceMembershipRow(Base):
|
||||||
|
__tablename__ = "workspace_memberships"
|
||||||
|
|
||||||
|
workspace_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("workspaces.id", ondelete="CASCADE"),
|
||||||
|
primary_key=True,
|
||||||
|
comment="所属 workspace;workspace 删除时级联清掉成员记录",
|
||||||
|
)
|
||||||
|
user_id: Mapped[str] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("users.id", ondelete="CASCADE"),
|
||||||
|
primary_key=True,
|
||||||
|
comment="成员 user_id;用户删除时级联清掉成员记录",
|
||||||
|
)
|
||||||
|
role: Mapped[str] = mapped_column(
|
||||||
|
String(16),
|
||||||
|
nullable=False,
|
||||||
|
comment='角色字符串:Stage 0 仅写 "owner";Stage 2 RBAC 打开 "admin"/"member"。用 String(16) 而非 enum 以便未来加 "viewer"/"auditor" 不动 schema',
|
||||||
|
)
|
||||||
|
invited_by: Mapped[str | None] = mapped_column(
|
||||||
|
String(36),
|
||||||
|
ForeignKey("users.id", ondelete="SET NULL"),
|
||||||
|
nullable=True,
|
||||||
|
comment="邀请人 user_id(Stage 2 invitation 流程才写);邀请人被删时此字段清空(不影响成员记录本身)",
|
||||||
|
)
|
||||||
|
joined_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True),
|
||||||
|
nullable=False,
|
||||||
|
default=lambda: datetime.now(UTC),
|
||||||
|
comment="加入 workspace 的时间(UTC)",
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# 倒查索引:列出 user 所属的所有 workspace(/auth/me 用)。
|
||||||
|
# 复合主键 (workspace_id, user_id) 的前导列是 workspace_id,
|
||||||
|
# 所以"按 user_id 查"需要单独的索引。
|
||||||
|
Index("idx_workspace_memberships_user", "user_id", "workspace_id"),
|
||||||
|
# 一个 workspace 严格 1 个 owner —— partial unique on role='owner'。
|
||||||
|
# SQLite + Postgres 都支持 WHERE 子句的 partial unique;双驱动
|
||||||
|
# 维护两套等价 where 表达式。
|
||||||
|
Index(
|
||||||
|
"idx_one_owner_per_workspace",
|
||||||
|
"workspace_id",
|
||||||
|
unique=True,
|
||||||
|
sqlite_where=text("role = 'owner'"),
|
||||||
|
postgresql_where=text("role = 'owner'"),
|
||||||
|
),
|
||||||
|
{"comment": ("工作空间成员表(multi-tenant RBAC)。复合 PK (workspace_id, user_id);每个 workspace 必有恰好 1 个 owner(partial unique 约束保证)。Stage 0 仅写 owner;Stage 2 RBAC 打开 admin/member。")},
|
||||||
|
)
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
"""SQLAlchemy-backed workspace membership repository.
|
||||||
|
|
||||||
|
Manages the ``workspace_memberships`` join table. Stage 0 only writes
|
||||||
|
``role='owner'`` (single-user workspaces); the repository accepts the
|
||||||
|
full Stage 2 RBAC role enum so the schema is forward-compatible.
|
||||||
|
|
||||||
|
Owner transfer is intentionally NOT modelled here as a single method —
|
||||||
|
it requires a two-row transactional swap with careful retry semantics,
|
||||||
|
and belongs in the auth router (PR4+) where it can be wrapped in a
|
||||||
|
permission check.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import delete, select, update
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||||
|
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
|
||||||
|
# 允许的 role 字面值。Stage 0 实际仅写 owner;admin/member 留给 Stage 2 RBAC。
|
||||||
|
_VALID_ROLES = frozenset({"owner", "admin", "member"})
|
||||||
|
|
||||||
|
|
||||||
|
class MembershipValidationError(ValueError):
|
||||||
|
"""Raised when role is not in the allowed enum."""
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_role(role: str) -> None:
|
||||||
|
if role not in _VALID_ROLES:
|
||||||
|
raise MembershipValidationError(f"role {role!r} is not in allowed set {_VALID_ROLES!r}")
|
||||||
|
|
||||||
|
|
||||||
|
class WorkspaceMembershipRepository:
|
||||||
|
def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
|
||||||
|
self._sf = session_factory
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _row_to_dict(row: WorkspaceMembershipRow) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"workspace_id": row.workspace_id,
|
||||||
|
"user_id": row.user_id,
|
||||||
|
"role": row.role,
|
||||||
|
"invited_by": row.invited_by,
|
||||||
|
"joined_at": row.joined_at.isoformat() if row.joined_at else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
async def add(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
workspace_id: str,
|
||||||
|
user_id: str,
|
||||||
|
role: str,
|
||||||
|
invited_by: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Insert a new membership row.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
- :class:`MembershipValidationError` for unknown role values
|
||||||
|
- :class:`sqlalchemy.exc.IntegrityError` for:
|
||||||
|
- duplicate (workspace_id, user_id) — composite PK collision
|
||||||
|
- second ``owner`` in the same workspace — partial unique index
|
||||||
|
- invalid workspace_id / user_id / invited_by FK
|
||||||
|
"""
|
||||||
|
_validate_role(role)
|
||||||
|
row = WorkspaceMembershipRow(
|
||||||
|
workspace_id=workspace_id,
|
||||||
|
user_id=user_id,
|
||||||
|
role=role,
|
||||||
|
invited_by=invited_by,
|
||||||
|
joined_at=datetime.now(UTC),
|
||||||
|
)
|
||||||
|
async with self._sf() as session:
|
||||||
|
session.add(row)
|
||||||
|
await session.commit()
|
||||||
|
await session.refresh(row)
|
||||||
|
return self._row_to_dict(row)
|
||||||
|
|
||||||
|
async def remove(self, *, workspace_id: str, user_id: str) -> bool:
|
||||||
|
"""Delete a membership row; returns True if a row was deleted.
|
||||||
|
|
||||||
|
Owner removal is allowed at the repository layer — auth router
|
||||||
|
(PR4) layers the "cannot remove last owner" rule on top.
|
||||||
|
"""
|
||||||
|
async with self._sf() as session:
|
||||||
|
result = await session.execute(
|
||||||
|
delete(WorkspaceMembershipRow).where(
|
||||||
|
WorkspaceMembershipRow.workspace_id == workspace_id,
|
||||||
|
WorkspaceMembershipRow.user_id == user_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
return (result.rowcount or 0) > 0
|
||||||
|
|
||||||
|
async def list_by_user(self, *, user_id: str) -> list[dict[str, Any]]:
|
||||||
|
"""Return all memberships for ``user_id``, ordered by joined_at desc.
|
||||||
|
|
||||||
|
No contextvar resolution here — caller is responsible for passing
|
||||||
|
the correct user_id. Used by ``/auth/me`` to list workspaces a
|
||||||
|
user belongs to.
|
||||||
|
"""
|
||||||
|
async with self._sf() as session:
|
||||||
|
result = await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.user_id == user_id).order_by(WorkspaceMembershipRow.joined_at.desc()))
|
||||||
|
return [self._row_to_dict(r) for r in result.scalars()]
|
||||||
|
|
||||||
|
async def list_by_workspace(self, *, workspace_id: str) -> list[dict[str, Any]]:
|
||||||
|
"""Return all members of a workspace, ordered by joined_at asc."""
|
||||||
|
async with self._sf() as session:
|
||||||
|
result = await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.workspace_id == workspace_id).order_by(WorkspaceMembershipRow.joined_at.asc()))
|
||||||
|
return [self._row_to_dict(r) for r in result.scalars()]
|
||||||
|
|
||||||
|
async def get_role(self, *, workspace_id: str, user_id: str) -> str | None:
|
||||||
|
"""Return the role string, or None if user is not a member."""
|
||||||
|
async with self._sf() as session:
|
||||||
|
result = await session.execute(
|
||||||
|
select(WorkspaceMembershipRow.role).where(
|
||||||
|
WorkspaceMembershipRow.workspace_id == workspace_id,
|
||||||
|
WorkspaceMembershipRow.user_id == user_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return result.scalar_one_or_none()
|
||||||
|
|
||||||
|
async def change_role(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
workspace_id: str,
|
||||||
|
user_id: str,
|
||||||
|
new_role: str,
|
||||||
|
) -> bool:
|
||||||
|
"""Update a member's role; returns True iff a row was updated.
|
||||||
|
|
||||||
|
Validates ``new_role`` against the allowed enum. Owner-transfer
|
||||||
|
flow needs to swap two rows atomically — do that with a manual
|
||||||
|
transaction in the caller; this method is for non-owner changes.
|
||||||
|
"""
|
||||||
|
_validate_role(new_role)
|
||||||
|
async with self._sf() as session:
|
||||||
|
result = await session.execute(
|
||||||
|
update(WorkspaceMembershipRow)
|
||||||
|
.where(
|
||||||
|
WorkspaceMembershipRow.workspace_id == workspace_id,
|
||||||
|
WorkspaceMembershipRow.user_id == user_id,
|
||||||
|
)
|
||||||
|
.values(role=new_role)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
return (result.rowcount or 0) > 0
|
||||||
@@ -20,6 +20,7 @@ from __future__ import annotations
|
|||||||
import asyncio
|
import asyncio
|
||||||
import contextlib
|
import contextlib
|
||||||
import logging
|
import logging
|
||||||
|
import re
|
||||||
from collections.abc import AsyncIterator
|
from collections.abc import AsyncIterator
|
||||||
|
|
||||||
from langgraph.types import Checkpointer
|
from langgraph.types import Checkpointer
|
||||||
@@ -114,7 +115,14 @@ async def _async_checkpointer_from_database(db_config) -> AsyncIterator[Checkpoi
|
|||||||
if not db_config.postgres_url:
|
if not db_config.postgres_url:
|
||||||
raise ValueError("database.postgres_url is required for the postgres backend")
|
raise ValueError("database.postgres_url is required for the postgres backend")
|
||||||
|
|
||||||
async with AsyncPostgresSaver.from_conn_string(db_config.postgres_url) as saver:
|
# LangGraph's AsyncPostgresSaver wraps psycopg directly and expects a
|
||||||
|
# libpq-style conninfo (`postgresql://...`). DeerFlow's own SQLAlchemy
|
||||||
|
# engine uses the same `postgres_url` but needs the `+asyncpg` dialect
|
||||||
|
# prefix. Strip the dialect prefix here so the same env var/config
|
||||||
|
# value satisfies both paths.
|
||||||
|
lg_conn_str = re.sub(r"^postgresql\+\w+://", "postgresql://", db_config.postgres_url)
|
||||||
|
|
||||||
|
async with AsyncPostgresSaver.from_conn_string(lg_conn_str) as saver:
|
||||||
await saver.setup()
|
await saver.setup()
|
||||||
yield saver
|
yield saver
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -17,6 +17,14 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|||||||
from deerflow.persistence.models.run_event import RunEventRow
|
from deerflow.persistence.models.run_event import RunEventRow
|
||||||
from deerflow.runtime.events.store.base import RunEventStore
|
from deerflow.runtime.events.store.base import RunEventStore
|
||||||
from deerflow.runtime.user_context import AUTO, _AutoSentinel, get_current_user, resolve_user_id
|
from deerflow.runtime.user_context import AUTO, _AutoSentinel, get_current_user, resolve_user_id
|
||||||
|
from deerflow.runtime.workspace_context import AUTO as WORKSPACE_AUTO
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
_AutoSentinel as _WorkspaceAutoSentinel,
|
||||||
|
)
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
get_current_workspace,
|
||||||
|
resolve_workspace_id,
|
||||||
|
)
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -86,6 +94,19 @@ class DbRunEventStore(RunEventStore):
|
|||||||
user = get_current_user()
|
user = get_current_user()
|
||||||
return str(user.id) if user is not None else None
|
return str(user.id) if user is not None else None
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _workspace_id_from_context() -> str | None:
|
||||||
|
"""Soft read of workspace_id from contextvar for write paths.
|
||||||
|
|
||||||
|
Mirrors :meth:`_user_id_from_context`. Returns ``None`` (no stamp)
|
||||||
|
when no workspace is in context — typical for background worker
|
||||||
|
writes that fire outside an HTTP request. The DB column is
|
||||||
|
nullable through PR5 and becomes NOT NULL only after the alembic
|
||||||
|
0003 migration runs (verified by the backfill path).
|
||||||
|
"""
|
||||||
|
workspace = get_current_workspace()
|
||||||
|
return str(workspace.id) if workspace is not None else None
|
||||||
|
|
||||||
async def put(self, *, thread_id, run_id, event_type, category, content="", metadata=None, created_at=None): # noqa: D401
|
async def put(self, *, thread_id, run_id, event_type, category, content="", metadata=None, created_at=None): # noqa: D401
|
||||||
"""Write a single event — low-frequency path only.
|
"""Write a single event — low-frequency path only.
|
||||||
|
|
||||||
@@ -98,6 +119,7 @@ class DbRunEventStore(RunEventStore):
|
|||||||
content, metadata = self._truncate_trace(category, content, metadata)
|
content, metadata = self._truncate_trace(category, content, metadata)
|
||||||
db_content, metadata = self._content_to_db(content, metadata)
|
db_content, metadata = self._content_to_db(content, metadata)
|
||||||
user_id = self._user_id_from_context()
|
user_id = self._user_id_from_context()
|
||||||
|
workspace_id = self._workspace_id_from_context()
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
async with session.begin():
|
async with session.begin():
|
||||||
# Use FOR UPDATE to serialize seq assignment within a thread.
|
# Use FOR UPDATE to serialize seq assignment within a thread.
|
||||||
@@ -109,6 +131,7 @@ class DbRunEventStore(RunEventStore):
|
|||||||
thread_id=thread_id,
|
thread_id=thread_id,
|
||||||
run_id=run_id,
|
run_id=run_id,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
|
workspace_id=workspace_id,
|
||||||
event_type=event_type,
|
event_type=event_type,
|
||||||
category=category,
|
category=category,
|
||||||
content=db_content,
|
content=db_content,
|
||||||
@@ -123,6 +146,7 @@ class DbRunEventStore(RunEventStore):
|
|||||||
if not events:
|
if not events:
|
||||||
return []
|
return []
|
||||||
user_id = self._user_id_from_context()
|
user_id = self._user_id_from_context()
|
||||||
|
workspace_id = self._workspace_id_from_context()
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
async with session.begin():
|
async with session.begin():
|
||||||
# Get max seq for the thread (assume all events in batch belong to same thread).
|
# Get max seq for the thread (assume all events in batch belong to same thread).
|
||||||
@@ -143,6 +167,7 @@ class DbRunEventStore(RunEventStore):
|
|||||||
thread_id=e["thread_id"],
|
thread_id=e["thread_id"],
|
||||||
run_id=e["run_id"],
|
run_id=e["run_id"],
|
||||||
user_id=e.get("user_id", user_id),
|
user_id=e.get("user_id", user_id),
|
||||||
|
workspace_id=e.get("workspace_id", workspace_id),
|
||||||
event_type=e["event_type"],
|
event_type=e["event_type"],
|
||||||
category=category,
|
category=category,
|
||||||
content=db_content,
|
content=db_content,
|
||||||
@@ -162,9 +187,13 @@ class DbRunEventStore(RunEventStore):
|
|||||||
before_seq=None,
|
before_seq=None,
|
||||||
after_seq=None,
|
after_seq=None,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages")
|
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.list_messages")
|
||||||
stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message")
|
stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message")
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
||||||
if before_seq is not None:
|
if before_seq is not None:
|
||||||
@@ -194,9 +223,13 @@ class DbRunEventStore(RunEventStore):
|
|||||||
event_types=None,
|
event_types=None,
|
||||||
limit=500,
|
limit=500,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_events")
|
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_events")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.list_events")
|
||||||
stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id)
|
stmt = select(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
||||||
if event_types:
|
if event_types:
|
||||||
@@ -215,13 +248,17 @@ class DbRunEventStore(RunEventStore):
|
|||||||
before_seq=None,
|
before_seq=None,
|
||||||
after_seq=None,
|
after_seq=None,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages_by_run")
|
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.list_messages_by_run")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.list_messages_by_run")
|
||||||
stmt = select(RunEventRow).where(
|
stmt = select(RunEventRow).where(
|
||||||
RunEventRow.thread_id == thread_id,
|
RunEventRow.thread_id == thread_id,
|
||||||
RunEventRow.run_id == run_id,
|
RunEventRow.run_id == run_id,
|
||||||
RunEventRow.category == "message",
|
RunEventRow.category == "message",
|
||||||
)
|
)
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
||||||
if before_seq is not None:
|
if before_seq is not None:
|
||||||
@@ -246,9 +283,13 @@ class DbRunEventStore(RunEventStore):
|
|||||||
thread_id,
|
thread_id,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.count_messages")
|
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.count_messages")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.count_messages")
|
||||||
stmt = select(func.count()).select_from(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message")
|
stmt = select(func.count()).select_from(RunEventRow).where(RunEventRow.thread_id == thread_id, RunEventRow.category == "message")
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
stmt = stmt.where(RunEventRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
stmt = stmt.where(RunEventRow.user_id == resolved_user_id)
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
@@ -259,10 +300,14 @@ class DbRunEventStore(RunEventStore):
|
|||||||
thread_id,
|
thread_id,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_thread")
|
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_thread")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.delete_by_thread")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
count_conditions = [RunEventRow.thread_id == thread_id]
|
count_conditions = [RunEventRow.thread_id == thread_id]
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
count_conditions.append(RunEventRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
count_conditions.append(RunEventRow.user_id == resolved_user_id)
|
count_conditions.append(RunEventRow.user_id == resolved_user_id)
|
||||||
count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions)
|
count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions)
|
||||||
@@ -278,10 +323,14 @@ class DbRunEventStore(RunEventStore):
|
|||||||
run_id,
|
run_id,
|
||||||
*,
|
*,
|
||||||
user_id: str | None | _AutoSentinel = AUTO,
|
user_id: str | None | _AutoSentinel = AUTO,
|
||||||
|
workspace_id: str | None | _WorkspaceAutoSentinel = WORKSPACE_AUTO,
|
||||||
):
|
):
|
||||||
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_run")
|
resolved_user_id = resolve_user_id(user_id, method_name="DbRunEventStore.delete_by_run")
|
||||||
|
resolved_workspace_id = resolve_workspace_id(workspace_id, method_name="DbRunEventStore.delete_by_run")
|
||||||
async with self._sf() as session:
|
async with self._sf() as session:
|
||||||
count_conditions = [RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id]
|
count_conditions = [RunEventRow.thread_id == thread_id, RunEventRow.run_id == run_id]
|
||||||
|
if resolved_workspace_id is not None:
|
||||||
|
count_conditions.append(RunEventRow.workspace_id == resolved_workspace_id)
|
||||||
if resolved_user_id is not None:
|
if resolved_user_id is not None:
|
||||||
count_conditions.append(RunEventRow.user_id == resolved_user_id)
|
count_conditions.append(RunEventRow.user_id == resolved_user_id)
|
||||||
count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions)
|
count_stmt = select(func.count()).select_from(RunEventRow).where(*count_conditions)
|
||||||
|
|||||||
@@ -0,0 +1,182 @@
|
|||||||
|
"""Request-scoped workspace context for multi-tenant authorization.
|
||||||
|
|
||||||
|
Sibling of :mod:`deerflow.runtime.user_context`. Holds a
|
||||||
|
:class:`~contextvars.ContextVar` that the gateway's auth middleware sets
|
||||||
|
after JWT verification (PR4 will wire this up). Repository methods read
|
||||||
|
the contextvar via a sentinel default parameter, letting routers stay
|
||||||
|
free of ``workspace_id`` boilerplate.
|
||||||
|
|
||||||
|
Three-state semantics for the repository ``workspace_id`` parameter:
|
||||||
|
|
||||||
|
- ``AUTO`` (sentinel, default): read from contextvar; raise
|
||||||
|
:class:`RuntimeError` if unset.
|
||||||
|
- Explicit ``str``: use the provided value, overriding contextvar.
|
||||||
|
- Explicit ``None``: no WHERE clause — used only by migration scripts
|
||||||
|
and admin CLIs that intentionally bypass workspace isolation.
|
||||||
|
|
||||||
|
Concept boundary
|
||||||
|
----------------
|
||||||
|
A workspace is the multi-tenant scope: a single-user free account is
|
||||||
|
its own 1-person workspace; a team subscription is a multi-member
|
||||||
|
workspace. The user_id contextvar narrows further to "which member of
|
||||||
|
the workspace", letting some operations be member-scoped while others
|
||||||
|
(skill install, billing, etc.) are workspace-scoped.
|
||||||
|
|
||||||
|
Dependency direction
|
||||||
|
--------------------
|
||||||
|
``persistence`` (lower layer) reads from this module; ``gateway.auth``
|
||||||
|
(higher layer) writes to it. ``CurrentWorkspace`` is defined here as a
|
||||||
|
:class:`typing.Protocol` so that ``persistence`` never needs to import
|
||||||
|
the concrete ``Workspace`` row class from ``deerflow.persistence.workspace``.
|
||||||
|
Any object with ``.id: str`` and ``.role: str`` attributes structurally
|
||||||
|
satisfies the protocol.
|
||||||
|
|
||||||
|
Asyncio semantics
|
||||||
|
-----------------
|
||||||
|
Identical to ``user_context``: ``ContextVar`` is task-local under asyncio.
|
||||||
|
``asyncio.create_task`` inherits the parent task's workspace context;
|
||||||
|
threading.Timer does **not** (callers spawning timers must capture
|
||||||
|
``get_effective_workspace_id()`` at enqueue time, the same way
|
||||||
|
:mod:`deerflow.agents.memory.queue` captures ``user_id``).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from contextvars import ContextVar, Token
|
||||||
|
from typing import Final, Protocol, runtime_checkable
|
||||||
|
|
||||||
|
|
||||||
|
@runtime_checkable
|
||||||
|
class CurrentWorkspace(Protocol):
|
||||||
|
"""Structural type for the current active workspace.
|
||||||
|
|
||||||
|
Any object with ``.id: str`` and ``.role: str`` attributes satisfies
|
||||||
|
this protocol. Concrete implementations live in
|
||||||
|
``app.gateway.auth.models`` (PR4 will add them).
|
||||||
|
|
||||||
|
``role`` is the *caller's* role within this workspace
|
||||||
|
(``'owner'`` / ``'admin'`` / ``'member'``), not the workspace's
|
||||||
|
own metadata. Stage 0 sees ``'owner'`` only — Stage 2 RBAC rollout
|
||||||
|
opens up the other values.
|
||||||
|
"""
|
||||||
|
|
||||||
|
id: str
|
||||||
|
role: str
|
||||||
|
|
||||||
|
|
||||||
|
_current_workspace: Final[ContextVar[CurrentWorkspace | None]] = ContextVar("deerflow_current_workspace", default=None)
|
||||||
|
|
||||||
|
|
||||||
|
def set_current_workspace(workspace: CurrentWorkspace) -> Token[CurrentWorkspace | None]:
|
||||||
|
"""Set the current workspace for this async task.
|
||||||
|
|
||||||
|
Returns a reset token that should be passed to
|
||||||
|
:func:`reset_current_workspace` in a ``finally`` block to restore
|
||||||
|
the previous context.
|
||||||
|
"""
|
||||||
|
return _current_workspace.set(workspace)
|
||||||
|
|
||||||
|
|
||||||
|
def reset_current_workspace(token: Token[CurrentWorkspace | None]) -> None:
|
||||||
|
"""Restore the context to the state captured by ``token``."""
|
||||||
|
_current_workspace.reset(token)
|
||||||
|
|
||||||
|
|
||||||
|
def get_current_workspace() -> CurrentWorkspace | None:
|
||||||
|
"""Return the current workspace, or ``None`` if unset.
|
||||||
|
|
||||||
|
Safe to call in any context. Used by code paths that can proceed
|
||||||
|
without a workspace (migration scripts, public endpoints).
|
||||||
|
"""
|
||||||
|
return _current_workspace.get()
|
||||||
|
|
||||||
|
|
||||||
|
def require_current_workspace() -> CurrentWorkspace:
|
||||||
|
"""Return the current workspace, or raise :class:`RuntimeError`.
|
||||||
|
|
||||||
|
Used by repository code that must not be called outside a
|
||||||
|
request-authenticated context. The error message is phrased so
|
||||||
|
that a caller debugging a stack trace can locate the offending
|
||||||
|
code path.
|
||||||
|
"""
|
||||||
|
workspace = _current_workspace.get()
|
||||||
|
if workspace is None:
|
||||||
|
raise RuntimeError("repository accessed without workspace context")
|
||||||
|
return workspace
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Effective workspace_id helpers (filesystem isolation)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
DEFAULT_WORKSPACE_ID: Final[str] = "default"
|
||||||
|
|
||||||
|
|
||||||
|
def get_effective_workspace_id() -> str:
|
||||||
|
"""Return the current workspace id as a string, or DEFAULT_WORKSPACE_ID if unset.
|
||||||
|
|
||||||
|
Unlike :func:`require_current_workspace` this never raises — it is
|
||||||
|
designed for filesystem-path resolution where a valid workspace
|
||||||
|
bucket is always needed (PR6 will switch
|
||||||
|
``Paths.thread_dir(workspace_id=...)`` to read from here).
|
||||||
|
"""
|
||||||
|
workspace = _current_workspace.get()
|
||||||
|
if workspace is None:
|
||||||
|
return DEFAULT_WORKSPACE_ID
|
||||||
|
return str(workspace.id)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Sentinel-based workspace_id resolution
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# Repository methods accept a ``workspace_id`` keyword-only argument that
|
||||||
|
# defaults to ``AUTO``. The three possible values drive distinct
|
||||||
|
# behaviours; see the docstring on :func:`resolve_workspace_id`.
|
||||||
|
|
||||||
|
|
||||||
|
class _AutoSentinel:
|
||||||
|
"""Singleton marker meaning 'resolve workspace_id from contextvar'."""
|
||||||
|
|
||||||
|
_instance: _AutoSentinel | None = None
|
||||||
|
|
||||||
|
def __new__(cls) -> _AutoSentinel:
|
||||||
|
if cls._instance is None:
|
||||||
|
cls._instance = super().__new__(cls)
|
||||||
|
return cls._instance
|
||||||
|
|
||||||
|
def __repr__(self) -> str:
|
||||||
|
return "<AUTO>"
|
||||||
|
|
||||||
|
|
||||||
|
AUTO: Final[_AutoSentinel] = _AutoSentinel()
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_workspace_id(
|
||||||
|
value: str | None | _AutoSentinel,
|
||||||
|
*,
|
||||||
|
method_name: str = "repository method",
|
||||||
|
) -> str | None:
|
||||||
|
"""Resolve the workspace_id parameter passed to a repository method.
|
||||||
|
|
||||||
|
Three-state semantics:
|
||||||
|
|
||||||
|
- :data:`AUTO` (default): read from contextvar; raise
|
||||||
|
:class:`RuntimeError` if no workspace is in context. This is the
|
||||||
|
common case for request-scoped calls.
|
||||||
|
- Explicit ``str``: use the provided id verbatim, overriding any
|
||||||
|
contextvar value. Useful for tests and admin-override flows.
|
||||||
|
- Explicit ``None``: no filter — the repository should skip the
|
||||||
|
workspace_id WHERE clause entirely. Reserved for migration scripts
|
||||||
|
and CLI tools that intentionally bypass workspace isolation.
|
||||||
|
"""
|
||||||
|
if isinstance(value, _AutoSentinel):
|
||||||
|
workspace = _current_workspace.get()
|
||||||
|
if workspace is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"{method_name} called with workspace_id=AUTO but no workspace context is set; pass an explicit workspace_id, set the contextvar via auth middleware, or opt out with workspace_id=None for migration/CLI paths."
|
||||||
|
)
|
||||||
|
# Coerce to ``str`` at the boundary; persistence stores
|
||||||
|
# ``workspace_id`` as ``String(36)`` (UUID v4 text).
|
||||||
|
return str(workspace.id)
|
||||||
|
return value
|
||||||
@@ -46,6 +46,10 @@ postgres = [
|
|||||||
"psycopg[binary]>=3.3.3",
|
"psycopg[binary]>=3.3.3",
|
||||||
"psycopg-pool>=3.3.0",
|
"psycopg-pool>=3.3.0",
|
||||||
]
|
]
|
||||||
|
postgres-test = [
|
||||||
|
"deerflow-harness[postgres]",
|
||||||
|
"testcontainers[postgres]>=4.0",
|
||||||
|
]
|
||||||
pymupdf = ["pymupdf4llm>=0.0.17"]
|
pymupdf = ["pymupdf4llm>=0.0.17"]
|
||||||
|
|
||||||
[build-system]
|
[build-system]
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ dependencies = [
|
|||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
postgres = ["deerflow-harness[postgres]"]
|
postgres = ["deerflow-harness[postgres]"]
|
||||||
|
postgres-test = ["deerflow-harness[postgres-test]"]
|
||||||
|
|
||||||
[dependency-groups]
|
[dependency-groups]
|
||||||
dev = [
|
dev = [
|
||||||
@@ -36,7 +37,9 @@ dev = [
|
|||||||
|
|
||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
markers = [
|
markers = [
|
||||||
"no_auto_user: disable the conftest autouse contextvar fixture for this test",
|
"no_auto_user: disable the conftest autouse user contextvar fixture for this test",
|
||||||
|
"no_auto_workspace: disable the conftest autouse workspace contextvar fixture for this test",
|
||||||
|
"postgres: requires a Postgres testcontainer (Docker daemon); skipped if unavailable",
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.uv]
|
[tool.uv]
|
||||||
|
|||||||
@@ -0,0 +1,306 @@
|
|||||||
|
"""Backfill ``workspace_id`` on PR5 business tables.
|
||||||
|
|
||||||
|
Three-step backfill (each idempotent — re-running picks up where a crash
|
||||||
|
left off because every step's WHERE clause filters already-processed rows):
|
||||||
|
|
||||||
|
1. For each user without ``default_workspace_id``: create a personal
|
||||||
|
workspace + ``owner`` membership + write back the user's
|
||||||
|
``default_workspace_id``.
|
||||||
|
2. ``UPDATE`` each of ``threads_meta`` / ``runs`` / ``feedback`` /
|
||||||
|
``run_events`` setting ``workspace_id`` from the row's owner's
|
||||||
|
``users.default_workspace_id``. Only touches rows where
|
||||||
|
``workspace_id IS NULL`` and ``user_id IS NOT NULL``.
|
||||||
|
3. Any rows still with ``workspace_id IS NULL`` (truly orphan — they had
|
||||||
|
``user_id = NULL`` to begin with) are assigned the *legacy* workspace
|
||||||
|
UUID ``00000000-0000-0000-0000-000000000000``. The script creates
|
||||||
|
that workspace on demand, owned by the platform admin.
|
||||||
|
|
||||||
|
Usage::
|
||||||
|
|
||||||
|
PYTHONPATH=. python scripts/backfill_workspace_id.py [--dry-run]
|
||||||
|
|
||||||
|
T5.4 only ships the skeleton — the three step bodies are filled in by
|
||||||
|
T5.5 / T5.6 / T5.7 along with their per-step tests.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import func, select, update
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||||
|
|
||||||
|
from app.gateway.auth.workspace_slug import auto_slug_from_email, next_available_slug
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
from deerflow.persistence.feedback.model import FeedbackRow
|
||||||
|
from deerflow.persistence.models.run_event import RunEventRow
|
||||||
|
from deerflow.persistence.run.model import RunRow
|
||||||
|
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace import WorkspaceRepository
|
||||||
|
from deerflow.persistence.workspace.sql import SLUG_BLACKLIST
|
||||||
|
from deerflow.persistence.workspace_membership import WorkspaceMembershipRepository
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# The legacy workspace anchor. Stage 0 LOCK'd UUID — chosen as the standard
|
||||||
|
# nil UUID so SQL log scans can spot it instantly.
|
||||||
|
LEGACY_WORKSPACE_ID = "00000000-0000-0000-0000-000000000000"
|
||||||
|
LEGACY_WORKSPACE_SLUG = "legacy"
|
||||||
|
LEGACY_WORKSPACE_NAME = "Legacy Workspace"
|
||||||
|
|
||||||
|
# The four business tables that gained ``workspace_id`` in alembic 0002.
|
||||||
|
_BUSINESS_TABLES: tuple[str, ...] = ("threads_meta", "runs", "feedback", "run_events")
|
||||||
|
|
||||||
|
# Map table-name to ORM class so we can build a portable correlated UPDATE
|
||||||
|
# using SQLAlchemy expression language (SQLite < 3.33 lacks UPDATE-FROM
|
||||||
|
# but supports correlated subqueries on every version we ship).
|
||||||
|
_TABLE_MODELS: dict[str, type[Base]] = {
|
||||||
|
"threads_meta": ThreadMetaRow,
|
||||||
|
"runs": RunRow,
|
||||||
|
"feedback": FeedbackRow,
|
||||||
|
"run_events": RunEventRow,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def _step1_create_workspaces_for_users(
|
||||||
|
session_factory: async_sessionmaker[AsyncSession],
|
||||||
|
*,
|
||||||
|
dry_run: bool,
|
||||||
|
) -> int:
|
||||||
|
"""Create one workspace + owner membership for each user missing default_workspace_id.
|
||||||
|
|
||||||
|
Returns the count of users a workspace was created for. Idempotent —
|
||||||
|
users that already have ``default_workspace_id`` are skipped so a
|
||||||
|
crashed run can resume safely.
|
||||||
|
"""
|
||||||
|
async with session_factory() as session:
|
||||||
|
result = await session.execute(select(UserRow.id, UserRow.email).where(UserRow.default_workspace_id.is_(None)))
|
||||||
|
candidates = [(row.id, row.email) for row in result]
|
||||||
|
|
||||||
|
if not candidates:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
ws_repo = WorkspaceRepository(session_factory)
|
||||||
|
m_repo = WorkspaceMembershipRepository(session_factory)
|
||||||
|
created = 0
|
||||||
|
for user_id, email in candidates:
|
||||||
|
base_slug = auto_slug_from_email(email)
|
||||||
|
|
||||||
|
async def slug_exists(s: str) -> bool:
|
||||||
|
if s in SLUG_BLACKLIST:
|
||||||
|
return True
|
||||||
|
return (await ws_repo.get_by_slug(s)) is not None
|
||||||
|
|
||||||
|
unique_slug = await next_available_slug(base_slug, exists_check=slug_exists)
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
logger.info("WOULD create workspace for user=%s email=%s slug=%s", user_id, email, unique_slug)
|
||||||
|
created += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
display_local = email.split("@", 1)[0]
|
||||||
|
workspace = await ws_repo.create(
|
||||||
|
name=f"{display_local}'s Workspace"[:64],
|
||||||
|
slug=unique_slug,
|
||||||
|
owner_id=user_id,
|
||||||
|
)
|
||||||
|
await m_repo.add(workspace_id=workspace["id"], user_id=user_id, role="owner")
|
||||||
|
async with session_factory() as session:
|
||||||
|
await session.execute(update(UserRow).where(UserRow.id == user_id).values(default_workspace_id=workspace["id"]))
|
||||||
|
await session.commit()
|
||||||
|
created += 1
|
||||||
|
logger.info("Created workspace %s (slug=%s) for user=%s", workspace["id"], unique_slug, user_id)
|
||||||
|
|
||||||
|
return created
|
||||||
|
|
||||||
|
|
||||||
|
async def _step2_update_table_from_users(
|
||||||
|
session_factory: async_sessionmaker[AsyncSession],
|
||||||
|
table: str,
|
||||||
|
*,
|
||||||
|
dry_run: bool,
|
||||||
|
) -> int:
|
||||||
|
"""UPDATE *table* setting workspace_id from owner's users.default_workspace_id.
|
||||||
|
|
||||||
|
Uses a correlated subquery (portable across SQLite + Postgres). Filters
|
||||||
|
``workspace_id IS NULL AND user_id IS NOT NULL`` so already-set rows
|
||||||
|
and truly orphan rows are skipped (Step 3 handles the latter).
|
||||||
|
|
||||||
|
Returns the number of rows updated (or that *would* be updated under
|
||||||
|
``dry_run``).
|
||||||
|
"""
|
||||||
|
model = _TABLE_MODELS[table]
|
||||||
|
workspace_col = model.workspace_id
|
||||||
|
user_col = model.user_id
|
||||||
|
|
||||||
|
# Subquery: pull the user's default_workspace_id for each row.
|
||||||
|
correlated_default = select(UserRow.default_workspace_id).where(UserRow.id == user_col).scalar_subquery()
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
# Count rows whose owner has a default_workspace_id assigned — only
|
||||||
|
# those would get touched by the actual UPDATE.
|
||||||
|
count_stmt = select(func.count()).select_from(model).join(UserRow, UserRow.id == user_col).where(workspace_col.is_(None), user_col.is_not(None), UserRow.default_workspace_id.is_not(None))
|
||||||
|
async with session_factory() as session:
|
||||||
|
count = (await session.execute(count_stmt)).scalar_one() or 0
|
||||||
|
logger.info("WOULD update %d rows in %s from users.default_workspace_id", count, table)
|
||||||
|
return int(count)
|
||||||
|
|
||||||
|
stmt = update(model).where(workspace_col.is_(None), user_col.is_not(None)).values(workspace_id=correlated_default)
|
||||||
|
async with session_factory() as session:
|
||||||
|
result = await session.execute(stmt)
|
||||||
|
await session.commit()
|
||||||
|
rowcount = result.rowcount or 0
|
||||||
|
logger.info("Updated %d rows in %s from users.default_workspace_id", rowcount, table)
|
||||||
|
return int(rowcount)
|
||||||
|
|
||||||
|
|
||||||
|
async def _ensure_legacy_workspace(
|
||||||
|
session_factory: async_sessionmaker[AsyncSession],
|
||||||
|
*,
|
||||||
|
dry_run: bool,
|
||||||
|
) -> bool:
|
||||||
|
"""Create the ``legacy_workspace`` anchor row idempotently.
|
||||||
|
|
||||||
|
The anchor is needed before Step 3 can point orphan rows at it. We
|
||||||
|
pick the platform admin (``system_role='admin'``) as owner; if no
|
||||||
|
admin exists yet we fall back to the oldest user. If the database
|
||||||
|
has no users at all we refuse to continue — running this script on
|
||||||
|
an unbootstrapped DB would create a workspace with no owner and the
|
||||||
|
FK to ``users`` would fail anyway.
|
||||||
|
|
||||||
|
Returns True if the workspace was just created (or would be, under
|
||||||
|
``dry_run``). False if it already existed.
|
||||||
|
"""
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
async with session_factory() as session:
|
||||||
|
existing = await session.get(WorkspaceRow, LEGACY_WORKSPACE_ID)
|
||||||
|
if existing is not None:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async with session_factory() as session:
|
||||||
|
admin_id = (await session.execute(select(UserRow.id).where(UserRow.system_role == "admin").order_by(UserRow.created_at).limit(1))).scalar_one_or_none()
|
||||||
|
if admin_id is None:
|
||||||
|
admin_id = (await session.execute(select(UserRow.id).order_by(UserRow.created_at).limit(1))).scalar_one_or_none()
|
||||||
|
|
||||||
|
if admin_id is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"Cannot create legacy_workspace: no users exist. Bootstrap an admin via /auth/initialize before running backfill.",
|
||||||
|
)
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
logger.info("WOULD create legacy_workspace (id=%s) owned by user=%s", LEGACY_WORKSPACE_ID, admin_id)
|
||||||
|
return True
|
||||||
|
|
||||||
|
ws_repo = WorkspaceRepository(session_factory)
|
||||||
|
await ws_repo.create(
|
||||||
|
workspace_id=LEGACY_WORKSPACE_ID,
|
||||||
|
name=LEGACY_WORKSPACE_NAME,
|
||||||
|
slug=LEGACY_WORKSPACE_SLUG,
|
||||||
|
owner_id=admin_id,
|
||||||
|
)
|
||||||
|
m_repo = WorkspaceMembershipRepository(session_factory)
|
||||||
|
await m_repo.add(workspace_id=LEGACY_WORKSPACE_ID, user_id=admin_id, role="owner")
|
||||||
|
logger.info("Created legacy_workspace (id=%s) owned by user=%s", LEGACY_WORKSPACE_ID, admin_id)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
async def _step3_assign_legacy_workspace(
|
||||||
|
session_factory: async_sessionmaker[AsyncSession],
|
||||||
|
table: str,
|
||||||
|
*,
|
||||||
|
dry_run: bool,
|
||||||
|
) -> int:
|
||||||
|
"""Assign LEGACY_WORKSPACE_ID to *table* rows still missing workspace_id.
|
||||||
|
|
||||||
|
Callers should ensure :func:`_ensure_legacy_workspace` has run first;
|
||||||
|
the orchestrator does this between Step 2 and Step 3. Orphan rows are
|
||||||
|
rows whose ``user_id`` was already NULL (or pointed at a deleted user)
|
||||||
|
so Step 2's correlated subquery left them untouched.
|
||||||
|
"""
|
||||||
|
model = _TABLE_MODELS[table]
|
||||||
|
workspace_col = model.workspace_id
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
count_stmt = select(func.count()).select_from(model).where(workspace_col.is_(None))
|
||||||
|
async with session_factory() as session:
|
||||||
|
count = (await session.execute(count_stmt)).scalar_one() or 0
|
||||||
|
logger.info("WOULD assign %d orphan row(s) in %s to legacy_workspace", count, table)
|
||||||
|
return int(count)
|
||||||
|
|
||||||
|
stmt = update(model).where(workspace_col.is_(None)).values(workspace_id=LEGACY_WORKSPACE_ID)
|
||||||
|
async with session_factory() as session:
|
||||||
|
result = await session.execute(stmt)
|
||||||
|
await session.commit()
|
||||||
|
rowcount = result.rowcount or 0
|
||||||
|
logger.info("Assigned %d orphan row(s) in %s to legacy_workspace", rowcount, table)
|
||||||
|
return int(rowcount)
|
||||||
|
|
||||||
|
|
||||||
|
async def backfill(
|
||||||
|
session_factory: async_sessionmaker[AsyncSession],
|
||||||
|
*,
|
||||||
|
dry_run: bool = False,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Run all three backfill steps; return a per-step row-count report.
|
||||||
|
|
||||||
|
Order matters: Step 1 must populate ``users.default_workspace_id``
|
||||||
|
before Step 2 can correlate business rows back through ``users``.
|
||||||
|
"""
|
||||||
|
report: dict[str, Any] = {"dry_run": dry_run}
|
||||||
|
|
||||||
|
report["users_workspaces_created"] = await _step1_create_workspaces_for_users(session_factory, dry_run=dry_run)
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
report[f"{table}_from_users"] = await _step2_update_table_from_users(session_factory, table, dry_run=dry_run)
|
||||||
|
report["legacy_workspace_created"] = await _ensure_legacy_workspace(session_factory, dry_run=dry_run)
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
report[f"{table}_legacy"] = await _step3_assign_legacy_workspace(session_factory, table, dry_run=dry_run)
|
||||||
|
|
||||||
|
return report
|
||||||
|
|
||||||
|
|
||||||
|
def _build_session_factory_from_config() -> async_sessionmaker[AsyncSession]:
|
||||||
|
"""Build an async session factory from the active config.yaml.
|
||||||
|
|
||||||
|
Avoids importing on module load so unit tests can stub
|
||||||
|
``session_factory`` directly without booting the full config pipeline.
|
||||||
|
"""
|
||||||
|
from deerflow.config import get_app_config
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine_from_config
|
||||||
|
|
||||||
|
asyncio.run(init_engine_from_config(get_app_config().database))
|
||||||
|
sf = get_session_factory()
|
||||||
|
if sf is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"database.backend=memory: nothing to backfill. Switch config.yaml to sqlite/postgres first.",
|
||||||
|
)
|
||||||
|
return sf
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Backfill workspace_id on Stage 0 business tables (idempotent).")
|
||||||
|
parser.add_argument(
|
||||||
|
"--dry-run",
|
||||||
|
action="store_true",
|
||||||
|
help="Print the rows each step would touch without writing.",
|
||||||
|
)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||||
|
|
||||||
|
sf = _build_session_factory_from_config()
|
||||||
|
report = asyncio.run(backfill(sf, dry_run=args.dry_run))
|
||||||
|
|
||||||
|
logger.info("Backfill report (dry_run=%s):", args.dry_run)
|
||||||
|
for key, value in report.items():
|
||||||
|
if key == "dry_run":
|
||||||
|
continue
|
||||||
|
logger.info(" %s: %s", key, value)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,212 @@
|
|||||||
|
"""One-time migration: lift per-user thread / memory / agent dirs into per-workspace layout.
|
||||||
|
|
||||||
|
PR4 introduced ``{base_dir}/users/{user_id}/...`` for per-user isolation. PR6
|
||||||
|
adds the multi-tenant top dimension: ``{base_dir}/workspaces/{wid}/...``,
|
||||||
|
with per-user state nested under each workspace
|
||||||
|
(``{base_dir}/workspaces/{wid}/users/{uid}/memory.json`` etc.). This script
|
||||||
|
walks the old PR4 layout, looks up each user's ``default_workspace_id``
|
||||||
|
from the ``users`` table, and rewrites the path.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
PYTHONPATH=. python scripts/migrate_paths_to_workspace.py [--dry-run] [--default-workspace WID]
|
||||||
|
|
||||||
|
The script is idempotent — re-running it after a successful migration is a no-op.
|
||||||
|
A ``--dry-run`` invocation must not write anything; it only logs what would happen.
|
||||||
|
|
||||||
|
Mapping rules:
|
||||||
|
|
||||||
|
- ``{base_dir}/users/{uid}/threads/{tid}/`` -> ``{base_dir}/workspaces/{wid}/threads/{tid}/``
|
||||||
|
- ``{base_dir}/users/{uid}/memory.json`` -> ``{base_dir}/workspaces/{wid}/users/{uid}/memory.json``
|
||||||
|
- ``{base_dir}/users/{uid}/agents/{name}/`` -> ``{base_dir}/workspaces/{wid}/users/{uid}/agents/{name}/``
|
||||||
|
|
||||||
|
``{wid}`` is read from ``users.default_workspace_id``. Users without one
|
||||||
|
fall through to ``--default-workspace`` (defaults to the special bucket
|
||||||
|
``"legacy_workspace"`` so they can be triaged manually). Pre-existing
|
||||||
|
destinations are preserved; legacy copies are moved to
|
||||||
|
``{base_dir}/migration-conflicts/<...>`` for human review.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import logging
|
||||||
|
import shutil
|
||||||
|
import sqlite3
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from deerflow.config.paths import Paths, get_paths
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
LEGACY_WORKSPACE_FALLBACK = "legacy_workspace"
|
||||||
|
|
||||||
|
|
||||||
|
def _load_user_workspaces(paths: Paths) -> dict[str, str | None]:
|
||||||
|
"""Read ``user_id -> default_workspace_id`` from the local sqlite DB.
|
||||||
|
|
||||||
|
Returns an empty dict when the database does not exist (fresh install
|
||||||
|
/ Postgres-only deployments). The Postgres path is out of scope for
|
||||||
|
this script — the operator should run it with ``--default-workspace``
|
||||||
|
set to the target workspace and skip the DB lookup.
|
||||||
|
"""
|
||||||
|
db_path = paths.base_dir / "deer-flow.db"
|
||||||
|
if not db_path.exists():
|
||||||
|
logger.info("No sqlite database at %s — every user will use the fallback workspace.", db_path)
|
||||||
|
return {}
|
||||||
|
|
||||||
|
conn = sqlite3.connect(str(db_path))
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
cursor = conn.execute("SELECT id, default_workspace_id FROM users")
|
||||||
|
except sqlite3.OperationalError as e:
|
||||||
|
logger.warning("Failed to query users.default_workspace_id: %s", e)
|
||||||
|
return {}
|
||||||
|
return {row[0]: row[1] for row in cursor.fetchall()}
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_workspace(user_id: str, user_workspaces: dict[str, str | None], fallback: str) -> str:
|
||||||
|
wid = user_workspaces.get(user_id)
|
||||||
|
return wid or fallback
|
||||||
|
|
||||||
|
|
||||||
|
def _move(src: Path, dest: Path, conflict_root: Path, *, dry_run: bool, label: str) -> str:
|
||||||
|
"""Move ``src`` to ``dest``; on conflict, divert legacy under ``conflict_root``.
|
||||||
|
|
||||||
|
Returns a short string describing what happened (for the report).
|
||||||
|
"""
|
||||||
|
if dest.exists():
|
||||||
|
conflict_dest = conflict_root / label / src.name
|
||||||
|
if not dry_run:
|
||||||
|
conflict_dest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
shutil.move(str(src), str(conflict_dest))
|
||||||
|
logger.warning("Conflict for %s/%s: legacy copy diverted to %s", label, src.name, conflict_dest)
|
||||||
|
return f"conflict -> {conflict_dest}"
|
||||||
|
|
||||||
|
if not dry_run:
|
||||||
|
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
shutil.move(str(src), str(dest))
|
||||||
|
return f"moved -> {dest}"
|
||||||
|
|
||||||
|
|
||||||
|
def migrate_user_tree(
|
||||||
|
paths: Paths,
|
||||||
|
user_id: str,
|
||||||
|
workspace_id: str,
|
||||||
|
*,
|
||||||
|
dry_run: bool,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Lift one user's tree under a workspace. Returns per-asset report rows."""
|
||||||
|
report: list[dict] = []
|
||||||
|
legacy_user_dir = paths.base_dir / "users" / user_id
|
||||||
|
if not legacy_user_dir.exists():
|
||||||
|
return report
|
||||||
|
|
||||||
|
conflict_root = paths.base_dir / "migration-conflicts" / "workspace-migration"
|
||||||
|
|
||||||
|
# 1. threads/{tid}/ -> workspaces/{wid}/threads/{tid}/
|
||||||
|
legacy_threads = legacy_user_dir / "threads"
|
||||||
|
if legacy_threads.exists():
|
||||||
|
for thread_dir in sorted(legacy_threads.iterdir()):
|
||||||
|
if not thread_dir.is_dir():
|
||||||
|
continue
|
||||||
|
dest = paths.thread_dir(thread_dir.name, workspace_id=workspace_id)
|
||||||
|
action = _move(thread_dir, dest, conflict_root, dry_run=dry_run, label=f"threads/{workspace_id}")
|
||||||
|
report.append({"asset": "thread", "user_id": user_id, "workspace_id": workspace_id, "name": thread_dir.name, "action": action})
|
||||||
|
if not dry_run and legacy_threads.exists() and not any(legacy_threads.iterdir()):
|
||||||
|
legacy_threads.rmdir()
|
||||||
|
|
||||||
|
# 2. memory.json -> workspaces/{wid}/users/{uid}/memory.json
|
||||||
|
legacy_mem = legacy_user_dir / "memory.json"
|
||||||
|
if legacy_mem.exists():
|
||||||
|
dest = paths.user_memory_file(user_id, workspace_id=workspace_id)
|
||||||
|
action = _move(legacy_mem, dest, conflict_root, dry_run=dry_run, label=f"users/{user_id}/memory")
|
||||||
|
report.append({"asset": "memory", "user_id": user_id, "workspace_id": workspace_id, "name": "memory.json", "action": action})
|
||||||
|
|
||||||
|
# 3. agents/{name}/ -> workspaces/{wid}/users/{uid}/agents/{name}/
|
||||||
|
legacy_agents = legacy_user_dir / "agents"
|
||||||
|
if legacy_agents.exists():
|
||||||
|
for agent_dir in sorted(legacy_agents.iterdir()):
|
||||||
|
if not agent_dir.is_dir():
|
||||||
|
continue
|
||||||
|
dest = paths.user_agent_dir(user_id, agent_dir.name, workspace_id=workspace_id)
|
||||||
|
action = _move(agent_dir, dest, conflict_root, dry_run=dry_run, label=f"users/{user_id}/agents")
|
||||||
|
report.append({"asset": "agent", "user_id": user_id, "workspace_id": workspace_id, "name": agent_dir.name, "action": action})
|
||||||
|
if not dry_run and legacy_agents.exists() and not any(legacy_agents.iterdir()):
|
||||||
|
legacy_agents.rmdir()
|
||||||
|
|
||||||
|
# 4. If the user dir is now empty, remove it so lifespan warnings clear.
|
||||||
|
if not dry_run and legacy_user_dir.exists() and not any(legacy_user_dir.iterdir()):
|
||||||
|
legacy_user_dir.rmdir()
|
||||||
|
|
||||||
|
return report
|
||||||
|
|
||||||
|
|
||||||
|
def migrate(
|
||||||
|
paths: Paths,
|
||||||
|
*,
|
||||||
|
user_workspaces: dict[str, str | None],
|
||||||
|
fallback_workspace: str,
|
||||||
|
dry_run: bool,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Top-level entry point: iterate every user under ``{base_dir}/users``."""
|
||||||
|
legacy_users = paths.base_dir / "users"
|
||||||
|
if not legacy_users.exists():
|
||||||
|
logger.info("No legacy ``users/`` directory under %s — nothing to migrate.", paths.base_dir)
|
||||||
|
return []
|
||||||
|
|
||||||
|
report: list[dict] = []
|
||||||
|
for user_dir in sorted(legacy_users.iterdir()):
|
||||||
|
if not user_dir.is_dir():
|
||||||
|
continue
|
||||||
|
user_id = user_dir.name
|
||||||
|
workspace_id = _resolve_workspace(user_id, user_workspaces, fallback_workspace)
|
||||||
|
logger.info("Migrating user %s -> workspace %s", user_id, workspace_id)
|
||||||
|
report.extend(migrate_user_tree(paths, user_id, workspace_id, dry_run=dry_run))
|
||||||
|
|
||||||
|
if not dry_run and legacy_users.exists() and not any(legacy_users.iterdir()):
|
||||||
|
legacy_users.rmdir()
|
||||||
|
|
||||||
|
return report
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Lift legacy per-user paths into per-workspace layout (PR6).")
|
||||||
|
parser.add_argument("--dry-run", action="store_true", help="Log actions without making changes.")
|
||||||
|
parser.add_argument(
|
||||||
|
"--default-workspace",
|
||||||
|
default=LEGACY_WORKSPACE_FALLBACK,
|
||||||
|
metavar="WID",
|
||||||
|
help=(f"Workspace id to use for users without a ``default_workspace_id`` in the DB. Defaults to ``{LEGACY_WORKSPACE_FALLBACK}`` (matches the orphan-row bucket used by PR5 backfill)."),
|
||||||
|
)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
|
||||||
|
|
||||||
|
paths = get_paths()
|
||||||
|
logger.info("Base directory: %s", paths.base_dir)
|
||||||
|
logger.info("Dry run: %s", args.dry_run)
|
||||||
|
logger.info("Fallback workspace: %s", args.default_workspace)
|
||||||
|
|
||||||
|
user_workspaces = _load_user_workspaces(paths)
|
||||||
|
logger.info("Loaded %d user->workspace mappings from DB", len(user_workspaces))
|
||||||
|
|
||||||
|
report = migrate(
|
||||||
|
paths,
|
||||||
|
user_workspaces=user_workspaces,
|
||||||
|
fallback_workspace=args.default_workspace,
|
||||||
|
dry_run=args.dry_run,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not report:
|
||||||
|
logger.info("Nothing to migrate.")
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info("Migration report (%d entries):", len(report))
|
||||||
|
for entry in report:
|
||||||
|
logger.info(" asset=%s user=%s workspace=%s name=%s action=%s", entry["asset"], entry["user_id"], entry["workspace_id"], entry["name"], entry["action"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Executable
+420
@@ -0,0 +1,420 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# verify_stage0.sh — systematic verification of the Stage 0 multi-tenant rollout.
|
||||||
|
#
|
||||||
|
# Layers (each can be run individually; defaults run all that are applicable):
|
||||||
|
# static — full pytest suite + lint + boundary scan (no external deps)
|
||||||
|
# paths — local filesystem layout: legacy users/ should be empty after migration
|
||||||
|
# rds — schema + index + alembic state on the remote Postgres (needs DATABASE_URL)
|
||||||
|
# runtime — gateway health probes (needs `make dev` running)
|
||||||
|
# e2e — register 2 users via curl, each creates a thread, cross-access must 404
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# ./scripts/verify_stage0.sh # run everything that has prerequisites
|
||||||
|
# ./scripts/verify_stage0.sh static rds # only those two
|
||||||
|
# DATABASE_URL=postgres://... ./scripts/verify_stage0.sh
|
||||||
|
# GATEWAY_URL=http://localhost:8001 ./scripts/verify_stage0.sh runtime e2e
|
||||||
|
#
|
||||||
|
# Exit code: 0 if every executed assertion passes, 1 if any fail.
|
||||||
|
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
# ── colours ──────────────────────────────────────────────────────────────
|
||||||
|
if [ -t 1 ]; then
|
||||||
|
C_RED=$'\033[0;31m'; C_GREEN=$'\033[0;32m'; C_YELLOW=$'\033[0;33m'
|
||||||
|
C_BLUE=$'\033[0;34m'; C_BOLD=$'\033[1m'; C_DIM=$'\033[2m'; C_RESET=$'\033[0m'
|
||||||
|
else
|
||||||
|
C_RED=''; C_GREEN=''; C_YELLOW=''; C_BLUE=''; C_BOLD=''; C_DIM=''; C_RESET=''
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ── counters ─────────────────────────────────────────────────────────────
|
||||||
|
PASS_COUNT=0
|
||||||
|
FAIL_COUNT=0
|
||||||
|
WARN_COUNT=0
|
||||||
|
FAILED_STEPS=()
|
||||||
|
SKIPPED_PHASES=()
|
||||||
|
|
||||||
|
ok() { printf ' %s✓%s %s\n' "$C_GREEN" "$C_RESET" "$1"; PASS_COUNT=$((PASS_COUNT+1)); }
|
||||||
|
fail() { printf ' %s✗%s %s\n' "$C_RED" "$C_RESET" "$1"; FAIL_COUNT=$((FAIL_COUNT+1)); FAILED_STEPS+=("$1"); }
|
||||||
|
warn() { printf ' %s!%s %s\n' "$C_YELLOW" "$C_RESET" "$1"; WARN_COUNT=$((WARN_COUNT+1)); }
|
||||||
|
info() { printf ' %s·%s %s\n' "$C_DIM" "$C_RESET" "$1"; }
|
||||||
|
phase() { printf '\n%s== %s ==%s\n' "$C_BLUE$C_BOLD" "$1" "$C_RESET"; }
|
||||||
|
|
||||||
|
# Run-cmd helpers — capture both streams for grep but return original exit code.
|
||||||
|
run() {
|
||||||
|
local label="$1"; shift
|
||||||
|
local out
|
||||||
|
out=$("$@" 2>&1)
|
||||||
|
local rc=$?
|
||||||
|
if [ "$rc" -eq 0 ]; then
|
||||||
|
ok "$label"
|
||||||
|
printf '%s' "$out"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fail "$label (exit $rc)"
|
||||||
|
printf '%s%s%s\n' "$C_DIM" "$out" "$C_RESET" >&2
|
||||||
|
return "$rc"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── env ──────────────────────────────────────────────────────────────────
|
||||||
|
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||||
|
BACKEND_DIR="$REPO_ROOT/backend"
|
||||||
|
GATEWAY_URL="${GATEWAY_URL:-http://localhost:8001}"
|
||||||
|
DEER_FLOW_HOME="${DEER_FLOW_HOME:-$REPO_ROOT/.deer-flow}"
|
||||||
|
|
||||||
|
# ── phase: static ────────────────────────────────────────────────────────
|
||||||
|
phase_static() {
|
||||||
|
phase "STATIC — unit + boundary + full pytest"
|
||||||
|
cd "$BACKEND_DIR"
|
||||||
|
|
||||||
|
# boundary scans first — fast and high-signal
|
||||||
|
info "running boundary scans (harness + workspace)"
|
||||||
|
if PYTHONPATH=. uv run pytest -q \
|
||||||
|
tests/test_harness_boundary.py \
|
||||||
|
tests/test_workspace_boundary.py \
|
||||||
|
tests/test_workspace_boundary_self.py >/tmp/verify_boundary.log 2>&1; then
|
||||||
|
local boundary_passed
|
||||||
|
boundary_passed=$(grep -Eo '[0-9]+ passed' /tmp/verify_boundary.log | head -1)
|
||||||
|
ok "boundary scans green ($boundary_passed)"
|
||||||
|
else
|
||||||
|
fail "boundary scans red — see /tmp/verify_boundary.log"
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "running full pytest (this takes ~90-130s)"
|
||||||
|
local pytest_log=/tmp/verify_full_pytest.log
|
||||||
|
PYTHONPATH=. uv run pytest -q >"$pytest_log" 2>&1
|
||||||
|
local rc=$?
|
||||||
|
local last_line
|
||||||
|
last_line=$(tail -1 "$pytest_log")
|
||||||
|
info "result: $last_line"
|
||||||
|
# Parse "<X> passed, <Y> failed, <Z> skipped"
|
||||||
|
local passed_n failed_n
|
||||||
|
passed_n=$(printf '%s' "$last_line" | grep -Eo '[0-9]+ passed' | grep -Eo '[0-9]+' | head -1 || echo 0)
|
||||||
|
failed_n=$(printf '%s' "$last_line" | grep -Eo '[0-9]+ failed' | grep -Eo '[0-9]+' | head -1 || echo 0)
|
||||||
|
if [ "${passed_n:-0}" -ge 3250 ]; then
|
||||||
|
ok "pytest pass count $passed_n ≥ 3250 (PR8 baseline)"
|
||||||
|
else
|
||||||
|
fail "pytest pass count $passed_n < 3250 — regression suspected"
|
||||||
|
fi
|
||||||
|
if [ "${failed_n:-0}" -le 18 ]; then
|
||||||
|
ok "pytest fail count $failed_n ≤ 18 (Stage 0 known-flake ceiling)"
|
||||||
|
[ "${failed_n:-0}" -gt 0 ] && warn "non-zero failures expected to be in the 18 caplog flake set; cross-check tail of /tmp/verify_full_pytest.log"
|
||||||
|
else
|
||||||
|
fail "pytest fail count $failed_n > 18 — new failures introduced beyond known caplog flake set"
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "running ruff lint"
|
||||||
|
if make lint >/tmp/verify_lint.log 2>&1; then
|
||||||
|
ok "ruff lint clean"
|
||||||
|
else
|
||||||
|
fail "ruff lint dirty — see /tmp/verify_lint.log"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── phase: paths ─────────────────────────────────────────────────────────
|
||||||
|
phase_paths() {
|
||||||
|
phase "PATHS — legacy users/ migration state"
|
||||||
|
if [ ! -d "$DEER_FLOW_HOME" ]; then
|
||||||
|
warn "DEER_FLOW_HOME ($DEER_FLOW_HOME) does not exist — fresh install, nothing to migrate"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "DEER_FLOW_HOME = $DEER_FLOW_HOME"
|
||||||
|
local legacy_dir="$DEER_FLOW_HOME/users"
|
||||||
|
if [ ! -d "$legacy_dir" ]; then
|
||||||
|
ok "no legacy users/ directory present (PR6 migration not needed or already done)"
|
||||||
|
else
|
||||||
|
local count
|
||||||
|
count=$(find "$legacy_dir" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | wc -l | tr -d ' ')
|
||||||
|
if [ "$count" -eq 0 ]; then
|
||||||
|
ok "legacy users/ is empty (migration complete or no prior data)"
|
||||||
|
else
|
||||||
|
warn "legacy users/ still has $count user dir(s) — run \`make migrate-paths\` (after \`make migrate-paths DRY_RUN=1\` to preview)"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
local workspace_dir="$DEER_FLOW_HOME/workspaces"
|
||||||
|
if [ -d "$workspace_dir" ]; then
|
||||||
|
local wcount
|
||||||
|
wcount=$(find "$workspace_dir" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | wc -l | tr -d ' ')
|
||||||
|
ok "workspaces/ layout in place ($wcount workspace dir(s))"
|
||||||
|
else
|
||||||
|
warn "workspaces/ dir does not exist yet — will be created when the first thread is opened"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Stage 0 PR6 dry-run check: just exercise the script flag, do not perform writes.
|
||||||
|
info "exercising migrate-paths dry-run (no writes)"
|
||||||
|
cd "$REPO_ROOT"
|
||||||
|
if make migrate-paths DRY_RUN=1 >/tmp/verify_migrate_dry.log 2>&1; then
|
||||||
|
ok "make migrate-paths DRY_RUN=1 ran without error"
|
||||||
|
else
|
||||||
|
fail "make migrate-paths DRY_RUN=1 failed — see /tmp/verify_migrate_dry.log"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── phase: rds ───────────────────────────────────────────────────────────
|
||||||
|
phase_rds() {
|
||||||
|
phase "RDS — schema + alembic + indexes"
|
||||||
|
if [ -z "${DATABASE_URL:-}" ]; then
|
||||||
|
warn "DATABASE_URL not set — skipping RDS phase"
|
||||||
|
SKIPPED_PHASES+=("rds (DATABASE_URL unset)")
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
if ! command -v psql >/dev/null 2>&1; then
|
||||||
|
fail "psql not installed — cannot run RDS phase"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
info "DATABASE_URL host: $(printf '%s' "$DATABASE_URL" | sed -E 's#.*@([^/]+)/.*#\1#')"
|
||||||
|
|
||||||
|
# PG version sanity
|
||||||
|
local pg_version
|
||||||
|
pg_version=$(psql "$DATABASE_URL" -tA -c "SELECT version()" 2>/dev/null | head -1)
|
||||||
|
if [ -z "$pg_version" ]; then
|
||||||
|
fail "cannot connect to RDS — check DATABASE_URL"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
ok "PG reachable: $pg_version"
|
||||||
|
|
||||||
|
# Alembic head
|
||||||
|
info "checking alembic head against expected 0003"
|
||||||
|
cd "$BACKEND_DIR"
|
||||||
|
if PYTHONPATH=. uv run alembic current 2>/tmp/verify_alembic.err | tee /tmp/verify_alembic.log | grep -q "^0003"; then
|
||||||
|
ok "alembic current is at 0003 (workspace_id NOT NULL + UNIQUE(wid, tid))"
|
||||||
|
else
|
||||||
|
fail "alembic current is not at 0003 — see /tmp/verify_alembic.log"
|
||||||
|
fi
|
||||||
|
cd "$REPO_ROOT"
|
||||||
|
|
||||||
|
# PR8 tables present
|
||||||
|
info "checking PR8 tables exist"
|
||||||
|
local pr8_count
|
||||||
|
pr8_count=$(psql "$DATABASE_URL" -tA -c "
|
||||||
|
SELECT count(*) FROM information_schema.tables
|
||||||
|
WHERE table_schema = current_schema()
|
||||||
|
AND table_name IN ('service_accounts','api_keys','external_users');
|
||||||
|
" 2>/dev/null | tr -d ' ')
|
||||||
|
if [ "${pr8_count:-0}" -eq 3 ]; then
|
||||||
|
ok "service_accounts + api_keys + external_users all present"
|
||||||
|
else
|
||||||
|
fail "PR8 tables missing — expected 3, got ${pr8_count:-0}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Partial index on api_keys: WHERE revoked_at IS NULL
|
||||||
|
info "checking idx_api_keys_active partial-index predicate"
|
||||||
|
local idx_def
|
||||||
|
idx_def=$(psql "$DATABASE_URL" -tA -c "
|
||||||
|
SELECT indexdef FROM pg_indexes
|
||||||
|
WHERE schemaname = current_schema()
|
||||||
|
AND indexname = 'idx_api_keys_active';
|
||||||
|
" 2>/dev/null | head -1)
|
||||||
|
if printf '%s' "$idx_def" | grep -qi 'WHERE.*revoked_at IS NULL'; then
|
||||||
|
ok "idx_api_keys_active has 'WHERE revoked_at IS NULL' predicate"
|
||||||
|
else
|
||||||
|
fail "idx_api_keys_active missing or wrong predicate — got: ${idx_def:-<none>}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 4 business tables workspace_id NOT NULL
|
||||||
|
info "checking workspace_id NOT NULL on 4 business tables"
|
||||||
|
local nullable_rows
|
||||||
|
nullable_rows=$(psql "$DATABASE_URL" -tA -c "
|
||||||
|
SELECT table_name FROM information_schema.columns
|
||||||
|
WHERE table_schema = current_schema()
|
||||||
|
AND column_name = 'workspace_id'
|
||||||
|
AND table_name IN ('thread_meta','runs','feedback','run_events')
|
||||||
|
AND is_nullable = 'YES';
|
||||||
|
" 2>/dev/null)
|
||||||
|
if [ -z "$nullable_rows" ]; then
|
||||||
|
ok "thread_meta + runs + feedback + run_events all have workspace_id NOT NULL"
|
||||||
|
else
|
||||||
|
fail "workspace_id is nullable in: $(printf '%s' "$nullable_rows" | tr '\n' ' ')"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Unique (workspace_id, thread_id) on thread_meta
|
||||||
|
info "checking UNIQUE(workspace_id, thread_id) on thread_meta"
|
||||||
|
local uq_count
|
||||||
|
uq_count=$(psql "$DATABASE_URL" -tA -c "
|
||||||
|
SELECT count(*) FROM pg_indexes
|
||||||
|
WHERE schemaname = current_schema()
|
||||||
|
AND tablename = 'thread_meta'
|
||||||
|
AND indexdef ILIKE '%UNIQUE%workspace_id%thread_id%';
|
||||||
|
" 2>/dev/null | tr -d ' ')
|
||||||
|
if [ "${uq_count:-0}" -ge 1 ]; then
|
||||||
|
ok "UNIQUE(workspace_id, thread_id) constraint/index present"
|
||||||
|
else
|
||||||
|
fail "no UNIQUE(workspace_id, thread_id) on thread_meta"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── phase: runtime ───────────────────────────────────────────────────────
|
||||||
|
phase_runtime() {
|
||||||
|
phase "RUNTIME — gateway health"
|
||||||
|
if ! command -v curl >/dev/null 2>&1; then
|
||||||
|
fail "curl missing — cannot run runtime phase"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
info "probing $GATEWAY_URL/health"
|
||||||
|
local health
|
||||||
|
health=$(curl -sS -o /dev/null -w '%{http_code}' --max-time 5 "$GATEWAY_URL/health" 2>/dev/null || true)
|
||||||
|
if [ "$health" = "200" ]; then
|
||||||
|
ok "gateway /health returns 200"
|
||||||
|
else
|
||||||
|
fail "gateway /health returned '$health' — is \`make dev\` running?"
|
||||||
|
SKIPPED_PHASES+=("e2e (gateway not reachable)")
|
||||||
|
SKIP_E2E=1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── phase: e2e ───────────────────────────────────────────────────────────
|
||||||
|
phase_e2e() {
|
||||||
|
phase "E2E — register 2 users + cross-workspace 404"
|
||||||
|
if [ "${SKIP_E2E:-0}" = "1" ]; then
|
||||||
|
warn "skipped because gateway probe failed"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
if ! command -v curl >/dev/null 2>&1; then
|
||||||
|
fail "curl missing"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
local tag
|
||||||
|
tag=$(date +%s)
|
||||||
|
local alice="alice-${tag}@verify.local"
|
||||||
|
local bob="bob-${tag}@verify.local"
|
||||||
|
local pw="VerifyStage0_${tag}"
|
||||||
|
local jar_a=/tmp/verify_alice_${tag}.cookies
|
||||||
|
local jar_b=/tmp/verify_bob_${tag}.cookies
|
||||||
|
rm -f "$jar_a" "$jar_b"
|
||||||
|
|
||||||
|
register_user() {
|
||||||
|
local jar="$1"; local email="$2"
|
||||||
|
curl -sS -c "$jar" -o /tmp/verify_register_$$.json -w '%{http_code}' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "{\"email\":\"$email\",\"password\":\"$pw\"}" \
|
||||||
|
"$GATEWAY_URL/api/auth/register"
|
||||||
|
}
|
||||||
|
|
||||||
|
info "registering Alice + Bob"
|
||||||
|
local code_a code_b
|
||||||
|
code_a=$(register_user "$jar_a" "$alice")
|
||||||
|
code_b=$(register_user "$jar_b" "$bob")
|
||||||
|
if [ "$code_a" = "201" ] && [ "$code_b" = "201" ]; then
|
||||||
|
ok "both users registered (201 / 201)"
|
||||||
|
else
|
||||||
|
fail "registration failed: alice=$code_a, bob=$code_b"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Pull CSRF tokens from cookie jars (csrf_token cookie value, 4th-from-last field).
|
||||||
|
csrf_from_jar() {
|
||||||
|
awk '$6 == "csrf_token" { print $7 }' "$1" | tail -1
|
||||||
|
}
|
||||||
|
local csrf_a csrf_b
|
||||||
|
csrf_a=$(csrf_from_jar "$jar_a")
|
||||||
|
csrf_b=$(csrf_from_jar "$jar_b")
|
||||||
|
if [ -n "$csrf_a" ] && [ -n "$csrf_b" ]; then
|
||||||
|
ok "CSRF token captured for both sessions"
|
||||||
|
else
|
||||||
|
fail "CSRF cookie missing (alice='${csrf_a:0:8}...', bob='${csrf_b:0:8}...')"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
create_thread() {
|
||||||
|
local jar="$1"; local csrf="$2"; local tid="$3"
|
||||||
|
curl -sS -b "$jar" -o /tmp/verify_thread_$$.json -w '%{http_code}' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-H "X-CSRF-Token: $csrf" \
|
||||||
|
-d "{\"thread_id\":\"$tid\"}" \
|
||||||
|
"$GATEWAY_URL/api/threads"
|
||||||
|
}
|
||||||
|
|
||||||
|
local tid_a="verify-alice-${tag}"
|
||||||
|
local tid_b="verify-bob-${tag}"
|
||||||
|
info "Alice creates thread $tid_a; Bob creates thread $tid_b"
|
||||||
|
local cta ctb
|
||||||
|
cta=$(create_thread "$jar_a" "$csrf_a" "$tid_a")
|
||||||
|
ctb=$(create_thread "$jar_b" "$csrf_b" "$tid_b")
|
||||||
|
if [ "$cta" = "200" ] && [ "$ctb" = "200" ]; then
|
||||||
|
ok "both threads created (200 / 200)"
|
||||||
|
else
|
||||||
|
fail "thread creation failed: alice=$cta, bob=$ctb"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Cross access: Alice tries to GET Bob's thread → must be 404 (per PR6 contract:
|
||||||
|
# cross-workspace returns 404, not 403, to avoid leaking existence).
|
||||||
|
info "Alice → Bob's thread (GET)"
|
||||||
|
local cross_get
|
||||||
|
cross_get=$(curl -sS -b "$jar_a" -o /dev/null -w '%{http_code}' "$GATEWAY_URL/api/threads/$tid_b")
|
||||||
|
if [ "$cross_get" = "404" ]; then
|
||||||
|
ok "cross-workspace GET returns 404 (no existence leak)"
|
||||||
|
else
|
||||||
|
fail "cross-workspace GET returned '$cross_get', expected 404 — PR6 isolation broken"
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "Alice → Bob's thread (DELETE)"
|
||||||
|
local cross_del
|
||||||
|
cross_del=$(curl -sS -b "$jar_a" -X DELETE -H "X-CSRF-Token: $csrf_a" \
|
||||||
|
-o /dev/null -w '%{http_code}' "$GATEWAY_URL/api/threads/$tid_b")
|
||||||
|
if [ "$cross_del" = "404" ]; then
|
||||||
|
ok "cross-workspace DELETE returns 404"
|
||||||
|
else
|
||||||
|
fail "cross-workspace DELETE returned '$cross_del', expected 404"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Same-workspace GET — sanity check Alice can still reach her own thread.
|
||||||
|
info "Alice → Alice's thread (sanity)"
|
||||||
|
local same_get
|
||||||
|
same_get=$(curl -sS -b "$jar_a" -o /dev/null -w '%{http_code}' "$GATEWAY_URL/api/threads/$tid_a")
|
||||||
|
if [ "$same_get" = "200" ]; then
|
||||||
|
ok "same-workspace GET returns 200 (isolation is not over-blocking)"
|
||||||
|
else
|
||||||
|
fail "same-workspace GET returned '$same_get', expected 200"
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "cookie jars left in /tmp for debugging: $jar_a $jar_b"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── main ─────────────────────────────────────────────────────────────────
|
||||||
|
main() {
|
||||||
|
local args=("$@")
|
||||||
|
if [ ${#args[@]} -eq 0 ]; then
|
||||||
|
args=(static paths rds runtime e2e)
|
||||||
|
fi
|
||||||
|
|
||||||
|
printf '%s%sStage 0 verification — %s%s\n' "$C_BOLD" "$C_BLUE" "$(date)" "$C_RESET"
|
||||||
|
printf 'Repo root: %s\n' "$REPO_ROOT"
|
||||||
|
printf 'Phases: %s\n' "${args[*]}"
|
||||||
|
|
||||||
|
for phase_name in "${args[@]}"; do
|
||||||
|
case "$phase_name" in
|
||||||
|
static) phase_static ;;
|
||||||
|
paths) phase_paths ;;
|
||||||
|
rds) phase_rds ;;
|
||||||
|
runtime) phase_runtime ;;
|
||||||
|
e2e) phase_e2e ;;
|
||||||
|
all)
|
||||||
|
phase_static; phase_paths; phase_rds; phase_runtime; phase_e2e
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
warn "unknown phase: $phase_name"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
printf '\n%s%s──── summary ────%s\n' "$C_BOLD" "$C_BLUE" "$C_RESET"
|
||||||
|
printf ' %s%d passed%s %s%d failed%s %s%d warn%s\n' \
|
||||||
|
"$C_GREEN" "$PASS_COUNT" "$C_RESET" \
|
||||||
|
"$C_RED" "$FAIL_COUNT" "$C_RESET" \
|
||||||
|
"$C_YELLOW" "$WARN_COUNT" "$C_RESET"
|
||||||
|
if [ ${#SKIPPED_PHASES[@]} -gt 0 ]; then
|
||||||
|
printf ' skipped: %s\n' "${SKIPPED_PHASES[*]}"
|
||||||
|
fi
|
||||||
|
if [ ${#FAILED_STEPS[@]} -gt 0 ]; then
|
||||||
|
printf '\n%sfailing steps:%s\n' "$C_RED" "$C_RESET"
|
||||||
|
for step in "${FAILED_STEPS[@]}"; do
|
||||||
|
printf ' ✗ %s\n' "$step"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
[ "$FAIL_COUNT" -eq 0 ]
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
@@ -36,8 +36,16 @@ from fastapi import FastAPI, Request, Response
|
|||||||
from starlette.middleware.base import BaseHTTPMiddleware
|
from starlette.middleware.base import BaseHTTPMiddleware
|
||||||
from starlette.types import ASGIApp
|
from starlette.types import ASGIApp
|
||||||
|
|
||||||
from app.gateway.auth.models import User
|
from app.gateway.auth.models import ActiveWorkspace, User
|
||||||
from app.gateway.authz import AuthContext, Permissions
|
from app.gateway.authz import AuthContext, Permissions
|
||||||
|
from deerflow.runtime.user_context import (
|
||||||
|
reset_current_user,
|
||||||
|
set_current_user,
|
||||||
|
)
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
reset_current_workspace,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
# Default permission set granted to the stub user. Mirrors `_ALL_PERMISSIONS`
|
# Default permission set granted to the stub user. Mirrors `_ALL_PERMISSIONS`
|
||||||
# in authz.py — kept inline so the tests don't import a private symbol.
|
# in authz.py — kept inline so the tests don't import a private symbol.
|
||||||
@@ -67,22 +75,55 @@ class _StubAuthMiddleware(BaseHTTPMiddleware):
|
|||||||
Mirrors what production ``AuthMiddleware`` does after the JWT decode
|
Mirrors what production ``AuthMiddleware`` does after the JWT decode
|
||||||
+ DB lookup short-circuit, so ``@require_permission`` finds an
|
+ DB lookup short-circuit, so ``@require_permission`` finds an
|
||||||
authenticated context and skips its own re-authentication path.
|
authenticated context and skips its own re-authentication path.
|
||||||
|
|
||||||
|
Optionally stamps the workspace contextvar too — needed by the PR6
|
||||||
|
decorator path that calls ``get_effective_workspace_id()`` before
|
||||||
|
delegating to ``check_access``.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, app: ASGIApp, user_factory: Callable[[], User]) -> None:
|
def __init__(
|
||||||
|
self,
|
||||||
|
app: ASGIApp,
|
||||||
|
user_factory: Callable[[], User],
|
||||||
|
workspace_factory: Callable[[], ActiveWorkspace | None] | None = None,
|
||||||
|
override_user_contextvar: bool = False,
|
||||||
|
) -> None:
|
||||||
super().__init__(app)
|
super().__init__(app)
|
||||||
self._user_factory = user_factory
|
self._user_factory = user_factory
|
||||||
|
self._workspace_factory = workspace_factory
|
||||||
|
# Tests that only need ``request.state.auth`` (the @require_permission
|
||||||
|
# path) keep the autouse user contextvar — flipping it to a per-call
|
||||||
|
# UUID would break legacy tests whose routes resolve paths via
|
||||||
|
# ``get_effective_user_id()``. Cross-user / cross-workspace tests opt
|
||||||
|
# in by setting this flag so the contextvar matches the request user.
|
||||||
|
self._override_user_contextvar = override_user_contextvar
|
||||||
|
|
||||||
async def dispatch(self, request: Request, call_next: Callable) -> Response:
|
async def dispatch(self, request: Request, call_next: Callable) -> Response:
|
||||||
user = self._user_factory()
|
user = self._user_factory()
|
||||||
request.state.user = user
|
request.state.user = user
|
||||||
request.state.auth = AuthContext(user=user, permissions=list(_STUB_PERMISSIONS))
|
request.state.auth = AuthContext(user=user, permissions=list(_STUB_PERMISSIONS))
|
||||||
return await call_next(request)
|
|
||||||
|
user_token = set_current_user(user) if self._override_user_contextvar else None
|
||||||
|
ws_token = None
|
||||||
|
if self._workspace_factory is not None:
|
||||||
|
workspace = self._workspace_factory()
|
||||||
|
if workspace is not None:
|
||||||
|
request.state.workspace = workspace
|
||||||
|
ws_token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
return await call_next(request)
|
||||||
|
finally:
|
||||||
|
if ws_token is not None:
|
||||||
|
reset_current_workspace(ws_token)
|
||||||
|
if user_token is not None:
|
||||||
|
reset_current_user(user_token)
|
||||||
|
|
||||||
|
|
||||||
def make_authed_test_app(
|
def make_authed_test_app(
|
||||||
*,
|
*,
|
||||||
user_factory: Callable[[], User] | None = None,
|
user_factory: Callable[[], User] | None = None,
|
||||||
|
workspace_factory: Callable[[], ActiveWorkspace | None] | None = None,
|
||||||
|
override_user_contextvar: bool = False,
|
||||||
owner_check_passes: bool = True,
|
owner_check_passes: bool = True,
|
||||||
) -> FastAPI:
|
) -> FastAPI:
|
||||||
"""Build a FastAPI test app with stub auth + permissive thread_store.
|
"""Build a FastAPI test app with stub auth + permissive thread_store.
|
||||||
@@ -103,7 +144,12 @@ def make_authed_test_app(
|
|||||||
"""
|
"""
|
||||||
factory = user_factory or _make_stub_user
|
factory = user_factory or _make_stub_user
|
||||||
app = FastAPI()
|
app = FastAPI()
|
||||||
app.add_middleware(_StubAuthMiddleware, user_factory=factory)
|
app.add_middleware(
|
||||||
|
_StubAuthMiddleware,
|
||||||
|
user_factory=factory,
|
||||||
|
workspace_factory=workspace_factory,
|
||||||
|
override_user_contextvar=override_user_contextvar,
|
||||||
|
)
|
||||||
|
|
||||||
repo = MagicMock()
|
repo = MagicMock()
|
||||||
repo.check_access = AsyncMock(return_value=owner_check_passes)
|
repo.check_access = AsyncMock(return_value=owner_check_passes)
|
||||||
|
|||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# PR7 — boundary scan allowlist for direct LangGraph checkpoint/saver imports.
|
||||||
|
#
|
||||||
|
# Paths are relative to backend/. A path here means: "this file is permitted to
|
||||||
|
# import langgraph.checkpoint.* at runtime". Adding to this list requires a
|
||||||
|
# matching justification in the PR that introduces the new importer.
|
||||||
|
#
|
||||||
|
# Consumed by tests/test_workspace_boundary.py.
|
||||||
|
#
|
||||||
|
# Imports inside `if TYPE_CHECKING:` blocks are exempt automatically — they do
|
||||||
|
# not pull the symbol into runtime — so type-only references (e.g. annotations
|
||||||
|
# on `BaseCheckpointSaver` parameters) do NOT need to be listed here.
|
||||||
|
|
||||||
|
langgraph_checkpoint_importers = [
|
||||||
|
# Gateway thread plumbing: constructs an empty checkpoint when initialising
|
||||||
|
# a new thread's state via the LangGraph runtime.
|
||||||
|
"app/gateway/routers/threads.py",
|
||||||
|
|
||||||
|
# Harness checkpointer factories: the single authorised place to construct
|
||||||
|
# InMemorySaver / SqliteSaver / PostgresSaver implementations. Everywhere
|
||||||
|
# else must obtain a checkpointer via `app.gateway.deps.get_checkpointer`
|
||||||
|
# or `deerflow.runtime.checkpointer` helpers.
|
||||||
|
"packages/harness/deerflow/runtime/checkpointer/async_provider.py",
|
||||||
|
"packages/harness/deerflow/runtime/checkpointer/provider.py",
|
||||||
|
|
||||||
|
# Background run worker: uses `empty_checkpoint` to seed state for runs
|
||||||
|
# resumed from a missing/expired checkpoint id.
|
||||||
|
"packages/harness/deerflow/runtime/runs/worker.py",
|
||||||
|
]
|
||||||
@@ -15,6 +15,13 @@ import pytest
|
|||||||
# Make 'app' and 'deerflow' importable from any working directory
|
# Make 'app' and 'deerflow' importable from any working directory
|
||||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "scripts"))
|
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "scripts"))
|
||||||
|
# Make 'fixtures.*' importable as plugin modules from this conftest.
|
||||||
|
sys.path.insert(0, str(Path(__file__).parent))
|
||||||
|
|
||||||
|
# Register fixture plugin modules so tests can request fixtures by name
|
||||||
|
# without ad-hoc imports. ``fixtures.postgres`` provides
|
||||||
|
# ``postgres_container`` (session-scoped) and ``postgres_url`` (per-test).
|
||||||
|
pytest_plugins = ["fixtures.postgres"]
|
||||||
|
|
||||||
# Break the circular import chain that exists in production code:
|
# Break the circular import chain that exists in production code:
|
||||||
# deerflow.subagents.__init__
|
# deerflow.subagents.__init__
|
||||||
@@ -38,6 +45,95 @@ _executor_mock.get_background_task_result = MagicMock()
|
|||||||
sys.modules["deerflow.subagents.executor"] = _executor_mock
|
sys.modules["deerflow.subagents.executor"] = _executor_mock
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Auto-seed test workspace + user when Base.metadata.create_all() runs
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# PR6 makes every business-row INSERT carry ``workspace_id`` (resolved
|
||||||
|
# from the autouse workspace contextvar = "test-workspace-autouse"). The
|
||||||
|
# Stage 0 schema has a NOT NULL FK from those rows to ``workspaces`` and
|
||||||
|
# from ``workspaces.owner_id`` to ``users``. Without the seed below,
|
||||||
|
# every legacy repo test would fail with a FOREIGN KEY error the moment
|
||||||
|
# it tries to insert a thread.
|
||||||
|
#
|
||||||
|
# We register an ``after_create`` hook on ``Base.metadata`` so that
|
||||||
|
# whenever ``init_engine`` finishes ``create_all()`` (the auto-create
|
||||||
|
# path used by tests and dev), the two anchor rows are present. Alembic
|
||||||
|
# migration tests don't trigger create_all so they are unaffected and
|
||||||
|
# keep exercising real FK constraints in isolation.
|
||||||
|
|
||||||
|
|
||||||
|
def _register_test_seed_listener() -> None:
|
||||||
|
"""Attach an after_create hook that seeds the autouse user + workspace."""
|
||||||
|
try:
|
||||||
|
from sqlalchemy import event, update
|
||||||
|
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||||
|
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||||
|
|
||||||
|
from deerflow.persistence.base import Base
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
except ImportError:
|
||||||
|
return
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
def _seed(_target, connection, **kw): # noqa: ARG001
|
||||||
|
tables = {t.name for t in kw.get("tables", []) or []}
|
||||||
|
if "users" not in tables or "workspaces" not in tables:
|
||||||
|
return
|
||||||
|
|
||||||
|
dialect = connection.dialect.name
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
|
||||||
|
# Seed both rows with a consistent ``default_workspace_id`` so the
|
||||||
|
# PR5 backfill script (which scans ``users.default_workspace_id IS
|
||||||
|
# NULL``) does not pick up the test fixtures as candidates.
|
||||||
|
# Insert user first with NULL default_workspace_id (chicken-and-egg
|
||||||
|
# with workspaces.owner_id FK), then workspace, then UPDATE the user
|
||||||
|
# row to point at the workspace so the PR5 backfill script does not
|
||||||
|
# pick up the autouse user as a candidate.
|
||||||
|
user_values = {
|
||||||
|
"id": "test-user-autouse",
|
||||||
|
"email": "test-user-autouse@local",
|
||||||
|
"password_hash": None,
|
||||||
|
"system_role": "user",
|
||||||
|
"created_at": now,
|
||||||
|
"oauth_provider": None,
|
||||||
|
"oauth_id": None,
|
||||||
|
"needs_setup": False,
|
||||||
|
"token_version": 0,
|
||||||
|
"default_workspace_id": None,
|
||||||
|
}
|
||||||
|
workspace_values = {
|
||||||
|
"id": "test-workspace-autouse",
|
||||||
|
"name": "Autouse Test Workspace",
|
||||||
|
"slug": "autouse-test",
|
||||||
|
"status": "active",
|
||||||
|
"owner_id": "test-user-autouse",
|
||||||
|
"created_at": now,
|
||||||
|
"updated_at": now,
|
||||||
|
}
|
||||||
|
|
||||||
|
if dialect == "sqlite":
|
||||||
|
user_stmt = sqlite_insert(UserRow.__table__).values(**user_values).on_conflict_do_nothing(index_elements=["id"])
|
||||||
|
ws_stmt = sqlite_insert(WorkspaceRow.__table__).values(**workspace_values).on_conflict_do_nothing(index_elements=["id"])
|
||||||
|
elif dialect == "postgresql":
|
||||||
|
user_stmt = pg_insert(UserRow.__table__).values(**user_values).on_conflict_do_nothing(index_elements=["id"])
|
||||||
|
ws_stmt = pg_insert(WorkspaceRow.__table__).values(**workspace_values).on_conflict_do_nothing(index_elements=["id"])
|
||||||
|
else:
|
||||||
|
return
|
||||||
|
|
||||||
|
connection.execute(user_stmt)
|
||||||
|
connection.execute(ws_stmt)
|
||||||
|
connection.execute(update(UserRow.__table__).where(UserRow.__table__.c.id == "test-user-autouse").where(UserRow.__table__.c.default_workspace_id.is_(None)).values(default_workspace_id="test-workspace-autouse"))
|
||||||
|
|
||||||
|
event.listen(Base.metadata, "after_create", _seed)
|
||||||
|
|
||||||
|
|
||||||
|
_register_test_seed_listener()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
@pytest.fixture()
|
||||||
def provisioner_module():
|
def provisioner_module():
|
||||||
"""Load docker/provisioner/app.py as an importable test module.
|
"""Load docker/provisioner/app.py as an importable test module.
|
||||||
@@ -110,3 +206,34 @@ def _auto_user_context(request):
|
|||||||
yield
|
yield
|
||||||
finally:
|
finally:
|
||||||
reset_current_user(token)
|
reset_current_user(token)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _auto_workspace_context(request):
|
||||||
|
"""Inject a default ``test-workspace-autouse`` into the workspace contextvar.
|
||||||
|
|
||||||
|
Mirror of :func:`_auto_user_context`. PR6 adds ``workspace_id=AUTO``
|
||||||
|
sentinels to every repository method; without an autouse workspace
|
||||||
|
fixture every legacy persistence test would raise RuntimeError.
|
||||||
|
|
||||||
|
Opt-out via ``@pytest.mark.no_auto_workspace``.
|
||||||
|
"""
|
||||||
|
if request.node.get_closest_marker("no_auto_workspace"):
|
||||||
|
yield
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
reset_current_workspace,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
except ImportError:
|
||||||
|
yield
|
||||||
|
return
|
||||||
|
|
||||||
|
workspace = SimpleNamespace(id="test-workspace-autouse", role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|||||||
Vendored
+82
@@ -0,0 +1,82 @@
|
|||||||
|
"""Postgres testcontainer fixtures for Stage 0 PR1.
|
||||||
|
|
||||||
|
Provides per-test ephemeral database isolation atop a single session-scoped
|
||||||
|
container. Tests marked ``@pytest.mark.postgres`` request the ``postgres_url``
|
||||||
|
fixture, which yields an asyncpg connection URL pointing at a freshly-created
|
||||||
|
database. The database is force-dropped after the test (any leaked connections
|
||||||
|
get pg_terminate_backend'd first).
|
||||||
|
|
||||||
|
Why per-database rather than per-schema:
|
||||||
|
asyncpg (the SQLAlchemy async driver we use) doesn't honor URL-embedded
|
||||||
|
search_path the way psycopg does. Per-database isolation is one extra
|
||||||
|
CREATE/DROP per test (~50ms), but lets test app code use its full schema
|
||||||
|
unchanged.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import secrets
|
||||||
|
from collections.abc import Iterator
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="session")
|
||||||
|
def postgres_container():
|
||||||
|
"""Session-scoped Postgres 16 container shared across all postgres tests.
|
||||||
|
|
||||||
|
Started once per pytest session. Subsequent tests piggy-back on the same
|
||||||
|
container; each gets its own database via the ``postgres_url`` fixture.
|
||||||
|
|
||||||
|
Skipped (and the test marked skip) if Docker is unavailable on the host —
|
||||||
|
testcontainers raises ``DockerException`` when it can't reach the daemon.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from testcontainers.postgres import PostgresContainer
|
||||||
|
except ImportError as exc: # pragma: no cover - install boundary
|
||||||
|
pytest.skip(f"testcontainers[postgres] not installed: {exc}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Image tag aligned with the production Aliyun RDS (PostgreSQL 17.9
|
||||||
|
# confirmed by `make doctor` 2026-05-11). Bump together with RDS upgrades.
|
||||||
|
with PostgresContainer("postgres:17-alpine") as pg:
|
||||||
|
yield pg
|
||||||
|
except Exception as exc: # pragma: no cover - environment-dependent
|
||||||
|
# DockerException, ConnectionError, etc. — surface a skip rather than
|
||||||
|
# an error so devs without Docker can still run the rest of the suite.
|
||||||
|
pytest.skip(f"could not start Postgres container ({exc})")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def postgres_url(postgres_container) -> Iterator[str]:
|
||||||
|
"""Per-test ephemeral database URL (asyncpg dialect).
|
||||||
|
|
||||||
|
Each invocation creates a unique database on the shared container and
|
||||||
|
yields its URL. Teardown force-drops the database, terminating any
|
||||||
|
backend connections the test forgot to close.
|
||||||
|
"""
|
||||||
|
import psycopg
|
||||||
|
from psycopg import sql
|
||||||
|
|
||||||
|
db_name = f"test_{secrets.token_hex(8)}"
|
||||||
|
raw = postgres_container.get_connection_url() # postgresql+psycopg2://...
|
||||||
|
# Strip the SQLAlchemy dialect prefix so plain psycopg can connect.
|
||||||
|
base = raw.replace("postgresql+psycopg2://", "postgresql://")
|
||||||
|
parent_url = base.rsplit("/", 1)[0] + "/postgres"
|
||||||
|
|
||||||
|
# CREATE DATABASE must run outside a transaction; psycopg autocommit=True.
|
||||||
|
with psycopg.connect(parent_url, autocommit=True) as conn:
|
||||||
|
conn.execute(sql.SQL("CREATE DATABASE {}").format(sql.Identifier(db_name)))
|
||||||
|
|
||||||
|
asyncpg_url = raw.replace("postgresql+psycopg2://", "postgresql+asyncpg://").rsplit("/", 1)[0] + f"/{db_name}"
|
||||||
|
|
||||||
|
try:
|
||||||
|
yield asyncpg_url
|
||||||
|
finally:
|
||||||
|
with psycopg.connect(parent_url, autocommit=True) as conn:
|
||||||
|
# Kick any leaked connections so DROP DATABASE doesn't block.
|
||||||
|
conn.execute(
|
||||||
|
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = %s AND pid <> pg_backend_pid()",
|
||||||
|
(db_name,),
|
||||||
|
)
|
||||||
|
conn.execute(sql.SQL("DROP DATABASE IF EXISTS {}").format(sql.Identifier(db_name)))
|
||||||
@@ -0,0 +1,357 @@
|
|||||||
|
"""Alembic 0002 / 0003 round-trips on both Postgres and SQLite.
|
||||||
|
|
||||||
|
PR5 has two migrations:
|
||||||
|
|
||||||
|
* ``0002_business_tables_workspace`` — adds *nullable* ``workspace_id``
|
||||||
|
+ FK to ``workspaces`` on ``threads_meta`` / ``runs`` / ``feedback`` /
|
||||||
|
``run_events``, plus a composite index on threads_meta for the common
|
||||||
|
"list a workspace's threads for a user, newest first" query.
|
||||||
|
* ``0003_business_tables_workspace_not_null`` — flips the column to
|
||||||
|
``NOT NULL`` (refusing to upgrade if NULL rows remain) and adds the
|
||||||
|
UNIQUE (workspace_id, thread_id) index on ``threads_meta``.
|
||||||
|
|
||||||
|
Pattern mirrors ``test_alembic_default_workspace_id.py``: synchronous
|
||||||
|
test bodies (alembic's command layer is sync, ``env.py`` calls
|
||||||
|
``asyncio.run`` internally — running under pytest-anyio would explode
|
||||||
|
that nested event loop). The pre-migration schema is bootstrapped with
|
||||||
|
just the columns those migrations touch, so we don't depend on the
|
||||||
|
production ORM staying frozen.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from alembic import command
|
||||||
|
from alembic.config import Config
|
||||||
|
from sqlalchemy import create_engine, inspect, text
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
_ALEMBIC_INI = Path(__file__).resolve().parents[1] / "packages" / "harness" / "deerflow" / "persistence" / "migrations" / "alembic.ini"
|
||||||
|
|
||||||
|
# Tables the PR5 migrations touch.
|
||||||
|
_BUSINESS_TABLES = ("threads_meta", "runs", "feedback", "run_events")
|
||||||
|
|
||||||
|
|
||||||
|
def _make_alembic_config(url: str) -> Config:
|
||||||
|
cfg = Config(str(_ALEMBIC_INI))
|
||||||
|
cfg.set_main_option("sqlalchemy.url", url)
|
||||||
|
return cfg
|
||||||
|
|
||||||
|
|
||||||
|
def _bootstrap_pre_pr5_schema(sync_url: str) -> None:
|
||||||
|
"""Create the post-PR4 schema PR5 migrations expect.
|
||||||
|
|
||||||
|
Includes ``workspaces`` (FK target), ``users`` (already has
|
||||||
|
``default_workspace_id`` from 0001 — but we skip 0001 here and bootstrap
|
||||||
|
the columns directly so the migration's behaviour can be tested in
|
||||||
|
isolation), and the four business tables.
|
||||||
|
"""
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE workspaces (
|
||||||
|
id VARCHAR(36) PRIMARY KEY,
|
||||||
|
name VARCHAR(64) NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE users (
|
||||||
|
id VARCHAR(36) PRIMARY KEY,
|
||||||
|
email VARCHAR(255) NOT NULL,
|
||||||
|
default_workspace_id VARCHAR(36)
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE threads_meta (
|
||||||
|
thread_id VARCHAR(64) PRIMARY KEY,
|
||||||
|
user_id VARCHAR(64),
|
||||||
|
updated_at TIMESTAMP
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE runs (
|
||||||
|
run_id VARCHAR(64) PRIMARY KEY,
|
||||||
|
thread_id VARCHAR(64) NOT NULL,
|
||||||
|
user_id VARCHAR(64)
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE feedback (
|
||||||
|
feedback_id VARCHAR(64) PRIMARY KEY,
|
||||||
|
thread_id VARCHAR(64) NOT NULL,
|
||||||
|
run_id VARCHAR(64) NOT NULL,
|
||||||
|
user_id VARCHAR(64),
|
||||||
|
rating INTEGER NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE run_events (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
thread_id VARCHAR(64) NOT NULL,
|
||||||
|
run_id VARCHAR(64) NOT NULL,
|
||||||
|
user_id VARCHAR(64),
|
||||||
|
event_type VARCHAR(32) NOT NULL,
|
||||||
|
category VARCHAR(16) NOT NULL,
|
||||||
|
content TEXT,
|
||||||
|
seq INTEGER NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
# Pretend 0001 has already run so 0002 is the next revision applied.
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE alembic_version (version_num VARCHAR(32) NOT NULL PRIMARY KEY)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conn.execute(text("INSERT INTO alembic_version (version_num) VALUES ('0001_users_default_workspace')"))
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_workspace_id_present(sync_url: str, *, nullable: bool) -> None:
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
insp = inspect(engine)
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
cols = {c["name"]: c for c in insp.get_columns(table)}
|
||||||
|
assert "workspace_id" in cols, f"{table}: workspace_id missing; got {list(cols)}"
|
||||||
|
col = cols["workspace_id"]
|
||||||
|
assert col["nullable"] is nullable, f"{table}.workspace_id nullable expected {nullable}, got {col['nullable']}"
|
||||||
|
fks = insp.get_foreign_keys(table)
|
||||||
|
ws_fks = [fk for fk in fks if fk.get("referred_table") == "workspaces" and fk.get("constrained_columns") == ["workspace_id"]]
|
||||||
|
assert ws_fks, f"{table}: FK to workspaces missing; got {fks}"
|
||||||
|
idxs = {i["name"] for i in insp.get_indexes("threads_meta")}
|
||||||
|
assert "idx_threads_meta_workspace_user_updated" in idxs, f"threads_meta composite index missing; got {idxs}"
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_workspace_id_absent(sync_url: str) -> None:
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
insp = inspect(engine)
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
cols = {c["name"] for c in insp.get_columns(table)}
|
||||||
|
assert "workspace_id" not in cols, f"{table}: workspace_id still present; got {cols}"
|
||||||
|
idxs = {i["name"] for i in insp.get_indexes("threads_meta")}
|
||||||
|
assert "idx_threads_meta_workspace_user_updated" not in idxs, f"threads_meta composite index still present; got {idxs}"
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- SQLite tests ----------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_sqlite_upgrade_0002_adds_workspace_id_column() -> None:
|
||||||
|
"""0002 adds nullable workspace_id + FK + composite index on SQLite."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
db_path = Path(tmp) / "test.db"
|
||||||
|
sync_url = f"sqlite:///{db_path}"
|
||||||
|
async_url = f"sqlite+aiosqlite:///{db_path}"
|
||||||
|
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
cfg = _make_alembic_config(async_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
_assert_workspace_id_present(sync_url, nullable=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sqlite_downgrade_0002_removes_workspace_id_column() -> None:
|
||||||
|
"""0002 downgrade drops the column + FK + composite index cleanly."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
db_path = Path(tmp) / "test.db"
|
||||||
|
sync_url = f"sqlite:///{db_path}"
|
||||||
|
async_url = f"sqlite+aiosqlite:///{db_path}"
|
||||||
|
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
cfg = _make_alembic_config(async_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
_assert_workspace_id_present(sync_url, nullable=True)
|
||||||
|
command.downgrade(cfg, "-1")
|
||||||
|
_assert_workspace_id_absent(sync_url)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- Postgres tests --------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.postgres
|
||||||
|
def test_postgres_upgrade_0002_adds_workspace_id_column(postgres_url: str) -> None:
|
||||||
|
"""Postgres: 0002 adds nullable workspace_id + FK + composite index."""
|
||||||
|
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(postgres_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
_assert_workspace_id_present(sync_url, nullable=True)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.postgres
|
||||||
|
def test_postgres_downgrade_0002_removes_workspace_id_column(postgres_url: str) -> None:
|
||||||
|
"""Postgres: 0002 downgrade drops column + FK + index cleanly."""
|
||||||
|
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(postgres_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
_assert_workspace_id_present(sync_url, nullable=True)
|
||||||
|
command.downgrade(cfg, "-1")
|
||||||
|
_assert_workspace_id_absent(sync_url)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 0003 helpers ----------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _populate_business_rows_for_0003(sync_url: str, workspace_id: str = "w1", thread_id: str = "t1", run_id: str = "r1") -> None:
|
||||||
|
"""Insert one row per business table with workspace_id populated.
|
||||||
|
|
||||||
|
Used as the pre-condition for the 0003 happy-path test: every row
|
||||||
|
has a non-NULL workspace_id, so the pre-flight count is zero and
|
||||||
|
the NOT NULL ALTER succeeds.
|
||||||
|
"""
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(text("INSERT INTO workspaces (id, name) VALUES (:id, :name)"), {"id": workspace_id, "name": "W"})
|
||||||
|
conn.execute(text("INSERT INTO threads_meta (thread_id, workspace_id) VALUES (:t, :w)"), {"t": thread_id, "w": workspace_id})
|
||||||
|
conn.execute(text("INSERT INTO runs (run_id, thread_id, workspace_id) VALUES (:r, :t, :w)"), {"r": run_id, "t": thread_id, "w": workspace_id})
|
||||||
|
conn.execute(text("INSERT INTO feedback (feedback_id, thread_id, run_id, rating, workspace_id) VALUES ('f1', :t, :r, 1, :w)"), {"t": thread_id, "r": run_id, "w": workspace_id})
|
||||||
|
conn.execute(text("INSERT INTO run_events (thread_id, run_id, event_type, category, seq, workspace_id) VALUES (:t, :r, 'x', 'lifecycle', 1, :w)"), {"t": thread_id, "r": run_id, "w": workspace_id})
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_workspace_id_not_null_and_unique_index(sync_url: str) -> None:
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
insp = inspect(engine)
|
||||||
|
for table in _BUSINESS_TABLES:
|
||||||
|
cols = {c["name"]: c for c in insp.get_columns(table)}
|
||||||
|
assert cols["workspace_id"]["nullable"] is False, f"{table}.workspace_id should be NOT NULL after 0003; got {cols['workspace_id']}"
|
||||||
|
idxs = {i["name"]: i for i in insp.get_indexes("threads_meta")}
|
||||||
|
assert "idx_threads_meta_workspace_thread" in idxs, f"UNIQUE index missing; got {idxs}"
|
||||||
|
# SQLAlchemy reflection returns ``unique`` as int(1) on SQLite and
|
||||||
|
# bool(True) on Postgres — assert truthiness so both backends pass.
|
||||||
|
assert idxs["idx_threads_meta_workspace_thread"]["unique"], f"index should be UNIQUE; got {idxs['idx_threads_meta_workspace_thread']}"
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 0003 SQLite tests -----------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_sqlite_upgrade_0003_succeeds_with_populated_workspace_id() -> None:
|
||||||
|
"""0003 NOT NULL ALTER + UNIQUE index lands when no NULL rows remain."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
db_path = Path(tmp) / "test.db"
|
||||||
|
sync_url = f"sqlite:///{db_path}"
|
||||||
|
async_url = f"sqlite+aiosqlite:///{db_path}"
|
||||||
|
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
cfg = _make_alembic_config(async_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
_populate_business_rows_for_0003(sync_url)
|
||||||
|
|
||||||
|
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
|
||||||
|
_assert_workspace_id_not_null_and_unique_index(sync_url)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sqlite_upgrade_0003_requires_no_null_workspace_id() -> None:
|
||||||
|
"""0003 refuses to upgrade if any business row still has workspace_id=NULL."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
db_path = Path(tmp) / "test.db"
|
||||||
|
sync_url = f"sqlite:///{db_path}"
|
||||||
|
async_url = f"sqlite+aiosqlite:///{db_path}"
|
||||||
|
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
cfg = _make_alembic_config(async_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
|
||||||
|
# Leave one threads_meta row with workspace_id NULL.
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(text("INSERT INTO threads_meta (thread_id) VALUES ('t-orphan')"))
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
with pytest.raises(RuntimeError, match="Cannot ALTER"):
|
||||||
|
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
|
||||||
|
|
||||||
|
|
||||||
|
def test_sqlite_threads_meta_unique_workspace_thread() -> None:
|
||||||
|
"""After 0003, duplicate (workspace_id, thread_id) raises IntegrityError on insert."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
db_path = Path(tmp) / "test.db"
|
||||||
|
sync_url = f"sqlite:///{db_path}"
|
||||||
|
async_url = f"sqlite+aiosqlite:///{db_path}"
|
||||||
|
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
cfg = _make_alembic_config(async_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
_populate_business_rows_for_0003(sync_url)
|
||||||
|
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
|
||||||
|
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
# threads_meta.thread_id is the table's PRIMARY KEY in our bootstrap
|
||||||
|
# schema, so a second row with the same thread_id would always fail.
|
||||||
|
# Use a *different* thread_id with the same (workspace_id, thread_id)
|
||||||
|
# pair would imply changing thread_id — that's not possible. Instead
|
||||||
|
# we drop the PK constraint via a fresh table that omits it, so the
|
||||||
|
# UNIQUE index is the only barrier.
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(text("CREATE TABLE threads_meta_test_unique (id INTEGER PRIMARY KEY, thread_id VARCHAR(64), workspace_id VARCHAR(36))"))
|
||||||
|
conn.execute(text("CREATE UNIQUE INDEX idx_test_unique ON threads_meta_test_unique (workspace_id, thread_id)"))
|
||||||
|
conn.execute(text("INSERT INTO threads_meta_test_unique (thread_id, workspace_id) VALUES ('t', 'w')"))
|
||||||
|
|
||||||
|
with pytest.raises(IntegrityError):
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(text("INSERT INTO threads_meta_test_unique (thread_id, workspace_id) VALUES ('t', 'w')"))
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 0003 Postgres tests ---------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.postgres
|
||||||
|
def test_postgres_upgrade_0003_succeeds_with_populated_workspace_id(postgres_url: str) -> None:
|
||||||
|
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(postgres_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
_populate_business_rows_for_0003(sync_url)
|
||||||
|
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
|
||||||
|
_assert_workspace_id_not_null_and_unique_index(sync_url)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.postgres
|
||||||
|
def test_postgres_upgrade_0003_requires_no_null_workspace_id(postgres_url: str) -> None:
|
||||||
|
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
|
||||||
|
_bootstrap_pre_pr5_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(postgres_url)
|
||||||
|
command.upgrade(cfg, "0002_business_tables_workspace")
|
||||||
|
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(text("INSERT INTO threads_meta (thread_id) VALUES ('t-orphan')"))
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
with pytest.raises(RuntimeError, match="Cannot ALTER"):
|
||||||
|
command.upgrade(cfg, "0003_business_tables_workspace_not_null")
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
"""Alembic revision 0001 round-trips on both Postgres and SQLite.
|
||||||
|
|
||||||
|
Verifies the first DeerFlow migration adds `users.default_workspace_id`
|
||||||
|
(with FK to `workspaces`) on upgrade and removes it on downgrade. Both
|
||||||
|
backends are exercised because the migration relies on
|
||||||
|
`op.batch_alter_table` for SQLite ALTER compatibility — we want to know
|
||||||
|
if either dialect regresses.
|
||||||
|
|
||||||
|
These tests are synchronous: alembic's command layer is sync, and our
|
||||||
|
`env.py` calls `asyncio.run(...)` internally. Running under
|
||||||
|
pytest-anyio would put us inside an event loop and crash that
|
||||||
|
`asyncio.run` call, so we keep the test bodies plain `def`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from alembic import command
|
||||||
|
from alembic.config import Config
|
||||||
|
from sqlalchemy import create_engine, inspect, text
|
||||||
|
|
||||||
|
_ALEMBIC_INI = Path(__file__).resolve().parents[1] / "packages" / "harness" / "deerflow" / "persistence" / "migrations" / "alembic.ini"
|
||||||
|
|
||||||
|
|
||||||
|
def _make_alembic_config(url: str) -> Config:
|
||||||
|
cfg = Config(str(_ALEMBIC_INI))
|
||||||
|
cfg.set_main_option("sqlalchemy.url", url)
|
||||||
|
return cfg
|
||||||
|
|
||||||
|
|
||||||
|
def _bootstrap_pre_pr4_schema(sync_url: str) -> None:
|
||||||
|
"""Create the minimal pre-PR4 schema the migration needs to ALTER.
|
||||||
|
|
||||||
|
Only `users` (without `default_workspace_id`) and `workspaces` (just `id`)
|
||||||
|
are required so the FK target resolves. The rest of the production
|
||||||
|
schema is irrelevant to this migration.
|
||||||
|
"""
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
with engine.begin() as conn:
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE workspaces (
|
||||||
|
id VARCHAR(36) PRIMARY KEY,
|
||||||
|
name VARCHAR(64) NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
text(
|
||||||
|
"""
|
||||||
|
CREATE TABLE users (
|
||||||
|
id VARCHAR(36) PRIMARY KEY,
|
||||||
|
email VARCHAR(255) NOT NULL
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_column_present(sync_url: str) -> None:
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
insp = inspect(engine)
|
||||||
|
cols = {c["name"] for c in insp.get_columns("users")}
|
||||||
|
assert "default_workspace_id" in cols, f"column missing; got {cols}"
|
||||||
|
fks = insp.get_foreign_keys("users")
|
||||||
|
fk_to_ws = [fk for fk in fks if fk.get("referred_table") == "workspaces"]
|
||||||
|
assert fk_to_ws, f"FK to workspaces missing; got {fks}"
|
||||||
|
assert fk_to_ws[0]["constrained_columns"] == ["default_workspace_id"]
|
||||||
|
assert fk_to_ws[0]["referred_columns"] == ["id"]
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_column_absent(sync_url: str) -> None:
|
||||||
|
engine = create_engine(sync_url)
|
||||||
|
insp = inspect(engine)
|
||||||
|
cols = {c["name"] for c in insp.get_columns("users")}
|
||||||
|
assert "default_workspace_id" not in cols, f"column still present; got {cols}"
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- SQLite tests ----------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_sqlite_upgrade_adds_default_workspace_id_with_fk() -> None:
|
||||||
|
"""SQLite: upgrade 0001 adds column + FK; batch_alter_table works."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
db_path = Path(tmp) / "test.db"
|
||||||
|
sync_url = f"sqlite:///{db_path}"
|
||||||
|
async_url = f"sqlite+aiosqlite:///{db_path}"
|
||||||
|
|
||||||
|
_bootstrap_pre_pr4_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(async_url)
|
||||||
|
command.upgrade(cfg, "0001_users_default_workspace")
|
||||||
|
|
||||||
|
_assert_column_present(sync_url)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sqlite_downgrade_removes_default_workspace_id() -> None:
|
||||||
|
"""SQLite: downgrade 0001 removes the column it added."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
db_path = Path(tmp) / "test.db"
|
||||||
|
sync_url = f"sqlite:///{db_path}"
|
||||||
|
async_url = f"sqlite+aiosqlite:///{db_path}"
|
||||||
|
|
||||||
|
_bootstrap_pre_pr4_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(async_url)
|
||||||
|
command.upgrade(cfg, "0001_users_default_workspace")
|
||||||
|
_assert_column_present(sync_url)
|
||||||
|
|
||||||
|
command.downgrade(cfg, "-1")
|
||||||
|
_assert_column_absent(sync_url)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- Postgres tests --------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.postgres
|
||||||
|
def test_postgres_upgrade_adds_default_workspace_id_with_fk(postgres_url: str) -> None:
|
||||||
|
"""Postgres: upgrade 0001 adds column + FK pointing at workspaces(id)."""
|
||||||
|
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
|
||||||
|
_bootstrap_pre_pr4_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(postgres_url)
|
||||||
|
command.upgrade(cfg, "head")
|
||||||
|
|
||||||
|
_assert_column_present(sync_url)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.postgres
|
||||||
|
def test_postgres_downgrade_removes_default_workspace_id(postgres_url: str) -> None:
|
||||||
|
"""Postgres: downgrade 0001 cleanly drops the FK and column."""
|
||||||
|
sync_url = postgres_url.replace("+asyncpg", "+psycopg")
|
||||||
|
_bootstrap_pre_pr4_schema(sync_url)
|
||||||
|
|
||||||
|
cfg = _make_alembic_config(postgres_url)
|
||||||
|
command.upgrade(cfg, "head")
|
||||||
|
_assert_column_present(sync_url)
|
||||||
|
|
||||||
|
command.downgrade(cfg, "-1")
|
||||||
|
_assert_column_absent(sync_url)
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
"""Schema tests for ``ApiKeyRow`` (Stage 0 PR8).
|
||||||
|
|
||||||
|
PR8 is schema-only — no repository class, no API. Tests exercise raw
|
||||||
|
ORM behaviour: column-level UNIQUE on key_prefix, the dual-dialect
|
||||||
|
partial index DDL (sqlite_where + postgresql_where), and CASCADE on
|
||||||
|
service_account delete.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import delete
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from deerflow.persistence.api_key import ApiKeyRow
|
||||||
|
from deerflow.persistence.service_account import ServiceAccountRow
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.anyio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
async def _setup(tmp_path):
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
return get_session_factory()
|
||||||
|
|
||||||
|
|
||||||
|
async def _cleanup():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_parents(sf) -> None:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(UserRow(id="u-alice", email="alice@example.com"))
|
||||||
|
await session.commit()
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(WorkspaceRow(id="w-1", name="Alice WS", slug="alice", owner_id="u-alice"))
|
||||||
|
await session.commit()
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(
|
||||||
|
ServiceAccountRow(
|
||||||
|
id="sa-1",
|
||||||
|
workspace_id="w-1",
|
||||||
|
name="bot",
|
||||||
|
role="member",
|
||||||
|
identity_mode="collapsed",
|
||||||
|
status="active",
|
||||||
|
created_by="u-alice",
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _make_key(*, key_id: str, prefix: str, revoked_at: datetime | None = None) -> ApiKeyRow:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
return ApiKeyRow(
|
||||||
|
id=key_id,
|
||||||
|
service_account_id="sa-1",
|
||||||
|
key_prefix=prefix,
|
||||||
|
key_hash="0" * 64,
|
||||||
|
name="default",
|
||||||
|
scopes="",
|
||||||
|
rate_limit_rpm=None,
|
||||||
|
expires_at=None,
|
||||||
|
last_used_at=None,
|
||||||
|
revoked_at=revoked_at,
|
||||||
|
created_at=now,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.4-1 — column-level UNIQUE on key_prefix
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_unique_key_prefix_enforced(tmp_path):
|
||||||
|
"""Two api_key rows sharing the same key_prefix raise IntegrityError."""
|
||||||
|
sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_parents(sf)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(_make_key(key_id="ak-1", prefix="dfk_live_abc12345"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
with pytest.raises(IntegrityError):
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(_make_key(key_id="ak-2", prefix="dfk_live_abc12345"))
|
||||||
|
await session.commit()
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.4-2 — partial index DDL covers both SQLite and Postgres
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_active_index_declares_both_dialect_where_clauses():
|
||||||
|
"""idx_api_keys_active must compile to a partial index on both drivers.
|
||||||
|
|
||||||
|
The plan locks ``sqlite_where`` *and* ``postgresql_where`` so the same
|
||||||
|
Index emits a partial index regardless of backend (Stage 0 dev still
|
||||||
|
runs SQLite locally; production is Postgres). Both ``dialect_options``
|
||||||
|
entries must be present.
|
||||||
|
"""
|
||||||
|
active_idx = next((idx for idx in ApiKeyRow.__table__.indexes if idx.name == "idx_api_keys_active"), None)
|
||||||
|
assert active_idx is not None, "idx_api_keys_active not declared"
|
||||||
|
sqlite_where = active_idx.dialect_options.get("sqlite", {}).get("where")
|
||||||
|
postgres_where = active_idx.dialect_options.get("postgresql", {}).get("where")
|
||||||
|
assert sqlite_where is not None, "sqlite_where missing on idx_api_keys_active"
|
||||||
|
assert postgres_where is not None, "postgresql_where missing on idx_api_keys_active"
|
||||||
|
assert "revoked_at IS NULL" in str(sqlite_where)
|
||||||
|
assert "revoked_at IS NULL" in str(postgres_where)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.4-3 — CASCADE on service_account delete
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cascade_on_service_account_delete(tmp_path):
|
||||||
|
"""Deleting the parent service_account removes all child api_keys."""
|
||||||
|
sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_parents(sf)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(_make_key(key_id="ak-cascade-1", prefix="dfk_live_cascade1"))
|
||||||
|
session.add(_make_key(key_id="ak-cascade-2", prefix="dfk_live_cascade2"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
await session.execute(delete(ServiceAccountRow).where(ServiceAccountRow.id == "sa-1"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
row1 = await session.get(ApiKeyRow, "ak-cascade-1")
|
||||||
|
row2 = await session.get(ApiKeyRow, "ak-cascade-2")
|
||||||
|
assert row1 is None
|
||||||
|
assert row2 is None
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
@@ -101,7 +101,7 @@ def test_create_and_decode_token():
|
|||||||
import os
|
import os
|
||||||
|
|
||||||
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
||||||
token = create_access_token(user_id)
|
token = create_access_token(user_id, workspace_id="ws-test", role="owner")
|
||||||
assert isinstance(token, str)
|
assert isinstance(token, str)
|
||||||
|
|
||||||
payload = decode_token(token)
|
payload = decode_token(token)
|
||||||
@@ -132,7 +132,7 @@ def test_decode_token_invalid():
|
|||||||
def test_create_token_custom_expiry():
|
def test_create_token_custom_expiry():
|
||||||
"""Custom expiry is respected."""
|
"""Custom expiry is respected."""
|
||||||
user_id = str(uuid4())
|
user_id = str(uuid4())
|
||||||
token = create_access_token(user_id, expires_delta=timedelta(hours=1))
|
token = create_access_token(user_id, expires_delta=timedelta(hours=1), workspace_id="ws-test", role="owner")
|
||||||
payload = decode_token(token)
|
payload = decode_token(token)
|
||||||
assert payload is not None
|
assert payload is not None
|
||||||
assert payload.sub == user_id
|
assert payload.sub == user_id
|
||||||
@@ -420,7 +420,7 @@ def test_jwt_encodes_ver():
|
|||||||
from app.gateway.auth.errors import TokenError
|
from app.gateway.auth.errors import TokenError
|
||||||
|
|
||||||
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
||||||
token = create_access_token(str(uuid4()), token_version=3)
|
token = create_access_token(str(uuid4()), token_version=3, workspace_id="ws-test", role="owner")
|
||||||
payload = decode_token(token)
|
payload = decode_token(token)
|
||||||
assert not isinstance(payload, TokenError)
|
assert not isinstance(payload, TokenError)
|
||||||
assert payload.ver == 3
|
assert payload.ver == 3
|
||||||
@@ -433,7 +433,7 @@ def test_jwt_default_ver_zero():
|
|||||||
from app.gateway.auth.errors import TokenError
|
from app.gateway.auth.errors import TokenError
|
||||||
|
|
||||||
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
||||||
token = create_access_token(str(uuid4()))
|
token = create_access_token(str(uuid4()), workspace_id="ws-test", role="owner")
|
||||||
payload = decode_token(token)
|
payload = decode_token(token)
|
||||||
assert not isinstance(payload, TokenError)
|
assert not isinstance(payload, TokenError)
|
||||||
assert payload.ver == 0
|
assert payload.ver == 0
|
||||||
@@ -447,7 +447,7 @@ def test_token_version_mismatch_rejects():
|
|||||||
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
os.environ["AUTH_JWT_SECRET"] = "test-secret-key-for-jwt-testing-minimum-32-chars"
|
||||||
|
|
||||||
user_id = str(uuid4())
|
user_id = str(uuid4())
|
||||||
token = create_access_token(user_id, token_version=0)
|
token = create_access_token(user_id, token_version=0, workspace_id="ws-test", role="owner")
|
||||||
|
|
||||||
mock_user = User(id=user_id, email="test@example.com", password_hash="hash", token_version=1)
|
mock_user = User(id=user_id, email="test@example.com", password_hash="hash", token_version=1)
|
||||||
|
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ def test_decode_token_returns_token_error_on_malformed():
|
|||||||
|
|
||||||
def test_decode_token_returns_payload_on_valid():
|
def test_decode_token_returns_payload_on_valid():
|
||||||
_setup_config()
|
_setup_config()
|
||||||
token = create_access_token("user-123")
|
token = create_access_token("user-123", workspace_id="ws-test", role="owner")
|
||||||
result = decode_token(token)
|
result = decode_token(token)
|
||||||
assert not isinstance(result, TokenError)
|
assert not isinstance(result, TokenError)
|
||||||
assert result.sub == "user-123"
|
assert result.sub == "user-123"
|
||||||
|
|||||||
@@ -0,0 +1,52 @@
|
|||||||
|
"""JWT carries wid + role claims (Stage 0 PR4).
|
||||||
|
|
||||||
|
Tokens issued after PR4 must include `wid` (workspace_id) and `role`
|
||||||
|
(owner/admin/member) so the AuthMiddleware can resolve the active
|
||||||
|
workspace without a DB hit. Legacy token compatibility lives in
|
||||||
|
:mod:`test_legacy_token_compat`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import jwt
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.gateway.auth import create_access_token, decode_token
|
||||||
|
from app.gateway.auth.config import get_auth_config
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _stable_jwt_secret(monkeypatch):
|
||||||
|
"""Pin a deterministic JWT secret across tests in this module."""
|
||||||
|
monkeypatch.setenv("AUTH_JWT_SECRET", "test-secret-key-for-jwt-testing-minimum-32-chars")
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def test_jwt_includes_wid_and_role() -> None:
|
||||||
|
"""create_access_token records wid + role in the encoded payload."""
|
||||||
|
user_id = str(uuid4())
|
||||||
|
workspace_id = str(uuid4())
|
||||||
|
|
||||||
|
token = create_access_token(user_id, workspace_id=workspace_id, role="owner")
|
||||||
|
|
||||||
|
raw = jwt.decode(token, get_auth_config().jwt_secret, algorithms=["HS256"])
|
||||||
|
assert raw["sub"] == user_id
|
||||||
|
assert raw["wid"] == workspace_id
|
||||||
|
assert raw["role"] == "owner"
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_round_trip_keeps_wid_and_role() -> None:
|
||||||
|
"""decode_token returns a TokenPayload exposing wid + role attributes."""
|
||||||
|
user_id = str(uuid4())
|
||||||
|
workspace_id = str(uuid4())
|
||||||
|
|
||||||
|
token = create_access_token(user_id, workspace_id=workspace_id, role="member")
|
||||||
|
payload = decode_token(token)
|
||||||
|
|
||||||
|
# decode_token returns TokenError on failure — must be the success branch here.
|
||||||
|
assert hasattr(payload, "wid"), f"got {payload!r}"
|
||||||
|
assert payload.sub == user_id
|
||||||
|
assert payload.wid == workspace_id
|
||||||
|
assert payload.role == "member"
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
"""``GET /auth/me`` returns the user's workspace memberships.
|
||||||
|
|
||||||
|
Stage 0 PR4 T4.11. After PR4 the frontend needs to discover which
|
||||||
|
workspaces the current user belongs to (eventually for a picker UI).
|
||||||
|
Each membership entry surfaces ``id``, ``name``, ``slug``, ``role``
|
||||||
|
so the picker can render the list without a second roundtrip.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
os.environ.setdefault("AUTH_JWT_SECRET", "test-secret-key-auth-me-workspaces-32+")
|
||||||
|
|
||||||
|
from app.gateway.auth.config import AuthConfig, set_auth_config
|
||||||
|
|
||||||
|
_TEST_SECRET = "test-secret-key-auth-me-workspaces-32+"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _setup_auth(tmp_path):
|
||||||
|
from app.gateway import deps
|
||||||
|
from app.gateway.routers.auth import _SETUP_STATUS_COOLDOWN
|
||||||
|
from deerflow.persistence.engine import close_engine, init_engine
|
||||||
|
|
||||||
|
set_auth_config(AuthConfig(jwt_secret=_TEST_SECRET))
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path}/auth_me.db"
|
||||||
|
asyncio.run(init_engine("sqlite", url=url, sqlite_dir=str(tmp_path)))
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
_SETUP_STATUS_COOLDOWN.clear()
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
_SETUP_STATUS_COOLDOWN.clear()
|
||||||
|
asyncio.run(close_engine())
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client(_setup_auth):
|
||||||
|
from app.gateway.app import create_app
|
||||||
|
|
||||||
|
set_auth_config(AuthConfig(jwt_secret=_TEST_SECRET))
|
||||||
|
yield TestClient(create_app())
|
||||||
|
|
||||||
|
|
||||||
|
def test_auth_me_returns_workspaces_list(client):
|
||||||
|
"""After registration, /auth/me lists the user's single personal workspace."""
|
||||||
|
client.post("/api/v1/auth/initialize", json={"email": "admin@example.com", "password": "Str0ng!Pass99"})
|
||||||
|
reg = client.post("/api/v1/auth/register", json={"email": "alice@example.com", "password": "Tr0ub4dor3a-strong!"})
|
||||||
|
user_id = reg.json()["id"]
|
||||||
|
|
||||||
|
resp = client.get("/api/v1/auth/me")
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
|
||||||
|
body = resp.json()
|
||||||
|
assert body["id"] == user_id
|
||||||
|
assert body["email"] == "alice@example.com"
|
||||||
|
assert body["default_workspace_id"], "user should land with a default workspace"
|
||||||
|
|
||||||
|
assert len(body["workspaces"]) == 1, body
|
||||||
|
ws = body["workspaces"][0]
|
||||||
|
assert ws["id"] == body["default_workspace_id"]
|
||||||
|
assert ws["slug"] == "alice"
|
||||||
|
assert ws["role"] == "owner"
|
||||||
|
assert ws["name"] # whatever the helper picks — just sanity check it's non-empty
|
||||||
|
|
||||||
|
|
||||||
|
def test_auth_me_workspaces_isolated_per_user(client):
|
||||||
|
"""Two users see only their own workspaces in /auth/me."""
|
||||||
|
client.post("/api/v1/auth/initialize", json={"email": "admin@example.com", "password": "Str0ng!Pass99"})
|
||||||
|
client.post("/api/v1/auth/register", json={"email": "alice@example.com", "password": "Tr0ub4dor3a-strong!"})
|
||||||
|
a_id = client.get("/api/v1/auth/me").json()["id"]
|
||||||
|
a_workspaces = client.get("/api/v1/auth/me").json()["workspaces"]
|
||||||
|
|
||||||
|
client.cookies.clear()
|
||||||
|
client.post("/api/v1/auth/register", json={"email": "bob@example.com", "password": "Tr0ub4dor3a-strong!"})
|
||||||
|
b_id = client.get("/api/v1/auth/me").json()["id"]
|
||||||
|
b_workspaces = client.get("/api/v1/auth/me").json()["workspaces"]
|
||||||
|
|
||||||
|
assert a_id != b_id
|
||||||
|
assert {w["id"] for w in a_workspaces}.isdisjoint({w["id"] for w in b_workspaces})
|
||||||
|
assert {w["slug"] for w in a_workspaces} == {"alice"}
|
||||||
|
assert {w["slug"] for w in b_workspaces} == {"bob"}
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
"""AuthMiddleware injects the workspace ContextVar (Stage 0 PR4 T4.7).
|
||||||
|
|
||||||
|
After PR4 every authenticated request has a workspace bound on
|
||||||
|
``deerflow.runtime.workspace_context._current_workspace``. The middleware
|
||||||
|
populates it from the JWT's ``wid`` / ``role`` claims, mirrors what it
|
||||||
|
already does for ``user_context``, and tears both down in a single
|
||||||
|
``try/finally`` so leaks don't cross requests.
|
||||||
|
|
||||||
|
Legacy 4-field tokens (no ``wid``) are rejected upstream by
|
||||||
|
``decode_token`` (T4.6) — those should never reach the workspace
|
||||||
|
injection branch; this file pins that the 401 they trigger carries
|
||||||
|
``AuthErrorCode.WORKSPACE_REQUIRED``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from unittest.mock import AsyncMock, patch
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import jwt
|
||||||
|
import pytest
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from starlette.testclient import TestClient
|
||||||
|
|
||||||
|
from app.gateway.auth import create_access_token
|
||||||
|
from app.gateway.auth.config import get_auth_config
|
||||||
|
from app.gateway.auth.models import User
|
||||||
|
from app.gateway.auth_middleware import AuthMiddleware
|
||||||
|
from deerflow.runtime.workspace_context import get_current_workspace
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _stable_jwt_secret(monkeypatch):
|
||||||
|
monkeypatch.setenv("AUTH_JWT_SECRET", "test-secret-key-for-jwt-testing-minimum-32-chars")
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def _make_app() -> FastAPI:
|
||||||
|
"""App with AuthMiddleware + an inspect route that surfaces the contextvar."""
|
||||||
|
app = FastAPI()
|
||||||
|
app.add_middleware(AuthMiddleware)
|
||||||
|
|
||||||
|
@app.get("/api/v1/auth/setup-status") # public — never gates on wid
|
||||||
|
async def setup_status():
|
||||||
|
return {"needs_setup": False}
|
||||||
|
|
||||||
|
@app.get("/api/models") # protected — exercises wid injection
|
||||||
|
async def inspect_workspace():
|
||||||
|
ws = get_current_workspace()
|
||||||
|
if ws is None:
|
||||||
|
return {"workspace": None}
|
||||||
|
return {"workspace": {"id": ws.id, "role": ws.role}}
|
||||||
|
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
def _make_user(uid: str) -> User:
|
||||||
|
return User(id=uid, email="t@example.com", password_hash="hash", token_version=0)
|
||||||
|
|
||||||
|
|
||||||
|
def _make_legacy_token() -> str:
|
||||||
|
"""Encode a pre-PR4 JWT (no wid/role) directly."""
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
payload = {
|
||||||
|
"sub": str(uuid4()),
|
||||||
|
"exp": now + timedelta(hours=1),
|
||||||
|
"iat": now,
|
||||||
|
"ver": 0,
|
||||||
|
}
|
||||||
|
return jwt.encode(payload, get_auth_config().jwt_secret, algorithm="HS256")
|
||||||
|
|
||||||
|
|
||||||
|
def test_new_jwt_injects_workspace_into_contextvar() -> None:
|
||||||
|
"""Cookie with wid+role → route observes the workspace via the contextvar."""
|
||||||
|
uid = str(uuid4())
|
||||||
|
token = create_access_token(uid, workspace_id="ws-abc", role="owner")
|
||||||
|
|
||||||
|
with patch("app.gateway.deps.get_local_provider") as fn:
|
||||||
|
fn.return_value.get_user = AsyncMock(return_value=_make_user(uid))
|
||||||
|
client = TestClient(_make_app())
|
||||||
|
res = client.get("/api/models", cookies={"access_token": token})
|
||||||
|
|
||||||
|
assert res.status_code == 200, res.text
|
||||||
|
assert res.json() == {"workspace": {"id": "ws-abc", "role": "owner"}}
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_jwt_rejected_with_workspace_required() -> None:
|
||||||
|
"""No-wid tokens get 401 with AuthErrorCode.WORKSPACE_REQUIRED, not generic token_invalid."""
|
||||||
|
client = TestClient(_make_app())
|
||||||
|
res = client.get("/api/models", cookies={"access_token": _make_legacy_token()})
|
||||||
|
|
||||||
|
assert res.status_code == 401
|
||||||
|
assert res.json()["detail"]["code"] == "workspace_required"
|
||||||
|
|
||||||
|
|
||||||
|
def test_public_path_skips_workspace_check() -> None:
|
||||||
|
"""Public whitelist (e.g. /api/v1/auth/setup-status) does not require wid."""
|
||||||
|
client = TestClient(_make_app())
|
||||||
|
res = client.get("/api/v1/auth/setup-status") # no cookie at all
|
||||||
|
assert res.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_workspace_contextvar_resets_between_requests() -> None:
|
||||||
|
"""After dispatch returns the contextvar must be clear (no leak across requests).
|
||||||
|
|
||||||
|
Why we test this: if the try/finally is wired only for user_context but
|
||||||
|
not workspace_context, two back-to-back requests can see each other's
|
||||||
|
workspace under asyncio task switching.
|
||||||
|
"""
|
||||||
|
uid = str(uuid4())
|
||||||
|
token = create_access_token(uid, workspace_id="ws-first", role="owner")
|
||||||
|
|
||||||
|
# First request resolves to ws-first
|
||||||
|
with patch("app.gateway.deps.get_local_provider") as fn:
|
||||||
|
fn.return_value.get_user = AsyncMock(return_value=_make_user(uid))
|
||||||
|
client = TestClient(_make_app())
|
||||||
|
res1 = client.get("/api/models", cookies={"access_token": token})
|
||||||
|
assert res1.json() == {"workspace": {"id": "ws-first", "role": "owner"}}
|
||||||
|
|
||||||
|
# Outside the request scope the contextvar must be empty again.
|
||||||
|
assert get_current_workspace() is None
|
||||||
|
|
||||||
|
# Second request with a different workspace must not see ws-first.
|
||||||
|
token2 = create_access_token(uid, workspace_id="ws-second", role="owner")
|
||||||
|
with patch("app.gateway.deps.get_local_provider") as fn:
|
||||||
|
fn.return_value.get_user = AsyncMock(return_value=_make_user(uid))
|
||||||
|
client = TestClient(_make_app())
|
||||||
|
res2 = client.get("/api/models", cookies={"access_token": token2})
|
||||||
|
assert res2.json() == {"workspace": {"id": "ws-second", "role": "owner"}}
|
||||||
@@ -0,0 +1,367 @@
|
|||||||
|
"""Tests for ``scripts/backfill_workspace_id.py`` (Stage 0 PR5).
|
||||||
|
|
||||||
|
Each step in the three-step backfill is exercised in isolation against
|
||||||
|
a SQLite-on-disk database. The script wires in ``app.gateway.auth.workspace_slug``,
|
||||||
|
so we get the same slug semantics that the registration flow uses.
|
||||||
|
|
||||||
|
Pattern mirrors :mod:`test_workspace_repo`: ``init_engine`` + per-test
|
||||||
|
tmp_path, with explicit ``close_engine`` teardown so the singleton
|
||||||
|
session factory does not leak across tests.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from deerflow.persistence.feedback.model import FeedbackRow
|
||||||
|
from deerflow.persistence.models.run_event import RunEventRow
|
||||||
|
from deerflow.persistence.run.model import RunRow
|
||||||
|
from deerflow.persistence.thread_meta.model import ThreadMetaRow
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
from scripts.backfill_workspace_id import (
|
||||||
|
LEGACY_WORKSPACE_ID,
|
||||||
|
_ensure_legacy_workspace,
|
||||||
|
_step1_create_workspaces_for_users,
|
||||||
|
_step2_update_table_from_users,
|
||||||
|
_step3_assign_legacy_workspace,
|
||||||
|
backfill,
|
||||||
|
)
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.anyio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
_BUSINESS_ROWS = (ThreadMetaRow, RunRow, FeedbackRow, RunEventRow)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _relax_workspace_id_nullable():
|
||||||
|
"""Simulate alembic 0002 (pre-backfill) state during these tests.
|
||||||
|
|
||||||
|
PR6 T5.11 flipped ``workspace_id`` to ``nullable=False`` on the four
|
||||||
|
business ORM models — production correctness comes from alembic 0003.
|
||||||
|
The backfill script's job is precisely to fill the rows that were
|
||||||
|
inserted between 0002 (column added, nullable) and 0003 (NOT NULL),
|
||||||
|
so tests for it must be able to insert NULL rows. We mutate
|
||||||
|
``column.nullable`` for the four tables before ``create_all`` runs,
|
||||||
|
then restore on teardown so other tests see the production shape.
|
||||||
|
"""
|
||||||
|
saved: list[tuple] = []
|
||||||
|
for model in _BUSINESS_ROWS:
|
||||||
|
col = model.__table__.c.workspace_id
|
||||||
|
saved.append((col, col.nullable))
|
||||||
|
col.nullable = True
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
for col, original in saved:
|
||||||
|
col.nullable = original
|
||||||
|
|
||||||
|
|
||||||
|
async def _init_engine(tmp_path):
|
||||||
|
from sqlalchemy import delete
|
||||||
|
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
sf = get_session_factory()
|
||||||
|
# The PR6 conftest seeds an autouse user + workspace so business-row FKs
|
||||||
|
# resolve in the wider test suite. Backfill tests model "fresh DB needs
|
||||||
|
# backfill" semantics, so wipe those rows here. Order: clear the FK
|
||||||
|
# pointer first, then the rows.
|
||||||
|
async with sf() as session:
|
||||||
|
await session.execute(delete(WorkspaceMembershipRow))
|
||||||
|
await session.execute(delete(WorkspaceRow).where(WorkspaceRow.id == "test-workspace-autouse"))
|
||||||
|
await session.execute(delete(UserRow).where(UserRow.id == "test-user-autouse"))
|
||||||
|
await session.commit()
|
||||||
|
return sf
|
||||||
|
|
||||||
|
|
||||||
|
async def _close():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_user(sf, *, email: str, default_workspace_id: str | None = None, system_role: str = "user") -> str:
|
||||||
|
user_id = str(uuid.uuid4())
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(UserRow(id=user_id, email=email, default_workspace_id=default_workspace_id, system_role=system_role))
|
||||||
|
await session.commit()
|
||||||
|
return user_id
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_business_row(sf, model, **fields) -> None:
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(model(**fields))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Step 1: per-user workspace creation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_creates_workspace_per_user_without_default(tmp_path):
|
||||||
|
"""Each user with NULL default_workspace_id gets a workspace + owner membership."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
u_alice = await _seed_user(sf, email="alice@example.com")
|
||||||
|
u_bob = await _seed_user(sf, email="bob+spam@example.com")
|
||||||
|
|
||||||
|
count = await _step1_create_workspaces_for_users(sf, dry_run=False)
|
||||||
|
assert count == 2
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
workspaces = (await session.execute(select(WorkspaceRow))).scalars().all()
|
||||||
|
memberships = (await session.execute(select(WorkspaceMembershipRow))).scalars().all()
|
||||||
|
users = {u.id: u for u in (await session.execute(select(UserRow))).scalars().all()}
|
||||||
|
|
||||||
|
# 2 workspaces, each with exactly one owner membership matching its user.
|
||||||
|
assert len(workspaces) == 2
|
||||||
|
assert len(memberships) == 2
|
||||||
|
owners_by_ws = {m.workspace_id: m.user_id for m in memberships if m.role == "owner"}
|
||||||
|
assert {m.role for m in memberships} == {"owner"}
|
||||||
|
for ws in workspaces:
|
||||||
|
assert owners_by_ws[ws.id] == ws.owner_id
|
||||||
|
assert users[ws.owner_id].default_workspace_id == ws.id
|
||||||
|
|
||||||
|
# Slug semantics: alice@ → "alice", bob+spam@ → "bob-spam".
|
||||||
|
slugs = {ws.slug for ws in workspaces}
|
||||||
|
assert slugs == {"alice", "bob-spam"}
|
||||||
|
_ = u_alice, u_bob # captured for readability
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_step1_is_idempotent(tmp_path):
|
||||||
|
"""Second run is a no-op when every user already has a default_workspace_id."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, email="carol@example.com")
|
||||||
|
first = await _step1_create_workspaces_for_users(sf, dry_run=False)
|
||||||
|
assert first == 1
|
||||||
|
# Re-running picks up the just-populated default_workspace_id, so the
|
||||||
|
# candidate set is empty.
|
||||||
|
second = await _step1_create_workspaces_for_users(sf, dry_run=False)
|
||||||
|
assert second == 0
|
||||||
|
async with sf() as session:
|
||||||
|
ws_count = len((await session.execute(select(WorkspaceRow))).scalars().all())
|
||||||
|
mem_count = len((await session.execute(select(WorkspaceMembershipRow))).scalars().all())
|
||||||
|
assert ws_count == 1
|
||||||
|
assert mem_count == 1
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_backfill_updates_4_tables_from_users(tmp_path):
|
||||||
|
"""Step 2 propagates each user's default_workspace_id into 4 business tables."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
user_id = await _seed_user(sf, email="dave@example.com")
|
||||||
|
# Pre-seed business rows owned by the user with NULL workspace_id.
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-1", user_id=user_id)
|
||||||
|
await _seed_business_row(sf, RunRow, run_id="r-1", thread_id="t-1", user_id=user_id)
|
||||||
|
await _seed_business_row(sf, FeedbackRow, feedback_id="f-1", thread_id="t-1", run_id="r-1", user_id=user_id, rating=1)
|
||||||
|
await _seed_business_row(sf, RunEventRow, thread_id="t-1", run_id="r-1", user_id=user_id, event_type="lifecycle_started", category="lifecycle", seq=1)
|
||||||
|
|
||||||
|
# Step 1 first so users.default_workspace_id is populated.
|
||||||
|
await _step1_create_workspaces_for_users(sf, dry_run=False)
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
ws_id = (await session.execute(select(WorkspaceRow.id))).scalar_one()
|
||||||
|
|
||||||
|
# Step 2 updates each table.
|
||||||
|
for table in ("threads_meta", "runs", "feedback", "run_events"):
|
||||||
|
count = await _step2_update_table_from_users(sf, table, dry_run=False)
|
||||||
|
assert count == 1, table
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
tm = (await session.execute(select(ThreadMetaRow))).scalar_one()
|
||||||
|
run = (await session.execute(select(RunRow))).scalar_one()
|
||||||
|
fb = (await session.execute(select(FeedbackRow))).scalar_one()
|
||||||
|
ev = (await session.execute(select(RunEventRow))).scalar_one()
|
||||||
|
assert tm.workspace_id == ws_id
|
||||||
|
assert run.workspace_id == ws_id
|
||||||
|
assert fb.workspace_id == ws_id
|
||||||
|
assert ev.workspace_id == ws_id
|
||||||
|
|
||||||
|
# Re-running Step 2 is a no-op (filtered by workspace_id IS NULL).
|
||||||
|
for table in ("threads_meta", "runs", "feedback", "run_events"):
|
||||||
|
assert await _step2_update_table_from_users(sf, table, dry_run=False) == 0
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_backfill_step2_isolates_per_user(tmp_path):
|
||||||
|
"""Two users with different default workspaces get their own threads tagged independently."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
u_eve = await _seed_user(sf, email="eve@example.com")
|
||||||
|
u_frank = await _seed_user(sf, email="frank@example.com")
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-eve", user_id=u_eve)
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-frank", user_id=u_frank)
|
||||||
|
|
||||||
|
await _step1_create_workspaces_for_users(sf, dry_run=False)
|
||||||
|
await _step2_update_table_from_users(sf, "threads_meta", dry_run=False)
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
rows = {r.thread_id: r.workspace_id for r in (await session.execute(select(ThreadMetaRow))).scalars().all()}
|
||||||
|
users = {u.id: u.default_workspace_id for u in (await session.execute(select(UserRow))).scalars().all()}
|
||||||
|
assert rows["t-eve"] == users[u_eve]
|
||||||
|
assert rows["t-frank"] == users[u_frank]
|
||||||
|
assert rows["t-eve"] != rows["t-frank"]
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_step1_skips_blacklisted_base_slug(tmp_path):
|
||||||
|
"""A user with email like admin@... gets bumped past the slug blacklist via the walker."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, email="admin@example.com")
|
||||||
|
await _step1_create_workspaces_for_users(sf, dry_run=False)
|
||||||
|
async with sf() as session:
|
||||||
|
ws = (await session.execute(select(WorkspaceRow))).scalar_one()
|
||||||
|
# The walker treats "admin" as taken (blacklisted), so it falls
|
||||||
|
# through to "admin-2" — the same behaviour the registration flow
|
||||||
|
# uses for reserved slugs.
|
||||||
|
assert ws.slug == "admin-2"
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Step 3: orphan rows -> legacy_workspace
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_backfill_orphan_rows_go_to_legacy_workspace(tmp_path):
|
||||||
|
"""Rows with user_id=NULL get assigned the legacy_workspace UUID after Step 3."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
# Seed a platform admin so the legacy workspace has an owner.
|
||||||
|
await _seed_user(sf, email="admin@example.com", system_role="admin")
|
||||||
|
# Orphan business rows (user_id=NULL): legacy data from before auth.
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-orphan", user_id=None)
|
||||||
|
await _seed_business_row(sf, RunRow, run_id="r-orphan", thread_id="t-orphan", user_id=None)
|
||||||
|
await _seed_business_row(sf, FeedbackRow, feedback_id="f-orphan", thread_id="t-orphan", run_id="r-orphan", user_id=None, rating=1)
|
||||||
|
await _seed_business_row(sf, RunEventRow, thread_id="t-orphan", run_id="r-orphan", user_id=None, event_type="legacy", category="lifecycle", seq=1)
|
||||||
|
|
||||||
|
# Ensure the anchor + reassign per table.
|
||||||
|
created = await _ensure_legacy_workspace(sf, dry_run=False)
|
||||||
|
assert created is True
|
||||||
|
for table in ("threads_meta", "runs", "feedback", "run_events"):
|
||||||
|
count = await _step3_assign_legacy_workspace(sf, table, dry_run=False)
|
||||||
|
assert count == 1, table
|
||||||
|
|
||||||
|
# Re-running the anchor helper is a no-op.
|
||||||
|
assert await _ensure_legacy_workspace(sf, dry_run=False) is False
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
tm = (await session.execute(select(ThreadMetaRow))).scalar_one()
|
||||||
|
run = (await session.execute(select(RunRow))).scalar_one()
|
||||||
|
fb = (await session.execute(select(FeedbackRow))).scalar_one()
|
||||||
|
ev = (await session.execute(select(RunEventRow))).scalar_one()
|
||||||
|
legacy = (await session.execute(select(WorkspaceRow).where(WorkspaceRow.id == LEGACY_WORKSPACE_ID))).scalar_one()
|
||||||
|
legacy_mem = (await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.workspace_id == LEGACY_WORKSPACE_ID))).scalar_one()
|
||||||
|
|
||||||
|
assert tm.workspace_id == LEGACY_WORKSPACE_ID
|
||||||
|
assert run.workspace_id == LEGACY_WORKSPACE_ID
|
||||||
|
assert fb.workspace_id == LEGACY_WORKSPACE_ID
|
||||||
|
assert ev.workspace_id == LEGACY_WORKSPACE_ID
|
||||||
|
assert legacy.slug == "legacy"
|
||||||
|
assert legacy_mem.role == "owner"
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_ensure_legacy_workspace_refuses_when_no_users(tmp_path):
|
||||||
|
"""ensure_legacy_workspace raises a clear error if the DB has no users."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
with pytest.raises(RuntimeError, match="no users exist"):
|
||||||
|
await _ensure_legacy_workspace(sf, dry_run=False)
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_backfill_dry_run_does_not_write(tmp_path):
|
||||||
|
"""``backfill(..., dry_run=True)`` reports counts but writes nothing."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, email="admin@example.com", system_role="admin")
|
||||||
|
user_id = await _seed_user(sf, email="helen@example.com")
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-owned", user_id=user_id)
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-orphan", user_id=None)
|
||||||
|
await _seed_business_row(sf, RunRow, run_id="r-owned", thread_id="t-owned", user_id=user_id)
|
||||||
|
|
||||||
|
# Snapshot row counts BEFORE the dry run so we can confirm
|
||||||
|
# nothing changed AFTER.
|
||||||
|
async with sf() as session:
|
||||||
|
ws_before = len((await session.execute(select(WorkspaceRow))).scalars().all())
|
||||||
|
mem_before = len((await session.execute(select(WorkspaceMembershipRow))).scalars().all())
|
||||||
|
users_with_default_before = len((await session.execute(select(UserRow).where(UserRow.default_workspace_id.is_not(None)))).scalars().all())
|
||||||
|
|
||||||
|
report = await backfill(sf, dry_run=True)
|
||||||
|
assert report["dry_run"] is True
|
||||||
|
# Step 1 reports 2 candidates (admin + helen, both without default).
|
||||||
|
assert report["users_workspaces_created"] == 2
|
||||||
|
# Step 2 reports 0 because Step 1 didn't actually populate
|
||||||
|
# users.default_workspace_id under dry_run — the JOIN comes up empty.
|
||||||
|
assert report["threads_meta_from_users"] == 0
|
||||||
|
assert report["runs_from_users"] == 0
|
||||||
|
# Step 3 reports the 3 NULL business rows (t-owned, t-orphan, r-owned).
|
||||||
|
assert report["legacy_workspace_created"] is True
|
||||||
|
assert report["threads_meta_legacy"] == 2
|
||||||
|
assert report["runs_legacy"] == 1
|
||||||
|
|
||||||
|
# State did not change.
|
||||||
|
async with sf() as session:
|
||||||
|
ws_after = len((await session.execute(select(WorkspaceRow))).scalars().all())
|
||||||
|
mem_after = len((await session.execute(select(WorkspaceMembershipRow))).scalars().all())
|
||||||
|
users_with_default_after = len((await session.execute(select(UserRow).where(UserRow.default_workspace_id.is_not(None)))).scalars().all())
|
||||||
|
rows = (await session.execute(select(ThreadMetaRow.workspace_id))).scalars().all()
|
||||||
|
assert ws_after == ws_before
|
||||||
|
assert mem_after == mem_before
|
||||||
|
assert users_with_default_after == users_with_default_before
|
||||||
|
assert all(w is None for w in rows)
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_full_backfill_orchestrator(tmp_path):
|
||||||
|
"""End-to-end: backfill() runs all three steps and reports per-step counts."""
|
||||||
|
sf = await _init_engine(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, email="admin@example.com", system_role="admin")
|
||||||
|
user_id = await _seed_user(sf, email="gina@example.com")
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-owned", user_id=user_id)
|
||||||
|
await _seed_business_row(sf, ThreadMetaRow, thread_id="t-orphan", user_id=None)
|
||||||
|
|
||||||
|
report = await backfill(sf, dry_run=False)
|
||||||
|
assert report["dry_run"] is False
|
||||||
|
# Two users were missing a default workspace (admin too — we
|
||||||
|
# didn't pre-populate admin's default_workspace_id).
|
||||||
|
assert report["users_workspaces_created"] == 2
|
||||||
|
assert report["threads_meta_from_users"] == 1
|
||||||
|
assert report["legacy_workspace_created"] is True
|
||||||
|
assert report["threads_meta_legacy"] == 1
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
rows = {r.thread_id: r.workspace_id for r in (await session.execute(select(ThreadMetaRow))).scalars().all()}
|
||||||
|
assert rows["t-orphan"] == LEGACY_WORKSPACE_ID
|
||||||
|
assert rows["t-owned"] != LEGACY_WORKSPACE_ID
|
||||||
|
assert rows["t-owned"] is not None
|
||||||
|
finally:
|
||||||
|
await _close()
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
"""``change_password`` and ``login_local`` re-issue JWTs carrying wid + role.
|
||||||
|
|
||||||
|
Stage 0 PR4 T4.12. The contract:
|
||||||
|
|
||||||
|
- After ``POST /auth/change-password`` the new session cookie's JWT must
|
||||||
|
still encode the user's workspace under ``wid`` (and ``role='owner'``),
|
||||||
|
with ``ver`` bumped. Dropping ``wid`` here would lock the user out of
|
||||||
|
every protected endpoint immediately after a password change.
|
||||||
|
- Same for ``POST /auth/login/local`` — it issues a fresh JWT and must
|
||||||
|
also encode ``wid``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
|
||||||
|
import jwt
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
os.environ.setdefault("AUTH_JWT_SECRET", "test-secret-key-change-password-wid-32x")
|
||||||
|
|
||||||
|
from app.gateway.auth.config import AuthConfig, set_auth_config
|
||||||
|
|
||||||
|
_TEST_SECRET = "test-secret-key-change-password-wid-32x"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _setup_auth(tmp_path):
|
||||||
|
from app.gateway import deps
|
||||||
|
from app.gateway.routers.auth import _SETUP_STATUS_COOLDOWN
|
||||||
|
from deerflow.persistence.engine import close_engine, init_engine
|
||||||
|
|
||||||
|
set_auth_config(AuthConfig(jwt_secret=_TEST_SECRET))
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path}/change_pwd.db"
|
||||||
|
asyncio.run(init_engine("sqlite", url=url, sqlite_dir=str(tmp_path)))
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
_SETUP_STATUS_COOLDOWN.clear()
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
_SETUP_STATUS_COOLDOWN.clear()
|
||||||
|
asyncio.run(close_engine())
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client(_setup_auth):
|
||||||
|
from app.gateway.app import create_app
|
||||||
|
|
||||||
|
set_auth_config(AuthConfig(jwt_secret=_TEST_SECRET))
|
||||||
|
yield TestClient(create_app())
|
||||||
|
|
||||||
|
|
||||||
|
def _decode(token: str) -> dict:
|
||||||
|
return jwt.decode(token, _TEST_SECRET, algorithms=["HS256"])
|
||||||
|
|
||||||
|
|
||||||
|
def _bootstrap_user(client) -> tuple[str, dict]:
|
||||||
|
"""Register a user and return (user_id, initial claims)."""
|
||||||
|
client.post("/api/v1/auth/initialize", json={"email": "admin@example.com", "password": "Str0ng!Pass99"})
|
||||||
|
resp = client.post("/api/v1/auth/register", json={"email": "alice@example.com", "password": "Tr0ub4dor3a-strong!"})
|
||||||
|
assert resp.status_code == 201, resp.text
|
||||||
|
return resp.json()["id"], _decode(resp.cookies["access_token"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_change_password_keeps_wid_and_bumps_ver(client):
|
||||||
|
"""change_password re-signs the JWT with wid + role; ver moves forward."""
|
||||||
|
user_id, initial_claims = _bootstrap_user(client)
|
||||||
|
|
||||||
|
# /register set the csrf cookie; the matching header is required on
|
||||||
|
# the change-password POST (Double Submit Cookie pattern).
|
||||||
|
csrf = client.cookies.get("csrf_token")
|
||||||
|
assert csrf, "register must have set csrf_token cookie"
|
||||||
|
|
||||||
|
resp = client.post(
|
||||||
|
"/api/v1/auth/change-password",
|
||||||
|
json={"current_password": "Tr0ub4dor3a-strong!", "new_password": "Tr0ub4dor3a-strong2!"},
|
||||||
|
headers={"X-CSRF-Token": csrf},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
|
||||||
|
new_claims = _decode(resp.cookies["access_token"])
|
||||||
|
assert new_claims["sub"] == user_id
|
||||||
|
assert new_claims["wid"] == initial_claims["wid"], "wid must survive a password change"
|
||||||
|
assert new_claims["role"] == "owner"
|
||||||
|
assert new_claims["ver"] == initial_claims["ver"] + 1, "token_version must advance"
|
||||||
|
|
||||||
|
|
||||||
|
def test_login_issues_wid_carrying_jwt(client):
|
||||||
|
"""/auth/login/local issues a JWT that the workspace middleware will accept."""
|
||||||
|
user_id, _ = _bootstrap_user(client)
|
||||||
|
|
||||||
|
# Clear the session cookie set by /register so the login response is observed in isolation.
|
||||||
|
client.cookies.clear()
|
||||||
|
resp = client.post(
|
||||||
|
"/api/v1/auth/login/local",
|
||||||
|
data={"username": "alice@example.com", "password": "Tr0ub4dor3a-strong!"},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
|
||||||
|
claims = _decode(resp.cookies["access_token"])
|
||||||
|
assert claims["sub"] == user_id
|
||||||
|
assert claims.get("wid"), "login must include wid"
|
||||||
|
assert claims["role"] == "owner"
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
"""Stage 0 PR2 · default backend regression + sqlite-still-works.
|
||||||
|
|
||||||
|
Two facts pinned here:
|
||||||
|
T2.3 — explicit ``database.backend: sqlite`` still produces a working
|
||||||
|
config (backwards-compat for users who deliberately stay on SQLite).
|
||||||
|
T2.4 — ``config.example.yaml`` default ``database.backend`` is ``postgres``
|
||||||
|
(Stage 0 PR2 flipped this from sqlite).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
from deerflow.config.app_config import AppConfig
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
|
||||||
|
|
||||||
|
def _write_extensions_config(path: Path) -> None:
|
||||||
|
path.write_text(json.dumps({"mcpServers": {}, "skills": {}}), encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T2.3 · sqlite backend regression
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_explicit_sqlite_backend_still_works(tmp_path, monkeypatch) -> None:
|
||||||
|
"""A config with explicit ``database.backend: sqlite`` must still parse.
|
||||||
|
|
||||||
|
Pin: PR2 made postgres the example default. This test guarantees that
|
||||||
|
users who copy the SQLite fallback block to their config.yaml do not
|
||||||
|
silently regress.
|
||||||
|
"""
|
||||||
|
config_path = tmp_path / "config.yaml"
|
||||||
|
extensions_path = tmp_path / "extensions_config.json"
|
||||||
|
_write_extensions_config(extensions_path)
|
||||||
|
config_path.write_text(
|
||||||
|
yaml.safe_dump(
|
||||||
|
{
|
||||||
|
"models": [
|
||||||
|
{
|
||||||
|
"name": "test",
|
||||||
|
"use": "langchain_openai:ChatOpenAI",
|
||||||
|
"model": "gpt-4",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"database": {
|
||||||
|
"backend": "sqlite",
|
||||||
|
"sqlite_dir": "/custom/sqlite/path",
|
||||||
|
},
|
||||||
|
"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
monkeypatch.setenv("DEER_FLOW_EXTENSIONS_CONFIG_PATH", str(extensions_path))
|
||||||
|
|
||||||
|
config = AppConfig.from_file(str(config_path))
|
||||||
|
|
||||||
|
assert config.database.backend == "sqlite"
|
||||||
|
assert config.database.sqlite_dir == "/custom/sqlite/path"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T2.4 · default config.example.yaml uses postgres
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_config_example_default_backend_is_postgres(monkeypatch) -> None:
|
||||||
|
"""Pin Stage 0 PR2's commitment: the example config defaults to Postgres.
|
||||||
|
|
||||||
|
Loaded directly from the on-disk ``config.example.yaml`` so any future
|
||||||
|
accidental flip back to sqlite would fail this test.
|
||||||
|
"""
|
||||||
|
example_path = REPO_ROOT / "config.example.yaml"
|
||||||
|
if not example_path.exists():
|
||||||
|
pytest.skip(f"config.example.yaml not found at {example_path}")
|
||||||
|
|
||||||
|
raw = yaml.safe_load(example_path.read_text(encoding="utf-8")) or {}
|
||||||
|
db = raw.get("database") or {}
|
||||||
|
|
||||||
|
assert db.get("backend") == "postgres", f"config.example.yaml database.backend is {db.get('backend')!r}; PR2 requires 'postgres' as the default"
|
||||||
|
# The PG URL must come from the env (referenced as $DATABASE_URL),
|
||||||
|
# never hardcoded with credentials.
|
||||||
|
assert db.get("postgres_url") == "$DATABASE_URL", f"postgres_url should be '$DATABASE_URL' env reference, got {db.get('postgres_url')!r}"
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
"""Lifespan hook backfills missing workspaces for pre-PR4 admins.
|
||||||
|
|
||||||
|
Stage 0 PR4 T4.13. Production scenario: a deployment that pre-dates
|
||||||
|
PR4 has an admin user whose ``users.default_workspace_id`` is NULL.
|
||||||
|
After the upgrade, the first time the app boots, the lifespan hook
|
||||||
|
must create the admin's personal workspace + owner membership so the
|
||||||
|
admin can immediately log in without hitting the post-PR4 workspace
|
||||||
|
gate (T4.7).
|
||||||
|
|
||||||
|
This test directly invokes ``_ensure_admin_user`` against a fixture
|
||||||
|
SQLite DB, simulating the upgrade path.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from app.gateway.auth.config import AuthConfig, set_auth_config
|
||||||
|
|
||||||
|
_TEST_SECRET = "test-secret-key-admin-backfill-32-chars"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _setup(tmp_path):
|
||||||
|
from app.gateway import deps
|
||||||
|
from deerflow.persistence.engine import close_engine, init_engine
|
||||||
|
|
||||||
|
set_auth_config(AuthConfig(jwt_secret=_TEST_SECRET))
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path}/admin_backfill.db"
|
||||||
|
asyncio.run(init_engine("sqlite", url=url, sqlite_dir=str(tmp_path)))
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
asyncio.run(close_engine())
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_pre_pr4_admin(email: str = "admin@example.com") -> str:
|
||||||
|
"""Insert an admin user with default_workspace_id=NULL (pre-PR4 state)."""
|
||||||
|
from app.gateway.deps import get_local_provider
|
||||||
|
|
||||||
|
provider = get_local_provider()
|
||||||
|
user = await provider.create_user(email=email, password="Str0ng!Pass99", system_role="admin")
|
||||||
|
# Belt + suspenders: pretend this user pre-dates PR4 even if the
|
||||||
|
# provider added a workspace_id (it does not today, but explicit
|
||||||
|
# is better).
|
||||||
|
user.default_workspace_id = None
|
||||||
|
await provider.update_user(user)
|
||||||
|
return str(user.id)
|
||||||
|
|
||||||
|
|
||||||
|
async def _read_workspace_state(user_id: str) -> dict:
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from deerflow.persistence.engine import get_session_factory
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
|
||||||
|
sf = get_session_factory()
|
||||||
|
async with sf() as session:
|
||||||
|
user = await session.get(UserRow, user_id)
|
||||||
|
memberships = (await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.user_id == user_id))).scalars().all()
|
||||||
|
workspaces = []
|
||||||
|
if memberships:
|
||||||
|
workspaces = (await session.execute(select(WorkspaceRow).where(WorkspaceRow.id.in_([m.workspace_id for m in memberships])))).scalars().all()
|
||||||
|
return {
|
||||||
|
"default_workspace_id": user.default_workspace_id if user else None,
|
||||||
|
"memberships": [(m.workspace_id, m.role) for m in memberships],
|
||||||
|
"workspaces": [(w.id, w.slug) for w in workspaces],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_ensure_admin_user_creates_missing_workspace():
|
||||||
|
"""Pre-PR4 admin without default_workspace_id → lifespan backfills it."""
|
||||||
|
from app.gateway.app import _ensure_admin_user
|
||||||
|
|
||||||
|
admin_id = asyncio.run(_seed_pre_pr4_admin())
|
||||||
|
before = asyncio.run(_read_workspace_state(admin_id))
|
||||||
|
assert before["default_workspace_id"] is None
|
||||||
|
assert before["workspaces"] == []
|
||||||
|
|
||||||
|
asyncio.run(_ensure_admin_user(FastAPI()))
|
||||||
|
|
||||||
|
after = asyncio.run(_read_workspace_state(admin_id))
|
||||||
|
assert after["default_workspace_id"] is not None, "lifespan should set default_workspace_id"
|
||||||
|
assert len(after["workspaces"]) == 1
|
||||||
|
ws_id, _slug = after["workspaces"][0]
|
||||||
|
assert after["memberships"] == [(ws_id, "owner")]
|
||||||
|
|
||||||
|
|
||||||
|
def test_ensure_admin_user_is_idempotent():
|
||||||
|
"""Running the lifespan hook twice does not create duplicate workspaces."""
|
||||||
|
from app.gateway.app import _ensure_admin_user
|
||||||
|
|
||||||
|
admin_id = asyncio.run(_seed_pre_pr4_admin())
|
||||||
|
asyncio.run(_ensure_admin_user(FastAPI()))
|
||||||
|
state_after_first = asyncio.run(_read_workspace_state(admin_id))
|
||||||
|
|
||||||
|
asyncio.run(_ensure_admin_user(FastAPI()))
|
||||||
|
state_after_second = asyncio.run(_read_workspace_state(admin_id))
|
||||||
|
|
||||||
|
assert state_after_first == state_after_second, "second run must be a no-op"
|
||||||
|
assert len(state_after_second["workspaces"]) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_ensure_admin_user_skips_when_admin_already_has_workspace():
|
||||||
|
"""An admin with a workspace already set should not get a second one."""
|
||||||
|
from app.gateway.app import _ensure_admin_user
|
||||||
|
from app.gateway.deps import get_local_provider
|
||||||
|
|
||||||
|
admin_id = asyncio.run(_seed_pre_pr4_admin())
|
||||||
|
|
||||||
|
async def _set_default(workspace_id: str):
|
||||||
|
provider = get_local_provider()
|
||||||
|
user = await provider.get_user(admin_id)
|
||||||
|
user.default_workspace_id = workspace_id
|
||||||
|
await provider.update_user(user)
|
||||||
|
|
||||||
|
# Run the lifespan hook once to seed a workspace, then re-run.
|
||||||
|
asyncio.run(_ensure_admin_user(FastAPI()))
|
||||||
|
state_seeded = asyncio.run(_read_workspace_state(admin_id))
|
||||||
|
assert len(state_seeded["workspaces"]) == 1
|
||||||
|
seeded_ws_id = state_seeded["workspaces"][0][0]
|
||||||
|
|
||||||
|
asyncio.run(_set_default(seeded_ws_id)) # ensure default is still pointing at it
|
||||||
|
asyncio.run(_ensure_admin_user(FastAPI()))
|
||||||
|
|
||||||
|
final = asyncio.run(_read_workspace_state(admin_id))
|
||||||
|
assert final["default_workspace_id"] == seeded_ws_id
|
||||||
|
assert [w[0] for w in final["workspaces"]] == [seeded_ws_id]
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
"""Schema tests for ``ExternalUserRow`` (Stage 0 PR8).
|
||||||
|
|
||||||
|
PR8 is schema-only — no repository class, no API. Tests exercise raw
|
||||||
|
ORM behaviour: the composite UNIQUE constraint (service_account_id,
|
||||||
|
external_id) and CASCADE on service_account delete.
|
||||||
|
|
||||||
|
An external user is the end-user identity passed through by a
|
||||||
|
service_account whose ``identity_mode`` is ``external_passthrough`` or
|
||||||
|
``both``: every call carries an ``X-External-User-Id`` header which is
|
||||||
|
upserted into this table for audit / quota attribution.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import delete
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from deerflow.persistence.external_user import ExternalUserRow
|
||||||
|
from deerflow.persistence.service_account import ServiceAccountRow
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.anyio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
async def _setup(tmp_path):
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
return get_session_factory()
|
||||||
|
|
||||||
|
|
||||||
|
async def _cleanup():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_parents(sf) -> None:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(UserRow(id="u-alice", email="alice@example.com"))
|
||||||
|
await session.commit()
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(WorkspaceRow(id="w-1", name="Alice WS", slug="alice", owner_id="u-alice"))
|
||||||
|
await session.commit()
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(
|
||||||
|
ServiceAccountRow(
|
||||||
|
id="sa-1",
|
||||||
|
workspace_id="w-1",
|
||||||
|
name="passthrough-bot",
|
||||||
|
role="member",
|
||||||
|
identity_mode="external_passthrough",
|
||||||
|
status="active",
|
||||||
|
created_by="u-alice",
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _make_external_user(*, eu_id: str, external_id: str) -> ExternalUserRow:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
return ExternalUserRow(
|
||||||
|
id=eu_id,
|
||||||
|
workspace_id="w-1",
|
||||||
|
service_account_id="sa-1",
|
||||||
|
external_id=external_id,
|
||||||
|
display_name=None,
|
||||||
|
metadata_json={},
|
||||||
|
created_at=now,
|
||||||
|
last_seen_at=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.5-1 — UNIQUE (service_account_id, external_id)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_unique_service_account_id_plus_external_id(tmp_path):
|
||||||
|
"""The same external_id may be inserted twice only under different SAs."""
|
||||||
|
sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_parents(sf)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(_make_external_user(eu_id="eu-1", external_id="client-42"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
with pytest.raises(IntegrityError):
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(_make_external_user(eu_id="eu-2", external_id="client-42"))
|
||||||
|
await session.commit()
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.5-2 — CASCADE on service_account delete
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cascade_on_service_account_delete(tmp_path):
|
||||||
|
"""Deleting the parent service_account removes all child external_users rows."""
|
||||||
|
sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_parents(sf)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(_make_external_user(eu_id="eu-c1", external_id="endpoint-A"))
|
||||||
|
session.add(_make_external_user(eu_id="eu-c2", external_id="endpoint-B"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
await session.execute(delete(ServiceAccountRow).where(ServiceAccountRow.id == "sa-1"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
row1 = await session.get(ExternalUserRow, "eu-c1")
|
||||||
|
row2 = await session.get(ExternalUserRow, "eu-c2")
|
||||||
|
assert row1 is None
|
||||||
|
assert row2 is None
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
@@ -74,7 +74,7 @@ def test_expired_jwt_raises_401():
|
|||||||
|
|
||||||
|
|
||||||
def test_user_not_found_raises_401():
|
def test_user_not_found_raises_401():
|
||||||
token = create_access_token("ghost")
|
token = create_access_token("ghost", workspace_id="ws-test", role="owner")
|
||||||
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(None)):
|
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(None)):
|
||||||
with pytest.raises(Auth.exceptions.HTTPException) as exc:
|
with pytest.raises(Auth.exceptions.HTTPException) as exc:
|
||||||
asyncio.run(authenticate(_req({"access_token": token})))
|
asyncio.run(authenticate(_req({"access_token": token})))
|
||||||
@@ -84,7 +84,7 @@ def test_user_not_found_raises_401():
|
|||||||
|
|
||||||
def test_token_version_mismatch_raises_401():
|
def test_token_version_mismatch_raises_401():
|
||||||
user = _user(token_version=2)
|
user = _user(token_version=2)
|
||||||
token = create_access_token(str(user.id), token_version=1)
|
token = create_access_token(str(user.id), token_version=1, workspace_id="ws-test", role="owner")
|
||||||
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
||||||
with pytest.raises(Auth.exceptions.HTTPException) as exc:
|
with pytest.raises(Auth.exceptions.HTTPException) as exc:
|
||||||
asyncio.run(authenticate(_req({"access_token": token})))
|
asyncio.run(authenticate(_req({"access_token": token})))
|
||||||
@@ -94,7 +94,7 @@ def test_token_version_mismatch_raises_401():
|
|||||||
|
|
||||||
def test_valid_token_returns_user_id():
|
def test_valid_token_returns_user_id():
|
||||||
user = _user(token_version=0)
|
user = _user(token_version=0)
|
||||||
token = create_access_token(str(user.id), token_version=0)
|
token = create_access_token(str(user.id), token_version=0, workspace_id="ws-test", role="owner")
|
||||||
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
||||||
result = asyncio.run(authenticate(_req({"access_token": token})))
|
result = asyncio.run(authenticate(_req({"access_token": token})))
|
||||||
assert result == str(user.id)
|
assert result == str(user.id)
|
||||||
@@ -102,7 +102,7 @@ def test_valid_token_returns_user_id():
|
|||||||
|
|
||||||
def test_valid_token_matching_version():
|
def test_valid_token_matching_version():
|
||||||
user = _user(token_version=5)
|
user = _user(token_version=5)
|
||||||
token = create_access_token(str(user.id), token_version=5)
|
token = create_access_token(str(user.id), token_version=5, workspace_id="ws-test", role="owner")
|
||||||
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
||||||
result = asyncio.run(authenticate(_req({"access_token": token})))
|
result = asyncio.run(authenticate(_req({"access_token": token})))
|
||||||
assert result == str(user.id)
|
assert result == str(user.id)
|
||||||
@@ -113,7 +113,7 @@ def test_valid_token_matching_version():
|
|||||||
|
|
||||||
def test_provider_exception_propagates():
|
def test_provider_exception_propagates():
|
||||||
"""Provider raises → should not be swallowed silently."""
|
"""Provider raises → should not be swallowed silently."""
|
||||||
token = create_access_token("user-1")
|
token = create_access_token("user-1", workspace_id="ws-test", role="owner")
|
||||||
p = AsyncMock()
|
p = AsyncMock()
|
||||||
p.get_user = AsyncMock(side_effect=RuntimeError("DB down"))
|
p.get_user = AsyncMock(side_effect=RuntimeError("DB down"))
|
||||||
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=p):
|
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=p):
|
||||||
@@ -126,7 +126,11 @@ def test_jwt_missing_ver_defaults_to_zero():
|
|||||||
import jwt as pyjwt
|
import jwt as pyjwt
|
||||||
|
|
||||||
uid = str(uuid4())
|
uid = str(uuid4())
|
||||||
raw = pyjwt.encode({"sub": uid, "exp": 9999999999, "iat": 1000000000}, _JWT_SECRET, algorithm="HS256")
|
raw = pyjwt.encode(
|
||||||
|
{"sub": uid, "wid": "ws-test", "role": "owner", "exp": 9999999999, "iat": 1000000000},
|
||||||
|
_JWT_SECRET,
|
||||||
|
algorithm="HS256",
|
||||||
|
)
|
||||||
user = _user(user_id=uid, token_version=0)
|
user = _user(user_id=uid, token_version=0)
|
||||||
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
||||||
result = asyncio.run(authenticate(_req({"access_token": raw})))
|
result = asyncio.run(authenticate(_req({"access_token": raw})))
|
||||||
@@ -138,7 +142,11 @@ def test_jwt_missing_ver_rejected_when_user_version_nonzero():
|
|||||||
import jwt as pyjwt
|
import jwt as pyjwt
|
||||||
|
|
||||||
uid = str(uuid4())
|
uid = str(uuid4())
|
||||||
raw = pyjwt.encode({"sub": uid, "exp": 9999999999, "iat": 1000000000}, _JWT_SECRET, algorithm="HS256")
|
raw = pyjwt.encode(
|
||||||
|
{"sub": uid, "wid": "ws-test", "role": "owner", "exp": 9999999999, "iat": 1000000000},
|
||||||
|
_JWT_SECRET,
|
||||||
|
algorithm="HS256",
|
||||||
|
)
|
||||||
user = _user(user_id=uid, token_version=1)
|
user = _user(user_id=uid, token_version=1)
|
||||||
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
with patch("app.gateway.langgraph_auth.get_local_provider", return_value=_mock_provider(user)):
|
||||||
with pytest.raises(Auth.exceptions.HTTPException) as exc:
|
with pytest.raises(Auth.exceptions.HTTPException) as exc:
|
||||||
@@ -221,7 +229,7 @@ def test_filter_with_empty_metadata():
|
|||||||
|
|
||||||
|
|
||||||
def test_shared_jwt_secret():
|
def test_shared_jwt_secret():
|
||||||
token = create_access_token("user-1", token_version=3)
|
token = create_access_token("user-1", token_version=3, workspace_id="ws-test", role="owner")
|
||||||
payload = decode_token(token)
|
payload = decode_token(token)
|
||||||
from app.gateway.auth.errors import TokenError
|
from app.gateway.auth.errors import TokenError
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,60 @@
|
|||||||
|
"""Legacy 4-field JWT compatibility (Stage 0 PR4 T4.6).
|
||||||
|
|
||||||
|
Before PR4 every JWT carried only `{sub, exp, iat, ver}`. After PR4 the
|
||||||
|
server expects `wid` (workspace_id) on every protected request. Old
|
||||||
|
cookies in the wild must NOT collapse into ``TokenError.MALFORMED`` —
|
||||||
|
that hides the actual problem (workspace required) and prevents the
|
||||||
|
frontend from steering the user to ``/select-workspace``.
|
||||||
|
|
||||||
|
The contract: ``decode_token`` returns ``TokenError.WORKSPACE_MISSING``
|
||||||
|
specifically when the JWT signature checks out and the payload is
|
||||||
|
otherwise well-formed but does NOT carry a ``wid`` claim.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
|
||||||
|
import jwt
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.gateway.auth.config import get_auth_config
|
||||||
|
from app.gateway.auth.errors import TokenError
|
||||||
|
from app.gateway.auth.jwt import decode_token
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _stable_jwt_secret(monkeypatch):
|
||||||
|
monkeypatch.setenv("AUTH_JWT_SECRET", "test-secret-key-for-jwt-testing-minimum-32-chars")
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def _make_legacy_token(*, expired: bool = False) -> str:
|
||||||
|
"""Encode a pre-PR4 JWT directly (bypassing create_access_token)."""
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
payload = {
|
||||||
|
"sub": "u-legacy",
|
||||||
|
"exp": now + (timedelta(seconds=-1) if expired else timedelta(hours=1)),
|
||||||
|
"iat": now,
|
||||||
|
"ver": 0,
|
||||||
|
}
|
||||||
|
return jwt.encode(payload, get_auth_config().jwt_secret, algorithm="HS256")
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_legacy_token_returns_workspace_missing_error() -> None:
|
||||||
|
"""4-field token (no wid) → TokenError.WORKSPACE_MISSING (not MALFORMED)."""
|
||||||
|
token = _make_legacy_token()
|
||||||
|
result = decode_token(token)
|
||||||
|
assert result == TokenError.WORKSPACE_MISSING
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_legacy_token_with_expired_still_reports_expired() -> None:
|
||||||
|
"""Expired legacy tokens keep reporting EXPIRED — that signal takes priority.
|
||||||
|
|
||||||
|
Why: an expired token must trigger /auth/refresh logic before we
|
||||||
|
decide it also lacks workspace; reporting WORKSPACE_MISSING on an
|
||||||
|
expired token would steer the user to /select-workspace instead.
|
||||||
|
"""
|
||||||
|
token = _make_legacy_token(expired=True)
|
||||||
|
result = decode_token(token)
|
||||||
|
assert result == TokenError.EXPIRED
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
"""PR6 T6.13 — workspace-path migration script tests.
|
||||||
|
|
||||||
|
Builds the legacy ``users/{uid}/...`` tree under a temp base dir,
|
||||||
|
points a populated sqlite ``users`` table at it via the conventional
|
||||||
|
``deer-flow.db`` location, and asserts the migration produces the new
|
||||||
|
``workspaces/{wid}/...`` layout. Covers:
|
||||||
|
|
||||||
|
- threads / memory.json / custom agents all rewritten under the workspace
|
||||||
|
- ``--dry-run`` writes nothing
|
||||||
|
- pre-existing destinations get diverted to ``migration-conflicts/``
|
||||||
|
- users without a ``default_workspace_id`` fall back to the explicit flag
|
||||||
|
- empty legacy dirs are cleaned up
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sqlite3
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
from scripts.migrate_paths_to_workspace import (
|
||||||
|
LEGACY_WORKSPACE_FALLBACK,
|
||||||
|
_load_user_workspaces,
|
||||||
|
migrate,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_legacy_tree(base: Path, *, user_id: str, thread_ids: tuple[str, ...] = (), with_memory: bool = False, agent_names: tuple[str, ...] = ()) -> None:
|
||||||
|
user_root = base / "users" / user_id
|
||||||
|
user_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
for tid in thread_ids:
|
||||||
|
(user_root / "threads" / tid / "user-data" / "workspace").mkdir(parents=True, exist_ok=True)
|
||||||
|
(user_root / "threads" / tid / "user-data" / "workspace" / "marker.txt").write_text(f"{user_id}/{tid}", encoding="utf-8")
|
||||||
|
if with_memory:
|
||||||
|
(user_root / "memory.json").write_text(f'{{"user_id": "{user_id}"}}', encoding="utf-8")
|
||||||
|
for name in agent_names:
|
||||||
|
(user_root / "agents" / name).mkdir(parents=True, exist_ok=True)
|
||||||
|
(user_root / "agents" / name / "SOUL.md").write_text(f"# {name}", encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_db(base: Path, *, users: dict[str, str | None]) -> None:
|
||||||
|
db_path = base / "deer-flow.db"
|
||||||
|
conn = sqlite3.connect(str(db_path))
|
||||||
|
try:
|
||||||
|
conn.execute("CREATE TABLE users (id TEXT PRIMARY KEY, default_workspace_id TEXT)")
|
||||||
|
conn.executemany("INSERT INTO users (id, default_workspace_id) VALUES (?, ?)", list(users.items()))
|
||||||
|
conn.commit()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def base(tmp_path: Path) -> Path:
|
||||||
|
return tmp_path
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_threads_under_workspace(base: Path):
|
||||||
|
_build_legacy_tree(base, user_id="alice", thread_ids=("t1",))
|
||||||
|
_seed_db(base, users={"alice": "ws-alpha"})
|
||||||
|
paths = Paths(base)
|
||||||
|
|
||||||
|
report = migrate(paths, user_workspaces=_load_user_workspaces(paths), fallback_workspace=LEGACY_WORKSPACE_FALLBACK, dry_run=False)
|
||||||
|
|
||||||
|
assert (base / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "workspace" / "marker.txt").read_text(encoding="utf-8") == "alice/t1"
|
||||||
|
assert not (base / "users" / "alice" / "threads").exists()
|
||||||
|
assert {entry["asset"] for entry in report} == {"thread"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_memory_and_agents_nested_under_workspace_and_user(base: Path):
|
||||||
|
_build_legacy_tree(base, user_id="alice", with_memory=True, agent_names=("code-reviewer",))
|
||||||
|
_seed_db(base, users={"alice": "ws-alpha"})
|
||||||
|
paths = Paths(base)
|
||||||
|
|
||||||
|
migrate(paths, user_workspaces=_load_user_workspaces(paths), fallback_workspace=LEGACY_WORKSPACE_FALLBACK, dry_run=False)
|
||||||
|
|
||||||
|
assert (base / "workspaces" / "ws-alpha" / "users" / "alice" / "memory.json").exists()
|
||||||
|
assert (base / "workspaces" / "ws-alpha" / "users" / "alice" / "agents" / "code-reviewer" / "SOUL.md").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_dry_run_writes_nothing(base: Path):
|
||||||
|
_build_legacy_tree(base, user_id="alice", thread_ids=("t1",), with_memory=True, agent_names=("a1",))
|
||||||
|
_seed_db(base, users={"alice": "ws-alpha"})
|
||||||
|
paths = Paths(base)
|
||||||
|
|
||||||
|
report = migrate(paths, user_workspaces=_load_user_workspaces(paths), fallback_workspace=LEGACY_WORKSPACE_FALLBACK, dry_run=True)
|
||||||
|
|
||||||
|
# Source unchanged
|
||||||
|
assert (base / "users" / "alice" / "threads" / "t1" / "user-data" / "workspace" / "marker.txt").exists()
|
||||||
|
assert (base / "users" / "alice" / "memory.json").exists()
|
||||||
|
assert (base / "users" / "alice" / "agents" / "a1" / "SOUL.md").exists()
|
||||||
|
# No destination created
|
||||||
|
assert not (base / "workspaces").exists()
|
||||||
|
# Report still populated so operator sees what *would* happen
|
||||||
|
assert len(report) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_fallback_workspace_used_when_user_has_no_default(base: Path):
|
||||||
|
_build_legacy_tree(base, user_id="alice", thread_ids=("t1",))
|
||||||
|
_seed_db(base, users={"alice": None})
|
||||||
|
paths = Paths(base)
|
||||||
|
|
||||||
|
migrate(paths, user_workspaces=_load_user_workspaces(paths), fallback_workspace="legacy_workspace", dry_run=False)
|
||||||
|
|
||||||
|
assert (base / "workspaces" / "legacy_workspace" / "threads" / "t1" / "user-data" / "workspace" / "marker.txt").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_conflict_routes_legacy_to_migration_conflicts(base: Path):
|
||||||
|
# Pre-create the destination with a different marker so the move sees a conflict.
|
||||||
|
_build_legacy_tree(base, user_id="alice", thread_ids=("t1",))
|
||||||
|
(base / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "workspace").mkdir(parents=True)
|
||||||
|
(base / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "workspace" / "marker.txt").write_text("preexisting", encoding="utf-8")
|
||||||
|
_seed_db(base, users={"alice": "ws-alpha"})
|
||||||
|
paths = Paths(base)
|
||||||
|
|
||||||
|
report = migrate(paths, user_workspaces=_load_user_workspaces(paths), fallback_workspace=LEGACY_WORKSPACE_FALLBACK, dry_run=False)
|
||||||
|
|
||||||
|
assert (base / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "workspace" / "marker.txt").read_text(encoding="utf-8") == "preexisting"
|
||||||
|
conflict_marker = base / "migration-conflicts" / "workspace-migration" / "threads/ws-alpha" / "t1"
|
||||||
|
assert conflict_marker.exists()
|
||||||
|
assert any("conflict" in entry["action"] for entry in report)
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_users_dir_removed_after_full_migration(base: Path):
|
||||||
|
_build_legacy_tree(base, user_id="alice", thread_ids=("t1",))
|
||||||
|
_seed_db(base, users={"alice": "ws-alpha"})
|
||||||
|
paths = Paths(base)
|
||||||
|
|
||||||
|
migrate(paths, user_workspaces=_load_user_workspaces(paths), fallback_workspace=LEGACY_WORKSPACE_FALLBACK, dry_run=False)
|
||||||
|
|
||||||
|
assert not (base / "users").exists(), "Empty legacy users/ dir should be cleaned up"
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_users_directory_is_noop(base: Path):
|
||||||
|
paths = Paths(base)
|
||||||
|
report = migrate(paths, user_workspaces={}, fallback_workspace=LEGACY_WORKSPACE_FALLBACK, dry_run=False)
|
||||||
|
assert report == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_db_returns_empty_mapping(base: Path):
|
||||||
|
paths = Paths(base)
|
||||||
|
assert _load_user_workspaces(paths) == {}
|
||||||
|
|
||||||
|
|
||||||
|
def test_db_without_users_table_returns_empty(base: Path):
|
||||||
|
db_path = base / "deer-flow.db"
|
||||||
|
conn = sqlite3.connect(str(db_path))
|
||||||
|
try:
|
||||||
|
conn.execute("CREATE TABLE other_table (x INTEGER)")
|
||||||
|
conn.commit()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
paths = Paths(base)
|
||||||
|
assert _load_user_workspaces(paths) == {}
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
"""PR6 T6.14 — lifespan warns when legacy users/ tree still has content."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.gateway.app import _check_path_migration_pending
|
||||||
|
|
||||||
|
|
||||||
|
class _FakePaths:
|
||||||
|
def __init__(self, base: Path):
|
||||||
|
self.base_dir = base
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def base(tmp_path: Path) -> Path:
|
||||||
|
return tmp_path
|
||||||
|
|
||||||
|
|
||||||
|
def _patch_paths(base: Path):
|
||||||
|
return patch("deerflow.config.paths.get_paths", return_value=_FakePaths(base))
|
||||||
|
|
||||||
|
|
||||||
|
def test_warns_when_legacy_users_dir_has_content(base: Path, caplog: pytest.LogCaptureFixture):
|
||||||
|
(base / "users" / "alice" / "threads" / "t1").mkdir(parents=True)
|
||||||
|
with _patch_paths(base), caplog.at_level(logging.WARNING, logger="app.gateway.app"):
|
||||||
|
_check_path_migration_pending(app=None) # type: ignore[arg-type]
|
||||||
|
assert any("make migrate-paths" in rec.message for rec in caplog.records)
|
||||||
|
|
||||||
|
|
||||||
|
def test_silent_when_legacy_users_dir_missing(base: Path, caplog: pytest.LogCaptureFixture):
|
||||||
|
with _patch_paths(base), caplog.at_level(logging.WARNING, logger="app.gateway.app"):
|
||||||
|
_check_path_migration_pending(app=None) # type: ignore[arg-type]
|
||||||
|
assert not any("migrate-paths" in rec.message for rec in caplog.records)
|
||||||
|
|
||||||
|
|
||||||
|
def test_silent_when_legacy_users_dir_empty(base: Path, caplog: pytest.LogCaptureFixture):
|
||||||
|
(base / "users").mkdir()
|
||||||
|
with _patch_paths(base), caplog.at_level(logging.WARNING, logger="app.gateway.app"):
|
||||||
|
_check_path_migration_pending(app=None) # type: ignore[arg-type]
|
||||||
|
assert not any("migrate-paths" in rec.message for rec in caplog.records)
|
||||||
@@ -0,0 +1,115 @@
|
|||||||
|
"""PR6 T6.9 / T6.10 — workspace-scoped path resolution.
|
||||||
|
|
||||||
|
The Paths class learns a new top-level dimension for multi-tenant
|
||||||
|
filesystems: ``{base_dir}/workspaces/{wid}/...``. Precedence:
|
||||||
|
|
||||||
|
- ``workspace_id`` given → new shape ``workspaces/{wid}/threads/{tid}/...``
|
||||||
|
- only ``user_id`` given → legacy shape ``users/{uid}/threads/{tid}/...``
|
||||||
|
- neither → very-legacy shape ``threads/{tid}/...``
|
||||||
|
|
||||||
|
Per-user filesystem state (memory.json, custom agents) lives under the
|
||||||
|
workspace too: ``workspaces/{wid}/users/{uid}/memory.json`` etc. — so a
|
||||||
|
user's memory cannot be reused across workspaces by mistake.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def paths(tmp_path: Path) -> Paths:
|
||||||
|
return Paths(tmp_path)
|
||||||
|
|
||||||
|
|
||||||
|
class TestValidateWorkspaceId:
|
||||||
|
def test_valid_workspace_id(self, paths: Paths):
|
||||||
|
d = paths.workspace_dir("ws-abc-123")
|
||||||
|
assert d == paths.base_dir / "workspaces" / "ws-abc-123"
|
||||||
|
|
||||||
|
def test_rejects_path_traversal(self, paths: Paths):
|
||||||
|
with pytest.raises(ValueError, match="Invalid workspace_id"):
|
||||||
|
paths.workspace_dir("../escape")
|
||||||
|
|
||||||
|
def test_rejects_slash(self, paths: Paths):
|
||||||
|
with pytest.raises(ValueError, match="Invalid workspace_id"):
|
||||||
|
paths.workspace_dir("ws/bar")
|
||||||
|
|
||||||
|
def test_rejects_empty(self, paths: Paths):
|
||||||
|
with pytest.raises(ValueError, match="Invalid workspace_id"):
|
||||||
|
paths.workspace_dir("")
|
||||||
|
|
||||||
|
|
||||||
|
class TestWorkspaceScopedThreadDir:
|
||||||
|
def test_workspace_takes_precedence_over_user(self, paths: Paths):
|
||||||
|
"""When both are given, workspace wins — user_id is recorded in the row, not the filesystem."""
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "threads" / "t1"
|
||||||
|
assert paths.thread_dir("t1", workspace_id="ws-alpha", user_id="alice") == expected
|
||||||
|
|
||||||
|
def test_workspace_only(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "threads" / "t1"
|
||||||
|
assert paths.thread_dir("t1", workspace_id="ws-alpha") == expected
|
||||||
|
|
||||||
|
def test_user_only_still_legacy(self, paths: Paths):
|
||||||
|
"""Legacy callers keep the user_id-only shape until migration runs."""
|
||||||
|
expected = paths.base_dir / "users" / "alice" / "threads" / "t1"
|
||||||
|
assert paths.thread_dir("t1", user_id="alice") == expected
|
||||||
|
|
||||||
|
def test_no_ids_very_legacy(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "threads" / "t1"
|
||||||
|
assert paths.thread_dir("t1") == expected
|
||||||
|
|
||||||
|
|
||||||
|
class TestEnsureThreadDirsWorkspace:
|
||||||
|
def test_creates_workspace_layout(self, paths: Paths):
|
||||||
|
paths.ensure_thread_dirs("t1", workspace_id="ws-alpha")
|
||||||
|
root = paths.base_dir / "workspaces" / "ws-alpha" / "threads" / "t1"
|
||||||
|
for sub in ("user-data/workspace", "user-data/uploads", "user-data/outputs", "acp-workspace"):
|
||||||
|
assert (root / sub).is_dir(), f"missing {sub}"
|
||||||
|
|
||||||
|
|
||||||
|
class TestSandboxDirsWorkspace:
|
||||||
|
def test_sandbox_work_dir(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "workspace"
|
||||||
|
assert paths.sandbox_work_dir("t1", workspace_id="ws-alpha") == expected
|
||||||
|
|
||||||
|
def test_sandbox_uploads_dir(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "uploads"
|
||||||
|
assert paths.sandbox_uploads_dir("t1", workspace_id="ws-alpha") == expected
|
||||||
|
|
||||||
|
def test_sandbox_outputs_dir(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "outputs"
|
||||||
|
assert paths.sandbox_outputs_dir("t1", workspace_id="ws-alpha") == expected
|
||||||
|
|
||||||
|
|
||||||
|
class TestUserMemoryUnderWorkspace:
|
||||||
|
def test_user_memory_file_under_workspace(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "users" / "alice" / "memory.json"
|
||||||
|
assert paths.user_memory_file("alice", workspace_id="ws-alpha") == expected
|
||||||
|
|
||||||
|
def test_user_memory_file_legacy_without_workspace(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "users" / "alice" / "memory.json"
|
||||||
|
assert paths.user_memory_file("alice") == expected
|
||||||
|
|
||||||
|
def test_user_agents_dir_under_workspace(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "users" / "alice" / "agents"
|
||||||
|
assert paths.user_agents_dir("alice", workspace_id="ws-alpha") == expected
|
||||||
|
|
||||||
|
def test_user_agent_memory_file_under_workspace(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "users" / "alice" / "agents" / "code-reviewer" / "memory.json"
|
||||||
|
assert paths.user_agent_memory_file("alice", "code-reviewer", workspace_id="ws-alpha") == expected
|
||||||
|
|
||||||
|
|
||||||
|
class TestVirtualPathResolutionWorkspace:
|
||||||
|
def test_resolve_virtual_path_workspace_scope(self, paths: Paths):
|
||||||
|
expected = paths.base_dir / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data" / "outputs" / "x.json"
|
||||||
|
actual = paths.resolve_virtual_path("t1", "/mnt/user-data/outputs/x.json", workspace_id="ws-alpha")
|
||||||
|
assert actual == expected.resolve()
|
||||||
|
|
||||||
|
def test_resolve_virtual_path_rejects_traversal_under_workspace(self, paths: Paths):
|
||||||
|
with pytest.raises(ValueError, match="path traversal"):
|
||||||
|
paths.resolve_virtual_path("t1", "/mnt/user-data/../../etc/passwd", workspace_id="ws-alpha")
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
"""Stage 0 PR1 · Postgres smoke tests via testcontainers.
|
||||||
|
|
||||||
|
These tests exercise the postgres_url fixture and verify that DeerFlow's
|
||||||
|
existing ``init_engine`` + ORM ``Base.metadata.create_all`` works against
|
||||||
|
Postgres exactly the same way it works against SQLite (no Stage 0
|
||||||
|
schema changes here — that's PR3+).
|
||||||
|
|
||||||
|
All tests are gated by ``@pytest.mark.postgres`` and skip cleanly when
|
||||||
|
Docker is unavailable (the postgres_container fixture handles that).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import secrets
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
# Mark every test in this module as `postgres` so they only run when
|
||||||
|
# explicitly requested with ``pytest -m postgres``.
|
||||||
|
pytestmark = [pytest.mark.postgres, pytest.mark.anyio]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
"""anyio uses asyncio backend (matches DeerFlow's runtime)."""
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T1.4 · Fixture self-test — verify per-test isolation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_postgres_url_creates_isolated_database(postgres_url: str) -> None:
|
||||||
|
"""The fixture should yield a usable URL pointing at a unique database."""
|
||||||
|
import asyncpg
|
||||||
|
|
||||||
|
# The URL form is `postgresql+asyncpg://user:pass@host:port/test_<hex>`.
|
||||||
|
assert postgres_url.startswith("postgresql+asyncpg://")
|
||||||
|
assert "/test_" in postgres_url
|
||||||
|
|
||||||
|
# Connect with raw asyncpg (strip SQLAlchemy dialect prefix) and confirm
|
||||||
|
# we landed in the named test DB.
|
||||||
|
raw = postgres_url.replace("postgresql+asyncpg://", "postgresql://")
|
||||||
|
conn = await asyncpg.connect(raw)
|
||||||
|
try:
|
||||||
|
current = await conn.fetchval("SELECT current_database()")
|
||||||
|
assert current.startswith("test_"), f"expected test_*, got {current!r}"
|
||||||
|
finally:
|
||||||
|
await conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_postgres_url_isolates_between_tests(postgres_container, postgres_url: str) -> None:
|
||||||
|
"""Two invocations of the fixture should yield two distinct databases.
|
||||||
|
|
||||||
|
Hand-rolls a second database via the same recipe to prove isolation
|
||||||
|
without depending on pytest's own per-test invocation timing.
|
||||||
|
"""
|
||||||
|
import asyncpg
|
||||||
|
import psycopg
|
||||||
|
from psycopg import sql
|
||||||
|
|
||||||
|
# Read what DB we're in.
|
||||||
|
raw = postgres_url.replace("postgresql+asyncpg://", "postgresql://")
|
||||||
|
conn = await asyncpg.connect(raw)
|
||||||
|
try:
|
||||||
|
db_a = await conn.fetchval("SELECT current_database()")
|
||||||
|
finally:
|
||||||
|
await conn.close()
|
||||||
|
|
||||||
|
# Create a *second* DB on the same container directly.
|
||||||
|
container_url = postgres_container.get_connection_url().replace("postgresql+psycopg2://", "postgresql://")
|
||||||
|
parent_url = container_url.rsplit("/", 1)[0] + "/postgres"
|
||||||
|
db_b_name = f"test_{secrets.token_hex(8)}"
|
||||||
|
with psycopg.connect(parent_url, autocommit=True) as c:
|
||||||
|
c.execute(sql.SQL("CREATE DATABASE {}").format(sql.Identifier(db_b_name)))
|
||||||
|
try:
|
||||||
|
assert db_a != db_b_name, "fixture should not reuse a DB name across tests"
|
||||||
|
finally:
|
||||||
|
with psycopg.connect(parent_url, autocommit=True) as c:
|
||||||
|
c.execute(sql.SQL("DROP DATABASE IF EXISTS {}").format(sql.Identifier(db_b_name)))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T1.5 · init_engine smoke — Base.metadata.create_all() works on Postgres
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_init_engine_postgres_creates_tables(postgres_url: str) -> None:
|
||||||
|
"""``init_engine`` against Postgres must auto-create existing ORM tables.
|
||||||
|
|
||||||
|
Verifies the four current business tables (users, threads_meta, runs,
|
||||||
|
feedback) plus run_events appear in information_schema after init —
|
||||||
|
proving that DeerFlow's ``Base.metadata.create_all()`` path works
|
||||||
|
identically on PG and SQLite.
|
||||||
|
"""
|
||||||
|
from sqlalchemy import text
|
||||||
|
|
||||||
|
from deerflow.persistence.engine import close_engine, get_session_factory, init_engine
|
||||||
|
|
||||||
|
await init_engine("postgres", url=postgres_url)
|
||||||
|
try:
|
||||||
|
sf = get_session_factory()
|
||||||
|
async with sf() as session:
|
||||||
|
result = await session.execute(text("SELECT table_name FROM information_schema.tables WHERE table_schema = 'public'"))
|
||||||
|
tables = {row[0] for row in result.all()}
|
||||||
|
|
||||||
|
# Existing tables before any Stage 0 schema additions:
|
||||||
|
expected = {"users", "threads_meta", "runs", "feedback", "run_events"}
|
||||||
|
missing = expected - tables
|
||||||
|
assert not missing, f"create_all() did not produce {missing}; got {tables}"
|
||||||
|
finally:
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T1.6 · Repository round-trip on Postgres
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_thread_meta_repo_postgres_round_trip(postgres_url: str) -> None:
|
||||||
|
"""ThreadMetaRepository.create + get must behave identically on PG.
|
||||||
|
|
||||||
|
Pin: this is a regression net for any SQLAlchemy / asyncpg surprise
|
||||||
|
that diverges from SQLite behavior in dict shape, default values,
|
||||||
|
or timestamp precision.
|
||||||
|
"""
|
||||||
|
from deerflow.persistence.engine import close_engine, get_session_factory, init_engine
|
||||||
|
from deerflow.persistence.thread_meta import ThreadMetaRepository
|
||||||
|
|
||||||
|
await init_engine("postgres", url=postgres_url)
|
||||||
|
try:
|
||||||
|
repo = ThreadMetaRepository(get_session_factory())
|
||||||
|
# Note: conftest's `_auto_user_context` autouse fixture has already
|
||||||
|
# injected `id="test-user-autouse"` so create() picks it via AUTO.
|
||||||
|
created = await repo.create(
|
||||||
|
thread_id="thread-1",
|
||||||
|
display_name="hello",
|
||||||
|
metadata={"k": "v"},
|
||||||
|
)
|
||||||
|
assert created["thread_id"] == "thread-1"
|
||||||
|
assert created["user_id"] == "test-user-autouse"
|
||||||
|
assert created["display_name"] == "hello"
|
||||||
|
assert created["metadata"] == {"k": "v"}
|
||||||
|
|
||||||
|
fetched = await repo.get("thread-1")
|
||||||
|
assert fetched is not None
|
||||||
|
assert fetched["thread_id"] == "thread-1"
|
||||||
|
assert fetched["user_id"] == "test-user-autouse"
|
||||||
|
finally:
|
||||||
|
await close_engine()
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
"""Acceptance test for PR8: ``Base.metadata.create_all()`` automatically
|
||||||
|
provisions ``service_accounts`` / ``api_keys`` / ``external_users``.
|
||||||
|
|
||||||
|
The harness layer registers all ORM models through ``deerflow.persistence.models``
|
||||||
|
(imported for side effects from ``engine.init_engine``). This test guards
|
||||||
|
against a row class being defined but accidentally left out of the
|
||||||
|
registration entry point — a class table that never gets created at
|
||||||
|
``init_engine`` time would otherwise silently break Stage 1 once the
|
||||||
|
API-key auth layer starts inserting rows.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import inspect
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.anyio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_pr8_tables_present_after_init_engine(tmp_path):
|
||||||
|
from deerflow.persistence.engine import close_engine, get_engine, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
try:
|
||||||
|
engine = get_engine()
|
||||||
|
assert engine is not None
|
||||||
|
|
||||||
|
def _table_names(sync_conn):
|
||||||
|
return set(inspect(sync_conn).get_table_names())
|
||||||
|
|
||||||
|
async with engine.connect() as conn:
|
||||||
|
tables = await conn.run_sync(_table_names)
|
||||||
|
|
||||||
|
assert {"service_accounts", "api_keys", "external_users"}.issubset(tables), f"PR8 tables missing from create_all: have {sorted(tables)}"
|
||||||
|
finally:
|
||||||
|
await close_engine()
|
||||||
@@ -0,0 +1,172 @@
|
|||||||
|
"""Registration endpoints (POST /initialize, POST /register) auto-create a workspace.
|
||||||
|
|
||||||
|
Stage 0 PR4 T4.8 + T4.9. Every newly registered user must end up with:
|
||||||
|
|
||||||
|
- a single ``workspaces`` row (their personal workspace),
|
||||||
|
- a single ``workspace_memberships`` row with ``role='owner'``,
|
||||||
|
- ``users.default_workspace_id`` pointing at that workspace,
|
||||||
|
- a session cookie whose JWT carries the workspace as the ``wid`` claim.
|
||||||
|
|
||||||
|
Tests run against a per-test SQLite engine bootstrapped by the
|
||||||
|
fixture; the registration router is exercised through the real
|
||||||
|
TestClient so the full handler + DB transaction path is covered.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
|
||||||
|
import jwt
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
os.environ.setdefault("AUTH_JWT_SECRET", "test-secret-key-register-workspace-32+")
|
||||||
|
|
||||||
|
from app.gateway.auth.config import AuthConfig, set_auth_config
|
||||||
|
|
||||||
|
_TEST_SECRET = "test-secret-key-register-workspace-32+"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _setup_auth(tmp_path):
|
||||||
|
from app.gateway import deps
|
||||||
|
from app.gateway.routers.auth import _SETUP_STATUS_COOLDOWN
|
||||||
|
from deerflow.persistence.engine import close_engine, init_engine
|
||||||
|
|
||||||
|
set_auth_config(AuthConfig(jwt_secret=_TEST_SECRET))
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path}/register_ws.db"
|
||||||
|
asyncio.run(init_engine("sqlite", url=url, sqlite_dir=str(tmp_path)))
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
_SETUP_STATUS_COOLDOWN.clear()
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
deps._cached_local_provider = None
|
||||||
|
deps._cached_repo = None
|
||||||
|
_SETUP_STATUS_COOLDOWN.clear()
|
||||||
|
asyncio.run(close_engine())
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client(_setup_auth):
|
||||||
|
from app.gateway.app import create_app
|
||||||
|
|
||||||
|
set_auth_config(AuthConfig(jwt_secret=_TEST_SECRET))
|
||||||
|
app = create_app()
|
||||||
|
yield TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def _init_payload(**extra):
|
||||||
|
return {"email": "admin@example.com", "password": "Str0ng!Pass99", **extra}
|
||||||
|
|
||||||
|
|
||||||
|
def _register_payload(email: str = "alice@example.com", **extra):
|
||||||
|
return {"email": email, "password": "Tr0ub4dor3a-strong!", **extra}
|
||||||
|
|
||||||
|
|
||||||
|
def _decode(token: str) -> dict:
|
||||||
|
"""Decode a JWT (signature-checked) and return the raw payload."""
|
||||||
|
return jwt.decode(token, _TEST_SECRET, algorithms=["HS256"])
|
||||||
|
|
||||||
|
|
||||||
|
async def _read_workspace_state(user_id: str) -> dict:
|
||||||
|
"""Inspect the per-user workspace state after a registration call.
|
||||||
|
|
||||||
|
Returns a dict with the workspace row, the owner membership row and
|
||||||
|
the user's default_workspace_id, so each test can pick what it
|
||||||
|
cares about.
|
||||||
|
"""
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from deerflow.persistence.engine import get_session_factory
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
|
||||||
|
sf = get_session_factory()
|
||||||
|
async with sf() as session:
|
||||||
|
user = await session.get(UserRow, user_id)
|
||||||
|
memberships = (await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.user_id == user_id))).scalars().all()
|
||||||
|
workspaces = []
|
||||||
|
if memberships:
|
||||||
|
workspaces = (await session.execute(select(WorkspaceRow).where(WorkspaceRow.id.in_([m.workspace_id for m in memberships])))).scalars().all()
|
||||||
|
return {
|
||||||
|
"user_default_workspace_id": getattr(user, "default_workspace_id", None) if user else None,
|
||||||
|
"memberships": [(m.workspace_id, m.user_id, m.role) for m in memberships],
|
||||||
|
"workspaces": [(w.id, w.slug, w.owner_id) for w in workspaces],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- T4.8 — /initialize ---------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_initialize_creates_admin_with_default_workspace(client):
|
||||||
|
"""POST /initialize → admin + workspace + owner membership + wid cookie."""
|
||||||
|
resp = client.post("/api/v1/auth/initialize", json=_init_payload())
|
||||||
|
assert resp.status_code == 201, resp.text
|
||||||
|
user_id = resp.json()["id"]
|
||||||
|
|
||||||
|
state = asyncio.run(_read_workspace_state(user_id))
|
||||||
|
|
||||||
|
assert len(state["workspaces"]) == 1, state
|
||||||
|
ws_id, ws_slug, ws_owner = state["workspaces"][0]
|
||||||
|
assert ws_owner == user_id
|
||||||
|
# base slug "admin" is reserved, walker bumps to first free suffix
|
||||||
|
assert ws_slug == "admin-2"
|
||||||
|
|
||||||
|
assert state["memberships"] == [(ws_id, user_id, "owner")]
|
||||||
|
assert state["user_default_workspace_id"] == ws_id
|
||||||
|
|
||||||
|
token = resp.cookies["access_token"]
|
||||||
|
claims = _decode(token)
|
||||||
|
assert claims["wid"] == ws_id
|
||||||
|
assert claims["role"] == "owner"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- T4.9 — /register -----------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_register_creates_user_with_default_workspace(client):
|
||||||
|
"""POST /register → user + workspace + owner membership + wid cookie."""
|
||||||
|
# Initialize an admin first so the system is past first-boot.
|
||||||
|
client.post("/api/v1/auth/initialize", json=_init_payload())
|
||||||
|
|
||||||
|
resp = client.post("/api/v1/auth/register", json=_register_payload())
|
||||||
|
assert resp.status_code == 201, resp.text
|
||||||
|
user_id = resp.json()["id"]
|
||||||
|
|
||||||
|
state = asyncio.run(_read_workspace_state(user_id))
|
||||||
|
|
||||||
|
assert len(state["workspaces"]) == 1, state
|
||||||
|
ws_id, ws_slug, ws_owner = state["workspaces"][0]
|
||||||
|
assert ws_owner == user_id
|
||||||
|
assert ws_slug == "alice"
|
||||||
|
|
||||||
|
assert state["memberships"] == [(ws_id, user_id, "owner")]
|
||||||
|
assert state["user_default_workspace_id"] == ws_id
|
||||||
|
|
||||||
|
token = resp.cookies["access_token"]
|
||||||
|
claims = _decode(token)
|
||||||
|
assert claims["wid"] == ws_id
|
||||||
|
assert claims["role"] == "owner"
|
||||||
|
|
||||||
|
|
||||||
|
def test_two_registrations_isolate_workspaces_and_avoid_slug_collision(client):
|
||||||
|
"""Two users with colliding email local-parts → distinct workspaces, slug suffix bump."""
|
||||||
|
client.post("/api/v1/auth/initialize", json=_init_payload())
|
||||||
|
|
||||||
|
r1 = client.post("/api/v1/auth/register", json=_register_payload(email="alice@example.com"))
|
||||||
|
r2 = client.post("/api/v1/auth/register", json=_register_payload(email="alice@somewhere.else"))
|
||||||
|
assert r1.status_code == 201, r1.text
|
||||||
|
assert r2.status_code == 201, r2.text
|
||||||
|
|
||||||
|
s1 = asyncio.run(_read_workspace_state(r1.json()["id"]))
|
||||||
|
s2 = asyncio.run(_read_workspace_state(r2.json()["id"]))
|
||||||
|
|
||||||
|
ws1 = s1["workspaces"][0]
|
||||||
|
ws2 = s2["workspaces"][0]
|
||||||
|
assert ws1[0] != ws2[0], "workspaces must be distinct"
|
||||||
|
assert ws1[1] == "alice"
|
||||||
|
assert ws2[1] == "alice-2", "slug collision walker should land on -2"
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
"""PR6 T6.6 — `@require_permission(owner_check=True)` workspace upgrade.
|
||||||
|
|
||||||
|
The decorator must:
|
||||||
|
|
||||||
|
1. Pull workspace_id from ``get_effective_workspace_id()`` (set by
|
||||||
|
AuthMiddleware per request) and pass it as the third positional to
|
||||||
|
``ThreadMetaStore.check_access``.
|
||||||
|
2. Raise **HTTPException 404** when check_access returns False — never
|
||||||
|
403 — so a cross-workspace request cannot distinguish "thread exists
|
||||||
|
in another tenant" from "thread does not exist".
|
||||||
|
|
||||||
|
These tests build a fake router with the same decorator usage as the
|
||||||
|
production code and verify the decorator's behaviour via a Mock
|
||||||
|
``thread_store`` whose ``check_access`` call we inspect, plus a TestClient
|
||||||
|
exercising the full HTTP boundary.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from _router_auth_helpers import make_authed_test_app
|
||||||
|
from fastapi import APIRouter, Request
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from app.gateway.auth.models import ActiveWorkspace
|
||||||
|
from app.gateway.authz import require_permission
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
reset_current_workspace,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _make_workspace(wid: str) -> Callable[[], ActiveWorkspace]:
|
||||||
|
"""Factory closure stable per call so the middleware reinjects the same id."""
|
||||||
|
|
||||||
|
def _factory() -> ActiveWorkspace:
|
||||||
|
return ActiveWorkspace(id=wid, role="owner")
|
||||||
|
|
||||||
|
return _factory
|
||||||
|
|
||||||
|
|
||||||
|
def _mount_routes(app):
|
||||||
|
router = APIRouter()
|
||||||
|
|
||||||
|
@router.delete("/probe/{thread_id}")
|
||||||
|
@require_permission("threads", "delete", owner_check=True, require_existing=True)
|
||||||
|
async def _delete_probe(thread_id: str, request: Request): # noqa: ARG001
|
||||||
|
return {"ok": True, "thread_id": thread_id}
|
||||||
|
|
||||||
|
@router.get("/probe/{thread_id}")
|
||||||
|
@require_permission("threads", "read", owner_check=True)
|
||||||
|
async def _get_probe(thread_id: str, request: Request): # noqa: ARG001
|
||||||
|
return {"ok": True, "thread_id": thread_id}
|
||||||
|
|
||||||
|
app.include_router(router)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
def test_cross_workspace_returns_404():
|
||||||
|
"""check_access returning False surfaces as 404, never 403."""
|
||||||
|
app = make_authed_test_app(
|
||||||
|
workspace_factory=_make_workspace("ws-alpha"),
|
||||||
|
owner_check_passes=False,
|
||||||
|
)
|
||||||
|
_mount_routes(app)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.delete("/probe/t1")
|
||||||
|
assert response.status_code == 404
|
||||||
|
assert "not found" in response.json()["detail"].lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_same_workspace_delete_allowed():
|
||||||
|
"""check_access returning True lets the route execute."""
|
||||||
|
app = make_authed_test_app(
|
||||||
|
workspace_factory=_make_workspace("ws-alpha"),
|
||||||
|
owner_check_passes=True,
|
||||||
|
)
|
||||||
|
_mount_routes(app)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.delete("/probe/t1")
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json()["ok"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_workspace_id_passed_to_check_access():
|
||||||
|
"""check_access receives the contextvar workspace_id as the 3rd positional."""
|
||||||
|
app = make_authed_test_app(
|
||||||
|
workspace_factory=_make_workspace("ws-alpha"),
|
||||||
|
owner_check_passes=True,
|
||||||
|
)
|
||||||
|
_mount_routes(app)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
client.delete("/probe/t1")
|
||||||
|
|
||||||
|
call = app.state.thread_store.check_access.call_args
|
||||||
|
assert call is not None
|
||||||
|
args = call.args
|
||||||
|
# (thread_id, user_id, workspace_id)
|
||||||
|
assert args[0] == "t1"
|
||||||
|
assert args[2] == "ws-alpha"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_no_workspace_in_context_falls_back_to_default():
|
||||||
|
"""No-auth dev mode (no workspace contextvar) uses DEFAULT_WORKSPACE_ID."""
|
||||||
|
app = make_authed_test_app(workspace_factory=None, owner_check_passes=True)
|
||||||
|
_mount_routes(app)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
client.delete("/probe/t1")
|
||||||
|
|
||||||
|
args = app.state.thread_store.check_access.call_args.args
|
||||||
|
assert args[2] == "default"
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_route_also_uses_workspace_id():
|
||||||
|
"""Read-style routes (require_existing=False) also pass workspace_id through."""
|
||||||
|
app = make_authed_test_app(
|
||||||
|
workspace_factory=_make_workspace("ws-beta"),
|
||||||
|
owner_check_passes=True,
|
||||||
|
)
|
||||||
|
_mount_routes(app)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
client.get("/probe/t-read")
|
||||||
|
args = app.state.thread_store.check_access.call_args.args
|
||||||
|
assert args[2] == "ws-beta"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def _reset_ws():
|
||||||
|
"""Helper for direct-call paths that mutate the contextvar."""
|
||||||
|
tokens: list = []
|
||||||
|
yield lambda wid: tokens.append(set_current_workspace(ActiveWorkspace(id=wid, role="owner")))
|
||||||
|
for token in reversed(tokens):
|
||||||
|
reset_current_workspace(token)
|
||||||
@@ -0,0 +1,181 @@
|
|||||||
|
"""Tests for Run/Feedback/RunEvent repository workspace_id filtering (PR6 T6.5)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
reset_current_workspace,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _init_engine(tmp_path, *, workspaces: tuple[str, ...] = ()):
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
for wid in workspaces:
|
||||||
|
await _seed_workspace(wid)
|
||||||
|
return get_session_factory()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_workspace(wid: str) -> None:
|
||||||
|
from deerflow.persistence.engine import get_session_factory
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
factory = get_session_factory()
|
||||||
|
async with factory() as session:
|
||||||
|
if await session.get(WorkspaceRow, wid) is not None:
|
||||||
|
return
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
session.add(
|
||||||
|
WorkspaceRow(
|
||||||
|
id=wid,
|
||||||
|
name=f"WS {wid}",
|
||||||
|
slug=wid.replace("_", "-")[:32],
|
||||||
|
status="active",
|
||||||
|
owner_id="test-user-autouse",
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
async def _cleanup():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
def _use_workspace(wid: str):
|
||||||
|
return set_current_workspace(SimpleNamespace(id=wid, role="owner"))
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunRepositoryWorkspace:
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_put_records_workspace_id(self, tmp_path):
|
||||||
|
from deerflow.persistence.run import RunRepository
|
||||||
|
|
||||||
|
sf = await _init_engine(tmp_path, workspaces=("ws-alpha",))
|
||||||
|
repo = RunRepository(sf)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.put("r1", thread_id="t1", user_id="alice")
|
||||||
|
record = await repo.get("r1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert record["workspace_id"] == "ws-alpha"
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_get_filters_cross_workspace(self, tmp_path):
|
||||||
|
from deerflow.persistence.run import RunRepository
|
||||||
|
|
||||||
|
sf = await _init_engine(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
repo = RunRepository(sf)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.put("r1", thread_id="t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
assert await repo.get("r1", user_id="alice") is None
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_list_by_thread_filters_workspace(self, tmp_path):
|
||||||
|
from deerflow.persistence.run import RunRepository
|
||||||
|
|
||||||
|
sf = await _init_engine(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
repo = RunRepository(sf)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.put("r1", thread_id="t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
await repo.put("r2", thread_id="t1", user_id="alice")
|
||||||
|
rows = await repo.list_by_thread("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert [r["run_id"] for r in rows] == ["r2"]
|
||||||
|
|
||||||
|
|
||||||
|
class TestFeedbackRepositoryWorkspace:
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_create_records_workspace_id(self, tmp_path):
|
||||||
|
from deerflow.persistence.feedback.sql import FeedbackRepository
|
||||||
|
|
||||||
|
sf = await _init_engine(tmp_path, workspaces=("ws-alpha",))
|
||||||
|
repo = FeedbackRepository(sf)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
row = await repo.create(run_id="r1", thread_id="t1", rating=1, user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert row["workspace_id"] == "ws-alpha"
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_list_by_thread_filters_workspace(self, tmp_path):
|
||||||
|
from deerflow.persistence.feedback.sql import FeedbackRepository
|
||||||
|
|
||||||
|
sf = await _init_engine(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
repo = FeedbackRepository(sf)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create(run_id="r1", thread_id="t1", rating=1, user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
rows = await repo.list_by_thread("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert rows == []
|
||||||
|
|
||||||
|
|
||||||
|
class TestRunEventStoreWorkspace:
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_put_records_workspace_id(self, tmp_path):
|
||||||
|
from deerflow.runtime.events.store.db import DbRunEventStore
|
||||||
|
|
||||||
|
sf = await _init_engine(tmp_path, workspaces=("ws-alpha",))
|
||||||
|
store = DbRunEventStore(sf)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
row = await store.put(thread_id="t1", run_id="r1", event_type="msg", category="message", content="hi")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert row["workspace_id"] == "ws-alpha"
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_list_messages_filters_cross_workspace(self, tmp_path):
|
||||||
|
from deerflow.runtime.events.store.db import DbRunEventStore
|
||||||
|
|
||||||
|
sf = await _init_engine(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
store = DbRunEventStore(sf)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await store.put(thread_id="t1", run_id="r1", event_type="msg", category="message", content="from-alpha")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
rows = await store.list_messages("t1", user_id="test-user-autouse")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert rows == []
|
||||||
@@ -0,0 +1,176 @@
|
|||||||
|
"""Schema tests for ``ServiceAccountRow`` (Stage 0 PR8).
|
||||||
|
|
||||||
|
Pattern mirrors :mod:`test_workspace_repo`: ephemeral SQLite per test via
|
||||||
|
``tmp_path``, no Postgres required at this layer.
|
||||||
|
|
||||||
|
PR8 is schema-only — no repository class, no API. Tests exercise raw
|
||||||
|
ORM behaviour: insert smoke, CASCADE on workspace delete, and RESTRICT
|
||||||
|
on the ``created_by`` user FK.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import delete
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from deerflow.persistence.service_account import ServiceAccountRow
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.anyio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
async def _setup(tmp_path):
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
return get_session_factory()
|
||||||
|
|
||||||
|
|
||||||
|
async def _cleanup():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_user(sf, user_id: str = "u-alice", email: str = "alice@example.com") -> None:
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(UserRow(id=user_id, email=email))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_workspace(sf, workspace_id: str = "w-1", owner_id: str = "u-alice", slug: str = "alice") -> None:
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(WorkspaceRow(id=workspace_id, name="Alice's WS", slug=slug, owner_id=owner_id))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.1 — insert smoke
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_insert_smoke(tmp_path):
|
||||||
|
"""A minimal service_account row can be inserted and read back."""
|
||||||
|
sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf)
|
||||||
|
await _seed_workspace(sf)
|
||||||
|
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(
|
||||||
|
ServiceAccountRow(
|
||||||
|
id="sa-1",
|
||||||
|
workspace_id="w-1",
|
||||||
|
name="ci-bot",
|
||||||
|
role="member",
|
||||||
|
identity_mode="collapsed",
|
||||||
|
status="active",
|
||||||
|
created_by="u-alice",
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
row = await session.get(ServiceAccountRow, "sa-1")
|
||||||
|
assert row is not None
|
||||||
|
assert row.workspace_id == "w-1"
|
||||||
|
assert row.name == "ci-bot"
|
||||||
|
assert row.role == "member"
|
||||||
|
assert row.identity_mode == "collapsed"
|
||||||
|
assert row.status == "active"
|
||||||
|
assert row.created_by == "u-alice"
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.2 — CASCADE on workspace delete
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cascade_on_workspace_delete(tmp_path):
|
||||||
|
"""Deleting the parent workspace removes the service_account row (FK CASCADE)."""
|
||||||
|
sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf)
|
||||||
|
await _seed_workspace(sf)
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(
|
||||||
|
ServiceAccountRow(
|
||||||
|
id="sa-2",
|
||||||
|
workspace_id="w-1",
|
||||||
|
name="bot",
|
||||||
|
role="member",
|
||||||
|
identity_mode="collapsed",
|
||||||
|
status="active",
|
||||||
|
created_by="u-alice",
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
await session.execute(delete(WorkspaceRow).where(WorkspaceRow.id == "w-1"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
row = await session.get(ServiceAccountRow, "sa-2")
|
||||||
|
assert row is None
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T8.3 — RESTRICT on created_by user delete
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_restrict_on_created_by_user_delete(tmp_path):
|
||||||
|
"""Deleting the creator user is blocked while their service_account survives."""
|
||||||
|
sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf)
|
||||||
|
await _seed_workspace(sf)
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(
|
||||||
|
ServiceAccountRow(
|
||||||
|
id="sa-3",
|
||||||
|
workspace_id="w-1",
|
||||||
|
name="bot",
|
||||||
|
role="member",
|
||||||
|
identity_mode="collapsed",
|
||||||
|
status="active",
|
||||||
|
created_by="u-alice",
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
with pytest.raises(IntegrityError):
|
||||||
|
async with sf() as session:
|
||||||
|
await session.execute(delete(UserRow).where(UserRow.id == "u-alice"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
# The service_account is still there after the rollback.
|
||||||
|
async with sf() as session:
|
||||||
|
row = await session.get(ServiceAccountRow, "sa-3")
|
||||||
|
assert row is not None
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
@@ -0,0 +1,88 @@
|
|||||||
|
"""PR6 T6.11 — ThreadDataMiddleware writes under workspace layout."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from deerflow.agents.middlewares.thread_data_middleware import ThreadDataMiddleware
|
||||||
|
from deerflow.config.paths import Paths
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
reset_current_workspace,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeRuntime:
|
||||||
|
def __init__(self, *, thread_id: str = "t1", run_id: str = "r1"):
|
||||||
|
self.context = {"thread_id": thread_id, "run_id": run_id}
|
||||||
|
|
||||||
|
|
||||||
|
def test_paths_resolve_under_workspace(tmp_path):
|
||||||
|
paths = Paths(tmp_path)
|
||||||
|
middleware = ThreadDataMiddleware(base_dir=str(tmp_path), lazy_init=True)
|
||||||
|
middleware._paths = paths
|
||||||
|
|
||||||
|
token = set_current_workspace(SimpleNamespace(id="ws-alpha", role="owner"))
|
||||||
|
try:
|
||||||
|
out = middleware.before_agent({"messages": []}, _FakeRuntime())
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
expected_root = tmp_path / "workspaces" / "ws-alpha" / "threads" / "t1" / "user-data"
|
||||||
|
assert out["thread_data"]["workspace_path"] == str(expected_root / "workspace")
|
||||||
|
assert out["thread_data"]["uploads_path"] == str(expected_root / "uploads")
|
||||||
|
assert out["thread_data"]["outputs_path"] == str(expected_root / "outputs")
|
||||||
|
assert out["thread_data"]["workspace_id"] == "ws-alpha"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_falls_back_to_default_workspace(tmp_path):
|
||||||
|
"""Without a workspace contextvar, `get_effective_workspace_id` returns 'default'."""
|
||||||
|
paths = Paths(tmp_path)
|
||||||
|
middleware = ThreadDataMiddleware(base_dir=str(tmp_path), lazy_init=True)
|
||||||
|
middleware._paths = paths
|
||||||
|
|
||||||
|
out = middleware.before_agent({"messages": []}, _FakeRuntime())
|
||||||
|
|
||||||
|
expected_root = tmp_path / "workspaces" / "default" / "threads" / "t1" / "user-data"
|
||||||
|
assert out["thread_data"]["workspace_path"] == str(expected_root / "workspace")
|
||||||
|
assert out["thread_data"]["workspace_id"] == "default"
|
||||||
|
|
||||||
|
|
||||||
|
def test_eager_creates_directories_under_workspace(tmp_path):
|
||||||
|
paths = Paths(tmp_path)
|
||||||
|
middleware = ThreadDataMiddleware(base_dir=str(tmp_path), lazy_init=False)
|
||||||
|
middleware._paths = paths
|
||||||
|
|
||||||
|
token = set_current_workspace(SimpleNamespace(id="ws-beta", role="owner"))
|
||||||
|
try:
|
||||||
|
middleware.before_agent({"messages": []}, _FakeRuntime(thread_id="t2"))
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
root = tmp_path / "workspaces" / "ws-beta" / "threads" / "t2" / "user-data"
|
||||||
|
assert (root / "workspace").is_dir()
|
||||||
|
assert (root / "uploads").is_dir()
|
||||||
|
assert (root / "outputs").is_dir()
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_config_fallback_still_workspace_scoped(tmp_path):
|
||||||
|
"""Thread_id resolution via LangGraph config still routes through workspace."""
|
||||||
|
paths = Paths(tmp_path)
|
||||||
|
middleware = ThreadDataMiddleware(base_dir=str(tmp_path), lazy_init=True)
|
||||||
|
middleware._paths = paths
|
||||||
|
|
||||||
|
class _Runtime:
|
||||||
|
context: dict = {}
|
||||||
|
|
||||||
|
with patch("deerflow.agents.middlewares.thread_data_middleware.get_config", return_value={"configurable": {"thread_id": "t-cfg"}}):
|
||||||
|
token = set_current_workspace(SimpleNamespace(id="ws-gamma", role="owner"))
|
||||||
|
try:
|
||||||
|
out = middleware.before_agent({"messages": []}, _Runtime())
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
assert "workspaces/ws-gamma/threads/t-cfg/user-data/workspace" in out["thread_data"]["workspace_path"]
|
||||||
@@ -64,21 +64,21 @@ class TestThreadMetaRepository:
|
|||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
async def test_check_access_no_record_allows(self, tmp_path):
|
async def test_check_access_no_record_allows(self, tmp_path):
|
||||||
repo = await _make_repo(tmp_path)
|
repo = await _make_repo(tmp_path)
|
||||||
assert await repo.check_access("unknown", "user1") is True
|
assert await repo.check_access("unknown", "user1", "test-workspace-autouse") is True
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
async def test_check_access_owner_matches(self, tmp_path):
|
async def test_check_access_owner_matches(self, tmp_path):
|
||||||
repo = await _make_repo(tmp_path)
|
repo = await _make_repo(tmp_path)
|
||||||
await repo.create("t1", user_id="user1")
|
await repo.create("t1", user_id="user1")
|
||||||
assert await repo.check_access("t1", "user1") is True
|
assert await repo.check_access("t1", "user1", "test-workspace-autouse") is True
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
async def test_check_access_owner_mismatch(self, tmp_path):
|
async def test_check_access_owner_mismatch(self, tmp_path):
|
||||||
repo = await _make_repo(tmp_path)
|
repo = await _make_repo(tmp_path)
|
||||||
await repo.create("t1", user_id="user1")
|
await repo.create("t1", user_id="user1")
|
||||||
assert await repo.check_access("t1", "user2") is False
|
assert await repo.check_access("t1", "user2", "test-workspace-autouse") is False
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
@@ -87,7 +87,7 @@ class TestThreadMetaRepository:
|
|||||||
# Explicit user_id=None to bypass the new AUTO default that
|
# Explicit user_id=None to bypass the new AUTO default that
|
||||||
# would otherwise pick up the test user from the autouse fixture.
|
# would otherwise pick up the test user from the autouse fixture.
|
||||||
await repo.create("t1", user_id=None)
|
await repo.create("t1", user_id=None)
|
||||||
assert await repo.check_access("t1", "anyone") is True
|
assert await repo.check_access("t1", "anyone", "test-workspace-autouse") is True
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
@@ -99,21 +99,21 @@ class TestThreadMetaRepository:
|
|||||||
caller "claim" it as untracked. The strict mode demands a row.
|
caller "claim" it as untracked. The strict mode demands a row.
|
||||||
"""
|
"""
|
||||||
repo = await _make_repo(tmp_path)
|
repo = await _make_repo(tmp_path)
|
||||||
assert await repo.check_access("never-existed", "user1", require_existing=True) is False
|
assert await repo.check_access("never-existed", "user1", "test-workspace-autouse", require_existing=True) is False
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
async def test_check_access_strict_owner_match_allowed(self, tmp_path):
|
async def test_check_access_strict_owner_match_allowed(self, tmp_path):
|
||||||
repo = await _make_repo(tmp_path)
|
repo = await _make_repo(tmp_path)
|
||||||
await repo.create("t1", user_id="user1")
|
await repo.create("t1", user_id="user1")
|
||||||
assert await repo.check_access("t1", "user1", require_existing=True) is True
|
assert await repo.check_access("t1", "user1", "test-workspace-autouse", require_existing=True) is True
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
async def test_check_access_strict_owner_mismatch_denied(self, tmp_path):
|
async def test_check_access_strict_owner_mismatch_denied(self, tmp_path):
|
||||||
repo = await _make_repo(tmp_path)
|
repo = await _make_repo(tmp_path)
|
||||||
await repo.create("t1", user_id="user1")
|
await repo.create("t1", user_id="user1")
|
||||||
assert await repo.check_access("t1", "user2", require_existing=True) is False
|
assert await repo.check_access("t1", "user2", "test-workspace-autouse", require_existing=True) is False
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
@@ -126,7 +126,7 @@ class TestThreadMetaRepository:
|
|||||||
"""
|
"""
|
||||||
repo = await _make_repo(tmp_path)
|
repo = await _make_repo(tmp_path)
|
||||||
await repo.create("t1", user_id=None)
|
await repo.create("t1", user_id=None)
|
||||||
assert await repo.check_access("t1", "anyone", require_existing=True) is True
|
assert await repo.check_access("t1", "anyone", "test-workspace-autouse", require_existing=True) is True
|
||||||
await _cleanup()
|
await _cleanup()
|
||||||
|
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
|
|||||||
@@ -0,0 +1,296 @@
|
|||||||
|
"""Tests for ThreadMetaRepository workspace_id filtering (PR6 T6.1-T6.4).
|
||||||
|
|
||||||
|
The repository's three-state ``workspace_id`` semantics mirror ``user_id``:
|
||||||
|
|
||||||
|
- :data:`AUTO` (default): read from workspace contextvar
|
||||||
|
- Explicit ``str``: use the provided id
|
||||||
|
- Explicit ``None``: bypass workspace filter (migration / CLI)
|
||||||
|
|
||||||
|
Cross-workspace access (a thread in workspace A queried with workspace B)
|
||||||
|
must return ``None``, never the row. This is the load-bearing isolation
|
||||||
|
boundary tested here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from deerflow.persistence.thread_meta import ThreadMetaRepository
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
reset_current_workspace,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_workspace(wid: str, *, owner_id: str = "test-user-autouse") -> None:
|
||||||
|
"""Insert a workspace row so threads_meta.workspace_id FK resolves."""
|
||||||
|
from deerflow.persistence.engine import get_session_factory
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
factory = get_session_factory()
|
||||||
|
async with factory() as session:
|
||||||
|
existing = await session.get(WorkspaceRow, wid)
|
||||||
|
if existing is not None:
|
||||||
|
return
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
session.add(
|
||||||
|
WorkspaceRow(
|
||||||
|
id=wid,
|
||||||
|
name=f"WS {wid}",
|
||||||
|
slug=wid.replace("_", "-")[:32],
|
||||||
|
status="active",
|
||||||
|
owner_id=owner_id,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
async def _make_repo(tmp_path, *, workspaces: tuple[str, ...] = ()):
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
for wid in workspaces:
|
||||||
|
await _seed_workspace(wid)
|
||||||
|
return ThreadMetaRepository(get_session_factory())
|
||||||
|
|
||||||
|
|
||||||
|
async def _cleanup():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
def _use_workspace(wid: str, role: str = "owner"):
|
||||||
|
"""Replace the autouse workspace contextvar inside a single test."""
|
||||||
|
return set_current_workspace(SimpleNamespace(id=wid, role=role))
|
||||||
|
|
||||||
|
|
||||||
|
class TestCreateWorkspace:
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_create_uses_workspace_context(self, tmp_path):
|
||||||
|
"""AUTO sentinel pulls workspace_id from the contextvar."""
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha",))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
record = await repo.create("t1")
|
||||||
|
assert record["workspace_id"] == "ws-alpha"
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_create_explicit_workspace_overrides_context(self, tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
record = await repo.create("t1", workspace_id="ws-beta")
|
||||||
|
assert record["workspace_id"] == "ws-beta"
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_create_workspace_none_rejected_by_orm(self, tmp_path):
|
||||||
|
"""After T5.11 (ORM nullable=False) explicit None creates fail at the DB layer.
|
||||||
|
|
||||||
|
Pre-PR6 the migration scripts relied on `workspace_id=None` to insert
|
||||||
|
orphan rows; that use is now restricted to **read** paths (filter
|
||||||
|
bypass). Writes must always carry a workspace.
|
||||||
|
"""
|
||||||
|
import sqlalchemy
|
||||||
|
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
with pytest.raises((sqlalchemy.exc.IntegrityError, sqlalchemy.exc.DBAPIError)):
|
||||||
|
await repo.create("t1", workspace_id=None)
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
class TestGetWorkspace:
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_get_filters_by_workspace(self, tmp_path):
|
||||||
|
"""Cross-workspace get returns None even when user_id matches."""
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
assert await repo.get("t1", user_id="alice") is None
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_get_returns_row_in_same_workspace(self, tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha",))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
record = await repo.get("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert record is not None
|
||||||
|
assert record["thread_id"] == "t1"
|
||||||
|
assert record["workspace_id"] == "ws-alpha"
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_get_workspace_none_bypasses_filter(self, tmp_path):
|
||||||
|
"""Explicit workspace_id=None lets migration scripts see any row."""
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha",))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
assert await repo.get("t1", user_id=None, workspace_id=None) is not None
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
class TestSearchUpdateDeleteWorkspace:
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_search_only_returns_current_workspace(self, tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
await repo.create("t2", user_id="alice")
|
||||||
|
rows = await repo.search(user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
ids = {r["thread_id"] for r in rows}
|
||||||
|
assert ids == {"t2"}
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_update_status_blocked_across_workspace(self, tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
await repo.update_status("t1", "busy", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
row = await repo.get("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert row["status"] == "idle"
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_update_display_name_blocked_across_workspace(self, tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice", display_name="A")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
await repo.update_display_name("t1", "B", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
row = await repo.get("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert row["display_name"] == "A"
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_update_metadata_blocked_across_workspace(self, tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice", metadata={"k": "alpha"})
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
await repo.update_metadata("t1", {"k": "beta"}, user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
row = await repo.get("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert row["metadata"] == {"k": "alpha"}
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_check_access_cross_workspace_false(self, tmp_path):
|
||||||
|
"""`check_access` returns False for cross-workspace, even with matching user_id."""
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
try:
|
||||||
|
assert await repo.check_access("t1", "alice", "ws-beta") is False
|
||||||
|
assert await repo.check_access("t1", "alice", "ws-alpha") is True
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_check_access_strict_cross_workspace_false(self, tmp_path):
|
||||||
|
"""require_existing=True path also denies cross-workspace."""
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
try:
|
||||||
|
assert await repo.check_access("t1", "alice", "ws-beta", require_existing=True) is False
|
||||||
|
assert await repo.check_access("t1", "alice", "ws-alpha", require_existing=True) is True
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_delete_blocked_across_workspace(self, tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path, workspaces=("ws-alpha", "ws-beta"))
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
await repo.create("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-beta")
|
||||||
|
try:
|
||||||
|
await repo.delete("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
token = _use_workspace("ws-alpha")
|
||||||
|
try:
|
||||||
|
row = await repo.get("t1", user_id="alice")
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
await _cleanup()
|
||||||
|
assert row is not None and row["thread_id"] == "t1"
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
"""PR6 T6.7 — POST /api/threads writes workspace_id from contextvar.
|
||||||
|
|
||||||
|
The `routers/threads.py:create_thread` path delegates to
|
||||||
|
``ThreadMetaStore.create`` *without* an explicit ``workspace_id`` — it
|
||||||
|
relies on the AUTO sentinel pulling the value from the active workspace
|
||||||
|
contextvar that AuthMiddleware (or the test stub) sets. This test
|
||||||
|
covers the integration through the FastAPI TestClient stack: post a
|
||||||
|
thread under workspace A, then verify the persisted record carries
|
||||||
|
``workspace_id="ws-alpha"``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
from _router_auth_helpers import make_authed_test_app
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from langgraph.checkpoint.memory import InMemorySaver
|
||||||
|
from langgraph.store.memory import InMemoryStore
|
||||||
|
|
||||||
|
from app.gateway.auth.models import ActiveWorkspace
|
||||||
|
from app.gateway.routers import threads
|
||||||
|
from deerflow.persistence.thread_meta.memory import MemoryThreadMetaStore
|
||||||
|
|
||||||
|
|
||||||
|
def _workspace_factory(wid: str) -> Callable[[], ActiveWorkspace]:
|
||||||
|
def _factory() -> ActiveWorkspace:
|
||||||
|
return ActiveWorkspace(id=wid, role="owner")
|
||||||
|
|
||||||
|
return _factory
|
||||||
|
|
||||||
|
|
||||||
|
def _build_app(workspace_id: str):
|
||||||
|
app = make_authed_test_app(workspace_factory=_workspace_factory(workspace_id))
|
||||||
|
store = InMemoryStore()
|
||||||
|
checkpointer = InMemorySaver()
|
||||||
|
app.state.store = store
|
||||||
|
app.state.checkpointer = checkpointer
|
||||||
|
app.state.thread_store = MemoryThreadMetaStore(store)
|
||||||
|
app.include_router(threads.router)
|
||||||
|
return app, store
|
||||||
|
|
||||||
|
|
||||||
|
def test_post_thread_stamps_workspace_id_from_contextvar():
|
||||||
|
app, store = _build_app("ws-alpha")
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.post("/api/threads", json={"thread_id": "t1", "metadata": {}})
|
||||||
|
assert response.status_code == 200, response.text
|
||||||
|
|
||||||
|
item = store.get(("threads",), "t1")
|
||||||
|
assert item is not None
|
||||||
|
assert item.value["workspace_id"] == "ws-alpha"
|
||||||
|
|
||||||
|
|
||||||
|
def test_post_thread_under_different_workspace():
|
||||||
|
app, store = _build_app("ws-beta")
|
||||||
|
with TestClient(app) as client:
|
||||||
|
client.post("/api/threads", json={"thread_id": "t2", "metadata": {}})
|
||||||
|
assert store.get(("threads",), "t2").value["workspace_id"] == "ws-beta"
|
||||||
@@ -27,21 +27,21 @@ class _PermissiveThreadMetaStore(MemoryThreadMetaStore):
|
|||||||
timestamp wire format.
|
timestamp wire format.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
async def _get_owned_record(self, thread_id, user_id, method_name): # type: ignore[override]
|
async def _get_owned_record(self, thread_id, user_id, workspace_id, method_name): # type: ignore[override]
|
||||||
item = await self._store.aget(THREADS_NS, thread_id)
|
item = await self._store.aget(THREADS_NS, thread_id)
|
||||||
return dict(item.value) if item is not None else None
|
return dict(item.value) if item is not None else None
|
||||||
|
|
||||||
async def check_access(self, thread_id, user_id, *, require_existing=False): # type: ignore[override]
|
async def check_access(self, thread_id, user_id, workspace_id, *, require_existing=False): # type: ignore[override]
|
||||||
item = await self._store.aget(THREADS_NS, thread_id)
|
item = await self._store.aget(THREADS_NS, thread_id)
|
||||||
if item is None:
|
if item is None:
|
||||||
return not require_existing
|
return not require_existing
|
||||||
return True
|
return True
|
||||||
|
|
||||||
async def create(self, thread_id, *, assistant_id=None, user_id=None, display_name=None, metadata=None): # type: ignore[override]
|
async def create(self, thread_id, *, assistant_id=None, user_id=None, workspace_id=None, display_name=None, metadata=None): # type: ignore[override]
|
||||||
return await super().create(thread_id, assistant_id=assistant_id, user_id=None, display_name=display_name, metadata=metadata)
|
return await super().create(thread_id, assistant_id=assistant_id, user_id=None, workspace_id=None, display_name=display_name, metadata=metadata)
|
||||||
|
|
||||||
async def search(self, *, metadata=None, status=None, limit=100, offset=0, user_id=None): # type: ignore[override]
|
async def search(self, *, metadata=None, status=None, limit=100, offset=0, user_id=None, workspace_id=None): # type: ignore[override]
|
||||||
return await super().search(metadata=metadata, status=status, limit=limit, offset=offset, user_id=None)
|
return await super().search(metadata=metadata, status=status, limit=limit, offset=offset, user_id=None, workspace_id=None)
|
||||||
|
|
||||||
|
|
||||||
def _build_thread_app() -> tuple[FastAPI, InMemoryStore, InMemorySaver]:
|
def _build_thread_app() -> tuple[FastAPI, InMemoryStore, InMemorySaver]:
|
||||||
|
|||||||
@@ -0,0 +1,127 @@
|
|||||||
|
"""Boundary check: only allowlisted modules may directly import LangGraph
|
||||||
|
checkpoint/saver clients.
|
||||||
|
|
||||||
|
Direct use of the LangGraph checkpoint API anywhere outside the gateway thread
|
||||||
|
plumbing and the harness checkpointer factory is a workspace-isolation hazard:
|
||||||
|
arbitrary code paths could otherwise reach across threads/workspaces by
|
||||||
|
constructing their own savers. PR7 enforces this with an AST static scan.
|
||||||
|
|
||||||
|
Imports inside ``if TYPE_CHECKING:`` blocks are intentionally ignored — they
|
||||||
|
never execute at runtime and therefore cannot bypass the boundary.
|
||||||
|
|
||||||
|
Allowlist lives in ``tests/boundary_allowlist.toml`` as a plain list of paths
|
||||||
|
relative to ``backend/``. Adding a new legitimate importer means appending a
|
||||||
|
line there in the same PR that introduces the import.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import ast
|
||||||
|
import tomllib
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
BACKEND_ROOT = Path(__file__).parent.parent # backend/
|
||||||
|
ALLOWLIST_FILE = Path(__file__).parent / "boundary_allowlist.toml"
|
||||||
|
|
||||||
|
# Match any submodule of these top-level packages.
|
||||||
|
TARGET_MODULE_PREFIXES: tuple[str, ...] = (
|
||||||
|
"langgraph.checkpoint",
|
||||||
|
"langgraph_checkpoint_postgres",
|
||||||
|
"langgraph_checkpoint_sqlite",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Directories under backend/ that are not part of the running app.
|
||||||
|
EXCLUDED_TOP_LEVEL = ("tests", ".venv", "build", "dist", "docs", ".pytest_cache", "node_modules")
|
||||||
|
|
||||||
|
|
||||||
|
def _matches_target(name: str) -> bool:
|
||||||
|
return any(name == prefix or name.startswith(prefix + ".") for prefix in TARGET_MODULE_PREFIXES)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_parent_map(tree: ast.AST) -> dict[int, ast.AST]:
|
||||||
|
parents: dict[int, ast.AST] = {}
|
||||||
|
for parent in ast.walk(tree):
|
||||||
|
for child in ast.iter_child_nodes(parent):
|
||||||
|
parents[id(child)] = parent
|
||||||
|
return parents
|
||||||
|
|
||||||
|
|
||||||
|
def _is_type_checking_test(test: ast.expr) -> bool:
|
||||||
|
"""Return True for ``TYPE_CHECKING`` or ``typing.TYPE_CHECKING`` conditions."""
|
||||||
|
if isinstance(test, ast.Name) and test.id == "TYPE_CHECKING":
|
||||||
|
return True
|
||||||
|
if isinstance(test, ast.Attribute) and test.attr == "TYPE_CHECKING":
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _inside_type_checking(node: ast.AST, parents: dict[int, ast.AST]) -> bool:
|
||||||
|
current = parents.get(id(node))
|
||||||
|
while current is not None:
|
||||||
|
if isinstance(current, ast.If) and _is_type_checking_test(current.test):
|
||||||
|
return True
|
||||||
|
current = parents.get(id(current))
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def collect_runtime_checkpoint_imports(filepath: Path) -> list[tuple[int, str]]:
|
||||||
|
"""Return ``(lineno, module_path)`` for every runtime import that targets a banned module."""
|
||||||
|
source = filepath.read_text(encoding="utf-8")
|
||||||
|
try:
|
||||||
|
tree = ast.parse(source, filename=str(filepath))
|
||||||
|
except SyntaxError:
|
||||||
|
return []
|
||||||
|
|
||||||
|
parents = _build_parent_map(tree)
|
||||||
|
hits: list[tuple[int, str]] = []
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if isinstance(node, ast.ImportFrom):
|
||||||
|
module = node.module or ""
|
||||||
|
if _matches_target(module) and not _inside_type_checking(node, parents):
|
||||||
|
hits.append((node.lineno, module))
|
||||||
|
elif isinstance(node, ast.Import):
|
||||||
|
for alias in node.names:
|
||||||
|
if _matches_target(alias.name) and not _inside_type_checking(node, parents):
|
||||||
|
hits.append((node.lineno, alias.name))
|
||||||
|
return hits
|
||||||
|
|
||||||
|
|
||||||
|
def _iter_backend_py_files() -> list[Path]:
|
||||||
|
candidates: list[Path] = []
|
||||||
|
for path in sorted(BACKEND_ROOT.rglob("*.py")):
|
||||||
|
rel_parts = path.relative_to(BACKEND_ROOT).parts
|
||||||
|
if rel_parts and rel_parts[0] in EXCLUDED_TOP_LEVEL:
|
||||||
|
continue
|
||||||
|
if "__pycache__" in rel_parts:
|
||||||
|
continue
|
||||||
|
candidates.append(path)
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
|
||||||
|
def _load_allowlist() -> set[str]:
|
||||||
|
data = tomllib.loads(ALLOWLIST_FILE.read_text(encoding="utf-8"))
|
||||||
|
raw = data.get("langgraph_checkpoint_importers", [])
|
||||||
|
return set(raw)
|
||||||
|
|
||||||
|
|
||||||
|
def scan_violations(allowlist: set[str]) -> list[str]:
|
||||||
|
"""Return formatted violation lines (one per banned import outside the allowlist)."""
|
||||||
|
violations: list[str] = []
|
||||||
|
for py in _iter_backend_py_files():
|
||||||
|
rel = py.relative_to(BACKEND_ROOT).as_posix()
|
||||||
|
for lineno, module in collect_runtime_checkpoint_imports(py):
|
||||||
|
if rel in allowlist:
|
||||||
|
continue
|
||||||
|
violations.append(f" {rel}:{lineno} imports {module}")
|
||||||
|
return violations
|
||||||
|
|
||||||
|
|
||||||
|
def test_only_allowlisted_modules_import_langgraph_checkpoint() -> None:
|
||||||
|
allowlist = _load_allowlist()
|
||||||
|
violations = scan_violations(allowlist)
|
||||||
|
assert not violations, (
|
||||||
|
"Unauthorized direct imports of langgraph.checkpoint.* detected. "
|
||||||
|
"Either route the access through `app.gateway.deps.get_checkpointer` "
|
||||||
|
"or, if this is a legitimate new importer, add the path to "
|
||||||
|
"tests/boundary_allowlist.toml in the same PR.\n" + "\n".join(violations)
|
||||||
|
)
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
"""Self-tests for the boundary scanner in ``test_workspace_boundary``.
|
||||||
|
|
||||||
|
These tests guard against the scanner silently passing because it cannot
|
||||||
|
detect anything. They feed synthetic ``.py`` snippets to the per-file
|
||||||
|
collector and assert it produces (or correctly suppresses) the expected
|
||||||
|
hits — so we know the integration test in ``test_workspace_boundary.py``
|
||||||
|
would actually fail if a real violation appeared.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from test_workspace_boundary import collect_runtime_checkpoint_imports
|
||||||
|
|
||||||
|
|
||||||
|
def _write(tmp_path: Path, body: str) -> Path:
|
||||||
|
target = tmp_path / "sample.py"
|
||||||
|
target.write_text(body, encoding="utf-8")
|
||||||
|
return target
|
||||||
|
|
||||||
|
|
||||||
|
def test_flags_direct_from_import(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
"from langgraph.checkpoint.postgres import AsyncPostgresSaver\n",
|
||||||
|
)
|
||||||
|
hits = collect_runtime_checkpoint_imports(target)
|
||||||
|
assert hits == [(1, "langgraph.checkpoint.postgres")]
|
||||||
|
|
||||||
|
|
||||||
|
def test_flags_bare_module_import(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
"import langgraph.checkpoint.memory # noqa: F401\n",
|
||||||
|
)
|
||||||
|
hits = collect_runtime_checkpoint_imports(target)
|
||||||
|
assert hits == [(1, "langgraph.checkpoint.memory")]
|
||||||
|
|
||||||
|
|
||||||
|
def test_flags_third_party_checkpoint_package(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
"from langgraph_checkpoint_postgres import PostgresSaver\nfrom langgraph_checkpoint_sqlite import SqliteSaver\n",
|
||||||
|
)
|
||||||
|
hits = collect_runtime_checkpoint_imports(target)
|
||||||
|
assert (1, "langgraph_checkpoint_postgres") in hits
|
||||||
|
assert (2, "langgraph_checkpoint_sqlite") in hits
|
||||||
|
|
||||||
|
|
||||||
|
def test_skips_type_checking_block_name_form(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
"from typing import TYPE_CHECKING\n\nif TYPE_CHECKING:\n from langgraph.checkpoint.base import BaseCheckpointSaver # noqa: F401\n",
|
||||||
|
)
|
||||||
|
assert collect_runtime_checkpoint_imports(target) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_skips_type_checking_block_attribute_form(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
"import typing\n\nif typing.TYPE_CHECKING:\n from langgraph.checkpoint.base import BaseCheckpointSaver # noqa: F401\n",
|
||||||
|
)
|
||||||
|
assert collect_runtime_checkpoint_imports(target) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_skips_nested_type_checking_block(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
"from typing import TYPE_CHECKING\n\nif True:\n if TYPE_CHECKING:\n from langgraph.checkpoint.postgres import AsyncPostgresSaver # noqa: F401\n",
|
||||||
|
)
|
||||||
|
assert collect_runtime_checkpoint_imports(target) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_does_not_flag_unrelated_imports(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
"import os\nfrom langgraph.graph.state import CompiledStateGraph # noqa: F401\nfrom app.gateway.deps import get_checkpointer # noqa: F401\n",
|
||||||
|
)
|
||||||
|
assert collect_runtime_checkpoint_imports(target) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_does_not_flag_string_literal_with_module_name(tmp_path: Path) -> None:
|
||||||
|
target = _write(
|
||||||
|
tmp_path,
|
||||||
|
'TARGET = "langgraph.checkpoint.postgres"\n',
|
||||||
|
)
|
||||||
|
assert collect_runtime_checkpoint_imports(target) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_handles_syntax_error_gracefully(tmp_path: Path) -> None:
|
||||||
|
target = _write(tmp_path, "def broken(:\n")
|
||||||
|
assert collect_runtime_checkpoint_imports(target) == []
|
||||||
@@ -0,0 +1,176 @@
|
|||||||
|
"""Tests for runtime.workspace_context — workspace contextvar semantics.
|
||||||
|
|
||||||
|
Mirrors :mod:`test_user_context` but for the workspace contextvar
|
||||||
|
introduced in Stage 0 PR3. No autouse workspace fixture exists yet
|
||||||
|
(PR4 will add it together with the AuthMiddleware injection), so these
|
||||||
|
tests run against a clean contextvar.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
AUTO,
|
||||||
|
DEFAULT_WORKSPACE_ID,
|
||||||
|
CurrentWorkspace,
|
||||||
|
get_current_workspace,
|
||||||
|
get_effective_workspace_id,
|
||||||
|
require_current_workspace,
|
||||||
|
reset_current_workspace,
|
||||||
|
resolve_workspace_id,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# get_current_workspace / require_current_workspace / set+reset round-trip
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_default_is_none():
|
||||||
|
"""Before any set, contextvar returns None."""
|
||||||
|
assert get_current_workspace() is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_set_and_reset_roundtrip():
|
||||||
|
"""set_current_workspace returns a token that reset restores."""
|
||||||
|
workspace = SimpleNamespace(id="ws-1", role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
assert get_current_workspace() is workspace
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
assert get_current_workspace() is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_require_current_workspace_raises_when_unset():
|
||||||
|
"""require_current_workspace raises RuntimeError if contextvar is unset."""
|
||||||
|
assert get_current_workspace() is None
|
||||||
|
with pytest.raises(RuntimeError, match="without workspace context"):
|
||||||
|
require_current_workspace()
|
||||||
|
|
||||||
|
|
||||||
|
def test_require_current_workspace_returns_workspace_when_set():
|
||||||
|
"""require_current_workspace returns the workspace when contextvar is set."""
|
||||||
|
workspace = SimpleNamespace(id="ws-2", role="admin")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
assert require_current_workspace() is workspace
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# CurrentWorkspace Protocol — must require BOTH .id and .role
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_protocol_accepts_id_and_role():
|
||||||
|
"""CurrentWorkspace is satisfied by any object with .id and .role."""
|
||||||
|
workspace = SimpleNamespace(id="ws-3", role="member")
|
||||||
|
assert isinstance(workspace, CurrentWorkspace)
|
||||||
|
|
||||||
|
|
||||||
|
def test_protocol_rejects_missing_role():
|
||||||
|
"""An object with only .id (no .role) is NOT a workspace."""
|
||||||
|
user_shaped = SimpleNamespace(id="ws-4")
|
||||||
|
assert not isinstance(user_shaped, CurrentWorkspace)
|
||||||
|
|
||||||
|
|
||||||
|
def test_protocol_rejects_no_id():
|
||||||
|
"""An object without .id does not satisfy CurrentWorkspace."""
|
||||||
|
not_a_workspace = SimpleNamespace(role="owner")
|
||||||
|
assert not isinstance(not_a_workspace, CurrentWorkspace)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# get_effective_workspace_id / DEFAULT_WORKSPACE_ID tests
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_workspace_id_is_default():
|
||||||
|
assert DEFAULT_WORKSPACE_ID == "default"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_effective_workspace_id_returns_default_when_no_workspace():
|
||||||
|
"""No workspace in context -> fallback to DEFAULT_WORKSPACE_ID."""
|
||||||
|
assert get_effective_workspace_id() == "default"
|
||||||
|
|
||||||
|
|
||||||
|
def test_effective_workspace_id_returns_workspace_id_when_set():
|
||||||
|
workspace = SimpleNamespace(id="ws-abc-123", role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
assert get_effective_workspace_id() == "ws-abc-123"
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
|
||||||
|
def test_effective_workspace_id_coerces_to_str():
|
||||||
|
"""workspace.id might be a UUID object; must come back as str."""
|
||||||
|
wid = uuid.uuid4()
|
||||||
|
workspace = SimpleNamespace(id=wid, role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
assert get_effective_workspace_id() == str(wid)
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# resolve_workspace_id three-state semantics
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_auto_reads_from_contextvar():
|
||||||
|
workspace = SimpleNamespace(id="ws-resolve-1", role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
assert resolve_workspace_id(AUTO) == "ws-resolve-1"
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_workspace
|
||||||
|
def test_resolve_auto_raises_when_unset():
|
||||||
|
assert get_current_workspace() is None
|
||||||
|
with pytest.raises(RuntimeError, match="workspace_id=AUTO but no workspace"):
|
||||||
|
resolve_workspace_id(AUTO, method_name="TestRepo.search")
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_explicit_str_overrides_contextvar():
|
||||||
|
workspace = SimpleNamespace(id="ws-ctx", role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
# Explicit value beats contextvar — admin override / test path.
|
||||||
|
assert resolve_workspace_id("ws-explicit") == "ws-explicit"
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_explicit_none_means_no_filter():
|
||||||
|
workspace = SimpleNamespace(id="ws-ctx-2", role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
# Explicit None opts out of workspace filtering (migration scripts).
|
||||||
|
assert resolve_workspace_id(None) is None
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_auto_coerces_uuid_to_str():
|
||||||
|
"""resolve_workspace_id with AUTO returns str even if workspace.id is UUID."""
|
||||||
|
wid = uuid.uuid4()
|
||||||
|
workspace = SimpleNamespace(id=wid, role="owner")
|
||||||
|
token = set_current_workspace(workspace)
|
||||||
|
try:
|
||||||
|
resolved = resolve_workspace_id(AUTO)
|
||||||
|
assert resolved == str(wid)
|
||||||
|
assert isinstance(resolved, str)
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
"""PR6 T6.8 — cross-workspace isolation boundary (e2e).
|
||||||
|
|
||||||
|
Wires a ``MemoryThreadMetaStore`` (LangGraph BaseStore backed) into a
|
||||||
|
stub-authed FastAPI app and asserts that any request from workspace B
|
||||||
|
against a thread created in workspace A returns **404**, regardless of
|
||||||
|
matching user_id. The cross-workspace block fires in
|
||||||
|
``ThreadMetaStore.check_access`` and is converted to 404 by
|
||||||
|
``@require_permission(owner_check=True)``.
|
||||||
|
|
||||||
|
We use the memory-backed implementation so the test stays in the test
|
||||||
|
event loop end-to-end (the SQL engine binds to whatever loop owns
|
||||||
|
``init_engine`` and the TestClient spins its own loop, which would
|
||||||
|
collide). The decorator path it exercises is the same as production;
|
||||||
|
the SQL repository's identical workspace filter is unit-covered by
|
||||||
|
``test_thread_meta_workspace_filter.py``.
|
||||||
|
|
||||||
|
Covers:
|
||||||
|
- ``GET /api/threads/{tid}`` — read (require_existing=False)
|
||||||
|
- ``DELETE /api/threads/{tid}`` — destructive (require_existing=True)
|
||||||
|
- ``PATCH /api/threads/{tid}`` — destructive write
|
||||||
|
- positive control: same-workspace GET still succeeds
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from _router_auth_helpers import make_authed_test_app
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from langgraph.checkpoint.memory import InMemorySaver
|
||||||
|
from langgraph.store.memory import InMemoryStore
|
||||||
|
|
||||||
|
from app.gateway.auth.models import ActiveWorkspace, User
|
||||||
|
from app.gateway.routers import threads
|
||||||
|
from deerflow.persistence.thread_meta.memory import MemoryThreadMetaStore
|
||||||
|
from deerflow.runtime.workspace_context import (
|
||||||
|
reset_current_workspace,
|
||||||
|
set_current_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _workspace_factory(wid: str) -> Callable[[], ActiveWorkspace]:
|
||||||
|
def _factory() -> ActiveWorkspace:
|
||||||
|
return ActiveWorkspace(id=wid, role="owner")
|
||||||
|
|
||||||
|
return _factory
|
||||||
|
|
||||||
|
|
||||||
|
def _user_factory(uid: str) -> Callable[[], User]:
|
||||||
|
def _factory() -> User:
|
||||||
|
return User(email=f"{uid}@example.com", password_hash="x", system_role="user", id=uid)
|
||||||
|
|
||||||
|
return _factory
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_thread(store, *, thread_id: str, user_id: str, workspace_id: str) -> None:
|
||||||
|
"""Insert a thread record under a specific workspace, bypassing the autouse fixture."""
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
async def _go():
|
||||||
|
meta_store = MemoryThreadMetaStore(store)
|
||||||
|
token = set_current_workspace(SimpleNamespace(id=workspace_id, role="owner"))
|
||||||
|
try:
|
||||||
|
await meta_store.create(thread_id, user_id=user_id)
|
||||||
|
finally:
|
||||||
|
reset_current_workspace(token)
|
||||||
|
|
||||||
|
asyncio.run(_go())
|
||||||
|
|
||||||
|
|
||||||
|
def _build_app(*, user_id: str, workspace_id: str):
|
||||||
|
app = make_authed_test_app(
|
||||||
|
user_factory=_user_factory(user_id),
|
||||||
|
workspace_factory=_workspace_factory(workspace_id),
|
||||||
|
override_user_contextvar=True,
|
||||||
|
)
|
||||||
|
store = InMemoryStore()
|
||||||
|
app.state.store = store
|
||||||
|
app.state.checkpointer = InMemorySaver()
|
||||||
|
app.state.thread_store = MemoryThreadMetaStore(store)
|
||||||
|
app.include_router(threads.router)
|
||||||
|
return app, store
|
||||||
|
|
||||||
|
|
||||||
|
def test_cross_workspace_get_returns_404():
|
||||||
|
user_id = str(uuid4())
|
||||||
|
app, store = _build_app(user_id=user_id, workspace_id="ws-beta")
|
||||||
|
_seed_thread(store, thread_id="t1", user_id=user_id, workspace_id="ws-alpha")
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.get("/api/threads/t1")
|
||||||
|
assert response.status_code == 404
|
||||||
|
assert "not found" in response.json()["detail"].lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cross_workspace_delete_returns_404():
|
||||||
|
user_id = str(uuid4())
|
||||||
|
app, store = _build_app(user_id=user_id, workspace_id="ws-beta")
|
||||||
|
_seed_thread(store, thread_id="t1", user_id=user_id, workspace_id="ws-alpha")
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.delete("/api/threads/t1")
|
||||||
|
assert response.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_cross_workspace_patch_returns_404():
|
||||||
|
user_id = str(uuid4())
|
||||||
|
app, store = _build_app(user_id=user_id, workspace_id="ws-beta")
|
||||||
|
_seed_thread(store, thread_id="t1", user_id=user_id, workspace_id="ws-alpha")
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.patch("/api/threads/t1", json={"metadata": {"k": "v"}})
|
||||||
|
assert response.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_same_workspace_get_succeeds():
|
||||||
|
"""Positive control: when the workspace matches, the row is returned."""
|
||||||
|
user_id = str(uuid4())
|
||||||
|
app, store = _build_app(user_id=user_id, workspace_id="ws-alpha")
|
||||||
|
_seed_thread(store, thread_id="t1", user_id=user_id, workspace_id="ws-alpha")
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.get("/api/threads/t1")
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json()["thread_id"] == "t1"
|
||||||
@@ -0,0 +1,216 @@
|
|||||||
|
"""Tests for WorkspaceMembershipRepository (Stage 0 PR3 T3.7)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace import WorkspaceRepository
|
||||||
|
from deerflow.persistence.workspace_membership import (
|
||||||
|
MembershipValidationError,
|
||||||
|
WorkspaceMembershipRepository,
|
||||||
|
)
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.anyio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
async def _setup(tmp_path):
|
||||||
|
"""Create both repos against a fresh SQLite DB."""
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
sf = get_session_factory()
|
||||||
|
return (
|
||||||
|
WorkspaceRepository(sf),
|
||||||
|
WorkspaceMembershipRepository(sf),
|
||||||
|
sf,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _cleanup():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_user(sf, user_id: str, email: str) -> None:
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(UserRow(id=user_id, email=email))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# add / remove smoke
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_add_then_get_role(tmp_path):
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-1", "u1@example.com")
|
||||||
|
ws = await ws_repo.create(name="W1", slug="w-1", owner_id="u-1")
|
||||||
|
|
||||||
|
added = await m_repo.add(workspace_id=ws["id"], user_id="u-1", role="owner")
|
||||||
|
assert added["role"] == "owner"
|
||||||
|
|
||||||
|
role = await m_repo.get_role(workspace_id=ws["id"], user_id="u-1")
|
||||||
|
assert role == "owner"
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_remove_returns_true_when_deleted_false_when_missing(tmp_path):
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-1", "u1@example.com")
|
||||||
|
ws = await ws_repo.create(name="W1", slug="rm-w", owner_id="u-1")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-1", role="owner")
|
||||||
|
|
||||||
|
assert await m_repo.remove(workspace_id=ws["id"], user_id="u-1") is True
|
||||||
|
# second remove of same row -> nothing to delete
|
||||||
|
assert await m_repo.remove(workspace_id=ws["id"], user_id="u-1") is False
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# partial unique: exactly one owner per workspace
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cannot_add_second_owner(tmp_path):
|
||||||
|
"""Inserting a second role='owner' in the same workspace must raise IntegrityError."""
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-1", "u1@example.com")
|
||||||
|
await _seed_user(sf, "u-2", "u2@example.com")
|
||||||
|
ws = await ws_repo.create(name="W", slug="one-owner", owner_id="u-1")
|
||||||
|
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-1", role="owner")
|
||||||
|
with pytest.raises(IntegrityError):
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-2", role="owner")
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_admin_and_member_dont_trigger_partial_unique(tmp_path):
|
||||||
|
"""Multiple admin/member rows in one workspace are fine (Stage 2 forward compat)."""
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-1", "u1@example.com")
|
||||||
|
await _seed_user(sf, "u-2", "u2@example.com")
|
||||||
|
await _seed_user(sf, "u-3", "u3@example.com")
|
||||||
|
ws = await ws_repo.create(name="W", slug="multi-admin", owner_id="u-1")
|
||||||
|
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-1", role="owner")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-2", role="admin")
|
||||||
|
# Second admin OK
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-3", role="admin")
|
||||||
|
|
||||||
|
members = await m_repo.list_by_workspace(workspace_id=ws["id"])
|
||||||
|
assert len(members) == 3
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# CASCADE: deleting a user wipes their memberships
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cascade_delete_user_removes_memberships(tmp_path):
|
||||||
|
"""FK ON DELETE CASCADE on user_id."""
|
||||||
|
from sqlalchemy import delete
|
||||||
|
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-keep", "keep@example.com")
|
||||||
|
await _seed_user(sf, "u-purge", "purge@example.com")
|
||||||
|
ws = await ws_repo.create(name="W", slug="cascade", owner_id="u-keep")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-keep", role="owner")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-purge", role="admin")
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
await session.execute(delete(UserRow).where(UserRow.id == "u-purge"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
members = await m_repo.list_by_workspace(workspace_id=ws["id"])
|
||||||
|
member_ids = [m["user_id"] for m in members]
|
||||||
|
assert "u-purge" not in member_ids
|
||||||
|
assert "u-keep" in member_ids
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# list_by_user ordering (most recent joined_at first)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_list_by_user_orders_recent_first(tmp_path):
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-1", "u1@example.com")
|
||||||
|
await _seed_user(sf, "u-2", "u2@example.com")
|
||||||
|
|
||||||
|
# u-1 owns workspace A
|
||||||
|
ws_a = await ws_repo.create(name="A", slug="ord-a", owner_id="u-1")
|
||||||
|
await m_repo.add(workspace_id=ws_a["id"], user_id="u-1", role="owner")
|
||||||
|
await asyncio.sleep(0.01) # ensure distinct joined_at
|
||||||
|
|
||||||
|
# u-1 later joins workspace C as a member (u-2 owns it)
|
||||||
|
ws_c = await ws_repo.create(name="C", slug="ord-c", owner_id="u-2")
|
||||||
|
await m_repo.add(workspace_id=ws_c["id"], user_id="u-1", role="member")
|
||||||
|
|
||||||
|
memberships = await m_repo.list_by_user(user_id="u-1")
|
||||||
|
slugs_in_order = [(m["workspace_id"], m["role"]) for m in memberships]
|
||||||
|
# 'ws_c member' joined AFTER 'ws_a owner' → ws_c first
|
||||||
|
assert slugs_in_order[0] == (ws_c["id"], "member")
|
||||||
|
assert slugs_in_order[1] == (ws_a["id"], "owner")
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# role validation + change_role
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_add_rejects_unknown_role(tmp_path):
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-1", "u1@example.com")
|
||||||
|
ws = await ws_repo.create(name="W", slug="invrole", owner_id="u-1")
|
||||||
|
with pytest.raises(MembershipValidationError, match="allowed set"):
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-1", role="viewer")
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_change_role_admin_to_member(tmp_path):
|
||||||
|
ws_repo, m_repo, sf = await _setup(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(sf, "u-1", "u1@example.com")
|
||||||
|
await _seed_user(sf, "u-2", "u2@example.com")
|
||||||
|
ws = await ws_repo.create(name="W", slug="chg", owner_id="u-1")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-1", role="owner")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="u-2", role="admin")
|
||||||
|
|
||||||
|
ok = await m_repo.change_role(workspace_id=ws["id"], user_id="u-2", new_role="member")
|
||||||
|
assert ok is True
|
||||||
|
assert await m_repo.get_role(workspace_id=ws["id"], user_id="u-2") == "member"
|
||||||
|
|
||||||
|
# change_role on a non-member returns False
|
||||||
|
miss = await m_repo.change_role(workspace_id=ws["id"], user_id="u-nonexistent", new_role="member")
|
||||||
|
assert miss is False
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
"""Partial unique index `idx_one_owner_per_workspace` works on Postgres.
|
||||||
|
|
||||||
|
The same assertion against SQLite is in
|
||||||
|
:mod:`test_workspace_membership_repo::test_cannot_add_second_owner`.
|
||||||
|
This file adds the Postgres twin via the @pytest.mark.postgres
|
||||||
|
testcontainers fixture from PR1.
|
||||||
|
|
||||||
|
Why duplicate the test:
|
||||||
|
- SQLite and Postgres parse ``WHERE`` clauses differently. We declare
|
||||||
|
both ``sqlite_where`` and ``postgresql_where`` on the Index; this
|
||||||
|
test pins that Postgres genuinely enforces the partial-unique
|
||||||
|
constraint, not just that SQLAlchemy emits the DDL.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace import WorkspaceRepository
|
||||||
|
from deerflow.persistence.workspace_membership import WorkspaceMembershipRepository
|
||||||
|
|
||||||
|
pytestmark = [pytest.mark.postgres, pytest.mark.anyio]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
async def test_partial_unique_on_owner_enforced_on_postgres(postgres_url: str) -> None:
|
||||||
|
"""Postgres: inserting a 2nd owner must raise IntegrityError, same as SQLite."""
|
||||||
|
from deerflow.persistence.engine import close_engine, get_session_factory, init_engine
|
||||||
|
|
||||||
|
await init_engine("postgres", url=postgres_url)
|
||||||
|
try:
|
||||||
|
sf = get_session_factory()
|
||||||
|
ws_repo = WorkspaceRepository(sf)
|
||||||
|
m_repo = WorkspaceMembershipRepository(sf)
|
||||||
|
|
||||||
|
# Seed two users
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(UserRow(id="pg-u1", email="pg1@example.com"))
|
||||||
|
session.add(UserRow(id="pg-u2", email="pg2@example.com"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
ws = await ws_repo.create(name="PG W", slug="pg-one-owner", owner_id="pg-u1")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="pg-u1", role="owner")
|
||||||
|
|
||||||
|
with pytest.raises(IntegrityError):
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="pg-u2", role="owner")
|
||||||
|
finally:
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_multiple_admins_allowed_on_postgres(postgres_url: str) -> None:
|
||||||
|
"""Postgres: partial unique on owner must NOT block multiple admins."""
|
||||||
|
from deerflow.persistence.engine import close_engine, get_session_factory, init_engine
|
||||||
|
|
||||||
|
await init_engine("postgres", url=postgres_url)
|
||||||
|
try:
|
||||||
|
sf = get_session_factory()
|
||||||
|
ws_repo = WorkspaceRepository(sf)
|
||||||
|
m_repo = WorkspaceMembershipRepository(sf)
|
||||||
|
|
||||||
|
async with sf() as session:
|
||||||
|
session.add(UserRow(id="pg-a1", email="a1@example.com"))
|
||||||
|
session.add(UserRow(id="pg-a2", email="a2@example.com"))
|
||||||
|
session.add(UserRow(id="pg-a3", email="a3@example.com"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
ws = await ws_repo.create(name="PG W", slug="pg-multi-admin", owner_id="pg-a1")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="pg-a1", role="owner")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="pg-a2", role="admin")
|
||||||
|
await m_repo.add(workspace_id=ws["id"], user_id="pg-a3", role="admin")
|
||||||
|
|
||||||
|
members = await m_repo.list_by_workspace(workspace_id=ws["id"])
|
||||||
|
assert {(m["user_id"], m["role"]) for m in members} == {
|
||||||
|
("pg-a1", "owner"),
|
||||||
|
("pg-a2", "admin"),
|
||||||
|
("pg-a3", "admin"),
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
await close_engine()
|
||||||
@@ -0,0 +1,296 @@
|
|||||||
|
"""Tests for WorkspaceRepository (Stage 0 PR3).
|
||||||
|
|
||||||
|
Pattern mirrors :mod:`test_feedback`: SQLite ephemeral DB per test via
|
||||||
|
tmp_path, no real Postgres needed at the unit-test layer. Partial-unique
|
||||||
|
double-driver validation lives in :mod:`test_workspace_partial_unique`
|
||||||
|
(T3.8, runs against both backends).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
|
from deerflow.persistence.user.model import UserRow
|
||||||
|
from deerflow.persistence.workspace import WorkspaceRepository, WorkspaceValidationError
|
||||||
|
|
||||||
|
|
||||||
|
async def _make_repo(tmp_path):
|
||||||
|
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||||
|
|
||||||
|
url = f"sqlite+aiosqlite:///{tmp_path / 'test.db'}"
|
||||||
|
await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path))
|
||||||
|
return WorkspaceRepository(get_session_factory())
|
||||||
|
|
||||||
|
|
||||||
|
async def _cleanup():
|
||||||
|
from deerflow.persistence.engine import close_engine
|
||||||
|
|
||||||
|
await close_engine()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_user(repo, user_id: str = "u-alice", email: str = "alice@example.com") -> None:
|
||||||
|
"""Create a user row so workspace.owner_id FK is satisfied."""
|
||||||
|
async with repo._sf() as session:
|
||||||
|
session.add(UserRow(id=user_id, email=email))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.anyio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def anyio_backend() -> str:
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# create / get_by_slug round-trip
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_then_lookup_by_slug(tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo)
|
||||||
|
created = await repo.create(name="Alice's Workspace", slug="alice", owner_id="u-alice")
|
||||||
|
assert created["slug"] == "alice"
|
||||||
|
assert created["status"] == "active"
|
||||||
|
assert created["owner_id"] == "u-alice"
|
||||||
|
assert len(created["id"]) == 36 # UUID v4
|
||||||
|
|
||||||
|
fetched = await repo.get_by_slug("alice")
|
||||||
|
assert fetched is not None
|
||||||
|
assert fetched["id"] == created["id"]
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_get_by_slug_returns_none_when_missing(tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
assert await repo.get_by_slug("nonexistent") is None
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# slug uniqueness + format + blacklist
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_rejects_duplicate_slug(tmp_path):
|
||||||
|
"""Two workspaces with the same slug — second raises IntegrityError."""
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo)
|
||||||
|
await repo.create(name="A", slug="dup", owner_id="u-alice")
|
||||||
|
with pytest.raises(IntegrityError):
|
||||||
|
await repo.create(name="B", slug="dup", owner_id="u-alice")
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"bad_slug",
|
||||||
|
[
|
||||||
|
"ab", # too short
|
||||||
|
"x" * 33, # too long
|
||||||
|
"UPPER", # uppercase
|
||||||
|
"has space", # space
|
||||||
|
"-start-with-dash", # bad start
|
||||||
|
"end-with-dash-", # bad end
|
||||||
|
"double--dash", # consecutive dashes
|
||||||
|
"underscore_not_ok", # underscore
|
||||||
|
],
|
||||||
|
)
|
||||||
|
async def test_create_rejects_invalid_slug_pattern(tmp_path, bad_slug):
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo)
|
||||||
|
with pytest.raises(WorkspaceValidationError, match="(pattern|length)"):
|
||||||
|
await repo.create(name="x", slug=bad_slug, owner_id="u-alice")
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("reserved", ["admin", "api", "auth", "settings", "billing", "select-workspace"])
|
||||||
|
async def test_create_rejects_reserved_slug(tmp_path, reserved):
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo)
|
||||||
|
with pytest.raises(WorkspaceValidationError, match="reserved"):
|
||||||
|
await repo.create(name="x", slug=reserved, owner_id="u-alice")
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# status state machine
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_status_state_transitions(tmp_path):
|
||||||
|
"""active → suspended → deleted are all accepted."""
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo)
|
||||||
|
ws = await repo.create(name="x", slug="trans", owner_id="u-alice")
|
||||||
|
assert ws["status"] == "active"
|
||||||
|
|
||||||
|
await repo.update_status(ws["id"], "suspended")
|
||||||
|
async with repo._sf() as session:
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
row = await session.get(WorkspaceRow, ws["id"])
|
||||||
|
assert row.status == "suspended"
|
||||||
|
|
||||||
|
await repo.update_status(ws["id"], "deleted")
|
||||||
|
async with repo._sf() as session:
|
||||||
|
from deerflow.persistence.workspace.model import WorkspaceRow
|
||||||
|
|
||||||
|
row = await session.get(WorkspaceRow, ws["id"])
|
||||||
|
assert row.status == "deleted"
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_update_status_rejects_unknown_value(tmp_path):
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo)
|
||||||
|
ws = await repo.create(name="x", slug="rejstat", owner_id="u-alice")
|
||||||
|
with pytest.raises(WorkspaceValidationError, match="allowed set"):
|
||||||
|
await repo.update_status(ws["id"], "weird-state")
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# CASCADE: workspace.delete() drops dependent memberships
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def test_delete_cascades_to_memberships(tmp_path):
|
||||||
|
"""Deleting a workspace removes its membership rows (FK CASCADE)."""
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo)
|
||||||
|
ws = await repo.create(name="x", slug="casc", owner_id="u-alice")
|
||||||
|
# Insert an owner membership manually (repository pattern is single-
|
||||||
|
# responsibility; registration flow will normally insert both rows
|
||||||
|
# in one transaction).
|
||||||
|
async with repo._sf() as session:
|
||||||
|
session.add(WorkspaceMembershipRow(workspace_id=ws["id"], user_id="u-alice", role="owner"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
await repo.delete(ws["id"])
|
||||||
|
|
||||||
|
async with repo._sf() as session:
|
||||||
|
remaining = (await session.execute(select(WorkspaceMembershipRow).where(WorkspaceMembershipRow.workspace_id == ws["id"]))).scalars().all()
|
||||||
|
assert remaining == [], "memberships should be CASCADE-deleted with workspace"
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# membership-aware get + list_by_user
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_user
|
||||||
|
async def test_get_returns_none_for_non_member(tmp_path):
|
||||||
|
"""User-A creates a workspace; User-B's `get(wsA)` returns None."""
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
from deerflow.runtime.user_context import reset_current_user, set_current_user
|
||||||
|
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
# Seed two users
|
||||||
|
async with repo._sf() as session:
|
||||||
|
session.add(UserRow(id="u-A", email="a@example.com"))
|
||||||
|
session.add(UserRow(id="u-B", email="b@example.com"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
# User A creates workspace + becomes owner
|
||||||
|
ws = await repo.create(name="A's WS", slug="a-ws", owner_id="u-A")
|
||||||
|
async with repo._sf() as session:
|
||||||
|
session.add(WorkspaceMembershipRow(workspace_id=ws["id"], user_id="u-A", role="owner"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
# User B attempts to read it via contextvar
|
||||||
|
user_b = SimpleNamespace(id="u-B")
|
||||||
|
token = set_current_user(user_b)
|
||||||
|
try:
|
||||||
|
assert await repo.get(ws["id"]) is None
|
||||||
|
finally:
|
||||||
|
reset_current_user(token)
|
||||||
|
|
||||||
|
# User A's own get succeeds
|
||||||
|
user_a = SimpleNamespace(id="u-A")
|
||||||
|
token = set_current_user(user_a)
|
||||||
|
try:
|
||||||
|
row = await repo.get(ws["id"])
|
||||||
|
assert row is not None
|
||||||
|
assert row["slug"] == "a-ws"
|
||||||
|
finally:
|
||||||
|
reset_current_user(token)
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.no_auto_user
|
||||||
|
async def test_list_by_user_excludes_other_workspaces(tmp_path):
|
||||||
|
from deerflow.persistence.workspace_membership.model import WorkspaceMembershipRow
|
||||||
|
from deerflow.runtime.user_context import reset_current_user, set_current_user
|
||||||
|
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
async with repo._sf() as session:
|
||||||
|
session.add(UserRow(id="u-A", email="a@example.com"))
|
||||||
|
session.add(UserRow(id="u-B", email="b@example.com"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
ws_a = await repo.create(name="A", slug="ws-a", owner_id="u-A")
|
||||||
|
ws_b = await repo.create(name="B", slug="ws-b", owner_id="u-B")
|
||||||
|
async with repo._sf() as session:
|
||||||
|
session.add(WorkspaceMembershipRow(workspace_id=ws_a["id"], user_id="u-A", role="owner"))
|
||||||
|
session.add(WorkspaceMembershipRow(workspace_id=ws_b["id"], user_id="u-B", role="owner"))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
token = set_current_user(SimpleNamespace(id="u-A"))
|
||||||
|
try:
|
||||||
|
workspaces = await repo.list_by_user()
|
||||||
|
assert [w["slug"] for w in workspaces] == ["ws-a"]
|
||||||
|
finally:
|
||||||
|
reset_current_user(token)
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_list_by_user_bypass_returns_all(tmp_path):
|
||||||
|
"""user_id=None opts out of membership filter (migration path)."""
|
||||||
|
repo = await _make_repo(tmp_path)
|
||||||
|
try:
|
||||||
|
await _seed_user(repo, "u-A", "a@example.com")
|
||||||
|
await _seed_user(repo, "u-B", "b@example.com")
|
||||||
|
await repo.create(name="A", slug="all-a", owner_id="u-A")
|
||||||
|
await repo.create(name="B", slug="all-b", owner_id="u-B")
|
||||||
|
|
||||||
|
workspaces = await repo.list_by_user(user_id=None)
|
||||||
|
# PR6 conftest auto-seeds an "autouse-test" workspace via the
|
||||||
|
# ``Base.metadata.after_create`` hook so business-row FKs resolve.
|
||||||
|
# ``user_id=None`` bypasses the membership filter, so it surfaces
|
||||||
|
# alongside the two rows the test inserted — that is the intended
|
||||||
|
# "no filter" behaviour. Assert the inserted ones are present.
|
||||||
|
slugs = sorted(w["slug"] for w in workspaces)
|
||||||
|
assert "all-a" in slugs
|
||||||
|
assert "all-b" in slugs
|
||||||
|
finally:
|
||||||
|
await _cleanup()
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
"""Slug helpers for registration / initialize (Stage 0 PR4 T4.10).
|
||||||
|
|
||||||
|
Two surfaces under test:
|
||||||
|
|
||||||
|
- ``auto_slug_from_email`` — pure transform; no DB dependency.
|
||||||
|
- ``next_available_slug`` — async collision walker; we stub the
|
||||||
|
``exists_check`` callable so the test stays a unit test.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.gateway.auth.workspace_slug import auto_slug_from_email, next_available_slug
|
||||||
|
|
||||||
|
# Mirror of the schema's slug pattern. Keeping it inline keeps this
|
||||||
|
# test self-contained — if the schema regex changes we want this test
|
||||||
|
# to refuse to lie about validity.
|
||||||
|
_SLUG_PATTERN = re.compile(r"^[a-z0-9](-?[a-z0-9])*$")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("email", "expected"),
|
||||||
|
[
|
||||||
|
("foo@example.com", "foo"),
|
||||||
|
("foo.bar@example.com", "foo-bar"),
|
||||||
|
("foo+spam@example.com", "foo-spam"),
|
||||||
|
("foo_bar@example.com", "foo-bar"),
|
||||||
|
("Foo.Bar@example.com", "foo-bar"),
|
||||||
|
("foo.bar+spam@example.com", "foo-bar-spam"),
|
||||||
|
("aaaaaaaaaabbbbbbbbbbccccccccccddddd@example.com", "aaaaaaaaaabbbbbbbbbbccccccccccdd"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_auto_slug_known_inputs(email: str, expected: str) -> None:
|
||||||
|
"""Deterministic mapping for the inputs called out in the design doc."""
|
||||||
|
assert auto_slug_from_email(email) == expected
|
||||||
|
assert _SLUG_PATTERN.fullmatch(auto_slug_from_email(email)), "schema regex must accept the output"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"email",
|
||||||
|
[
|
||||||
|
"@example.com", # no local part
|
||||||
|
"a@example.com", # too short
|
||||||
|
"ab@example.com", # still too short
|
||||||
|
"...+_+...@example.com", # only separators
|
||||||
|
"---@example.com", # only hyphens
|
||||||
|
"🎉@example.com", # non-ASCII
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_auto_slug_falls_back_when_unusable(email: str) -> None:
|
||||||
|
"""Pathological emails fall back to ``user-{token}`` so the slug is always valid."""
|
||||||
|
slug = auto_slug_from_email(email)
|
||||||
|
assert slug.startswith("user-"), f"expected fallback, got {slug!r}"
|
||||||
|
assert _SLUG_PATTERN.fullmatch(slug)
|
||||||
|
assert 3 <= len(slug) <= 32
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
@pytest.mark.parametrize("anyio_backend", ["asyncio"])
|
||||||
|
async def test_next_available_slug_returns_base_when_free(anyio_backend) -> None:
|
||||||
|
"""No collision → ``base`` is returned unchanged."""
|
||||||
|
seen: set[str] = set()
|
||||||
|
|
||||||
|
async def exists(s: str) -> bool:
|
||||||
|
return s in seen
|
||||||
|
|
||||||
|
assert await next_available_slug("foo", exists_check=exists) == "foo"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
@pytest.mark.parametrize("anyio_backend", ["asyncio"])
|
||||||
|
async def test_next_available_slug_walks_through_collisions(anyio_backend) -> None:
|
||||||
|
"""``foo``, ``foo-2`` taken → walker lands on ``foo-3``."""
|
||||||
|
seen = {"foo", "foo-2"}
|
||||||
|
|
||||||
|
async def exists(s: str) -> bool:
|
||||||
|
return s in seen
|
||||||
|
|
||||||
|
assert await next_available_slug("foo", exists_check=exists) == "foo-3"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
@pytest.mark.parametrize("anyio_backend", ["asyncio"])
|
||||||
|
async def test_next_available_slug_truncates_base_to_fit_suffix(anyio_backend) -> None:
|
||||||
|
"""A 32-char base + ``-2`` would exceed the limit → base is shortened."""
|
||||||
|
base = "a" * 32 # exactly at the limit
|
||||||
|
seen = {base}
|
||||||
|
|
||||||
|
async def exists(s: str) -> bool:
|
||||||
|
return s in seen
|
||||||
|
|
||||||
|
result = await next_available_slug(base, exists_check=exists)
|
||||||
|
assert len(result) <= 32
|
||||||
|
assert result.endswith("-2")
|
||||||
|
assert _SLUG_PATTERN.fullmatch(result)
|
||||||
Generated
+45
-2
@@ -766,6 +766,9 @@ dependencies = [
|
|||||||
postgres = [
|
postgres = [
|
||||||
{ name = "deerflow-harness", extra = ["postgres"] },
|
{ name = "deerflow-harness", extra = ["postgres"] },
|
||||||
]
|
]
|
||||||
|
postgres-test = [
|
||||||
|
{ name = "deerflow-harness", extra = ["postgres-test"] },
|
||||||
|
]
|
||||||
|
|
||||||
[package.dev-dependencies]
|
[package.dev-dependencies]
|
||||||
dev = [
|
dev = [
|
||||||
@@ -780,6 +783,7 @@ requires-dist = [
|
|||||||
{ name = "bcrypt", specifier = ">=4.0.0" },
|
{ name = "bcrypt", specifier = ">=4.0.0" },
|
||||||
{ name = "deerflow-harness", editable = "packages/harness" },
|
{ name = "deerflow-harness", editable = "packages/harness" },
|
||||||
{ name = "deerflow-harness", extras = ["postgres"], marker = "extra == 'postgres'", editable = "packages/harness" },
|
{ name = "deerflow-harness", extras = ["postgres"], marker = "extra == 'postgres'", editable = "packages/harness" },
|
||||||
|
{ name = "deerflow-harness", extras = ["postgres-test"], marker = "extra == 'postgres-test'", editable = "packages/harness" },
|
||||||
{ name = "dingtalk-stream", specifier = ">=0.24.3" },
|
{ name = "dingtalk-stream", specifier = ">=0.24.3" },
|
||||||
{ name = "email-validator", specifier = ">=2.0.0" },
|
{ name = "email-validator", specifier = ">=2.0.0" },
|
||||||
{ name = "fastapi", specifier = ">=0.115.0" },
|
{ name = "fastapi", specifier = ">=0.115.0" },
|
||||||
@@ -795,7 +799,7 @@ requires-dist = [
|
|||||||
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.34.0" },
|
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.34.0" },
|
||||||
{ name = "wecom-aibot-python-sdk", specifier = ">=0.1.6" },
|
{ name = "wecom-aibot-python-sdk", specifier = ">=0.1.6" },
|
||||||
]
|
]
|
||||||
provides-extras = ["postgres"]
|
provides-extras = ["postgres", "postgres-test"]
|
||||||
|
|
||||||
[package.metadata.requires-dev]
|
[package.metadata.requires-dev]
|
||||||
dev = [
|
dev = [
|
||||||
@@ -854,6 +858,13 @@ postgres = [
|
|||||||
{ name = "psycopg", extra = ["binary"] },
|
{ name = "psycopg", extra = ["binary"] },
|
||||||
{ name = "psycopg-pool" },
|
{ name = "psycopg-pool" },
|
||||||
]
|
]
|
||||||
|
postgres-test = [
|
||||||
|
{ name = "asyncpg" },
|
||||||
|
{ name = "langgraph-checkpoint-postgres" },
|
||||||
|
{ name = "psycopg", extra = ["binary"] },
|
||||||
|
{ name = "psycopg-pool" },
|
||||||
|
{ name = "testcontainers" },
|
||||||
|
]
|
||||||
pymupdf = [
|
pymupdf = [
|
||||||
{ name = "pymupdf4llm" },
|
{ name = "pymupdf4llm" },
|
||||||
]
|
]
|
||||||
@@ -866,6 +877,7 @@ requires-dist = [
|
|||||||
{ name = "alembic", specifier = ">=1.13" },
|
{ name = "alembic", specifier = ">=1.13" },
|
||||||
{ name = "asyncpg", marker = "extra == 'postgres'", specifier = ">=0.29" },
|
{ name = "asyncpg", marker = "extra == 'postgres'", specifier = ">=0.29" },
|
||||||
{ name = "ddgs", specifier = ">=9.10.0" },
|
{ name = "ddgs", specifier = ">=9.10.0" },
|
||||||
|
{ name = "deerflow-harness", extras = ["postgres"], marker = "extra == 'postgres-test'", editable = "packages/harness" },
|
||||||
{ name = "dotenv", specifier = ">=0.9.9" },
|
{ name = "dotenv", specifier = ">=0.9.9" },
|
||||||
{ name = "duckdb", specifier = ">=1.4.4" },
|
{ name = "duckdb", specifier = ">=1.4.4" },
|
||||||
{ name = "exa-py", specifier = ">=1.0.0" },
|
{ name = "exa-py", specifier = ">=1.0.0" },
|
||||||
@@ -897,9 +909,10 @@ requires-dist = [
|
|||||||
{ name = "readabilipy", specifier = ">=0.3.0" },
|
{ name = "readabilipy", specifier = ">=0.3.0" },
|
||||||
{ name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0,<3.0" },
|
{ name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0,<3.0" },
|
||||||
{ name = "tavily-python", specifier = ">=0.7.17" },
|
{ name = "tavily-python", specifier = ">=0.7.17" },
|
||||||
|
{ name = "testcontainers", extras = ["postgres"], marker = "extra == 'postgres-test'", specifier = ">=4.0" },
|
||||||
{ name = "tiktoken", specifier = ">=0.8.0" },
|
{ name = "tiktoken", specifier = ">=0.8.0" },
|
||||||
]
|
]
|
||||||
provides-extras = ["ollama", "postgres", "pymupdf"]
|
provides-extras = ["ollama", "postgres", "postgres-test", "pymupdf"]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "defusedxml"
|
name = "defusedxml"
|
||||||
@@ -941,6 +954,20 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/ba/5a/18ad964b0086c6e62e2e7500f7edc89e3faa45033c71c1893d34eed2b2de/dnspython-2.8.0-py3-none-any.whl", hash = "sha256:01d9bbc4a2d76bf0db7c1f729812ded6d912bd318d3b1cf81d30c0f845dbf3af", size = 331094, upload-time = "2025-09-07T18:57:58.071Z" },
|
{ url = "https://files.pythonhosted.org/packages/ba/5a/18ad964b0086c6e62e2e7500f7edc89e3faa45033c71c1893d34eed2b2de/dnspython-2.8.0-py3-none-any.whl", hash = "sha256:01d9bbc4a2d76bf0db7c1f729812ded6d912bd318d3b1cf81d30c0f845dbf3af", size = 331094, upload-time = "2025-09-07T18:57:58.071Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "docker"
|
||||||
|
version = "7.1.0"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "pywin32", marker = "sys_platform == 'win32'" },
|
||||||
|
{ name = "requests" },
|
||||||
|
{ name = "urllib3" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/91/9b/4a2ea29aeba62471211598dac5d96825bb49348fa07e906ea930394a83ce/docker-7.1.0.tar.gz", hash = "sha256:ad8c70e6e3f8926cb8a92619b832b4ea5299e2831c14284663184e200546fa6c", size = 117834, upload-time = "2024-05-23T11:13:57.216Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/e3/26/57c6fb270950d476074c087527a558ccb6f4436657314bfb6cdf484114c4/docker-7.1.0-py3-none-any.whl", hash = "sha256:c96b93b7f0a746f9e77d325bcfb87422a3d8bd4f03136ae8a85b37f1898d5fc0", size = 147774, upload-time = "2024-05-23T11:13:55.01Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "docstring-parser"
|
name = "docstring-parser"
|
||||||
version = "0.18.0"
|
version = "0.18.0"
|
||||||
@@ -4124,6 +4151,22 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/d7/c1/eb8f9debc45d3b7918a32ab756658a0904732f75e555402972246b0b8e71/tenacity-9.1.4-py3-none-any.whl", hash = "sha256:6095a360c919085f28c6527de529e76a06ad89b23659fa881ae0649b867a9d55", size = 28926, upload-time = "2026-02-07T10:45:32.24Z" },
|
{ url = "https://files.pythonhosted.org/packages/d7/c1/eb8f9debc45d3b7918a32ab756658a0904732f75e555402972246b0b8e71/tenacity-9.1.4-py3-none-any.whl", hash = "sha256:6095a360c919085f28c6527de529e76a06ad89b23659fa881ae0649b867a9d55", size = 28926, upload-time = "2026-02-07T10:45:32.24Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "testcontainers"
|
||||||
|
version = "4.14.2"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "docker" },
|
||||||
|
{ name = "python-dotenv" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
{ name = "urllib3" },
|
||||||
|
{ name = "wrapt" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/ca/ac/a597c3a0e02b26cbed6dd07df68be1e57684766fd1c381dee9b170a99690/testcontainers-4.14.2.tar.gz", hash = "sha256:1340ccf16fe3acd9389a6c9e1d9ab21d9fe99a8afdf8165f89c3e69c1967d239", size = 166841, upload-time = "2026-03-18T05:19:16.696Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/13/2d/26b8b30067d94339afee62c3edc9b803a6eb9332f521ba77d8aaab5de873/testcontainers-4.14.2-py3-none-any.whl", hash = "sha256:0d0522c3cd8f8d9627cda41f7a6b51b639fa57bdc492923c045117933c668d68", size = 125712, upload-time = "2026-03-18T05:19:15.29Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "tiktoken"
|
name = "tiktoken"
|
||||||
version = "0.12.0"
|
version = "0.12.0"
|
||||||
|
|||||||
+11
-8
@@ -15,7 +15,7 @@
|
|||||||
# ============================================================================
|
# ============================================================================
|
||||||
# Bump this number when the config schema changes.
|
# Bump this number when the config schema changes.
|
||||||
# Run `make config-upgrade` to merge new fields into your local config.yaml.
|
# Run `make config-upgrade` to merge new fields into your local config.yaml.
|
||||||
config_version: 9
|
config_version: 10
|
||||||
|
|
||||||
# ============================================================================
|
# ============================================================================
|
||||||
# Logging
|
# Logging
|
||||||
@@ -878,16 +878,19 @@ skill_evolution:
|
|||||||
# NOTE: When both `checkpointer` and `database` are configured,
|
# NOTE: When both `checkpointer` and `database` are configured,
|
||||||
# `checkpointer` takes precedence for LangGraph state persistence.
|
# `checkpointer` takes precedence for LangGraph state persistence.
|
||||||
# If you use `database`, you can remove the `checkpointer` section.
|
# If you use `database`, you can remove the `checkpointer` section.
|
||||||
|
# Stage 0+ default is postgres (parity with production, room for future RLS).
|
||||||
|
# Set DATABASE_URL in .env. SQLite is preserved as an offline-dev fallback —
|
||||||
|
# uncomment the SQLite block below and comment out the Postgres block to use it.
|
||||||
|
#
|
||||||
|
# Postgres (default):
|
||||||
|
database:
|
||||||
|
backend: postgres
|
||||||
|
postgres_url: $DATABASE_URL
|
||||||
|
|
||||||
|
# SQLite fallback (offline dev):
|
||||||
# database:
|
# database:
|
||||||
# backend: sqlite
|
# backend: sqlite
|
||||||
# sqlite_dir: .deer-flow/data
|
# sqlite_dir: .deer-flow/data
|
||||||
#
|
|
||||||
# database:
|
|
||||||
# backend: postgres
|
|
||||||
# postgres_url: $DATABASE_URL
|
|
||||||
database:
|
|
||||||
backend: sqlite
|
|
||||||
sqlite_dir: .deer-flow/data
|
|
||||||
|
|
||||||
# ============================================================================
|
# ============================================================================
|
||||||
# Run Events Configuration
|
# Run Events Configuration
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
# Usage: docker-compose -f docker-compose-dev.yaml up --build
|
# Usage: docker-compose -f docker-compose-dev.yaml up --build
|
||||||
#
|
#
|
||||||
# Services:
|
# Services:
|
||||||
|
# - postgres: PostgreSQL 16 (port 5432) — Stage 0+ default DB backend
|
||||||
# - nginx: Reverse proxy (port 2026)
|
# - nginx: Reverse proxy (port 2026)
|
||||||
# - frontend: Frontend Next.js dev server (port 3000)
|
# - frontend: Frontend Next.js dev server (port 3000)
|
||||||
# - gateway: Backend Gateway API + agent runtime (port 8001)
|
# - gateway: Backend Gateway API + agent runtime (port 8001)
|
||||||
@@ -13,6 +14,30 @@
|
|||||||
# Access: http://localhost:2026
|
# Access: http://localhost:2026
|
||||||
|
|
||||||
services:
|
services:
|
||||||
|
# ── Database (Stage 0+ default backend) ────────────────────────────────
|
||||||
|
# Postgres for local dev. Production may point DATABASE_URL at a remote RDS;
|
||||||
|
# this service can then be omitted via `docker compose --profile <other>`.
|
||||||
|
postgres:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
container_name: deer-flow-postgres
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: ${POSTGRES_USER:-deerflow}
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-deerflow_dev}
|
||||||
|
POSTGRES_DB: ${POSTGRES_DB:-deerflow}
|
||||||
|
ports:
|
||||||
|
- "${POSTGRES_PORT:-5432}:5432"
|
||||||
|
volumes:
|
||||||
|
- postgres-data:/var/lib/postgresql/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-deerflow} -d ${POSTGRES_DB:-deerflow}"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
start_period: 10s
|
||||||
|
networks:
|
||||||
|
- deer-flow-dev
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
# ── Sandbox Provisioner ────────────────────────────────────────────────
|
# ── Sandbox Provisioner ────────────────────────────────────────────────
|
||||||
# Manages per-sandbox Pod + Service lifecycle in the host Kubernetes
|
# Manages per-sandbox Pod + Service lifecycle in the host Kubernetes
|
||||||
# cluster via the K8s API.
|
# cluster via the K8s API.
|
||||||
@@ -125,6 +150,9 @@ services:
|
|||||||
UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:0.7.20}
|
UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:0.7.20}
|
||||||
UV_INDEX_URL: ${UV_INDEX_URL:-https://pypi.org/simple}
|
UV_INDEX_URL: ${UV_INDEX_URL:-https://pypi.org/simple}
|
||||||
container_name: deer-flow-gateway
|
container_name: deer-flow-gateway
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
command: sh -c "{ cd backend && (uv sync || (echo '[startup] uv sync failed; recreating .venv and retrying once' && uv venv --allow-existing .venv && uv sync)) && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 --reload --reload-include='*.yaml .env'; } > /app/logs/gateway.log 2>&1"
|
command: sh -c "{ cd backend && (uv sync || (echo '[startup] uv sync failed; recreating .venv and retrying once' && uv venv --allow-existing .venv && uv sync)) && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001 --reload --reload-include='*.yaml .env'; } > /app/logs/gateway.log 2>&1"
|
||||||
volumes:
|
volumes:
|
||||||
- ../backend/:/app/backend/
|
- ../backend/:/app/backend/
|
||||||
@@ -178,6 +206,8 @@ volumes:
|
|||||||
# image build are not shadowed by the host backend/ directory mount.
|
# image build are not shadowed by the host backend/ directory mount.
|
||||||
gateway-venv:
|
gateway-venv:
|
||||||
gateway-uv-cache:
|
gateway-uv-cache:
|
||||||
|
# Persist Postgres data across container restarts.
|
||||||
|
postgres-data:
|
||||||
|
|
||||||
networks:
|
networks:
|
||||||
deer-flow-dev:
|
deer-flow-dev:
|
||||||
|
|||||||
@@ -0,0 +1,188 @@
|
|||||||
|
# DeerFlow 整体架构鸟瞰
|
||||||
|
|
||||||
|
按"自外向内、自顶向下"分层讲,并指出每一层对应的代码位置,方便后续深入。
|
||||||
|
|
||||||
|
## 一、进程与部署拓扑
|
||||||
|
|
||||||
|
DeerFlow 表面是 4 个端口,本质是 **3 个进程 + 1 个反向代理**。
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────────┐
|
||||||
|
浏览器 / IM ─────────▶│ nginx :2026 │ 统一入口
|
||||||
|
│ (含 CORS、SSE 透传) │
|
||||||
|
└──────────┬───────────┘
|
||||||
|
│
|
||||||
|
┌──────────────────┴──────────────────┐
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
┌────────────────────┐ ┌──────────────────────────┐
|
||||||
|
│ Frontend (Next.js) │ │ Gateway (uvicorn) │
|
||||||
|
│ :3000 │ │ :8001 │
|
||||||
|
│ pnpm dev / preview │ │ ┌──────────────────────┐ │
|
||||||
|
└────────────────────┘ │ │ FastAPI 路由层 │ │
|
||||||
|
│ │ /api/models, /skills │ │
|
||||||
|
│ │ /threads, /runs ... │ │
|
||||||
|
│ ├──────────────────────┤ │
|
||||||
|
│ │ LangGraph Runtime │ │
|
||||||
|
│ │ (RunManager, │ │
|
||||||
|
│ │ StreamBridge, │ │
|
||||||
|
│ │ Checkpointer) │ │
|
||||||
|
│ ├──────────────────────┤ │
|
||||||
|
│ │ lead_agent 图 │ │
|
||||||
|
│ │ + 18 个中间件 │ │
|
||||||
|
│ │ + Sandbox / Tools │ │
|
||||||
|
│ └──────────────────────┘ │
|
||||||
|
└────────────┬─────────────┘
|
||||||
|
│
|
||||||
|
┌──────────────────────────┼─────────────────────────┐
|
||||||
|
▼ ▼ ▼
|
||||||
|
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
||||||
|
│ Sandbox │ │ MCP Servers │ │ LLM 提供商 │
|
||||||
|
│ Local / AIO Docker│ │ (stdio/sse/http) │ │ OpenAI/Anthropic │
|
||||||
|
│ 提供 bash/fs │ │ │ │ /vLLM/Codex CLI │
|
||||||
|
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
关键事实(容易踩坑):
|
||||||
|
|
||||||
|
- **LangGraph 运行时不是独立进程**,而是嵌在 Gateway 这同一个 uvicorn 进程里。`scripts/serve.sh:225` 只起了一个后端进程:`uvicorn app.gateway.app:app`。
|
||||||
|
- nginx 把 `/api/langgraph/*` 重写成 `/api/*` 再代理到 Gateway(`docker/nginx/nginx.local.conf:49-73`),所以前端用的"标准 LangGraph SDK 协议"和 DeerFlow 自己的 REST 是同一个 8001 端口。
|
||||||
|
- 路由协议在 nginx 里特意为 SSE 关闭了缓冲(`proxy_buffering off; X-Accel-Buffering no`),否则流式响应会被吃掉。
|
||||||
|
|
||||||
|
## 二、后端代码二分:Harness vs App
|
||||||
|
|
||||||
|
后端最重要的边界,**决定新写代码该放哪里**。
|
||||||
|
|
||||||
|
```
|
||||||
|
backend/
|
||||||
|
├── packages/harness/deerflow/ ← 可发布的智能体框架(import: deerflow.*)
|
||||||
|
│ ├── agents/ lead_agent + memory + middlewares + ThreadState
|
||||||
|
│ ├── runtime/ checkpointer / runs / stream_bridge / events / store
|
||||||
|
│ ├── sandbox/ Sandbox 抽象 + local 实现 + 文件/bash 工具
|
||||||
|
│ ├── subagents/ 子代理注册表 + 后台执行池
|
||||||
|
│ ├── tools/ 内建工具(present_files / ask_clarification / view_image)
|
||||||
|
│ ├── mcp/ MultiServerMCPClient + 缓存 + OAuth
|
||||||
|
│ ├── skills/ SKILL.md 加载、工具白名单
|
||||||
|
│ ├── models/ 模型工厂、vLLM/Codex/Claude 自定义 provider
|
||||||
|
│ ├── community/ tavily / jina / firecrawl / aio_sandbox(可选实现)
|
||||||
|
│ ├── memory/ 长期记忆(事实抽取、debounce 队列)
|
||||||
|
│ ├── persistence/ SQLAlchemy 模型(用户、运行、事件、反馈)
|
||||||
|
│ ├── guardrails/ 工具调用前置鉴权(可插拔 provider)
|
||||||
|
│ ├── tracing/ LangSmith / Langfuse callback
|
||||||
|
│ ├── reflection/ "module:variable" 字符串 → 实例(配置驱动的关键)
|
||||||
|
│ ├── uploads/ 上传文件转换 markdown
|
||||||
|
│ └── client.py DeerFlowClient(嵌入式 Python 客户端)
|
||||||
|
│
|
||||||
|
└── app/ ← 应用代码(import: app.*)
|
||||||
|
├── gateway/
|
||||||
|
│ ├── app.py FastAPI 入口 + lifespan
|
||||||
|
│ ├── auth_middleware.py 会话/Token 鉴权
|
||||||
|
│ ├── csrf_middleware.py 双重 cookie CSRF
|
||||||
|
│ ├── langgraph_auth.py 注入到 langgraph.json 的鉴权钩子
|
||||||
|
│ └── routers/ ↓ 下表
|
||||||
|
└── channels/ IM(Slack/Telegram/Feishu/DingTalk/微信/企微)
|
||||||
|
```
|
||||||
|
|
||||||
|
**铁律:app 可以 import deerflow,deerflow 不能 import app**(CI 用 `tests/test_harness_boundary.py` 强制)。这意味着 harness 必须自给自足——任何"agent 运行时需要的能力"都要在 harness 里完成抽象,app 层只做 HTTP/IM 适配。
|
||||||
|
|
||||||
|
## 三、Gateway 路由总览
|
||||||
|
|
||||||
|
`backend/app/gateway/routers/` 14 个路由文件,分三类职责:
|
||||||
|
|
||||||
|
| 类别 | 路由 | 干什么 |
|
||||||
|
|---|---|---|
|
||||||
|
| **配置/资源管理** | `models` `skills` `mcp` `memory` `agents` | 列出/启停 LLM 模型、技能、MCP、自定义 agent |
|
||||||
|
| **会话/数据** | `threads` `uploads` `artifacts` `suggestions` | 管理线程、上传文件、产物下载、追问建议 |
|
||||||
|
| **运行(核心)** | `thread_runs` `runs` `feedback` `assistants_compat` | 创建运行、SSE 流、消息分页、反馈打分、LangGraph 兼容协议 |
|
||||||
|
| **横切** | `auth` `channels` | 用户登录注册、IM 渠道状态 |
|
||||||
|
|
||||||
|
`assistants_compat.py` 是关键:它把前端用的 LangGraph SDK 协议(`POST /threads/{id}/runs/stream`、`messages-tuple` 流模式等)翻译成 DeerFlow 内部的 `RunManager` 调用——这就是 nginx 那条 `/api/langgraph/*` 重写规则的接收端。
|
||||||
|
|
||||||
|
## 四、一次对话的完整生命周期
|
||||||
|
|
||||||
|
把上面所有零件串起来——用户在前端输入一句话,会发生这些事:
|
||||||
|
|
||||||
|
```
|
||||||
|
1. 前端 useThreadStream hook
|
||||||
|
└─▶ LangGraph SDK 调用 POST /api/langgraph/threads/{id}/runs/stream
|
||||||
|
(stream_mode=["values","messages-tuple","custom"])
|
||||||
|
|
||||||
|
2. nginx 重写 → /api/threads/{id}/runs/stream → Gateway
|
||||||
|
|
||||||
|
3. Gateway thread_runs 路由
|
||||||
|
├─▶ AuthMiddleware 解析 session → user_id 注入到 user_context(contextvar)
|
||||||
|
├─▶ CSRFMiddleware 校验
|
||||||
|
└─▶ runtime.RunManager 创建 Run → 落库(runs / run_events 表)
|
||||||
|
|
||||||
|
4. RunManager 调用 lead_agent 图(langgraph.json: deerflow.agents:make_lead_agent)
|
||||||
|
├─▶ 解析 configurable: model_name / thinking_enabled / is_plan_mode / subagent_enabled
|
||||||
|
├─▶ create_chat_model() 实例化 LLM(reflection 从 "module:Class" 字符串实例化)
|
||||||
|
└─▶ create_agent(model, tools, middlewares, state_schema=ThreadState)
|
||||||
|
|
||||||
|
5. 18 个中间件按顺序拦截每一轮 model→tool→model:
|
||||||
|
ThreadDataMiddleware 创建 .deer-flow/users/{uid}/threads/{tid}/...
|
||||||
|
UploadsMiddleware 注入新上传文件
|
||||||
|
SandboxMiddleware acquire 沙箱,state.sandbox_id 写入
|
||||||
|
DanglingToolCall 修复中断的 tool_call 序列
|
||||||
|
LLMErrorHandling LLM 报错降级
|
||||||
|
Guardrail 工具调用前鉴权(可选)
|
||||||
|
SandboxAudit 记录 bash/fs 操作
|
||||||
|
ToolErrorHandling tool 异常 → ToolMessage 不中断
|
||||||
|
Summarization token 接近上限时压缩历史
|
||||||
|
TodoList plan_mode 才挂
|
||||||
|
TokenUsage 累计 token
|
||||||
|
Title 首轮后自动起标题
|
||||||
|
Memory 队列异步抽取记忆
|
||||||
|
ViewImage 视觉模型注入 base64
|
||||||
|
DeferredToolFilter 需要时才暴露 tool schema
|
||||||
|
SubagentLimit 限制 task 并发到 3
|
||||||
|
LoopDetection 检测重复工具循环
|
||||||
|
Clarification ask_clarification 触发 interrupt(END)
|
||||||
|
|
||||||
|
6. Tools 由 get_available_tools() 拼装:
|
||||||
|
├─ Sandbox 工具:bash / ls / read_file / write_file / str_replace
|
||||||
|
├─ 内建工具:present_files / ask_clarification / view_image / setup_agent
|
||||||
|
├─ MCP 工具:从 extensions_config.json 启用的 server 拉取
|
||||||
|
├─ Community 工具:tavily / jina / firecrawl / image_search(按 config.yaml)
|
||||||
|
└─ task 工具(可选):派遣 subagent
|
||||||
|
|
||||||
|
7. StreamBridge 把图执行的事件流转换成 SSE:
|
||||||
|
- "values" 完整状态快照
|
||||||
|
- "messages-tuple" 增量 token / 工具调用 / 工具返回
|
||||||
|
- "custom" StreamWriter 自定义事件
|
||||||
|
- "end" 收尾,附 token usage
|
||||||
|
|
||||||
|
8. 前端 LangGraph SDK 接 SSE,按 message id 累加 delta,更新 UI
|
||||||
|
|
||||||
|
9. 运行结束后,MemoryMiddleware 后台 30s debounce 抽取记忆事实写入
|
||||||
|
.deer-flow/users/{uid}/memory.json
|
||||||
|
```
|
||||||
|
|
||||||
|
## 五、状态与持久化的几条线
|
||||||
|
|
||||||
|
DeerFlow 的状态被有意拆成"快/慢/历史"三层,因为它要同时支持长会话、跨进程恢复、文件级产物:
|
||||||
|
|
||||||
|
| 状态 | 位置 | 谁写 |
|
||||||
|
|---|---|---|
|
||||||
|
| **会话状态(messages, todos, artifacts)** | LangGraph checkpointer(内置 SQLite/PG,路径在 `runtime/checkpointer/async_provider.py`) | 每个 step 自动 |
|
||||||
|
| **运行元数据/事件流** | `persistence/` 下 SQLAlchemy 模型(`runs`、`run_events`、`feedback`、`threads_meta`) | RunManager + StreamBridge |
|
||||||
|
| **每用户每线程文件** | `.deer-flow/users/{uid}/threads/{tid}/user-data/{workspace,uploads,outputs}` | ThreadDataMiddleware + 沙箱工具 |
|
||||||
|
| **长期记忆** | `.deer-flow/users/{uid}/memory.json`(可叠加 per-agent) | MemoryMiddleware(异步) |
|
||||||
|
| **配置** | `config.yaml`(模型、工具、沙箱、记忆…) + `extensions_config.json`(MCP、技能开关) | `make setup` 或 Gateway PUT |
|
||||||
|
|
||||||
|
agent 看到的永远是 **虚拟路径** `/mnt/user-data/...` 和 `/mnt/skills/...`,由 `sandbox/tools.py` 的 `replace_virtual_path()` 翻译成上面物理路径。这层抽象让"本地沙箱"和"Docker 沙箱"对 agent 完全透明。
|
||||||
|
|
||||||
|
## 六、前端架构(一行总结)
|
||||||
|
|
||||||
|
`frontend/src/core/threads/hooks.ts` 里的 `useThreadStream` / `useSubmitThread` / `useThreads` 是整个前端的"主动脉"——它们包了 LangGraph SDK 单例(`core/api/`),所有 UI 组件订阅 thread 状态做渲染。Server Components 默认,需要交互的才 `"use client"`。`core/` 下其它子目录(artifacts/skills/mcp/memory/settings)都是为这条主动脉提供周边能力。
|
||||||
|
|
||||||
|
## 七、一图记住"它在做什么"
|
||||||
|
|
||||||
|
DeerFlow 本质上是一个 **"LangGraph 智能体 + 18 段切面 + 沙箱 + 记忆"** 的组合:
|
||||||
|
|
||||||
|
- **LangGraph** 提供图执行、checkpoint、stream 协议
|
||||||
|
- **18 个中间件** 是 DeerFlow 自己加的"切面层",每个解决一个具体的健壮性/能力问题(错误恢复、上下文压缩、记忆、子代理限流……)
|
||||||
|
- **沙箱+技能+MCP+工具** 是 agent 的"手脚"
|
||||||
|
- **Gateway + IM Channels + 嵌入式 Client** 是同一个 agent 的三种暴露方式(HTTP/聊天/Python 直调)
|
||||||
|
|
||||||
|
> 想继续往里钻的话,建议下一步选三个之一:(a)走读 lead_agent + 中间件链,理解 agent 一轮 think/act 的完整代码路径;(b)走读 sandbox + tools,理解虚拟路径和工具拼装;(c)走读 runtime + StreamBridge,理解 SSE 协议怎么映射回 LangGraph SDK。
|
||||||
@@ -0,0 +1,239 @@
|
|||||||
|
# ADR-001 · 数据隔离模型
|
||||||
|
|
||||||
|
| 项目 | 内容 |
|
||||||
|
|---|---|
|
||||||
|
| 状态 | 草稿(Draft) · 2026-05-09 据 spike 结果修订 §4.1.1 / §4.1.2 / §4.2 |
|
||||||
|
| 决策日期 | TBD |
|
||||||
|
| 决策者 | CTO + 架构 + 后端 lead |
|
||||||
|
| 关联 ADR | ADR-004 租户层级、ADR-005 存储拓扑、ADR-006 运行时与渠道 |
|
||||||
|
| 关联 spike / 审计 | [adr-vs-code-audit](./adr-vs-code-audit.zh-CN.md) · [langgraph-postgres spike](./adr-spike-langgraph-postgres.zh-CN.md) |
|
||||||
|
| 代码命名 | 本 ADR 写 `tenant_id`,落代码统一读作 `workspace_id`(详 [workspace-schema-design §1](./workspace-schema-design.zh-CN.md#1-命名约定--workspace-vs-tenant)) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 背景
|
||||||
|
|
||||||
|
DeerFlow 当前是 **"多用户单租户"** 模型:所有用户的会话、运行、记忆、产物都共用同一套表,仓储层用 `user_id` WHERE 过滤做个人空间隔离(`runtime/user_context.py:138-167`)。这套机制设计良好——repository 用 ContextVar + `AUTO` 哨兵自动注入当前用户,下层不依赖上层(harness 不能 import app)。
|
||||||
|
|
||||||
|
多租户化需要在 `user_id` 之上再加一层 `tenant_id`。问题是:**用什么物理隔离强度?**
|
||||||
|
|
||||||
|
三种主流方案:
|
||||||
|
|
||||||
|
| 维度 | 行级(tenant_id WHERE) | per-tenant schema | per-tenant DB |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 实现成本 | 低 | 中 | 高 |
|
||||||
|
| 跨租户 bug 爆炸半径 | 高 | 中 | 极低 |
|
||||||
|
| 备份/恢复粒度 | 全量 | 按 schema | 按 DB |
|
||||||
|
| 合规友好度(SOC2/HIPAA) | 一般 | 好 | 最好 |
|
||||||
|
| 跨租户分析查询 | 容易 | 中 | 难 |
|
||||||
|
| 升级 schema | 一次完成 | 要遍历所有 schema | 要遍历所有 DB |
|
||||||
|
| 适用客户规模 | <10k 租户 | 10k–100 大客户 | <100 大客户 |
|
||||||
|
| 运维复杂度 | 低 | 中 | 高 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 决策
|
||||||
|
|
||||||
|
**采用 行级 `tenant_id` + Postgres Row-Level Security(RLS)** 作为双保险。
|
||||||
|
|
||||||
|
理由:
|
||||||
|
|
||||||
|
1. **DeerFlow 仓储层现状几乎平行扩展**——已经有 `resolve_user_id()` 哨兵模式,把 `tenant_id` 按同样模式补一遍,改造面集中、风险可控。
|
||||||
|
2. **Postgres RLS 是 DB 层兜底**——即使应用层有 bug 漏写 `WHERE tenant_id = ...`,DB 也会强制过滤,第二道防线。
|
||||||
|
3. **覆盖目标客户规模**:B2B 中小客户为主、租户数 1k–10k,行级方案足够。
|
||||||
|
4. **不放弃跨租户分析能力**:平台需要做用量统计、监控、健康检查,单库行级最方便。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 备选方案与拒绝理由
|
||||||
|
|
||||||
|
### A. per-tenant schema(同库不同 schema)
|
||||||
|
|
||||||
|
**拒绝。** 看似比行级更隔离,实际坑很多:
|
||||||
|
|
||||||
|
- **schema 数量爆炸**:1000 个租户 = 1000 个 schema × 每张表,pg_class 体积膨胀,连接池里 search_path 切换有性能抖动
|
||||||
|
- **schema migration 痛苦**:每发布一次 schema 改动要遍历所有 schema 跑 migration,失败回滚极复杂
|
||||||
|
- **跨租户查询难**:要写 `UNION ALL` 跨所有 schema,运营仪表盘几乎无法实现
|
||||||
|
- **依然需要应用层过滤**:连接进哪个 schema 仍由应用层决定,没真正消除"应用层 bug 跨租户"
|
||||||
|
|
||||||
|
### B. per-tenant database(独立物理库)
|
||||||
|
|
||||||
|
**拒绝(默认场景)。** 隔离最强但成本极高:
|
||||||
|
|
||||||
|
- **运维负担**:1000 个 DB = 1000 套备份/恢复/监控/连接池
|
||||||
|
- **冷启动延迟**:每个租户新建 DB 时间从秒级飙到分钟级
|
||||||
|
- **跨租户操作不可能**:平台级查询、聚合、迁移全部失效
|
||||||
|
- **连接池复杂度爆炸**:每租户独立连接池或者共用动态切库,都是噩梦
|
||||||
|
|
||||||
|
**仅在两种情况切换到此方案:** ① 拿到强合规客户(金融/医疗/政府),合同里写明物理数据隔离;② 客户付费足够覆盖每租户独立 DB 的运维成本(典型企业级订阅)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 落地影响
|
||||||
|
|
||||||
|
### 4.1 表结构改造
|
||||||
|
|
||||||
|
所有业务表加 `tenant_id` 列 + 复合索引(`tenant_id` 作为前导列):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
ALTER TABLE threads_meta ADD COLUMN tenant_id UUID NOT NULL;
|
||||||
|
CREATE INDEX idx_threads_meta_tenant_user ON threads_meta (tenant_id, user_id, updated_at DESC);
|
||||||
|
|
||||||
|
ALTER TABLE runs ADD COLUMN tenant_id UUID NOT NULL;
|
||||||
|
CREATE INDEX idx_runs_tenant_created ON runs (tenant_id, created_at DESC);
|
||||||
|
|
||||||
|
ALTER TABLE run_events ADD COLUMN tenant_id UUID NOT NULL;
|
||||||
|
CREATE INDEX idx_run_events_tenant_run ON run_events (tenant_id, run_id, seq);
|
||||||
|
|
||||||
|
ALTER TABLE feedback ADD COLUMN tenant_id UUID NOT NULL;
|
||||||
|
CREATE INDEX idx_feedback_tenant_run ON feedback (tenant_id, run_id);
|
||||||
|
|
||||||
|
-- ADR-005 引入的新表也要带 tenant_id(建表时就有)
|
||||||
|
-- agent_configs, memory_facts, memory_context,
|
||||||
|
-- tenant_skill_state, tenant_mcp_configs, tenant_secrets, tenant_quotas
|
||||||
|
```
|
||||||
|
|
||||||
|
**关键索引原则**:每个 `tenant_id` 都必须是复合索引的**第一列**——RLS policy 走的就是这条路径,前导列错了 RLS 会全表扫。
|
||||||
|
|
||||||
|
#### 4.1.1 LangGraph 自有表(checkpoints / checkpoint_writes / checkpoint_blobs / checkpoint_migrations)
|
||||||
|
|
||||||
|
`runtime/checkpointer/async_provider.py` 用的是 LangGraph 内置 `AsyncPostgresSaver`,**表结构不在 DeerFlow 控制下**。原稿讨论过两条路(subquery RLS / 列升级),spike([adr-spike-langgraph-postgres](./adr-spike-langgraph-postgres.zh-CN.md))验证后我们改用 **两层隔离模型**:
|
||||||
|
|
||||||
|
| 表归属 | 隔离机制 | 防线性质 |
|
||||||
|
|---|---|---|
|
||||||
|
| **DeerFlow 自有表**(threads_meta、runs、run_events、feedback、users、tenant_*) | RLS + `SET LOCAL app.tenant_id` via SQLAlchemy session(DeerFlow 完全控制 conn pool) | DB 强约束 |
|
||||||
|
| **LangGraph checkpoint 表** | **应用层强校验**——入口路由在调 LangGraph 前必查 `threads_meta` 上的 `(tenant_id, thread_id)` 归属 | 应用层强约束 + 表 unique constraint 兜底 |
|
||||||
|
|
||||||
|
**为何对 LangGraph 表放弃 RLS**:
|
||||||
|
|
||||||
|
- `langgraph-checkpoint-postgres==3.0.5` **不存在 `connection_factory` 参数**(spike §2 实测);其连接池注入路径只有 `__init__(conn=AsyncConnectionPool)` 这一个口子,且 `psycopg_pool` 自带的 `configure` callback 只在物理连接首次创建时跑——拿不到运行期 ContextVar 里的 tenant_id
|
||||||
|
- 子类化 `AsyncConnectionPool` 重写 `getconn` 注入 `SET app.tenant_id`/`RESET` 是可行的 hack,但侵入 psycopg-pool 内部,库升级风险高(spike §3.1)
|
||||||
|
- 给 LangGraph 表 ALTER 加 `tenant_id` 列同样不可取——LangGraph 用 `MIGRATIONS` 数组管理 schema,每次升级都要 diff 防漏(spike §3.4)
|
||||||
|
|
||||||
|
**LangGraph 表的安全模型**(接受的 trade-off):
|
||||||
|
|
||||||
|
- 安全等级从"DB 强约束"降级为"应用层强约束 + 表 unique constraint"
|
||||||
|
- 强约束点是 **`threads_meta` 表上的 `UNIQUE (tenant_id, thread_id)` 复合索引** + 入口路由的强校验:任何代码路径要写 LangGraph 表前必须先在 `threads_meta` 找到对应行,且行的 `tenant_id` 与当前 ContextVar 一致
|
||||||
|
- CI 加 boundary 测试,禁止任何路径绕过 `threads.py` / `thread_runs.py` 直连 LangGraph saver(包括 LangGraph Studio 必须走相同入口或显式审批)
|
||||||
|
- 平台 admin 路径走 `BYPASSRLS` role 时同样必须经过应用层 audit,不直接跳过 thread 归属检查
|
||||||
|
|
||||||
|
**未来可升级路径**(不阻塞 phase-0):若上游接受 PR 加入 `connection_factory`,可平滑切回"DeerFlow 表 + LangGraph 表统一 RLS"模型。
|
||||||
|
|
||||||
|
#### 4.1.2 第一道防线:thread_id ↔ tenant_id 校验
|
||||||
|
|
||||||
|
LangGraph 调用入口在 **`app/gateway/routers/threads.py`**(thread CRUD)和 **`app/gateway/routers/thread_runs.py`**(run 创建/恢复/事件流)。这两个路由在调用 LangGraph 之前**必须先用 `threads_meta` 校验 `(tenant_id, thread_id)` 归属**:
|
||||||
|
|
||||||
|
- **创建路径**:先在 `threads_meta` 写入 `(tenant_id=current, thread_id, user_id=current)`,依赖 `UNIQUE (tenant_id, thread_id)` 防重;再调 LangGraph 创建对应 thread
|
||||||
|
- **读/写路径**:先用 `(current_tenant_id, requested_thread_id)` SELECT `threads_meta`,未命中即 404;命中后才允许调 LangGraph
|
||||||
|
|
||||||
|
> 注:原稿写"在 `AssistantsCompat` 路由强制校验"是错的——`assistants_compat.py:1-50` 只服务 `assistants.search/get` 静态 stub,**不**触达 thread 入口(审计报告 §ADR-001 已修正)。
|
||||||
|
|
||||||
|
应用层校验是第一道防线、`UNIQUE` 约束是 DB 层兜底——任何对 LangGraph 表的访问都经过这一关。
|
||||||
|
|
||||||
|
### 4.2 RLS policy 模板
|
||||||
|
|
||||||
|
所有带 `tenant_id` 的表都加同样形态的 policy:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
ALTER TABLE threads_meta ENABLE ROW LEVEL SECURITY;
|
||||||
|
ALTER TABLE threads_meta FORCE ROW LEVEL SECURITY; -- 即使表所有者也走 policy
|
||||||
|
|
||||||
|
CREATE POLICY tenant_isolation ON threads_meta
|
||||||
|
USING (tenant_id = current_setting('app.tenant_id', true)::uuid)
|
||||||
|
WITH CHECK (tenant_id = current_setting('app.tenant_id', true)::uuid);
|
||||||
|
```
|
||||||
|
|
||||||
|
应用层在每次拿到连接时,先 `SET LOCAL app.tenant_id = '<uuid>'`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# packages/harness/deerflow/persistence/engine.py
|
||||||
|
async def _set_session_tenant(session: AsyncSession, tenant_id: str) -> None:
|
||||||
|
"""Bind session to tenant; RLS policy enforces filter."""
|
||||||
|
await session.execute(text("SET LOCAL app.tenant_id = :tid"), {"tid": tenant_id})
|
||||||
|
```
|
||||||
|
|
||||||
|
每个仓储方法的开头自动调用,从 ContextVar 取 tenant_id(仿照现有 `resolve_user_id` 模式)。
|
||||||
|
|
||||||
|
**LangGraph 自有连接池不参与 SET LOCAL**:DeerFlow 仓储和 LangGraph checkpointer 是**两套连接池**——前者是 SQLAlchemy `AsyncSession`(DeerFlow 控制),后者是 LangGraph 自己持有的 psycopg 池(DeerFlow 不可控)。按 §4.1.1 的两层模型:
|
||||||
|
|
||||||
|
- **DeerFlow 自有表**:上面 `_set_session_tenant` helper 在每次仓储调用前注入 `SET LOCAL`,RLS 兜底
|
||||||
|
- **LangGraph 表**:**不注入 `SET LOCAL`**——LangGraph 表上不启用 RLS,租户隔离靠应用层强校验(§4.1.2)实现。`AsyncPostgresSaver` 仍按现状用 `from_conn_string`,无侵入
|
||||||
|
|
||||||
|
> 原稿设想的"自定义 `connection_factory` 注入到 saver 构造"在 `langgraph-checkpoint-postgres==3.0.5` 不可行——库不存在该参数([spike](./adr-spike-langgraph-postgres.zh-CN.md) §2.2/2.4)。详细备选方案与拒绝理由见 spike §3。
|
||||||
|
|
||||||
|
**关键约束**:所有"会查 DeerFlow 自有表"的代码路径都必须保证调用栈上已注入 `app.tenant_id`,否则 RLS 会把整个会话过滤成空集。CI 加冒烟测试确认这点。LangGraph 表上的访问则必须经过 §4.1.2 的入口校验。
|
||||||
|
|
||||||
|
### 4.3 ContextVar 扩展
|
||||||
|
|
||||||
|
在 `runtime/user_context.py` 旁边加 `tenant_context.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
_current_tenant: Final[ContextVar[CurrentTenant | None]] = ContextVar("deerflow_current_tenant", default=None)
|
||||||
|
|
||||||
|
class _AutoSentinel: ... # 同 user_id 模式
|
||||||
|
AUTO: Final[_AutoSentinel] = _AutoSentinel()
|
||||||
|
|
||||||
|
def resolve_tenant_id(value, *, method_name) -> str:
|
||||||
|
"""与 resolve_user_id 同款三态:AUTO / 显式 str / 显式 None。
|
||||||
|
SaaS 模式下 None 是 forbidden(除非显式 admin override)。"""
|
||||||
|
```
|
||||||
|
|
||||||
|
`AuthMiddleware` 在解析完 JWT 后两个 ContextVar 同时注入。
|
||||||
|
|
||||||
|
### 4.4 SQLite → Postgres 迁移
|
||||||
|
|
||||||
|
**SQLite 不支持 RLS**,多租户上线必须切 Postgres。迁移路径:
|
||||||
|
|
||||||
|
1. 第 0 阶段:在 dev 环境同时跑 SQLite 和 Postgres,用 `aiosqlite` / `asyncpg` 双驱动
|
||||||
|
2. 第 1 阶段:生产切 Postgres,老数据用 `pg_loader` 导入;DeerFlow 现有的 `Base.metadata.create_all()` 直接接 Postgres
|
||||||
|
3. 老用户的 `user_id` 在没有 tenant_id 时归到一个"legacy_tenant",迁移脚本同步把 `tenant_id` 回填
|
||||||
|
|
||||||
|
### 4.5 跨租户操作(平台后台)
|
||||||
|
|
||||||
|
平台 admin / 运维需要跨租户查询时,不能简单"绕过 RLS"——而是用一个**专用 role** 配 `BYPASSRLS`,只给受限运维账号使用,操作审计入库:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE ROLE deerflow_admin BYPASSRLS;
|
||||||
|
-- 应用层 admin 路由用这个 role 的连接池,并强制审计日志
|
||||||
|
```
|
||||||
|
|
||||||
|
**绝不允许应用主连接池有 `BYPASSRLS`。**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 风险与缓解
|
||||||
|
|
||||||
|
| 风险 | 缓解 |
|
||||||
|
|---|---|
|
||||||
|
| 应用层漏写 tenant_id WHERE | RLS 是兜底(DeerFlow 表);CI 加静态检查(detect SQL 不带 tenant_id) |
|
||||||
|
| **LangGraph 表无 RLS,仅应用层强约束**(§4.1.1 trade-off) | `threads_meta` `UNIQUE (tenant_id, thread_id)` 兜底;CI boundary 测试禁止绕过 `threads.py` / `thread_runs.py` 直连 saver;定期审计任何新增的 LangGraph 直连路径 |
|
||||||
|
| 索引前导列错了走全表扫 | DBA 评审所有 EXPLAIN;上线前压测 |
|
||||||
|
| `current_setting('app.tenant_id')` 没设导致 RLS 全过滤掉 | 应用层 fail-closed;监控空集查询率 |
|
||||||
|
| 跨租户分析需求多 | 提供受控的 admin role + 审计日志 |
|
||||||
|
| SQLite 开发 vs Postgres 生产差异 | 测试集成层用 testcontainers 跑 Postgres;不允许用 SQLite 跑 RLS 相关测试 |
|
||||||
|
| **当前不存在 Postgres 测试夹具基础设施**(审计报告 §ADR-001 highest-risk gap) | phase-0 必须先落 testcontainers + RLS 冒烟测试,再做仓储改造;否则 RLS bug 进生产 |
|
||||||
|
| 单 DB 容量上限(>1TB 后维护困难) | 监控 DB 体积;超过阈值切 per-tenant DB(推翻方案) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 推翻条件
|
||||||
|
|
||||||
|
切换到 **per-tenant DB** 当且仅当:
|
||||||
|
|
||||||
|
1. 拿到强合规客户(金融/医疗/政府),合同要求物理数据隔离
|
||||||
|
2. 单 DB 容量 / 写 TPS 触顶,垂直扩展不经济
|
||||||
|
3. 出现一次跨租户数据泄露事故,董事会要求最强隔离
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 默认假设
|
||||||
|
|
||||||
|
| 项 | 默认 |
|
||||||
|
|---|---|
|
||||||
|
| 数据库 | PostgreSQL 16+ |
|
||||||
|
| RLS 启用 | 所有带 `tenant_id` 的业务表 |
|
||||||
|
| 主键 | UUID v7(时间排序) |
|
||||||
|
| 索引前导列 | `tenant_id` |
|
||||||
|
| Connection pool | 每应用进程 20–50 conns,PgBouncer transaction mode |
|
||||||
|
| 备份 | 每日全量 + WAL streaming,保留 30 天 |
|
||||||
|
| 跨租户查询 | 仅通过 `deerflow_admin` role + 审计 |
|
||||||
@@ -0,0 +1,372 @@
|
|||||||
|
# ADR-002 · 沙箱隔离模型
|
||||||
|
|
||||||
|
| 项目 | 内容 |
|
||||||
|
|---|---|
|
||||||
|
| 状态 | 草稿(Draft) |
|
||||||
|
| 决策日期 | TBD |
|
||||||
|
| 决策者 | 安全 + 架构 + SRE |
|
||||||
|
| 关联 ADR | ADR-001 数据隔离、ADR-005 存储拓扑 |
|
||||||
|
| 关联审计 | [adr-vs-code-audit](./adr-vs-code-audit.zh-CN.md) — 注意:现有 `AioSandboxProvider` 出网/资源/cosign 缺位;K8sSandboxProvider 几乎从零开工(实际工作量大于本 ADR §5 估算) |
|
||||||
|
| 代码命名 | 本 ADR 写 `tenant_id` / `tenant-{tenant_id}` namespace,落代码统一读作 `workspace_id` / `ws-{workspace_id}`(详 [workspace-schema-design §1](./workspace-schema-design.zh-CN.md#1-命名约定--workspace-vs-tenant)) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. 概念前提
|
||||||
|
|
||||||
|
本 ADR 反复出现 **K8s Namespace + gVisor/Kata 运行时 + NetworkPolicy** 三件套——它们在不同层把租户的代码运行环境关起来,缺一不可。先用一段话讲清楚是什么、防什么、不防什么,再读后面的决策细节会顺很多。
|
||||||
|
|
||||||
|
### 0.1 K8s Namespace —— 资源/视图隔离
|
||||||
|
|
||||||
|
Kubernetes 的逻辑分区。一个集群里跑多租户,每租户分一个 namespace(如 `tenant-acme`、`tenant-bigco`),其中的 Pod / Service / Secret / ConfigMap 互相看不见。配套:
|
||||||
|
|
||||||
|
- **ResourceQuota**:限制 namespace 总用量(CPU、内存、Pod 数、存储)
|
||||||
|
- **LimitRange**:单 Pod 兜底(默认 request/limit、单 Pod 上限)
|
||||||
|
- **RBAC**:把租户管理员权限只绑到自己的 namespace
|
||||||
|
|
||||||
|
⚠️ **不是安全边界**。namespace 只让你"看不到",不是"碰不到"。两租户的 Pod 若都跑在默认 `runc` 上、共享同一个 Linux 内核,**任何一个内核 0day 都能让 A 容器逃逸到宿主,进而看到 B 容器**。所以需要下一层。
|
||||||
|
|
||||||
|
> 类比:办公楼里不同公司的门禁卡。同事进不了别人的工位,但墙不会自己变厚。
|
||||||
|
|
||||||
|
### 0.2 gVisor / Kata —— 内核级隔离
|
||||||
|
|
||||||
|
把容器从宿主内核上"再隔一层"。DeerFlow 沙箱要跑租户上传的任意 Python 代码,必须比 runc 更硬。
|
||||||
|
|
||||||
|
**gVisor(Google)**
|
||||||
|
- 用户态实现一个沙箱内核(Sentry),拦截容器所有 syscall 自己模拟,再用极少几个 syscall 跟真内核打交道
|
||||||
|
- 攻击面:先攻破 Sentry,再攻破真内核——多一道
|
||||||
|
- 代价:每个 syscall 中转,IO 密集型 workload 慢 ~10-30%
|
||||||
|
- 集成:Pod spec 写 `runtimeClassName: gvisor`
|
||||||
|
|
||||||
|
**Kata Containers**
|
||||||
|
- 给每个 Pod 起一个轻量虚拟机(QEMU 或 Firecracker 后端),容器跑在 VM 内独立内核里
|
||||||
|
- 隔离强度 ≈ 真 VM,启动几百毫秒
|
||||||
|
- 代价:每 Pod 多占 ~50-150 MB 内存、冷启动比 gVisor 慢一点
|
||||||
|
- 集成:`runtimeClassName: kata-qemu` / `kata-fc`
|
||||||
|
|
||||||
|
> 本 ADR 取舍:默认 **gVisor**(性价比平衡),监管/付费档位切 **Kata-Firecracker**(接近 VM 强度,单独定价)。配套 Cosign 镜像签名 + 只读根文件系统 + drop ALL caps 是纵深防御。
|
||||||
|
|
||||||
|
### 0.3 NetworkPolicy —— 出/入流量白名单
|
||||||
|
|
||||||
|
K8s 原生防火墙,按 namespace / Pod label 控制谁能跟谁通信。**默认拒绝 + 显式放行**是标准姿势:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
spec:
|
||||||
|
podSelector: {} # 命中 namespace 内所有 Pod
|
||||||
|
policyTypes: [Egress]
|
||||||
|
egress: [] # 空白名单 = 全部禁止
|
||||||
|
```
|
||||||
|
|
||||||
|
为什么对 DeerFlow 是关键——租户代码可能尝试访问:
|
||||||
|
|
||||||
|
| 目标 | 风险 |
|
||||||
|
|---|---|
|
||||||
|
| `169.254.169.254`(云元数据) | 偷 IAM 凭据、节点 token,直接拿下集群 |
|
||||||
|
| 平台内网 DB / Redis | 横向打到其他租户的数据 |
|
||||||
|
| 其他租户的 namespace IP | 跨租户监听/嗅探 |
|
||||||
|
| 互联网 C2 服务器 | 数据外泄、挖矿、僵尸网络 |
|
||||||
|
|
||||||
|
默认全部拒绝后,仅放行:DNS(CoreDNS)+ 出口走 **Egress Gateway**(Envoy/Squid 做域名白名单,允许 `api.openai.com`、`pypi.org` 等,拒绝其余)。
|
||||||
|
|
||||||
|
> 执行靠 CNI 插件(Calico / Cilium)。Cilium 还支持 L7 策略(如"允许 GET /v1/chat/completions、禁 POST /admin"),是更强的备选。
|
||||||
|
|
||||||
|
### 0.4 三件套合起来看
|
||||||
|
|
||||||
|
```
|
||||||
|
租户的 Python 代码
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌────────────────────────────────────────┐
|
||||||
|
│ Pod (tenant-acme namespace) │ ← K8s Namespace:逻辑隔离 + 配额
|
||||||
|
│ ├─ runtimeClassName: gvisor │ ← gVisor:内核级攻击面隔离
|
||||||
|
│ ├─ readOnlyRootFilesystem │
|
||||||
|
│ └─ capabilities.drop: ["ALL"] │
|
||||||
|
└────────────────────────────────────────┘
|
||||||
|
│ egress
|
||||||
|
▼
|
||||||
|
┌────────────────────────────────────────┐
|
||||||
|
│ NetworkPolicy: default-deny │ ← NetworkPolicy:网络层白名单
|
||||||
|
│ → 仅允许 Egress Gateway / DNS │
|
||||||
|
└────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Egress Gateway(域名白名单)
|
||||||
|
```
|
||||||
|
|
||||||
|
- 没有 namespace:租户互相能看见对方的资源对象
|
||||||
|
- 没有 gVisor:一个内核 0day 全集群陪葬
|
||||||
|
- 没有 NetworkPolicy:租户代码 `curl 169.254.169.254` 就能拿走节点凭据
|
||||||
|
|
||||||
|
三层都套上,才是本 ADR 想要的"对抗任意租户代码"的最低防御姿势。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 背景
|
||||||
|
|
||||||
|
> **分期落地提示**:本 ADR 描述的 K8s + gVisor + NetworkPolicy 全套架构是**目标态**。按 [phased-rollout-by-scale](../02-rollout/phased-rollout-by-scale.zh-CN.md) 实际落地节奏:
|
||||||
|
> - **Stage 1**:仅做 §3 威胁模型的"出网默认禁 + cgroup CPU/memory 限额",落到现有 `AioSandboxProvider` 上(轻量补丁版)。**不上 K8s**。
|
||||||
|
> - **Stage 3**:才换 `K8sSandboxProvider`,引入 namespace + gVisor + NetworkPolicy + Pod Security Standard 全套(§5 全文落地)。
|
||||||
|
> - **Stage 4 / premium**:按合同切 Kata-Firecracker 或独立 nodepool。
|
||||||
|
>
|
||||||
|
> 读 §2~§9 时把它当作"Stage 3 完成态",Stage 1 落地时只摘 §3 出网/资源那两行就好。
|
||||||
|
|
||||||
|
沙箱是多租户里**爆炸半径最大**的组件:客户的 agent 可以跑任意 bash 命令、读写文件、调用 MCP 工具。如果隔离不够强,一个客户能:
|
||||||
|
|
||||||
|
- **读到其他客户的数据**(容器逃逸 / 共享卷误用)
|
||||||
|
- **薅云元数据**(`curl http://169.254.169.254/...` 偷 IAM 凭证)
|
||||||
|
- **横向移动**(同 namespace 的其他容器、同节点的 hostNetwork)
|
||||||
|
- **耗尽资源**(fork bomb、无限循环、磁盘填满)
|
||||||
|
|
||||||
|
DeerFlow 现状有两个 sandbox provider:
|
||||||
|
|
||||||
|
| Provider | 强度 | 多租户可用 |
|
||||||
|
|---|---|---|
|
||||||
|
| `LocalSandboxProvider` | bash/fs **直接落主机**,零隔离 | ❌ 绝对不能用 |
|
||||||
|
| `AioSandboxProvider` | Docker 容器(社区实现,`packages/harness/deerflow/community/aio_sandbox/`) | ⚠️ 当前配置不够 |
|
||||||
|
|
||||||
|
`AioSandboxProvider` 起点不错(每 thread 一个容器、虚拟路径翻译已有),但默认配置缺少多租户必需的几条隔离:默认出网未禁、CPU/内存 limits 未强制、根文件系统未只读、镜像未签名校验。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 决策
|
||||||
|
|
||||||
|
**采用 K8s + 强隔离运行时(gVisor 或 Kata Containers)+ NetworkPolicy 默认禁出网 + per-tenant Namespace。**
|
||||||
|
|
||||||
|
| 层 | 作用 |
|
||||||
|
|---|---|
|
||||||
|
| **K8s Namespace per tenant** | 资源逻辑隔离;NetworkPolicy 起效边界 |
|
||||||
|
| **gVisor (runsc) 运行时** | 用户态系统调用拦截,容器逃逸到宿主难度大幅提升;性能损失 ~5-15%(多数 agent 任务可接受) |
|
||||||
|
| **NetworkPolicy 默认 DENY** | 出网白名单:只允许到 LLM endpoint、配置的 MCP servers、搜索 API |
|
||||||
|
| **ResourceQuota + LimitRange** | per-namespace CPU/内存上限;单 pod CPU/内存上限 |
|
||||||
|
| **Pod Security Standard: restricted** | 禁 root、禁 privileged、只读根文件系统、drop ALL caps |
|
||||||
|
| **emptyDir 临时卷** | 数据靠 ADR-005 同步对象存储,pod 销毁即清 |
|
||||||
|
| **镜像签名校验**(Cosign) | 启动 pod 前验证 sandbox 镜像签名,防供应链攻击 |
|
||||||
|
|
||||||
|
**Premium 客户档位**:在此之上再加 per-tenant **物理节点池** + **Firecracker microVM**(kata-fc),把"租户 X 的沙箱永远不和别人共享物理节点"做到 SLA 里。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 威胁模型
|
||||||
|
|
||||||
|
| 攻击场景 | 共享 Docker(现状) | per-tenant K8s NS + gVisor(决策) | per-tenant Firecracker |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 容器逃逸到宿主 | 全员沦陷 | 单租户沦陷(gVisor 大幅降低成功率) | 单租户沦陷(VM 边界,逃逸难度极高) |
|
||||||
|
| 容器间横向移动(同节点) | 可行 | NetworkPolicy 禁止 + namespace 隔离 | 不可能(独立 VM) |
|
||||||
|
| 出网到云元数据 169.254.169.254 | 默认可行 | NetworkPolicy 禁 + IMDSv2 强制 token | 同左 |
|
||||||
|
| 出网到内部服务(DB / 内网) | 可行 | NetworkPolicy 禁 + egress gateway 白名单 | 同左 |
|
||||||
|
| 侧信道(CPU 缓存 / Spectre) | 可行 | 减弱(gVisor 隔离系统调用,但 CPU 共享仍有风险) | 显著减弱(独立 VM、独立 vCPU) |
|
||||||
|
| 资源耗尽(fork bomb / OOM) | 影响同节点全部容器 | LimitRange 强制 cgroup;超限 OOMKill | VM 内独立调度 |
|
||||||
|
| 持久化攻击(写定时任务) | 可写主机 cron | 只读根文件系统 + ephemeral pod 重建即清 | 同左 |
|
||||||
|
| 提权 | 看 Docker 配置(默认 root) | restricted PSS 禁 root、禁 capabilities | 同左 |
|
||||||
|
| 镜像被替换(供应链) | 不校验 | Cosign 验证签名 + admission controller 拦截 | 同左 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 备选方案与拒绝理由
|
||||||
|
|
||||||
|
### A. 共享 Docker(保留 AioSandboxProvider 当前模式)
|
||||||
|
|
||||||
|
**拒绝。** 即使加固到极致,根本问题仍在:
|
||||||
|
|
||||||
|
- 所有租户共享 dockerd / containerd,一次容器逃逸 = 全员沦陷
|
||||||
|
- Docker 的 NetworkPolicy 等价物(user-defined network)粒度粗
|
||||||
|
- 资源限制依赖 cgroup v1/v2 配置一致,运维难统一
|
||||||
|
|
||||||
|
### B. per-tenant Firecracker microVM(默认)
|
||||||
|
|
||||||
|
**拒绝(作为默认)。** 隔离最强但成本最高:
|
||||||
|
|
||||||
|
- 冷启动比 K8s pod 慢 2-5×(500ms vs 几秒)
|
||||||
|
- 运维需要专门团队(kata-fc / cloud-hypervisor 都不是开箱即用)
|
||||||
|
- AWS / 阿里云的部分托管 K8s 不直接支持 Firecracker,需要自建 nodepool
|
||||||
|
|
||||||
|
**保留作为 premium 档位**:把 Firecracker 当付费 SLA 卖给监管类客户。
|
||||||
|
|
||||||
|
### C. 共享 K8s Namespace + 仅靠 NetworkPolicy + cgroup
|
||||||
|
|
||||||
|
**拒绝。** namespace 不分隔意味着:
|
||||||
|
|
||||||
|
- pod-to-pod 通信默认开放(NetworkPolicy 是白名单制,漏一条就全开)
|
||||||
|
- ServiceAccount 共享,越权读 secret 风险大
|
||||||
|
- ResourceQuota 是 namespace 级别,没法精细分配到租户
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 落地影响
|
||||||
|
|
||||||
|
### 5.1 新建 `K8sSandboxProvider`
|
||||||
|
|
||||||
|
```python
|
||||||
|
# packages/harness/deerflow/sandbox/k8s/provider.py
|
||||||
|
class K8sSandboxProvider(SandboxProvider):
|
||||||
|
"""每 thread 一个 Pod,按 tenant 落到对应 Namespace。
|
||||||
|
|
||||||
|
生命周期:
|
||||||
|
acquire(thread_id) → 创建 Pod(gVisor runtime, restricted PSS, NetworkPolicy 已挂)
|
||||||
|
get(sandbox_id) → 返回与运行中 Pod 通信的客户端(kubectl exec / WebSocket)
|
||||||
|
release(sandbox_id) → delete Pod(emptyDir 自动回收)
|
||||||
|
"""
|
||||||
|
```
|
||||||
|
|
||||||
|
替换 `LocalSandboxProvider`(开发/测试用)和 `AioSandboxProvider`(保留作为单机部署 fallback)。
|
||||||
|
|
||||||
|
### 5.2 K8s 资源(每租户 Namespace 一份)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# tenant onboarding 时自动渲染、apply
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Namespace
|
||||||
|
metadata:
|
||||||
|
name: tenant-{tenant_id}
|
||||||
|
labels:
|
||||||
|
pod-security.kubernetes.io/enforce: restricted
|
||||||
|
deerflow.io/tenant-id: {tenant_id}
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ResourceQuota
|
||||||
|
metadata:
|
||||||
|
namespace: tenant-{tenant_id}
|
||||||
|
spec:
|
||||||
|
hard:
|
||||||
|
cpu: "16"
|
||||||
|
memory: 32Gi
|
||||||
|
pods: "20"
|
||||||
|
requests.storage: 100Gi
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: LimitRange
|
||||||
|
metadata:
|
||||||
|
namespace: tenant-{tenant_id}
|
||||||
|
spec:
|
||||||
|
limits:
|
||||||
|
- type: Container
|
||||||
|
default: { cpu: "500m", memory: 1Gi }
|
||||||
|
defaultRequest: { cpu: "100m", memory: 256Mi }
|
||||||
|
max: { cpu: "2", memory: 4Gi }
|
||||||
|
---
|
||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: NetworkPolicy
|
||||||
|
metadata:
|
||||||
|
name: deny-all-egress
|
||||||
|
namespace: tenant-{tenant_id}
|
||||||
|
spec:
|
||||||
|
podSelector: {}
|
||||||
|
policyTypes: [Egress]
|
||||||
|
egress: # 仅允许下面这些
|
||||||
|
- to:
|
||||||
|
- namespaceSelector: { matchLabels: { name: kube-system } }
|
||||||
|
podSelector: { matchLabels: { k8s-app: kube-dns } }
|
||||||
|
ports: [{ protocol: UDP, port: 53 }]
|
||||||
|
- to: # 通过 egress gateway 出外网
|
||||||
|
- podSelector: { matchLabels: { app: deerflow-egress-gateway } }
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 Egress gateway
|
||||||
|
|
||||||
|
放一个集中的出口代理(envoy / squid)做:
|
||||||
|
|
||||||
|
- LLM endpoint 白名单(OpenAI / Anthropic / vLLM 内网)
|
||||||
|
- MCP server 白名单(per-tenant 启用列表)
|
||||||
|
- 搜索 API 白名单(Tavily / Jina / Brave / DuckDuckGo)
|
||||||
|
- **黑名单**:169.254.169.254(cloud metadata)、10.0.0.0/8 / 172.16.0.0/12 / 192.168.0.0/16(内部网络,除非白名单)
|
||||||
|
- 全量审计(按租户记录每次出网)
|
||||||
|
|
||||||
|
### 5.4 Pod Spec 关键字段
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
spec:
|
||||||
|
runtimeClassName: gvisor # 或 kata-fc(premium)
|
||||||
|
automountServiceAccountToken: false # 沙箱不该有 SA token
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
runAsUser: 65532
|
||||||
|
seccompProfile:
|
||||||
|
type: RuntimeDefault
|
||||||
|
containers:
|
||||||
|
- name: sandbox
|
||||||
|
image: registry.deerflow.io/sandbox:v2.3.4@sha256:... # 强制 digest pin
|
||||||
|
securityContext:
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
readOnlyRootFilesystem: true
|
||||||
|
capabilities: { drop: [ALL] }
|
||||||
|
volumeMounts:
|
||||||
|
- { name: workspace, mountPath: /mnt/user-data }
|
||||||
|
- { name: skills, mountPath: /mnt/skills, readOnly: true }
|
||||||
|
volumes:
|
||||||
|
- { name: workspace, emptyDir: { sizeLimit: 10Gi } }
|
||||||
|
- { name: skills, emptyDir: { sizeLimit: 5Gi } } # 启动时从 S3 拉
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.5 镜像签名
|
||||||
|
|
||||||
|
- CI 用 cosign sign 给 sandbox 镜像签名
|
||||||
|
- K8s admission controller(policy-controller / Kyverno)启动 pod 前验证 cosign signature
|
||||||
|
- 拒绝任何未签名 / 签名不匹配的镜像
|
||||||
|
|
||||||
|
### 5.6 SandboxAuditMiddleware 增强
|
||||||
|
|
||||||
|
现有 `SandboxAuditMiddleware` 记录了工具调用,多租户必须:
|
||||||
|
|
||||||
|
- 每条审计带 `(tenant_id, user_id, thread_id, tool_name, args_hash)`
|
||||||
|
- 异步推到独立审计存储(不与业务库共用)
|
||||||
|
- 保留至少 90 天(合规要求常见值)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 性能影响
|
||||||
|
|
||||||
|
| 指标 | 估计 |
|
||||||
|
|---|---|
|
||||||
|
| Pod 冷启动(gVisor + 镜像 pre-warm) | 1.5-3s |
|
||||||
|
| Pod 冷启动(Firecracker) | 3-8s |
|
||||||
|
| Bash 命令延迟(vs 宿主) | gVisor +5-15%;Firecracker +10-30% |
|
||||||
|
| 文件 I/O(小文件) | gVisor 显著慢(系统调用拦截);用 emptyDir tmpfs 缓解 |
|
||||||
|
| 网络(egress gateway) | +1-3ms 单跳 |
|
||||||
|
|
||||||
|
**冷启动是最敏感的指标**——靠两条路径优化:
|
||||||
|
|
||||||
|
1. **Pod prewarm**:每个 namespace 维护 N 个空闲 pod 池(`PodReadinessProbe` 通过即可服用,按需绑定 thread)
|
||||||
|
2. **镜像层缓存**:每节点预拉镜像(DaemonSet image-puller)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 风险与缓解
|
||||||
|
|
||||||
|
| 风险 | 缓解 |
|
||||||
|
|---|---|
|
||||||
|
| gVisor 与某些 syscall 不兼容(agent bash 跑不动某些工具) | sandbox 镜像里预装常用工具;CI 跑兼容性测试集 |
|
||||||
|
| 出网白名单维护负担 | Per-tenant MCP/搜索配置自动生成 NetworkPolicy;运维 admin UI 一键加 |
|
||||||
|
| Firecracker 运维复杂 | 仅作为 premium 档位,不强制全量上 |
|
||||||
|
| 节点 noisy neighbor(CPU 共享导致侧信道) | 高敏租户走专属 nodepool(taints/tolerations) |
|
||||||
|
| Pod prewarm 池资源浪费 | 按租户活跃度动态调节池大小;闲置超过阈值缩到 0 |
|
||||||
|
| 镜像供应链攻击 | Cosign 强制 + SBOM + 漏洞扫描 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. 推翻条件
|
||||||
|
|
||||||
|
切换到 **per-tenant Firecracker(默认)** 当且仅当:
|
||||||
|
|
||||||
|
1. 实测 gVisor 在某条关键 syscall 上有不可绕过的兼容性问题(且 sandbox 镜像无法预装替代品)
|
||||||
|
2. 拿到合同要求"强物理隔离"的监管客户,付费档位要求覆盖运维成本
|
||||||
|
3. 出现一次容器逃逸 PoC 影响多租户
|
||||||
|
|
||||||
|
切换到 **共享 Docker(极端简化)** 当且仅当:
|
||||||
|
|
||||||
|
- 公司决定退回单租户产品形态——多租户上线后基本不应回退
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. 默认假设
|
||||||
|
|
||||||
|
| 项 | 默认 |
|
||||||
|
|---|---|
|
||||||
|
| 集群 | EKS / ACK / GKE,K8s 1.29+ |
|
||||||
|
| 沙箱 runtime | gVisor (runsc) |
|
||||||
|
| Premium runtime | Kata Containers + Firecracker |
|
||||||
|
| Pod 隔离粒度 | per-thread(不复用) |
|
||||||
|
| 冷启动 SLO | P50 < 2s, P99 < 5s |
|
||||||
|
| Pod CPU/Mem 上限 | 2 CPU / 4 GiB(单 pod) |
|
||||||
|
| Namespace 配额 | 16 CPU / 32 GiB / 20 pods(按 plan 调) |
|
||||||
|
| Egress 白名单数量 | <30 个域名 / 租户 |
|
||||||
|
| 审计保留期 | 90 天热 + 1 年冷 |
|
||||||
|
| 镜像签名 | cosign + Kyverno 强制 |
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user