20 Commits
Author SHA1 Message Date
jiang d320972d57 fix(docker): pin GeoServer platform manifest 2026-10-22 13:03:26 +08:00
jiang b4f3204489 chore(docker): advance PostGIS 14 patch line 2026-10-22 12:30:30 +08:00
jiang 1e71f8953b test(server): align service access delegation expectation 2026-10-22 12:18:12 +08:00
jiang f3818b3fed fix(docker): include runtime samples in image 2026-10-22 12:14:40 +08:00
jiang 024abf07a8 chore(docker): pin secure runtime and resolve MCP dependencies 2026-10-22 12:08:36 +08:00
jiang be6317eb48 feat(server): integrate MCP queries and investigation messaging
Add project-bound OAuth MCP queries and controlled analysis with scoped evidence,
limits, timeouts and a nearby-model spatial query using EPSG:4326 inputs.
Default nearby searches to 200 meters and five results.

Integrate investigation messaging/presentation, health/readiness and typed history;
remove duplicate web search and fix online simulation and compressed baseline writes.
Include project migrations 023-025; keep local credentials and demo seeding excluded.

Validation: 319 focused pytest checks; real PostGIS temporary-table regression;
read-only Jinqian bridge radius comparison; cross-language MCP discovery/calls.
2026-10-19 18:46:58 +08:00
jiang 2b6fc5aa52 feat(tasks): define investigation report contract and history 2026-10-10 16:02:46 +08:00
jiang 4a63fcd7ff fix(workbench): serialize events and target work order sync 2026-10-08 15:41:49 +08:00
jiang b70283c2ff feat(work-orders): synchronize peripheral changes through events 2026-10-08 11:50:15 +08:00
jiang afcbd52a5a refactor(tasks): centralize access request lookup 2026-09-24 17:11:21 +08:00
jiang 540d73add8 docs(workbench): clarify active notification counting 2026-09-24 17:00:41 +08:00
jiang ad813a7c16 feat(workbench): add dynamic task approval lifecycle 2026-09-24 16:36:12 +08:00
jiang db092b0f5e feat(workbench): keep only actionable notifications 2026-09-24 15:18:18 +08:00
jiang e374b5b3cb fix(workbench): remove synthetic check projections 2026-09-24 12:01:01 +08:00
jiang 1a6ff64bbe feat(db): add workbench schema comments 2026-09-24 10:33:12 +08:00
jiang 91afe76c6d docs(workbench): gate database rollout by project 2026-09-24 10:13:52 +08:00
jiang ed06bc2bfc fix(workbench): configure psycopg snapshot transaction
Open the psycopg transaction without unsupported SQLAlchemy-style options, then set the PostgreSQL isolation and read-only characteristics before the first query. Cover the exact snapshot endpoint path with a regression test.
2026-09-24 09:49:01 +08:00
jiang 9335bdc1ea feat(workbench): persist operational workflows and events 2026-09-24 04:19:02 +08:00
jiang baff7dfe9e chore(api): refresh workbench contracts 2026-09-22 10:44:43 +08:00
jiang 7ab9b36a5e feat(workbench): add runtime GIS and sample integrations 2026-09-22 10:44:31 +08:00
220 changed files with 46430 additions and 528 deletions
+46 -9
View File
@@ -29,6 +29,8 @@ TIMESCALEDB_DB_HOST="localhost"
TIMESCALEDB_DB_PORT="5433"
TIMESCALEDB_DB_USER="tjwater"
TIMESCALEDB_DB_PASSWORD="password"
# 实时快照替换事务的解压预算;仅事务内生效,禁止0(无限制)。
TIMESCALEDB_REALTIME_MAX_DECOMPRESSED_TUPLES=1000000
# ============================================
# 元数据数据库配置 (Metadata DB)
@@ -74,17 +76,52 @@ KEYCLOAK_ALGORITHM=RS256
KEYCLOAK_AUDIENCE="account"
KEYCLOAK_ACCESS_TOKEN_MAX_AGE_SECONDS=900
# ============================================
# 工作台事件同步
# ============================================
# 认证接口必须使用明确来源,逗号分隔;不要配置为 *。
CORS_ALLOWED_ORIGINS="http://127.0.0.1:5173,http://localhost:5173"
WORKBENCH_EVENTS_ENABLED=true
WORKBENCH_EVENT_RETENTION_DAYS=7
# 每个项目启动一个 worker 实例。长期运行优先使用 OAuth 客户端凭据。
OPERATIONAL_WORKER_PROJECT_ID=""
OPERATIONAL_WORKER_SERVER_URL="http://127.0.0.1:8000"
OPERATIONAL_WORKER_AGENT_URL="http://127.0.0.1:8787"
OPERATIONAL_WORKER_RECONCILE_SECONDS=30
OPERATIONAL_WORKER_MAX_CONCURRENCY=4
OPERATIONAL_WORKER_TOKEN_URL=""
OPERATIONAL_WORKER_CLIENT_ID=""
OPERATIONAL_WORKER_CLIENT_SECRET=""
OPERATIONAL_WORKER_TOKEN_REFRESH_SECONDS=300
OPERATIONAL_DATA_DELEGATION_BEARER=""
# ============================================
# Bocha Web Search API
# Tianditu shared API key and service URLs
# ============================================
BOCHA_API_KEY="sk-your-bocha-api-key"
BOCHA_WEB_SEARCH_URL="https://api.bochaai.com/v1/web-search"
BOCHA_WEB_SEARCH_TIMEOUT_SECONDS=30
# ============================================
# Tianditu Geocoding API
# ============================================
TIANDITU_GEOCODER_TOKEN="your-tianditu-geocoder-token"
TIANDITU_TOKEN="your-tianditu-token"
TIANDITU_GEOCODER_URL="https://api.tianditu.gov.cn/geocoder"
TIANDITU_GEOCODER_TIMEOUT_SECONDS=30
# 地名搜索与地理编码共用 TIANDITU_TOKEN;也可沿用 TJWATER_TIANDITU_TOKEN。
TIANDITU_SEARCH_URL="https://api.tianditu.gov.cn/v2/search"
TIANDITU_SEARCH_TIMEOUT_SECONDS=15
# MCP query resource, off until OAuth and every project BizDB are provisioned.
# 仅 ENVIRONMENT=development 时覆盖 worker 取令牌地址;留空沿用 OPERATIONAL_WORKER_TOKEN_URL。
OPERATIONAL_WORKER_DEV_TOKEN_URL=
TJWATER_MCP_ENABLED=false
TJWATER_MCP_ISSUER_URI=
# 仅 ENVIRONMENT=development,可填写与 issuer 完全相同的内网 HTTP IPv4 issuer。
# 正式环境必须留空;此例外不放行远程 HTTP MCP resource。
TJWATER_MCP_DEV_HTTP_ISSUER_URI=
TJWATER_MCP_RESOURCE_URI=
TJWATER_MCP_AUDIENCE=tjwater-mcp
TJWATER_MCP_HOST_CLIENT_ID=tjwater-agent-mcp
TJWATER_MCP_ALLOWED_HOSTS=["localhost:*","127.0.0.1:*"]
TJWATER_MCP_ALLOWED_ORIGINS=[]
# 普通查询预算;计算工具由 Agent 用 timeout_seconds 按次指定(默认120秒)。
TJWATER_MCP_QUERY_TIMEOUT_SECONDS=30
# 计算等待上限,默认1小时;管理员可配置120至86400秒,实际上限通过 tools/list 发布。
TJWATER_MCP_MAX_CALCULATION_TIMEOUT_SECONDS=3600
+9 -4
View File
@@ -1,13 +1,16 @@
FROM condaforge/miniforge3:latest AS base
FROM condaforge/miniforge3:26.7.2-0@sha256:569afc65b5338d35e276adf9a860025cd37a677659d19dcb71fc1d4c2063a94e AS base
WORKDIR /app
ENV CONDA_ENV_PATH=/opt/server \
PATH="/opt/server/bin:${PATH}"
ENV PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple \
PIP_TRUSTED_HOST=pypi.tuna.tsinghua.edu.cn \
UV_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
# 安装 Python 3.12 和 pymetis (通过 conda-forge 避免编译问题)
RUN mamba install -y python=3.12 pymetis && \
# 在独立环境中安装 Python 3.12 和 pymetis,避免修改 Miniforge 基础环境。
RUN mamba create -y -p "$CONDA_ENV_PATH" python=3.12 pymetis pip && \
mamba clean -afy
COPY requirements.txt .
@@ -20,6 +23,8 @@ COPY app ./app
COPY contracts ./contracts
COPY infra ./infra
COPY resources ./resources
COPY scripts/run_operational_worker.py scripts/sync_peripheral_work_orders.py ./scripts/
COPY samples/peripheral ./samples/peripheral
RUN python -c "from pathlib import Path; from zipfile import ZipFile; model_dir = Path('app/algorithms/pipe_health_prediction/model'); zip_path = model_dir / 'my_survival_forest_model_quxi.zip'; joblib_name = 'my_survival_forest_model_quxi.joblib'; joblib_path = model_dir / joblib_name; assert zip_path.exists(), f'Model archive not found: {zip_path}'; archive = ZipFile(zip_path); archive.extract(joblib_name, model_dir); archive.close(); assert joblib_path.exists(), f'Model file not extracted: {joblib_path}'" && \
rm -f app/algorithms/pipe_health_prediction/model/my_survival_forest_model_quxi.zip
RUN mkdir -p db_inp temp data inp
@@ -33,7 +38,7 @@ COPY tests ./tests
RUN python -m compileall -q app && \
python -c "import app.main" && \
python scripts/check_openapi.py && \
pytest -q tests
pytest -q --import-mode=importlib tests
FROM base AS runner
+34
View File
@@ -43,8 +43,18 @@ conda run -n server uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
conda activate server
```
`GET /health` 是无需鉴权的 HTTP 服务存活检查,返回 `200` 和 `{"status":"ok"}`,不查询数据库。
`GET /ready` 检查应用启动状态、元数据库 `SELECT 1` 及已启用的 MCP 传输生命周期;
全部就绪返回 `200`,否则返回 `503`,元数据库探测预算为2秒。MCP 关闭时标记为
`disabled`,不阻止就绪。返回结果不包含数据库地址、凭据或原始异常,禁止缓存。
就绪状态不等同于 OAuth、用户权限或真实工具调用成功;指定项目的业务库和时序库仍通过
带身份与 `X-Project-Id` 的 `GET /api/v1/projects/current/database-health` 检查。
## 常用命令
外围接入样板是独立进程,复用 PostgreSQL,不自动建表;启动、授权边界和契约见 [外围样板说明](docs/peripheral-sample.md)。
外围样板提供 synthetic 测试工单和测试数据的版本化 CRUD;工单经同步器投影为 `workbench.operational_tasks` 中的外围来源任务。固定演示时窗和造数方式见 [合成样本说明](docs/peripheral-generated-data.md),运行态读取边界见 [工作台运行态说明](docs/workbench-runtime.md)。这些数据不代表现场事实,也不会触发设备控制。
```bash
conda run -n server python -m pytest tests -q
conda run -n server python scripts/run_server.py
@@ -73,6 +83,16 @@ docker compose -f infra/docker/docker-compose.yml config
- `iot_data` DSN 用于 TimescaleDB,始终使用元数据配置的完整 DSN,不再从项目代码推导数据库名。
- 元数据、业务库和 TimescaleDB 可以部署在同一主机,也可以分别部署。
在线重算会原子替换一个时刻的节点、管段基线。历史分块已经压缩时,写入事务使用
`TIMESCALEDB_REALTIME_MAX_DECOMPRESSED_TUPLES` 设置解压预算,默认100万条;配置必须
为正整数,不允许0。该设置仅在实时快照写入事务内生效,不改变数据库全局设置或普通读取。
预算按需根据受影响压缩批次的规模调整,超过预算仍会失败并回滚当前快照;此前完成的其他
时刻不会一起回滚。同步、异步及单表批量快照入口使用同一预算。
压缩写入回归位于 `tests/integration/test_realtime_compression.py`;仅在设置
`REALTIME_COMPRESSION_TEST_DSN` 时执行,并且只接受库名以 `tjw_test_realtime_` 开头的
独立可丢弃数据库。测试会创建和删除该库内的 `realtime` schema,禁止指向项目数据库。
完整新建供水项目使用 `POST /api/v1/admin/project-provisions`,以
`multipart/form-data` 同时提交 `name`、小写 `code`、可选的
`description`、`gs_workspace`、`map_zoom` 和 INP `file`。工作流会按顺序完成:
@@ -87,6 +107,16 @@ docker compose -f infra/docker/docker-compose.yml config
使用模板复制或临时方案库的模拟功能时,`biz_data` 账号必须具备数据库创建和删除权限;只有显式删除项目时才会终止该项目的现有数据库会话,普通复制不会主动中断复制源会话。
奉贤项目的原始 GIS 与水质监测点使用独立脚本维护。导入脚本读取两份交付目录并写入项目 BizDB;水质脚本调用现有灵敏度分析服务生成 50 个监测点;发布脚本创建 10 个项目图层并刷新 GeoServer 缓存:
```bash
conda run -n server python scripts/import_fengxian_gis.py --database tjwater_v2
conda run -n server python scripts/generate_fengxian_water_quality_points.py --project tjwater_v2
conda run -n server python scripts/publish_fengxian_map_layers.py --database tjwater_v2 --workspace tjwater_next
```
发布命令需要配置 `GEOSERVER_URL`、`GEOSERVER_USERNAME` 和 `GEOSERVER_PASSWORD`。其中 7 个基础管网图层仍由项目供应流程创建,奉贤发布脚本追加 9 个原始 GIS 图层和 1 个水质监测点图层。
## 测试与发布
提交前根据改动范围运行最小有效测试:
@@ -106,3 +136,7 @@ Gitea 包工作流位于 `.gitea/workflows/package.yml`,通常由 tag 触发
## 安全规则
不要提交 `.env`、客户数据、数据库 dump、日志、生成缓存、`db_inp/`、`temp/`、`data/` 或本地密钥。CI/CD 凭据应放在 Gitea secrets 和仓库变量中。
## 产品 MCP 查询
Server 可选启用 MCP 2026-07-28 的25项项目查询及10项受控分析,默认关闭。它使用专用 OAuth resource Bearer、宿主绑定上下文及现有项目/任务权限,调查查询返回前持久化证据。启用需更新依赖、应用项目 SQL 迁移和配置 Keycloak token exchange;这些生产操作尚未执行。详见 [实施与启用](docs/mcp-query-implementation.md) 和 [协议/旧 CLI 能力对照](docs/mcp-query-design.md)。
+58
View File
@@ -0,0 +1,58 @@
"""Public process probes, separate from authenticated project diagnostics."""
import asyncio
from fastapi import APIRouter, Request
from fastapi.responses import JSONResponse
from sqlalchemy import text
from app.infra.db.metadb.database import engine
router = APIRouter(include_in_schema=False)
METADATA_TIMEOUT_SECONDS = 2
async def metadata_database_ready() -> bool:
try:
async with asyncio.timeout(METADATA_TIMEOUT_SECONDS):
async with engine.connect() as connection:
result = await connection.execute(text("SELECT 1"))
return result.scalar_one() == 1
except Exception:
# Public probes must not expose connection strings or driver errors.
return False
@router.get("/health")
async def health() -> dict[str, str]:
"""Check HTTP liveness without project or database dependencies."""
return {"status": "ok"}
@router.get("/ready")
async def readiness(request: Request) -> JSONResponse:
"""Check startup, metadata connectivity and the enabled MCP lifecycle."""
started = getattr(request.app.state, "startup_complete", False)
runtime = getattr(request.app.state, "mcp_runtime", None)
mcp_ready = runtime is None or runtime.ready
metadata_ready = await metadata_database_ready() if started else False
ready = started and metadata_ready and mcp_ready
return JSONResponse(
status_code=200 if ready else 503,
content={
"status": "ready" if ready else "not_ready",
"checks": {
"startup": "ready" if started else "not_ready",
"metadata_database": (
"not_checked" if not started
else "ready" if metadata_ready else "unavailable"
),
"mcp_transport": (
"disabled" if runtime is None
else "ready" if mcp_ready else "not_ready"
),
},
},
headers={"Cache-Control": "no-store"},
)
+47 -5
View File
@@ -22,6 +22,10 @@ class ProblemDetails(BaseModel):
code: str
trace_id: str
errors: list[dict[str, Any]] = Field(default_factory=list)
retryable: bool = False
related_refs: list[Any] = Field(default_factory=list)
recovery_action: str = "none"
retry_after_seconds: int | None = None
def _trace_id(request: Request) -> str:
@@ -36,6 +40,10 @@ def _problem_response(
detail: str,
code: str,
errors: list[dict[str, Any]] | None = None,
retryable: bool = False,
related_refs: list[Any] | None = None,
recovery_action: str = "none",
retry_after_seconds: int | None = None,
) -> JSONResponse:
problem = ProblemDetails(
type=f"https://tjwater.example/problems/{code.replace('_', '-')}",
@@ -46,6 +54,10 @@ def _problem_response(
code=code,
trace_id=_trace_id(request),
errors=errors or [],
retryable=retryable,
related_refs=related_refs or [],
recovery_action=recovery_action,
retry_after_seconds=retry_after_seconds,
)
return JSONResponse(
status_code=status_code,
@@ -89,7 +101,6 @@ def install_problem_details_handlers(app: FastAPI) -> None:
@app.exception_handler(HTTPException)
async def http_error_handler(request: Request, exc: HTTPException) -> JSONResponse:
detail = exc.detail if isinstance(exc.detail, str) else str(exc.detail)
code_by_status = {
401: "unauthenticated",
403: "forbidden",
@@ -98,12 +109,43 @@ def install_problem_details_handlers(app: FastAPI) -> None:
422: "validation_error",
503: "dependency_unavailable",
}
fallback_code = code_by_status.get(exc.status_code, "request_error")
detail_payload = exc.detail if isinstance(exc.detail, dict) else None
code = (
detail_payload.get("code")
if detail_payload and isinstance(detail_payload.get("code"), str)
else fallback_code
)
detail = exc.detail if isinstance(exc.detail, str) else None
if detail_payload:
candidate = detail_payload.get("message") or detail_payload.get("detail")
if isinstance(candidate, str):
detail = candidate
if detail is None:
detail = code.replace("_", " ").title()
related_refs = detail_payload.get("related_refs", []) if detail_payload else []
if not isinstance(related_refs, list):
related_refs = []
recovery_action = (
detail_payload.get("recovery_action", "none") if detail_payload else "none"
)
if not isinstance(recovery_action, str):
recovery_action = "none"
retry_after_seconds = (
detail_payload.get("retry_after_seconds") if detail_payload else None
)
if not isinstance(retry_after_seconds, int):
retry_after_seconds = None
return _problem_response(
request,
status_code=exc.status_code,
title=code_by_status.get(exc.status_code, "request_error")
.replace("_", " ")
.title(),
title=fallback_code.replace("_", " ").title(),
detail=detail,
code=code_by_status.get(exc.status_code, "request_error"),
code=code,
retryable=(
bool(detail_payload.get("retryable", False)) if detail_payload else False
),
related_refs=related_refs,
recovery_action=recovery_action,
retry_after_seconds=retry_after_seconds,
)
+101
View File
@@ -0,0 +1,101 @@
"""Actor-private messaging APIs; the Agent verifies its session ownership."""
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query
from psycopg import AsyncConnection
from app.auth.permissions import OPERATIONAL_TASK_VIEW, OPERATIONAL_TASK_WRITE, require_permission
from app.auth.project_dependencies import ProjectContext, get_project_pg_connection
from app.domain.domain_errors import DomainError
from app.domain.schemas.agent_messages import (
AgentMessage, AgentMessageCreate, AgentMessageDelivery, AgentMessagePage, AgentMessageReply,
)
from app.infra.db.postgresql.agent_messages import AgentMessageRepository
from app.services.agent_messages import AgentMessageService
router = APIRouter(prefix="/agent-messages")
def _service(ctx: ProjectContext, conn: AsyncConnection) -> AgentMessageService:
return AgentMessageService(AgentMessageRepository(conn, ctx.project_id), ctx)
def _raise_domain(exc: DomainError):
raise HTTPException(status_code=exc.status, detail=exc.payload()) from exc
@router.post("", response_model=AgentMessage, status_code=201)
async def create_agent_message(
command: AgentMessageCreate,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).create(command)
except DomainError as exc:
_raise_domain(exc)
@router.get("", response_model=AgentMessagePage)
async def list_agent_messages(
session_id: str = Query(min_length=1, max_length=500),
limit: int = Query(default=50, ge=1, le=100),
cursor: str | None = None,
pending_only: bool = False,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).list_session(session_id, limit, cursor, pending_only)
except DomainError as exc:
_raise_domain(exc)
@router.get("/pending", response_model=AgentMessagePage)
async def list_pending_agent_messages(
limit: int = Query(default=50, ge=1, le=100),
cursor: str | None = None,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).pending(limit, cursor)
except DomainError as exc:
_raise_domain(exc)
@router.get("/{message_id}", response_model=AgentMessage)
async def get_agent_message(
message_id: UUID,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).get(message_id)
except DomainError as exc:
_raise_domain(exc)
@router.post("/{message_id}/reply", response_model=AgentMessage, status_code=201)
async def reply_agent_message(
message_id: UUID, command: AgentMessageReply,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).reply(message_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post("/{message_id}/delivery", response_model=AgentMessage)
async def record_agent_message_delivery(
message_id: UUID, command: AgentMessageDelivery,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).delivery(message_id, command)
except DomainError as exc:
_raise_domain(exc)
+60
View File
@@ -0,0 +1,60 @@
"""Host-only MCP bindings: declared in REST contracts even when MCP is disabled."""
from fastapi import APIRouter, Header, HTTPException, Request, Response
from psycopg.errors import UndefinedTable, InvalidSchemaName
from app.mcp.models import BindInput, BindResponse
from app.infra.db.postgresql.mcp_queries import McpQueryRepository
router = APIRouter(tags=["MCP Query Context"])
def runtime_for(request: Request):
runtime = getattr(request.app.state, "mcp_runtime", None)
if runtime is None:
raise HTTPException(503, "Product MCP is disabled")
return runtime
@router.post("/mcp-query-contexts", response_model=BindResponse)
async def create_context(
command: BindInput,
request: Request,
x_project_id: str = Header(alias="X-Project-Id"),
) -> BindResponse:
runtime = runtime_for(request)
principal = await runtime.identity(request)
from app.mcp.contexts import bind_context
try:
async with runtime.scope_factory(principal, x_project_id) as scope:
binding = await bind_context(scope, principal, command, runtime.settings)
except (UndefinedTable, InvalidSchemaName):
raise HTTPException(
503, "MCP project storage has not been provisioned"
) from None
return BindResponse(
context_handle=binding.context_handle,
expires_at=binding.expires_at,
task_id=binding.task_id,
run_id=binding.run_id,
)
@router.delete("/mcp-query-contexts/{context_handle}", status_code=204)
async def delete_context(
context_handle: str,
request: Request,
x_project_id: str = Header(alias="X-Project-Id"),
) -> Response:
runtime = runtime_for(request)
principal = await runtime.identity(request)
try:
async with runtime.scope_factory(principal, x_project_id) as scope:
await McpQueryRepository(scope.connection, scope.project.project_id).revoke(
context_handle, principal.subject, principal.client_id
)
except (UndefinedTable, InvalidSchemaName):
raise HTTPException(
503, "MCP project storage has not been provisioned"
) from None
return Response(status_code=204)
+773
View File
@@ -0,0 +1,773 @@
"""Project-scoped APIs for operational tasks, evidence and approved actions."""
from __future__ import annotations
from datetime import datetime
from typing import Literal
from uuid import UUID
from fastapi import APIRouter, Depends, Header, HTTPException, Query, status
from psycopg import AsyncConnection
from app.auth.permissions import (
OPERATIONAL_TASK_EXECUTE,
OPERATIONAL_TASK_EXTERNAL_SYNC,
OPERATIONAL_TASK_VIEW,
OPERATIONAL_TASK_WRITE,
SCADA_VIEW,
SIMULATION_VIEW,
permissions_for_context,
require_permission,
)
from app.auth.project_dependencies import (
ProjectContext,
get_project_context,
get_project_pg_connection,
)
from app.domain.domain_errors import DomainError
from app.domain.schemas.workbench_runtime import WorkbenchDateTime
from app.domain.schemas.operational_data import TaskDataQuery, TaskDataSnapshot
from app.domain.schemas.operational_task import (
ActionDecisionCommand,
ActionExecuteCommand,
ActionProposalCommand,
ActionReconcileCommand,
WorkOrderDraftUpdateCommand,
WorkOrderConfirmDispatchCommand,
TaskAccessRequest,
TaskAccessRequestCreate,
TaskAccessRequestDecision,
AgentArtifactsCommand,
AgentArtifactsResult,
AgentRunCancelCommand,
AgentRunCommand,
AgentRunDispatchFailureCommand,
AgentRunFailureCommand,
AgentTaskContext,
DispatchDueSchedulesCommand,
DispatchDueSchedulesResult,
EvidenceRecord,
ExternalFeedbackCommand,
ExternalFeedbackResult,
ExternalSourceWithdrawalCommand,
HumanApprovalAction,
HumanEvidenceCommand,
InvestigationReportPage,
InvestigationReportRevision,
NotificationAcknowledgeCommand,
OperationalTaskCreate,
OperationalTaskDetail,
OperationalTaskNotification,
OperationalTaskNotificationPage,
OperationalTaskPage,
OperationalTaskRetryCommand,
OperationalTaskRun,
OperationalTaskRunPage,
OperationalTaskSchedule,
OperationalTaskScheduleCreate,
OperationalTaskSchedulePage,
OperationalTaskScheduleUpdate,
RecoverExpiredLeasesCommand,
RecoverExpiredLeasesResult,
TaskStatus,
TaskDataQueryResult,
TaskType,
TaskPresentationEventCommand,
TaskPresentationEventResult,
TaskPresentationCursor,
)
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.infra.integrations.operational_actions import (
OperationalActionDispatcher,
get_operational_action_dispatcher,
)
from app.infra.integrations.operational_data import TaskDataError, read_task_data
from app.services.operational_tasks import OperationalTaskService
router = APIRouter()
def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskService:
return OperationalTaskService(
OperationalTaskRepository(conn, ctx.project_id),
ctx,
_task_permissions(ctx),
)
def _task_permissions(ctx: ProjectContext) -> set[str]:
# Investigations use preconfigured project access, including service accounts.
# Do not force an already-authorized reader through legacy per-task delegation.
return set(permissions_for_context(ctx))
def _raise_domain(exc: DomainError):
raise HTTPException(status_code=exc.status, detail=exc.payload()) from exc
@router.get("/operational-tasks/{task_id}/reports", response_model=InvestigationReportPage)
async def list_investigation_reports(
task_id: UUID,
limit: int = Query(default=20, ge=1, le=100),
offset: int = Query(default=0, ge=0),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
items, next_offset = await _service(ctx, conn).list_reports(task_id, limit, offset)
return InvestigationReportPage(items=items, next_offset=next_offset)
except DomainError as exc:
_raise_domain(exc)
@router.get("/operational-tasks/{task_id}/reports/{revision}", response_model=InvestigationReportRevision)
async def get_investigation_report(
task_id: UUID,
revision: int,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).report(task_id, revision)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks",
response_model=OperationalTaskDetail,
status_code=status.HTTP_201_CREATED,
)
async def create_operational_task(
command: OperationalTaskCreate,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).create(command)
except DomainError as exc:
_raise_domain(exc)
@router.get("/operational-tasks", response_model=OperationalTaskPage)
async def list_operational_tasks(
task_type: TaskType | None = None,
task_status: TaskStatus | None = Query(default=None, alias="status"),
limit: int = Query(default=50, ge=1, le=100),
offset: int = Query(default=0, ge=0),
peripheral_work_order_id: UUID | None = Query(
default=None, description="Only tasks sourced from or linked to this peripheral work order",
),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
items, next_offset = await _service(ctx, conn).list(
task_type=task_type, status=task_status, limit=limit, offset=offset,
**({"peripheral_work_order_id": peripheral_work_order_id}
if peripheral_work_order_id is not None else {}),
)
return OperationalTaskPage(
items=items, limit=limit, offset=offset, next_offset=next_offset
)
except DomainError as exc:
_raise_domain(exc)
@router.get("/operational-task-runs", response_model=OperationalTaskRunPage)
async def list_operational_task_runs(
run_status: Literal[
"queued", "claimed", "running", "succeeded", "failed", "timed_out", "cancelled"
] | None = Query(default=None, alias="status"),
task_type: TaskType | None = None,
task_created_from: WorkbenchDateTime | None = None,
limit: int = Query(default=50, ge=1, le=100),
cursor: str | None = None,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
cursor_created_at = None
cursor_run_id = None
if cursor:
try:
timestamp, run_id = cursor.rsplit("|", 1)
cursor_created_at = datetime.fromisoformat(timestamp)
cursor_run_id = UUID(run_id)
except (ValueError, TypeError) as exc:
raise HTTPException(status_code=400, detail="Invalid run cursor") from exc
try:
items, next_cursor = await _service(ctx, conn).list_runs(
status=run_status,
task_type=task_type,
task_created_from=task_created_from,
limit=limit,
cursor_created_at=cursor_created_at,
cursor_run_id=cursor_run_id,
)
return OperationalTaskRunPage(items=items, next_cursor=next_cursor)
except DomainError as exc:
_raise_domain(exc)
@router.get("/operational-tasks/{task_id}/runs", response_model=OperationalTaskRunPage)
async def list_task_run_history(
task_id: UUID,
limit: int = Query(default=20, ge=1, le=100),
offset: int = Query(default=0, ge=0),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return OperationalTaskRunPage(items=await _service(ctx, conn).task_runs(task_id, limit, offset))
except DomainError as exc:
_raise_domain(exc)
@router.get("/operational-tasks/{task_id}/runs/{run_id}", response_model=OperationalTaskRun)
async def get_task_run(
task_id: UUID, run_id: UUID,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).task_run(task_id, run_id)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/schedules",
response_model=OperationalTaskSchedule,
status_code=status.HTTP_201_CREATED,
)
async def create_operational_schedule(
command: OperationalTaskScheduleCreate,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).create_schedule(command)
except DomainError as exc:
_raise_domain(exc)
@router.get(
"/operational-tasks/schedules", response_model=OperationalTaskSchedulePage
)
async def list_operational_schedules(
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return OperationalTaskSchedulePage(items=await _service(ctx, conn).list_schedules())
except DomainError as exc:
_raise_domain(exc)
@router.patch(
"/operational-tasks/schedules/{schedule_id}",
response_model=OperationalTaskSchedule,
)
async def update_operational_schedule(
schedule_id: UUID,
command: OperationalTaskScheduleUpdate,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).update_schedule(schedule_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/schedules/dispatch-due",
response_model=DispatchDueSchedulesResult,
)
async def dispatch_due_operational_schedules(
command: DispatchDueSchedulesCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).dispatch_due_schedules(command.limit)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/leases/recover",
response_model=RecoverExpiredLeasesResult,
)
async def recover_operational_task_leases(
command: RecoverExpiredLeasesCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).recover_expired_leases(
command.limit, task_type=command.task_type,
task_created_from=command.task_created_from,
)
except DomainError as exc:
_raise_domain(exc)
@router.get(
"/operational-task-notifications",
response_model=OperationalTaskNotificationPage,
)
async def list_operational_task_notifications(
unread_only: bool = False,
limit: int = Query(default=50, ge=1, le=100),
offset: int = Query(default=0, ge=0),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
items, next_offset = await _service(ctx, conn).list_notifications(
unread_only=unread_only, limit=limit, offset=offset
)
return OperationalTaskNotificationPage(
items=items, limit=limit, offset=offset, next_offset=next_offset
)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-task-notifications/{notification_id}/acknowledge",
response_model=OperationalTaskNotification,
)
async def acknowledge_operational_task_notification(
notification_id: UUID,
command: NotificationAcknowledgeCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).acknowledge_notification(notification_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.get("/operational-tasks/{task_id}", response_model=OperationalTaskDetail)
async def get_operational_task(
task_id: UUID,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).detail(task_id)
except DomainError as exc:
_raise_domain(exc)
@router.post("/operational-tasks/{task_id}/retry", response_model=OperationalTaskDetail)
async def retry_operational_task(
task_id: UUID,
command: OperationalTaskRetryCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).retry_task(task_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/runs/{run_id}/claim",
response_model=OperationalTaskRun,
)
async def claim_operational_task_run(
task_id: UUID,
run_id: UUID,
command: AgentRunCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).claim_run(task_id, run_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/runs/{run_id}/heartbeat",
response_model=OperationalTaskRun,
)
async def heartbeat_operational_task_run(
task_id: UUID,
run_id: UUID,
command: AgentRunCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).heartbeat_run(task_id, run_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/runs/{run_id}/complete",
response_model=OperationalTaskDetail,
)
async def complete_operational_task_run(
task_id: UUID,
run_id: UUID,
command: AgentRunCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).complete_run(task_id, run_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/runs/{run_id}/fail",
response_model=OperationalTaskDetail,
)
async def fail_operational_task_run(
task_id: UUID,
run_id: UUID,
command: AgentRunFailureCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).fail_run(task_id, run_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/runs/{run_id}/cancel",
response_model=OperationalTaskDetail,
)
async def cancel_operational_task_run(
task_id: UUID,
run_id: UUID,
command: AgentRunCancelCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).cancel_run(task_id, run_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/runs/{run_id}/dispatch-failure",
response_model=OperationalTaskRun,
)
async def record_operational_task_run_dispatch_failure(
task_id: UUID,
run_id: UUID,
command: AgentRunDispatchFailureCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).record_dispatch_failure(task_id, run_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.get(
"/operational-tasks/{task_id}/agent-context", response_model=AgentTaskContext
)
async def get_operational_task_agent_context(
task_id: UUID,
report_revision: int | None = Query(default=None, ge=1),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).agent_context(task_id, report_revision)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/agent-artifacts",
response_model=AgentArtifactsResult,
)
async def publish_operational_task_artifacts(
task_id: UUID,
command: AgentArtifactsCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).publish_artifacts(task_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.get("/operational-tasks/{task_id}/presentation-cursor", response_model=TaskPresentationCursor)
async def get_operational_task_presentation_cursor(
task_id: UUID,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).presentation_cursor(task_id)
except DomainError as exc:
_raise_domain(exc)
@router.post("/operational-tasks/{task_id}/presentation-events", response_model=TaskPresentationEventResult)
async def publish_operational_task_presentation_event(
task_id: UUID,
command: TaskPresentationEventCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).publish_presentation_event(task_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/data-query", response_model=TaskDataQueryResult
)
async def query_operational_task_data(
task_id: UUID,
query: TaskDataQuery,
authorization: str | None = Header(default=None, alias="Authorization"),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
service = _service(ctx, conn)
try:
if not await service.can_read_source(task_id, query.source):
raise DomainError("PERMISSION_DENIED", "Task has no approved access to this data source", status=403)
source_permission = SCADA_VIEW if query.source.startswith("scada_") else SIMULATION_VIEW
snapshot = await read_task_data(
context=ctx, query=query, credential=authorization,
delegated=source_permission not in _task_permissions(ctx),
)
evidence, stored_snapshot = await service.record_data_snapshot(task_id, snapshot)
return TaskDataQueryResult(evidence=evidence, snapshot=stored_snapshot)
except TaskDataError as exc:
raise HTTPException(status_code=exc.status, detail=exc.detail) from exc
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/access-requests",
response_model=TaskAccessRequest,
status_code=status.HTTP_201_CREATED,
)
async def request_operational_task_access(
task_id: UUID,
command: TaskAccessRequestCreate,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).request_access(task_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/access-requests/{access_request_id}/decision",
response_model=TaskAccessRequest,
)
async def decide_operational_task_access(
task_id: UUID,
access_request_id: UUID,
command: TaskAccessRequestDecision,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).decide_access(task_id, access_request_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/human-evidence",
response_model=OperationalTaskDetail,
)
async def add_operational_task_human_evidence(
task_id: UUID,
command: HumanEvidenceCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).add_human_evidence(task_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.get(
"/operational-tasks/{task_id}/evidence/{evidence_id}/snapshot",
response_model=TaskDataSnapshot,
)
async def get_operational_task_data_snapshot(
task_id: UUID,
evidence_id: str,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).data_snapshot(task_id, evidence_id)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/actions",
response_model=HumanApprovalAction,
status_code=status.HTTP_201_CREATED,
)
async def propose_operational_task_action(
task_id: UUID,
command: ActionProposalCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_WRITE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).propose_action(task_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/actions/{action_id}/decision",
response_model=HumanApprovalAction,
)
async def decide_operational_task_action(
task_id: UUID,
action_id: UUID,
command: ActionDecisionCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).decide_action(task_id, action_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.patch(
"/operational-tasks/{task_id}/actions/{action_id}/draft",
response_model=HumanApprovalAction,
)
async def update_operational_work_order_draft(
task_id: UUID,
action_id: UUID,
command: WorkOrderDraftUpdateCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).update_work_order_draft(task_id, action_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/actions/{action_id}/confirm-dispatch",
response_model=HumanApprovalAction,
)
async def confirm_operational_work_order_dispatch(
task_id: UUID,
action_id: UUID,
command: WorkOrderConfirmDispatchCommand,
authorization: str | None = Header(default=None, alias="Authorization"),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
dispatcher: OperationalActionDispatcher = Depends(get_operational_action_dispatcher),
):
try:
return await _service(ctx, conn).confirm_work_order_dispatch(
task_id, action_id, command, dispatcher, authorization
)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/actions/{action_id}/execute",
response_model=HumanApprovalAction,
)
async def execute_operational_task_action(
task_id: UUID,
action_id: UUID,
command: ActionExecuteCommand,
authorization: str | None = Header(default=None, alias="Authorization"),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
dispatcher: OperationalActionDispatcher = Depends(get_operational_action_dispatcher),
):
try:
return await _service(ctx, conn).submit_action(
task_id, action_id, command, dispatcher, authorization
)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/actions/{action_id}/reconcile",
response_model=HumanApprovalAction,
)
async def reconcile_operational_task_action(
task_id: UUID,
action_id: UUID,
command: ActionReconcileCommand,
authorization: str | None = Header(default=None, alias="Authorization"),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
dispatcher: OperationalActionDispatcher = Depends(get_operational_action_dispatcher),
):
try:
return await _service(ctx, conn).reconcile_action(
task_id, action_id, command, dispatcher, authorization
)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/external-feedback",
response_model=ExternalFeedbackResult,
)
async def synchronize_operational_task_feedback(
task_id: UUID,
command: ExternalFeedbackCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXTERNAL_SYNC)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).external_feedback(task_id, command)
except DomainError as exc:
_raise_domain(exc)
@router.post(
"/operational-tasks/{task_id}/withdraw-external-source",
response_model=OperationalTaskDetail,
)
async def withdraw_operational_task_external_source(
task_id: UUID,
command: ExternalSourceWithdrawalCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXTERNAL_SYNC)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
return await _service(ctx, conn).withdraw_external_source(task_id, command)
except DomainError as exc:
_raise_domain(exc)
+19
View File
@@ -0,0 +1,19 @@
from fastapi import APIRouter, HTTPException
from app.domain.schemas.place_search import PlaceSearchRequest, PlaceSearchResponse
from app.services.place_search import PlaceSearchError, search_tianditu_places
router = APIRouter()
@router.post(
"/place-search-requests",
response_model=PlaceSearchResponse,
summary="Search Fengxian places",
description="调用天地图地名搜索,固定上海市奉贤区范围,只返回前 10 个可定位候选,不提供分页。",
)
async def tianditu_place_search(request: PlaceSearchRequest) -> PlaceSearchResponse:
try:
return await search_tianditu_places(request)
except PlaceSearchError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
+107 -35
View File
@@ -1,47 +1,116 @@
import logging
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query
from psycopg import AsyncConnection
from app.infra.db.postgresql.analysis import AnalysisRepository
from app.auth.project_dependencies import get_project_pg_connection
from app.domain.schemas.gis_attributes import (
ElementGisProperties,
GisElementType,
SourceGisFeatureProperties,
)
from app.infra.db.postgresql.analysis import AnalysisRepository
from app.infra.db.postgresql.gis_attributes import GisAttributeRepository
router = APIRouter()
logger = logging.getLogger(__name__)
async def get_database_connection(
@router.get(
"/source-gis/layers/{layer_key}/features/{source_fid}/properties",
response_model=SourceGisFeatureProperties,
summary="获取源 GIS 要素台账属性",
)
async def get_source_gis_feature_properties(
layer_key: str,
source_fid: int,
conn: AsyncConnection = Depends(get_project_pg_connection),
):
"""获取数据库连接"""
yield conn
normalized_layer_key = layer_key.strip()
if (
not normalized_layer_key
or len(normalized_layer_key) > 80
or source_fid < 0
):
raise HTTPException(status_code=422, detail="无效的源 GIS 要素编号")
try:
group = await GisAttributeRepository.get_source_feature_group(
conn,
normalized_layer_key,
source_fid,
)
if group is None:
raise HTTPException(status_code=404, detail="源 GIS 要素不存在")
return SourceGisFeatureProperties(
layer_key=normalized_layer_key,
source_fid=source_fid,
group=group,
)
except HTTPException:
raise
except Exception as exc:
logger.warning("Source GIS feature properties are unavailable", exc_info=True)
raise HTTPException(status_code=500, detail="查询源 GIS 要素属性失败") from exc
@router.get(
"/elements/{element_type}/{element_id}/gis-properties",
response_model=ElementGisProperties,
summary="获取元素 GIS 台账属性",
)
async def get_element_gis_properties(
element_type: GisElementType,
element_id: str,
conn: AsyncConnection = Depends(get_project_pg_connection),
):
"""Return source GIS records linked to one current-project map element."""
normalized_id = element_id.strip()
if not normalized_id or len(normalized_id) > 160:
raise HTTPException(status_code=422, detail="无效的元素编号")
try:
groups = await GisAttributeRepository.list_element_groups(
conn,
element_type,
normalized_id,
)
return ElementGisProperties(
element_type=element_type,
element_id=normalized_id,
groups=groups,
)
except Exception as exc:
logger.warning("Element GIS properties are unavailable", exc_info=True)
raise HTTPException(status_code=500, detail="查询 GIS 台账属性失败") from exc
@router.get("/analysis/runs", summary="获取分析运行列表")
async def get_analysis_runs(
conn: AsyncConnection = Depends(get_database_connection),
limit: int = Query(default=100, ge=1, le=100),
offset: int = Query(default=0, ge=0),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
"""
获取所有方案信息
返回项目中所有方案的详细信息
"""
"""分页返回当前项目的持久分析运行。"""
try:
runs = await AnalysisRepository.list_runs(conn)
return {"success": True, "data": runs, "count": len(runs)}
except Exception as e:
raise HTTPException(status_code=500, detail=f"查询分析运行时发生错误: {str(e)}")
runs = await AnalysisRepository.list_runs(conn, limit=limit, offset=offset)
return {
"success": True,
"data": runs,
"count": len(runs),
"limit": limit,
"offset": offset,
}
except Exception as exc:
logger.warning("Analysis run page is unavailable", exc_info=True)
raise HTTPException(status_code=500, detail="查询分析运行失败") from exc
@router.get("/analysis/runs/{run_id}", summary="获取分析运行")
async def get_analysis_run(
run_id: UUID,
conn: AsyncConnection = Depends(get_database_connection),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
"""
获取所有爆管定位结果
返回项目中所有的爆管定位分析结果
"""
"""返回当前项目的一条持久分析运行。"""
try:
run = await AnalysisRepository.get_run(conn, run_id)
if run is None:
@@ -49,28 +118,31 @@ async def get_analysis_run(
return run
except HTTPException:
raise
except Exception as e:
raise HTTPException(
status_code=500, detail=f"查询分析运行时发生错误: {str(e)}"
)
except Exception as exc:
logger.warning("Analysis run is unavailable", exc_info=True)
raise HTTPException(status_code=500, detail="查询分析运行失败") from exc
@router.get("/analysis/runs/{run_id}/results", summary="获取分析结果")
async def get_analysis_results(
run_id: UUID,
result_type: str | None = Query(default=None, description="结果类型"),
conn: AsyncConnection = Depends(get_database_connection),
limit: int = Query(default=100, ge=1, le=500),
offset: int = Query(default=0, ge=0),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
"""
根据爆管事件ID查询爆管定位结果
参数:
burst_incident: 爆管事件的唯一标识符
"""
"""分页返回一条分析运行的持久结果。"""
try:
return await AnalysisRepository.list_results(conn, run_id, result_type)
except Exception as e:
return await AnalysisRepository.list_results(
conn,
run_id,
result_type,
limit=limit,
offset=offset,
)
except Exception as exc:
logger.warning("Analysis result page is unavailable", exc_info=True)
raise HTTPException(
status_code=500,
detail=f"查询分析结果时发生错误: {str(e)}",
)
detail="查询分析结果失败",
) from exc
+1
View File
@@ -146,6 +146,7 @@ async def optimize_sensor_placement_scheme(
sensor_count=payload.sensor_count,
min_diameter=payload.min_diameter,
created_by=current_user.username,
sensor_type=payload.sensor_type,
)
run = get_sensor_placement_run(network, created["run_id"])
return {**run, "can_edit": True}
+3 -22
View File
@@ -1,5 +1,5 @@
from typing import Any, List, Literal, Optional
from datetime import datetime, timedelta
from datetime import datetime
from fastapi import APIRouter, Body, Depends, HTTPException, Query
from fastapi.responses import PlainTextResponse
from app.auth.keycloak_dependencies import get_current_keycloak_username
@@ -18,7 +18,6 @@ from app.services.simulation_scenarios import (
from app.services.valve_isolation import analyze_valve_isolation
from app.domain.time import (
parse_aware_time,
parse_clock_duration_seconds,
parse_utc_time,
)
from pydantic import BaseModel, Field, field_validator
@@ -50,27 +49,9 @@ class PressureRegulation(BaseModel):
def run_simulation_manually_by_date(
network_name: str, start_time: datetime, duration: int
) -> None:
end_datetime = start_time + timedelta(minutes=duration)
time_properties = simulation.get_time(network_name)
hydraulic_step_seconds = parse_clock_duration_seconds(
time_properties["HYDRAULIC TIMESTEP"],
field_name="HYDRAULIC TIMESTEP",
simulation.run_online_simulation(
name=network_name, start_time=start_time, duration_seconds=duration * 60,
)
if hydraulic_step_seconds <= 0:
raise ValueError("HYDRAULIC TIMESTEP must be greater than 0.")
hydraulic_step = timedelta(seconds=hydraulic_step_seconds)
scada_mappings = simulation.query_corresponding_element_id_and_query_id(
network_name
)
current_time = start_time
while current_time < end_datetime:
simulation.run_simulation(
name=network_name,
simulation_type="realtime",
modify_pattern_start_time=current_time.isoformat(timespec="seconds"),
scada_mappings=scada_mappings,
)
current_time += hydraulic_step
# 必须用这个PlainTextResponse,不然每个key都有引号
+2 -1
View File
@@ -11,9 +11,10 @@ from app.services.timeseries_history import TimeseriesHistoryService
from .dependencies import get_timescale_connection, get_postgres_connection
router = APIRouter()
history_router = APIRouter()
@router.post(
@history_router.post(
"/timeseries/views/element-history/query",
summary="批量查询管网元素历史数据",
response_model=ElementHistoryResponse,
-29
View File
@@ -1,29 +0,0 @@
from typing import Any
from fastapi import APIRouter, HTTPException, status
from app.services.web_search import (
BochaSearchAPIError,
BochaSearchConfigError,
WebSearchRequest,
search_bocha_web,
)
router = APIRouter()
@router.post(
"/web-searches",
summary="Web Search",
description="调用 Bocha Web Search API 获取实时网页搜索结果",
)
async def web_search(request: WebSearchRequest) -> dict[str, Any]:
try:
return await search_bocha_web(request)
except BochaSearchConfigError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=str(exc),
) from exc
except BochaSearchAPIError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
+156
View File
@@ -0,0 +1,156 @@
"""Authenticated workbench snapshot and resumable Server-Sent Events."""
from __future__ import annotations
import asyncio
import json
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, Header, HTTPException, Query
from fastapi.responses import StreamingResponse
from psycopg import AsyncConnection
from psycopg.conninfo import conninfo_to_dict
from app.api.v1.endpoints.workbench_runtime import workbench_runtime
from app.auth.permissions import OPERATIONAL_TASK_VIEW, permissions_for_context, require_permission
from app.auth.project_dependencies import (
ProjectContext,
get_metadata_repository,
get_project_business_routing,
get_project_pg_connection,
)
from app.core.config import settings
from app.domain.schemas.workbench_events import WorkbenchEvent, WorkbenchSyncSnapshot
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.infra.db.postgresql.workbench_events import WorkbenchEventRepository
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
from app.infra.db.project_routing import ActiveProjectRouting
from app.services.operational_tasks import OperationalTaskService
from app.infra.db.postgresql.workbench_event_hub import workbench_event_hubs
router = APIRouter()
def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskService:
return OperationalTaskService(
OperationalTaskRepository(conn, ctx.project_id),
ctx,
permissions_for_context(ctx),
)
@router.get("/workbench/sync-snapshot", response_model=WorkbenchSyncSnapshot)
async def workbench_sync_snapshot(
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
async with conn.transaction():
await conn.execute(
"SET TRANSACTION ISOLATION LEVEL REPEATABLE READ READ ONLY"
)
cursor = await WorkbenchEventRepository(conn, ctx.project_id).current_cursor()
runtime = await workbench_runtime(ctx=ctx, conn=conn)
tasks, _ = await _service(ctx, conn).list(
task_type=None, status=None, limit=50, offset=0
)
notifications, _ = await _service(ctx, conn).list_notifications(
unread_only=False, limit=100, offset=0
)
return WorkbenchSyncSnapshot(
schema="workbench-sync-snapshot@1",
event_cursor=cursor,
runtime=runtime,
tasks=tasks,
notifications=notifications,
generated_at=datetime.now(timezone.utc),
)
@router.get("/workbench/events")
async def workbench_events(
after: int | None = Query(default=None, ge=0),
last_event_id: str | None = Header(default=None, alias="Last-Event-ID"),
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_VIEW)),
routing: ActiveProjectRouting = Depends(get_project_business_routing),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
):
if not settings.WORKBENCH_EVENTS_ENABLED:
raise HTTPException(status_code=503, detail="Workbench events are disabled")
cursor = after
if cursor is None and last_event_id:
try:
cursor = int(last_event_id)
except ValueError as exc:
raise HTTPException(status_code=400, detail="Invalid Last-Event-ID") from exc
cursor = cursor or 0
current_identity = _database_identity(routing.business_dsn)
try:
active_routings = await metadata_repo.list_active_project_db_routings("biz_data")
finally:
# FastAPI keeps yield dependencies alive for a StreamingResponse. The
# stream only needs immutable routing values, so release the shared
# metadata checkout before the five-minute SSE body begins.
await metadata_repo.session.close()
if any(
item.project_id != ctx.project_id
and _database_identity(item.dsn) == current_identity
for item in active_routings
):
raise HTTPException(
status_code=503,
detail="Workbench events require one dedicated business database per project",
)
async def stream():
started = asyncio.get_running_loop().time()
last_keepalive = started
async with workbench_event_hubs.subscribe(ctx.project_id, routing.business_dsn) as hub:
oldest = await hub.oldest_cursor()
latest = await hub.current_cursor()
if cursor > 0 and (
oldest is None or cursor < oldest - 1 or cursor > latest
):
payload = {"schema": "workbench-event@1", "cursor": cursor}
yield f"event: reset_required\ndata: {json.dumps(payload, separators=(',', ':'))}\n\n"
return
current = cursor
wake_generation = hub.generation
while asyncio.get_running_loop().time() - started < 300:
rows = await hub.read_after(current)
for row in rows:
event = WorkbenchEvent(
schema="workbench-event@1",
**row,
)
current = event.event_seq
data = event.model_dump_json(by_alias=True)
yield f"id: {current}\nevent: workbench_event\ndata: {data}\n\n"
if rows:
continue
now = asyncio.get_running_loop().time()
if now - last_keepalive >= 15:
yield f": keepalive {int(now)}\n\n"
last_keepalive = now
# The Hub invalidates shared reads on NOTIFY and every 30s.
# Per-client timeout only keeps the transport alive, not SQL.
wake_generation = await hub.wait_for_wake(wake_generation, timeout=15)
yield "event: reconnect\ndata: {}\n\n"
return StreamingResponse(
stream(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache, no-transform",
"X-Accel-Buffering": "no",
"Content-Encoding": "identity",
},
)
def _database_identity(dsn: str) -> tuple[str, str, str]:
values = conninfo_to_dict(dsn)
return (
values.get("host") or values.get("hostaddr") or "localhost",
values.get("port") or "5432",
values.get("dbname") or values.get("user") or "",
)
+155
View File
@@ -0,0 +1,155 @@
"""Read-only workbench runtime aggregation."""
import logging
from contextlib import asynccontextmanager
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query, status
from psycopg import AsyncConnection
from pydantic import ValidationError
from app.auth.project_dependencies import (
ProjectContext,
get_project_context,
get_project_pg_connection,
)
from app.domain.schemas.workbench_runtime import (
WorkbenchAnalysisHistoryPage,
WorkbenchPersistedAlert,
WorkbenchRuntimeLatestAnalysis,
WorkbenchRuntimeSnapshot,
)
from app.domain.schemas.task_alert import TaskAlertCommand, TaskAlertEvent, TaskAlertState
from app.infra.db.postgresql.analysis import AnalysisRepository
from app.infra.db.postgresql.workbench_operations import (
WorkbenchOperationsRepository,
)
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
from app.services.task_alerts import TaskAlertService
from app.domain.domain_errors import DomainError
from app.services.workbench_runtime import (
build_workbench_runtime_snapshot,
project_analysis_history_item,
project_latest_analysis,
project_sensor_location_map_payload,
sensor_location_ids_for_map_projection,
)
router = APIRouter()
logger = logging.getLogger(__name__)
@asynccontextmanager
async def _projection_transaction(conn: AsyncConnection):
"""Isolate an optional projection so one SQL failure cannot poison others."""
transaction = getattr(conn, "transaction", None)
if transaction is None: # Lightweight endpoint test doubles.
yield
return
async with transaction():
yield
@router.get(
"/workbench/analysis-runs",
response_model=WorkbenchAnalysisHistoryPage,
)
async def workbench_analysis_runs(
limit: int = Query(default=20, ge=1, le=100),
offset: int = Query(default=0, ge=0),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
"""Return a bounded, sanitized analysis history page for the workbench."""
rows = await AnalysisRepository.list_run_summaries(
conn,
limit=limit + 1,
offset=offset,
)
has_more = len(rows) > limit
items = rows[:limit]
return {
"items": [project_analysis_history_item(run) for run in items],
"limit": limit,
"offset": offset,
"next_offset": offset + limit if has_more else None,
}
@router.get(
"/workbench/analysis-runs/{run_id}",
response_model=WorkbenchRuntimeLatestAnalysis,
)
async def workbench_analysis_run(
run_id: UUID,
conn: AsyncConnection = Depends(get_project_pg_connection),
):
"""Return one authorized project's analysis run as a typed workbench artifact."""
run = await AnalysisRepository.get_run(conn, run_id)
if run is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Analysis run not found",
)
results = await AnalysisRepository.list_recent_results(conn, run_id, limit=100)
try:
referenced_map_payload = await _sensor_location_map_payload(conn, results)
except Exception:
referenced_map_payload = None
logger.warning(
"Workbench analysis spatial references are unavailable",
exc_info=True,
)
return project_latest_analysis(
run,
results,
referenced_map_payload=referenced_map_payload,
)
@router.post("/workbench/task-alert-events", response_model=TaskAlertState)
async def push_task_alert_event(
command: TaskAlertCommand,
ctx: ProjectContext = Depends(get_project_context),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
try:
event = TaskAlertEvent(
**command.model_dump(),
producer_id=f"principal:{ctx.user_id}",
)
return await TaskAlertService(
TaskAlertRepository(conn, ctx.project_id), ctx
).push(event)
except DomainError as exc:
raise HTTPException(status_code=exc.status, detail=exc.message) from exc
@router.get("/workbench/runtime", response_model=WorkbenchRuntimeSnapshot)
async def workbench_runtime(
ctx: ProjectContext = Depends(get_project_context),
conn: AsyncConnection = Depends(get_project_pg_connection),
):
from app.services.workbench_runtime import read_workbench_runtime
return await read_workbench_runtime(ctx, conn)
async def _sensor_location_map_payload(
conn: AsyncConnection, results: list[dict]
) -> dict | None:
node_ids = sensor_location_ids_for_map_projection(results)
if not node_ids:
return None
map_points = await AnalysisRepository.list_junction_map_points(conn, node_ids)
return project_sensor_location_map_payload(node_ids, map_points)
def _validate_documents(documents, model, label: str) -> tuple[list[dict], int]:
valid: list[dict] = []
rejected = 0
for document in documents:
try:
valid.append(model.model_validate(document).model_dump())
except ValidationError:
rejected += 1
logger.warning("Workbench %s document was rejected", label)
return valid, rejected
+29 -5
View File
@@ -4,6 +4,7 @@ from app.api.v1.endpoints import (
access,
admin_metadata,
agent_auth,
agent_messages,
audit,
burst_detection,
burst_location,
@@ -11,12 +12,15 @@ from app.api.v1.endpoints import (
leakage,
meta,
model_import,
operational_tasks,
place_search,
project,
project_data,
scada,
sensor_placement,
simulation,
web_search,
workbench_runtime,
workbench_events,
)
from app.api.v1.endpoints.components import (
controls,
@@ -85,8 +89,22 @@ burst_run_access = Depends(require_permission(BURST_RUN))
optimization_run_access = Depends(require_permission(OPTIMIZATION_RUN))
# Core services
api_router.include_router(
workbench_runtime.router,
tags=["Workbench Runtime"],
dependencies=[simulation_access],
)
api_router.include_router(
workbench_events.router,
tags=["Workbench Events"],
)
api_router.include_router(
operational_tasks.router,
tags=["Operational Tasks"],
)
api_router.include_router(access.router, tags=["Access Control"])
api_router.include_router(agent_auth.router, tags=["Agent Auth"])
api_router.include_router(agent_messages.router, tags=["Agent Messages"])
api_router.include_router(
admin_metadata.router,
tags=["Metadata Admin"],
@@ -143,13 +161,13 @@ api_router.include_router(
dependencies=[optimization_run_access],
)
api_router.include_router(
web_search.router,
tags=["Web Search"],
geocoding.router,
tags=["Geocoding"],
dependencies=[webgis_view_access],
)
api_router.include_router(
geocoding.router,
tags=["Geocoding"],
place_search.router,
tags=["Place Search"],
dependencies=[webgis_view_access],
)
api_router.include_router(
@@ -179,6 +197,12 @@ for endpoint_router, tag in (
dependencies=[simulation_access],
)
api_router.include_router(
ts_composite.history_router,
tags=["TimescaleDB - Composite"],
dependencies=[Depends(require_permission(SCADA_VIEW))],
)
for endpoint_router, tag in (
(ts_scada.router, "TimescaleDB - SCADA"),
(ts_composite.router, "TimescaleDB - Composite"),
+10
View File
@@ -6,6 +6,9 @@ from fastapi.security import OAuth2PasswordBearer
from jose import JWTError, jwt
from app.core.config import settings
from app.mcp.config import McpSettings
mcp_resource_settings = McpSettings()
oauth2_optional = OAuth2PasswordBearer(
tokenUrl="keycloak", auto_error=False
@@ -29,6 +32,13 @@ def _decode_keycloak_token(token: str) -> dict:
algorithms=[settings.KEYCLOAK_ALGORITHM],
audience=settings.KEYCLOAK_AUDIENCE or None,
)
# A resource-bound MCP token must never authorize product REST APIs,
# including when another audience overlaps the configured API audience.
audience = payload.get("aud", [])
audiences = [audience] if isinstance(audience, str) else audience
if (isinstance(audiences, list) and mcp_resource_settings.audience in audiences
or payload.get("azp") == mcp_resource_settings.host_client_id):
raise JWTError("MCP resource tokens cannot authorize product REST APIs")
if settings.KEYCLOAK_ACCESS_TOKEN_MAX_AGE_SECONDS <= 0:
return payload
+33
View File
@@ -21,6 +21,15 @@ MODEL_IMPORT = "model.import"
AUDIT_VIEW = "audit.view"
ENVIRONMENT_MANAGE = "environment.manage"
MEMBERSHIP_MANAGE = "membership.manage"
OPERATIONAL_TASK_VIEW = "operational_task.view"
OPERATIONAL_TASK_WRITE = "operational_task.write"
OPERATIONAL_TASK_EXECUTE = "operational_task.execute"
OPERATIONAL_TASK_EXTERNAL_SYNC = "operational_task.external_sync"
OPERATIONAL_TASK_ACCESS_APPROVE = "operational_task.access.approve"
OPERATIONAL_ACTION_WORK_ORDER_APPROVE = "operational_action.work_order.approve"
OPERATIONAL_ACTION_WORK_ORDER_EXECUTE = "operational_action.work_order.execute"
OPERATIONAL_ACTION_COMMAND_APPROVE = "operational_action.command.approve"
OPERATIONAL_ACTION_COMMAND_EXECUTE = "operational_action.command.execute"
PROJECT_MEMBER_PERMISSIONS = frozenset(
{
@@ -36,6 +45,11 @@ PROJECT_MEMBER_PERMISSIONS = frozenset(
RISK_RUN,
OPTIMIZATION_VIEW,
OPTIMIZATION_RUN,
OPERATIONAL_TASK_VIEW,
OPERATIONAL_TASK_WRITE,
OPERATIONAL_TASK_ACCESS_APPROVE,
OPERATIONAL_ACTION_WORK_ORDER_APPROVE,
OPERATIONAL_ACTION_WORK_ORDER_EXECUTE,
}
)
@@ -44,6 +58,7 @@ PROJECT_VIEWER_PERMISSIONS = frozenset(
WEBGIS_VIEW,
SCADA_VIEW,
SIMULATION_VIEW,
OPERATIONAL_TASK_VIEW,
}
)
@@ -53,6 +68,15 @@ SYSTEM_ADMIN_PERMISSIONS = frozenset(
AUDIT_VIEW,
ENVIRONMENT_MANAGE,
MEMBERSHIP_MANAGE,
OPERATIONAL_TASK_EXECUTE,
OPERATIONAL_TASK_EXTERNAL_SYNC,
}
)
SYSTEM_SERVICE_PERMISSIONS = frozenset(
{
OPERATIONAL_TASK_EXECUTE,
OPERATIONAL_TASK_EXTERNAL_SYNC,
}
)
@@ -71,6 +95,15 @@ def resolve_permissions(
permissions = set(PROJECT_ROLE_PERMISSIONS.get(project_role or "", frozenset()))
if is_superuser or system_role == "admin":
permissions.update(SYSTEM_ADMIN_PERMISSIONS)
if project_role == "member":
permissions.update(
{
OPERATIONAL_ACTION_COMMAND_APPROVE,
OPERATIONAL_ACTION_COMMAND_EXECUTE,
}
)
elif system_role == "service" and project_role == "member":
permissions.update(SYSTEM_SERVICE_PERMISSIONS)
return frozenset(permissions)
+2
View File
@@ -129,6 +129,7 @@ async def resolve_project_business_routing(
return ActiveProjectRouting(
project_code=ctx.project_code,
business_dsn=business.dsn,
project_id=ctx.project_id,
)
@@ -161,6 +162,7 @@ async def resolve_project_simulation_routing(
project_code=ctx.project_code,
business_dsn=business.dsn,
timescale_dsn=timescale.dsn,
project_id=ctx.project_id,
)
+28 -7
View File
@@ -1,6 +1,7 @@
from pathlib import Path
from typing import Optional
from urllib.parse import quote_plus
from pydantic import AliasChoices, Field
from pydantic_settings import BaseSettings, SettingsConfigDict
@@ -9,7 +10,6 @@ class Settings(BaseSettings):
ENVIRONMENT: str = "production"
API_V1_STR: str = "/api/v1"
NETWORK_NAME: str = "default_network"
# 敏感配置加密密钥 (Fernet)
DATABASE_ENCRYPTION_KEY: str = ""
@@ -26,6 +26,9 @@ class Settings(BaseSettings):
TIMESCALEDB_DB_PORT: str = "5433"
TIMESCALEDB_DB_USER: str = "postgres"
TIMESCALEDB_DB_PASSWORD: str = "password"
TIMESCALEDB_REALTIME_MAX_DECOMPRESSED_TUPLES: int = Field(
default=1_000_000, ge=1, le=2_147_483_647,
)
# Metadata Database Config (PostgreSQL)
METADATA_DB_NAME: str = "system_hub"
METADATA_DB_HOST: str = "localhost"
@@ -63,16 +66,34 @@ class Settings(BaseSettings):
KEYCLOAK_AUDIENCE: str = ""
KEYCLOAK_ACCESS_TOKEN_MAX_AGE_SECONDS: int = 900
# Bocha Web Search API
BOCHA_API_KEY: str = ""
BOCHA_WEB_SEARCH_URL: str = "https://api.bochaai.com/v1/web-search"
BOCHA_WEB_SEARCH_TIMEOUT_SECONDS: float = 30.0
# Authenticated workbench synchronization.
WORKBENCH_EVENTS_ENABLED: bool = True
WORKBENCH_EVENT_RETENTION_DAYS: int = 7
CORS_ALLOWED_ORIGINS: str = (
"http://127.0.0.1:5173,http://localhost:5173,"
"http://127.0.0.1:15173,http://localhost:15173"
)
# Tianditu Geocoding API
TIANDITU_GEOCODER_TOKEN: str = ""
@property
def cors_allowed_origins(self) -> list[str]:
origins = [value.strip() for value in self.CORS_ALLOWED_ORIGINS.split(",") if value.strip()]
if "*" in origins:
raise ValueError("CORS_ALLOWED_ORIGINS must list explicit origins")
return origins
# One shared Tianditu key for geocoding, place search and other services.
TIANDITU_TOKEN: str = Field(
"",
validation_alias=AliasChoices(
"TIANDITU_TOKEN", "TJWATER_TIANDITU_TOKEN", "TIANDITU_GEOCODER_TOKEN"
),
)
TIANDITU_GEOCODER_URL: str = "https://api.tianditu.gov.cn/geocoder"
TIANDITU_GEOCODER_TIMEOUT_SECONDS: float = 30.0
TIANDITU_SEARCH_URL: str = "https://api.tianditu.gov.cn/v2/search"
TIANDITU_SEARCH_TIMEOUT_SECONDS: float = 15.0
@property
def SQLALCHEMY_DATABASE_URI(self) -> str:
db_password = quote_plus(self.DB_PASSWORD)
+17
View File
@@ -0,0 +1,17 @@
import logging
import re
class TiandituTokenFilter(logging.Filter):
def filter(self, record: logging.LogRecord) -> bool:
message = record.getMessage()
if "tianditu.gov.cn" in message:
record.msg = re.sub(r"([?&]tk=)[^&\s\"']+", r"\1[redacted]", message)
record.args = ()
return True
def install_tianditu_token_filter() -> None:
logger = logging.getLogger("httpx")
if not any(isinstance(item, TiandituTokenFilter) for item in logger.filters):
logger.addFilter(TiandituTokenFilter())
+22
View File
@@ -0,0 +1,22 @@
from __future__ import annotations
class DomainError(Exception):
"""A domain rejection with a stable API error payload."""
def __init__(self, code: str, message: str, *, refs: list | None = None,
recovery: str = "none", status: int = 422):
super().__init__(message)
self.code, self.message, self.status = code, message, status
self.refs, self.recovery = refs or [], recovery
def payload(self) -> dict:
return {"code": self.code, "message": self.message,
"retryable": self.recovery == "retry_same_request",
"related_refs": self.refs, "recovery_action": self.recovery,
"retry_after_seconds": None}
def require(condition: bool, code: str, message: str, **kwargs) -> None:
if not condition:
raise DomainError(code, message, **kwargs)
+1 -1
View File
@@ -5,7 +5,7 @@ from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, model_validator
BusinessRole = Literal["admin", "user"]
BusinessRole = Literal["admin", "service", "user"]
ProjectRole = Literal["member", "viewer"]
ProjectStatus = Literal["active", "inactive", "archived"]
ProjectDbRole = Literal["biz_data", "iot_data"]
+59
View File
@@ -0,0 +1,59 @@
"""Private, durable messages between an ordinary chat and one investigation run."""
from typing import Annotated, Literal
from uuid import UUID
from pydantic import Field
from app.domain.schemas.shared import NonEmptyStr, StrictModel
from app.domain.schemas.workbench_runtime import WorkbenchDateTime
MessageText = Annotated[str, Field(min_length=1, max_length=12000, pattern=r"\S")]
MessageStatus = Literal["pending", "enqueued", "delivered", "failed"]
class AgentMessageCreate(StrictModel):
request_id: UUID
task_id: UUID
run_id: UUID
source_session_id: NonEmptyStr
text: MessageText
class AgentMessageReply(StrictModel):
request_id: UUID
source_session_id: NonEmptyStr
text: MessageText
class AgentMessageDelivery(StrictModel):
status: Literal["enqueued", "delivered", "failed"]
target_session_id: NonEmptyStr | None = None
native_message_id: NonEmptyStr | None = None
failure_detail: Annotated[str, Field(max_length=500)] | None = None
class AgentMessage(StrictModel):
project_id: UUID
message_id: UUID
request_id: UUID
task_id: UUID
run_id: UUID
source_session_id: NonEmptyStr
source_owner_id: UUID
conversation_owner_id: UUID
target_session_id: NonEmptyStr | None = None
direction: Literal["to_task", "to_chat"]
in_reply_to: UUID | None = None
text: MessageText
status: MessageStatus
native_message_id: NonEmptyStr
failure_detail: str | None = None
created_at: WorkbenchDateTime
updated_at: WorkbenchDateTime
delivered_at: WorkbenchDateTime | None = None
class AgentMessagePage(StrictModel):
items: list[AgentMessage]
next_cursor: str | None = None
+47
View File
@@ -0,0 +1,47 @@
"""Schemas for project-scoped source GIS attributes."""
from datetime import date
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field
GisElementType = Literal[
"pipes",
"junctions",
"valves",
"reservoirs",
"pumps",
"tanks",
"scada",
]
class GisAttributeGroup(BaseModel):
model_config = ConfigDict(extra="forbid")
dataset_key: str
dataset_title: str
source_date: date
layer_key: str
layer_name: str
source_fid: int
match_method: Literal["node_pair", "geometry", "source"]
match_distance_m: float | None = None
attributes: dict[str, Any] = Field(default_factory=dict)
class ElementGisProperties(BaseModel):
model_config = ConfigDict(extra="forbid")
element_type: GisElementType
element_id: str
groups: list[GisAttributeGroup] = Field(default_factory=list)
class SourceGisFeatureProperties(BaseModel):
model_config = ConfigDict(extra="forbid")
layer_key: str
source_fid: int
group: GisAttributeGroup
+83
View File
@@ -0,0 +1,83 @@
"""Finite read scopes available to operational investigations."""
from datetime import timedelta
from typing import Literal
from pydantic import Field, JsonValue, model_validator
from app.domain.schemas.shared import NonEmptyStr, StrictModel
from app.domain.schemas.workbench_runtime import WorkbenchDateTime
TaskDataSource = Literal[
"workbench_runtime", "scada_devices", "scada_readings", "peripheral_points",
"peripheral_work_orders", "peripheral_observations", "peripheral_test_data",
]
McpSnapshotSource = Literal[
'run_simulation',
'run_contaminant_simulation',
'analyze_valve_closure',
'analyze_flushing',
'analyze_pressure_regulation',
'optimize_sensor_placement',
'run_burst_detection',
'run_burst_location',
'analyze_valve_isolation',
'run_leakage_identification',
'get_element_properties',
'list_elements',
'get_network_options',
'get_network_schema',
'list_scada_devices',
'get_scada_device',
'query_scada_readings',
'query_model_baseline_series',
'get_model_baseline_at_time',
'query_model_baseline_field',
'query_analysis_series',
'query_analysis_values',
'list_analysis_runs',
'get_analysis_run',
'list_analysis_results',
'list_sensor_placement_runs',
'get_sensor_placement_run',
'query_element_history',
'get_element_gis_properties',
'get_source_gis_properties',
'get_workbench_runtime',
'get_task_context',
'query_peripheral_data',
'get_task_evidence',
]
TaskSnapshotSource = TaskDataSource | McpSnapshotSource
class TaskDataQuery(StrictModel):
source: TaskDataSource
device_ids: list[NonEmptyStr] = Field(default_factory=list, max_length=20)
point_id: NonEmptyStr | None = None
start: WorkbenchDateTime | None = None
end: WorkbenchDateTime | None = None
@model_validator(mode="after")
def validate_window(self):
if self.source in {"scada_readings", "peripheral_observations"}:
if self.start is None or self.end is None:
raise ValueError("time series queries require start and end")
if not timedelta(0) < self.end - self.start <= timedelta(days=1):
raise ValueError("query window must be positive and at most 24 hours")
if self.source == "scada_readings" and not self.device_ids:
raise ValueError("SCADA queries require device_ids")
if self.source == "peripheral_observations" and not self.point_id:
raise ValueError("peripheral queries require point_id")
return self
class TaskDataSnapshot(StrictModel):
source: TaskSnapshotSource
source_ref: NonEmptyStr
queried_at: WorkbenchDateTime
origin: Literal["system_data", "synthetic"]
data: JsonValue
truncated: bool
+880
View File
@@ -0,0 +1,880 @@
"""Contracts for project-scoped operational tasks and human-approved actions."""
from __future__ import annotations
import re
from typing import Annotated, Literal
from uuid import UUID
from pydantic import Field, JsonValue, model_validator
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictBool, StrictModel
from app.domain.schemas.operational_data import TaskDataSnapshot, TaskDataSource
from app.domain.schemas.workbench_runtime import (
WorkbenchDateTime,
WorkOrderActivity,
)
TaskType = Literal["scheduled_flow", "external_work_order", "event_investigation"]
TaskStatus = Literal[
"queued", "running", "waiting_human", "completed", "failed", "cancelled"
]
TaskLifecycleStatus = Literal[
"open", "waiting_human", "completed", "failed", "cancelled"
]
OperationalTaskRunStatus = Literal[
"queued", "claimed", "running", "succeeded", "failed", "timed_out", "cancelled"
]
TaskPriority = Literal["normal", "urgent"]
TaskSourceKind = Literal["schedule", "external_system", "operator", "agent"]
TriggerKind = Literal["schedule", "external_sync", "event", "manual"]
EvidenceKind = Literal["system_data", "external_feedback", "human_input", "synthetic"]
PlanNodeStatus = Literal[
"pending", "waiting", "running", "completed", "failed", "skipped", "cancelled"
]
ActionKind = Literal["create_work_order", "issue_command"]
ApprovalStatus = Literal["pending", "approved", "rejected"]
ExecutionStatus = Literal[
"not_submitted", "submitted", "succeeded", "failed", "unsupported", "unknown"
]
class TaskSource(StrictModel):
kind: TaskSourceKind
external_id: NonEmptyStr | None = None
external_revision: PositiveInt | None = None
event_kind: NonEmptyStr | None = None
deleted_at: WorkbenchDateTime | None = None
class TaskTrigger(StrictModel):
kind: TriggerKind
occurred_at: WorkbenchDateTime
detail: NonEmptyStr
class OperationalTaskCreate(StrictModel):
request_id: UUID
task_type: TaskType
workflow: NonEmptyStr
title: NonEmptyStr
summary: NonEmptyStr
priority: TaskPriority = "normal"
source: TaskSource
trigger: TaskTrigger
scheduled_for: WorkbenchDateTime | None = None
agent_session_id: NonEmptyStr | None = None
class ExecutionPlanNode(StrictModel):
id: NonEmptyStr
title: NonEmptyStr
owner: Literal["agent", "human"]
status: PlanNodeStatus
detail: NonEmptyStr
evidence_refs: list[NonEmptyStr] = Field(default_factory=list)
action_refs: list[UUID] = Field(default_factory=list)
kind: Literal["activity", "decision", "parallel_split", "parallel_join", "gate"] | None = None
selected_edge_id: NonEmptyStr | None = None
gate_ref: UUID | None = None
revisits_node_id: NonEmptyStr | None = None
class ExecutionPlanEdge(StrictModel):
id: NonEmptyStr | None = None
from_node: NonEmptyStr = Field(alias="from")
to_node: NonEmptyStr = Field(alias="to")
condition: NonEmptyStr | None = None
class ExecutionPlanInput(StrictModel):
schema_name: Literal["execution-plan@1", "execution-plan@2"] = Field(alias="schema")
title: NonEmptyStr
reason: NonEmptyStr
nodes: Annotated[list[ExecutionPlanNode], Field(min_length=1, max_length=100)]
edges: Annotated[list[ExecutionPlanEdge], Field(max_length=300)] = Field(
default_factory=list
)
@model_validator(mode="after")
def validate_graph(self):
node_ids = [node.id for node in self.nodes]
if len(set(node_ids)) != len(node_ids):
raise ValueError("execution plan node ids must be unique")
known = set(node_ids)
graph = {node_id: [] for node_id in node_ids}
indegree = {node_id: 0 for node_id in node_ids}
seen_edges: set[tuple[str, str]] = set()
edge_ids: set[str] = set()
for edge in self.edges:
pair = (edge.from_node, edge.to_node)
if edge.from_node not in known or edge.to_node not in known:
raise ValueError("execution plan edges must reference known nodes")
if edge.from_node == edge.to_node or pair in seen_edges:
raise ValueError("execution plan edges must be unique and non-reflexive")
seen_edges.add(pair)
if self.schema_name == "execution-plan@2":
if edge.id is None or edge.id in edge_ids:
raise ValueError("v2 execution plan edge ids must be unique")
edge_ids.add(edge.id)
graph[edge.from_node].append(edge.to_node)
indegree[edge.to_node] += 1
pending = [node_id for node_id, value in indegree.items() if value == 0]
visited = 0
while pending:
current = pending.pop()
visited += 1
for target in graph[current]:
indegree[target] -= 1
if indegree[target] == 0:
pending.append(target)
if visited != len(node_ids):
raise ValueError("execution plan must be acyclic")
if self.schema_name == "execution-plan@2" and sum(
not any(edge.to_node == node_id for edge in self.edges) for node_id in node_ids
) != 1:
raise ValueError("v2 execution plan requires one start node")
if self.schema_name == "execution-plan@2":
by_id = {node.id: node for node in self.nodes}
for node in self.nodes:
if node.kind is None:
raise ValueError("v2 execution plan nodes require kind")
outgoing = [edge for edge in self.edges if edge.from_node == node.id]
incoming = [edge for edge in self.edges if edge.to_node == node.id]
if node.kind == "decision":
if len(outgoing) < 2 or any(not edge.condition for edge in outgoing):
raise ValueError("decision requires at least two labeled routes")
if node.selected_edge_id is not None and node.selected_edge_id not in {edge.id for edge in outgoing}:
raise ValueError("decision selected_edge_id must reference an outgoing route")
if node.status == "completed" and node.selected_edge_id is None:
raise ValueError("completed decision requires a selected route")
if node.selected_edge_id is not None:
for edge in outgoing:
target_status = by_id[edge.to_node].status
if edge.id == node.selected_edge_id and target_status in {"skipped", "cancelled"}:
raise ValueError("selected route cannot be skipped")
if edge.id != node.selected_edge_id and target_status in {"running", "completed", "waiting"}:
raise ValueError("unchosen route cannot be active")
elif node.selected_edge_id is not None:
raise ValueError("only decision nodes may select routes")
if node.kind not in {"decision", "parallel_split"} and len(outgoing) > 1:
raise ValueError("only decision or parallel split may branch")
if node.kind == "parallel_split" and len(outgoing) < 2:
raise ValueError("parallel split requires at least two routes")
if node.kind == "parallel_join" and len(incoming) < 2:
raise ValueError("parallel join requires at least two inputs")
if node.kind == "gate" and node.gate_ref is None:
raise ValueError("gate requires a persisted reference")
if node.kind != "gate" and node.gate_ref is not None:
raise ValueError("only gate nodes may have gate_ref")
if node.revisits_node_id is not None and node.revisits_node_id not in by_id:
raise ValueError("revisited node must exist in the plan")
return self
class EvidenceInput(StrictModel):
evidence_id: NonEmptyStr
kind: EvidenceKind
summary: NonEmptyStr
source_label: NonEmptyStr
source_ref: NonEmptyStr | None = None
observed_at: WorkbenchDateTime
class EvidenceRecord(EvidenceInput):
recorded_at: WorkbenchDateTime
recorded_by: NonEmptyStr
has_snapshot: bool = False
class StoredEvidenceRecord(EvidenceRecord):
data_snapshot: TaskDataSnapshot | None = None
class InvestigationDataGap(StrictModel):
gap_id: NonEmptyStr
node_id: NonEmptyStr | None = None
missing_data: NonEmptyStr
scope: NonEmptyStr
impact: NonEmptyStr
required_data: NonEmptyStr
evidence_refs: list[NonEmptyStr] = Field(min_length=1)
blocking: StrictBool
TaskMapLayer = Literal["junctions", "pipes", "valves", "pumps", "tanks", "reservoirs", "scada"]
TaskFeatureId = Annotated[str, Field(min_length=1, max_length=128)]
class InvestigationMapTarget(StrictModel):
source_id: TaskMapLayer
feature_id: TaskFeatureId
class InvestigationDiagramNodeRef(StrictModel):
node_id: Annotated[str, Field(pattern=r"^[A-Za-z_][A-Za-z0-9_-]{0,79}$")]
label: NonEmptyStr
evidence_refs: list[NonEmptyStr] = Field(default_factory=list)
action_refs: list[UUID] = Field(default_factory=list)
status: Literal[
"pending", "running", "completed", "waiting_human", "failed", "skipped", "cancelled"
] | None = None
map_targets: list[InvestigationMapTarget] = Field(default_factory=list, max_length=100)
class InvestigationDiagram(StrictModel):
title: NonEmptyStr
mermaid: Annotated[str, Field(min_length=1, max_length=20000)]
node_refs: list[InvestigationDiagramNodeRef] = Field(default_factory=list, max_length=100)
@model_validator(mode="after")
def validate_description(self):
if not re.match(r"^\s*(?:flowchart|graph)\s+(?:TD|TB|BT|LR|RL)\b", self.mermaid):
raise ValueError("investigation diagrams must be Mermaid flowcharts")
if "```" in self.mermaid or "%%{" in self.mermaid or re.search(r"(?:^|[;\n])\s*click\b", self.mermaid):
raise ValueError("diagram descriptions cannot contain fences, directives or click commands")
ids = [node.node_id for node in self.node_refs]
if len(ids) != len(set(ids)):
raise ValueError("diagram node references must be unique")
return self
class InvestigationReport(StrictModel):
"""Read model: absent historical facts remain unknown, never inferred."""
title: NonEmptyStr
current_conclusion: NonEmptyStr
conclusion_boundary: NonEmptyStr
evidence_refs: list[NonEmptyStr] = Field(default_factory=list)
recommendations: list[NonEmptyStr] = Field(default_factory=list)
pending_items: list[NonEmptyStr] = Field(default_factory=list)
stage: Literal["interim", "final"] | None = None
outcome: Literal["concluded", "insufficient_data"] | None = None
stop_reason: NonEmptyStr | None = None
data_gaps: list[InvestigationDataGap] | None = None
diagram: InvestigationDiagram | None = None
@model_validator(mode="after")
def validate_result(self):
if self.outcome is None:
if self.stage is not None or self.stop_reason is not None or self.data_gaps is not None:
raise ValueError("structured report fields must be supplied together")
return self
if self.stage is None or self.data_gaps is None:
raise ValueError("structured reports require stage and data_gaps")
ids = [gap.gap_id for gap in self.data_gaps]
if len(ids) != len(set(ids)):
raise ValueError("data gap ids must be unique")
blocking = any(gap.blocking for gap in self.data_gaps)
if self.outcome == "insufficient_data":
if self.stage != "final" or not self.stop_reason or not blocking:
raise ValueError("insufficient data requires a final report, stop reason and blocking gap")
if self.recommendations or self.pending_items:
raise ValueError("insufficient data ends the investigation without downstream recommendations or pending work")
elif self.stop_reason is not None or blocking:
raise ValueError("concluded report cannot contain a stop reason or blocking gap")
return self
class InvestigationReportInput(InvestigationReport):
"""Write model: every new report explicitly declares its business result."""
stage: Literal["interim", "final"]
outcome: Literal["concluded", "insufficient_data"]
data_gaps: list[InvestigationDataGap]
class InvestigationProgress(StrictModel):
"""A complete reading snapshot, not a plan or a business result."""
summary: Annotated[str, Field(min_length=1, max_length=20000)]
diagram: InvestigationDiagram | None = None
class InvestigationProgressSnapshot(StrictModel):
revision: PositiveInt
run_id: UUID
execution_generation: PositiveInt
published_at: WorkbenchDateTime
source_session_id: NonEmptyStr
progress: InvestigationProgress
class ExecutionPlanSnapshot(StrictModel):
revision: PositiveInt
execution_generation: PositiveInt | None = None
source_session_id: NonEmptyStr
published_at: WorkbenchDateTime
published_by: NonEmptyStr
plan: ExecutionPlanInput
class InvestigationReportRevision(StrictModel):
revision: PositiveInt
execution_generation: PositiveInt | None = None
source_session_id: NonEmptyStr
published_at: WorkbenchDateTime
published_by: NonEmptyStr
report: InvestigationReport
class InvestigationReportPage(StrictModel):
items: list[InvestigationReportRevision]
next_offset: int | None = Field(default=None, ge=0)
class InvestigationReportReference(StrictModel):
execution_generation: PositiveInt
revision: PositiveInt
class AgentArtifactsCommand(StrictModel):
request_id: UUID
source_session_id: NonEmptyStr
run_id: UUID | None = None
evidence: list[EvidenceInput] = Field(default_factory=list, max_length=100)
plan: ExecutionPlanInput | None = None
report: InvestigationReportInput | None = None
progress: InvestigationProgress | None = None
@model_validator(mode="after")
def require_artifact(self):
if not self.evidence and self.plan is None and self.report is None and self.progress is None:
raise ValueError("at least one artifact must be supplied")
return self
class TaskAccessRequestCreate(StrictModel):
request_id: UUID
source_session_id: NonEmptyStr
node_id: NonEmptyStr
scopes: Annotated[list[TaskDataSource], Field(min_length=1, max_length=7)]
reason: NonEmptyStr
class TaskAccessRequestDecision(StrictModel):
request_id: UUID
expected_version: PositiveInt
decision: Literal["approved", "rejected"]
approved_scopes: list[TaskDataSource] = Field(default_factory=list)
comment: NonEmptyStr | None = None
class TaskAccessRequest(StrictModel):
access_request_id: UUID
task_id: UUID
node_id: NonEmptyStr
scopes: list[TaskDataSource]
reason: NonEmptyStr
status: Literal["pending", "approved", "rejected"]
approved_scopes: list[TaskDataSource]
version: PositiveInt
requested_by: NonEmptyStr
requested_at: WorkbenchDateTime
decided_by: NonEmptyStr | None = None
decided_at: WorkbenchDateTime | None = None
comment: NonEmptyStr | None = None
class CreateWorkOrderPayload(StrictModel):
title: NonEmptyStr
description: NonEmptyStr
work_order_type: NonEmptyStr
priority: TaskPriority = "normal"
scheduled_at: WorkbenchDateTime | None = None
location: NonEmptyStr | None = None
assignee: NonEmptyStr | None = None
class CommandTarget(StrictModel):
kind: NonEmptyStr
id: NonEmptyStr
class IssueCommandPayload(StrictModel):
command_type: NonEmptyStr
target: CommandTarget
parameters: dict[str, JsonValue]
impact_summary: NonEmptyStr
preconditions: list[NonEmptyStr] = Field(min_length=1)
expires_at: WorkbenchDateTime
ActionPayload = CreateWorkOrderPayload | IssueCommandPayload
class ActionProposalCommand(StrictModel):
request_id: UUID
kind: ActionKind
reason: NonEmptyStr
payload: ActionPayload
evidence_refs: list[NonEmptyStr] = Field(default_factory=list)
node_id: NonEmptyStr | None = None
@model_validator(mode="after")
def validate_payload_kind(self):
expected = CreateWorkOrderPayload if self.kind == "create_work_order" else IssueCommandPayload
if not isinstance(self.payload, expected):
raise ValueError("action payload does not match its kind")
return self
class ActionDecisionCommand(StrictModel):
request_id: UUID
expected_version: PositiveInt
decision: Literal["approved", "rejected"]
comment: NonEmptyStr | None = None
class WorkOrderDraftUpdateCommand(StrictModel):
request_id: UUID
expected_version: PositiveInt
payload: CreateWorkOrderPayload
class WorkOrderConfirmDispatchCommand(StrictModel):
request_id: UUID
expected_version: PositiveInt
class ActionExecuteCommand(StrictModel):
request_id: UUID
expected_version: PositiveInt
class ActionReconcileCommand(StrictModel):
request_id: UUID
expected_version: PositiveInt
class ActionReceipt(StrictModel):
adapter: NonEmptyStr
receipt_id: NonEmptyStr
submitted_at: WorkbenchDateTime
completed_at: WorkbenchDateTime | None = None
external_status: NonEmptyStr
detail: NonEmptyStr
class HumanApprovalAction(StrictModel):
action_id: UUID
task_id: UUID
version: PositiveInt
kind: ActionKind
reason: NonEmptyStr
payload: ActionPayload
evidence_refs: list[NonEmptyStr]
node_id: NonEmptyStr | None = None
required_approval_permission: NonEmptyStr
required_execution_permission: NonEmptyStr
approval_status: ApprovalStatus
execution_status: ExecutionStatus
execution_request_id: UUID | None = None
proposed_by: NonEmptyStr
proposed_at: WorkbenchDateTime
edited_by: NonEmptyStr | None = None
edited_at: WorkbenchDateTime | None = None
decided_by: NonEmptyStr | None = None
decided_at: WorkbenchDateTime | None = None
decision_comment: NonEmptyStr | None = None
executed_by: NonEmptyStr | None = None
executed_at: WorkbenchDateTime | None = None
receipt: ActionReceipt | None = None
execution_detail: NonEmptyStr | None = None
class ExternalWorkOrderSnapshot(StrictModel):
work_order_id: UUID
revision: PositiveInt
title: NonEmptyStr
summary: NonEmptyStr
status: Literal[
"draft", "open", "in_progress", "awaiting_reply", "completed", "cancelled"
]
priority: Literal["low", "normal", "high", "urgent"]
assignee: NonEmptyStr | None = None
scheduled_at: WorkbenchDateTime | None = None
work_order_type: Literal["general", "water_quality_complaint", "burst_report"] = "general"
location: NonEmptyStr | None = None
scenario_code: NonEmptyStr | None = None
feedback_requirements: list[NonEmptyStr] = Field(default_factory=list)
activity: list[WorkOrderActivity] = Field(default_factory=list)
origin: Literal["synthetic"]
updated_at: WorkbenchDateTime
@model_validator(mode="before")
@classmethod
def accept_legacy_snapshot(cls, value):
if isinstance(value, dict):
return {key: item for key, item in value.items() if key != "subtasks"}
return value
class OperationalTaskRun(StrictModel):
run_id: UUID
task_id: UUID
generation: PositiveInt
execution_request_id: UUID
status: OperationalTaskRunStatus
revision: PositiveInt
agent_session_id: NonEmptyStr | None = None
lease_expires_at: WorkbenchDateTime | None = None
heartbeat_at: WorkbenchDateTime | None = None
started_at: WorkbenchDateTime | None = None
finished_at: WorkbenchDateTime | None = None
failure_code: NonEmptyStr | None = None
failure_detail: NonEmptyStr | None = None
dispatch_attempts: int = Field(default=0, ge=0)
next_dispatch_at: WorkbenchDateTime
created_at: WorkbenchDateTime
updated_at: WorkbenchDateTime
class OperationalTaskSummary(StrictModel):
project_id: UUID
task_id: UUID
task_type: TaskType
workflow: NonEmptyStr
lifecycle_status: TaskLifecycleStatus = "open"
active_run: OperationalTaskRun | None = None
latest_run: OperationalTaskRun | None = None
# Compatibility fields for the staged 014 -> 015 rollout. New consumers
# must use lifecycle_status and active_run.
status: TaskStatus
priority: TaskPriority
revision: PositiveInt
execution_generation: PositiveInt
execution_request_id: UUID
title: NonEmptyStr
summary: NonEmptyStr
source: TaskSource
trigger: TaskTrigger
scheduled_for: WorkbenchDateTime | None
agent_session_id: NonEmptyStr | None
agent_lease_expires_at: WorkbenchDateTime | None
current_plan_revision: PositiveInt | None
current_report_revision: PositiveInt | None
created_at: WorkbenchDateTime
updated_at: WorkbenchDateTime
class OperationalTaskPermissions(StrictModel):
can_publish_artifacts: bool
can_propose_actions: bool
can_sync_external_feedback: bool
can_decide_work_orders: bool
can_execute_work_orders: bool
can_decide_commands: bool
can_execute_commands: bool
can_decide_access: bool = False
class OperationalTaskDetail(OperationalTaskSummary):
latest_plan: ExecutionPlanSnapshot | None
latest_report: InvestigationReportRevision | None
latest_progress: InvestigationProgressSnapshot | None = None
evidence: list[EvidenceRecord]
actions: list[HumanApprovalAction]
access_requests: list[TaskAccessRequest] = Field(default_factory=list)
permissions: OperationalTaskPermissions
work_order_snapshot: ExternalWorkOrderSnapshot | None = None
class OperationalTaskPage(StrictModel):
items: list[OperationalTaskSummary]
limit: PositiveInt
offset: int = Field(ge=0)
next_offset: int | None = Field(default=None, ge=0)
class OperationalTaskRunPage(StrictModel):
items: list[OperationalTaskRun]
next_cursor: NonEmptyStr | None = None
class AgentTaskContext(StrictModel):
task: OperationalTaskDetail
approved_data_scopes: list[NonEmptyStr]
selected_report: InvestigationReportRevision | None = None
class AgentRunCommand(StrictModel):
request_id: UUID
expected_revision: PositiveInt
source_session_id: NonEmptyStr
class AgentRunFailureCommand(AgentRunCommand):
failure_code: NonEmptyStr
failure_detail: NonEmptyStr
class AgentRunCancelCommand(StrictModel):
request_id: UUID
expected_revision: PositiveInt
reason: NonEmptyStr
class AgentRunDispatchFailureCommand(StrictModel):
request_id: UUID
expected_revision: PositiveInt
failure_code: NonEmptyStr
failure_detail: NonEmptyStr
retryable: bool = True
class OperationalTaskRetryCommand(StrictModel):
request_id: UUID
reason: NonEmptyStr
class RecoverExpiredLeasesCommand(StrictModel):
limit: int = Field(default=20, ge=1, le=100)
task_type: TaskType | None = None
task_created_from: WorkbenchDateTime | None = None
class RecoverExpiredLeasesResult(StrictModel):
task_ids: list[UUID]
recovered_at: WorkbenchDateTime
class AgentArtifactsResult(StrictModel):
task: OperationalTaskDetail
plan_revision: PositiveInt | None
report_revision: PositiveInt | None
progress_revision: PositiveInt | None = None
class TaskSafeChartSeries(StrictModel):
name: Annotated[str, Field(max_length=128)]
data: Annotated[list[Annotated[float, Field(strict=True, allow_inf_nan=False)]], Field(min_length=1, max_length=100)]
class TaskSafeChart(StrictModel):
title: Annotated[str, Field(max_length=200)]
chart_type: Literal["line", "bar"]
x_data: Annotated[list[Annotated[str, Field(max_length=128)]], Field(min_length=1, max_length=100)]
series: Annotated[list[TaskSafeChartSeries], Field(min_length=1, max_length=8)]
@model_validator(mode="after")
def validate_alignment(self):
if any(len(series.data) != len(self.x_data) for series in self.series):
raise ValueError("chart series must align with x_data")
return self
class TaskHighlightFeatures(StrictModel):
action: Literal["highlight_features"]
layer: TaskMapLayer
feature_ids: Annotated[list[TaskFeatureId], Field(min_length=1, max_length=100)]
effect: Literal["steady", "pulse"] = "steady"
class TaskClearHighlights(StrictModel):
action: Literal["clear_highlights"]
class TaskOpenChart(StrictModel):
action: Literal["open_chart"]
chart_id: TaskFeatureId
chart: TaskSafeChart
evidence_refs: Annotated[list[NonEmptyStr], Field(min_length=1, max_length=100)]
class TaskCloseChart(StrictModel):
action: Literal["close_chart"]
chart_id: TaskFeatureId
class TaskSetOverlay(StrictModel):
action: Literal["set_overlay"]
overlay_id: TaskFeatureId
title: Annotated[str, Field(min_length=1, max_length=200)]
mode: Literal["geometry", "heatmap"]
evidence_id: NonEmptyStr
class TaskClearOverlay(StrictModel):
action: Literal["clear_overlay"]
overlay_id: TaskFeatureId
TaskPresentationCommand = Annotated[
TaskHighlightFeatures | TaskClearHighlights | TaskOpenChart | TaskCloseChart | TaskSetOverlay | TaskClearOverlay,
Field(discriminator="action"),
]
class TaskPresentationEventCommand(StrictModel):
request_id: UUID
run_id: UUID
source_session_id: NonEmptyStr
command: TaskPresentationCommand
class TaskPresentationEventResult(StrictModel):
recorded: Literal[True] = True
event_seq: PositiveInt
run_id: UUID
execution_generation: PositiveInt
class TaskPresentationActiveRun(StrictModel):
run_id: UUID
generation: PositiveInt
status: OperationalTaskRunStatus
class TaskPresentationCursor(StrictModel):
task_id: UUID
execution_generation: PositiveInt
active_run: TaskPresentationActiveRun | None
event_cursor: int = Field(ge=0)
class TaskDataQueryResult(StrictModel):
evidence: EvidenceRecord
snapshot: TaskDataSnapshot
class ExternalFeedbackEvidence(StrictModel):
evidence_id: NonEmptyStr
kind: Literal["external_feedback", "synthetic"] = "external_feedback"
summary: NonEmptyStr
source_label: NonEmptyStr
source_ref: NonEmptyStr | None = None
observed_at: WorkbenchDateTime
class ExternalFeedbackCommand(StrictModel):
request_id: UUID
source_event_id: NonEmptyStr
source_revision: PositiveInt
status: Literal[
"scheduled", "assigned", "in_progress", "awaiting_feedback",
"completed", "cancelled", "failed"
]
occurred_at: WorkbenchDateTime
detail: NonEmptyStr
evidence: ExternalFeedbackEvidence | None = None
work_order_snapshot: ExternalWorkOrderSnapshot | None = None
@model_validator(mode="after")
def snapshot_matches_feedback_revision(self):
if self.work_order_snapshot is not None:
work_order_id = str(self.work_order_snapshot.work_order_id)
if self.source_event_id not in {
work_order_id, f"peripheral_sample:{work_order_id}"
}:
raise ValueError("work_order_snapshot id must match source_event_id")
if self.work_order_snapshot.revision != self.source_revision:
raise ValueError("work_order_snapshot revision must match source_revision")
expected_status = {
"draft": "scheduled",
"open": "assigned",
"in_progress": "in_progress",
"awaiting_reply": "awaiting_feedback",
"completed": "completed",
"cancelled": "cancelled",
}[self.work_order_snapshot.status]
if self.status != expected_status:
raise ValueError("work_order_snapshot status must match feedback status")
return self
class ExternalFeedbackResult(StrictModel):
task: OperationalTaskDetail
source_event_id: NonEmptyStr
source_revision: PositiveInt
accepted_at: WorkbenchDateTime
class ExternalSourceWithdrawalCommand(StrictModel):
request_id: UUID
source_event_id: NonEmptyStr
class HumanEvidenceCommand(StrictModel):
request_id: UUID
evidence: EvidenceInput
class OperationalTaskNotification(StrictModel):
notification_id: UUID
task_id: UUID
kind: Literal[
"task_created", "action_pending", "report_updated", "task_completed",
"task_failed", "run_delivery_degraded", "access_approval_pending",
"action_approval_pending", "human_input_required",
"external_coordination_failed"
]
title: NonEmptyStr
description: NonEmptyStr
required_permission: NonEmptyStr | None = None
lifecycle_status: Literal["active", "resolved"] = "active"
subject_type: Literal["task", "run", "access_request", "action"] = "task"
subject_id: UUID | None = None
report_reference: InvestigationReportReference | None = None
resolved_at: WorkbenchDateTime | None = None
resolution_reason: NonEmptyStr | None = None
status: Literal["unread", "read"]
version: PositiveInt
created_at: WorkbenchDateTime
read_at: WorkbenchDateTime | None = None
read_by: NonEmptyStr | None = None
class OperationalTaskNotificationPage(StrictModel):
items: list[OperationalTaskNotification]
limit: PositiveInt
offset: int = Field(ge=0)
next_offset: int | None = Field(default=None, ge=0)
class NotificationAcknowledgeCommand(StrictModel):
request_id: UUID
expected_version: PositiveInt
class OperationalTaskScheduleCreate(StrictModel):
request_id: UUID
schedule_key: NonEmptyStr
title: NonEmptyStr
summary: NonEmptyStr
interval_seconds: int = Field(ge=60, le=31_536_000)
next_run_at: WorkbenchDateTime
priority: TaskPriority = "normal"
class OperationalTaskSchedule(StrictModel):
schedule_id: UUID
schedule_key: NonEmptyStr
title: NonEmptyStr
summary: NonEmptyStr
interval_seconds: int = Field(ge=60, le=31_536_000)
next_run_at: WorkbenchDateTime
priority: TaskPriority
enabled: bool
version: PositiveInt
created_at: WorkbenchDateTime
updated_at: WorkbenchDateTime
class OperationalTaskSchedulePage(StrictModel):
items: list[OperationalTaskSchedule]
class OperationalTaskScheduleUpdate(StrictModel):
request_id: UUID
expected_version: PositiveInt
enabled: bool
class DispatchDueSchedulesCommand(StrictModel):
limit: int = Field(default=20, ge=1, le=100)
class DispatchDueSchedulesResult(StrictModel):
task_ids: list[UUID]
dispatched_at: WorkbenchDateTime
+28
View File
@@ -0,0 +1,28 @@
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
MAX_PLACE_SEARCH_RESULTS = 10
class PlaceSearchRequest(BaseModel):
model_config = ConfigDict(str_strip_whitespace=True, extra="forbid")
keyword: str = Field(..., min_length=1, max_length=120)
class PlaceSearchItem(BaseModel):
id: str = Field(..., min_length=1)
name: str = Field(..., min_length=1)
address: str
longitude: float = Field(..., ge=-180, le=180, allow_inf_nan=False)
latitude: float = Field(..., ge=-90, le=90, allow_inf_nan=False)
category: str | None = None
district: Literal["奉贤区"] = "奉贤区"
class PlaceSearchResponse(BaseModel):
items: list[PlaceSearchItem] = Field(..., max_length=MAX_PLACE_SEARCH_RESULTS)
total: int = Field(..., ge=0, le=MAX_PLACE_SEARCH_RESULTS)
truncated: bool = False
region: Literal["奉贤区"] = "奉贤区"
+3 -1
View File
@@ -25,7 +25,7 @@ class SensorPlacementOptimizeRequest(BaseModel):
pattern=r"^[^/\\\x00]+$",
)
run_name: str = Field(..., min_length=1, max_length=64)
sensor_type: Literal["pressure"]
sensor_type: Literal["pressure", "water_quality"]
method: Literal["sensitivity", "kmeans"]
sensor_count: int = Field(..., gt=0, le=200)
min_diameter: int = Field(default=0, ge=0)
@@ -89,6 +89,8 @@ class SensorPlacementSchemeResponse(BaseModel):
created_by: str
created_at: datetime
status: str
sensor_type: Literal["pressure", "water_quality"] = "pressure"
method: Literal["sensitivity", "kmeans"] | None = None
sensor_locations: list[str]
sensor_points: list[SensorPointResponse]
can_edit: bool = False
+58
View File
@@ -0,0 +1,58 @@
"""Small strict value types shared by independent backend features."""
from datetime import datetime
from typing import Annotated, Literal
from uuid import UUID
from pydantic import BaseModel, BeforeValidator, ConfigDict, Field, StringConstraints, model_validator
NonEmptyStr = Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)]
StrictBool = Annotated[bool, Field(strict=True)]
PositiveInt = Annotated[int, Field(strict=True, ge=1)]
NonNegativeInt = Annotated[int, Field(strict=True, ge=0)]
def _datetime_wire_value(value: object) -> datetime | str:
if not isinstance(value, (datetime, str)):
raise ValueError("datetime input must be an RFC3339 string or datetime instance")
return value
WireDateTime = Annotated[datetime, BeforeValidator(_datetime_wire_value)]
class StrictModel(BaseModel):
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
ObjectType = Literal[
"analysis_task",
"goal",
"state_snapshot",
"action_contract",
"validation",
"run",
"outcome",
"evidence",
"artifact",
"operation",
]
class ObjectRef(StrictModel):
object_type: ObjectType
object_id: UUID
revision: PositiveInt
class TimeRange(StrictModel):
start: WireDateTime
end: WireDateTime
@model_validator(mode="after")
def validate_range(self) -> "TimeRange":
if self.start.tzinfo is None or self.end.tzinfo is None:
raise ValueError("timestamps must include a timezone")
if self.start >= self.end:
raise ValueError("start must be before end")
return self
+38
View File
@@ -0,0 +1,38 @@
"""Idempotent alert events emitted by scheduled and other backend tasks."""
from __future__ import annotations
from typing import Literal
from uuid import UUID
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictModel
from app.domain.schemas.workbench_runtime import WorkbenchAlertSource, WorkbenchDateTime
class TaskAlertCommand(StrictModel):
event_id: UUID
alert_id: NonEmptyStr
event_type: Literal["raised", "acknowledged", "resolved"]
occurred_at: WorkbenchDateTime
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
title: NonEmptyStr
description: NonEmptyStr
severity: Literal["warning", "critical"]
class TaskAlertEvent(TaskAlertCommand):
producer_id: NonEmptyStr
class TaskAlertState(StrictModel):
alert_id: NonEmptyStr
revision: PositiveInt
producer_id: NonEmptyStr
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
title: NonEmptyStr
description: NonEmptyStr
occurred_at: WorkbenchDateTime
severity: Literal["warning", "critical"]
status: Literal["active", "acknowledged", "resolved"]
updated_at: WorkbenchDateTime
+1
View File
@@ -86,3 +86,4 @@ class ElementHistorySeries(BaseModel):
class ElementHistoryResponse(BaseModel):
series: list[ElementHistorySeries]
truncated: bool = Field(default=False, exclude=True)
+51
View File
@@ -0,0 +1,51 @@
"""Stable workbench event-stream and synchronization contracts."""
from __future__ import annotations
from typing import Literal
from uuid import UUID
from pydantic import Field, JsonValue
from app.domain.schemas.operational_task import (
OperationalTaskNotification,
OperationalTaskSummary,
)
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictModel
from app.domain.schemas.workbench_runtime import WorkbenchDateTime, WorkbenchRuntimeSnapshot
WorkbenchEventTopic = Literal[
"operational.task.changed",
"operational.run.queued",
"operational.run.changed",
"operational.report.published",
"operational.progress.changed",
"operational.presentation.requested",
"operational.action.changed",
"operational.notification.changed",
"workbench.alert.changed",
"workbench.work_order.changed",
"analysis.run.changed",
]
class WorkbenchEvent(StrictModel):
schema_name: Literal["workbench-event@1"] = Field(alias="schema")
event_seq: PositiveInt
event_id: UUID
project_id: UUID
topic: WorkbenchEventTopic
aggregate_type: NonEmptyStr
aggregate_id: NonEmptyStr
aggregate_revision: PositiveInt
occurred_at: WorkbenchDateTime
metadata: dict[str, JsonValue]
class WorkbenchSyncSnapshot(StrictModel):
schema_name: Literal["workbench-sync-snapshot@1"] = Field(alias="schema")
event_cursor: int = Field(ge=0)
runtime: WorkbenchRuntimeSnapshot
tasks: list[OperationalTaskSummary]
notifications: list[OperationalTaskNotification]
generated_at: WorkbenchDateTime
+192
View File
@@ -0,0 +1,192 @@
"""Typed read-only projections consumed by the supply workbench."""
from typing import Annotated, Literal
from pydantic import AwareDatetime, BeforeValidator, Field, JsonValue, model_validator
from app.domain.schemas.shared import (
NonEmptyStr,
NonNegativeInt,
PositiveInt,
StrictBool,
StrictModel,
)
def _workbench_datetime_wire_value(value: object):
from datetime import datetime
if not isinstance(value, (datetime, str)):
raise ValueError("datetime input must be an RFC3339 string or datetime instance")
return value
WorkbenchDateTime = Annotated[
AwareDatetime, BeforeValidator(_workbench_datetime_wire_value)
]
ProjectionAvailability = Literal["ready", "not_configured", "degraded"]
class WorkbenchRuntimeCondition(StrictModel):
id: NonEmptyStr
task_id: Literal["backend-schedule"]
session_id: NonEmptyStr
title: NonEmptyStr
summary: NonEmptyStr
status: Literal["pending", "running", "completed", "warning", "error"]
risk_level: Literal["normal", "attention", "critical"]
scheduled_at: WorkbenchDateTime
updated_at: WorkbenchDateTime
detail: NonEmptyStr
recommendation: NonEmptyStr | None
evidence: list[NonEmptyStr]
model_name: Literal["TJWater scheduler"]
class WorkbenchAlertSource(StrictModel):
task_type: Literal["scheduled_task", "analysis_task", "other_task"]
task_id: NonEmptyStr
run_id: NonEmptyStr
class WorkbenchRuntimeAlert(StrictModel):
id: NonEmptyStr
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
title: NonEmptyStr
description: NonEmptyStr
occurred_at: WorkbenchDateTime
severity: Literal["warning", "critical"]
class WorkbenchPersistedAlert(StrictModel):
alert_id: NonEmptyStr
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
title: NonEmptyStr
description: NonEmptyStr
occurred_at: WorkbenchDateTime
severity: Literal["warning", "critical"]
status: Literal["active", "acknowledged", "resolved"]
producer_id: NonEmptyStr | None = None
updated_at: WorkbenchDateTime | None = None
class WorkbenchRuntimeScenario(StrictModel):
id: NonEmptyStr
run_id: NonEmptyStr
name: NonEmptyStr
description: NonEmptyStr
status: Literal["draft", "active", "review"]
run_type: NonEmptyStr
run_type_label: NonEmptyStr
run_status: NonEmptyStr
created_at: WorkbenchDateTime
class WorkbenchRuntimeAnalysisFact(StrictModel):
label: NonEmptyStr
value: NonEmptyStr
class WorkbenchRuntimeAnalysisResult(StrictModel):
result_type: NonEmptyStr
created_at: WorkbenchDateTime
facts: list[WorkbenchRuntimeAnalysisFact]
class WorkbenchRuntimeLatestAnalysis(StrictModel):
run_id: NonEmptyStr
name: NonEmptyStr
run_type: NonEmptyStr
run_type_label: NonEmptyStr
status: NonEmptyStr
status_label: NonEmptyStr
created_by: NonEmptyStr
created_at: WorkbenchDateTime
started_at: WorkbenchDateTime
results: list[WorkbenchRuntimeAnalysisResult]
map_payload: dict[str, JsonValue] | None
class WorkbenchAnalysisHistoryItem(StrictModel):
run_id: NonEmptyStr
name: NonEmptyStr
run_type: NonEmptyStr
run_type_label: NonEmptyStr
status: NonEmptyStr
status_label: NonEmptyStr
created_by: NonEmptyStr
created_at: WorkbenchDateTime
started_at: WorkbenchDateTime
has_results: StrictBool
class WorkbenchAnalysisHistoryPage(StrictModel):
items: list[WorkbenchAnalysisHistoryItem]
limit: PositiveInt
offset: NonNegativeInt
next_offset: NonNegativeInt | None
class WorkOrderActivity(StrictModel):
sequence: PositiveInt
occurred_at: WorkbenchDateTime
actor: NonEmptyStr
kind: Literal["created", "updated", "feedback"]
title: NonEmptyStr
detail: NonEmptyStr
class WorkbenchRuntimeWorkOrder(StrictModel):
id: NonEmptyStr
code: NonEmptyStr
scheduled_at: WorkbenchDateTime
title: NonEmptyStr
summary: NonEmptyStr
status: Literal["pending", "running", "awaiting_reply", "completed"]
risk_level: Literal["normal", "attention", "critical"]
updated_at: WorkbenchDateTime
detail: NonEmptyStr
recommendation: NonEmptyStr | None
duration_minutes: PositiveInt
source: NonEmptyStr
dispatcher: NonEmptyStr
location: NonEmptyStr | None = None
assignee: NonEmptyStr | None = None
priority: Literal["normal", "urgent"]
feedback_requirements: list[NonEmptyStr] = Field(default_factory=list)
activity: list[WorkOrderActivity] = Field(default_factory=list)
@model_validator(mode="before")
@classmethod
def accept_legacy_document(cls, value):
if not isinstance(value, dict) or "subtasks" not in value:
return value
document = {key: item for key, item in value.items() if key != "subtasks"}
subtasks = value.get("subtasks") or []
first = subtasks[0] if subtasks else {}
document.setdefault("location", first.get("location"))
document.setdefault("assignee", first.get("assignee"))
document.setdefault("feedback_requirements", list(dict.fromkeys(
requirement for subtask in subtasks
for requirement in subtask.get("reply_requirements", [])
)))
document.setdefault("activity", [])
return document
class WorkbenchRuntimeSnapshot(StrictModel):
availability: ProjectionAvailability
source: Literal["none", "scheduled_tasks"]
scenario_availability: ProjectionAvailability
artifact_availability: ProjectionAvailability
schedule_availability: ProjectionAvailability
work_order_availability: ProjectionAvailability
alert_availability: ProjectionAvailability
generated_at: WorkbenchDateTime
conditions: list[WorkbenchRuntimeCondition]
alerts: list[WorkbenchRuntimeAlert]
scenarios: list[WorkbenchRuntimeScenario]
latest_analysis: WorkbenchRuntimeLatestAnalysis | None
work_orders: list[WorkbenchRuntimeWorkOrder]
+3
View File
@@ -61,7 +61,10 @@ class AuditMiddleware(BaseHTTPMiddleware):
"/api/v1/audit/session-events",
"/audit/session-events",
}
# Query handles and source arguments are not legacy REST audit payloads.
# Task queries persist scoped evidence/receipts in their own transaction.
EXCLUDED_PATH_PREFIXES = (
"/api/v1/mcp-query-contexts",
)
async def dispatch(self, request: Request, call_next: Callable) -> Response:
@@ -525,6 +525,39 @@ class MetadataRepository:
pool_max_size=record.pool_max_size,
)
async def list_active_project_db_routings(
self, db_role: str
) -> List[ProjectDbRouting]:
result = await self.session.execute(
select(models.ProjectDatabase)
.join(models.Project, models.Project.id == models.ProjectDatabase.project_id)
.where(
models.ProjectDatabase.db_role == db_role,
models.Project.status == "active",
)
.order_by(models.ProjectDatabase.project_id)
)
if not is_database_encryption_configured():
raise ValueError("DATABASE_ENCRYPTION_KEY is not configured")
encryptor = get_database_encryptor()
try:
return [
ProjectDbRouting(
project_id=record.project_id,
db_role=record.db_role,
db_type=record.db_type,
dsn=_normalize_postgres_dsn(encryptor.decrypt(record.dsn_encrypted)),
pool_min_size=record.pool_min_size,
pool_max_size=record.pool_max_size,
)
for record in result.scalars().all()
]
except InvalidToken as exc:
raise ValueError(
"Failed to decrypt project DB DSN: DATABASE_ENCRYPTION_KEY mismatch "
"or invalid dsn_encrypted value"
) from exc
async def list_projects_for_user(self, user_id: UUID) -> List[ProjectSummary]:
stmt = (
select(models.Project, models.UserProjectMembership.project_role)
+88
View File
@@ -0,0 +1,88 @@
"""Project scoped private message storage, sharing task transaction locks."""
from psycopg import AsyncConnection
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
class AgentMessageRepository(OperationalTaskRepository):
def __init__(self, connection: AsyncConnection, project_id):
super().__init__(connection, project_id)
async def get_message(self, message_id: str) -> dict | None:
cursor = await self.connection.execute(
"SELECT * FROM workbench.agent_messages WHERE project_id=%s AND message_id=%s",
(self.project_id, message_id),
)
row = await cursor.fetchone()
return self._message(row) if row else None
async def get_message_request(self, owner_id: str, session_id: str, request_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT * FROM workbench.agent_messages WHERE project_id=%s
AND source_owner_id=%s AND source_session_id=%s AND request_id=%s""",
(self.project_id, owner_id, session_id, request_id),
)
row = await cursor.fetchone()
return self._message(row) if row else None
@staticmethod
def _message(row) -> dict:
return dict(row)
async def insert_message(self, message: dict) -> None:
keys = [key for key in message if key != "project_id"]
await self.connection.execute(
f"INSERT INTO workbench.agent_messages (project_id,{','.join(keys)}) "
f"VALUES ({','.join(['%s'] * (len(keys) + 1))})",
(self.project_id, *(message[key] for key in keys)),
)
async def list_session_messages(
self, owner_id: str, session_id: str, limit: int, *, cursor_at=None, cursor_id=None, pending_only=False,
) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT * FROM workbench.agent_messages WHERE project_id=%s
AND conversation_owner_id=%s AND (source_session_id=%s OR
(direction='to_chat' AND target_session_id=%s))
AND (%s::timestamptz IS NULL OR (created_at,message_id)<(%s::timestamptz,%s::uuid))
AND (NOT %s OR status IN ('pending','enqueued'))
ORDER BY created_at DESC,message_id DESC LIMIT %s""",
(self.project_id, owner_id, session_id, session_id, cursor_at, cursor_at, cursor_id, pending_only, limit),
)
return [self._message(row) for row in await cursor.fetchall()]
async def list_run_session_messages(
self, session_id: str, limit: int, *, cursor_at=None, cursor_id=None, pending_only=False,
) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT m.* FROM workbench.agent_messages m
JOIN workbench.operational_task_runs r ON r.project_id=m.project_id AND r.run_id=m.run_id
WHERE m.project_id=%s AND r.agent_session_id=%s
AND r.status IN ('claimed','running')
AND (%s::timestamptz IS NULL OR (m.created_at,m.message_id)<(%s::timestamptz,%s::uuid))
AND (NOT %s OR (m.direction='to_task' AND m.status IN ('pending','enqueued')))
ORDER BY m.created_at DESC,m.message_id DESC LIMIT %s""",
(self.project_id, session_id, cursor_at, cursor_at, cursor_id, pending_only, limit),
)
return [self._message(row) for row in await cursor.fetchall()]
async def list_pending_messages(self, limit: int, *, cursor_at=None, cursor_id=None) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT m.* FROM workbench.agent_messages m
JOIN workbench.operational_task_runs r
ON r.project_id=m.project_id AND r.run_id=m.run_id
WHERE m.project_id=%s AND m.status IN ('pending','enqueued')
AND (m.direction='to_chat' OR r.status IN ('claimed','running'))
AND (%s::timestamptz IS NULL OR (m.created_at,m.message_id)>(%s::timestamptz,%s::uuid))
ORDER BY m.created_at,m.message_id LIMIT %s""",
(self.project_id, cursor_at, cursor_at, cursor_id, limit),
)
return [self._message(row) for row in await cursor.fetchall()]
async def update_message_delivery(self, message_id: str, update: dict) -> None:
keys = list(update)
await self.connection.execute(
f"UPDATE workbench.agent_messages SET {','.join(key+'=%s' for key in keys)} "
"WHERE project_id=%s AND message_id=%s",
(*(update[key] for key in keys), self.project_id, message_id),
)
+149 -7
View File
@@ -4,6 +4,7 @@ from uuid import UUID, uuid4
from psycopg import AsyncConnection, Connection
from psycopg.types.json import Jsonb
from app.infra.db.postgresql.workbench_events import lock_event_transaction_sync
class AnalysisRepository:
@@ -19,6 +20,7 @@ class AnalysisRepository:
parameters: dict[str, Any],
run_id: UUID | None = None,
) -> dict[str, Any]:
lock_event_transaction_sync(conn)
execution_id = run_id or uuid4()
with conn.cursor() as cur:
cur.execute(
@@ -53,18 +55,53 @@ class AnalysisRepository:
status: str,
created_by: str,
parameters: dict[str, Any],
) -> None:
) -> int:
lock_event_transaction_sync(conn)
with conn.cursor() as cur:
cur.execute(
"""
UPDATE analysis.runs
SET created_by = %s, status = %s, parameters = %s
SET created_by = %s, status = %s, parameters = %s,
revision = revision + 1
WHERE run_id = %s
RETURNING revision
""",
(created_by, status, Jsonb(parameters), run_id),
)
if cur.rowcount != 1:
row = cur.fetchone()
if row is None:
raise LookupError(f"analysis run {run_id} does not exist")
return int(row["revision"])
@staticmethod
def append_run_changed_event_sync(
conn: Connection,
*,
project_id: UUID,
run_id: UUID,
revision: int,
status: str,
occurred_at: datetime,
results_committed: bool,
) -> None:
with conn.cursor() as cur:
cur.execute(
"""INSERT INTO workbench.domain_event_log
(project_id,event_id,topic,aggregate_type,aggregate_id,
aggregate_revision,occurred_at,metadata)
VALUES (%s,%s,'analysis.run.changed','analysis_run',%s,%s,%s,%s)
RETURNING event_seq""",
(
project_id, uuid4(), str(run_id), revision, occurred_at,
Jsonb({"run_id": str(run_id), "status": status,
"results_committed": results_committed}),
),
)
row = cur.fetchone()
cur.execute(
"SELECT pg_notify('workbench_events', %s)",
(f'{project_id}:{row["event_seq"]}',),
)
@staticmethod
def insert_result_sync(
@@ -101,7 +138,11 @@ class AnalysisRepository:
return cur.fetchone()
@staticmethod
async def list_runs(conn: AsyncConnection) -> list[dict]:
async def list_runs(
conn: AsyncConnection, *, limit: int = 100, offset: int = 0
) -> list[dict]:
if not 1 <= limit <= 100 or offset < 0:
raise ValueError("analysis run page is out of range")
async with conn.cursor() as cur:
await cur.execute(
"""
@@ -109,7 +150,52 @@ class AnalysisRepository:
started_at, status, parameters
FROM analysis.runs
ORDER BY created_at DESC, run_id
LIMIT %s OFFSET %s
""",
(limit, offset),
)
return await cur.fetchall()
@staticmethod
async def list_recent_runs(
conn: AsyncConnection, *, limit: int = 100
) -> list[dict]:
if not 1 <= limit <= 100:
raise ValueError("recent analysis run limit must be between 1 and 100")
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT run_id, name, run_type, created_by, created_at,
started_at, status, parameters
FROM analysis.runs
ORDER BY created_at DESC, run_id
LIMIT %s
""",
(limit,),
)
return await cur.fetchall()
@staticmethod
async def list_run_summaries(
conn: AsyncConnection, *, limit: int = 21, offset: int = 0
) -> list[dict]:
if not 1 <= limit <= 101 or offset < 0:
raise ValueError("analysis run summary page is out of range")
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT r.run_id, r.name, r.run_type, r.created_by,
r.created_at, r.started_at, r.status,
EXISTS (
SELECT 1
FROM analysis.results AS result
WHERE result.run_id = r.run_id
) AS has_results
FROM analysis.runs AS r
ORDER BY r.created_at DESC, r.run_id
LIMIT %s OFFSET %s
""",
(limit, offset),
)
return await cur.fetchall()
@@ -129,20 +215,76 @@ class AnalysisRepository:
@staticmethod
async def list_results(
conn: AsyncConnection, run_id: UUID, result_type: str | None = None
conn: AsyncConnection,
run_id: UUID,
result_type: str | None = None,
*,
limit: int = 100,
offset: int = 0,
lookahead: bool = False,
) -> list[dict]:
if not 1 <= limit <= 500 or offset < 0:
raise ValueError("analysis result page is out of range")
query = """
SELECT result_id, run_id, result_type, node_id, link_id,
payload, created_at
FROM analysis.results
WHERE run_id = %s
"""
params: tuple[UUID] | tuple[UUID, str] = (run_id,)
params: tuple = (run_id,)
if result_type is not None:
query += " AND result_type = %s"
params = (run_id, result_type)
query += " ORDER BY created_at, result_id"
query += " ORDER BY created_at, result_id LIMIT %s OFFSET %s"
params += (limit + int(lookahead), offset)
async with conn.cursor() as cur:
await cur.execute(query, params)
return await cur.fetchall()
@staticmethod
async def list_recent_results(
conn: AsyncConnection, run_id: UUID, *, limit: int = 100
) -> list[dict]:
if not 1 <= limit <= 100:
raise ValueError("recent analysis result limit must be between 1 and 100")
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT result_id, run_id, result_type, node_id, link_id,
payload, created_at
FROM (
SELECT result_id, run_id, result_type, node_id, link_id,
payload, created_at
FROM analysis.results
WHERE run_id = %s
ORDER BY created_at DESC, result_id DESC
LIMIT %s
) AS recent_results
ORDER BY created_at, result_id
""",
(run_id, limit),
)
return await cur.fetchall()
@staticmethod
async def list_junction_map_points(
conn: AsyncConnection, node_ids: list[str]
) -> list[dict]:
if not node_ids:
return []
if len(node_ids) > 500:
raise ValueError("analysis map projection supports at most 500 junctions")
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT g.id AS node_id,
ST_X(ST_Transform(g.geom, 4326)) AS longitude,
ST_Y(ST_Transform(g.geom, 4326)) AS latitude
FROM gis.junctions AS g
WHERE g.id = ANY(%s)
ORDER BY g.id
""",
(node_ids,),
)
return await cur.fetchall()
+80
View File
@@ -0,0 +1,80 @@
"""Read-only access to imported source GIS attributes."""
from typing import Any
from psycopg import AsyncConnection
class GisAttributeRepository:
@staticmethod
async def list_element_groups(
conn: AsyncConnection,
element_type: str,
element_id: str,
) -> list[dict[str, Any]]:
async with conn.cursor() as cur:
await cur.execute(
"SELECT to_regclass('source_gis.element_links') AS relation"
)
availability = await cur.fetchone()
if not availability or availability["relation"] is None:
return []
await cur.execute(
"""
SELECT
dataset.dataset_key,
dataset.title AS dataset_title,
dataset.source_date,
layer.layer_key,
layer.display_name AS layer_name,
feature.source_fid,
link.match_method,
link.distance_m AS match_distance_m,
feature.attributes
FROM source_gis.element_links AS link
JOIN source_gis.features AS feature
ON feature.layer_key = link.layer_key
AND feature.source_fid = link.source_fid
JOIN source_gis.layers AS layer
ON layer.layer_key = feature.layer_key
JOIN source_gis.datasets AS dataset
ON dataset.dataset_key = layer.dataset_key
WHERE link.element_type = %s
AND link.element_id = %s
ORDER BY dataset.source_date DESC, layer.display_name, feature.source_fid
""",
(element_type, element_id),
)
return [dict(row) for row in await cur.fetchall()]
@staticmethod
async def get_source_feature_group(
conn: AsyncConnection,
layer_key: str,
source_fid: int,
) -> dict[str, Any] | None:
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT
dataset.dataset_key,
dataset.title AS dataset_title,
dataset.source_date,
layer.layer_key,
layer.display_name AS layer_name,
feature.source_fid,
'source' AS match_method,
NULL::double precision AS match_distance_m,
feature.attributes
FROM source_gis.features AS feature
JOIN source_gis.layers AS layer
ON layer.layer_key = feature.layer_key
JOIN source_gis.datasets AS dataset
ON dataset.dataset_key = layer.dataset_key
WHERE feature.layer_key = %s
AND feature.source_fid = %s
""",
(layer_key, source_fid),
)
row = await cur.fetchone()
return dict(row) if row else None
+91
View File
@@ -0,0 +1,91 @@
"""Persistent MCP context and receipt storage in the selected project's BizDB."""
from datetime import datetime, timezone
from psycopg import AsyncConnection
from psycopg.types.json import Jsonb
from app.mcp.models import BoundContext
class McpQueryRepository:
def __init__(self, conn: AsyncConnection, project_id):
self.conn, self.project_id = conn, project_id
async def create(self, context: BoundContext):
async with self.conn.transaction(), self.conn.cursor() as cur:
# Bounded maintenance; expired/revoked handles remain invalid even if cleanup fails.
await cur.execute(
"""DELETE FROM workbench.mcp_query_contexts WHERE context_handle IN
(SELECT context_handle FROM workbench.mcp_query_contexts
WHERE project_id=%s AND (expires_at < now() OR revoked_at IS NOT NULL)
ORDER BY expires_at LIMIT 100 FOR UPDATE SKIP LOCKED)""",
(self.project_id,),
)
await cur.execute(
"""INSERT INTO workbench.mcp_query_contexts
(context_handle,project_id,subject,client_id,document,expires_at)
VALUES (%s,%s,%s,%s,%s,%s)""",
(
context.context_handle,
self.project_id,
context.subject,
context.client_id,
Jsonb(context.model_dump(mode="json")),
context.expires_at,
),
)
async def get(self, handle: str) -> BoundContext | None:
async with self.conn.transaction(), self.conn.cursor() as cur:
await cur.execute(
"""SELECT document,revoked_at FROM workbench.mcp_query_contexts
WHERE context_handle=%s AND project_id=%s""",
(handle, self.project_id),
)
row = await cur.fetchone()
if not row:
return None
document = dict(row["document"])
document["revoked_at"] = row["revoked_at"]
return BoundContext.model_validate(document)
async def revoke(self, handle: str, subject: str, client_id: str):
async with self.conn.transaction(), self.conn.cursor() as cur:
await cur.execute(
"""UPDATE workbench.mcp_query_contexts SET revoked_at=COALESCE(revoked_at,%s)
WHERE context_handle=%s AND project_id=%s AND subject=%s AND client_id=%s""",
(
datetime.now(timezone.utc),
handle,
self.project_id,
subject,
client_id,
),
)
async def receipt(self, scope_key: str, call_id: str) -> dict | None:
async with self.conn.transaction(), self.conn.cursor() as cur:
await cur.execute(
"""SELECT parameters,result FROM workbench.mcp_query_receipts
WHERE project_id=%s AND scope_key=%s AND call_id=%s""",
(self.project_id, scope_key, call_id),
)
return await cur.fetchone()
async def save_receipt(
self, scope_key: str, call_id: str, parameters: dict, result: dict
):
# Caller owns the short evidence + receipt transaction.
async with self.conn.cursor() as cur:
await cur.execute(
"""INSERT INTO workbench.mcp_query_receipts
(project_id,scope_key,call_id,parameters,result) VALUES (%s,%s,%s,%s,%s)
ON CONFLICT (project_id,scope_key,call_id) DO NOTHING""",
(self.project_id, scope_key, call_id, Jsonb(parameters), Jsonb(result)),
)
async def lock_receipt(self, scope_key: str, call_id: str):
async with self.conn.cursor() as cur:
await cur.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s,0))",
(f"mcp:{self.project_id}:{scope_key}:{call_id}",),
)
@@ -0,0 +1,823 @@
"""PostgreSQL persistence for durable operational tasks."""
from __future__ import annotations
from contextlib import asynccontextmanager
from datetime import datetime
from typing import Any
from psycopg import AsyncConnection
from psycopg.errors import UndefinedTable
from psycopg.types.json import Jsonb
from uuid import uuid4
from app.domain.domain_errors import DomainError
from app.infra.db.postgresql.workbench_events import lock_event_transaction
from app.infra.db.postgresql.workbench_events import WorkbenchEventRepository
class OperationalTaskRepository:
def __init__(self, connection: AsyncConnection, project_id):
self.connection = connection
self.project_id = str(project_id)
@asynccontextmanager
async def transaction(self):
try:
async with self.connection.transaction():
await lock_event_transaction(self.connection)
yield self
except UndefinedTable as exc:
raise DomainError(
"DEPENDENCY_UNAVAILABLE",
"Project operational-task storage is not provisioned",
status=503,
) from exc
async def lock(self, key: str) -> None:
await self.connection.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s, 0))",
(f"tjwater-operational:{self.project_id}:{key}",),
)
async def try_lock(self, key: str) -> bool:
cursor = await self.connection.execute(
"SELECT pg_try_advisory_xact_lock(hashtextextended(%s, 0)) AS acquired",
(f"tjwater-operational:{self.project_id}:{key}",),
)
return bool((await cursor.fetchone())["acquired"])
async def get_request(self, scope: str, request_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT request_digest,response
FROM workbench.operational_task_requests
WHERE project_id=%s AND scope=%s AND request_id=%s""",
(self.project_id, scope, request_id),
)
row = await cursor.fetchone()
return dict(row) if row else None
async def save_request(
self, scope: str, request_id: str, request_digest: str, response: dict
) -> None:
cursor = await self.connection.execute(
"""INSERT INTO workbench.operational_task_requests
(project_id,scope,request_id,request_digest,response)
VALUES (%s,%s,%s,%s,%s)
ON CONFLICT(project_id,scope,request_id) DO UPDATE SET
response=excluded.response
WHERE operational_task_requests.request_digest=excluded.request_digest""",
(self.project_id, scope, request_id, request_digest, Jsonb(response)),
)
if cursor.rowcount != 1:
raise DomainError(
"IDEMPOTENCY_CONFLICT",
"request_id was used with different content",
status=409,
)
async def get_task(self, task_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_tasks
WHERE project_id=%s AND task_id=%s""",
(self.project_id, task_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def get_active_run(self, task_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
agent_session_id,lease_expires_at,heartbeat_at,started_at,finished_at,
failure_code,failure_detail,dispatch_attempts,next_dispatch_at,
created_at,updated_at
FROM workbench.operational_task_runs
WHERE project_id=%s AND task_id=%s
AND status IN ('queued','claimed','running')
ORDER BY generation DESC LIMIT 1""",
(self.project_id, task_id),
)
row = await cursor.fetchone()
return dict(row) if row else None
async def get_run(self, run_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
agent_session_id,lease_expires_at,heartbeat_at,started_at,finished_at,
failure_code,failure_detail,dispatch_attempts,next_dispatch_at,
created_at,updated_at
FROM workbench.operational_task_runs
WHERE project_id=%s AND run_id=%s""",
(self.project_id, run_id),
)
row = await cursor.fetchone()
return dict(row) if row else None
async def list_task_runs(self, task_id: str, limit: int, offset: int) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT * FROM workbench.operational_task_runs
WHERE project_id=%s AND task_id=%s
ORDER BY generation DESC LIMIT %s OFFSET %s""",
(self.project_id, task_id, limit, offset),
)
return [{key: value for key, value in row.items() if key != "project_id"}
for row in await cursor.fetchall()]
async def get_latest_run(self, task_id: str) -> dict | None:
rows = await self.list_task_runs(task_id, 1, 0)
return rows[0] if rows else None
async def require_execution_storage(self) -> None:
"""Reject claims before changing state if the inbox cannot be used."""
cursor = await self.connection.execute(
"SELECT to_regclass('workbench.agent_messages') IS NOT NULL AS available"
)
if not (await cursor.fetchone())["available"]:
raise DomainError(
"DEPENDENCY_UNAVAILABLE",
"Agent message storage is not provisioned; apply "
"resources/sql/024_agent_messages.sql to this project database before starting runs",
status=503,
)
async def pending_run_messages(self, run_id: str) -> list[str]:
cursor = await self.connection.execute(
"""SELECT message_id FROM workbench.agent_messages
WHERE project_id=%s AND run_id=%s AND direction='to_task'
AND status IN ('pending','enqueued') ORDER BY created_at LIMIT 100""",
(self.project_id, run_id),
)
return [str(row["message_id"]) for row in await cursor.fetchall()]
async def fail_run_messages(self, run_id: str, failure_detail: str, now: datetime) -> None:
await self.connection.execute(
"""UPDATE workbench.agent_messages SET status='failed',failure_detail=%s,updated_at=%s
WHERE project_id=%s AND run_id=%s AND direction='to_task'
AND status IN ('pending','enqueued')""",
(failure_detail, now, self.project_id, run_id),
)
async def insert_run(self, run: dict) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_runs
(project_id,run_id,task_id,generation,execution_request_id,status,revision,
agent_session_id,lease_expires_at,heartbeat_at,started_at,finished_at,
failure_code,failure_detail,dispatch_attempts,next_dispatch_at,
created_at,updated_at)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
(
self.project_id, run["run_id"], run["task_id"], run["generation"],
run["execution_request_id"], run["status"], run["revision"],
run.get("agent_session_id"), run.get("lease_expires_at"),
run.get("heartbeat_at"), run.get("started_at"), run.get("finished_at"),
run.get("failure_code"), run.get("failure_detail"),
run.get("dispatch_attempts", 0), run["next_dispatch_at"],
run["created_at"], run["updated_at"],
),
)
async def update_run(self, run: dict, expected_revision: int) -> None:
cursor = await self.connection.execute(
"""UPDATE workbench.operational_task_runs SET
status=%s,revision=%s,agent_session_id=%s,lease_expires_at=%s,
heartbeat_at=%s,started_at=%s,finished_at=%s,failure_code=%s,
failure_detail=%s,dispatch_attempts=%s,next_dispatch_at=%s,updated_at=%s
WHERE project_id=%s AND run_id=%s AND revision=%s""",
(
run["status"], run["revision"], run.get("agent_session_id"),
run.get("lease_expires_at"), run.get("heartbeat_at"),
run.get("started_at"), run.get("finished_at"), run.get("failure_code"),
run.get("failure_detail"), run.get("dispatch_attempts", 0),
run["next_dispatch_at"], run["updated_at"], self.project_id,
run["run_id"], expected_revision,
),
)
if cursor.rowcount != 1:
raise DomainError("VERSION_CONFLICT", "Run revision changed", status=409)
async def list_runs(
self, *, status: str | None, limit: int, cursor_created_at=None, cursor_run_id=None,
task_type: str | None = None, task_created_from: datetime | None = None,
) -> list[dict]:
result = await self.connection.execute(
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
agent_session_id,lease_expires_at,heartbeat_at,started_at,finished_at,
failure_code,failure_detail,dispatch_attempts,next_dispatch_at,
created_at,updated_at
FROM workbench.operational_task_runs
WHERE project_id=%s
AND (%s::text IS NULL OR status=%s::text)
AND (%s::timestamptz IS NULL OR (created_at,run_id)>(%s::timestamptz,%s::uuid))
AND (status<>'queued' OR next_dispatch_at<=now())
AND EXISTS (
SELECT 1 FROM workbench.operational_tasks t
WHERE t.project_id=operational_task_runs.project_id
AND t.task_id=operational_task_runs.task_id
AND (%s::text IS NULL OR t.task_type=%s::text)
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
)
ORDER BY created_at,run_id LIMIT %s""",
(
self.project_id, status, status, cursor_created_at,
cursor_created_at, cursor_run_id,
task_type, task_type, task_created_from, task_created_from, limit,
),
)
return [dict(row) for row in await result.fetchall()]
async def claim_expired_runs(
self, limit: int, *, task_type: str | None = None, task_created_from: datetime | None = None,
) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
agent_session_id,lease_expires_at,heartbeat_at,started_at,finished_at,
failure_code,failure_detail,dispatch_attempts,next_dispatch_at,
created_at,updated_at
FROM workbench.operational_task_runs
WHERE project_id=%s AND status IN ('claimed','running')
AND lease_expires_at IS NOT NULL AND lease_expires_at<=now()
AND EXISTS (
SELECT 1 FROM workbench.operational_tasks t
WHERE t.project_id=operational_task_runs.project_id
AND t.task_id=operational_task_runs.task_id
AND (%s::text IS NULL OR t.task_type=%s::text)
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
)
ORDER BY lease_expires_at,run_id FOR UPDATE SKIP LOCKED LIMIT %s""",
(self.project_id, task_type, task_type, task_created_from, task_created_from, limit),
)
return [dict(row) for row in await cursor.fetchall()]
async def consecutive_timed_out_runs(self, task_id: str) -> int:
cursor = await self.connection.execute(
"""SELECT status FROM workbench.operational_task_runs
WHERE project_id=%s AND task_id=%s ORDER BY generation DESC""",
(self.project_id, task_id),
)
count = 0
for row in await cursor.fetchall():
if row["status"] != "timed_out":
break
count += 1
return count
async def append_event(
self, *, topic: str, aggregate_type: str, aggregate_id: str,
aggregate_revision: int, occurred_at, metadata: dict | None = None,
) -> int:
event_id = uuid4()
cursor = await self.connection.execute(
"""INSERT INTO workbench.domain_event_log
(project_id,event_id,topic,aggregate_type,aggregate_id,
aggregate_revision,occurred_at,metadata)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s) RETURNING event_seq""",
(
self.project_id, event_id, topic, aggregate_type, aggregate_id,
aggregate_revision, occurred_at, Jsonb(metadata or {}),
),
)
row = await cursor.fetchone()
event_seq = int(row["event_seq"])
await self.connection.execute(
"SELECT pg_notify('workbench_events', %s)",
(f'{self.project_id}:{event_seq}',),
)
return event_seq
async def current_event_cursor(self) -> int:
return await WorkbenchEventRepository(self.connection, self.project_id).current_cursor()
async def get_task_by_create_request(self, request_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_tasks
WHERE project_id=%s AND create_request_id=%s""",
(self.project_id, request_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def get_task_by_external_source(
self, source_kind: str, external_id: str
) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_tasks
WHERE project_id=%s
AND document#>>'{source,kind}'=%s
AND document#>>'{source,external_id}'=%s""",
(self.project_id, source_kind, external_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def list_tasks(
self,
*,
task_type: str | None,
status: str | None,
limit: int,
offset: int,
peripheral_work_order_id: str | None = None,
) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT task.document,
CASE WHEN active_run.run_id IS NULL THEN NULL
ELSE to_jsonb(active_run) - 'project_id' END AS active_run,
CASE WHEN latest_run.run_id IS NULL THEN NULL
ELSE to_jsonb(latest_run) - 'project_id' END AS latest_run
FROM workbench.operational_tasks AS task
LEFT JOIN LATERAL (
SELECT run.* FROM workbench.operational_task_runs AS run
WHERE run.project_id=task.project_id AND run.task_id=task.task_id
AND run.status IN ('queued','claimed','running')
ORDER BY run.generation DESC LIMIT 1
) AS active_run ON true
LEFT JOIN LATERAL (
SELECT run.* FROM workbench.operational_task_runs AS run
WHERE run.project_id=task.project_id AND run.task_id=task.task_id
ORDER BY run.generation DESC LIMIT 1
) AS latest_run ON true
WHERE task.project_id=%s
AND (task.document#>>'{source,deleted_at}') IS NULL
AND (%s::text IS NULL OR task.task_type=%s::text)
AND (%s::text IS NULL OR task.status=%s::text)
AND (%s::text IS NULL
OR (task.document#>>'{source,kind}'='external_system'
AND task.document#>>'{source,external_id}' IN (%s,%s))
OR EXISTS (
SELECT 1 FROM workbench.operational_task_actions AS action
WHERE action.project_id=task.project_id AND action.task_id=task.task_id
AND action.document->>'kind'='create_work_order'
AND action.document#>>'{receipt,adapter}'='peripheral_sample'
AND action.document#>>'{receipt,receipt_id}'=%s
))
ORDER BY task.updated_at DESC,task.task_id
LIMIT %s OFFSET %s""",
(
self.project_id,
task_type,
task_type,
status,
status,
peripheral_work_order_id,
peripheral_work_order_id,
f"peripheral_sample:{peripheral_work_order_id}" if peripheral_work_order_id else None,
peripheral_work_order_id,
limit,
offset,
),
)
return [
{**row["document"], "active_run": row["active_run"], "latest_run": row["latest_run"]}
for row in await cursor.fetchall()
]
async def insert_task(self, task: dict, create_request_id: str) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_tasks
(project_id,task_id,create_request_id,task_type,workflow,status,lifecycle_status,priority,
revision,execution_generation,execution_request_id,agent_session_id,
agent_lease_expires_at,current_plan_revision,current_report_revision,
scheduled_for,created_at,updated_at,document)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
self._task_params(task, create_request_id),
)
async def update_task(self, task: dict) -> None:
cursor = await self.connection.execute(
"""UPDATE workbench.operational_tasks SET
status=%s,lifecycle_status=%s,revision=%s,execution_generation=%s,execution_request_id=%s,
agent_session_id=%s,agent_lease_expires_at=%s,current_plan_revision=%s,
current_report_revision=%s,updated_at=%s,document=%s
WHERE project_id=%s AND task_id=%s AND revision=%s""",
(
task["status"], task["lifecycle_status"], task["revision"], task["execution_generation"],
task["execution_request_id"], task.get("agent_session_id"),
task.get("agent_lease_expires_at"), task.get("current_plan_revision"),
task.get("current_report_revision"), task["updated_at"], Jsonb(task),
self.project_id, task["task_id"], task["revision"] - 1,
),
)
if cursor.rowcount != 1:
raise DomainError("VERSION_CONFLICT", "Task revision changed", status=409)
def _task_params(self, task: dict, create_request_id: str) -> tuple:
return (
self.project_id, task["task_id"], create_request_id, task["task_type"],
task["workflow"], task["status"], task["lifecycle_status"], task["priority"], task["revision"],
task["execution_generation"], task["execution_request_id"],
task.get("agent_session_id"), task.get("agent_lease_expires_at"),
task.get("current_plan_revision"), task.get("current_report_revision"),
task.get("scheduled_for"), task["created_at"], task["updated_at"], Jsonb(task),
)
async def get_evidence(self, task_id: str, evidence_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_evidence
WHERE project_id=%s AND task_id=%s AND evidence_id=%s""",
(self.project_id, task_id, evidence_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def list_evidence(self, task_id: str) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_evidence
WHERE project_id=%s AND task_id=%s ORDER BY created_at,evidence_id""",
(self.project_id, task_id),
)
return [row["document"] for row in await cursor.fetchall()]
async def insert_evidence(self, task_id: str, evidence: dict) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_evidence
(project_id,task_id,evidence_id,document,created_at)
VALUES (%s,%s,%s,%s,%s)""",
(self.project_id, task_id, evidence["evidence_id"], Jsonb(evidence),
evidence["recorded_at"]),
)
async def access_requests_available(self) -> bool:
# Older project databases can serve existing tasks before migration 019.
# Keep approval writes fail-closed until the table is provisioned.
availability = await self.connection.execute(
"SELECT to_regclass('workbench.operational_task_access_requests') AS relation"
)
return (await availability.fetchone())["relation"] is not None
async def list_access_requests(self, task_id: str) -> list[dict]:
if not await self.access_requests_available():
return []
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_access_requests
WHERE project_id=%s AND task_id=%s
ORDER BY requested_at,access_request_id""",
(self.project_id, task_id),
)
return [row["document"] for row in await cursor.fetchall()]
async def get_access_request(self, task_id: str, access_request_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_access_requests
WHERE project_id=%s AND task_id=%s AND access_request_id=%s""",
(self.project_id, task_id, access_request_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def insert_access_request(self, task_id: str, document: dict) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_access_requests
(project_id,task_id,access_request_id,node_id,status,version,document,requested_at,updated_at)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
(self.project_id, task_id, document["access_request_id"], document["node_id"],
document["status"], document["version"], Jsonb(document),
document["requested_at"], document["requested_at"]),
)
async def update_access_request(self, task_id: str, document: dict, expected_version: int) -> None:
cursor = await self.connection.execute(
"""UPDATE workbench.operational_task_access_requests
SET status=%s,version=%s,document=%s,updated_at=%s
WHERE project_id=%s AND task_id=%s AND access_request_id=%s AND version=%s""",
(document["status"], document["version"], Jsonb(document), document["decided_at"],
self.project_id, task_id, document["access_request_id"], expected_version),
)
if cursor.rowcount != 1:
raise DomainError("VERSION_CONFLICT", "Access request version changed", status=409)
async def get_latest_plan(self, task_id: str) -> dict | None:
return await self._latest_artifact("operational_task_plans", task_id)
async def get_latest_report(self, task_id: str) -> dict | None:
return await self._latest_artifact("operational_task_reports", task_id)
async def get_report(self, task_id: str, revision: int) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_reports
WHERE project_id=%s AND task_id=%s AND revision=%s""",
(self.project_id, task_id, revision),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def list_reports(self, task_id: str, limit: int, offset: int) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_reports
WHERE project_id=%s AND task_id=%s ORDER BY revision DESC
LIMIT %s OFFSET %s""",
(self.project_id, task_id, limit, offset),
)
return [row["document"] for row in await cursor.fetchall()]
async def _latest_artifact(self, table: str, task_id: str) -> dict | None:
if table not in {"operational_task_plans", "operational_task_reports"}:
raise ValueError("unsupported artifact table")
cursor = await self.connection.execute(
f"""SELECT document FROM workbench.{table}
WHERE project_id=%s AND task_id=%s ORDER BY revision DESC LIMIT 1""",
(self.project_id, task_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def insert_plan(self, task_id: str, snapshot: dict) -> None:
await self._insert_artifact("operational_task_plans", task_id, snapshot)
async def insert_report(self, task_id: str, snapshot: dict) -> None:
await self._insert_artifact("operational_task_reports", task_id, snapshot)
async def get_latest_progress(self, task_id: str, generation: int) -> dict | None:
# Read compatibility before the coordinated progress migration is applied.
availability = await self.connection.execute(
"SELECT to_regclass('workbench.operational_task_progress') AS relation"
)
if not (await availability.fetchone())["relation"]:
return None
cursor = await self.connection.execute(
"""SELECT progress.document FROM workbench.operational_task_progress AS progress
JOIN workbench.operational_task_runs AS run
ON run.project_id=progress.project_id AND run.run_id=progress.run_id
AND run.task_id=progress.task_id
WHERE progress.project_id=%s AND progress.task_id=%s AND run.generation=%s""",
(self.project_id, task_id, generation),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def save_progress(self, task_id: str, snapshot: dict) -> None:
cursor = await self.connection.execute(
"""INSERT INTO workbench.operational_task_progress
(project_id,task_id,run_id,revision,document,published_at)
VALUES (%s,%s,%s,%s,%s,%s)
ON CONFLICT(project_id,task_id,run_id) DO UPDATE SET
revision=excluded.revision,document=excluded.document,published_at=excluded.published_at
WHERE workbench.operational_task_progress.revision=excluded.revision-1""",
(self.project_id, task_id, snapshot["run_id"], snapshot["revision"],
Jsonb(snapshot), snapshot["published_at"]),
)
if cursor.rowcount != 1:
raise DomainError("VERSION_CONFLICT", "Investigation progress revision changed", status=409)
async def _insert_artifact(self, table: str, task_id: str, snapshot: dict) -> None:
if table not in {"operational_task_plans", "operational_task_reports"}:
raise ValueError("unsupported artifact table")
await self.connection.execute(
f"""INSERT INTO workbench.{table}
(project_id,task_id,revision,document,created_at)
VALUES (%s,%s,%s,%s,%s)""",
(self.project_id, task_id, snapshot["revision"], Jsonb(snapshot),
snapshot["published_at"]),
)
async def get_action(self, task_id: str, action_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_actions
WHERE project_id=%s AND task_id=%s AND action_id=%s""",
(self.project_id, task_id, action_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def list_actions(self, task_id: str) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_actions
WHERE project_id=%s AND task_id=%s ORDER BY created_at,action_id""",
(self.project_id, task_id),
)
return [row["document"] for row in await cursor.fetchall()]
async def insert_action(self, task_id: str, action: dict) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_actions
(project_id,task_id,action_id,version,approval_status,execution_status,
document,created_at,updated_at)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
(self.project_id, task_id, action["action_id"], action["version"],
action["approval_status"], action["execution_status"], Jsonb(action),
action["proposed_at"], action["proposed_at"]),
)
await self._insert_action_revision(task_id, action)
async def update_action(self, task_id: str, action: dict) -> None:
await self._insert_action_revision(task_id, action)
await self.connection.execute(
"""UPDATE workbench.operational_task_actions SET
version=%s,approval_status=%s,execution_status=%s,document=%s,updated_at=%s
WHERE project_id=%s AND task_id=%s AND action_id=%s""",
(action["version"], action["approval_status"], action["execution_status"],
Jsonb(action), action.get("executed_at") or action.get("decided_at") or action.get("edited_at") or
action["proposed_at"], self.project_id, task_id, action["action_id"]),
)
async def _insert_action_revision(self, task_id: str, action: dict) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_action_revisions
(project_id,task_id,action_id,version,document,recorded_at)
VALUES (%s,%s,%s,%s,%s,%s)""",
(
self.project_id, task_id, action["action_id"], action["version"],
Jsonb(action), action.get("executed_at") or action.get("decided_at") or action.get("edited_at")
or action["proposed_at"],
),
)
async def get_feedback(self, task_id: str, source_event_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT source_revision,document,accepted_at
FROM workbench.operational_task_external_feedback
WHERE project_id=%s AND task_id=%s AND source_event_id=%s""",
(self.project_id, task_id, source_event_id),
)
row = await cursor.fetchone()
return dict(row) if row else None
async def upsert_feedback(self, task_id: str, feedback: dict, accepted_at) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_external_feedback
(project_id,task_id,source_event_id,source_revision,document,accepted_at)
VALUES (%s,%s,%s,%s,%s,%s)
ON CONFLICT(project_id,task_id,source_event_id) DO UPDATE SET
source_revision=excluded.source_revision,document=excluded.document,
accepted_at=excluded.accepted_at""",
(self.project_id, task_id, feedback["source_event_id"],
feedback["source_revision"], Jsonb(feedback), accepted_at),
)
async def insert_notification(self, notification: dict) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_notifications
(project_id,notification_id,task_id,created_at,document)
VALUES (%s,%s,%s,%s,%s)""",
(self.project_id, notification["notification_id"], notification["task_id"],
notification["created_at"], Jsonb(notification)),
)
async def resolve_notifications(
self,
task_id: str,
*,
kinds: list[str],
subject_type: str,
subject_id: str,
resolved_at,
resolution_reason: str,
) -> list[dict]:
cursor = await self.connection.execute(
"""UPDATE workbench.operational_task_notifications
SET document=document || jsonb_build_object(
'lifecycle_status','resolved',
'resolved_at',to_jsonb(%s::timestamptz),
'resolution_reason',%s::text
)
WHERE project_id=%s AND task_id=%s
AND COALESCE(document->>'lifecycle_status','active')='active'
AND document->>'kind'=ANY(%s)
AND document->>'subject_type'=%s
AND document->>'subject_id'=%s
RETURNING document""",
(
resolved_at,
resolution_reason,
self.project_id,
task_id,
kinds,
subject_type,
subject_id,
),
)
return [row["document"] for row in await cursor.fetchall()]
async def get_notification(self, notification_id: str, user_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT n.document || jsonb_build_object(
'status',CASE WHEN r.read_at IS NULL THEN 'unread' ELSE 'read' END,
'version',COALESCE(r.version,1),
'read_at',r.read_at,'read_by',
CASE WHEN r.user_id IS NULL THEN NULL ELSE 'principal:' || r.user_id::text END
) AS document
FROM workbench.operational_task_notifications n
LEFT JOIN workbench.operational_task_notification_reads r
ON r.project_id=n.project_id AND r.notification_id=n.notification_id
AND r.user_id=%s
WHERE n.project_id=%s AND n.notification_id=%s""",
(user_id, self.project_id, notification_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def mark_notification_read(
self, notification_id: str, user_id: str, version: int, read_at
) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_notification_reads
(project_id,notification_id,user_id,version,read_at)
VALUES (%s,%s,%s,%s,%s)
ON CONFLICT(project_id,notification_id,user_id) DO UPDATE SET
version=excluded.version,read_at=excluded.read_at""",
(self.project_id, notification_id, user_id, version, read_at),
)
async def list_notifications(
self, *, user_id: str, allowed_permissions: list[str],
unread_only: bool, limit: int, offset: int
) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT n.document || jsonb_build_object(
'status',CASE WHEN r.read_at IS NULL THEN 'unread' ELSE 'read' END,
'version',COALESCE(r.version,1),
'read_at',r.read_at,'read_by',
CASE WHEN r.user_id IS NULL THEN NULL ELSE 'principal:' || r.user_id::text END
) AS document
FROM workbench.operational_task_notifications n
LEFT JOIN workbench.operational_task_notification_reads r
ON r.project_id=n.project_id AND r.notification_id=n.notification_id
AND r.user_id=%s
WHERE n.project_id=%s
AND COALESCE(n.document->>'lifecycle_status','active')='active'
AND (%s=false OR r.read_at IS NULL)
AND (n.document->>'required_permission' IS NULL
OR n.document->>'required_permission'=ANY(%s))
ORDER BY n.created_at DESC,n.notification_id LIMIT %s OFFSET %s""",
(user_id, self.project_id, unread_only, allowed_permissions, limit, offset),
)
return [row["document"] for row in await cursor.fetchall()]
async def get_schedule_by_request(self, request_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_schedules
WHERE project_id=%s AND create_request_id=%s""",
(self.project_id, request_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def insert_schedule(self, schedule: dict, request_id: str) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_schedules
(project_id,schedule_id,create_request_id,schedule_key,next_run_at,
interval_seconds,enabled,version,document,created_at,updated_at)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
(self.project_id, schedule["schedule_id"], request_id,
schedule["schedule_key"], schedule["next_run_at"],
schedule["interval_seconds"], schedule["enabled"], schedule["version"],
Jsonb(schedule), schedule["created_at"], schedule["updated_at"]),
)
async def list_schedules(self) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_schedules
WHERE project_id=%s ORDER BY next_run_at,schedule_id""",
(self.project_id,),
)
return [row["document"] for row in await cursor.fetchall()]
async def get_schedule(self, schedule_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_schedules
WHERE project_id=%s AND schedule_id=%s""",
(self.project_id, schedule_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def update_schedule(self, schedule: dict) -> None:
cursor = await self.connection.execute(
"""UPDATE workbench.operational_task_schedules SET
enabled=%s,version=%s,updated_at=%s,document=%s
WHERE project_id=%s AND schedule_id=%s AND version=%s""",
(
schedule["enabled"], schedule["version"], schedule["updated_at"],
Jsonb(schedule), self.project_id, schedule["schedule_id"],
schedule["version"] - 1,
),
)
if cursor.rowcount != 1:
raise DomainError("VERSION_CONFLICT", "Schedule version changed", status=409)
async def claim_due_schedules(self, limit: int) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_task_schedules
WHERE project_id=%s AND enabled=true AND next_run_at <= now()
ORDER BY next_run_at,schedule_id FOR UPDATE SKIP LOCKED LIMIT %s""",
(self.project_id, limit),
)
return [row["document"] for row in await cursor.fetchall()]
async def advance_schedule(self, schedule: dict) -> None:
await self.connection.execute(
"""UPDATE workbench.operational_task_schedules SET
next_run_at=%s,version=%s,updated_at=%s,document=%s
WHERE project_id=%s AND schedule_id=%s""",
(schedule["next_run_at"], schedule["version"], schedule["updated_at"],
Jsonb(schedule), self.project_id, schedule["schedule_id"]),
)
async def insert_occurrence(self, schedule_id: str, due_at, task_id: str, created_at) -> bool:
cursor = await self.connection.execute(
"""INSERT INTO workbench.operational_schedule_occurrences
(project_id,schedule_id,due_at,task_id,created_at)
VALUES (%s,%s,%s,%s,%s)
ON CONFLICT(project_id,schedule_id,due_at) DO NOTHING
RETURNING task_id""",
(self.project_id, schedule_id, due_at, task_id, created_at),
)
return await cursor.fetchone() is not None
+406
View File
@@ -0,0 +1,406 @@
"""Bounded read primitives over existing project views and time-series tables."""
from datetime import datetime
from typing import Any
from uuid import UUID
from psycopg import AsyncConnection
from psycopg import sql
from psycopg.rows import dict_row
from fastapi import HTTPException
from app.infra.db.postgresql.scada import _SCADA_VIEW_SELECT, _device
from app.infra.db.timescaledb.repositories.realtime import RealtimeRepository
from app.infra.db.timescaledb.repositories.analysis import AnalysisResultsRepository
ELEMENT_VIEWS = {
v: v + "s" for v in ("junction", "pipe", "reservoir", "tank", "pump", "valve")
}
MAX_QUERY_BYTES = 8 * 1024 * 1024
async def read_rows(conn, statement, params=(), *, timeout_ms=30000):
async with conn.transaction(), conn.cursor(row_factory=dict_row) as cur:
await cur.execute(
"SELECT set_config('statement_timeout',%s,true)", (str(timeout_ms),)
)
await cur.execute(statement, params)
return await cur.fetchall()
async def asset_rows(conn, element_type, *, element_id=None, limit=100, offset=0):
view = ELEMENT_VIEWS[element_type]
where = sql.SQL("WHERE v.id=%s") if element_id else sql.SQL("")
params = (element_id,) if element_id else ()
statement = sql.SQL(
"SELECT to_jsonb(v)-'geom' AS record FROM gis.{} v {} ORDER BY v.id LIMIT %s OFFSET %s"
).format(sql.Identifier(view), where)
rows = await read_rows(conn, statement, params + (limit, offset))
return [row["record"] for row in rows]
async def scada_devices(conn, *, device_id=None, limit=100, offset=0):
where = " WHERE id=%s" if device_id else ""
params = (device_id,) if device_id else ()
rows = await read_rows(
conn,
_SCADA_VIEW_SELECT + where + " ORDER BY device_id LIMIT %s OFFSET %s",
params + (limit, offset),
)
return [_device(row) for row in rows]
async def series_rows(
conn,
*,
source,
element_type,
ids,
start=None,
end=None,
time=None,
field=None,
run_id=None,
limit=1000,
offset=0,
end_exclusive=False,
):
node = element_type == "junction"
if source == "scada":
table, id_column, fields = (
"measurements",
"device_id",
{"monitored_value", "cleaned_value"},
)
else:
table, id_column = (
("node_results", "node_id") if node else ("link_results", "link_id")
)
fields = (
(
AnalysisResultsRepository.NODE_FIELDS
if node
else AnalysisResultsRepository.LINK_FIELDS
)
if source == "analysis"
else (
RealtimeRepository.NODE_FIELDS
if node
else RealtimeRepository.LINK_FIELDS
)
)
if field is not None and field not in fields:
raise ValueError("unsupported field for this data source and element type")
columns = (
sql.SQL("{},time,{} AS value").format(
sql.Identifier(id_column), sql.Identifier(field)
)
if field
else sql.SQL("*")
)
filters = []
params = []
if ids:
filters.append(sql.SQL("{}=ANY(%s)").format(sql.Identifier(id_column)))
params.append(ids)
if run_id:
filters.append(sql.SQL("run_id=%s"))
params.append(run_id)
if start is not None:
filters.append(
sql.SQL(
"time >= %s AND time < %s"
if end_exclusive
else "time >= %s AND time <= %s"
)
)
params.extend((start, end))
else:
# Existing point-query semantics use +/- one second.
filters.append(
sql.SQL("time BETWEEN %s-interval '1 second' AND %s+interval '1 second'")
)
params.extend((time, time))
statement = sql.SQL(
"SELECT {} FROM {}.{} WHERE {} ORDER BY time,{} LIMIT %s OFFSET %s"
).format(
columns,
sql.Identifier(source),
sql.Identifier(table),
sql.SQL(" AND ").join(filters),
sql.Identifier(id_column),
)
return await read_rows(conn, statement, tuple(params) + (limit, offset))
async def sensor_runs(conn, *, run_id=None, limit=100, offset=0):
where = " AND r.run_id=%s" if run_id else ""
params = (run_id,) if run_id else ()
return await read_rows(
conn,
"""SELECT r.run_id,r.name,r.created_by,r.created_at,r.status,
result.payload FROM analysis.runs r JOIN LATERAL
(SELECT payload FROM analysis.results WHERE run_id=r.run_id
AND result_type='sensor_placement' ORDER BY created_at DESC,result_id DESC LIMIT 1) result ON true
WHERE r.run_type='sensor_placement'"""
+ where
+ " ORDER BY r.created_at DESC,r.run_id LIMIT %s OFFSET %s",
params + (limit, offset),
)
async def history_rows(
conn: AsyncConnection, descriptors: list[dict[str, Any]], *,
start: datetime, end: datetime, run_id: UUID | None, limit: int, offset: int,
) -> list[dict[str, Any]]:
"""Page points across raw, cleaned and model series in one SQL statement."""
parts, params = [], []
for index, item in enumerate(descriptors):
source, field = item["source"], item["field"]
if source == "scada":
table, id_column, fields = "measurements", "device_id", {"monitored_value", "cleaned_value"}
elif source in {"realtime", "analysis"}:
node = item["element_type"] == "junction"
table, id_column = ("node_results", "node_id") if node else ("link_results", "link_id")
fields = {"pressure"} if node else {"flow"}
else:
raise ValueError("unsupported history source")
if field not in fields or (source == "analysis" and run_id is None):
raise ValueError("unsupported history field or missing run")
parts.append(sql.SQL(
"SELECT %s::integer AS series_index,time,{} AS value FROM {}.{} "
"WHERE {}=%s AND time >= %s AND time <= %s{}"
).format(
sql.Identifier(field), sql.Identifier(source), sql.Identifier(table),
sql.Identifier(id_column), sql.SQL(" AND run_id=%s" if source == "analysis" else ""),
))
params.extend((index, item["id"], start, end))
if source == "analysis":
params.append(run_id)
if not parts:
return []
statement = sql.SQL(
"SELECT series_index,time,value FROM ({}) points "
"ORDER BY series_index, time LIMIT %s OFFSET %s"
).format(sql.SQL(" UNION ALL ").join(parts))
return await read_rows(conn, statement, tuple(params) + (limit, offset))
async def evidence_page(
conn: AsyncConnection, *, project_id: UUID, task_id: UUID, evidence_id: str, data_path: list[str],
limit: int, offset: int, mcp_sources: list[str], full_snapshot: bool = False,
) -> dict[str, Any]:
"""Project the immutable snapshot in PostgreSQL before transferring JSON."""
return await _snapshot_page(conn, stored_sql="""
SELECT document->'data_snapshot' AS snapshot
FROM workbench.operational_task_evidence
WHERE project_id=%s AND task_id=%s AND evidence_id=%s
AND jsonb_typeof(document->'data_snapshot')='object'
""", stored_params=(project_id, task_id, evidence_id), data_path=data_path,
limit=limit, offset=offset, mcp_sources=mcp_sources, full_snapshot=full_snapshot)
async def _snapshot_page(
conn: AsyncConnection, *, stored_sql: str, stored_params: tuple, data_path: list[str],
limit: int, offset: int, mcp_sources: list[str], full_snapshot: bool = False,
max_bytes: int = MAX_QUERY_BYTES,
) -> dict[str, Any]:
"""Shared JSONB projection; callers supply only fixed, scoped SQL sources."""
def descriptor(value, path):
return f"""jsonb_build_object(
'collection_type',jsonb_typeof({value}),
'count',CASE jsonb_typeof({value}) WHEN 'array' THEN jsonb_array_length({value})
WHEN 'string' THEN char_length({value} #>> '{{}}')
ELSE (SELECT count(*) FROM jsonb_object_keys({value})) END,
'data_path',to_jsonb({path}))"""
collection = descriptor("p.value", "p.path")
child = descriptor("child.item", "p.path || ARRAY[child.key]")
statement = """
WITH stored AS MATERIALIZED (
STORED_QUERY
), captured AS (
SELECT snapshot,
snapshot->>'source'=ANY(%s::text[])
AND jsonb_typeof(snapshot->'data')='object' AS is_mcp
FROM stored
), selected AS MATERIALIZED (
SELECT snapshot,
CASE WHEN is_mcp THEN (snapshot->'data')-'data' END AS query_metadata,
(CASE WHEN is_mcp THEN snapshot->'data'->'data'
ELSE snapshot->'data' END) #> %s::text[] AS value
FROM captured
), entries AS (
SELECT ordinality AS position, NULL::text AS key, item AS value
FROM selected, jsonb_array_elements(
CASE WHEN jsonb_typeof(value)='array' THEN value ELSE '[]'::jsonb END
) WITH ORDINALITY AS a(item, ordinality)
UNION ALL
SELECT row_number() OVER (ORDER BY e.key COLLATE "C"), e.key, e.item
FROM selected, jsonb_each(
CASE WHEN jsonb_typeof(value)='object' THEN value ELSE '{}'::jsonb END
) AS e(key,item)
), paged AS MATERIALIZED (
SELECT * FROM entries ORDER BY position LIMIT %s OFFSET %s
), paths AS (
SELECT paged.*, jsonb_typeof(selected.value) AS parent_type,
%s::text[] || ARRAY[CASE WHEN jsonb_typeof(selected.value)='array'
THEN (position-1)::text ELSE key END] AS path
FROM paged,selected
), summaries AS (
SELECT p.position,p.key,p.value AS original,
CASE WHEN jsonb_typeof(p.value) IN ('array','object') OR
(jsonb_typeof(p.value)='string' AND char_length(p.value #>> '{}')>256) THEN
CASE WHEN p.parent_type='object' OR jsonb_typeof(p.value) IN ('array','string')
THEN COLLECTION_PREVIEW
ELSE COALESCE((SELECT jsonb_object_agg(child.key,
CASE WHEN jsonb_typeof(child.item) IN ('array','object') OR
(jsonb_typeof(child.item)='string' AND char_length(child.item #>> '{}')>256)
THEN CHILD_PREVIEW ELSE child.item END)
FROM jsonb_each(p.value) AS child(key,item)), '{}'::jsonb)
END ELSE p.value END AS value
FROM paths p
), projected AS (
SELECT CASE WHEN %s THEN snapshot ELSE jsonb_build_object(
'snapshot_metadata',snapshot-'data', 'query_metadata',query_metadata,
'nested',EXISTS(SELECT 1 FROM summaries WHERE original<>value),
'value_type',jsonb_typeof(value),
'total',CASE jsonb_typeof(value)
WHEN 'array' THEN jsonb_array_length(value)
WHEN 'object' THEN (SELECT count(*) FROM jsonb_object_keys(value))
WHEN 'string' THEN char_length(value #>> '{}') END,
'value',CASE jsonb_typeof(value)
WHEN 'array' THEN COALESCE((SELECT jsonb_agg(value ORDER BY position) FROM summaries),'[]'::jsonb)
WHEN 'object' THEN COALESCE((SELECT jsonb_object_agg(key,value) FROM summaries),'{}'::jsonb)
WHEN 'string' THEN to_jsonb(substring(value #>> '{}' FROM %s FOR %s))
ELSE value END
) END AS payload, value IS NULL AS missing_path
FROM selected
), sized AS MATERIALIZED (
SELECT payload,missing_path,octet_length(payload::text)>%s AS too_large
FROM projected
)
SELECT CASE WHEN NOT too_large THEN payload END AS payload,too_large,missing_path
FROM sized
"""
statement = statement.replace("STORED_QUERY", stored_sql).replace("COLLECTION_PREVIEW", collection).replace("CHILD_PREVIEW", child)
rows = await read_rows(conn, statement, (
*stored_params, mcp_sources, data_path, limit, offset,
data_path, full_snapshot, offset + 1, limit, max_bytes,
))
if not rows:
raise HTTPException(404, "Stored query document was not found")
row = rows[0]
if row["too_large"]:
raise HTTPException(413, "Query exceeds the byte budget; select a deeper data_path or smaller page")
if row.get("missing_path") and not full_snapshot:
raise HTTPException(422, "Evidence data_path does not name a stored value")
return row["payload"]
async def task_context_summary(conn: AsyncConnection, *, project_id: UUID, task_id: UUID, report_revision: int | None) -> dict:
"""Read fixed-size metadata and explicit previews, never a complete report."""
report = """jsonb_build_object('revision',r.revision,
'execution_generation',r.document->'execution_generation',
'stage',r.document#>'{report,stage}','outcome',r.document#>'{report,outcome}',
'current_conclusion_preview',left(r.document#>>'{report,current_conclusion}',200),
'preview_only',true,'detail',jsonb_build_object('section','report','report_revision',r.revision))"""
rows = await read_rows(conn, """
SELECT jsonb_build_object('task',jsonb_build_object(
'task_id',t.task_id,'task_type',t.task_type,'workflow',t.document->>'workflow','status',t.status,
'lifecycle_status',t.lifecycle_status,'priority',t.priority,'revision',t.revision,
'execution_generation',t.execution_generation,
'current_report_revision',t.current_report_revision,
'title_preview',left(t.document->>'title',160),'preview_only',true,
'detail',jsonb_build_object('section','task')),
'latest_report',(SELECT REPORT_SUMMARY FROM workbench.operational_task_reports r
WHERE r.project_id=t.project_id AND r.task_id=t.task_id ORDER BY r.revision DESC LIMIT 1),
'selected_report',CASE WHEN %s::int IS NULL THEN NULL ELSE
(SELECT REPORT_SUMMARY FROM workbench.operational_task_reports r
WHERE r.project_id=t.project_id AND r.task_id=t.task_id AND r.revision=%s) END
) AS payload
FROM workbench.operational_tasks t WHERE t.project_id=%s AND t.task_id=%s
""".replace("REPORT_SUMMARY", report), (report_revision, report_revision, project_id, task_id))
if not rows:
raise HTTPException(404, "Task was not found in this project")
result = rows[0]["payload"]
if report_revision is not None and result["selected_report"] is None:
raise HTTPException(404, "Selected report revision was not found")
return result
async def task_context_page(
conn: AsyncConnection, *, project_id: UUID, task_id: UUID, section: str,
report_revision: int | None, data_path: list[str], limit: int, offset: int,
) -> dict:
"""Page only the requested task section inside PostgreSQL, before transfer."""
collections = {
"evidence": ("operational_task_evidence", "evidence_id"),
"actions": ("operational_task_actions", "action_id"),
"access_requests": ("operational_task_access_requests", "access_request_id"),
}
documents = {"task": "operational_tasks", "report": "operational_task_reports", "plan": "operational_task_plans"}
if section in {"access_requests", "progress"}:
table = "operational_task_access_requests" if section == "access_requests" else "operational_task_progress"
available = await read_rows(conn, "SELECT to_regclass(%s) AS relation", ("workbench." + table,))
if not available[0]["relation"]:
return {"snapshot_metadata": {"section": section, "available": False}, "value": None, "value_type": "null", "total": None, "nested": False}
projection_offset = offset
if section in collections:
table, key = collections[section]
# These identifiers come exclusively from the fixed table above.
relation = sql.Identifier("workbench", table).as_string()
identifier = sql.Identifier(key).as_string()
if data_path:
stored = f"""SELECT jsonb_build_object('section',%s::text,'data',jsonb_build_object({identifier}::text,document)) AS snapshot
FROM {relation} WHERE project_id=%s AND task_id=%s AND {identifier}::text=%s"""
params = (section, project_id, task_id, data_path[0])
else:
# Fetch just this page of rows. Collection keys remain stable across
# paging, so every nested data_path uses the real evidence/action ID.
stored = f"""WITH items AS MATERIALIZED (
SELECT {identifier}::text AS key,document FROM {relation}
WHERE project_id=%s AND task_id=%s
ORDER BY {identifier}::text COLLATE "C" LIMIT %s OFFSET %s)
SELECT jsonb_build_object('section',%s::text,
'total',(SELECT count(*) FROM {relation} WHERE project_id=%s AND task_id=%s),
'data',COALESCE(jsonb_object_agg(key,document),'{{}}'::jsonb)) AS snapshot FROM items"""
params = (project_id, task_id, limit + 1, offset, section, project_id, task_id)
projection_offset = 0
elif section in documents:
relation = sql.Identifier("workbench", documents[section]).as_string()
where = ""
params = (section, project_id, task_id)
if section == "report":
where = " AND (%s::int IS NULL OR revision=%s) ORDER BY revision DESC LIMIT 1"
params += (report_revision, report_revision)
elif section == "plan":
where = " ORDER BY revision DESC LIMIT 1"
stored = f"SELECT jsonb_build_object('section',%s::text,'data',document) AS snapshot FROM {relation} WHERE project_id=%s AND task_id=%s{where}"
elif section == "run":
stored = """SELECT jsonb_build_object('section','run','data',to_jsonb(r)-'project_id') AS snapshot
FROM workbench.operational_task_runs r WHERE project_id=%s AND task_id=%s ORDER BY generation DESC LIMIT 1"""
params = (project_id, task_id)
elif section == "progress":
stored = """SELECT jsonb_build_object('section','progress','data',p.document) AS snapshot
FROM workbench.operational_task_progress p
JOIN workbench.operational_task_runs r ON r.project_id=p.project_id AND r.task_id=p.task_id AND r.run_id=p.run_id
JOIN workbench.operational_tasks t ON t.project_id=p.project_id AND t.task_id=p.task_id AND t.execution_generation=r.generation
WHERE p.project_id=%s AND p.task_id=%s"""
params = (project_id, task_id)
else:
raise HTTPException(422, "Unknown task context section")
return await _snapshot_page(conn, stored_sql=stored, stored_params=params,
data_path=data_path, limit=limit, offset=projection_offset, mcp_sources=[], max_bytes=8000)
async def task_data_scopes(conn: AsyncConnection, *, project_id: UUID, task_id: UUID, permissions: set[str]) -> list[str]:
"""Bound scope discovery to seven known sources, without loading task detail."""
known = ["workbench_runtime", "peripheral_points", "peripheral_work_orders", "peripheral_observations", "peripheral_test_data", "scada_devices", "scada_readings"]
scopes = {name for name in known if ("scada.view" if name.startswith("scada_") else "simulation.view") in permissions}
available = await read_rows(conn, "SELECT to_regclass('workbench.operational_task_access_requests') AS relation")
if available[0]["relation"]:
rows = await read_rows(conn, """SELECT DISTINCT scope FROM workbench.operational_task_access_requests,
LATERAL jsonb_array_elements_text(COALESCE(document->'approved_scopes','[]'::jsonb)) AS granted(scope)
WHERE project_id=%s AND task_id=%s AND status='approved' AND scope=ANY(%s::text[])""", (project_id, task_id, known))
scopes.update(row["scope"] for row in rows)
return [name for name in known if name in scopes]
+4 -2
View File
@@ -111,14 +111,16 @@ class ScadaInfoRepository:
conn: AsyncConnection,
node_ids: list[str],
link_ids: list[str],
*, row_limit: int | None = None,
) -> list[dict[str, Any]]:
if not node_ids and not link_ids:
return []
async with conn.cursor() as cur:
await cur.execute(
_SCADA_VIEW_SELECT
+ " WHERE node_id = ANY(%s) OR link_id = ANY(%s) ORDER BY device_id",
(node_ids, link_ids),
+ " WHERE node_id = ANY(%s) OR link_id = ANY(%s) ORDER BY device_id"
+ (" LIMIT %s" if row_limit is not None else ""),
(node_ids, link_ids) + ((row_limit,) if row_limit is not None else ()),
)
return [_device(record) for record in await cur.fetchall()]
@@ -24,6 +24,8 @@ def _placement_row(row: dict[str, Any]) -> dict[str, Any]:
"created_by": row["created_by"],
"created_at": row["created_at"],
"status": row["status"],
"sensor_type": payload.get("sensor_type", "pressure"),
"method": payload.get("method"),
"sensor_locations": locations,
}
@@ -57,11 +59,15 @@ def create_sensor_placement(
min_diameter: int,
created_by: str,
sensor_locations: list[str],
sensor_type: str = "pressure",
method: str | None = None,
) -> dict[str, Any]:
run_id = uuid4()
payload = {
"sensor_number": len(sensor_locations),
"minimum_diameter": min_diameter,
"sensor_type": sensor_type,
"method": method,
"sensor_locations": sensor_locations,
}
with project_connection(name) as conn, conn.transaction():
@@ -0,0 +1,71 @@
"""Bounded nearest-model queries on the already authorized project connection."""
from math import asin, cos, degrees, radians, sin
from typing import Any
from psycopg import AsyncConnection, sql
from app.infra.db.postgresql.read_queries import ELEMENT_VIEWS, read_rows
_MODEL_RELATIONS = {
kind: sql.Identifier("gis", view) for kind, view in ELEMENT_VIEWS.items()
}
def _search_boxes(
longitude: float, latitude: float, radius_m: float,
) -> list[tuple[float, float, float, float]]:
# A deliberately smaller-than-WGS84 curvature radius gives a conservative
# geographic envelope. It is only a GiST candidate filter; geography below
# determines the actual radius and distances. Split at the antimeridian.
angle = (radius_m + 10) / 6_300_000
low, high = latitude - degrees(angle), latitude + degrees(angle)
if low < -85 or high > 85:
# Avoid Mercator's polar singularity; retain exact geography filtering.
return []
width = degrees(asin(min(1.0, sin(angle) / cos(radians(latitude)))))
left, right = longitude - width, longitude + width
if left < -180:
return [(-180, low, right, high), (left + 360, low, 180, high)]
if right > 180:
return [(left, low, 180, high), (-180, low, right - 360, high)]
return [(left, low, right, high)]
async def nearby_elements(
conn: AsyncConnection,
element_type: str,
*,
center: tuple[float, float],
radius_m: float,
limit: int,
offset: int,
) -> list[dict[str, Any]]:
"""Rank complete model geometries by spheroidal distance, never endpoints."""
relation = _MODEL_RELATIONS[element_type]
filters = []
parameters: list[Any] = [*center]
for box in _search_boxes(*center, radius_m):
filters.append(sql.SQL(
"v.geom && ST_Transform(ST_MakeEnvelope(%s,%s,%s,%s,4326),3857)"
))
parameters.extend(box)
candidate_filter = sql.SQL(" OR ").join(filters) if filters else sql.SQL("TRUE")
statement = sql.SQL("""
WITH target AS (
SELECT ST_SetSRID(ST_MakePoint(%s,%s),4326)::geography AS point
), candidates AS MATERIALIZED (
SELECT v.id, to_jsonb(v)-'geom' AS properties,
ST_Transform(v.geom,4326)::geography AS shape
FROM {} v
WHERE v.geom IS NOT NULL AND NOT ST_IsEmpty(v.geom) AND ({})
)
SELECT c.id AS element_id, c.properties,
ST_Distance(c.shape,t.point,true) AS distance_m
FROM candidates c CROSS JOIN target t
WHERE ST_DWithin(c.shape,t.point,%s,true)
ORDER BY distance_m,c.id
LIMIT %s OFFSET %s
""").format(relation, candidate_filter)
parameters.extend((radius_m, limit, offset))
return await read_rows(conn, statement, tuple(parameters))
+109
View File
@@ -0,0 +1,109 @@
"""PostgreSQL persistence for task-produced alert events and current state."""
from __future__ import annotations
from contextlib import asynccontextmanager
from psycopg import AsyncConnection
from psycopg.errors import UndefinedTable
from psycopg.types.json import Jsonb
from uuid import uuid4
from app.domain.domain_errors import DomainError
from app.infra.db.postgresql.workbench_events import lock_event_transaction
class TaskAlertRepository:
def __init__(self, connection: AsyncConnection, project_id):
self.connection = connection
self.project_id = str(project_id)
@asynccontextmanager
async def transaction(self):
try:
async with self.connection.transaction():
await lock_event_transaction(self.connection)
yield self
except UndefinedTable as exc:
raise DomainError(
"DEPENDENCY_UNAVAILABLE",
"Project alert storage is not provisioned",
status=503,
) from exc
async def lock_alert(self, alert_id: str) -> None:
await self.connection.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s, 0))",
(f"tjwater-alert:{self.project_id}:{alert_id}",),
)
async def get_event(self, producer_id: str, event_id: str) -> dict | None:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.alert_events
WHERE project_id=%s AND producer_id=%s AND event_id=%s""",
(self.project_id, producer_id, event_id),
)
row = await cursor.fetchone()
return row["document"] if row else None
async def get_alert(self, alert_id: str) -> tuple[int, dict] | None:
cursor = await self.connection.execute(
"""SELECT revision,document FROM workbench.alerts
WHERE project_id=%s AND alert_id=%s""",
(self.project_id, alert_id),
)
row = await cursor.fetchone()
return (row["revision"], row["document"]) if row else None
async def save_event(self, event: dict, resulting_state: dict) -> None:
document = event | {"resulting_state": resulting_state}
await self.connection.execute(
"""INSERT INTO workbench.alert_events
(project_id,producer_id,event_id,alert_id,event_type,occurred_at,document)
VALUES (%s,%s,%s,%s,%s,%s,%s)""",
(
self.project_id,
event["producer_id"],
event["event_id"],
event["alert_id"],
event["event_type"],
event["occurred_at"],
Jsonb(document),
),
)
async def save_alert(self, state: dict) -> None:
await self.connection.execute(
"""INSERT INTO workbench.alerts
(project_id,alert_id,revision,status,occurred_at,updated_at,document)
VALUES (%s,%s,%s,%s,%s,%s,%s)
ON CONFLICT(project_id,alert_id) DO UPDATE SET
revision=excluded.revision,status=excluded.status,
updated_at=excluded.updated_at,document=excluded.document""",
(
self.project_id,
state["alert_id"],
state["revision"],
state["status"],
state["occurred_at"],
state["updated_at"],
Jsonb({key: value for key, value in state.items() if key != "revision"}),
),
)
async def append_changed_event(self, state: dict) -> None:
cursor = await self.connection.execute(
"""INSERT INTO workbench.domain_event_log
(project_id,event_id,topic,aggregate_type,aggregate_id,
aggregate_revision,occurred_at,metadata)
VALUES (%s,%s,'workbench.alert.changed','workbench_alert',%s,%s,%s,%s)
RETURNING event_seq""",
(
self.project_id, uuid4(), state["alert_id"], state["revision"],
state["updated_at"], Jsonb({"alert_id": state["alert_id"]}),
),
)
row = await cursor.fetchone()
await self.connection.execute(
"SELECT pg_notify('workbench_events', %s)",
(f'{self.project_id}:{row["event_seq"]}',),
)
@@ -0,0 +1,221 @@
"""Per-process, project-scoped PostgreSQL LISTEN hubs for workbench SSE."""
from __future__ import annotations
import asyncio
import contextlib
import logging
from collections import OrderedDict
from contextlib import asynccontextmanager
from uuid import UUID
from psycopg import AsyncConnection
from psycopg_pool import AsyncConnectionPool
from psycopg.rows import dict_row
from app.infra.db.postgresql.workbench_events import WorkbenchEventRepository
logger = logging.getLogger(__name__)
class WorkbenchEventHub:
"""Own one LISTEN connection and fan wake-ups out to SSE subscribers."""
def __init__(self, project_id: UUID, dsn: str) -> None:
self.project_id = project_id
self.dsn = dsn
self.subscribers = 0
self._connection: AsyncConnection | None = None
self._pool: AsyncConnectionPool | None = None
self._connection_lock = asyncio.Lock()
self._read_lock = asyncio.Lock()
self._reads: OrderedDict[tuple, tuple[int, object]] = OrderedDict()
self._condition = asyncio.Condition()
self._generation = 0
self._ready = asyncio.Event()
self._closed = False
self._listener_task: asyncio.Task | None = None
self._idle_task: asyncio.Task | None = None
async def start(self) -> None:
if self._listener_task is None or self._listener_task.done():
self._closed = False
self._ready.clear()
self._listener_task = asyncio.create_task(
self._listen_loop(), name=f"workbench-events:{self.project_id}"
)
await asyncio.wait_for(self._ready.wait(), timeout=10)
async def _connect(self) -> AsyncConnection:
if self._pool is None:
self._pool = AsyncConnectionPool(
conninfo=self.dsn,
min_size=0,
max_size=1,
open=False,
kwargs={"autocommit": True, "row_factory": dict_row},
)
await self._pool.open()
connection = await self._pool.getconn()
await connection.execute("LISTEN workbench_events")
return connection
async def _listen_loop(self) -> None:
retry_seconds = 1
next_reconcile = asyncio.get_running_loop().time() + 30
while not self._closed:
try:
if self._connection is None or self._connection.closed:
async with self._connection_lock:
if self._connection is None or self._connection.closed:
self._connection = await self._connect()
self._ready.set()
retry_seconds = 1
notified = False
async with self._connection_lock:
connection = self._connection
if connection is None:
continue
async for notification in connection.notifies(timeout=1, stop_after=1):
if notification.payload.startswith(f"{self.project_id}:"):
notified = True
break
now = asyncio.get_running_loop().time()
if notified or now >= next_reconcile:
await self._wake()
if now >= next_reconcile:
next_reconcile = now + 30
await asyncio.sleep(0)
except asyncio.CancelledError:
raise
except Exception:
logger.exception("Workbench event listener failed for project %s", self.project_id)
await self._drop_connection()
await self._wake()
await asyncio.sleep(retry_seconds)
retry_seconds = min(retry_seconds * 2, 30)
async def _wake(self) -> None:
async with self._condition:
self._generation += 1
self._condition.notify_all()
async def wait_for_wake(self, generation: int, timeout: float) -> int:
async with self._condition:
if self._generation != generation:
return self._generation
with contextlib.suppress(asyncio.TimeoutError):
await asyncio.wait_for(
self._condition.wait_for(lambda: self._generation != generation),
timeout=timeout,
)
return self._generation
@property
def generation(self) -> int:
return self._generation
async def oldest_cursor(self) -> int | None:
return await self._read(("oldest",), lambda repository: repository.oldest_cursor())
async def current_cursor(self) -> int:
return await self._read(("current",), lambda repository: repository.current_cursor())
async def read_after(self, after: int) -> list[dict]:
return await self._read(("after", after), lambda repository: repository.read_after(after))
async def _read(self, key: tuple, operation):
await self.start()
async with self._read_lock:
cached = self._reads.get(key)
if cached is not None and cached[0] == self.generation:
self._reads.move_to_end(key)
return cached[1]
# Record the generation before the query: a concurrent NOTIFY must
# invalidate even an empty result from an older database snapshot.
generation = self.generation
async with self._connection_lock:
if self._connection is None or self._connection.closed:
self._connection = await self._connect()
self._ready.set()
repository = WorkbenchEventRepository(self._connection, self.project_id)
result = await operation(repository)
self._reads[key] = (generation, result)
self._reads.move_to_end(key)
while len(self._reads) > 32:
self._reads.popitem(last=False)
return result
async def _drop_connection(self) -> None:
async with self._connection_lock:
connection, self._connection = self._connection, None
if connection is not None and self._pool is not None:
with contextlib.suppress(Exception):
await self._pool.putconn(connection)
async def close(self) -> None:
self._closed = True
self._reads.clear()
if self._idle_task is not None:
self._idle_task.cancel()
self._idle_task = None
if self._listener_task is not None:
self._listener_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._listener_task
self._listener_task = None
await self._drop_connection()
if self._pool is not None:
await self._pool.close()
self._pool = None
class WorkbenchEventHubManager:
def __init__(self, idle_seconds: float = 60) -> None:
self.idle_seconds = idle_seconds
self._hubs: dict[tuple[UUID, str], WorkbenchEventHub] = {}
self._lock = asyncio.Lock()
@asynccontextmanager
async def subscribe(self, project_id: UUID, dsn: str):
key = (project_id, dsn)
async with self._lock:
hub = self._hubs.get(key)
if hub is None:
hub = WorkbenchEventHub(project_id, dsn)
self._hubs[key] = hub
if hub._idle_task is not None:
hub._idle_task.cancel()
hub._idle_task = None
hub.subscribers += 1
try:
await hub.start()
yield hub
finally:
async with self._lock:
hub.subscribers = max(0, hub.subscribers - 1)
if hub.subscribers == 0 and hub._idle_task is None:
hub._idle_task = asyncio.create_task(self._close_when_idle(key, hub))
async def _close_when_idle(
self, key: tuple[UUID, str], hub: WorkbenchEventHub
) -> None:
try:
await asyncio.sleep(self.idle_seconds)
async with self._lock:
if hub.subscribers or self._hubs.get(key) is not hub:
return
self._hubs.pop(key, None)
hub._idle_task = None
await hub.close()
except asyncio.CancelledError:
return
async def close_all(self) -> None:
async with self._lock:
hubs = list(self._hubs.values())
self._hubs.clear()
await asyncio.gather(*(hub.close() for hub in hubs), return_exceptions=True)
workbench_event_hubs = WorkbenchEventHubManager()
@@ -0,0 +1,61 @@
"""Cursor reads for the durable project-scoped workbench event log."""
from __future__ import annotations
from psycopg import AsyncConnection, Connection
EVENT_ORDER_LOCK = "tjwater-workbench:event-order"
async def lock_event_transaction(connection: AsyncConnection) -> None:
"""Serialize v2 event-producing transactions before business locks/writes.
PostgreSQL sequence allocation is not commit order. All producers must use
this same transaction lock so a committed higher cursor cannot hide an
earlier, still-uncommitted event. Other BizDBs remain rollout opt-out.
"""
if getattr(getattr(connection, "info", None), "dbname", None) == "tjwater_v2":
await connection.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s,0))", (EVENT_ORDER_LOCK,)
)
def lock_event_transaction_sync(connection: Connection) -> None:
"""Synchronous counterpart using the identical PostgreSQL lock key."""
if getattr(getattr(connection, "info", None), "dbname", None) == "tjwater_v2":
connection.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s,0))", (EVENT_ORDER_LOCK,)
)
class WorkbenchEventRepository:
def __init__(self, connection: AsyncConnection, project_id):
self.connection = connection
self.project_id = str(project_id)
async def current_cursor(self) -> int:
cursor = await self.connection.execute(
"SELECT COALESCE(max(event_seq),0) AS cursor FROM workbench.domain_event_log WHERE project_id=%s",
(self.project_id,),
)
row = await cursor.fetchone()
return int(row["cursor"])
async def oldest_cursor(self) -> int | None:
cursor = await self.connection.execute(
"SELECT min(event_seq) AS cursor FROM workbench.domain_event_log WHERE project_id=%s",
(self.project_id,),
)
row = await cursor.fetchone()
return int(row["cursor"]) if row and row["cursor"] is not None else None
async def read_after(self, after: int, limit: int = 200) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT event_seq,event_id,project_id,topic,aggregate_type,aggregate_id,
aggregate_revision,occurred_at,metadata
FROM workbench.domain_event_log
WHERE project_id=%s AND event_seq>%s
ORDER BY event_seq LIMIT %s""",
(self.project_id, after, limit),
)
return [dict(row) for row in await cursor.fetchall()]
@@ -0,0 +1,49 @@
"""Optional project-local alert projection."""
from dataclasses import dataclass
from typing import Any
from psycopg import AsyncConnection
@dataclass(frozen=True)
class WorkbenchOperationsAvailability:
alerts: bool
schedules: bool = False
class WorkbenchOperationsRepository:
"""Read provisioned workbench records without creating or mutating storage."""
@staticmethod
async def availability(
conn: AsyncConnection,
) -> WorkbenchOperationsAvailability:
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT to_regclass('workbench.alerts') IS NOT NULL AS alerts
"""
)
row = await cur.fetchone()
return WorkbenchOperationsAvailability(
alerts=bool(row and row["alerts"]),
)
@staticmethod
async def list_alerts(
conn: AsyncConnection, project_id: str, *, limit: int = 100
) -> list[dict[str, Any]]:
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT document
FROM workbench.alerts
WHERE project_id = %s
AND status = 'active'
ORDER BY occurred_at DESC, alert_id
LIMIT %s
""",
(project_id, limit),
)
return [row["document"] for row in await cur.fetchall()]
+2
View File
@@ -4,6 +4,7 @@ from contextlib import contextmanager
from contextvars import ContextVar, Token
from dataclasses import dataclass
from typing import Iterator
from uuid import UUID
from psycopg.conninfo import conninfo_to_dict, make_conninfo
@@ -15,6 +16,7 @@ class ActiveProjectRouting:
project_code: str
business_dsn: str
timescale_dsn: str | None = None
project_id: UUID | None = None
@property
def business_database_name(self) -> str:
@@ -184,6 +184,7 @@ class AnalysisResultsRepository:
start_time: datetime,
end_time: datetime,
field: str,
*, row_limit: int | None = None,
) -> dict[str, list[dict[str, Any]]]:
if element_type == "node":
table_name, id_column, valid_fields = (
@@ -214,8 +215,10 @@ class AnalysisResultsRepository:
sql.Identifier(id_column),
sql.Identifier(id_column),
)
if row_limit is not None:
query += sql.SQL(" LIMIT %s")
async with conn.cursor() as cur:
await cur.execute(query, (run_id, element_ids, start_time, end_time))
await cur.execute(query, (run_id, element_ids, start_time, end_time) + ((row_limit,) if row_limit is not None else ()))
for row in await cur.fetchall():
result.setdefault(str(row["element_id"]), []).append(
{"time": row["time"], "value": row["value"]}
@@ -3,9 +3,28 @@ from datetime import datetime, timedelta
from collections import defaultdict
from psycopg import AsyncConnection, Connection, sql
from app.domain.time import parse_utc_time
from app.core.config import settings
class RealtimeRepository:
# The third set_config argument makes this transaction-local, so a pooled
# connection returns to its original guard after commit or rollback.
SNAPSHOT_BUDGET_SQL = "SELECT set_config('timescaledb.max_tuples_decompressed_per_dml_transaction', %s, true)"
@staticmethod
async def _configure_snapshot_write(cur) -> None:
await cur.execute(
RealtimeRepository.SNAPSHOT_BUDGET_SQL,
(str(settings.TIMESCALEDB_REALTIME_MAX_DECOMPRESSED_TUPLES),),
)
@staticmethod
def _configure_snapshot_write_sync(cur) -> None:
cur.execute(
RealtimeRepository.SNAPSHOT_BUDGET_SQL,
(str(settings.TIMESCALEDB_REALTIME_MAX_DECOMPRESSED_TUPLES),),
)
LINK_FIELDS = frozenset(
{
"flow",
@@ -93,6 +112,7 @@ class RealtimeRepository:
# 使用事务确保原子性
async with conn.transaction():
async with conn.cursor() as cur:
await RealtimeRepository._configure_snapshot_write(cur)
await cur.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s::text, 1))",
(target_time,),
@@ -116,6 +136,7 @@ class RealtimeRepository:
# 使用事务确保原子性
with conn.transaction():
with conn.cursor() as cur:
RealtimeRepository._configure_snapshot_write_sync(cur)
cur.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s::text, 1))",
(target_time,),
@@ -163,6 +184,7 @@ class RealtimeRepository:
end_time: datetime,
link_ids: list[str],
field: str,
*, row_limit: int | None = None,
) -> dict[str, list[dict[str, Any]]]:
if field not in RealtimeRepository.LINK_FIELDS:
raise ValueError(f"Invalid field: {field}")
@@ -174,8 +196,10 @@ class RealtimeRepository:
"WHERE time BETWEEN %s AND %s AND link_id = ANY(%s) "
"ORDER BY link_id, time"
).format(sql.Identifier(field))
if row_limit is not None:
query += sql.SQL(" LIMIT %s")
async with conn.cursor() as cur:
await cur.execute(query, (start_time, end_time, link_ids))
await cur.execute(query, (start_time, end_time, link_ids) + ((row_limit,) if row_limit is not None else ()))
for row in await cur.fetchall():
result.setdefault(str(row["link_id"]), []).append(
{"time": row["time"].isoformat(), "value": row[field]}
@@ -282,6 +306,7 @@ class RealtimeRepository:
# 使用事务确保原子性
async with conn.transaction():
async with conn.cursor() as cur:
await RealtimeRepository._configure_snapshot_write(cur)
await cur.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s::text, 1))",
(target_time,),
@@ -304,6 +329,7 @@ class RealtimeRepository:
# 使用事务确保原子性
with conn.transaction():
with conn.cursor() as cur:
RealtimeRepository._configure_snapshot_write_sync(cur)
cur.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s::text, 1))",
(target_time,),
@@ -351,6 +377,7 @@ class RealtimeRepository:
end_time: datetime,
node_ids: list[str],
field: str,
*, row_limit: int | None = None,
) -> dict[str, list[dict[str, Any]]]:
if field not in RealtimeRepository.NODE_FIELDS:
raise ValueError(f"Invalid field: {field}")
@@ -362,8 +389,10 @@ class RealtimeRepository:
"WHERE time BETWEEN %s AND %s AND node_id = ANY(%s) "
"ORDER BY node_id, time"
).format(sql.Identifier(field))
if row_limit is not None:
query += sql.SQL(" LIMIT %s")
async with conn.cursor() as cur:
await cur.execute(query, (start_time, end_time, node_ids))
await cur.execute(query, (start_time, end_time, node_ids) + ((row_limit,) if row_limit is not None else ()))
for row in await cur.fetchall():
result.setdefault(str(row["node_id"]), []).append(
{"time": row["time"].isoformat(), "value": row[field]}
@@ -482,6 +511,7 @@ class RealtimeRepository:
# table. Copy-only helpers avoid repeating replacement SQL.
async with conn.transaction():
async with conn.cursor() as cur:
await RealtimeRepository._configure_snapshot_write(cur)
await cur.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s::text, 1))",
(simulation_time,),
@@ -559,6 +589,7 @@ class RealtimeRepository:
# table. Copy-only helpers avoid repeating replacement SQL.
with conn.transaction():
with conn.cursor() as cur:
RealtimeRepository._configure_snapshot_write_sync(cur)
cur.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s::text, 1))",
(simulation_time,),
@@ -34,13 +34,15 @@ class ScadaRepository:
device_ids: List[str],
start_time: datetime,
end_time: datetime,
*, row_limit: int | None = None,
) -> List[dict]:
async with conn.cursor() as cur:
await cur.execute(
f"SELECT {ScadaRepository.RESULT_COLUMNS} FROM scada.measurements "
"WHERE device_id = ANY(%s) "
"AND time >= %s AND time <= %s ORDER BY device_id, time",
(device_ids, start_time, end_time),
"AND time >= %s AND time <= %s ORDER BY device_id, time"
+ (" LIMIT %s" if row_limit is not None else ""),
(device_ids, start_time, end_time) + ((row_limit,) if row_limit is not None else ()),
)
return await cur.fetchall()
+83
View File
@@ -94,6 +94,89 @@ class GeoServerAdminClient:
)
return True
def feature_type_exists(
self,
*,
workspace: str,
datastore: str,
layer: str,
) -> bool:
response = self._client.get(
f"/rest/workspaces/{workspace}/datastores/{datastore}/featuretypes/{layer}.json"
)
if response.status_code == 404:
return False
if response.is_error:
raise GeoServerProvisioningError(
f"GeoServer feature type check failed with HTTP {response.status_code}"
)
return True
def publish_feature_type(
self,
*,
workspace: str,
datastore: str,
layer: str,
title: str,
map_bbox: tuple[float, float, float, float],
) -> None:
minx, miny, maxx, maxy = map_bbox
payload = {
"featureType": {
"name": layer,
"nativeName": layer,
"title": title,
"srs": "EPSG:3857",
"projectionPolicy": "FORCE_DECLARED",
"enabled": True,
"advertised": True,
"nativeBoundingBox": {
"minx": minx,
"miny": miny,
"maxx": maxx,
"maxy": maxy,
"crs": "EPSG:3857",
},
"latLonBoundingBox": {
"minx": _longitude(minx),
"miny": _latitude(miny),
"maxx": _longitude(maxx),
"maxy": _latitude(maxy),
"crs": "EPSG:4326",
},
}
}
if self.feature_type_exists(
workspace=workspace,
datastore=datastore,
layer=layer,
):
self._request(
"PUT",
f"/rest/workspaces/{workspace}/datastores/{datastore}/featuretypes/{layer}",
json=payload,
)
else:
self._request(
"POST",
f"/rest/workspaces/{workspace}/datastores/{datastore}/featuretypes",
json=payload,
)
self._set_client_cache(workspace, layer)
def truncate_layer_cache(self, workspace: str, layer: str) -> None:
self._request(
"POST",
"/gwc/rest/masstruncate",
content=(
"<truncateLayer><layerName>"
f"{workspace}:{layer}"
"</layerName></truncateLayer>"
),
headers={"Content-Type": "text/xml"},
)
def create_project_workspace(
self,
*,
@@ -0,0 +1,218 @@
"""Typed external action adapters. Credentials are request-scoped, never persisted."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Literal, Protocol
from uuid import UUID
import httpx
from pydantic import BaseModel, ConfigDict, Field, ValidationError
from pydantic_settings import BaseSettings, SettingsConfigDict
from app.auth.project_dependencies import ProjectContext
from app.domain.schemas.operational_task import (
ActionReceipt, CreateWorkOrderPayload, HumanApprovalAction,
IssueCommandPayload, OperationalTaskDetail,
)
@dataclass(frozen=True)
class ActionDispatchOutcome:
status: Literal["submitted", "succeeded", "failed", "unsupported", "unknown"]
detail: str
receipt: ActionReceipt | None = None
class OperationalActionDispatcher(Protocol):
async def dispatch(
self, *, context: ProjectContext, task: OperationalTaskDetail,
action: HumanApprovalAction, credential: str | None,
) -> ActionDispatchOutcome: ...
async def reconcile(
self, *, context: ProjectContext, task: OperationalTaskDetail,
action: HumanApprovalAction, credential: str | None,
) -> ActionDispatchOutcome: ...
class OperationalActionSettings(BaseSettings):
peripheral_url: str = ""
command_gateway_url: str = ""
timeout_seconds: float = Field(default=10, ge=1, le=60)
model_config = SettingsConfigDict(
env_prefix="OPERATIONAL_ACTION_", extra="ignore",
env_file=Path(__file__).resolve().parents[3] / ".env",
)
class SampleWorkOrderReceipt(BaseModel):
model_config = ConfigDict(extra="ignore")
work_order_id: UUID
project_id: UUID
created_at: datetime
origin: Literal["synthetic"]
status: Literal[
"draft", "open", "in_progress", "awaiting_reply", "completed", "cancelled"
]
class CommandGatewayReceipt(BaseModel):
model_config = ConfigDict(extra="forbid")
request_id: UUID
project_id: UUID
receipt_id: str = Field(min_length=1)
status: Literal["accepted", "running", "completed", "failed"]
submitted_at: datetime
completed_at: datetime | None = None
detail: str = Field(min_length=1)
class HttpOperationalActionDispatcher:
def __init__(
self, settings: OperationalActionSettings | None = None,
*, transport: httpx.AsyncBaseTransport | None = None,
):
self.settings = settings or OperationalActionSettings()
self.transport = transport
async def dispatch(
self, *, context: ProjectContext, task: OperationalTaskDetail,
action: HumanApprovalAction, credential: str | None,
) -> ActionDispatchOutcome:
base = (self.settings.peripheral_url if action.kind == "create_work_order"
else self.settings.command_gateway_url)
if not base:
return ActionDispatchOutcome("unsupported", "尚未配置该动作的外部执行适配器")
if not credential or not credential.lower().startswith("bearer "):
return ActionDispatchOutcome("failed", "缺少当前用户的产品身份,未提交外部动作")
if action.task_id != task.task_id or task.project_id != context.project_id:
return ActionDispatchOutcome("failed", "动作与当前项目任务不匹配,未提交")
if action.approval_status != "approved":
return ActionDispatchOutcome("failed", "动作尚未通过人工确认,未提交")
request_id = action.execution_request_id
if request_id is None:
return ActionDispatchOutcome("failed", "缺少持久化执行编号,未提交")
headers = {"Authorization": credential, "X-Project-Id": str(context.project_id)}
try:
if action.kind == "create_work_order":
payload = CreateWorkOrderPayload.model_validate(action.payload)
body = {
"request_id": str(request_id), "title": payload.title,
"summary": payload.description, "status": "open",
"priority": payload.priority, "assignee": payload.assignee,
"scheduled_at": payload.scheduled_at.isoformat() if payload.scheduled_at else None,
"location": payload.location,
# An approved action creates a child execution order, not a
# new inbound complaint/burst event for the sync loop.
"work_order_type": "general",
}
path = "/api/sample/v1/work-orders"
recovery_path = f"/api/sample/v1/requests/{request_id}"
else:
payload = IssueCommandPayload.model_validate(action.payload)
if payload.expires_at <= datetime.now(timezone.utc):
return ActionDispatchOutcome("failed", "命令确认窗口已过期,未提交")
body = {
"request_id": str(request_id), "project_id": str(context.project_id),
"task_id": str(task.task_id), "action_id": str(action.action_id),
"action_version": action.version,
"approved_by": action.decided_by,
"approved_at": action.decided_at.isoformat() if action.decided_at else None,
"command": payload.model_dump(mode="json"),
}
path = "/api/v1/commands"
recovery_path = f"/api/v1/commands/requests/{request_id}"
except (ValidationError, TypeError, ValueError):
return ActionDispatchOutcome("failed", "动作参数不符合执行契约,未提交")
async with httpx.AsyncClient(
base_url=base.rstrip("/"), headers=headers, timeout=self.settings.timeout_seconds,
follow_redirects=False, trust_env=False, transport=self.transport,
) as client:
try:
response = await client.post(path, json=body)
if response.status_code in {400, 401, 403, 404, 409, 422}:
# Sample responses explicitly state whether any write occurred.
# An untyped rejection is not proof that no effect took place.
error = response.json()
if isinstance(error, dict) and error.get("effect") == "none":
return ActionDispatchOutcome("failed", "外部系统拒绝了动作,未生效")
response.raise_for_status()
return self._receipt(action.kind, response.json(), request_id, context.project_id)
except (httpx.HTTPError, ValueError, KeyError, ValidationError):
# A lost response can follow a successful write. Recover using the
# same idempotency key, never manufacture success or blindly retry.
try:
recovered = await client.get(recovery_path)
recovered.raise_for_status()
value = recovered.json()
if action.kind == "create_work_order":
if not isinstance(value, dict) or value.get("request_id") != str(request_id):
raise ValueError("request receipt mismatch")
value = value["result"]
return self._receipt(action.kind, value, request_id, context.project_id)
except (httpx.HTTPError, ValueError, KeyError, ValidationError):
return ActionDispatchOutcome("unknown", "外部执行结果待核实,请通过回执核对,勿重复下发")
async def reconcile(
self, *, context: ProjectContext, task: OperationalTaskDetail,
action: HumanApprovalAction, credential: str | None,
) -> ActionDispatchOutcome:
request_id = action.execution_request_id
if not credential or request_id is None or action.task_id != task.task_id or task.project_id != context.project_id:
return ActionDispatchOutcome("unknown", "无法核实当前动作的身份或执行编号")
base = self.settings.peripheral_url if action.kind == "create_work_order" else self.settings.command_gateway_url
if not base:
return ActionDispatchOutcome("unknown", "回执查询服务尚未配置")
path = (f"/api/sample/v1/requests/{request_id}" if action.kind == "create_work_order"
else f"/api/v1/commands/requests/{request_id}")
try:
async with httpx.AsyncClient(
base_url=base.rstrip("/"), timeout=self.settings.timeout_seconds,
follow_redirects=False, trust_env=False, transport=self.transport,
) as client:
response = await client.get(path, headers={
"Authorization": credential, "X-Project-Id": str(context.project_id),
})
response.raise_for_status()
value = response.json()
if action.kind == "create_work_order":
if not isinstance(value, dict) or value.get("request_id") != str(request_id):
raise ValueError("request receipt mismatch")
value = value["result"]
return self._receipt(action.kind, value, request_id, context.project_id)
except (httpx.HTTPError, ValueError, KeyError, ValidationError):
return ActionDispatchOutcome("unknown", "尚未取得可核实的执行回执")
@staticmethod
def _receipt(kind: str, value: object, request_id: UUID, project_id: UUID) -> ActionDispatchOutcome:
if kind == "create_work_order":
order = SampleWorkOrderReceipt.model_validate(value)
if order.project_id != project_id or order.created_at.tzinfo is None:
raise ValueError("invalid external project or timestamp")
receipt = ActionReceipt(
adapter="peripheral_sample", receipt_id=str(order.work_order_id),
submitted_at=order.created_at, completed_at=order.created_at,
external_status=order.status, detail="合成外围工单已创建,现场工作进度由外围系统继续同步",
)
return ActionDispatchOutcome("succeeded", receipt.detail, receipt)
result = CommandGatewayReceipt.model_validate(value)
if result.project_id != project_id or result.request_id != request_id:
raise ValueError("command receipt does not match submission")
if result.submitted_at.tzinfo is None or (result.completed_at and result.completed_at.tzinfo is None):
raise ValueError("command receipt timestamps must include timezone")
if result.status == "completed" and result.completed_at is None:
raise ValueError("command has no completion receipt")
receipt = ActionReceipt(
adapter="command_gateway", receipt_id=result.receipt_id,
submitted_at=result.submitted_at, completed_at=result.completed_at,
external_status=result.status, detail=result.detail,
)
status = "succeeded" if result.status == "completed" else "failed" if result.status == "failed" else "submitted"
return ActionDispatchOutcome(status, result.detail, receipt)
def get_operational_action_dispatcher() -> OperationalActionDispatcher:
return HttpOperationalActionDispatcher()
+185
View File
@@ -0,0 +1,185 @@
"""Read-only source adapters with bounded queries and explicit provenance."""
from __future__ import annotations
import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path
import httpx
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict
from app.auth.permissions import SCADA_VIEW, SIMULATION_VIEW, permissions_for_context
from app.auth.project_dependencies import ProjectContext
from app.domain.schemas.operational_data import TaskDataQuery, TaskDataSnapshot
class TaskDataSettings(BaseSettings):
server_url: str = "http://127.0.0.1:8000"
peripheral_url: str = ""
timeout_seconds: float = Field(default=15, ge=1, le=60)
delegation_bearer: str = ""
model_config = SettingsConfigDict(
env_prefix="OPERATIONAL_DATA_", extra="ignore",
env_file=Path(__file__).resolve().parents[3] / ".env",
)
class TaskDataError(Exception):
def __init__(self, status: int, detail: str):
super().__init__(detail)
self.status = status
self.detail = detail
def _matches_project(value: object, project_id: str) -> bool:
if isinstance(value, list):
return all(_matches_project(item, project_id) for item in value)
if isinstance(value, dict):
if "project_id" in value and value["project_id"] != project_id:
return False
return all(_matches_project(item, project_id) for item in value.values())
return True
def _select_demo_observations(
payload: object, query: TaskDataQuery, project_id: str,
) -> dict[str, object]:
"""Project a bounded DEMO-* observation query from the stored demo dataset."""
if not isinstance(payload, dict) or not _matches_project(payload, project_id):
raise TaskDataError(502, "数据源返回了其他项目的数据")
points = payload.get("points")
observations = payload.get("observations")
if not isinstance(points, list) or not isinstance(observations, list):
raise TaskDataError(502, "外围合成样本格式无效")
if not any(isinstance(point, dict) and point.get("point_id") == query.point_id for point in points):
raise TaskDataError(404, "合成样本测点不存在")
assert query.start is not None and query.end is not None
items: list[object] = []
try:
for item in observations:
if not isinstance(item, dict) or item.get("point_id") != query.point_id:
continue
observed_at = datetime.fromisoformat(str(item.get("observed_at")))
if query.start <= observed_at < query.end:
items.append(item)
except (TypeError, ValueError) as exc:
raise TaskDataError(502, "外围合成样本格式无效") from exc
return {
"origin": "synthetic",
"source_id": payload.get("source_id"),
"source_revision": payload.get("source_revision"),
"point_id": query.point_id,
"window": {
"start": query.start.isoformat(),
"end": query.end.isoformat(),
},
"items": items,
"truncated": False,
}
async def read_task_data(
*, context: ProjectContext, query: TaskDataQuery, credential: str | None,
delegated: bool = False,
settings: TaskDataSettings | None = None,
transport: httpx.AsyncBaseTransport | None = None,
) -> TaskDataSnapshot:
config = settings or TaskDataSettings()
permission = SCADA_VIEW if query.source.startswith("scada_") else SIMULATION_VIEW
if permission not in permissions_for_context(context) and not delegated:
raise TaskDataError(403, "当前身份没有该数据源的读取权限")
if delegated:
credential = config.delegation_bearer
if not credential:
raise TaskDataError(503, "任务授权已批准,但数据源委托身份尚未配置")
if not credential.lower().startswith("bearer "):
credential = f"Bearer {credential}"
if not credential or not credential.lower().startswith("bearer "):
raise TaskDataError(401, "数据查询需要当前产品身份")
peripheral = query.source.startswith("peripheral_")
base = config.peripheral_url if peripheral else config.server_url
if not base:
raise TaskDataError(503, "调查数据源尚未配置")
paths = {
"workbench_runtime": "/api/v1/workbench/runtime",
"scada_devices": "/api/v1/scada-devices",
"scada_readings": "/api/v1/timeseries/scada-readings",
"peripheral_points": "/api/sample/v1/monitoring/points",
"peripheral_work_orders": "/api/sample/v1/work-orders",
"peripheral_observations": "/api/sample/v1/monitoring/observations",
"peripheral_test_data": "/api/sample/v1/test-data",
}
demo_observations = (
query.source == "peripheral_observations"
and query.point_id is not None
and query.point_id.startswith("DEMO-")
)
path = "/api/sample/v1/demo-data" if demo_observations else paths[query.source]
params: dict[str, str | int] = {"limit": 100}
if query.source == "scada_readings":
params.update(device_ids=",".join(query.device_ids), start_time=query.start.isoformat(), end_time=query.end.isoformat())
elif query.source == "peripheral_observations" and not demo_observations:
params = {"point_id": query.point_id, "start": query.start.isoformat(), "end": query.end.isoformat()}
elif demo_observations:
params = {}
try:
async with httpx.AsyncClient(
base_url=base.rstrip("/"), timeout=config.timeout_seconds, follow_redirects=False,
trust_env=False, transport=transport,
) as client:
async with client.stream("GET", path, params=params, headers={
"Authorization": credential, "X-Project-Id": str(context.project_id),
}) as response:
if response.status_code in {401, 403}:
raise TaskDataError(response.status_code, "数据源拒绝当前身份或项目权限")
response.raise_for_status()
body = bytearray()
async for chunk in response.aiter_bytes():
body.extend(chunk)
if len(body) > 2_000_000:
raise TaskDataError(422, "查询结果过大,请缩短时间范围或减少设备数量")
data = json.loads(body)
except (httpx.HTTPError, ValueError) as exc:
raise TaskDataError(502, "调查数据查询失败,不能据此形成已核实结论") from exc
if demo_observations:
data = _select_demo_observations(data, query, str(context.project_id))
elif not _matches_project(data, str(context.project_id)):
raise TaskDataError(502, "数据源返回了其他项目的数据")
origin = "synthetic" if peripheral else "system_data"
historical_replay = False
if query.source in {"peripheral_points", "peripheral_observations"}:
source_origin = data.get("origin") if isinstance(data, dict) else None
if source_origin not in {"historical_replay", "synthetic"}:
raise TaskDataError(502, "监测数据缺少明确来源,不能作为已核实证据")
historical_replay = source_origin == "historical_replay"
origin = "system_data" if historical_replay else "synthetic"
truncated = isinstance(data, dict) and bool(data.get("truncated") or data.get("next_cursor") or data.get("next_offset"))
if isinstance(data, list) and len(data) > 1000:
data, truncated = data[:1000], True
if isinstance(data, dict) and isinstance(data.get("items"), list):
if len(data["items"]) > 1000:
data = {**data, "items": data["items"][:1000]}
truncated = True
if isinstance(data.get("total"), int) and data["total"] > len(data["items"]):
truncated = True
# The digest identifies the returned snapshot, not merely a mutable URL.
digest = hashlib.sha256(json.dumps(data, ensure_ascii=False, sort_keys=True).encode()).hexdigest()
if demo_observations:
assert query.start is not None and query.end is not None
source_ref = (
f"peripheral_demo_data:{query.point_id}:"
f"{query.start.isoformat()}:{query.end.isoformat()}:sha256:{digest}"
)
elif historical_replay:
# Use a distinct identity: old snapshots may have incorrectly labeled
# this same historical response synthetic. Immutable evidence stays intact.
source_ref = f"peripheral_historical_replay:{query.source}:sha256:{digest}"
else:
source_ref = f"{query.source}:sha256:{digest}"
return TaskDataSnapshot(
source=query.source, source_ref=source_ref,
queried_at=datetime.now(timezone.utc), origin=origin,
data=data, truncated=truncated,
)
+38 -10
View File
@@ -1,13 +1,15 @@
from fastapi import FastAPI
from contextlib import asynccontextmanager
from contextlib import asynccontextmanager, AsyncExitStack
from fastapi.middleware.gzip import GZipMiddleware
from fastapi.middleware.cors import CORSMiddleware
import logging
from datetime import datetime
from app.api.problem_details import install_problem_details_handlers
from app.api.health import router as health_router
from app.api.v1.rest_router import api_router
from app.infra.db.dynamic_manager import project_connection_manager
from app.infra.db.postgresql.workbench_event_hub import workbench_event_hubs
from app.infra.db.metadb.database import close_metadata_engine
from app.infra.db.timescaledb.sync_pool import close_all_timescale_pools
from app.native.wndb.core.connection import close_all_project_pools
@@ -23,20 +25,38 @@ logger.setLevel(logging.INFO)
@asynccontextmanager
async def lifespan(app: FastAPI):
app.state.startup_complete = False
logger.info("**********************************************************")
logger.info(str(datetime.now()))
logger.info("TJWater CloudService is starting...")
logger.info("**********************************************************")
yield
# 清理资源
await project_connection_manager.close_all()
close_all_timescale_pools()
close_all_project_pools()
await close_metadata_engine()
logger.info("Database connections closed")
try:
async with AsyncExitStack() as stack:
if mcp_runtime is not None:
await stack.enter_async_context(mcp_runtime.lifespan())
app.state.startup_complete = True
try:
yield
finally:
app.state.startup_complete = False
finally:
await workbench_event_hubs.close_all()
await project_connection_manager.close_all()
close_all_timescale_pools()
close_all_project_pools()
await close_metadata_engine()
logger.info("Database connections closed")
# Disabled deployments do not import the MCP SDK or acquire business credentials.
from app.mcp.config import McpSettings
mcp_settings = McpSettings()
mcp_runtime = None
if mcp_settings.enabled:
from app.mcp.application import McpQueryApplication
mcp_runtime = McpQueryApplication(mcp_settings)
# 根据环境配置决定是否启用文档
is_production = settings.ENVIRONMENT.lower() == "production"
@@ -53,20 +73,28 @@ app = FastAPI(
# Include Routers
app.state.startup_complete = False
app.include_router(health_router)
app.include_router(api_router, prefix="/api/v1")
install_problem_details_handlers(app)
# Legcy Routers without version prefix
from app.api.v1.endpoints.mcp_queries import router as mcp_context_router
app.state.mcp_runtime = mcp_runtime
app.include_router(mcp_context_router, prefix="/api/v1")
# Legacy routers without version prefix
# app.include_router(api_router)
# 配置中间件
app.add_middleware(GZipMiddleware, minimum_size=1000)
# 添加审计中间件(可选,记录关键操作)
app.add_middleware(AuditMiddleware)
if mcp_runtime is not None:
from app.mcp.dispatch import McpDispatchMiddleware
app.add_middleware(McpDispatchMiddleware, runtime=mcp_runtime)
# 配置 CORS 中间件
# 确保这是你最后一个添加的 app.add_middleware
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # 允许所有来源
allow_origins=settings.cors_allowed_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
View File
+139
View File
@@ -0,0 +1,139 @@
"""Bounded calculation inputs; identity, routing and output paths are host-owned."""
from datetime import datetime, timedelta
from typing import Annotated, Literal
from uuid import UUID
from pydantic import Field, model_validator
from app.mcp.models import Id, StrictInput
MAX_CALCULATION_TIMEOUT_SECONDS = 86400
class CalculationInput(StrictInput):
timeout_seconds: int = Field(
default=120, ge=1, le=MAX_CALCULATION_TIMEOUT_SECONDS, strict=True,
description="本次计算等待预算,秒;由Agent按模型规模和任务时长选择,受工具目录中的服务端上限约束;与模拟duration不同,超时不保证已启动的计算停止或回滚",
)
class BurstDetectionInput(CalculationInput):
target_time: datetime = Field(description="带时区的侦测时刻;读取此前15天同周期压力数据,不是单小时定位")
sensor_nodes: list[Id] = Field(min_length=5, max_length=20, description="已核实的压力监测节点ID,至少5个完整传感器")
sampling_interval_minutes: int = Field(default=15, ge=1, le=60)
@model_validator(mode="after")
def valid_target(self):
if self.target_time.tzinfo is None:
raise ValueError("target_time requires explicit timezone")
if 1440 % self.sampling_interval_minutes:
raise ValueError("sampling interval must divide a day")
if len(set(self.sensor_nodes)) != len(self.sensor_nodes):
raise ValueError("sensor_nodes must be distinct")
return self
class BurstLocationInput(CalculationInput):
data_source: Literal["monitoring", "simulation"] = "monitoring"
simulation_run_id: UUID | None = Field(default=None, description="已有分析运行UUID,不是方案名称;simulation模式只读取已有异常结果及模型基线")
burst_leakage: float = Field(gt=0, allow_inf_nan=False, description="有依据的假定漏水流量,WNTR SI单位m³/s;不是已知爆管管段,不能随意猜测")
pressure_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="压力监测设备关联的模型节点ID,不是设备查询编号;省略时自动读取当前项目SCADA配置中的压力测点,不需要猜测节点ID")
flow_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="流量监测设备关联的模型要素ID,不是设备查询编号")
scada_burst_start: datetime
scada_burst_end: datetime
scada_normal_start: datetime | None = Field(default=None, description="monitoring模式正常对照窗开始,须与结束时间成对提供;均省略时用爆管窗前一天相同时段,不代表该时段已确认正常")
scada_normal_end: datetime | None = Field(default=None, description="monitoring模式正常对照窗结束,须与开始时间成对提供;simulation模式不填写,使用同时间窗模型基线")
@model_validator(mode="after")
def valid_windows(self):
pairs = [(self.scada_burst_start, self.scada_burst_end)]
if self.data_source == "monitoring":
if self.simulation_run_id is not None:
raise ValueError("monitoring cannot use simulation_run_id")
if self.scada_normal_start is not None or self.scada_normal_end is not None:
pairs.append((self.scada_normal_start, self.scada_normal_end))
elif self.simulation_run_id is None or self.scada_normal_start is not None or self.scada_normal_end is not None:
raise ValueError("simulation requires run_id and uses the same-window model baseline")
for start, end in pairs:
if start is None or end is None or start.tzinfo is None or end.tzinfo is None:
raise ValueError("explicit timezone-aware burst and normal windows are required")
if not timedelta(0) < end - start <= timedelta(days=1):
raise ValueError("each window must be positive and at most 24 hours")
for ids in (self.pressure_scada_ids, self.flow_scada_ids):
if ids is not None and len(set(ids)) != len(ids):
raise ValueError("sensor IDs must be distinct")
return self
class ValveIsolationInput(CalculationInput):
accident_element: list[Id] = Field(min_length=1, max_length=20, description="经核实或明确作为候选的事故管段/节点ID")
disabled_valves: list[Id] = Field(default_factory=list, max_length=200, description="已知无法关闭的阀门;未提供不代表现场确认全部可用")
class LeakageIdentificationInput(CalculationInput):
scada_start: datetime
scada_end: datetime
sensor_nodes: list[Id] = Field(min_length=1, max_length=20)
q_sum: float = Field(gt=0, allow_inf_nan=False, description="有依据的总漏损流量约束,m³/s,不得使用无依据默认值")
start_time: float = Field(ge=0, lt=24, allow_inf_nan=False, description="模型起算小时;需与SCADA时间窗及模型时钟对齐")
timestep: int = Field(default=15, ge=5, le=60, description="计算步长,分钟;需与观测采样对应")
dma_count: int = Field(ge=1, le=20)
@model_validator(mode="after")
def valid_window(self):
if self.scada_start.tzinfo is None or self.scada_end.tzinfo is None:
raise ValueError("timestamps require explicit timezone")
duration = self.scada_end - self.scada_start
if not timedelta(0) < duration <= timedelta(days=1):
raise ValueError("window must be positive and at most 24 hours")
if duration.total_seconds() % (self.timestep * 60):
raise ValueError("window must contain whole calculation steps")
if len(set(self.sensor_nodes)) != len(self.sensor_nodes):
raise ValueError("sensor_nodes must be distinct")
return self
class ScenarioInput(CalculationInput):
start_time: datetime
duration: int = Field(default=900, ge=60, le=86400, description="模拟时长,秒;首次优先短时间窗")
output_node_ids: list[Id] = Field(min_length=1, max_length=20, description="需要返回结果的节点;不把局部结果解释为全网无影响")
output_link_ids: list[Id] = Field(default_factory=list, max_length=20)
@model_validator(mode="after")
def timezone_required(self):
if self.start_time.tzinfo is None:
raise ValueError("start_time requires explicit timezone")
return self
class ContaminantSimulationInput(ScenarioInput):
source: Id = Field(description="假设污染源节点;这是正向传播模拟,不是自动反演污染源")
concentration: float = Field(gt=0, allow_inf_nan=False, description="假设源浓度,mg/L")
pattern: Id | None = Field(default=None, description="已存在的污染源时间模式ID;不填为持续恒定源")
class ValveClosureInput(ScenarioInput):
valves: list[Id] = Field(min_length=1, max_length=20, description="在临时模型中关闭的阀门,不下发现场控制")
class FlushingInput(ScenarioInput):
drainage_node_id: Id
flushing_flow_m3h: float = Field(gt=0, allow_inf_nan=False, description="明确的冲洗流量,m³/h;不使用无依据的自动排放量")
closed_valves: list[Id] = Field(default_factory=list, max_length=20)
NonnegativeValue = Annotated[float, Field(ge=0, allow_inf_nan=False)]
PumpSeries = Annotated[list[NonnegativeValue], Field(min_length=1, max_length=96)]
class PressureRegulationInput(ScenarioInput):
pump_control: dict[Id, PumpSeries] = Field(min_length=1, max_length=20, description="已登记泵ID及运行序列;工频值为0/1,变频值为频率Hz,每项对应模型水力步长,不自动优化")
tank_init_level: dict[Id, Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(default_factory=dict, max_length=20, description="水箱初始水位,须在模型允许范围内")
class SensorPlacementInput(CalculationInput):
method: Literal["sensitivity", "kmeans"]
sensor_count: int = Field(ge=1, le=20)
min_diameter: int = Field(default=0, ge=0, le=5000, description="最小管径,mm")
+69
View File
@@ -0,0 +1,69 @@
"""Side-effect-limited adapters for existing domain calculations."""
import asyncio
from threading import BoundedSemaphore
from typing import Any
from fastapi import HTTPException
from pydantic import BaseModel
from app.mcp.scenario_service import SCENARIO_TOOLS, calculate_scenario
# These services synchronously read SCADA/model series or write online results.
# Topology-only isolation and model-based sensor placement need only BizDB.
TIMESCALE_CALCULATIONS = SCENARIO_TOOLS | frozenset({
"run_burst_detection", "run_burst_location", "run_leakage_identification",
})
# A timed-out HTTP request cannot kill a native algorithm thread. Keep its slot
# until the calculation really exits, so retries cannot accumulate CPU jobs.
_slots = BoundedSemaphore(2)
def _calculate(name: str, parameters: dict[str, Any], network: str, username: str) -> dict[str, Any]:
if not _slots.acquire(blocking=False):
raise HTTPException(503, "Calculation capacity is occupied")
try:
if name in SCENARIO_TOOLS:
return calculate_scenario(name, parameters, network=network, username=username)
if name == "optimize_sensor_placement":
from app.services.sensor_placement import optimize_sensor_placement_by_kmeans, optimize_sensor_placement_by_sensitivity
parameters = dict(parameters)
method = parameters.pop("method")
optimizer = optimize_sensor_placement_by_sensitivity if method == "sensitivity" else optimize_sensor_placement_by_kmeans
return optimizer(project_code=network, run_name="MCP preview", created_by=username,
persist_results=False, **parameters)
if name == "run_leakage_identification":
from app.services.dma_leakage_estimation import run_leakage_identification
duration = (parameters["scada_end"] - parameters["scada_start"]).total_seconds() / 3600
return run_leakage_identification(
network=network, username=username, scheme_name=None,
duration=duration, q_sum_unit="m3/s", output_flow_unit="m3/s",
n_workers=1, pop_size=50, max_gen=100, **parameters,
)
if name == "run_burst_detection":
from app.services.burst_detection import run_burst_detection
return run_burst_detection(network=network, username=username, scheme_name=None, **parameters)
if name == "run_burst_location":
from app.services.burst_location import run_burst_location_by_network
return run_burst_location_by_network(
network=network, username=username, scheme_name=None,
final_candidates_csv_path=None, **parameters,
)
if name == "analyze_valve_isolation":
from app.services.valve_isolation import analyze_valve_isolation
return analyze_valve_isolation(network=network, **parameters)
raise ValueError("Calculation is not registered")
finally:
_slots.release()
async def calculate(name: str, args: BaseModel, *, network: str, username: str) -> dict[str, Any]:
# to_thread copies the validated project's routing context. Never accept
# database routes, actor identity, arbitrary measurements or paths from MCP.
return await asyncio.to_thread(
_calculate, name, args.model_dump(exclude={"context_handle", "timeout_seconds"}, exclude_none=True), network, username,
)
+505
View File
@@ -0,0 +1,505 @@
"""Official SDK transport with product authorization, persistence and query handlers."""
import asyncio
import json
import logging
from contextlib import asynccontextmanager
from urllib.parse import urlsplit
from uuid import uuid5, NAMESPACE_URL
from typing import Any
from fastapi import HTTPException
from mcp import types
from mcp.server import Server
from mcp.server.auth.middleware.auth_context import get_access_token
from mcp.server.auth.settings import AuthSettings
from mcp.server.auth.routes import create_protected_resource_routes
from mcp.server.caching import CacheHint
from mcp.server.transport_security import TransportSecuritySettings
from mcp.shared.exceptions import MCPError
from psycopg.errors import UndefinedTable, InvalidSchemaName
from pydantic import ValidationError
from starlette.responses import JSONResponse
from app.mcp.auth import (
McpTokenVerifier,
QueryScopeMiddleware,
QUERY_SCOPE,
CONTEXT_SCOPE,
challenge,
)
from app.mcp.catalog import TOOLS
from app.mcp.config import McpSettings
from app.mcp.contexts import resolve_context, verify_task
from app.mcp.models import QueryResult
from app.mcp.scope import project_scope
from app.mcp.query_service import execute_query, task_service, authorize_query
from app.infra.db.postgresql.mcp_queries import McpQueryRepository
from app.domain.schemas.operational_data import TaskDataSnapshot
logger = logging.getLogger(__name__)
def tool_error(code, message):
return types.CallToolResult(
isError=True,
content=[types.TextContent(type="text", text=message)],
structuredContent={"error": {"code": code, "message": message}},
)
class ModernOnlyMiddleware:
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] != "http" or scope.get("path") != "/mcp":
return await self.app(scope, receive, send)
if scope["method"] in {"GET", "DELETE"}:
return await JSONResponse(
{"error": "modern_protocol_required"},
status_code=405,
headers={"Allow": "POST"},
)(scope, receive, send)
headers = {
k.decode("latin1").lower(): v.decode("latin1")
for k, v in scope.get("headers", [])
}
version = headers.get("mcp-protocol-version")
if version != "2026-07-28":
body = bytearray()
request_id = None
body_version = None
while True:
message = await receive()
if message["type"] == "http.disconnect":
return
body.extend(message.get("body", b""))
if len(body) > 1024 * 1024:
return await JSONResponse(
{"error": "request_too_large"}, status_code=413
)(scope, receive, send)
if not message.get("more_body"):
break
try:
payload = json.loads(body)
request_id = payload.get("id")
body_version = payload.get("params", {}).get("_meta", {}).get(
types.PROTOCOL_VERSION_META_KEY
)
except (ValueError, AttributeError):
pass
mismatch = not version or (
isinstance(body_version, str) and body_version != version
)
code = -32020 if mismatch else -32022
result = {
"jsonrpc": "2.0",
"id": request_id,
"error": {
"code": code,
"message": (
"MCP-Protocol-Version header is missing or differs from request metadata"
if mismatch else "MCP 2026-07-28 is required"
),
"data": {"supported": ["2026-07-28"], "requested": version},
},
}
return await JSONResponse(result, status_code=400)(scope, receive, send)
await self.app(scope, receive, send)
class McpQueryApplication:
def __init__(
self,
settings: McpSettings,
*,
verifier=None,
scope_factory=project_scope,
executor=execute_query,
):
self.settings = settings
self.verifier = verifier or McpTokenVerifier(settings)
self.scope_factory = scope_factory
self.executor = executor
self.active = {}
self.ready = False
self.server = Server(
"tjwater-query",
version="1.0.0",
instructions="Project-bound water queries. Model baselines are not observations. Truncated results do not cover the full range.",
cache_hints={
"server/discover": CacheHint(300000, "public"),
"tools/list": CacheHint(300000, "public"),
},
on_list_tools=self.list_tools,
on_call_tool=self.call_tool,
get_tool_input_schema=self.tool_input_schema,
on_list_resources=self.list_resources,
on_list_resource_templates=self.list_templates,
on_read_resource=self.read_resource,
)
self.server.add_request_handler(
"server/discover", types.RequestParams, self.discover
)
app = self.server.streamable_http_app(
stateless_http=True,
json_response=True,
max_request_body_size=1024 * 1024,
auth=AuthSettings(
issuer_url=settings.issuer_uri,
resource_server_url=settings.resource_uri,
required_scopes=["mcp.catalog.read"],
validate_token_resource=True,
),
token_verifier=self.verifier,
transport_security=TransportSecuritySettings(
allowed_hosts=settings.allowed_hosts,
allowed_origins=settings.allowed_origins,
),
)
# Catalog authentication and advertised resource scopes are different concerns.
metadata = create_protected_resource_routes(
resource_url=settings.resource_uri,
authorization_servers=[settings.issuer_uri],
scopes_supported=["mcp.catalog.read", QUERY_SCOPE, CONTEXT_SCOPE],
resource_name="TJWater Query",
)
metadata_path = metadata[0].path
app.router.routes = [
route for route in app.router.routes if route.path != metadata_path
] + metadata
self.asgi = ModernOnlyMiddleware(
QueryScopeMiddleware(app, self.verifier, settings)
)
from app.api.v1.endpoints.mcp_queries import router
self.router = router
@asynccontextmanager
async def lifespan(self):
async with self.server.session_manager.run():
self.ready = True
try:
yield
finally:
self.ready = False
async def discover(self, ctx, params):
return types.DiscoverResult(
supported_versions=["2026-07-28"],
capabilities=self.server.get_capabilities(
protocol_version=ctx.protocol_version,
extensions={"io.modelcontextprotocol/oauth-client-credentials": {}},
),
instructions=self.server.instructions,
)
def tool_input_schema(self, name: str) -> dict[str, Any] | None:
tool = TOOLS.get(name)
if tool is None:
return None
schema = tool.model.model_json_schema()
if tool.computation:
schema["properties"]["timeout_seconds"]["maximum"] = self.settings.max_calculation_timeout_seconds
return schema
async def list_tools(self, ctx, params):
if params and params.cursor:
raise MCPError(-32602, "The query catalog has no continuation page")
tools = []
for name in sorted(TOOLS):
tool = TOOLS[name].protocol_tool()
tool.input_schema.update(self.tool_input_schema(name))
tools.append(tool)
return types.ListToolsResult(
tools=tools,
ttlMs=300000,
cacheScope="public",
)
async def list_resources(self, ctx, params):
return types.ListResourcesResult(resources=[], ttlMs=0, cacheScope="private")
async def list_templates(self, ctx, params):
return types.ListResourceTemplatesResult(
resourceTemplates=[
types.ResourceTemplate(
name="task-evidence",
uriTemplate="tjwater://tasks/{task_id}/evidence/{evidence_id}",
mimeType="application/json",
)
],
ttlMs=300000,
cacheScope="public",
)
async def read_resource(self, ctx, params):
meta = ctx.meta or {}
handle = meta.get("io.tjwater/contextHandle")
url = urlsplit(str(params.uri))
parts = url.path.strip("/").split("/")
if (
url.scheme != "tjwater"
or url.netloc != "tasks"
or len(parts) != 3
or parts[1] != "evidence"
or url.query
or url.fragment
):
raise MCPError(-32602, "Unknown query resource")
result = await self._query(
ctx,
"get_task_evidence",
{"context_handle": handle, "evidence_id": parts[2]},
resource_task=parts[0],
include_full=True,
)
if result.isError:
raise MCPError(
-32602, "Evidence is unavailable in this authorization context"
)
return types.ReadResourceResult(
contents=[
types.TextResourceContents(
uri=str(params.uri),
mimeType="application/json",
text=json.dumps(result.structuredContent, ensure_ascii=False),
)
],
ttlMs=0,
cacheScope="private",
)
async def call_tool(self, ctx, params):
return await self._query(ctx, params.name, params.arguments or {})
async def _query(
self, ctx, name, arguments, resource_task=None, include_full=False
):
tool = TOOLS.get(name)
if tool is None:
raise MCPError(-32602, "Unknown query tool")
principal = get_access_token()
if principal is None or QUERY_SCOPE not in principal.scopes:
return tool_error("PERMISSION_DENIED", "Query permission is required")
try:
args = tool.model.model_validate(arguments)
except ValidationError:
return tool_error(
"INVALID_QUERY", "Query parameters do not match the tool schema"
)
timeout_seconds = (
args.timeout_seconds if tool.computation else self.settings.query_timeout_seconds
)
if tool.computation and timeout_seconds > self.settings.max_calculation_timeout_seconds:
return tool_error("INVALID_QUERY", "Calculation timeout exceeds the server limit")
request = ctx.request
project = request.headers.get("x-project-id", "") if request else ""
meta = ctx.meta or {}
call_id = meta.get("io.tjwater/callId")
if not isinstance(call_id, str) or not 0 < len(call_id) <= 128:
return tool_error(
"CALL_ID_REQUIRED", "A host-owned call identifier is required"
)
key = (principal.subject, project)
if self.active.get(key, 0) >= 4:
return tool_error(
"RATE_LIMITED", "At most four queries may run concurrently"
)
self.active[key] = self.active.get(key, 0) + 1
try:
async with (
asyncio.timeout(timeout_seconds),
self.scope_factory(principal, project) as scope,
):
binding = await resolve_context(scope, principal, args.context_handle)
if resource_task is not None and str(binding.task_id) != resource_task:
raise HTTPException(403, "Resource task mismatch")
await authorize_query(scope, binding, tool, args)
repo = McpQueryRepository(scope.connection, scope.project.project_id)
parameters = {
"tool": name,
"arguments": args.model_dump(
mode="json", exclude={"context_handle"}
),
}
scope_key = json.dumps(
[
binding.subject,
binding.client_id,
binding.native_session_id,
str(binding.task_id),
str(binding.run_id),
str(binding.execution_request_id),
],
separators=(",", ":"),
)
previous = (
await repo.receipt(scope_key, call_id) if binding.task_id else None
)
if previous:
if previous["parameters"] != parameters:
raise HTTPException(
409, "Call identifier has different query parameters"
)
result = QueryResult.model_validate(previous["result"])
else:
result = (
await self.executor(
scope, binding, tool, args, full_snapshot=True
)
if include_full
else await self.executor(scope, binding, tool, args)
)
# Never serialize an unbounded response into a model message or database receipt.
if len(result.model_dump_json().encode()) > 8 * 1024 * 1024:
return tool_error(
"RESULT_TOO_LARGE",
"Narrow the query or request a smaller page",
)
if binding.task_id and name not in {
"get_task_context",
"get_task_evidence",
}:
async with scope.connection.transaction():
await repo.lock_receipt(scope_key, call_id)
previous = await repo.receipt(scope_key, call_id)
if previous:
if previous["parameters"] != parameters:
raise HTTPException(
409,
"Call identifier has different query parameters",
)
result = QueryResult.model_validate(previous["result"])
else:
task_repo = task_service(scope).repo
# Lock the task row only after external/data-source work has finished.
await task_repo.lock(f"task:{binding.task_id}")
await verify_task(scope, binding)
result.source_ref = "mcp:" + str(
uuid5(NAMESPACE_URL, scope_key + ":" + call_id)
)
snapshot = TaskDataSnapshot(
source=name,
source_ref=result.source_ref,
queried_at=result.queried_at,
origin=result.origin,
data=result.model_dump(mode="json"),
truncated=result.truncated,
)
evidence, _ = await task_service(
scope
).record_data_snapshot(binding.task_id, snapshot)
result.evidence_id = evidence.evidence_id
await repo.save_receipt(
scope_key,
call_id,
parameters,
result.model_dump(mode="json"),
)
output = result.model_dump(mode="json")
content = []
if (
not include_full
and len(json.dumps(output, ensure_ascii=False).encode()) > 12000
):
evidence_id = result.evidence_id or (
args.evidence_id if name == "get_task_evidence" else None
)
if binding.task_id and evidence_id:
uri = (
f"tjwater://tasks/{binding.task_id}/evidence/{evidence_id}"
)
output["data"] = {
"resource_uri": uri,
"preview": "The complete query snapshot is saved in task evidence",
}
output["truncated"] = True
output["data_gaps"] = [
*result.data_gaps,
"Inline result abbreviated; read the saved evidence resource",
]
content.append(
types.ResourceLink(
type="resource_link",
uri=uri,
name="query-evidence",
mimeType="application/json",
)
)
else:
return tool_error(
"RESULT_TOO_LARGE",
"Request a smaller page or narrower window",
)
content.insert(
0,
types.TextContent(
type="text", text=json.dumps(output, ensure_ascii=False)
),
)
return types.CallToolResult(content=content, structuredContent=output)
except HTTPException as exc:
codes = {
401: "AUTH_REQUIRED",
403: "QUERY_CONTEXT_DENIED",
404: "NOT_FOUND",
409: "QUERY_CONTEXT_STALE",
413: "RESULT_TOO_LARGE",
422: "INVALID_QUERY",
503: "SOURCE_UNAVAILABLE",
}
return tool_error(
codes.get(exc.status_code, "QUERY_FAILED"),
str(exc.detail)
if isinstance(exc.detail, str)
and exc.status_code in {403, 404, 409, 413, 422}
else "Query is unavailable",
)
except (UndefinedTable, InvalidSchemaName):
return tool_error(
"MCP_STORAGE_UNAVAILABLE",
"MCP project storage has not been provisioned",
)
except TimeoutError:
return tool_error("QUERY_TIMEOUT", "Query execution timed out")
except ValueError:
return tool_error(
"INVALID_QUERY", "Invalid query value or incompatible source data"
)
except Exception:
logger.warning(
"MCP query unavailable", extra={"tool": name, "project_id": project}
)
return tool_error("SOURCE_UNAVAILABLE", "Query source is unavailable")
finally:
remaining = self.active[key] - 1
if remaining:
self.active[key] = remaining
else:
self.active.pop(key, None)
async def identity(self, request):
auth = request.headers.get("authorization", "")
try:
principal = (
await self.verifier.verify_token(auth[7:])
if auth.lower().startswith("bearer ")
else None
)
except Exception:
raise HTTPException(503, "OAuth key verification is unavailable") from None
if not principal:
raise HTTPException(
401,
"MCP authentication required",
headers=challenge(self.settings, CONTEXT_SCOPE),
)
if CONTEXT_SCOPE not in principal.scopes:
raise HTTPException(
403,
"MCP context scope required",
headers=challenge(self.settings, CONTEXT_SCOPE, insufficient=True),
)
return principal
+169
View File
@@ -0,0 +1,169 @@
"""Verify resource-bound Keycloak tokens without passing them to business APIs."""
import asyncio
import time
from uuid import UUID
import httpx
from jose import JWTError, jwt
from mcp.server.auth.provider import AccessToken
from starlette.responses import JSONResponse
from app.mcp.config import McpSettings
CATALOG_SCOPE = "mcp.catalog.read"
QUERY_SCOPE = "mcp.query"
CONTEXT_SCOPE = "mcp.context.bind"
class McpTokenVerifier:
def __init__(self, settings: McpSettings, client: httpx.AsyncClient | None = None):
self.settings = settings
self.client = client
self._keys: list[dict] = []
self._expires = 0.0
self._last_refresh = 0.0
self._lock = asyncio.Lock()
async def _load_keys(self, kid: str) -> list[dict]:
now = time.monotonic()
if self._expires > now and any(k.get("kid") == kid for k in self._keys):
return self._keys
async with self._lock:
now = time.monotonic()
if self._expires > now and any(k.get("kid") == kid for k in self._keys):
return self._keys
if self._expires > now and now - self._last_refresh < 20:
return self._keys
url = (
self.settings.issuer_uri.rstrip("/") + "/protocol/openid-connect/certs"
)
self._last_refresh = now
if self.client:
response = await self.client.get(url)
else:
async with httpx.AsyncClient(
timeout=5, trust_env=False, follow_redirects=False
) as client:
response = await client.get(url)
response.raise_for_status()
try:
payload = response.json()
except ValueError:
raise httpx.RequestError("invalid OAuth key set response") from None
keys = payload.get("keys") if isinstance(payload, dict) else None
if not isinstance(keys, list) or len(keys) > 100:
raise httpx.RequestError("invalid OAuth key set")
self._keys = [
k
for k in keys
if isinstance(k, dict)
and k.get("kty") == "RSA"
and k.get("use", "sig") == "sig"
and k.get("alg", "RS256") == "RS256"
]
self._expires = now + 300
return self._keys
async def verify_token(self, token: str) -> AccessToken | None:
if not token or len(token) > 16384:
return None
try:
header = jwt.get_unverified_header(token)
kid = header.get("kid")
if (
header.get("alg") != "RS256"
or not isinstance(kid, str)
or not 0 < len(kid) <= 128
):
return None
keys = await self._load_keys(kid)
key = next((k for k in keys if k.get("kid") == kid), None)
if key is None:
return None
claims = jwt.decode(
token,
key,
algorithms=["RS256"],
audience=self.settings.audience,
issuer=self.settings.issuer_uri,
options={"require_exp": True, "require_iat": True, "require_sub": True},
)
UUID(claims["sub"])
client_id = claims.get("azp")
if not isinstance(client_id, str) or not client_id:
return None
if (
not isinstance(claims.get("iat"), (int, float))
or not time.time() - 900 < claims["iat"] <= time.time() + 30
or claims["exp"] > claims["iat"] + 900
):
return None
return AccessToken(
token=token,
client_id=client_id,
subject=claims["sub"],
scopes=str(claims.get("scope", "")).split(),
expires_at=int(claims["exp"]),
resource=self.settings.resource_uri,
claims=claims,
)
except (JWTError, ValueError, KeyError, TypeError):
return None
# JWKS/network failures propagate as unavailable, never as empty data.
def challenge(
settings: McpSettings, scope: str, *, insufficient: bool = False
) -> dict[str, str]:
from mcp.server.auth.routes import build_resource_metadata_url
metadata = build_resource_metadata_url(settings.resource_uri)
suffix = ', error="insufficient_scope"' if insufficient else ""
return {
"WWW-Authenticate": f'Bearer resource_metadata="{metadata}", scope="{scope}"{suffix}'
}
class QueryScopeMiddleware:
"""HTTP scope challenges before JSON-RPC dispatch; SDK validates mirrored headers."""
def __init__(self, app, verifier: McpTokenVerifier, settings: McpSettings):
self.app, self.verifier, self.settings = app, verifier, settings
async def __call__(self, scope, receive, send):
if scope["type"] != "http" or scope.get("path") != "/mcp":
return await self.app(scope, receive, send)
headers = {
k.decode("latin1").lower(): v.decode("latin1")
for k, v in scope.get("headers", [])
}
auth = headers.get("authorization", "")
try:
token = (
await self.verifier.verify_token(auth[7:])
if auth.lower().startswith("bearer ")
else None
)
except (httpx.HTTPError, ValueError):
return await JSONResponse(
{"error": "authentication_unavailable"}, status_code=503
)(scope, receive, send)
required = (
QUERY_SCOPE
if headers.get("mcp-method") in {"tools/call", "resources/read"}
else CATALOG_SCOPE
)
if token is None:
return await JSONResponse(
{"error": "invalid_token"},
status_code=401,
headers=challenge(self.settings, required),
)(scope, receive, send)
if required not in token.scopes:
return await JSONResponse(
{"error": "insufficient_scope"},
status_code=403,
headers=challenge(self.settings, required, insufficient=True),
)(scope, receive, send)
await self.app(scope, receive, send)
+221
View File
@@ -0,0 +1,221 @@
"""One finite query catalog; names, schemas and permissions come from this registry."""
from dataclasses import dataclass
from pydantic import BaseModel
from mcp import types
from app.mcp import models as m
from app.mcp.analysis_models import BurstDetectionInput, BurstLocationInput, ValveIsolationInput, LeakageIdentificationInput
from app.mcp.analysis_models import ScenarioInput, ContaminantSimulationInput, ValveClosureInput, FlushingInput, PressureRegulationInput, SensorPlacementInput
@dataclass(frozen=True)
class QueryTool:
name: str
description: str
model: type[BaseModel]
permission: str
semantics: str
task_only: bool = False
computation: bool = False
def protocol_tool(self) -> types.Tool:
if self.name == "run_simulation":
suffix = "。任务调用复用证据留存。"
elif self.computation:
suffix = "。仅计算建议,不保存方案、不控制设备;任务调用复用证据留存。"
elif self.name == "query_nearby_elements":
suffix = "。仅查询当前已授权项目的模型管网,按距离和元素ID稳定排序;按需分页,不自动遍历全量;空间邻近不代表管网连通或供水关系。"
else:
suffix = (
"。先限定对象和时间范围,再读取明细;集合默认20条,按需逐页扩大,"
"不要自动遍历全量;截断或非零offset不能证明完整范围无异常。"
)
return types.Tool(
name=self.name,
description=self.description + suffix,
inputSchema=self.model.model_json_schema(),
outputSchema=m.QueryResult.model_json_schema(),
annotations=types.ToolAnnotations(
readOnlyHint=False, destructiveHint=self.name == "run_simulation", openWorldHint=False
),
)
_specs = [
("run_simulation", "按水力步长在在线业务模型上执行指定时间段的实时模拟,不复制数据库;更新该时段的模型条件及实时基线,返回指定节点和管段结果;duration单位为秒,不创建方案或正式分析运行,结果不是实测", ScenarioInput, "simulation.run", "analysis", False, True),
("run_contaminant_simulation", "在隔离模型中模拟假设污染源的浓度传播;返回指定节点/管段结果,不反演污染源,不代表实测污染范围", ContaminantSimulationInput, "simulation.run", "analysis", False, True),
("analyze_valve_closure", "在隔离模型中模拟关阀的水力影响,返回指定节点/管段结果;不操作真实阀门", ValveClosureInput, "simulation.run", "analysis", False, True),
("analyze_flushing", "在隔离模型中评估指定排放节点、冲洗流量和关阀组合的水力结果;不是现场冲洗或消毒合格证明", FlushingInput, "simulation.run", "analysis", False, True),
("analyze_pressure_regulation", "在隔离模型中评估给定调泵序列和水箱初始水位的压力影响;不自动优化调度,不实际控制泵", PressureRegulationInput, "simulation.run", "analysis", False, True),
("optimize_sensor_placement", "使用灵敏度或K-means算法推荐压力监测点;仅返回候选节点,不保存正式方案,不保证污染溯源可观测性", SensorPlacementInput, "optimization.run", "analysis", False, True),
("run_leakage_identification", "基于SCADA压力与模型进行DMA漏损分配识别,需有依据的总漏损流量和时间对齐;返回区域估计,不是已确认漏点。内部执行遗传算法与水力计算", LeakageIdentificationInput, "burst.run", "analysis", False, True),
("run_burst_detection", "爆管异常侦测:使用至少5个完整压力传感器此前15天数据;只提示压力异常,不定位、不证明爆管", BurstDetectionInput, "burst.run", "analysis", False, True),
("run_burst_location", "爆管候选定位:对比异常与正常压力,可辅助流量;需有依据的漏水量假设。压力节点可省略,自动读取项目SCADA压力测点;monitoring正常窗可成对省略,默认前一天同时间段,须说明该对照假设。simulation须有已有分析运行UUID并读取同窗模型基线,不是实测;定位内部会执行水力计算,不创建新的事件方案", BurstLocationInput, "burst.run", "analysis", False, True),
("analyze_valve_isolation", "按事故候选要素分析拓扑隔离边界、建议关闭阀门和受影响节点;不是污染扩散或水力影响范围,不执行关阀,须现场核实", ValveIsolationInput, "simulation.run", "analysis", False, True),
(
"get_element_properties",
"读取当前项目的单个管网元素属性",
m.ElementInput,
"webgis.view",
"asset",
),
(
"list_elements",
"分页读取管网元素,不自动读取全网",
m.ElementPageInput,
"webgis.view",
"asset",
),
(
"query_nearby_elements",
"查询坐标附近的模型元素,默认pipe;center为EPSG:4326 [经度,纬度],可直接使用geocode.center;radius_m默认200米、最大10000米,limit默认5、最大100,支持offset。按点到完整几何的WGS84椭球最短距离筛选并排序,返回element_id、distance_m和属性;不查询原始GIS管线、不自动扩大半径、不解析道路交叉点",
m.NearbyElementsInput,
"webgis.view",
"asset",
),
(
"get_network_options",
"读取模型时间、能源或网络选项",
m.OptionsInput,
"webgis.view",
"asset",
),
(
"get_network_schema",
"读取当前支持的元素、SCADA和选项结构",
m.SchemaInput,
"webgis.view",
"asset",
),
(
"list_scada_devices",
"分页读取SCADA设备和物理单位",
m.PageInput,
"scada.view",
"asset",
),
("get_scada_device", "读取一个SCADA设备", m.DeviceInput, "scada.view", "asset"),
(
"query_scada_readings",
"查询带时区且最多24小时的原始或清洗监测数据",
m.ScadaInput,
"scada.view",
"observed",
),
(
"query_model_baseline_series",
"读取最多20个元素的模型基线,不能称为实测",
m.BaselineSeriesInput,
"simulation.view",
"model_baseline",
),
(
"get_model_baseline_at_time",
"读取单个元素指定时刻的模型基线",
m.BaselineAtTimeInput,
"simulation.view",
"model_baseline",
),
(
"query_model_baseline_field",
"分页读取指定时刻的模型字段",
m.BaselineFieldInput,
"simulation.view",
"model_baseline",
),
(
"query_analysis_series",
"读取指定分析运行和元素字段时序",
m.AnalysisSeriesInput,
"simulation.view",
"analysis",
),
(
"query_analysis_values",
"分页读取指定分析运行某时刻的元素字段",
m.AnalysisValuesInput,
"simulation.view",
"analysis",
),
(
"list_analysis_runs",
"分页读取持久分析运行",
m.RunsInput,
"webgis.view",
"analysis",
),
("get_analysis_run", "读取单个持久分析运行", m.RunInput, "webgis.view", "analysis"),
(
"list_analysis_results",
"分页读取分析成果,可筛选result_type",
m.ResultsInput,
"webgis.view",
"analysis",
),
(
"list_sensor_placement_runs",
"分页读取已有监测点选址运行",
m.PageInput,
"optimization.run",
"analysis",
),
(
"get_sensor_placement_run",
"读取已有监测点选址成果",
m.RunInput,
"optimization.run",
"analysis",
),
(
"query_element_history",
"比较实测与模型基线或分析结果;analysis_comparison要求run_id;limit/offset按序列、时间排序的数据点分页",
m.HistoryInput,
"scada.view",
"observed",
),
(
"get_element_gis_properties",
"读取元素关联GIS台账",
m.ElementInput,
"webgis.view",
"asset",
),
(
"get_source_gis_properties",
"读取指定图层源要素台账",
m.SourceGisInput,
"webgis.view",
"asset",
),
(
"get_workbench_runtime",
"读取工作台已有运行态",
m.StrictInput,
"simulation.view",
"task",
),
(
"get_task_context",
"读取绑定任务的有界摘要及handling_skill元数据;section=skill读取Server按任务workflow分发的业务处置Skill正文与版本,属于流程指引而非事件证据;其他section选择task/report/evidence/actions/progress/access_requests/plan/run,data_path按返回路径读取嵌套字段,limit/offset分页;report_revision保留选定报告版本",
m.TaskContextInput,
"operational_task.view",
"task",
True,
),
(
"query_peripheral_data",
"查询有限外围数据源;synthetic是合成证据",
m.PeripheralInput,
"simulation.view",
"peripheral",
),
(
"get_task_evidence",
"读取当前绑定任务已保存的证据快照",
m.EvidenceInput,
"operational_task.view",
"task",
True,
),
]
TOOLS = {row[0]: QueryTool(*row) for row in _specs}
+76
View File
@@ -0,0 +1,76 @@
"""MCP is opt-in; an incomplete OAuth resource must never start listening."""
from pathlib import Path
from ipaddress import IPv4Address, IPv4Network
from urllib.parse import urlsplit
from pydantic import Field, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
class McpSettings(BaseSettings):
enabled: bool = False
issuer_uri: str = ""
dev_http_issuer_uri: str = ""
environment: str = Field(default="production", validation_alias="ENVIRONMENT")
resource_uri: str = ""
audience: str = "tjwater-mcp"
host_client_id: str = "tjwater-agent-mcp"
allowed_hosts: list[str] = Field(
default_factory=lambda: ["127.0.0.1:*", "localhost:*"]
)
allowed_origins: list[str] = Field(default_factory=list)
context_ttl_seconds: int = Field(default=600, ge=30, le=600)
query_timeout_seconds: int = Field(default=30, ge=1, le=60)
max_calculation_timeout_seconds: int = Field(default=3600, ge=120, le=86400)
model_config = SettingsConfigDict(
env_prefix="TJWATER_MCP_",
extra="ignore",
populate_by_name=True,
env_file=Path(__file__).resolve().parents[2] / ".env",
)
@model_validator(mode="after")
def validate_resource(self):
if self.dev_http_issuer_uri:
dev_url = urlsplit(self.dev_http_issuer_uri)
try:
address = IPv4Address(dev_url.hostname or "")
private_ipv4 = any(address in IPv4Network(network) for network in (
"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"
))
except ValueError:
private_ipv4 = False
if (
self.environment != "development"
or dev_url.scheme != "http"
or not private_ipv4
or dev_url.username or dev_url.password or dev_url.query or dev_url.fragment
or self.dev_http_issuer_uri != self.issuer_uri
):
raise ValueError("development HTTP issuer requires development environment and an exact private IPv4 issuer")
if not self.enabled:
return self
for index, value in enumerate((self.issuer_uri, self.resource_uri)):
url = urlsplit(value)
if (
url.scheme not in {"http", "https"}
or not url.hostname
or url.username
or url.password
or url.query
or url.fragment
):
raise ValueError("MCP requires explicit issuer and resource URIs")
if url.scheme == "http" and url.hostname not in {
"localhost",
"127.0.0.1",
"::1",
}:
if index != 0 or value != self.dev_http_issuer_uri:
raise ValueError("non-loopback MCP and OAuth require HTTPS")
if urlsplit(self.resource_uri).path != "/mcp":
raise ValueError("MCP resource URI must identify the /mcp endpoint")
if not self.audience.strip() or not self.host_client_id.strip():
raise ValueError("MCP audience and host client ID are required")
return self
+112
View File
@@ -0,0 +1,112 @@
"""An OAuth identity and a Server-owned binding are both required for data queries."""
import secrets
from datetime import datetime, timedelta, timezone
from uuid import UUID
from fastapi import HTTPException
from app.auth.permissions import (
permissions_for_context,
OPERATIONAL_TASK_EXECUTE,
OPERATIONAL_TASK_VIEW,
)
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.infra.db.postgresql.mcp_queries import McpQueryRepository
from app.mcp.models import BindInput, BoundContext
from app.mcp.scope import QueryScope
from app.mcp.config import McpSettings
from mcp.server.auth.provider import AccessToken
def task_repository(scope: QueryScope) -> OperationalTaskRepository:
return OperationalTaskRepository(scope.connection, scope.project.project_id)
async def verify_task(scope: QueryScope, binding: BoundContext) -> None:
permissions = permissions_for_context(scope.project)
if OPERATIONAL_TASK_VIEW not in permissions:
raise HTTPException(403, "Task read permission is required")
repo = task_repository(scope)
async with scope.connection.transaction():
task = await repo.get_task(str(binding.task_id))
if not task:
raise HTTPException(404, "Task is not in this project")
if binding.purpose == "task_collaboration":
return
if OPERATIONAL_TASK_EXECUTE not in permissions:
raise HTTPException(403, "Task execution permission is required")
run = await repo.get_active_run(str(binding.task_id))
now = datetime.now(timezone.utc)
owner = binding.parent_session_id or binding.native_session_id
if (
not run
or run["status"] not in {"claimed", "running"}
or run.get("agent_session_id") != owner
or not run.get("lease_expires_at")
or run["lease_expires_at"] <= now
or binding.run_id is not None
and str(run["run_id"]) != str(binding.run_id)
or binding.execution_request_id is not None
and str(run["execution_request_id"]) != str(binding.execution_request_id)
):
raise HTTPException(
409, "Investigation run is no longer active for this session"
)
binding.run_id = UUID(str(run["run_id"]))
binding.execution_request_id = UUID(str(run["execution_request_id"]))
async def bind_context(
scope: QueryScope, principal: AccessToken, command: BindInput, settings: McpSettings
) -> BoundContext:
if principal.client_id != settings.host_client_id:
raise HTTPException(
403, "Only the trusted product host may bind query contexts"
)
from app.mcp.catalog import TOOLS
binding = BoundContext(
context_handle=secrets.token_urlsafe(32),
project_id=scope.project.project_id,
subject=principal.subject,
client_id=principal.client_id,
tool_cap=sorted(name for name, tool in TOOLS.items() if command.purpose != "task_reader" or not tool.computation),
permission_cap=sorted(permissions_for_context(scope.project)),
native_session_id=command.native_session_id,
purpose=command.purpose,
parent_session_id=command.parent_session_id,
task_id=command.task_id,
expires_at=datetime.now(timezone.utc)
+ timedelta(seconds=settings.context_ttl_seconds),
)
if binding.task_id:
await verify_task(scope, binding)
from app.infra.db.postgresql.read_queries import task_data_scopes
async with scope.connection.transaction():
binding.task_scope_cap = await task_data_scopes(scope.connection,
project_id=binding.project_id, task_id=binding.task_id,
permissions=permissions_for_context(scope.project))
await McpQueryRepository(scope.connection, scope.project.project_id).create(binding)
return binding
async def resolve_context(
scope: QueryScope, principal: AccessToken, handle: str
) -> BoundContext:
binding = await McpQueryRepository(scope.connection, scope.project.project_id).get(
handle
)
if (
not binding
or binding.revoked_at
or binding.expires_at <= datetime.now(timezone.utc)
or binding.subject != principal.subject
or binding.client_id != principal.client_id
or binding.project_id != scope.project.project_id
):
raise HTTPException(
403, "Query context is unavailable for this identity and project"
)
if binding.task_id:
await verify_task(scope, binding)
return binding
+15
View File
@@ -0,0 +1,15 @@
"""Delegate MCP transport/metadata with the original request path intact."""
class McpDispatchMiddleware:
def __init__(self, app, runtime):
self.app, self.runtime = app, runtime
async def __call__(self, scope, receive, send):
path = scope.get("path", "")
if scope["type"] == "http" and (
path == "/mcp" or path.startswith("/.well-known/oauth-protected-resource")
):
await self.runtime.asgi(scope, receive, send)
else:
await self.app(scope, receive, send)
+293
View File
@@ -0,0 +1,293 @@
"""Strict business inputs and provenance envelopes shared by MCP handlers."""
from datetime import datetime, timedelta
from typing import Annotated, Literal
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, JsonValue, model_validator
Id = Annotated[str, Field(min_length=1, max_length=160, pattern=r"^\S(?:.*\S)?$")]
ElementType = Literal["junction", "pipe", "valve", "pump", "tank", "reservoir"]
SeriesType = Literal["junction", "pipe"]
class StrictInput(BaseModel):
model_config = ConfigDict(extra="forbid")
context_handle: Annotated[str, Field(min_length=32, max_length=128)]
class PageInput(StrictInput):
limit: int = Field(default=20, ge=1, le=500)
offset: int = Field(default=0, ge=0, le=10000)
class TaskContextInput(PageInput):
section: Literal["summary", "task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill"] = "summary"
report_revision: int | None = Field(default=None, ge=1)
data_path: list[Id] = Field(default_factory=list, max_length=8)
@model_validator(mode="after")
def summary_is_fixed(self):
if self.section in {"summary", "skill"} and (self.data_path or self.offset):
raise ValueError("Select a detail section before using data_path or offset")
return self
class ElementInput(StrictInput):
element_type: ElementType
element_id: Id
class ElementPageInput(PageInput):
element_type: ElementType
class NearbyElementsInput(PageInput):
element_type: ElementType = "pipe"
center: tuple[
Annotated[float, Field(ge=-180, le=180, allow_inf_nan=False)],
Annotated[float, Field(ge=-90, le=90, allow_inf_nan=False)],
] = Field(description="EPSG:4326 [longitude, latitude], directly from geocode.center")
radius_m: float = Field(default=200, gt=0, le=10000, allow_inf_nan=False)
limit: int = Field(default=5, ge=1, le=100)
class OptionsInput(StrictInput):
kind: Literal["time", "energy", "pump-energy", "network"]
pump_id: Id | None = None
@model_validator(mode="after")
def pump_required(self):
if self.kind == "pump-energy" and not self.pump_id:
raise ValueError("pump_id is required for pump-energy")
if self.kind != "pump-energy" and self.pump_id:
raise ValueError("pump_id is only valid for pump-energy")
return self
class SchemaInput(StrictInput):
kind: Literal[
"junction",
"pipe",
"valve",
"pump",
"tank",
"reservoir",
"time",
"energy",
"pump-energy",
"option",
"scada-device",
]
class DeviceInput(StrictInput):
device_id: Id
class WindowInput(PageInput):
start: datetime
end: datetime
limit: int = Field(default=20, ge=1, le=10000)
@model_validator(mode="after")
def window_valid(self):
if self.start.tzinfo is None or self.end.tzinfo is None:
raise ValueError("timestamps require explicit timezone")
if not timedelta(0) < self.end - self.start <= timedelta(days=1):
raise ValueError("time window must be positive and at most 24 hours")
return self
class ScadaInput(WindowInput):
device_ids: list[Id] = Field(min_length=1, max_length=20)
field: Literal["monitored_value", "cleaned_value"] = "monitored_value"
class BaselineSeriesInput(WindowInput):
element_type: SeriesType
element_ids: list[Id] = Field(min_length=1, max_length=20)
class TimeInput(StrictInput):
time: datetime
@model_validator(mode="after")
def timezone_required(self):
if self.time.tzinfo is None:
raise ValueError("timestamp requires explicit timezone")
return self
class BaselineAtTimeInput(TimeInput):
element_type: SeriesType
element_id: Id
class BaselineFieldInput(TimeInput):
element_type: SeriesType
field: Id
limit: int = Field(default=20, ge=1, le=500)
offset: int = Field(default=0, ge=0, le=10000)
class AnalysisSeriesInput(WindowInput):
run_id: UUID
element_type: SeriesType
element_id: Id
field: Id
class AnalysisValuesInput(BaselineFieldInput):
run_id: UUID
class RunInput(StrictInput):
run_id: UUID
class RunsInput(PageInput):
limit: int = Field(default=20, ge=1, le=100)
class ResultsInput(PageInput):
run_id: UUID
result_type: Id | None = None
class HistoryTarget(BaseModel):
model_config = ConfigDict(extra="forbid")
element_type: SeriesType
element_id: Id
device_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20)
class HistoryInput(WindowInput):
elements: list[HistoryTarget] = Field(min_length=1, max_length=20)
mode: Literal["observed", "realtime_comparison", "analysis_comparison"] = "observed"
run_id: UUID | None = None
@model_validator(mode="after")
def analysis_required(self):
if (self.mode == "analysis_comparison") != (self.run_id is not None):
raise ValueError("run_id is required only for analysis_comparison")
return self
class SourceGisInput(StrictInput):
layer_key: Annotated[str, Field(min_length=1, max_length=80)]
source_fid: int = Field(ge=0, le=9223372036854775807)
class PeripheralInput(PageInput):
source: Literal[
"peripheral_points",
"peripheral_work_orders",
"peripheral_observations",
"peripheral_test_data",
]
point_id: Id | None = None
start: datetime | None = None
end: datetime | None = None
@model_validator(mode="after")
def observation_window(self):
if self.source == "peripheral_observations":
if not self.point_id or self.start is None or self.end is None:
raise ValueError("observations require point_id, start and end")
if (
self.start.tzinfo is None
or self.end.tzinfo is None
or not timedelta(0) < self.end - self.start <= timedelta(days=1)
):
raise ValueError(
"observation window requires timezone and at most 24 hours"
)
elif (
self.point_id is not None or self.start is not None or self.end is not None
):
raise ValueError(
"point_id and window are only valid for peripheral_observations"
)
return self
class EvidenceInput(PageInput):
evidence_id: Id
data_path: list[Id] = Field(default_factory=list, max_length=8)
class BindInput(BaseModel):
model_config = ConfigDict(extra="forbid")
native_session_id: Id
purpose: Literal[
"ordinary_chat", "task_execution", "task_reader", "task_collaboration"
]
task_id: UUID | None = None
parent_session_id: Id | None = None
@model_validator(mode="after")
def task_scope(self):
if self.purpose == "ordinary_chat" and (self.task_id or self.parent_session_id):
raise ValueError("ordinary chat cannot bind a task")
if self.purpose != "ordinary_chat" and self.task_id is None:
raise ValueError("task context requires task_id")
if (self.purpose == "task_reader") != (self.parent_session_id is not None):
raise ValueError("only a task reader requires parent_session_id")
return self
class BindResponse(BaseModel):
context_handle: str
expires_at: datetime
task_id: UUID | None
run_id: UUID | None
class PageInfo(BaseModel):
limit: int
offset: int
total: int | None = None
has_more: bool
class QueryResult(BaseModel):
model_config = ConfigDict(extra="forbid")
source: str
source_ref: str
queried_at: datetime
origin: Literal["system_data", "synthetic"] = "system_data"
semantics: Literal[
"asset",
"observed",
"model_baseline",
"analysis",
"comparison",
"task",
"peripheral",
]
data: JsonValue
page: PageInfo | None = None
window: dict[str, datetime] | None = None
units: dict[str, str] = Field(default_factory=dict)
truncated: bool = False
data_gaps: list[str] = Field(default_factory=list)
evidence_id: str | None = None
class BoundContext(BaseModel):
context_handle: str
project_id: UUID
subject: str
client_id: str
tool_cap: list[str] = Field(default_factory=list)
permission_cap: list[str] = Field(default_factory=list)
task_scope_cap: list[str] = Field(default_factory=list)
native_session_id: str
purpose: str
parent_session_id: str | None = None
task_id: UUID | None = None
run_id: UUID | None = None
execution_request_id: UUID | None = None
expires_at: datetime
revoked_at: datetime | None = None
+132
View File
@@ -0,0 +1,132 @@
"""Read existing project-owned peripheral facts without forwarding MCP tokens."""
from datetime import datetime, timezone
from uuid import uuid4
from fastapi import HTTPException
from app.infra.db.postgresql.read_queries import MAX_QUERY_BYTES, read_rows, scada_devices, series_rows
from app.domain.schemas.operational_data import TaskDataSnapshot, TaskDataQuery
from app.services.content_digest import digest
from samples.peripheral.store import PostgresStore
from app.infra.integrations.operational_data import _select_demo_observations
async def read_peripheral_query(scope, args):
conn = scope.connection
project = str(scope.project.project_id)
truncated = False
origin = "system_data"
if args.source in {"peripheral_work_orders", "peripheral_test_data"}:
table, column = (
("work_orders", "work_order_id")
if args.source == "peripheral_work_orders"
else ("test_data_records", "record_id")
)
rows = await read_rows(
conn,
"SELECT document FROM peripheral_sample."
+ table
+ " WHERE project_id=%s AND deleted_at IS NULL ORDER BY "
+ column
+ " LIMIT %s OFFSET %s",
(project, args.limit + 1, args.offset),
)
items = [
PostgresStore._active_managed_document(row["document"])
for row in rows[: args.limit]
]
truncated = len(rows) > args.limit
if any(item.get("project_id") != project for item in items):
raise HTTPException(502, "Peripheral project mismatch")
data = {
"items": items,
"truncated": truncated,
"limit": args.limit,
"offset": args.offset,
}
origin = (
"synthetic"
if items and any(item.get("origin") == "synthetic" for item in items)
else "system_data"
)
elif args.source == "peripheral_points":
rows = await scada_devices(conn, limit=args.limit + 1, offset=args.offset)
data = {
"origin": "historical_replay",
"items": [
{
"point_id": row["device_id"],
"label": row["device_id"] + " · " + row["device_type"],
"unit": row["measurement_unit"],
"status": "unknown",
"origin": "historical_replay",
}
for row in rows[: args.limit]
],
"truncated": len(rows) > args.limit,
}
truncated = data["truncated"]
elif args.point_id.startswith("DEMO-"):
rows = await read_rows(
conn,
"SELECT CASE WHEN octet_length(document::text)<=%s THEN document END AS document,"
"digest,octet_length(document::text)>%s AS too_large "
"FROM peripheral_sample.demo_datasets WHERE project_id=%s AND dataset_id='water-demo-v1'",
(MAX_QUERY_BYTES, MAX_QUERY_BYTES, project),
)
if rows and rows[0]["too_large"]:
raise HTTPException(413, "Synthetic dataset exceeds the integrity-check byte budget")
if not rows or digest(rows[0]["document"]) != rows[0]["digest"]:
raise HTTPException(503, "Synthetic dataset is unavailable or invalid")
query = TaskDataQuery(
source=args.source, point_id=args.point_id, start=args.start, end=args.end
)
data = _select_demo_observations(
{**rows[0]["document"], "project_id": project}, query, project
)
items = data["items"]
truncated = len(items) > args.limit + args.offset
data["items"] = items[args.offset : args.offset + args.limit]
data["truncated"] = truncated
origin = "synthetic"
else:
devices = await scada_devices(conn, device_id=args.point_id, limit=1)
if not devices:
raise HTTPException(404, "Peripheral point was not found")
device = devices[0]
if not device.get("api_query_id"):
raise HTTPException(503, "Peripheral history mapping is unavailable")
async with scope.timescale() as ts:
rows = await series_rows(
ts,
source="scada",
element_type="junction",
ids=[device["api_query_id"]],
start=args.start,
end=args.end,
field="monitored_value",
limit=args.limit + 1,
offset=args.offset,
end_exclusive=True,
)
truncated = len(rows) > args.limit
data = {
"origin": "historical_replay",
"items": [
{
"point_id": args.point_id,
"observed_at": row["time"].isoformat(),
"value": row["value"],
"unit": device["measurement_unit"],
}
for row in rows[: args.limit]
],
"truncated": truncated,
}
return TaskDataSnapshot(
source=args.source,
source_ref="mcp:" + str(uuid4()),
queried_at=datetime.now(timezone.utc),
data=data,
origin=origin,
truncated=truncated,
)
+492
View File
@@ -0,0 +1,492 @@
"""MCP domain dispatch reuses Server repositories and services, never an HTTP loopback."""
import asyncio
import math
from contextlib import nullcontext
from typing import Any
from pydantic import BaseModel
from app.mcp.scope import QueryScope
from app.mcp.catalog import QueryTool, TOOLS
from app.mcp.models import BoundContext
from datetime import datetime, timezone
from uuid import uuid4
from fastapi import HTTPException
from fastapi.encoders import jsonable_encoder
from app.auth.permissions import permissions_for_context
from app.infra.db.postgresql import read_queries as reads
from app.infra.db.postgresql.spatial_queries import nearby_elements
from app.infra.db.postgresql.analysis import AnalysisRepository
from app.infra.db.postgresql.gis_attributes import GisAttributeRepository
from app.infra.db.postgresql.network_settings import NetworkSettingsRepository
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.services.operational_tasks import OperationalTaskService
from app.mcp.models import QueryResult, PageInfo
from app.mcp.scenario_service import SCENARIO_TOOLS
from app.mcp.work_order_skills import read_work_order_skill
def finite_data(value: Any) -> tuple[Any, int]:
"""JSON cannot represent infinity/NaN; preserve the gap as null with provenance."""
if isinstance(value, float) and not math.isfinite(value):
return None, 1
count = 0
if isinstance(value, dict):
result = {}
for key, item in value.items():
result[key], replaced = finite_data(item)
count += replaced
return result, count
if isinstance(value, list):
result = []
for item in value:
clean, replaced = finite_data(item)
result.append(clean)
count += replaced
return result, count
return value, 0
def task_service(scope: QueryScope) -> OperationalTaskService:
return OperationalTaskService(
OperationalTaskRepository(scope.connection, scope.project.project_id),
scope.project,
set(permissions_for_context(scope.project)),
)
def page_result(rows, args):
has_more = len(rows) > args.limit
return (
rows[: args.limit],
PageInfo(limit=args.limit, offset=args.offset, total=None, has_more=has_more),
has_more,
)
async def authorize_query(
scope: QueryScope, binding: BoundContext, tool: QueryTool, args: BaseModel
) -> None:
if tool.name not in binding.tool_cap:
raise HTTPException(403, "Query is outside the captured tool catalog")
permissions = permissions_for_context(scope.project)
if tool.computation:
if tool.name == "optimize_sensor_placement" and scope.project.project_role != "member":
raise HTTPException(403, "Sensor placement requires project member access")
if binding.purpose == "task_reader":
raise HTTPException(403, "Read-only task agents cannot start calculations")
required = {tool.permission}
if tool.name in SCENARIO_TOOLS:
required.add("scada.view")
if tool.name in {"run_burst_detection", "run_burst_location", "run_leakage_identification"}:
required.add("simulation.view" if getattr(args, "data_source", "monitoring") == "simulation" else "scada.view")
if not required.issubset(permissions) or not required.issubset(binding.permission_cap):
raise HTTPException(403, "Calculation and input data permissions are required")
permission = (
"scada.view"
if tool.name == "get_network_schema" and args.kind == "scada-device"
else tool.permission
)
if tool.task_only and binding.task_id is None:
raise HTTPException(403, "This query requires a bound task")
granted = permission in permissions and permission in binding.permission_cap
source = {
"list_scada_devices": "scada_devices",
"get_scada_device": "scada_devices",
"query_scada_readings": "scada_readings",
"query_element_history": "scada_readings",
"get_workbench_runtime": "workbench_runtime",
}.get(tool.name)
if tool.name == "query_peripheral_data":
source = args.source
if not granted and binding.task_id and source in binding.task_scope_cap:
async with scope.connection.transaction():
granted = await task_service(scope).can_read_source(binding.task_id, source)
if not granted:
raise HTTPException(403, "Data source permission is required")
if (
tool.name == "query_element_history"
and args.mode != "observed"
and (
"simulation.view" not in permissions
or "simulation.view" not in binding.permission_cap
)
):
raise HTTPException(403, "Model comparison permission is required")
async def execute_query(
scope: QueryScope,
binding: BoundContext,
tool: QueryTool,
args: BaseModel,
*,
full_snapshot: bool = False,
) -> QueryResult:
await authorize_query(scope, binding, tool, args)
name = tool.name
conn = scope.connection
page = None
truncated = False
units = {}
origin = "system_data"
gaps = []
source_ref = "mcp:" + str(uuid4())
evidence_id = None
if tool.computation:
from app.mcp.analysis_service import TIMESCALE_CALCULATIONS, calculate
# Context/receipt reads may have started an implicit transaction. Release
# it before potentially long CPU/IO work; evidence uses a new transaction.
await conn.commit()
async with (
scope.simulation_routing()
if name in TIMESCALE_CALCULATIONS
else nullcontext()
):
data = await calculate(
name, args, network=scope.project.project_code,
username=scope.user.username,
)
data = {
"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
"result": data,
}
gaps = [{
"run_simulation": "Online hydraulic recalculation updated the realtime baseline; results cover requested elements only, not observed SCADA data. No standalone analysis run was saved.",
"run_contaminant_simulation": "Forward model simulation for an assumed source, not source inversion or observed contamination; only requested elements are returned.",
"analyze_valve_closure": "Simulated hydraulic impact for requested elements only; no physical valves were operated.",
"analyze_flushing": "Simulated flushing hydraulics only; no field action or water-quality clearance was verified.",
"analyze_pressure_regulation": "Evaluation of a supplied pump schedule, not automatic optimization or device control; requested elements only.",
"optimize_sensor_placement": "Candidate pressure monitoring locations, not a saved deployment scheme or a guarantee of contaminant source observability.",
"run_leakage_identification": "DMA leakage estimates depend on the total leakage constraint and aligned model/SCADA time; they do not confirm an exact leak location.",
"run_burst_detection": "Pressure anomalies alone do not confirm a burst or locate a pipe.",
"run_burst_location": "Candidate ranking depends on the leakage assumption, model and sensor coverage; field confirmation is required.",
"analyze_valve_isolation": "Topology-only isolation advice; valve operability, hydraulic impact and field actions are not verified or executed.",
}[name]]
if getattr(args, "data_source", None) == "simulation":
gaps.append("Input is an existing analysis run and model baseline, not observed SCADA data.")
elif name == "query_nearby_elements":
rows = await nearby_elements(
conn, args.element_type, center=args.center, radius_m=args.radius_m,
limit=args.limit + 1, offset=args.offset,
)
items, page, truncated = page_result(rows, args)
truncated = truncated or args.offset > 0
data = {
"source": "model_network", "element_type": args.element_type,
"center": list(args.center), "center_crs": "EPSG:4326",
"radius_m": args.radius_m, "distance_method": "WGS84_spheroid",
"items": items,
}
units = {"distance_m": "m", "radius_m": "m"}
gaps = ["Spatial proximity does not establish hydraulic connectivity or a supply relationship"]
elif name in {"get_element_properties", "list_elements"}:
rows = await reads.asset_rows(
conn,
args.element_type,
element_id=getattr(args, "element_id", None),
limit=getattr(args, "limit", 1) + 1,
offset=getattr(args, "offset", 0),
)
if name == "get_element_properties":
if not rows:
raise HTTPException(404, "Element was not found")
data = rows[0]
else:
data, page, truncated = page_result(rows, args)
elif name in {"list_scada_devices", "get_scada_device"}:
rows = await reads.scada_devices(
conn,
device_id=getattr(args, "device_id", None),
limit=getattr(args, "limit", 1) + 1,
offset=getattr(args, "offset", 0),
)
if name == "get_scada_device":
if not rows:
raise HTTPException(404, "SCADA device was not found")
data = rows[0]
else:
data, page, truncated = page_result(rows, args)
elif name == "get_network_schema":
from app.services import tjnetwork
mapping = {
"option": "get_option_v3_schema",
"scada-device": "get_scada_info_schema",
"pump-energy": "get_pump_energy_schema",
}
function = getattr(
tjnetwork, mapping.get(args.kind, "get_" + args.kind + "_schema")
)
data = await asyncio.to_thread(function, scope.project.project_code)
elif name == "get_network_options":
if args.kind == "pump-energy":
rows = await reads.read_rows(
conn,
"SELECT price,pattern_id AS pattern,efficiency_curve_id AS effic FROM network.pump_energy_settings WHERE pump_id=%s",
(args.pump_id,),
)
data = {"pump": args.pump_id, "price": None, "pattern": None, "effic": None}
if rows:
data.update(rows[0])
else:
table = {
"time": "time_settings",
"energy": "energy_settings",
"network": "simulation_settings",
}[args.kind]
where = " WHERE engine_version='v3'" if args.kind == "network" else ""
rows = await reads.read_rows(
conn,
"SELECT key,value FROM network."
+ table
+ where
+ " ORDER BY key LIMIT 501",
)
data = {str(row["key"]): str(row["value"]) for row in rows[:500]}
truncated = len(rows) > 500
elif name in {
"query_scada_readings",
"query_model_baseline_series",
"get_model_baseline_at_time",
"query_model_baseline_field",
"query_analysis_series",
"query_analysis_values",
}:
source = (
"scada"
if name == "query_scada_readings"
else "analysis"
if name.startswith("query_analysis")
else "realtime"
)
ids = (
getattr(args, "device_ids", None)
or getattr(args, "element_ids", None)
or ([args.element_id] if hasattr(args, "element_id") else [])
)
async with scope.timescale() as ts:
rows = await reads.series_rows(
ts,
source=source,
element_type=getattr(args, "element_type", "junction"),
ids=ids,
start=getattr(args, "start", None),
end=getattr(args, "end", None),
time=getattr(args, "time", None),
field=getattr(args, "field", None),
run_id=getattr(args, "run_id", None),
limit=getattr(args, "limit", 1) + 1,
offset=getattr(args, "offset", 0),
)
if name == "get_model_baseline_at_time":
data = rows[:1]
else:
data, page, truncated = page_result(rows, args)
if source != "scada":
async with conn.transaction():
units = await NetworkSettingsRepository.get_result_units(conn)
if source == "analysis":
run = await AnalysisRepository.get_run(conn, args.run_id)
if not run:
raise HTTPException(404, "Analysis run was not found")
recorded = (run.get("parameters") or {}).get("result_units")
if isinstance(recorded, dict):
units = recorded
else:
gaps.append(
"Analysis units inferred from current network settings"
)
if hasattr(args, "field") and args.field not in units:
gaps.append(f"Unit metadata is unavailable for field {args.field}")
else:
metadata = [
r
for device in ids
for r in await reads.scada_devices(conn, device_id=device, limit=1)
]
units = {
r["device_id"]: r["measurement_unit"]
for r in metadata
if r["device_id"] in ids
}
elif name in {"list_analysis_runs", "get_analysis_run", "list_analysis_results"}:
async with conn.transaction():
if name == "get_analysis_run":
data = await AnalysisRepository.get_run(conn, args.run_id)
if not data:
raise HTTPException(404, "Analysis run was not found")
elif name == "list_analysis_runs":
# Fetch one additional row with the repository's supported summary-page bound.
rows = await AnalysisRepository.list_run_summaries(
conn, limit=args.limit + 1, offset=args.offset
)
data, page, truncated = page_result(rows, args)
else:
rows = await AnalysisRepository.list_results(
conn,
args.run_id,
args.result_type,
limit=args.limit,
lookahead=True,
offset=args.offset,
)
data, page, truncated = page_result(rows, args)
elif name in {"list_sensor_placement_runs", "get_sensor_placement_run"}:
rows = await reads.sensor_runs(
conn,
run_id=getattr(args, "run_id", None),
limit=getattr(args, "limit", 1) + 1,
offset=getattr(args, "offset", 0),
)
if name == "get_sensor_placement_run":
if not rows:
raise HTTPException(404, "Sensor placement run was not found")
data = rows[0]
else:
data, page, truncated = page_result(rows, args)
elif name == "query_element_history":
from app.domain.schemas.timeseries_history import ElementHistoryQuery
from app.services.timeseries_history import TimeseriesHistoryService
query = ElementHistoryQuery(
start_time=args.start,
end_time=args.end,
mode=args.mode,
run_id=args.run_id,
elements=[e.model_dump() for e in args.elements],
)
async with scope.timescale() as ts, conn.transaction(), ts.transaction():
result = await TimeseriesHistoryService.query(
ts, conn, query, row_limit=args.limit, row_offset=args.offset
)
data = result.model_dump(mode="json")
truncated = result.truncated
page = PageInfo(limit=args.limit, offset=args.offset, has_more=truncated)
truncated = truncated or args.offset > 0
elif name in {"get_element_gis_properties", "get_source_gis_properties"}:
async with conn.transaction():
if name == "get_element_gis_properties":
data = await GisAttributeRepository.list_element_groups(
conn, args.element_type, args.element_id
)
else:
data = await GisAttributeRepository.get_source_feature_group(
conn, args.layer_key, args.source_fid
)
if data is None:
raise HTTPException(404, "Source GIS feature was not found")
elif name == "get_task_context":
if args.section in {"summary", "skill"}:
data = await reads.task_context_summary(conn, project_id=binding.project_id,
task_id=binding.task_id, report_revision=args.report_revision)
skill = read_work_order_skill(
data.get("task", {}).get("workflow"), include_content=args.section == "skill"
)
if args.section == "skill":
data = {"section": "skill", "skill": skill}
else:
data["handling_skill"] = skill
data["display_time_zone"] = "Asia/Shanghai"
data["approved_data_scopes"] = binding.task_scope_cap
data["bound_run_id"] = str(binding.run_id) if binding.run_id else None
data["available_sections"] = ["task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill"]
truncated = True
gaps.append("Task context is a bounded summary; use section, data_path, limit and offset for detail. Previews are not complete reports.")
else:
projection = await reads.task_context_page(conn, project_id=binding.project_id, task_id=binding.task_id,
section=args.section, report_revision=args.report_revision, data_path=args.data_path, limit=args.limit, offset=args.offset)
total = projection["total"]
if args.section in {"evidence", "actions", "access_requests"} and not args.data_path:
total = projection["snapshot_metadata"].get("total", total)
if projection["value_type"] in {"array", "object", "string"}:
page = PageInfo(limit=args.limit, offset=args.offset, total=total, has_more=args.offset + args.limit < total)
elif args.offset:
raise HTTPException(422, "Scalar task detail cannot use an offset")
truncated = bool(projection.get("nested") or args.offset or page and page.has_more)
data = {"section": args.section, "selected_path": args.data_path, "value": projection["value"], "metadata": projection["snapshot_metadata"]}
if projection.get("nested"):
gaps.append("Nested values have data_path descriptors; select a descriptor path for its values")
elif name == "get_task_evidence":
projection = await reads.evidence_page(
conn, project_id=scope.project.project_id, task_id=binding.task_id,
evidence_id=args.evidence_id, data_path=args.data_path,
limit=args.limit, offset=args.offset, mcp_sources=list(TOOLS),
full_snapshot=full_snapshot,
)
document = projection if full_snapshot else projection["snapshot_metadata"]
origin = document["origin"]
source_ref = document["source_ref"]
evidence_id = args.evidence_id
truncated = document.get("truncated", False)
if full_snapshot:
data = document
else:
selected = projection["value"]
if projection["value_type"] in {"array", "object", "string"}:
total = projection["total"]
page = PageInfo(
limit=args.limit,
offset=args.offset,
total=total,
has_more=args.offset + args.limit < total,
)
truncated = truncated or page.has_more or args.offset > 0
elif args.offset:
raise HTTPException(422, "Scalar evidence cannot use an offset")
if projection.get("nested"):
truncated = True
gaps.append(
"Nested collections have data_path descriptors; query a descriptor path for their values"
)
data = {
"snapshot_metadata": document,
"query_metadata": projection["query_metadata"],
"selected_path": args.data_path,
"value": selected,
}
elif name == "get_workbench_runtime":
from app.services.workbench_runtime import read_workbench_runtime
data = (await read_workbench_runtime(scope.project, conn)).model_dump(
mode="json"
)
elif name == "query_peripheral_data":
from app.mcp.peripheral import read_peripheral_query
snapshot = await read_peripheral_query(scope, args)
data = snapshot.data
origin = snapshot.origin
truncated = snapshot.truncated
else:
raise ValueError("query handler is not registered")
window = (
{"start": args.start, "end": args.end}
if hasattr(args, "start") and args.start is not None
else None
)
data, nonfinite = finite_data(jsonable_encoder(data))
semantics = (
"comparison"
if name == "query_element_history" and args.mode != "observed"
else tool.semantics
)
return QueryResult(
source=name,
source_ref=source_ref,
evidence_id=evidence_id,
queried_at=datetime.now(timezone.utc),
semantics=semantics,
data=data,
data_gaps=gaps
+ (
[f"{nonfinite} non-finite numeric values were returned as null"]
if nonfinite
else []
),
page=page,
window=window,
units=units,
truncated=truncated,
origin=origin,
)
+70
View File
@@ -0,0 +1,70 @@
"""MCP online recalculation and isolated scenario adapters."""
from typing import Any
SCENARIO_TOOLS = frozenset({
"run_simulation",
"run_contaminant_simulation", "analyze_valve_closure",
"analyze_flushing", "analyze_pressure_regulation",
})
def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, username: str) -> dict[str, Any]:
from app.services import simulation_scenarios as scenarios
parameters = dict(parameters)
nodes = parameters.pop("output_node_ids")
links = parameters.pop("output_link_ids")
start = parameters.pop("start_time")
duration = parameters.pop("duration")
common = dict(name=network, modify_pattern_start_time=start.isoformat(),
modify_total_duration=duration, persist_results=False, scheme_name=None)
if name == "run_simulation":
from app.services.simulation import run_online_simulation
result = run_online_simulation(
network, start, duration, output_node_ids=nodes, output_link_ids=links,
)
elif name == "run_contaminant_simulation":
result = scenarios.contaminant_simulation(**common, username=username,
source=parameters["source"], concentration=parameters["concentration"], source_pattern=parameters.get("pattern"))
elif name == "analyze_valve_closure":
result = scenarios.valve_close_analysis(**common,
modify_valve_opening={valve: 0.0 for valve in parameters["valves"]})
elif name == "analyze_flushing":
# The service consumes m³/h (the older REST label incorrectly says L/s).
# Reject unsupported conversion rather than silently ignore the flow.
from app.native.wndb.model.options import get_option
if get_option(network).get("UNITS") not in {"LPS", "CMH"}:
raise ValueError("Flushing currently supports LPS or CMH models only")
result = scenarios.flushing_analysis(**common, username=username,
drainage_node_ID=parameters["drainage_node_id"], flushing_flow=parameters["flushing_flow_m3h"],
modify_valve_opening={valve: 0.0 for valve in parameters["closed_valves"]})
elif name == "analyze_pressure_regulation":
from app.services.simulation import query_corresponding_element_id_and_query_id
mappings = query_corresponding_element_id_and_query_id(network)
controls = parameters["pump_control"]
if set(controls) - (set(mappings.fixed_pumps) | set(mappings.variable_pumps)):
raise ValueError("Pump controls must reference registered model pumps")
if any(value not in (0, 1) for key, values in controls.items() if key in mappings.fixed_pumps for value in values):
raise ValueError("Fixed-speed pump controls must be 0 or 1")
result = scenarios.pressure_regulation(**common,
modify_fixed_pump_pattern={key: value for key, value in controls.items() if key in mappings.fixed_pumps},
modify_variable_pump_pattern={key: value for key, value in controls.items() if key in mappings.variable_pumps},
modify_tank_initial_level=parameters["tank_init_level"], scada_mappings=mappings)
else:
raise ValueError("Scenario is not registered")
if not isinstance(result, dict):
raise ValueError("Scenario did not return calculation results")
selected_nodes = [row for row in result.get("node_results", []) if str(row.get("node")) in nodes]
selected_links = [row for row in result.get("link_results", []) if str(row.get("link")) in links]
return {
"input_source": "online_model_simulation" if name == "run_simulation" else "isolated_model_simulation",
"persisted_baseline": name == "run_simulation", "persisted_scheme": False,
"start_time": start.isoformat(), "duration_seconds": duration,
"units": result.get("units", {}), "times": result.get("times", {}),
"node_results": selected_nodes, "link_results": selected_links,
"coverage": {"requested_nodes": nodes, "requested_links": links,
"missing_nodes": sorted(set(nodes) - {str(row["node"]) for row in selected_nodes}),
"missing_links": sorted(set(links) - {str(row["link"]) for row in selected_links})},
}
+76
View File
@@ -0,0 +1,76 @@
"""Resolve authoritative metadata and project routing for each MCP request."""
from contextlib import asynccontextmanager
from dataclasses import dataclass
from typing import Any, AsyncIterator
from psycopg import AsyncConnection
from app.auth.project_dependencies import ProjectContext
from app.infra.db.project_routing import ActiveProjectRouting
from fastapi import HTTPException
from mcp.server.auth.provider import AccessToken
from app.auth.metadata_dependencies import get_current_metadata_user
from app.auth.project_dependencies import (
resolve_project_context,
resolve_project_business_routing,
resolve_project_simulation_routing,
get_project_pg_connection,
get_project_timescale_connection,
)
from app.infra.db.metadb.database import get_metadata_session
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
from app.infra.db.project_routing import activate_project_routing
class QueryMetadataRepository(MetadataRepository):
async def get_project_db_routing(self, project_id, role):
routing = await super().get_project_db_routing(project_id, role)
# Existing connection generators obtain routing before awaiting a pool.
# End that implicit SELECT transaction here, before the network wait.
await self.session.commit()
return routing
@dataclass
class QueryScope:
project: ProjectContext
user: Any
connection: AsyncConnection
routing: ActiveProjectRouting
metadata: QueryMetadataRepository
@asynccontextmanager
async def simulation_routing(self) -> AsyncIterator[ActiveProjectRouting]:
"""Resolve both databases before a native calculation can modify its model."""
routing = await resolve_project_simulation_routing(self.project, self.metadata)
await self.metadata.session.commit()
with activate_project_routing(routing):
yield routing
@asynccontextmanager
async def timescale(self):
# Metadata repository here is used only to resolve a known project DB.
async with asynccontextmanager(get_project_timescale_connection)(
self.project, self.metadata
) as conn:
await self.metadata.session.commit()
yield conn
@asynccontextmanager
async def project_scope(principal: AccessToken, project_id: str):
if not principal.claims:
raise HTTPException(401, "MCP identity is missing")
async with asynccontextmanager(get_metadata_session)() as session:
metadata = QueryMetadataRepository(session)
user = await get_current_metadata_user(principal.claims, metadata)
project = await resolve_project_context(project_id, user, metadata)
routing = await resolve_project_business_routing(project, metadata)
# End the metadata read before any external query/IO waits.
await session.commit()
async with asynccontextmanager(get_project_pg_connection)(
project, metadata
) as conn:
await session.commit()
with activate_project_routing(routing):
yield QueryScope(project, user, conn, routing, metadata)
+40
View File
@@ -0,0 +1,40 @@
"""Server-owned business instructions, selected from the persisted task workflow."""
from hashlib import sha256
from pathlib import Path
from typing import Any
import yaml
SKILL_ROOT = Path(__file__).resolve().parents[2] / "resources" / "work-order-skills"
WORKFLOW_SKILLS = {
"burst_report": "burst-report",
"water_quality_complaint": "water-quality-complaint",
"general": "appointment-follow-up",
}
def read_work_order_skill(workflow: str | None, *, include_content: bool = False) -> dict[str, Any]:
"""Read only packaged, allowlisted instructions; never interpret a task path."""
name = WORKFLOW_SKILLS.get(workflow)
if name is None:
return {"available": False, "workflow": workflow, "reason": "No business skill is configured for this workflow"}
content = (SKILL_ROOT / name / "SKILL.md").read_text(encoding="utf-8")
_, frontmatter, body = content.split("---", 2)
header = yaml.safe_load(frontmatter)
if header["name"] != name:
raise ValueError("Packaged work-order skill name does not match its directory")
result = {
"available": True,
"kind": "business_instructions",
"workflow": workflow,
"name": header["name"],
"description": header["description"],
"version": str(header["metadata"]["version"]),
"content_sha256": sha256(content.encode("utf-8")).hexdigest(),
"read": {"section": "skill"},
}
if include_content:
result["instructions"] = body.strip()
return result
+2 -2
View File
@@ -120,7 +120,7 @@ def dump_inp(project: str, inp: str, version: str = '3'):
file.write('\n'.join(inp_out_demand(project)))
elif name == STATUS:
file.write('\n'.join(inp_out_status(project)))
file.write('\n'.join(inp_out_status(project, version=version)))
elif name == PATTERNS:
if version == '3':
@@ -232,7 +232,7 @@ def export_inp(project: str, version: str = '3') -> ChangeSet:
inp += '\n'.join(inp_out_demand(project))
elif name == STATUS:
inp += '\n'.join(inp_out_status(project))
inp += '\n'.join(inp_out_status(project, version=version))
elif name == PATTERNS:
if version == '3':
+4 -2
View File
@@ -93,14 +93,16 @@ def inp_in_status(line: str) -> str:
return str(f"insert into network.link_initial_settings (link_id, status, setting) values ({sql_literal(link)}, null, {sql_literal(float(value))});")
def inp_out_status(name: str) -> list[str]:
def inp_out_status(name: str, version: str = '2') -> list[str]:
lines = []
objs = read_all(name, 'select link_id as link, status, setting from network.link_initial_settings order by link_id')
for obj in objs:
link = obj['link']
status = obj['status'] if obj['status'] != None else ''
setting = obj['setting'] if obj['setting'] != None else ''
if status != '':
# EPANET 2 valves are active at their configured setting by default.
# ACTIVE is an internal state, not a legal [STATUS] keyword in v2.
if status != '' and not (version == '2' and status.upper() == LINK_STATUS_ACTIVE):
lines.append(f'{link} {status}')
if setting != '':
lines.append(f'{link} {setting}')
+186
View File
@@ -0,0 +1,186 @@
"""Messages never change task generation or silently target a replacement run."""
from datetime import datetime, timezone
from uuid import UUID, uuid4
from app.auth.permissions import OPERATIONAL_TASK_EXECUTE, permissions_for_context
from app.domain.domain_errors import DomainError, require
from app.domain.schemas.agent_messages import (
AgentMessage, AgentMessageCreate, AgentMessageDelivery, AgentMessagePage, AgentMessageReply,
)
def _message_cursor(cursor: str | None):
if not cursor:
return None, None
try:
stamp, identifier = cursor.rsplit("|", 1)
cursor_at, cursor_id = datetime.fromisoformat(stamp), UUID(identifier)
require(cursor_at.tzinfo is not None, "INVALID_CURSOR", "Cursor needs a timezone", status=400)
return cursor_at, cursor_id
except (ValueError, TypeError) as exc:
raise DomainError("INVALID_CURSOR", "Invalid message cursor", status=400) from exc
class AgentMessageService:
def __init__(self, repository, context, *, clock=None):
self.repo = repository
self.context = context
self.clock = clock or (lambda: datetime.now(timezone.utc))
def _require_service(self) -> None:
require(self.context.system_role == "service" and
OPERATIONAL_TASK_EXECUTE in permissions_for_context(self.context), "SERVICE_REQUIRED",
"Message dispatch requires a project service account", status=403)
async def get(self, message_id: UUID) -> AgentMessage:
if self.context.system_role == "service":
self._require_service()
row = await self.repo.get_message(str(message_id))
require(row is not None, "MESSAGE_NOT_FOUND", "Message not found", status=404)
require(self.context.system_role == "service" or
str(row["conversation_owner_id"]) == str(self.context.user_id),
"MESSAGE_NOT_FOUND", "Message not found", status=404)
return AgentMessage.model_validate(row)
async def _active_run(self, task_id: UUID, run_id: UUID) -> dict:
run = await self.repo.get_run(str(run_id))
require(run is not None and str(run["task_id"]) == str(task_id),
"RUN_NOT_FOUND", "Run does not belong to the task", status=404)
task = await self.repo.get_task(str(task_id))
require(task is not None and task["task_type"] == "event_investigation",
"INVALID_TASK_TYPE", "Messages require an investigation task", status=409)
require(run["status"] in {"queued", "claimed", "running"},
"RUN_TERMINAL", "Run no longer accepts messages", status=409)
if run["status"] != "queued":
expires = run.get("lease_expires_at")
if isinstance(expires, str):
expires = datetime.fromisoformat(expires.replace("Z", "+00:00"))
require(expires is not None and expires > self.clock(),
"LEASE_EXPIRED", "Run lease expired", status=409)
return run
async def _insert(self, *, command, owner_id, conversation_owner_id, direction, task_id, run_id,
target_session_id, in_reply_to=None) -> AgentMessage:
prior = await self.repo.get_message_request(
str(owner_id), command.source_session_id, str(command.request_id),
)
expected = {
"task_id": str(task_id), "run_id": str(run_id),
"direction": direction, "in_reply_to": str(in_reply_to) if in_reply_to else None,
"text": command.text,
}
if prior is not None:
require(all((str(prior[key]) if prior[key] is not None else None) == value
for key, value in expected.items()),
"IDEMPOTENCY_CONFLICT", "request_id was used with different content", status=409)
return AgentMessage.model_validate(prior)
# The run lock serializes message creation with completion/cancellation.
await self._active_run(task_id, run_id)
message_id = uuid4()
now = self.clock()
message = AgentMessage(
project_id=self.context.project_id,
message_id=message_id, request_id=command.request_id,
task_id=task_id, run_id=run_id, source_session_id=command.source_session_id,
source_owner_id=owner_id, conversation_owner_id=conversation_owner_id,
target_session_id=target_session_id,
direction=direction, in_reply_to=in_reply_to, text=command.text,
status="pending", native_message_id=f"msg_{message_id}",
created_at=now, updated_at=now,
)
await self.repo.insert_message(message.model_dump(mode="json"))
return message
async def create(self, command: AgentMessageCreate) -> AgentMessage:
async with self.repo.transaction():
await self.repo.lock(f"run:{command.run_id}")
run = await self.repo.get_run(str(command.run_id))
return await self._insert(
command=command, owner_id=self.context.user_id,
conversation_owner_id=self.context.user_id, direction="to_task",
task_id=command.task_id, run_id=command.run_id,
target_session_id=run.get("agent_session_id") if run else None,
)
async def reply(self, message_id: UUID, command: AgentMessageReply) -> AgentMessage:
async with self.repo.transaction():
original = await self.get(message_id)
await self.repo.lock(f"run:{original.run_id}")
run = await self.repo.get_run(str(original.run_id))
if original.direction == "to_task":
self._require_service()
require(run is not None and run.get("agent_session_id") == command.source_session_id,
"SESSION_MISMATCH", "Session does not own the message run", status=409)
direction, target = "to_chat", original.source_session_id
else:
require(original.conversation_owner_id == self.context.user_id,
"MESSAGE_NOT_FOUND", "Message not found", status=404)
require(command.source_session_id == original.target_session_id,
"SESSION_MISMATCH", "Only the original chat can reply", status=409)
direction = "to_task"
target = run.get("agent_session_id") if run else None
return await self._insert(
command=command, owner_id=self.context.user_id,
conversation_owner_id=original.conversation_owner_id, direction=direction,
task_id=original.task_id, run_id=original.run_id,
target_session_id=target, in_reply_to=message_id,
)
async def list_session(
self, session_id: str, limit: int, cursor: str | None, pending_only: bool = False,
) -> AgentMessagePage:
cursor_at, cursor_id = _message_cursor(cursor)
kwargs = {"cursor_at": cursor_at, "cursor_id": cursor_id, "pending_only": pending_only}
if self.context.system_role == "service":
self._require_service()
rows = await self.repo.list_run_session_messages(session_id, limit + 1, **kwargs)
if not rows:
rows = await self.repo.list_session_messages(str(self.context.user_id), session_id, limit + 1, **kwargs)
else:
rows = await self.repo.list_session_messages(str(self.context.user_id), session_id, limit + 1, **kwargs)
items = [AgentMessage.model_validate(row) for row in rows[:limit]]
next_cursor = f"{items[-1].created_at.isoformat()}|{items[-1].message_id}" if len(rows) > limit else None
return AgentMessagePage(items=items, next_cursor=next_cursor)
async def pending(self, limit: int, cursor: str | None = None) -> AgentMessagePage:
self._require_service()
cursor_at, cursor_id = _message_cursor(cursor)
rows = await self.repo.list_pending_messages(limit + 1, cursor_at=cursor_at, cursor_id=cursor_id)
items = [AgentMessage.model_validate(row) for row in rows[:limit]]
next_cursor = f"{items[-1].created_at.isoformat()}|{items[-1].message_id}" if len(rows) > limit else None
return AgentMessagePage(items=items, next_cursor=next_cursor)
async def delivery(self, message_id: UUID, command: AgentMessageDelivery) -> AgentMessage:
async with self.repo.transaction():
message = await self.get(message_id)
if self.context.system_role == "service":
self._require_service()
if self.context.system_role != "service":
require(message.direction == "to_chat",
"SERVICE_REQUIRED", "Only the receiver can acknowledge a chat reply", status=403)
await self.repo.lock(f"run:{message.run_id}")
await self.repo.lock(f"message:{message_id}")
message = await self.get(message_id)
if message.status in {"delivered", "failed"}:
require(message.status == command.status, "MESSAGE_TERMINAL",
"Message already has a terminal delivery result", status=409)
return message
if message.direction == "to_task" and command.status != "failed":
run = await self._active_run(message.task_id, message.run_id)
target = run.get("agent_session_id")
require(target is not None and command.target_session_id == target,
"SESSION_MISMATCH", "Delivery must target the owning run session", status=409)
else:
target = message.target_session_id
require(command.target_session_id in {None, target}, "SESSION_MISMATCH",
"Delivery cannot change the reply channel", status=409)
require(command.native_message_id in {None, message.native_message_id},
"MESSAGE_ID_MISMATCH", "Native message identity is immutable", status=409)
now = self.clock()
update = {
"status": command.status, "target_session_id": target,
"failure_detail": command.failure_detail, "updated_at": now,
"delivered_at": now if command.status == "delivered" else None,
}
await self.repo.update_message_delivery(str(message_id), update)
return message.model_copy(update=update)
+2
View File
@@ -63,6 +63,7 @@ def run_burst_location_by_network(
use_scada_flow: bool = False,
scheme_name: str | None = None,
simulation_run_id: UUID | None = None,
final_candidates_csv_path: str | None = "temp/burst_location/final_round_candidates.csv",
) -> dict[str, Any]:
if not network:
raise ValueError("network is required.")
@@ -312,6 +313,7 @@ def run_burst_location_by_network(
min_dpressure=min_dpressure,
basic_pressure=basic_pressure,
visualize_partition=False,
final_candidates_csv_path=final_candidates_csv_path,
)
payload: dict[str, Any] = {
+18
View File
@@ -0,0 +1,18 @@
"""RFC 8785 canonical JSON hashing shared by persisted records."""
from hashlib import sha256
import rfc8785
from app.domain.domain_errors import DomainError
def canonical(value: object) -> bytes:
try:
return rfc8785.dumps(value)
except rfc8785.CanonicalizationError as exc:
raise DomainError("SCHEMA_INVALID", "Content is not canonicalizable JSON") from exc
def digest(value: object) -> str:
return "sha256:" + sha256(canonical(value)).hexdigest()
+6 -3
View File
@@ -5,6 +5,9 @@ import httpx
from pydantic import AliasChoices, BaseModel, Field
from app.core.config import settings
from app.core.tianditu_logging import install_tianditu_token_filter
install_tianditu_token_filter()
class TiandituGeocodeRequest(BaseModel):
@@ -32,12 +35,12 @@ async def geocode_tianditu(
*,
client: httpx.AsyncClient | None = None,
) -> dict[str, Any]:
if not settings.TIANDITU_GEOCODER_TOKEN:
raise TiandituGeocodingConfigError("TIANDITU_GEOCODER_TOKEN is not configured")
if not settings.TIANDITU_TOKEN:
raise TiandituGeocodingConfigError("TIANDITU_TOKEN is not configured")
params = {
"ds": json.dumps({"keyWord": request.keyword}, ensure_ascii=False),
"tk": settings.TIANDITU_GEOCODER_TOKEN,
"tk": settings.TIANDITU_TOKEN,
}
if client is not None:
+99
View File
@@ -0,0 +1,99 @@
"""Resolve browser overlays only from immutable Server-ingested snapshots."""
from __future__ import annotations
from math import isfinite
from app.domain.domain_errors import require
from app.domain.schemas.operational_data import TaskDataSnapshot
def _number(value) -> bool:
return isinstance(value, (int, float)) and not isinstance(value, bool) and isfinite(value)
def _position(value):
require(isinstance(value, list) and len(value) in {2, 3}
and all(_number(item) for item in value)
and -180 <= value[0] <= 180 and -90 <= value[1] <= 90,
"OVERLAY_GEOMETRY_INVALID", "Overlay coordinates must be finite WGS84 positions", status=422)
return value[:2]
def _geometry(value):
require(isinstance(value, dict), "OVERLAY_GEOMETRY_INVALID", "Overlay geometry is missing", status=422)
kind = value.get("type")
depths = {"Point": 0, "MultiPoint": 1, "LineString": 1,
"MultiLineString": 2, "Polygon": 2, "MultiPolygon": 3}
require(kind in depths, "OVERLAY_GEOMETRY_INVALID", "Unsupported overlay geometry", status=422)
positions = 0
def coordinates(items, depth):
nonlocal positions
if depth == 0:
positions += 1
require(positions <= 20000, "OVERLAY_TOO_LARGE", "Overlay geometry is too large", status=422)
return _position(items)
require(isinstance(items, list) and 0 < len(items) <= 20000,
"OVERLAY_GEOMETRY_INVALID", "Overlay coordinates are empty or too large", status=422)
return [coordinates(item, depth - 1) for item in items]
normalized = coordinates(value.get("coordinates"), depths[kind])
lines = ([normalized] if kind == "LineString" else normalized if kind == "MultiLineString" else [])
require(all(len(line) >= 2 for line in lines), "OVERLAY_GEOMETRY_INVALID",
"Overlay lines need at least two positions", status=422)
rings = (normalized if kind == "Polygon" else [ring for polygon in normalized for ring in polygon]
if kind == "MultiPolygon" else [])
require(all(len(ring) >= 4 and ring[0] == ring[-1] for ring in rings),
"OVERLAY_GEOMETRY_INVALID", "Overlay polygon rings must be closed", status=422)
return {"type": kind, "coordinates": normalized}
def resolve_overlay(snapshot: TaskDataSnapshot, mode: str) -> dict:
"""Use recorded GeoJSON or recorded coordinates; never geocode or infer a boundary."""
data = snapshot.data
candidates = [data]
if isinstance(data, dict):
candidates.extend([data.get("map_payload"), data.get("drawing_payload")])
latest = data.get("latest_analysis")
if isinstance(latest, dict):
candidates.append(latest.get("map_payload"))
collection = next((item for item in candidates
if isinstance(item, dict) and item.get("type") == "FeatureCollection"), None)
if collection is not None:
raw_features = collection.get("features")
require(isinstance(raw_features, list), "OVERLAY_GEOMETRY_INVALID",
"Snapshot FeatureCollection is invalid", status=422)
else:
rows = data.get("items") if isinstance(data, dict) else data
require(isinstance(rows, list), "OVERLAY_DATA_MISSING",
"Evidence snapshot has no recorded coordinates or GeoJSON", status=422)
raw_features = []
for row in rows:
if not isinstance(row, dict) or row.get("longitude") is None or row.get("latitude") is None:
continue
raw_features.append({
"type": "Feature", "geometry": {"type": "Point", "coordinates": [row["longitude"], row["latitude"]]},
"id": row.get("device_id") or row.get("point_id") or row.get("work_order_id") or row.get("id"),
"properties": {"weight": row.get("weight", 1)},
})
require(0 < len(raw_features) <= 1000, "OVERLAY_DATA_MISSING",
"Overlay needs between one and 1000 recorded features", status=422)
features = []
for feature in raw_features:
require(isinstance(feature, dict) and feature.get("type") == "Feature",
"OVERLAY_GEOMETRY_INVALID", "Snapshot contains an invalid feature", status=422)
geometry = _geometry(feature.get("geometry"))
require(mode != "heatmap" or geometry["type"] == "Point", "OVERLAY_HEATMAP_POINTS_REQUIRED",
"Heatmaps require recorded points, not inferred polygon centroids", status=422)
properties = feature.get("properties") or {}
require(isinstance(properties, dict), "OVERLAY_GEOMETRY_INVALID", "Feature properties are invalid", status=422)
normalized = {"type": "Feature", "geometry": geometry, "properties": dict(properties)}
if isinstance(feature.get("id"), (str, int)) and not isinstance(feature.get("id"), bool):
normalized["id"] = feature["id"]
if mode == "heatmap":
weight = properties.get("weight", 1)
require(_number(weight) and weight >= 0, "OVERLAY_WEIGHT_INVALID",
"Heatmap weight must be a finite nonnegative number", status=422)
normalized["properties"]["weight"] = weight
features.append(normalized)
return {"type": "FeatureCollection", "features": features}
File diff suppressed because it is too large Load Diff
+115
View File
@@ -0,0 +1,115 @@
import json
from typing import Any
import httpx
from pydantic import ValidationError
from app.core.config import settings
from app.core.tianditu_logging import install_tianditu_token_filter
from app.domain.schemas.place_search import (
MAX_PLACE_SEARCH_RESULTS,
PlaceSearchItem,
PlaceSearchRequest,
PlaceSearchResponse,
)
FENGXIAN_ADMIN_CODE = "156310120"
install_tianditu_token_filter()
class PlaceSearchError(RuntimeError):
def __init__(self, status_code: int, detail: str):
super().__init__(detail)
self.status_code = status_code
self.detail = detail
async def search_tianditu_places(
request: PlaceSearchRequest,
*,
client: httpx.AsyncClient | None = None,
) -> PlaceSearchResponse:
token = settings.TIANDITU_TOKEN
if not token:
raise PlaceSearchError(503, "地名搜索服务未配置")
params = {
"postStr": json.dumps(
{
"keyWord": request.keyword,
"queryType": 12,
"specify": FENGXIAN_ADMIN_CODE,
"start": 0,
"count": MAX_PLACE_SEARCH_RESULTS,
"show": 2,
},
ensure_ascii=False,
),
"type": "query",
"tk": token,
}
try:
if client is not None:
response = await client.get(settings.TIANDITU_SEARCH_URL, params=params)
else:
async with httpx.AsyncClient(
timeout=settings.TIANDITU_SEARCH_TIMEOUT_SECONDS
) as managed_client:
response = await managed_client.get(
settings.TIANDITU_SEARCH_URL, params=params
)
response.raise_for_status()
except httpx.TimeoutException:
raise PlaceSearchError(504, "地名搜索超时,请稍后重试") from None
except httpx.HTTPError:
# Request URLs contain the provider key and must not reach the client.
raise PlaceSearchError(502, "天地图地名搜索服务暂不可用") from None
try:
return _parse_response(response.json())
except (ValueError, TypeError, KeyError, AttributeError, ValidationError):
raise PlaceSearchError(502, "天地图返回了无效的地名搜索结果") from None
def _parse_response(payload: dict[str, Any]) -> PlaceSearchResponse:
code = str(payload.get("status", {}).get("infocode", ""))
if code == "3001":
return PlaceSearchResponse(items=[], total=0)
if code != "1000":
raise PlaceSearchError(502, "天地图地名搜索请求失败,请稍后重试")
result_type = str(payload.get("resultType", ""))
if result_type not in {"1", "3"}:
raise PlaceSearchError(502, "未获得可定位的地名结果,请细化关键字")
raw_items = payload.get("pois", []) if result_type == "1" else payload.get("area", [])
if not isinstance(raw_items, list) or len(raw_items) > MAX_PLACE_SEARCH_RESULTS:
raise ValueError("Invalid search results")
items = []
for raw in raw_items:
admin_code = str(raw.get("provinceCode") or raw.get("adminCode") or "")
if admin_code and not admin_code.startswith(("156310", "310")):
raise ValueError("Result outside Shanghai")
county_code = str(raw.get("countyCode") or (raw.get("adminCode") if result_type == "3" else "") or "")
county = raw.get("county")
if county_code and not county_code.startswith((FENGXIAN_ADMIN_CODE, "310120")):
raise ValueError("Result outside Fengxian")
if county and county != "奉贤区" or not county_code and county != "奉贤区":
raise ValueError("Missing or mismatched district")
lon, lat = str(raw["lonlat"]).split(",")
name = raw["name"]
if not isinstance(name, str) or not name.strip():
raise ValueError("Missing place name")
name = name.strip()
category = raw.get("typeName") or ("公交站" if str(raw.get("poiType")) == "102" else None)
items.append(PlaceSearchItem(
id=str(raw.get("hotPointID") or raw.get("adminCode") or f"{name}:{lon},{lat}"),
name=name,
address=str(raw.get("address") or name),
longitude=float(lon), latitude=float(lat),
category="行政区" if result_type == "3" else category,
district="奉贤区",
))
total = int(payload.get("count", 0))
if total < len(items):
raise ValueError("Invalid search total")
return PlaceSearchResponse(
items=items, total=len(items), truncated=total > MAX_PLACE_SEARCH_RESULTS,
)
+52 -18
View File
@@ -1,9 +1,10 @@
from datetime import datetime
from datetime import datetime, timezone
from typing import Any
from uuid import UUID, uuid4
from app.infra.db.postgresql.analysis import AnalysisRepository
from app.native.wndb.core.connection import project_connection, project_transaction
from app.infra.db.project_routing import get_active_project_routing
from app.domain.time import parse_utc_time
@@ -40,16 +41,21 @@ def create_analysis_run(
started_at = parse_utc_time(scheme_start_time, field_name="scheme_start_time")
run_id = uuid4()
with project_connection(name) as conn:
AnalysisRepository.create_run_sync(
conn,
run_id=run_id,
name=scheme_name,
run_type=scheme_type,
created_by=username,
started_at=started_at,
status=status,
parameters=scheme_detail,
)
with conn.transaction():
AnalysisRepository.create_run_sync(
conn,
run_id=run_id,
name=scheme_name,
run_type=scheme_type,
created_by=username,
started_at=started_at,
status=status,
parameters=scheme_detail,
)
_append_run_event(
conn, run_id, revision=1, status=status,
occurred_at=started_at, results_committed=False,
)
return run_id
@@ -84,6 +90,10 @@ def store_analysis_run_with_result(
result_type=result_type,
payload=result_payload,
)
_append_run_event(
conn, run_id, revision=1, status="completed",
occurred_at=started_at, results_committed=True,
)
return run_id
@@ -96,13 +106,37 @@ def update_analysis_run(
scheme_detail: dict[str, Any],
) -> None:
with project_connection(name) as conn:
AnalysisRepository.update_run_sync(
conn,
run_id,
status=status,
created_by=username,
parameters=scheme_detail,
)
with conn.transaction():
revision = AnalysisRepository.update_run_sync(
conn,
run_id,
status=status,
created_by=username,
parameters=scheme_detail,
)
_append_run_event(
conn, run_id, revision=revision, status=status,
occurred_at=datetime.now(timezone.utc),
results_committed=status == "completed",
)
def _append_run_event(
conn, run_id: UUID, *, revision: int, status: str,
occurred_at: datetime, results_committed: bool,
) -> None:
routing = get_active_project_routing()
if routing is None or routing.project_id is None:
return
AnalysisRepository.append_run_changed_event_sync(
conn,
project_id=routing.project_id,
run_id=run_id,
revision=revision,
status=status,
occurred_at=occurred_at,
results_committed=results_committed,
)
def get_analysis_run(name: str, run_id: UUID) -> dict[str, Any]:
+22
View File
@@ -80,14 +80,26 @@ def _create_validated_placement(
min_diameter: int,
created_by: str,
sensor_locations: list[str],
sensor_type: str = "pressure",
method: str | None = None,
persist_results: bool = True,
) -> dict[str, Any]:
validate_sensor_placement_nodes(project_code, sensor_locations)
if not persist_results:
return {
"sensor_locations": sensor_locations,
"sensor_points": _sensor_points(project_code, sensor_locations),
"method": method,
"sensor_type": sensor_type,
}
return sensor_placement_repository.create_sensor_placement(
project_code,
run_name=run_name,
min_diameter=min_diameter,
created_by=created_by,
sensor_locations=sensor_locations,
sensor_type=sensor_type,
method=method,
)
@@ -97,6 +109,8 @@ def optimize_sensor_placement_by_sensitivity(
sensor_count: int,
min_diameter: int,
created_by: str,
sensor_type: str = "pressure",
persist_results: bool = True,
) -> dict[str, Any]:
"""Run sensitivity placement and persist the validated result."""
@@ -112,6 +126,9 @@ def optimize_sensor_placement_by_sensitivity(
min_diameter=min_diameter,
created_by=created_by,
sensor_locations=sensor_locations,
sensor_type=sensor_type,
method="sensitivity",
persist_results=persist_results,
)
@@ -121,6 +138,8 @@ def optimize_sensor_placement_by_kmeans(
sensor_count: int,
min_diameter: int,
created_by: str,
sensor_type: str = "pressure",
persist_results: bool = True,
) -> dict[str, Any]:
"""Export the model, run K-means placement, and persist the result."""
@@ -136,6 +155,9 @@ def optimize_sensor_placement_by_kmeans(
min_diameter=min_diameter,
created_by=created_by,
sensor_locations=sensor_locations,
sensor_type=sensor_type,
method="kmeans",
persist_results=persist_results,
)
+74 -2
View File
@@ -11,7 +11,7 @@ from app.native.wndb.model.tanks import get_tank, set_tank
from app.native.wndb.model.times import get_time, set_time
# from get_real_status import *
from datetime import datetime
from datetime import datetime, timedelta
from math import modf
import json
import pytz
@@ -150,6 +150,69 @@ def _apply_valve_control(
set_status(project_name, cs)
def run_online_simulation(
name: str,
start_time: datetime,
duration_seconds: int,
*,
output_node_ids: list[str] | None = None,
output_link_ids: list[str] | None = None,
) -> dict:
"""Recalculate online baseline snapshots without cloning a model database."""
if start_time.tzinfo is None or duration_seconds <= 0:
raise ValueError("Online simulation requires an aware start time and positive duration")
step = parse_clock_duration_seconds(
get_time(name)["HYDRAULIC TIMESTEP"], field_name="HYDRAULIC TIMESTEP"
)
if step <= 0:
raise ValueError("HYDRAULIC TIMESTEP must be greater than 0.")
mappings = query_corresponding_element_id_and_query_id(name)
selected = {
"node": {node: [] for node in output_node_ids or []},
"link": {link: [] for link in output_link_ids or []},
}
sample_times = []
units = {}
current = start_time
end = start_time + timedelta(seconds=duration_seconds)
while current < end:
output = run_simulation(
name=name,
simulation_type="realtime",
modify_pattern_start_time=current.isoformat(timespec="seconds"),
scada_mappings=mappings,
)
if not isinstance(output, dict):
raise RuntimeError("Online simulation did not return calculation results")
units = output.get("units", {})
sample_times.append(current.isoformat(timespec="seconds"))
for kind, elements in selected.items():
if not elements:
continue
rows = {
str(row[kind]): row.get("result", [])
for row in output.get(f"{kind}_results", []) if str(row[kind]) in elements
}
for element, values in elements.items():
samples = rows.get(element, [])
# Each realtime run is a single snapshot. Missing samples remain
# explicit so later values cannot shift to an earlier timestamp.
values.append(samples[0] if samples else None)
current += timedelta(seconds=step)
return {
"units": units,
"times": {
"report_start": 0, "report_step": step,
"sim_duration": duration_seconds, "num_periods": len(sample_times),
"sample_times": sample_times,
},
**{f"{kind}_results": [
{kind: element, "result": values}
for element, values in elements.items() if any(value is not None for value in values)
] for kind, elements in selected.items()},
}
# 2025/01/11
def run_simulation(
name: str,
@@ -170,7 +233,8 @@ def run_simulation(
scheme_username: str = "system",
scheme_detail: dict | None = None,
scada_mappings: ScadaElementMappings | None = None,
) -> UUID | None:
persist_results: bool = True,
) -> UUID | dict | None:
"""
传入需要修改的参数,改变数据库中对应位置的值,然后计算,返回结果
:param name: 模型名称,数据库中对应的名字
@@ -189,6 +253,10 @@ def run_simulation(
:param scheme_name:模拟方案名称
:return:
"""
if not persist_results and (
not name.startswith("tjw_tmp_") or simulation_type.upper() != "EXTENDED"
):
raise ValueError("Non-persistent simulation requires an isolated temporary model")
# 记录开始时间
time_cost_start = time.perf_counter()
@@ -514,6 +582,8 @@ def run_simulation(
num_periods_result = times_info.get("num_periods")
if num_periods_result is None:
raise RuntimeError("run_project output missing times.num_periods")
if not persist_results:
return output_data
print("simulation_type", simulation_type)
print("before store result")
# print(num_periods_result)
@@ -583,4 +653,6 @@ def run_simulation(
# TimescaleInternalQueries.fill_scheme_simulation_result_to_SCADA(scheme_type=scheme_type, scheme_name=scheme_name)
print("after store result")
if simulation_type.upper() == "REALTIME":
return output_data
return run_id if simulation_type.upper() == "EXTENDED" else None
+20 -8
View File
@@ -222,7 +222,8 @@ def valve_close_analysis(
modify_valve_opening: dict[str, float] = None,
scheme_name: str = None,
_temporary_project: str | None = None,
) -> None:
persist_results: bool = True,
) -> dict | str | None:
"""
关阀模拟
:param name: 模型名称,数据库中对应的名字
@@ -271,7 +272,7 @@ def valve_close_analysis(
set_option(new_name, cs_options)
# result = run_simulation_ex(new_name,'realtime', modify_pattern_start_time, modify_pattern_start_time, modify_total_duration,
# downloading_prohibition=True)
simulation.run_simulation(
result = simulation.run_simulation(
name=new_name,
simulation_type="extended",
modify_pattern_start_time=modify_pattern_start_time,
@@ -280,7 +281,9 @@ def valve_close_analysis(
scheme_type="valve_close_analysis",
scheme_name=scheme_name,
result_db_name=name,
persist_results=persist_results,
)
return result if not persist_results else None
# step 3. restore the base model
# for valve in valves:
# execute_undo(name)
@@ -303,7 +306,8 @@ def flushing_analysis(
username: str | None = None,
valve_control: dict[str, dict] = None,
_temporary_project: str | None = None,
) -> None:
persist_results: bool = True,
) -> dict | str | None:
"""
管道冲洗模拟
:param name: 模型名称,数据库中对应的名字
@@ -425,7 +429,7 @@ def flushing_analysis(
set_option(new_name, cs_options)
# result = run_simulation_ex(new_name,'realtime', modify_pattern_start_time, modify_pattern_start_time, modify_total_duration,
# downloading_prohibition=True)
simulation.run_simulation(
result = simulation.run_simulation(
name=new_name,
simulation_type="extended",
modify_pattern_start_time=modify_pattern_start_time,
@@ -437,7 +441,9 @@ def flushing_analysis(
result_db_name=name,
scheme_username=username,
scheme_detail=scheme_detail,
persist_results=persist_results,
)
return result if not persist_results else None
# step 4. restore the base model
# return result
@@ -457,7 +463,8 @@ def contaminant_simulation(
source_pattern: str = None, # 污染源时间变化模式名称
username: str | None = None,
_temporary_project: str | None = None,
) -> None:
persist_results: bool = True,
) -> dict | str | None:
"""
污染模拟
:param name: 模型名称,数据库中对应的名字
@@ -568,7 +575,7 @@ def contaminant_simulation(
# step 5. run simulation
# result = run_simulation_ex(new_name,'realtime', modify_pattern_start_time, modify_pattern_start_time, modify_total_duration,
# downloading_prohibition=True)
simulation.run_simulation(
result = simulation.run_simulation(
name=new_name,
simulation_type="extended",
modify_pattern_start_time=modify_pattern_start_time,
@@ -578,7 +585,9 @@ def contaminant_simulation(
result_db_name=name,
scheme_username=username,
scheme_detail=scheme_detail,
persist_results=persist_results,
)
return result if not persist_results else None
# for i in range(1,operation_step):
# execute_undo(name)
@@ -600,7 +609,8 @@ def pressure_regulation(
scheme_name: str = None,
scada_mappings: simulation.ScadaElementMappings | None = None,
_temporary_project: str | None = None,
) -> None:
persist_results: bool = True,
) -> dict | str | None:
"""
区域调压模拟,用来模拟未来15分钟内,开关水泵对区域压力的影响
:param name: 模型名称,数据库中对应的名字
@@ -645,7 +655,7 @@ def pressure_regulation(
# pump_control=pump_control,
# tank_initial_level_control=tank_initial_level_control,
# downloading_prohibition=True)
simulation.run_simulation(
result = simulation.run_simulation(
name=new_name,
simulation_type="extended",
modify_pattern_start_time=modify_pattern_start_time,
@@ -657,5 +667,7 @@ def pressure_regulation(
scheme_name=scheme_name,
result_db_name=name,
scada_mappings=scada_mappings,
persist_results=persist_results,
)
return result if not persist_results else None
# return result
+127
View File
@@ -0,0 +1,127 @@
"""Monotonic, idempotent state machine for task-produced alerts."""
from __future__ import annotations
from datetime import datetime, timezone
from app.domain.schemas.task_alert import TaskAlertEvent, TaskAlertState
from app.domain.domain_errors import require
from app.services.content_digest import digest
class TaskAlertService:
def __init__(
self,
repository,
context,
*,
clock=None,
internal_producer_id: str | None = None,
):
self.repo = repository
self.context = context
self.clock = clock or (lambda: datetime.now(timezone.utc))
self.internal_producer_id = internal_producer_id
async def push(self, command: TaskAlertEvent) -> TaskAlertState:
if self.internal_producer_id is None:
require(
self.context is not None and self.context.project_role == "member",
"PERMISSION_DENIED",
"Alert events require project write permission",
status=403,
)
require(
command.producer_id == f"principal:{self.context.user_id}",
"PERMISSION_DENIED",
"Alert producer must match the authenticated project principal",
status=403,
)
else:
require(
command.producer_id == self.internal_producer_id,
"PERMISSION_DENIED",
"Internal alert producer identity does not match",
status=403,
)
event = command.model_dump(mode="json")
async with self.repo.transaction():
await self.repo.lock_alert(command.alert_id)
prior_event = await self.repo.get_event(
command.producer_id, str(command.event_id)
)
if prior_event is not None:
prior_command = {
key: value for key, value in prior_event.items()
if key != "resulting_state"
}
require(
digest(prior_command) == digest(event),
"IDEMPOTENCY_CONFLICT",
"event_id was used with a different alert event",
status=409,
)
return TaskAlertState.model_validate(prior_event["resulting_state"])
current = await self.repo.get_alert(command.alert_id)
state = self._transition(command, current)
state_document = state.model_dump(mode="json")
await self.repo.save_event(event, state_document)
await self.repo.save_alert(state_document)
append_changed_event = getattr(self.repo, "append_changed_event", None)
if append_changed_event is not None:
await append_changed_event(state_document)
return state
def _transition(
self, command: TaskAlertEvent, current: tuple[int, dict] | None
) -> TaskAlertState:
now = self.clock()
require(now.tzinfo is not None, "SCHEMA_INVALID", "Alert clock must be timezone-aware")
if current is None:
require(command.event_type == "raised", "INVALID_TRANSITION",
"An alert must be raised before it can change state", status=409)
revision = 1
status = "active"
else:
revision, document = current
require(document.get("producer_id") == command.producer_id,
"PERMISSION_DENIED", "Only the originating producer can update an alert", status=403)
require(document.get("source") == command.source.model_dump(mode="json"),
"SCOPE_MISMATCH", "Alert source cannot change", status=409)
require(
all(
document.get(field) == event_value
for field, event_value in {
"condition_id": command.condition_id,
"title": command.title,
"description": command.description,
"occurred_at": command.model_dump(mode="json")["occurred_at"],
"severity": command.severity,
}.items()
),
"IDEMPOTENCY_CONFLICT",
"Alert facts cannot change after the alert is raised",
status=409,
)
allowed = {
("active", "acknowledged"): "acknowledged",
("active", "resolved"): "resolved",
("acknowledged", "resolved"): "resolved",
}
status = allowed.get((document["status"], command.event_type))
require(status is not None, "INVALID_TRANSITION",
"Alert state transition is not permitted", status=409)
revision += 1
return TaskAlertState(
alert_id=command.alert_id,
revision=revision,
producer_id=command.producer_id,
condition_id=command.condition_id,
source=command.source,
title=command.title,
description=command.description,
occurred_at=command.occurred_at,
severity=command.severity,
status=status,
updated_at=now,
)
+66 -1
View File
@@ -16,6 +16,7 @@ from app.domain.schemas.timeseries_history import (
from app.infra.db.postgresql.analysis import AnalysisRepository
from app.infra.db.postgresql.network_settings import NetworkSettingsRepository
from app.infra.db.postgresql.scada import ScadaInfoRepository
from app.infra.db.postgresql import read_queries
from app.infra.db.timescaledb.repositories.analysis import AnalysisResultsRepository
from app.infra.db.timescaledb.repositories.realtime import RealtimeRepository
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
@@ -56,13 +57,69 @@ def _points(
]
async def _query_page(timescale_conn, postgres_conn, query, selected_devices, limit, offset):
descriptors = []
series = []
def add(*, source, element_type, element_id, device_id=None, field, unit, unit_inferred=False):
descriptors.append({
"source": source, "element_type": element_type,
"id": device_id if source == "scada" else element_id, "field": field,
})
metric = _target_metric(element_type)
series.append(ElementHistorySeries(
element_id=element_id, element_type=element_type, device_id=device_id,
metric=metric, source=(
HistorySource.SCADA_RAW if field == "monitored_value" else HistorySource.SCADA_CLEANED
) if source == "scada" else (
HistorySource.ANALYSIS_SIMULATION if source == "analysis" else HistorySource.REALTIME_SIMULATION
),
source_unit=unit, display_unit=display_unit_for_metric(metric.value),
unit_inferred=unit_inferred, points=[],
))
# Stable series ordering makes a page crossing raw/cleaned/model boundaries repeatable.
for device_id, (device, element_type, element_id) in sorted(selected_devices.items()):
for field in ("monitored_value", "cleaned_value"):
add(source="scada", element_type=element_type, element_id=element_id,
device_id=device_id, field=field, unit=device["measurement_unit"])
if query.mode.value != "observed":
units = await NetworkSettingsRepository.get_result_units(postgres_conn)
source = "analysis" if query.mode.value == "analysis_comparison" else "realtime"
inferred = False
if source == "analysis":
run = await AnalysisRepository.get_run(postgres_conn, query.run_id)
if run is None:
raise ValueError(f"analysis run does not exist: {query.run_id}")
recorded = (run.get("parameters") or {}).get("result_units")
inferred = not isinstance(recorded, dict)
if not inferred:
units = recorded
targets = {(t.element_type, t.element_id) for t in query.elements}
for element_type, element_id in sorted(targets):
metric = _target_metric(element_type)
add(source=source, element_type=element_type, element_id=element_id,
field=metric.value, unit=str(units.get(metric.value, "")).strip(), unit_inferred=inferred)
rows = await read_queries.history_rows(
timescale_conn, descriptors, start=query.start_time, end=query.end_time,
run_id=query.run_id, limit=limit + 1, offset=offset,
)
for row in rows[:limit]:
series[row["series_index"]].points.append(HistoryPoint(time=row["time"], value=row["value"]))
return ElementHistoryResponse(series=[s for s in series if s.points], truncated=len(rows) > limit)
class TimeseriesHistoryService:
@staticmethod
async def query(
timescale_conn: AsyncConnection,
postgres_conn: AsyncConnection,
query: ElementHistoryQuery,
*, row_limit: int | None = None, row_offset: int = 0,
) -> ElementHistoryResponse:
if row_offset < 0 or (row_offset and row_limit is None) or (row_limit is not None and row_limit < 1):
raise ValueError("row_offset requires a bounded page")
pipe_ids = list(
dict.fromkeys(
target.element_id
@@ -78,8 +135,10 @@ class TimeseriesHistoryService:
)
)
devices = await ScadaInfoRepository.get_scadas_for_elements(
postgres_conn, junction_ids, pipe_ids
postgres_conn, junction_ids, pipe_ids, **({"row_limit": 1001} if row_limit is not None else {})
)
if row_limit is not None and len(devices) > 1000:
raise ValueError("too many device mappings; narrow the element selection")
devices_by_element: dict[tuple[HistoryElementType, str], list[dict[str, Any]]] = (
defaultdict(list)
)
@@ -122,6 +181,12 @@ class TimeseriesHistoryService:
target.element_id,
)
if row_limit is not None and len(selected_devices) > 20:
raise ValueError("select at most 20 devices explicitly")
if row_limit is not None:
return await _query_page(
timescale_conn, postgres_conn, query, selected_devices, row_limit, row_offset,
)
scada_rows = await ScadaRepository.get_scada_by_ids_time_range(
timescale_conn,
list(selected_devices),
-93
View File
@@ -1,93 +0,0 @@
from typing import Any, Literal
import httpx
from pydantic import BaseModel, Field
from app.core.config import settings
Freshness = Literal["noLimit", "oneDay", "oneWeek", "oneMonth", "oneYear"]
class WebSearchRequest(BaseModel):
query: str = Field(..., min_length=1, description="搜索关键词")
freshness: Freshness | str = Field(
default="noLimit",
description="时间范围:noLimit、oneDay、oneWeek、oneMonth、oneYear 或日期范围",
)
summary: bool = Field(default=True, description="是否返回网页摘要")
count: int = Field(default=10, ge=1, le=50, description="返回结果数量")
include: list[str] | None = Field(default=None, description="限定搜索域名")
exclude: list[str] | None = Field(default=None, description="排除搜索域名")
class BochaSearchConfigError(RuntimeError):
pass
class BochaSearchAPIError(RuntimeError):
def __init__(self, status_code: int, detail: Any):
super().__init__("Bocha Web Search API request failed")
self.status_code = status_code
self.detail = detail
def _build_payload(request: WebSearchRequest) -> dict[str, Any]:
payload = request.model_dump(exclude_none=True)
if request.include:
payload["include"] = ",".join(request.include)
if request.exclude:
payload["exclude"] = ",".join(request.exclude)
return payload
async def search_bocha_web(
request: WebSearchRequest,
*,
client: httpx.AsyncClient | None = None,
) -> dict[str, Any]:
if not settings.BOCHA_API_KEY:
raise BochaSearchConfigError("BOCHA_API_KEY is not configured")
headers = {
"Authorization": f"Bearer {settings.BOCHA_API_KEY}",
"Content-Type": "application/json",
}
payload = _build_payload(request)
if client is not None:
response = await client.post(
settings.BOCHA_WEB_SEARCH_URL,
headers=headers,
json=payload,
)
return _parse_response(response)
async with httpx.AsyncClient(
timeout=settings.BOCHA_WEB_SEARCH_TIMEOUT_SECONDS
) as managed_client:
response = await managed_client.post(
settings.BOCHA_WEB_SEARCH_URL,
headers=headers,
json=payload,
)
return _parse_response(response)
def _parse_response(response: httpx.Response) -> dict[str, Any]:
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
raise BochaSearchAPIError(
exc.response.status_code,
_response_detail(exc.response),
) from exc
return response.json()
def _response_detail(response: httpx.Response) -> Any:
try:
return response.json()
except ValueError:
return response.text
+482
View File
@@ -0,0 +1,482 @@
"""Project persisted runtime state into the read-only supply workbench feed."""
from collections.abc import Callable
from datetime import datetime, timezone
from math import isfinite
from typing import Any
from app.domain.schemas.workbench_runtime import WorkbenchPersistedAlert
_ANALYSIS_TYPE_LABELS = {
"burst_analysis": "爆管分析",
"burst_detection": "爆管检测",
"burst_location": "爆管定位",
"contaminant_analysis": "污染物分析",
"contaminant_simulation": "污染物模拟",
"dma_leak_identification": "DMA 漏损识别",
"flushing_analysis": "冲洗分析",
"pressure_regulation": "压力调控",
"sensor_placement": "传感器布点",
"valve_close_analysis": "关阀分析",
}
_ANALYSIS_STATUS_LABELS = {
"queued": "排队中",
"pending": "等待中",
"running": "运行中",
"completed": "已完成",
"succeeded": "已成功",
"failed": "已失败",
"error": "异常",
"cancelled": "已取消",
}
async def build_workbench_runtime_snapshot(
*,
analysis_runs: list[dict[str, Any]] | None = None,
latest_analysis_results: list[dict[str, Any]] | None = None,
latest_analysis_map_payload: dict[str, Any] | None = None,
alert_storage_configured: bool = False,
persisted_alerts: list[dict[str, Any]] | None = None,
scenario_availability: str | None = None,
artifact_availability: str | None = None,
schedule_availability: str | None = None,
alert_availability: str | None = None,
projection_degraded: bool = False,
clock: Callable[[], datetime] = lambda: datetime.now(timezone.utc),
) -> dict[str, Any]:
persisted_analysis_runs = (analysis_runs or [])[:100]
scenarios = [project_analysis_scenario(run) for run in persisted_analysis_runs[:20]]
latest_analysis = (
project_latest_analysis(
persisted_analysis_runs[0],
latest_analysis_results or [],
referenced_map_payload=latest_analysis_map_payload,
)
if persisted_analysis_runs
else None
)
stored_alerts = [
WorkbenchPersistedAlert.model_validate(item)
for item in (
(persisted_alerts or [])[:100]
if alert_storage_configured
else []
)
]
active_stored_alerts = [
{
"id": f"backend-alert-{alert.alert_id}",
"condition_id": alert.condition_id,
"source": alert.source.model_dump(),
"title": alert.title,
"description": alert.description,
"occurred_at": alert.occurred_at,
"severity": alert.severity,
}
for alert in stored_alerts
if alert.status == "active"
]
projected_alerts = active_stored_alerts
projected_alerts.sort(
key=lambda alert: _timestamp(alert["occurred_at"]),
reverse=True,
)
source = "none"
resolved_scenario_availability = scenario_availability or (
"ready" if scenarios else "not_configured"
)
resolved_artifact_availability = artifact_availability or (
"ready" if latest_analysis else "not_configured"
)
resolved_schedule_availability = schedule_availability or "not_configured"
resolved_work_order_availability = "not_configured"
resolved_alert_availability = alert_availability or (
"ready" if alert_storage_configured else "not_configured"
)
section_availabilities = (
resolved_scenario_availability,
resolved_artifact_availability,
resolved_schedule_availability,
resolved_work_order_availability,
resolved_alert_availability,
)
return {
"availability": (
"degraded"
if projection_degraded or "degraded" in section_availabilities
else
"ready"
if (
source != "none"
or alert_storage_configured
)
else "not_configured"
),
"source": source,
"scenario_availability": resolved_scenario_availability,
"artifact_availability": resolved_artifact_availability,
"schedule_availability": resolved_schedule_availability,
"work_order_availability": resolved_work_order_availability,
"alert_availability": resolved_alert_availability,
"generated_at": clock(),
"conditions": [],
"alerts": projected_alerts,
"scenarios": scenarios,
"latest_analysis": latest_analysis,
"work_orders": [],
}
def project_analysis_scenario(run: dict[str, Any]) -> dict[str, Any]:
run_id = str(run["run_id"])
run_type = str(run.get("run_type") or "analysis")
run_status = str(run.get("status") or "unknown").lower()
type_label = _analysis_type_label(run_type)
return {
"id": f"analysis-run-{run_id}",
"run_id": run_id,
"name": str(run.get("name") or type_label),
"description": (
f"{type_label} · {_analysis_status_label(run_status)} · "
f"创建人 {run.get('created_by') or 'unknown'}"
),
"status": _analysis_scenario_state(run_status),
"run_type": run_type,
"run_type_label": type_label,
"run_status": run_status,
"created_at": run["created_at"],
}
def project_analysis_history_item(run: dict[str, Any]) -> dict[str, Any]:
run_type = str(run.get("run_type") or "analysis")
run_status = str(run.get("status") or "unknown").lower()
created_at = run["created_at"]
return {
"run_id": str(run["run_id"]),
"name": str(run.get("name") or _analysis_type_label(run_type)),
"run_type": run_type,
"run_type_label": _analysis_type_label(run_type),
"status": run_status,
"status_label": _analysis_status_label(run_status),
"created_by": str(run.get("created_by") or "unknown"),
"created_at": created_at,
"started_at": run.get("started_at") or created_at,
"has_results": bool(run.get("has_results")),
}
def project_latest_analysis(
run: dict[str, Any],
results: list[dict[str, Any]],
*,
referenced_map_payload: dict[str, Any] | None = None,
) -> dict[str, Any]:
run_type = str(run.get("run_type") or "analysis")
run_status = str(run.get("status") or "unknown").lower()
created_at = run["created_at"]
return {
"run_id": str(run["run_id"]),
"name": str(run.get("name") or _analysis_type_label(run_type)),
"run_type": run_type,
"run_type_label": _analysis_type_label(run_type),
"status": run_status,
"status_label": _analysis_status_label(run_status),
"created_by": str(run.get("created_by") or "unknown"),
"created_at": created_at,
"started_at": run.get("started_at") or created_at,
"results": [
{
"result_type": str(result.get("result_type") or "analysis_result"),
"created_at": result.get("created_at") or created_at,
"facts": _analysis_result_facts(result),
}
for result in results
],
"map_payload": _analysis_map_payload(results) or referenced_map_payload,
}
def sensor_location_ids_for_map_projection(
results: list[dict[str, Any]],
) -> list[str]:
if _analysis_map_payload(results) is not None:
return []
for result in reversed(results):
if result.get("result_type") != "sensor_placement":
continue
payload = result.get("payload")
if not isinstance(payload, dict):
continue
locations = _non_empty_text_list(payload.get("sensor_locations"))
if locations:
return list(dict.fromkeys(locations))[:500]
return []
def project_sensor_location_map_payload(
node_ids: list[str], map_points: list[dict[str, Any]]
) -> dict[str, Any] | None:
points_by_id = {
str(point.get("node_id")): point
for point in map_points
if point.get("node_id") is not None
}
features: list[dict[str, Any]] = []
for position, node_id in enumerate(node_ids, start=1):
point = points_by_id.get(node_id)
if point is None:
continue
longitude = point.get("longitude")
latitude = point.get("latitude")
if (
isinstance(longitude, bool)
or isinstance(latitude, bool)
or not isinstance(longitude, (int, float))
or not isinstance(latitude, (int, float))
or not isfinite(float(longitude))
or not isfinite(float(latitude))
):
continue
features.append({
"type": "Feature",
"id": node_id,
"properties": {
"label": f"监测点 {position} · {node_id}",
"node_id": node_id,
"result_type": "sensor_placement",
"position": position,
},
"geometry": {
"type": "Point",
"coordinates": [float(longitude), float(latitude)],
},
})
if not features:
return None
return {"type": "FeatureCollection", "features": features}
def _analysis_result_facts(result: dict[str, Any]) -> list[dict[str, str]]:
facts: list[dict[str, str]] = []
node_id = _non_empty_text(result.get("node_id"))
link_id = _non_empty_text(result.get("link_id"))
if node_id:
facts.append({"label": "关联节点", "value": node_id})
if link_id:
facts.append({"label": "关联管段", "value": link_id})
payload = result.get("payload")
if not isinstance(payload, dict):
return facts
result_type = str(result.get("result_type") or "")
if result_type == "sensor_placement":
locations = _non_empty_text_list(payload.get("sensor_locations"))
sensor_count = _finite_number_text(payload.get("sensor_number"))
if sensor_count is None and locations:
sensor_count = str(len(locations))
if sensor_count is not None:
facts.append({"label": "测点数量", "value": sensor_count})
minimum_diameter = _finite_number_text(payload.get("minimum_diameter"))
if minimum_diameter is not None:
facts.append({"label": "最小管径参数", "value": minimum_diameter})
if locations:
facts.append({
"label": "测点位置",
"value": _summarize_identifiers(locations),
})
elif result_type == "leakage_identification":
run_status = _non_empty_text(payload.get("run_status"))
if run_status:
facts.append({"label": "结果状态", "value": run_status})
sensor_nodes = _non_empty_text_list(payload.get("sensor_nodes"))
if sensor_nodes:
facts.append({"label": "传感器节点数", "value": str(len(sensor_nodes))})
rows = payload.get("rows")
if isinstance(rows, list):
facts.append({"label": "结果行数", "value": str(len(rows))})
areas = payload.get("areas")
if isinstance(areas, list):
facts.append({"label": "分区数量", "value": str(len(areas))})
return facts
def _non_empty_text(value: object) -> str | None:
if not isinstance(value, str):
return None
normalized = value.strip()
return normalized or None
def _non_empty_text_list(value: object) -> list[str]:
if not isinstance(value, list):
return []
return [text for item in value if (text := _non_empty_text(item)) is not None]
def _finite_number_text(value: object) -> str | None:
if isinstance(value, bool) or not isinstance(value, (int, float)):
return None
if isinstance(value, float) and not isfinite(value):
return None
return str(value)
def _summarize_identifiers(values: list[str], limit: int = 12) -> str:
visible = values[:limit]
suffix = f" 等 {len(values)} 个" if len(values) > limit else ""
return "、".join(visible) + suffix
def _analysis_map_payload(
results: list[dict[str, Any]],
) -> dict[str, Any] | None:
for result in reversed(results):
payload = result.get("payload")
if not isinstance(payload, dict):
continue
candidate = payload.get("drawing_payload")
if not isinstance(candidate, dict):
candidate = payload
if (
candidate.get("type") == "FeatureCollection"
and isinstance(candidate.get("features"), list)
and candidate["features"]
):
return candidate
return None
def _analysis_type_label(run_type: str) -> str:
return _ANALYSIS_TYPE_LABELS.get(run_type, run_type)
def _analysis_status_label(status: str) -> str:
return _ANALYSIS_STATUS_LABELS.get(status, status)
def _analysis_scenario_state(status: str) -> str:
if status == "running":
return "active"
if status in {"completed", "succeeded", "failed", "error"}:
return "review"
return "draft"
def _timestamp(value: datetime | str) -> float:
parsed = (
value
if isinstance(value, datetime)
else datetime.fromisoformat(value.replace("Z", "+00:00"))
)
return parsed.timestamp()
# Persisted runtime queries shared by REST and MCP.
from contextlib import asynccontextmanager
import logging
from pydantic import ValidationError
from app.auth.project_dependencies import ProjectContext
from app.infra.db.postgresql.analysis import AnalysisRepository
from app.infra.db.postgresql.workbench_operations import WorkbenchOperationsRepository
from app.domain.schemas.workbench_runtime import WorkbenchRuntimeSnapshot
logger = logging.getLogger(__name__)
@asynccontextmanager
async def _projection_transaction(conn):
transaction = getattr(conn, "transaction", None)
if transaction is None:
yield
return
async with transaction():
yield
async def _sensor_location_map_payload(conn, results):
node_ids = sensor_location_ids_for_map_projection(results)
if not node_ids:
return None
points = await AnalysisRepository.list_junction_map_points(conn,node_ids)
return project_sensor_location_map_payload(node_ids,points)
def _validate_documents(documents, model, label):
valid=[];rejected=0
for document in documents:
try:valid.append(model.model_validate(document).model_dump())
except ValidationError:
rejected+=1
logger.warning("Workbench %s document was rejected",label)
return valid,rejected
async def read_workbench_runtime(ctx: ProjectContext, conn):
analysis_runs: list[dict] | None = None
latest_results: list[dict] | None = None
latest_analysis_map_payload: dict | None = None
analysis_availability = "not_configured"
alert_storage_configured = False
alerts: list[dict] | None = None
schedule_availability = "not_configured"
alert_availability = "not_configured"
try:
async with _projection_transaction(conn):
analysis_runs = await AnalysisRepository.list_recent_runs(conn, limit=100)
if analysis_runs:
latest_results = await AnalysisRepository.list_recent_results(
conn, analysis_runs[0]["run_id"], limit=100
)
analysis_availability = "ready" if analysis_runs else "not_configured"
except Exception:
logger.warning("Workbench analysis projection is unavailable", exc_info=True)
analysis_runs = None
latest_results = None
analysis_availability = "degraded"
if analysis_runs and latest_results:
try:
async with _projection_transaction(conn):
latest_analysis_map_payload = await _sensor_location_map_payload(
conn, latest_results
)
except Exception:
logger.warning(
"Workbench analysis spatial references are unavailable",
exc_info=True,
)
try:
async with _projection_transaction(conn):
operations_availability = await WorkbenchOperationsRepository.availability(conn)
alert_storage_configured = operations_availability.alerts
except Exception:
logger.warning("Workbench operations storage probe is unavailable", exc_info=True)
schedule_availability = "degraded"
alert_availability = "degraded"
if alert_storage_configured:
try:
async with _projection_transaction(conn):
alert_documents = await WorkbenchOperationsRepository.list_alerts(
conn, str(ctx.project_id)
)
alerts, invalid_alerts = _validate_documents(
alert_documents, WorkbenchPersistedAlert, "alert"
)
alert_availability = "degraded" if invalid_alerts else "ready"
except Exception:
alerts = None
alert_availability = "degraded"
logger.warning("Workbench alert projection is unavailable", exc_info=True)
return await build_workbench_runtime_snapshot(
analysis_runs=analysis_runs,
latest_analysis_results=latest_results,
latest_analysis_map_payload=latest_analysis_map_payload,
alert_storage_configured=alert_storage_configured,
persisted_alerts=alerts,
scenario_availability=analysis_availability,
artifact_availability=analysis_availability,
schedule_availability=schedule_availability,
alert_availability=alert_availability,
projection_degraded=False,
)
+1 -1
View File
@@ -3,7 +3,7 @@
"contracts": {
"server": {
"file": "server-v1.openapi.json",
"sha256": "b565d841061c9091f48ff3118bcc0cbb1b918b0cb8c2316d177570e8b2d8ba29"
"sha256": "949077b0e1c711cfef78319d24cbe081e8d71444e7b25018507e12ba374018a3"
}
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+53
View File
@@ -0,0 +1,53 @@
# 动态调查计划数据契约
调查任务从工单投诉内容或人工请求创建。Agent 在本次执行中生成 `execution-plan@2`,根据新证据追加修订;数据库保存不可变计划快照的 JSONB 文档,并不保存固定流程模板。任务、Run、证据、访问申请、审核动作和报告各有独立生命周期。计划中的 `gate_ref` 只引用已经持久化的访问申请或动作,不能把审批结果写成模型自行判断的文本。
## `execution-plan@2`
| 字段 | 含义 |
| --- | --- |
| `schema` | 固定为 `execution-plan@2`;旧快照继续按 `execution-plan@1` 读取 |
| `title`、`reason` | 本次计划名称和制定或修订原因 |
| `nodes[]` | 具有稳定 `id`、`title`、`owner`、`status`、`detail`、`kind`、`evidence_refs[]`、`action_refs[]` 的步骤 |
| `nodes[].kind` | `activity` 单项执行;`decision` 条件判断;`parallel_split` 并行分叉;`parallel_join` 并行汇合;`gate` 人工审核或授权等待点 |
| `nodes[].selected_edge_id` | 仅判断节点使用。判断完成时指向实际选择的出边;未选择时为 `null` |
| `nodes[].gate_ref` | 仅审核节点使用。UUID 指向任务内已有的访问申请或待确认动作 |
| `nodes[].revisits_node_id` | 再次核验时,新节点引用先前节点 ID;不创建回环连线 |
| `edges[]` | 每条有稳定 `id`、`from`、`to` 和可空 `condition`;判断路线必须有条件文字 |
图必须无环且只有一个起点。只有 `decision` 和 `parallel_split` 可以有多条出边;`parallel_join` 至少有两条入边。选中路线的直接目标不能标为跳过;未选路线的直接目标不能标为执行中、待人工或已完成。执行中和已完成的步骤,前置步骤必须完成或跳过。已完成的业务判断与活动必须引用真实证据或动作;纯分叉、汇合和审核节点不要求伪造证据。服务端在发布时再次核验引用、当前 Run 和会话租约。
下面是最小示例;`gate_ref` 所指申请先通过独立接口创建,再发布包含审核节点的计划。审批后,Agent 从当前任务上下文读取结果并发布下一版计划。
```json
{
"schema": "execution-plan@2",
"title": "水质投诉调查",
"reason": "核对工单位置和当前风险",
"nodes": [
{"id": "risk", "title": "判断紧急风险", "owner": "agent", "status": "completed", "detail": "根据投诉记录判断", "kind": "decision", "selected_edge_id": "urgent", "evidence_refs": ["complaint-1"], "action_refs": []},
{"id": "field", "title": "安排现场采样", "owner": "human", "status": "pending", "detail": "待安排现场人员", "kind": "activity", "evidence_refs": [], "action_refs": []},
{"id": "routine", "title": "常规核查", "owner": "human", "status": "skipped", "detail": "本次选择紧急路线", "kind": "activity", "evidence_refs": [], "action_refs": []},
{"id": "split", "title": "并行核对", "owner": "agent", "status": "pending", "detail": "授权后同时核对现场和数据", "kind": "parallel_split", "evidence_refs": [], "action_refs": []},
{"id": "upstream", "title": "上游监测", "owner": "agent", "status": "pending", "detail": "待批准读取", "kind": "activity", "evidence_refs": [], "action_refs": []},
{"id": "network", "title": "管网运行", "owner": "agent", "status": "pending", "detail": "待核对运行事件", "kind": "activity", "evidence_refs": [], "action_refs": []},
{"id": "join", "title": "汇总调查证据", "owner": "agent", "status": "waiting", "detail": "等待现场与数据结果", "kind": "parallel_join", "evidence_refs": [], "action_refs": []},
{"id": "access", "title": "审核数据范围", "owner": "human", "status": "waiting", "detail": "等待项目成员审核", "kind": "gate", "gate_ref": "84000000-0000-4000-8000-000000000001", "evidence_refs": [], "action_refs": []}
],
"edges": [
{"id": "urgent", "from": "risk", "to": "access", "condition": "有紧急风险"},
{"id": "normal", "from": "risk", "to": "routine", "condition": "无紧急风险"},
{"id": "access-split", "from": "access", "to": "split", "condition": null},
{"id": "field-route", "from": "split", "to": "field", "condition": "现场"},
{"id": "upstream-route", "from": "split", "to": "upstream", "condition": "上游"},
{"id": "network-route", "from": "split", "to": "network", "condition": "管网"},
{"id": "field-join", "from": "field", "to": "join", "condition": null},
{"id": "upstream-join", "from": "upstream", "to": "join", "condition": null},
{"id": "network-join", "from": "network", "to": "join", "condition": null}
]
}
```
实际调查可以把现场路径与 Agent 路径放在同一个并行分叉下,并在下一次修订中补充更多判断或并行组。已发布计划保留原样;拒绝授权、人工补充、复测或回访触发新 Run 时,Agent 读取当前任务事实后继续规划。旧 `execution-plan@1` 只有依赖关系,前端按原始前置条件展示,不推断它代表条件选择还是并行执行。
访问申请独立存于 `workbench.operational_task_access_requests`,包括请求范围、理由、申请节点、审核人、批准范围和版本;工单草稿及其审批、回执仍存于动作表。前端默认呈现选中路线,并在明确的分叉与汇合处组成并行组;未选路线折叠保留供核对。
+22
View File
@@ -0,0 +1,22 @@
# 15173 联调 UI Keycloak 客户端
2026-09-23 已在 `tjwater` realm 验证独立 public client
`tjwater-integration-ui`。该客户端只用于本机 15173 联调页面,不替代或修改
现有 `next-tjwater` 客户端。
已验证配置:
- Standard Authorization Code Flow:开启。
- PKCE:强制 `S256`。
- Implicit Flow、Direct Access Grants、Service Accounts:关闭。
- Public client:开启,不保存客户端密钥。
- Full Scope Allowed:关闭;未授予 realm-management 或系统管理员角色。
- Redirect URI:`http://127.0.0.1:15173/*`、`http://localhost:15173/*`。
- Web Origin:`http://127.0.0.1:15173`、`http://localhost:15173`。
可使用以下幂等脚本复核或恢复同一配置;管理员密码仅从隐藏提示或进程环境
读取,不会写入文件:
```bash
conda run -n server python scripts/provision_integration_ui_client.py
```
@@ -0,0 +1,116 @@
# 调查进展与实时展示
调查执行中可持续发布正文和完整 Mermaid 图,尚未形成结论时也能发布。
Server 将阅读进展、业务报告和浏览器展示请求分别保存。运行结束仍要求同轮最终报告,
进展中的节点状态不参与调度、审批或结束判断。
## 进展快照
`POST /api/v1/operational-tasks/{task_id}/agent-artifacts` 新增可选 `progress`:
```json
{
"request_id": "<uuid>",
"run_id": "<current-run-uuid>",
"source_session_id": "<claimed-session-id>",
"progress": {
"summary": "正在核对投诉时段的监测记录,尚未形成结论。",
"diagram": {
"title": "当前调查进展",
"mermaid": "flowchart LR\nread[核查监测记录] --> judge[判断证据范围]",
"node_refs": [{
"node_id": "read",
"label": "核查监测记录",
"status": "running",
"evidence_refs": [],
"action_refs": [],
"map_targets": [{"source_id": "pipes", "feature_id": "<known-pipe-id>"}]
}]
}
}
}
```
示例中的编号由实际任务和资产替换。正文最长 20,000 字符,`diagram` 可空。
图示沿用 [调查报告契约](investigation-report-contract.md) 的 Mermaid 安全限制。
节点状态可空,或为 `pending/running/completed/waiting_human/failed/skipped/cancelled`。
`map_targets` 只接受 `junctions/pipes/valves/pumps/tanks/reservoirs/scada` 七类来源和要素编号。
图中节点引用在本次发布内唯一,证据和动作引用必须属于该任务。跨修订可以修改节点编号、
分支和文字,Server 不验证依赖执行顺序或节点完工。
每次传入 `progress` 都完整替换该 run 的快照,传入 `diagram: null` 清除上一版图示。
省略 `progress` 保留原快照。同一个请求编号重试只发布一次。
`workbench.operational_task_progress` 每个任务和 run 保存最新一份快照,独立于七天事件清理。
任务详情和 Agent 上下文返回 `latest_progress`,包含 `revision/run_id/execution_generation/`
`published_at/source_session_id/progress`。只读取当前执行代次,新轮次尚未发布时返回空。
快照更新发出 `operational.progress.changed`,前端据此读取最新内容。
## 浏览器实时联动
`POST /api/v1/operational-tasks/{task_id}/presentation-events` 接收
`request_id/run_id/source_session_id/command`,要求任务执行权限。
Server 校验当前 run、绑定会话、有效租约和幂等身份,记录 `operational.presentation.requested`。
响应为 `recorded/event_seq/run_id/execution_generation`,表示展示请求已保存,
不表示浏览器已显示或现场动作已执行。
| command.action | 参数 |
| --- | --- |
| highlight_features | layer、feature_ids、effect:steady 或 pulse |
| clear_highlights | 无 |
| open_chart | chart_id、chart、非空 evidence_refs |
| close_chart | chart_id |
| set_overlay | overlay_id、title、mode:geometry 或 heatmap、evidence_id |
| clear_overlay | overlay_id |
图表限定折线和柱形,包含 `title/chart_type/x_data/series`。最多 100 个横轴值和 8 组有限数值序列,
序列长度与横轴对齐,引用证据必须属于当前任务。地图高亮来源沿用上述七类。
覆盖层输入不接受模型提供的 `data`。Server 从证据的不可变数据快照提取现有 GeoJSON,
或使用已记录的 `longitude/latitude` 点位,将 `FeatureCollection` 加入事件中的 `command.data`。
热力图只接受记录中的点,权重为有限非负数,未记录权重时取 1。
没有坐标的地址、空数据、非法几何、模型自绘区域均不能生成覆盖层。该接口不做地址解析、
范围推算或污染传播计算。展示事件总大小上限为 512 KiB,超限返回 422。
`GET /api/v1/operational-tasks/{task_id}/presentation-cursor` 在同一事务中读取
`task_id/execution_generation/active_run/event_cursor`,其中活跃 run 仅返回 `run_id/generation/status`。
浏览器打开 Dev 联动、切换任务或断线重连时,以此建立新基线,只消费之后的展示事件。
跟随开关只影响当前浏览器的临时展示,正文和图示更新始终可读,调查仍由后台继续执行。
关闭 Dev 面板不改变跟随开关。关闭跟随开关、切换任务或断线时清理临时效果,
执行结束停止执行态动效,已显示的图表与覆盖层保留供阅读。不回放历史请求,不重新执行调查。
## 人工反馈续调
外围同步器根据动作回执找到关联子工单。样板创建固定为 `revision=1`,首次创建回执不作为
新的现场反馈入队。后续版本可唤醒已结束调查,旧报告和证据保留,生成新的执行代次。
现场备注从工单 `activity.kind=feedback` 的记录进入反馈证据,仍标明样板合成来源。
重复请求或同一源版本重放不增加执行代次,取消调查拒绝新反馈。
同步器先送达各父调查反馈,再推进预约工单投影的源版本。定向源事件和动作回执补查保留,
处理源更新早于回执保存的情况。全量对账只为源版本变化补查已结束调查,本次不回填历史反馈。
原投诉收到新版本后同步推进其已有来源修订号。数据不足仍正常结束本轮,
只有新的可信业务输入进入后续轮次,不依赖图示节点制造人工等待。
## 部署与验证
新增 [023 迁移](../resources/sql/023_operational_task_progress.sql) 仅适用于 `tjwater_v2`,
增加进展表和两个事件 topic 的 CHECK 取值。2026-10-10 已在目标库完成全库备份、
迁移事务回滚演练及正式应用,其他项目和旧任务未改动。
实际 worker 曾成功投递指定调查,模型发布进展和报告,但因 service 账号的任务权限过滤
导致 `approved_data_scopes` 为空,误作数据不足结束。这不是业务数据确实不足。
用户确认开放规则后,任务 API 已移除该额外过滤,沿用账号已有的项目读取权限。
在线上下文返回七类来源,同一身份实际读取 SCADA 设备和外围合成工单成功;没有新增
角色权限或放宽跨项目、审批和设备控制边界。旧委托路径保留兼容,已有权限无需委托凭据。
用户级 `tjwater-operational-worker.service` 已重新启用。旧报告保留,未自动重新调查,
不能把旧报告视为修复后的业务验收。任务创建时间下界和并发配置见
[执行 worker](operational-task-workers.md),未回填旧任务。
进展读接口在表尚未部署时返回空,写入口须在迁移与 Server 更新后启用。
按现有数据库发布流程应用迁移,再更新 Server、Agent 和前端,运行中的 worker 需要加载更新。
聚焦检查:
```bash
conda run -n server python -m pytest tests/unit/test_operational_tasks.py tests/unit/test_operational_presentation.py tests/unit/test_peripheral_task_sync.py tests/unit/test_peripheral_push_sync.py tests/api/test_operational_task_dynamic_flow.py tests/api/test_openapi_contract.py -q
git diff --check
```
这些检查覆盖隔离 HTTP 与状态机,不代替共享数据库迁移、真实模型运行或现场复测验收。
+166
View File
@@ -0,0 +1,166 @@
# 调查报告数据契约
本契约适用于供水事件调查,由 Server 的 `InvestigationReportInput`(写入)和
`InvestigationReport`(读取)定义。Agent 和前端采用同样的字段和约束。
不增加格式版本标签;原字段的类型保持稳定。`revision` 是报告修订号,
`execution_generation` 是调查轮次,两者均不用于选择格式解析器。
## 数据模型
| 字段 | 类型 | 新写入规则 |
| --- | --- | --- |
| title | 非空字符串 | 报告标题 |
| current_conclusion | 非空字符串 | 当前判断文字;无结论时明确写无法判断,不编造原因 |
| conclusion_boundary | 非空字符串 | 判断范围及限制 |
| evidence_refs | 字符串数组 | 引用该任务中已持久化、不可变的证据编号 |
| recommendations | 字符串数组 | 保持原类型,处置建议不等于已执行动作 |
| pending_items | 字符串数组 | 保持原类型,不从文字推断状态 |
| stage | interim / final | 必填;阶段性报告不能用于结束运行 |
| outcome | concluded / insufficient_data | 必填;业务调查结果,与运行成功/失败独立 |
| stop_reason | 非空字符串或 null | 证据不足时必填;已形成结论时必须为 null |
| data_gaps | 缺口对象数组 | 必填;空数组表示已记录且没有缺口 |
| diagram | 调查脉络对象或 null | 可选;给人阅读的 Mermaid 说明,不控制执行或任务状态 |
缺口对象除 node_id 外的字段必填:
| 字段 | 类型 | 含义 |
| --- | --- | --- |
| gap_id | 非空字符串 | 报告内唯一的缺口编号 |
| node_id | 非空字符串或 null | 可选的展示关联,不要求有执行计划 |
| missing_data | 非空字符串 | 缺失数据内容 |
| scope | 非空字符串 | 查询对象、地点及时间范围 |
| impact | 非空字符串 | 因此无法回答的调查问题 |
| required_data | 非空字符串 | 新一轮调查需要补齐的具体数据 |
| evidence_refs | 至少一个证据编号 | 实际查询或可信输入记录;必须也出现在报告 evidence_refs 中 |
| blocking | boolean | 是否阻止形成结论 |
禁止未知字段、未知枚举和重复 gap_id。各引用在 Server 中校验。
查询失败、权限配置异常不能作为“没有数据”的证明。
调查脉络对象包含 `title`、`mermaid` 和可选的 `node_refs`(默认空数组)。
`mermaid` 最长 20,000 字符,必须以 `flowchart` 或 `graph` 及方向开头,
不包含 Markdown 围栏、初始化指令和 click 命令。前端使用 strict 安全级别渲染 SVG。
完整图语法的解析失败只影响图示,正文、依据和审核仍可阅读。
每条 node_ref 包含 `node_id`、`label`、`evidence_refs`、`action_refs`;后两项默认空数组。
node_id 为最多 80 字符的 ASCII 标识符,必须以字母或下划线开头,在本图引用内唯一;
最多 100 条引用。证据必须来自报告 evidence_refs,动作必须属于当前任务。
节点可以说明候选方向与条件关系;图中的分支、状态文字不能证明实际并发、动作执行或现场完成。
图的颜色、字体和交互由前端统一配置。正文不包含 SVG 或 HTML。
## 业务约束
1. `concluded` 允许 interim/final,stop_reason 为 null,不允许 blocking 缺口。
2. `insufficient_data` 只能是 final,至少一个 blocking 缺口,必须提供 stop_reason;
recommendations 和 pending_items 必须为空,不发起补数据的工单动作。
3. 报告必须来自当前执行轮次、会话和有效租约;执行计划与图示均可缺省。
历史计划或未结束的解释节点不阻止发布与正常收尾。运行状态来自 Server run,
审核和业务结果来自真实动作与回执,图示不参与计算。
4. 尚有待审核、已批准未提交、已提交或结果未知的动作时,不能发布数据不足退出报告。
不自动撤销已有动作,不影响独立现场处置。
5. 数据不足报告保存后,本执行代不再接受新的报告。补齐数据应使用已有 retry 接口
创建新 execution_generation,原报告、证据和回执保持可追溯。
6. 事件调查提议动作前必须已保存本轮 outcome=concluded 的报告。
顺序为阶段性结论、动作提议、最终报告、运行收尾。图示可在报告中关联动作编号。
7. 数据不足时顺序为查询证据、最终报告、运行收尾。
运行正常结束为 succeeded,调查生命周期为 completed;页面显示“已结束·证据不足”,
不显示正常水质、不自动关闭原投诉工单,不等待人工授权或补数据。
8. 数据读取范围由系统配置;本场景的人工审核只针对具体业务动作。
既有数据访问审核接口仅保留兼容其他调用,不作为事件调查的产品入口。
## 历史读取与兼容
旧报告缺少 stage/outcome/stop_reason/data_gaps 时,Server 统一返回 null。
null 表示未记录,不能转换成 concluded、空缺口清单或正常水质。
前端兼容滚动发布期间的缺省/null,未知字段仍严格拒绝;一旦提供 outcome,
其余结构化字段和业务约束必须完整满足。新写入不接受全空的历史格式。
历史导入脚本的 dry-run 继续校验原始记录;缺少结构化结果的历史报告不能直接发布为新调查报告,实写在创建任务前拒绝,避免按旧正文编造结果。
无需改写历史报告或新增数据库表,新增字段保存在原报告 document 中。
上线顺序:先部署能读取可选新字段的前端,再协调更新 Server 新写入契约与 Agent。
在切换写入端前应结束旧 Agent 在途运行,旧写入格式会被明确拒绝,不能静默推断结果。
回退前须保留支持新字段的读取端;不能直接回退到会拒绝新字段的旧前端。
## 接口与错误
所有路径均以 `/api/v1` 开头,保留现有 Bearer 身份与项目范围校验。
| 方法与路径 | 用途 |
| --- | --- |
| GET /operational-tasks/{task_id} | 读取统一的报告模型,旧结构化字段返回 null |
| GET /operational-tasks/{task_id}/reports?limit=20&offset=0 | 按修订号倒序读取历史,返回 items 与 next_offset |
| GET /operational-tasks/{task_id}/reports/{revision} | 读取指定报告快照,任务及项目范围校验 |
| GET /operational-tasks/{task_id}/agent-context?report_revision=1 | selected_report 是所选报告,task 保留当前真实状态;未传则 selected_report 为 null |
| POST /operational-tasks/{task_id}/agent-artifacts | 写入报告和计划;保留 request_id 幂等、run_id 与会话归属校验 |
| POST /operational-tasks/{task_id}/runs/{run_id}/complete | 事件调查校验本轮最终报告后结束运行;检查 expected_revision |
| POST /operational-tasks/{task_id}/actions | 仅本轮已保存 concluded 报告的事件调查允许提议动作 |
| POST /operational-tasks/{task_id}/retry | 创建新调查轮次,不覆盖上一轮报告 |
| 错误码 | HTTP | 含义 |
| --- | --- | --- |
| 请求体校验错误 | 422 | 新报告缺字段、未知枚举、无关键缺口或相互矛盾的字段 |
| REPORT_GAP_EVIDENCE_INVALID | 422 | 缺口依据不在报告的证据集合内 |
| REPORT_NOT_FOUND | 404 | 指定任务中的报告修订不存在 |
| REPORT_ACTION_PENDING | 409 | 缺数退出仍有关联的未决动作 |
| INVESTIGATION_ENDED | 409 | 本轮已经发布缺数退出报告,再写成果需要新轮次 |
| ACTION_CONCLUSION_REQUIRED | 409 | 没有本轮已形成结论的报告,不得生成动作 |
complete 接口保留已有返回方式:不完整成果会把 run 标为 failed,
failure_code 为 ARTIFACT_INCOMPLETE;调用方须读取返回的业务状态,不能只看 HTTP 200。
## 缺数报告示例
以下为 agent-artifacts 请求中 report 字段的完整示例。
`sample-query-empty` 必须是同一任务内已持久化的空查询证据;示例本身不构成真实查询事实。
```json
{
"title": "水质投诉调查报告",
"current_conclusion": "缺少投诉时段的末梢采样,无法确定异常来源。",
"conclusion_boundary": "已核对可用记录,本轮未确认水质异常原因。",
"evidence_refs": ["sample-query-empty"],
"recommendations": [],
"pending_items": [],
"stage": "final",
"outcome": "insufficient_data",
"stop_reason": "关键采样数据缺失,本轮调查结束。",
"data_gaps": [{
"gap_id": "terminal-sample-gap",
"node_id": null,
"missing_data": "投诉时段末梢水样检测结果",
"scope": "投诉小区,2026-09-24 08:00 至 09:00(北京时间)",
"impact": "无法判断异常来自供水管网还是用户端。",
"required_data": "该地点和时段的采样检测记录",
"evidence_refs": ["sample-query-empty"],
"blocking": true
}]
}
```
## 验收
- 新写入缺字段、不一致结果、无证据的关键缺口、未知业务证据或动作引用均被拒绝。
- 关键数据不足保存报告并正常结束;无运行失败、无待人工通知、无后续动作。
- 阶段性报告不能冒充最终报告;旧代结论不能授权新代动作。
- 已有未决动作阻止矛盾的缺数退出;重试幂等,不重复生成报告。
- 新轮次保留旧报告历史标识,不能显示成当前轮次已结束。
- 桌面及 320/375px 展示结果、缺口、证据,审批区不出现数据授权入口。
## 发布、阅读与协作
执行中尚未形成结论的正文和 Mermaid 使用独立的 `progress` 快照持续更新,
见 [调查进展与实时展示](investigation-progress-and-presentation.md)。
报告的 interim/final 与 concluded/insufficient_data 结果契约保持不变。
每次发布保留 `operational.report.published` 同步事件。首次 final、阶段报告转 final、
新轮次 final 或判断/边界/依据/建议/缺口实质变化时,生成 `report_updated` 通知,
其中 `report_reference = {execution_generation, revision}`。纯图示修订不生成额外成果提醒。
新提醒取代同任务之前的成果提醒;读取相应报告后按用户确认已读。
前端在调查列表显示“新报告”,这类提醒不进入“待办与告警”。
正在阅读的报告和图示固定在所选修订,收到更新后显示“查看新报告”,不自动替换正文。
当前待审核动作单独读取最新真实状态,即使用户正在阅读旧报告也不会审批旧版本。
协作请求 `task_context` 可携带 report_revision、evidence_id、gap_id;轮次和报告修订号由
Agent 服务根据 Server 返回的快照生成 `data-task_context`,随回答流和历史消息持久化。
该引用说明回答依据,不授予访问权限,也不成为执行约束。
+71
View File
@@ -0,0 +1,71 @@
# Agent 业务分析 MCP
Agent 原生工具统一使用 `mcp_tjwater_` 前缀。Server目录共35项:25项查询、下表10项分析;合并6项宿主查询后共41项。工具参数来自Server实际MCP schema,不维护第二套参数契约。
| 工具名(不含前缀) | 能力与边界 | 必要权限 |
| --- | --- | --- |
| `run_simulation` | 在线模拟:按水力步长直接计算业务库,不复制数据库;更新模型条件和实时基线,返回指定节点/管段,不创建方案或正式分析运行 | `simulation.run` + `scada.view` |
| `run_burst_detection` | 指定时刻的15天压力异常侦测,至少5个完整监测节点;不定位 | `burst.run` + `scada.view` |
| `run_burst_location` | 异常/正常压力、可选流量及漏水量假设的候选管段排序 | `burst.run` + 对应数据的 `scada.view` 或 `simulation.view` |
| `analyze_valve_isolation` | 拓扑隔离边界、候选关阀及影响节点;不是水力计算或真实关阀 | `simulation.run` |
| `run_leakage_identification` | SCADA压力约束下的DMA漏损分配估计;需总漏损量依据 | `burst.run` + `scada.view` |
| `run_contaminant_simulation` | 已假设污染源的正向浓度传播;不是源头反演 | `simulation.run` + `scada.view` |
| `analyze_valve_closure` | 给定关阀组合的水力影响 | `simulation.run` + `scada.view` |
| `analyze_flushing` | 指定排放节点和流量的冲洗水力影响;不证明消毒合格 | `simulation.run` + `scada.view` |
| `analyze_pressure_regulation` | 给定调泵序列、水箱初始水位的压力影响;不自动优化调度 | `simulation.run` + `scada.view` |
| `optimize_sensor_placement` | 灵敏度或K-means压力监测点建议;不保存正式布点方案 | `optimization.run`,项目成员角色 |
## 执行与证据
标准水力模拟示例(只展示参数,不执行):
```json
{"start_time":"2026-10-08T12:00:00+08:00","duration":3600,"output_node_ids":["<已核实节点ID>"],"output_link_ids":["540672"]}
```
工具完整 ID 为 `mcp_tjwater_run_simulation`。开始时间必须带时区,时长为60至86400秒,
默认900秒;至少指定一个节点,节点和管段各最多20个。项目与身份由宿主绑定。
输出沿用场景结果信封,包含 `node_results`、`link_results`、单位、时间和覆盖范围。
它复用 REST `/api/v1/simulation-runs` 的在线计算服务:REST 时长仍为分钟,MCP 时长为秒。
每个水力步长执行一次 `realtime` 快照,更新在线模型条件和该时刻的实时基线;不复制模型数据库。
结果中的 `times.sample_times` 对齐采样点,窗口不包含结束时刻;例如一步长15分钟、时长3600秒时返回4个采样点。
`persisted_baseline=true` 表示在线基线已重算,`persisted_scheme=false` 表示没有创建方案。
该工具的 MCP annotation 为 `readOnlyHint=false`、`destructiveHint=true`,因为重算会替换对应时刻的基线。
它不会创建 `analysis.run_id`,不能将本次计算冒充已有爆管事件运行;调查调用仍可保存证据。
计算读取现有模型和对应SCADA条件,不从摘要虚构观测,也不自动补齐数据或执行现场操作。
爆管定位 `run_burst_location` 默认使用 `data_source="monitoring"`。`pressure_scada_ids` 可省略,由后端从当前项目SCADA配置自动获取压力测点对应的模型节点;显式指定时仍限制为1至20个不同节点。正常对照时间窗 `scada_normal_start` / `scada_normal_end` 可成对省略,沿用后端默认的前一天同时间段;返回结果中的 `pressure_scada_ids` 和 `scada_window` 说明实际测点和时间窗。默认对照不等于已确认该时段正常。正常窗无有效压力数据或两窗没有共同有效测点时计算失败,不补造数据、不自动换窗。正常窗只有部分测点缺数时,沿用服务逻辑使用两窗共同有效测点,结合返回测点范围解释结果。
仅提供异常时段和有依据的漏水量即可使用上述默认值,例如:
```json
{"scada_burst_start":"2026-10-08T12:00:00+08:00","scada_burst_end":"2026-10-08T13:00:00+08:00","burst_leakage":0.0274}
```
`data_source="simulation"` 仍须提供已有 `simulation_run_id`,使用同窗模型基线作正常对照,不填写正常时间窗;在线 `run_simulation` 不创建该运行UUID,不能替代已有分析运行。
- 身份、项目路由、任务和运行由宿主绑定;模型不能提供network、username、凭据、文件路径、原始观测数组或方案保存名称。
- 依赖时序数据的8项计算通过权限校验后,先从项目元数据解析并激活 BizDB 与 TimescaleDB 完整路由,再启动计算线程;时序库未配置时在修改模型前失败。普通业务查询、纯拓扑隔离及监测点选址不强制要求时序库路由。
- 普通项目会话、任务协作和调查主Agent可调用;个人会话、学习复盘和只读reader不能调用分析。Server每次同时校验当前权限与绑定时的权限上限;任务数据读取scope不能替代计算权限。管理员显式deny继续生效。
- 复用现有业务函数。在线模拟直接调用业务库;定位关闭共享候选CSV输出;识别不保存方案;选址使用不持久化模式。其余四项场景模拟复用项目专用`<业务库>_template`及唯一临时模型库,校验模板同步就绪,沿用临时库并发限制和清理;不是复制在线业务库。模板复制仍会短暂停止模板连接/订阅工作进程,不能称为完全没有基础设施副作用。
- 场景`persist_results=False`仅允许隔离临时模型,跳过正式analysis运行、Timescale结果和基线写入;既有REST默认保存行为不变。仅返回请求的最多20个节点及20个管段,包含单位、时间和缺失对象;不能用局部结果断言全网无影响。
- 数据源和计算假设必须有依据。定位漏水量和DMA总漏损约束使用m³/s;冲洗使用服务实际接收的m³/h,不沿用旧REST误标的L/s。工频泵序列0/1,变频泵序列为Hz;每项对应模型水力步长。冲洗当前限LPS/CMH模型。选址当前推荐压力点,不冒充污染溯源优化。
- 调查结果经现有快照/证据/回执链保存,只有返回evidence_id后才引用为系统证据;普通聊天不自动保存调查证据。模拟结果始终是analysis而非实测。已知模拟事件源管段是输入背景,不是算法定位准确性的证明。
## 资源边界与发布
10项计算工具均提供 `timeout_seconds`,由Agent根据模型规模、模拟步数及已有耗时按次选择;它表示实际等待预算,与 `duration` 表示的模拟时间不同。未填写默认120秒。服务端上限由 `TJWATER_MCP_MAX_CALCULATION_TIMEOUT_SECONDS` 控制,默认3600秒,管理员可设120至86400秒;`tools/list` schema 发布实际有效上限,超限在计算前拒绝。
例如 `duration=3600, timeout_seconds=900` 表示模拟1小时、允许等待15分钟。Agent的上游HTTP/SDK预算为所选预算加5秒,整个宿主调用另留60秒处理认证、上下文绑定和清理;OpenCode连接上限从工具目录计算并写入宿主连接文件。本地MCP请求关闭Bun空闲断连,仍受上述调用截止时间控制。普通查询继续使用原30秒Server预算与35秒宿主预算。
计算在线程执行,不阻塞事件循环,单Server进程最多两项计算。超时不能杀死底层原生计算,槽位一直保留到计算真正退出;超时结果不会补写为调查成功结果,不自动反复重试。本次未增加后台计算作业队列。普通聊天大结果仍受现有12KB内联限制,调查结果可经证据分页读取。
在线重算逐个时刻写入基线,超时或中途失败可能已有部分时刻更新,不能把工具失败解释为整个窗口已回滚;不要自动重试整个窗口。
无需新增数据库迁移或模型配置。须同步发布Server、Agent和前端证据枚举;先更新Server目录,再重启Agent,client模式还需重载原生连接。保持既有OAuth、MCP开关及数据库路由配置,不自动更改权限或重启调查。
本次验证限schema、权限、协议目录和mock业务函数,不运行真实模拟、优化或付费模型。真实项目的模板同步状态、运算耗时、SCADA覆盖和结果准确性仍需另行验收。
2026-10-19 修复:EPANET 2 导出省略阀门默认 `ACTIVE` 状态行并保留数值设置,EPANET 3 导出保留原表达;覆盖文件和文本导出。MCP 在线模拟改为共用在线计算服务,方案模拟继续隔离。回归验证包含原生 EPANET 输入校验及 mock 在线基线写入,不重算真实项目历史数据。
标准水力模拟接入已通过后端schema/权限/隔离适配测试、Agent目录与权限测试、前端证据解析测试;Agent和前端类型检查、前端改动文件ESLint通过。隔离HTTP smoke使用真实两端SDK发现34项Server工具及40项合并工具,并核对 `run_simulation` 输入schema和上下文绑定/查询;身份服务及数据执行为fixture,不等同生产验收。测试启动器剔除宿主MCP环境配置,避免私有issuer覆盖测试的loopback设置。运行中的服务仍需同步重载后确认新目录,未启动真实模拟。
+88
View File
@@ -0,0 +1,88 @@
# MCP 接入的 Keycloak 管理员配置清单
这里创建的是 OAuth 认证配置。OpenCode 到 Agent、Agent 到 Server 的 MCP 协议客户端已实现,无需再开发或安装 MCP 客户端程序。
以下使用已创建的请求方客户端 `agent-mcp`,MCP 资源的 audience 使用 `tjwater-mcp`。它们位于现有业务登录使用的同一个 realm。
## 1. 请求方客户端 agent-mcp
在 Keycloak 的 Clients 中打开 `agent-mcp`:
| 设置 | 值 |
| --- | --- |
| Client type / Protocol | OpenID Connect |
| Client authentication | On;这是 confidential 客户端 |
| Service accounts roles | On;用于启动时读取 MCP 工具目录 |
| Standard token exchange | On;用于按当前用户或 worker 的身份换取 MCP 令牌 |
| Standard flow | Off;该客户端不负责浏览器登录 |
| Direct access grants | Off |
| Access token lifespan | 建议300秒,不超过900秒 |
| 签名算法 | RS256 |
保存后,在 Credentials 中取得 Client Secret。密钥只写入 Agent 的忽略文件 `.local.env`:
```dotenv
TJWATER_MCP_CLIENT_ID='agent-mcp'
TJWATER_MCP_CLIENT_SECRET='<Credentials 中的 Client Secret>'
```
不要把密钥放入聊天、前端配置、OpenCode 配置或 Git。Server 只配置 `TJWATER_MCP_HOST_CLIENT_ID='agent-mcp'`,不需要保存这个客户端的密钥。
## 2. 资源客户端及 scopes
创建或复用表示 MCP 服务的 OIDC 客户端 `tjwater-mcp`;关闭浏览器登录、直接密码登录和服务账户。这个 ID 是目标 audience,不是另一个需要部署的服务。
创建以下 OIDC Client scopes,并作为 Optional scopes 分配给 `agent-mcp`;启用 Include in token scope:
- `mcp.catalog.read`
- `mcp.query`
- `mcp.context.bind`
在这些 scopes 中添加 Audience mapper,使对应 token 的 `aud` 包含目标客户端 `tjwater-mcp`。应用按实际 grant 请求 scope:服务账户请求只有 `mcp.catalog.read`;用户/worker 的交换请求才包含查询和上下文 scope。不要给服务账户额外分配项目成员、管理或设备控制权限。
## 3. 允许当前业务身份交换
在原有前端登录客户端和 worker 客户端的专用 scope 中添加 Audience mapper,使源 token 的 `aud` 包含请求方 `agent-mcp`。保留原业务 API 所需的 audience;不要给源 token 添加 MCP 目标 `tjwater-mcp`。
交换后必须保持用户主体 `sub` 不变,`azp=agent-mcp`,目标 `aud` 包含 `tjwater-mcp`。Keycloak 的 `audience` 参数只筛选已有映射的 audience,不能代替 Audience mapper。
版本和界面名称以当前 Keycloak 为准。配置依据:[Keycloak Standard Token Exchange](https://www.keycloak.org/securing-apps/token-exchange)。仅在公开发现中看到 token exchange,并不能证明具体客户端已启用或映射正确。
## 4. 地址、数据库和启用顺序
正式环境提供该 realm 的 HTTPS issuer。本地开发可显式允许一个 RFC1918 IPv4 HTTP issuer:Agent 设置 `NODE_ENV=development`,Server 设置 `ENVIRONMENT=development`,两端 `TJWATER_MCP_DEV_HTTP_ISSUER_URI` 均填写与各自 `TJWATER_MCP_ISSUER_URI` 完全相同的地址(含 realm)。此例外不放行远程 HTTP MCP resource;Agent 只允许该 issuer 同源的远程 HTTP 发现及令牌端点,继续拒绝重定向。正式环境必须清空此项并使用 HTTPS 域名。配置值必须与发现文档和令牌 `iss` 一致,不能仅替换访问地址。
例如,同机开发时 Server `.env` 可配置为(内网地址按实际认证服务替换):
```dotenv
ENVIRONMENT=development
TJWATER_MCP_ENABLED=true
TJWATER_MCP_ISSUER_URI=http://192.168.1.114:8088/realms/tjwater
TJWATER_MCP_DEV_HTTP_ISSUER_URI=http://192.168.1.114:8088/realms/tjwater
TJWATER_MCP_RESOURCE_URI=http://127.0.0.1:8000/mcp
```
Agent `.local.env` 使用相同的三个 URI,并设置 `NODE_ENV=development`、
`TJWATER_MCP_ENABLED=true`。开发模式不跳过签名、issuer、audience、scope 或项目权限校验,
也不自动接受另一种 issuer。切换 Server 环境还会开启开发 API 文档页面。
修改环境文件后需重启相应进程;已有进程不一定自动重读 `.env`,系统环境变量也可能覆盖文件。
应先确认 Server 的 `/.well-known/oauth-protected-resource/mcp` 公布的新 issuer,
再启动 Agent 检查工具目录。切回生产时,两端恢复相同 HTTPS issuer 并删除 HTTP 白名单。
业务身份也必须使用同一 issuer:供水前端只在被 Git 忽略的
`.env.development.local` 中将 `TJWATER_KEYCLOAK_ISSUER` 设为该内网地址,
保留 `.env.local` 的 HTTPS 配置供正常 production 构建使用。
Server 本机 `.env` 可额外设置 `OPERATIONAL_WORKER_DEV_TOKEN_URL` 为同一内网 issuer
发现文档中的 `token_endpoint`;该值仅在 `ENVIRONMENT=development` 时生效,
不替换原来的 `OPERATIONAL_WORKER_TOKEN_URL`。重启开发前端和 worker,并在浏览器重新登录。
旧 HTTPS 令牌不能用于内网 issuer 的交换;工具目录读取成功不等于业务身份交换成功。
接入人员随后执行:
1. 安装 MCP SDK 2 对应的 Server 依赖。
2. 对选定项目业务库备份并应用 [025_mcp_query_contexts.sql](../resources/sql/025_mcp_query_contexts.sql)。
3. 在 Agent 和 Server 的私有环境中配置相同的 issuer、resource、audience;resource 必须指向实际 Server `/mcp`,请求方客户端 ID 均使用 `agent-mcp`。
4. 验证带标准 `resource` 的目录 grant 和用户/worker token exchange 都成功;若身份平台拒绝,先解决兼容性,不移除参数重试。
5. 在受控测试实例中开启两端 `TJWATER_MCP_ENABLED` 并重载;通过测试项目查询、任务证据和越权拒绝验收后,再按部署流程启用正式环境。
完整环境变量与验收要求见 [MCP 实施文档](mcp-query-implementation.md)。在以上步骤完成前保持 MCP 开关关闭。
+280
View File
@@ -0,0 +1,280 @@
# NextTJWater MCP 查询与认证设计
日期:2026-10-15。状态:查询与认证代码已在本地实现,认证发现、resource 参数和版本头体校验已补齐,默认关闭,未部署;真实 Keycloak 与项目数据库验收待完成。
本设计将现有后端查询能力通过 MCP 开放给普通项目 Agent、调查主 Agent 和调查只读子 Agent。采用 MCP 2026-07-28、Streamable HTTP、现有 Python Server 内挂载服务、可信 Agent 宿主按请求代理认证。旧 CLI 用于查找功能和接口线索,业务实现继续由当前 Server 负责。
第一版完成查询、权限、取消、分页、调查证据和凭据续期的完整闭环。已有报告发布、工单审批、设备控制、调查运行生命周期继续使用当前受控工具;不通过 MCP 自动开放旧 CLI 中的分析执行命令。
## 当前系统与功能核对
Agent 的独立只读查询现统一通过原生 `mcp_tjwater` 连接调用。重复的项目查询插件与自定义 HTTP 查询桥已移除;`memory_manager`、`skill_manager` 保留完整读写插件,发布、动作、消息发送和界面展示也保留插件形式。普通会话凭据保存在宿主的 `src/product/credential-refresh.ts`;调查由 worker 以服务身份投递并续期。调查子 Agent 通过父任务绑定访问数据,不持有独立产品凭据。
实施前从 `app.main.app.openapi()` 核对了 266 路径的契约,其路径集合及各 HTTP operation 与工作区 `contracts/server-v1.openapi.json` 一致。公开 API 经 `app/api/v1/rest_router.py` 适配,项目字段注入和分页须看最终注册契约,不能只看底层 handler 签名。旧 CLI 的 41 条业务命令中,38 条能匹配当前 HTTP 方法与路径:29 条读命令、9 条执行命令。另有 3 条需要替换或不开放,详见附表。路径匹配不代表参数兼容、数据可用或算法已完成业务验收。
实施前 `requirements.txt` 和 conda 环境均为 `mcp==1.9.4`、`fastmcp==2.9.2`,当时没有 Server MCP 注册实现。Agent 使用的 OpenCode 2.0.27 配置类型支持 `legacy`、`auto`、`2026-07-28`,宿主认证代理现已补充,见实施说明。
## 协议选择
官方 `specification/latest` 指向 [MCP 2026-07-28](https://modelcontextprotocol.io/specification/2026-07-28)。新实现按该正式版本建设;产品内部连接固定要求这个版本,版本不支持时返回明确错误。第一版不提供旧 `initialize` 握手兼容层。
| 协议要求 | 本系统设计 |
| --- | --- |
| 无状态请求与逐请求能力声明 | 每次请求发送标准 `_meta` 中的版本、客户端信息和能力;不以连接识别用户或调查轮次 |
| `server/discover` | 返回版本、服务信息和实际支持能力;不能用自报服务名建立信任 |
| HTTP 请求镜像头 | 由 SDK发送并校验 `MCP-Protocol-Version`、`Mcp-Method`、适用时的 `Mcp-Name`;头与正文不一致拒绝 |
| 普通结果 | 返回 `resultType=complete`、`structuredContent`、匹配的 `outputSchema`,以及简短文字说明 |
| 列表和资源缓存提示 | 返回 `ttlMs`、`cacheScope`;缓存不作为权限证明 |
| 取消与断线 | 取消传到 SDK、数据库和外部请求;断开的流不按旧 SSE 游标恢复 |
这些改变来自[正式版变更说明](https://modelcontextprotocol.io/specification/2026-07-28/changelog)和 [Streamable HTTP 规范](https://modelcontextprotocol.io/specification/2026-07-28/basic/transports/streamable-http)。协议支持情况以实际发现与调用验收为准,不以软件版本字符串代替。
第一版声明 tools 和 resources;不声明 prompts、subscriptions、MRTR、Tasks、MCP Apps、Sampling、Roots 或 Logging 能力。查询不要求模型补填身份、密码或令牌。现有产品调查任务不会替换为 MCP Tasks。
Schema 使用 JSON Schema 2020-12,模型输入对象拒绝未声明字段;只使用服务自带的本地定义,不在校验时从任意外部 URL 获取 `$ref`。宿主校验模型投影,Server 再校验完整线协议参数。工具发现、结构化结果、状态句柄和执行错误遵循[正式工具规范](https://modelcontextprotocol.io/specification/2026-07-28/server/tools)。
| 失败类型 | 响应与宿主处理 |
| --- | --- |
| 未认证或过期 | HTTP 401 和 Bearer challenge;宿主最多续期一次 |
| scope 或产品权限不足 | HTTP 403;scope 不足时带 insufficient_scope,不自动扩大产品权限 |
| 请求头与正文不一致 | HTTP 400、JSON-RPC -32020 |
| 协议版本不支持 | HTTP 400、JSON-RPC -32022,列出支持版本 |
| RPC 方法不存在 | HTTP 404、JSON-RPC -32601 |
| 工具不存在或 RPC 参数结构无效 | JSON-RPC -32602 |
| 业务参数、数据源或上下文失败 | resultType=complete、isError=true;返回稳定业务错误码和可操作说明 |
旧客户端 GET/DELETE 到 `/mcp` 返回 405;旧 Mcp-Session-Id 和 Last-Event-ID 不建立或恢复身份。产品上下文删除使用独立 REST 控制路径,不混用协议会话删除。
`server/discover` 和固定查询目录按工具名稳定排序,缓存提示取 `ttlMs=300000`、`cacheScope=public`,正文仅包含通用定义。业务结果资源取 `ttlMs=0`、`cacheScope=private`;宿主按令牌、主体、项目和上下文分别保存缓存,不跨授权上下文复用。[缓存规范](https://modelcontextprotocol.io/specification/2026-07-28/server/utilities/caching)
## 服务与调用结构
在现有 FastAPI Server 挂载规范路径 `/mcp`,通过官方 Python SDK v2 承担协议处理。ASGI 的生命周期纳入现有应用启动和关闭;不得用根路径挂载遮蔽 REST 路由。官方 SDK 支持[在现有 ASGI 应用中挂载](https://py.sdk.modelcontextprotocol.io/run/asgi/),其[协议版本说明](https://py.sdk.modelcontextprotocol.io/protocol-versions/)覆盖 2026-07-28。
查询适配器复用现有 service、repository、项目路由和业务权限,不从 MCP 向自己的 REST API再发一次 HTTP 请求,不转发收到的 MCP Bearer。需要从 endpoint 提取逻辑时,提取供 REST 和 MCP 共用的 service,而不直接调用带 FastAPI Depends 默认值的 endpoint 函数。
```text
模型调用查询工具
→ OpenCode 原生 MCP(tools/call,_meta 中关联原生 sessionID)
→ Agent 本地 MCP 服务(内部密钥认证、宿主会话和直接父子关系检查)
→ 宿主专用产品 MCP 客户端
→ Keycloak(按主体获取面向 MCP 的令牌)
→ Server /mcp(标准 MCP 请求 + 宿主绑定的上下文)
→ 项目与任务权限检查
→ 共用查询 service
→ BizDB / TimescaleDB / 既有外围适配器
→ 调查证据保存和有范围的数据结果
```
OpenCode 原生发现本地 MCP 的工具目录,工具 ID 为 `mcp_tjwater_<name>`。本地 MCP 默认提供地址解析、会话搜索、调查任务列表与详情、消息读取、待办读取六项查询;启用产品 MCP 后,宿主从真实 Server `tools/list` 发现24项项目查询,投影并合并到本地目录,共30项。公开网页搜索使用 OpenCode 原生 `websearch`,Server 不提供搜索接口。可信插件只配置连接和权限策略,不注册独立查询工具或自定义执行器。待办读取经宿主 RPC 访问原插件存储,保留已有清单。
本地服务使用官方 TypeScript Server SDK 2.3.1、2026-07-28 Streamable HTTP,绑定127.0.0.1临时端口,拒绝浏览器来源和错误内部密钥。OpenCode 的 `_meta["ai.opencode/sessionID"]` 仅是关联值;通过内部通道认证后,服务再解析宿主身份或已核验的直接只读子会话。未知会话、个人项目查询和学习复盘均拒绝。URL、身份、项目、网络、产品认证头和上下文句柄不属于模型参数。产品 MCP 未启用时项目查询不可用,没有旧桥回退。取消信号贯穿整个调用链。[OpenCode MCP 会话上下文](https://dev.opencode.ai/v2/docs/mcp-servers/#context)
现有 OpenCode 共享 MCP 配置缺少逐会话 OAuth 提供器,不能把一份用户 Bearer 填进 `mcp.servers.headers`。产品内部连接由 Agent 宿主按请求认证,外部 MCP 继续保持原配置路径和现有策略,二者不能误注册为同名工具或形成两份不同凭据的业务连接。
## 认证与上下文绑定
### OAuth 资源与现有登录
Keycloak 继续作为授权服务器。MCP 是独立的受保护资源,资源标识由 `TJWATER_MCP_RESOURCE_URI` 指定为部署的规范 `/mcp` URI;令牌目标 audience 取专用客户端 `tjwater-mcp`。Agent 宿主使用预注册 confidential client `tjwater-agent-mcp`。用户继续使用当前前端标准流程和 S256 PKCE 登录,不新增产品内的第二次登录。
OAuth scope 分成 `mcp.catalog.read`、`mcp.query` 和 `mcp.context.bind`。宿主启动时以自身 client_credentials 取得仅含 catalog scope 的令牌,发现真实目录;该身份不获得项目业务读取能力。会话查询令牌必须具有 query scope,绑定接口还要求 context.bind。scope 只开放接口类别,具体数据权限仍由产品主体、项目成员关系和工具 registry 求交。
MCP 端提供 `/.well-known/oauth-protected-resource/mcp`,401 响应包含对应的 `WWW-Authenticate` 资源发现信息。验证签名、固定 issuer、audience、exp、iat、客户端来源和查询 scope;对新 MCP 使用独立验证器,不依赖当前 REST 验证器可为空的 audience,也不通过修改全局 audience 破坏现有 REST。
标准 OAuth 客户端在授权和 token 请求中携带规范资源 URI。MCP 仅接收确实面向它发行的访问令牌,令牌每次通过 Authorization 头发送,不出现在 URL、工具参数或模型文本中。[MCP 授权规范](https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization)、[授权服务器发现](https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization/authorization-server-discovery)
产品宿主先发送无凭据的标准发现请求,从401 challenge 读取资源元数据地址;无地址时按 MCP 路径、根路径依次发现 well-known。资源 URI 必须匹配已配置资源,authorization_servers 必须包含预注册凭据所属的固定 issuer。随后使用官方 SDK 按规范顺序发现 OAuth/OIDC 元数据,严格比较 issuer,从元数据取得 token endpoint;不向其他 issuer 发送原凭据。公开发现请求不带产品 Bearer 或 secret,单份文档最多64KiB,总预算10秒;拒绝重定向和远程明文地址。发现缓存最多5分钟,MCP 401 后失效重查。根据元数据使用 client_secret_post 或 client_secret_basic。
产品内部宿主基于当前产品凭据使用 Keycloak Standard Token Exchange V2 换取面向 `tjwater-mcp` 的短期访问令牌;保留原主体,限定查询 scope,避免共享一个超级服务账号访问所有普通用户数据。专用令牌只保存在 Agent 主进程内存中,刷新令牌仍由现有登录机制持有。
Keycloak V2 需要 confidential requester、启用 Standard token exchange、subject token 对 requester 的 audience,以及目标 audience/scope 配置。client_credentials 与 token exchange 均发送 `resource=规范MCP URI`;exchange 另保留产品要求的 audience。身份平台必须将该资源映射到专用 audience,并核验两种 grant 返回的令牌。若部署版本拒绝 resource,应先升级或提供兼容适配再启用;不能删除标准参数后重试。不增加第三方客户端授权码流程。[Keycloak Token Exchange](https://www.keycloak.org/securing-apps/token-exchange)
上述 realm/client 能力尚未在线核验,是部署前提,由身份平台配置负责人落实。达不到前提时停止启用内部 MCP;不以普通 API Bearer 转发、共享管理员 token 或关闭 audience 校验降级。
### 每次调用绑定项目与调查轮次
OAuth 令牌只证明主体与可用 scope,业务范围由 Server 签发的 `context_handle` 约束。它是应用上下文,不是 MCP 协议会话,也不能单独作为认证凭据。
可信 Agent 宿主通过受认证的 `POST /api/v1/mcp-query-contexts` 申请上下文。请求中的 session、purpose、项目和任务信息均由原生会话及宿主已有绑定读取;仅允许 `azp=tjwater-agent-mcp` 的宿主客户端创建任务上下文。Server 从令牌确定主体、从成员关系确定项目读取权限,验证 task、run、execution_request 和父会话占用的租约;原生父子关系由可信宿主先核验;不接受自报用户、权限或数据 scope。
Server 保存的绑定字段包括:主体、client ID、project ID、native session ID、purpose、可用查询工具、任务及运行标识、父会话、创建与过期时间、撤销状态。句柄使用至少 256 位随机值,10 分钟过期。普通上下文到期由宿主重新申请;任务上下文有效性另外取决于当前轮次和 lease,不能用续租让旧轮次重新获得访问权。
所有业务 MCP tool 的线协议 schema 具有宿主字段 `context_handle`。Agent 宿主客户端在发布给本地 MCP 的 schema 投影中移除它,并在请求 Server 时覆盖注入;它不使用 `x-mcp-header`,也不进入聊天 transcript 或普通日志。若输入包含同名字段、身份字段或保留 scope 字段,宿主直接拒绝。resources/read 由宿主在应用命名空间 `_meta` 中注入同一上下文,URI 仅标识任务证据而不携带句柄或令牌。外部客户端若将来接入,应自己调用绑定接口并在宿主侧选择项目,不从模型内容决定项目。
Server 每次同时核验 Bearer 与句柄所属的 `sub/azp`,重新校验成员关系和当前权限;任务调用还检查项目、任务、run、execution request 和 lease。句柄复制到其他用户、客户端或项目无效。切换项目新建会话,关闭/删除会话撤销绑定;调用 `DELETE /api/v1/mcp-query-contexts/{context_handle}` 时检查绑定所有者。令牌在新 scope 下续期时不会自动扩大已有上下文的工具授权。
普通会话的 `purpose=ordinary_chat`,仅访问其绑定项目。调查主 Agent 的 `purpose=task_execution`,使用 worker 的现有服务主体和本轮任务范围。只读子 Agent 的 `purpose=task_reader`,宿主根据已经验证的父子关系绑定父轮次;绑定只允许查询和系统证据追加,不授予报告发布、审批、设备控制、Shell 或再次委派权限。个人会话不创建项目查询上下文,返回现有 `PROJECT_CONTEXT_REQUIRED`。
### 续期和退出
普通项目会话复用现有 `credential_refresh_required` 回传与前端 Keycloak `updateToken`。收到更新的产品 access token 后,宿主重新 exchange 并继续原查询,用户只在现有登录本身失效时重新登录。并发续期按主体与原凭据版本合并,最终缓存按 issuer、resource、主体、scope 和原凭据版本隔离。
后台调查不依赖浏览器。worker 保持当前 client_credentials 续期;新服务 Bearer 到达后由 Agent 更新原会话产品凭据,再 exchange。MCP 声明官方 `io.modelcontextprotocol/oauth-client-credentials` 授权扩展以支持服务身份;首次范围仍由服务账号成员关系与任务授权决定。[Client Credentials 扩展](https://modelcontextprotocol.io/extensions/auth/oauth-client-credentials)
MCP 401 清除本主体缓存并续期一次;403 不反复刷新、不自动申请更大 scope。认证服务不可用时返回认证暂不可用,不能返回空数组掩盖故障。注销时撤销产品绑定、取消调用并清除宿主缓存。JWT exchange 的源 token 撤销不必然撤销已交换 token,因此 MCP 令牌短期有效,Server 的绑定撤销和每次业务权限检查仍然必须执行。
## 查询工具目录
以下为第一版明确目录。名称为 MCP 服务端名字,OpenCode 原生发现后添加 `mcp_tjwater_` 前缀。元素枚举固定为 junction、pipe、valve、pump、tank、reservoir;时间必须带时区;身份、项目和 network 都不在模型输入中。HTTP 路径表示现有能力来源,MCP 实现调用其共用 service。
| 工具 | 业务输入 | 当前能力来源与限制 |
| --- | --- | --- |
| `get_element_properties` | element_type、element_id | 六类 `/{type}s/properties` |
| `list_elements` | element_type、limit、offset | 六类 `/{type}s`;返回一页,不暗中获取全网 |
| `query_nearby_elements` | center、可选 element_type、radius_m、limit、offset | 当前项目模型管网空间邻近查询;直接接收地理编码的 EPSG:4326 `[经度,纬度]`,默认pipe、200米、5条,最大10000米、单页100条;返回element_id、distance_m及properties,不查询原始GIS源要素 |
| `get_network_options` | kind、可选 pump_id | network-options 及 time、energy、pump-energy;泵能源必须给泵 ID |
| `get_network_schema` | kind | 固定 network-schemas 枚举,以当前注册路径为准 |
| `list_scada_devices` | limit、offset | `/scada-devices`,显式分页 |
| `get_scada_device` | device_id | `/scada-devices/{device_id}` |
| `query_scada_readings` | device_ids、start、end、field | scada-readings/fields;区分 monitored_value 和 cleaned_value |
| `query_model_baseline_series` | element_type、start、end、元素筛选 | realtime/links、realtime/nodes;标记为模型基线,不称为实测 |
| `get_model_baseline_at_time` | element_type、element_id、time | realtime/simulation-results |
| `query_model_baseline_field` | element_type、field、time | realtime/records;只允许当前字段枚举 |
| `query_analysis_series` | run_id、element_type、element_id、field、start、end | analysis/runs/{run_id}/links 或 nodes/{id} |
| `query_analysis_values` | run_id、element_type、field、time | analysis/runs/{run_id}/values |
| `list_analysis_runs` | limit、offset | `/analysis/runs` |
| `get_analysis_run` | run_id | `/analysis/runs/{run_id}` |
| `list_analysis_results` | run_id、result_type、limit、offset | `/analysis/runs/{run_id}/results`;保留 result_type |
| `list_sensor_placement_runs` | limit、offset | sensor-placement-runs;公开 REST 已包装分页,但底层 service 读取整数组,MCP 应推动共用 service 做真实有界查询 |
| `get_sensor_placement_run` | run_id | sensor-placement-runs/{run_id};REST 适配层已注入 network,MCP 共用 service 也从可信项目上下文解析 |
| `query_element_history` | elements、start、end、mode、可选 run_id | element-history/query;observed、realtime_comparison、analysis_comparison |
| `get_element_gis_properties` | element_type、element_id | elements/{type}/{id}/gis-properties |
| `get_source_gis_properties` | layer_key、source_fid | source-gis/layers/{key}/features/{id}/properties |
| `get_workbench_runtime` | 已有运行态查询参数 | workbench/runtime;只读取 |
| `get_task_context` | section、data_path、limit、offset、可选 report_revision;无模型可选 task_id | 当前绑定任务的固定摘要及分章节、路径分页详情;预览不等于完整报告 |
| `query_peripheral_data` | source、point_id、start、end、limit、offset | 固定外围 points、work_orders、observations、test_data 适配器;保留 synthetic |
| `get_task_evidence` | evidence_id、data_path、limit、offset | 当前任务的 evidence/{id}/snapshot;项目、任务及可读轮次由绑定约束 |
初版为24项查询;新增空间邻近查询后,当前为25项查询,连同10项分析共35项Server工具。旧 CLI 的 `data pipeline-health` 实际触发模型预测,不能包装成“读取已保存结果”;它作为 `predict_pipeline_health` 的后续计算候选,先核对模型资产、执行成本和计算权限。旧 CLI 的 9 条 POST 执行命令也仅列作后续候选,不加入查询目录。
空间邻近查询沿用 `webgis.view` 和宿主绑定项目,数据库连接不从工具参数选择。`app/infra/db/postgresql/spatial_queries.py` 使用既有 `gis.*` 模型视图及GiST几何索引做保守候选筛选,再转换到WGS84 geography,按点到完整几何的椭球最短距离以米筛选和排序,最后在数据库中执行LIMIT/OFFSET,不把全网几何交给Agent。跨日期变更线时拆分候选包围框,极区跳过Mercator候选筛选;空或缺失几何不参与查询。距离语义依据 [PostGIS ST_DWithin](https://postgis.net/docs/ST_DWithin.html) 和 [ST_Distance](https://postgis.net/docs/ST_Distance.html),不把EPSG:3857平面距离直接当作实际米数。
输出 `data.source=model_network`、`center_crs=EPSG:4326`、`distance_method=WGS84_spheroid`,`data.items` 包含 `element_id`、`distance_m` 和 `properties`;`page.has_more` 与 `truncated` 保留分页边界。没有命中返回空items,不自动扩大半径;空间邻近不证明拓扑连接或供水归属,道路/河流交叉位置必须由调用方先确认。查询预算仍为30秒,调查调用沿用现有证据与回执保存,不新增REST查询桥或数据库迁移。
聚焦验证:`python -m pytest tests/unit/test_mcp_spatial.py tests/unit/test_mcp_query.py -q`;设置 `MCP_SPATIAL_TEST_DSN` 后运行 `python -m pytest tests/integration/test_mcp_spatial_postgis.py -q` 可验证实际PostGIS距离与分页,测试只创建事务内TEMP表并回滚,不写业务数据。
权限保持当前真实规则:管网、GIS 和分析运行读取沿用 webgis.view,SCADA 沿用 scada.view,模型基线和分析时序沿用 simulation.view;监测点选址路由当前整体要求 optimization.run,MCP 不擅自改成 optimization.view。历史比较模式额外核对其中使用的模拟数据权限。拆分读权限需要单独的后端授权设计,不能借 MCP 接入顺便降级。
新源 registry 明确 source、所需权限、查询参数 Schema、数据语义、结果上限、任务 scope 和是否允许证据保存。原 7 个 TaskDataSource 与新源按明确规则扩展,历史证据仍可反序列化。既有已批准 scope 只迁移到与其原意等价的查询能力,不用“只读”作为放开所有源的理由。
## 分页 大结果与数据语义
集合查询默认 limit=20,最大 500;分析运行最大 100。offset 最大10000,超过时缩小时间窗或目标。常规集合在 SQL 中限制返回量。带总量的集合返回 items、total、limit、offset、has_more;没有稳定总量时 total=null,不伪造总数。Agent 首次优先单对象、必要的短时间窗和小页;只有明确问题仍缺证据且 has_more=true 时继续翻页,不自动遍历全网。
历史组合查询按稳定的序列顺序(设备ID、原始/清洗,再按类型和元素ID排列模型序列)及时间对数据点分页;在同一条 UNION ALL SQL 上应用全局 limit/offset,并多读一个点计算 has_more。证据查询按绑定项目、任务和证据ID,在数据库中选取 data_path 和页;嵌套集合返回路径描述,数组保持索引顺序,对象按 C 排序,字符串按 Unicode 字符分页。证据页和完整资源在返回数据库客户端前检查8MiB上限。此限制约束传输和应用进程,不代表数据库无需读取或解压 JSONB。
DEMO 样本仍需校验整份 RFC 8785 摘要,不能仅验证所选页。数据库先检查整份 JSON 文本大小,超过8MiB则拒绝传输;预算内读取完整样本,校验后筛选和分页。超限须由数据源拆分样本,不能通过反复缩小查询窗口消除该限制。
时序默认最多 20 个目标、24 小时,最多返回 10000 条或 8 MiB。全网模型基线接口当前没有元素筛选和结果分页,接入前必须在共用 service 增加有界筛选/游标;MCP 不直接代理当前无限制接口。批量历史采用其现有 mode 和 run_id 语义,不假定新接口完全等价于旧三类 composite 查询。
结果超过内联 12000 字节时返回摘要、数量、时间范围、truncated、继续查询参数或受保护 resource_link。第一版 resources 暴露已有任务证据快照,普通查询用分页继续读取;不新增全局 result-ref 存储或任意主机文件路径 API。资源读取同样核验令牌和上下文,不能凭 URI 得到数据。
共用输出对象包含 source、source_ref、queried_at、origin、data、window、units、page、truncated、data_gaps,以及调查时的 evidence_id。原 service 返回结构由各工具的 Pydantic 输出模型明确描述,不靠自由文本或一个无约束的 data 字段作为最终契约。第一版每主体/项目最多 4 个并发查询;超过并发返回可重试限流结果,队列不无限增长。查询执行预算沿用 Server 已配置的访问超时,认证续期等待单独计时。
普通原始查询错误返回结构化业务错误,不改成空数据。结果标明数据来源、查询时间、源时间窗口、单位、SCADA 实测或清洗、模型基线或分析运行、synthetic、缺失及截断。`truncated=true` 不能支持“完整范围未见异常”的结论。
## 调查证据与幂等
共享查询入口根据已验证 purpose 选择包装方式。普通聊天返回查询结果;调查及 reader 查询在返回前调用现有 `OperationalTaskService.record_data_snapshot`,保存范围与来源,再返回 snapshot 和 evidence_id。外部 MCP 返回内容不能通过这一入口被重新标成内部系统证据。
原生 callID 经宿主转为 application request key。Server 按主体、项目、会话、任务、run、callID 和规范参数记录查询回执;同 key、同参数重试读取已保存 snapshot,同 key、不同参数拒绝。MCP 断线重发使用新 JSON-RPC ID,但保持同一 application request key;证据存储提交失败就返回失败,不能先返回成功再异步补证据。
调查取数前后都校验本轮仍然有效。长查询期间不持有任务行锁;完成后用短事务确认轮次并保存证据。查询途中被新轮次替代时,不把晚到结果写进新轮次,也不复活旧运行。外部网络请求与数据库读写使用短事务,不跨网络等待持锁。
查询业务源是只读,但调查包装会追加证据,因此相应工具不能仅根据 GET 方法声明 `readOnlyHint=true`。固定目录内可能保存证据的工具保守标为 `readOnlyHint=false`、`destructiveHint=false`,执行策略按已审查的精确工具名允许“读源并追加系统证据”,不由 annotation 决定授权。精确 grant 不覆盖报告发布、工单或设备动作。
## 客户端策略与生命周期
内部业务查询在项目会话、调查执行和已验证 task_reader 中开放对应精确工具。reader 的精确白名单仅增加可信 `mcp_tjwater_*` 项目查询目录;现有外部 MCP 禁用规则继续执行,没有向模型开放通用 resources 桥或子 Agent 写权限。
取消传播至 OpenCode 原生 MCP、本地 MCP、产品 MCP 客户端、Server handler 和底层工作。接口断线时停止请求,宿主不会无条件后台补跑。读查询只允许一次认证重试和一次可判定未完成的传输重试;调查依 application request key 防止重复保存。
本地 MCP 采用已有内部 token 和浏览器来源拒绝规则;它只证明可信 OpenCode 到宿主的通道,不能替代 Server 的 OAuth 主体认证。日志保留主体、项目、任务、run、工具名、耗时、数量、取消与错误类别,不保存 Bearer、secret、context_handle 或原始敏感数据。
MCP 服务失败时其调用返回可重试错误,现有 REST 和原生查询仍可使用;如果管理员启用 MCP 作为必需产品能力,Agent 启动预热必须验证 server/discover、实际 tools/list、Schema、身份路径和取消,不调用付费推理,不用“连接成功”代替能力就绪。
## 实施范围与验收
这是两仓库的中等规模改动,预计超过 8 个文件;第一版挂载于已有 Server,不新增独立部署服务。前端正常情况下继续复用现有凭据刷新事件,只有验收发现契约缺口时才扩展。
| 仓库 | 实施落点 |
| --- | --- |
| Server | app/main.py 挂载和 lifespan;app/core/config.py;新增 app/mcp 的协议、工具与认证适配模块 |
| Server | app/services 的共用查询与上下文绑定服务;app/domain/schemas 的查询/输出契约;任务源与证据回执的持久存储要求 |
| Server | 现有 API service 提取、有界查询;独立 MCP 令牌验证;requirements.txt;.env.example;tests/api、tests/auth、tests/unit;OpenAPI 导出 |
| Agent | src/product 的 MCP 客户端与令牌缓存;现有 credential-refresh;新增内部 MCP call 桥 |
| Agent | .opencode/plugins/tjwater-tools.ts;MCP 和 task-reader 策略;工具 schema 投影;真实目录启动检查;package.json 与锁文件;聚焦测试 |
升级 Server 的官方 mcp SDK 到已发布 v2,按官方迁移文档核对 lifespan、认证、协议模式和结果类型后固定实际发布版本。不能仅升级 fastmcp 并假定新协议成立;已有依赖是否删除,须先确认真实使用与兼容关系。当前 Python SDK 文档示例同时包含 legacy 配置,协议行为以 2026-07-28 规范和实际验收为准。[Python SDK v2 迁移说明](https://py.sdk.modelcontextprotocol.io/migration/)
部署配置包含固定 Keycloak issuer、规范 MCP resource URI、专用 audience、Agent confidential client ID 与 secret。secret 只在 Agent 宿主环境或密钥管理中,Keycloak 签名密钥由 JWKS 获取并校验固定 issuer,不发给 OpenCode。沿用 worker 自有 client_credentials 配置,不把 worker secret 转交给 Agent。
Server 需提供上下文绑定、撤销和调查查询回执的持久存储;多进程读写须一致,不能用单进程 dict 宣称重启或横向扩容可用。应用层只保存令牌派生身份和范围,不保存 token。存储由后端按现有数据库管理流程交付,本设计不执行数据库迁移。
验收必须覆盖:
1. 实际 `server/discover`、24 工具目录、版本固定、标准 `_meta`、resultType、缓存提示;头与正文不一致、未知工具、非法 schema、旧握手的错误行为。
2. 两个用户两个项目并发,交错调用也不串凭据;句柄换用户、项目、client、task、run 或失效均拒绝;个人会话不得获得业务数据。
3. 普通登录无需再次登录 MCP;401续期一次、403不升权;并发 refresh 合并;注销、成员撤销和背景服务凭据续期。
4. worker 无浏览器完成查询;reader 只取父任务范围;不获得报告、审批、控制、Shell 或其他外部 MCP。
5. 管网属性与分页、SCADA 原值/清洗值、模型基线、分析 run、历史三种 mode、GIS 与外围 synthetic;不支持的旧功能返回明确能力缺口。
6. 24小时/目标数/分页边界、实际服务端限量、资源读取授权;超大结果不会全量进入内存或聊天。
7. 查询成功且证据已经持久化;断线重试仅一个证据回执;查询中轮次被替代、取消、DB失败和认证服务失败均不误报成功。
8. REST 与 MCP 的权限和数据结果相同;新 SDK 不改变原 API、worker 或报告读取;启动真实目录检查不调用模型推理。
使用 Server 现有 conda 环境运行新增 MCP 聚焦 tests/api、tests/auth、tests/unit;Agent 执行 bun run check 与新增凭据、MCP桥、策略、目录聚焦测试。用户登录、refresh 和两项目隔离需用现有前端做一次实际浏览器验收。设计文档本身仅进行路径检查与 git diff --check,不执行业务写请求。
交付可以分两次独立上线:先增加服务端只读 MCP、OAuth与上下文,保留现有产品调用;再接入 Agent 宿主客户端和 reader,验证后切换产品查询。任何一步失败都可以回到已有原生工具,不能在单次失败时自动切换凭据绕过权限。回滚停用 MCP 入口/宿主注册并撤销上下文,保留已经保存的调查证据与旧报告。提交、身份平台变更、数据库供给和部署另按实际实施授权执行。
## 旧 CLI 与当前后端对照
路径均为 `/api/v1` 下的当前能力;“执行候选”不加入第一版查询 MCP。此表依据当前源码生成 OpenAPI 和旧 CLI handler,而非线上算法成功证明。
| 旧 CLI 命令 | 当前路径或处理 | 第一版决定 |
| --- | --- | --- |
| network get-junction-properties | GET /junctions/properties | get_element_properties |
| network get-pipe-properties | GET /pipes/properties | get_element_properties |
| network get-all-pipes-properties | GET /pipes | list_elements 分页 |
| network get-reservoir-properties | GET /reservoirs/properties | get_element_properties |
| network get-all-reservoirs-properties | GET /reservoirs | list_elements 分页 |
| network get-tank-properties | GET /tanks/properties | get_element_properties |
| network get-all-tanks-properties | GET /tanks | list_elements 分页 |
| network get-pump-properties | GET /pumps/properties | get_element_properties |
| network get-all-pumps-properties | GET /pumps | list_elements 分页 |
| network get-valve-properties | GET /valves/properties | get_element_properties |
| network get-all-valves-properties | GET /valves | list_elements 分页 |
| component option schema | GET /network-schemas/{kind} | get_network_schema |
| component option get | GET /network-options 及 time、energy、pump-energy | get_network_options |
| simulation run | POST /simulation-runs | 执行候选,duration 为分钟 |
| analysis burst | POST /burst-analyses | 执行候选 |
| analysis valve isolation | POST /valve-isolation-analyses | 执行候选 |
| analysis flushing | POST /flushing-analyses | 执行候选 |
| analysis age | /water-age-analyses 未注册 | 不开放,不假定算法仍可运行 |
| analysis contaminant | POST /contaminant-simulations | 执行候选 |
| analysis sensor-placement run | POST /sensor-placement-runs | 执行候选 |
| analysis sensor-placement list | GET /sensor-placement-runs | list_sensor_placement_runs,有界适配 |
| analysis sensor-placement get | GET /sensor-placement-runs/{run_id} | get_sensor_placement_run,读取项目路由中的既有成果 |
| analysis runs list | GET /analysis/runs | list_analysis_runs |
| analysis runs get | GET /analysis/runs/{run_id} | get_analysis_run |
| analysis runs results | GET /analysis/runs/{run_id}/results | list_analysis_results |
| analysis leakage identify | POST /leakage-identifications | 执行候选 |
| analysis burst-detection detect | POST /burst-detections | 执行候选 |
| analysis burst-location locate | POST /burst-locations | 执行候选 |
| data timeseries realtime links | GET /timeseries/realtime/links | query_model_baseline_series,已补最多20个元素筛选及SQL限量 |
| data timeseries realtime nodes | GET /timeseries/realtime/nodes | query_model_baseline_series,已补最多20个元素筛选及SQL限量 |
| data timeseries realtime simulation-by-id-time | GET /timeseries/realtime/simulation-results | get_model_baseline_at_time |
| data timeseries realtime simulation-by-time-property | GET /timeseries/realtime/records | query_model_baseline_field |
| data timeseries analysis link-field | GET /timeseries/analysis/runs/{run_id}/links/{link_id} | query_analysis_series |
| data timeseries analysis node-field | GET /timeseries/analysis/runs/{run_id}/nodes/{node_id} | query_analysis_series |
| data timeseries analysis values | GET /timeseries/analysis/runs/{run_id}/values | query_analysis_values |
| data timeseries scada query | GET /timeseries/scada-readings 及 fields | query_scada_readings |
| data timeseries composite | 原三个 views 路径未注册;现有 POST /timeseries/views/element-history/query | 按三种 history mode 重新适配,不直接宣称等价 |
| data pipeline-health | GET /pipeline-health-predictions,实际进行预测 | 后续计算候选,先验证模型资产 |
| data scada get | 原 /scada-devices/detail 未注册;现有 GET /scada-devices/{device_id} | 修正为 get_scada_device |
| data scada list | GET /scada-devices | list_scada_devices 分页 |
| data scada schema | GET /network-schemas/scada-device | get_network_schema |
当前 Agent 的 `ProjectDataClient` 读取单个元素等能力仍较窄;其中 SCADA 设备读取仅列表,缺少单设备新路径入口。MCP 切换前以本表和当前 Schema 对齐,不能从旧 CLI 直接复制参数或依赖其全量分页行为。
任务上下文查询支持可选的正整数 `report_revision`;未显式指定时宿主保留协作会话绑定的报告修订号,避免将历史报告替换为最新报告。

Some files were not shown because too many files have changed in this diff Show More