Compare commits
25
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b45c5c6f46 | ||
|
|
2e4f6cb93b | ||
|
|
471ad92a47 | ||
|
|
1892b1d1f0 | ||
|
|
2e8fe86f3f | ||
|
|
b853b61a0b | ||
|
|
e47c61c121 | ||
|
|
6a4478bfe4 | ||
|
|
5197a93163 | ||
|
|
549019507e | ||
|
|
acc8f8f7cb | ||
|
|
b8359ba83c | ||
|
|
c0dea2aacf | ||
|
|
33df25f22b | ||
|
|
2a7b96f813 | ||
|
|
06d8539aa8 | ||
|
|
2552a9b40d | ||
|
|
01bad116c1 | ||
|
|
bf8ea45d40 | ||
|
|
dc83cf5c88 | ||
|
|
ff8aba4964 | ||
|
|
26fa5d1909 | ||
|
|
f25423dec4 | ||
|
|
e48ccfba9e | ||
|
|
f2cfce8192 |
+5
-4
@@ -6,6 +6,8 @@
|
||||
# 基础运行
|
||||
PROJECT_NAME="TJWater Server"
|
||||
ENVIRONMENT="production"
|
||||
# 任务领取范围;留空时沿用 ENVIRONMENT。历史任务需先执行 034 迁移。
|
||||
OPERATIONAL_TASK_EXECUTION_SCOPE=""
|
||||
API_V1_STR="/api/v1"
|
||||
NETWORK_NAME="tjwater"
|
||||
|
||||
@@ -150,10 +152,9 @@ TJWATER_MCP_MAX_REQUEST_TIMEOUT_SECONDS=240
|
||||
# 后台计算独立执行时限为 120–86400 秒,不受工具 timeout_seconds 控制。
|
||||
TJWATER_MCP_MAX_CALCULATION_TIMEOUT_SECONDS=3600
|
||||
|
||||
# 独立计算队列消费者;显式项目 UUID,以逗号分隔。不会在 API lifespan 内启动。
|
||||
# 先在每个目标项目 BizDB 依次应用 026_calculation_jobs.sql 和 027_calculation_job_lifecycle.sql,再启动计算服务。
|
||||
CALCULATION_WORKER_PROJECT_IDS=""
|
||||
# 所有配置项目合计并发,独立于 Uvicorn --workers 4;单实例锁防止重复消费者扩容。
|
||||
# 独立计算队列消费者;从 MetaDB 动态发现活动项目,不会在 API lifespan 内启动。
|
||||
# 先在每个项目 BizDB 应用 026、027 和 030 迁移,再启动计算服务。
|
||||
# 所有活动项目合计并发,独立于 Uvicorn --workers 4;单实例锁防止重复消费者扩容。
|
||||
CALCULATION_WORKER_MAX_CONCURRENCY=2
|
||||
CALCULATION_WORKER_POLL_SECONDS=2
|
||||
CALCULATION_WORKER_LEASE_SECONDS=30
|
||||
|
||||
@@ -12,7 +12,7 @@ Use the existing conda environment when available:
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests/unit tests/auth -q
|
||||
conda run -n server uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
conda run -n server python scripts/run_server.py --reload
|
||||
docker build -t tjwater-server:local .
|
||||
docker compose -f infra/docker/docker-compose.yml config
|
||||
```
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
`next-tjwater-server` 是 NextTJWater 供水系统的 FastAPI 后端,提供认证、项目、管网、模拟、爆管、漏损、SCADA、地图服务、工况任务和受控 MCP 能力。业务数据和时序数据按已认证项目路由,后台工作进程也必须使用明确的项目范围。
|
||||
|
||||
协议、工作进程、数据库、SCADA 与历史资料索引见 [文档目录](docs/README.md)。
|
||||
|
||||
## 技术栈
|
||||
|
||||
- Python 3.12
|
||||
@@ -42,7 +44,7 @@ cp .env.example .env
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests/unit tests/auth -q
|
||||
conda run -n server uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
conda run -n server python scripts/run_server.py --reload
|
||||
```
|
||||
|
||||
如需要进入环境:
|
||||
@@ -84,11 +86,12 @@ CORS 匹配浏览器页面来源,不是 Nginx 连接 IP 或域名解析结果
|
||||
|
||||
## 业务 Skill 与闭环
|
||||
|
||||
[爆管处置工程图](docs/burst-response-workflow.md)覆盖上报、候选定位、隔离阀门、影响分析、地图与数据展示、审批处置和反馈闭环。Server 通过任务 MCP 按 workflow 分发以下业务 Skill,当前版本均为 `2.0.0`:
|
||||
[爆管事件处理流程](docs/burst-response-workflow.md)覆盖工单接收、资产关联、隔离阀门、影响说明、审批处置和反馈闭环。Server 按任务 workflow 与回放来源分发独立 Skill,版本以实际返回为准:
|
||||
|
||||
| workflow | Skill | 调查重点 |
|
||||
| --- | --- | --- |
|
||||
| burst_report | [爆管处置](resources/work-order-skills/burst-report/SKILL.md) | 候选定位、隔离阀门、影响分析、抢修及恢复闭环 |
|
||||
| burst_report(工单) | [爆管事件处理](resources/work-order-skills/burst-event-handling/SKILL.md) | 资产关联、隔离方案、影响说明、处置及恢复闭环 |
|
||||
| scada_anomaly/历史回放 | [SCADA 异常调查](resources/work-order-skills/scada-anomaly/SKILL.md) | 冻结数据诊断、候选定位与隔离影响分析 |
|
||||
| water_quality_complaint | [水质投诉](resources/work-order-skills/water-quality-complaint/SKILL.md) | 投诉关联、监测与采样、候选影响、处置及复测回访 |
|
||||
| general | [预约跟进](resources/work-order-skills/appointment-follow-up/SKILL.md) | 审批与回执、现场履约、反馈复核及与原事件的闭环区别 |
|
||||
|
||||
@@ -97,7 +100,7 @@ JSONL 整理遵守会话文件权限,后台图表和表格通过既有报告 `
|
||||
## 常用命令
|
||||
|
||||
外围接入样板作为 `/peripheral` 子应用随主 API 进程启动,复用 PostgreSQL 且不自动建表;授权边界和契约见 [外围样板说明](docs/peripheral-sample.md)。
|
||||
外围样板提供 synthetic 测试工单和测试数据的版本化 CRUD;工单经同步器投影为 `workbench.operational_tasks` 中的外围来源任务。固定演示时窗和造数方式见 [合成样本说明](docs/peripheral-generated-data.md),运行态读取边界见 [工作台运行态说明](docs/workbench-runtime.md)。这些数据不代表现场事实,也不会触发设备控制。
|
||||
外围系统提供业务工单和反馈的版本化 CRUD,工单来源固定为 `system_data`,经同步器投影为 `workbench.operational_tasks` 中的外围来源任务。系统上报与已核实事实分别记录,现场处置通过审批流程执行。独立测试数据和监测演示的固定时窗及造数方式见 [合成样本说明](docs/peripheral-generated-data.md),运行态读取边界见 [工作台运行态说明](docs/workbench-runtime.md)。
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests -q
|
||||
@@ -111,6 +114,14 @@ docker compose -f infra/docker/docker-compose.yml config
|
||||
- `docker build`:构建后端镜像。
|
||||
- `docker compose config`:检查 compose 配置和变量展开。
|
||||
|
||||
开发时使用 `python scripts/run_server.py --reload` 启用进程交接式重载:父进程持有监听端口,
|
||||
候选进程完成 ASGI 初始化、通过元数据库就绪检查并接管监听后,才通知旧进程停止接收新连接。
|
||||
旧 HTTP 请求和 SSE 连接继续由旧进程完成,期间新请求由新进程响应;不迁移已建立的 TCP 连接,
|
||||
也不通过强制取消旧请求来加快重载。候选启动失败或超出 `--startup-timeout`(默认 60 秒)时,
|
||||
保留旧进程继续服务。快速连续修改最多保留两个正在排空的旧进程,其余修改合并后再重载。
|
||||
默认监视 `app/` 和 `samples/peripheral/` 下的 Python 文件,可重复传入 `--reload-dir` 指定目录。
|
||||
直接执行 `uvicorn ... --reload` 仍使用其原有先停后启流程;已经运行的旧启动命令不会自动切换到新机制。
|
||||
|
||||
## 开发规范
|
||||
|
||||
- Python 文件、函数、变量、Pydantic 字段、JSON body 字段和 query 参数使用 `snake_case`。
|
||||
@@ -204,7 +215,7 @@ Gitea Actions 工作流位于 `.gitea/workflows/package.yml`,可手动触发
|
||||
|
||||
## 产品 MCP 查询
|
||||
|
||||
Server 可选启用 MCP 2026-07-28,默认关闭。当前目录包含 67 项类型化工具,其中 42 项查询、12 项计算、13 项预览式写入。MCP 使用专用 OAuth resource Bearer、宿主绑定上下文和现有项目或任务权限,不提供自由命令执行。
|
||||
Server 可选启用 MCP 2026-07-28,默认关闭。目录包含查询、分析计算、预览式写入和计算任务管理工具;可用项以 `app/mcp/catalog.py` 及当前授权目录为准。MCP 使用专用 OAuth resource Bearer、宿主绑定上下文和现有项目或任务权限,不提供自由命令执行。
|
||||
|
||||
写入工具默认只执行 `dry_run`。真实写入必须使用同一 `request_id`,带上预览返回的 `expected_digest`,并通过宿主原生许可和项目权限检查。超时或结果不明时先查询操作状态,不盲目重试。实施、启用和授权边界见 [MCP 实施与启用](docs/mcp-query-implementation.md) 和 [协议与旧 CLI 能力对照](docs/mcp-query-design.md)。
|
||||
|
||||
|
||||
@@ -49,6 +49,7 @@ def _project_response(project: models.Project) -> AdminProjectResponse:
|
||||
gs_workspace=project.gs_workspace,
|
||||
map_extent=project.map_extent,
|
||||
map_config=getattr(project, "map_config", {}) or {},
|
||||
settings=getattr(project, "settings", {}) or {},
|
||||
status=project.status,
|
||||
created_at=project.created_at,
|
||||
updated_at=project.updated_at,
|
||||
@@ -276,6 +277,7 @@ async def create_admin_project(
|
||||
gs_workspace=payload.gs_workspace,
|
||||
map_extent=payload.map_extent,
|
||||
map_config=payload.map_config.model_dump(exclude_unset=True),
|
||||
settings=payload.settings.model_dump(exclude_unset=True),
|
||||
status=payload.status,
|
||||
creator_user_id=current_user.id,
|
||||
)
|
||||
|
||||
@@ -45,6 +45,7 @@ async def get_project_metadata(
|
||||
gs_workspace=project.gs_workspace,
|
||||
map_extent=project.map_extent,
|
||||
map_config=getattr(project, "map_config", {}) or {},
|
||||
settings=getattr(project, "settings", {}) or {},
|
||||
status=project.status,
|
||||
project_role=ctx.project_role,
|
||||
)
|
||||
|
||||
@@ -320,6 +320,7 @@ async def provision_project(
|
||||
gs_workspace=project.gs_workspace,
|
||||
map_extent=project.map_extent,
|
||||
map_config=getattr(project, "map_config", {}) or {},
|
||||
settings=getattr(project, "settings", {}) or {},
|
||||
status=project.status,
|
||||
created_at=project.created_at,
|
||||
updated_at=project.updated_at,
|
||||
|
||||
@@ -7,7 +7,9 @@ from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Query, status
|
||||
from psycopg import AsyncConnection
|
||||
from app.auth.metadata_dependencies import get_metadata_repository
|
||||
|
||||
from app.core.config import settings
|
||||
from app.auth.permissions import (
|
||||
OPERATIONAL_TASK_EXECUTE,
|
||||
OPERATIONAL_TASK_EXTERNAL_SYNC,
|
||||
@@ -22,6 +24,7 @@ from app.auth.project_dependencies import (
|
||||
ProjectContext,
|
||||
get_project_context,
|
||||
get_project_pg_connection,
|
||||
get_project_timescale_connection,
|
||||
)
|
||||
from app.domain.domain_errors import DomainError
|
||||
from app.domain.schemas.workbench_runtime import WorkbenchDateTime
|
||||
@@ -90,11 +93,37 @@ from app.services.operational_tasks import OperationalTaskService
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/operational-tasks/recovery/reconcile")
|
||||
async def reconcile_task_recoveries(
|
||||
command: RecoverExpiredLeasesCommand,
|
||||
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
|
||||
conn: AsyncConnection = Depends(get_project_pg_connection),
|
||||
timescale=Depends(get_project_timescale_connection),
|
||||
metadata=Depends(get_metadata_repository),
|
||||
) -> dict[str, int]:
|
||||
from app.domain.schemas.project_settings import ProjectSettings
|
||||
from app.services.operational_recovery import reconcile_recoveries
|
||||
|
||||
if SCADA_VIEW not in permissions_for_context(ctx):
|
||||
raise HTTPException(403, "恢复监测需要 SCADA 查看权限")
|
||||
project = await metadata.get_project_by_id(ctx.project_id)
|
||||
if project is None:
|
||||
raise HTTPException(404, "当前项目不存在")
|
||||
try:
|
||||
resumed = await reconcile_recoveries(_service(ctx, conn), timescale,
|
||||
ProjectSettings.model_validate(project.settings or {}), limit=command.limit,
|
||||
created_from=command.task_created_from)
|
||||
return {"resumed": resumed}
|
||||
except DomainError as exc:
|
||||
_raise_domain(exc)
|
||||
|
||||
|
||||
def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskService:
|
||||
return OperationalTaskService(
|
||||
OperationalTaskRepository(conn, ctx.project_id),
|
||||
ctx,
|
||||
_task_permissions(ctx),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
|
||||
|
||||
@@ -567,10 +596,24 @@ async def query_operational_task_data(
|
||||
if not await service.can_read_source(task_id, query.source):
|
||||
raise DomainError("PERMISSION_DENIED", "Task has no approved access to this data source", status=403)
|
||||
source_permission = SCADA_VIEW if query.source.startswith("scada_") else SIMULATION_VIEW
|
||||
snapshot = await read_task_data(
|
||||
context=ctx, query=query, credential=authorization,
|
||||
delegated=source_permission not in _task_permissions(ctx),
|
||||
)
|
||||
from app.services.scada_replay_queries import replay_snapshot_rows
|
||||
replay_context = await service.replay_context(task_id)
|
||||
if replay_context:
|
||||
if query.source != "scada_readings":
|
||||
raise HTTPException(403, "回放调查数据接口仅开放本轮 SCADA 快照,请通过 MCP 查询设备目录")
|
||||
rows = await replay_snapshot_rows(conn, ctx.project_id, replay_context, query.device_ids,
|
||||
query.start, query.end, 1001)
|
||||
from app.services.content_digest import digest
|
||||
from datetime import datetime, timezone
|
||||
snapshot = TaskDataSnapshot(
|
||||
source="scada_readings", source_ref=f"scada_replay:{replay_context['replay_id']}:{digest(rows)}",
|
||||
queried_at=datetime.now(timezone.utc), origin="synthetic", data=rows[:1000], truncated=len(rows) > 1000,
|
||||
)
|
||||
else:
|
||||
snapshot = await read_task_data(
|
||||
context=ctx, query=query, credential=authorization,
|
||||
delegated=source_permission not in _task_permissions(ctx),
|
||||
)
|
||||
evidence, stored_snapshot = await service.record_data_snapshot(task_id, snapshot)
|
||||
return TaskDataQueryResult(evidence=evidence, snapshot=stored_snapshot)
|
||||
except TaskDataError as exc:
|
||||
|
||||
@@ -35,6 +35,7 @@ async def get_project_info_endpoint(
|
||||
gs_workspace=project_detail.gs_workspace,
|
||||
map_extent=project_detail.map_extent,
|
||||
map_config=getattr(project_detail, "map_config", {}) or {},
|
||||
settings=getattr(project_detail, "settings", {}) or {},
|
||||
status=project_detail.status,
|
||||
project_role="viewer", # Default role for public access
|
||||
)
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
"""SCADA replay controls. Manual steps are processed by the command request."""
|
||||
from uuid import UUID
|
||||
from datetime import datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, HTTPException
|
||||
|
||||
from app.auth.metadata_dependencies import get_metadata_repository
|
||||
from app.auth.permissions import SCADA_VIEW, SCADA_CLEAN, require_permission, permissions_for_context
|
||||
from app.auth.project_dependencies import ProjectContext, get_project_pg_connection, get_project_timescale_connection
|
||||
from app.domain.schemas.scada_replay import (
|
||||
ScadaReplayCapabilities, ScadaReplayCreate, ScadaReplayCommand, ScadaReplaySnapshot,
|
||||
ScadaReplayEventsPage, ScadaReplayReadings,
|
||||
)
|
||||
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
|
||||
from app.domain.schemas.project_settings import ProjectSettings
|
||||
from app.services.scada_replay import ScadaReplayService, snapshot
|
||||
|
||||
router = APIRouter(prefix="/scada-replays", tags=["SCADA Replay"])
|
||||
|
||||
|
||||
async def repository(
|
||||
ctx: ProjectContext = Depends(require_permission(SCADA_VIEW)),
|
||||
connection=Depends(get_project_pg_connection),
|
||||
):
|
||||
return ScadaReplayRepository(connection, ctx.project_id)
|
||||
|
||||
|
||||
async def project_settings(
|
||||
ctx: ProjectContext = Depends(require_permission(SCADA_VIEW)),
|
||||
metadata_repo=Depends(get_metadata_repository),
|
||||
) -> ProjectSettings:
|
||||
project = await metadata_repo.get_project_by_id(ctx.project_id)
|
||||
if project is None:
|
||||
raise HTTPException(404, "当前项目不存在")
|
||||
return ProjectSettings.model_validate(project.settings or {})
|
||||
|
||||
|
||||
@router.get("/capabilities", response_model=ScadaReplayCapabilities)
|
||||
async def replay_capabilities(repo=Depends(repository), ctx=Depends(require_permission(SCADA_VIEW)),
|
||||
settings=Depends(project_settings)):
|
||||
available = await repo.available()
|
||||
current = await repo.current() if available else None
|
||||
monitoring = settings.scada_monitoring
|
||||
monitoring_ready = bool(
|
||||
monitoring
|
||||
and len(monitoring.pressure_device_ids) >= 5
|
||||
and (monitoring.decision_policy == "weighted" or monitoring.outlet_flow_device_ids)
|
||||
)
|
||||
if monitoring_ready:
|
||||
monitoring_reason = None
|
||||
elif monitoring is None:
|
||||
monitoring_reason = "项目尚未配置压力监测角色及组合判定规则"
|
||||
elif len(monitoring.pressure_device_ids) < 5:
|
||||
monitoring_reason = "爆管侦测至少需要 5 个压力监测设备"
|
||||
else:
|
||||
monitoring_reason = "项目尚未配置出厂边界流量设备"
|
||||
return ScadaReplayCapabilities(
|
||||
available=available, can_write=SCADA_CLEAN in permissions_for_context(ctx),
|
||||
reason=None if available else "需先部署 SCADA 回放存储结构",
|
||||
monitoring_interval_seconds=settings.scada_monitoring_interval_seconds,
|
||||
burst_monitoring_ready=monitoring_ready,
|
||||
burst_monitoring_reason=monitoring_reason,
|
||||
monitoring_config=monitoring,
|
||||
current=snapshot(current) if current else None,
|
||||
)
|
||||
|
||||
|
||||
@router.post("", response_model=ScadaReplaySnapshot, status_code=201)
|
||||
async def create_replay(command: ScadaReplayCreate, repo=Depends(repository),
|
||||
ctx=Depends(require_permission(SCADA_CLEAN)), settings=Depends(project_settings)):
|
||||
return await ScadaReplayService(repo, ctx).create(
|
||||
command,
|
||||
monitoring_interval_seconds=settings.scada_monitoring_interval_seconds,
|
||||
monitoring_config=settings.scada_monitoring,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{replay_id}", response_model=ScadaReplaySnapshot)
|
||||
async def get_replay(replay_id: UUID, repo=Depends(repository)):
|
||||
await ScadaReplayService(repo, None).ensure_storage()
|
||||
return snapshot(await repo.get(replay_id))
|
||||
|
||||
|
||||
@router.post("/{replay_id}/commands", response_model=ScadaReplaySnapshot)
|
||||
async def replay_command(replay_id: UUID, command: ScadaReplayCommand, repo=Depends(repository),
|
||||
ctx=Depends(require_permission(SCADA_CLEAN)),
|
||||
timescale=Depends(get_project_timescale_connection)):
|
||||
service = ScadaReplayService(repo, ctx, timescale)
|
||||
result = await service.command(replay_id, command)
|
||||
if command.action in {"step", "monitor", "retry"} and result.step_pending:
|
||||
return await service.process_pending_step(replay_id)
|
||||
return result
|
||||
|
||||
|
||||
@router.get("/{replay_id}/events", response_model=ScadaReplayEventsPage)
|
||||
async def replay_events(replay_id: UUID, offset: int = Query(0, ge=0), limit: int = Query(100, ge=1, le=200),
|
||||
repo=Depends(repository)):
|
||||
await ScadaReplayService(repo, None).ensure_storage()
|
||||
await repo.get(replay_id)
|
||||
items = await repo.events(replay_id, offset, limit + 1)
|
||||
return ScadaReplayEventsPage(items=items[:limit], next_offset=offset + limit if len(items) > limit else None)
|
||||
|
||||
|
||||
@router.get("/{replay_id}/observations", response_model=ScadaReplayReadings)
|
||||
async def replay_observations(replay_id: UUID, through: datetime, repo=Depends(repository)):
|
||||
await ScadaReplayService(repo, None).ensure_storage()
|
||||
document = await repo.get(replay_id)
|
||||
if (through.tzinfo is None or not document.get("delivered_until")
|
||||
or through > datetime.fromisoformat(document["delivered_until"])
|
||||
or through < datetime.fromisoformat(document["start_time"])):
|
||||
raise HTTPException(422, "回看仅允许本轮已读取的时间区间")
|
||||
return ScadaReplayReadings(
|
||||
items=await repo.latest_outputs(replay_id, through),
|
||||
through=through,
|
||||
)
|
||||
@@ -1,10 +1,13 @@
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Path, Body
|
||||
from typing import List
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
from psycopg import AsyncConnection
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from app.domain.schemas.scada import ScadaLatestReadingsResponse
|
||||
from app.domain.schemas.scada import ScadaLatestReadingsResponse, ScadaRawPage
|
||||
from app.auth.project_dependencies import get_project_context, ProjectContext
|
||||
from app.services.scada_raw import read_raw_page, validate_window
|
||||
from app.infra.db.postgresql.scada import ScadaInfoRepository
|
||||
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
from .dependencies import get_postgres_connection, get_timescale_connection
|
||||
@@ -13,6 +16,46 @@ router = APIRouter()
|
||||
SCADA_BATCH_MAX_ITEMS = 10_000
|
||||
|
||||
|
||||
@router.get("/timeseries/scada-readings/raw", response_model=ScadaRawPage,
|
||||
summary="按时间顺序分页读取 SCADA 原始值")
|
||||
async def get_scada_raw(
|
||||
start_time: datetime, end_time: datetime, device_ids: str,
|
||||
limit: int = Query(500, ge=1, le=500),
|
||||
cursor: str | None = Query(None, max_length=2048),
|
||||
replay_id: UUID | None = None,
|
||||
conn: AsyncConnection = Depends(get_timescale_connection),
|
||||
postgres_conn: AsyncConnection = Depends(get_postgres_connection),
|
||||
ctx: ProjectContext = Depends(get_project_context),
|
||||
) -> ScadaRawPage:
|
||||
try:
|
||||
validate_window(start_time, end_time)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(422, str(exc)) from exc
|
||||
ids = list(dict.fromkeys(item.strip() for item in device_ids.split(",") if item.strip()))
|
||||
if not ids or len(ids) > 500:
|
||||
raise HTTPException(422, "请选择 1 至 500 个 SCADA 设备")
|
||||
existing = await ScadaInfoRepository.get_existing_device_ids(postgres_conn, ids)
|
||||
if set(ids) != existing:
|
||||
raise HTTPException(422, "查询包含当前项目未配置的 SCADA 设备")
|
||||
try:
|
||||
through = None
|
||||
if replay_id is not None:
|
||||
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
|
||||
from app.services.scada_replay import ScadaReplayService
|
||||
repo = ScadaReplayRepository(postgres_conn, ctx.project_id)
|
||||
await ScadaReplayService(repo, ctx).ensure_storage()
|
||||
replay = await repo.get(replay_id)
|
||||
if not set(ids) <= set(replay["config"]["device_ids"]):
|
||||
raise HTTPException(403, "设备超出回放范围")
|
||||
if start_time < datetime.fromisoformat(replay["start_time"]) or end_time > datetime.fromisoformat(replay["end_time"]):
|
||||
raise HTTPException(422, "查询超出选定回放区间")
|
||||
through = datetime.fromisoformat(replay["virtual_now"])
|
||||
return await read_raw_page(conn, project_id=str(ctx.project_id), device_ids=ids,
|
||||
start=start_time, end=end_time, limit=limit, cursor=cursor, through=through)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(422, str(exc)) from exc
|
||||
|
||||
|
||||
class ScadaReadingBatchItem(BaseModel):
|
||||
time: datetime
|
||||
device_id: str = Field(min_length=1)
|
||||
|
||||
@@ -36,6 +36,7 @@ def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskServi
|
||||
OperationalTaskRepository(conn, ctx.project_id),
|
||||
ctx,
|
||||
permissions_for_context(ctx),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -61,8 +61,10 @@ from app.auth.permissions import (
|
||||
require_method_permission,
|
||||
require_permission,
|
||||
)
|
||||
from app.api.v1.endpoints import scada_replay
|
||||
|
||||
api_router = APIRouter()
|
||||
api_router.include_router(scada_replay.router)
|
||||
|
||||
webgis_access = Depends(
|
||||
require_method_permission(
|
||||
|
||||
@@ -8,6 +8,12 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
class Settings(BaseSettings):
|
||||
PROJECT_NAME: str = "TJWater Server"
|
||||
ENVIRONMENT: str = "production"
|
||||
OPERATIONAL_TASK_EXECUTION_SCOPE: str = ""
|
||||
|
||||
@property
|
||||
def operational_task_execution_scope(self) -> str:
|
||||
return self.OPERATIONAL_TASK_EXECUTION_SCOPE.strip() or self.ENVIRONMENT
|
||||
|
||||
API_V1_STR: str = "/api/v1"
|
||||
NETWORK_NAME: str = "default_network"
|
||||
# 敏感配置加密密钥 (Fernet)
|
||||
|
||||
@@ -2,9 +2,10 @@ from datetime import datetime
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
|
||||
|
||||
from app.domain.schemas.project_map_config import ProjectMapConfig, ProjectMapConfigUpdate
|
||||
from app.domain.schemas.project_settings import ProjectSettings
|
||||
|
||||
|
||||
BusinessRole = Literal["admin", "service", "user"]
|
||||
@@ -72,6 +73,7 @@ class ProjectMemberResponse(BaseModel):
|
||||
|
||||
|
||||
class AdminProjectCreateRequest(BaseModel):
|
||||
settings: ProjectSettings = Field(default_factory=ProjectSettings)
|
||||
name: str = Field(..., min_length=1, max_length=100)
|
||||
code: str = Field(..., min_length=1, max_length=50)
|
||||
description: str | None = None
|
||||
@@ -82,6 +84,10 @@ class AdminProjectCreateRequest(BaseModel):
|
||||
|
||||
|
||||
class AdminProjectUpdateRequest(ProjectMapConfigUpdate):
|
||||
settings: ProjectSettings | None = Field(
|
||||
default=None,
|
||||
description="按顶层键合并项目配置;省略的键保持原值,嵌套对象整体替换",
|
||||
)
|
||||
name: str | None = Field(default=None, min_length=1, max_length=100)
|
||||
code: str | None = Field(default=None, min_length=1, max_length=50)
|
||||
description: str | None = None
|
||||
@@ -89,8 +95,16 @@ class AdminProjectUpdateRequest(ProjectMapConfigUpdate):
|
||||
map_extent: dict | None = None
|
||||
status: ProjectStatus | None = None
|
||||
|
||||
@field_validator("settings")
|
||||
@classmethod
|
||||
def reject_null_settings(cls, value: ProjectSettings | None) -> ProjectSettings:
|
||||
if value is None:
|
||||
raise ValueError("settings must be an object")
|
||||
return value
|
||||
|
||||
|
||||
class AdminProjectResponse(BaseModel):
|
||||
settings: ProjectSettings = Field(default_factory=ProjectSettings)
|
||||
project_id: UUID
|
||||
name: str
|
||||
code: str
|
||||
|
||||
@@ -3,12 +3,28 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from jsonschema import Draft7Validator
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_serializer, model_validator
|
||||
|
||||
CHART_SAMPLE_TIME_PATTERN = (
|
||||
r"^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9]"
|
||||
r"(?:\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
)
|
||||
|
||||
|
||||
def validate_chart_sample_time(value: str) -> None:
|
||||
"""Match the bounded frontend RFC3339 spelling and an actual calendar date."""
|
||||
if len(value) > 64 or re.fullmatch(CHART_SAMPLE_TIME_PATTERN, value) is None:
|
||||
raise ValueError("Chart sample times require RFC3339 seconds and a timezone")
|
||||
if datetime.fromisoformat(value).tzinfo is None:
|
||||
raise ValueError("Chart sample times require a timezone")
|
||||
|
||||
|
||||
_catalog = json.loads(
|
||||
(Path(__file__).resolve().parents[3] / "contracts/answer-ui-catalog.json").read_text()
|
||||
)
|
||||
@@ -71,8 +87,18 @@ class AnswerUiBlock(UiModel):
|
||||
if element.type == "Chart":
|
||||
if any(len(series["data"]) != len(props["x_data"]) for series in props["series"]):
|
||||
raise ValueError("Chart series must match the axis")
|
||||
if any(not math.isfinite(value) for series in props["series"] for value in series["data"]):
|
||||
if any(value is not None and not math.isfinite(value) for series in props["series"] for value in series["data"]):
|
||||
raise ValueError("Chart values must be finite")
|
||||
for series in props["series"]:
|
||||
times = series.get("sample_times")
|
||||
if times is not None:
|
||||
if len(times) != len(series["data"]):
|
||||
raise ValueError("Chart sample times must match the axis")
|
||||
for value, at in zip(series["data"], times, strict=True):
|
||||
if at is not None:
|
||||
if value is None:
|
||||
raise ValueError("Chart sample times require an observed value")
|
||||
validate_chart_sample_time(at)
|
||||
if element.type == "Table" and any(len(row) != len(props["columns"]) for row in props["rows"]):
|
||||
raise ValueError("Table row width must match its columns")
|
||||
if element.type == "FollowUpForm":
|
||||
|
||||
@@ -4,9 +4,11 @@ from uuid import UUID
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from app.domain.schemas.project_map_config import ProjectMapConfig
|
||||
from app.domain.schemas.project_settings import ProjectSettings
|
||||
|
||||
|
||||
class ProjectMetaResponse(BaseModel):
|
||||
settings: ProjectSettings = Field(default_factory=ProjectSettings)
|
||||
project_id: UUID
|
||||
name: str
|
||||
code: str
|
||||
|
||||
@@ -51,6 +51,7 @@ McpSnapshotSource = Literal[
|
||||
'cancel_task_run',
|
||||
'run_simulation',
|
||||
'run_contaminant_simulation',
|
||||
'run_contaminant_source_trace',
|
||||
'analyze_valve_closure',
|
||||
'analyze_flushing',
|
||||
'analyze_pressure_regulation',
|
||||
@@ -58,6 +59,7 @@ McpSnapshotSource = Literal[
|
||||
'run_burst_detection',
|
||||
'run_burst_location',
|
||||
'analyze_valve_isolation',
|
||||
'generate_node_coverage',
|
||||
'run_leakage_identification',
|
||||
'get_element_properties',
|
||||
'list_elements',
|
||||
|
||||
@@ -5,11 +5,16 @@ import re
|
||||
from typing import Annotated, Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import Field, JsonValue, model_validator
|
||||
from app.domain.schemas.answer_ui import ReportUiBlock, validate_report_ui_blocks
|
||||
from pydantic import Field, JsonValue, StringConstraints, field_validator, model_serializer, model_validator
|
||||
from app.domain.schemas.answer_ui import (
|
||||
CHART_SAMPLE_TIME_PATTERN, ReportUiBlock, validate_chart_sample_time, validate_report_ui_blocks,
|
||||
)
|
||||
|
||||
from app.domain.schemas.task_question import TaskQuestionRequest
|
||||
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictBool, StrictModel
|
||||
from app.domain.schemas.report_authoring import (
|
||||
validate_presentation_command_text, validate_report_text, validate_report_ui_text,
|
||||
)
|
||||
from app.domain.schemas.operational_data import TaskDataSnapshot, TaskDataSource
|
||||
from app.domain.schemas.workbench_runtime import (
|
||||
WorkbenchDateTime,
|
||||
@@ -41,12 +46,19 @@ ExecutionStatus = Literal[
|
||||
]
|
||||
|
||||
|
||||
class ScadaReplayTaskContext(StrictModel):
|
||||
replay_id: UUID
|
||||
knowledge_cutoff: WorkbenchDateTime
|
||||
batch_sequence: PositiveInt
|
||||
|
||||
|
||||
class TaskSource(StrictModel):
|
||||
kind: TaskSourceKind
|
||||
external_id: NonEmptyStr | None = None
|
||||
external_revision: PositiveInt | None = None
|
||||
event_kind: NonEmptyStr | None = None
|
||||
deleted_at: WorkbenchDateTime | None = None
|
||||
replay_context: ScadaReplayTaskContext | None = None
|
||||
|
||||
|
||||
class TaskTrigger(StrictModel):
|
||||
@@ -213,25 +225,65 @@ class InvestigationMapTarget(StrictModel):
|
||||
feature_id: TaskFeatureId
|
||||
|
||||
|
||||
class InvestigationDiagramNodeRef(StrictModel):
|
||||
InvestigationNodeAnnotation = Literal["reference", "recorded", "needs_input", "failed", "not_taken"]
|
||||
LEGACY_INVESTIGATION_NODE_ANNOTATIONS = {
|
||||
"pending": "reference", "running": "reference", "completed": "recorded",
|
||||
"waiting": "needs_input", "waiting_human": "needs_input",
|
||||
"skipped": "not_taken", "cancelled": "not_taken",
|
||||
}
|
||||
|
||||
|
||||
class _InvestigationDiagramNode(StrictModel):
|
||||
node_id: Annotated[str, Field(pattern=r"^[A-Za-z_][A-Za-z0-9_-]{0,79}$")]
|
||||
label: NonEmptyStr
|
||||
detail: Annotated[str, Field(min_length=1, max_length=2000)] | None = None
|
||||
evidence_refs: list[NonEmptyStr] = Field(default_factory=list)
|
||||
action_refs: list[UUID] = Field(default_factory=list)
|
||||
status: Literal[
|
||||
"pending", "running", "completed", "waiting", "waiting_human", "failed", "skipped", "cancelled"
|
||||
] | None = Field(default=None, description=(
|
||||
"Evidence-confirmed node state; waiting means prerequisites/external/system result, "
|
||||
"waiting_human requires manual authorization/approval/feedback. "
|
||||
"Unknown stays omitted/null; host controls colors."
|
||||
))
|
||||
map_targets: list[InvestigationMapTarget] = Field(default_factory=list, max_length=100)
|
||||
|
||||
|
||||
class InvestigationDiagramNodeRef(_InvestigationDiagramNode):
|
||||
status: InvestigationNodeAnnotation | None = Field(default=None, description=(
|
||||
"A saved process-reference annotation, not live execution state. "
|
||||
"recorded means a finding exists, not that a task or field action completed."
|
||||
))
|
||||
|
||||
@field_validator("status", mode="before")
|
||||
@classmethod
|
||||
def read_legacy_annotation(cls, value: object) -> object:
|
||||
return LEGACY_INVESTIGATION_NODE_ANNOTATIONS.get(value, value) if isinstance(value, str) else value
|
||||
|
||||
|
||||
class InvestigationGraphNodeUpdate(_InvestigationDiagramNode):
|
||||
label: NonEmptyStr | None = Field(default=None, description=(
|
||||
"Required for a new node. Omit or use null to retain an existing node's label. "
|
||||
"Use the same node_id to revise status or detail without adding another step."
|
||||
))
|
||||
status: InvestigationNodeAnnotation | None = Field(default=None, description=(
|
||||
"Optional reference annotation: reference, recorded finding, needs_input, failed attempt, or not_taken. "
|
||||
"No live node/task binding or per-step synchronization is required."
|
||||
))
|
||||
|
||||
|
||||
class InvestigationGraphEdge(StrictModel):
|
||||
source: Annotated[str, Field(pattern=r"^[A-Za-z_][A-Za-z0-9_-]{0,79}$")]
|
||||
target: Annotated[str, Field(pattern=r"^[A-Za-z_][A-Za-z0-9_-]{0,79}$")]
|
||||
label: Annotated[str, Field(min_length=1, max_length=200)] | None = None
|
||||
|
||||
|
||||
class InvestigationGraphUpdate(StrictModel):
|
||||
title: Annotated[str, Field(min_length=1, max_length=200)] | None = None
|
||||
nodes: list[InvestigationGraphNodeUpdate] = Field(default_factory=list, max_length=100)
|
||||
edges: list[InvestigationGraphEdge] = Field(default_factory=list, max_length=200)
|
||||
|
||||
|
||||
class InvestigationDiagram(StrictModel):
|
||||
title: NonEmptyStr
|
||||
mermaid: Annotated[str, Field(min_length=1, max_length=20000)]
|
||||
node_refs: list[InvestigationDiagramNodeRef] = Field(default_factory=list, max_length=100)
|
||||
# Present only on a graph maintained by the host. Legacy Mermaid stays readable.
|
||||
edges: list[InvestigationGraphEdge] | None = Field(default=None, max_length=200)
|
||||
source_mermaid: Annotated[str, Field(min_length=1, max_length=20000)] | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_description(self):
|
||||
@@ -252,7 +304,7 @@ def _undeclared_diagram_node_refs(diagram: InvestigationDiagram | None) -> list[
|
||||
for node in diagram.node_refs:
|
||||
escaped = re.escape(node.node_id)
|
||||
if not re.search(
|
||||
rf"(?<![A-Za-z0-9_-]){escaped}(?=\s*(?:[\[({{@;]|--|==|-\.|$))",
|
||||
rf"(?<![A-Za-z0-9_-]){escaped}(?=\s*(?:[\[({{@;]|:::|--|==|-\.|$))",
|
||||
diagram.mermaid,
|
||||
re.MULTILINE,
|
||||
):
|
||||
@@ -275,6 +327,7 @@ class InvestigationReport(StrictModel):
|
||||
data_gaps: list[InvestigationDataGap] | None = None
|
||||
diagram: InvestigationDiagram | None = None
|
||||
ui_blocks: list["ReportUiBlock"] = Field(default_factory=list, max_length=8)
|
||||
scada_device_ids: list[TaskFeatureId] | None = Field(default=None, max_length=1000)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_result(self):
|
||||
@@ -299,12 +352,44 @@ class InvestigationReport(StrictModel):
|
||||
return self
|
||||
|
||||
|
||||
class InvestigationDataGapInput(InvestigationDataGap):
|
||||
"""New gap descriptions use business prose; references remain opaque."""
|
||||
|
||||
missing_data: NonEmptyStr = Field(max_length=4000, description="缺失的业务资料,不写执行字段")
|
||||
scope: NonEmptyStr = Field(max_length=4000, description="缺口涉及的对象、地点与当地时间范围")
|
||||
impact: NonEmptyStr = Field(max_length=4000, description="资料缺项对当前判断的实际影响")
|
||||
required_data: NonEmptyStr = Field(max_length=4000, description="需要补充的具体业务资料")
|
||||
|
||||
@field_validator("missing_data", "scope", "impact", "required_data")
|
||||
@classmethod
|
||||
def validate_reader_text(cls, value: str) -> str:
|
||||
return validate_report_text(value)
|
||||
|
||||
|
||||
class InvestigationReportInput(InvestigationReport):
|
||||
"""Write model: every new report explicitly declares its business result."""
|
||||
|
||||
title: NonEmptyStr = Field(max_length=80, description="简短中文业务标题,写明对象和当前结果,不含执行轮次、流程版本或原始时间戳")
|
||||
current_conclusion: NonEmptyStr = Field(max_length=300, description="先写当前业务判断,再写关键依据与下一步,建议 150 字以内;完整数值和对象比较使用 ui_blocks.Table")
|
||||
conclusion_boundary: NonEmptyStr = Field(max_length=240, description="只写影响判断的证据范围、不确定性和适用限制,不复述结论、执行步骤或参数")
|
||||
recommendations: list[Annotated[NonEmptyStr, Field(max_length=160)]] = Field(default_factory=list)
|
||||
pending_items: list[Annotated[NonEmptyStr, Field(max_length=160)]] = Field(default_factory=list)
|
||||
stop_reason: Annotated[NonEmptyStr, Field(max_length=160)] | None = None
|
||||
stage: Literal["interim", "final"]
|
||||
outcome: Literal["concluded", "insufficient_data"]
|
||||
data_gaps: list[InvestigationDataGap]
|
||||
data_gaps: list[InvestigationDataGapInput]
|
||||
|
||||
@field_validator("title", "current_conclusion", "conclusion_boundary", "stop_reason")
|
||||
@classmethod
|
||||
def validate_reader_text(cls, value: str | None) -> str | None:
|
||||
return validate_report_text(value) if value is not None else value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_authoring(self):
|
||||
for value in [*self.recommendations, *self.pending_items]:
|
||||
validate_report_text(value)
|
||||
validate_report_ui_text(self.ui_blocks)
|
||||
return self
|
||||
|
||||
|
||||
class InvestigationProgress(StrictModel):
|
||||
@@ -314,6 +399,22 @@ class InvestigationProgress(StrictModel):
|
||||
diagram: InvestigationDiagram | None = None
|
||||
|
||||
|
||||
class InvestigationProgressInput(InvestigationProgress):
|
||||
summary: NonEmptyStr = Field(max_length=300, description="用简短中文说明本轮新增发现、当前依赖和下一步;不复制报告、执行字段、版本号或证据编号")
|
||||
graph_update: InvestigationGraphUpdate | None = None
|
||||
|
||||
@field_validator("summary")
|
||||
@classmethod
|
||||
def validate_reader_summary(cls, value: str) -> str:
|
||||
return validate_report_text(value)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_graph_input(self):
|
||||
if self.graph_update is not None and self.diagram is not None:
|
||||
raise ValueError("use graph_update without a generated Mermaid diagram")
|
||||
return self
|
||||
|
||||
|
||||
class InvestigationProgressSnapshot(StrictModel):
|
||||
revision: PositiveInt
|
||||
run_id: UUID
|
||||
@@ -358,7 +459,7 @@ class AgentArtifactsCommand(StrictModel):
|
||||
evidence: list[EvidenceInput] = Field(default_factory=list, max_length=100)
|
||||
plan: ExecutionPlanInput | None = None
|
||||
report: InvestigationReportInput | None = None
|
||||
progress: InvestigationProgress | None = None
|
||||
progress: InvestigationProgressInput | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_artifacts(self):
|
||||
@@ -368,6 +469,8 @@ class AgentArtifactsCommand(StrictModel):
|
||||
self.report.diagram if self.report is not None else None,
|
||||
self.progress.diagram if self.progress is not None else None,
|
||||
):
|
||||
if diagram is not None and (diagram.edges is not None or diagram.source_mermaid is not None):
|
||||
raise ValueError("structured graph metadata is host-owned; use progress.graph_update")
|
||||
missing = _undeclared_diagram_node_refs(diagram)
|
||||
if missing:
|
||||
raise ValueError(
|
||||
@@ -565,6 +668,7 @@ class OperationalTaskRun(StrictModel):
|
||||
|
||||
|
||||
class OperationalTaskSummary(StrictModel):
|
||||
execution_scope: NonEmptyStr
|
||||
project_id: UUID
|
||||
task_id: UUID
|
||||
task_type: TaskType
|
||||
@@ -581,6 +685,7 @@ class OperationalTaskSummary(StrictModel):
|
||||
execution_request_id: UUID
|
||||
title: NonEmptyStr
|
||||
summary: NonEmptyStr
|
||||
handling_context: dict[str, JsonValue] = Field(default_factory=dict)
|
||||
source: TaskSource
|
||||
trigger: TaskTrigger
|
||||
scheduled_for: WorkbenchDateTime | None
|
||||
@@ -683,17 +788,64 @@ class AgentArtifactsResult(StrictModel):
|
||||
progress_revision: PositiveInt | None = None
|
||||
|
||||
|
||||
class ScadaChartGap(StrictModel):
|
||||
device_id: TaskFeatureId
|
||||
day_offset: Annotated[int, Field(ge=0, le=7)]
|
||||
missing_count: Annotated[int, Field(ge=1, le=100)]
|
||||
reason: Literal["no_readings", "outside_frozen_scope", "outside_frozen_window", "no_scheduled_samples"]
|
||||
|
||||
|
||||
class ScadaChartContext(StrictModel):
|
||||
role: Literal["current", "comparison"]
|
||||
metric: Literal["pressure", "flow", "quality"]
|
||||
unit: Annotated[str, Field(min_length=1, max_length=128)]
|
||||
window_start: WorkbenchDateTime
|
||||
window_end: WorkbenchDateTime
|
||||
value_field: Literal["monitored_value", "cleaned_value"] = "monitored_value"
|
||||
gaps: list[ScadaChartGap] = Field(default_factory=list, max_length=8)
|
||||
|
||||
|
||||
class TaskSafeChartSeries(StrictModel):
|
||||
name: Annotated[str, Field(max_length=128)]
|
||||
data: Annotated[list[Annotated[float, Field(strict=True, allow_inf_nan=False)]], Field(min_length=1, max_length=100)]
|
||||
device_id: TaskFeatureId | None = None
|
||||
day_offset: Annotated[int, Field(ge=0, le=7)] | None = None
|
||||
type: Literal["line", "bar", "scatter"] | None = None
|
||||
data: Annotated[list[Annotated[float, Field(strict=True, allow_inf_nan=False)] | None], Field(min_length=1, max_length=100)]
|
||||
sample_times: Annotated[list[Annotated[str, Field(
|
||||
strict=True, max_length=64,
|
||||
pattern=CHART_SAMPLE_TIME_PATTERN,
|
||||
)] | None], Field(min_length=1, max_length=100)] | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def serialize_optional_fields(self, handler):
|
||||
return {key: value for key, value in handler(self).items() if value is not None}
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_sample_times(self):
|
||||
if self.sample_times is not None:
|
||||
if len(self.sample_times) != len(self.data):
|
||||
raise ValueError("chart sample_times must align with data")
|
||||
for value, at in zip(self.data, self.sample_times, strict=True):
|
||||
if at is not None:
|
||||
if value is None:
|
||||
raise ValueError("chart sample_times require an observed value")
|
||||
validate_chart_sample_time(at)
|
||||
return self
|
||||
|
||||
|
||||
class TaskSafeChart(StrictModel):
|
||||
title: Annotated[str, Field(max_length=200)]
|
||||
chart_type: Literal["line", "bar"]
|
||||
scada: ScadaChartContext | None = None
|
||||
chart_type: Literal["line", "bar", "time_comparison"]
|
||||
x_axis_name: Annotated[str, Field(max_length=128)] | None = None
|
||||
y_axis_name: Annotated[str, Field(max_length=128)] | None = None
|
||||
x_data: Annotated[list[Annotated[str, Field(max_length=128)]], Field(min_length=1, max_length=100)]
|
||||
series: Annotated[list[TaskSafeChartSeries], Field(min_length=1, max_length=8)]
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def serialize_optional_fields(self, handler):
|
||||
return {key: value for key, value in handler(self).items() if value is not None}
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_alignment(self):
|
||||
if any(len(series.data) != len(self.x_data) for series in self.series):
|
||||
@@ -797,11 +949,72 @@ TaskPresentationCommand = Annotated[
|
||||
]
|
||||
|
||||
|
||||
class TaskPresentationNarration(StrictModel):
|
||||
stage: Literal[
|
||||
"anomaly_detection", "evidence_verification", "asset_localization",
|
||||
"event_impact", "source_control", "isolation_plan", "flushing_plan",
|
||||
"field_feedback", "clearance_verification", "recovery_verification",
|
||||
]
|
||||
summary: Annotated[str, Field(strict=True, min_length=1, max_length=160)]
|
||||
evidence_refs: Annotated[list[TaskFeatureId], Field(min_length=1, max_length=100)]
|
||||
next_step: Annotated[str, Field(strict=True, min_length=1, max_length=120)] | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_narration(self):
|
||||
if not self.summary.strip() or (self.next_step is not None and not self.next_step.strip()):
|
||||
raise ValueError("presentation narration must contain readable text")
|
||||
if len(set(self.evidence_refs)) != len(self.evidence_refs):
|
||||
raise ValueError("presentation narration evidence IDs must be unique")
|
||||
return self
|
||||
|
||||
|
||||
TaskPresentationPurpose = Annotated[
|
||||
str, StringConstraints(strict=True, strip_whitespace=True, min_length=1, max_length=160),
|
||||
]
|
||||
|
||||
|
||||
class TaskPresentationNarrationInput(TaskPresentationNarration):
|
||||
"""New live findings use business prose; saved narration remains readable."""
|
||||
|
||||
summary: Annotated[str, Field(strict=True, min_length=1, max_length=160,
|
||||
description="本步已取得的关键发现,建议 80 字以内;区分观测、模型候选和现场反馈,不复述动作或核查目的")]
|
||||
next_step: Annotated[str, Field(strict=True, min_length=1, max_length=120,
|
||||
description="实际下一步或等待条件,建议 60 字以内;无后续时省略,不把展示当作现场执行")] | None = None
|
||||
|
||||
@field_validator("summary", "next_step")
|
||||
@classmethod
|
||||
def validate_reader_text(cls, value: str | None) -> str | None:
|
||||
return validate_report_text(value) if value is not None else value
|
||||
|
||||
|
||||
class TaskPresentationEventCommand(StrictModel):
|
||||
request_id: UUID
|
||||
run_id: UUID
|
||||
source_session_id: NonEmptyStr
|
||||
command: TaskPresentationCommand
|
||||
narration: TaskPresentationNarrationInput | None = None
|
||||
purpose: TaskPresentationPurpose | None = Field(default=None,
|
||||
description="一句中文说明本步为何查看这些对象,建议 60 字以内;不堆砌编号、执行参数或复述当前发现")
|
||||
|
||||
@field_validator("purpose")
|
||||
@classmethod
|
||||
def validate_reader_purpose(cls, value: str | None) -> str | None:
|
||||
return validate_report_text(value) if value is not None else value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_visible_labels(self):
|
||||
validate_presentation_command_text(self.command.model_dump())
|
||||
return self
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def serialize_legacy_command(self, handler):
|
||||
value = handler(self)
|
||||
# Keep old idempotency digests stable when no narration was supplied.
|
||||
if self.narration is None:
|
||||
value.pop("narration", None)
|
||||
if self.purpose is None:
|
||||
value.pop("purpose", None)
|
||||
return value
|
||||
|
||||
|
||||
class TaskPresentationEventResult(StrictModel):
|
||||
@@ -832,6 +1045,8 @@ class TaskPresentationHistoryItem(StrictModel):
|
||||
# Stored records are validated against the current command contract by the
|
||||
# reader; an obsolete record must not prevent reading the rest of the page.
|
||||
command: dict[str, JsonValue]
|
||||
narration: TaskPresentationNarration | None = None
|
||||
purpose: TaskPresentationPurpose | None = None
|
||||
|
||||
|
||||
class TaskPresentationHistoryPage(StrictModel):
|
||||
@@ -908,6 +1123,13 @@ class HumanEvidenceCommand(StrictModel):
|
||||
evidence: EvidenceInput
|
||||
|
||||
|
||||
class WorkOrderNotificationReference(StrictModel):
|
||||
source_event_id: NonEmptyStr
|
||||
source_revision: PositiveInt
|
||||
evidence_id: NonEmptyStr | None = None
|
||||
status: Literal["scheduled", "assigned", "in_progress", "awaiting_feedback", "completed", "cancelled", "failed"]
|
||||
|
||||
|
||||
class OperationalTaskNotification(StrictModel):
|
||||
notification_id: UUID
|
||||
task_id: UUID
|
||||
@@ -915,7 +1137,7 @@ class OperationalTaskNotification(StrictModel):
|
||||
"task_created", "action_pending", "report_updated", "task_completed",
|
||||
"task_failed", "run_delivery_degraded", "access_approval_pending",
|
||||
"action_approval_pending", "human_input_required",
|
||||
"external_coordination_failed"
|
||||
"external_coordination_failed", "work_order_feedback"
|
||||
]
|
||||
title: NonEmptyStr
|
||||
description: NonEmptyStr
|
||||
@@ -924,6 +1146,8 @@ class OperationalTaskNotification(StrictModel):
|
||||
subject_type: Literal["task", "run", "access_request", "action", "question"] = "task"
|
||||
subject_id: UUID | None = None
|
||||
report_reference: InvestigationReportReference | None = None
|
||||
work_order_action: HumanApprovalAction | None = None
|
||||
work_order_reference: WorkOrderNotificationReference | None = None
|
||||
resolved_at: WorkbenchDateTime | None = None
|
||||
resolution_reason: NonEmptyStr | None = None
|
||||
status: Literal["unread", "read"]
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
from math import isclose
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, JsonValue, model_validator
|
||||
|
||||
|
||||
class ScadaMonitoringWeights(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
burst_detection: float = Field(default=0.5, ge=0, le=1)
|
||||
pressure: float = Field(default=0.4, ge=0, le=1)
|
||||
outlet_flow: float = Field(default=0.1, ge=0, le=1)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def normalized(self):
|
||||
if not isclose(self.burst_detection + self.pressure + self.outlet_flow, 1.0, abs_tol=1e-9):
|
||||
raise ValueError("SCADA 三项监测权重之和必须为 1")
|
||||
return self
|
||||
|
||||
|
||||
class ScadaMonitoringSettings(BaseModel):
|
||||
"""Project-owned device roles used by the manual SCADA monitoring demo."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
rule_version: str = Field(
|
||||
default="burst-monitor-v2",
|
||||
min_length=1,
|
||||
max_length=80,
|
||||
pattern=r"^[a-zA-Z0-9_.-]+$",
|
||||
)
|
||||
pressure_device_ids: list[str] = Field(default_factory=list, max_length=20)
|
||||
outlet_flow_device_ids: list[str] = Field(default_factory=list, max_length=20)
|
||||
decision_policy: Literal["all_signals", "weighted"] = "weighted"
|
||||
weights: ScadaMonitoringWeights = Field(default_factory=ScadaMonitoringWeights)
|
||||
alert_threshold: float = Field(default=0.8, gt=0, le=1)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def distinct_roles(self):
|
||||
for values in (self.pressure_device_ids, self.outlet_flow_device_ids):
|
||||
if len(set(values)) != len(values) or any(not value.strip() for value in values):
|
||||
raise ValueError("SCADA 监测角色中的设备标识不能为空或重复")
|
||||
if set(self.pressure_device_ids) & set(self.outlet_flow_device_ids):
|
||||
raise ValueError("同一设备不能同时作为压力与出厂边界流量")
|
||||
return self
|
||||
|
||||
|
||||
class ProjectSettings(BaseModel):
|
||||
"""Extensible, non-secret project parameters stored in MetaDB as JSON."""
|
||||
|
||||
model_config = ConfigDict(extra="allow", allow_inf_nan=False)
|
||||
|
||||
__pydantic_extra__: dict[str, JsonValue] = Field(init=False)
|
||||
scada_monitoring_interval_seconds: int | None = Field(
|
||||
default=None,
|
||||
strict=True,
|
||||
ge=60,
|
||||
le=31_536_000,
|
||||
description="SCADA 主动监测演示步长,单位秒;null 表示未配置",
|
||||
)
|
||||
scada_monitoring: ScadaMonitoringSettings | None = Field(
|
||||
default=None,
|
||||
description="主动监测使用的项目设备角色;不创建定时任务",
|
||||
)
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Reader-facing text checks for new publications, never historical reads."""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.domain.schemas.answer_ui import ReportUiBlock
|
||||
|
||||
|
||||
_execution_text = re.compile(
|
||||
r"\b(?:execution_generation|handling_context|verification_ready|waiting_work_order|"
|
||||
r"waiting_scada|returned_at|verified_through|required_samples|interval_seconds|"
|
||||
r"knowledge_cutoff|batch_sequence|query_complete|evidence_id|action_id|run_id|"
|
||||
r"task_id|source_session_id|approval_status|execution_status|verification_evidence_id|"
|
||||
r"normal_day_count|baseline_method|selection_method|get_task_context|query_scada_readings|"
|
||||
r"run_burst_location|publish_task_report|publish_task_progress)\b|"
|
||||
r"\b(?:status|phase|interval)\s*[:=]|\b(?:mcp|recovery|sha256):|"
|
||||
r"\bskill(?:\.(?:name|version))?\s*[:=]|\bskill\s+[\w-]+\s+v?\d|"
|
||||
r"\b(?:final|interim)/concluded\b|"
|
||||
r"[0-9a-f]{8}-(?:[0-9a-f]{4}-){3}[0-9a-f]{12}",
|
||||
re.IGNORECASE | re.ASCII,
|
||||
)
|
||||
_source_time = re.compile(
|
||||
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}|UTC\s*[+-]\s*\d|Asia/Shanghai|上海时间|"
|
||||
r"\d{2}:\d{2}(?::\d{2})?\s*\+\d{2}:\d{2}",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_unrendered_markup = re.compile(r"`|\*\*|</?[A-Za-z][^>]*>|^\s{0,3}#{1,6}\s", re.MULTILINE)
|
||||
|
||||
|
||||
def validate_report_text(value: str) -> str:
|
||||
if _execution_text.search(value):
|
||||
raise ValueError("请将执行字段和状态码改写为中文业务说明,完整编号保留在结构化引用中")
|
||||
if _source_time.search(value):
|
||||
raise ValueError("请将来源时间转换为当地日期和时刻,正文省略时区名称、偏移和 ISO 时间戳")
|
||||
if _unrendered_markup.search(value):
|
||||
raise ValueError("可见文字使用纯文本;重点与明细使用对应结果组件,不写 Markdown 或 HTML")
|
||||
return value
|
||||
|
||||
|
||||
def validate_report_ui_text(blocks: list[ReportUiBlock]) -> None:
|
||||
for item in blocks:
|
||||
validate_report_text(item.block.fallback_text)
|
||||
for element in item.block.spec.elements.values():
|
||||
props = element.props
|
||||
for key in ("title", "summary", "text", "description", "x_axis_name", "y_axis_name"):
|
||||
if isinstance(props.get(key), str):
|
||||
validate_report_text(props[key])
|
||||
for value in props.get("columns", []):
|
||||
validate_report_text(value)
|
||||
for row in props.get("rows", []):
|
||||
for value in row:
|
||||
validate_report_text(value)
|
||||
for value in props.get("x_data", []):
|
||||
validate_report_text(value)
|
||||
for series in props.get("series", []):
|
||||
validate_report_text(series["name"])
|
||||
|
||||
|
||||
def validate_presentation_command_text(command: dict[str, Any]) -> None:
|
||||
"""Check visible labels without touching geometry, references or timestamps."""
|
||||
if command["action"] == "set_overlay":
|
||||
validate_report_text(command["title"])
|
||||
elif command["action"] == "open_chart":
|
||||
chart = command["chart"]
|
||||
for key in ("title", "x_axis_name", "y_axis_name"):
|
||||
if isinstance(chart.get(key), str):
|
||||
validate_report_text(chart[key])
|
||||
for value in chart["x_data"]:
|
||||
validate_report_text(value)
|
||||
for series in chart["series"]:
|
||||
validate_report_text(series["name"])
|
||||
@@ -1,8 +1,8 @@
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
|
||||
class ScadaDeviceResponse(BaseModel):
|
||||
"""Project SCADA metadata keyed by the canonical device identifier."""
|
||||
|
||||
@@ -47,3 +47,20 @@ class ScadaLatestReadingsResponse(BaseModel):
|
||||
offset: int = Field(ge=0)
|
||||
observed_count: int = Field(ge=0)
|
||||
latest_observed_at: datetime | None = None
|
||||
|
||||
|
||||
class ScadaRawReading(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
time: datetime
|
||||
device_id: str
|
||||
monitored_value: float | None
|
||||
quality: Literal["unvalidated", "missing", "invalid"] = "unvalidated"
|
||||
|
||||
|
||||
class ScadaRawPage(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
items: list[ScadaRawReading]
|
||||
next_cursor: str | None
|
||||
scanned_through: datetime
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
"""Persisted replay controls and explicit, unit-aware demonstration rules."""
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
|
||||
|
||||
class ReplayModel(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
|
||||
class ScadaReplayRule(ReplayModel):
|
||||
rule_id: str = Field(min_length=1, max_length=80, pattern=r"^[a-zA-Z0-9_-]+$")
|
||||
device_ids: list[str] = Field(min_length=1, max_length=500)
|
||||
kind: Literal["relative_change", "bounds"]
|
||||
direction: Literal["both", "increase", "decrease"] = "both"
|
||||
threshold: float | None = Field(None, gt=0)
|
||||
lower: float | None = None
|
||||
upper: float | None = None
|
||||
unit: str = Field(min_length=1, max_length=40)
|
||||
metric_name: str = Field(default="", max_length=80)
|
||||
minimum_samples: int = Field(3, ge=1, le=96)
|
||||
window_samples: int = Field(4, ge=1, le=96)
|
||||
consecutive_samples: int = Field(1, ge=1, le=20)
|
||||
recovery_samples: int = Field(2, ge=1, le=20)
|
||||
minimum_devices: int = Field(1, ge=1, le=500)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def parameters(self):
|
||||
if len(set(self.device_ids)) != len(self.device_ids):
|
||||
raise ValueError("规则内设备不能重复")
|
||||
if self.minimum_devices > len(self.device_ids):
|
||||
raise ValueError("同时异常设备数超过规则设备数")
|
||||
if self.kind == "relative_change" and (self.threshold is None or self.minimum_samples > self.window_samples):
|
||||
raise ValueError("变化率规则需要阈值和足够的历史窗口")
|
||||
if self.kind == "bounds" and self.lower is None and self.upper is None:
|
||||
raise ValueError("上下限规则至少配置一个阈值")
|
||||
if self.lower is not None and self.upper is not None and self.lower >= self.upper:
|
||||
raise ValueError("规则下限必须小于上限")
|
||||
return self
|
||||
|
||||
|
||||
class ScadaQualityRange(ReplayModel):
|
||||
lower: float | None = None
|
||||
upper: float | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def bounds(self):
|
||||
if self.lower is not None and self.upper is not None and self.lower >= self.upper:
|
||||
raise ValueError("仪表量程下限必须小于上限")
|
||||
return self
|
||||
|
||||
|
||||
class ScadaReplayCreate(ReplayModel):
|
||||
request_id: UUID
|
||||
start_time: datetime
|
||||
end_time: datetime
|
||||
device_ids: list[str] = Field(min_length=1, max_length=500)
|
||||
rules: list[ScadaReplayRule] = Field(min_length=1, max_length=30)
|
||||
quality_ranges: dict[str, ScadaQualityRange] = Field(default_factory=dict)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def configuration(self):
|
||||
if (self.start_time.tzinfo is None or self.end_time.tzinfo is None
|
||||
or not timedelta(0) < self.end_time - self.start_time <= timedelta(days=31)):
|
||||
raise ValueError("请选择带时区、最长 31 天的有效时间区间")
|
||||
ids = set(self.device_ids)
|
||||
if len(ids) != len(self.device_ids) or any(not value.strip() for value in ids):
|
||||
raise ValueError("设备标识不能为空或重复")
|
||||
if len({rule.rule_id for rule in self.rules}) != len(self.rules):
|
||||
raise ValueError("规则标识不能重复")
|
||||
if any(not set(rule.device_ids) <= ids for rule in self.rules) or not set(self.quality_ranges) <= ids:
|
||||
raise ValueError("规则及量程设备必须属于本次回放")
|
||||
return self
|
||||
|
||||
|
||||
class ScadaReplayCommand(ReplayModel):
|
||||
request_id: UUID
|
||||
expected_revision: int = Field(ge=1)
|
||||
action: Literal["step", "monitor", "stop", "retry"]
|
||||
at_time: datetime | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def monitoring_time(self):
|
||||
if self.action == "monitor":
|
||||
if self.at_time is None or self.at_time.tzinfo is None:
|
||||
raise ValueError("选定时刻监测需要提供带时区的 at_time")
|
||||
elif self.at_time is not None:
|
||||
raise ValueError("只有选定时刻监测可以指定 at_time")
|
||||
return self
|
||||
|
||||
|
||||
class ScadaMonitoringIncident(ReplayModel):
|
||||
incident_id: UUID
|
||||
status: Literal["active", "resolved"]
|
||||
action: Literal["raised", "ongoing", "resolved"]
|
||||
started_at: datetime
|
||||
last_observed_at: datetime
|
||||
recovery_count: int = Field(ge=0, le=2)
|
||||
|
||||
|
||||
class ScadaMonitoringResult(ReplayModel):
|
||||
monitoring_id: UUID
|
||||
period_start: datetime
|
||||
period_end: datetime
|
||||
knowledge_cutoff: datetime
|
||||
batch_sequence: int = Field(ge=1)
|
||||
rule_version: str = Field(min_length=1, max_length=80)
|
||||
status: Literal["normal", "attention", "insufficient_data", "suspected_burst"]
|
||||
detail: str = Field(min_length=1, max_length=1000)
|
||||
task_id: UUID | None = None
|
||||
pressure: dict
|
||||
outlet_flow: dict
|
||||
burst_detection: dict
|
||||
weighted: dict | None = None
|
||||
incident: ScadaMonitoringIncident | None = None
|
||||
role_gaps: list[str] = Field(default_factory=list)
|
||||
anomaly_signals: list[dict] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ScadaReplaySnapshot(ReplayModel):
|
||||
replay_id: UUID
|
||||
project_id: UUID
|
||||
revision: int
|
||||
status: Literal["ready", "paused", "completed", "stopped", "failed"]
|
||||
start_time: datetime
|
||||
end_time: datetime
|
||||
monitoring_interval_seconds: int | None = Field(None, ge=60, le=31_536_000)
|
||||
step_pending: bool = False
|
||||
monitoring_count: int = 0
|
||||
last_monitoring: ScadaMonitoringResult | None = None
|
||||
monitoring_config: ScadaMonitoringSettings | None = None
|
||||
monitoring_calibration: dict | None = None
|
||||
virtual_now: datetime
|
||||
delivered_until: datetime | None = None
|
||||
processed_until: datetime | None = None
|
||||
processed_count: int = 0
|
||||
anomaly_count: int = 0
|
||||
devices: list[dict]
|
||||
rules: list[ScadaReplayRule]
|
||||
latest: list[dict]
|
||||
active_events: list[dict]
|
||||
error: str | None = None
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class ScadaReplayCapabilities(ReplayModel):
|
||||
available: bool
|
||||
can_write: bool
|
||||
reason: str | None = None
|
||||
monitoring_interval_seconds: int | None = Field(None, ge=60, le=31_536_000)
|
||||
burst_monitoring_ready: bool = False
|
||||
burst_monitoring_reason: str | None = None
|
||||
monitoring_config: ScadaMonitoringSettings | None = None
|
||||
current: ScadaReplaySnapshot | None = None
|
||||
|
||||
|
||||
class ScadaReplayEventsPage(ReplayModel):
|
||||
items: list[dict]
|
||||
next_offset: int | None
|
||||
|
||||
|
||||
class ScadaReplayReadings(ReplayModel):
|
||||
items: list[dict]
|
||||
through: datetime
|
||||
@@ -5,7 +5,7 @@ from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictModel
|
||||
from app.domain.schemas.workbench_runtime import WorkbenchAlertSource, WorkbenchDateTime
|
||||
from app.domain.schemas.workbench_runtime import WorkbenchAlertSource, WorkbenchScadaReplaySource, WorkbenchDateTime
|
||||
|
||||
|
||||
class TaskAlertCommand(StrictModel):
|
||||
@@ -14,7 +14,7 @@ class TaskAlertCommand(StrictModel):
|
||||
event_type: Literal["raised", "acknowledged", "resolved"]
|
||||
occurred_at: WorkbenchDateTime
|
||||
condition_id: NonEmptyStr | None
|
||||
source: WorkbenchAlertSource
|
||||
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
|
||||
title: NonEmptyStr
|
||||
description: NonEmptyStr
|
||||
severity: Literal["warning", "critical"]
|
||||
@@ -29,7 +29,7 @@ class TaskAlertState(StrictModel):
|
||||
revision: PositiveInt
|
||||
producer_id: NonEmptyStr
|
||||
condition_id: NonEmptyStr | None
|
||||
source: WorkbenchAlertSource
|
||||
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
|
||||
title: NonEmptyStr
|
||||
description: NonEmptyStr
|
||||
occurred_at: WorkbenchDateTime
|
||||
|
||||
@@ -49,10 +49,17 @@ class WorkbenchAlertSource(StrictModel):
|
||||
run_id: NonEmptyStr
|
||||
|
||||
|
||||
class WorkbenchScadaReplaySource(StrictModel):
|
||||
kind: Literal["scada_replay"]
|
||||
replay_id: NonEmptyStr
|
||||
rule_id: NonEmptyStr
|
||||
batch_sequence: int = Field(ge=1)
|
||||
|
||||
|
||||
class WorkbenchRuntimeAlert(StrictModel):
|
||||
id: NonEmptyStr
|
||||
condition_id: NonEmptyStr | None
|
||||
source: WorkbenchAlertSource
|
||||
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
|
||||
title: NonEmptyStr
|
||||
description: NonEmptyStr
|
||||
occurred_at: WorkbenchDateTime
|
||||
@@ -62,7 +69,7 @@ class WorkbenchRuntimeAlert(StrictModel):
|
||||
class WorkbenchPersistedAlert(StrictModel):
|
||||
alert_id: NonEmptyStr
|
||||
condition_id: NonEmptyStr | None
|
||||
source: WorkbenchAlertSource
|
||||
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
|
||||
title: NonEmptyStr
|
||||
description: NonEmptyStr
|
||||
occurred_at: WorkbenchDateTime
|
||||
|
||||
@@ -7,6 +7,29 @@ from uuid import UUID
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
async def discover_project_ids() -> tuple[str, ...]:
|
||||
"""Return every active PostgreSQL-backed project from authoritative metadata."""
|
||||
from sqlalchemy import select
|
||||
from app.infra.db.metadb.database import SessionLocal
|
||||
from app.infra.db.metadb import models
|
||||
|
||||
async with SessionLocal() as session:
|
||||
rows = await session.scalars(
|
||||
select(models.Project.id)
|
||||
.join(
|
||||
models.ProjectDatabase,
|
||||
models.ProjectDatabase.project_id == models.Project.id,
|
||||
)
|
||||
.where(
|
||||
models.Project.status == "active",
|
||||
models.ProjectDatabase.db_role == "biz_data",
|
||||
models.ProjectDatabase.db_type == "postgresql",
|
||||
)
|
||||
.order_by(models.Project.id)
|
||||
)
|
||||
return tuple(str(project_id) for project_id in rows.all())
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def queue_repository(project_id):
|
||||
"""Resolve the current project route and open its calculation queue."""
|
||||
|
||||
@@ -44,6 +44,7 @@ class Project(Base):
|
||||
gs_workspace: Mapped[str] = mapped_column(String(100), unique=True)
|
||||
map_extent: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||
map_config: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict, server_default="{}")
|
||||
settings: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict, server_default="{}")
|
||||
status: Mapped[str] = mapped_column(String(20), default="active")
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), default=datetime.utcnow
|
||||
|
||||
@@ -67,6 +67,7 @@ class ProjectDetail:
|
||||
map_extent: Optional[dict]
|
||||
status: str
|
||||
map_config: dict = field(default_factory=dict)
|
||||
settings: dict = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -223,6 +224,7 @@ class MetadataRepository:
|
||||
status: str,
|
||||
creator_user_id: UUID | None = None,
|
||||
map_config: dict | None = None,
|
||||
settings: dict | None = None,
|
||||
) -> models.Project:
|
||||
project = models.Project(
|
||||
id=uuid4(),
|
||||
@@ -233,6 +235,7 @@ class MetadataRepository:
|
||||
map_extent=map_extent,
|
||||
status=status,
|
||||
map_config=map_config or {},
|
||||
settings=settings or {},
|
||||
created_at=_utcnow(),
|
||||
updated_at=_utcnow(),
|
||||
)
|
||||
@@ -263,6 +266,7 @@ class MetadataRepository:
|
||||
timescale_dsn: str,
|
||||
pool_min_size: int = 1,
|
||||
pool_max_size: int = 4,
|
||||
settings: dict | None = None,
|
||||
) -> models.Project:
|
||||
"""Atomically expose a fully provisioned project and both DB routes."""
|
||||
business_secret = _encrypt_database_secret(business_dsn)
|
||||
@@ -276,6 +280,7 @@ class MetadataRepository:
|
||||
map_extent=map_extent,
|
||||
status="active",
|
||||
map_config={},
|
||||
settings=settings or {},
|
||||
created_at=_utcnow(),
|
||||
updated_at=_utcnow(),
|
||||
)
|
||||
@@ -333,6 +338,8 @@ class MetadataRepository:
|
||||
setattr(project, field, updates[field])
|
||||
if "map_config" in updates:
|
||||
project.map_config = {**(project.map_config or {}), **updates["map_config"]}
|
||||
if "settings" in updates:
|
||||
project.settings = {**(project.settings or {}), **updates["settings"]}
|
||||
project.updated_at = _utcnow()
|
||||
await self.session.commit()
|
||||
await self.session.refresh(project)
|
||||
@@ -350,6 +357,7 @@ class MetadataRepository:
|
||||
gs_workspace=project.gs_workspace,
|
||||
map_extent=project.map_extent,
|
||||
map_config=project.map_config or {},
|
||||
settings=project.settings or {},
|
||||
status=project.status,
|
||||
)
|
||||
|
||||
|
||||
@@ -7,10 +7,10 @@ from app.mcp.errors import CalculationCapacityExceeded
|
||||
|
||||
|
||||
# Explicit metadata projection prevents status polling from decoding large JSON results.
|
||||
JOB_COLUMNS = "job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,status,error,lease_generation,lease_owner,lease_expires_at,created_at,updated_at,started_at,finished_at,queue_expires_at,result_expires_at"
|
||||
JOB_COLUMNS = "job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,status,error,lease_generation,lease_owner,lease_expires_at,created_at,updated_at,started_at,finished_at,result_expires_at"
|
||||
|
||||
|
||||
PUBLIC_JOB_COLUMNS = "job_id,status,tool,error,created_at,updated_at,started_at,finished_at,queue_expires_at,result_expires_at"
|
||||
PUBLIC_JOB_COLUMNS = "job_id,status,tool,error,created_at,updated_at,started_at,finished_at,result_expires_at"
|
||||
|
||||
|
||||
class CalculationJobRepository:
|
||||
@@ -19,12 +19,6 @@ class CalculationJobRepository:
|
||||
self.project_id = str(project_id)
|
||||
|
||||
async def maintain(self):
|
||||
await self.connection.execute(
|
||||
"""UPDATE workbench.calculation_jobs SET status='timed_out',
|
||||
error='{"code":"QUEUE_TIMEOUT"}'::jsonb,finished_at=clock_timestamp(),updated_at=clock_timestamp()
|
||||
WHERE project_id=%s AND status='queued' AND queue_expires_at<=clock_timestamp()""",
|
||||
(self.project_id,),
|
||||
)
|
||||
await self.connection.execute(
|
||||
"""UPDATE workbench.calculation_jobs SET result=NULL
|
||||
WHERE project_id=%s AND result IS NOT NULL AND result_expires_at<=clock_timestamp()""",
|
||||
@@ -32,14 +26,14 @@ class CalculationJobRepository:
|
||||
)
|
||||
|
||||
async def enqueue(self, *, owner_user_id, request_key, tool, parameters, binding, timeout_seconds,
|
||||
queue_wait_seconds=3600, max_active_jobs=100):
|
||||
max_active_jobs=100):
|
||||
async with self.connection.transaction():
|
||||
return await self._enqueue(owner_user_id=owner_user_id, request_key=request_key,
|
||||
tool=tool, parameters=parameters, binding=binding, timeout_seconds=timeout_seconds,
|
||||
queue_wait_seconds=queue_wait_seconds, max_active_jobs=max_active_jobs)
|
||||
max_active_jobs=max_active_jobs)
|
||||
|
||||
async def _enqueue(self, *, owner_user_id, request_key, tool, parameters, binding,
|
||||
timeout_seconds, queue_wait_seconds, max_active_jobs):
|
||||
timeout_seconds, max_active_jobs):
|
||||
# Serialize admission within the caller's transaction, across every API process.
|
||||
await self.connection.execute(
|
||||
"SELECT pg_advisory_xact_lock(hashtextextended(%s, 1748531781))", (self.project_id,))
|
||||
@@ -64,11 +58,11 @@ class CalculationJobRepository:
|
||||
raise CalculationCapacityExceeded()
|
||||
cursor = await self.connection.execute(
|
||||
f"""INSERT INTO workbench.calculation_jobs
|
||||
(job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,queue_expires_at)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,clock_timestamp()+%s*interval '1 second')
|
||||
(job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s)
|
||||
RETURNING {JOB_COLUMNS}""",
|
||||
(uuid4(), self.project_id, owner_user_id, request_key, tool,
|
||||
Jsonb(parameters), Jsonb(binding), timeout_seconds, queue_wait_seconds),
|
||||
Jsonb(parameters), Jsonb(binding), timeout_seconds),
|
||||
)
|
||||
return dict(await cursor.fetchone())
|
||||
|
||||
@@ -118,7 +112,7 @@ class CalculationJobRepository:
|
||||
lease_generation=j.lease_generation+1,
|
||||
lease_expires_at=clock_timestamp()+%s*interval '1 second',updated_at=clock_timestamp()
|
||||
WHERE j.job_id=(SELECT job_id FROM workbench.calculation_jobs
|
||||
WHERE project_id=%s AND status='queued' AND queue_expires_at>clock_timestamp() ORDER BY created_at,job_id
|
||||
WHERE project_id=%s AND status='queued' ORDER BY created_at,job_id
|
||||
FOR UPDATE SKIP LOCKED LIMIT 1) RETURNING j.*""",
|
||||
(owner, lease_seconds, self.project_id),
|
||||
)
|
||||
|
||||
@@ -6,6 +6,24 @@ from psycopg import AsyncConnection
|
||||
|
||||
|
||||
class NetworkAssetRepository:
|
||||
@staticmethod
|
||||
async def get_isolation_model(conn: AsyncConnection) -> list[dict[str, Any]]:
|
||||
"""Read source nodes, link state and pipe lengths in one model snapshot."""
|
||||
from app.infra.db.postgresql.read_queries import read_rows
|
||||
|
||||
return await read_rows(conn, """
|
||||
SELECT l.id, l.link_type::text AS kind,
|
||||
l.start_node_id, l.end_node_id, p.length, p.diameter,
|
||||
COALESCE(s.status::text, p.status::text, '') AS status
|
||||
FROM network.links l
|
||||
LEFT JOIN network.pipes p ON p.link_id = l.id
|
||||
LEFT JOIN network.link_initial_settings s ON s.link_id = l.id
|
||||
UNION ALL
|
||||
SELECT n.id, n.node_type::text, NULL, NULL, NULL, NULL, ''
|
||||
FROM network.nodes n
|
||||
WHERE n.node_type::text IN ('reservoir', 'tank')
|
||||
""")
|
||||
|
||||
@staticmethod
|
||||
async def get_pipes_by_ids(
|
||||
conn: AsyncConnection,
|
||||
|
||||
@@ -164,7 +164,7 @@ class OperationalTaskRepository:
|
||||
ORDER BY generation DESC LIMIT %s OFFSET %s""",
|
||||
(self.project_id, task_id, limit, offset),
|
||||
)
|
||||
return [{key: value for key, value in row.items() if key != "project_id"}
|
||||
return [{key: value for key, value in row.items() if key not in {"project_id", "artifact_manifest"}}
|
||||
for row in await cursor.fetchall()]
|
||||
|
||||
async def get_latest_run(self, task_id: str) -> dict | None:
|
||||
@@ -264,6 +264,7 @@ class OperationalTaskRepository:
|
||||
async def list_runs(
|
||||
self, *, status: str | None, limit: int, cursor_created_at=None, cursor_run_id=None,
|
||||
task_type: str | None = None, task_created_from: datetime | None = None,
|
||||
execution_scope: str = "production",
|
||||
) -> list[dict]:
|
||||
result = await self.connection.execute(
|
||||
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
|
||||
@@ -282,18 +283,20 @@ class OperationalTaskRepository:
|
||||
AND t.task_id=operational_task_runs.task_id
|
||||
AND (%s::text IS NULL OR t.task_type=%s::text)
|
||||
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
|
||||
AND t.document->>'execution_scope'=%s
|
||||
)
|
||||
ORDER BY created_at,run_id LIMIT %s""",
|
||||
(
|
||||
self.project_id, status, status, cursor_created_at,
|
||||
cursor_created_at, cursor_run_id,
|
||||
task_type, task_type, task_created_from, task_created_from, limit,
|
||||
task_type, task_type, task_created_from, task_created_from, execution_scope, limit,
|
||||
),
|
||||
)
|
||||
return [dict(row) for row in await result.fetchall()]
|
||||
|
||||
async def claim_expired_runs(
|
||||
self, limit: int, *, task_type: str | None = None, task_created_from: datetime | None = None,
|
||||
execution_scope: str = "production",
|
||||
) -> list[dict]:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
|
||||
@@ -310,9 +313,10 @@ class OperationalTaskRepository:
|
||||
AND t.task_id=operational_task_runs.task_id
|
||||
AND (%s::text IS NULL OR t.task_type=%s::text)
|
||||
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
|
||||
AND t.document->>'execution_scope'=%s
|
||||
)
|
||||
ORDER BY lease_expires_at,run_id FOR UPDATE SKIP LOCKED LIMIT %s""",
|
||||
(self.project_id, task_type, task_type, task_created_from, task_created_from, limit),
|
||||
(self.project_id, task_type, task_type, task_created_from, task_created_from, execution_scope, limit),
|
||||
)
|
||||
return [dict(row) for row in await cursor.fetchall()]
|
||||
|
||||
@@ -367,6 +371,34 @@ class OperationalTaskRepository:
|
||||
)
|
||||
return [dict(row) for row in await cursor.fetchall()]
|
||||
|
||||
async def get_scada_presentation(self, task_id: str, run_id: str) -> dict | None:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT EXISTS (SELECT 1 FROM pg_attribute
|
||||
WHERE attrelid=to_regclass('workbench.operational_task_runs')
|
||||
AND attname='artifact_manifest' AND NOT attisdropped) AS available"""
|
||||
)
|
||||
if not (await cursor.fetchone())["available"]:
|
||||
raise DomainError("DEPENDENCY_UNAVAILABLE",
|
||||
"SCADA display coverage requires resources/sql/035_scada_presentation_coverage.sql in this project database",
|
||||
status=503)
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT artifact_manifest->'scada_presentation' AS document FROM workbench.operational_task_runs
|
||||
WHERE project_id=%s AND task_id=%s AND run_id=%s""",
|
||||
(self.project_id, task_id, run_id),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def save_scada_presentation(self, task_id: str, run_id: str, document: dict, now: datetime) -> None:
|
||||
cursor = await self.connection.execute(
|
||||
"""UPDATE workbench.operational_task_runs
|
||||
SET artifact_manifest=jsonb_set(artifact_manifest,'{scada_presentation}',%s::jsonb,true)
|
||||
WHERE project_id=%s AND task_id=%s AND run_id=%s""",
|
||||
(Jsonb({**document, "updated_at": now.isoformat()}), self.project_id, task_id, run_id),
|
||||
)
|
||||
if cursor.rowcount != 1:
|
||||
raise DomainError("RUN_NOT_FOUND", "SCADA display coverage targets an unknown run", status=409)
|
||||
|
||||
async def get_task_by_create_request(self, request_id: str) -> dict | None:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.operational_tasks
|
||||
@@ -401,9 +433,9 @@ class OperationalTaskRepository:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT task.document,
|
||||
CASE WHEN active_run.run_id IS NULL THEN NULL
|
||||
ELSE to_jsonb(active_run) - 'project_id' END AS active_run,
|
||||
ELSE to_jsonb(active_run) - 'project_id' - 'artifact_manifest' END AS active_run,
|
||||
CASE WHEN latest_run.run_id IS NULL THEN NULL
|
||||
ELSE to_jsonb(latest_run) - 'project_id' END AS latest_run
|
||||
ELSE to_jsonb(latest_run) - 'project_id' - 'artifact_manifest' END AS latest_run
|
||||
FROM workbench.operational_tasks AS task
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT run.* FROM workbench.operational_task_runs AS run
|
||||
@@ -624,6 +656,37 @@ class OperationalTaskRepository:
|
||||
row = await cursor.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def get_latest_graph_progress(self, task_id: str) -> dict | None:
|
||||
availability = await self.connection.execute(
|
||||
"SELECT to_regclass('workbench.operational_task_progress') AS relation"
|
||||
)
|
||||
if not (await availability.fetchone())["relation"]:
|
||||
return None
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.operational_task_progress
|
||||
WHERE project_id=%s AND task_id=%s
|
||||
AND jsonb_typeof(document#>'{progress,diagram,edges}')='array'
|
||||
ORDER BY (document->>'execution_generation')::bigint DESC, revision DESC LIMIT 1""",
|
||||
(self.project_id, task_id),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def get_latest_diagram(self, task_id: str) -> dict | None:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT diagram FROM (
|
||||
SELECT document#>'{progress,diagram}' AS diagram, published_at AS occurred_at
|
||||
FROM workbench.operational_task_progress WHERE project_id=%s AND task_id=%s
|
||||
UNION ALL
|
||||
SELECT document#>'{report,diagram}', created_at
|
||||
FROM workbench.operational_task_reports WHERE project_id=%s AND task_id=%s
|
||||
) AS diagrams WHERE jsonb_typeof(diagram)='object'
|
||||
ORDER BY occurred_at DESC LIMIT 1""",
|
||||
(self.project_id, task_id, self.project_id, task_id),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return row["diagram"] if row else None
|
||||
|
||||
async def save_progress(self, task_id: str, snapshot: dict) -> None:
|
||||
cursor = await self.connection.execute(
|
||||
"""INSERT INTO workbench.operational_task_progress
|
||||
@@ -711,6 +774,28 @@ class OperationalTaskRepository:
|
||||
row = await cursor.fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
async def recovery_candidates(self, limit: int, created_from=None, execution_scope: str = "production") -> list[dict]:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.operational_tasks
|
||||
WHERE project_id=%s AND task_type='event_investigation'
|
||||
AND status NOT IN ('cancelled','failed')
|
||||
AND document#>>'{handling_context,phase}'='waiting_scada'
|
||||
AND document->>'execution_scope'=%s
|
||||
AND (%s::timestamptz IS NULL OR created_at>=%s)
|
||||
ORDER BY COALESCE((document#>>'{handling_context,last_checked_at}')::timestamptz,created_at),task_id LIMIT %s""",
|
||||
(self.project_id, execution_scope, created_from, created_from, limit),
|
||||
)
|
||||
return [row["document"] for row in await cursor.fetchall()]
|
||||
|
||||
async def list_investigation_alerts(self, task_id: str) -> list[dict]:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.alerts WHERE project_id=%s
|
||||
AND status IN ('active','acknowledged')
|
||||
AND document->>'condition_id'=%s ORDER BY alert_id""",
|
||||
(self.project_id, f"operational-task:{task_id}"),
|
||||
)
|
||||
return [row["document"] for row in await cursor.fetchall()]
|
||||
|
||||
async def upsert_feedback(self, task_id: str, feedback: dict, accepted_at) -> None:
|
||||
await self.connection.execute(
|
||||
"""INSERT INTO workbench.operational_task_external_feedback
|
||||
|
||||
@@ -7,7 +7,9 @@ from uuid import UUID
|
||||
from psycopg import AsyncConnection
|
||||
from psycopg import sql
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from fastapi import HTTPException
|
||||
from app.domain.schemas.operational_task import LEGACY_INVESTIGATION_NODE_ANNOTATIONS
|
||||
from app.infra.db.postgresql.scada import _SCADA_VIEW_SELECT, _device
|
||||
from app.infra.db.timescaledb.repositories.realtime import RealtimeRepository
|
||||
from app.infra.db.timescaledb.repositories.analysis import AnalysisResultsRepository
|
||||
@@ -312,6 +314,7 @@ async def task_context_summary(conn: AsyncConnection, *, project_id: UUID, task_
|
||||
'current_report_revision',t.current_report_revision,
|
||||
'title_preview',left(t.document->>'title',160),'preview_only',true,
|
||||
'detail',jsonb_build_object('section','task')),
|
||||
'handling_context',COALESCE(t.document->'handling_context','{}'::jsonb),
|
||||
'latest_report',(SELECT REPORT_SUMMARY FROM workbench.operational_task_reports r
|
||||
WHERE r.project_id=t.project_id AND r.task_id=t.task_id ORDER BY r.revision DESC LIMIT 1),
|
||||
'selected_report',CASE WHEN %s::int IS NULL THEN NULL ELSE
|
||||
@@ -328,6 +331,29 @@ async def task_context_summary(conn: AsyncConnection, *, project_id: UUID, task_
|
||||
return result
|
||||
|
||||
|
||||
def _reference_diagram_projection(stored_sql: str, stored_params: tuple, section: str) -> tuple[str, tuple]:
|
||||
"""Normalize only saved diagram annotations before any data_path or page projection."""
|
||||
if section not in {"progress", "report"}:
|
||||
raise ValueError("reference diagrams only belong to progress or report sections")
|
||||
path = "{data," + section + ",diagram,node_refs}"
|
||||
statement = f"""
|
||||
WITH reference_documents AS (
|
||||
{stored_sql}
|
||||
), reference_aliases AS (
|
||||
SELECT %s::jsonb AS annotations
|
||||
)
|
||||
SELECT CASE WHEN jsonb_typeof(snapshot#>'{path}')='array' THEN
|
||||
jsonb_set(snapshot,'{path}',COALESCE((
|
||||
SELECT jsonb_agg(CASE WHEN annotations ? (node->>'status')
|
||||
THEN jsonb_set(node,'{{status}}',annotations->(node->>'status'))
|
||||
ELSE node END ORDER BY position)
|
||||
FROM jsonb_array_elements(snapshot#>'{path}') WITH ORDINALITY AS nodes(node,position)
|
||||
),'[]'::jsonb)) ELSE snapshot END AS snapshot
|
||||
FROM reference_documents CROSS JOIN reference_aliases
|
||||
"""
|
||||
return statement, (*stored_params, Jsonb(LEGACY_INVESTIGATION_NODE_ANNOTATIONS))
|
||||
|
||||
|
||||
async def task_context_page(
|
||||
conn: AsyncConnection, *, project_id: UUID, task_id: UUID, section: str,
|
||||
report_revision: int | None, data_path: list[str], limit: int, offset: int,
|
||||
@@ -378,18 +404,23 @@ async def task_context_page(
|
||||
where = " ORDER BY revision DESC LIMIT 1"
|
||||
stored = f"SELECT jsonb_build_object('section',%s::text,'data',document) AS snapshot FROM {relation} WHERE project_id=%s AND task_id=%s{where}"
|
||||
elif section == "run":
|
||||
stored = """SELECT jsonb_build_object('section','run','data',to_jsonb(r)-'project_id') AS snapshot
|
||||
stored = """SELECT jsonb_build_object('section','run','data',to_jsonb(r)-'project_id'-'artifact_manifest') AS snapshot
|
||||
FROM workbench.operational_task_runs r WHERE project_id=%s AND task_id=%s ORDER BY generation DESC LIMIT 1"""
|
||||
params = (project_id, task_id)
|
||||
elif section == "progress":
|
||||
stored = """SELECT jsonb_build_object('section','progress','data',p.document) AS snapshot
|
||||
FROM workbench.operational_task_progress p
|
||||
JOIN workbench.operational_task_runs r ON r.project_id=p.project_id AND r.task_id=p.task_id AND r.run_id=p.run_id
|
||||
JOIN workbench.operational_tasks t ON t.project_id=p.project_id AND t.task_id=p.task_id AND t.execution_generation=r.generation
|
||||
WHERE p.project_id=%s AND p.task_id=%s"""
|
||||
JOIN workbench.operational_tasks t ON t.project_id=p.project_id AND t.task_id=p.task_id
|
||||
WHERE p.project_id=%s AND p.task_id=%s AND r.generation<=t.execution_generation
|
||||
AND (t.execution_generation=r.generation
|
||||
OR jsonb_typeof(p.document#>'{progress,diagram,edges}')='array')
|
||||
ORDER BY (t.execution_generation=r.generation) DESC,r.generation DESC,p.revision DESC LIMIT 1"""
|
||||
params = (project_id, task_id)
|
||||
else:
|
||||
raise HTTPException(422, "Unknown task context section")
|
||||
if section in {"progress", "report"}:
|
||||
stored, params = _reference_diagram_projection(stored, params, section)
|
||||
return await _snapshot_page(conn, stored_sql=stored, stored_params=params,
|
||||
data_path=data_path, limit=limit, offset=projection_offset, mcp_sources=[], max_bytes=8000)
|
||||
|
||||
|
||||
@@ -0,0 +1,207 @@
|
||||
"""Short-transaction storage for replay control and resumable batches."""
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from fastapi import HTTPException
|
||||
from psycopg.types.json import Jsonb
|
||||
|
||||
|
||||
class ScadaReplayRepository:
|
||||
def __init__(self, connection, project_id):
|
||||
self.connection, self.project_id = connection, str(project_id)
|
||||
self.lock_key = f"scada-replay:{self.project_id}"
|
||||
|
||||
async def available(self) -> bool:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT to_regclass('peripheral_sample.scada_replays') IS NOT NULL "
|
||||
"AND to_regclass('peripheral_sample.scada_replay_batches') IS NOT NULL AS ready"
|
||||
)
|
||||
return (await cur.fetchone())["ready"]
|
||||
|
||||
@asynccontextmanager
|
||||
async def control(self):
|
||||
async with self.connection.transaction():
|
||||
await self.connection.execute("SELECT pg_advisory_xact_lock(hashtextextended(%s,0))", (self.lock_key,))
|
||||
yield
|
||||
|
||||
@asynccontextmanager
|
||||
async def processing(self):
|
||||
# Session lock outlives short transactions, never an open SQL transaction
|
||||
# across Timescale IO. A dead connection automatically releases ownership.
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute("SELECT pg_try_advisory_lock(hashtextextended(%s,0)) AS owned", (self.lock_key,))
|
||||
owned = (await cur.fetchone())["owned"]
|
||||
try:
|
||||
yield owned
|
||||
finally:
|
||||
if owned:
|
||||
async with self.connection.transaction():
|
||||
await self.connection.execute("SELECT pg_advisory_unlock(hashtextextended(%s,0))", (self.lock_key,))
|
||||
|
||||
async def get(self, replay_id) -> dict:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT document FROM peripheral_sample.scada_replays WHERE project_id=%s AND replay_id=%s",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
row = await cur.fetchone()
|
||||
if not row:
|
||||
raise HTTPException(404, "当前项目不存在此 SCADA 回放")
|
||||
return row["document"]
|
||||
|
||||
async def current(self) -> dict | None:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT document FROM peripheral_sample.scada_replays WHERE project_id=%s"
|
||||
" ORDER BY created_at DESC LIMIT 1", (self.project_id,),
|
||||
)
|
||||
row = await cur.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def by_request(self, request_id) -> dict | None:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT document FROM peripheral_sample.scada_replays WHERE project_id=%s AND request_id=%s",
|
||||
(self.project_id, str(request_id)),
|
||||
)
|
||||
row = await cur.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def active_other(self, replay_id) -> bool:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT 1 FROM peripheral_sample.scada_replays WHERE project_id=%s AND replay_id<>%s "
|
||||
"AND status IN ('ready','paused') LIMIT 1",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
return await cur.fetchone() is not None
|
||||
|
||||
async def insert(self, document: dict):
|
||||
await self.connection.execute(
|
||||
"INSERT INTO peripheral_sample.scada_replays(project_id,replay_id,request_id,status,document) "
|
||||
"VALUES(%s,%s,%s,%s,%s)",
|
||||
(self.project_id, document["replay_id"], document["config"]["request_id"], document["status"], Jsonb(document)),
|
||||
)
|
||||
|
||||
async def save(self, document: dict):
|
||||
document["updated_at"] = datetime.now(timezone.utc).isoformat()
|
||||
await self.connection.execute(
|
||||
"UPDATE peripheral_sample.scada_replays SET status=%s,document=%s,updated_at=now() "
|
||||
"WHERE project_id=%s AND replay_id=%s",
|
||||
(document["status"], Jsonb(document), self.project_id, document["replay_id"]),
|
||||
)
|
||||
|
||||
async def pending_batch(self, replay_id) -> dict | None:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT document FROM peripheral_sample.scada_replay_batches "
|
||||
"WHERE project_id=%s AND replay_id=%s AND sequence>0 AND stage<>'detected' ORDER BY sequence LIMIT 1",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
row = await cur.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def save_batch(self, replay_id, batch: dict):
|
||||
await self.connection.execute(
|
||||
"INSERT INTO peripheral_sample.scada_replay_batches(project_id,replay_id,sequence,stage,document) "
|
||||
"VALUES(%s,%s,%s,%s,%s) ON CONFLICT(project_id,replay_id,sequence) "
|
||||
"DO UPDATE SET stage=excluded.stage,document=excluded.document",
|
||||
(self.project_id, str(replay_id), batch["sequence"], batch["stage"], Jsonb(batch)),
|
||||
)
|
||||
|
||||
async def clear_unpublished_baseline(self, replay_id) -> None:
|
||||
"""Discard only an incomplete attempt, retaining every published snapshot."""
|
||||
await self.connection.execute(
|
||||
"DELETE FROM peripheral_sample.scada_replay_batches "
|
||||
"WHERE project_id=%s AND replay_id=%s AND sequence<0 "
|
||||
"AND stage='clean_prepared' AND document->>'kind'='baseline'",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
|
||||
async def publish_baseline(self, replay_id) -> None:
|
||||
"""Publish with the replay's baseline metadata in the caller's transaction."""
|
||||
await self.connection.execute(
|
||||
"UPDATE peripheral_sample.scada_replay_batches "
|
||||
"SET stage='detected',document=jsonb_set(document,'{stage}','\"detected\"'::jsonb) "
|
||||
"WHERE project_id=%s AND replay_id=%s AND sequence<0 "
|
||||
"AND stage='clean_prepared' AND document->>'kind'='baseline'",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
|
||||
async def events(self, replay_id, offset=0, limit=100) -> list[dict]:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT e.value AS event FROM peripheral_sample.scada_replay_batches b "
|
||||
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'events','[]'::jsonb)) e "
|
||||
"WHERE b.project_id=%s AND b.replay_id=%s ORDER BY b.sequence DESC LIMIT %s OFFSET %s",
|
||||
(self.project_id, str(replay_id), limit, offset),
|
||||
)
|
||||
return [r["event"] for r in await cur.fetchall()]
|
||||
|
||||
async def monitoring_outputs(
|
||||
self,
|
||||
replay_id,
|
||||
device_ids: list[str],
|
||||
start: datetime,
|
||||
end: datetime,
|
||||
before_sequence: int,
|
||||
) -> list[dict]:
|
||||
"""Read only committed replay outputs available before the current batch."""
|
||||
if not device_ids:
|
||||
return []
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT r.value AS reading FROM peripheral_sample.scada_replay_batches b "
|
||||
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'outputs','[]'::jsonb)) r "
|
||||
"WHERE b.project_id=%s AND b.replay_id=%s AND b.stage='detected' AND b.sequence<%s "
|
||||
"AND r.value->>'device_id'=ANY(%s) "
|
||||
"AND (r.value->>'time')::timestamptz>=%s AND (r.value->>'time')::timestamptz<=%s "
|
||||
"ORDER BY (r.value->>'time')::timestamptz,r.value->>'device_id'",
|
||||
(self.project_id, str(replay_id), before_sequence, device_ids, start, end),
|
||||
)
|
||||
return [row["reading"] for row in await cur.fetchall()]
|
||||
|
||||
async def detected_outputs(
|
||||
self,
|
||||
replay_id,
|
||||
device_ids: list[str],
|
||||
start: datetime,
|
||||
end: datetime,
|
||||
before_sequence: int,
|
||||
*,
|
||||
limit: int | None = None,
|
||||
offset: int = 0,
|
||||
) -> list[dict]:
|
||||
"""Read immutable detector outputs available to a bounded replay investigation."""
|
||||
statement = (
|
||||
"SELECT r.value AS reading FROM peripheral_sample.scada_replay_batches b "
|
||||
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'outputs','[]'::jsonb)) r "
|
||||
"WHERE b.project_id=%s AND b.replay_id=%s AND b.sequence<=%s AND b.stage='detected' "
|
||||
"AND r.value->>'device_id'=ANY(%s) "
|
||||
"AND (r.value->>'time')::timestamptz>=%s AND (r.value->>'time')::timestamptz<=%s "
|
||||
"ORDER BY (r.value->>'time')::timestamptz,r.value->>'device_id'"
|
||||
)
|
||||
params: tuple = (
|
||||
self.project_id, str(replay_id), before_sequence, device_ids, start, end,
|
||||
)
|
||||
if limit is not None:
|
||||
statement += " LIMIT %s OFFSET %s"
|
||||
params += (limit, offset)
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(statement, params)
|
||||
return [row["reading"] for row in await cur.fetchall()]
|
||||
|
||||
async def latest_outputs(self, replay_id, through: datetime, limit: int = 500) -> list[dict]:
|
||||
"""Return the latest committed cleaned output for each replay device."""
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT DISTINCT ON(r.value->>'device_id') r.value AS reading "
|
||||
"FROM peripheral_sample.scada_replay_batches b "
|
||||
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'outputs','[]'::jsonb)) r "
|
||||
"WHERE b.project_id=%s AND b.replay_id=%s AND b.sequence>0 AND b.stage IN ('cleaned','detected') "
|
||||
"AND (r.value->>'time')::timestamptz<=%s "
|
||||
"ORDER BY r.value->>'device_id',(r.value->>'time')::timestamptz DESC LIMIT %s",
|
||||
(self.project_id, str(replay_id), through, limit),
|
||||
)
|
||||
return [row["reading"] for row in await cur.fetchall()]
|
||||
@@ -1,5 +1,5 @@
|
||||
from typing import List, Any
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timedelta
|
||||
from collections import defaultdict
|
||||
from psycopg import AsyncConnection, Connection, sql
|
||||
from psycopg.rows import dict_row
|
||||
@@ -9,6 +9,91 @@ class ScadaRepository:
|
||||
VALUE_FIELDS = frozenset({"monitored_value", "cleaned_value"})
|
||||
RESULT_COLUMNS = "time, device_id, monitored_value, cleaned_value"
|
||||
|
||||
@staticmethod
|
||||
async def begin_replay_baseline_snapshot(conn: AsyncConnection) -> None:
|
||||
"""Set the caller's source transaction to a repeatable read-only snapshot."""
|
||||
await conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
|
||||
|
||||
@staticmethod
|
||||
async def get_warmup_values(
|
||||
conn: AsyncConnection,
|
||||
device_ids: list[str],
|
||||
before: datetime,
|
||||
per_device_limit: int,
|
||||
) -> list[dict]:
|
||||
"""Return the latest raw values before a replay starts for detector warmup."""
|
||||
async with conn.cursor() as cur:
|
||||
await cur.execute(
|
||||
"SELECT d.device_id,h.monitored_value FROM unnest(%s::text[]) d(device_id) "
|
||||
"CROSS JOIN LATERAL (SELECT monitored_value,time FROM scada.measurements "
|
||||
"WHERE device_id=d.device_id AND time<%s ORDER BY time DESC LIMIT %s) h "
|
||||
"ORDER BY d.device_id,h.time",
|
||||
(device_ids, before, per_device_limit),
|
||||
)
|
||||
return [dict(row) for row in await cur.fetchall()]
|
||||
|
||||
@staticmethod
|
||||
async def get_raw_page(
|
||||
conn: AsyncConnection, device_ids: list[str], start: datetime,
|
||||
end: datetime, through: datetime, limit: int,
|
||||
after: tuple[datetime, str] | None = None, *, frame: bool = False,
|
||||
) -> list[dict]:
|
||||
"""Read raw only, with a stable composite key and a bounded frame/page."""
|
||||
predicate = "device_id=ANY(%s) AND time>=%s AND time<%s AND time<=%s"
|
||||
params: tuple = (device_ids, start, end, through)
|
||||
if after is not None:
|
||||
predicate += " AND (time,device_id)>(%s,%s)"
|
||||
params += after
|
||||
if frame:
|
||||
statement = (
|
||||
"WITH next_time AS (SELECT min(time) AS time FROM scada.measurements WHERE "
|
||||
+ predicate + ") SELECT m.time,m.device_id,m.monitored_value,"
|
||||
"md5(COALESCE(m.monitored_value::text,'NULL')) AS source_hash "
|
||||
"FROM scada.measurements m WHERE m.device_id=ANY(%s) "
|
||||
"AND m.time=(SELECT time FROM next_time) ORDER BY m.time,m.device_id LIMIT %s"
|
||||
)
|
||||
params += (device_ids, limit)
|
||||
else:
|
||||
statement = (
|
||||
"SELECT time,device_id,monitored_value,"
|
||||
"md5(COALESCE(monitored_value::text,'NULL')) AS source_hash "
|
||||
"FROM scada.measurements WHERE " + predicate
|
||||
+ " ORDER BY time,device_id LIMIT %s"
|
||||
)
|
||||
params += (limit,)
|
||||
async with conn.cursor() as cur:
|
||||
await cur.execute(statement, params)
|
||||
return [dict(row) for row in await cur.fetchall()]
|
||||
|
||||
@staticmethod
|
||||
async def apply_replay_cleaning(conn: AsyncConnection, rows: list[dict]) -> None:
|
||||
"""Guard every raw version; update corrected values without inserting raw."""
|
||||
if not rows:
|
||||
return
|
||||
async with conn.transaction():
|
||||
async with conn.cursor() as cur:
|
||||
await cur.execute(
|
||||
"SELECT m.time,m.device_id,md5(COALESCE(m.monitored_value::text,'NULL')) AS source_hash "
|
||||
"FROM scada.measurements m JOIN unnest(%s::timestamptz[],%s::text[]) "
|
||||
"b(time,device_id) USING(time,device_id) FOR UPDATE OF m",
|
||||
([r["time"] for r in rows], [r["device_id"] for r in rows]),
|
||||
)
|
||||
existing = {(r["time"].isoformat(), r["device_id"]): r["source_hash"]
|
||||
for r in await cur.fetchall()}
|
||||
if len(existing) != len(rows) or any(
|
||||
existing.get((datetime.fromisoformat(r["time"]).isoformat(), r["device_id"]))
|
||||
!= r["source_hash"] for r in rows
|
||||
):
|
||||
raise ValueError("SOURCE_CHANGED: SCADA 原始值在处理期间发生变化")
|
||||
await cur.execute(
|
||||
"UPDATE scada.measurements m SET cleaned_value=b.value "
|
||||
"FROM unnest(%s::timestamptz[],%s::text[],%s::double precision[]) "
|
||||
"b(time,device_id,value) WHERE m.time=b.time AND m.device_id=b.device_id "
|
||||
"AND m.cleaned_value IS DISTINCT FROM b.value",
|
||||
([r["time"] for r in rows], [r["device_id"] for r in rows],
|
||||
[r["cleaned_value"] for r in rows]),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
async def insert_scada_batch(conn: AsyncConnection, data: List[dict]):
|
||||
if not data:
|
||||
@@ -46,6 +131,29 @@ class ScadaRepository:
|
||||
)
|
||||
return await cur.fetchall()
|
||||
|
||||
@staticmethod
|
||||
async def get_recovery_scada_readings(
|
||||
conn: AsyncConnection, device_ids: list[str], incident_at: datetime,
|
||||
returned_at: datetime, through: datetime, *, baseline_limit: int = 100_001,
|
||||
) -> list[dict]:
|
||||
"""Read the causal reference and exactly the first stored frame after T."""
|
||||
if not device_ids or through <= returned_at:
|
||||
return []
|
||||
async with conn.cursor() as cur:
|
||||
await cur.execute(
|
||||
"WITH next_frame AS (SELECT min(time) AS time FROM scada.measurements "
|
||||
"WHERE device_id=ANY(%s) AND time>%s AND time<=%s), "
|
||||
"reference AS (SELECT time,device_id,monitored_value,cleaned_value "
|
||||
"FROM scada.measurements WHERE device_id=ANY(%s) AND time>=%s AND time<%s "
|
||||
"ORDER BY time,device_id LIMIT %s) "
|
||||
"SELECT time,device_id,monitored_value,cleaned_value FROM reference UNION ALL "
|
||||
"SELECT m.time,m.device_id,m.monitored_value,m.cleaned_value FROM scada.measurements m "
|
||||
"JOIN next_frame f ON m.time=f.time WHERE m.device_id=ANY(%s) ORDER BY time,device_id",
|
||||
(device_ids, returned_at, through, device_ids, incident_at - timedelta(days=15),
|
||||
incident_at, baseline_limit, device_ids),
|
||||
)
|
||||
return [dict(row) for row in await cur.fetchall()]
|
||||
|
||||
@staticmethod
|
||||
async def get_latest_scada_readings(
|
||||
conn: AsyncConnection,
|
||||
|
||||
@@ -194,6 +194,7 @@ class HttpOperationalActionDispatcher:
|
||||
raise ValueError("invalid external project or timestamp")
|
||||
receipt = ActionReceipt(
|
||||
adapter="peripheral_sample", receipt_id=str(order.work_order_id),
|
||||
# This is the creation action's completion, not field repair.
|
||||
submitted_at=order.created_at, completed_at=order.created_at,
|
||||
external_status=order.status,
|
||||
detail="外围工单已创建,现场工作进度由外围系统继续同步",
|
||||
|
||||
+81
-19
@@ -4,7 +4,7 @@ from datetime import datetime, timedelta
|
||||
from typing import Annotated, Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import Field, model_validator
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from app.mcp.models import Id, StrictInput
|
||||
|
||||
@@ -15,12 +15,42 @@ MAX_REQUEST_TIMEOUT_SECONDS = 240
|
||||
class CalculationInput(StrictInput):
|
||||
timeout_seconds: int = Field(
|
||||
default=120, ge=1, le=MAX_REQUEST_TIMEOUT_SECONDS, strict=True,
|
||||
description="本次HTTP请求响应等待预算,秒,默认120秒;不是后台计算执行时限,受工具目录中的请求上限约束;后台任务通过状态和结果工具查询",
|
||||
description="本次HTTP请求响应等待预算,秒,默认120秒;后台执行时限由作业系统管理,受工具目录中的请求上限约束;后台任务通过状态和结果工具查询",
|
||||
)
|
||||
|
||||
|
||||
class NodeCoverageGroup(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
group_id: Id = Field(max_length=80)
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
node_ids: list[Id] = Field(min_length=1, max_length=10000)
|
||||
|
||||
|
||||
class NodeCoverageInput(CalculationInput):
|
||||
groups: list[NodeCoverageGroup] = Field(min_length=1, max_length=8,
|
||||
description="已核实的模型节点集合;各组为需要比较的业务或空间集合,可重叠,位置通过模型节点关联获取")
|
||||
concave_ratio: float = Field(default=0.25, ge=0, le=1, allow_inf_nan=False,
|
||||
description="凹包比例;越接近1越平滑接近凸包,参数只决定示意形状")
|
||||
buffer_m: float = Field(default=25, gt=0, le=1000, allow_inf_nan=False,
|
||||
description="展示缓冲距离,米;用于几何展示")
|
||||
split_by_connectivity: bool = Field(default=False, strict=True,
|
||||
description="需要按模型连接分组时设true;地理分布比较默认false,供水关系按拓扑核验")
|
||||
excluded_link_ids: list[Id] = Field(default_factory=list, max_length=200,
|
||||
description="连通分组时额外视为断开的管段/阀门,例如已有隔离建议中的关阀ID;仅影响绘图分组")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def bounded_groups(self):
|
||||
if len({group.group_id for group in self.groups}) != len(self.groups):
|
||||
raise ValueError("coverage group IDs must be distinct")
|
||||
if len({node for group in self.groups for node in group.node_ids}) > 10000:
|
||||
raise ValueError("coverage supports at most 10000 distinct nodes")
|
||||
if self.excluded_link_ids and not self.split_by_connectivity:
|
||||
raise ValueError("excluded_link_ids requires split_by_connectivity")
|
||||
return self
|
||||
|
||||
|
||||
class BurstDetectionInput(CalculationInput):
|
||||
target_time: datetime = Field(description="带时区的侦测时刻;读取此前15天同周期压力数据,不是单小时定位")
|
||||
target_time: datetime = Field(description="带时区的侦测时刻;读取此前15天同周期压力数据,返回压力异常诊断")
|
||||
sensor_nodes: list[Id] = Field(min_length=5, max_length=20, description="已核实的压力监测节点ID,至少5个完整传感器")
|
||||
sampling_interval_minutes: int = Field(default=15, ge=1, le=60)
|
||||
|
||||
@@ -37,14 +67,14 @@ class BurstDetectionInput(CalculationInput):
|
||||
|
||||
class BurstLocationInput(CalculationInput):
|
||||
data_source: Literal["monitoring", "simulation"] = "monitoring"
|
||||
simulation_run_id: UUID | None = Field(default=None, description="已有分析运行UUID,不是方案名称;simulation模式只读取已有异常结果及模型基线")
|
||||
burst_leakage: float = Field(gt=0, allow_inf_nan=False, description="当前爆管定位算法必需的假定漏水流量,WNTR SI单位m³/s;可用有数据和方法依据的估计或明确给定的情景假设,说明来源与不确定性,不要求现场实测确认;不能任填数值")
|
||||
pressure_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="压力监测设备关联的模型节点ID,不是设备查询编号;省略时自动读取当前项目SCADA配置中的压力测点,不需要猜测节点ID")
|
||||
flow_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="流量监测设备关联的模型要素ID,不是设备查询编号")
|
||||
simulation_run_id: UUID | None = Field(default=None, description="已有分析运行UUID;simulation模式只读取已有异常结果及模型基线")
|
||||
burst_leakage: float | None = Field(default=None, gt=0, allow_inf_nan=False, description="水力定位必需的假定漏水流量,WNTR SI单位m³/s;使用有依据的估计或明确给定的情景假设,保留估计方法与输入来源。仅 Server 明确开放的候选定位路径可省略,该路径直接返回候选管段与相似度;其他项目和时段省略将被拒绝")
|
||||
pressure_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="压力监测设备关联的模型节点ID,查询编号通过设备映射转换;省略时读取项目配置的压力测点")
|
||||
flow_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="流量监测设备关联的模型要素ID,由设备映射取得")
|
||||
scada_burst_start: datetime
|
||||
scada_burst_end: datetime
|
||||
scada_normal_start: datetime | None = Field(default=None, description="monitoring模式正常对照窗开始,须与结束时间成对提供;均省略时用爆管窗前一天相同时段,不代表该时段已确认正常")
|
||||
scada_normal_end: datetime | None = Field(default=None, description="monitoring模式正常对照窗结束,须与开始时间成对提供;simulation模式不填写,使用同时间窗模型基线")
|
||||
scada_normal_start: datetime | None = Field(default=None, description="monitoring模式正常对照窗开始,须与结束时间成对提供;均省略时用爆管窗前一天相同时段,使用前须核对正常对照有效性")
|
||||
scada_normal_end: datetime | None = Field(default=None, description="monitoring模式正常对照窗结束,须与开始时间成对提供;simulation模式省略此项,使用同时间窗模型基线")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_windows(self):
|
||||
@@ -67,17 +97,34 @@ class BurstLocationInput(CalculationInput):
|
||||
return self
|
||||
|
||||
|
||||
class ContaminantSourceTraceInput(CalculationInput):
|
||||
candidate_node_ids: list[Id] = Field(min_length=1, max_length=20,
|
||||
description="根据事件位置、附近资产或水质设备映射取得的候选模型节点;Server核验节点存在后稳定返回一个候选。当前为临时定位接口,尚未执行污染反演")
|
||||
start: datetime = Field(description="带时区的事件核查窗口开始")
|
||||
end: datetime = Field(description="带时区的事件核查窗口结束")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_window(self):
|
||||
if self.start.tzinfo is None or self.end.tzinfo is None:
|
||||
raise ValueError("timestamps require explicit timezone")
|
||||
if not timedelta(0) < self.end - self.start <= timedelta(days=1):
|
||||
raise ValueError("window must be positive and at most 24 hours")
|
||||
if len(set(self.candidate_node_ids)) != len(self.candidate_node_ids):
|
||||
raise ValueError("candidate_node_ids must be distinct")
|
||||
return self
|
||||
|
||||
|
||||
class ValveIsolationInput(CalculationInput):
|
||||
accident_element: list[Id] = Field(min_length=1, max_length=20, description="经核实或明确作为候选的事故管段/节点ID")
|
||||
accident_element_type: Literal["auto", "link", "node"] = Field(default="auto", description="事故要素类型;auto优先识别管段,再识别节点;节点与管段重号时,节点分析须指定node,纯管段分析可指定link")
|
||||
disabled_valves: list[Id] = Field(default_factory=list, max_length=200, description="已知无法关闭的阀门;未提供不代表现场确认全部可用")
|
||||
disabled_valves: list[Id] = Field(default_factory=list, max_length=200, description="已知无法关闭的阀门;现场可操作性按执行前核验记录确认")
|
||||
|
||||
|
||||
class LeakageIdentificationInput(CalculationInput):
|
||||
scada_start: datetime
|
||||
scada_end: datetime
|
||||
sensor_nodes: list[Id] = Field(min_length=1, max_length=20)
|
||||
q_sum: float = Field(gt=0, allow_inf_nan=False, description="有依据的总漏损流量约束,m³/s,不得使用无依据默认值")
|
||||
q_sum: float = Field(gt=0, allow_inf_nan=False, description="有依据的总漏损流量约束,m³/s,保留数据来源与估算方法")
|
||||
start_time: float = Field(ge=0, lt=24, allow_inf_nan=False, description="模型起算小时;需与SCADA时间窗及模型时钟对齐")
|
||||
timestep: int = Field(default=15, ge=5, le=60, description="计算步长,分钟;需与观测采样对应")
|
||||
dma_count: int = Field(ge=1, le=20)
|
||||
@@ -99,7 +146,7 @@ class LeakageIdentificationInput(CalculationInput):
|
||||
class ScenarioInput(CalculationInput):
|
||||
start_time: datetime
|
||||
duration: int = Field(default=900, ge=60, le=86400, description="模拟时长,秒;首次优先短时间窗")
|
||||
output_node_ids: list[Id] = Field(min_length=1, max_length=20, description="需要返回结果的节点;不把局部结果解释为全网无影响")
|
||||
output_node_ids: list[Id] = Field(min_length=1, max_length=20, description="需要返回结果的节点;结论范围限定为返回对象")
|
||||
output_link_ids: list[Id] = Field(default_factory=list, max_length=20)
|
||||
|
||||
@model_validator(mode="after")
|
||||
@@ -110,19 +157,34 @@ class ScenarioInput(CalculationInput):
|
||||
|
||||
|
||||
class ContaminantSimulationInput(ScenarioInput):
|
||||
source: Id = Field(description="假设污染源节点;这是正向传播模拟,不是自动反演污染源")
|
||||
source: Id = Field(description="假设污染源节点;这是正向传播模拟,结果按给定源解释")
|
||||
concentration: float = Field(gt=0, allow_inf_nan=False, description="假设源浓度,mg/L")
|
||||
pattern: Id | None = Field(default=None, description="已存在的污染源时间模式ID;不填为持续恒定源")
|
||||
pattern: Id | None = Field(default=None, description="已存在的污染源时间模式ID;省略时采用持续恒定源")
|
||||
include_impact_coverage: bool = Field(default=False, strict=True,
|
||||
description="估计处置前的污染传播范围;在筛选展示节点前汇总全网节点整个模拟时段的浓度,保存影响节点、首个超过阈值的报告时刻、峰值及地图覆盖")
|
||||
impact_threshold_mg_l: float = Field(default=0, ge=0, allow_inf_nan=False,
|
||||
description="模型影响范围的浓度阈值,mg/L,按严格大于判定;默认0表示模型中出现正浓度,仅为情景展示条件,不是饮用水安全限值。沿用模型初始水质与反应条件")
|
||||
|
||||
|
||||
class ValveClosureInput(ScenarioInput):
|
||||
valves: list[Id] = Field(min_length=1, max_length=20, description="在临时模型中关闭的阀门,不下发现场控制")
|
||||
valves: list[Id] = Field(min_length=1, max_length=20, description="在临时模型中关闭的阀门,现场操作另走审批流程")
|
||||
|
||||
|
||||
class FlushingInput(ScenarioInput):
|
||||
drainage_node_id: Id
|
||||
flushing_flow_m3h: float = Field(gt=0, allow_inf_nan=False, description="明确的冲洗流量,m³/h;不使用无依据的自动排放量")
|
||||
closed_valves: list[Id] = Field(default_factory=list, max_length=20)
|
||||
drainage_node_id: Id = Field(description="已核实的冲洗排放节点;清洁水来源、流向及污染水处置须在方案中说明")
|
||||
flushing_flow_m3h: float = Field(gt=0, allow_inf_nan=False, description="明确的冲洗流量,m³/h;提供流量来源与估算依据")
|
||||
closed_valves: list[Id] = Field(default_factory=list, max_length=20,
|
||||
description="本次冲洗情景中保持关闭的完整阀门清单,用于隔离用户或约束流向")
|
||||
opened_valves: list[Id] = Field(default_factory=list, max_length=20,
|
||||
description="本次冲洗情景中明确打开的阀门,如清洁水进水和导流阀;不表示现场已开阀或恢复用户供水。未列阀门沿用当前模型状态,不继承上次独立模拟")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def distinct_valve_states(self):
|
||||
if any(len(values) != len(set(values)) for values in (self.closed_valves, self.opened_valves)):
|
||||
raise ValueError("flushing valve lists must contain distinct IDs")
|
||||
if set(self.closed_valves) & set(self.opened_valves):
|
||||
raise ValueError("a flushing valve cannot be both opened and closed")
|
||||
return self
|
||||
|
||||
|
||||
NonnegativeValue = Annotated[float, Field(ge=0, allow_inf_nan=False)]
|
||||
@@ -130,7 +192,7 @@ PumpSeries = Annotated[list[NonnegativeValue], Field(min_length=1, max_length=96
|
||||
|
||||
|
||||
class PressureRegulationInput(ScenarioInput):
|
||||
pump_control: dict[Id, PumpSeries] = Field(min_length=1, max_length=20, description="已登记泵ID及运行序列;工频值为0/1,变频值为频率Hz,每项对应模型水力步长,不自动优化")
|
||||
pump_control: dict[Id, PumpSeries] = Field(min_length=1, max_length=20, description="已登记泵ID及运行序列;工频值为0/1,变频值为频率Hz,每项对应模型水力步长,按给定序列评估")
|
||||
tank_init_level: dict[Id, Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(default_factory=dict, max_length=20, description="水箱初始水位,须在模型允许范围内")
|
||||
|
||||
|
||||
|
||||
@@ -71,6 +71,10 @@ def _calculate(name: str, parameters: dict[str, Any], network: str, username: st
|
||||
from app.services.valve_isolation import analyze_valve_isolation
|
||||
|
||||
return analyze_valve_isolation(network=network, **parameters)
|
||||
if name == "generate_node_coverage":
|
||||
from app.services.node_coverage import generate_node_coverage
|
||||
|
||||
return generate_node_coverage(network=network, **parameters)
|
||||
raise ValueError("Calculation is not registered")
|
||||
finally:
|
||||
_slots.release()
|
||||
@@ -94,3 +98,46 @@ async def bounded_calculation(function, *args, **kwargs):
|
||||
finally:
|
||||
_slots.release()
|
||||
return await asyncio.to_thread(run)
|
||||
|
||||
|
||||
async def calculate_replay_burst_location(
|
||||
parameters: dict[str, Any], *, network: str, username: str,
|
||||
) -> dict[str, Any]:
|
||||
"""Run location with Server-built frozen replay series and no live SCADA query."""
|
||||
from app.services.burst_location import run_burst_location_by_network
|
||||
|
||||
return await bounded_calculation(
|
||||
run_burst_location_by_network,
|
||||
network=network,
|
||||
username=username,
|
||||
scheme_name=None,
|
||||
final_candidates_csv_path=None,
|
||||
**parameters,
|
||||
)
|
||||
|
||||
|
||||
async def calculate_replay_burst_simulation(args: BaseModel, *, network: str, username: str) -> dict[str, Any]:
|
||||
"""An independent current-model scenario, with no live SCADA substitution."""
|
||||
return await bounded_calculation(
|
||||
calculate_scenario, "run_burst_simulation",
|
||||
args.model_dump(exclude={"context_handle", "timeout_seconds"}, exclude_none=True),
|
||||
network=network, username=username, model_only=True,
|
||||
)
|
||||
|
||||
|
||||
async def calculate_replay_contaminant_simulation(args: BaseModel, *, network: str, username: str) -> dict[str, Any]:
|
||||
"""Estimate propagation in a current-model scenario without live SCADA substitution."""
|
||||
return await bounded_calculation(
|
||||
calculate_scenario, "run_contaminant_simulation",
|
||||
args.model_dump(exclude={"context_handle", "timeout_seconds"}, exclude_none=True),
|
||||
network=network, username=username, model_only=True,
|
||||
)
|
||||
|
||||
|
||||
async def calculate_replay_flushing(args: BaseModel, *, network: str, username: str) -> dict[str, Any]:
|
||||
"""Evaluate a model-only flushing plan without live SCADA or field execution."""
|
||||
return await bounded_calculation(
|
||||
calculate_scenario, "analyze_flushing",
|
||||
args.model_dump(exclude={"context_handle", "timeout_seconds"}, exclude_none=True),
|
||||
network=network, username=username, model_only=True,
|
||||
)
|
||||
|
||||
+20
-3
@@ -32,6 +32,8 @@ from app.mcp.catalog import TOOLS
|
||||
from app.mcp.config import McpSettings
|
||||
from app.mcp.contexts import resolve_context, verify_task
|
||||
from app.mcp.models import QueryResult
|
||||
from app.mcp.display_time import display_query_result, display_time_text
|
||||
from app.mcp.scada_catalog import compact_scada_catalog
|
||||
from app.mcp.errors import CalculationCapacityExceeded
|
||||
from app.mcp.scope import project_scope
|
||||
from app.mcp.query_service import execute_query, task_service, authorize_query
|
||||
@@ -43,6 +45,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def tool_error(code, message):
|
||||
message = display_time_text(message)
|
||||
return types.CallToolResult(
|
||||
isError=True,
|
||||
content=[types.TextContent(type="text", text=message)],
|
||||
@@ -358,7 +361,8 @@ class McpQueryApplication:
|
||||
result = QueryResult.model_validate(previous["result"])
|
||||
else:
|
||||
from app.mcp.calculation_jobs import should_queue, submit_calculation
|
||||
if should_queue(tool, args):
|
||||
queued = should_queue(tool, args, project_code=getattr(scope.project, "project_code", None))
|
||||
if queued:
|
||||
result = await submit_calculation(
|
||||
scope, binding, tool, args, call_id,
|
||||
job_timeout_seconds=self.settings.max_calculation_timeout_seconds,
|
||||
@@ -377,7 +381,7 @@ class McpQueryApplication:
|
||||
"RESULT_TOO_LARGE",
|
||||
"Narrow the query or request a smaller page",
|
||||
)
|
||||
if binding.task_id and not should_queue(tool, args) and name not in {
|
||||
if binding.task_id and not queued and name not in {
|
||||
"get_calculation_job", "get_calculation_result", "cancel_calculation_job", "list_calculation_jobs",
|
||||
"get_task_context",
|
||||
"get_task_evidence",
|
||||
@@ -422,11 +426,20 @@ class McpQueryApplication:
|
||||
parameters,
|
||||
result.model_dump(mode="json"),
|
||||
)
|
||||
output = result.model_dump(mode="json")
|
||||
output = display_query_result(result.model_dump(mode="json"))
|
||||
content = []
|
||||
from app.mcp.extended_service import ARTIFACT_TOOLS
|
||||
# Complete SCADA inputs must reach the model intact, not as a first-page preview.
|
||||
# The shared 8 MiB result ceiling above still applies before persistence and output.
|
||||
complete_scada = name == "query_scada_readings" and args.device_ids is None
|
||||
if (not include_full and name == "list_scada_devices"
|
||||
and len(json.dumps(output, ensure_ascii=False).encode()) > 12000):
|
||||
preview = compact_scada_catalog(output, budget=12000)
|
||||
if preview is not None:
|
||||
output = preview
|
||||
if (
|
||||
not include_full
|
||||
and not complete_scada
|
||||
and name not in ARTIFACT_TOOLS
|
||||
and len(json.dumps(output, ensure_ascii=False).encode()) > 12000
|
||||
):
|
||||
@@ -441,6 +454,10 @@ class McpQueryApplication:
|
||||
"resource_uri": uri,
|
||||
"preview": "The complete query snapshot is saved in task evidence",
|
||||
}
|
||||
from app.mcp.presentation_preview import saved_presentation_preview
|
||||
presentation = saved_presentation_preview(result)
|
||||
if presentation is not None:
|
||||
output["data"]["presentation"] = display_query_result(presentation)
|
||||
output["truncated"] = True
|
||||
output["data_gaps"] = [
|
||||
*result.data_gaps,
|
||||
|
||||
@@ -29,7 +29,11 @@ def restore_binding(document: dict) -> BoundContext:
|
||||
return BoundContext.model_validate({**document, "context_handle": "durable-calculation", "expires_at": datetime.max.replace(tzinfo=timezone.utc)})
|
||||
|
||||
|
||||
def should_queue(tool, args) -> bool:
|
||||
def should_queue(tool, args, *, project_code: str | None = None) -> bool:
|
||||
if tool.name == "run_burst_location":
|
||||
from app.services.burst_location_demo import is_location_demo
|
||||
if is_location_demo(project_code, args):
|
||||
return False
|
||||
return tool.name in QUEUED_TOOLS and (not tool.mutating or not args.dry_run)
|
||||
|
||||
|
||||
@@ -39,7 +43,7 @@ def envelope(source: str, data, *, job_id="list", page=None) -> QueryResult:
|
||||
|
||||
|
||||
def public_job(row: dict) -> dict:
|
||||
fields = {"job_id", "status", "tool", "created_at", "started_at", "finished_at", "updated_at", "error", "queue_expires_at", "result_expires_at"}
|
||||
fields = {"job_id", "status", "tool", "created_at", "started_at", "finished_at", "updated_at", "error", "result_expires_at"}
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
expiry = row.get("result_expires_at")
|
||||
available = row.get("status") == "succeeded" and (expiry is None or expiry > datetime.now(timezone.utc))
|
||||
@@ -162,6 +166,19 @@ async def read_calculation_job(scope, binding, tool, args) -> QueryResult:
|
||||
raise HTTPException(422, "offset requires a collection data_path")
|
||||
else:
|
||||
result.data = value
|
||||
if not args.data_path and binding.task_id:
|
||||
from app.mcp.presentation_preview import saved_burst_result_preview, saved_presentation_preview
|
||||
|
||||
presentation = saved_presentation_preview(result)
|
||||
if presentation is not None and isinstance(result.data, dict):
|
||||
result.data = {**result.data, "presentation": presentation}
|
||||
if len(result.model_dump_json().encode()) > 11000:
|
||||
preview = saved_burst_result_preview(result, str(binding.task_id))
|
||||
if preview is not None:
|
||||
result.data = preview
|
||||
result.truncated = True
|
||||
result.data_gaps = [*result.data_gaps,
|
||||
"Inline burst impact summary only; full node comparisons and geometry are saved in task evidence. Read the evidence resource or select a narrower data_path."]
|
||||
if len(result.model_dump_json().encode()) > 11000:
|
||||
raise HTTPException(413, "Select a narrower data_path or smaller result page")
|
||||
return result
|
||||
|
||||
+48
-35
@@ -6,7 +6,7 @@ from mcp import types
|
||||
from app.mcp import models as m
|
||||
from app.mcp import calculation_models as jobs
|
||||
from app.mcp import extended_models as e
|
||||
from app.mcp.analysis_models import BurstDetectionInput, BurstLocationInput, ValveIsolationInput, LeakageIdentificationInput
|
||||
from app.mcp.analysis_models import BurstDetectionInput, BurstLocationInput, ContaminantSourceTraceInput, ValveIsolationInput, LeakageIdentificationInput, NodeCoverageInput
|
||||
from app.mcp.analysis_models import ScenarioInput, ContaminantSimulationInput, ValveClosureInput, FlushingInput, PressureRegulationInput, SensorPlacementInput
|
||||
|
||||
|
||||
@@ -23,22 +23,33 @@ class QueryTool:
|
||||
|
||||
def protocol_tool(self) -> types.Tool:
|
||||
if self.mutating:
|
||||
suffix = "。默认dry_run只预览;执行必须携带预览摘要、相同request_id并通过宿主原生许可。仅项目普通聊天可用,后台调查和reader禁止;失败或超时先核实状态,不自动重试。"
|
||||
suffix = "。默认dry_run只预览;执行必须携带预览摘要、相同request_id并通过宿主原生许可。由项目成员的普通聊天执行写入;失败或超时先核实状态,再按原请求接续。"
|
||||
elif self.name == "run_simulation":
|
||||
suffix = "。任务调用复用证据留存。"
|
||||
elif self.computation:
|
||||
suffix = "。仅计算建议,不保存方案、不控制设备;任务调用复用证据留存。"
|
||||
suffix = "。返回分析建议并留存任务证据;正式方案与现场操作按各自业务流程执行。"
|
||||
elif self.name == "query_nearby_elements":
|
||||
suffix = "。仅查询当前已授权项目的模型管网,按距离和元素ID稳定排序;按需分页,不自动遍历全量;空间邻近不代表管网连通或供水关系。"
|
||||
suffix = "。仅查询当前已授权项目的模型管网,按距离和元素ID稳定排序;按核查目标分页;连接与供水关系由拓扑分析核验。"
|
||||
elif self.name == "list_scada_devices":
|
||||
suffix = (
|
||||
"。默认每页100个,最多500个;page.has_more=true时继续递增offset读取后续页面。"
|
||||
"大目录内联保留本页全部设备的编号、类型和单位,模型映射及坐标用get_scada_device核验。"
|
||||
)
|
||||
elif self.name == "query_scada_readings":
|
||||
suffix = (
|
||||
"。首次调查省略device_ids,Server读取完整设备属性并自动翻页读取窗口内全部授权观测,"
|
||||
"返回devices、readings和coverage,包括缺测及冻结范围外设备;全量结果完整内联。"
|
||||
"显式device_ids仅用于后续局部复核,不能替代全量核查。"
|
||||
)
|
||||
else:
|
||||
suffix = (
|
||||
"。先限定对象和时间范围,再读取明细;集合默认20条,按需逐页扩大,"
|
||||
"不要自动遍历全量;截断或非零offset不能证明完整范围无异常。"
|
||||
"按当前目标读取必要页面,并说明截断和分页覆盖范围。"
|
||||
)
|
||||
return types.Tool(
|
||||
name=self.name,
|
||||
description=self.description + suffix + (
|
||||
"。执行返回持久job_id;排队回执不是分析证据,使用get_calculation_job查询状态、get_calculation_result分页读取成功结果,不重复提交。"
|
||||
"。返回queued/running和job_id时,使用get_calculation_job查询状态、get_calculation_result读取成功结果,排队回执用于跟踪;直接返回分析结果时立即读取。"
|
||||
if self.name in {"run_burst_location", "run_leakage_identification", "run_burst_detection", "run_simulation", "run_burst_simulation", "run_contaminant_simulation", "analyze_valve_closure", "analyze_flushing", "analyze_pressure_regulation", "optimize_sensor_placement", "save_simulation_scenario", "create_sensor_placement"}
|
||||
else ""
|
||||
),
|
||||
@@ -51,16 +62,18 @@ class QueryTool:
|
||||
|
||||
|
||||
_specs = [
|
||||
("run_simulation", "按水力步长在在线业务模型上执行指定时间段的实时模拟,不复制数据库;更新该时段的模型条件及实时基线,返回指定节点和管段结果;duration单位为秒,不创建方案或正式分析运行,结果不是实测", ScenarioInput, "simulation.run", "analysis", False, True),
|
||||
("run_contaminant_simulation", "在隔离模型中模拟假设污染源的浓度传播;返回指定节点/管段结果,不反演污染源,不代表实测污染范围", ContaminantSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_valve_closure", "在隔离模型中模拟关阀的水力影响,返回指定节点/管段结果;不操作真实阀门", ValveClosureInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_flushing", "在隔离模型中评估指定排放节点、冲洗流量和关阀组合的水力结果;不是现场冲洗或消毒合格证明", FlushingInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_pressure_regulation", "在隔离模型中评估给定调泵序列和水箱初始水位的压力影响;不自动优化调度,不实际控制泵", PressureRegulationInput, "simulation.run", "analysis", False, True),
|
||||
("optimize_sensor_placement", "使用灵敏度或K-means算法推荐压力监测点;仅返回候选节点,不保存正式方案,不保证污染溯源可观测性", SensorPlacementInput, "optimization.run", "analysis", False, True),
|
||||
("run_leakage_identification", "基于SCADA压力与模型进行DMA漏损分配识别,需有依据的总漏损流量和时间对齐;返回区域估计,不是已确认漏点。内部执行遗传算法与水力计算", LeakageIdentificationInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_detection", "爆管异常侦测:使用至少5个完整压力传感器此前15天数据;只提示压力异常,不定位、不证明爆管", BurstDetectionInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_location", "爆管候选定位:对比异常与正常压力,可辅助流量;burst_leakage必需,可使用有依据的估计或明确给定的情景假设,不要求现场确诊;本工具不反演总漏水量。压力节点可省略,自动读取项目SCADA压力测点;monitoring正常窗可成对省略,默认前一天同时间段,须说明该对照假设。simulation须有已有分析运行UUID并读取同窗模型基线,不是实测;定位内部会执行水力计算,不创建新的事件方案", BurstLocationInput, "burst.run", "analysis", False, True),
|
||||
("analyze_valve_isolation", "按事故候选要素分析拓扑隔离边界、建议关闭阀门和受影响节点;候选分析无需等待现场确诊。输出不是污染扩散或水力影响范围;现场操作前须核实阀门映射及可用性,本工具不执行关阀", ValveIsolationInput, "simulation.run", "analysis", False, True),
|
||||
("run_contaminant_source_trace", "水污染候选溯源:提交由事件位置或监测映射取得的候选模型节点及事件窗,快速返回确定的已核验节点。当前为临时稳定选择,正式反演算法后续接入;不生成置信度。取得located_node后先展示节点、保存阶段结论并提现场核查/采样处置工单;工单与污染传播模拟、accident_element_type=node的隔离和影响分析并行,结果补入同一调查,不因算法来源停止", ContaminantSourceTraceInput, "simulation.run", "analysis", False, True),
|
||||
("run_simulation", "按水力步长在在线业务模型上执行指定时间段的实时模拟;更新该时段的模型条件及实时基线,返回指定节点和管段结果;duration单位为秒,输出为在线模型结果,正式分析成果通过方案工具保存", ScenarioInput, "simulation.run", "analysis", False, True),
|
||||
("run_contaminant_simulation", "在隔离模型中模拟假设污染源的浓度传播;include_impact_coverage=true时汇总全网节点整个模拟时段超过impact_threshold_mg_l的累计范围、首个超阈值报告时刻和峰值,并保存地图证据。阈值为情景展示条件,非饮用水安全限值;回放按当前模型独立估计,不读取实时SCADA。定位后先提现场核查/采样处置工单,工单审核派发与本计算并行,结果补入同一调查", ContaminantSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_valve_closure", "在隔离模型中模拟关阀的水力影响,返回指定节点/管段结果;现场阀门操作按业务审批执行", ValveClosureInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_flushing", "在独立模型中评估指定排放节点、流量及明确开关阀组合的冲洗水力结果;未列阀门沿用当前模型,不继承上次模拟;不判定污染清除或恢复供水", FlushingInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_pressure_regulation", "在隔离模型中评估给定调泵序列和水箱初始水位的压力影响;输出为给定方案评估,现场调度按审批执行", PressureRegulationInput, "simulation.run", "analysis", False, True),
|
||||
("optimize_sensor_placement", "使用灵敏度或K-means算法推荐压力监测点;返回候选节点,正式部署另行保存并核验观测能力", SensorPlacementInput, "optimization.run", "analysis", False, True),
|
||||
("run_leakage_identification", "基于SCADA压力与模型进行DMA漏损分配识别,需有依据的总漏损流量和时间对齐;返回待现场核验的区域估计。内部执行遗传算法与水力计算", LeakageIdentificationInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_detection", "爆管异常侦测:使用至少5个完整压力传感器此前15天数据;返回压力异常信号,候选管段由定位工具分析", BurstDetectionInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_location", "爆管候选定位:对比异常与正常压力,可辅助流量;依据现有异常形成候选,水力算法使用有依据的burst_leakage作为输入。任务上下文location_capability.available=true时可省略漏量,Server直接返回候选管段与相似度,随后继续地图展示、隔离边界与影响分析。压力节点可省略,自动读取项目配置;monitoring正常窗省略时默认前一天同时间段,须说明假设;回放必须显式提供冻结正常窗。simulation须有已有分析运行UUID及同窗模型基线,结果标注模型来源", BurstLocationInput, "burst.run", "analysis", False, True),
|
||||
("analyze_valve_isolation", "按事故候选要素分析拓扑隔离边界、建议关闭阀门和受影响节点;候选分析无需等待现场确诊。输出为拓扑隔离与供水失源范围;水力影响另用关阀模拟评估,现场操作前核实阀门映射及可用性", ValveIsolationInput, "simulation.run", "analysis", False, True),
|
||||
("generate_node_coverage", "为当前项目已核实的模型节点集合生成空间覆盖示意,适用于候选分布、监测覆盖比较、区域成员、隔离或模拟结果节点等,支持各类项目分析。支持最多8组、合计10000个唯一节点,从真实GIS查询坐标;监测设备、投诉或地址须先关联模型节点。可按连接分组,记录凹包/缓冲参数、缺失节点和display_only;结果为节点空间覆盖示意,保留分组及几何覆盖范围。返回result.map_payload及coverage,任务中用set_overlay引用本次evidence_id、省略data展示;地图通过展示工具发布", NodeCoverageInput, "webgis.view", "analysis", False, True),
|
||||
(
|
||||
"get_element_properties",
|
||||
"读取当前项目的单个管网元素属性",
|
||||
@@ -70,14 +83,14 @@ _specs = [
|
||||
),
|
||||
(
|
||||
"list_elements",
|
||||
"分页读取管网元素,不自动读取全网",
|
||||
"按查询目标分页读取管网元素",
|
||||
m.ElementPageInput,
|
||||
"webgis.view",
|
||||
"asset",
|
||||
),
|
||||
(
|
||||
"query_nearby_elements",
|
||||
"查询坐标附近的模型元素,默认pipe;center为EPSG:4326 [经度,纬度],可直接使用geocode.center;radius_m默认200米、最大10000米,limit默认5、最大100,支持offset。按点到完整几何的WGS84椭球最短距离筛选并排序,返回element_id、distance_m和属性;不查询原始GIS管线、不自动扩大半径、不解析道路交叉点",
|
||||
"查询坐标附近的模型元素,默认pipe;center为EPSG:4326 [经度,纬度],可直接使用geocode.center;radius_m默认200米、最大10000米,limit默认5、最大100,支持offset。按点到完整几何的WGS84椭球最短距离筛选并排序,返回element_id、distance_m和属性;查询范围为当前模型管网;原始GIS、范围调整与歧义地点按各自入口核验",
|
||||
m.NearbyElementsInput,
|
||||
"webgis.view",
|
||||
"asset",
|
||||
@@ -99,7 +112,7 @@ _specs = [
|
||||
(
|
||||
"list_scada_devices",
|
||||
"分页读取SCADA设备和物理单位",
|
||||
m.PageInput,
|
||||
m.ScadaDeviceListInput,
|
||||
"scada.view",
|
||||
"asset",
|
||||
),
|
||||
@@ -113,7 +126,7 @@ _specs = [
|
||||
),
|
||||
(
|
||||
"query_model_baseline_series",
|
||||
"读取最多20个元素的模型基线,不能称为实测",
|
||||
"读取最多20个元素的模型基线,结果标注模型来源",
|
||||
m.BaselineSeriesInput,
|
||||
"simulation.view",
|
||||
"model_baseline",
|
||||
@@ -205,7 +218,7 @@ _specs = [
|
||||
),
|
||||
(
|
||||
"get_task_context",
|
||||
"读取绑定任务的有界摘要及handling_skill元数据;section=skill读取Server按任务workflow分发的业务处置Skill正文与版本,属于流程指引而非事件证据;其他section选择task/report/evidence/actions/progress/access_requests/plan/run,data_path按返回路径读取嵌套字段,limit/offset分页;report_revision保留选定报告版本",
|
||||
"读取绑定任务的有界摘要及handling_skill元数据;section=skill读取Server按任务workflow分发的业务处置Skill正文与版本,属于流程指引而非事件证据;其他section选择task/report/evidence/actions/progress/access_requests/plan/run,data_path按返回路径读取嵌套字段,limit/offset分页;progress/report图节点仅为流程参考标记,旧执行态兼容转换,不代表实时进度;实际执行状态读run、动作状态读actions;report_revision保留选定报告版本",
|
||||
m.TaskContextInput,
|
||||
"operational_task.view",
|
||||
"task",
|
||||
@@ -229,40 +242,40 @@ _specs = [
|
||||
]
|
||||
_specs += [
|
||||
("list_regions", "分页查询当前项目区域/DMA列表", m.PageInput, "webgis.view", "asset"),
|
||||
("get_region", "查询当前项目区域边界,返回EPSG:4326;不等于供水拓扑", e.RegionInput, "webgis.view", "asset"),
|
||||
("get_region", "查询当前项目区域边界,返回EPSG:4326几何;供水关系按拓扑核验", e.RegionInput, "webgis.view", "asset"),
|
||||
("get_region_nodes", "分页查询区域配置的成员节点,未配置时使用区域几何筛选", e.RegionInput, "webgis.view", "asset"),
|
||||
("get_node_connections", "分页查询节点直接连接的管线,不自动遍历全网", e.NodeConnectionsInput, "webgis.view", "asset"),
|
||||
("query_network_extent", "按EPSG:4326矩形范围分页查询当前项目模型几何;无GIS配置不跨项目回退", e.ExtentInput, "webgis.view", "asset"),
|
||||
("get_node_connections", "按目标节点分页查询直接连接的管线", e.NodeConnectionsInput, "webgis.view", "asset"),
|
||||
("query_network_extent", "按EPSG:4326矩形范围分页查询当前项目模型几何;GIS缺失时返回当前项目的具体缺口", e.ExtentInput, "webgis.view", "asset"),
|
||||
("list_major_nodes", "按关联管道最小管径查询骨干节点,经纬度EPSG:4326", e.MajorNodesInput, "webgis.view", "asset"),
|
||||
("get_model_configuration", "读取模式、曲线、控制规则、需水量、水质及反应等模型配置;集合按页返回", e.ModelConfigurationInput, "webgis.view", "asset"),
|
||||
("list_model_configurations", "分页读取模式或曲线ID", e.ConfigurationListInput, "webgis.view", "asset"),
|
||||
("search_project_places", "按项目配置的行政区搜索地名,最多10个候选,依赖天地图配置", e.PlaceSearchInput, "webgis.view", "asset"),
|
||||
("reverse_geocode", "把EPSG:4326地图取点转为项目范围内地名,保留输入坐标", e.ReverseGeocodeInput, "webgis.view", "asset"),
|
||||
("get_project_info", "读取当前项目概况与地图范围,不返回数据库路由或凭据", m.StrictInput, "webgis.view", "asset"),
|
||||
("get_project_info", "读取当前项目概况与地图范围,公开字段限于项目业务概况", m.StrictInput, "webgis.view", "asset"),
|
||||
("export_project_model", "从当前项目数据库导出INP或ChangeSet压缩文件,由宿主保存到会话文件", e.ProjectExportInput, "webgis.view", "asset"),
|
||||
("get_sensor_placement_candidate", "读取监测点候选节点工程属性", e.SensorCandidateInput, "optimization.run", "asset"),
|
||||
("export_sensor_placement", "导出已有监测点方案的Excel工程清单,由宿主保存到会话文件", e.SensorExportInput, "optimization.run", "analysis"),
|
||||
("list_task_schedules", "分页读取当前项目定时任务配置,不启动调度", m.PageInput, "operational_task.view", "task"),
|
||||
("list_task_schedules", "分页读取当前项目定时任务配置,执行状态由调度系统管理", m.PageInput, "operational_task.view", "task"),
|
||||
("get_task_run_history", "分页读取指定项目任务的运行历史", e.TaskHistoryInput, "operational_task.view", "task"),
|
||||
("get_mutation_status", "读取当前身份和会话内的写入执行状态;started只表示已启动,超时后须核实目标数据,不自动重试", e.MutationStatusInput, "webgis.view", "task"),
|
||||
("calculate_demand_allocation", "按管长权重分配总需水量到节点、区域或全网,只计算不修改模型,按页返回", e.DemandAllocationInput, "webgis.view", "analysis", False, True),
|
||||
("run_burst_simulation", "在隔离模型中模拟假设管道破口面积的水力影响;不是爆管侦测或定位,不保存正式方案", e.BurstSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("edit_model_element", "预览或增删改模型节点、管段、泵阀、水池和区域;仅修改模型,不控制现场设备", e.ModelElementMutationInput, "webgis.edit", "asset", False, False, True),
|
||||
("get_mutation_status", "读取当前身份和会话内的写入执行状态;started只表示已启动,超时后先核实目标数据,再按原请求接续", e.MutationStatusInput, "webgis.view", "task"),
|
||||
("calculate_demand_allocation", "按管长权重分配总需水量到节点、区域或全网,返回分配建议,模型变更通过业务写入执行,按页返回", e.DemandAllocationInput, "webgis.view", "analysis", False, True),
|
||||
("run_burst_simulation", "在隔离模型中模拟管道破口;include_impact_coverage=true时按起算同刻已有在线模拟计算全网节点压力下降及1/3/5米水头分档覆盖,保存为展示证据;回放仅用当前模型情景,不替代冻结观测", e.BurstSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("edit_model_element", "预览或增删改模型节点、管段、泵阀、水池和区域;变更范围为业务模型,现场设备操作按对应流程执行", e.ModelElementMutationInput, "webgis.edit", "asset", False, False, True),
|
||||
("edit_model_configuration", "预览或修改模型需水量、模式、曲线、控制规则、水质及模拟选项", e.ConfigurationMutationInput, "webgis.edit", "asset", False, False, True),
|
||||
("clean_scada_readings", "预览或清洗明确设备与24小时内时间窗的数据,写入cleaned_value", e.ScadaCleaningInput, "scada.clean", "observed", False, False, True),
|
||||
("write_scada_readings", "预览或补录/修正SCADA数据,要求真实来源依据;不把模拟当实测", e.ScadaWriteInput, "scada.clean", "observed", False, False, True),
|
||||
("write_scada_readings", "预览或补录/修正SCADA数据,要求真实来源依据;观测来源按真实记录核验", e.ScadaWriteInput, "scada.clean", "observed", False, False, True),
|
||||
("delete_scada_readings", "预览或删除明确设备与24小时内时间窗的SCADA记录", e.ScadaDeleteInput, "scada.clean", "observed", False, False, True),
|
||||
("save_simulation_scenario", "预览或重新计算并保存正式隔离场景成果;有真实计算开销,不保存模型生成的观测数组", e.SavedScenarioInput, "simulation.run", "analysis", False, False, True),
|
||||
("save_simulation_scenario", "预览或重新计算并保存正式隔离场景成果;有真实计算开销,观测数据按真实来源记录", e.SavedScenarioInput, "simulation.run", "analysis", False, False, True),
|
||||
("create_sensor_placement", "预览或计算并保存正式压力监测点选址方案", e.SensorCreateInput, "optimization.run", "analysis", False, False, True),
|
||||
("update_sensor_placement", "预览或修改本人有权编辑的监测点方案,保留旧点位乐观校验", e.SensorUpdateInput, "optimization.run", "analysis", False, False, True),
|
||||
("create_operational_task", "预览或创建项目任务,不伪造Agent运行完成", e.TaskCreateInput, "operational_task.write", "task", False, False, True),
|
||||
("create_operational_task", "预览或创建项目任务,Agent运行终态由执行系统管理", e.TaskCreateInput, "operational_task.write", "task", False, False, True),
|
||||
("create_task_schedule", "预览或创建定时任务配置", e.ScheduleCreateInput, "operational_task.write", "task", False, False, True),
|
||||
("update_task_schedule", "预览或启停定时任务配置,校验版本", e.ScheduleUpdateInput, "operational_task.write", "task", False, False, True),
|
||||
("retry_operational_task", "预览或重新入队已终止任务;不自动启动新Agent会话", e.TaskRetryInput, "operational_task.write", "task", False, False, True),
|
||||
("retry_operational_task", "预览或重新入队已终止任务;后续会话由任务调度系统创建", e.TaskRetryInput, "operational_task.write", "task", False, False, True),
|
||||
("cancel_task_run", "预览或取消指定任务运行,校验运行版本", e.TaskCancelInput, "operational_task.write", "task", False, False, True),
|
||||
]
|
||||
_specs += [
|
||||
("get_calculation_job", "读取当前身份与会话的持久计算状态;排队回执不是分析证据", jobs.CalculationJobInput, "webgis.view", "task"),
|
||||
("get_calculation_job", "读取当前身份与会话的持久计算状态;排队回执用于跟踪,分析依据读取成功结果", jobs.CalculationJobInput, "webgis.view", "task"),
|
||||
("get_calculation_result", "读取成功计算的原始结果;data_path选择嵌套集合,limit/offset按需分页", jobs.CalculationResultInput, "webgis.view", "task"),
|
||||
("cancel_calculation_job", "请求取消本人当前会话的计算;运行中取消由独立计算进程处理,须继续核实最终状态", jobs.CalculationJobInput, "webgis.view", "task"),
|
||||
("list_calculation_jobs", "分页读取本人当前会话及任务范围的计算状态", jobs.CalculationJobsInput, "webgis.view", "task"),
|
||||
|
||||
@@ -15,6 +15,7 @@ from app.mcp.models import BindInput, BoundContext
|
||||
from app.mcp.scope import QueryScope
|
||||
from app.mcp.config import McpSettings
|
||||
from mcp.server.auth.provider import AccessToken
|
||||
from app.mcp.work_order_skills import skill_name_for_task
|
||||
|
||||
|
||||
def task_repository(scope: QueryScope) -> OperationalTaskRepository:
|
||||
@@ -30,6 +31,9 @@ async def verify_task(scope: QueryScope, binding: BoundContext) -> None:
|
||||
task = await repo.get_task(str(binding.task_id))
|
||||
if not task:
|
||||
raise HTTPException(404, "Task is not in this project")
|
||||
binding.replay_context = task.get("source", {}).get("replay_context")
|
||||
if skill_name_for_task(task.get("workflow"), binding.replay_context) == "burst-event-handling":
|
||||
binding.tool_cap = [name for name in binding.tool_cap if name != "run_burst_location"]
|
||||
if binding.purpose == "task_collaboration":
|
||||
return
|
||||
if OPERATIONAL_TASK_EXECUTE not in permissions:
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Project model-facing timestamps after saving the original query evidence."""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
DISPLAY_TIME_ZONE = "Asia/Shanghai"
|
||||
DISPLAY_OFFSET = timezone(timedelta(hours=8))
|
||||
_INSTANT = re.compile(
|
||||
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?P<fraction>\.\d+)?(?:Z|[+-]\d{2}:\d{2})"
|
||||
)
|
||||
_PROSE_FIELDS = frozenset({
|
||||
"title", "summary", "detail", "description", "notice", "label", "message",
|
||||
"current_conclusion", "conclusion_boundary", "recommendations", "pending_items",
|
||||
"missing_data", "required_data", "impact", "scope", "fallback_text", "text",
|
||||
})
|
||||
_OPAQUE_FIELDS = frozenset({"id", "cursor", "url", "uri", "path", "sha256", "instructions", "code", "sql"})
|
||||
_OPAQUE_SUFFIXES = ("_id", "_ids", "_ref", "_refs", "_cursor", "_digest", "_sha256", "_url", "_uri", "_path")
|
||||
|
||||
|
||||
def _opaque(key: str) -> bool:
|
||||
return key in _OPAQUE_FIELDS or key.endswith(_OPAQUE_SUFFIXES)
|
||||
|
||||
|
||||
def _display_instant(value: str) -> str:
|
||||
match = _INSTANT.fullmatch(value)
|
||||
if match is None:
|
||||
return value
|
||||
try:
|
||||
local = datetime.fromisoformat(value).astimezone(DISPLAY_OFFSET)
|
||||
except (ValueError, OverflowError):
|
||||
return value
|
||||
fraction = match.group("fraction") or ""
|
||||
return (f"{local.year:04d}-{local.month:02d}-{local.day:02d}T"
|
||||
f"{local.hour:02d}:{local.minute:02d}:{local.second:02d}{fraction}+08:00")
|
||||
|
||||
|
||||
def display_time_text(value: str) -> str:
|
||||
"""Convert explicit instants in prose while retaining links and quoted code."""
|
||||
parts = re.split(r"(https?://\S+|tjwater://\S+|`[^`]*`)", value)
|
||||
for index in range(0, len(parts), 2):
|
||||
parts[index] = re.sub(
|
||||
r"(?<![A-Za-z0-9_/+:.-])" + _INSTANT.pattern + r"(?![A-Za-z0-9_:+/-])",
|
||||
lambda match: _display_instant(match.group()), parts[index],
|
||||
)
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def display_query_result(value: Any, *, field: str = "") -> Any:
|
||||
"""Keep instants, fractions and identifiers intact; change display offsets."""
|
||||
if _opaque(field):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
path = value.get("selected_path") or []
|
||||
opaque_selection = isinstance(path, list) and bool(path) and _opaque(str(path[-1]))
|
||||
return {key: item if key == "value" and opaque_selection else display_query_result(item, field=key)
|
||||
for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [display_query_result(item, field=field) for item in value]
|
||||
if isinstance(value, str):
|
||||
if field in _PROSE_FIELDS:
|
||||
return display_time_text(value)
|
||||
return _display_instant(value) if len(value) <= 128 else value
|
||||
return value
|
||||
@@ -28,7 +28,7 @@ class NodeConnectionsInput(PageInput):
|
||||
|
||||
class ExtentInput(PageInput):
|
||||
element_type: Literal["junction", "pipe", "valve", "pump", "tank", "reservoir"] = "pipe"
|
||||
bounds: tuple[float, float, float, float] = Field(description="EPSG:4326 [west,south,east,north]; 不跨180度经线")
|
||||
bounds: tuple[float, float, float, float] = Field(description="EPSG:4326 [west,south,east,north];经度范围满足west<=east")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_bounds(self):
|
||||
@@ -116,8 +116,9 @@ class DemandAllocationInput(CalculationInput):
|
||||
|
||||
|
||||
class BurstSimulationInput(ScenarioInput):
|
||||
include_impact_coverage: bool = Field(default=False, strict=True, description="比较起算时刻全网节点与已有在线模拟压力,按下降1≤Δ<3、3≤Δ<5、Δ≥5米水头生成事件影响覆盖;不重算或覆盖在线基线")
|
||||
burst_pipe_ids: list[Id] = Field(min_length=1, max_length=20)
|
||||
orifice_areas_cm2: list[Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(min_length=1, max_length=20, description="破口面积cm²,不是流量;必须有依据")
|
||||
orifice_areas_cm2: list[Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(min_length=1, max_length=20, description="破口面积,单位cm²;保留面积来源及估算依据")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def paired_bursts(self):
|
||||
@@ -177,7 +178,7 @@ class ConfigurationMutationInput(MutationInput):
|
||||
class ScadaCleaningInput(MutationInput):
|
||||
start: datetime
|
||||
end: datetime
|
||||
device_ids: list[Id] = Field(min_length=1, max_length=20, description="必须显式列出设备;不支持all")
|
||||
device_ids: list[Id] = Field(min_length=1, max_length=20, description="显式列出本次处理的实际设备ID")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_window(self):
|
||||
@@ -205,7 +206,7 @@ class ScadaRecord(BaseModel):
|
||||
class ScadaWriteInput(MutationInput):
|
||||
operation: Literal["insert", "update"]
|
||||
records: list[ScadaRecord] = Field(min_length=1, max_length=500)
|
||||
provenance: str = Field(min_length=1, max_length=2000, description="补录/修正的真实来源与依据,不得把模型值冒充实测")
|
||||
provenance: str = Field(min_length=1, max_length=2000, description="补录/修正观测的真实来源与依据,模型结果按其来源另行记录")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def bounded_records(self):
|
||||
|
||||
+20
-2
@@ -21,6 +21,10 @@ class PageInput(StrictInput):
|
||||
offset: int = Field(default=0, ge=0, le=10000)
|
||||
|
||||
|
||||
class ScadaDeviceListInput(PageInput):
|
||||
limit: int = Field(default=100, ge=1, le=500)
|
||||
|
||||
|
||||
class TaskContextInput(PageInput):
|
||||
section: Literal["summary", "task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill", "questions"] = "summary"
|
||||
report_revision: int | None = Field(default=None, ge=1)
|
||||
@@ -102,8 +106,20 @@ class WindowInput(PageInput):
|
||||
|
||||
|
||||
class ScadaInput(WindowInput):
|
||||
device_ids: list[Id] = Field(min_length=1, max_length=20)
|
||||
field: Literal["monitored_value", "cleaned_value"] = "monitored_value"
|
||||
device_ids: list[Id] | None = Field(
|
||||
default=None, min_length=1, max_length=20,
|
||||
description="Omit to read the complete SCADA catalogue and all authorized readings in the window; explicit IDs select a paginated detail query.",
|
||||
)
|
||||
field: Literal["monitored_value", "cleaned_value"] = Field(
|
||||
default="monitored_value",
|
||||
description="Field for explicit-device detail queries; complete all-device reads return both raw and cleaned values.",
|
||||
)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def complete_query_starts_at_zero(self):
|
||||
if self.device_ids is None and self.offset:
|
||||
raise ValueError("All-device SCADA queries read every page internally; offset must be zero")
|
||||
return self
|
||||
|
||||
|
||||
class BaselineSeriesInput(WindowInput):
|
||||
@@ -255,6 +271,7 @@ class PageInfo(BaseModel):
|
||||
|
||||
class QueryResult(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
display_time_zone: Literal["Asia/Shanghai"] = "Asia/Shanghai"
|
||||
source: str
|
||||
source_ref: str
|
||||
queried_at: datetime
|
||||
@@ -291,5 +308,6 @@ class BoundContext(BaseModel):
|
||||
task_id: UUID | None = None
|
||||
run_id: UUID | None = None
|
||||
execution_request_id: UUID | None = None
|
||||
replay_context: dict | None = None
|
||||
expires_at: datetime
|
||||
revoked_at: datetime | None = None
|
||||
|
||||
@@ -189,11 +189,12 @@ def saved_scenario(network: str, username: str, args: BaseModel) -> Any:
|
||||
modify_valve_opening={valve: 0.0 for valve in parameters["valves"]})
|
||||
if args.scenario == "flushing":
|
||||
from app.native.wndb.model.options import get_option
|
||||
from app.mcp.scenario_service import flushing_valve_controls
|
||||
if get_option(network).get("UNITS") not in {"LPS", "CMH"}:
|
||||
raise ValueError("Flushing supports LPS or CMH models only")
|
||||
return scenarios.flushing_analysis(**common, username=username,
|
||||
drainage_node_ID=parameters["drainage_node_id"], flushing_flow=parameters["flushing_flow_m3h"],
|
||||
modify_valve_opening={valve: 0.0 for valve in parameters["closed_valves"]})
|
||||
valve_control=flushing_valve_controls(parameters))
|
||||
mappings = query_corresponding_element_id_and_query_id(network)
|
||||
controls = parameters["pump_control"]
|
||||
if set(controls) - (set(mappings.fixed_pumps) | set(mappings.variable_pumps)):
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Keep a saved geometry discoverable even when its inline response is shortened."""
|
||||
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
from app.mcp.models import QueryResult
|
||||
|
||||
|
||||
def saved_burst_result_preview(result: QueryResult, task_id: str) -> dict | None:
|
||||
"""Keep durable full-network results usable without copying node arrays inline."""
|
||||
if result.source != "run_burst_simulation" or not result.evidence_id:
|
||||
return None
|
||||
data = result.data if isinstance(result.data, dict) else {}
|
||||
analysis = data.get("result", {})
|
||||
impact = analysis.get("impact") if isinstance(analysis, dict) else None
|
||||
if not isinstance(impact, dict):
|
||||
return None
|
||||
summary = {key: value for key, value in impact.items()
|
||||
if key not in {"comparisons", "grades", "unavailable_node_ids"}}
|
||||
summary["unavailable_node_count"] = len(impact.get("unavailable_node_ids", []))
|
||||
summary["grades"] = [
|
||||
{**{key: value for key, value in grade.items() if key != "node_ids"},
|
||||
"node_count": len(grade.get("node_ids", []))} for grade in impact.get("grades", [])
|
||||
]
|
||||
preview = {
|
||||
"resource_uri": f"tjwater://tasks/{task_id}/evidence/{result.evidence_id}",
|
||||
"parameters": data.get("parameters", {}), "result": {"impact": summary},
|
||||
}
|
||||
presentation = saved_presentation_preview(result)
|
||||
if presentation is not None:
|
||||
preview["presentation"] = presentation
|
||||
return preview
|
||||
|
||||
|
||||
def saved_presentation_preview(result: QueryResult) -> dict | None:
|
||||
if result.source not in {"generate_node_coverage", "run_burst_simulation", "run_contaminant_simulation"} or not result.evidence_id:
|
||||
return None
|
||||
analysis = result.data.get("result") if isinstance(result.data, dict) else None
|
||||
if not isinstance(analysis, dict):
|
||||
return None
|
||||
payload = analysis.get("map_payload")
|
||||
if not isinstance(payload, dict) or not payload.get("features"):
|
||||
return None
|
||||
coverage = analysis.get("coverage", {})
|
||||
burst = result.source == "run_burst_simulation"
|
||||
contaminant = result.source == "run_contaminant_simulation"
|
||||
if burst and analysis.get("impact", {}).get("phase") != "after_burst_before_intervention":
|
||||
return None
|
||||
if contaminant and analysis.get("impact", {}).get("phase") != "after_contamination_before_intervention":
|
||||
return None
|
||||
return {
|
||||
"command": {
|
||||
"action": "set_overlay", "mode": "geometry", "evidence_id": result.evidence_id,
|
||||
"overlay_id": f"node-coverage-{uuid5(NAMESPACE_URL, result.evidence_id)}",
|
||||
"title": ("爆管后、处置前的预计影响范围(米水头)" if burst else
|
||||
"水污染传播预计影响范围(处置前情景)" if contaminant else "节点空间覆盖示意"),
|
||||
},
|
||||
"display_only": True,
|
||||
"coverage_status": coverage.get("status"),
|
||||
"groups": {role: {
|
||||
"node_count": len(group.get("node_ids", [])),
|
||||
"mapped_node_count": group.get("mapped_node_count", 0),
|
||||
"missing_node_count": len(group.get("missing_node_ids", [])),
|
||||
} for role, group in coverage.get("groups", {}).items()},
|
||||
}
|
||||
+154
-4
@@ -12,6 +12,7 @@ from datetime import datetime, timezone
|
||||
from uuid import uuid4
|
||||
from fastapi import HTTPException
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
from app.core.config import settings
|
||||
from app.auth.permissions import permissions_for_context
|
||||
from app.infra.db.postgresql import read_queries as reads
|
||||
from app.infra.db.postgresql.spatial_queries import nearby_elements
|
||||
@@ -20,6 +21,7 @@ from app.infra.db.postgresql.gis_attributes import GisAttributeRepository
|
||||
from app.infra.db.postgresql.network_settings import NetworkSettingsRepository
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
from app.services.burst_location_demo import location_demo_result
|
||||
from app.mcp.models import QueryResult, PageInfo
|
||||
from app.mcp.scenario_service import SCENARIO_TOOLS
|
||||
from app.mcp.work_order_skills import read_work_order_skill
|
||||
@@ -51,6 +53,7 @@ def task_service(scope: QueryScope) -> OperationalTaskService:
|
||||
OperationalTaskRepository(scope.connection, scope.project.project_id),
|
||||
scope.project,
|
||||
set(permissions_for_context(scope.project)),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
|
||||
|
||||
@@ -66,8 +69,39 @@ def page_result(rows, args):
|
||||
async def authorize_query(
|
||||
scope: QueryScope, binding: BoundContext, tool: QueryTool, args: BaseModel
|
||||
) -> None:
|
||||
replay_context = getattr(binding, "replay_context", None)
|
||||
if tool.name not in binding.tool_cap:
|
||||
raise HTTPException(403, "Query is outside the captured tool catalog")
|
||||
if replay_context:
|
||||
from app.services.scada_replay_queries import check_replay_window
|
||||
allowed = {"get_task_context", "get_task_evidence", "list_scada_devices", "get_scada_device",
|
||||
"query_scada_readings", "get_network_schema", "run_burst_location", "run_contaminant_source_trace",
|
||||
"get_element_properties", "list_elements", "query_nearby_elements", "get_network_options",
|
||||
"get_node_connections", "get_element_gis_properties", "query_network_extent",
|
||||
"list_regions", "get_region", "get_region_nodes",
|
||||
"analyze_valve_isolation", "generate_node_coverage", "run_burst_simulation", "run_contaminant_simulation", "analyze_flushing",
|
||||
"get_calculation_job", "get_calculation_result", "list_calculation_jobs"}
|
||||
if tool.name not in allowed:
|
||||
raise HTTPException(403, "回放调查仅开放冻结 SCADA、资产、拓扑分析及指定模型情景;时序观测请使用 query_scada_readings")
|
||||
if tool.name == "run_burst_simulation":
|
||||
if not args.include_impact_coverage:
|
||||
raise HTTPException(422, "回放爆管情景必须启用影响覆盖,按当前模型估计并保存同刻在线基线")
|
||||
check_replay_window(replay_context, args.start_time, args.start_time)
|
||||
if tool.name == "run_contaminant_simulation":
|
||||
if not args.include_impact_coverage:
|
||||
raise HTTPException(422, "回放污染情景必须启用影响覆盖,按当前模型估计传播范围")
|
||||
check_replay_window(replay_context, args.start_time, args.start_time)
|
||||
if tool.name == "analyze_flushing":
|
||||
check_replay_window(replay_context, args.start_time, args.start_time)
|
||||
if tool.name == "query_scada_readings":
|
||||
check_replay_window(replay_context, args.start, args.end)
|
||||
if tool.name == "run_contaminant_source_trace":
|
||||
check_replay_window(replay_context, args.start, args.end)
|
||||
if tool.name == "run_burst_location":
|
||||
if args.data_source != "monitoring" or args.scada_normal_start is None or args.scada_normal_end is None:
|
||||
raise HTTPException(422, "回放定位只支持显式异常窗和正常对照窗的 monitoring 输入")
|
||||
check_replay_window(replay_context, args.scada_burst_start, args.scada_burst_end)
|
||||
check_replay_window(replay_context, args.scada_normal_start, args.scada_normal_end)
|
||||
permissions = permissions_for_context(scope.project)
|
||||
if tool.mutating:
|
||||
if binding.purpose != "ordinary_chat" or binding.task_id is not None or scope.project.project_role != "member":
|
||||
@@ -85,6 +119,8 @@ async def authorize_query(
|
||||
required = {tool.permission}
|
||||
if tool.name in SCENARIO_TOOLS:
|
||||
required.add("scada.view")
|
||||
if tool.name == "run_burst_simulation" and args.include_impact_coverage:
|
||||
required.add("simulation.view")
|
||||
if tool.name in {"run_burst_detection", "run_burst_location", "run_leakage_identification"}:
|
||||
required.add("simulation.view" if getattr(args, "data_source", "monitoring") == "simulation" else "scada.view")
|
||||
if not required.issubset(permissions) or not required.issubset(binding.permission_cap):
|
||||
@@ -109,8 +145,18 @@ async def authorize_query(
|
||||
if not granted and binding.task_id and source in binding.task_scope_cap:
|
||||
async with scope.connection.transaction():
|
||||
granted = await task_service(scope).can_read_source(binding.task_id, source)
|
||||
if granted and tool.name == "query_scada_readings" and args.device_ids is None:
|
||||
granted = (
|
||||
"scada_devices" in binding.task_scope_cap
|
||||
and await task_service(scope).can_read_source(binding.task_id, "scada_devices")
|
||||
)
|
||||
if not granted:
|
||||
raise HTTPException(403, "Data source permission is required")
|
||||
if tool.name == "run_burst_location" and args.burst_leakage is None:
|
||||
from app.services.burst_location_demo import is_location_demo
|
||||
|
||||
if not is_location_demo(getattr(scope.project, "project_code", None), args):
|
||||
raise HTTPException(422, "水力定位需要有依据的 burst_leakage;仅已开放的候选定位路径可省略漏量")
|
||||
if (
|
||||
tool.name == "query_element_history"
|
||||
and args.mode != "observed"
|
||||
@@ -149,7 +195,87 @@ async def execute_query(
|
||||
gaps = []
|
||||
source_ref = "mcp:" + str(uuid4())
|
||||
evidence_id = None
|
||||
if tool.computation:
|
||||
replay_context = getattr(binding, "replay_context", None)
|
||||
if name == "run_contaminant_source_trace":
|
||||
from app.services.contaminant_source_trace import trace_contaminant_source
|
||||
|
||||
data = {"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
|
||||
"result": await trace_contaminant_source(conn, args)}
|
||||
origin = "synthetic"
|
||||
if data["result"]["located_node"] is None:
|
||||
gaps.append(data["result"]["notice"])
|
||||
elif name == "query_scada_readings" and args.device_ids is None:
|
||||
from app.mcp.scada_readings import read_complete_scada
|
||||
data, units, gaps, origin = await read_complete_scada(scope, binding, args)
|
||||
elif name == "query_scada_readings" and replay_context:
|
||||
from app.services.scada_replay_queries import replay_snapshot_rows
|
||||
rows = await replay_snapshot_rows(conn, scope.project.project_id, replay_context,
|
||||
args.device_ids, args.start, args.end, args.limit + 1, args.offset)
|
||||
data, page, truncated = page_result(rows, args)
|
||||
origin = "synthetic"
|
||||
elif name == "run_burst_simulation" and replay_context:
|
||||
from app.mcp.analysis_service import calculate_replay_burst_simulation
|
||||
|
||||
await conn.commit()
|
||||
async with scope.simulation_routing():
|
||||
result = await calculate_replay_burst_simulation(
|
||||
args, network=scope.project.project_code, username=scope.user.username,
|
||||
)
|
||||
data = {"parameters": args.model_dump(mode="json", exclude={"context_handle"}), "result": result}
|
||||
origin = "synthetic"
|
||||
gaps = ["回放爆管影响为当前模型的独立假设情景,不读取实时SCADA;同刻在线模拟为计算时读取的模型参考,非冻结观测或历史模型重建。未来模拟样本仅为预测。"]
|
||||
elif name == "run_contaminant_simulation" and replay_context:
|
||||
from app.mcp.analysis_service import calculate_replay_contaminant_simulation
|
||||
|
||||
await conn.commit()
|
||||
async with scope.simulation_routing():
|
||||
result = await calculate_replay_contaminant_simulation(
|
||||
args, network=scope.project.project_code, username=scope.user.username,
|
||||
)
|
||||
data = {"parameters": args.model_dump(mode="json", exclude={"context_handle"}), "result": result}
|
||||
origin = "synthetic"
|
||||
gaps = ["回放污染传播为当前模型的独立假设情景,不读取实时SCADA,不代表冻结观测或历史模型重建。未来模拟样本仅为预测;浓度阈值仅用于情景展示,不是饮用水安全限值。"]
|
||||
elif name == "analyze_flushing" and replay_context:
|
||||
from app.mcp.analysis_service import calculate_replay_flushing
|
||||
|
||||
await conn.commit()
|
||||
async with scope.simulation_routing():
|
||||
result = await calculate_replay_flushing(
|
||||
args, network=scope.project.project_code, username=scope.user.username,
|
||||
)
|
||||
data = {"parameters": args.model_dump(mode="json", exclude={"context_handle"}), "result": result}
|
||||
origin = "synthetic"
|
||||
gaps = ["回放冲洗为当前模型的独立水力情景,不读取实时SCADA,不继承上次模拟;未来模拟样本仅为预测,不验证污染清除、水质合格或现场操作。"]
|
||||
elif name == "run_burst_location" and replay_context:
|
||||
from app.mcp.analysis_service import calculate_replay_burst_location
|
||||
from app.services.scada_replay_queries import replay_burst_location_inputs
|
||||
|
||||
parameters = await replay_burst_location_inputs(
|
||||
conn, scope.project.project_id, replay_context, args,
|
||||
)
|
||||
result = await location_demo_result(conn, scope.project.project_code, args)
|
||||
await conn.commit()
|
||||
if result is None:
|
||||
async with scope.simulation_routing():
|
||||
result = await calculate_replay_burst_location(
|
||||
parameters,
|
||||
network=scope.project.project_code,
|
||||
username=scope.user.username,
|
||||
)
|
||||
data = {
|
||||
"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
|
||||
"result": result,
|
||||
}
|
||||
origin = "synthetic"
|
||||
gaps = [result["notice"]] if result.get("notice") else []
|
||||
elif name == "run_burst_location" and (
|
||||
demo_result := await location_demo_result(conn, scope.project.project_code, args)
|
||||
) is not None:
|
||||
data = {"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
|
||||
"result": demo_result}
|
||||
origin = "synthetic"
|
||||
gaps = [demo_result["notice"]] if demo_result.get("notice") else []
|
||||
elif tool.computation:
|
||||
from app.mcp.analysis_service import TIMESCALE_CALCULATIONS, calculate
|
||||
|
||||
# Context/receipt reads may have started an implicit transaction. Release
|
||||
@@ -170,7 +296,7 @@ async def execute_query(
|
||||
}
|
||||
gaps = [{
|
||||
"run_simulation": "Online hydraulic recalculation updated the realtime baseline; results cover requested elements only, not observed SCADA data. No standalone analysis run was saved.",
|
||||
"run_contaminant_simulation": "Forward model simulation for an assumed source, not source inversion or observed contamination; only requested elements are returned.",
|
||||
"run_contaminant_simulation": "Forward model simulation for an assumed source, not source inversion or observed contamination. Selected series only; opt-in impact coverage uses all native nodes over the simulation window, with an explicit display threshold and existing model initial quality/reactions, not a drinking-water safety limit.",
|
||||
"analyze_valve_closure": "Simulated hydraulic impact for requested elements only; no physical valves were operated.",
|
||||
"analyze_flushing": "Simulated flushing hydraulics only; no field action or water-quality clearance was verified.",
|
||||
"analyze_pressure_regulation": "Evaluation of a supplied pump schedule, not automatic optimization or device control; requested elements only.",
|
||||
@@ -179,8 +305,9 @@ async def execute_query(
|
||||
"run_burst_detection": "Pressure anomalies alone do not confirm a burst or locate a pipe.",
|
||||
"run_burst_location": "Candidate ranking depends on the leakage assumption, model and sensor coverage; field confirmation is required.",
|
||||
"analyze_valve_isolation": "Topology-only isolation advice; valve operability, hydraulic impact and field actions are not verified or executed.",
|
||||
"generate_node_coverage": "Derived node coverage for display only; geometry does not establish a fault, contamination, outage or service boundary. Missing GIS positions are reported separately.",
|
||||
"calculate_demand_allocation": "Pipe-length-weighted allocation only; model demands were not modified. Units follow the current model.",
|
||||
"run_burst_simulation": "Forward hydraulic simulation for assumed orifice areas, not detection or confirmed burst location; only requested elements returned.",
|
||||
"run_burst_simulation": "Forward hydraulic simulation for assumed orifice areas; selected series only. Opt-in impact coverage compares all native nodes at the start time with existing online results, using current model settings for baseline units. This is an estimated pre-intervention range, not a confirmed outage or historical model reconstruction.",
|
||||
}[name]]
|
||||
if name == "calculate_demand_allocation":
|
||||
truncated = data["result"]["has_more"] or args.offset > 0
|
||||
@@ -322,6 +449,9 @@ async def execute_query(
|
||||
if hasattr(args, "field") and args.field not in units:
|
||||
gaps.append(f"Unit metadata is unavailable for field {args.field}")
|
||||
else:
|
||||
# Keep the selected field in durable evidence; `value` alone cannot
|
||||
# distinguish raw observations from cleaned observations.
|
||||
data = [{**row, args.field: row["value"]} if "value" in row else row for row in data]
|
||||
metadata = [
|
||||
r
|
||||
for device in ids
|
||||
@@ -403,7 +533,8 @@ async def execute_query(
|
||||
data = await reads.task_context_summary(conn, project_id=binding.project_id,
|
||||
task_id=binding.task_id, report_revision=args.report_revision)
|
||||
skill = read_work_order_skill(
|
||||
data.get("task", {}).get("workflow"), include_content=args.section == "skill"
|
||||
data.get("task", {}).get("workflow"), include_content=args.section == "skill",
|
||||
replay_context=replay_context,
|
||||
)
|
||||
if args.section == "skill":
|
||||
data = {"section": "skill", "skill": skill}
|
||||
@@ -415,6 +546,25 @@ async def execute_query(
|
||||
data["available_sections"] = ["task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill", "questions"]
|
||||
truncated = True
|
||||
gaps.append("Task context is a bounded summary; use section, data_path, limit and offset for detail. Previews are not complete reports.")
|
||||
if skill.get("name") == "water-pollution-event":
|
||||
data["source_trace_capability"] = {
|
||||
"available": "run_contaminant_source_trace" in binding.tool_cap,
|
||||
"tool": "run_contaminant_source_trace",
|
||||
"algorithm_status": "provisional",
|
||||
"required_inputs": ["candidate_node_ids", "start", "end"],
|
||||
"next_steps": ["present_node", "interim_report", "propose_task_action", "run_contaminant_simulation",
|
||||
"source_control", "analyze_valve_isolation", "generate_node_coverage",
|
||||
"analyze_flushing", "field_feedback", "clearance_verification", "report"],
|
||||
"work_order_wait_blocks_analysis": False,
|
||||
"simulation_parameters": {"include_impact_coverage": True},
|
||||
}
|
||||
if (skill.get("name") == "scada-anomaly" and replay_context
|
||||
and "run_burst_location" in binding.tool_cap):
|
||||
from app.services.burst_location_demo import location_demo_context
|
||||
|
||||
data["location_capability"] = location_demo_context(
|
||||
getattr(scope.project, "project_code", None), replay_context,
|
||||
)
|
||||
else:
|
||||
projection = await reads.task_context_page(conn, project_id=binding.project_id, task_id=binding.task_id,
|
||||
section=args.section, report_revision=args.report_revision, data_path=args.data_path, limit=args.limit, offset=args.offset)
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
"""Keep a full page of device identities visible within the inline response budget."""
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
|
||||
def compact_scada_catalog(output: dict[str, Any], budget: int) -> dict[str, Any] | None:
|
||||
"""Project attributes, never rows; full task evidence is persisted before this step."""
|
||||
rows = output.get("data")
|
||||
if output.get("source") != "list_scada_devices" or not isinstance(rows, list):
|
||||
return None
|
||||
if not all(isinstance(row, dict) and "device_id" in row for row in rows):
|
||||
return None
|
||||
fields = ("device_id", "device_type", "measurement_unit")
|
||||
preview = {
|
||||
**output,
|
||||
"data": [{key: row[key] for key in fields if key in row} for row in rows],
|
||||
"truncated": True,
|
||||
"data_gaps": [
|
||||
*output.get("data_gaps", []),
|
||||
"All devices in this page are listed with IDs, types and units; "
|
||||
"other attributes are omitted. Use get_scada_device for model mapping, "
|
||||
"coordinates and other device details. Use page.has_more to check for more devices.",
|
||||
],
|
||||
}
|
||||
if len(json.dumps(preview, ensure_ascii=False).encode()) > budget:
|
||||
return None
|
||||
return preview
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Complete project SCADA reads with explicit frozen-scope and missing-data accounting."""
|
||||
|
||||
import json
|
||||
from collections import Counter
|
||||
|
||||
from fastapi import HTTPException
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
|
||||
from app.mcp.models import BoundContext, ScadaInput
|
||||
from app.mcp.scope import QueryScope
|
||||
from app.infra.db.postgresql import read_queries as reads
|
||||
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
|
||||
from app.services.scada_replay_queries import check_replay_window, check_snapshot_start
|
||||
|
||||
|
||||
MAX_COMPLETE_SCADA_BYTES = 8 * 1024 * 1024
|
||||
|
||||
|
||||
async def read_complete_scada(
|
||||
scope: QueryScope, binding: BoundContext, args: ScadaInput,
|
||||
) -> tuple[dict, dict, list[str], str]:
|
||||
"""Exhaust source pages; never treat a first page or selected device subset as a full scan."""
|
||||
devices = []
|
||||
offset = 0
|
||||
size = 0
|
||||
|
||||
def account(items):
|
||||
nonlocal size
|
||||
size += len(json.dumps(jsonable_encoder(items), ensure_ascii=False).encode())
|
||||
if size > MAX_COMPLETE_SCADA_BYTES:
|
||||
raise HTTPException(413, "Complete SCADA result exceeds 8 MiB; split the time window and retain every device")
|
||||
|
||||
while True:
|
||||
page = await reads.scada_devices(scope.connection, limit=501, offset=offset)
|
||||
batch = page[:500]
|
||||
account(batch)
|
||||
devices.extend(batch)
|
||||
if len(page) <= 500:
|
||||
break
|
||||
offset += len(batch)
|
||||
|
||||
project_device_count = len(devices)
|
||||
context = getattr(binding, "replay_context", None)
|
||||
catalog = {str(device["device_id"]): device for device in devices}
|
||||
outside_scope = []
|
||||
if context:
|
||||
check_replay_window(context, args.start, args.end)
|
||||
repository = ScadaReplayRepository(scope.connection, scope.project.project_id)
|
||||
run = await repository.get(context["replay_id"])
|
||||
if not run:
|
||||
raise HTTPException(404, "SCADA replay was not found in the current project")
|
||||
check_snapshot_start(run, args.start)
|
||||
frozen = {str(device["device_id"]): device for device in run.get("devices", [])}
|
||||
selected = list(dict.fromkeys(run["config"]["device_ids"]))
|
||||
outside_scope = sorted(set(catalog) - set(selected))
|
||||
# Retain devices removed from the current catalogue but present in the frozen source.
|
||||
for device in selected:
|
||||
if device not in catalog:
|
||||
catalog[device] = frozen.get(device, {"device_id": device})
|
||||
devices = list(catalog.values())
|
||||
else:
|
||||
selected = list(catalog)
|
||||
|
||||
rows = []
|
||||
offset = 0
|
||||
# The public limit controls source page size, not completeness of this operation.
|
||||
page_size = min(max(args.limit, 1000), 10000)
|
||||
|
||||
async def read_pages(connection=None):
|
||||
nonlocal offset
|
||||
while selected:
|
||||
if context:
|
||||
page = await repository.detected_outputs(
|
||||
context["replay_id"], selected, args.start, args.end, context["batch_sequence"],
|
||||
limit=page_size + 1, offset=offset,
|
||||
)
|
||||
else:
|
||||
page = await reads.series_rows(
|
||||
connection, source="scada", element_type="junction", ids=selected,
|
||||
start=args.start, end=args.end, field=None,
|
||||
limit=page_size + 1, offset=offset,
|
||||
)
|
||||
batch = page[:page_size]
|
||||
if context:
|
||||
# Match the existing frozen-reading contract; internal source hashes are not observations.
|
||||
fields = ("time", "device_id", "monitored_value", "cleaned_value", "quality", "reason")
|
||||
batch = [{key: row[key] for key in fields if key in row} for row in batch]
|
||||
account(batch)
|
||||
rows.extend(batch)
|
||||
if len(page) <= page_size:
|
||||
break
|
||||
offset += len(batch)
|
||||
|
||||
if context:
|
||||
await read_pages()
|
||||
elif selected:
|
||||
async with scope.timescale() as connection:
|
||||
await read_pages(connection)
|
||||
|
||||
counts = Counter(str(row["device_id"]) for row in rows)
|
||||
missing = sorted(set(selected) - counts.keys())
|
||||
coverage = {
|
||||
"mode": "all_scada", "query_complete": True,
|
||||
"project_device_count": project_device_count, "queried_device_count": len(selected),
|
||||
"devices_with_readings": len(counts), "reading_count": len(rows),
|
||||
"queried_device_ids": selected, "missing_device_ids": missing,
|
||||
"outside_frozen_scope_device_ids": outside_scope,
|
||||
"readings_per_device": {device: counts[device] for device in selected},
|
||||
"replay_id": context["replay_id"] if context else None,
|
||||
"batch_sequence": context["batch_sequence"] if context else None,
|
||||
}
|
||||
if context:
|
||||
coverage["frozen_start"] = (run.get("baseline") or {}).get("start_time") or run["start_time"]
|
||||
data = {"devices": devices, "readings": rows, "coverage": coverage}
|
||||
account([coverage])
|
||||
gaps = []
|
||||
if missing:
|
||||
gaps.append(f"{len(missing)} queried SCADA devices have no readings in this window; see coverage.missing_device_ids")
|
||||
if outside_scope:
|
||||
gaps.append(f"{len(outside_scope)} project devices are outside this frozen replay; see coverage.outside_frozen_scope_device_ids")
|
||||
units = {device: catalog[device]["measurement_unit"] for device in selected
|
||||
if catalog[device].get("measurement_unit")}
|
||||
return data, units, gaps, "synthetic" if context else "system_data"
|
||||
@@ -10,7 +10,14 @@ SCENARIO_TOOLS = frozenset({
|
||||
})
|
||||
|
||||
|
||||
def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, username: str) -> dict[str, Any]:
|
||||
def flushing_valve_controls(parameters: dict[str, Any]) -> dict[str, dict[str, str]]:
|
||||
"""Apply explicit states without changing regulating-valve settings."""
|
||||
return {**{valve: {"status": "CLOSED"} for valve in parameters.get("closed_valves", [])},
|
||||
**{valve: {"status": "OPEN"} for valve in parameters.get("opened_valves", [])}}
|
||||
|
||||
|
||||
def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, username: str,
|
||||
model_only: bool = False) -> dict[str, Any]:
|
||||
from app.services import simulation_scenarios as scenarios
|
||||
|
||||
parameters = dict(parameters)
|
||||
@@ -18,6 +25,8 @@ def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, u
|
||||
links = parameters.pop("output_link_ids")
|
||||
start = parameters.pop("start_time")
|
||||
duration = parameters.pop("duration")
|
||||
include_impact = parameters.pop("include_impact_coverage", False)
|
||||
impact_threshold = parameters.pop("impact_threshold_mg_l", 0)
|
||||
common = dict(name=network, modify_pattern_start_time=start.isoformat(),
|
||||
modify_total_duration=duration, persist_results=False, scheme_name=None)
|
||||
if name == "run_simulation":
|
||||
@@ -27,11 +36,27 @@ def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, u
|
||||
network, start, duration, output_node_ids=nodes, output_link_ids=links,
|
||||
)
|
||||
elif name == "run_burst_simulation":
|
||||
boundary = {}
|
||||
if model_only:
|
||||
from app.infra.db.postgresql.scada import ScadaElementMappings
|
||||
# Replay scenarios use current model assumptions, never mutable live
|
||||
# SCADA readings in place of the investigation's frozen observations.
|
||||
boundary["scada_mappings"] = ScadaElementMappings(
|
||||
reservoirs={}, tanks={}, fixed_pumps={}, variable_pumps={},
|
||||
pressure={}, demand={}, quality={},
|
||||
)
|
||||
result = scenarios.burst_analysis(**common, username=username,
|
||||
burst_ID=parameters["burst_pipe_ids"], burst_size=parameters["orifice_areas_cm2"])
|
||||
burst_ID=parameters["burst_pipe_ids"], burst_size=parameters["orifice_areas_cm2"], **boundary)
|
||||
elif name == "run_contaminant_simulation":
|
||||
boundary = {}
|
||||
if model_only:
|
||||
from app.infra.db.postgresql.scada import ScadaElementMappings
|
||||
boundary["scada_mappings"] = ScadaElementMappings(
|
||||
reservoirs={}, tanks={}, fixed_pumps={}, variable_pumps={},
|
||||
pressure={}, demand={}, quality={},
|
||||
)
|
||||
result = scenarios.contaminant_simulation(**common, username=username,
|
||||
source=parameters["source"], concentration=parameters["concentration"], source_pattern=parameters.get("pattern"))
|
||||
source=parameters["source"], concentration=parameters["concentration"], source_pattern=parameters.get("pattern"), **boundary)
|
||||
elif name == "analyze_valve_closure":
|
||||
result = scenarios.valve_close_analysis(**common,
|
||||
modify_valve_opening={valve: 0.0 for valve in parameters["valves"]})
|
||||
@@ -41,9 +66,16 @@ def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, u
|
||||
from app.native.wndb.model.options import get_option
|
||||
if get_option(network).get("UNITS") not in {"LPS", "CMH"}:
|
||||
raise ValueError("Flushing currently supports LPS or CMH models only")
|
||||
boundary = {}
|
||||
if model_only:
|
||||
from app.infra.db.postgresql.scada import ScadaElementMappings
|
||||
boundary["scada_mappings"] = ScadaElementMappings(
|
||||
reservoirs={}, tanks={}, fixed_pumps={}, variable_pumps={},
|
||||
pressure={}, demand={}, quality={},
|
||||
)
|
||||
result = scenarios.flushing_analysis(**common, username=username,
|
||||
drainage_node_ID=parameters["drainage_node_id"], flushing_flow=parameters["flushing_flow_m3h"],
|
||||
modify_valve_opening={valve: 0.0 for valve in parameters["closed_valves"]})
|
||||
valve_control=flushing_valve_controls(parameters), **boundary)
|
||||
elif name == "analyze_pressure_regulation":
|
||||
from app.services.simulation import query_corresponding_element_id_and_query_id
|
||||
mappings = query_corresponding_element_id_and_query_id(network)
|
||||
@@ -62,7 +94,7 @@ def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, u
|
||||
raise ValueError("Scenario did not return calculation results")
|
||||
selected_nodes = [row for row in result.get("node_results", []) if str(row.get("node")) in nodes]
|
||||
selected_links = [row for row in result.get("link_results", []) if str(row.get("link")) in links]
|
||||
return {
|
||||
output = {
|
||||
"input_source": "online_model_simulation" if name == "run_simulation" else "isolated_model_simulation",
|
||||
"persisted_baseline": name == "run_simulation", "persisted_scheme": False,
|
||||
"start_time": start.isoformat(), "duration_seconds": duration,
|
||||
@@ -72,3 +104,28 @@ def calculate_scenario(name: str, parameters: dict[str, Any], *, network: str, u
|
||||
"missing_nodes": sorted(set(nodes) - {str(row["node"]) for row in selected_nodes}),
|
||||
"missing_links": sorted(set(links) - {str(row["link"]) for row in selected_links})},
|
||||
}
|
||||
if name == "run_burst_simulation" and include_impact:
|
||||
from app.services.burst_impact import estimate_burst_impact
|
||||
impact = estimate_burst_impact(network, start, result, model_only=model_only)
|
||||
impact["coverage"] = {**output["coverage"], **impact.get("coverage", {})}
|
||||
output.update(impact)
|
||||
if name == "run_contaminant_simulation" and include_impact:
|
||||
from app.services.contaminant_impact import estimate_contaminant_impact
|
||||
impact = estimate_contaminant_impact(network, start, duration, result,
|
||||
threshold_mg_l=impact_threshold, model_only=model_only)
|
||||
impact["coverage"] = {**output["coverage"], **impact.get("coverage", {})}
|
||||
output.update(impact)
|
||||
if name == "analyze_flushing":
|
||||
output["flushing_plan"] = {
|
||||
"opened_valves": parameters.get("opened_valves", []),
|
||||
"closed_valves": parameters.get("closed_valves", []),
|
||||
"drainage_node_id": parameters["drainage_node_id"],
|
||||
"flushing_flow_m3h": parameters["flushing_flow_m3h"],
|
||||
"estimated_discharge_volume_m3": parameters["flushing_flow_m3h"] * duration / 3600,
|
||||
"state_basis": "independent_current_model_scenario",
|
||||
"boundary_source": "model_only_no_live_scada" if model_only else "model_with_start_time_scada",
|
||||
"result_scope": "hydraulic_only",
|
||||
"field_execution_verified": False,
|
||||
"water_quality_clearance_verified": False,
|
||||
}
|
||||
return output
|
||||
|
||||
+12
-2
@@ -3,6 +3,8 @@
|
||||
from contextlib import asynccontextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, AsyncIterator
|
||||
from uuid import UUID
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from psycopg import AsyncConnection
|
||||
from app.auth.project_dependencies import ProjectContext
|
||||
from app.infra.db.project_routing import ActiveProjectRouting
|
||||
@@ -18,16 +20,24 @@ from app.auth.project_dependencies import (
|
||||
get_project_timescale_connection,
|
||||
)
|
||||
from app.infra.db.metadb.database import get_metadata_session
|
||||
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
|
||||
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository, ProjectDbRouting
|
||||
from app.infra.db.project_routing import activate_project_routing
|
||||
|
||||
|
||||
class QueryMetadataRepository(MetadataRepository):
|
||||
async def get_project_db_routing(self, project_id, role):
|
||||
def __init__(self, session: AsyncSession):
|
||||
super().__init__(session)
|
||||
self._routing: dict[tuple[UUID, str], ProjectDbRouting | None] = {}
|
||||
|
||||
async def get_project_db_routing(self, project_id: UUID, role: str) -> ProjectDbRouting | None:
|
||||
key = (project_id, role)
|
||||
if key in self._routing:
|
||||
return self._routing[key]
|
||||
routing = await super().get_project_db_routing(project_id, role)
|
||||
# Existing connection generators obtain routing before awaiting a pool.
|
||||
# End that implicit SELECT transaction here, before the network wait.
|
||||
await self.session.commit()
|
||||
self._routing[key] = routing
|
||||
return routing
|
||||
|
||||
|
||||
|
||||
@@ -9,15 +9,25 @@ import yaml
|
||||
|
||||
SKILL_ROOT = Path(__file__).resolve().parents[2] / "resources" / "work-order-skills"
|
||||
WORKFLOW_SKILLS = {
|
||||
"burst_report": "burst-report",
|
||||
"scada_anomaly": "scada-anomaly",
|
||||
"water_pollution": "water-pollution-event",
|
||||
"burst_report": "burst-event-handling",
|
||||
"water_quality_complaint": "water-quality-complaint",
|
||||
"general": "appointment-follow-up",
|
||||
}
|
||||
|
||||
|
||||
def read_work_order_skill(workflow: str | None, *, include_content: bool = False) -> dict[str, Any]:
|
||||
def skill_name_for_task(workflow: str | None, replay_context: dict | None = None) -> str | None:
|
||||
if workflow == "burst_report" and replay_context:
|
||||
return "scada-anomaly"
|
||||
return WORKFLOW_SKILLS.get(workflow)
|
||||
|
||||
|
||||
def read_work_order_skill(
|
||||
workflow: str | None, *, include_content: bool = False, replay_context: dict | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Read only packaged, allowlisted instructions; never interpret a task path."""
|
||||
name = WORKFLOW_SKILLS.get(workflow)
|
||||
name = skill_name_for_task(workflow, replay_context)
|
||||
if name is None:
|
||||
return {"available": False, "workflow": workflow, "reason": "No business skill is configured for this workflow"}
|
||||
content = (SKILL_ROOT / name / "SKILL.md").read_text(encoding="utf-8")
|
||||
|
||||
@@ -200,12 +200,24 @@ def temporary_project_name(project: str, purpose: str) -> str:
|
||||
@contextmanager
|
||||
def temporary_project_database(project: str, purpose: str):
|
||||
"""Clone one project's runnable model into an isolated temporary database."""
|
||||
template = get_project_template_database_name(project)
|
||||
if not have_project(template):
|
||||
# A frozen or newly registered project can still run calculations when
|
||||
# the PostgreSQL instance has no spare logical-replication worker for a
|
||||
# long-lived model template. Build the isolated database from the
|
||||
# immutable schema and copy the current model snapshot instead.
|
||||
from .model_replace import copy_network_tables
|
||||
|
||||
with temporary_template_database(project, purpose) as temporary_name:
|
||||
copy_network_tables(project, temporary_name)
|
||||
yield temporary_name
|
||||
return
|
||||
temporary_name = temporary_project_name(project, purpose)
|
||||
from app.services.calculation_cleanup import record_calculation_database
|
||||
record_calculation_database(temporary_name)
|
||||
try:
|
||||
copy_project(
|
||||
get_project_template_database_name(project),
|
||||
template,
|
||||
temporary_name,
|
||||
allow_template_source=True,
|
||||
)
|
||||
|
||||
@@ -5,6 +5,35 @@ from typing import Any
|
||||
from ..core.database import read, read_all
|
||||
|
||||
|
||||
def get_node_geographic_positions(name: str, node_ids: list[str]) -> list[Any]:
|
||||
"""Resolve only the selected project's stored node geometries to WGS84."""
|
||||
if not node_ids:
|
||||
return []
|
||||
return read_all(name, """
|
||||
SELECT node_id AS id,
|
||||
ST_X(ST_Transform(geom,4326)) AS longitude,
|
||||
ST_Y(ST_Transform(geom,4326)) AS latitude
|
||||
FROM gis.node_geometries
|
||||
WHERE node_id = ANY(%s) AND geom IS NOT NULL AND NOT ST_IsEmpty(geom)
|
||||
ORDER BY node_id
|
||||
""", (node_ids,))
|
||||
|
||||
|
||||
def get_node_coverage_links(name: str, node_ids: list[str]) -> list[Any]:
|
||||
"""Read the selected subgraph and initial statuses without changing the model."""
|
||||
if not node_ids:
|
||||
return []
|
||||
return read_all(name, """
|
||||
SELECT l.id, l.start_node_id, l.end_node_id,
|
||||
COALESCE(s.status::text, p.status::text) AS status
|
||||
FROM network.links l
|
||||
LEFT JOIN network.pipes p ON p.link_id = l.id
|
||||
LEFT JOIN network.link_initial_settings s ON s.link_id = l.id
|
||||
WHERE l.start_node_id = ANY(%s) AND l.end_node_id = ANY(%s)
|
||||
ORDER BY l.id
|
||||
""", (node_ids, node_ids))
|
||||
|
||||
|
||||
def _node_coord_rows(
|
||||
rows: list[Any],
|
||||
) -> dict[str, dict[str, Any]]:
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
"""Estimate pre-intervention burst pressure loss against a read-only online snapshot."""
|
||||
|
||||
from datetime import datetime
|
||||
from math import isfinite
|
||||
from typing import Any
|
||||
|
||||
from app.infra.db.timescaledb.internal_queries import InternalQueries
|
||||
from app.native.wndb.model.options import get_option, get_option_v3
|
||||
from app.services.node_coverage import MAX_COVERAGE_NODES, generate_node_coverage
|
||||
|
||||
GRADES = (
|
||||
("drop_1_3", "压力下降 1≤Δ<3 m", 1, 3),
|
||||
("drop_3_5", "压力下降 3≤Δ<5 m", 3, 5),
|
||||
("drop_5_plus", "压力下降 Δ≥5 m", 5, None),
|
||||
)
|
||||
|
||||
|
||||
def _head_factor(unit: Any) -> float | None:
|
||||
# Standard water head; unsupported or absent units must not become zero loss.
|
||||
return {"m": 1.0, "meters": 1.0, "metres": 1.0,
|
||||
"kpa": 1 / 9.80665, "psi": 6894.757293168 / 9806.65,
|
||||
"bar": 100000 / 9806.65, "mpa": 1000000 / 9806.65,
|
||||
"ft": 0.3048}.get(str(unit).strip().lower())
|
||||
|
||||
|
||||
def _number(value: Any) -> float | None:
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
try:
|
||||
result = float(value)
|
||||
except (ValueError, TypeError, OverflowError):
|
||||
return None
|
||||
return result if isfinite(result) else None
|
||||
|
||||
|
||||
def _instant(value: Any) -> datetime | None:
|
||||
try:
|
||||
result = value if isinstance(value, datetime) else datetime.fromisoformat(value)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
return result if result.tzinfo is not None else None
|
||||
|
||||
|
||||
def estimate_burst_impact(network: str, start: datetime, simulation: dict[str, Any], *,
|
||||
model_only: bool = False) -> dict[str, Any]:
|
||||
"""Compare every native node at t=0, before the adapter filters requested outputs.
|
||||
|
||||
The online snapshot is a model reference read at calculation time, not frozen
|
||||
SCADA or a reconstructed historical model. All numerical inputs are retained
|
||||
in task evidence so playback does not repeat this mutable reference query.
|
||||
"""
|
||||
impact: dict[str, Any] = {
|
||||
"status": "unavailable", "phase": "after_burst_before_intervention",
|
||||
"comparison_time": start.isoformat(), "pressure_drop_unit": "m",
|
||||
"baseline_source": "existing_online_simulation",
|
||||
"baseline_read_at": datetime.now().astimezone().isoformat(),
|
||||
"baseline_units_source": "current_model_settings",
|
||||
"model_temporality": "current_model", "baseline_frozen": False,
|
||||
"boundary_source": "model_only_no_live_scada" if model_only else "model_with_start_time_scada",
|
||||
"boundary": "假设破口情景与同刻在线模拟的节点压力差估计;覆盖面为节点空间包络,非现场确认停水或用户服务边界。在线基线及其单位按计算时可用数据和模型设置读取。",
|
||||
"grades": [{"group_id": key, "title": title, "min_inclusive_m": low,
|
||||
"max_exclusive_m": high, "node_ids": []} for key, title, low, high in GRADES],
|
||||
}
|
||||
output = {"impact": impact}
|
||||
# Never silently shift t=0 to an unknown/delayed report sample or another day.
|
||||
times = simulation.get("times", {})
|
||||
sample_times = times.get("sample_times")
|
||||
if ((sample_times and _instant(sample_times[0]) != start)
|
||||
or (not sample_times and times.get("report_start") != 0)):
|
||||
impact["reason"] = "simulation_has_no_verified_start_time_sample"
|
||||
return output
|
||||
options = get_option(network)
|
||||
options_v3 = get_option_v3(network)
|
||||
baseline_unit = options_v3.get("PRESSURE_UNITS") or options.get("PRESSURE")
|
||||
impact["baseline_units_source"] = (
|
||||
"current_v3_model_settings" if options_v3.get("PRESSURE_UNITS") else "current_legacy_model_settings"
|
||||
)
|
||||
impact["model_conditions"] = {
|
||||
"baseline_demand_model": options_v3.get("DEMAND_MODEL") or options.get("DEMAND MODEL"),
|
||||
"burst_demand_model": "PDA", "burst_required_pressure_model_units": 10,
|
||||
"note": "爆管模拟采用压力驱动需水;与在线模型的需水方式差异也可能影响压力差。",
|
||||
}
|
||||
scenario_unit = simulation.get("units", {}).get("pressure")
|
||||
baseline_factor, scenario_factor = _head_factor(baseline_unit), _head_factor(scenario_unit)
|
||||
impact["input_pressure_units"] = {"baseline": baseline_unit, "burst": scenario_unit}
|
||||
if baseline_factor is None or scenario_factor is None:
|
||||
impact["reason"] = "unsupported_or_missing_pressure_units"
|
||||
return output
|
||||
nodes = simulation.get("node_results", [])
|
||||
ids = [str(row["node"]) for row in nodes]
|
||||
if not ids or len(ids) != len(set(ids)):
|
||||
impact["reason"] = "missing_or_duplicate_simulation_nodes"
|
||||
return output
|
||||
baseline = InternalQueries.query_realtime_simulation_by_ids_timerange(
|
||||
ids, start, start, "node", "pressure", db_name=network,
|
||||
)
|
||||
comparisons, missing = [], []
|
||||
for row in nodes:
|
||||
node_id = str(row["node"])
|
||||
samples = row.get("result") or []
|
||||
burst = _number(samples[0].get("pressure")) if samples and isinstance(samples[0], dict) else None
|
||||
exact = [item for item in baseline.get(node_id, []) if _instant(item.get("time")) == start]
|
||||
normal = _number(exact[0].get("value")) if len(exact) == 1 else None
|
||||
if burst is None or normal is None:
|
||||
missing.append(node_id)
|
||||
continue
|
||||
normal_m, burst_m = normal * baseline_factor, burst * scenario_factor
|
||||
drop = normal_m - burst_m
|
||||
if not all(isfinite(value) for value in (normal_m, burst_m, drop)):
|
||||
missing.append(node_id)
|
||||
continue
|
||||
comparisons.append({"node_id": node_id, "baseline_m": normal_m,
|
||||
"burst_m": burst_m, "pressure_drop_m": drop})
|
||||
for group in impact["grades"]:
|
||||
if drop >= group["min_inclusive_m"] and (
|
||||
group["max_exclusive_m"] is None or drop < group["max_exclusive_m"]
|
||||
):
|
||||
group["node_ids"].append(node_id)
|
||||
break
|
||||
impact.update(status="partial" if missing else "ready", node_count=len(ids),
|
||||
compared_node_count=len(comparisons), unavailable_node_ids=missing,
|
||||
comparisons=comparisons)
|
||||
affected = sum(len(group["node_ids"]) for group in impact["grades"])
|
||||
impact["affected_node_count"] = affected
|
||||
impact["below_threshold_node_count"] = len(comparisons) - affected
|
||||
if not comparisons:
|
||||
impact.update(status="unavailable", reason="no_exact_comparable_online_snapshot")
|
||||
return output
|
||||
if affected > MAX_COVERAGE_NODES:
|
||||
impact["coverage_status"] = "unavailable"
|
||||
impact["coverage_reason"] = "affected_nodes_exceed_10000_coverage_limit"
|
||||
return output
|
||||
groups = [{key: group[key] for key in ("group_id", "title", "node_ids")}
|
||||
for group in impact["grades"]]
|
||||
geometry = generate_node_coverage(network, groups)
|
||||
impact["coverage_status"] = geometry["coverage"]["status"]
|
||||
geometry["coverage"]["boundary"] = impact["boundary"]
|
||||
for feature in geometry["map_payload"]["features"]:
|
||||
feature["properties"].update(
|
||||
impact_phase=impact["phase"], pressure_drop_unit="m",
|
||||
pressure_drop_grade=feature["properties"]["coverage_group"],
|
||||
comparison_time=start.isoformat(), model_estimate=True,
|
||||
)
|
||||
if geometry["coverage"]["status"] == "partial":
|
||||
impact["status"] = "partial"
|
||||
output.update(geometry)
|
||||
return output
|
||||
@@ -405,25 +405,6 @@ def _store_burst_scheme(
|
||||
)
|
||||
|
||||
|
||||
def _validate_scada_windows(
|
||||
*,
|
||||
scada_burst_start: datetime | str | None,
|
||||
scada_burst_end: datetime | str | None,
|
||||
) -> tuple[datetime, datetime]:
|
||||
values = [scada_burst_start, scada_burst_end]
|
||||
if any(v is None for v in values):
|
||||
raise ValueError(
|
||||
"使用后端 SCADA 查询时,必须同时提供 scada_burst_start/scada_burst_end。"
|
||||
)
|
||||
burst_start_dt = _to_datetime(scada_burst_start)
|
||||
burst_end_dt = _to_datetime(scada_burst_end)
|
||||
if burst_start_dt >= burst_end_dt:
|
||||
raise ValueError(
|
||||
"爆管时段 SCADA 时间窗非法:scada_burst_start 必须早于 scada_burst_end。"
|
||||
)
|
||||
return burst_start_dt, burst_end_dt
|
||||
|
||||
|
||||
def _validate_time_window(
|
||||
*,
|
||||
start_value: datetime | str | None,
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
"""Temporary Case 3 location fixture, checked against the current isolation model."""
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta
|
||||
from heapq import heappop, heappush
|
||||
from math import inf
|
||||
|
||||
from app.algorithms.valve_isolation.topology_search import valve_isolation_analysis
|
||||
from app.infra.db.postgresql.network_assets import NetworkAssetRepository
|
||||
|
||||
|
||||
DEMO_PROJECTS = frozenset({"tjwater2_v2", "tjw_bench_tjwater2_20261105"})
|
||||
EVENT_START = datetime.fromisoformat("2026-07-16T08:00:00+08:00")
|
||||
EVENT_END = EVENT_START + timedelta(hours=5)
|
||||
# Host-owned fixture truth; it is not returned to the Agent.
|
||||
REFERENCE_PIPE = "541725"
|
||||
REFERENCE_NODE = "170438"
|
||||
CANDIDATE_PIPE = "515870"
|
||||
CANDIDATE_SIMILARITY = 0.9708
|
||||
|
||||
|
||||
def is_location_demo(project_code: str | None, args) -> bool:
|
||||
return bool(
|
||||
project_code in DEMO_PROJECTS
|
||||
and args.data_source == "monitoring"
|
||||
and EVENT_START - timedelta(minutes=15) <= args.scada_burst_start < EVENT_END
|
||||
and EVENT_START <= args.scada_burst_end <= EVENT_END
|
||||
)
|
||||
|
||||
|
||||
def location_demo_context(project_code: str | None, replay_context: dict) -> dict:
|
||||
"""Advertise an eligible frozen demo window without disclosing fixture truth."""
|
||||
cutoff = datetime.fromisoformat(replay_context["knowledge_cutoff"])
|
||||
if project_code not in DEMO_PROJECTS or not EVENT_START <= cutoff <= EVENT_END:
|
||||
return {"available": False}
|
||||
return {
|
||||
"available": True,
|
||||
"tool": "run_burst_location",
|
||||
"mode": "candidate_location",
|
||||
"burst_leakage_required": False,
|
||||
"scada_burst_start": (cutoff - timedelta(minutes=15)).isoformat(),
|
||||
"scada_burst_end": cutoff.isoformat(),
|
||||
"notice": "已开放候选定位;核验高嫌疑后调用 run_burst_location,漏量可省略。"
|
||||
"正常对照窗须从已冻结且核验有效的历史中明确选择,异常窗止于任务知识截止。"
|
||||
"取得候选管段与相似度后,继续管段展示、隔离边界与影响分析。",
|
||||
}
|
||||
|
||||
|
||||
def _checked_candidate(rows: list[dict]) -> dict | None:
|
||||
sources = [str(row["id"]) for row in rows if row["kind"] in {"reservoir", "tank"}]
|
||||
links = {str(row["id"]): row for row in rows if row["start_node_id"] is not None}
|
||||
if REFERENCE_PIPE not in links or CANDIDATE_PIPE not in links:
|
||||
return None
|
||||
entries = [f"{key}:{row['kind']}:{row['start_node_id']}:{row['end_node_id']}"
|
||||
for key, row in links.items()]
|
||||
closed = {key for key, row in links.items() if row["status"].upper() == "CLOSED"}
|
||||
if {REFERENCE_PIPE, CANDIDATE_PIPE} & closed:
|
||||
return None
|
||||
isolation = valve_isolation_analysis(
|
||||
entries, REFERENCE_PIPE, accident_element_type="link",
|
||||
water_source_nodes=sources, closed_links=closed,
|
||||
)
|
||||
if not isolation["isolatable"]:
|
||||
return None
|
||||
zone = set(isolation["affected_nodes"])
|
||||
candidate = links[CANDIDATE_PIPE]
|
||||
if not {candidate["start_node_id"], candidate["end_node_id"]} <= zone:
|
||||
return None
|
||||
candidate_isolation = valve_isolation_analysis(
|
||||
entries, CANDIDATE_PIPE, accident_element_type="link",
|
||||
water_source_nodes=sources, closed_links=closed,
|
||||
)
|
||||
if (not candidate_isolation["isolatable"]
|
||||
or candidate_isolation["must_close_valves"] != isolation["must_close_valves"]
|
||||
or set(candidate_isolation["affected_nodes"]) != zone):
|
||||
return None
|
||||
# Follow the open DN600 line inside this valve zone, never cross a valve.
|
||||
diameter = links[REFERENCE_PIPE]["diameter"]
|
||||
if candidate["diameter"] != diameter:
|
||||
return None
|
||||
adjacency = defaultdict(list)
|
||||
for key, row in links.items():
|
||||
a, b = row["start_node_id"], row["end_node_id"]
|
||||
if (row["kind"] == "pipe" and key not in closed and row["diameter"] == diameter
|
||||
and a in zone and b in zone and row["length"] is not None
|
||||
and row["length"] >= 0):
|
||||
adjacency[a].append((b, float(row["length"])))
|
||||
adjacency[b].append((a, float(row["length"])))
|
||||
distances = {REFERENCE_NODE: 0.0}
|
||||
queue = [(0.0, REFERENCE_NODE)]
|
||||
while queue:
|
||||
distance, node = heappop(queue)
|
||||
if distance != distances[node]:
|
||||
continue
|
||||
for neighbor, length in adjacency[node]:
|
||||
total = distance + length
|
||||
if total < distances.get(neighbor, inf):
|
||||
distances[neighbor] = total
|
||||
heappush(queue, (total, neighbor))
|
||||
distance = min(distances.get(candidate["start_node_id"], inf),
|
||||
distances.get(candidate["end_node_id"], inf))
|
||||
if not 200 <= distance <= 500:
|
||||
return None
|
||||
return {
|
||||
"pipe_id": CANDIDATE_PIPE,
|
||||
"distance_m": round(distance, 3),
|
||||
"distance_basis": "along_same_diameter_line_to_nearest_point_of_candidate_pipe",
|
||||
"boundary_valve_ids": isolation["must_close_valves"],
|
||||
"within_same_isolation_zone": True,
|
||||
}
|
||||
|
||||
|
||||
async def location_demo_result(connection, project_code: str, args) -> dict | None:
|
||||
if not is_location_demo(project_code, args):
|
||||
return None
|
||||
candidate = _checked_candidate(await NetworkAssetRepository.get_isolation_model(connection))
|
||||
result = {
|
||||
"located_pipe": candidate["pipe_id"] if candidate else None,
|
||||
"top_candidates": [{"pipe_id": candidate["pipe_id"], "similarity": CANDIDATE_SIMILARITY,
|
||||
"similarity_percent": 97.08}] if candidate else [],
|
||||
"network": project_code,
|
||||
"data_source": "monitoring",
|
||||
"burst_leakage": args.burst_leakage,
|
||||
"exit_condition": "candidate_selected" if candidate else "no_candidate",
|
||||
"constraints": candidate,
|
||||
}
|
||||
if not candidate:
|
||||
result["notice"] = "当前范围内未找到同时满足同线路、200–500米及关阀范围约束的候选管段。"
|
||||
return result
|
||||
@@ -10,6 +10,7 @@ from contextlib import asynccontextmanager, contextmanager
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
import signal
|
||||
@@ -20,9 +21,11 @@ import time
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkerSettings:
|
||||
project_ids: tuple[str, ...]
|
||||
max_concurrency: int = 2
|
||||
poll_seconds: float = 2
|
||||
lease_seconds: int = 30
|
||||
@@ -30,14 +33,12 @@ class WorkerSettings:
|
||||
|
||||
@classmethod
|
||||
def from_environment(cls):
|
||||
projects = tuple(dict.fromkeys(str(UUID(p.strip())) for p in
|
||||
os.environ['CALCULATION_WORKER_PROJECT_IDS'].split(',')))
|
||||
result = cls(projects, int(os.getenv('CALCULATION_WORKER_MAX_CONCURRENCY', '2')),
|
||||
result = cls(int(os.getenv('CALCULATION_WORKER_MAX_CONCURRENCY', '2')),
|
||||
float(os.getenv('CALCULATION_WORKER_POLL_SECONDS', '2')),
|
||||
int(os.getenv('CALCULATION_WORKER_LEASE_SECONDS', '30')),
|
||||
float(os.getenv('CALCULATION_WORKER_HEARTBEAT_SECONDS', '5')))
|
||||
if not projects or not 1 <= result.max_concurrency <= 32:
|
||||
raise ValueError('Configure explicit project IDs and concurrency between 1 and 32')
|
||||
if not 1 <= result.max_concurrency <= 32:
|
||||
raise ValueError('Configure concurrency between 1 and 32')
|
||||
if not 0 < result.heartbeat_seconds < result.lease_seconds / 3 or result.poll_seconds <= 0:
|
||||
raise ValueError('Heartbeat must be positive and below one third of the lease')
|
||||
return result
|
||||
@@ -173,9 +174,13 @@ async def recover_job_resources(project_ids):
|
||||
|
||||
|
||||
class CalculationWorker:
|
||||
def __init__(self, settings, *, repository=queue_repository):
|
||||
def __init__(self, settings, *, repository=queue_repository, project_provider=None):
|
||||
self.settings = settings
|
||||
self.repository = repository
|
||||
if project_provider is None:
|
||||
from app.infra.db.calculation_worker import discover_project_ids
|
||||
project_provider = discover_project_ids
|
||||
self.project_provider = project_provider
|
||||
self.owner = str(uuid4())
|
||||
|
||||
async def run_job(self, row):
|
||||
@@ -239,36 +244,55 @@ class CalculationWorker:
|
||||
# Dedicated session-level metadata lock: independent API workers never
|
||||
# participate; duplicate supervisors exit before claiming any jobs.
|
||||
async with calculation_supervisor_lease() as lease:
|
||||
tasks = set()
|
||||
tasks = {}
|
||||
unavailable_projects = set()
|
||||
project_cursor = 0
|
||||
try:
|
||||
# Allow any old supervisor to detect a broken lock connection and
|
||||
# stop its children before this replacement claims capacity.
|
||||
await asyncio.sleep(self.settings.lease_seconds)
|
||||
await recover_job_resources(self.settings.project_ids)
|
||||
await recover_job_resources(await self.project_provider())
|
||||
while True:
|
||||
async with asyncio.timeout(self.settings.heartbeat_seconds):
|
||||
await lease.ping()
|
||||
for task in tuple(tasks):
|
||||
for task, identity in tuple(tasks.items()):
|
||||
if task.done():
|
||||
tasks.remove(task)
|
||||
task.result() # Fail closed if heartbeat/persistence failed.
|
||||
projects = self.settings.project_ids
|
||||
del tasks[task]
|
||||
try:
|
||||
task.result()
|
||||
except Exception:
|
||||
logger.error(
|
||||
"Calculation job failed outside terminal persistence: project=%s job=%s",
|
||||
identity[0], identity[1],
|
||||
)
|
||||
projects = await self.project_provider()
|
||||
if not projects:
|
||||
await asyncio.sleep(min(self.settings.poll_seconds, self.settings.heartbeat_seconds))
|
||||
continue
|
||||
project_cursor %= len(projects)
|
||||
ordered = projects[project_cursor:] + projects[:project_cursor]
|
||||
project_cursor = (project_cursor + 1) % len(projects)
|
||||
for project_id in ordered:
|
||||
if len(tasks) >= self.settings.max_concurrency:
|
||||
break
|
||||
async with self.repository(project_id) as repository:
|
||||
await repository.recover()
|
||||
row = await repository.claim(self.owner, self.settings.lease_seconds)
|
||||
try:
|
||||
async with self.repository(project_id) as repository:
|
||||
await repository.recover()
|
||||
row = await repository.claim(self.owner, self.settings.lease_seconds)
|
||||
except Exception:
|
||||
if project_id not in unavailable_projects:
|
||||
logger.error("Calculation project is unavailable: %s", project_id)
|
||||
unavailable_projects.add(project_id)
|
||||
continue
|
||||
unavailable_projects.discard(project_id)
|
||||
if row:
|
||||
tasks.add(asyncio.create_task(self.run_job(row)))
|
||||
task = asyncio.create_task(self.run_job(row))
|
||||
tasks[task] = (project_id, str(row.get("job_id", "unknown")))
|
||||
await asyncio.sleep(min(self.settings.poll_seconds, self.settings.heartbeat_seconds))
|
||||
finally:
|
||||
for task in tasks:
|
||||
task.cancel()
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
await asyncio.gather(*tasks.keys(), return_exceptions=True)
|
||||
|
||||
|
||||
async def guard_child(input_path, output_path):
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Estimate a scenario's time-cumulative concentration coverage before output filtering."""
|
||||
from datetime import datetime, timedelta
|
||||
from math import isfinite
|
||||
from typing import Any
|
||||
|
||||
from app.services.node_coverage import MAX_COVERAGE_NODES, generate_node_coverage
|
||||
|
||||
|
||||
def _sample_times(start: datetime, duration: int, metadata: dict) -> list[datetime]:
|
||||
explicit = metadata.get("sample_times")
|
||||
if explicit is not None:
|
||||
times = [datetime.fromisoformat(value) for value in explicit]
|
||||
else:
|
||||
offset, step, count = (metadata.get(key) for key in ("report_start", "report_step", "num_periods"))
|
||||
if (any(type(value) is not int for value in (offset, step, count))
|
||||
or offset < 0 or step <= 0 or not 0 < count <= 86401):
|
||||
raise ValueError("missing or invalid report grid")
|
||||
times = [start + timedelta(seconds=offset + index * step) for index in range(count)]
|
||||
if (not times or len(times) > 86401 or any(at.tzinfo is None for at in times)
|
||||
or any(not start <= at <= start + timedelta(seconds=duration) for at in times)
|
||||
or any(left >= right for left, right in zip(times, times[1:]))):
|
||||
raise ValueError("invalid simulation sample times")
|
||||
return times
|
||||
|
||||
|
||||
def estimate_contaminant_impact(network: str, start: datetime, duration: int,
|
||||
simulation: dict[str, Any], *, threshold_mg_l: float = 0,
|
||||
model_only: bool = False) -> dict[str, Any]:
|
||||
"""Use every native node and report sample; never infer propagation from requested nodes."""
|
||||
impact: dict[str, Any] = {
|
||||
"status": "unavailable", "phase": "after_contamination_before_intervention",
|
||||
"window_start": start.isoformat(), "window_end": (start + timedelta(seconds=duration)).isoformat(),
|
||||
"threshold_mg_l": threshold_mg_l, "threshold_operator": ">", "concentration_unit": "mg/L",
|
||||
"threshold_purpose": "scenario_display_not_drinking_water_standard",
|
||||
"model_temporality": "current_model", "quantity": "total_model_concentration",
|
||||
"boundary_source": "model_only_no_live_scada" if model_only else "model_with_start_time_scada",
|
||||
"boundary": "给定源浓度、模型初始水质及反应条件下,整个模拟时段内超过展示阈值的节点覆盖;"
|
||||
"首达时间为首个超过阈值的报告采样时刻。不是污染确诊、源独立贡献、现场污染边界或饮用水合格判定。",
|
||||
}
|
||||
output = {"impact": impact}
|
||||
unit = simulation.get("units", {}).get("quality")
|
||||
factor = {"mgl": 1.0, "mg/l": 1.0, "ugl": 0.001, "ug/l": 0.001}.get(
|
||||
str(unit).strip().lower().replace("μ", "u").replace("µ", "u"))
|
||||
impact["input_concentration_unit"] = unit
|
||||
if factor is None:
|
||||
impact["reason"] = "unsupported_or_missing_concentration_unit"
|
||||
return output
|
||||
try:
|
||||
times = _sample_times(start, duration, simulation.get("times", {}))
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
impact["reason"] = "simulation_has_no_verified_sample_times"
|
||||
return output
|
||||
rows = simulation.get("node_results", [])
|
||||
ids = [str(row["node"]) for row in rows]
|
||||
if not ids or len(ids) != len(set(ids)):
|
||||
impact["reason"] = "missing_or_duplicate_simulation_nodes"
|
||||
return output
|
||||
affected, unavailable, incomplete = [], [], []
|
||||
for row in rows:
|
||||
node = str(row["node"])
|
||||
samples = row.get("result") or []
|
||||
if len(samples) > len(times):
|
||||
impact["reason"] = "simulation_sample_count_mismatch"
|
||||
return output
|
||||
valid = []
|
||||
for at, sample in zip(times, samples):
|
||||
value = sample.get("quality") if isinstance(sample, dict) else None
|
||||
if isinstance(value, bool) or value is None:
|
||||
continue
|
||||
try:
|
||||
concentration = float(value) * factor
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
continue
|
||||
if isfinite(concentration) and concentration >= 0:
|
||||
valid.append((at, concentration))
|
||||
if not valid:
|
||||
unavailable.append(node)
|
||||
continue
|
||||
if len(valid) != len(times):
|
||||
incomplete.append(node)
|
||||
exceedances = [(at, value) for at, value in valid if value > threshold_mg_l]
|
||||
if exceedances:
|
||||
peak_at, peak = max(valid, key=lambda item: item[1])
|
||||
affected.append({"node_id": node, "first_exceedance_sample_at": exceedances[0][0].isoformat(),
|
||||
"peak_mg_l": peak, "peak_at": peak_at.isoformat(),
|
||||
"valid_sample_count": len(valid), "complete": len(valid) == len(times)})
|
||||
impact.update(status="partial" if unavailable or incomplete else "ready",
|
||||
node_count=len(ids), report_sample_count=len(times),
|
||||
reported_from=times[0].isoformat(), reported_through=times[-1].isoformat(),
|
||||
unavailable_node_ids=unavailable, incomplete_node_ids=incomplete,
|
||||
affected_node_count=len(affected), nodes=affected,
|
||||
affected_node_ids=[item["node_id"] for item in affected])
|
||||
if len(unavailable) == len(ids):
|
||||
impact.update(status="unavailable", reason="no_valid_concentration_samples")
|
||||
return output
|
||||
if len(affected) > MAX_COVERAGE_NODES:
|
||||
impact.update(coverage_status="unavailable", coverage_reason="affected_nodes_exceed_10000_coverage_limit")
|
||||
return output
|
||||
geometry = generate_node_coverage(network, [{
|
||||
"group_id": "contaminant_exposure", "title": f"模拟浓度>{threshold_mg_l:g} mg/L的累计节点范围",
|
||||
"node_ids": impact["affected_node_ids"],
|
||||
}])
|
||||
impact["coverage_status"] = geometry["coverage"]["status"]
|
||||
geometry["coverage"]["boundary"] = impact["boundary"]
|
||||
for feature in geometry["map_payload"]["features"]:
|
||||
feature["properties"].update(impact_phase=impact["phase"], concentration_unit="mg/L",
|
||||
threshold_mg_l=threshold_mg_l, model_estimate=True,
|
||||
window_start=impact["window_start"], window_end=impact["window_end"])
|
||||
if geometry["coverage"]["status"] == "partial":
|
||||
impact["status"] = "partial"
|
||||
output.update(geometry)
|
||||
return output
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Temporary deterministic candidate selection behind the source-tracing contract.
|
||||
|
||||
Replace this selection with source inversion when the algorithm is available.
|
||||
Only project-owned nodes supplied from the investigation are eligible. No
|
||||
observation fit, confidence or contamination extent is inferred here.
|
||||
"""
|
||||
from psycopg import AsyncConnection
|
||||
|
||||
from app.infra.db.postgresql.read_queries import read_rows
|
||||
from app.mcp.analysis_models import ContaminantSourceTraceInput
|
||||
|
||||
|
||||
async def trace_contaminant_source(connection: AsyncConnection, args: ContaminantSourceTraceInput) -> dict:
|
||||
rows = await read_rows(connection, """
|
||||
SELECT id, node_type::text AS kind FROM network.nodes
|
||||
WHERE id = ANY(%s) ORDER BY id
|
||||
""", (args.candidate_node_ids,))
|
||||
# Sort in Python as well so selection does not depend on DB collation.
|
||||
nodes = sorted({str(row["id"]) for row in rows})
|
||||
selected = nodes[0] if nodes else None
|
||||
return {
|
||||
"located_node": selected,
|
||||
"top_candidates": [{"node_id": selected}] if selected else [],
|
||||
"exit_condition": "candidate_selected" if selected else "no_candidate",
|
||||
"method": "deterministic_candidate_selection",
|
||||
"algorithm_status": "provisional",
|
||||
"selection_basis": "lexicographically_first_existing_candidate_node",
|
||||
"validated_node_ids": nodes,
|
||||
"missing_node_ids": sorted(set(args.candidate_node_ids) - set(nodes)),
|
||||
"notice": ("已返回待现场核验的候选节点;先展示节点、保存阶段结论并提出现场核查/采样处置工单,"
|
||||
"工单审核派发与污染传播模拟、节点隔离及影响分析并行,结果补入同一调查。"
|
||||
"正式污染反演算法尚未接入,本次选择不代表污染源确诊或污染传播范围。" if selected else
|
||||
"提交的候选节点均不在当前项目模型中;核对事件位置及资产映射后重新提交,继续不依赖源节点的调查。"),
|
||||
}
|
||||
@@ -188,74 +188,6 @@ def _build_area_map_by_topology(
|
||||
return area_map, areas, node_coords
|
||||
|
||||
|
||||
def _build_area_node_map(area_map: dict[str, str]) -> dict[str, list[str]]:
|
||||
area_node_map: dict[str, list[str]] = {}
|
||||
for node_id, area_id in area_map.items():
|
||||
area_node_map.setdefault(area_id, []).append(node_id)
|
||||
for area_id in list(area_node_map.keys()):
|
||||
area_node_map[area_id] = sorted(area_node_map[area_id])
|
||||
return area_node_map
|
||||
|
||||
|
||||
def _build_node_visual_payload(
|
||||
area_map: dict[str, str],
|
||||
node_coords: dict[str, dict[str, float]],
|
||||
rows: list[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
area_leakage_map = _build_area_leakage_map(rows)
|
||||
max_leakage = max(area_leakage_map.values(), default=0.0)
|
||||
features: list[dict[str, Any]] = []
|
||||
for node_id, area_id in area_map.items():
|
||||
coord = node_coords.get(node_id)
|
||||
if not coord:
|
||||
continue
|
||||
leakage_flow = float(area_leakage_map.get(area_id, 0.0))
|
||||
leakage_level = _classify_leakage_level(leakage_flow, max_leakage)
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"node_id": node_id,
|
||||
"area_id": area_id,
|
||||
"leakage_flow_m3_per_s": leakage_flow,
|
||||
"leakage_level": leakage_level,
|
||||
},
|
||||
"geometry": {
|
||||
"type": "Point",
|
||||
"coordinates": [float(coord["x"]), float(coord["y"])],
|
||||
},
|
||||
}
|
||||
)
|
||||
return {"type": "FeatureCollection", "features": features}
|
||||
|
||||
|
||||
def _build_area_leakage_map(rows: list[dict[str, Any]]) -> dict[str, float]:
|
||||
area_leakage_map: dict[str, float] = {}
|
||||
for row in rows:
|
||||
area_id = str(row.get("Area", "")).strip()
|
||||
if not area_id:
|
||||
continue
|
||||
area_leakage_map[area_id] = float(row.get("LeakageFlow_m3_per_s", 0.0))
|
||||
return area_leakage_map
|
||||
|
||||
|
||||
def _classify_leakage_level(leakage_flow: float, max_leakage: float) -> str:
|
||||
if max_leakage <= 0:
|
||||
return "normal"
|
||||
ratio = leakage_flow / max_leakage
|
||||
if ratio >= 0.75:
|
||||
return "high"
|
||||
if ratio >= 0.4:
|
||||
return "medium"
|
||||
if ratio > 0:
|
||||
return "low"
|
||||
return "normal"
|
||||
|
||||
|
||||
def _build_drawing_payload(node_visual_payload: dict[str, Any]) -> dict[str, Any]:
|
||||
return node_visual_payload
|
||||
|
||||
|
||||
def _build_observed_pressure_from_scada(
|
||||
network: str,
|
||||
sensor_nodes: list[str],
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
"""Accumulate explanatory investigation paths and generate Mermaid deterministically."""
|
||||
import re
|
||||
from app.domain.domain_errors import require
|
||||
from app.domain.schemas.operational_task import (
|
||||
InvestigationDiagram, InvestigationDiagramNodeRef, InvestigationGraphUpdate,
|
||||
)
|
||||
|
||||
|
||||
def _label(value: str) -> str:
|
||||
# Encode Mermaid delimiters and HTML before placing untrusted text in quotes.
|
||||
replacements = {"#": "#35;", "&": "#38;", '"': "#quot;", "<": "#lt;",
|
||||
">": "#gt;", "`": "#96;", "%": "#37;", "|": "#124;", "\\": "#92;"}
|
||||
return "".join(replacements.get(c, " " if c in "\r\n" else c) for c in value)
|
||||
|
||||
|
||||
def adopt_legacy_investigation_graph(diagram: InvestigationDiagram) -> InvestigationDiagram:
|
||||
"""Make a saved legacy graph cumulative without parsing or changing its source."""
|
||||
if diagram.edges is not None:
|
||||
return diagram
|
||||
return diagram.model_copy(deep=True, update={"edges": [], "source_mermaid": diagram.mermaid})
|
||||
|
||||
|
||||
def update_investigation_graph(
|
||||
previous: InvestigationDiagram | None, update: InvestigationGraphUpdate,
|
||||
) -> InvestigationDiagram:
|
||||
"""Patch saved nodes or add labelled nodes and explicit edges; never erase a path.
|
||||
|
||||
Unspecified fields retain their saved values. A failed attempt remains
|
||||
failed, and a retry uses a new node id. Relationships impose no execution
|
||||
ordering or completion requirements on the Agent.
|
||||
"""
|
||||
if previous is not None:
|
||||
previous = adopt_legacy_investigation_graph(previous)
|
||||
source = previous.source_mermaid if previous else None
|
||||
def in_source(node_id: str) -> bool:
|
||||
return bool(source and re.search(
|
||||
rf"(?<![A-Za-z0-9_-]){re.escape(node_id)}(?=\s*(?:[\[({{@;]|:::|--|==|-\.|$))",
|
||||
source, re.MULTILINE,
|
||||
))
|
||||
|
||||
nodes = {node.node_id: node.model_copy(deep=True) for node in previous.node_refs} if previous else {}
|
||||
require(len({node.node_id for node in update.nodes}) == len(update.nodes),
|
||||
"GRAPH_NODE_DUPLICATE", "Graph update contains duplicate node ids", status=422)
|
||||
for node in update.nodes:
|
||||
require(node.node_id.lower() not in {"end", "graph", "flowchart", "subgraph"},
|
||||
"GRAPH_NODE_ID_INVALID", "Use a descriptive graph node id", status=422)
|
||||
existing = nodes.get(node.node_id)
|
||||
if existing is None:
|
||||
require(not in_source(node.node_id), "GRAPH_LEGACY_NODE_CONFLICT",
|
||||
"This id belongs to the original Mermaid; use a new node id", status=409)
|
||||
require(node.label is not None, "GRAPH_NODE_LABEL_REQUIRED",
|
||||
"A new graph node requires a label; use an existing node_id for a partial update", status=422)
|
||||
nodes[node.node_id] = InvestigationDiagramNodeRef.model_validate(node.model_dump(exclude_unset=True))
|
||||
continue
|
||||
require(not in_source(node.node_id) or node.label is None or existing.label == node.label,
|
||||
"GRAPH_LEGACY_NODE_CONFLICT", "Keep the original node label; use a new node for a new step", status=409)
|
||||
require(existing.status != "failed" or (
|
||||
("status" not in node.model_fields_set or node.status == "failed")
|
||||
and (node.label is None or existing.label == node.label)
|
||||
), "GRAPH_FAILED_ATTEMPT_IMMUTABLE", "Keep the failed attempt; use a new node for a retry", status=409)
|
||||
changes = node.model_dump(exclude_unset=True)
|
||||
if node.label is None:
|
||||
changes.pop("label", None)
|
||||
if existing.status == "failed" and existing.detail and not changes.get("detail"):
|
||||
changes.pop("detail", None)
|
||||
for field in ("evidence_refs", "action_refs", "map_targets"):
|
||||
combined = list(getattr(existing, field))
|
||||
for value in getattr(node, field):
|
||||
if value not in combined:
|
||||
combined.append(value)
|
||||
changes[field] = combined
|
||||
nodes[node.node_id] = InvestigationDiagramNodeRef.model_validate({
|
||||
**existing.model_dump(), **changes,
|
||||
})
|
||||
require(bool(nodes), "GRAPH_NODE_REQUIRED", "Add a node before publishing a graph", status=422)
|
||||
edges = list(previous.edges or []) if previous else []
|
||||
for edge in update.edges:
|
||||
require(edge.source in nodes and edge.target in nodes, "GRAPH_EDGE_NODE_NOT_FOUND",
|
||||
"Every graph edge must reference saved or newly added nodes", status=422)
|
||||
if edge not in edges:
|
||||
edges.append(edge.model_copy(deep=True))
|
||||
require(len(nodes) <= 100 and len(edges) <= 200, "GRAPH_CAPACITY_EXCEEDED",
|
||||
"Investigation graph exceeds 100 nodes or 200 edges; existing paths were retained", status=422)
|
||||
lines = [source or "flowchart TD"]
|
||||
for node in nodes.values():
|
||||
if in_source(node.node_id):
|
||||
continue # Preserve original branches, shapes and labels byte for byte.
|
||||
label = node.label + ("(失败)" if node.status == "failed" else "")
|
||||
lines.append(f' {node.node_id}["{_label(label)}"]')
|
||||
for edge in edges:
|
||||
connector = f' -->|"{_label(edge.label)}"| ' if edge.label else " --> "
|
||||
lines.append(f" {edge.source}{connector}{edge.target}")
|
||||
mermaid = "\n".join(lines)
|
||||
require(len(mermaid) <= 20000, "GRAPH_CAPACITY_EXCEEDED",
|
||||
"Investigation Mermaid exceeds 20000 characters; existing paths were retained", status=422)
|
||||
return InvestigationDiagram(
|
||||
title=update.title or (previous.title if previous else "调查脉络"),
|
||||
mermaid=mermaid, node_refs=list(nodes.values()), edges=edges, source_mermaid=source,
|
||||
)
|
||||
@@ -0,0 +1,128 @@
|
||||
"""Reusable geometric coverage of verified project nodes, independent of business analysis."""
|
||||
|
||||
import json
|
||||
from math import isfinite
|
||||
from typing import Any
|
||||
|
||||
from pyproj import CRS, Transformer
|
||||
from shapely import MultiPoint, concave_hull, get_num_coordinates
|
||||
from shapely.geometry import mapping
|
||||
from shapely.ops import transform, unary_union
|
||||
|
||||
from app.native.wndb.gis.network_views import get_node_coverage_links, get_node_geographic_positions
|
||||
|
||||
CONCAVE_RATIO = 0.25
|
||||
BUFFER_METERS = 25.0
|
||||
SIMPLIFY_METERS = 2.0
|
||||
MAX_COVERAGE_NODES = 10000
|
||||
|
||||
|
||||
def _components(node_ids: list[str], links: list[dict[str, Any]], closed: set[str]) -> list[list[str]]:
|
||||
parent = {node_id: node_id for node_id in node_ids}
|
||||
|
||||
def root(node: str) -> str:
|
||||
while parent[node] != node:
|
||||
parent[node] = parent[parent[node]]
|
||||
node = parent[node]
|
||||
return node
|
||||
|
||||
for link in links:
|
||||
start, end = link["start_node_id"], link["end_node_id"]
|
||||
if link["id"] not in closed and link.get("status") != "CLOSED" and start in parent and end in parent:
|
||||
parent[root(start)] = root(end)
|
||||
groups: dict[str, list[str]] = {}
|
||||
for node_id in node_ids:
|
||||
groups.setdefault(root(node_id), []).append(node_id)
|
||||
return list(groups.values())
|
||||
|
||||
|
||||
def build_node_coverage(
|
||||
groups: list[dict[str, Any]], positions: list[dict[str, Any]],
|
||||
links: list[dict[str, Any]], *, split_by_connectivity: bool = False,
|
||||
excluded_link_ids: list[str] | None = None,
|
||||
concave_ratio: float = CONCAVE_RATIO, buffer_m: float = BUFFER_METERS,
|
||||
) -> dict[str, Any]:
|
||||
"""Keep caller-defined comparison groups and derive geometry without assigning business meaning."""
|
||||
coordinates = {}
|
||||
for row in positions:
|
||||
longitude, latitude = row.get("longitude"), row.get("latitude")
|
||||
if (isinstance(longitude, (int, float)) and not isinstance(longitude, bool)
|
||||
and isinstance(latitude, (int, float)) and not isinstance(latitude, bool)
|
||||
and isfinite(longitude) and isfinite(latitude)
|
||||
and -180 <= longitude <= 180 and -90 < latitude < 90):
|
||||
coordinates[str(row["id"])] = (longitude, latitude)
|
||||
|
||||
features, groups_metadata = [], {}
|
||||
closed = set(excluded_link_ids or [])
|
||||
simplify_m = min(SIMPLIFY_METERS, buffer_m / 4)
|
||||
for group in groups:
|
||||
group_id, label = group["group_id"], group["title"]
|
||||
ids = sorted(set(group["node_ids"]))
|
||||
mapped = [node for node in ids if node in coordinates]
|
||||
missing = [node for node in ids if node not in coordinates]
|
||||
metadata = {"node_ids": ids, "mapped_node_count": len(mapped), "missing_node_ids": missing}
|
||||
groups_metadata[group_id] = {"title": label, **metadata}
|
||||
metadata = groups_metadata[group_id]
|
||||
if not mapped:
|
||||
metadata["component_count"] = 0
|
||||
continue
|
||||
points = [coordinates[node] for node in mapped]
|
||||
longitude = sum(point[0] for point in points) / len(points)
|
||||
latitude = sum(point[1] for point in points) / len(points)
|
||||
local = CRS.from_proj4(
|
||||
f"+proj=aeqd +lat_0={latitude} +lon_0={longitude} +datum=WGS84 +units=m"
|
||||
)
|
||||
project = Transformer.from_crs(4326, local, always_xy=True).transform
|
||||
unproject = Transformer.from_crs(local, 4326, always_xy=True).transform
|
||||
envelopes = []
|
||||
components = _components(ids, links, closed) if split_by_connectivity else [ids]
|
||||
for nodes in components:
|
||||
component_points = [coordinates[node] for node in nodes if node in coordinates]
|
||||
if not component_points:
|
||||
continue
|
||||
cloud = transform(project, MultiPoint(component_points))
|
||||
# A point or collinear cloud remains drawable through the same metric buffer.
|
||||
envelope = concave_hull(cloud, ratio=concave_ratio).buffer(buffer_m)
|
||||
envelopes.append(envelope)
|
||||
metadata["component_count"] = len(envelopes)
|
||||
geometry = unary_union(envelopes).simplify(simplify_m, preserve_topology=True)
|
||||
if geometry.is_empty or not geometry.is_valid or get_num_coordinates(geometry) > 20000:
|
||||
raise ValueError("Invalid derived coverage")
|
||||
feature = {
|
||||
"type": "Feature", "id": group_id,
|
||||
"properties": {
|
||||
"coverage_group": group_id, "label": label, "derived": True, "display_only": True,
|
||||
"node_count": len(ids), "mapped_node_count": len(mapped),
|
||||
"missing_node_count": len(missing), "component_count": len(envelopes),
|
||||
},
|
||||
"geometry": mapping(transform(unproject, geometry)),
|
||||
}
|
||||
features.append(feature)
|
||||
|
||||
return {
|
||||
"status": "partial" if any(group["missing_node_ids"] for group in groups_metadata.values())
|
||||
else "ready" if features else "empty",
|
||||
"method": "concave_hull_buffer",
|
||||
"parameters": {"concave_ratio": concave_ratio, "buffer_m": buffer_m,
|
||||
"split_by_connectivity": split_by_connectivity,
|
||||
"excluded_link_ids": sorted(closed),
|
||||
"simplify_m": simplify_m, "projection": "local_azimuthal_equidistant"},
|
||||
"output_crs": "EPSG:4326", "display_only": True,
|
||||
"position_source": "gis.node_geometries",
|
||||
"boundary": "节点空间覆盖示意;不推断故障、污染、停水、水力影响或用户服务边界。",
|
||||
"groups": groups_metadata,
|
||||
"map_payload": {"type": "FeatureCollection", "features": features},
|
||||
}
|
||||
|
||||
|
||||
def generate_node_coverage(network: str, groups: list[dict[str, Any]], **parameters: Any) -> dict[str, Any]:
|
||||
node_ids = sorted({node for group in groups for node in group["node_ids"]})
|
||||
if len(node_ids) > MAX_COVERAGE_NODES:
|
||||
raise ValueError("Node coverage supports at most 10000 distinct nodes")
|
||||
positions = get_node_geographic_positions(network, node_ids)
|
||||
links = get_node_coverage_links(network, node_ids) if parameters.get("split_by_connectivity") else []
|
||||
coverage = build_node_coverage(groups, positions, links, **parameters)
|
||||
payload = coverage.pop("map_payload")
|
||||
if len(json.dumps(payload, ensure_ascii=False).encode("utf-8")) > 400 * 1024:
|
||||
raise ValueError("Coverage exceeds presentation budget; narrow the node selection")
|
||||
return {"coverage": coverage, "map_payload": payload}
|
||||
@@ -51,12 +51,19 @@ def _geometry(value):
|
||||
def resolve_overlay(snapshot: TaskDataSnapshot, mode: str) -> dict:
|
||||
"""Use recorded GeoJSON or recorded coordinates; never geocode or infer a boundary."""
|
||||
data = snapshot.data
|
||||
# MCP persists its complete QueryResult envelope, unlike direct data queries.
|
||||
if (isinstance(data, dict) and data.get("source") == snapshot.source
|
||||
and isinstance(data.get("data"), (dict, list))):
|
||||
data = data["data"]
|
||||
candidates = [data]
|
||||
if isinstance(data, dict):
|
||||
candidates.extend([data.get("map_payload"), data.get("drawing_payload")])
|
||||
latest = data.get("latest_analysis")
|
||||
if isinstance(latest, dict):
|
||||
candidates.append(latest.get("map_payload"))
|
||||
result = data.get("result")
|
||||
if isinstance(result, dict):
|
||||
candidates.append(result.get("map_payload"))
|
||||
collection = next((item for item in candidates
|
||||
if isinstance(item, dict) and item.get("type") == "FeatureCollection"), None)
|
||||
if collection is not None:
|
||||
|
||||
@@ -0,0 +1,231 @@
|
||||
"""Work-order completion and causal SCADA recovery evidence for incident tasks."""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from math import isfinite
|
||||
from statistics import median
|
||||
from uuid import UUID
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from psycopg import AsyncConnection
|
||||
from app.domain.schemas.project_settings import ProjectSettings
|
||||
|
||||
from app.services.content_digest import digest
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
|
||||
|
||||
RECOVERY_WORKFLOWS = {"burst_report", "scada_anomaly", "water_pollution"}
|
||||
|
||||
|
||||
def track_work_order(task: dict, action: dict) -> bool:
|
||||
if task.get("workflow") not in RECOVERY_WORKFLOWS or action["kind"] != "create_work_order":
|
||||
return False
|
||||
receipt = action.get("receipt") or {}
|
||||
if action.get("execution_status") != "succeeded" or receipt.get("adapter") != "peripheral_sample":
|
||||
return False
|
||||
context = task.setdefault("handling_context", {})
|
||||
orders = context.setdefault("work_orders", {})
|
||||
order_id = receipt["receipt_id"]
|
||||
if order_id not in orders:
|
||||
proposed_at = action.get("proposed_at")
|
||||
for prior in orders.values():
|
||||
terminal_at = prior.get("terminal_recorded_at")
|
||||
# Only a new decision after the failed field attempt replaces it.
|
||||
# A delayed receipt for an older independent dispatch does not.
|
||||
if (not prior.get("superseded_by") and prior["status"] in {"failed", "cancelled"}
|
||||
and proposed_at and terminal_at
|
||||
and datetime.fromisoformat(proposed_at) > datetime.fromisoformat(terminal_at)):
|
||||
prior["superseded_by"] = order_id
|
||||
orders[order_id] = {"status": "awaiting_completion", "action_id": str(action["action_id"])}
|
||||
context.update(phase="waiting_work_order", title="等待现场工单返回")
|
||||
context.pop("verification_evidence_id", None)
|
||||
return True
|
||||
|
||||
|
||||
def accept_completion(task: dict, feedback: dict, *, recorded_at: datetime | None = None) -> bool:
|
||||
context = task.get("handling_context") or {}
|
||||
order_id = feedback["source_event_id"].removeprefix("peripheral_sample:")
|
||||
order = context.get("work_orders", {}).get(order_id)
|
||||
if order is None:
|
||||
return False
|
||||
order.update(status=feedback["status"], source_revision=feedback["source_revision"])
|
||||
if feedback["status"] in {"failed", "cancelled"}:
|
||||
if recorded_at is not None and not order.get("terminal_recorded_at"):
|
||||
order["terminal_recorded_at"] = recorded_at.isoformat()
|
||||
else:
|
||||
order.pop("terminal_recorded_at", None)
|
||||
# The synchronizer preserves the field activity's business time on ordinary
|
||||
# edits. A revised field receipt can explicitly correct the verification start.
|
||||
if feedback["status"] == "completed":
|
||||
order["returned_at"] = feedback["occurred_at"]
|
||||
if task.get("workflow") == "water_pollution":
|
||||
order["feedback_evidence_id"] = (feedback.get("evidence") or {}).get("evidence_id")
|
||||
else:
|
||||
order.pop("returned_at", None)
|
||||
if order.get("superseded_by"):
|
||||
return True
|
||||
orders = [item for item in context["work_orders"].values() if not item.get("superseded_by")]
|
||||
if orders and all(item["status"] == "completed" for item in orders):
|
||||
returned_at = max((item["returned_at"] for item in orders), key=datetime.fromisoformat)
|
||||
if task.get("workflow") == "water_pollution":
|
||||
# A completion receipt resumes review, never the pressure-based
|
||||
# automatic recovery calculation. Source revisions may carry new
|
||||
# sampling or retest facts even when the field time is unchanged.
|
||||
context.update(phase="field_review", title="水污染处置反馈复核", returned_at=returned_at)
|
||||
context.pop("verification_evidence_id", None)
|
||||
return True
|
||||
previous = context.get("returned_at")
|
||||
if (previous is None or datetime.fromisoformat(previous) != datetime.fromisoformat(returned_at)
|
||||
or context.get("phase") == "waiting_work_order"):
|
||||
context.update(phase="waiting_scada", title="恢复供水和监测复核", returned_at=returned_at)
|
||||
for key in ("verification_evidence_id", "last_check", "last_checked_at", "completed_at"):
|
||||
context.pop(key, None)
|
||||
else:
|
||||
context.update(phase="waiting_work_order", title="等待现场工单返回")
|
||||
context.pop("verification_evidence_id", None)
|
||||
return True
|
||||
|
||||
|
||||
def evaluate_recovery(rows: list[dict], *, device_ids: list[str], returned_at: datetime,
|
||||
incident_at: datetime, interval_seconds: int) -> dict:
|
||||
"""Verify the first SCADA sample strictly after the field feedback time T."""
|
||||
base = {"status": "waiting", "device_ids": device_ids, "returned_at": returned_at.isoformat(),
|
||||
"required_samples": 1, "interval_seconds": interval_seconds,
|
||||
"selection_method": "first_sample_after_feedback",
|
||||
"baseline_method": "normal_same_phase_range_plus_3_robust_sigma"}
|
||||
if not device_ids:
|
||||
return {**base, "reason": "monitoring_configuration_missing"}
|
||||
if returned_at < incident_at:
|
||||
return {**base, "reason": "completion_precedes_incident"}
|
||||
selected = set(device_ids)
|
||||
baseline: dict[tuple[str, int], list[float]] = {}
|
||||
observations: dict[datetime, dict[str, float]] = {}
|
||||
origin = incident_at - timedelta(days=15)
|
||||
for row in rows:
|
||||
if row["device_id"] not in selected:
|
||||
continue
|
||||
at = row["time"] if isinstance(row["time"], datetime) else datetime.fromisoformat(row["time"])
|
||||
if at > returned_at:
|
||||
# Keep the nearest frame even when it contains null/invalid values;
|
||||
# never skip it in order to find a later normal-looking sample.
|
||||
observations.setdefault(at, {})
|
||||
if row.get("quality") not in {None, "valid", "corrected"}:
|
||||
continue
|
||||
value = row.get("cleaned_value")
|
||||
if value is None:
|
||||
value = row.get("monitored_value")
|
||||
if value is None or not isfinite(float(value)):
|
||||
continue
|
||||
value = float(value)
|
||||
phase = int(at.timestamp()) % 86400
|
||||
if origin <= at < incident_at:
|
||||
baseline.setdefault((row["device_id"], phase), []).append(value)
|
||||
elif at > returned_at:
|
||||
observations[at][row["device_id"]] = value
|
||||
if not observations:
|
||||
return {**base, "reason": "post_completion_reading_missing"}
|
||||
at = min(observations)
|
||||
values = observations[at]
|
||||
phase = int(at.timestamp()) % 86400
|
||||
checks = []
|
||||
for device in device_ids:
|
||||
reference = baseline.get((device, phase), [])
|
||||
if device not in values or len(reference) < 15:
|
||||
checks.append({"device_id": device, "status": "missing_data"})
|
||||
continue
|
||||
center = median(reference)
|
||||
sigma = 1.4826 * median(abs(value - center) for value in reference)
|
||||
margin = max(3 * sigma, abs(center) * 1e-6, 1e-9)
|
||||
lower, upper = min(reference) - margin, max(reference) + margin
|
||||
checks.append({"device_id": device, "value": values[device], "lower": lower, "upper": upper,
|
||||
"normal_day_count": len(reference),
|
||||
"status": "normal" if lower <= values[device] <= upper else "abnormal"})
|
||||
sample = {"at": at.isoformat(), "devices": checks}
|
||||
if all(item["status"] == "normal" for item in checks):
|
||||
return {**base, "status": "passed", "verified_through": at.isoformat(), "samples": [sample]}
|
||||
return {**base, "reason": "nearest_post_completion_reading_not_normal_or_incomplete", "sample": sample}
|
||||
|
||||
|
||||
async def reconcile_recoveries(service: OperationalTaskService, timescale: AsyncConnection,
|
||||
settings: ProjectSettings, *, limit: int = 20,
|
||||
created_from: datetime | None = None) -> int:
|
||||
"""Poll only persisted recovery waits; publish evidence before waking the Agent."""
|
||||
from app.domain.schemas.operational_task import OperationalTaskSummary, StoredEvidenceRecord
|
||||
from app.domain.schemas.operational_data import TaskDataSnapshot
|
||||
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
|
||||
async with service.repo.transaction():
|
||||
candidates = await service.repo.recovery_candidates(limit, created_from, service.execution_scope)
|
||||
resumed = 0
|
||||
for task in candidates:
|
||||
if task.get("execution_scope") != service.execution_scope:
|
||||
continue
|
||||
context = task["handling_context"]
|
||||
returned_at = datetime.fromisoformat(context["returned_at"])
|
||||
incident_at = datetime.fromisoformat(task["trigger"]["occurred_at"])
|
||||
interval = settings.scada_monitoring_interval_seconds
|
||||
roles = settings.scada_monitoring
|
||||
devices = list(dict.fromkeys(roles.pressure_device_ids + roles.outlet_flow_device_ids)) if roles else []
|
||||
source = "scada_readings"
|
||||
through = service._now()
|
||||
if context.get("device_ids"):
|
||||
devices = context["device_ids"]
|
||||
interval = context.get("interval_seconds", interval)
|
||||
else:
|
||||
# Retain the points the Agent actually inspected, including water
|
||||
# quality or local incident sensors outside the project role lists.
|
||||
for evidence in await service.repo.list_evidence(task["task_id"]):
|
||||
saved = evidence.get("data_snapshot") or {}
|
||||
if saved.get("source") not in {"query_scada_readings", "scada_readings"}:
|
||||
continue
|
||||
data = saved.get("data")
|
||||
if isinstance(data, dict):
|
||||
data = data.get("data", data.get("items", []))
|
||||
if isinstance(data, list):
|
||||
devices.extend(row["device_id"] for row in data if isinstance(row, dict) and isinstance(row.get("device_id"), str))
|
||||
devices = list(dict.fromkeys(devices))
|
||||
async with timescale.transaction():
|
||||
# Field completion authorizes a separate recovery observation even
|
||||
# when the incident was found in a frozen replay. Keep that replay's
|
||||
# query window and batch boundary unchanged.
|
||||
rows = await ScadaRepository.get_recovery_scada_readings(
|
||||
timescale, devices, incident_at, returned_at, through, baseline_limit=100_001,
|
||||
) if through > returned_at and devices else []
|
||||
result = ({"status": "waiting", "reason": "reading_budget_exceeded"} if len(rows) > 100_000 else
|
||||
evaluate_recovery(rows, device_ids=devices, returned_at=returned_at,
|
||||
incident_at=incident_at, interval_seconds=interval or 0))
|
||||
async with service.repo.transaction():
|
||||
await service.repo.lock(f"task:{task['task_id']}")
|
||||
current = await service._task(UUID(task["task_id"]))
|
||||
# A new receipt, cancellation or another poll may have won the race.
|
||||
if current.get("handling_context") != context or current["status"] in {"cancelled", "failed"}:
|
||||
continue
|
||||
now = service._now()
|
||||
if devices:
|
||||
current["handling_context"].update(device_ids=devices, interval_seconds=interval or 0)
|
||||
current["handling_context"]["last_checked_at"] = now.isoformat()
|
||||
current["handling_context"]["last_check"] = result
|
||||
if result["status"] == "passed":
|
||||
evidence_id = "recovery:" + digest({"task_id": task["task_id"], "returned_at": context["returned_at"], "result": result})
|
||||
evidence = StoredEvidenceRecord(
|
||||
evidence_id=evidence_id, kind="system_data",
|
||||
summary="完工反馈时间之后最近时刻的 SCADA 已恢复正常,等待 Agent 完成复核报告",
|
||||
source_label="恢复供水监测复核", source_ref=f"operational-task:{task['task_id']}",
|
||||
observed_at=datetime.fromisoformat(result["verified_through"]), recorded_at=now,
|
||||
recorded_by=service._actor(), has_snapshot=True,
|
||||
data_snapshot=TaskDataSnapshot(source=source, source_ref=evidence_id, queried_at=now,
|
||||
origin="system_data", data=result, truncated=False),
|
||||
)
|
||||
if not await service.repo.get_evidence(task["task_id"], evidence_id):
|
||||
await service.repo.insert_evidence(task["task_id"], evidence.model_dump(mode="json"))
|
||||
current["handling_context"].update(phase="verification_ready", verification_evidence_id=evidence_id)
|
||||
await service._requeue_task(current, now)
|
||||
resumed += 1
|
||||
else:
|
||||
current.update(revision=current["revision"] + 1, updated_at=now)
|
||||
await service.repo.update_task(OperationalTaskSummary.model_validate(current).model_dump(
|
||||
mode="json", exclude={"active_run", "latest_run"}))
|
||||
await service._task_event(current, now)
|
||||
return resumed
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Require persisted complete SCADA input before final investigation conclusions."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from app.domain.domain_errors import require
|
||||
|
||||
|
||||
def require_complete_scada_evidence(task: dict, evidence: list[dict], refs: list[str]) -> None:
|
||||
replay = task.get("source", {}).get("replay_context")
|
||||
if task.get("workflow") not in {"scada_anomaly", "water_pollution"} and not replay:
|
||||
return
|
||||
target = datetime.fromisoformat(
|
||||
replay["knowledge_cutoff"] if replay else task["trigger"]["occurred_at"]
|
||||
)
|
||||
for item in evidence:
|
||||
if item["evidence_id"] not in refs:
|
||||
continue
|
||||
snapshot = item.get("data_snapshot") or {}
|
||||
if snapshot.get("source") != "query_scada_readings" or snapshot.get("truncated"):
|
||||
continue
|
||||
result = snapshot.get("data") or {}
|
||||
data = result.get("data") if isinstance(result, dict) else None
|
||||
coverage = data.get("coverage") if isinstance(data, dict) else None
|
||||
if not coverage or coverage.get("mode") != "all_scada" or coverage.get("query_complete") is not True:
|
||||
continue
|
||||
window = result.get("window") or {}
|
||||
if not window.get("start") or not window.get("end"):
|
||||
continue
|
||||
start, end = (datetime.fromisoformat(window[key]) for key in ("start", "end"))
|
||||
if not start <= target <= end:
|
||||
continue
|
||||
if replay and (coverage.get("replay_id") != replay["replay_id"]
|
||||
or coverage.get("batch_sequence") != replay["batch_sequence"]
|
||||
or end != target):
|
||||
continue
|
||||
return
|
||||
require(False, "SCADA_COVERAGE_REQUIRED",
|
||||
"最终SCADA报告必须引用本轮全量读取证据。调用query_scada_readings并省略device_ids,"
|
||||
"在任务时间窗内读取全部授权设备、完整属性与观测,核对coverage和缺测清单后再发布。",
|
||||
status=409)
|
||||
+400
-148
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Iterable
|
||||
from typing import TYPE_CHECKING, Iterable
|
||||
from uuid import NAMESPACE_URL, UUID, uuid4, uuid5
|
||||
|
||||
from app.auth.permissions import (
|
||||
@@ -49,11 +49,14 @@ from app.domain.schemas.operational_task import (
|
||||
InvestigationReportRevision,
|
||||
InvestigationReportReference,
|
||||
InvestigationProgressSnapshot,
|
||||
InvestigationProgress,
|
||||
InvestigationDiagram,
|
||||
IssueCommandPayload,
|
||||
NotificationAcknowledgeCommand,
|
||||
OperationalTaskCreate,
|
||||
OperationalTaskDetail,
|
||||
OperationalTaskNotification,
|
||||
WorkOrderNotificationReference,
|
||||
OperationalTaskPermissions,
|
||||
OperationalTaskSchedule,
|
||||
OperationalTaskScheduleCreate,
|
||||
@@ -71,7 +74,16 @@ from app.domain.schemas.operational_task import (
|
||||
)
|
||||
from app.domain.schemas.operational_data import TaskDataSnapshot, TaskDataSource
|
||||
from app.services.content_digest import digest
|
||||
from app.services.investigation_graph import adopt_legacy_investigation_graph, update_investigation_graph
|
||||
from app.services.operational_presentation import resolve_overlay
|
||||
from app.services.operational_recovery import accept_completion, track_work_order
|
||||
from app.services.operational_scada_coverage import require_complete_scada_evidence
|
||||
from app.services import scada_presentation_coverage
|
||||
from app.services.time_comparison_chart import validate_time_comparison_chart
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
|
||||
|
||||
LEASE_DURATION = timedelta(seconds=60)
|
||||
@@ -86,11 +98,13 @@ def _task_json(model: OperationalTaskSummary) -> dict:
|
||||
|
||||
|
||||
class OperationalTaskService:
|
||||
def __init__(self, repository, context, permissions: Iterable[str], *, clock=None):
|
||||
def __init__(self, repository: OperationalTaskRepository, context, permissions: Iterable[str], *, clock=None,
|
||||
execution_scope: str = "production"):
|
||||
self.repo = repository
|
||||
self.context = context
|
||||
self.permissions = frozenset(permissions)
|
||||
self.clock = clock or (lambda: datetime.now(timezone.utc))
|
||||
self.execution_scope = execution_scope
|
||||
|
||||
def _now(self) -> datetime:
|
||||
now = self.clock()
|
||||
@@ -109,42 +123,14 @@ class OperationalTaskService:
|
||||
require(owner == command.agent_instance_id,
|
||||
"AGENT_INSTANCE_MISMATCH", "Agent instance does not own this run", status=409)
|
||||
|
||||
@property
|
||||
def _supports_runs(self) -> bool:
|
||||
return hasattr(self.repo, "get_active_run") and hasattr(self.repo, "update_run")
|
||||
|
||||
async def _active_run(self, task: dict) -> dict | None:
|
||||
if self._supports_runs:
|
||||
return await self.repo.get_active_run(str(task["task_id"]))
|
||||
if task.get("status") not in {"queued", "running"}:
|
||||
return None
|
||||
now = self._now()
|
||||
return _json(OperationalTaskRun(
|
||||
run_id=uuid5(NAMESPACE_URL, f"legacy-run:{task['task_id']}:{task['execution_generation']}"),
|
||||
task_id=task["task_id"],
|
||||
generation=task["execution_generation"],
|
||||
execution_request_id=task["execution_request_id"],
|
||||
status="running" if task["status"] == "running" else "queued",
|
||||
revision=task["revision"],
|
||||
agent_session_id=task.get("agent_session_id"),
|
||||
lease_expires_at=task.get("agent_lease_expires_at"),
|
||||
heartbeat_at=task.get("updated_at") if task["status"] == "running" else None,
|
||||
started_at=task.get("created_at") if task["status"] == "running" else None,
|
||||
dispatch_attempts=0,
|
||||
next_dispatch_at=task.get("created_at") or now,
|
||||
created_at=task.get("created_at") or now,
|
||||
updated_at=task.get("updated_at") or now,
|
||||
))
|
||||
return await self.repo.get_active_run(str(task["task_id"]))
|
||||
|
||||
async def _append_event(self, **kwargs) -> int | None:
|
||||
append_event = getattr(self.repo, "append_event", None)
|
||||
return await append_event(**kwargs) if append_event is not None else None
|
||||
async def _append_event(self, **kwargs) -> int:
|
||||
return await self.repo.append_event(**kwargs)
|
||||
|
||||
async def _list_access_requests(self, task_id: UUID | str) -> list[dict]:
|
||||
list_access_requests = getattr(self.repo, "list_access_requests", None)
|
||||
if list_access_requests is None:
|
||||
return []
|
||||
return await list_access_requests(str(task_id))
|
||||
return await self.repo.list_access_requests(str(task_id))
|
||||
|
||||
def _permission_view(self) -> OperationalTaskPermissions:
|
||||
permissions = self.permissions
|
||||
@@ -161,6 +147,14 @@ class OperationalTaskService:
|
||||
)
|
||||
|
||||
async def create(self, command: OperationalTaskCreate) -> OperationalTaskDetail:
|
||||
if command.source.replay_context:
|
||||
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
|
||||
replay_context = command.source.replay_context
|
||||
replay_doc = await ScadaReplayRepository(self.repo.connection, self.context.project_id).get(replay_context.replay_id)
|
||||
require(replay_doc.get("delivered_until") is not None
|
||||
and replay_context.knowledge_cutoff <= datetime.fromisoformat(replay_doc["delivered_until"])
|
||||
and replay_context.batch_sequence <= replay_doc["batch_sequence"],
|
||||
"REPLAY_SCOPE_INVALID", "回放调查不能使用尚未读取的批次", status=422)
|
||||
body = _json(command)
|
||||
request_id = str(command.request_id)
|
||||
scope = "task:create"
|
||||
@@ -185,6 +179,7 @@ class OperationalTaskService:
|
||||
now = self._now()
|
||||
task_id = uuid4()
|
||||
task = OperationalTaskSummary(
|
||||
execution_scope=self.execution_scope,
|
||||
project_id=self.context.project_id,
|
||||
task_id=task_id,
|
||||
task_type=command.task_type,
|
||||
@@ -219,6 +214,27 @@ class OperationalTaskService:
|
||||
await self._task_event(task_document, now)
|
||||
return await self.detail(task_id)
|
||||
|
||||
async def advance_replay_investigation(self, task_id: UUID, replay_context: dict, request_id: UUID) -> None:
|
||||
"""A recovery is new evidence, coalesced through the existing run lifecycle."""
|
||||
scope = f"task:{task_id}:replay-evidence"
|
||||
async with self.repo.transaction():
|
||||
if await self._request_replay(scope, request_id, replay_context):
|
||||
return
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
prior = task["source"].get("replay_context")
|
||||
require(prior and prior["replay_id"] == replay_context["replay_id"], "REPLAY_SCOPE_INVALID",
|
||||
"Replay evidence belongs to another investigation", status=409)
|
||||
require(datetime.fromisoformat(prior["knowledge_cutoff"]) <= datetime.fromisoformat(replay_context["knowledge_cutoff"]),
|
||||
"REPLAY_SCOPE_INVALID", "Replay evidence cannot move backwards", status=409)
|
||||
task["source"]["replay_context"] = replay_context
|
||||
task["source"]["external_revision"] = task["source"].get("external_revision", 1) + 1
|
||||
now = self._now()
|
||||
await self._requeue_task(task, now)
|
||||
await self.repo.update_task(_task_json(OperationalTaskSummary.model_validate(task)))
|
||||
await self._task_event(task, now)
|
||||
await self._save_request(scope, request_id, replay_context, {"task_id": str(task_id)})
|
||||
|
||||
async def list(
|
||||
self, *, task_type: str | None, status: str | None, limit: int, offset: int,
|
||||
peripheral_work_order_id: UUID | None = None,
|
||||
@@ -251,9 +267,10 @@ class OperationalTaskService:
|
||||
actions = await self.repo.list_actions(str(task_id))
|
||||
access_requests = await self._list_access_requests(task_id)
|
||||
active_run = await self._active_run(task)
|
||||
latest_run = (await self.repo.get_latest_run(str(task_id))
|
||||
if hasattr(self.repo, "get_latest_run") else active_run)
|
||||
latest_run = await self.repo.get_latest_run(str(task_id))
|
||||
progress = await self.repo.get_latest_progress(str(task_id), task["execution_generation"])
|
||||
if progress is None:
|
||||
progress = await self.repo.get_latest_graph_progress(str(task_id))
|
||||
work_order_snapshot = None
|
||||
if task["task_type"] == "external_work_order" and task["source"].get("external_id"):
|
||||
feedback = await self.repo.get_feedback(
|
||||
@@ -336,6 +353,9 @@ class OperationalTaskService:
|
||||
selected_report=await self.report(task_id, report_revision) if report_revision is not None else None,
|
||||
)
|
||||
|
||||
async def replay_context(self, task_id: UUID) -> dict | None:
|
||||
return (await self._task(task_id)).get("source", {}).get("replay_context")
|
||||
|
||||
async def can_read_source(self, task_id: UUID, source: TaskDataSource) -> bool:
|
||||
task = await self._task(task_id)
|
||||
permission = SCADA_VIEW if source.startswith("scada_") else SIMULATION_VIEW
|
||||
@@ -359,9 +379,8 @@ class OperationalTaskService:
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return TaskAccessRequest.model_validate(replay["access_request"])
|
||||
availability = getattr(self.repo, "access_requests_available", None)
|
||||
require(
|
||||
availability is None or await availability(),
|
||||
await self.repo.access_requests_available(),
|
||||
"DEPENDENCY_UNAVAILABLE",
|
||||
"Task data access requests are not enabled for this project database",
|
||||
status=503,
|
||||
@@ -455,6 +474,7 @@ class OperationalTaskService:
|
||||
task_type: str | None = None, task_created_from: datetime | None = None,
|
||||
) -> tuple[list[OperationalTaskRun], str | None]:
|
||||
rows = await self.repo.list_runs(
|
||||
execution_scope=self.execution_scope,
|
||||
status=status,
|
||||
limit=limit + 1,
|
||||
cursor_created_at=cursor_created_at,
|
||||
@@ -476,6 +496,9 @@ class OperationalTaskService:
|
||||
scope = f"run:{run_id}:claim"
|
||||
async with self.repo.transaction():
|
||||
await self.repo.require_execution_storage()
|
||||
task = await self._task(task_id)
|
||||
require(task.get("execution_scope") == self.execution_scope,
|
||||
"EXECUTION_SCOPE_MISMATCH", "当前环境不能领取其他执行范围的任务", status=403)
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return OperationalTaskRun.model_validate(replay["run"])
|
||||
@@ -538,6 +561,8 @@ class OperationalTaskService:
|
||||
return OperationalTaskRun.model_validate(replay["run"])
|
||||
await self.repo.lock(f"run:{run_id}")
|
||||
run = await self._run(task_id, run_id)
|
||||
require(run.get("failure_code") != "SUPERSEDED_BY_REQUEUE", "RUN_SUPERSEDED",
|
||||
"New task input superseded this run; the replacement run will continue", status=409)
|
||||
require(run["revision"] == command.expected_revision, "VERSION_CONFLICT",
|
||||
"Run revision changed", status=409)
|
||||
require(run["status"] in {"claimed", "running"}, "INVALID_TRANSITION",
|
||||
@@ -574,8 +599,7 @@ class OperationalTaskService:
|
||||
return OperationalTaskRun.model_validate(await self._run(task_id, run_id))
|
||||
|
||||
async def _fail_run_messages(self, run_id, reason: str, now: datetime) -> None:
|
||||
if hasattr(self.repo, "fail_run_messages"):
|
||||
await self.repo.fail_run_messages(str(run_id), reason, now)
|
||||
await self.repo.fail_run_messages(str(run_id), reason, now)
|
||||
|
||||
async def fail_run(
|
||||
self, task_id: UUID, run_id: UUID, command: AgentRunFailureCommand
|
||||
@@ -725,11 +749,12 @@ class OperationalTaskService:
|
||||
async with self.repo.transaction():
|
||||
for run in await self.repo.claim_expired_runs(
|
||||
limit, task_type=task_type, task_created_from=task_created_from,
|
||||
execution_scope=self.execution_scope,
|
||||
):
|
||||
# Lease recovery already holds a row lock. Try rather than wait
|
||||
# for the shared run lock to avoid reversing the lock order of
|
||||
# message/finish transactions; a busy run is checked next tick.
|
||||
if hasattr(self.repo, "try_lock") and not await self.repo.try_lock(f"run:{run['run_id']}"):
|
||||
if not await self.repo.try_lock(f"run:{run['run_id']}"):
|
||||
continue
|
||||
task_id = UUID(str(run["task_id"]))
|
||||
expected = run["revision"]
|
||||
@@ -741,6 +766,7 @@ class OperationalTaskService:
|
||||
timed_out = OperationalTaskRun.model_validate(run)
|
||||
await self.repo.update_run(_json(timed_out), expected)
|
||||
await self._fail_run_messages(run["run_id"], "Run lease expired", now)
|
||||
await task_questions.cancel_pending(self, task_id, now)
|
||||
await self._run_event(timed_out, now)
|
||||
task = await self._task(task_id)
|
||||
timeout_count = await self.repo.consecutive_timed_out_runs(str(task_id))
|
||||
@@ -795,6 +821,11 @@ class OperationalTaskService:
|
||||
"Only a running run accepts Agent artifacts", status=409)
|
||||
now = self._now()
|
||||
self._require_active_run_lease(active_run, now)
|
||||
if command.report:
|
||||
for item in command.report.ui_blocks:
|
||||
for element in item.block.spec.elements.values():
|
||||
if element.type == "Chart":
|
||||
validate_time_comparison_chart(element.props)
|
||||
if command.report and (
|
||||
command.report.stage == "final" or command.report.outcome == "insufficient_data"
|
||||
):
|
||||
@@ -828,11 +859,27 @@ class OperationalTaskService:
|
||||
for item in await self.repo.list_actions(str(task_id))
|
||||
}
|
||||
progress_revision = None
|
||||
saved_diagram = None
|
||||
if command.progress is not None or command.report is not None:
|
||||
stored = await self.repo.get_latest_graph_progress(str(task_id))
|
||||
if stored is not None:
|
||||
saved_diagram = InvestigationProgressSnapshot.model_validate(stored).progress.diagram
|
||||
elif command.progress is not None:
|
||||
legacy = await self.repo.get_latest_diagram(str(task_id))
|
||||
saved_diagram = InvestigationDiagram.model_validate(legacy) if legacy else None
|
||||
if command.progress is not None:
|
||||
require(command.run_id is not None, "RUN_REQUIRED",
|
||||
"Progress must identify its current execution run", status=422)
|
||||
if command.progress.diagram:
|
||||
for node in command.progress.diagram.node_refs:
|
||||
diagram = command.progress.diagram
|
||||
if command.progress.graph_update is not None:
|
||||
diagram = update_investigation_graph(saved_diagram, command.progress.graph_update)
|
||||
elif saved_diagram is not None:
|
||||
require(diagram is None or saved_diagram.edges is None, "GRAPH_UPDATE_REQUIRED",
|
||||
"Use graph_update to patch saved nodes or extend the investigation graph", status=422)
|
||||
if diagram is None:
|
||||
diagram = adopt_legacy_investigation_graph(saved_diagram)
|
||||
if diagram:
|
||||
for node in diagram.node_refs:
|
||||
self._require_evidence_refs(node.evidence_refs, known_evidence)
|
||||
require(all(str(ref) in actions for ref in node.action_refs),
|
||||
"ACTION_NOT_FOUND", "Progress references an unknown action", status=422)
|
||||
@@ -841,9 +888,11 @@ class OperationalTaskService:
|
||||
snapshot = InvestigationProgressSnapshot(
|
||||
revision=progress_revision, run_id=active_run["run_id"],
|
||||
execution_generation=task["execution_generation"],
|
||||
source_session_id=command.source_session_id, published_at=now, progress=command.progress,
|
||||
source_session_id=command.source_session_id, published_at=now,
|
||||
progress=InvestigationProgress(summary=command.progress.summary, diagram=diagram),
|
||||
)
|
||||
await self.repo.save_progress(str(task_id), _json(snapshot))
|
||||
saved_diagram = diagram
|
||||
await self._append_event(
|
||||
topic="operational.progress.changed", aggregate_type="operational_task_progress",
|
||||
aggregate_id=str(task_id), aggregate_revision=progress_revision, occurred_at=now,
|
||||
@@ -895,17 +944,32 @@ class OperationalTaskService:
|
||||
await self.repo.insert_plan(str(task_id), _json(snapshot))
|
||||
current_plan = snapshot
|
||||
if command.report:
|
||||
self._require_evidence_refs(command.report.evidence_refs, known_evidence)
|
||||
for gap in command.report.data_gaps:
|
||||
require(set(gap.evidence_refs) <= set(command.report.evidence_refs),
|
||||
report_input = command.report
|
||||
if saved_diagram is not None:
|
||||
diagram = adopt_legacy_investigation_graph(saved_diagram)
|
||||
report_input = report_input.model_copy(update={
|
||||
"diagram": diagram,
|
||||
"evidence_refs": list(dict.fromkeys([
|
||||
*report_input.evidence_refs,
|
||||
*(ref for node in diagram.node_refs for ref in node.evidence_refs),
|
||||
])),
|
||||
})
|
||||
self._require_evidence_refs(report_input.evidence_refs, known_evidence)
|
||||
if report_input.stage == "final":
|
||||
require_complete_scada_evidence(
|
||||
task, await self.repo.list_evidence(str(task_id)), report_input.evidence_refs,
|
||||
)
|
||||
await self._verify_scada_report_presentation(task, active_run, _json(report_input), now)
|
||||
for gap in report_input.data_gaps:
|
||||
require(set(gap.evidence_refs) <= set(report_input.evidence_refs),
|
||||
"REPORT_GAP_EVIDENCE_INVALID",
|
||||
"Gap evidence must be included in report evidence", status=422)
|
||||
if command.report.diagram:
|
||||
for node in command.report.diagram.node_refs:
|
||||
self._require_evidence_refs(node.evidence_refs, set(command.report.evidence_refs))
|
||||
if report_input.diagram:
|
||||
for node in report_input.diagram.node_refs:
|
||||
self._require_evidence_refs(node.evidence_refs, set(report_input.evidence_refs))
|
||||
require(all(str(ref) in actions for ref in node.action_refs),
|
||||
"ACTION_NOT_FOUND", "Diagram references an unknown action", status=422)
|
||||
if command.report.outcome == "insufficient_data":
|
||||
if report_input.outcome == "insufficient_data":
|
||||
require(not any(action.approval_status == "pending" or (
|
||||
action.approval_status == "approved" and action.execution_status in {"not_submitted", "submitted", "unknown"}
|
||||
) for action in actions.values()), "REPORT_ACTION_PENDING",
|
||||
@@ -915,7 +979,7 @@ class OperationalTaskService:
|
||||
revision=report_revision,
|
||||
execution_generation=task["execution_generation"],
|
||||
source_session_id=command.source_session_id,
|
||||
published_at=now, published_by=self._actor(), report=command.report,
|
||||
published_at=now, published_by=self._actor(), report=report_input,
|
||||
)
|
||||
report = await self._report_with_progress_diagram(task_id, report)
|
||||
await self.repo.insert_report(str(task_id), _json(report))
|
||||
@@ -928,20 +992,21 @@ class OperationalTaskService:
|
||||
metadata={"task_id": str(task_id), "run_id": str(active_run["run_id"])},
|
||||
)
|
||||
previous_result = previous_report["report"] if previous_report else None
|
||||
if command.report.stage == "final" and (
|
||||
effective_report = _json(report.report)
|
||||
if report.report.stage == "final" and (
|
||||
previous_report is None
|
||||
or previous_report.get("execution_generation") != task["execution_generation"]
|
||||
or previous_result.get("stage") != "final"
|
||||
or any(previous_result.get(key, [] if key == "ui_blocks" else None) != _json(command.report).get(key)
|
||||
or any(previous_result.get(key, [] if key == "ui_blocks" else None) != effective_report.get(key)
|
||||
for key in ("current_conclusion", "conclusion_boundary", "outcome", "stop_reason",
|
||||
"evidence_refs", "recommendations", "pending_items", "data_gaps", "ui_blocks"))
|
||||
"evidence_refs", "recommendations", "pending_items", "data_gaps", "ui_blocks", "scada_device_ids"))
|
||||
):
|
||||
await self._resolve_notifications(
|
||||
task_id, kinds=["report_updated"], subject_type="task", subject_id=task_id,
|
||||
now=now, reason="report_superseded",
|
||||
)
|
||||
await self._notify(
|
||||
task_id, "report_updated", task["title"], command.report.current_conclusion, now,
|
||||
task_id, "report_updated", task["title"], report.report.current_conclusion, now,
|
||||
report_reference=InvestigationReportReference(
|
||||
execution_generation=task["execution_generation"], revision=report_revision,
|
||||
),
|
||||
@@ -965,25 +1030,9 @@ class OperationalTaskService:
|
||||
lease_expires_at=now + LEASE_DURATION,
|
||||
updated_at=now,
|
||||
)
|
||||
if self._supports_runs:
|
||||
await self.repo.update_run(
|
||||
_json(OperationalTaskRun.model_validate(active_run)), run_expected
|
||||
)
|
||||
else:
|
||||
# Compatibility for pre-015 repository adapters only.
|
||||
if command.report:
|
||||
if task["task_type"] == "event_investigation":
|
||||
task["status"] = self._investigation_status(actions.values()) if command.report.stage == "final" else "running"
|
||||
else:
|
||||
task["status"] = self._status_from_plan(current_plan.plan.nodes, actions.values()) if current_plan else "running"
|
||||
if task["status"] in {"waiting_human", "completed", "failed", "cancelled"}:
|
||||
task["lifecycle_status"] = task["status"]
|
||||
task["agent_lease_expires_at"] = (
|
||||
now + LEASE_DURATION if task["status"] == "running" else None
|
||||
)
|
||||
await self.repo.update_task(
|
||||
_task_json(OperationalTaskSummary.model_validate(task))
|
||||
)
|
||||
await self.repo.update_run(
|
||||
_json(OperationalTaskRun.model_validate(active_run)), run_expected
|
||||
)
|
||||
await self._task_event(task, now)
|
||||
response = {"plan_revision": plan_revision, "report_revision": report_revision}
|
||||
if progress_revision is not None:
|
||||
@@ -1018,6 +1067,8 @@ class OperationalTaskService:
|
||||
run_id=row["metadata"]["run_id"],
|
||||
execution_generation=row["metadata"]["execution_generation"],
|
||||
command=row["metadata"]["command"],
|
||||
narration=row["metadata"].get("narration"),
|
||||
purpose=row["metadata"].get("purpose"),
|
||||
) for row in rows[:limit]]
|
||||
return TaskPresentationHistoryPage(
|
||||
task_id=task_id, event_cursor=boundary, items=items,
|
||||
@@ -1048,7 +1099,14 @@ class OperationalTaskService:
|
||||
now = self._now()
|
||||
self._require_active_run_lease(run, now)
|
||||
payload = _json(command.command)
|
||||
if command.narration is not None:
|
||||
known = {item["evidence_id"] for item in await self.repo.list_evidence(str(task_id))}
|
||||
self._require_evidence_refs(command.narration.evidence_refs, known)
|
||||
await self._record_scada_presentation(task, run, payload, now)
|
||||
if command.command.action == "open_chart":
|
||||
validate_time_comparison_chart(payload["chart"], allow_missing_current=bool(
|
||||
payload["chart"].get("scada") and scada_presentation_coverage.applies(task),
|
||||
))
|
||||
known = {item["evidence_id"] for item in await self.repo.list_evidence(str(task_id))}
|
||||
self._require_evidence_refs(command.command.evidence_refs, known)
|
||||
elif command.command.action == "set_overlay":
|
||||
@@ -1069,6 +1127,10 @@ class OperationalTaskService:
|
||||
payload["data"] = resolved
|
||||
metadata = {"task_id": str(task_id), "run_id": str(run["run_id"]),
|
||||
"execution_generation": run["generation"], "command": payload}
|
||||
if command.narration is not None:
|
||||
metadata["narration"] = _json(command.narration)
|
||||
if command.purpose is not None:
|
||||
metadata["purpose"] = command.purpose
|
||||
try:
|
||||
size = len(json.dumps(metadata, ensure_ascii=False, allow_nan=False).encode("utf-8"))
|
||||
except (ValueError, TypeError) as exc:
|
||||
@@ -1086,6 +1148,61 @@ class OperationalTaskService:
|
||||
await self._save_request(scope, command.request_id, body, _json(result))
|
||||
return result
|
||||
|
||||
async def _record_scada_presentation(self, task: dict, run: dict, payload: dict, now: datetime) -> None:
|
||||
if not scada_presentation_coverage.applies(task):
|
||||
return
|
||||
action = payload["action"]
|
||||
targets = []
|
||||
if action == "render_scada_analysis":
|
||||
targets = [item["device_id"] for item in payload["items"]]
|
||||
elif action in {"locate_features", "highlight_features"} and payload["layer"] == "scada":
|
||||
targets = payload["feature_ids"]
|
||||
elif action != "open_chart":
|
||||
return
|
||||
evidence = await self.repo.list_evidence(task["task_id"])
|
||||
observations = scada_presentation_coverage.ScadaObservations(task, evidence)
|
||||
document = await self.repo.get_scada_presentation(task["task_id"], str(run["run_id"]))
|
||||
if action == "open_chart":
|
||||
if payload["chart"].get("scada"):
|
||||
known = {item["evidence_id"] for item in evidence}
|
||||
self._require_evidence_refs(payload["evidence_refs"], known)
|
||||
chart = scada_presentation_coverage.verify_chart(task, payload["chart"], evidence, payload["evidence_refs"])
|
||||
targets = chart["device_ids"]
|
||||
duplicates = [chart_id for chart_id, stored in (document or {}).get("charts", {}).items()
|
||||
if chart_id != payload["chart_id"] and stored["role"] == chart["role"] == "comparison"
|
||||
and any(set(stored.get("coverage", {}).get(device, {}).get("covered", []))
|
||||
& set(chart["coverage"][device]["covered"])
|
||||
for device in set(stored["device_ids"]) & set(targets))]
|
||||
require(not duplicates, "SCADA_CHART_DUPLICATE_DEVICE",
|
||||
f"设备同期图已存在,请更新原chart_id:{', '.join(duplicates)}", refs=duplicates)
|
||||
document = scada_presentation_coverage.add_targets(document, targets, observations)
|
||||
document["charts"][payload["chart_id"]] = chart
|
||||
elif document and payload["chart_id"] in document["charts"]:
|
||||
# Replacing a typed chart with a generic one removes its coverage,
|
||||
# while cumulative targets prevent shrinking the required set.
|
||||
del document["charts"][payload["chart_id"]]
|
||||
else:
|
||||
return
|
||||
else:
|
||||
document = scada_presentation_coverage.add_targets(document, targets, observations)
|
||||
await self.repo.save_scada_presentation(task["task_id"], str(run["run_id"]), document, now)
|
||||
|
||||
async def _verify_scada_report_presentation(self, task: dict, run: dict, report: dict, now: datetime) -> None:
|
||||
if not scada_presentation_coverage.applies(task):
|
||||
return
|
||||
if report.get("stage") != "final" and not report.get("scada_device_ids"):
|
||||
return
|
||||
evidence = await self.repo.list_evidence(task["task_id"])
|
||||
observations = scada_presentation_coverage.ScadaObservations(task, evidence)
|
||||
document = await self.repo.get_scada_presentation(task["task_id"], str(run["run_id"]))
|
||||
targets = list(report.get("scada_device_ids") or [])
|
||||
if targets:
|
||||
document = scada_presentation_coverage.add_targets(document, targets, observations)
|
||||
if report.get("stage") == "final":
|
||||
scada_presentation_coverage.require_complete(task, document, observations)
|
||||
if document:
|
||||
await self.repo.save_scada_presentation(task["task_id"], str(run["run_id"]), document, now)
|
||||
|
||||
async def propose_action(
|
||||
self, task_id: UUID, command: ActionProposalCommand
|
||||
) -> HumanApprovalAction:
|
||||
@@ -1099,6 +1216,7 @@ class OperationalTaskService:
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
self._require_real_action(task, command.kind)
|
||||
self._validate_action_payload(command)
|
||||
if task["task_type"] == "event_investigation":
|
||||
report = await self.repo.get_latest_report(str(task_id))
|
||||
@@ -1150,6 +1268,7 @@ class OperationalTaskService:
|
||||
task_id, "action_approval_pending", task["title"], command.reason, now,
|
||||
required_permission=approval_permission,
|
||||
subject_type="action", subject_id=action.action_id,
|
||||
work_order_action=action if action.kind == "create_work_order" else None,
|
||||
)
|
||||
await self._save_request(
|
||||
scope, command.request_id, body, {"action_id": str(action.action_id)}
|
||||
@@ -1167,8 +1286,10 @@ class OperationalTaskService:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
self._require_actionable_task(await self._task(task_id))
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
action = await self._action(task_id, action_id)
|
||||
self._require_real_action(task, action.kind)
|
||||
require(action.kind == "create_work_order", "INVALID_ACTION_KIND",
|
||||
"Only work-order drafts can be edited", status=422)
|
||||
require(action.version == command.expected_version, "VERSION_CONFLICT",
|
||||
@@ -1221,8 +1342,10 @@ class OperationalTaskService:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
self._require_actionable_task(await self._task(task_id))
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
action = await self._action(task_id, action_id)
|
||||
self._require_real_action(task, action.kind)
|
||||
require(action.version == command.expected_version, "VERSION_CONFLICT",
|
||||
"Action version changed", status=409)
|
||||
require(action.approval_status == "pending", "INVALID_TRANSITION",
|
||||
@@ -1266,8 +1389,10 @@ class OperationalTaskService:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
self._require_actionable_task(await self._task(task_id))
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
action = await self._action(task_id, action_id)
|
||||
self._require_real_action(task, action.kind)
|
||||
require(action.version == command.expected_version, "VERSION_CONFLICT",
|
||||
"Action version changed", status=409)
|
||||
require(action.approval_status == "approved", "INVALID_TRANSITION",
|
||||
@@ -1327,6 +1452,7 @@ class OperationalTaskService:
|
||||
|
||||
async with self.repo.transaction():
|
||||
await self.repo.lock(f"request:{scope}:{command.request_id}")
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
current = await self._action(task_id, action_id)
|
||||
require(current.version == action.version and current.execution_status == "unknown",
|
||||
@@ -1362,7 +1488,12 @@ class OperationalTaskService:
|
||||
)
|
||||
if outcome.status in {"succeeded", "failed", "unsupported"}:
|
||||
task_document = await self._task(task_id)
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
tracked = track_work_order(task_document, _json(current))
|
||||
if (tracked or task_document["status"] == "cancelled"
|
||||
or (task_document.get("source") or {}).get("deleted_at")):
|
||||
await self._keep_work_order_wait(task_document, outcome_recorded_at)
|
||||
else:
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
await self.repo.update_task(
|
||||
_task_json(OperationalTaskSummary.model_validate(task_document))
|
||||
)
|
||||
@@ -1398,6 +1529,7 @@ class OperationalTaskService:
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
current = await self._action(task_id, action_id)
|
||||
require(current.version == action.version, "VERSION_CONFLICT",
|
||||
@@ -1435,7 +1567,12 @@ class OperationalTaskService:
|
||||
)
|
||||
if outcome.status in {"succeeded", "failed", "unsupported"}:
|
||||
task_document = await self._task(task_id)
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
tracked = track_work_order(task_document, _json(current))
|
||||
if (tracked or task_document["status"] == "cancelled"
|
||||
or (task_document.get("source") or {}).get("deleted_at")):
|
||||
await self._keep_work_order_wait(task_document, outcome_recorded_at)
|
||||
else:
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
await self.repo.update_task(
|
||||
_task_json(OperationalTaskSummary.model_validate(task_document))
|
||||
)
|
||||
@@ -1451,11 +1588,13 @@ class OperationalTaskService:
|
||||
body = _json(command)
|
||||
scope = f"task:{task_id}:external-feedback"
|
||||
async with self.repo.transaction():
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
require(task.get("execution_scope") == self.execution_scope,
|
||||
"EXECUTION_SCOPE_MISMATCH", "当前环境不能同步其他执行范围的任务", status=403)
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return ExternalFeedbackResult(task=await self.detail(task_id), **replay)
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
if task["task_type"] == "event_investigation":
|
||||
self._require_actionable_task(task)
|
||||
own_source = task["source"].get("external_id")
|
||||
@@ -1537,8 +1676,8 @@ class OperationalTaskService:
|
||||
"EVIDENCE_CONFLICT", "Evidence ids are immutable", status=409)
|
||||
else:
|
||||
await self.repo.insert_evidence(str(task_id), _json(evidence))
|
||||
self._apply_feedback_status(task, command.status)
|
||||
if task["task_type"] == "external_work_order":
|
||||
self._apply_feedback_status(task, command.status)
|
||||
task["source"] = {
|
||||
**task["source"],
|
||||
"external_revision": command.source_revision,
|
||||
@@ -1555,9 +1694,30 @@ class OperationalTaskService:
|
||||
),
|
||||
)
|
||||
if task["task_type"] == "event_investigation":
|
||||
# Recover older tasks whose successful dispatch predates tracking.
|
||||
for item in await self.repo.list_actions(str(task_id)):
|
||||
track_work_order(task, item)
|
||||
linked_work_order = accept_completion(task, body, recorded_at=now)
|
||||
if matching_source:
|
||||
task["source"] = {**task["source"], "external_revision": command.source_revision}
|
||||
await self._requeue_task(task, now)
|
||||
feedback_order = (task.get("handling_context") or {}).get("work_orders", {}).get(
|
||||
command.source_event_id.removeprefix("peripheral_sample:"), {})
|
||||
if linked_work_order and (feedback_order.get("superseded_by")
|
||||
or task["handling_context"].get("phase") == "completed"):
|
||||
# A receipt edit with the same completion time does not
|
||||
# reopen an incident that already passed recovery review.
|
||||
task.update(revision=task["revision"] + 1, updated_at=now)
|
||||
elif (linked_work_order and command.status not in {"failed", "cancelled"}
|
||||
and task["handling_context"].get("phase") in {"waiting_work_order", "waiting_scada"}):
|
||||
# Progress is persisted, not a reason to restart analysis.
|
||||
# Completed field work waits for causal SCADA evidence;
|
||||
# reconcile_recoveries wakes the same task when it is ready.
|
||||
await self._keep_work_order_wait(task, now)
|
||||
elif (linked_work_order and task["handling_context"].get("phase") == "verification_ready"
|
||||
and await self.repo.get_active_run(str(task_id)) is not None):
|
||||
await self._keep_work_order_wait(task, now)
|
||||
else:
|
||||
await self._requeue_task(task, now)
|
||||
else:
|
||||
task.update(revision=task["revision"] + 1, updated_at=now)
|
||||
await self.repo.update_task(_task_json(OperationalTaskSummary.model_validate(task)))
|
||||
@@ -1594,6 +1754,29 @@ class OperationalTaskService:
|
||||
task_id, "task_failed",
|
||||
task["title"], command.detail, now,
|
||||
)
|
||||
if command.source_revision > 1 and task["task_type"] in {"external_work_order", "event_investigation"}:
|
||||
# The feedback, its notification and outbox event commit together.
|
||||
# Duplicate request/source versions return above, before publishing.
|
||||
work_order_action = next((item for item in await self.repo.list_actions(str(task_id))
|
||||
if item["kind"] == "create_work_order" and item.get("receipt")
|
||||
and command.source_event_id == f"{item['receipt']['adapter']}:{item['receipt']['receipt_id']}"), None)
|
||||
is_work_order_feedback = (task["task_type"] == "external_work_order"
|
||||
or work_order_action is not None
|
||||
or (task["task_type"] == "event_investigation" and matching_source))
|
||||
if is_work_order_feedback:
|
||||
await self._notify(
|
||||
task_id, "work_order_feedback", "收到工单反馈",
|
||||
command.evidence.summary if command.evidence else command.detail, now,
|
||||
subject_type="action" if work_order_action else "task",
|
||||
subject_id=UUID(str(work_order_action["action_id"])) if work_order_action else task_id,
|
||||
work_order_action=HumanApprovalAction.model_validate(work_order_action) if work_order_action else None,
|
||||
work_order_reference=WorkOrderNotificationReference(
|
||||
source_event_id=command.source_event_id,
|
||||
source_revision=command.source_revision,
|
||||
evidence_id=command.evidence.evidence_id if command.evidence else None,
|
||||
status=command.status,
|
||||
),
|
||||
)
|
||||
response = {
|
||||
"source_event_id": command.source_event_id,
|
||||
"source_revision": command.source_revision,
|
||||
@@ -1608,11 +1791,13 @@ class OperationalTaskService:
|
||||
body = _json(command)
|
||||
scope = f"task:{task_id}:withdraw-external-source"
|
||||
async with self.repo.transaction():
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
require(task.get("execution_scope") == self.execution_scope,
|
||||
"EXECUTION_SCOPE_MISMATCH", "当前环境不能同步其他执行范围的任务", status=403)
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return await self.detail(task_id)
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
source = task["source"]
|
||||
require(
|
||||
task["task_type"] == "external_work_order"
|
||||
@@ -1918,13 +2103,12 @@ class OperationalTaskService:
|
||||
return DispatchDueSchedulesResult(task_ids=task_ids, dispatched_at=now)
|
||||
|
||||
async def _require_no_pending_calculations(self, task_id: UUID, run: dict) -> None:
|
||||
if hasattr(self.repo, "pending_calculation_jobs"):
|
||||
pending = await self.repo.pending_calculation_jobs(
|
||||
str(task_id), str(run["run_id"]), str(run["execution_request_id"]),
|
||||
)
|
||||
require(not pending, "PENDING_CALCULATIONS",
|
||||
"Wait for submitted calculations before publishing a final report or completing the run",
|
||||
refs=pending, status=409, recovery="retry_same_request")
|
||||
pending = await self.repo.pending_calculation_jobs(
|
||||
str(task_id), str(run["run_id"]), str(run["execution_request_id"]),
|
||||
)
|
||||
require(not pending, "PENDING_CALCULATIONS",
|
||||
"Wait for submitted calculations before publishing a final report or completing the run",
|
||||
refs=pending, status=409, recovery="retry_same_request")
|
||||
|
||||
async def _finish_run(
|
||||
self, task_id: UUID, run_id: UUID, command: AgentRunCommand,
|
||||
@@ -1954,11 +2138,10 @@ class OperationalTaskService:
|
||||
actions = []
|
||||
if outcome == "succeeded":
|
||||
await self._require_no_pending_calculations(task_id, run)
|
||||
if hasattr(self.repo, "pending_run_messages"):
|
||||
pending_messages = await self.repo.pending_run_messages(str(run_id))
|
||||
require(not pending_messages, "PENDING_AGENT_MESSAGES",
|
||||
"Run has messages not yet entered into Agent context",
|
||||
status=409, recovery="retry_same_request", refs=pending_messages)
|
||||
pending_messages = await self.repo.pending_run_messages(str(run_id))
|
||||
require(not pending_messages, "PENDING_AGENT_MESSAGES",
|
||||
"Run has messages not yet entered into Agent context",
|
||||
status=409, recovery="retry_same_request", refs=pending_messages)
|
||||
plan_document = await self.repo.get_latest_plan(str(task_id))
|
||||
report_document = await self.repo.get_latest_report(str(task_id))
|
||||
plan = (
|
||||
@@ -1979,8 +2162,22 @@ class OperationalTaskService:
|
||||
and plan.source_session_id == command.source_session_id
|
||||
))
|
||||
)
|
||||
actions = [HumanApprovalAction.model_validate(item)
|
||||
for item in await self.repo.list_actions(str(task_id))]
|
||||
work_order_wait = task["task_type"] == "event_investigation" and (
|
||||
(task.get("handling_context") or {}).get("phase") in {"waiting_work_order", "waiting_scada"}
|
||||
or any(action.kind == "create_work_order" and (
|
||||
action.approval_status == "pending" or (
|
||||
action.approval_status == "approved"
|
||||
and action.execution_status in {"not_submitted", "submitted", "unknown"}
|
||||
)) for action in actions)
|
||||
)
|
||||
if pending_question:
|
||||
lifecycle = "waiting_human"
|
||||
elif same_generation and work_order_wait and report.report.stage == "interim":
|
||||
# Ending a run is distinct from closing an incident. Keep
|
||||
# the current stage report and release the execution lease.
|
||||
lifecycle = "waiting_human"
|
||||
elif not same_generation or report.report.stage != "final":
|
||||
final_outcome = "failed"
|
||||
lifecycle = "failed"
|
||||
@@ -1989,16 +2186,33 @@ class OperationalTaskService:
|
||||
"Run requires a final report from the same generation"
|
||||
)
|
||||
else:
|
||||
actions = [
|
||||
HumanApprovalAction.model_validate(item)
|
||||
for item in await self.repo.list_actions(str(task_id))
|
||||
]
|
||||
await self._verify_scada_report_presentation(task, run, _json(report.report), now)
|
||||
access_requests = [
|
||||
TaskAccessRequest.model_validate(item)
|
||||
for item in await self._list_access_requests(task_id)
|
||||
]
|
||||
if task["task_type"] == "event_investigation":
|
||||
lifecycle = self._investigation_status(actions)
|
||||
lifecycle = self._investigation_status(actions, task)
|
||||
recovery = task.get("handling_context") or {}
|
||||
if lifecycle == "completed" and recovery:
|
||||
if task.get("workflow") == "water_pollution":
|
||||
orders = [item for item in recovery.get("work_orders", {}).values()
|
||||
if not item.get("superseded_by")]
|
||||
feedback_refs = [item.get("feedback_evidence_id") for item in orders]
|
||||
require(recovery.get("phase") in {"field_review", "completed"} and bool(orders)
|
||||
and all(item["status"] == "completed" for item in orders)
|
||||
and all(ref and ref in report.report.evidence_refs for ref in feedback_refs),
|
||||
"POLLUTION_FEEDBACK_REQUIRED", "水污染最终报告必须复核并引用全部最新现场反馈证据", status=409)
|
||||
verification_id = feedback_refs[-1]
|
||||
else:
|
||||
verification_id = recovery.get("verification_evidence_id")
|
||||
require(verification_id in report.report.evidence_refs,
|
||||
"RECOVERY_EVIDENCE_REQUIRED", "最终复核报告必须引用恢复监测证据", status=409)
|
||||
require(report.report.outcome == "concluded", "RECOVERY_EVIDENCE_REQUIRED",
|
||||
"恢复复核通过后才能结束事件", status=409)
|
||||
recovery.update(phase="completed", title=("水污染处置反馈已复核" if task.get("workflow") == "water_pollution"
|
||||
else "恢复供水复核成功"), completed_at=now.isoformat())
|
||||
await self._resolve_investigation_alerts(str(task_id), verification_id)
|
||||
elif report.report.outcome == "insufficient_data":
|
||||
terminal = all(node.status in {"completed", "skipped", "cancelled"}
|
||||
for node in plan.plan.nodes)
|
||||
@@ -2047,7 +2261,8 @@ class OperationalTaskService:
|
||||
failure_detail or "工况任务执行失败",
|
||||
now,
|
||||
)
|
||||
elif lifecycle == "waiting_human" and not pending_question:
|
||||
elif (lifecycle == "waiting_human" and not pending_question
|
||||
and (task.get("handling_context") or {}).get("phase") not in {"waiting_work_order", "waiting_scada"}):
|
||||
pending_actions = any(
|
||||
action.approval_status == "pending" for action in actions
|
||||
)
|
||||
@@ -2074,11 +2289,31 @@ class OperationalTaskService:
|
||||
)
|
||||
return run
|
||||
|
||||
async def _resolve_investigation_alerts(self, task_id: str, evidence_id: str) -> None:
|
||||
from app.domain.schemas.task_alert import TaskAlertEvent
|
||||
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
|
||||
from app.services.task_alerts import TaskAlertService
|
||||
|
||||
for alert in await self.repo.list_investigation_alerts(task_id):
|
||||
command = TaskAlertEvent.model_validate({
|
||||
**{key: value for key, value in alert.items() if key not in {"status", "updated_at", "revision"}},
|
||||
"event_id": uuid5(NAMESPACE_URL, f"recovery:{task_id}:{evidence_id}:{alert['alert_id']}"),
|
||||
"event_type": "resolved",
|
||||
})
|
||||
await TaskAlertService(TaskAlertRepository(self.repo.connection, self.repo.project_id), None,
|
||||
internal_producer_id=command.producer_id).push(command)
|
||||
|
||||
async def _task(self, task_id: UUID) -> dict:
|
||||
task = await self.repo.get_task(str(task_id))
|
||||
require(task is not None, "TASK_NOT_FOUND", "Operational task not found", status=404)
|
||||
return task
|
||||
|
||||
@staticmethod
|
||||
def _require_real_action(task: dict, kind: str) -> None:
|
||||
require(kind == "create_work_order" or not (task.get("source") or {}).get("replay_context"),
|
||||
"REPLAY_ACTION_DISABLED",
|
||||
"回放任务不支持设备命令;可生成工单草稿,由有权限的用户确认发布", status=409)
|
||||
|
||||
@staticmethod
|
||||
def _require_actionable_task(task: dict) -> None:
|
||||
source = task.get("source") or {}
|
||||
@@ -2142,8 +2377,10 @@ class OperationalTaskService:
|
||||
return "running"
|
||||
|
||||
@staticmethod
|
||||
def _investigation_status(actions: Iterable[HumanApprovalAction]) -> str:
|
||||
"""Only recorded business actions can keep an investigation waiting."""
|
||||
def _investigation_status(actions: Iterable[HumanApprovalAction], task: dict | None = None) -> str:
|
||||
"""Field completion and recovery evidence precede incident closure."""
|
||||
if (task or {}).get("handling_context", {}).get("phase") in {"waiting_work_order", "waiting_scada"}:
|
||||
return "waiting_human"
|
||||
return "waiting_human" if any(
|
||||
action.approval_status == "pending" or (
|
||||
action.approval_status == "approved"
|
||||
@@ -2259,44 +2496,58 @@ class OperationalTaskService:
|
||||
}[status]
|
||||
task["agent_lease_expires_at"] = None
|
||||
|
||||
async def _keep_work_order_wait(self, task: dict, now: datetime) -> None:
|
||||
"""Persist a field wait without revoking a run or allocating a generation."""
|
||||
if task["status"] != "cancelled" and not (task.get("source") or {}).get("deleted_at"):
|
||||
active = await self.repo.get_active_run(str(task["task_id"]))
|
||||
if active is not None:
|
||||
task.update(
|
||||
lifecycle_status="open", status="queued" if active["status"] == "queued" else "running",
|
||||
agent_session_id=active.get("agent_session_id"),
|
||||
agent_lease_expires_at=active.get("lease_expires_at"),
|
||||
)
|
||||
else:
|
||||
task.update(lifecycle_status="waiting_human", status="waiting_human",
|
||||
agent_session_id=None, agent_lease_expires_at=None)
|
||||
task.update(revision=task["revision"] + 1, updated_at=now)
|
||||
|
||||
async def _requeue_task(self, task: dict, now: datetime) -> None:
|
||||
await task_questions.cancel_pending(self, UUID(str(task["task_id"])), now)
|
||||
if self._supports_runs:
|
||||
active_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
while active_run is not None:
|
||||
await self.repo.lock(f"run:{active_run['run_id']}")
|
||||
# Completion may have won the run lock after our first read.
|
||||
refreshed_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
if refreshed_run is None or str(refreshed_run["run_id"]) == str(active_run["run_id"]):
|
||||
active_run = refreshed_run
|
||||
break
|
||||
active_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
while active_run is not None:
|
||||
await self.repo.lock(f"run:{active_run['run_id']}")
|
||||
# Completion may have won the run lock after our first read.
|
||||
refreshed_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
if refreshed_run is None or str(refreshed_run["run_id"]) == str(active_run["run_id"]):
|
||||
active_run = refreshed_run
|
||||
if active_run is not None and active_run["status"] == "queued":
|
||||
# Multiple inputs can arrive before the worker claims the next
|
||||
# generation. They are all visible when that queued run starts,
|
||||
# so coalesce them instead of violating the one-active-run rule.
|
||||
task.update(
|
||||
lifecycle_status="open", status="queued",
|
||||
revision=task["revision"] + 1,
|
||||
execution_generation=active_run["generation"],
|
||||
execution_request_id=str(active_run["execution_request_id"]),
|
||||
agent_session_id=None, agent_lease_expires_at=None,
|
||||
updated_at=now,
|
||||
)
|
||||
return
|
||||
if active_run is not None:
|
||||
expected = active_run["revision"]
|
||||
active_run.update(
|
||||
status="cancelled", revision=expected + 1,
|
||||
lease_expires_at=None, finished_at=now,
|
||||
failure_code="SUPERSEDED_BY_REQUEUE",
|
||||
failure_detail="New task input superseded the active run",
|
||||
updated_at=now,
|
||||
)
|
||||
cancelled = OperationalTaskRun.model_validate(active_run)
|
||||
await self.repo.update_run(_json(cancelled), expected)
|
||||
await self._fail_run_messages(active_run["run_id"], "Run superseded by requeue", now)
|
||||
await self._run_event(cancelled, now)
|
||||
break
|
||||
active_run = refreshed_run
|
||||
if active_run is not None and active_run["status"] == "queued":
|
||||
# Multiple inputs can arrive before the worker claims the next
|
||||
# generation. They are all visible when that queued run starts,
|
||||
# so coalesce them instead of violating the one-active-run rule.
|
||||
task.update(
|
||||
lifecycle_status="open", status="queued",
|
||||
revision=task["revision"] + 1,
|
||||
execution_generation=active_run["generation"],
|
||||
execution_request_id=str(active_run["execution_request_id"]),
|
||||
agent_session_id=None, agent_lease_expires_at=None,
|
||||
updated_at=now,
|
||||
)
|
||||
return
|
||||
if active_run is not None:
|
||||
expected = active_run["revision"]
|
||||
active_run.update(
|
||||
status="cancelled", revision=expected + 1,
|
||||
lease_expires_at=None, finished_at=now,
|
||||
failure_code="SUPERSEDED_BY_REQUEUE",
|
||||
failure_detail="New task input superseded the active run",
|
||||
updated_at=now,
|
||||
)
|
||||
cancelled = OperationalTaskRun.model_validate(active_run)
|
||||
await self.repo.update_run(_json(cancelled), expected)
|
||||
await self._fail_run_messages(active_run["run_id"], "Run superseded by requeue", now)
|
||||
await self._run_event(cancelled, now)
|
||||
generation = task["execution_generation"] + 1
|
||||
task_id = UUID(task["task_id"])
|
||||
task.update(
|
||||
@@ -2306,15 +2557,16 @@ class OperationalTaskService:
|
||||
agent_session_id=None, agent_lease_expires_at=None, updated_at=now,
|
||||
)
|
||||
run = self._new_run(task_id, generation, now)
|
||||
if self._supports_runs:
|
||||
await self.repo.insert_run(_json(run))
|
||||
await self._run_event(run, now, queued=True)
|
||||
await self.repo.insert_run(_json(run))
|
||||
await self._run_event(run, now, queued=True)
|
||||
|
||||
async def _notify(
|
||||
self, task_id: UUID, kind: str, title: str, description: str, now: datetime,
|
||||
*, required_permission: str | None = None,
|
||||
subject_type: str = "task", subject_id: UUID | None = None,
|
||||
report_reference: InvestigationReportReference | None = None,
|
||||
work_order_reference: WorkOrderNotificationReference | None = None,
|
||||
work_order_action: HumanApprovalAction | None = None,
|
||||
) -> None:
|
||||
notification = OperationalTaskNotification(
|
||||
notification_id=uuid4(), task_id=task_id, kind=kind,
|
||||
@@ -2323,6 +2575,8 @@ class OperationalTaskService:
|
||||
lifecycle_status="active", subject_type=subject_type,
|
||||
subject_id=subject_id or task_id,
|
||||
report_reference=report_reference,
|
||||
work_order_reference=work_order_reference,
|
||||
work_order_action=work_order_action,
|
||||
)
|
||||
await self.repo.insert_notification(_json(notification))
|
||||
await self._append_event(
|
||||
@@ -2344,10 +2598,7 @@ class OperationalTaskService:
|
||||
now: datetime,
|
||||
reason: str,
|
||||
) -> None:
|
||||
resolve = getattr(self.repo, "resolve_notifications", None)
|
||||
if resolve is None:
|
||||
return
|
||||
notifications = await resolve(
|
||||
notifications = await self.repo.resolve_notifications(
|
||||
str(task_id),
|
||||
kinds=kinds,
|
||||
subject_type=subject_type,
|
||||
@@ -2432,6 +2683,7 @@ class OperationalTaskService:
|
||||
due_at: datetime, now: datetime,
|
||||
) -> OperationalTaskSummary:
|
||||
return OperationalTaskSummary(
|
||||
execution_scope=self.execution_scope,
|
||||
project_id=self.context.project_id, task_id=task_id,
|
||||
task_type="scheduled_flow", workflow=schedule.schedule_key,
|
||||
lifecycle_status="open", status="queued", priority=schedule.priority, revision=1,
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Fit causal replay thresholds in bounded numeric arrays, before publishing history."""
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.services.scada_normal_profile import CALIBRATION_VERSION, DIAGNOSTIC_POLICY
|
||||
|
||||
|
||||
def adaptive_score_threshold(scores: list[float]) -> dict:
|
||||
"""Keep the cutoff below the normal lower tail, with a fixed noise margin."""
|
||||
values = np.asarray(scores, dtype=float)
|
||||
if len(values) < 5 or not np.isfinite(values).all():
|
||||
return {"status": "insufficient_data", "reason": "normal_score_history_incomplete"}
|
||||
center = float(np.median(values))
|
||||
sigma = float(1.4826 * np.median(np.abs(values - center)))
|
||||
lower_tail = float(np.quantile(values, 0.05))
|
||||
return {
|
||||
"status": "ready",
|
||||
"threshold": min(-0.04, center - 3 * sigma, lower_tail - 0.02),
|
||||
"normal_score_median": center,
|
||||
"normal_score_min": float(values.min()),
|
||||
"normal_score_max": float(values.max()),
|
||||
"normal_score_robust_sigma": sigma,
|
||||
"normal_score_lower_quantile": lower_tail,
|
||||
"normal_score_count": len(values),
|
||||
"method": "min(-0.04, median-3*robust_sigma, normal_q05-0.02)",
|
||||
}
|
||||
|
||||
|
||||
class MonitoringCalibrationBuilder:
|
||||
"""Retain bounded numeric series, never full raw pages or device batches."""
|
||||
|
||||
def __init__(self, document: dict, config: ScadaReplayCreate):
|
||||
self.settings = ScadaMonitoringSettings.model_validate(document["monitoring_config"])
|
||||
self.config = config
|
||||
self.cutoff = config.start_time
|
||||
self.start = self.cutoff - timedelta(days=15)
|
||||
self.interval = int(document["monitoring_interval_seconds"])
|
||||
self.supported = self.interval % 60 == 0 and 86400 % self.interval == 0
|
||||
self.count = 15 * (86400 // self.interval) if self.supported else 0
|
||||
ids = dict.fromkeys(self.settings.pressure_device_ids + self.settings.outlet_flow_device_ids
|
||||
+ [device for rule in config.rules if rule.kind == "relative_change"
|
||||
for device in rule.device_ids])
|
||||
self.values = {device: np.full(self.count, np.nan) for device in ids}
|
||||
|
||||
def add(self, rows: list[dict]) -> None:
|
||||
for row in rows:
|
||||
device = row["device_id"]
|
||||
if device not in self.values:
|
||||
continue
|
||||
at = datetime.fromisoformat(row["time"])
|
||||
if at >= self.cutoff:
|
||||
raise ValueError("动态阈值拟合不能读取回放开始时刻及之后的观测")
|
||||
offset = (at - self.start).total_seconds()
|
||||
if (not self.supported or offset < 0 or offset % self.interval
|
||||
or row.get("quality") not in {"valid", "corrected"}
|
||||
or row.get("cleaned_value") is None):
|
||||
continue
|
||||
index = int(offset // self.interval)
|
||||
if np.isfinite(self.values[device][index]):
|
||||
raise ValueError("动态阈值拟合存在重复的设备时间点")
|
||||
self.values[device][index] = float(row["cleaned_value"])
|
||||
|
||||
def fit(self) -> dict:
|
||||
base = {
|
||||
"version": CALIBRATION_VERSION,
|
||||
"diagnostic_policy": DIAGNOSTIC_POLICY,
|
||||
"knowledge_cutoff": self.cutoff.isoformat(),
|
||||
"history_start": self.start.isoformat(),
|
||||
"required_samples_per_device": self.count,
|
||||
"pressure": {}, "outlet_flow": {}, "diagnostic": {},
|
||||
}
|
||||
if not self.supported:
|
||||
return {**base, "status": "insufficient_data", "reason": "unsupported_sampling_interval"}
|
||||
available = {device: int(np.isfinite(values).sum()) for device, values in self.values.items()}
|
||||
# A regular diurnal rise/reversal can be a huge percentage of a small
|
||||
# preceding flow. Judge its absolute value against causal normal readings
|
||||
# at the same sampling phase before treating that percentage as abnormal.
|
||||
points_per_day = 86400 // self.interval
|
||||
for device, values in self.values.items():
|
||||
if available[device] != self.count:
|
||||
continue
|
||||
days = values.reshape(15, points_per_day)
|
||||
center = np.median(days, axis=0)
|
||||
sigma = 1.4826 * np.median(np.abs(days - center), axis=0)
|
||||
margin = np.maximum(3 * sigma, np.maximum(np.abs(center) * 1e-6, 1e-9))
|
||||
base["diagnostic"][device] = {
|
||||
"phase_origin": self.start.isoformat(), "interval_seconds": self.interval,
|
||||
"lower": (days.min(axis=0) - margin).tolist(),
|
||||
"upper": (days.max(axis=0) + margin).tolist(),
|
||||
"method": "normal_same_phase_range_plus_3_robust_sigma", "normal_day_count": 15,
|
||||
}
|
||||
for kind, ids, direction, floor in (
|
||||
("pressure", self.settings.pressure_device_ids, -1, 3.0),
|
||||
("outlet_flow", self.settings.outlet_flow_device_ids, 1, 5.0),
|
||||
):
|
||||
for device in ids:
|
||||
if available[device] != self.count:
|
||||
continue
|
||||
rule = next((rule for rule in self.config.rules
|
||||
if rule.kind == "relative_change" and device in rule.device_ids), None)
|
||||
if rule is None:
|
||||
continue
|
||||
series = pd.Series(self.values[device])
|
||||
reference = series.rolling(rule.window_samples, min_periods=rule.minimum_samples).median().shift(1)
|
||||
changes = (direction * 100 * (series - reference) / reference.abs()).to_numpy()
|
||||
changes = changes[np.isfinite(changes)]
|
||||
if not len(changes):
|
||||
continue
|
||||
center = float(np.median(changes))
|
||||
sigma = float(1.4826 * np.median(np.abs(changes - center)))
|
||||
base[kind][device] = {
|
||||
"threshold_percent": max(floor, center + 3 * sigma),
|
||||
"normal_change_median": center,
|
||||
"normal_change_robust_sigma": sigma,
|
||||
"normal_change_count": int(len(changes)),
|
||||
"method": "normal_recent_window_change_median_plus_3_robust_sigma",
|
||||
"normal_profile": base["diagnostic"].get(device),
|
||||
}
|
||||
pressure_ids = self.settings.pressure_device_ids
|
||||
incomplete = {device: available[device] for device in pressure_ids if available[device] != self.count}
|
||||
if len(pressure_ids) < 5 or incomplete:
|
||||
return {**base, "status": "insufficient_data", "reason": "pressure_history_incomplete",
|
||||
"available_samples": available}
|
||||
from app.services import scada_replay_monitoring as monitoring
|
||||
|
||||
index = pd.date_range(start=self.start, periods=self.count, freq=pd.Timedelta(seconds=self.interval))
|
||||
frame = pd.DataFrame({device: self.values[device] for device in pressure_ids}, index=index)
|
||||
result = monitoring.run_burst_detection(
|
||||
network="scada-replay-baseline", username="scada-replay",
|
||||
observed_pressure_data=frame, points_per_day=86400 // self.interval,
|
||||
mu=monitoring.TARGET_MU,
|
||||
iforest_params={"n_estimators": monitoring.TARGET_N_ESTIMATORS,
|
||||
"random_state": monitoring.TARGET_RANDOM_STATE, "contamination": "auto"},
|
||||
sensor_nodes=pressure_ids,
|
||||
)
|
||||
score = adaptive_score_threshold([float(row["Score"]) for row in result["rows"]])
|
||||
return {**base, "status": score["status"], "burst_detection": score}
|
||||
@@ -0,0 +1,98 @@
|
||||
"""Persist one accident across monitoring periods without moving its evidence cutoff."""
|
||||
from datetime import datetime
|
||||
from math import isfinite
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
|
||||
def _anchors(monitoring: dict, indicator: str = "pressure") -> dict:
|
||||
return {
|
||||
item["device_id"]: {"baseline": item["baseline"], "threshold_percent": item["threshold_percent"]}
|
||||
for item in monitoring.get(indicator, {}).get("devices", [])
|
||||
if item.get("hit") and item.get("baseline", 0) > 0
|
||||
and 0 < item.get("threshold_percent", 0) < 100
|
||||
}
|
||||
|
||||
|
||||
def update_incident(document: dict, monitoring: dict, sequence: int) -> tuple[dict | None, str | None]:
|
||||
incident = document.get("monitoring_incident")
|
||||
# Adopt the last task from an older replay document on its first new step.
|
||||
previous = document.get("last_monitoring") or {}
|
||||
if incident:
|
||||
incident.setdefault("burst_diagnosed", previous.get("status") == "suspected_burst")
|
||||
if not incident and previous.get("task_id"):
|
||||
incident = {
|
||||
"incident_id": str(uuid5(NAMESPACE_URL, f"scada_replay:{document['replay_id']}:monitor:"
|
||||
f"{previous['period_end']}:{previous['rule_version']}")),
|
||||
"started_at": previous["period_end"], "last_observed_at": previous["period_end"],
|
||||
"status": "active", "task_id": previous["task_id"], "recovery_count": 0,
|
||||
"pressure_anchors": _anchors(previous), "alert": None,
|
||||
"flow_anchors": _anchors(previous, "outlet_flow"),
|
||||
"burst_diagnosed": previous.get("status") == "suspected_burst",
|
||||
}
|
||||
document["monitoring_incident"] = incident
|
||||
if incident and incident.get("last_monitoring_id") == monitoring.get("monitoring_id") and monitoring.get("monitoring_id"):
|
||||
return incident, incident["last_action"]
|
||||
anomaly = (monitoring["status"] in {"attention", "suspected_burst"}
|
||||
or bool(monitoring.get("anomaly_signals"))
|
||||
or any(monitoring.get(key, {}).get("hit") is True
|
||||
for key in ("pressure", "outlet_flow", "burst_detection")))
|
||||
if anomaly and (not incident or incident["status"] == "resolved"):
|
||||
external = (f"scada_replay:{document['replay_id']}:monitor:"
|
||||
f"{monitoring['period_end']}:{monitoring['rule_version']}")
|
||||
incident = {
|
||||
"incident_id": str(uuid5(NAMESPACE_URL, external)), "external_id": external,
|
||||
"started_at": monitoring["period_end"], "status": "active", "task_id": None,
|
||||
"recovery_count": 0, "pressure_anchors": _anchors(monitoring), "alert": None,
|
||||
"flow_anchors": _anchors(monitoring, "outlet_flow"),
|
||||
"batch_sequence": sequence,
|
||||
}
|
||||
document["monitoring_incident"] = incident
|
||||
action = "raised"
|
||||
elif incident and incident["status"] == "active":
|
||||
# A later diagnostic stage may add pressure/flow evidence to the same task.
|
||||
# Freeze each newly abnormal device once; retain earlier references.
|
||||
for indicator, field in (("pressure", "pressure_anchors"), ("outlet_flow", "flow_anchors")):
|
||||
for device, anchor in _anchors(monitoring, indicator).items():
|
||||
incident.setdefault(field, {}).setdefault(device, anchor)
|
||||
current = {item["device_id"]: item for item in monitoring.get("pressure", {}).get("devices", [])}
|
||||
anchors = incident["pressure_anchors"]
|
||||
flow = {item["device_id"]: item for item in monitoring.get("outlet_flow", {}).get("devices", [])}
|
||||
# Rolling relative changes can become normal while absolute pressure stays low.
|
||||
# Confirm recovery against the frozen pre-accident references as well.
|
||||
recovered = monitoring["status"] == "normal" and not anomaly and all(
|
||||
monitoring.get(key, {}).get("status") == "ready"
|
||||
and monitoring[key].get("hit") is False for key in ("pressure", "burst_detection")
|
||||
) and all(
|
||||
device in current and isfinite(current[device]["value"])
|
||||
and current[device].get("observed_at") is not None
|
||||
and datetime.fromisoformat(current[device]["observed_at"]) > datetime.fromisoformat(incident["last_observed_at"])
|
||||
and current[device]["value"] >= anchor["baseline"] * (1 - anchor["threshold_percent"] / 100)
|
||||
for device, anchor in anchors.items()
|
||||
) and all(
|
||||
device in flow and isfinite(flow[device]["value"])
|
||||
and flow[device].get("observed_at") is not None
|
||||
and datetime.fromisoformat(flow[device]["observed_at"]) > datetime.fromisoformat(incident["last_observed_at"])
|
||||
and flow[device]["value"] <= anchor["baseline"] * (1 + anchor["threshold_percent"] / 100)
|
||||
for device, anchor in incident.get("flow_anchors", {}).items()
|
||||
)
|
||||
contiguous = bool(monitoring.get("period_start")) and (
|
||||
datetime.fromisoformat(monitoring["period_start"]) == datetime.fromisoformat(incident["last_observed_at"])
|
||||
)
|
||||
incident["recovery_count"] = (incident["recovery_count"] + 1 if contiguous else 1) if recovered else 0
|
||||
action = "resolved" if incident["recovery_count"] >= 2 else "ongoing"
|
||||
if action == "resolved":
|
||||
incident["status"] = "resolved"
|
||||
else:
|
||||
return None, None
|
||||
incident.update(last_observed_at=monitoring["period_end"],
|
||||
last_monitoring_id=monitoring.get("monitoring_id"), last_action=action)
|
||||
return incident, action
|
||||
|
||||
|
||||
def attach_incident(monitoring: dict, incident: dict, action: str) -> None:
|
||||
monitoring["task_id"] = incident["task_id"]
|
||||
monitoring["incident"] = {
|
||||
key: incident[key] for key in (
|
||||
"incident_id", "status", "started_at", "last_observed_at", "recovery_count",
|
||||
)
|
||||
} | {"action": action}
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Clock-phase normal envelopes frozen before replay; never inspect future readings."""
|
||||
from datetime import datetime
|
||||
from math import isfinite
|
||||
|
||||
CALIBRATION_VERSION = "scada-monitor-calibration@2"
|
||||
DIAGNOSTIC_POLICY = "normal-phase-envelope@1"
|
||||
|
||||
|
||||
def weighted_diagnosis(document: dict) -> bool:
|
||||
return (document.get("monitoring_config") or {}).get("decision_policy") == "weighted"
|
||||
|
||||
|
||||
def current_calibration(document: dict, start: datetime) -> dict | None:
|
||||
calibration = document.get("monitoring_calibration") or {}
|
||||
if (calibration.get("version") == CALIBRATION_VERSION
|
||||
and calibration.get("diagnostic_policy") == DIAGNOSTIC_POLICY
|
||||
and calibration.get("knowledge_cutoff") == start.isoformat()):
|
||||
return calibration
|
||||
return None
|
||||
|
||||
|
||||
def pollution_signal(document: dict, event: dict) -> bool:
|
||||
"""Water-quality measurements have their own incident investigation workflow."""
|
||||
if event.get("category") == "data_quality":
|
||||
return False
|
||||
devices = {item["device_id"]: item.get("device_type") for item in document.get("devices", [])}
|
||||
ids = event.get("device_ids") or next((item.get("device_ids", [])
|
||||
for item in (document.get("config") or {}).get("rules", [])
|
||||
if item["rule_id"] == event.get("rule_id")), [])
|
||||
return bool(ids) and all(devices.get(device) in {"quality", "water_quality"} for device in ids)
|
||||
|
||||
|
||||
def diagnostic_signal(document: dict, event: dict) -> bool:
|
||||
"""Never revive a persisted fixed-percentage event as new diagnosis evidence."""
|
||||
if event.get("category") == "data_quality" or pollution_signal(document, event):
|
||||
return False
|
||||
if not weighted_diagnosis(document):
|
||||
return True
|
||||
rules = (document.get("config") or {}).get("rules", [])
|
||||
rule = next((item for item in rules if item["rule_id"] == event["rule_id"]), None)
|
||||
return bool(rule and (rule["kind"] == "bounds" or event.get("diagnostic_policy") == DIAGNOSTIC_POLICY))
|
||||
|
||||
|
||||
def normal_profile_bounds(profile: dict | None, at: datetime) -> tuple[float, float] | None:
|
||||
if not profile:
|
||||
return None
|
||||
interval = profile.get("interval_seconds", 0)
|
||||
if not profile.get("phase_origin"):
|
||||
return None
|
||||
origin = datetime.fromisoformat(profile["phase_origin"])
|
||||
lower, upper = profile.get("lower", []), profile.get("upper", [])
|
||||
offset = (at - origin).total_seconds()
|
||||
if interval <= 0 or offset < 0 or offset % interval or not lower or len(lower) != len(upper):
|
||||
return None
|
||||
slot = int(offset // interval) % len(lower)
|
||||
low, high = lower[slot], upper[slot]
|
||||
if low is None or high is None or not isfinite(low) or not isfinite(high):
|
||||
return None
|
||||
return float(low), float(high)
|
||||
@@ -0,0 +1,268 @@
|
||||
"""Verify monitoring charts against observations and require run-local device coverage."""
|
||||
from __future__ import annotations
|
||||
|
||||
from copy import deepcopy
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from math import isclose
|
||||
import re
|
||||
|
||||
from app.domain.domain_errors import require
|
||||
|
||||
|
||||
DISPLAY_TIMEZONE = timezone(timedelta(hours=8))
|
||||
METRICS = {"pressure": "pressure", "pipe_flow": "flow", "flow": "flow", "quality": "quality", "water_quality": "quality"}
|
||||
|
||||
|
||||
def applies(task: dict) -> bool:
|
||||
if task.get("workflow") not in {"scada_anomaly", "water_pollution"} and not task.get("source", {}).get("replay_context"):
|
||||
return False
|
||||
# Recovery has its own Server-verified post-repair evidence and finalization gate.
|
||||
return (task.get("handling_context") or {}).get("phase") not in {"verification_ready", "field_review", "completed"}
|
||||
|
||||
|
||||
def cutoff(task: dict) -> datetime:
|
||||
replay = task.get("source", {}).get("replay_context") or {}
|
||||
return datetime.fromisoformat(replay.get("knowledge_cutoff") or task["trigger"]["occurred_at"])
|
||||
|
||||
|
||||
def _unit(value: str) -> str:
|
||||
return value.replace("³", "3").replace(" ", "").lower()
|
||||
|
||||
|
||||
class ScadaObservations:
|
||||
def __init__(self, task: dict, evidence: list[dict], refs: list[str] | None = None):
|
||||
self.devices: dict[str, dict] = {}
|
||||
self.values: dict[tuple[str, datetime], dict] = {}
|
||||
self.windows: list[tuple[set[str], set[str], datetime, datetime]] = []
|
||||
self.frozen_limits: list[tuple[set[str], datetime]] = []
|
||||
target = cutoff(task)
|
||||
replay = task.get("source", {}).get("replay_context") or {}
|
||||
for item in evidence:
|
||||
snapshot = item.get("data_snapshot") or {}
|
||||
result = snapshot.get("data") or {}
|
||||
if snapshot.get("source") != "query_scada_readings" or not isinstance(result, dict):
|
||||
continue
|
||||
data = result.get("data")
|
||||
coverage = data.get("coverage", {}) if isinstance(data, dict) else {}
|
||||
if replay and coverage and (coverage.get("replay_id") != replay["replay_id"]
|
||||
or coverage.get("batch_sequence") != replay["batch_sequence"]):
|
||||
continue
|
||||
window = result.get("window") or {}
|
||||
if not window.get("start") or not window.get("end"):
|
||||
continue
|
||||
start, end = (datetime.fromisoformat(window[key]) for key in ("start", "end"))
|
||||
if end > target:
|
||||
continue
|
||||
devices = data.get("devices", []) if isinstance(data, dict) else []
|
||||
for device in devices:
|
||||
self.devices[str(device["device_id"])] = device
|
||||
if refs is not None and item["evidence_id"] not in refs:
|
||||
continue
|
||||
rows = data.get("readings", []) if isinstance(data, dict) else data
|
||||
if not isinstance(rows, list):
|
||||
continue
|
||||
ids = {str(row["device_id"]) for row in rows if "device_id" in row}
|
||||
outside = set(coverage.get("outside_frozen_scope_device_ids", []))
|
||||
if coverage.get("query_complete") is True:
|
||||
ids.update(coverage.get("queried_device_ids", []))
|
||||
ids.update(str(device["device_id"]) for device in devices)
|
||||
page = result.get("page") or {}
|
||||
complete = not (snapshot.get("truncated") or result.get("truncated")
|
||||
or page.get("has_more") or page.get("offset", 0))
|
||||
if complete:
|
||||
self.windows.append((ids, outside, start, end))
|
||||
if replay and coverage.get("query_complete") and coverage.get("frozen_start"):
|
||||
self.frozen_limits.append((ids, datetime.fromisoformat(coverage["frozen_start"])))
|
||||
for row in rows:
|
||||
if "time" in row and "device_id" in row:
|
||||
at = datetime.fromisoformat(row["time"])
|
||||
if start <= at <= end:
|
||||
observed = self.values.setdefault((str(row["device_id"]), at), {})
|
||||
observed.update({field: row[field] for field in ("monitored_value", "cleaned_value") if field in row})
|
||||
|
||||
def absent_reason(self, device: str, at: datetime) -> str | None:
|
||||
for ids, outside, start, end in self.windows:
|
||||
if device in ids and start <= at <= end:
|
||||
return "outside_frozen_scope" if device in outside else "no_readings"
|
||||
for ids, frozen_start in self.frozen_limits:
|
||||
if device in ids and at < frozen_start:
|
||||
return "outside_frozen_window"
|
||||
return None
|
||||
|
||||
def window_is_empty(self, device: str, start: datetime, end: datetime) -> bool:
|
||||
if any(key == device and start <= at <= end for key, at in self.values):
|
||||
return False
|
||||
intervals = [(left, right) for ids, _, left, right in self.windows if device in ids]
|
||||
intervals.extend((start, frozen_start) for ids, frozen_start in self.frozen_limits if device in ids)
|
||||
through = start
|
||||
for left, right in sorted(intervals):
|
||||
if left > through:
|
||||
break
|
||||
through = max(through, right)
|
||||
if through >= end:
|
||||
return True
|
||||
return False
|
||||
|
||||
def period(self, device: str, start: datetime, end: datetime) -> timedelta:
|
||||
frequency = str(self.devices[device].get("transmission_frequency") or "")
|
||||
match = re.fullmatch(r"(\d+):([0-5]\d):([0-5]\d)", frequency)
|
||||
if match:
|
||||
hours, minutes, seconds = map(int, match.groups())
|
||||
period = timedelta(hours=hours, minutes=minutes, seconds=seconds)
|
||||
else:
|
||||
minutes = re.fullmatch(r"(\d+)\s*(?:min|分钟)", frequency)
|
||||
times = sorted(at for key, at in self.values if key == device and start <= at <= end)
|
||||
period = (timedelta(minutes=int(minutes[1])) if minutes else
|
||||
min((right - left for left, right in zip(times, times[1:])), default=timedelta(0)))
|
||||
require(period > timedelta(0), "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 缺少可核实的采样周期,请读取完整设备属性", refs=[device])
|
||||
return period
|
||||
|
||||
def slots(self, device: str, start: datetime, end: datetime) -> list[datetime]:
|
||||
"""Keep the business bounds exact, aligning slots to the observed sampling phase."""
|
||||
period = self.period(device, start, end)
|
||||
observed = sorted(at for key, at in self.values if key == device and start <= at <= end)
|
||||
historical = sorted(at for key, at in self.values if key == device and at < start)
|
||||
# Referenced historical observations preserve phase even if current data is absent.
|
||||
anchor = (observed[0] if observed else historical[-1] if historical else
|
||||
start.astimezone(DISPLAY_TIMEZONE).replace(hour=0, minute=0, second=0, microsecond=0))
|
||||
first = start + (anchor - start) % period
|
||||
return [first + period * index for index in range(int((end - first) // period) + 1)]
|
||||
|
||||
|
||||
def add_targets(document: dict | None, devices: list[str], observations: ScadaObservations) -> dict:
|
||||
result = deepcopy(document) if document else {"version": 1, "targets": [], "charts": {}}
|
||||
targets = set(result["targets"])
|
||||
for device in devices:
|
||||
require(device in observations.devices, "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 未出现在本轮截止时间内的完整设备证据中", refs=[device])
|
||||
if observations.devices[device].get("device_type") in METRICS:
|
||||
targets.add(device)
|
||||
result["targets"] = sorted(targets)
|
||||
return result
|
||||
|
||||
|
||||
def verify_chart(task: dict, chart: dict, evidence: list[dict], refs: list[str]) -> dict:
|
||||
"""Return verified coverage; mutate only the outgoing chart's Server-derived gaps."""
|
||||
context = chart["scada"]
|
||||
end = cutoff(task)
|
||||
start = end - timedelta(hours=3)
|
||||
require(datetime.fromisoformat(context["window_start"]) == start
|
||||
and datetime.fromisoformat(context["window_end"]) == end,
|
||||
"SCADA_CHART_WINDOW_INVALID", "SCADA图表必须使用本轮截止时刻之前的完整三小时窗口")
|
||||
observations = ScadaObservations(task, evidence, refs)
|
||||
series = chart["series"]
|
||||
ids = {item.get("device_id") for item in series}
|
||||
require(None not in ids and all(device in observations.devices for device in ids),
|
||||
"SCADA_CHART_EVIDENCE_REQUIRED", "每条SCADA序列须绑定已核实的 device_id", refs=sorted(ids - {None}))
|
||||
comparison = context["role"] == "comparison"
|
||||
require(chart["chart_type"] == ("time_comparison" if comparison else "line"),
|
||||
"SCADA_CHART_INVALID", "SCADA总览须使用line,同期图须使用time_comparison")
|
||||
bindings = [(item["device_id"], item.get("day_offset")) for item in series]
|
||||
require(len(bindings) == len(set(bindings)), "SCADA_CHART_INVALID", "SCADA序列的设备与日期绑定不能重复")
|
||||
require((len(ids) == 1 and {day for _, day in bindings} == set(range(8))) if comparison
|
||||
else all(day == 0 for _, day in bindings),
|
||||
"SCADA_CHART_INVALID", "总览每个设备绑定day_offset=0;同期图每个设备须有当天及此前七天共8组序列")
|
||||
field = context.get("value_field", "monitored_value")
|
||||
gaps = []
|
||||
coverage = {}
|
||||
for item in series:
|
||||
device, day = item["device_id"], item["day_offset"]
|
||||
metadata = observations.devices[device]
|
||||
require(METRICS.get(metadata.get("device_type")) == context["metric"]
|
||||
and _unit(metadata.get("measurement_unit") or "") == _unit(context["unit"]),
|
||||
"SCADA_CHART_INVALID", f"设备 {device} 的指标或单位与完整设备证据不一致", refs=[device])
|
||||
period = observations.period(device, start, end)
|
||||
full_slots = observations.slots(device, start, end)
|
||||
no_schedule = not full_slots
|
||||
if no_schedule:
|
||||
require(observations.window_is_empty(device, start - timedelta(days=day), end - timedelta(days=day)),
|
||||
"SCADA_CHART_EVIDENCE_REQUIRED", f"设备 {device} 无计划采样槽,须完整读取对应三小时窗口并证明无读数", refs=[device])
|
||||
# This is an empty display entry, never an observation or a missed scheduled sample.
|
||||
full_slots = [end]
|
||||
labels = [at.astimezone(DISPLAY_TIMEZONE).strftime("%H:%M:%S" if period.total_seconds() % 60 else "%H:%M")
|
||||
for at in full_slots]
|
||||
first = labels.index(chart["x_data"][0]) if chart["x_data"][0] in labels else -1
|
||||
count = len(chart["x_data"])
|
||||
require(first >= 0 and chart["x_data"] == labels[first:first + count]
|
||||
and len(item.get("sample_times") or []) == count,
|
||||
"SCADA_CHART_INVALID", f"设备 {device} 须保留连续真实采样槽和sample_times;超限按时间分图", refs=[device])
|
||||
slots = full_slots[first:first + count]
|
||||
coverage[device] = {"covered": [at.isoformat() for at in slots]}
|
||||
require((item.get("type") or "line") == ("scatter" if day else "line"),
|
||||
"SCADA_CHART_INVALID", "当前序列须为line,历史序列须为scatter")
|
||||
missing = 0
|
||||
reason = "no_readings"
|
||||
expected = {at - timedelta(days=day) for at in (full_slots[-3:] if day else full_slots)}
|
||||
# Off-grid observations cannot be silently discarded by a regular-looking chart.
|
||||
observed = {at for key, at in observations.values if key == device
|
||||
and min(expected) <= at <= max(expected)}
|
||||
require(observed <= expected, "SCADA_CHART_INVALID",
|
||||
f"设备 {device} 含非规则采样时刻,须明确处理,不能静默丢点", refs=[device])
|
||||
for index, slot in enumerate(slots):
|
||||
at = slot - timedelta(days=day)
|
||||
value, sample_time = item["data"][index], item["sample_times"][index]
|
||||
if at not in expected:
|
||||
require(value is None and sample_time is None, "SCADA_CHART_INVALID",
|
||||
"历史散点只保留每日末三个对应时刻")
|
||||
continue
|
||||
record = observations.values.get((device, at))
|
||||
require(record is None or field in record, "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 的证据未标明 {field},请重新读取明确的监测字段", refs=[device])
|
||||
actual = record.get(field) if record is not None else None
|
||||
if actual is None:
|
||||
absent = observations.absent_reason(device, at)
|
||||
require(record is not None or absent is not None, "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 的 {at.isoformat()} 未被完整读取,不能将未查询当作缺测", refs=[device])
|
||||
require(value is None and sample_time is None, "SCADA_CHART_DATA_MISMATCH",
|
||||
f"设备 {device} 缺测槽必须保留null,不能补零或推算", refs=[device])
|
||||
missing += 1
|
||||
if absent in {"outside_frozen_scope", "outside_frozen_window"}:
|
||||
reason = absent
|
||||
elif no_schedule:
|
||||
reason = "no_scheduled_samples"
|
||||
else:
|
||||
require(value is not None and sample_time is not None
|
||||
and datetime.fromisoformat(sample_time) == at
|
||||
and (isclose(value, actual, rel_tol=1e-9, abs_tol=1e-9)
|
||||
or any(isclose(value, round(actual, digits), rel_tol=0, abs_tol=1e-9)
|
||||
for digits in range(1, 7))),
|
||||
"SCADA_CHART_DATA_MISMATCH",
|
||||
f"设备 {device} 的 {at.isoformat()} 图表值或采样时间与证据不符", refs=[device])
|
||||
if missing:
|
||||
gaps.append({"device_id": device, "day_offset": day, "missing_count": missing, "reason": reason})
|
||||
context["gaps"] = gaps
|
||||
return {"role": context["role"], "metric": context["metric"], "device_ids": sorted(ids),
|
||||
"window_start": context["window_start"], "window_end": context["window_end"], "gaps": gaps,
|
||||
"coverage": coverage}
|
||||
|
||||
|
||||
def require_complete(task: dict, document: dict | None, observations: ScadaObservations) -> None:
|
||||
targets = set((document or {}).get("targets", []))
|
||||
supported = {key for key, device in observations.devices.items() if device.get("device_type") in METRICS}
|
||||
if not supported and not targets:
|
||||
return
|
||||
require(bool(targets), "SCADA_PRESENTATION_INCOMPLETE",
|
||||
"请声明本轮SCADA目标设备并发布当前总览与七天同期图;旧格式图表不计入设备覆盖", status=409)
|
||||
end = cutoff(task)
|
||||
start = end - timedelta(hours=3)
|
||||
covered = {"current": {}, "comparison": {}}
|
||||
for chart in document["charts"].values():
|
||||
if (datetime.fromisoformat(chart["window_start"]) == start
|
||||
and datetime.fromisoformat(chart["window_end"]) == end):
|
||||
for device, slots in chart.get("coverage", {}).items():
|
||||
if device not in chart["device_ids"]:
|
||||
continue
|
||||
covered[chart["role"]].setdefault(device, set()).update(
|
||||
datetime.fromisoformat(at) for at in slots["covered"])
|
||||
missing = {}
|
||||
for role in covered:
|
||||
missing[role] = sorted(device for device in targets
|
||||
if not covered[role].get(device) or not set(observations.slots(device, start, end) or [end])
|
||||
<= covered[role][device])
|
||||
missing_current, missing_comparison = missing["current"], missing["comparison"]
|
||||
require(not missing_current and not missing_comparison, "SCADA_PRESENTATION_INCOMPLETE",
|
||||
f"本轮SCADA目标共{len(targets)}个;缺少当前总览设备:{', '.join(missing_current) or '无'};"
|
||||
f"缺少七天同期对比设备:{', '.join(missing_comparison) or '无'}。请补图后重试;仅作参考不能豁免展示。",
|
||||
refs=sorted(set(missing_current + missing_comparison)), status=409,
|
||||
recovery="retry_same_request")
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Read-only chronological SCADA paging shared by REST and replay processing."""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from datetime import datetime
|
||||
|
||||
from app.domain.schemas.scada import ScadaRawPage, ScadaRawReading
|
||||
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
|
||||
|
||||
def validate_window(start: datetime, end: datetime) -> None:
|
||||
if start.tzinfo is None or end.tzinfo is None or start >= end:
|
||||
raise ValueError("需要带时区、结束晚于开始的 SCADA 时间区间")
|
||||
|
||||
|
||||
def raw_reading(row: dict) -> ScadaRawReading:
|
||||
value = row["monitored_value"]
|
||||
invalid = value is not None and not math.isfinite(float(value))
|
||||
return ScadaRawReading(
|
||||
time=row["time"], device_id=row["device_id"],
|
||||
monitored_value=None if invalid else value,
|
||||
quality="invalid" if invalid else "missing" if value is None else "unvalidated",
|
||||
)
|
||||
|
||||
|
||||
async def read_raw_page(
|
||||
conn, *, project_id: str, device_ids: list[str], start: datetime,
|
||||
end: datetime, limit: int = 500, cursor: str | None = None,
|
||||
through: datetime | None = None,
|
||||
) -> ScadaRawPage:
|
||||
validate_window(start, end)
|
||||
ids = sorted(set(device_ids))
|
||||
if not ids or len(ids) > 500 or not 1 <= limit <= 500:
|
||||
raise ValueError("请选择 1 至 500 个设备,每页最多 500 条")
|
||||
cap = min(through or end, end)
|
||||
if cap.tzinfo is None or cap < start:
|
||||
raise ValueError("模拟时间不在可读取区间内")
|
||||
signature = hashlib.sha256(json.dumps(
|
||||
[project_id, ids, start.isoformat(), end.isoformat()], separators=(",", ":")
|
||||
).encode()).hexdigest()
|
||||
after = None
|
||||
if cursor:
|
||||
try:
|
||||
data = json.loads(base64.urlsafe_b64decode(cursor.encode()))
|
||||
frozen = datetime.fromisoformat(data["through"])
|
||||
last = datetime.fromisoformat(data["time"])
|
||||
if (data["query"] != signature or frozen.tzinfo is None or last.tzinfo is None
|
||||
or frozen > cap or not start <= last <= frozen or data["device"] not in ids):
|
||||
raise ValueError()
|
||||
cap, after = frozen, (last, data["device"])
|
||||
except (ValueError, KeyError, TypeError, UnicodeError) as exc:
|
||||
raise ValueError("SCADA 分页游标无效或查询范围已改变") from exc
|
||||
rows = await ScadaRepository.get_raw_page(conn, ids, start, end, cap, limit + 1, after)
|
||||
selected = rows[:limit]
|
||||
next_cursor = None
|
||||
if len(rows) > limit:
|
||||
last = selected[-1]
|
||||
next_cursor = base64.urlsafe_b64encode(json.dumps({
|
||||
"query": signature, "through": cap.isoformat(),
|
||||
"time": last["time"].isoformat(), "device": last["device_id"],
|
||||
}, separators=(",", ":")).encode()).decode()
|
||||
return ScadaRawPage(items=[raw_reading(row) for row in selected],
|
||||
next_cursor=next_cursor, scanned_through=cap)
|
||||
@@ -0,0 +1,597 @@
|
||||
"""Durable, project-scoped SCADA playback; raw is never inserted or rewritten."""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import json
|
||||
from uuid import NAMESPACE_URL, uuid4, uuid5
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from fastapi import HTTPException
|
||||
from app.core.config import settings
|
||||
from app.auth.permissions import permissions_for_context
|
||||
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate, ScadaReplayCommand, ScadaReplaySnapshot
|
||||
from app.infra.db.postgresql.scada import ScadaInfoRepository
|
||||
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
from app.services.content_digest import digest
|
||||
from app.services.scada_replay_processing import prepare_cleaning, detect_frame
|
||||
from app.services.scada_replay_monitoring import evaluate_monitoring_period
|
||||
from app.services.scada_normal_profile import current_calibration, diagnostic_signal, weighted_diagnosis
|
||||
|
||||
|
||||
# 500 devices at 15-minute sampling need 720,000 rows for fifteen days.
|
||||
# Leave headroom without allowing dense sources to make preparation unbounded.
|
||||
BASELINE_PAGE_SIZE = 2000
|
||||
BASELINE_MAX_ROWS = 1_000_000
|
||||
BASELINE_MAX_BYTES = 512 * 1024 * 1024
|
||||
|
||||
|
||||
class BaselineBudgetExceeded(RuntimeError):
|
||||
"""The unpublished baseline exceeded its fixed preparation budget."""
|
||||
|
||||
|
||||
def parse_time(value: str) -> datetime:
|
||||
return datetime.fromisoformat(value)
|
||||
|
||||
|
||||
def snapshot(document: dict) -> ScadaReplaySnapshot:
|
||||
return ScadaReplaySnapshot(
|
||||
**{key: document[key] for key in (
|
||||
"replay_id", "project_id", "revision", "status", "start_time", "end_time",
|
||||
"virtual_now", "created_at", "updated_at",
|
||||
)},
|
||||
delivered_until=document.get("delivered_until"), processed_until=document.get("processed_until"),
|
||||
processed_count=document.get("processed_count", 0), anomaly_count=document.get("anomaly_count", 0),
|
||||
devices=document["devices"], rules=document["config"]["rules"],
|
||||
latest=list(document.get("latest", {}).values()),
|
||||
active_events=[state["event"] for state in document.get("rule_states", {}).values() if state.get("event")],
|
||||
error=document.get("error"),
|
||||
monitoring_interval_seconds=document.get("monitoring_interval_seconds"),
|
||||
step_pending=document.get("step_pending", False),
|
||||
monitoring_count=document.get("monitoring_count", 0),
|
||||
last_monitoring=document.get("last_monitoring"),
|
||||
# Old persisted documents without a policy still use three-signal detection.
|
||||
monitoring_config=({"decision_policy": "all_signals", **document["monitoring_config"]}
|
||||
if document.get("monitoring_config") else None),
|
||||
monitoring_calibration=document.get("monitoring_calibration"),
|
||||
)
|
||||
|
||||
|
||||
class ScadaReplayService:
|
||||
def __init__(self, repository, context, timescale=None, *, clock=None):
|
||||
self.repo, self.context, self.timescale = repository, context, timescale
|
||||
self.clock = clock or (lambda: datetime.now(timezone.utc))
|
||||
|
||||
async def ensure_storage(self):
|
||||
if not await self.repo.available():
|
||||
raise HTTPException(503, "SCADA 回放存储未配置,需要应用 031_scada_replays.sql")
|
||||
|
||||
async def create(
|
||||
self,
|
||||
command: ScadaReplayCreate,
|
||||
*,
|
||||
monitoring_interval_seconds: int | None,
|
||||
monitoring_config: ScadaMonitoringSettings | None = None,
|
||||
):
|
||||
await self.ensure_storage()
|
||||
async with self.repo.control():
|
||||
previous = await self.repo.by_request(command.request_id)
|
||||
config = command.model_dump(mode="json")
|
||||
if previous:
|
||||
if previous["config"] != config:
|
||||
raise HTTPException(409, "创建请求号已用于不同回放配置")
|
||||
return snapshot(previous)
|
||||
if monitoring_interval_seconds is None:
|
||||
raise HTTPException(422, "请先配置项目 settings.scada_monitoring_interval_seconds,再创建演示")
|
||||
replay_id = str(uuid4())
|
||||
if await self.repo.active_other(replay_id):
|
||||
raise HTTPException(409, "当前项目已有回放,请先停止原回放再新建")
|
||||
metadata = {row["device_id"]: row for row in await ScadaInfoRepository.get_scadas(self.repo.connection)}
|
||||
if not set(command.device_ids) <= metadata.keys():
|
||||
raise HTTPException(422, "设备不属于当前项目")
|
||||
for rule in command.rules:
|
||||
if any(metadata[device]["measurement_unit"] != rule.unit for device in rule.device_ids):
|
||||
raise HTTPException(422, "规则单位与设备单位不一致,请分组配置")
|
||||
if any(metadata[device]["device_type"] == "quality" for device in rule.device_ids) and not rule.metric_name.strip():
|
||||
raise HTTPException(422, "水质规则必须明确指标名称,不能仅凭单位解释读数")
|
||||
if monitoring_config:
|
||||
role_ids = set(monitoring_config.pressure_device_ids) | set(monitoring_config.outlet_flow_device_ids)
|
||||
if not role_ids <= metadata.keys():
|
||||
raise HTTPException(422, "项目 SCADA 监测角色包含不存在的设备")
|
||||
if any(metadata[device]["device_type"] != "pressure" for device in monitoring_config.pressure_device_ids):
|
||||
raise HTTPException(422, "压力监测角色必须引用 pressure 设备")
|
||||
if any(metadata[device]["device_type"] not in {"pipe_flow", "flow"}
|
||||
for device in monitoring_config.outlet_flow_device_ids):
|
||||
raise HTTPException(422, "出厂边界流量角色必须引用流量设备")
|
||||
now = self.clock().isoformat()
|
||||
document = {
|
||||
"replay_id": replay_id, "project_id": str(self.context.project_id),
|
||||
"created_by": str(self.context.user_id), "config": config,
|
||||
"revision": 1, "status": "ready", "start_time": config["start_time"],
|
||||
"end_time": config["end_time"], "virtual_now": config["start_time"],
|
||||
"created_at": now, "updated_at": now,
|
||||
"monitoring_interval_seconds": monitoring_interval_seconds,
|
||||
"monitoring_config": monitoring_config.model_dump(mode="json") if monitoring_config else None,
|
||||
"devices": [metadata[device] for device in command.device_ids],
|
||||
"batch_sequence": 0, "processed_count": 0, "anomaly_count": 0,
|
||||
"monitoring_count": 0, "latest": {}, "histories": {}, "rule_states": {}, "receipts": {},
|
||||
}
|
||||
await self.repo.insert(document)
|
||||
return snapshot(document)
|
||||
|
||||
async def command(self, replay_id, command: ScadaReplayCommand):
|
||||
await self.ensure_storage()
|
||||
async with self.repo.control():
|
||||
document = await self.repo.get(replay_id)
|
||||
key, body = str(command.request_id), command.model_dump(mode="json", exclude_none=True)
|
||||
receipt = document.setdefault("receipts", {}).get(key)
|
||||
if receipt:
|
||||
if receipt != body:
|
||||
raise HTTPException(409, "命令请求号已用于不同内容")
|
||||
return snapshot(document)
|
||||
if command.expected_revision != document["revision"]:
|
||||
raise HTTPException(409, "回放控制状态已更新,请刷新后重试")
|
||||
action, status = command.action, document["status"]
|
||||
if (action in {"step", "monitor", "retry"} and document.get("warmed")
|
||||
and weighted_diagnosis(document)
|
||||
and current_calibration(document, parse_time(document["start_time"])) is None):
|
||||
raise HTTPException(409, "旧回放缺少新版历史诊断校准,请停止后新建演示")
|
||||
if document.get("step_pending") and action in {"step", "monitor"}:
|
||||
raise HTTPException(409, "当前时间步正在处理,请完成后继续")
|
||||
if action in {"step", "monitor", "retry"} and await self.repo.active_other(replay_id):
|
||||
raise HTTPException(409, "当前项目已有另一条活动回放")
|
||||
if action == "step" and status in {"ready", "paused"}:
|
||||
interval = document.get("monitoring_interval_seconds")
|
||||
if interval is None:
|
||||
raise HTTPException(409, "旧回放未固定监测周期,请停止后新建演示")
|
||||
start = parse_time(document["start_time"])
|
||||
elapsed = (parse_time(document["virtual_now"]) - start).total_seconds()
|
||||
target = start + timedelta(seconds=(int(elapsed // interval) + 1) * interval)
|
||||
document["step_start"] = document["virtual_now"]
|
||||
document["step_anomaly_signals"] = []
|
||||
document["step_target"] = min(parse_time(document["end_time"]), target).isoformat()
|
||||
document["status"], document["step_pending"] = "paused", True
|
||||
elif action == "monitor" and status in {"ready", "paused", "completed"}:
|
||||
target = command.at_time
|
||||
if target is None:
|
||||
raise HTTPException(422, "选定时刻监测需要提供 at_time")
|
||||
start = parse_time(document["start_time"])
|
||||
current = parse_time(document["virtual_now"])
|
||||
if not start <= target <= parse_time(document["end_time"]):
|
||||
raise HTTPException(422, "监测时刻必须在本轮演示区间内")
|
||||
if target < current:
|
||||
raise HTTPException(409, "监测历史时刻需要新建一轮回放,避免使用后续状态")
|
||||
interval = document.get("monitoring_interval_seconds")
|
||||
if interval is None:
|
||||
raise HTTPException(409, "本轮未固定监测周期,请停止后新建演示")
|
||||
previous = document.get("last_monitoring") or {}
|
||||
# Reusing an immutable result must not count as another normal
|
||||
# period, resolve an incident, or create another investigation.
|
||||
if not previous.get("period_end") or parse_time(previous["period_end"]) != target:
|
||||
document["step_start"] = (current if target > current
|
||||
else max(start, target - timedelta(seconds=interval))).isoformat()
|
||||
document["step_target"] = target.isoformat()
|
||||
document["step_anomaly_signals"] = []
|
||||
document["status"], document["step_pending"] = "paused", True
|
||||
elif action == "stop" and status not in {"completed", "stopped"}:
|
||||
document["status"], document["step_pending"] = "stopped", False
|
||||
document.pop("step_target", None)
|
||||
document.pop("step_start", None)
|
||||
elif action == "retry" and status == "failed":
|
||||
if "SOURCE_CHANGED" in (document.get("error") or ""):
|
||||
raise HTTPException(409, "原始数据发生变化,需要停止并新建回放")
|
||||
document["status"], document["error"] = "paused", None
|
||||
document["step_pending"] = bool(document.get("step_target"))
|
||||
else:
|
||||
raise HTTPException(409, "当前回放状态不支持此操作")
|
||||
document["revision"] += 1
|
||||
document["receipts"][key] = body
|
||||
if len(document["receipts"]) > 256:
|
||||
del document["receipts"][next(iter(document["receipts"]))]
|
||||
await self.repo.save(document)
|
||||
return snapshot(document)
|
||||
|
||||
async def process_pending_step(self, replay_id):
|
||||
await self.ensure_storage()
|
||||
async with self.repo.processing() as owned:
|
||||
if not owned:
|
||||
raise HTTPException(409, "当前时间步正在由另一请求处理")
|
||||
while True:
|
||||
document = await self.repo.get(replay_id)
|
||||
if not document.get("step_pending"):
|
||||
return snapshot(document)
|
||||
try:
|
||||
await self._advance(document)
|
||||
except Exception as exc:
|
||||
# Failure never discards the prepared batch; retry resumes it.
|
||||
document = await self.repo.get(replay_id)
|
||||
document["status"], document["step_pending"] = "failed", False
|
||||
if isinstance(exc, HTTPException):
|
||||
document["error"] = str(exc.detail)
|
||||
elif isinstance(exc, BaselineBudgetExceeded):
|
||||
document["error"] = str(exc)
|
||||
else:
|
||||
document["error"] = (
|
||||
"SOURCE_CHANGED: 原始观测已改变,需要新建回放"
|
||||
if "SOURCE_CHANGED" in str(exc)
|
||||
else "SCADA 处理失败,已保留批次与进度;请检查服务日志后重试"
|
||||
)
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save(document)
|
||||
raise
|
||||
|
||||
async def _advance(self, document):
|
||||
config = ScadaReplayCreate.model_validate(document["config"])
|
||||
replay_id = document["replay_id"]
|
||||
if not document.get("warmed"):
|
||||
await self._freeze_baseline(document, config)
|
||||
if weighted_diagnosis(document) and current_calibration(document, config.start_time) is None:
|
||||
raise HTTPException(409, "旧回放缺少新版历史诊断校准,请停止后新建演示")
|
||||
batch = await self.repo.pending_batch(replay_id)
|
||||
if batch is None:
|
||||
step = document.get("step_pending", False)
|
||||
virtual = parse_time(document["virtual_now"])
|
||||
through = parse_time(document["step_target"]) if step else virtual
|
||||
after = ((parse_time(document["cursor"][0]), document["cursor"][1])
|
||||
if document.get("cursor") else None)
|
||||
async with self.timescale.transaction():
|
||||
rows = await ScadaRepository.get_raw_page(
|
||||
self.timescale, config.device_ids, config.start_time, config.end_time,
|
||||
through, 501, after, frame=True,
|
||||
)
|
||||
if len(rows) > 500:
|
||||
raise ValueError("SCADA frame exceeds selected device bound")
|
||||
at = rows[0]["time"] if rows else through
|
||||
if step:
|
||||
virtual = through
|
||||
document["virtual_now"] = virtual.isoformat()
|
||||
if not rows and document.get("processed_until") == at.isoformat() and not step:
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save(document)
|
||||
return
|
||||
sequence = document["batch_sequence"] + 1
|
||||
refs = [{"time": row["time"].isoformat(), "device_id": row["device_id"],
|
||||
"source_hash": row["source_hash"]} for row in rows]
|
||||
batch = {"sequence": sequence, "stage": "selected", "at": at.isoformat(),
|
||||
"refs": refs, "input_digest": digest(refs), "events": []}
|
||||
async with self.repo.connection.transaction():
|
||||
document["batch_sequence"] = sequence
|
||||
document["delivered_until"] = at.isoformat()
|
||||
await self.repo.save_batch(replay_id, batch)
|
||||
await self.repo.save(document)
|
||||
else:
|
||||
rows = None
|
||||
if batch["stage"] == "selected":
|
||||
if rows is None:
|
||||
# Only crash recovery reloads the selected raw frame.
|
||||
at = parse_time(batch["at"])
|
||||
async with self.timescale.transaction():
|
||||
rows = await ScadaRepository.get_raw_page(
|
||||
self.timescale, config.device_ids, at, config.end_time, at, 501,
|
||||
) if batch["refs"] else []
|
||||
actual = [{"time": row["time"].isoformat(), "device_id": row["device_id"],
|
||||
"source_hash": row["source_hash"]} for row in rows]
|
||||
if digest(actual) != batch["input_digest"]:
|
||||
raise ValueError("SOURCE_CHANGED: selected raw frame differs")
|
||||
batch["outputs"] = prepare_cleaning(rows, config)
|
||||
batch["stage"] = "clean_prepared"
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save_batch(replay_id, batch)
|
||||
if batch["stage"] == "clean_prepared":
|
||||
await ScadaRepository.apply_replay_cleaning(self.timescale, batch["outputs"])
|
||||
batch["stage"] = "cleaned"
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save_batch(replay_id, batch)
|
||||
if batch["stage"] == "cleaned":
|
||||
result, changes = detect_frame(document, batch["outputs"], config, parse_time(batch["at"]))
|
||||
result["processed_until"] = batch["at"]
|
||||
target = result.get("step_target")
|
||||
was_step = bool(target)
|
||||
result["step_pending"] = bool(target and parse_time(batch["at"]) < parse_time(target))
|
||||
if was_step:
|
||||
signals = {item["event_id"]: item for item in result.get("step_anomaly_signals", [])
|
||||
if diagnostic_signal(result, item)}
|
||||
for event in changes:
|
||||
if event["change"] == "raised" and diagnostic_signal(result, event):
|
||||
signals[event["event_id"]] = {key: event[key] for key in (
|
||||
"event_id", "rule_id", "device_ids", "detail", "diagnostic_policy"
|
||||
) if key in event}
|
||||
result["step_anomaly_signals"] = list(signals.values())
|
||||
monitoring = None
|
||||
if was_step and not result["step_pending"]:
|
||||
period_start = parse_time(result.get("step_start", result["start_time"]))
|
||||
period_end = parse_time(target)
|
||||
role = result.get("monitoring_config") or {}
|
||||
role_ids = list(dict.fromkeys(
|
||||
role.get("pressure_device_ids", []) + role.get("outlet_flow_device_ids", [])
|
||||
))
|
||||
previous = await self.repo.monitoring_outputs(
|
||||
replay_id, role_ids, period_end - timedelta(days=15),
|
||||
period_end, batch["sequence"],
|
||||
) if role_ids else []
|
||||
from app.mcp.analysis_service import bounded_calculation
|
||||
monitoring = await bounded_calculation(
|
||||
evaluate_monitoring_period,
|
||||
result,
|
||||
previous + batch["outputs"],
|
||||
period_start=period_start,
|
||||
period_end=period_end,
|
||||
batch_sequence=batch["sequence"],
|
||||
)
|
||||
# Any configured measurement rule can enter diagnosis, including
|
||||
# indicators outside the pressure/outlet roles used for burst scoring.
|
||||
signals = {item["event_id"]: item for item in result.get("step_anomaly_signals", [])
|
||||
if diagnostic_signal(result, item)}
|
||||
for state in result.get("rule_states", {}).values():
|
||||
event = state.get("event")
|
||||
if event and diagnostic_signal(result, event):
|
||||
signals[event["event_id"]] = {key: event[key] for key in (
|
||||
"event_id", "rule_id", "device_ids", "detail", "diagnostic_policy"
|
||||
) if key in event}
|
||||
monitoring["anomaly_signals"] = list(signals.values())
|
||||
if signals and monitoring["status"] == "normal":
|
||||
monitoring["status"] = "attention"
|
||||
monitoring["detail"] = "本周期曾检出测点异常,进入异常调查与爆管诊断"
|
||||
result["monitoring_count"] = result.get("monitoring_count", 0) + 1
|
||||
if not result["step_pending"]:
|
||||
result.pop("step_target", None)
|
||||
result.pop("step_start", None)
|
||||
result.pop("step_anomaly_signals", None)
|
||||
if batch["refs"]:
|
||||
last = batch["refs"][-1]
|
||||
result["cursor"] = [last["time"], last["device_id"]]
|
||||
if parse_time(batch["at"]) >= config.end_time:
|
||||
result["status"] = "completed"
|
||||
batch["stage"], batch["events"] = "detected", changes
|
||||
if monitoring is not None:
|
||||
batch["monitoring"] = monitoring
|
||||
async with self.repo.connection.transaction():
|
||||
await self._create_investigations(result, changes, batch, create_tasks=not was_step)
|
||||
if monitoring is not None:
|
||||
await self._create_monitoring_task(result, monitoring, batch)
|
||||
batch["monitoring"] = monitoring
|
||||
result["last_monitoring"] = monitoring
|
||||
await self.repo.save_batch(replay_id, batch)
|
||||
await self.repo.save(result)
|
||||
|
||||
async def _freeze_baseline(self, document, config):
|
||||
"""Stage bounded pages, then atomically publish one causal source snapshot."""
|
||||
if document.get("warmed"):
|
||||
return
|
||||
start = config.start_time - timedelta(days=15)
|
||||
batch_count, row_count, byte_count, after = 0, 0, 0, None
|
||||
counts = {device_id: 0 for device_id in config.device_ids}
|
||||
valid_counts = dict(counts)
|
||||
histories = {device_id: [] for device_id in config.device_ids}
|
||||
window = max(rule.window_samples for rule in config.rules)
|
||||
calibration = None
|
||||
if (document.get("monitoring_config") or {}).get("decision_policy") == "weighted":
|
||||
from app.services.scada_monitoring_calibration import MonitoringCalibrationBuilder
|
||||
calibration = MonitoringCalibrationBuilder(document, config)
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.clear_unpublished_baseline(document["replay_id"])
|
||||
# Keep only the current page in memory. Each BizDB write commits before
|
||||
# the next source read; clean_prepared baseline pages remain invisible.
|
||||
async with self.timescale.transaction():
|
||||
await ScadaRepository.begin_replay_baseline_snapshot(self.timescale)
|
||||
while True:
|
||||
rows = await ScadaRepository.get_raw_page(
|
||||
self.timescale, config.device_ids, start, config.start_time,
|
||||
config.start_time, min(BASELINE_PAGE_SIZE, BASELINE_MAX_ROWS - row_count + 1), after,
|
||||
)
|
||||
if not rows:
|
||||
break
|
||||
if row_count + len(rows) > BASELINE_MAX_ROWS:
|
||||
raise BaselineBudgetExceeded(
|
||||
f"BASELINE_BUDGET_EXCEEDED: 十五天历史基线超过 {BASELINE_MAX_ROWS:,} 条准备预算,"
|
||||
"尚未发布;请减少回放设备或核查历史采样密度后重试"
|
||||
)
|
||||
outputs = prepare_cleaning(rows, config)
|
||||
if calibration is not None:
|
||||
calibration.add(outputs)
|
||||
batch = {
|
||||
"sequence": -batch_count - 1, "stage": "clean_prepared", "kind": "baseline",
|
||||
"at": outputs[-1]["time"], "outputs": outputs, "events": [],
|
||||
}
|
||||
batch_bytes = len(json.dumps(batch, ensure_ascii=False, allow_nan=False,
|
||||
separators=(",", ":")).encode("utf-8"))
|
||||
if byte_count + batch_bytes > BASELINE_MAX_BYTES:
|
||||
raise BaselineBudgetExceeded(
|
||||
f"BASELINE_BUDGET_EXCEEDED: 十五天历史基线超过 {BASELINE_MAX_BYTES // (1024 * 1024)} MiB 准备预算,"
|
||||
"尚未发布;请减少回放设备或核查历史采样密度后重试"
|
||||
)
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save_batch(document["replay_id"], batch)
|
||||
batch_count += 1
|
||||
row_count += len(outputs)
|
||||
byte_count += batch_bytes
|
||||
for row in outputs:
|
||||
device_id = row["device_id"]
|
||||
counts[device_id] += 1
|
||||
if row["quality"] == "valid":
|
||||
valid_counts[device_id] += 1
|
||||
histories[device_id].append(row["cleaned_value"])
|
||||
histories[device_id] = histories[device_id][-window:]
|
||||
last = rows[-1]
|
||||
after = (last["time"], last["device_id"])
|
||||
document["baseline"] = {
|
||||
"start_time": start.isoformat(), "end_time": config.start_time.isoformat(),
|
||||
"end_exclusive": True, "sample_counts": counts, "valid_sample_counts": valid_counts,
|
||||
"row_count": row_count, "batch_count": batch_count, "byte_count": byte_count,
|
||||
}
|
||||
document["histories"] = histories
|
||||
if calibration is not None:
|
||||
from app.mcp.analysis_service import bounded_calculation
|
||||
document["monitoring_calibration"] = await bounded_calculation(calibration.fit)
|
||||
document["warmed"] = True
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.publish_baseline(document["replay_id"])
|
||||
await self.repo.save(document)
|
||||
|
||||
async def _create_investigations(self, document, changes, batch, *, create_tasks=True):
|
||||
from app.services.scada_normal_profile import pollution_signal
|
||||
from app.domain.schemas.operational_task import OperationalTaskCreate, TaskSource, TaskTrigger
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
from app.domain.schemas.task_alert import TaskAlertEvent
|
||||
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
|
||||
from app.services.task_alerts import TaskAlertService
|
||||
from uuid import UUID
|
||||
producer = "scada-replay:" + document["replay_id"]
|
||||
alerts = TaskAlertService(TaskAlertRepository(self.repo.connection, self.context.project_id),
|
||||
self.context, internal_producer_id=producer)
|
||||
tasks = OperationalTaskService(OperationalTaskRepository(self.repo.connection, self.context.project_id),
|
||||
self.context, permissions_for_context(self.context),
|
||||
execution_scope=settings.operational_task_execution_scope)
|
||||
for event in changes:
|
||||
pollution = pollution_signal(document, event)
|
||||
# Pressure/flow rules share period diagnosis during manual steps.
|
||||
# Water-quality incidents and telemetry-quality alerts stay independent.
|
||||
if (not create_tasks and not pollution and event.get("category") != "data_quality"
|
||||
and (event["change"] == "raised" or event.get("notification_suppressed"))):
|
||||
if event["change"] == "raised":
|
||||
event["batch_sequence"] = batch["sequence"]
|
||||
event["notification_suppressed"] = True
|
||||
document["rule_states"][event.get("state_key", event["rule_id"])]["event"].update(
|
||||
batch_sequence=batch["sequence"], notification_suppressed=True,
|
||||
)
|
||||
continue
|
||||
if event["change"] == "raised":
|
||||
event["batch_sequence"] = batch["sequence"]
|
||||
document["rule_states"][event.get("state_key", event["rule_id"])]["event"]["batch_sequence"] = batch["sequence"]
|
||||
if event["change"] == "raised" and event.get("category") != "data_quality" and (create_tasks or pollution):
|
||||
external = f"scada_replay:{document['replay_id']}:event:{event['event_id']}"
|
||||
command = OperationalTaskCreate(
|
||||
request_id=uuid5(NAMESPACE_URL, external), task_type="event_investigation",
|
||||
workflow="water_pollution" if pollution else "scada_anomaly",
|
||||
title=f"{'水污染事件调查' if pollution else 'SCADA 回放调查'} · {event['rule_id']}",
|
||||
summary=event["detail"], source=TaskSource(
|
||||
kind="external_system", external_id=external, external_revision=1,
|
||||
event_kind="water_pollution" if pollution else "scada_anomaly",
|
||||
replay_context={"replay_id": document["replay_id"],
|
||||
"knowledge_cutoff": batch["at"], "batch_sequence": batch["sequence"]},
|
||||
),
|
||||
trigger=TaskTrigger(kind="event", occurred_at=event["occurred_at"], detail=(
|
||||
"水质监测规则检出异常,启动水污染事件调查;污染源及影响待核验" if pollution else
|
||||
"已存 SCADA 历史的回放检测信号,原因尚待调查")),
|
||||
)
|
||||
task = await tasks.create(command)
|
||||
event["task_id"] = str(task.task_id)
|
||||
document["rule_states"][event.get("state_key", event["rule_id"])]["event"]["task_id"] = event["task_id"]
|
||||
await alerts.push(TaskAlertEvent(
|
||||
event_id=uuid5(NAMESPACE_URL, event["event_id"] + ":" + event["change"]),
|
||||
alert_id=event["event_id"], producer_id=producer, event_type=event["change"],
|
||||
occurred_at=event["occurred_at"],
|
||||
condition_id=f"operational-task:{event['task_id']}" if pollution and event.get("task_id") else None,
|
||||
source={"kind": "scada_replay", "replay_id": document["replay_id"],
|
||||
"rule_id": event["rule_id"], "batch_sequence": event["batch_sequence"]},
|
||||
title=f"{'水污染侦测异常' if pollution else 'SCADA 回放异常'} · {event['rule_id']}", description=event["detail"], severity="warning",
|
||||
))
|
||||
if event["change"] != "raised":
|
||||
if event.get("task_id"):
|
||||
await tasks.advance_replay_investigation(UUID(event["task_id"]), {
|
||||
"replay_id": document["replay_id"], "knowledge_cutoff": batch["at"],
|
||||
"batch_sequence": batch["sequence"],
|
||||
}, uuid5(NAMESPACE_URL, event["event_id"] + ":recovery"))
|
||||
continue
|
||||
|
||||
async def _create_monitoring_task(self, document, monitoring, batch):
|
||||
from app.services.scada_monitoring_incidents import attach_incident, update_incident
|
||||
incident, action = update_incident(document, monitoring, batch["sequence"])
|
||||
if incident is None:
|
||||
return
|
||||
if incident.get("task_id"):
|
||||
if monitoring["status"] == "suspected_burst" and not incident.get("burst_diagnosed"):
|
||||
from uuid import UUID
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
tasks = OperationalTaskService(
|
||||
OperationalTaskRepository(self.repo.connection, self.context.project_id),
|
||||
self.context, permissions_for_context(self.context),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
await tasks.advance_replay_investigation(UUID(incident["task_id"]), {
|
||||
"replay_id": document["replay_id"], "knowledge_cutoff": monitoring["knowledge_cutoff"],
|
||||
"batch_sequence": batch["sequence"],
|
||||
}, uuid5(NAMESPACE_URL, incident["incident_id"] + ":burst-evidence"))
|
||||
incident["burst_diagnosed"] = True
|
||||
if action == "resolved" and incident.get("alert"):
|
||||
await self._publish_monitoring_alert(document, incident, "resolved")
|
||||
attach_incident(monitoring, incident, action)
|
||||
return
|
||||
from app.domain.schemas.operational_task import OperationalTaskCreate, TaskSource, TaskTrigger
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
tasks = OperationalTaskService(
|
||||
OperationalTaskRepository(self.repo.connection, self.context.project_id),
|
||||
self.context,
|
||||
permissions_for_context(self.context),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
external = incident["external_id"]
|
||||
pressure = monitoring["pressure"]
|
||||
flow = monitoring["outlet_flow"]
|
||||
findings = []
|
||||
if pressure.get("hit_device_ids"):
|
||||
findings.append(f"{len(pressure['hit_device_ids'])} 个压力测点下降")
|
||||
if flow.get("hit_device_ids"):
|
||||
findings.append(f"{len(flow['hit_device_ids'])} 个出厂边界流量测点增加")
|
||||
summary = f"SCADA 监测发现{'、'.join(findings) or '指标异常'},已启动调查。"
|
||||
if monitoring.get("anomaly_signals"):
|
||||
signals = monitoring["anomaly_signals"]
|
||||
devices = {device for item in signals for device in item.get("device_ids", [])}
|
||||
summary += f"本次异常涉及 {len(devices)} 个测点。"
|
||||
summary += ("现有监测结果提示爆管嫌疑,需结合管网与现场核验。"
|
||||
if monitoring["status"] == "suspected_burst" else "异常原因及影响范围待核查。")
|
||||
event_time = parse_time(monitoring["period_end"]).astimezone(ZoneInfo("Asia/Shanghai"))
|
||||
command = OperationalTaskCreate(
|
||||
request_id=uuid5(NAMESPACE_URL, external),
|
||||
task_type="event_investigation",
|
||||
workflow="scada_anomaly",
|
||||
title=f"SCADA 异常调查 · {event_time:%Y-%m-%d %H:%M}",
|
||||
summary=summary,
|
||||
source=TaskSource(
|
||||
kind="external_system",
|
||||
external_id=external,
|
||||
external_revision=1,
|
||||
event_kind="scada_anomaly",
|
||||
replay_context={
|
||||
"replay_id": document["replay_id"],
|
||||
"knowledge_cutoff": monitoring["knowledge_cutoff"],
|
||||
"batch_sequence": batch["sequence"],
|
||||
},
|
||||
),
|
||||
trigger=TaskTrigger(
|
||||
kind="event",
|
||||
occurred_at=monitoring["period_end"],
|
||||
detail="SCADA 任一指标异常启动调查,核验数据后进行爆管诊断;异常不等于已确诊爆管",
|
||||
),
|
||||
)
|
||||
task = await tasks.create(command)
|
||||
incident["task_id"] = str(task.task_id)
|
||||
incident["burst_diagnosed"] = monitoring["status"] == "suspected_burst"
|
||||
incident["alert"] = {
|
||||
"alert_id": incident["incident_id"],
|
||||
"producer_id": "scada-replay:" + document["replay_id"],
|
||||
"occurred_at": incident["started_at"],
|
||||
"condition_id": f"operational-task:{task.task_id}",
|
||||
"source": {"kind": "scada_replay", "replay_id": document["replay_id"],
|
||||
"rule_id": "scada_diagnosis", "batch_sequence": batch["sequence"]},
|
||||
"title": "SCADA 指标异常", "description": "已启动调查,详细诊断见调查报告。", "severity": "warning",
|
||||
}
|
||||
await self._publish_monitoring_alert(document, incident, "raised")
|
||||
attach_incident(monitoring, incident, action)
|
||||
|
||||
async def _publish_monitoring_alert(self, document, incident, change):
|
||||
from app.domain.schemas.task_alert import TaskAlertEvent
|
||||
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
|
||||
from app.services.task_alerts import TaskAlertService
|
||||
alerts = TaskAlertService(
|
||||
TaskAlertRepository(self.repo.connection, self.context.project_id), self.context,
|
||||
internal_producer_id="scada-replay:" + document["replay_id"],
|
||||
)
|
||||
await alerts.push(TaskAlertEvent(
|
||||
**incident["alert"], event_type=change,
|
||||
event_id=uuid5(NAMESPACE_URL, incident["incident_id"] + ":" + change),
|
||||
))
|
||||
@@ -0,0 +1,287 @@
|
||||
"""Deterministic monitoring summaries built only from frozen replay outputs."""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from statistics import median
|
||||
from typing import Any
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
from app.services.scada_normal_profile import current_calibration, normal_profile_bounds
|
||||
from app.services.burst_detection import (
|
||||
TARGET_MU,
|
||||
TARGET_N_ESTIMATORS,
|
||||
TARGET_RANDOM_STATE,
|
||||
TARGET_SCORE_THRESHOLD,
|
||||
run_burst_detection,
|
||||
)
|
||||
|
||||
|
||||
def _time(value: str | datetime) -> datetime:
|
||||
return value if isinstance(value, datetime) else datetime.fromisoformat(value)
|
||||
|
||||
|
||||
def _valid_rows(rows: list[dict], device_id: str) -> list[dict]:
|
||||
return sorted(
|
||||
(
|
||||
row for row in rows
|
||||
if row.get("device_id") == device_id
|
||||
and row.get("quality") in {"valid", "corrected"}
|
||||
and row.get("cleaned_value") is not None
|
||||
),
|
||||
key=lambda row: _time(row["time"]),
|
||||
)
|
||||
|
||||
|
||||
def _relative_indicator(
|
||||
rows: list[dict],
|
||||
config: ScadaReplayCreate,
|
||||
device_ids: list[str],
|
||||
period_start: datetime,
|
||||
period_end: datetime,
|
||||
*,
|
||||
direction: str,
|
||||
calibration: dict | None = None,
|
||||
) -> dict[str, Any]:
|
||||
details: list[dict[str, Any]] = []
|
||||
gaps: list[dict[str, str]] = []
|
||||
for device_id in device_ids:
|
||||
rule = next((
|
||||
item for item in config.rules
|
||||
if item.kind == "relative_change" and device_id in item.device_ids
|
||||
and item.threshold is not None
|
||||
), None)
|
||||
if rule is None:
|
||||
gaps.append({"device_id": device_id, "reason": "relative_change_rule_missing"})
|
||||
continue
|
||||
profile = calibration.get(device_id) if calibration is not None else None
|
||||
if calibration is not None and profile is None:
|
||||
gaps.append({"device_id": device_id, "reason": "adaptive_history_not_ready"})
|
||||
continue
|
||||
threshold = float(profile["threshold_percent"]) if profile else rule.threshold
|
||||
valid = _valid_rows(rows, device_id)
|
||||
current = [row for row in valid if period_start < _time(row["time"]) <= period_end]
|
||||
latest = current[-1] if current else None
|
||||
if not current:
|
||||
gaps.append({"device_id": device_id, "reason": "period_reading_missing"})
|
||||
continue
|
||||
baseline = ([row for row in valid if _time(row["time"]) < _time(latest["time"])]
|
||||
[-rule.window_samples:]) if latest else []
|
||||
if len(baseline) < rule.minimum_samples:
|
||||
gaps.append({"device_id": device_id, "reason": "baseline_not_ready"})
|
||||
continue
|
||||
reference = median(float(row["cleaned_value"]) for row in baseline)
|
||||
if reference == 0:
|
||||
gaps.append({"device_id": device_id, "reason": "zero_baseline"})
|
||||
continue
|
||||
value = float(latest["cleaned_value"])
|
||||
change = (value - reference) / abs(reference) * 100
|
||||
hit = change <= -threshold if direction == "decrease" else change >= threshold
|
||||
bounds = normal_profile_bounds(profile.get("normal_profile"), _time(latest["time"])) if profile else None
|
||||
if profile and bounds is None:
|
||||
gaps.append({"device_id": device_id, "reason": "normal_phase_history_not_ready"})
|
||||
continue
|
||||
if bounds is not None:
|
||||
hit = hit and not bounds[0] <= value <= bounds[1]
|
||||
details.append({
|
||||
"device_id": device_id,
|
||||
"observed_at": _time(latest["time"]).isoformat(),
|
||||
"baseline": reference,
|
||||
"baseline_method": "recent_samples_median",
|
||||
"baseline_start": _time(baseline[0]["time"]).isoformat(),
|
||||
"baseline_end": _time(baseline[-1]["time"]).isoformat(),
|
||||
"baseline_sample_count": len(baseline),
|
||||
"value": value,
|
||||
"value_change": value - reference,
|
||||
"unit": rule.unit,
|
||||
"change_percent": change,
|
||||
"threshold_percent": threshold,
|
||||
**({"normal_phase_lower": bounds[0], "normal_phase_upper": bounds[1]} if bounds else {}),
|
||||
**({"threshold_method": profile["method"],
|
||||
"configured_threshold_percent": rule.threshold} if profile else {}),
|
||||
"hit": hit,
|
||||
})
|
||||
return {
|
||||
"status": "ready" if not gaps else "insufficient_data",
|
||||
"hit": any(item["hit"] for item in details),
|
||||
"hit_device_ids": [item["device_id"] for item in details if item["hit"]],
|
||||
"devices": details,
|
||||
"gaps": gaps,
|
||||
}
|
||||
|
||||
|
||||
def _burst_detection(
|
||||
rows: list[dict],
|
||||
pressure_device_ids: list[str],
|
||||
interval_seconds: int,
|
||||
period_end: datetime,
|
||||
*,
|
||||
score_threshold: float = TARGET_SCORE_THRESHOLD,
|
||||
) -> dict[str, Any]:
|
||||
if len(pressure_device_ids) < 5:
|
||||
return {"status": "sensor_coverage_insufficient", "hit": False,
|
||||
"required_sensors": 5, "available_sensors": len(pressure_device_ids)}
|
||||
if interval_seconds % 60 or 86400 % interval_seconds:
|
||||
return {"status": "unsupported_sampling_interval", "hit": False,
|
||||
"interval_seconds": interval_seconds}
|
||||
points_per_day = 86400 // interval_seconds
|
||||
count = 15 * points_per_day
|
||||
series_by_device: dict[str, dict[datetime, float]] = {}
|
||||
common_times: set[datetime] | None = None
|
||||
for device_id in pressure_device_ids:
|
||||
values = {_time(row["time"]): float(row["cleaned_value"])
|
||||
for row in _valid_rows(rows, device_id) if _time(row["time"]) <= period_end}
|
||||
series_by_device[device_id] = values
|
||||
common_times = set(values) if common_times is None else common_times & set(values)
|
||||
if not common_times:
|
||||
return {"status": "baseline_not_ready", "hit": False, "required_samples": count}
|
||||
target = period_end
|
||||
if target not in common_times:
|
||||
return {"status": "current_reading_missing", "hit": False,
|
||||
"target_time": target.isoformat(), "latest_common_time": max(common_times).isoformat()}
|
||||
expected = [target - timedelta(seconds=interval_seconds * offset)
|
||||
for offset in range(count - 1, -1, -1)]
|
||||
incomplete = [device_id for device_id, values in series_by_device.items()
|
||||
if any(at not in values for at in expected)]
|
||||
if incomplete:
|
||||
return {"status": "baseline_not_ready", "hit": False, "required_samples": count,
|
||||
"incomplete_device_ids": incomplete}
|
||||
frame = pd.DataFrame({device_id: [series_by_device[device_id][at] for at in expected]
|
||||
for device_id in pressure_device_ids}, index=expected)
|
||||
try:
|
||||
result = run_burst_detection(
|
||||
network="scada-replay",
|
||||
username="scada-replay",
|
||||
observed_pressure_data=frame,
|
||||
points_per_day=points_per_day,
|
||||
mu=TARGET_MU,
|
||||
iforest_params={
|
||||
"n_estimators": TARGET_N_ESTIMATORS,
|
||||
"random_state": TARGET_RANDOM_STATE,
|
||||
"contamination": "auto",
|
||||
},
|
||||
sensor_nodes=pressure_device_ids,
|
||||
)
|
||||
except (TypeError, ValueError) as exc:
|
||||
return {"status": "analysis_failed", "hit": False, "reason": str(exc)[:300]}
|
||||
latest = result["rows"][-1]
|
||||
score = float(latest["Score"])
|
||||
return {
|
||||
"status": "ready",
|
||||
"hit": score <= score_threshold,
|
||||
"target_time": target.isoformat(),
|
||||
"target_score": score,
|
||||
"score_interpretation": "lower_is_more_anomalous_not_probability",
|
||||
"score_threshold": score_threshold,
|
||||
"sensor_count": len(pressure_device_ids),
|
||||
"reference_day_count": 14,
|
||||
"latest_sensor_rankings": result["summary"].get("latest_sensor_rankings", []),
|
||||
}
|
||||
|
||||
|
||||
def evaluate_monitoring_period(
|
||||
document: dict,
|
||||
rows: list[dict],
|
||||
*,
|
||||
period_start: datetime,
|
||||
period_end: datetime,
|
||||
batch_sequence: int,
|
||||
) -> dict[str, Any]:
|
||||
config = ScadaReplayCreate.model_validate(document["config"])
|
||||
monitoring = document.get("monitoring_config")
|
||||
rule_version = (monitoring or {}).get("rule_version", "unconfigured")
|
||||
monitoring_id = str(uuid5(
|
||||
NAMESPACE_URL,
|
||||
f"{document['replay_id']}:{period_end.isoformat()}:{rule_version}",
|
||||
))
|
||||
base = {
|
||||
"monitoring_id": monitoring_id,
|
||||
"period_start": period_start.isoformat(),
|
||||
"period_end": period_end.isoformat(),
|
||||
"knowledge_cutoff": period_end.isoformat(),
|
||||
"batch_sequence": batch_sequence,
|
||||
"rule_version": rule_version,
|
||||
"task_id": None,
|
||||
}
|
||||
if not monitoring:
|
||||
return {**base, "status": "insufficient_data", "detail": "项目未配置 SCADA 监测设备角色",
|
||||
"pressure": {"status": "unconfigured", "hit": False},
|
||||
"outlet_flow": {"status": "unconfigured", "hit": False},
|
||||
"burst_detection": {"status": "not_run", "hit": False}}
|
||||
pressure_ids = monitoring.get("pressure_device_ids", [])
|
||||
flow_ids = monitoring.get("outlet_flow_device_ids", [])
|
||||
selected = set(config.device_ids)
|
||||
role_gaps = sorted((set(pressure_ids) | set(flow_ids)) - selected)
|
||||
weighted_policy = monitoring.get("decision_policy") == "weighted"
|
||||
if not weighted_policy and (not pressure_ids or not flow_ids or role_gaps):
|
||||
return {**base, "status": "insufficient_data", "detail": "监测角色为空或不属于本轮回放设备",
|
||||
"role_gaps": role_gaps,
|
||||
"pressure": {"status": "unconfigured", "hit": False},
|
||||
"outlet_flow": {"status": "unconfigured", "hit": False},
|
||||
"burst_detection": {"status": "not_run", "hit": False}}
|
||||
calibration = current_calibration(document, config.start_time) if weighted_policy else None
|
||||
if config.start_time > period_end:
|
||||
calibration = None
|
||||
base["role_gaps"] = role_gaps
|
||||
pressure = _relative_indicator(
|
||||
rows, config, pressure_ids, period_start, period_end, direction="decrease",
|
||||
calibration=(calibration or {}).get("pressure", {}) if weighted_policy else None,
|
||||
)
|
||||
outlet_flow = _relative_indicator(
|
||||
rows, config, flow_ids, period_start, period_end, direction="increase",
|
||||
calibration=(calibration or {}).get("outlet_flow", {}) if weighted_policy else None,
|
||||
) if flow_ids else {"status": "unconfigured", "hit": False, "hit_device_ids": [], "devices": [], "gaps": []}
|
||||
detector = {"status": "baseline_not_ready", "hit": False} if weighted_policy and (
|
||||
not calibration or calibration.get("status") != "ready"
|
||||
) else _burst_detection(
|
||||
rows, pressure_ids, int(document["monitoring_interval_seconds"]), period_end,
|
||||
score_threshold=calibration["burst_detection"]["threshold"] if calibration else TARGET_SCORE_THRESHOLD,
|
||||
)
|
||||
if weighted_policy:
|
||||
settings = ScadaMonitoringSettings.model_validate(monitoring)
|
||||
weights = settings.weights.model_dump()
|
||||
indicators = {"burst_detection": detector, "pressure": pressure, "outlet_flow": outlet_flow}
|
||||
contributions = {key: weights[key] if indicator["status"] == "ready" and indicator["hit"] else 0.0
|
||||
for key, indicator in indicators.items()}
|
||||
score = round(sum(contributions.values()), 10)
|
||||
complete = pressure["status"] == detector["status"] == "ready"
|
||||
diagnostic_missing = sorted({device for rule in config.rules if rule.kind == "relative_change"
|
||||
for device in rule.device_ids}
|
||||
- set((calibration or {}).get("diagnostic", {})))
|
||||
if not complete:
|
||||
status, detail = "insufficient_data", (
|
||||
"旧回放缺少新版历史诊断校准,请停止后新建演示" if calibration is None
|
||||
else "压力数据或爆管侦测基线不足,尚不能作出组合判定"
|
||||
)
|
||||
elif score >= settings.alert_threshold:
|
||||
status, detail = "suspected_burst", "爆管相关指标加权达到疑似爆管门槛"
|
||||
elif any(indicator["hit"] for indicator in indicators.values()):
|
||||
status, detail = "attention", "部分指标异常,进入调查;综合结果尚未达到爆管嫌疑门槛"
|
||||
elif diagnostic_missing:
|
||||
status, detail = "insufficient_data", "部分测点缺少完整历史同期基线,不能判定全部指标正常"
|
||||
else:
|
||||
status, detail = "normal", "本周期爆管相关指标均未命中"
|
||||
detail += f"(综合 {score:.2f} / 门槛 {settings.alert_threshold:.2f},流量权重 {weights['outlet_flow']:.0%})"
|
||||
return {**base, "status": status, "detail": detail, "pressure": pressure,
|
||||
"outlet_flow": outlet_flow, "burst_detection": detector,
|
||||
"weighted": {"score": score, "threshold": settings.alert_threshold,
|
||||
"weights": weights, "contributions": contributions,
|
||||
"unavailable_indicators": [key for key, value in indicators.items() if value["status"] != "ready"],
|
||||
"calibration_cutoff": calibration["knowledge_cutoff"] if calibration else None,
|
||||
"diagnostic_unavailable_device_ids": diagnostic_missing,
|
||||
"missing_weight_policy": "zero_contribution_without_renormalization"}}
|
||||
complete = pressure["status"] == outlet_flow["status"] == detector["status"] == "ready"
|
||||
hits = (bool(detector["hit"]), bool(pressure["hit"]), bool(outlet_flow["hit"]))
|
||||
if not complete:
|
||||
status, detail = "insufficient_data", "监测数据、设备角色或爆管侦测基线不足"
|
||||
elif all(hits):
|
||||
status, detail = "suspected_burst", "爆管侦测、压力下降与出厂边界流量增加相互支持"
|
||||
elif any(hits):
|
||||
status, detail = "attention", "仅部分爆管相关指标命中,需要继续观察或人工核查"
|
||||
else:
|
||||
status, detail = "normal", "本周期爆管相关指标均未命中"
|
||||
return {**base, "status": status, "detail": detail, "pressure": pressure,
|
||||
"outlet_flow": outlet_flow, "burst_detection": detector}
|
||||
@@ -0,0 +1,160 @@
|
||||
"""Causal batch quality checks and grouped anomaly episodes, no database IO."""
|
||||
from copy import deepcopy
|
||||
from datetime import datetime
|
||||
from math import isfinite
|
||||
from statistics import median
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
import re
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.services.scada_normal_profile import (
|
||||
DIAGNOSTIC_POLICY, current_calibration, normal_profile_bounds, weighted_diagnosis,
|
||||
)
|
||||
|
||||
|
||||
def prepare_cleaning(rows: list[dict], config: ScadaReplayCreate) -> list[dict]:
|
||||
result = []
|
||||
for row in rows:
|
||||
value = row["monitored_value"]
|
||||
quality, reason = "valid", "原始值通过数值与已配置量程校验"
|
||||
if value is None:
|
||||
quality, reason = "missing", "原始读数缺失"
|
||||
elif not isfinite(float(value)):
|
||||
quality, reason = "invalid", "原始读数不是有限数值"
|
||||
else:
|
||||
bounds = config.quality_ranges.get(row["device_id"])
|
||||
if bounds and ((bounds.lower is not None and value < bounds.lower)
|
||||
or (bounds.upper is not None and value > bounds.upper)):
|
||||
quality, reason = "invalid", "超出本次明确配置的仪表量程"
|
||||
result.append({
|
||||
"time": row["time"].isoformat() if isinstance(row["time"], datetime) else row["time"],
|
||||
"device_id": row["device_id"], "source_hash": row["source_hash"],
|
||||
"monitored_value": value if value is not None and isfinite(float(value)) else None,
|
||||
"cleaned_value": value if quality == "valid" else None,
|
||||
"quality": quality, "reason": reason,
|
||||
})
|
||||
return result
|
||||
|
||||
|
||||
def detect_frame(document: dict, rows: list[dict], config: ScadaReplayCreate, at: datetime) -> tuple[dict, list[dict]]:
|
||||
"""Consume a complete timestamp; never treat missing data as recovery."""
|
||||
result = deepcopy(document)
|
||||
histories = result.setdefault("histories", {})
|
||||
states = result.setdefault("rule_states", {})
|
||||
latest = result.setdefault("latest", {})
|
||||
for row in rows:
|
||||
latest[row["device_id"]] = row
|
||||
by_id = {row["device_id"]: row for row in rows}
|
||||
changes = []
|
||||
devices = {device["device_id"]: device for device in result.get("devices", [])}
|
||||
weighted = weighted_diagnosis(result)
|
||||
calibration = current_calibration(result, config.start_time) if weighted else None
|
||||
for device_id in config.device_ids:
|
||||
row = by_id.get(device_id)
|
||||
invalid = row is not None and row["quality"] != "valid"
|
||||
previous = latest.get(device_id)
|
||||
frequency = str(devices.get(device_id, {}).get("transmission_frequency", ""))
|
||||
match = re.fullmatch(r"(\d+):(\d+):(\d+)", frequency)
|
||||
if match:
|
||||
seconds = int(match[1]) * 3600 + int(match[2]) * 60 + int(match[3])
|
||||
else:
|
||||
minutes = re.fullmatch(r"(\d+)\s*(?:min|分钟)", frequency)
|
||||
seconds = int(minutes[1]) * 60 if minutes else 0
|
||||
prior_time = datetime.fromisoformat(previous["time"]) if previous else config.start_time
|
||||
missing = not row and seconds > 0 and (at - prior_time).total_seconds() >= seconds * 2
|
||||
key = "quality-" + device_id
|
||||
state = states.setdefault(key, {"event": None, "recoveries": 0})
|
||||
if invalid or missing:
|
||||
state["recoveries"] = 0
|
||||
if state["event"] is None:
|
||||
event = {"event_id": str(uuid5(NAMESPACE_URL, f"{result['replay_id']}:{key}:{at.isoformat()}")),
|
||||
"rule_id": key, "device_ids": [device_id], "occurred_at": at.isoformat(),
|
||||
"last_observed_at": at.isoformat(), "status": "active", "category": "data_quality",
|
||||
"detail": row["reason"] if invalid else "连续两个设备周期未读取到观测",
|
||||
"task_id": None}
|
||||
state["event"] = event
|
||||
changes.append({**event, "change": "raised"})
|
||||
result["anomaly_count"] = result.get("anomaly_count", 0) + 1
|
||||
elif row and row["quality"] == "valid" and state["event"]:
|
||||
state["recoveries"] += 1
|
||||
if state["recoveries"] >= 2:
|
||||
changes.append({**state["event"], "status": "resolved", "resolved_at": at.isoformat(), "change": "resolved"})
|
||||
state["event"] = None
|
||||
for rule in config.rules:
|
||||
state_key = f"{rule.rule_id}:{DIAGNOSTIC_POLICY}" if weighted and rule.kind == "relative_change" else rule.rule_id
|
||||
state = states.setdefault(state_key, {"hits": 0, "recoveries": 0, "event": None})
|
||||
hits, usable, reasons = [], [], []
|
||||
for device_id in rule.device_ids:
|
||||
row = by_id.get(device_id)
|
||||
if not row or row["quality"] != "valid":
|
||||
continue
|
||||
value = row["cleaned_value"]
|
||||
if rule.kind == "bounds":
|
||||
triggered = ((rule.lower is not None and value < rule.lower)
|
||||
or (rule.upper is not None and value > rule.upper))
|
||||
reason = f"{device_id}={value} {rule.unit},超出配置范围"
|
||||
elif weighted:
|
||||
# The percentage in the compatibility rule is not a diagnosis
|
||||
# threshold. Learn each device's normal same-phase range first.
|
||||
profile = (calibration or {}).get("diagnostic", {}).get(device_id)
|
||||
bounds = normal_profile_bounds(profile, at)
|
||||
if bounds is None:
|
||||
continue
|
||||
low, high = bounds
|
||||
triggered = (value < low or value > high if rule.direction == "both"
|
||||
else value > high if rule.direction == "increase" else value < low)
|
||||
reason = f"{device_id}={value} {rule.unit},超出冻结正常同时段范围 [{low:.4f}, {high:.4f}] {rule.unit}"
|
||||
else:
|
||||
previous = histories.get(device_id, [])[-rule.window_samples:]
|
||||
if len(previous) < rule.minimum_samples:
|
||||
continue
|
||||
baseline = median(previous)
|
||||
if baseline == 0:
|
||||
continue
|
||||
change = (value - baseline) / abs(baseline) * 100
|
||||
triggered = (abs(change) >= rule.threshold if rule.direction == "both"
|
||||
else change >= rule.threshold if rule.direction == "increase"
|
||||
else change <= -rule.threshold)
|
||||
reason = f"{device_id} 较历史中位值变化 {change:.1f}%,阈值 {rule.threshold}%"
|
||||
usable.append(device_id)
|
||||
if triggered:
|
||||
hits.append(device_id)
|
||||
reasons.append(reason)
|
||||
if len(hits) >= rule.minimum_devices:
|
||||
state["hits"] += 1
|
||||
state["recoveries"] = 0
|
||||
if state["event"] is None and state["hits"] >= rule.consecutive_samples:
|
||||
event_id = str(uuid5(NAMESPACE_URL, f"{result['replay_id']}:{state_key}:{at.isoformat()}"))
|
||||
event = {"event_id": event_id, "rule_id": rule.rule_id, "device_ids": hits,
|
||||
"occurred_at": at.isoformat(), "last_observed_at": at.isoformat(),
|
||||
"status": "active", "detail": ";".join(reasons)[:2000], "task_id": None}
|
||||
if weighted and rule.kind == "relative_change":
|
||||
event.update(diagnostic_policy=DIAGNOSTIC_POLICY, state_key=state_key)
|
||||
state["event"] = event
|
||||
changes.append({**event, "change": "raised"})
|
||||
result["anomaly_count"] = result.get("anomaly_count", 0) + 1
|
||||
elif state["event"]:
|
||||
state["event"]["last_observed_at"] = at.isoformat()
|
||||
elif len(usable) == len(rule.device_ids):
|
||||
state["hits"] = 0
|
||||
if state["event"]:
|
||||
state["recoveries"] += 1
|
||||
if state["recoveries"] >= rule.recovery_samples:
|
||||
event = {**state["event"], "status": "resolved", "resolved_at": at.isoformat(),
|
||||
"change": "resolved"}
|
||||
changes.append(event)
|
||||
state["event"] = None
|
||||
state["recoveries"] = 0
|
||||
# Incomplete frames cannot count as a consecutive hit/recovery.
|
||||
elif not usable and any(device in by_id for device in rule.device_ids):
|
||||
state["hits"] = state["recoveries"] = 0
|
||||
frozen = {device for rule in config.rules if states[
|
||||
f"{rule.rule_id}:{DIAGNOSTIC_POLICY}" if weighted and rule.kind == "relative_change" else rule.rule_id
|
||||
]["event"]
|
||||
for device in rule.device_ids}
|
||||
for row in rows:
|
||||
if row["quality"] == "valid" and row["device_id"] not in frozen:
|
||||
history = histories.setdefault(row["device_id"], [])
|
||||
history.append(row["cleaned_value"])
|
||||
histories[row["device_id"]] = history[-96:]
|
||||
result["processed_count"] = result.get("processed_count", 0) + len(rows)
|
||||
return result, changes
|
||||
@@ -0,0 +1,102 @@
|
||||
"""Server-enforced replay knowledge bounds and immutable corrected snapshots."""
|
||||
from datetime import datetime
|
||||
|
||||
from fastapi import HTTPException
|
||||
|
||||
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
|
||||
|
||||
|
||||
def check_replay_window(context, start, end):
|
||||
cap = datetime.fromisoformat(context["knowledge_cutoff"])
|
||||
if start is None or end is None or start.tzinfo is None or end.tzinfo is None or start > end or end > cap:
|
||||
raise HTTPException(422, f"回放查询需要明确时间范围,结束不得晚于 {cap.isoformat()}")
|
||||
|
||||
|
||||
def check_snapshot_start(run, start):
|
||||
replay_start = datetime.fromisoformat(run["start_time"])
|
||||
if start < replay_start:
|
||||
baseline = run.get("baseline")
|
||||
if not baseline:
|
||||
raise HTTPException(422, "本轮回放未冻结历史对照,请新建回放后查询历史基线")
|
||||
if start < datetime.fromisoformat(baseline["start_time"]):
|
||||
raise HTTPException(422, "查询起点早于本轮已冻结的历史基线")
|
||||
|
||||
|
||||
async def replay_snapshot_rows(connection, project_id, context, device_ids, start, end, limit, offset=0):
|
||||
check_replay_window(context, start, end)
|
||||
repository = ScadaReplayRepository(connection, project_id)
|
||||
run = await repository.get(context["replay_id"])
|
||||
if not set(device_ids) <= set(run["config"]["device_ids"]):
|
||||
raise HTTPException(403, "设备或回放不属于当前调查")
|
||||
check_snapshot_start(run, start)
|
||||
rows = await repository.detected_outputs(
|
||||
context["replay_id"], device_ids, start, end, context["batch_sequence"],
|
||||
limit=limit, offset=offset,
|
||||
)
|
||||
return [{key: row[key] for key in (
|
||||
"time", "device_id", "monitored_value", "cleaned_value", "quality", "reason"
|
||||
)} for row in rows]
|
||||
|
||||
|
||||
async def replay_burst_location_inputs(connection, project_id, context, args):
|
||||
"""Build location series from immutable replay outputs, never live TSDB rows."""
|
||||
if args.data_source != "monitoring":
|
||||
raise HTTPException(422, "回放定位只支持 monitoring 数据源")
|
||||
if args.scada_normal_start is None or args.scada_normal_end is None:
|
||||
raise HTTPException(422, "回放定位必须显式提供已处理范围内的正常对照窗口")
|
||||
check_replay_window(context, args.scada_burst_start, args.scada_burst_end)
|
||||
check_replay_window(context, args.scada_normal_start, args.scada_normal_end)
|
||||
repository = ScadaReplayRepository(connection, project_id)
|
||||
run = await repository.get(context["replay_id"])
|
||||
if run:
|
||||
check_snapshot_start(run, min(args.scada_burst_start, args.scada_normal_start))
|
||||
devices = {item["device_id"]: item for item in run.get("devices", [])}
|
||||
monitoring = run.get("monitoring_config") or {}
|
||||
pressure_devices = monitoring.get("pressure_device_ids", [])
|
||||
pressure_map = {str(devices[item]["node_id"]): item for item in pressure_devices
|
||||
if item in devices and devices[item].get("node_id")}
|
||||
requested_pressure = args.pressure_scada_ids or list(pressure_map)
|
||||
if not requested_pressure or not set(requested_pressure) <= set(pressure_map):
|
||||
raise HTTPException(422, "压力节点必须来自本轮已配置的回放监测角色")
|
||||
flow_devices = monitoring.get("outlet_flow_device_ids", [])
|
||||
flow_map = {str(devices[item]["link_id"]): item for item in flow_devices
|
||||
if item in devices and devices[item].get("link_id")}
|
||||
requested_flow = args.flow_scada_ids or []
|
||||
if not set(requested_flow) <= set(flow_map):
|
||||
raise HTTPException(422, "流量要素必须来自本轮已配置的出厂边界流量角色")
|
||||
selected = [pressure_map[item] for item in requested_pressure] + [flow_map[item] for item in requested_flow]
|
||||
rows = await repository.detected_outputs(
|
||||
context["replay_id"], selected,
|
||||
min(args.scada_burst_start, args.scada_normal_start),
|
||||
max(args.scada_burst_end, args.scada_normal_end),
|
||||
context["batch_sequence"],
|
||||
)
|
||||
|
||||
def averages(mapping, start, end):
|
||||
result = {}
|
||||
for element_id, device_id in mapping.items():
|
||||
values = [float(row["cleaned_value"]) for row in rows
|
||||
if row.get("device_id") == device_id
|
||||
and row.get("quality") in {"valid", "corrected"}
|
||||
and row.get("cleaned_value") is not None
|
||||
and start <= datetime.fromisoformat(row["time"]) <= end]
|
||||
if not values:
|
||||
raise HTTPException(422, f"回放窗口缺少 {device_id} 的有效清洗观测")
|
||||
result[element_id] = sum(values) / len(values)
|
||||
return result
|
||||
|
||||
pressure_mapping = {item: pressure_map[item] for item in requested_pressure}
|
||||
flow_mapping = {item: flow_map[item] for item in requested_flow}
|
||||
return {
|
||||
"data_source": "monitoring",
|
||||
"burst_leakage": args.burst_leakage,
|
||||
"pressure_scada_ids": requested_pressure,
|
||||
"burst_pressure": averages(pressure_mapping, args.scada_burst_start, args.scada_burst_end),
|
||||
"normal_pressure": averages(pressure_mapping, args.scada_normal_start, args.scada_normal_end),
|
||||
**({
|
||||
"flow_scada_ids": requested_flow,
|
||||
"burst_flow": averages(flow_mapping, args.scada_burst_start, args.scada_burst_end),
|
||||
"normal_flow": averages(flow_mapping, args.scada_normal_start, args.scada_normal_end),
|
||||
"use_scada_flow": True,
|
||||
} if requested_flow else {}),
|
||||
}
|
||||
@@ -29,6 +29,7 @@ from app.native.wndb.model.sources import (
|
||||
)
|
||||
from app.native.wndb.model.times import get_time, set_time
|
||||
from app.domain.time import parse_clock_duration_seconds
|
||||
from app.infra.db.postgresql.scada import ScadaElementMappings
|
||||
|
||||
|
||||
def _isolated_analysis(purpose: str):
|
||||
@@ -79,6 +80,7 @@ def burst_analysis(
|
||||
username: str | None = None,
|
||||
persist_results: bool = True,
|
||||
_temporary_project: str | None = None,
|
||||
scada_mappings: ScadaElementMappings | None = None,
|
||||
) -> dict | None:
|
||||
"""
|
||||
爆管模拟
|
||||
@@ -123,6 +125,7 @@ def burst_analysis(
|
||||
name=new_name,
|
||||
simulation_type="manually_temporary",
|
||||
modify_pattern_start_time=modify_pattern_start_time,
|
||||
scada_mappings=scada_mappings,
|
||||
)
|
||||
print(
|
||||
datetime.now(pytz.timezone("Asia/Shanghai")).strftime("%Y-%m-%d %H:%M:%S")
|
||||
@@ -203,6 +206,7 @@ def burst_analysis(
|
||||
modify_variable_pump_pattern=modify_variable_pump_pattern,
|
||||
modify_valve_opening=modify_valve_opening,
|
||||
scheme_type="burst_analysis",
|
||||
scada_mappings=scada_mappings,
|
||||
scheme_name=scheme_name,
|
||||
result_db_name=name,
|
||||
scheme_username=username,
|
||||
@@ -312,6 +316,7 @@ def flushing_analysis(
|
||||
valve_control: dict[str, dict] = None,
|
||||
_temporary_project: str | None = None,
|
||||
persist_results: bool = True,
|
||||
scada_mappings: ScadaElementMappings | None = None,
|
||||
) -> dict | str | None:
|
||||
"""
|
||||
管道冲洗模拟
|
||||
@@ -441,6 +446,7 @@ def flushing_analysis(
|
||||
modify_total_duration=modify_total_duration,
|
||||
modify_valve_opening=modify_valve_opening,
|
||||
valve_control=valve_control,
|
||||
scada_mappings=scada_mappings,
|
||||
scheme_type="flushing_analysis",
|
||||
scheme_name=scheme_name,
|
||||
result_db_name=name,
|
||||
@@ -469,6 +475,7 @@ def contaminant_simulation(
|
||||
username: str | None = None,
|
||||
_temporary_project: str | None = None,
|
||||
persist_results: bool = True,
|
||||
scada_mappings: ScadaElementMappings | None = None,
|
||||
) -> dict | str | None:
|
||||
"""
|
||||
污染模拟
|
||||
@@ -591,6 +598,7 @@ def contaminant_simulation(
|
||||
scheme_username=username,
|
||||
scheme_detail=scheme_detail,
|
||||
persist_results=persist_results,
|
||||
scada_mappings=scada_mappings,
|
||||
)
|
||||
return result if not persist_results else None
|
||||
|
||||
|
||||
@@ -13,9 +13,7 @@ if TYPE_CHECKING:
|
||||
|
||||
|
||||
async def list_questions(service: "OperationalTaskService", task_id: UUID) -> list[dict]:
|
||||
# Legacy test repositories do not implement question persistence.
|
||||
reader = getattr(service.repo, "list_questions", None)
|
||||
return await reader(str(task_id)) if reader else []
|
||||
return await service.repo.list_questions(str(task_id))
|
||||
|
||||
|
||||
async def cancel_pending(service: "OperationalTaskService", task_id: UUID, now: datetime) -> None:
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
"""Publication-only validation for same-time-of-day line/scatter comparisons."""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
|
||||
from app.domain.domain_errors import require
|
||||
|
||||
|
||||
DISPLAY_TIMEZONE = timezone(timedelta(hours=8))
|
||||
DAY_SECONDS = 86400
|
||||
|
||||
|
||||
def validate_time_comparison_chart(chart: Mapping[str, Any], *, allow_missing_current: bool = False) -> None:
|
||||
"""Validate new publications; persisted historical charts keep their read contract."""
|
||||
chart_type = chart["chart_type"]
|
||||
series = chart["series"]
|
||||
kinds = {item.get("type") or chart_type for item in series}
|
||||
if chart_type != "time_comparison":
|
||||
require(not {"line", "scatter"} <= kinds, "TIME_COMPARISON_INVALID",
|
||||
"曲线与历史散点复合图须使用 chart_type=time_comparison", status=422)
|
||||
return
|
||||
|
||||
def check(condition: bool, message: str) -> None:
|
||||
require(condition, "TIME_COMPARISON_INVALID", message, status=422)
|
||||
|
||||
kinds = {item.get("type") or "line" for item in series}
|
||||
check(kinds == {"line", "scatter"},
|
||||
"time_comparison 的 series 必须包含当前 line 和历史 scatter,不能混入 bar")
|
||||
labels = chart["x_data"]
|
||||
check(all(re.fullmatch(r"(?:[01][0-9]|2[0-3]):[0-5][0-9](?::[0-5][0-9])?", label)
|
||||
for label in labels),
|
||||
"time_comparison.x_data 只接受 HH:mm 或 HH:mm:ss 时刻;历史日期保留在 sample_times,不能追加到横轴")
|
||||
check(len({len(label) for label in labels}) == 1, "x_data 必须使用一致的时刻精度")
|
||||
seconds = [sum(int(part) * factor for part, factor in zip(label.split(":"), (3600, 60, 1)))
|
||||
for label in labels]
|
||||
offsets = [0]
|
||||
for index in range(1, len(seconds)):
|
||||
step = (seconds[index] - seconds[index - 1]) % DAY_SECONDS
|
||||
check(step > 0, f"x_data[{index}] 重复了上一时刻;历史点应复用当前槽位")
|
||||
offsets.append(offsets[-1] + step)
|
||||
check(offsets[-1] < DAY_SECONDS, "x_data 必须按时间递增且小于一天,不能追加多天历史槽位")
|
||||
|
||||
current_bases: set[datetime] = set()
|
||||
historical_bases: list[datetime] = []
|
||||
for series_index, item in enumerate(series):
|
||||
times = item.get("sample_times")
|
||||
path = f"series[{series_index}]"
|
||||
check(times is not None and len(times) == len(labels), f"{path}.sample_times 必须提供并与 x_data 等长")
|
||||
bases: set[datetime] = set()
|
||||
for index, (value, at) in enumerate(zip(item["data"], times, strict=True)):
|
||||
check((value is None) == (at is None),
|
||||
f"{path}.sample_times[{index}]:每个有效值必须保留真实带时区时间,缺测值和时间均填 null")
|
||||
if at is None:
|
||||
continue
|
||||
try:
|
||||
local = datetime.fromisoformat(at).astimezone(DISPLAY_TIMEZONE)
|
||||
base = local - timedelta(seconds=offsets[index])
|
||||
except OverflowError:
|
||||
check(False, f"{path}.sample_times[{index}] 超出同期对比可表达的日期范围")
|
||||
label = labels[index]
|
||||
expected = local.strftime("%H:%M:%S" if len(label) == 8 else "%H:%M")
|
||||
check(expected == label and local.microsecond == 0 and (len(label) == 8 or local.second == 0),
|
||||
f"{path}.sample_times[{index}] 换算为 UTC+8 后须与 x_data[{index}]={label} 对齐")
|
||||
bases.add(base)
|
||||
check(len(bases) <= 1, f"{path}.sample_times 必须按同一窗口对齐,不能将不同日期拼入同一组")
|
||||
if (item.get("type") or "line") == "line":
|
||||
current_bases.update(bases)
|
||||
else:
|
||||
historical_bases.extend(bases)
|
||||
scada = chart.get("scada")
|
||||
empty_scada = allow_missing_current and not current_bases and scada and scada["role"] == "comparison"
|
||||
check(len(current_bases) == 1 or bool(empty_scada),
|
||||
"当前 line 必须包含真实采样时间,且所有当前曲线使用同一窗口")
|
||||
# SCADA publication separately verifies every null against persisted reads.
|
||||
if empty_scada:
|
||||
window_start = datetime.fromisoformat(scada["window_start"]).astimezone(DISPLAY_TIMEZONE)
|
||||
start_seconds = window_start.hour * 3600 + window_start.minute * 60 + window_start.second
|
||||
current = window_start.replace(microsecond=0) + timedelta(seconds=(seconds[0] - start_seconds) % DAY_SECONDS)
|
||||
else:
|
||||
current = next(iter(current_bases))
|
||||
for historical in historical_bases:
|
||||
delta = (current - historical).total_seconds()
|
||||
check(delta > 0 and delta % DAY_SECONDS == 0,
|
||||
"历史 scatter 必须来自当前窗口之前的对应日期,不能使用当天或未来采样")
|
||||
@@ -1,11 +1,10 @@
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
from typing import Any, Dict, List
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from psycopg import AsyncConnection
|
||||
from uuid import UUID
|
||||
|
||||
from app.algorithms.pipe_health_prediction.survival_predictor import (
|
||||
PipeHealthSurvivalPredictor,
|
||||
@@ -15,14 +14,11 @@ from app.algorithms.scada_cleaning.pressure_series import clean_pressure_data_df
|
||||
from app.infra.db.postgresql.network_assets import NetworkAssetRepository
|
||||
from app.infra.db.postgresql.scada import ScadaInfoRepository
|
||||
from app.infra.db.timescaledb.repositories.realtime import RealtimeRepository
|
||||
from app.infra.db.timescaledb.repositories.analysis import AnalysisResultsRepository
|
||||
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
|
||||
|
||||
class TimeseriesAnalysisService:
|
||||
"""
|
||||
复合查询类,提供跨表查询功能
|
||||
"""
|
||||
"""SCADA 清洗与管道健康预测;历史查询由 TimeseriesHistoryService 负责。"""
|
||||
|
||||
@staticmethod
|
||||
async def _get_project_scada_index(
|
||||
@@ -31,294 +27,6 @@ class TimeseriesAnalysisService:
|
||||
scadas = await ScadaInfoRepository.get_scadas(postgres_conn)
|
||||
return {scada["device_id"]: scada for scada in scadas}
|
||||
|
||||
@staticmethod
|
||||
async def get_scada_associated_realtime_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
postgres_conn: AsyncConnection,
|
||||
device_ids: List[str],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 SCADA 关联的 link/node 模拟值
|
||||
|
||||
根据传入的 SCADA device_ids,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
postgres_conn: PostgreSQL 异步连接
|
||||
device_ids: SCADA 设备ID列表
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 device_id 为键,值为数据列表,每个数据包含 time, value 和 scada_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当 SCADA 设备未找到或字段无效时
|
||||
"""
|
||||
scada_by_id = await TimeseriesAnalysisService._get_project_scada_index(postgres_conn)
|
||||
link_devices: dict[str, str] = {}
|
||||
node_devices: dict[str, str] = {}
|
||||
for device_id in device_ids:
|
||||
target_scada = scada_by_id.get(device_id)
|
||||
if not target_scada:
|
||||
raise ValueError(f"SCADA device {device_id} not found")
|
||||
scada_type = target_scada["device_type"]
|
||||
if scada_type in {"pipe_flow", "flow"}:
|
||||
link_devices[device_id] = target_scada["link_id"]
|
||||
elif scada_type == "pressure":
|
||||
node_devices[device_id] = target_scada["node_id"]
|
||||
else:
|
||||
raise ValueError(f"Unknown SCADA type: {scada_type}")
|
||||
|
||||
link_series = await RealtimeRepository.get_link_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time,
|
||||
list(dict.fromkeys(link_devices.values())), "flow",
|
||||
)
|
||||
node_series = await RealtimeRepository.get_node_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time,
|
||||
list(dict.fromkeys(node_devices.values())), "pressure",
|
||||
)
|
||||
return {
|
||||
device_id: [
|
||||
{**item, "scada_id": device_id}
|
||||
for item in (
|
||||
link_series.get(element_id, [])
|
||||
if device_id in link_devices
|
||||
else node_series.get(element_id, [])
|
||||
)
|
||||
]
|
||||
for device_id, element_id in (link_devices | node_devices).items()
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_scada_associated_analysis_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
postgres_conn: AsyncConnection,
|
||||
device_ids: List[str],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
run_id: UUID,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 SCADA 关联的 link/node 分析模拟值
|
||||
|
||||
根据传入的 SCADA device_ids,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
postgres_conn: PostgreSQL 异步连接
|
||||
device_ids: SCADA 设备ID列表
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 device_id 为键,值为数据列表,每个数据包含 time, value 和 scada_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当 SCADA 设备未找到或字段无效时
|
||||
"""
|
||||
scada_by_id = await TimeseriesAnalysisService._get_project_scada_index(postgres_conn)
|
||||
link_devices: dict[str, str] = {}
|
||||
node_devices: dict[str, str] = {}
|
||||
for device_id in device_ids:
|
||||
target_scada = scada_by_id.get(device_id)
|
||||
if not target_scada:
|
||||
raise ValueError(f"SCADA device {device_id} not found")
|
||||
|
||||
scada_type = target_scada["device_type"]
|
||||
if scada_type in {"pipe_flow", "flow"}:
|
||||
link_devices[device_id] = target_scada["link_id"]
|
||||
elif scada_type == "pressure":
|
||||
node_devices[device_id] = target_scada["node_id"]
|
||||
else:
|
||||
raise ValueError(f"Unknown SCADA type: {scada_type}")
|
||||
|
||||
link_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "link",
|
||||
list(dict.fromkeys(link_devices.values())), start_time, end_time, "flow",
|
||||
)
|
||||
node_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "node",
|
||||
list(dict.fromkeys(node_devices.values())), start_time, end_time, "pressure",
|
||||
)
|
||||
return {
|
||||
device_id: [
|
||||
{**item, "scada_id": device_id}
|
||||
for item in (
|
||||
link_series.get(element_id, [])
|
||||
if device_id in link_devices
|
||||
else node_series.get(element_id, [])
|
||||
)
|
||||
]
|
||||
for device_id, element_id in (link_devices | node_devices).items()
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_realtime_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
feature_infos: List[Tuple[str, str]],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 link/node 模拟值
|
||||
|
||||
根据传入的 feature_infos,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
feature_infos: 传入的 feature 信息列表,包含 (element_id, type)
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 feature_id 为键,值为数据列表,每个数据包含 time, value 和 feature_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当 SCADA 设备未找到或字段无效时
|
||||
"""
|
||||
pipe_ids: list[str] = []
|
||||
junction_ids: list[str] = []
|
||||
for feature_id, feature_type in feature_infos:
|
||||
if feature_type.lower() == "pipe":
|
||||
pipe_ids.append(feature_id)
|
||||
elif feature_type.lower() == "junction":
|
||||
junction_ids.append(feature_id)
|
||||
else:
|
||||
raise ValueError(f"Unknown type: {feature_type}")
|
||||
link_series = await RealtimeRepository.get_link_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time, list(dict.fromkeys(pipe_ids)), "flow"
|
||||
)
|
||||
node_series = await RealtimeRepository.get_node_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time,
|
||||
list(dict.fromkeys(junction_ids)), "pressure",
|
||||
)
|
||||
return {
|
||||
feature_id: [
|
||||
{**item, "feature_id": feature_id}
|
||||
for item in (
|
||||
link_series.get(feature_id, [])
|
||||
if feature_type.lower() == "pipe"
|
||||
else node_series.get(feature_id, [])
|
||||
)
|
||||
]
|
||||
for feature_id, feature_type in feature_infos
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_analysis_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
feature_infos: List[Tuple[str, str]],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
run_id: UUID,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 link/node 分析模拟值
|
||||
|
||||
根据传入的 feature_infos,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
feature_infos: 传入的 feature 信息列表,包含 (element_id, type)
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
run_id: 分析运行 ID
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 feature_id 为键,值为数据列表,每个数据包含 time, value 和 feature_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当类型无效时
|
||||
"""
|
||||
pipe_ids: list[str] = []
|
||||
junction_ids: list[str] = []
|
||||
for feature_id, feature_type in feature_infos:
|
||||
if feature_type.lower() == "pipe":
|
||||
pipe_ids.append(feature_id)
|
||||
elif feature_type.lower() == "junction":
|
||||
junction_ids.append(feature_id)
|
||||
else:
|
||||
raise ValueError(f"Unknown type: {feature_type}")
|
||||
link_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "link", list(dict.fromkeys(pipe_ids)),
|
||||
start_time, end_time, "flow",
|
||||
)
|
||||
node_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "node", list(dict.fromkeys(junction_ids)),
|
||||
start_time, end_time, "pressure",
|
||||
)
|
||||
return {
|
||||
feature_id: [
|
||||
{**item, "feature_id": feature_id}
|
||||
for item in (
|
||||
link_series.get(feature_id, [])
|
||||
if feature_type.lower() == "pipe"
|
||||
else node_series.get(feature_id, [])
|
||||
)
|
||||
]
|
||||
for feature_id, feature_type in feature_infos
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_element_associated_scada_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
postgres_conn: AsyncConnection,
|
||||
element_id: str,
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
use_cleaned: bool = False,
|
||||
) -> Optional[Any]:
|
||||
"""
|
||||
获取 link/node 关联的 SCADA 监测值
|
||||
|
||||
根据传入的 link/node id,匹配 SCADA 信息,
|
||||
如果存在关联的 SCADA device_id,获取实际的监测数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
postgres_conn: PostgreSQL 异步连接
|
||||
element_id: link 或 node 的 ID
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
use_cleaned: 是否使用清洗后的数据 (True: "cleaned_value", False: "monitored_value")
|
||||
|
||||
Returns:
|
||||
SCADA 监测数据值,如果没有找到则返回 None
|
||||
|
||||
Raises:
|
||||
ValueError: 当元素类型无效时
|
||||
"""
|
||||
|
||||
scada_by_id = await TimeseriesAnalysisService._get_project_scada_index(postgres_conn)
|
||||
associated_scada = next(
|
||||
(
|
||||
scada
|
||||
for scada in scada_by_id.values()
|
||||
if (scada.get("node_id") or scada.get("link_id")) == element_id
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
if not associated_scada:
|
||||
return None
|
||||
|
||||
device_id = associated_scada["device_id"]
|
||||
|
||||
data_field = "cleaned_value" if use_cleaned else "monitored_value"
|
||||
|
||||
res = await ScadaRepository.get_scada_field_by_id_time_range(
|
||||
timescale_conn, [device_id], start_time, end_time, data_field
|
||||
)
|
||||
|
||||
return {element_id: res.get(device_id, [])}
|
||||
|
||||
@staticmethod
|
||||
async def clean_scada_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
|
||||
@@ -57,6 +57,33 @@ def _points(
|
||||
]
|
||||
|
||||
|
||||
def _history_series(
|
||||
*, element_type: HistoryElementType, element_id: str,
|
||||
source: HistorySource, source_unit: str, device_id: str | None = None,
|
||||
unit_inferred: bool = False, points: list[HistoryPoint] | None = None,
|
||||
) -> ElementHistorySeries:
|
||||
metric = _target_metric(element_type)
|
||||
return ElementHistorySeries(
|
||||
element_id=element_id, element_type=element_type, device_id=device_id,
|
||||
metric=metric, source=source, source_unit=source_unit,
|
||||
display_unit=display_unit_for_metric(metric.value),
|
||||
unit_inferred=unit_inferred, points=points if points is not None else [],
|
||||
)
|
||||
|
||||
|
||||
async def _resolve_result_units(
|
||||
postgres_conn: AsyncConnection, query: ElementHistoryQuery,
|
||||
) -> tuple[dict[str, Any], bool]:
|
||||
units = await NetworkSettingsRepository.get_result_units(postgres_conn)
|
||||
if query.mode.value != "analysis_comparison":
|
||||
return units, False
|
||||
run = await AnalysisRepository.get_run(postgres_conn, query.run_id)
|
||||
if run is None:
|
||||
raise ValueError(f"analysis run does not exist: {query.run_id}")
|
||||
recorded = (run.get("parameters") or {}).get("result_units")
|
||||
return (recorded, False) if isinstance(recorded, dict) else (units, True)
|
||||
|
||||
|
||||
async def _query_page(timescale_conn, postgres_conn, query, selected_devices, limit, offset):
|
||||
descriptors = []
|
||||
series = []
|
||||
@@ -66,16 +93,14 @@ async def _query_page(timescale_conn, postgres_conn, query, selected_devices, li
|
||||
"source": source, "element_type": element_type,
|
||||
"id": device_id if source == "scada" else element_id, "field": field,
|
||||
})
|
||||
metric = _target_metric(element_type)
|
||||
series.append(ElementHistorySeries(
|
||||
series.append(_history_series(
|
||||
element_id=element_id, element_type=element_type, device_id=device_id,
|
||||
metric=metric, source=(
|
||||
source=(
|
||||
HistorySource.SCADA_RAW if field == "monitored_value" else HistorySource.SCADA_CLEANED
|
||||
) if source == "scada" else (
|
||||
HistorySource.ANALYSIS_SIMULATION if source == "analysis" else HistorySource.REALTIME_SIMULATION
|
||||
),
|
||||
source_unit=unit, display_unit=display_unit_for_metric(metric.value),
|
||||
unit_inferred=unit_inferred, points=[],
|
||||
source_unit=unit, unit_inferred=unit_inferred,
|
||||
))
|
||||
|
||||
# Stable series ordering makes a page crossing raw/cleaned/model boundaries repeatable.
|
||||
@@ -84,17 +109,8 @@ async def _query_page(timescale_conn, postgres_conn, query, selected_devices, li
|
||||
add(source="scada", element_type=element_type, element_id=element_id,
|
||||
device_id=device_id, field=field, unit=device["measurement_unit"])
|
||||
if query.mode.value != "observed":
|
||||
units = await NetworkSettingsRepository.get_result_units(postgres_conn)
|
||||
units, inferred = await _resolve_result_units(postgres_conn, query)
|
||||
source = "analysis" if query.mode.value == "analysis_comparison" else "realtime"
|
||||
inferred = False
|
||||
if source == "analysis":
|
||||
run = await AnalysisRepository.get_run(postgres_conn, query.run_id)
|
||||
if run is None:
|
||||
raise ValueError(f"analysis run does not exist: {query.run_id}")
|
||||
recorded = (run.get("parameters") or {}).get("result_units")
|
||||
inferred = not isinstance(recorded, dict)
|
||||
if not inferred:
|
||||
units = recorded
|
||||
targets = {(t.element_type, t.element_id) for t in query.elements}
|
||||
for element_type, element_id in sorted(targets):
|
||||
metric = _target_metric(element_type)
|
||||
@@ -202,21 +218,18 @@ class TimeseriesHistoryService:
|
||||
rows = scada_by_device.get(device_id, [])
|
||||
if not rows:
|
||||
continue
|
||||
metric = _target_metric(element_type)
|
||||
unit = device["measurement_unit"]
|
||||
for source, field in (
|
||||
(HistorySource.SCADA_RAW, "monitored_value"),
|
||||
(HistorySource.SCADA_CLEANED, "cleaned_value"),
|
||||
):
|
||||
series.append(
|
||||
ElementHistorySeries(
|
||||
_history_series(
|
||||
element_id=element_id,
|
||||
element_type=element_type,
|
||||
device_id=device_id,
|
||||
metric=metric,
|
||||
source=source,
|
||||
source_unit=unit,
|
||||
display_unit=display_unit_for_metric(metric.value),
|
||||
points=_points(
|
||||
rows,
|
||||
value_field=field,
|
||||
@@ -227,15 +240,8 @@ class TimeseriesHistoryService:
|
||||
if query.mode.value == "observed":
|
||||
return ElementHistoryResponse(series=series)
|
||||
|
||||
current_units = await NetworkSettingsRepository.get_result_units(postgres_conn)
|
||||
result_units, unit_inferred = await _resolve_result_units(postgres_conn, query)
|
||||
if query.mode.value == "analysis_comparison":
|
||||
run = await AnalysisRepository.get_run(postgres_conn, query.run_id)
|
||||
if run is None:
|
||||
raise ValueError(f"analysis run does not exist: {query.run_id}")
|
||||
parameters = run.get("parameters") or {}
|
||||
recorded_units = parameters.get("result_units")
|
||||
unit_inferred = not isinstance(recorded_units, dict)
|
||||
result_units = recorded_units if not unit_inferred else current_units
|
||||
link_values = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn,
|
||||
query.run_id,
|
||||
@@ -256,8 +262,6 @@ class TimeseriesHistoryService:
|
||||
)
|
||||
source = HistorySource.ANALYSIS_SIMULATION
|
||||
else:
|
||||
unit_inferred = False
|
||||
result_units = current_units
|
||||
link_values = await RealtimeRepository.get_link_fields_by_ids_time_range(
|
||||
timescale_conn,
|
||||
query.start_time,
|
||||
@@ -285,13 +289,11 @@ class TimeseriesHistoryService:
|
||||
continue
|
||||
source_unit = str(result_units.get(metric.value, "")).strip()
|
||||
series.append(
|
||||
ElementHistorySeries(
|
||||
_history_series(
|
||||
element_id=target.element_id,
|
||||
element_type=target.element_type,
|
||||
metric=metric,
|
||||
source=source,
|
||||
source_unit=source_unit,
|
||||
display_unit=display_unit_for_metric(metric.value),
|
||||
unit_inferred=unit_inferred,
|
||||
points=_points(
|
||||
rows,
|
||||
|
||||
@@ -310,9 +310,18 @@
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar"
|
||||
"bar",
|
||||
"time_comparison"
|
||||
]
|
||||
},
|
||||
"x_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"y_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"x_data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
@@ -331,10 +340,43 @@
|
||||
"type": "string",
|
||||
"maxLength": 200
|
||||
},
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar",
|
||||
"scatter"
|
||||
]
|
||||
},
|
||||
"data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "number"
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
},
|
||||
"sample_times": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"contracts": {
|
||||
"server": {
|
||||
"file": "server-v1.openapi.json",
|
||||
"sha256": "fa5b186d0f1e4257352a812e578cb9901baf47beaca33c388e876f1a7b4c42ae"
|
||||
"sha256": "c31e9b4c71216951d1525acf1bcb8aeba4158b6cdc6b0b9dc01be02f9b6e6af3"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1687,6 +1687,18 @@
|
||||
],
|
||||
"title": "Location"
|
||||
},
|
||||
"occurred_at": {
|
||||
"anyOf": [
|
||||
{
|
||||
"format": "date-time",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Occurred At"
|
||||
},
|
||||
"priority": {
|
||||
"anyOf": [
|
||||
{
|
||||
|
||||
+2877
-16
File diff suppressed because it is too large
Load Diff
@@ -207,13 +207,114 @@
|
||||
"type": "string",
|
||||
"maxLength": 200
|
||||
},
|
||||
"scada": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"role": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"current",
|
||||
"comparison"
|
||||
]
|
||||
},
|
||||
"metric": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"pressure",
|
||||
"flow",
|
||||
"quality"
|
||||
]
|
||||
},
|
||||
"unit": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128
|
||||
},
|
||||
"window_start": {
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
"window_end": {
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
"value_field": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"monitored_value",
|
||||
"cleaned_value"
|
||||
]
|
||||
},
|
||||
"gaps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"device_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128
|
||||
},
|
||||
"day_offset": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 7
|
||||
},
|
||||
"missing_count": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 100
|
||||
},
|
||||
"reason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"no_readings",
|
||||
"outside_frozen_scope",
|
||||
"outside_frozen_window",
|
||||
"no_scheduled_samples"
|
||||
]
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"device_id",
|
||||
"day_offset",
|
||||
"missing_count",
|
||||
"reason"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"maxItems": 8
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"role",
|
||||
"metric",
|
||||
"unit",
|
||||
"window_start",
|
||||
"window_end"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"chart_type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar"
|
||||
"bar",
|
||||
"time_comparison"
|
||||
]
|
||||
},
|
||||
"x_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"y_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"x_data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
@@ -232,10 +333,53 @@
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"device_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128
|
||||
},
|
||||
"day_offset": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 7
|
||||
},
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar",
|
||||
"scatter"
|
||||
]
|
||||
},
|
||||
"data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "number"
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
},
|
||||
"sample_times": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# 供水 Server 文档目录
|
||||
|
||||
环境、启动和发布命令见 [仓库 README](../README.md),开发规则见 [AGENTS.md](../AGENTS.md)。整理日期:2026-11-10。下列文档按职责组织,不将历史迁移或本地验证当作现有服务状态。
|
||||
|
||||
## API、认证与数据
|
||||
|
||||
| 主题 | 维护入口 |
|
||||
| --- | --- |
|
||||
| API 命名、字段和生成契约 | [API 风格](api-style.md) · [OpenAPI](../contracts/server-v1.openapi.json) |
|
||||
| 身份、项目成员、管理接口与元数据 | [认证与用户管理](../AUTHENTICATION_AND_USER_MANAGEMENT.md) |
|
||||
| MCP 设计依据、启用与认证配置 | [查询设计](mcp-query-design.md) · [实施与启用](mcp-query-implementation.md) · [Keycloak 配置](mcp-keycloak-setup.md) |
|
||||
| MCP 分析、项目服务和后台计算 | [分析工具](mcp-analysis-tools.md) · [项目服务](mcp-project-services.md) · [计算队列](calculation-queue.md) |
|
||||
| 项目地名与空间覆盖 | [地名搜索](project-place-search.md) · [节点覆盖](node-coverage-tool.md) |
|
||||
| 项目 BizDB 与时序模型 | [数据库架构](../resources/db_v2/DATABASE_ARCHITECTURE.md) · [管网结构](../resources/db_v2/WNDB_STRUCTURE.md) |
|
||||
|
||||
工具目录以 `app/mcp/catalog.py` 为准;启用范围、项目权限和会话用途决定实际可用项,不以文档中的历史工具数量推定。
|
||||
|
||||
## 任务、调查与工作台
|
||||
|
||||
| 主题 | 维护入口 |
|
||||
| --- | --- |
|
||||
| 任务类型、调查轮次与协作 | [工况任务契约](operational-tasks.md) |
|
||||
| Worker、租约、持久消息、外部动作和部署 | [工况工作进程](operational-task-workers.md) |
|
||||
| 进展、展示事件与人工反馈续调 | [调查进展与展示](investigation-progress-and-presentation.md) |
|
||||
| 正式报告与稳定引用 | [调查报告契约](investigation-report-contract.md) |
|
||||
| 爆管工单到处置闭环 | [爆管处置路线](burst-response-workflow.md) |
|
||||
| 工作台运行态、外围工单与告警存储 | [运行态与存储](workbench-runtime.md) |
|
||||
| 事件顺序、SSE 与读取去重 | [工作台事件链路](workbench-event-refinement.md) |
|
||||
| 通知进入、退出与生命周期 | [通知路由](operational-notification-routing.md) |
|
||||
| 迁移门禁与逐项目验证 | [数据库变更门禁](workbench-database-rollout.md) |
|
||||
| 外围工单来源、项目路由和主动同步 | [外围服务](peripheral-sample.md) |
|
||||
|
||||
[旧动态计划协议](dynamic-investigation-plan.md) 用于兼容历史计划,当前任务和报告规则以上述专门契约为准。[15173 UI Keycloak 客户端](integration-ui-keycloak-client.md) 是特定联调环境说明,不能作为全部部署环境的通用配置。
|
||||
|
||||
## SCADA、回放与数据集
|
||||
|
||||
| 主题 | 维护入口 |
|
||||
| --- | --- |
|
||||
| 回放、时间轴、周期监测与调查边界 | [SCADA 回放设计与实现](scada-peripheral-replay-design.md) |
|
||||
| 外围测试样本与数据再现 | [合成样本](peripheral-generated-data.md) |
|
||||
| 凤阳历史爆管模拟 | [凤阳 SCADA 复制记录](fengyang-burst-scada.md) |
|
||||
| 奉贤独立 Agent benchmark | [tjwater_v2 benchmark](tjwater-v2-burst-benchmark.md) |
|
||||
| tjwater2 场景选取与观测 | [多距离与弱信号方案](tjwater2-v2-burst-balanced-design.md) |
|
||||
| tjwater2 水力校准方法 | [爆管模拟校准](tjwater2-v2-burst-simulation-design.md) |
|
||||
| 监测点算法与资源压力取舍 | [灵敏度优化](sensor-sensitivity-optimization.md) |
|
||||
|
||||
这些数据文档分别描述数据准备、算法设计或历史试算,不能互相替代,也不证明目标数据库已经加载或真实 Agent 已完成验收。
|
||||
|
||||
## 合并与历史资料
|
||||
|
||||
- `workbench-operations-storage.md` 已并入运行态文档的“工单与告警存储”。
|
||||
- `peripheral-work-order-events.md` 已并入外围服务文档的“工单事件主动同步”,旧固定库限制已按项目路由实现修订。
|
||||
- [历史命名审查](history/backend-naming-audit.md) 与 [失效接口排查](history/retired-api-investigation.md) 保留原始审查依据,不能作为当前 API 目录。
|
||||
- `scripts/missing_api_endpoints.md` 是历史能力盘点,新增能力前需重新核对当前契约与调用方。
|
||||
|
||||
设计与部署说明保留分工:设计解释选择与边界,运行文档给出可执行配置;不同数据库、不同数据集和不同生命周期的业务对象不为减少文件数而合并。
|
||||
+2
-1
@@ -1,7 +1,8 @@
|
||||
# TJWater REST API v1
|
||||
|
||||
This contract is the public API contract for coordinated TJWater
|
||||
Server, Agent, Frontend, and CLI releases.
|
||||
Server, Agent, and Frontend releases. Legacy CLI material is a historical
|
||||
capability reference, not a current runtime integration.
|
||||
|
||||
- Paths use lowercase kebab-case, have no trailing slash, and identify
|
||||
resources rather than handler actions.
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="720" height="362" viewBox="0 0 720 362" role="img" aria-labelledby="title description">
|
||||
<title id="title">SCADA 演示时间轴 FM 调频盘样式</title>
|
||||
<desc id="description">静态设计示意。选定区间为十月八日八点至十四点,当前时间十点十五分。半透明玻璃刻度窗内细密时间刻度横向排列,中央红色指示线固定,控制栏包含暂停、下一批和六十倍速。</desc>
|
||||
<defs>
|
||||
<linearGradient id="glass" x1="0" y1="0" x2="0" y2="1">
|
||||
<stop offset="0" stop-color="#fff" stop-opacity=".72"/>
|
||||
<stop offset=".5" stop-color="#f7faff" stop-opacity=".4"/>
|
||||
<stop offset="1" stop-color="#fff" stop-opacity=".64"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="edge" x1="0" y1="0" x2="1" y2="0">
|
||||
<stop offset="0" stop-color="#f7f9fc"/>
|
||||
<stop offset=".12" stop-color="#f7f9fc" stop-opacity="0"/>
|
||||
<stop offset=".88" stop-color="#f7f9fc" stop-opacity="0"/>
|
||||
<stop offset="1" stop-color="#f7f9fc"/>
|
||||
</linearGradient>
|
||||
<filter id="blur"><feGaussianBlur stdDeviation="16"/></filter>
|
||||
<filter id="shadow" x="-.1" y="-.3" width="1.2" height="1.6">
|
||||
<feDropShadow dx="0" dy="2" stdDeviation="4" flood-color="#243852" flood-opacity=".1"/>
|
||||
</filter>
|
||||
<clipPath id="window"><rect x="48" y="130" width="624" height="110" rx="12"/></clipPath>
|
||||
</defs>
|
||||
<rect width="720" height="362" rx="18" fill="#e9edf2"/>
|
||||
<rect x="24" y="22" width="672" height="318" rx="16" fill="#fafbfd" stroke="#d7dee7"/>
|
||||
<g font-family="'Noto Sans SC Variable', -apple-system, BlinkMacSystemFont, 'Segoe UI', 'PingFang SC', 'Noto Sans CJK SC', sans-serif" fill="#263347">
|
||||
<text x="48" y="54" font-size="13" font-weight="600">SCADA 时段演示</text>
|
||||
<text x="672" y="54" text-anchor="end" font-size="11" fill="#778398">静态样式示意</text>
|
||||
<text x="48" y="79" font-size="12" fill="#68758a">演示区间</text>
|
||||
<text x="114" y="79" font-size="12">10 月 8 日 08:00 — 14:00</text>
|
||||
<text x="360" y="118" text-anchor="middle" font-size="28" font-weight="500" font-variant-numeric="tabular-nums" letter-spacing="1">10:15:00</text>
|
||||
<rect x="48" y="130" width="624" height="110" rx="12" fill="#edf2f7" filter="url(#shadow)"/>
|
||||
<g clip-path="url(#window)">
|
||||
<g filter="url(#blur)" opacity=".6">
|
||||
<path d="M12 234 L280 132 L423 247 L749 142" fill="none" stroke="#c7d8e7" stroke-width="35"/>
|
||||
<path d="M63 134 L221 219 L438 128 L647 212" fill="none" stroke="#fff" stroke-width="38"/>
|
||||
</g>
|
||||
<rect x="48" y="130" width="624" height="110" fill="url(#glass)"/>
|
||||
<path d="M65 131 H655" stroke="#fff" stroke-width="2" opacity=".9"/>
|
||||
<path d="M49 138 V231 M671 138 V231" stroke="#fff" opacity=".6"/>
|
||||
<path d="M60 237 H660" stroke="#a8b6c8" opacity=".25"/>
|
||||
<g fill="none" stroke="#67778b" stroke-width="1">
|
||||
<path d="M90 166 V197 M180 166 V197 M270 166 V197 M360 166 V197 M450 166 V197 M540 166 V197 M630 166 V197"/>
|
||||
<path d="M60 179 V196 M120 179 V196 M150 179 V196 M210 179 V196 M240 179 V196 M300 179 V196 M330 179 V196 M390 179 V196 M420 179 V196 M480 179 V196 M510 179 V196 M570 179 V196 M600 179 V196 M660 179 V196" opacity=".65"/>
|
||||
</g>
|
||||
<g font-size="11" text-anchor="middle" font-variant-numeric="tabular-nums">
|
||||
<text x="90" y="216">09:30</text><text x="180" y="216">09:45</text>
|
||||
<text x="270" y="216">10:00</text><text x="360" y="216" font-weight="600">10:15</text>
|
||||
<g fill="#7b889a"><text x="450" y="216">10:30</text><text x="540" y="216">10:45</text><text x="630" y="216">11:00</text></g>
|
||||
</g>
|
||||
<rect x="48" y="131" width="624" height="108" fill="url(#edge)"/>
|
||||
</g>
|
||||
<rect x="48.5" y="130.5" width="623" height="109" rx="11.5" fill="none" stroke="#fff" stroke-opacity=".8"/>
|
||||
<path d="M356 141 H364 L360 147 Z" fill="#cf453f"/>
|
||||
<path d="M360 147 V201" stroke="#cf453f" stroke-width="2"/>
|
||||
<rect x="48" y="256" width="88" height="38" rx="8" fill="#4662ba"/>
|
||||
<path d="M65 269 V281 M70 269 V281" stroke="#fff" stroke-width="2"/>
|
||||
<text x="98" y="280" text-anchor="middle" fill="#fff" font-size="12">暂停</text>
|
||||
<rect x="148" y="256" width="88" height="38" rx="8" fill="#fff" stroke="#d7dee7"/>
|
||||
<text x="192" y="280" text-anchor="middle" font-size="12" fill="#7b8799">下一批</text>
|
||||
<rect x="578" y="256" width="94" height="38" rx="8" fill="#fff" stroke="#d7dee7"/>
|
||||
<text x="601" y="280" font-size="12">60×</text>
|
||||
<path d="M651 273 L655 277 L659 273" fill="none" stroke="#64738a" stroke-width="1.5"/>
|
||||
<text x="48" y="319" font-size="11" fill="#68758a">已读取 10:15 · 已检测 10:10</text>
|
||||
<text x="672" y="319" text-anchor="end" font-size="11" fill="#68758a">红线对应当前时间 · 横向拨动可回看</text>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.7 KiB |
+54
-217
@@ -1,248 +1,85 @@
|
||||
# 从爆管上报到处置闭环
|
||||
# 从爆管工单到事件处置闭环
|
||||
|
||||
本文定义爆管调查的完整业务路线,供维护者审阅和验收。运行时指令位于 [burst-report/SKILL.md](../resources/work-order-skills/burst-report/SKILL.md),由 Server 根据绑定任务的 `workflow=burst_report` 经 MCP 分发。
|
||||
工单触发的 `workflow=burst_report` 使用 [爆管事件处理 Skill](../resources/work-order-skills/burst-event-handling/SKILL.md)。Server 按持久化来源选择流程,Agent 接收同一任务的核验上下文;SCADA 异常和历史回放使用 [SCADA Skill](../resources/work-order-skills/scada-anomaly/SKILL.md)。
|
||||
|
||||
重点成果是有依据的事故候选位置、隔离阀门清单和影响范围,以及经审批执行、反馈核验后的闭环记录。Agent 根据证据选择、跳过或重做分支,不按图逐节点机械调用工具。工程图覆盖多轮调查和人工业务;每轮发布的进度图只标注实际进行的工作,不复制整图冒充执行记录。
|
||||
|
||||
## 完整工程图
|
||||
## 固定处理路线
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph intake[工单进入与后台任务]
|
||||
order[用户或外围系统提交爆管上报工单]
|
||||
source[保存原工单及来源版本、发生时间、位置和描述]
|
||||
sync[同步器按来源关联调查及工单投影]
|
||||
queue[Server 保存调查任务和 queued run]
|
||||
claim[Worker 投递,Agent 领取并维持租约]
|
||||
context[读取任务摘要、业务 Skill、历史报告和既有动作]
|
||||
order --> source --> sync --> queue --> claim --> context
|
||||
end
|
||||
|
||||
subgraph investigation[Agent 调查与工程分析]
|
||||
scope[确定事件时刻、地点、调查目标和真实或演练来源]
|
||||
location[核对已有资产编号;必要时解析地点和查询附近管段]
|
||||
monitor[核对相关测点映射、异常窗口、正常对照和数据质量]
|
||||
cover{查询是否实际覆盖事件与对照窗口}
|
||||
more[缩小对象和窗口,按需补页,核对缺测及零值]
|
||||
hypothesis{是否具备定位所需观测和漏水量依据}
|
||||
locate[计算爆管候选排序,保留假设、覆盖范围与证据]
|
||||
candidate[整合上报、资产关联、算法与现场反馈得到候选集合]
|
||||
usable{是否存在可用于隔离评估的候选资产}
|
||||
isolation[逐候选分析拓扑隔离边界、必关阀及受影响节点]
|
||||
isolateok{可隔离且阀门可用性有可核验依据}
|
||||
alternate[纳入已知不可关闭阀门,重新评估替代隔离方案]
|
||||
hydraulic[按需模拟候选关阀组合的压力和流量影响]
|
||||
impact[区分拓扑停水区、水力影响区和有资料支持的用户范围]
|
||||
findings[形成候选位置、阀门清单、影响对比和建议方案]
|
||||
fieldbasis{是否已有足以支持具体现场核验的结论}
|
||||
fielddraft[形成有位置、核验目标及回执要求的现场核验建议]
|
||||
gap[真实关键数据缺失:发布 final insufficient_data 并结束本轮]
|
||||
error[接口、权限或配置异常:记录具体失败并由宿主结束失败运行]
|
||||
scope --> location --> candidate
|
||||
scope --> monitor --> cover
|
||||
cover -->|未覆盖且可补齐| more --> monitor
|
||||
cover -->|已覆盖或确认实际缺失| hypothesis
|
||||
hypothesis -->|具备依据| locate --> candidate
|
||||
hypothesis -->|条件不具备,明确原因| candidate
|
||||
candidate --> usable
|
||||
usable -->|候选有效,标明待现场确认| isolation --> isolateok
|
||||
isolateok -->|有可评估方案| hydraulic --> impact --> findings
|
||||
isolateok -->|已知存在不可关阀| alternate --> isolation
|
||||
isolateok -->|无法形成方案| fieldbasis
|
||||
usable -->|尚无候选| fieldbasis
|
||||
fieldbasis -->|可形成具体核验建议| fielddraft
|
||||
fieldbasis -->|当前判断所需证据确实缺失| gap
|
||||
monitor -->|不可恢复的查询异常| error
|
||||
location -->|不可恢复的必要查询异常| error
|
||||
end
|
||||
|
||||
subgraph approval[报告、审批与现场处置]
|
||||
interim[保存本轮 interim concluded 阶段报告]
|
||||
proposals[创建具体待审批草稿,关联证据和执行前置条件]
|
||||
finalround[保存本轮 final 报告,由宿主结束本轮 run]
|
||||
review[有权限人员审核当前草稿版本]
|
||||
decision{审批结果}
|
||||
revise[驳回理由保存并触发原调查新一轮]
|
||||
submit[Server 通过配置的适配器幂等提交]
|
||||
receipt{外部提交结果}
|
||||
reconcile[使用同一执行请求号核对回执,不重复下发]
|
||||
unsupported[未配置或明确失败:记录原因并重新分析]
|
||||
workkind{本次获批的是核验还是处置}
|
||||
fieldcheck[现场核验准确位置、漏水迹象及资产,回传实际记录]
|
||||
notice[人工实施获批的通知、供水保障和应急安排]
|
||||
isolatefield[现场核验阀门及实际隔离结果]
|
||||
repair[现场定位、止水和抢修,回传实际故障管段及修复记录]
|
||||
clean[按现场风险及获批方案清洗、消毒或冲洗]
|
||||
restore[核验修复与适用复测,按批准方案恢复供水]
|
||||
follow[记录恢复后压力流量、漏水复查及用户回访]
|
||||
findings --> interim
|
||||
fielddraft --> interim
|
||||
interim --> proposals --> finalround --> review --> decision
|
||||
decision -->|驳回| revise
|
||||
decision -->|批准| submit --> receipt
|
||||
receipt -->|结果不明或仍在执行| reconcile --> receipt
|
||||
receipt -->|明确失败或不支持| unsupported
|
||||
receipt -->|工单已创建或设备动作有回执| workkind
|
||||
workkind -->|现场核验事项| fieldcheck
|
||||
workkind -->|有依据的处置事项| notice
|
||||
notice --> isolatefield --> repair --> clean --> restore --> follow
|
||||
end
|
||||
|
||||
subgraph closure[现场反馈、复核与原事件闭环]
|
||||
feedback[外围系统保存关联工单的新反馈和单调版本]
|
||||
requeue[Server 校验来源并为原调查排队新 generation]
|
||||
reread[重新读取现场事实、执行回执和最新监测证据]
|
||||
verify[对照预测与现场结果,复核修复、恢复和风险]
|
||||
faultstate{复核结果支持故障、误报还是尚待核验}
|
||||
falsealarm[核对排除故障的证据及人工确认]
|
||||
closed{处置及闭环证据是否齐全}
|
||||
remaining[列出未完成事项及对应已有或待审批动作]
|
||||
sourceclosed{原上报工单已结案且版本已核对}
|
||||
readyreport[发布具备结案条件报告并结束本轮]
|
||||
closedreport[发布闭环复核 final 报告并结束本轮]
|
||||
closeowner[有权限人员在原工单系统核对并完成原事件结案]
|
||||
closedcase[报告与来源事实一致,确认业务闭环]
|
||||
follow --> feedback --> requeue
|
||||
fieldcheck --> feedback
|
||||
reread --> verify --> faultstate
|
||||
faultstate -->|已确认故障| closed
|
||||
faultstate -->|有依据排除故障| falsealarm --> sourceclosed
|
||||
faultstate -->|尚需核验| remaining
|
||||
closed -->|尚未齐全或复测异常| remaining --> interim
|
||||
closed -->|证据齐全| sourceclosed
|
||||
sourceclosed -->|是| closedreport --> closedcase
|
||||
sourceclosed -->|否| readyreport --> closeowner --> feedback
|
||||
reread -->|现场位置或阀门条件改变| candidate
|
||||
end
|
||||
|
||||
subgraph presentation[贯穿调查的地图、图表与数据整理]
|
||||
progress[发布真实调查进展和 Mermaid 状态]
|
||||
map[高亮候选管段、闪烁建议关阀、展示已有证据覆盖层]
|
||||
chart[打开异常与对照曲线,展示候选及影响对比表]
|
||||
filepermit{会话是否已有文件写入权限}
|
||||
jsonl[按权限整理有来源引用的 JSONL 调查数据]
|
||||
reportui[通过报告 ui_blocks 发布 json-render 结构化界面]
|
||||
sourceview[展示保持证据来源,失败不阻断业务调查]
|
||||
progress --> map --> chart --> filepermit
|
||||
filepermit -->|已获准且需要导出| jsonl --> reportui
|
||||
filepermit -->|未获准或无需导出| reportui
|
||||
reportui --> sourceview
|
||||
end
|
||||
|
||||
context -->|首次或需要重新分析| scope
|
||||
context -->|已有新反馈待复核| reread
|
||||
revise --> requeue
|
||||
unsupported --> requeue
|
||||
requeue --> queue
|
||||
gap -->|补充事实被 Server 接受后才重开| requeue
|
||||
findings -->|无须新增动作| finalonly[无需新增动作时保存 final 报告并结束本轮]
|
||||
scope -.-> progress
|
||||
candidate -.-> map
|
||||
impact -.-> chart
|
||||
verify -.-> reportui
|
||||
flowchart TD
|
||||
order[爆管上报工单] --> context[读取原工单与当前任务]
|
||||
context --> asset[核对地点并关联管网资产]
|
||||
asset --> interim[保存定位阶段报告]
|
||||
interim --> approval[立即发起工单草稿、人工审核派发]
|
||||
interim --> burst[爆管模拟与同刻在线压力差]
|
||||
burst --> coverage[保存并展示爆管后、处置前影响覆盖]
|
||||
coverage --> isolation[分析隔离边界与阀门方案]
|
||||
isolation --> report[补充阶段报告与处置建议]
|
||||
approval --> feedback[接收实际处置与恢复反馈]
|
||||
feedback --> verify[复核影响与结案条件]
|
||||
verify --> close[Server 完成任务并解除告警]
|
||||
verify -->|仍异常或有未完成处置| asset
|
||||
```
|
||||
|
||||
图中的现场处置由实际审批、工单与回执推进;顺序按现场紧迫性和获批方案调整。必要的紧急通知、现场核验和供水保障不应等待可选的完整模型计算。`clean` 为按风险适用的步骤,不能把每次爆管都判断为污染。`hydraulic` 在缺少模型、时序输入或权限时记录具体限制,仍可保留已完成的拓扑隔离结果。
|
||||
1. **接收核对**:保留来源工单ID、单调版本、发生时间、位置、描述、现场反馈和已有动作;后续轮次从新增事实接续。
|
||||
2. **资产关联**:优先核对上报资产编号。坐标或地址通过地理编码、附近资产、属性和连接关系形成有依据的关联;位置歧义列为具体待确认项。
|
||||
3. **并行工单与影响估计**:定位到管道后保存 interim/concluded 阶段报告,随即提出现场核查/抢修 general 工单草稿;复用同一 action_id,人工审核、确认派发保持现有权限和版本校验。待审核或现场处理期间继续独立的影响模拟,不等待模型完成才发起工单。
|
||||
4. **隔离方案**:用 analyze_valve_isolation 分析关联资产,保留必关阀、可选阀、直接隔离与供水失源节点;已知失效阀门用于替代方案分析,现场可操作性列为执行条件。
|
||||
5. **影响说明**:先保留爆管后、处置前的压力下降覆盖步骤,再展示隔离方案的拓扑范围;结合可用SCADA、水力与用户关联说明影响。
|
||||
6. **报告与处置**:定位阶段已发起现场工单,后续复用原动作;更新报告后按新增需要提出通知、保障或恢复等动作草稿;关联现场任务采用 general 工单。审批、派发和现场完成分别核对回执。
|
||||
7. **反馈复核**:已派发的关联工单保持 `waiting_work_order`,全部收到有效 `completed` 反馈后进入 `waiting_scada`。以现场反馈业务时间为 T,取严格晚于 T 的最近一个 SCADA 采样时刻进行复核;正常范围使用事件前15天同采样相位的正常包络。数据未到或仍异常时保持等待,后台工作进程定期复核;通过后持久保存恢复证据并接续 Agent。Agent 最终报告必须引用该证据,运行完成后才能完成任务并解除对应告警。现场资料缺项仅限定未核实细节,不新增本任务的关闭门槛。来源系统另有明确行政结案流程时,按其实际状态另记。
|
||||
|
||||
`requeue` 表示 Server 接受了新业务事实后分配下一轮,不是 Agent 复活旧会话或自行循环等待。原工单结案由原系统有权限人员处理,后台 Agent 目前没有直接关闭原工单的工具。`completed`、`concluded`、动作 `succeeded` 和“业务闭环”分别核对,不能互相推导。
|
||||
SCADA作为事件影响的辅助证据,按目标测点和事件/对照窗口读取。具备水力输入且工具已授权时,使用 analyze_valve_closure 评估给定阀门方案;各步骤记录实际结果、适用范围或具体待补项。
|
||||
|
||||
## 将业务步骤映射为现有能力
|
||||
## 报告固定内容
|
||||
|
||||
以下产品查询与计算均使用 `mcp_tjwater_` 前缀。参数以本轮实际发现的工具 schema 为准。
|
||||
报告包含事件概况、地点与资产关联、隔离方案、影响说明、处置建议和待办。摘要以150字以内为目标,资产和测点明细放表格,完整依据引用证据。
|
||||
|
||||
| 业务动作 | 已有入口 | 必须形成的结果 |
|
||||
| --- | --- | --- |
|
||||
| 接续原任务 | `get_task_context`,依次读取 summary、skill 及所需分节;`get_task_run_history` | 原工单来源、事件时刻、调查代次、流程版本、既有动作与现场反馈 |
|
||||
| 查找地点和资产 | `geocode` / `search_project_places`、`query_nearby_elements`、`get_element_properties`、`get_node_connections` | 实际候选管段或节点、位置来源、匹配理由;邻近不代表故障确认 |
|
||||
| 核对监测 | `list_scada_devices`、`get_scada_device`、`query_scada_readings`、`query_element_history` | 设备和模型节点的映射、异常及对照窗口、单位、缺测、分页覆盖 |
|
||||
| 压力异常侦测 | `run_burst_detection` | 至少 5 个完整压力测点及此前 15 天数据满足时才调用;不把侦测当作定位 |
|
||||
| 爆管候选定位 | `run_burst_location` | 候选排序、算法实际返回的评价值、观测范围、漏水量假设及来源 |
|
||||
| 逐候选隔离 | `analyze_valve_isolation` | `accident_elements`、`isolatable`、`must_close_valves`、`optional_valves`、`affected_nodes`、`affected_node_count` 及已知 `disabled_valves` |
|
||||
| 水力影响 | `analyze_valve_closure` | 指定关阀组合、时间窗、输出对象和模拟压力流量;参数不足时保留明确原因 |
|
||||
| 用户影响 | 已授权资产属性、区域及外围资料查询 | 受影响对象、关联依据和覆盖限制;没有用户资料不估造户数 |
|
||||
| 发布和提议 | `publish_task_progress`、`publish_task_report`、`propose_task_action` | 持久化报告、证据引用和待审批动作;动作不是执行事实 |
|
||||
| 反馈接续 | Server `external-feedback` / `human-evidence` 及 worker | 同一任务的新轮次、真实来源版本、修复和复测证据 |
|
||||
“影响说明”逐项保留时间、对象、单位、范围、来源与实际或预计的性质:
|
||||
|
||||
定位的 `burst_leakage` 必须有依据,单位为 m³/s;`pressure_scada_ids` 是模型节点 ID,不能填设备查询编号,也可省略并使用服务的项目配置。monitoring 模式正常窗可成对省略,使用前一天同窗,但这只是一项对照假设,仍要核对数据有效性。simulation 模式必须引用已有分析运行 UUID,不把在线重算当成已有分析运行。已知事故管段可直接做候选隔离;不必先完成一次没有必要的定位计算,也不能把已知答案作为算法独立定位成功的证明。
|
||||
| 影响项 | 依据与输出 |
|
||||
| --- | --- |
|
||||
| 事件当前影响 | 已查询测点的压力、流量、持续时间和恢复情况 |
|
||||
| 直接隔离范围 | affected_nodes及对应计数、地图覆盖 |
|
||||
| 新增供水失源范围 | supply_affected_nodes及对应计数、地图覆盖 |
|
||||
| 水力影响 | 给定阀门组合、起算时间、模型条件下的压力和流量结果 |
|
||||
| 用户与保障影响 | 有资料关联的用户、重点设施、保障需求;缺少关联时报告节点与区域范围 |
|
||||
|
||||
隔离工具支持明确标注的候选管段,无须等到现场确认才评估方案。多个候选的比较优先分开调用;把多个 ID 一次传入表示联合事故场景,不等同于逐个比较。`disabled_valves` 仅填已有事实支持的不可关阀;空数组不证明现场阀门均可用。`isolatable=false` 不能解释为零影响,即使 `affected_nodes` 为空也须解释返回的计数和限制。
|
||||
变化量基于同口径对象、单位、时间与模型条件计算;现场实际、模型预测和空间示意分别标注。缺少可选模拟参数时,继续形成已有依据的隔离与保障建议。
|
||||
|
||||
## 像调查员一样查询、查看、比较和复核
|
||||
## 工具与入口约束
|
||||
|
||||
每一步先明确待回答的问题,再查小范围资料,观察实际结果后决定下一步。例如:核实地址,查附近管段并查看属性,关联压力点,比较事件前后及正常对照,检查候选,评估关阀影响。只把必要的查询、事实和判断依据写入进度及报告,不保存模型内部推理。
|
||||
工单事件从上报位置开展资产关联。Agent会话策略隐藏 run_burst_location,Server在绑定与后续解析时同步移除其调用能力;普通聊天和SCADA候选定位沿用各自策略。持久计算在执行前重新校验任务绑定。
|
||||
|
||||
- 不满足于“查过 SCADA”:必须确认返回的时间实际覆盖事件。查询 24 小时而只返回当天前 45 分钟时,应缩到事件前后的小窗,按需补页,不直接进入最终判断。
|
||||
- 正常基线、实测、模型模拟、外围演练和人工反馈分别标注;缺测不是零值,异常不是爆管确认,候选评分不是校准后的概率。
|
||||
- 查不到事故管段时先检查地址解析、附近资产与上下游关联;能查询的字段先查询,不能直接全部交给现场。
|
||||
- 既有证据通过 `get_task_evidence` 的有界分页读取,不重复拉取同一批事实。失败、权限拒绝和存储未配置不当作“无异常”或“无数据”。
|
||||
- 可以用最多两个前台 `investigation-reader` 并行核对无依赖的问题。读取、计算、发布和文件权限仍按角色执行;reader 只返回查询事实及真实证据编号。
|
||||
历史 burst_report 任务携带 replay_context 时,Server分发scada-anomaly,时序查询保持 knowledge_cutoff 与 batch_sequence,demo能力提示按冻结窗口返回。
|
||||
|
||||
## 地图与图表操作
|
||||
业务阶段和工单、返回时间、复核回执保存在任务文档 `handling_context` JSON 中,不增加数据库列。外围样板更新工单时可提供带时区的 `occurred_at`,同时必须填写现场反馈;该时间写入活动记录,系统的 `updated_at` 仍是实际记录时间。同步器将现场反馈时间传给调查,避免历史演示误用电脑当前时间。
|
||||
|
||||
后台调查统一调用 `publish_task_presentation`,每个不同命令使用不同 UUID `request_id`,相同命令重试复用原编号。以下为工具已有能力,不需要浏览器自动化。
|
||||
预约时间 `scheduled_at` 可在派单前的工单草稿和已创建工单的编辑表单中调整。实际完工反馈时间与预约安排分别记录;Agent 以实际反馈时间为恢复复核起点。新的有效现场反馈修订完工时间后,旧复核证据保留为历史记录,当前复核起点更新,重新读取严格晚于新反馈时间的最近一个 SCADA 采样时刻。只修改预约安排或其他工单信息,不改写已有现场反馈时间。
|
||||
|
||||
| 调查进展 | command.action | 取值及依据 |
|
||||
| --- | --- | --- |
|
||||
| 找到实际候选资产 | `highlight_features` | `layer=pipes` / `junctions`,`feature_ids` 为已核实的地图要素编号,`effect=steady` |
|
||||
| 正在核查候选或建议关阀 | `highlight_features` | 对应 pipes 或 valves,`effect=pulse`;闪烁只表示调查关注,不代表设备正在操作 |
|
||||
| 查看异常与对照 | `open_chart` | 有限长度的时间和值,引用真实 `evidence_refs`;不同日期的对照对齐方式在标题或来源中说明 |
|
||||
| 查看已有空间结果 | `set_overlay` | 提供 `overlay_id/title/mode/evidence_id`,Server 从证据提取几何,不接收手绘影响区 |
|
||||
| 撤下旧的临时展示 | `close_chart` / `clear_overlay` / `clear_highlights` | 清理自己记录的临时展示;清理失败不修改业务结果 |
|
||||
`POST /operational-tasks/recovery/reconcile` 由现有 operational worker 每轮调用,按项目及任务创建边界处理等待复核的任务;SCADA 读取失败不阻断其他调查派发。复核直接读取项目时序库中完工反馈时间之后最近时刻的 SCADA,不依赖回放状态或推进回放时间。
|
||||
|
||||
支持的图层为 junctions、pipes、valves、pumps、tanks、reservoirs、scada。先核实业务 ID 与地图 feature ID 的映射,不能把设备编号、节点编号与 GIS 编号混用。每次高亮最多 100 个要素,需要更多时说明所显示的子集;不要假定多次命令会累积成完整范围。没有几何时可以按已知 ID 高亮、用表格解释结果,不画包络多边形冒充影响范围。
|
||||
这条恢复链路同样跟踪带 `replay_context` 的调查工单。原异常快照的 `knowledge_cutoff` 和 `batch_sequence` 保留,普通回放查询继续受其限制;工单完工反馈单独触发 Server 的 `ScadaRepository.get_recovery_scada_readings()`,以实际完工业务时间为 T,读取严格晚于 T 的最近一帧。例如 2026-07-16 13:00 完工,15 分钟采样时复核 13:15,而非原异常截止的 08:00。Agent 从 `handling_context.verification_evidence_id` 读取保存的恢复证据,并与原异常证据共同引用,不需放开回放查询或重建复测工单。
|
||||
|
||||
图表最多 100 个横轴值和 8 组数值序列,单位须在系列名称或标题中明确。不能把 null 缺测填零以通过 schema;可以按有数据的区间分图,并保留缺测说明。`recorded` 仅表示请求保存。浏览器未开 Dev 跟随时仍继续调查,不能等待地图显示回执。后台接口目前没有摄像机缩放、平移、鼠标点击或任意图层开关命令;不要调用仅供聊天的 `locate_features`、`zoom_to_map` 或原生浏览器工具替代。
|
||||
已有调查在收到新的有效完工反馈时会补登记成功派发的关联工单。旧回报误用了电脑当前时间时,需在来源工单更正现场反馈业务时间并提交新版本;系统不猜测完工时刻,也不改写原报告或旧证据。
|
||||
|
||||
## JSONL 调查数据与 json-render 展示
|
||||
## 验证
|
||||
|
||||
JSONL 用于整理已经取得的数据与分析结果,不补造监测、阀门、现场反馈或审批。推荐每行一个对象,记录 `record_type/origin/source_evidence_refs`,并按对象保留时间窗、单位、模型或假设、覆盖限制和实际结果。`origin` 沿用来源分类;本地派生记录引用输入证据,并说明转换方法。不要把 JSONL 文件自身当作 Server 已保存的业务证据。
|
||||
检查Skill名称、版本、摘要与正文digest一致、内联体积、角色权限、工单工具隔离和SCADA定位回归。Agent类型检查与会话工具策略测试共同覆盖运行时入口。发布后分别验收工单事件处理报告和SCADA候选定位,原有报告按既有修订保留。
|
||||
|
||||
按内容选择 `event_context`、`monitoring_coverage`、`candidate`、`isolation`、`hydraulic_impact`、`field_feedback`、`closure_check`,无需为没有结果的类别制造空壳文件。候选记录写实际排序及评价值;隔离记录保留建议必关阀、可选阀、已知不可关阀及受影响节点;闭环记录逐项保留已核验、待核验或不适用及原因。JSONL 是导出约定,不新增 MCP 输入字段或服务端数据协议。
|
||||
关联契约:[分析 MCP](mcp-analysis-tools.md)、[调查展示](investigation-progress-and-presentation.md)、[报告契约](investigation-report-contract.md)、[任务与多轮反馈](operational-tasks.md)。
|
||||
|
||||
以下两行只演示文件格式,`synthetic`、`example_only` 与空证据引用表示它们不可用于实际报告或审批,不包含真实分析结果:
|
||||
## 爆管压力下降覆盖与历史还原
|
||||
|
||||
```jsonl
|
||||
{"record_type":"candidate","origin":"synthetic","example_only":true,"source_evidence_refs":[],"element_id":"DEMO-PIPE-01","rank":null,"evaluation_value":null,"status":"not_computed","limitations":["仅说明字段,未执行定位"]}
|
||||
{"record_type":"closure_check","origin":"synthetic","example_only":true,"source_evidence_refs":[],"item":"repair_verified","status":"not_verified","observed_at":null,"limitations":["无现场反馈,不能声称闭环"]}
|
||||
```
|
||||
`run_burst_simulation` 新增可选 `include_impact_coverage=true`,原默认行为保留。指定已定位管道、带时区的事件起算时刻及有来源或明确估计依据的破口面积(cm²)。计算保持隔离临时模型、不保存分析方案、不重算在线基线,不施加新的关阀或抢修操作。
|
||||
|
||||
如果当前会话已获准使用原生文件写入,则在当前 workspace 下写 `files/burst-investigation-<generation>.jsonl`,保留原始证据不变;通过允许的读取工具逐行复核 JSON、行数、对象 ID、时间和单位。只引用宿主绑定的 generation,不能从用户文本自选。不得读取兄弟会话、业务数据库、宿主源码或凭据;需要本地转换时只使用已获准的原生工具及沙箱 Shell,不引入 HTTP、SQL 或自由命令查询桥。
|
||||
在筛选最多20个展示节点之前,以原生模拟全网节点的起算首帧为计算对象,严格读取同一时刻已有在线模拟压力。不同压力单位换算为米水头,下降量为在线压力减爆管压力,互斥划分 `1≤Δ<3 m`、`3≤Δ<5 m`、`Δ≥5 m`。小于1米与缺少可比数据分别计数;错时、缺测、无效值或未知单位不补零、不取邻近值,不从代表测点外推全网。需要 `simulation.run`、`scada.view`,影响比较另需 `simulation.view`。
|
||||
|
||||
当前没有专用的免审批 JSONL 导出工具。文件权限未授予时,不通过 Shell 绕过、不让后台无限等待批准、不声称已生成文件;继续完成报告内的结构化数据,并如实说明文件未保存。恢复写入授权须由有权限的人或宿主处理,Skill 不扩大权限。
|
||||
输出 `impact` 保留每个可比节点的在线压力、爆管压力、下降量、分组、缺测清单、比较时刻和读取来源;`coverage` 和 `map_payload` 使用实际GIS节点坐标生成三档空间包络。缺坐标、覆盖节点超过10000或几何超预算时明确报告,不截取前若干节点冒充完整范围。地图采用黄/橙/红,图例保留每档已绘制/分组节点数。
|
||||
|
||||
图表、表格和指标使用 `publish_task_report.report.ui_blocks`,每块为 `answer-ui@1`,包含独立 `block_id`、`fallback_text` 和 json-render 的 `spec.root/elements`,外层附真实 `evidence_refs`。最多 8 块、总计 400 KB,单块还受现有 schema 限制。优先组织以下内容:
|
||||
爆管模型使用 PDA 需水方式和原有10模型压力单位的服务压力设定;在线模型方式、当前模型时态及在线压力单位推定来源一并保留。在线记录没有独立版本化单位元数据,不能把此估计称为严格历史同模型反事实或现场停水边界。
|
||||
|
||||
| 展示 | 建议组件 | 数据来源 |
|
||||
| --- | --- | --- |
|
||||
| 候选管段及理由 | Table、Evidence、MapLocation | 地点、资产与定位证据;明确候选和已确认的区别 |
|
||||
| 异常及正常对照 | Chart | 同单位的实际序列,说明时间对齐及缺测 |
|
||||
| 关阀与隔离范围 | Table、Metric、MapLocation | 隔离证据、阀门属性及候选场景 |
|
||||
| 水力与用户影响 | Table、Chart | 指定输出节点的模拟结果及真实用户关联资料 |
|
||||
| 处置与闭环核对 | Table、Evidence | 动作回执、现场反馈、恢复监测和原工单版本 |
|
||||
携带 `replay_context` 的调查只开放启用影响覆盖的爆管情景,起算时刻不得超过 `knowledge_cutoff`。使用当前模型的独立假设情景,初始化及扩展模拟均传入空SCADA边界映射,禁止从实时SCADA回填冻结观测;同刻在线结果仅作为计算时读取的模型参考,标明非冻结基线。未来样本属于预测,不能作为已发生的观测。
|
||||
|
||||
只用工具公布的组件属性。JSONL 可作为整理中间格式,但目前没有“把任意 JSONL 直接传给 json-render”的通道;需转换为已注册组件的 JSON 属性。后台不用 `present_answer`,不把原始 JSONL 输出到聊天正文,也不把表单提交当成已保存现场证据。小数据和简单结论可直接文字表达。
|
||||
完成持久计算后,Agent按回执 `data.presentation.command` 用证据编号发布覆盖,并保存三档计数、假设与缺口表。Server从任务所属的不可变证据解析几何,将完整展示命令写入事件。历史播放还原这份命令,前进到隔离范围和后退到事件影响均不重新查询在线模型,也不派单。工单审核、派发与反馈继续由通知和标准工单窗口记录;模型结果补入同一调查,不重复创建工单。
|
||||
|
||||
## 本轮成果与爆管闭环的验收
|
||||
|
||||
报告至少回答下列四类问题,逐项给出实际结果或经过查询证实的具体限制;无需固定排版或创建固定数量的动作。
|
||||
|
||||
1. **位置**:事故已确认还是仅有候选?每个候选的模型/地图编号、匹配或定位依据、排名及未确认事项是什么?
|
||||
2. **隔离**:每个候选场景是否可隔离?应关、可选和已知不可关的阀门分别是什么?有无替代方案及现场前置核验?
|
||||
3. **影响**:拓扑受影响节点、水力压降或失供范围、用户关联分别有哪些实际结果?未覆盖什么?没有水力或用户资料时分别说明,不能推定为零。
|
||||
4. **处置与闭环**:当前可审批的工作是什么?已有哪些动作、审批、提交回执和现场事实?还缺什么反馈才具备结案条件?
|
||||
|
||||
有具体结论和动作建议时,先保存 `interim/concluded`,再 `propose_task_action`,最后保存 `final`。人工执行事项使用 `work_order_type=general`,保留原爆管事件。工单描述写位置、任务内容、依赖、执行前核验、需回传的事实和责任要求;`propose_task_action` 当前 payload 没有独立的 dependencies 或 feedback_requirements 字段,不发明字段。没有客户通知适配器时,将通知作为人工工单事项,不宣称已发送。
|
||||
|
||||
闭环复核至少核对实际故障资产及原因、隔离/阀门实施记录、抢修结果、适用的清洗及水质复测、恢复供水记录和恢复后监测、用户通知及回访、未决动作与遗留风险、原上报工单结案状态。不适用事项必须有事件依据;未收到结果写待核验,不把记录创建时间当现场完成时间。只齐备处置证据而原工单未结案时,结论为“具备结案条件,待原系统确认”,而非“已闭环”。误报或未发现故障的结案也要有支持排除的实际核查与人工确认。
|
||||
|
||||
必要查询失败、权限拒绝和存储未配置属于执行异常,按 schema 修正可纠正问题;仍无法取得当前判断必需的资料时交宿主记录失败。已通过有效查询确认关键数据确实缺失且无法补齐,才发布 `final/insufficient_data`,提供 blocking 缺口和查询依据,建议与待办为空,不在该退出分支再生成补数据工单。能够明确形成一项现场核验结论时可提议具体核验动作,但不能把“尚未开展定位”包装为“已完成定位调查”。
|
||||
|
||||
## 能力边界与验证
|
||||
|
||||
本次交付为工程图和运行时 Skill 文本,使用现有 MCP、发布、审批与反馈能力;不新增自动关单、鼠标自动化、摄像机操作、JSONL 导出接口或数据库字段。Server 目前没有按 Skill 检查定位/隔离成果的强制验收器,本文的业务验收要求仍由 Agent 执行并由人复核。
|
||||
|
||||
当前 Skill 仅通过 `section=skill` 整份分发,没有相对路径 reference 读取协议,因此运行时指令保持自包含,不能要求 Agent 读取本工程文档。内容需控制在既有内联预算内。Server 更新资源后,应按原发布方式让实际 Server 实例取得新文件;Agent 读取时核对 `name/version/content_sha256`,不把本地文件更新当成运行中实例已生效。
|
||||
|
||||
验证包含 Mermaid 语法解析、本地路径和工具 ID 核对、Skill 分发/权限/体积测试及 `git diff --check`。后续真实行为验收应分别覆盖:有效观测和漏水量依据下的定位;已知候选但缺漏水量时的候选隔离;不可关阀替代方案;SCADA 首屏未覆盖事件时继续读取;接口失败明确终止;文件权限缺失不绕过;无浏览器时照常调查;实际反馈触发新轮及最终闭环复核。没有真实模型计算或现场回执,不报告定位准确性或业务闭环已通过。
|
||||
|
||||
关联契约:[分析 MCP](mcp-analysis-tools.md)、[调查展示](investigation-progress-and-presentation.md)、[报告契约](investigation-report-contract.md)、[任务与多轮反馈](operational-tasks.md)、[后台工作进程](operational-task-workers.md)。
|
||||
全网结果超过单次工具回执上限时,默认 `get_calculation_result` 返回标明缩略的计数与模型条件、保存证据资源链接及可发布地图命令。原始逐节点结果和几何保持完整,可通过证据资源或 `data_path=["result","impact","comparisons"]` 分页读取;不将全网计算缩为20个展示节点。
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# 动态调查计划数据契约
|
||||
# 旧动态调查计划兼容契约
|
||||
|
||||
调查任务从工单投诉内容或人工请求创建。Agent 在本次执行中生成 `execution-plan@2`,根据新证据追加修订;数据库保存不可变计划快照的 JSONB 文档,并不保存固定流程模板。任务、Run、证据、访问申请、审核动作和报告各有独立生命周期。计划中的 `gate_ref` 只引用已经持久化的访问申请或动作,不能把审批结果写成模型自行判断的文本。
|
||||
> 历史兼容文档:字段和示例用于读取已有 `execution-plan@1/@2` 快照,不要求当前调查生成或执行这些计划。当前进展与累计图见 [调查进展](investigation-progress-and-presentation.md),Agent 生命周期见 [任务执行](../../next-tjwater-agent/docs/execution-plan.md)。
|
||||
|
||||
早期设计由 Agent 生成 `execution-plan@2` 并按证据追加修订;数据库保存不可变计划快照的 JSONB 文档,不保存固定流程模板。任务、Run、证据、访问申请、审核动作和报告各有独立生命周期。计划中的 `gate_ref` 只引用已经持久化的访问申请或动作,不能把审批结果写成模型自行判断的文本。
|
||||
|
||||
## `execution-plan@2`
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Backend Naming Audit
|
||||
|
||||
> 历史接口盘点,保留旧名称与修正依据。本文中的旧仓库名、URL 和替代建议不作为当前可用接口清单;现行规范见 [API 风格](../api-style.md),实际契约见 [Server OpenAPI](../../contracts/server-v1.openapi.json)。
|
||||
|
||||
DOC-003 audit for the internal `TJWaterServerBinary` backend.
|
||||
|
||||
## Scope
|
||||
@@ -1,5 +1,7 @@
|
||||
# `app/api/v1/endpoints/` 失效 API 排查与修正
|
||||
|
||||
> 历史接口盘点,保留旧名称与修正依据。本文中的旧仓库名、URL 和替代建议不作为当前可用接口清单;现行规范见 [API 风格](../api-style.md),实际契约见 [Server OpenAPI](../../contracts/server-v1.openapi.json)。
|
||||
|
||||
排查范围:`app/api/v1/endpoints/`
|
||||
|
||||
结论:本次共确认 5 个问题接口,处理结果如下:
|
||||
@@ -67,7 +69,7 @@ DMA 计算请使用:
|
||||
|
||||
服务区计算请使用:
|
||||
|
||||
- `/api/v1/calculateservicearea/`
|
||||
- `/api/v1/calculateservicearea/`
|
||||
现在返回全部时间步结果,不再接收 `time_index`
|
||||
|
||||
## 变更文件
|
||||
@@ -1,8 +1,9 @@
|
||||
# 调查进展与实时展示
|
||||
|
||||
调查执行中可持续发布正文和完整 Mermaid 图,尚未形成结论时也能发布。
|
||||
调查执行中可按需整理正文和流程参考图,Server 累计保留路径并生成 Mermaid,尚未形成结论时也能发布。
|
||||
Server 将阅读进展、业务报告和浏览器展示请求分别保存。运行结束仍要求同轮最终报告,
|
||||
进展中的节点状态不参与调度、审批或结束判断。
|
||||
图节点不绑定实际执行任务,标记只解释最近整理的路线、结果和限制;不参与调度、审批、等待或结束判断。
|
||||
路线有实质调整、形成重要结果、需要保留失败尝试或整理报告时更新即可,不要求逐步发布、频繁刷新或及时同步。
|
||||
|
||||
## 进展快照
|
||||
|
||||
@@ -14,37 +15,92 @@ Server 将阅读进展、业务报告和浏览器展示请求分别保存。运
|
||||
"run_id": "<current-run-uuid>",
|
||||
"source_session_id": "<claimed-session-id>",
|
||||
"progress": {
|
||||
"summary": "正在核对投诉时段的监测记录,尚未形成结论。",
|
||||
"diagram": {
|
||||
"title": "当前调查进展",
|
||||
"mermaid": "flowchart LR\nread[核查监测记录] --> judge[判断证据范围]",
|
||||
"node_refs": [{
|
||||
"summary": "整理投诉时段的监测核查路线,尚未形成结论。",
|
||||
"graph_update": {
|
||||
"title": "调查流程参考",
|
||||
"nodes": [{
|
||||
"node_id": "read",
|
||||
"label": "核查监测记录",
|
||||
"status": "running",
|
||||
"status": "reference",
|
||||
"evidence_refs": [],
|
||||
"action_refs": [],
|
||||
"map_targets": [{"source_id": "pipes", "feature_id": "<known-pipe-id>"}]
|
||||
}]
|
||||
}],
|
||||
"edges": []
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
示例中的编号由实际任务和资产替换。正文最长 20,000 字符,`diagram` 可空。
|
||||
示例中的编号由实际任务和资产替换。新发布的 `summary` 最多 300 字符,建议 120 字以内,
|
||||
只说明本轮新增发现、当前依赖和下一步;使用中文业务语言与当地时间,不复制报告或执行参数。
|
||||
详细结果和引用留在图节点、证据与报告明细中,历史正文仍按原值读取。`graph_update` 可省略。
|
||||
图示沿用 [调查报告契约](investigation-report-contract.md) 的 Mermaid 安全限制。
|
||||
节点状态可空,或为 `pending/running/completed/waiting_human/failed/skipped/cancelled`。
|
||||
`map_targets` 只接受 `junctions/pipes/valves/pumps/tanks/reservoirs/scada` 七类来源和要素编号。
|
||||
图中节点引用在本次发布内唯一,证据和动作引用必须属于该任务。跨修订可以修改节点编号、
|
||||
分支和文字,Server 不验证依赖执行顺序或节点完工。
|
||||
节点标记可省略或为 null,均显示为参考步骤;新增量更新只接受以下五种:
|
||||
|
||||
每次传入 `progress` 都完整替换该 run 的快照,传入 `diagram: null` 清除上一版图示。
|
||||
省略 `progress` 保留原快照。同一个请求编号重试只发布一次。
|
||||
| status | 显示 | 含义 |
|
||||
| --- | --- | --- |
|
||||
| reference | 参考步骤 | 路线或分支说明,不断言是否执行 |
|
||||
| recorded | 已有结果 | 已整理出结果或依据,不表示任务、计算或现场动作完成 |
|
||||
| needs_input | 待补充条件 | 记录缺少的资料、权限或人工事实,不控制任务等待 |
|
||||
| failed | 尝试失败 | 保留一次失败尝试及原因,后续路线另建节点 |
|
||||
| not_taken | 未采用路线 | 未采用、不适用或改走其他分支 |
|
||||
|
||||
旧图读取兼容:pending/running 转为 reference,completed 转为 recorded,waiting/waiting_human 转为 needs_input,
|
||||
skipped/cancelled 转为 not_taken;failed 保留。仅规范化读取结果,不改写历史文档,不再显示“正在执行”或“记录时执行中”。
|
||||
REST 与 MCP 使用相同的旧标记映射。MCP 的 progress/report 图示在分页及 data_path 投影之前完成转换,
|
||||
读取整份说明、节点列表或单个 status 时含义一致;实际 task/run/action 状态和旧执行计划状态保持各自语义。
|
||||
`map_targets` 只接受 `junctions/pipes/valves/pumps/tanks/reservoirs/scada` 七类来源和要素编号。
|
||||
图中节点引用唯一,证据和动作引用必须属于该任务。新 Agent 使用下述增量更新,节点编号在任务各轮保持稳定。
|
||||
Server 不验证依赖执行顺序或推断节点完工。
|
||||
|
||||
`progress.graph_update` 包含可选 `title`、`nodes` 和 `edges`。节点沿用 `node_id/label/status/evidence_refs/action_refs/map_targets`,
|
||||
可用 `detail` 记录实际结果或失败原因;连线为 `{source,target,label?}`。仅提交新增或变化的内容,
|
||||
`update_investigation_graph` 合并节点、追加明确连线并生成 Mermaid,保留已有路径和引用。
|
||||
“保留路径”约束图的结构,不冻结已有节点的标记。`nodes` 按 `node_id` 合并:新增节点必须提供 `label`;
|
||||
修改已保存节点时可只提交 `node_id` 与变化字段,无需再提供 `label`、全部节点或连线。
|
||||
省略字段保留原值,`label: null` 也保留名称;`status: null` 显式恢复为参考步骤。
|
||||
证据、动作和地图引用继续按集合累加,空列表不移除已有引用。资料或条件补齐后,应显式修订原 `needs_input` 节点,
|
||||
新增下游节点不会自动改变上游标记,也不根据节点顺序推断执行结果。例如已有节点 `read` 可更新为:
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": "补充资料已核对,更新原核查节点。",
|
||||
"graph_update": {
|
||||
"nodes": [{"node_id": "read", "status": "recorded", "detail": "已核对收到的现场记录。"}]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
上述更新不增加节点或连线,也不改写已归档的报告图。旧图已保存于 `node_refs` 的节点支持相同的局部修订;
|
||||
仅存在于旧 Mermaid 源码、尚无节点元数据的编号仍不能作为新增步骤重用。
|
||||
尝试失败时标记 `failed`,重试或替代路线使用新节点及连线;原失败节点不能改写为成功。
|
||||
仅更新正文或传入 `diagram: null` 均保留累计图,原始 Mermaid 不再由新 Agent 生成。
|
||||
首次增量更新沿用最近保存的旧图,将原源码保存在 `source_mermaid`,保留原分支和形状后追加内容;
|
||||
旧图内节点不得改名或重用为其他步骤。已被旧实现覆盖且未存入报告的内容无法还原。
|
||||
尚未迁移的旧图仍接受旧客户端显式提交的完整图;仅更新正文或传入 `diagram: null` 时自动接管旧图,
|
||||
保留原源码并转为累计图。任务启用累计图后拒绝旧客户端重画覆盖,不提供通过空值清图的入口。
|
||||
省略 `progress` 保留原快照,同一个请求编号重试只发布一次。
|
||||
`workbench.operational_task_progress` 每个任务和 run 保存最新一份快照,独立于七天事件清理。
|
||||
任务详情和 Agent 上下文返回 `latest_progress`,包含 `revision/run_id/execution_generation/`
|
||||
`published_at/source_session_id/progress`。只读取当前执行代次,新轮次尚未发布时返回空。
|
||||
`published_at/source_session_id/progress`。优先返回当前执行代次;新轮次尚未发布时保留最近的累计图及其原始轮次。
|
||||
`get_task_context(section="progress")` 同样允许读取上轮累计图以接续,摘要和图示不证明新轮次已执行。
|
||||
报告发布时由 Server 存入累计图及其已有依据,历史报告保持独立快照;最多保留 100 个节点、200 条连线,超限显式报错,不静默裁剪。
|
||||
快照更新发出 `operational.progress.changed`,前端据此读取最新内容。
|
||||
|
||||
### 旧任务图迁移
|
||||
|
||||
在 Server 仓库运行 `python -m scripts.migrate_investigation_graphs --plan /受保护目录/plan.json --project-id <项目UUID>`;
|
||||
多个项目重复指定 `--project-id`。预览只读数据库,计划文件以私有权限保存原始进度文档和迁移后的完整文档。
|
||||
核对项目、业务库、任务和图数量后,运行 `python -m scripts.migrate_investigation_graphs --apply /受保护目录/plan.json`。
|
||||
连接使用 MetaDB 路由,计划中不保存连接凭据。迁移逐项目事务执行,与正常发布使用相同的事件锁和任务锁,
|
||||
预览后仍为旧图但已发生变化的行会使该项目回滚;重复执行跳过已经接管的图。
|
||||
|
||||
迁移仅更新已有进度图及其 revision,保留原 summary、发布时间、run、会话、节点说明和引用;
|
||||
原 Mermaid 字节保存在 `source_mermaid`,旧节点标记转换为参考标记,不解析或猜测原始连线。
|
||||
历史报告保持原样,通过读取兼容显示新标记。迁移逐项目校验任务、run 和报告整行摘要未变,
|
||||
并发出进度变更事件及写入独立执行回执;不产生报告更新通知。备份和回执不得提交到仓库。
|
||||
|
||||
## 浏览器实时联动
|
||||
|
||||
`POST /api/v1/operational-tasks/{task_id}/presentation-events` 接收
|
||||
@@ -53,6 +109,33 @@ Server 校验当前 run、绑定会话、有效租约和幂等身份,记录 `o
|
||||
响应为 `recorded/event_seq/run_id/execution_generation`,表示展示请求已保存,
|
||||
不表示浏览器已显示或现场动作已执行。
|
||||
|
||||
### 展示表达与阅读
|
||||
|
||||
实时展示与正式报告共用中文业务表达和时间要求,字段沿用现有结构:
|
||||
|
||||
| 字段 | 表达职责 | 长度 |
|
||||
| --- | --- | --- |
|
||||
| purpose | 为什么核查这些对象;不复述动作名称、当前发现或完整报告 | 建议 60 字以内,最多 160 字符 |
|
||||
| narration.stage | 本步所属业务阶段,前端显示中文阶段名称 | 使用既有枚举 |
|
||||
| narration.summary | 本步已有证据支持的关键发现,区分观测、候选、模型与现场反馈 | 建议 80 字以内,最多 160 字符 |
|
||||
| narration.next_step | 实际下一步或等待条件;无后续时省略 | 建议 60 字以内,最多 120 字符 |
|
||||
| narration.evidence_refs | 该发现引用的实际证据,不作为正文展示 | 保持现有引用校验 |
|
||||
|
||||
已有明确分析结果时提供 `narration`;纯视角调整或清除操作不编造发现。
|
||||
目的、发现和后续分别组织,不重复粘贴同一段文字。同一核查目的的批量图表可以复用有依据的
|
||||
相同讲解,保留合并阅读能力;不同发现分别保存,不为合并而抹去结论差异。
|
||||
|
||||
讲解、目的和图表、覆盖层的可见标签均采用纯文本,省略执行字段、版本号、技术状态码、
|
||||
完整 UUID、证据哈希、时区名称、偏移和 ISO 时间戳。必要设备编号、单位与历史日期保留;
|
||||
标题交代对象、指标或预计范围,算法参数留在证据明细。结构化采样时刻、设备标识、数值、
|
||||
几何和证据引用保持原值,不因文字整理改变分析范围。
|
||||
|
||||
Server 的新请求模型、Agent 原生工具及宿主入口校验这些文字规则;历史读取模型保持兼容。
|
||||
前端在当前动作说明中突出已保存的发现,再呈现目的和后续,缺少讲解的历史步骤只显示用途,
|
||||
不推断结论。讲解进入已有滚动阅读区,历史与播放控件留在标题区,长文字不挤出控件。
|
||||
历史选择固定在所选步骤;自动停留时间考虑实际可见的目的、发现与后续文字,
|
||||
用户手动选择的停留时间保持优先。上述文字和播放行为不控制后台调查、审批或现场执行。
|
||||
|
||||
| command.action | 参数 |
|
||||
| --- | --- |
|
||||
| locate_features | layer、feature_ids |
|
||||
@@ -68,7 +151,7 @@ Server 校验当前 run、绑定会话、有效租约和幂等身份,记录 `o
|
||||
|
||||
以上十种动作与 Chat 的 `present_view`、Dev JSONL 共用扁平 `command: {action, ...字段}`。Chat 使用 `frontend-action@2` 和执行回执;任务仍只记录展示请求,Dev 只做本地预览。三者使用独立的地图源、SCADA 图层和图表状态。Dev 的动作数据行必须为 `{"record_type":"presentation","command":{...}}`;界面块行使用 `record_type: ui_block`,新示例明确带 `phase: final`。普通调查数据保持来源字段,不执行其中的 action。仓库示例见[前端统一展示夹具](../../next-tjwater-frontend/tests/fixtures/unified-presentation.jsonl),其中编号来自测试夹具。
|
||||
|
||||
图表限定折线和柱形,包含 `title/chart_type/x_data/series`。最多 100 个横轴值和 8 组有限数值序列,
|
||||
图表使用普通折线 line、柱状 bar,或当前曲线加历史散点的 time_comparison,包含 `title/chart_type/x_data/series`。最多 100 个横轴值和 8 组有限数值序列,
|
||||
序列长度与横轴对齐,引用证据必须属于当前任务。地图高亮来源沿用上述七类。
|
||||
覆盖层的权威数据来自不可变证据。省略 `data` 时,Server 从证据快照提取现有 GeoJSON,
|
||||
或使用已记录的 `longitude/latitude` 点位,将 `FeatureCollection` 加入事件中的 `command.data`。
|
||||
@@ -77,6 +160,84 @@ Server 校验当前 run、绑定会话、有效租约和幂等身份,记录 `o
|
||||
没有坐标的地址、空数据、非法几何、模型自绘区域均不能生成覆盖层。该接口不做地址解析、
|
||||
范围推算或污染传播计算。展示事件总大小上限为 512 KiB,超限返回 422。
|
||||
|
||||
`generate_node_coverage` 是独立的节点空间覆盖工具,适用于普通项目聊天和后台调查,
|
||||
不绑定爆管、阀门隔离或固定执行顺序。调用者提供需要比较的节点分组,Server 从当前项目
|
||||
`gis.node_geometries` 读取真实位置。监测设备、投诉、地址等须先关联模型节点,
|
||||
不接受模型自由生成的坐标。具体适用范围和参数见[节点覆盖工具](node-coverage-tool.md)。
|
||||
|
||||
计算先投影到局部米制坐标,取凹包(默认 ratio=0.25)、加展示缓冲(默认 25 m),
|
||||
再以 2 m 容差简化并转回 WGS84;凹包比例与缓冲距离可在有界参数内选择。
|
||||
默认展示各组的地理包络,不判断连接关系;需要时可按模型连接拆分,额外指定视为断开的连接。
|
||||
少量点、重合点和共线点同样通过缓冲形成面。多组结果一起保存在 `result.map_payload`,
|
||||
避免连续 `set_overlay` 覆盖前一组;`result.coverage` 保存来源节点、映射完整性及算法参数。
|
||||
该面不推断故障、污染、实际停水、水力影响或用户服务边界,缓冲距离也不是供水服务半径。
|
||||
缺失几何时标记 `partial`,没有可用位置时不生成面。输入最多 8 组、合计 10000 个不同节点;
|
||||
几何输出遵守覆盖层顶点和大小限制,超限要求缩小范围,不静默抽样或改变原分析结果。
|
||||
|
||||
MCP 保存完整查询响应作为不可变证据,展示解析器同时支持响应信封中的
|
||||
`data.result.map_payload`。Agent 使用 `set_overlay` 引用本次 `evidence_id` 并省略 `data`。
|
||||
超过内联返回预算时,响应的 `data.presentation.command` 仍保留可发布的展示命令,
|
||||
`data.presentation.groups` 保留覆盖完整性计数,
|
||||
完整几何留在证据中,不能把缩略返回误认为没有空间成果。调查按实际步骤同步发布展示:
|
||||
地址得到明确坐标后缩放定位,取得候选资产后定位候选,检查影响节点后生成并展示覆盖示意,
|
||||
形成或调整关阀方案后定位阀门,读取监测对照后展示点位及序列。
|
||||
图表同样随结果返回发布:监测趋势与事件对照用折线图,可比候选评价或方案用柱状图,
|
||||
水力及污染传播等模拟按实际返回的时刻、对象和指标画图。压力、流量、浓度等不同单位分图,
|
||||
注明来源、范围与实测或模型属性;没有同口径基准时不计算或绘制改善量,缺测不补零。
|
||||
过程图表通过 `open_chart` 提供 `chart_id/chart/evidence_refs`,每图最多100个横轴值、8组序列,
|
||||
超限时分图或明确说明抽样、聚合范围;单值或无可比数据用文字或表格。
|
||||
同指标、单位和窗口的同期复合图按对象分别发布稳定 chart_id,标题为“对象ID 共同标题”,每条序列名以“对象ID ”开头。共同标题、chart_type、轴名、x_data、当前序列去掉对象前缀后的名称与 sample_times 完全一致时,浏览器将对象数据集聚合为一个子图表:右侧按钮切换对象,下方图例控制序列;窄屏对象列表移到下方。普通趋势总览、柱状图和结构化 Chart 保持独立,不新增 objects 字段。原 chart_id 的 close_chart 只移除对应对象,关闭聚合卡片移除组内对象。
|
||||
|
||||
浏览器的总图表窗口统一管理过程图组与报告 Chart,子图表仍可手动独立弹窗放大。只有正在跟随的运行中任务新收到 open_chart 时可自动弹出总窗口;后续命令更新内容,不自动打开子窗口,用户关闭后不重弹。任务打开、切换、刷新、已完成任务载入、SSE 重连历史补载及历史导航只恢复数据,不弹窗。窗口意图是浏览器临时状态,不写入事件或历史快照,不新增协议字段;open_chart 的内容加载与窗口是否弹出是两个不同状态,Server 的 recorded 也不表示浏览器已经打开窗口。
|
||||
|
||||
time_comparison 的 x_data 使用精度一致、按窗口排列的 UTC+8 HH:mm 或 HH:mm:ss 共享槽,允许跨午夜但短于一天,不追加历史日期。series 同时包含当前 line 与历史 scatter,不混入 bar;sample_times 与 data 等长,有效值和真实带时区时间成对保留,缺测值与时间均为 null。时间转换后须与横轴槽一致,当前线使用同一窗口,每条历史序列来自同一历史日的对应窗口,不接受当天或未来日期。新任务展示与报告发布不满足这些规则时返回 TIME_COMPARISON_INVALID,旧保存内容保持结构读取。普通 line/bar 跳过同期语义,但不能将 line/scatter 复合图标为 line。
|
||||
|
||||
前端维护的脚本 scripts/export-view-command-contract.mjs 同步 Agent/Server 的结构协议及 tests/fixtures 下的统一 JSONL 和 chart-publication-cases.json;JSON Schema 不代替跨字段语义校验。Server 的独立测试用相同正反例校验 Pydantic 边界和 validate_time_comparison_chart;Agent 主机、原生工具及 Dev 的新图表输入也执行对应规则。
|
||||
|
||||
关键趋势与比较结论保留到报告中,最终报告的图表不替代过程发布。
|
||||
|
||||
### SCADA 设备展示覆盖
|
||||
|
||||
SCADA 异常初次调查的监测过程图使用可选的 `chart.scada` 结构:
|
||||
`role=current|comparison`、`metric=pressure|flow`、`unit`、带时区的 `window_start/window_end`,
|
||||
窗口固定为本轮截止时刻之前三小时;采样槽保持真实采样相位,截止非整刻不平移。超过100槽按连续时间分图,使用不同稳定chart_id,元数据仍声明完整三小时窗口;Server逐点核实后合并覆盖,缺少任一槽仍拒绝收尾。同期历史只取完整窗口末三个采样槽,同设备同期分图不得重叠。低频设备窗口无计划采样槽时,完整读取对应当天/历史三小时并核实无读数后,以截止时刻的单个null保留设备入口,标记no_scheduled_samples,不计作漏采。`value_field` 默认为 `monitored_value`,可显式选 `cleaned_value`。
|
||||
每条序列绑定 `device_id` 和 `day_offset`(当天 0,此前七天 1 至 7)。
|
||||
同期图每个设备包含当天 line 和七组 scatter,历史仅末三个对应采样槽有值。
|
||||
Server 校验设备属性、单位、完整采样槽、真实时间与引用快照的对应值,允许数值四舍五入保留 1 至 6 位小数,
|
||||
保留负流量;缺测必须有完整查询或明确空值记录证明,未查询不能视为缺测。
|
||||
`chart.scada.gaps` 由 Server 生成并存入展示事件,前端保留设备入口并展示缺测点数。
|
||||
回放全量查询的 `coverage.frozen_start` 记录服务端已冻结的最早时刻;历史早于该边界时,
|
||||
以本轮匹配 replay/batch 的完整回执证明 `outside_frozen_window`,无需虚构一次成功的历史读取。
|
||||
|
||||
地图的 SCADA 定位/标记、已验证图表和最终报告 `scada_device_ids` 累计本轮目标;
|
||||
监测明细表使用该字段声明完整设备集合。按指标和单位分图,跨多张总览取设备并集,
|
||||
同一 `chart_id` 更新只采用最新覆盖,累计目标不缩小。关闭图表及清空地图只影响浏览器,
|
||||
不撤销产物覆盖。最终报告及成功结束均拒绝遗漏,返回 `409 SCADA_PRESENTATION_INCOMPLETE`,
|
||||
`related_refs` 列出待补设备;Agent 补图后可重试原请求。普通进度和 interim 报告可继续发布。
|
||||
“仅作参考”不豁免展示;确有缺测时仍发布有空槽的图表并保留设备,
|
||||
完整查询无法完成时保留 interim 和具体失败原因,不伪造缺测来通过 final。
|
||||
|
||||
覆盖事实保存在 `workbench.operational_task_runs.artifact_manifest.scada_presentation`,
|
||||
JSON 包含累计 `targets`、按 `chart_id` 保存的最新验证 `charts` 及 `updated_at`。
|
||||
它与 Run 一起保留,独立于七天展示事件清理;不依赖浏览器在线或回执。
|
||||
不同 Run、项目和截止窗口不能抵数。恢复复核阶段沿用恢复证据门禁,不要求重画异常轮次图表。
|
||||
历史读取不要求新字段,已完成报告不回写;旧 Agent 的无绑定图表仍可读,但不能通过新轮次的覆盖门禁。
|
||||
|
||||
部署按[工作台数据库变更门禁](workbench-database-rollout.md)核实目标 BizDB、备份和回滚演练。
|
||||
先部署已兼容内部清单读取的 Server,再暂停目标项目的任务写入并在维护窗口应用
|
||||
[035 迁移](../resources/sql/035_scada_presentation_coverage.sql),核对读取与存储就绪后发布 Agent 和前端。
|
||||
不要在旧 Server 仍处理请求时新增该列:旧版整行 Run 投影会将内部字段传入严格响应模型。
|
||||
新 Server 在迁移前可读取历史任务,SCADA 写入明确返回 503;完成迁移后再恢复目标项目写入。
|
||||
迁移只为 Run 新增通用 JSON 产物清单,默认空对象;不会自动回填旧图或自动迁移其他项目。
|
||||
未应用迁移的 SCADA 写入与收尾返回明确的 503,历史读取保持可用。
|
||||
Run 读取投影显式排除内部清单,REST 与 MCP 原有 Run 结构保持一致。
|
||||
若回退到本次读取兼容修复之前的 Server,须先备份清单再移除新增列;旧代码的整行读取不能直接带着新列回退。
|
||||
本次实现不代表迁移或上线已执行。
|
||||
业务 Skill 说明本步骤应展示什么,调查指令统一说明发布方法、非空几何检查、去重和失败处理。
|
||||
同一阶段批量展示,数据或方案没有变化时不重复请求;不为地图动作增加无关业务分析。
|
||||
报告的 `map_targets` 仅为手动定位入口。关键步骤的数据缺失或展示失败需记录具体限制,
|
||||
没有浏览器连接也不阻止分析或报告收尾。
|
||||
|
||||
`GET /api/v1/operational-tasks/{task_id}/presentation-cursor` 在同一事务中读取
|
||||
`task_id/execution_generation/active_run/event_cursor`,其中活跃 run 仅返回 `run_id/generation/status`。
|
||||
浏览器打开 Dev 联动、切换任务或断线重连时,以此建立新基线,只消费之后的展示事件。
|
||||
@@ -121,3 +282,12 @@ git diff --check
|
||||
```
|
||||
|
||||
这些检查覆盖隔离 HTTP 与状态机,不代替共享数据库迁移、真实模型运行或现场复测验收。
|
||||
|
||||
|
||||
## 工单通知与历史展示
|
||||
|
||||
工单审核通知保存提议时的 `work_order_action` 快照。外部工单的新来源修订在反馈事务中同时保存 `work_order_feedback` 通知与 `operational.notification.changed` 事件,`work_order_reference` 记录来源、修订、证据及状态;关联动作存在时一并保存反馈时的动作快照。重复请求或已处理来源修订不重复生成通知。字段保存在现有通知 JSON 文档中,无新增数据库迁移。
|
||||
|
||||
前端通过事件流刷新通知及关联任务详情,初始快照满页时补取后续通知,并补载首屏任务列表以外的未读通知任务。报告、通知和标准浮动工单窗口定位到同一任务/动作。已读状态与通知生命周期独立,关闭窗口不执行审核或派单。
|
||||
|
||||
历史演示保留已解决通知,并按通知发生时间还原审核、反馈窗口及历史提醒。历史节点只读,不能调用审批、派单或反馈写接口;打开当前工单须退出历史,使用当前动作及来源最新修订。旧通知缺少工单快照时只能标注当前资料,不宣称精确恢复当时的工单。
|
||||
|
||||
@@ -9,17 +9,19 @@
|
||||
|
||||
| 字段 | 类型 | 新写入规则 |
|
||||
| --- | --- | --- |
|
||||
| title | 非空字符串 | 报告标题 |
|
||||
| current_conclusion | 非空字符串 | 当前判断文字;无结论时明确写无法判断,不编造原因 |
|
||||
| conclusion_boundary | 非空字符串 | 判断范围及限制 |
|
||||
| title | 非空字符串,最多 80 字符 | 简短中文业务标题,建议 30 字以内,不包含执行轮次和流程版本 |
|
||||
| current_conclusion | 非空字符串,最多 300 字符 | 首句写当前判断,接关键依据和实际下一步,建议 150 字以内;完整数值放入表格 |
|
||||
| conclusion_boundary | 非空字符串,最多 240 字符 | 影响判断的证据范围、不确定性和适用限制;不复述执行过程、参数或完整资产清单 |
|
||||
| evidence_refs | 字符串数组 | 引用该任务中已持久化、不可变的证据编号 |
|
||||
| recommendations | 字符串数组 | 保持原类型,处置建议不等于已执行动作 |
|
||||
| pending_items | 字符串数组 | 保持原类型,不从文字推断状态 |
|
||||
| recommendations | 字符串数组,每项最多 160 字符 | 具体对象与后续动作;处置建议不等于已执行动作 |
|
||||
| pending_items | 字符串数组,每项最多 160 字符 | 实际尚未完成或等待外部条件的事项,不从文字推断状态 |
|
||||
| stage | interim / final | 必填;阶段性报告不能用于结束运行 |
|
||||
| outcome | concluded / insufficient_data | 必填;业务调查结果,与运行成功/失败独立 |
|
||||
| stop_reason | 非空字符串或 null | 证据不足时必填;已形成结论时必须为 null |
|
||||
| stop_reason | 最多 160 字符的非空字符串或 null | 证据不足时必填;已形成结论时必须为 null |
|
||||
| data_gaps | 缺口对象数组 | 必填;空数组表示已记录且没有缺口 |
|
||||
| diagram | 调查脉络对象或 null | 可选;给人阅读的 Mermaid 说明,不控制执行或任务状态 |
|
||||
| ui_blocks | 结果组件数组,最多 8 块、合计 400 KB | 以现有 Result、Section、Table 等组件表达重点与明细,块内引用必须属于报告证据 |
|
||||
| scada_device_ids | 设备编号数组,最多 1000 项 | 监测明细与展示目标的完整集合,不能因精简摘要删除目标 |
|
||||
|
||||
缺口对象除 node_id 外的字段必填:
|
||||
|
||||
@@ -27,10 +29,10 @@
|
||||
| --- | --- | --- |
|
||||
| gap_id | 非空字符串 | 报告内唯一的缺口编号 |
|
||||
| node_id | 非空字符串或 null | 可选的展示关联,不要求有执行计划 |
|
||||
| missing_data | 非空字符串 | 缺失数据内容 |
|
||||
| scope | 非空字符串 | 查询对象、地点及时间范围 |
|
||||
| impact | 非空字符串 | 因此无法回答的调查问题 |
|
||||
| required_data | 非空字符串 | 新一轮调查需要补齐的具体数据 |
|
||||
| missing_data | 非空字符串,最多 4000 字符 | 缺失的业务资料 |
|
||||
| scope | 非空字符串,最多 4000 字符 | 查询对象、地点及当地时间范围 |
|
||||
| impact | 非空字符串,最多 4000 字符 | 因此无法回答的调查问题 |
|
||||
| required_data | 非空字符串,最多 4000 字符 | 新一轮调查需要补齐的具体数据 |
|
||||
| evidence_refs | 至少一个证据编号 | 实际查询或可信输入记录;必须也出现在报告 evidence_refs 中 |
|
||||
| blocking | boolean | 是否阻止形成结论 |
|
||||
|
||||
@@ -42,11 +44,56 @@
|
||||
不包含 Markdown 围栏、初始化指令和 click 命令。前端使用 strict 安全级别渲染 SVG。
|
||||
完整图语法的解析失败只影响图示,正文、依据和审核仍可阅读。
|
||||
|
||||
## 报告表达与生成要求
|
||||
|
||||
报告面向供水运维人员,按当前判断、关键依据、判断限制、后续安排组织内容。
|
||||
字段仍按现有模型发布,不将多段报告重新堆入 `current_conclusion` 或 `conclusion_boundary`。
|
||||
|
||||
- 结论首句直接回答当前发生了什么、形成什么判断。随后只保留最关键的变化和实际下一步,
|
||||
正文最多举两个关键对象。评分权重、算法参数、全部测点、阀门和影响对象放入依据明细。
|
||||
- 多个对象或多组数值比较使用有证据引用的 `ui_blocks.Table`。表格按实际资料选择对象、
|
||||
指标、时刻、异常值、当前值、基线与结果等列,单位放在列名或单元格;缺值明确写未取得。
|
||||
全部目标按需分表,不以精简正文为由缩小调查、证据或 SCADA 展示覆盖。
|
||||
- 使用 `Result` 突出有依据的判断、关键变化或实际待办,使用带业务标题的 `Section` 组织明细。
|
||||
模型候选或尚待现场核验用 `unconfirmed`,部分完成用 `partial`;`success` 必须有相应通过依据。
|
||||
现有文字字段按纯文本展示,不写 Markdown 加粗、标题、表格或 HTML,也不引入自定义强调字段。
|
||||
- 正文、进展、缺口说明和组件可见文字采用中文业务语言。执行字段、技术状态码、流程版本、
|
||||
完整 UUID 和证据哈希留在结构化引用或执行记录中。必要资产编号、SCADA、DN600、单位、
|
||||
模型相似度和有业务用途的工单简称可以保留,不能机械删除所有英文或数字。
|
||||
- 阅读时间先换算为 `Asia/Shanghai`,再以当地日期和时刻显示,例如 `2026-07-16 13:15`。
|
||||
正文、标题、进展、表格和图表均省略时区名称、偏移及 ISO 原始时间戳。已有 `+08:00`
|
||||
不重复换算,跨日同步更新日期;查询时间、`sample_times` 和证据时间戳仍保留明确偏移。
|
||||
事件发生、工单完工、恢复采样和报告生成时间各自标明用途,不能互相代替。
|
||||
- 区分观测、模型估计、候选和现场反馈。相似度不是发生概率,提交工单不证明现场完工,
|
||||
单帧复核通过不证明长期稳定;不同来源或阶段的设备数分别说明范围,不能混用分母。
|
||||
单条流量规则未触发不表示所有流量未变化;不同影响集合不能直接相加。
|
||||
- 判断边界只保留影响理解的限制。实际资料缺项放入 `data_gaps`;未来常规采样和日常监测
|
||||
写入建议,不将其包装成当前关键缺口。非阻塞资料缺项不得新增事件闭环条件。
|
||||
|
||||
初次调查说明异常、候选及预计影响;等待处置说明已完成事项、实际等待条件和下一步;
|
||||
恢复报告先说明现场反馈与完工时间、恢复复核时刻与结果,再给出必要限制和后续监测安排。
|
||||
恢复明细来自本次恢复证据,原异常分析作为历史依据保留,不在摘要中重复全部定位与隔离参数。
|
||||
|
||||
`progress.summary` 最多 300 字符,建议 120 字以内,只写本轮新增发现、当前依赖和下一步,
|
||||
不复制正式报告、不汇报执行字段和流程版本。整理报告时核对脉络图:已解除的等待条件应按
|
||||
原节点更新业务说明和参考标记,保留失败尝试与原路线,不重复添加相同连线。
|
||||
这不要求逐步更新图示,也不改变任何运行、审批、等待或闭环判据。
|
||||
|
||||
Server 的新写入模型及 Agent 的原生工具、宿主发布入口校验长度、明显执行字段与技术编号、
|
||||
原始时间表达和无法渲染的格式标记;组件的可见文字同样校验,结构化证据编号与采样时间除外。
|
||||
校验错误返回具体原因,由 Agent 依据原证据改写后重试,不截断正文、不删除引用或补造缺口。
|
||||
语义准确性、信息取舍和表格完整性仍需遵循提示词与既有证据、覆盖校验,不能由正则代替。
|
||||
|
||||
上述长度和文字规则仅约束新发布,不应用于历史读取或原证据快照;不回写已有报告。
|
||||
|
||||
每条 node_ref 包含 `node_id`、`label`、`evidence_refs`、`action_refs`;后两项默认空数组。
|
||||
node_id 为最多 80 字符的 ASCII 标识符,必须以字母或下划线开头,在本图引用内唯一,
|
||||
并与 Mermaid 源码中实际节点使用完全相同的 ID;禁止为同一节点另造引用 ID;
|
||||
最多 100 条引用。证据必须来自报告 evidence_refs,动作必须属于当前任务。
|
||||
节点可以说明候选方向与条件关系;图中的分支、状态文字不能证明实际并发、动作执行或现场完成。
|
||||
可选 status 仅为参考标记:reference(参考步骤)、recorded(已有结果)、needs_input(待补充条件)、
|
||||
failed(尝试失败)、not_taken(未采用路线)。recorded 不证明业务完成,标记不与任务或工具运行状态绑定。
|
||||
旧运行状态按[兼容映射](investigation-progress-and-presentation.md)读取,未提供标记时保持中性的参考步骤。
|
||||
图的颜色、字体和交互由前端统一配置。正文不包含 SVG 或 HTML。
|
||||
|
||||
## 业务约束
|
||||
@@ -130,7 +177,7 @@ failure_code 为 ARTIFACT_INCOMPLETE;调用方须读取返回的业务状态
|
||||
"gap_id": "terminal-sample-gap",
|
||||
"node_id": null,
|
||||
"missing_data": "投诉时段末梢水样检测结果",
|
||||
"scope": "投诉小区,2026-09-24 08:00 至 09:00(北京时间)",
|
||||
"scope": "投诉小区,2026-09-24 08:00 至 09:00",
|
||||
"impact": "无法判断异常来自供水管网还是用户端。",
|
||||
"required_data": "该地点和时段的采样检测记录",
|
||||
"evidence_refs": ["sample-query-empty"],
|
||||
@@ -147,10 +194,15 @@ failure_code 为 ARTIFACT_INCOMPLETE;调用方须读取返回的业务状态
|
||||
- 已有未决动作阻止矛盾的缺数退出;重试幂等,不重复生成报告。
|
||||
- 新轮次保留旧报告历史标识,不能显示成当前轮次已结束。
|
||||
- 桌面及 320/375px 展示结果、缺口、证据,审批区不出现数据授权入口。
|
||||
- 新发布拒绝超长摘要、正文执行字段、完整技术编号、原始时间表达和未支持的格式标记;
|
||||
相同历史报告仍可读取,结构化证据引用与带偏移的采样时间保持不变。
|
||||
- 生成结果结论在前、明细入表,明确单位、时间和各自覆盖范围;
|
||||
候选、现场反馈、单帧复核与长期监测保持各自含义,进展和图示不与当前事实冲突。
|
||||
|
||||
## 发布、阅读与协作
|
||||
|
||||
执行中尚未形成结论的正文和 Mermaid 使用独立的 `progress` 快照持续更新,
|
||||
执行中尚未形成结论的正文和 Mermaid 使用独立的 `progress` 快照按需整理,
|
||||
在路线调整、重要结果或报告归纳时更新即可,不要求逐步更新或及时同步。
|
||||
见 [调查进展与实时展示](investigation-progress-and-presentation.md)。
|
||||
报告的 interim/final 与 concluded/insufficient_data 结果契约保持不变。
|
||||
|
||||
|
||||
@@ -5,13 +5,14 @@ Agent 原生工具统一使用 `mcp_tjwater_` 前缀。Server目录共67项:42
|
||||
| 工具名(不含前缀) | 能力与边界 | 必要权限 |
|
||||
| --- | --- | --- |
|
||||
| `run_simulation` | 在线模拟:按水力步长直接计算业务库,不复制数据库;更新模型条件和实时基线,返回指定节点/管段,不创建方案或正式分析运行 | `simulation.run` + `scada.view` |
|
||||
| `run_burst_simulation` | 假设管道破口正向模拟;可选全网同刻在线压力差与1/3/5米水头覆盖,详见[爆管流程](burst-response-workflow.md) | `simulation.run` + `scada.view`,影响比较另需 `simulation.view` |
|
||||
| `run_burst_detection` | 指定时刻的15天压力异常侦测,至少5个完整监测节点;不定位 | `burst.run` + `scada.view` |
|
||||
| `run_burst_location` | 异常/正常压力、可选流量及漏水量假设的候选管段排序 | `burst.run` + 对应数据的 `scada.view` 或 `simulation.view` |
|
||||
| `analyze_valve_isolation` | 拓扑隔离边界、候选关阀及影响节点;不是水力计算或真实关阀 | `simulation.run` |
|
||||
| `run_leakage_identification` | SCADA压力约束下的DMA漏损分配估计;需总漏损量依据 | `burst.run` + `scada.view` |
|
||||
| `run_contaminant_simulation` | 已假设污染源的正向浓度传播;不是源头反演 | `simulation.run` + `scada.view` |
|
||||
| `run_contaminant_simulation` | 假设源正向传播;可选按全网全时段浓度汇总累计范围、首个超阈值报告时刻与峰值,详见[污染流程](water-pollution-response-workflow.md);与工单并行,不承担反演或安全判定 | `simulation.run` + `scada.view` |
|
||||
| `analyze_valve_closure` | 给定关阀组合的水力影响 | `simulation.run` + `scada.view` |
|
||||
| `analyze_flushing` | 指定排放节点和流量的冲洗水力影响;不证明消毒合格 | `simulation.run` + `scada.view` |
|
||||
| `analyze_flushing` | 指定排放点、流量与互斥的 `opened_valves` / `closed_valves`,评估独立冲洗水力情景并保存阀态和预计排放量;不证明污染清除或水质合格 | `simulation.run` + `scada.view` |
|
||||
| `analyze_pressure_regulation` | 给定调泵序列、水箱初始水位的压力影响;不自动优化调度 | `simulation.run` + `scada.view` |
|
||||
| `optimize_sensor_placement` | 灵敏度或K-means压力监测点建议;不保存正式布点方案 | `optimization.run`,项目成员角色 |
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# NextTJWater MCP 查询与认证设计
|
||||
|
||||
日期:2026-10-15。状态:查询与认证代码已在本地实现,认证发现、resource 参数和版本头体校验已补齐,默认关闭,未部署;真实 Keycloak 与项目数据库验收待完成。
|
||||
初稿日期:2026-10-15。本文保留 MCP 查询与认证的设计依据,后续工具扩展及当前启用步骤见 [实施与启用](mcp-query-implementation.md)。早期本地实现、默认配置和验证快照不代表当前线上启用或部署状态;需核对目标环境。
|
||||
|
||||
本设计将现有后端查询能力通过 MCP 开放给普通项目 Agent、调查主 Agent 和调查只读子 Agent。采用 MCP 2026-07-28、Streamable HTTP、现有 Python Server 内挂载服务、可信 Agent 宿主按请求代理认证。旧 CLI 用于查找功能和接口线索,业务实现继续由当前 Server 负责。
|
||||
|
||||
第一版完成查询、权限、取消、分页、调查证据和凭据续期的完整闭环。已有报告发布、工单审批、设备控制、调查运行生命周期继续使用当前受控工具;不通过 MCP 自动开放旧 CLI 中的分析执行命令。
|
||||
本文第一版设计聚焦查询、权限、取消、分页、调查证据和凭据续期。后续已扩展受控分析、预览式写入及后台计算队列,详见实施说明;报告发布、审批、设备控制和调查运行仍遵循各自业务边界,不从旧 CLI 清单自动开放能力。
|
||||
|
||||
## 当前系统与功能核对
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 产品 MCP 查询:本地实施与启用
|
||||
|
||||
实现使用 MCP 2026-07-28、Python mcp 2.3.0、TypeScript @modelcontextprotocol/client 与 @modelcontextprotocol/server 2.3.1。Server 提供 `/mcp` 和71项工具(42项原查询、12项计算、13项预览式写入、4项计算任务管理);Agent 将其投影合并到本地 MCP,由 OpenCode 原生发现为 `mcp_tjwater_*`,普通项目会话、调查主 Agent 和已验证的直接只读子 Agent 共享42项查询;12项计算仅向非reader会话开放,13项写入仅向项目成员的普通聊天开放。个人会话不能获取项目凭据。旧 CLI 只作为能力清单,没有运行时 CLI 桥接。
|
||||
实现使用 MCP 2026-07-28、Python mcp 2.3.0、TypeScript @modelcontextprotocol/client 与 @modelcontextprotocol/server 2.3.1。Server 通过 `/mcp` 提供查询、分析计算、预览式写入和计算任务管理,具体目录以 `app/mcp/catalog.py` 为准;Agent 投影到本地 MCP,由 OpenCode 原生发现为 `mcp_tjwater_*`。普通项目会话、调查主 Agent 和已验证的直接只读子 Agent 在各自授权范围内查询;计算仅向非 reader 会话开放,写入仅向项目成员的普通聊天开放。个人会话不能获取项目凭据。旧 CLI 只作为能力清单,没有运行时 CLI 桥接。
|
||||
|
||||
`run_simulation` 是在线重算,复用 REST 在线计算服务而不复制数据库;按水力步长更新模型条件和实时基线,不创建正式方案。MCP 时长为秒,原 REST 时长仍为分钟。其 annotation 为 `readOnlyHint=false`、`destructiveHint=true`;只读 reader 不能调用,其他方案模拟继续使用隔离临时模型。详细边界见 [MCP 分析工具说明](mcp-analysis-tools.md)。
|
||||
|
||||
@@ -14,9 +14,15 @@ Agent 根据401 challenge 或路径/根 well-known 发现受保护资源元数
|
||||
|
||||
调查数据返回前,在短事务内追加现有任务证据并保存调用回执。同一主体、会话、任务、run 和原生 call ID 的重试复用证据;不同参数或失效范围明确报错。七类原有访问申请 scope 保持原契约,MCP 的新增来源只扩充快照来源枚举。没有新增报告发布、审批、控制、Shell 或递归委派权限。
|
||||
|
||||
普通查询默认20条;目录类最多500条,时序最多10000条、20个标识、24小时;offset 最大10000。Agent 首次优先单对象、必要的短时间窗和小页,仅在证据不足时按需扩大,不自动遍历全网。SQL 有 limit/offset 和 statement_timeout。历史对比在同一条SQL中按序列、时间对数据点分页,返回 page.has_more;非零offset也标明覆盖不完整。历史曲线展示调用初始每页500点,`RESULT_TOO_LARGE` 时原位减半重试,最多读取1000条、发出16次请求,共用120秒超时(单次请求同时受该剩余总预算约束),图表最多100个有效点;这不改变模型首次查询默认值或时序接口上限。
|
||||
普通查询默认20条;目录类最多500条,显式指定设备的时序查询最多10000条、20个标识、24小时;offset 最大10000。SCADA调查首次省略 `query_scada_readings.device_ids`,由Server读取项目完整设备目录及窗口内全部授权观测,内部自动翻页,`offset` 必须为0。其余查询按问题选定对象和窗口。SQL 有 limit/offset 和 statement_timeout。历史对比在同一条SQL中按序列、时间对数据点分页,返回 page.has_more;非零offset也标明覆盖不完整。历史曲线展示调用初始每页500点,`RESULT_TOO_LARGE` 时原位减半重试,最多读取1000条、发出16次请求,共用120秒超时(单次请求同时受该剩余总预算约束),图表最多100个有效点;这不改变模型首次查询默认值或时序接口上限。
|
||||
|
||||
单份结果超过8MiB报错;超过12KB的调查工具消息只返回证据资源引用,完整快照仍保存,Server 的 `resources/read` 可以读取它。Agent 本地 MCP 只提供工具,将上游资源引用转为 `evidence_read` 中的 `mcp_tjwater_get_task_evidence` 参数,不向 OpenCode 泄露不可在本地读取的链接。`get_task_evidence` 在数据库中按 data_path、limit、offset 选页,嵌套集合和超过256字符的字符串返回路径描述;字符串按 Unicode 字符分页。页和完整资源均在数据库返回前检查8MiB上限,保留原证据编号和合成来源;普通查询要求缩小范围。其他结果仍保留执行后的整体8MiB检查,不能把它理解为所有查询均有数据库端字节限制。原始/清洗实测、模型基线、分析结果、比较和 synthetic 均明确标记。非有限数值转为 null 并记录数据缺口。
|
||||
单份结果超过8MiB报错;超过12KB的调查工具消息默认返回证据资源引用,设备目录的精简路径见下段。完整快照仍保存,Server 的 `resources/read` 可以读取它。Agent 本地 MCP 只提供工具,将上游资源引用转为 `evidence_read` 中的 `mcp_tjwater_get_task_evidence` 参数,不向 OpenCode 泄露不可在本地读取的链接。`get_task_evidence` 在数据库中按 data_path、limit、offset 选页,嵌套集合和超过256字符的字符串返回路径描述;字符串按 Unicode 字符分页。页和完整资源均在数据库返回前检查8MiB上限,保留原证据编号和合成来源;普通查询要求缩小范围。其他结果仍保留执行后的整体8MiB检查,不能把它理解为所有查询均有数据库端字节限制。原始/清洗实测、模型基线、分析结果、比较和 synthetic 均明确标记。非有限数值转为 null 并记录数据缺口。
|
||||
|
||||
`list_scada_devices` 默认每页100个、最多500个,其他集合仍沿用各自的分页默认值。设备目录超过12,000字节时,先内联本页全部设备的 `device_id`、`device_type` 和 `measurement_unit`,省略其他属性并明确标注;模型映射、坐标等通过 `get_scada_device` 核验,任务证据和幂等回执保留完整属性。此时 `truncated` 表示属性省略,设备是否还有后续页以 `page.has_more` 为准,按本页实际设备数递增 `offset`。精简目录仍超预算时沿用证据引用或缩页路径。
|
||||
|
||||
全量查询同时要求设备目录与读数权限;通过任务授权读取时,捕获范围及当前批准状态都必须同时包含 `scada_devices` 和 `scada_readings`,显式设备读数查询仍沿用读数授权。
|
||||
|
||||
全量 `query_scada_readings` 返回 `data.devices` 的完整属性、`data.readings` 的原始和清洗值,以及 `data.coverage` 的查询设备、读数数量和缺测清单;该模式保留完整内联数据,不经过12KB缩略,仍执行8MiB整份结果上限。回放仅读取指定批次之前的冻结观测,项目中未纳入该回放的设备单列 `outside_frozen_scope_device_ids`。`query_complete=true` 表示授权范围内的查询已读完,不表示设备均有有效数据。SCADA及历史回放任务发布final时,必须引用窗口覆盖本轮时刻、且回放批次匹配的全量证据;单设备、局部集合、未引用或旧批次证据不能通过此校验。既有报告不回写,后续重跑适用新约束。
|
||||
|
||||
`get_task_context` 默认 section=summary,只查固定元数据、160字符任务标题及200字符结论预览,并返回 available_sections;preview_only 和 truncated 明确说明不是完整报告。按 section=task/report/evidence/actions/progress/access_requests/plan/run 读取详情,report_revision 保留选择旧报告的语义。data_path 选择嵌套值,limit 默认20、最多500,offset 最大10000;证据、动作和授权申请集合以稳定 ID 为键,按 ID 字典序分页。字符串按字符分页。详情在数据库返回前检查8KB预算,超限缩页或选更深路径。缺少可选进度或授权申请表时返回 metadata.available=false,不冒充空事实。绑定权限范围只读取七类既有 scope,不再加载整个任务上下文。不增加工具数量、数据库表或迁移,也不改变原 REST 完整上下文契约。
|
||||
|
||||
@@ -26,7 +32,9 @@ Agent 根据401 challenge 或路径/根 well-known 发现受保护资源元数
|
||||
|
||||
### Server 分发业务处置 Skill
|
||||
|
||||
业务流程源文件位于 `resources/work-order-skills/*/SKILL.md`,随 Server 镜像分发。`get_task_context` 摘要提供 `handling_skill`(适用 workflow、名称、版本和 SHA-256)及 `display_time_zone=Asia/Shanghai`;使用 `section="skill"` 读取完整正文。流程由服务端持久化任务的 workflow 选择,调用方不能传入文件路径或其他任务类型;未知 workflow 返回 `available=false`,不自动套用其他流程。
|
||||
业务流程源文件位于 `resources/work-order-skills/*/SKILL.md`,随 Server 镜像分发。`get_task_context` 摘要提供 `handling_skill`(适用 workflow、名称、版本和 SHA-256)及 `display_time_zone=Asia/Shanghai`;使用 `section="skill"` 读取完整正文。流程由持久化任务的 workflow 与 replay_context 选择:工单 burst_report 使用 burst-event-handling,scada_anomaly 及带回放来源的历史 burst_report 使用 scada-anomaly;未知 workflow 返回 available=false。工单事件处理的绑定工具范围移除 run_burst_location,保留资产关联、隔离和影响分析。
|
||||
|
||||
报告阅读时区统一为 UTC+8(Asia/Shanghai)。MCP 在原始证据和回执保存后,将模型可见结果中的显式带时区时间转换为 `+08:00`,保留同一时刻和源小数秒精度;存储、查询参数及冻结窗口沿用原始时刻,标识符与引用保持原文。前端对报告正文、交互表格、证据卡片、图表提示及数据表采用同一阅读时区,并兼容历史正文中的 `Z`、`UTC` 和 `UTC0` 标记。历史正文中省略时区的时间按读者默认 UTC+8 展示。
|
||||
|
||||
当前覆盖爆管上报、水质投诉与预约跟进。业务 Skill 描述风险核查、隔离停水、用户通知、现场处置和恢复条件;Agent 本地 Skill 维护通用执行方法。权限、审批和执行仍由 Server 代码控制。流程说明没有 evidence_id,不记录成事件证据。具体草稿由现有 `propose_task_action` 保存并等待人工审批;人工通知等步骤通过关联预约工单承接,不能当作通知已发送或现场工作已完成。
|
||||
|
||||
@@ -99,10 +107,10 @@ Agent:
|
||||
|
||||
```bash
|
||||
bun run check
|
||||
bun test tests/product/mcpAuthDiscovery.test.ts tests/product/mcpQueryClient.test.ts tests/product/credentialRefresh.test.ts tests/runtime/task-agent-policy.test.ts tests/runtime/opencodeConfig.test.ts tests/opencode/mcpPolicy.test.ts tests/opencode/productMcp.test.ts
|
||||
bun test tests/product/mcpAuthDiscovery.test.ts tests/product/mcpQueryClient.test.ts tests/product/credentialRefresh.test.ts tests/runtime/task-agent-policy.test.ts tests/runtime/opencodeConfig.test.ts tests/opencode/mcpPolicy.test.ts tests/opencode/queryMcp.test.ts tests/mcp/queryServer.test.ts
|
||||
MCP_TEST_PYTHON=<隔离Server环境Python> MCP_TEST_SERVER_ROOT=<Server仓库绝对路径> bun tests/integration/mcp-query-smoke.ts
|
||||
```
|
||||
|
||||
HTTP smoke 只启动自己的 loopback fixture,使用真实两端 SDK,无生产数据库、Keycloak或模型推理。它验证新版发现、真实资源元数据 HTTP 读取、授权服务器元数据及带 resource 的两种 grant(后两者为桩)、67项工具、主机绑定、查询及撤销;真实环境验收记录见上方实际运行环境一节,不能用 fixture 结果代替。前端严格解析已接受这些 MCP 快照来源,未加入浏览器 MCP 凭据或新 UI。
|
||||
HTTP smoke 只启动自己的 loopback fixture,使用真实两端 SDK,无生产数据库、Keycloak或模型推理。它验证新版发现、真实资源元数据 HTTP 读取、授权服务器元数据及带 resource 的两种 grant(后两者为桩)、对应源码版本的工具目录、主机绑定、查询及撤销;真实环境验收记录见上方实际运行环境一节,不能用 fixture 结果代替。前端严格解析已接受这些 MCP 快照来源,未加入浏览器 MCP 凭据或新 UI。
|
||||
|
||||
查询限制回归使用 `python -m pytest tests/unit/test_mcp_query_limits.py tests/unit/test_mcp_query.py tests/unit/test_timeseries_history.py tests/unit/test_timeseries_history_schema.py -q`。实际 PostgreSQL SQL 验证可设置 `MCP_QUERY_TEST_DSN` 后运行 `python -m pytest tests/integration/test_mcp_query_limits_postgres.py -q`;该 DSN 必须指向空的、可丢弃的独立数据库,测试创建临时业务同名 schema 并在事务结束时回滚。它验证数据点跨序列翻页、证据路径/Unicode分页、任务摘要与指定报告、章节稳定路径、项目/任务隔离、既有scope边界、字节门槛和合成摘要,不代替真实项目验收。
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# 节点空间覆盖工具
|
||||
|
||||
`generate_node_coverage` 增加 Agent 可选择的空间表达能力,将已取得的模型节点集合转为
|
||||
可解释、可复核的覆盖示意。它可以在项目聊天、任务协作和后台调查中使用,
|
||||
不要求爆管任务、隔离成功或特定报告。生成、发布展示和业务分析是三个独立操作。
|
||||
|
||||
## 适用范围
|
||||
|
||||
| 问题 | 输入依据 | 输出含义 |
|
||||
| --- | --- | --- |
|
||||
| 候选对象分布在哪里 | 已核实的候选模型节点 | 候选节点空间包络 |
|
||||
| 两组监测点覆盖有何不同 | 已核对的设备到模型节点映射 | 已接入节点分布比较,不证明监测可观测性 |
|
||||
| 区域成员分布在哪里 | 已查询的区域成员节点 | 成员节点包络,不替代已有行政区或 DMA 边界 |
|
||||
| 哪些节点属于模型影响集合 | 已有拓扑或模拟结果中的节点集合 | 结果节点的空间示意,不重新计算影响 |
|
||||
| 投诉关联的节点如何分布 | 已解析并核对的投诉到模型节点关联 | 已关联节点分布,不证明投诉或污染真实边界 |
|
||||
|
||||
只有地址或未关联设备时,先查询位置和资产关联;已有权威区域几何时优先展示原始边界。
|
||||
本工具不接收任意坐标、HTML、SQL、脚本或调用者指定的项目身份,不执行关阀、模拟或方案保存。
|
||||
|
||||
## 参数和结果
|
||||
|
||||
- `groups`:最多 8 组,每组提供唯一 `group_id`、`title` 和 `node_ids`;组之间可重叠,
|
||||
合计最多 10000 个不同节点。不按组名自动赋予隔离、污染或服务覆盖语义。
|
||||
- `concave_ratio`:0–1,默认 0.25;越接近 1 越接近凸包。分散的点取单一包络可能包含较大空白,
|
||||
应按实际比较目标分组,并明确示意含义。
|
||||
- `buffer_m`:大于 0、最多 1000 m,默认 25 m,仅是展示缓冲。
|
||||
- `split_by_connectivity`:默认 false。true 时按所选节点的模型连接拆分,并排除初始 CLOSED 连接。
|
||||
`excluded_link_ids` 可额外指定最多 200 个断开连接,适用于已有关阀建议等有依据的绘图情景。
|
||||
这不会修改模型,也不能替代完整拓扑分析。
|
||||
|
||||
`result.map_payload` 是多组共存的 GeoJSON;`result.coverage` 记录算法参数、节点 ID、
|
||||
已映射数量、缺失几何节点、分组数量和 `display_only` 边界。缺少部分位置时返回 `partial`,
|
||||
不补造坐标。单点、重合点或共线点通过米制缓冲生成可绘制面。
|
||||
|
||||
## 与展示工具配合
|
||||
|
||||
生成工具不会移动地图。项目聊天可用 `present_view` 展示本次返回的几何;后台调查用
|
||||
`publish_task_presentation` 的 `set_overlay`,引用生成结果自己的 `evidence_id` 并省略 `data`,
|
||||
Server 从不可变证据解析完整几何。大结果内联缩略时,`data.presentation.command` 保留可发布命令。
|
||||
`recorded` 仍只表示请求已记录,浏览器显示和后台调查完成各自独立。
|
||||
|
||||
本次爆管报告只是一个验证样例:可将 24 个直接隔离节点、295 个供水影响节点作为两组,
|
||||
按已有三阀建议设置连通分组;不将这种调用固定进 `analyze_valve_isolation`。
|
||||
@@ -150,7 +150,7 @@ conda run -n server python scripts/sync_peripheral_work_orders.py --once
|
||||
conda run -n server python scripts/sync_peripheral_work_orders.py
|
||||
```
|
||||
|
||||
使用相同工作身份,并配置 `OPERATIONAL_ACTION_PERIPHERAL_URL`。同步器订阅源工单变更事件,收到消息即读取分页工单,将外围版本按系统数据投影到 `external_work_order` 任务;默认每 300 秒兜底对账。工单完成只表示该外部任务完成,不代表其他调查自动完成。事件协议、`tjwater_v2` 范围与启用步骤见 [外围工单主动同步](peripheral-work-order-events.md)。
|
||||
使用相同工作身份,并配置 `OPERATIONAL_ACTION_PERIPHERAL_URL`。同步器订阅源工单变更事件,收到源工单事件后按 ID 定向读取工单,将外围版本按系统数据投影到 `external_work_order` 任务;默认每 300 秒兜底对账。工单完成只表示该外部任务完成,不代表其他调查自动完成。事件协议、项目存储范围与启用步骤见 [外围工单主动同步](peripheral-sample.md#工单事件主动同步)。
|
||||
|
||||
工况任务面板的刷新按钮只重新读取当前项目的工作台快照,不直接触发外围系统全量对账。外围工单由独立同步器使用服务身份响应后端事件,避免多标签页或普通用户刷新放大全量扫描,也避免浏览器项目与同步器固定项目不一致。
|
||||
|
||||
@@ -190,3 +190,29 @@ OPERATIONAL_TASKS_STORAGE_TEST=1 conda run -n server python -m pytest tests/inte
|
||||
```
|
||||
|
||||
第二条需要当前配置的 PostgreSQL 可连接及测试 DDL 权限。测试在强制回滚事务中创建表结构和隔离项目数据,结束后不保留这些修改;不代替正式迁移。外部工单调用仍使用隔离 HTTP transport,不会操作实际工单系统。该测试覆盖持久化报告、反馈重新排队、旧会话拒绝、人工确认与丢响应回执核对,而非真实模型推理质量。
|
||||
|
||||
### 共用数据库时的执行范围
|
||||
|
||||
任务创建时,Server 将 `OPERATIONAL_TASK_EXECUTION_SCOPE`(留空则使用
|
||||
`ENVIRONMENT`)写入 `workbench.operational_tasks.document.execution_scope`。
|
||||
本地配置 `development`,线上配置 `production`;浏览器不提交此字段,
|
||||
重试保留原范围。`execution_scope` 为必填字段,运行时代码不为缺失值兜底。
|
||||
现有任务通过 `resources/sql/034_operational_task_execution_scope.sql` 一次性补为
|
||||
`production`;已有明确范围的任务保持原值。此回填是历史任务的迁移策略,
|
||||
不代表通过旧记录推断出了创建环境。
|
||||
|
||||
迁移需要与新版本切换协调执行:暂停任务写入,执行每个业务库的 SQL,
|
||||
启动新版本并恢复任务写入。旧版本的严格模型不接受新增 JSON 字段,
|
||||
新版本也拒绝未迁移数据;不能在旧版本仍服务任务请求时单独回填。
|
||||
SQL 同时增加必填范围约束,不设置数据库默认值。回滚版本前需同步处理该约束
|
||||
与 JSON 字段,不得直接切回旧镜像。
|
||||
|
||||
Worker 应连接所属环境的 `OPERATIONAL_WORKER_SERVER_URL` 和
|
||||
`OPERATIONAL_WORKER_AGENT_URL`。Server 在运行列表、领取、租约回收与
|
||||
SCADA recovery 候选查询上限制执行范围;直接领取其他范围返回
|
||||
`403 EXECUTION_SCOPE_MISMATCH`。外部工单反馈与来源撤回也必须匹配任务的执行范围,
|
||||
并在幂等重放前校验;开发环境同步器不能更新生产任务或重置其恢复监测起点。
|
||||
这属于配置隔离,不能替代服务凭据授权:
|
||||
不要让本地 Worker 通过线上 Server 领取任务。普通任务与历史记录仍共用存储。
|
||||
|
||||
修改范围配置需要重启相应 Server。暂停的本地 Worker 不会因代码更新恢复。
|
||||
|
||||
@@ -52,7 +52,7 @@ conda run -n server python scripts/seed_peripheral_sample.py --project-id <proje
|
||||
|
||||
外围面板的“历史监测回放”读取现有项目历史监测源,与上述 `water-demo-v1` 合成数据集是两个独立区域。前端可通过 `TJWATER_PERIPHERAL_SAMPLE_TIME_MODE=demo` 默认使用已知有数据的固定窗口,并用 `TJWATER_PERIPHERAL_SAMPLE_DEMO_START`、`TJWATER_PERIPHERAL_SAMPLE_DEMO_END` 配置开始和结束时间;切到 `recent` 时按 `TJWATER_PERIPHERAL_SAMPLE_RECENT_HOURS` 查询最近时段。界面仍允许在运行时切换演示、最近和自定义窗口。
|
||||
|
||||
本地默认演示窗口为 UTC 2026-04-30 08:00 至 16:00。它只控制历史监测查询,不改变 `water-demo-v1` 自身的 UTC 2026-09-17 00:00 至 12:00 固定内容。
|
||||
前端源码默认窗口为 UTC 2026-09-17 00:00 至 12:00,可由运行时配置覆盖,见 [前端配置定义](../../next-tjwater-frontend/src/shared/config/env.ts)。它只控制历史监测查询,不改变 `water-demo-v1` 自身的固定内容;部署环境的实际配置与数据覆盖需现场核对。
|
||||
|
||||
## 接口与页面
|
||||
|
||||
@@ -62,5 +62,4 @@ conda run -n server python scripts/seed_peripheral_sample.py --project-id <proje
|
||||
|
||||
前端在“外围样板”中单独读取合成样本,展示版本、种子和摘要,以及读数、设备状态、告警和运行历史。加载失败不会清空或阻止既有核查任务及历史回放。
|
||||
|
||||
核查任务、反馈仍需用户依据已保存来源显式提交。生成器不会伪造 Next 来源、现场反馈或业务审批。
|
||||
已审核反馈现在可经人工确认[导入独立新证据](field-feedback-import.md),再另行确认建立[人工复核草稿修订](feedback-review-revisions.md)。专业重分析不在本次已确认的样板反馈范围内;生成器不会自动导入或修订成果。
|
||||
外围业务工单及现场反馈使用 `system_data` 来源,合成生成器不伪造业务上报、人工证据或审批。旧的“反馈导入后建立人工复核草稿”不再作为当前流程;人工事实、外部反馈与调查新轮次的边界见 [工况任务](operational-tasks.md#调查轮次与协作边界),工单来源与事件同步见 [外围服务](peripheral-sample.md)。
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user