Compare commits

...
Author SHA1 Message Date
jiang 549019507e fix(tasks): isolate peripheral feedback writes by execution scope
Generic Container CI/CD / test-build-publish (push) Successful in 4m15s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 4m17s
An old development synchronizer could overwrite a production recovery timestamp in a shared project database even after production deployment. Check task scope before feedback or source-withdrawal writes and before idempotency replay.

Cover cross-scope rejection, same-scope success and replay, and preserve actual field feedback time when later order metadata changes.
2026-11-08 04:07:07 +08:00
jiang acc8f8f7cb fix(skills): keep comparison guidance within inline response budget
Generic Container CI/CD / test-build-publish (push) Successful in 4m11s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 4m11s
2026-11-08 03:10:22 +08:00
jiang b8359ba83c fix(charts): validate same-time comparison publications
Generic Container CI/CD / test-build-publish (push) Failing after 2m10s
Server CI/CD v2 / build-test-publish-and-deploy (push) Failing after 2m10s
Prompt-only projection rules allowed historical samples to be appended to the current axis. Enforce typed comparison semantics at presentation and report publication, preserving historical read contracts and returning actionable validation errors.
2026-11-08 03:04:14 +08:00
jiang c0dea2aacf fix(contracts): export required task execution scope 2026-11-08 02:00:29 +08:00
jiang 33df25f22b fix(tasks): isolate worker execution scopes
Persist a required Server-owned execution scope and enforce it for listing, claiming, lease recovery and SCADA verification. Backfill existing tasks explicitly through migration 034 so local development workers no longer claim production tasks from the shared database.
2026-11-08 01:54:48 +08:00
jiang 2a7b96f813 fix(scada): verify recovery after replay work-order completion
Generic Container CI/CD / test-build-publish (push) Successful in 3m58s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 3m58s
2026-11-07 23:12:42 +08:00
jiang 06d8539aa8 feat(scada): require complete authorized investigation evidence
Generic Container CI/CD / test-build-publish (push) Successful in 5m52s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 5m52s
2026-11-07 21:15:01 +08:00
jiang 2552a9b40d fix(scada): require complete pressure and flow comparison charts 2026-11-07 21:03:43 +08:00
jiang 01bad116c1 fix(scada): align location capability and investigation guidance
Generic Container CI/CD / test-build-publish (push) Successful in 5m25s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 5m26s
2026-11-07 20:14:24 +08:00
jiang bf8ea45d40 test: support importlib mode for calibration fixtures
Generic Container CI/CD / test-build-publish (push) Successful in 5m17s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 5m17s
2026-11-07 19:54:42 +08:00
jiang dc83cf5c88 fix(scada): verify first sample after field completion
Generic Container CI/CD / test-build-publish (push) Failing after 47s
Server CI/CD v2 / build-test-publish-and-deploy (push) Failing after 47s
2026-11-07 19:50:31 +08:00
jiang ff8aba4964 feat(scada): add calibrated monitoring and incident recovery 2026-11-07 19:44:58 +08:00
jiang 26fa5d1909 feat(scada): add project monitoring replay
Generic Container CI/CD / test-build-publish (push) Successful in 3m39s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 3m39s
2026-11-04 14:27:21 +08:00
jiang f25423dec4 feat(benchmark): add tjwater2 burst calibration design 2026-11-03 17:26:38 +08:00
jiang e48ccfba9e test: update MCP catalog count for node coverage
Generic Container CI/CD / test-build-publish (push) Successful in 3m35s
Server CI/CD v2 / build-test-publish-and-deploy (push) Successful in 3m35s
2026-11-02 15:26:14 +08:00
jiang f2cfce8192 feat: add node coverage and improve investigation presentation
Generic Container CI/CD / test-build-publish (push) Failing after 1m50s
Server CI/CD v2 / build-test-publish-and-deploy (push) Failing after 1m51s
2026-11-02 15:23:34 +08:00
160 changed files with 16483 additions and 482 deletions
+5 -4
View File
@@ -6,6 +6,8 @@
# 基础运行
PROJECT_NAME="TJWater Server"
ENVIRONMENT="production"
# 任务领取范围;留空时沿用 ENVIRONMENT。历史任务需先执行 034 迁移。
OPERATIONAL_TASK_EXECUTION_SCOPE=""
API_V1_STR="/api/v1"
NETWORK_NAME="tjwater"
@@ -150,10 +152,9 @@ TJWATER_MCP_MAX_REQUEST_TIMEOUT_SECONDS=240
# 后台计算独立执行时限为 120–86400 秒,不受工具 timeout_seconds 控制。
TJWATER_MCP_MAX_CALCULATION_TIMEOUT_SECONDS=3600
# 独立计算队列消费者;显式项目 UUID,以逗号分隔。不会在 API lifespan 内启动。
# 先在每个目标项目 BizDB 依次应用 026_calculation_jobs.sql 和 027_calculation_job_lifecycle.sql,再启动计算服务。
CALCULATION_WORKER_PROJECT_IDS=""
# 所有配置项目合计并发,独立于 Uvicorn --workers 4;单实例锁防止重复消费者扩容。
# 独立计算队列消费者;从 MetaDB 动态发现活动项目,不会在 API lifespan 内启动。
# 先在每个项目 BizDB 应用 026、027 和 030 迁移,再启动计算服务。
# 所有活动项目合计并发,独立于 Uvicorn --workers 4;单实例锁防止重复消费者扩容。
CALCULATION_WORKER_MAX_CONCURRENCY=2
CALCULATION_WORKER_POLL_SECONDS=2
CALCULATION_WORKER_LEASE_SECONDS=30
+1 -1
View File
@@ -12,7 +12,7 @@ Use the existing conda environment when available:
```bash
conda run -n server python -m pytest tests/unit tests/auth -q
conda run -n server uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
conda run -n server python scripts/run_server.py --reload
docker build -t tjwater-server:local .
docker compose -f infra/docker/docker-compose.yml config
```
+13 -4
View File
@@ -42,7 +42,7 @@ cp .env.example .env
```bash
conda run -n server python -m pytest tests/unit tests/auth -q
conda run -n server uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
conda run -n server python scripts/run_server.py --reload
```
如需要进入环境:
@@ -84,11 +84,12 @@ CORS 匹配浏览器页面来源,不是 Nginx 连接 IP 或域名解析结果
## 业务 Skill 与闭环
[爆管处置工程图](docs/burst-response-workflow.md)覆盖上报、候选定位、隔离阀门、影响分析、地图与数据展示、审批处置和反馈闭环。Server 通过任务 MCP 按 workflow 分发以下业务 Skill,当前版本均为 `2.0.0`:
[爆管事件处理流程](docs/burst-response-workflow.md)覆盖工单接收、资产关联、隔离阀门、影响说明、审批处置和反馈闭环。Server 按任务 workflow 与回放来源分发独立 Skill,版本以实际返回为准:
| workflow | Skill | 调查重点 |
| --- | --- | --- |
| burst_report | [爆管处置](resources/work-order-skills/burst-report/SKILL.md) | 候选定位、隔离阀门、影响分析、抢修及恢复闭环 |
| burst_report(工单) | [爆管事件处理](resources/work-order-skills/burst-event-handling/SKILL.md) | 资产关联、隔离方案、影响说明、处置及恢复闭环 |
| scada_anomaly/历史回放 | [SCADA 异常调查](resources/work-order-skills/scada-anomaly/SKILL.md) | 冻结数据诊断、候选定位与隔离影响分析 |
| water_quality_complaint | [水质投诉](resources/work-order-skills/water-quality-complaint/SKILL.md) | 投诉关联、监测与采样、候选影响、处置及复测回访 |
| general | [预约跟进](resources/work-order-skills/appointment-follow-up/SKILL.md) | 审批与回执、现场履约、反馈复核及与原事件的闭环区别 |
@@ -97,7 +98,7 @@ JSONL 整理遵守会话文件权限,后台图表和表格通过既有报告 `
## 常用命令
外围接入样板作为 `/peripheral` 子应用随主 API 进程启动,复用 PostgreSQL 且不自动建表;授权边界和契约见 [外围样板说明](docs/peripheral-sample.md)。
外围样板提供 synthetic 测试工单和测试数据的版本化 CRUD;工单经同步器投影为 `workbench.operational_tasks` 中的外围来源任务。固定演示时窗和造数方式见 [合成样本说明](docs/peripheral-generated-data.md),运行态读取边界见 [工作台运行态说明](docs/workbench-runtime.md)。这些数据不代表现场事实,也不会触发设备控制。
外围系统提供业务工单和反馈的版本化 CRUD,工单来源固定为 `system_data`,经同步器投影为 `workbench.operational_tasks` 中的外围来源任务。系统上报与已核实事实分别记录,现场处置通过审批流程执行。独立测试数据和监测演示的固定时窗及造数方式见 [合成样本说明](docs/peripheral-generated-data.md),运行态读取边界见 [工作台运行态说明](docs/workbench-runtime.md)。
```bash
conda run -n server python -m pytest tests -q
@@ -111,6 +112,14 @@ docker compose -f infra/docker/docker-compose.yml config
- `docker build`:构建后端镜像。
- `docker compose config`:检查 compose 配置和变量展开。
开发时使用 `python scripts/run_server.py --reload` 启用进程交接式重载:父进程持有监听端口,
候选进程完成 ASGI 初始化、通过元数据库就绪检查并接管监听后,才通知旧进程停止接收新连接。
旧 HTTP 请求和 SSE 连接继续由旧进程完成,期间新请求由新进程响应;不迁移已建立的 TCP 连接,
也不通过强制取消旧请求来加快重载。候选启动失败或超出 `--startup-timeout`(默认 60 秒)时,
保留旧进程继续服务。快速连续修改最多保留两个正在排空的旧进程,其余修改合并后再重载。
默认监视 `app/` 和 `samples/peripheral/` 下的 Python 文件,可重复传入 `--reload-dir` 指定目录。
直接执行 `uvicorn ... --reload` 仍使用其原有先停后启流程;已经运行的旧启动命令不会自动切换到新机制。
## 开发规范
- Python 文件、函数、变量、Pydantic 字段、JSON body 字段和 query 参数使用 `snake_case`。
+2
View File
@@ -49,6 +49,7 @@ def _project_response(project: models.Project) -> AdminProjectResponse:
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
map_config=getattr(project, "map_config", {}) or {},
settings=getattr(project, "settings", {}) or {},
status=project.status,
created_at=project.created_at,
updated_at=project.updated_at,
@@ -276,6 +277,7 @@ async def create_admin_project(
gs_workspace=payload.gs_workspace,
map_extent=payload.map_extent,
map_config=payload.map_config.model_dump(exclude_unset=True),
settings=payload.settings.model_dump(exclude_unset=True),
status=payload.status,
creator_user_id=current_user.id,
)
+1
View File
@@ -45,6 +45,7 @@ async def get_project_metadata(
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
map_config=getattr(project, "map_config", {}) or {},
settings=getattr(project, "settings", {}) or {},
status=project.status,
project_role=ctx.project_role,
)
+1
View File
@@ -320,6 +320,7 @@ async def provision_project(
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
map_config=getattr(project, "map_config", {}) or {},
settings=getattr(project, "settings", {}) or {},
status=project.status,
created_at=project.created_at,
updated_at=project.updated_at,
+47 -4
View File
@@ -7,7 +7,9 @@ from uuid import UUID
from fastapi import APIRouter, Depends, Header, HTTPException, Query, status
from psycopg import AsyncConnection
from app.auth.metadata_dependencies import get_metadata_repository
from app.core.config import settings
from app.auth.permissions import (
OPERATIONAL_TASK_EXECUTE,
OPERATIONAL_TASK_EXTERNAL_SYNC,
@@ -22,6 +24,7 @@ from app.auth.project_dependencies import (
ProjectContext,
get_project_context,
get_project_pg_connection,
get_project_timescale_connection,
)
from app.domain.domain_errors import DomainError
from app.domain.schemas.workbench_runtime import WorkbenchDateTime
@@ -90,11 +93,37 @@ from app.services.operational_tasks import OperationalTaskService
router = APIRouter()
@router.post("/operational-tasks/recovery/reconcile")
async def reconcile_task_recoveries(
command: RecoverExpiredLeasesCommand,
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
conn: AsyncConnection = Depends(get_project_pg_connection),
timescale=Depends(get_project_timescale_connection),
metadata=Depends(get_metadata_repository),
) -> dict[str, int]:
from app.domain.schemas.project_settings import ProjectSettings
from app.services.operational_recovery import reconcile_recoveries
if SCADA_VIEW not in permissions_for_context(ctx):
raise HTTPException(403, "恢复监测需要 SCADA 查看权限")
project = await metadata.get_project_by_id(ctx.project_id)
if project is None:
raise HTTPException(404, "当前项目不存在")
try:
resumed = await reconcile_recoveries(_service(ctx, conn), timescale,
ProjectSettings.model_validate(project.settings or {}), limit=command.limit,
created_from=command.task_created_from)
return {"resumed": resumed}
except DomainError as exc:
_raise_domain(exc)
def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskService:
return OperationalTaskService(
OperationalTaskRepository(conn, ctx.project_id),
ctx,
_task_permissions(ctx),
execution_scope=settings.operational_task_execution_scope,
)
@@ -567,10 +596,24 @@ async def query_operational_task_data(
if not await service.can_read_source(task_id, query.source):
raise DomainError("PERMISSION_DENIED", "Task has no approved access to this data source", status=403)
source_permission = SCADA_VIEW if query.source.startswith("scada_") else SIMULATION_VIEW
snapshot = await read_task_data(
context=ctx, query=query, credential=authorization,
delegated=source_permission not in _task_permissions(ctx),
)
from app.services.scada_replay_queries import replay_snapshot_rows
replay_context = await service.replay_context(task_id)
if replay_context:
if query.source != "scada_readings":
raise HTTPException(403, "回放调查数据接口仅开放本轮 SCADA 快照,请通过 MCP 查询设备目录")
rows = await replay_snapshot_rows(conn, ctx.project_id, replay_context, query.device_ids,
query.start, query.end, 1001)
from app.services.content_digest import digest
from datetime import datetime, timezone
snapshot = TaskDataSnapshot(
source="scada_readings", source_ref=f"scada_replay:{replay_context['replay_id']}:{digest(rows)}",
queried_at=datetime.now(timezone.utc), origin="synthetic", data=rows[:1000], truncated=len(rows) > 1000,
)
else:
snapshot = await read_task_data(
context=ctx, query=query, credential=authorization,
delegated=source_permission not in _task_permissions(ctx),
)
evidence, stored_snapshot = await service.record_data_snapshot(task_id, snapshot)
return TaskDataQueryResult(evidence=evidence, snapshot=stored_snapshot)
except TaskDataError as exc:
+1
View File
@@ -35,6 +35,7 @@ async def get_project_info_endpoint(
gs_workspace=project_detail.gs_workspace,
map_extent=project_detail.map_extent,
map_config=getattr(project_detail, "map_config", {}) or {},
settings=getattr(project_detail, "settings", {}) or {},
status=project_detail.status,
project_role="viewer", # Default role for public access
)
+115
View File
@@ -0,0 +1,115 @@
"""SCADA replay controls. Manual steps are processed by the command request."""
from uuid import UUID
from datetime import datetime
from fastapi import APIRouter, Depends, Query, HTTPException
from app.auth.metadata_dependencies import get_metadata_repository
from app.auth.permissions import SCADA_VIEW, SCADA_CLEAN, require_permission, permissions_for_context
from app.auth.project_dependencies import ProjectContext, get_project_pg_connection, get_project_timescale_connection
from app.domain.schemas.scada_replay import (
ScadaReplayCapabilities, ScadaReplayCreate, ScadaReplayCommand, ScadaReplaySnapshot,
ScadaReplayEventsPage, ScadaReplayReadings,
)
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
from app.domain.schemas.project_settings import ProjectSettings
from app.services.scada_replay import ScadaReplayService, snapshot
router = APIRouter(prefix="/scada-replays", tags=["SCADA Replay"])
async def repository(
ctx: ProjectContext = Depends(require_permission(SCADA_VIEW)),
connection=Depends(get_project_pg_connection),
):
return ScadaReplayRepository(connection, ctx.project_id)
async def project_settings(
ctx: ProjectContext = Depends(require_permission(SCADA_VIEW)),
metadata_repo=Depends(get_metadata_repository),
) -> ProjectSettings:
project = await metadata_repo.get_project_by_id(ctx.project_id)
if project is None:
raise HTTPException(404, "当前项目不存在")
return ProjectSettings.model_validate(project.settings or {})
@router.get("/capabilities", response_model=ScadaReplayCapabilities)
async def replay_capabilities(repo=Depends(repository), ctx=Depends(require_permission(SCADA_VIEW)),
settings=Depends(project_settings)):
available = await repo.available()
current = await repo.current() if available else None
monitoring = settings.scada_monitoring
monitoring_ready = bool(
monitoring
and len(monitoring.pressure_device_ids) >= 5
and (monitoring.decision_policy == "weighted" or monitoring.outlet_flow_device_ids)
)
if monitoring_ready:
monitoring_reason = None
elif monitoring is None:
monitoring_reason = "项目尚未配置压力监测角色及组合判定规则"
elif len(monitoring.pressure_device_ids) < 5:
monitoring_reason = "爆管侦测至少需要 5 个压力监测设备"
else:
monitoring_reason = "项目尚未配置出厂边界流量设备"
return ScadaReplayCapabilities(
available=available, can_write=SCADA_CLEAN in permissions_for_context(ctx),
reason=None if available else "需先部署 SCADA 回放存储结构",
monitoring_interval_seconds=settings.scada_monitoring_interval_seconds,
burst_monitoring_ready=monitoring_ready,
burst_monitoring_reason=monitoring_reason,
monitoring_config=monitoring,
current=snapshot(current) if current else None,
)
@router.post("", response_model=ScadaReplaySnapshot, status_code=201)
async def create_replay(command: ScadaReplayCreate, repo=Depends(repository),
ctx=Depends(require_permission(SCADA_CLEAN)), settings=Depends(project_settings)):
return await ScadaReplayService(repo, ctx).create(
command,
monitoring_interval_seconds=settings.scada_monitoring_interval_seconds,
monitoring_config=settings.scada_monitoring,
)
@router.get("/{replay_id}", response_model=ScadaReplaySnapshot)
async def get_replay(replay_id: UUID, repo=Depends(repository)):
await ScadaReplayService(repo, None).ensure_storage()
return snapshot(await repo.get(replay_id))
@router.post("/{replay_id}/commands", response_model=ScadaReplaySnapshot)
async def replay_command(replay_id: UUID, command: ScadaReplayCommand, repo=Depends(repository),
ctx=Depends(require_permission(SCADA_CLEAN)),
timescale=Depends(get_project_timescale_connection)):
service = ScadaReplayService(repo, ctx, timescale)
result = await service.command(replay_id, command)
if command.action in {"step", "monitor", "retry"} and result.step_pending:
return await service.process_pending_step(replay_id)
return result
@router.get("/{replay_id}/events", response_model=ScadaReplayEventsPage)
async def replay_events(replay_id: UUID, offset: int = Query(0, ge=0), limit: int = Query(100, ge=1, le=200),
repo=Depends(repository)):
await ScadaReplayService(repo, None).ensure_storage()
await repo.get(replay_id)
items = await repo.events(replay_id, offset, limit + 1)
return ScadaReplayEventsPage(items=items[:limit], next_offset=offset + limit if len(items) > limit else None)
@router.get("/{replay_id}/observations", response_model=ScadaReplayReadings)
async def replay_observations(replay_id: UUID, through: datetime, repo=Depends(repository)):
await ScadaReplayService(repo, None).ensure_storage()
document = await repo.get(replay_id)
if (through.tzinfo is None or not document.get("delivered_until")
or through > datetime.fromisoformat(document["delivered_until"])
or through < datetime.fromisoformat(document["start_time"])):
raise HTTPException(422, "回看仅允许本轮已读取的时间区间")
return ScadaReplayReadings(
items=await repo.latest_outputs(replay_id, through),
through=through,
)
+44 -1
View File
@@ -1,10 +1,13 @@
from fastapi import APIRouter, Depends, HTTPException, Query, Path, Body
from typing import List
from datetime import datetime
from uuid import UUID
from psycopg import AsyncConnection
from pydantic import BaseModel, Field, field_validator
from app.domain.schemas.scada import ScadaLatestReadingsResponse
from app.domain.schemas.scada import ScadaLatestReadingsResponse, ScadaRawPage
from app.auth.project_dependencies import get_project_context, ProjectContext
from app.services.scada_raw import read_raw_page, validate_window
from app.infra.db.postgresql.scada import ScadaInfoRepository
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
from .dependencies import get_postgres_connection, get_timescale_connection
@@ -13,6 +16,46 @@ router = APIRouter()
SCADA_BATCH_MAX_ITEMS = 10_000
@router.get("/timeseries/scada-readings/raw", response_model=ScadaRawPage,
summary="按时间顺序分页读取 SCADA 原始值")
async def get_scada_raw(
start_time: datetime, end_time: datetime, device_ids: str,
limit: int = Query(500, ge=1, le=500),
cursor: str | None = Query(None, max_length=2048),
replay_id: UUID | None = None,
conn: AsyncConnection = Depends(get_timescale_connection),
postgres_conn: AsyncConnection = Depends(get_postgres_connection),
ctx: ProjectContext = Depends(get_project_context),
) -> ScadaRawPage:
try:
validate_window(start_time, end_time)
except ValueError as exc:
raise HTTPException(422, str(exc)) from exc
ids = list(dict.fromkeys(item.strip() for item in device_ids.split(",") if item.strip()))
if not ids or len(ids) > 500:
raise HTTPException(422, "请选择 1 至 500 个 SCADA 设备")
existing = await ScadaInfoRepository.get_existing_device_ids(postgres_conn, ids)
if set(ids) != existing:
raise HTTPException(422, "查询包含当前项目未配置的 SCADA 设备")
try:
through = None
if replay_id is not None:
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
from app.services.scada_replay import ScadaReplayService
repo = ScadaReplayRepository(postgres_conn, ctx.project_id)
await ScadaReplayService(repo, ctx).ensure_storage()
replay = await repo.get(replay_id)
if not set(ids) <= set(replay["config"]["device_ids"]):
raise HTTPException(403, "设备超出回放范围")
if start_time < datetime.fromisoformat(replay["start_time"]) or end_time > datetime.fromisoformat(replay["end_time"]):
raise HTTPException(422, "查询超出选定回放区间")
through = datetime.fromisoformat(replay["virtual_now"])
return await read_raw_page(conn, project_id=str(ctx.project_id), device_ids=ids,
start=start_time, end=end_time, limit=limit, cursor=cursor, through=through)
except ValueError as exc:
raise HTTPException(422, str(exc)) from exc
class ScadaReadingBatchItem(BaseModel):
time: datetime
device_id: str = Field(min_length=1)
+1
View File
@@ -36,6 +36,7 @@ def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskServi
OperationalTaskRepository(conn, ctx.project_id),
ctx,
permissions_for_context(ctx),
execution_scope=settings.operational_task_execution_scope,
)
+2
View File
@@ -61,8 +61,10 @@ from app.auth.permissions import (
require_method_permission,
require_permission,
)
from app.api.v1.endpoints import scada_replay
api_router = APIRouter()
api_router.include_router(scada_replay.router)
webgis_access = Depends(
require_method_permission(
+6
View File
@@ -8,6 +8,12 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
PROJECT_NAME: str = "TJWater Server"
ENVIRONMENT: str = "production"
OPERATIONAL_TASK_EXECUTION_SCOPE: str = ""
@property
def operational_task_execution_scope(self) -> str:
return self.OPERATIONAL_TASK_EXECUTION_SCOPE.strip() or self.ENVIRONMENT
API_V1_STR: str = "/api/v1"
NETWORK_NAME: str = "default_network"
# 敏感配置加密密钥 (Fernet)
+15 -1
View File
@@ -2,9 +2,10 @@ from datetime import datetime
from typing import Literal
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, model_validator
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from app.domain.schemas.project_map_config import ProjectMapConfig, ProjectMapConfigUpdate
from app.domain.schemas.project_settings import ProjectSettings
BusinessRole = Literal["admin", "service", "user"]
@@ -72,6 +73,7 @@ class ProjectMemberResponse(BaseModel):
class AdminProjectCreateRequest(BaseModel):
settings: ProjectSettings = Field(default_factory=ProjectSettings)
name: str = Field(..., min_length=1, max_length=100)
code: str = Field(..., min_length=1, max_length=50)
description: str | None = None
@@ -82,6 +84,10 @@ class AdminProjectCreateRequest(BaseModel):
class AdminProjectUpdateRequest(ProjectMapConfigUpdate):
settings: ProjectSettings | None = Field(
default=None,
description="按顶层键合并项目配置;省略的键保持原值,嵌套对象整体替换",
)
name: str | None = Field(default=None, min_length=1, max_length=100)
code: str | None = Field(default=None, min_length=1, max_length=50)
description: str | None = None
@@ -89,8 +95,16 @@ class AdminProjectUpdateRequest(ProjectMapConfigUpdate):
map_extent: dict | None = None
status: ProjectStatus | None = None
@field_validator("settings")
@classmethod
def reject_null_settings(cls, value: ProjectSettings | None) -> ProjectSettings:
if value is None:
raise ValueError("settings must be an object")
return value
class AdminProjectResponse(BaseModel):
settings: ProjectSettings = Field(default_factory=ProjectSettings)
project_id: UUID
name: str
code: str
+27 -1
View File
@@ -3,12 +3,28 @@ from __future__ import annotations
import json
import math
import re
from datetime import datetime
from pathlib import Path
from typing import Annotated, Any, Literal
from jsonschema import Draft7Validator
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_serializer, model_validator
CHART_SAMPLE_TIME_PATTERN = (
r"^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9]"
r"(?:\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
)
def validate_chart_sample_time(value: str) -> None:
"""Match the bounded frontend RFC3339 spelling and an actual calendar date."""
if len(value) > 64 or re.fullmatch(CHART_SAMPLE_TIME_PATTERN, value) is None:
raise ValueError("Chart sample times require RFC3339 seconds and a timezone")
if datetime.fromisoformat(value).tzinfo is None:
raise ValueError("Chart sample times require a timezone")
_catalog = json.loads(
(Path(__file__).resolve().parents[3] / "contracts/answer-ui-catalog.json").read_text()
)
@@ -71,8 +87,18 @@ class AnswerUiBlock(UiModel):
if element.type == "Chart":
if any(len(series["data"]) != len(props["x_data"]) for series in props["series"]):
raise ValueError("Chart series must match the axis")
if any(not math.isfinite(value) for series in props["series"] for value in series["data"]):
if any(value is not None and not math.isfinite(value) for series in props["series"] for value in series["data"]):
raise ValueError("Chart values must be finite")
for series in props["series"]:
times = series.get("sample_times")
if times is not None:
if len(times) != len(series["data"]):
raise ValueError("Chart sample times must match the axis")
for value, at in zip(series["data"], times, strict=True):
if at is not None:
if value is None:
raise ValueError("Chart sample times require an observed value")
validate_chart_sample_time(at)
if element.type == "Table" and any(len(row) != len(props["columns"]) for row in props["rows"]):
raise ValueError("Table row width must match its columns")
if element.type == "FollowUpForm":
+2
View File
@@ -4,9 +4,11 @@ from uuid import UUID
from pydantic import BaseModel, Field
from app.domain.schemas.project_map_config import ProjectMapConfig
from app.domain.schemas.project_settings import ProjectSettings
class ProjectMetaResponse(BaseModel):
settings: ProjectSettings = Field(default_factory=ProjectSettings)
project_id: UUID
name: str
code: str
+1
View File
@@ -58,6 +58,7 @@ McpSnapshotSource = Literal[
'run_burst_detection',
'run_burst_location',
'analyze_valve_isolation',
'generate_node_coverage',
'run_leakage_identification',
'get_element_properties',
'list_elements',
+43 -5
View File
@@ -5,8 +5,10 @@ import re
from typing import Annotated, Literal
from uuid import UUID
from pydantic import Field, JsonValue, model_validator
from app.domain.schemas.answer_ui import ReportUiBlock, validate_report_ui_blocks
from pydantic import Field, JsonValue, model_serializer, model_validator
from app.domain.schemas.answer_ui import (
CHART_SAMPLE_TIME_PATTERN, ReportUiBlock, validate_chart_sample_time, validate_report_ui_blocks,
)
from app.domain.schemas.task_question import TaskQuestionRequest
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictBool, StrictModel
@@ -41,12 +43,19 @@ ExecutionStatus = Literal[
]
class ScadaReplayTaskContext(StrictModel):
replay_id: UUID
knowledge_cutoff: WorkbenchDateTime
batch_sequence: PositiveInt
class TaskSource(StrictModel):
kind: TaskSourceKind
external_id: NonEmptyStr | None = None
external_revision: PositiveInt | None = None
event_kind: NonEmptyStr | None = None
deleted_at: WorkbenchDateTime | None = None
replay_context: ScadaReplayTaskContext | None = None
class TaskTrigger(StrictModel):
@@ -252,7 +261,7 @@ def _undeclared_diagram_node_refs(diagram: InvestigationDiagram | None) -> list[
for node in diagram.node_refs:
escaped = re.escape(node.node_id)
if not re.search(
rf"(?<![A-Za-z0-9_-]){escaped}(?=\s*(?:[\[({{@;]|--|==|-\.|$))",
rf"(?<![A-Za-z0-9_-]){escaped}(?=\s*(?:[\[({{@;]|:::|--|==|-\.|$))",
diagram.mermaid,
re.MULTILINE,
):
@@ -565,6 +574,7 @@ class OperationalTaskRun(StrictModel):
class OperationalTaskSummary(StrictModel):
execution_scope: NonEmptyStr
project_id: UUID
task_id: UUID
task_type: TaskType
@@ -581,6 +591,7 @@ class OperationalTaskSummary(StrictModel):
execution_request_id: UUID
title: NonEmptyStr
summary: NonEmptyStr
handling_context: dict[str, JsonValue] = Field(default_factory=dict)
source: TaskSource
trigger: TaskTrigger
scheduled_for: WorkbenchDateTime | None
@@ -685,15 +696,42 @@ class AgentArtifactsResult(StrictModel):
class TaskSafeChartSeries(StrictModel):
name: Annotated[str, Field(max_length=128)]
data: Annotated[list[Annotated[float, Field(strict=True, allow_inf_nan=False)]], Field(min_length=1, max_length=100)]
type: Literal["line", "bar", "scatter"] | None = None
data: Annotated[list[Annotated[float, Field(strict=True, allow_inf_nan=False)] | None], Field(min_length=1, max_length=100)]
sample_times: Annotated[list[Annotated[str, Field(
strict=True, max_length=64,
pattern=CHART_SAMPLE_TIME_PATTERN,
)] | None], Field(min_length=1, max_length=100)] | None = None
@model_serializer(mode="wrap")
def serialize_optional_fields(self, handler):
return {key: value for key, value in handler(self).items() if value is not None}
@model_validator(mode="after")
def validate_sample_times(self):
if self.sample_times is not None:
if len(self.sample_times) != len(self.data):
raise ValueError("chart sample_times must align with data")
for value, at in zip(self.data, self.sample_times, strict=True):
if at is not None:
if value is None:
raise ValueError("chart sample_times require an observed value")
validate_chart_sample_time(at)
return self
class TaskSafeChart(StrictModel):
title: Annotated[str, Field(max_length=200)]
chart_type: Literal["line", "bar"]
chart_type: Literal["line", "bar", "time_comparison"]
x_axis_name: Annotated[str, Field(max_length=128)] | None = None
y_axis_name: Annotated[str, Field(max_length=128)] | None = None
x_data: Annotated[list[Annotated[str, Field(max_length=128)]], Field(min_length=1, max_length=100)]
series: Annotated[list[TaskSafeChartSeries], Field(min_length=1, max_length=8)]
@model_serializer(mode="wrap")
def serialize_optional_fields(self, handler):
return {key: value for key, value in handler(self).items() if value is not None}
@model_validator(mode="after")
def validate_alignment(self):
if any(len(series.data) != len(self.x_data) for series in self.series):
+64
View File
@@ -0,0 +1,64 @@
from math import isclose
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, JsonValue, model_validator
class ScadaMonitoringWeights(BaseModel):
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
burst_detection: float = Field(default=0.5, ge=0, le=1)
pressure: float = Field(default=0.4, ge=0, le=1)
outlet_flow: float = Field(default=0.1, ge=0, le=1)
@model_validator(mode="after")
def normalized(self):
if not isclose(self.burst_detection + self.pressure + self.outlet_flow, 1.0, abs_tol=1e-9):
raise ValueError("SCADA 三项监测权重之和必须为 1")
return self
class ScadaMonitoringSettings(BaseModel):
"""Project-owned device roles used by the manual SCADA monitoring demo."""
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
rule_version: str = Field(
default="burst-monitor-v2",
min_length=1,
max_length=80,
pattern=r"^[a-zA-Z0-9_.-]+$",
)
pressure_device_ids: list[str] = Field(default_factory=list, max_length=20)
outlet_flow_device_ids: list[str] = Field(default_factory=list, max_length=20)
decision_policy: Literal["all_signals", "weighted"] = "weighted"
weights: ScadaMonitoringWeights = Field(default_factory=ScadaMonitoringWeights)
alert_threshold: float = Field(default=0.8, gt=0, le=1)
@model_validator(mode="after")
def distinct_roles(self):
for values in (self.pressure_device_ids, self.outlet_flow_device_ids):
if len(set(values)) != len(values) or any(not value.strip() for value in values):
raise ValueError("SCADA 监测角色中的设备标识不能为空或重复")
if set(self.pressure_device_ids) & set(self.outlet_flow_device_ids):
raise ValueError("同一设备不能同时作为压力与出厂边界流量")
return self
class ProjectSettings(BaseModel):
"""Extensible, non-secret project parameters stored in MetaDB as JSON."""
model_config = ConfigDict(extra="allow", allow_inf_nan=False)
__pydantic_extra__: dict[str, JsonValue] = Field(init=False)
scada_monitoring_interval_seconds: int | None = Field(
default=None,
strict=True,
ge=60,
le=31_536_000,
description="SCADA 主动监测演示步长,单位秒;null 表示未配置",
)
scada_monitoring: ScadaMonitoringSettings | None = Field(
default=None,
description="主动监测使用的项目设备角色;不创建定时任务",
)
+18 -1
View File
@@ -1,8 +1,8 @@
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
class ScadaDeviceResponse(BaseModel):
"""Project SCADA metadata keyed by the canonical device identifier."""
@@ -47,3 +47,20 @@ class ScadaLatestReadingsResponse(BaseModel):
offset: int = Field(ge=0)
observed_count: int = Field(ge=0)
latest_observed_at: datetime | None = None
class ScadaRawReading(BaseModel):
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
time: datetime
device_id: str
monitored_value: float | None
quality: Literal["unvalidated", "missing", "invalid"] = "unvalidated"
class ScadaRawPage(BaseModel):
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
items: list[ScadaRawReading]
next_cursor: str | None
scanned_through: datetime
+169
View File
@@ -0,0 +1,169 @@
"""Persisted replay controls and explicit, unit-aware demonstration rules."""
from datetime import datetime, timedelta
from typing import Literal
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, model_validator
from app.domain.schemas.project_settings import ScadaMonitoringSettings
class ReplayModel(BaseModel):
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
class ScadaReplayRule(ReplayModel):
rule_id: str = Field(min_length=1, max_length=80, pattern=r"^[a-zA-Z0-9_-]+$")
device_ids: list[str] = Field(min_length=1, max_length=500)
kind: Literal["relative_change", "bounds"]
direction: Literal["both", "increase", "decrease"] = "both"
threshold: float | None = Field(None, gt=0)
lower: float | None = None
upper: float | None = None
unit: str = Field(min_length=1, max_length=40)
metric_name: str = Field(default="", max_length=80)
minimum_samples: int = Field(3, ge=1, le=96)
window_samples: int = Field(4, ge=1, le=96)
consecutive_samples: int = Field(1, ge=1, le=20)
recovery_samples: int = Field(2, ge=1, le=20)
minimum_devices: int = Field(1, ge=1, le=500)
@model_validator(mode="after")
def parameters(self):
if len(set(self.device_ids)) != len(self.device_ids):
raise ValueError("规则内设备不能重复")
if self.minimum_devices > len(self.device_ids):
raise ValueError("同时异常设备数超过规则设备数")
if self.kind == "relative_change" and (self.threshold is None or self.minimum_samples > self.window_samples):
raise ValueError("变化率规则需要阈值和足够的历史窗口")
if self.kind == "bounds" and self.lower is None and self.upper is None:
raise ValueError("上下限规则至少配置一个阈值")
if self.lower is not None and self.upper is not None and self.lower >= self.upper:
raise ValueError("规则下限必须小于上限")
return self
class ScadaQualityRange(ReplayModel):
lower: float | None = None
upper: float | None = None
@model_validator(mode="after")
def bounds(self):
if self.lower is not None and self.upper is not None and self.lower >= self.upper:
raise ValueError("仪表量程下限必须小于上限")
return self
class ScadaReplayCreate(ReplayModel):
request_id: UUID
start_time: datetime
end_time: datetime
device_ids: list[str] = Field(min_length=1, max_length=500)
rules: list[ScadaReplayRule] = Field(min_length=1, max_length=30)
quality_ranges: dict[str, ScadaQualityRange] = Field(default_factory=dict)
@model_validator(mode="after")
def configuration(self):
if (self.start_time.tzinfo is None or self.end_time.tzinfo is None
or not timedelta(0) < self.end_time - self.start_time <= timedelta(days=31)):
raise ValueError("请选择带时区、最长 31 天的有效时间区间")
ids = set(self.device_ids)
if len(ids) != len(self.device_ids) or any(not value.strip() for value in ids):
raise ValueError("设备标识不能为空或重复")
if len({rule.rule_id for rule in self.rules}) != len(self.rules):
raise ValueError("规则标识不能重复")
if any(not set(rule.device_ids) <= ids for rule in self.rules) or not set(self.quality_ranges) <= ids:
raise ValueError("规则及量程设备必须属于本次回放")
return self
class ScadaReplayCommand(ReplayModel):
request_id: UUID
expected_revision: int = Field(ge=1)
action: Literal["step", "monitor", "stop", "retry"]
at_time: datetime | None = None
@model_validator(mode="after")
def monitoring_time(self):
if self.action == "monitor":
if self.at_time is None or self.at_time.tzinfo is None:
raise ValueError("选定时刻监测需要提供带时区的 at_time")
elif self.at_time is not None:
raise ValueError("只有选定时刻监测可以指定 at_time")
return self
class ScadaMonitoringIncident(ReplayModel):
incident_id: UUID
status: Literal["active", "resolved"]
action: Literal["raised", "ongoing", "resolved"]
started_at: datetime
last_observed_at: datetime
recovery_count: int = Field(ge=0, le=2)
class ScadaMonitoringResult(ReplayModel):
monitoring_id: UUID
period_start: datetime
period_end: datetime
knowledge_cutoff: datetime
batch_sequence: int = Field(ge=1)
rule_version: str = Field(min_length=1, max_length=80)
status: Literal["normal", "attention", "insufficient_data", "suspected_burst"]
detail: str = Field(min_length=1, max_length=1000)
task_id: UUID | None = None
pressure: dict
outlet_flow: dict
burst_detection: dict
weighted: dict | None = None
incident: ScadaMonitoringIncident | None = None
role_gaps: list[str] = Field(default_factory=list)
anomaly_signals: list[dict] = Field(default_factory=list)
class ScadaReplaySnapshot(ReplayModel):
replay_id: UUID
project_id: UUID
revision: int
status: Literal["ready", "paused", "completed", "stopped", "failed"]
start_time: datetime
end_time: datetime
monitoring_interval_seconds: int | None = Field(None, ge=60, le=31_536_000)
step_pending: bool = False
monitoring_count: int = 0
last_monitoring: ScadaMonitoringResult | None = None
monitoring_config: ScadaMonitoringSettings | None = None
monitoring_calibration: dict | None = None
virtual_now: datetime
delivered_until: datetime | None = None
processed_until: datetime | None = None
processed_count: int = 0
anomaly_count: int = 0
devices: list[dict]
rules: list[ScadaReplayRule]
latest: list[dict]
active_events: list[dict]
error: str | None = None
created_at: datetime
updated_at: datetime
class ScadaReplayCapabilities(ReplayModel):
available: bool
can_write: bool
reason: str | None = None
monitoring_interval_seconds: int | None = Field(None, ge=60, le=31_536_000)
burst_monitoring_ready: bool = False
burst_monitoring_reason: str | None = None
monitoring_config: ScadaMonitoringSettings | None = None
current: ScadaReplaySnapshot | None = None
class ScadaReplayEventsPage(ReplayModel):
items: list[dict]
next_offset: int | None
class ScadaReplayReadings(ReplayModel):
items: list[dict]
through: datetime
+3 -3
View File
@@ -5,7 +5,7 @@ from typing import Literal
from uuid import UUID
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictModel
from app.domain.schemas.workbench_runtime import WorkbenchAlertSource, WorkbenchDateTime
from app.domain.schemas.workbench_runtime import WorkbenchAlertSource, WorkbenchScadaReplaySource, WorkbenchDateTime
class TaskAlertCommand(StrictModel):
@@ -14,7 +14,7 @@ class TaskAlertCommand(StrictModel):
event_type: Literal["raised", "acknowledged", "resolved"]
occurred_at: WorkbenchDateTime
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
title: NonEmptyStr
description: NonEmptyStr
severity: Literal["warning", "critical"]
@@ -29,7 +29,7 @@ class TaskAlertState(StrictModel):
revision: PositiveInt
producer_id: NonEmptyStr
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
title: NonEmptyStr
description: NonEmptyStr
occurred_at: WorkbenchDateTime
+9 -2
View File
@@ -49,10 +49,17 @@ class WorkbenchAlertSource(StrictModel):
run_id: NonEmptyStr
class WorkbenchScadaReplaySource(StrictModel):
kind: Literal["scada_replay"]
replay_id: NonEmptyStr
rule_id: NonEmptyStr
batch_sequence: int = Field(ge=1)
class WorkbenchRuntimeAlert(StrictModel):
id: NonEmptyStr
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
title: NonEmptyStr
description: NonEmptyStr
occurred_at: WorkbenchDateTime
@@ -62,7 +69,7 @@ class WorkbenchRuntimeAlert(StrictModel):
class WorkbenchPersistedAlert(StrictModel):
alert_id: NonEmptyStr
condition_id: NonEmptyStr | None
source: WorkbenchAlertSource
source: WorkbenchAlertSource | WorkbenchScadaReplaySource
title: NonEmptyStr
description: NonEmptyStr
occurred_at: WorkbenchDateTime
+23
View File
@@ -7,6 +7,29 @@ from uuid import UUID
from sqlalchemy import text
async def discover_project_ids() -> tuple[str, ...]:
"""Return every active PostgreSQL-backed project from authoritative metadata."""
from sqlalchemy import select
from app.infra.db.metadb.database import SessionLocal
from app.infra.db.metadb import models
async with SessionLocal() as session:
rows = await session.scalars(
select(models.Project.id)
.join(
models.ProjectDatabase,
models.ProjectDatabase.project_id == models.Project.id,
)
.where(
models.Project.status == "active",
models.ProjectDatabase.db_role == "biz_data",
models.ProjectDatabase.db_type == "postgresql",
)
.order_by(models.Project.id)
)
return tuple(str(project_id) for project_id in rows.all())
@asynccontextmanager
async def queue_repository(project_id):
"""Resolve the current project route and open its calculation queue."""
+1
View File
@@ -44,6 +44,7 @@ class Project(Base):
gs_workspace: Mapped[str] = mapped_column(String(100), unique=True)
map_extent: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
map_config: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict, server_default="{}")
settings: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict, server_default="{}")
status: Mapped[str] = mapped_column(String(20), default="active")
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.utcnow
@@ -67,6 +67,7 @@ class ProjectDetail:
map_extent: Optional[dict]
status: str
map_config: dict = field(default_factory=dict)
settings: dict = field(default_factory=dict)
@dataclass(frozen=True)
@@ -223,6 +224,7 @@ class MetadataRepository:
status: str,
creator_user_id: UUID | None = None,
map_config: dict | None = None,
settings: dict | None = None,
) -> models.Project:
project = models.Project(
id=uuid4(),
@@ -233,6 +235,7 @@ class MetadataRepository:
map_extent=map_extent,
status=status,
map_config=map_config or {},
settings=settings or {},
created_at=_utcnow(),
updated_at=_utcnow(),
)
@@ -263,6 +266,7 @@ class MetadataRepository:
timescale_dsn: str,
pool_min_size: int = 1,
pool_max_size: int = 4,
settings: dict | None = None,
) -> models.Project:
"""Atomically expose a fully provisioned project and both DB routes."""
business_secret = _encrypt_database_secret(business_dsn)
@@ -276,6 +280,7 @@ class MetadataRepository:
map_extent=map_extent,
status="active",
map_config={},
settings=settings or {},
created_at=_utcnow(),
updated_at=_utcnow(),
)
@@ -333,6 +338,8 @@ class MetadataRepository:
setattr(project, field, updates[field])
if "map_config" in updates:
project.map_config = {**(project.map_config or {}), **updates["map_config"]}
if "settings" in updates:
project.settings = {**(project.settings or {}), **updates["settings"]}
project.updated_at = _utcnow()
await self.session.commit()
await self.session.refresh(project)
@@ -350,6 +357,7 @@ class MetadataRepository:
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
map_config=project.map_config or {},
settings=project.settings or {},
status=project.status,
)
+9 -15
View File
@@ -7,10 +7,10 @@ from app.mcp.errors import CalculationCapacityExceeded
# Explicit metadata projection prevents status polling from decoding large JSON results.
JOB_COLUMNS = "job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,status,error,lease_generation,lease_owner,lease_expires_at,created_at,updated_at,started_at,finished_at,queue_expires_at,result_expires_at"
JOB_COLUMNS = "job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,status,error,lease_generation,lease_owner,lease_expires_at,created_at,updated_at,started_at,finished_at,result_expires_at"
PUBLIC_JOB_COLUMNS = "job_id,status,tool,error,created_at,updated_at,started_at,finished_at,queue_expires_at,result_expires_at"
PUBLIC_JOB_COLUMNS = "job_id,status,tool,error,created_at,updated_at,started_at,finished_at,result_expires_at"
class CalculationJobRepository:
@@ -19,12 +19,6 @@ class CalculationJobRepository:
self.project_id = str(project_id)
async def maintain(self):
await self.connection.execute(
"""UPDATE workbench.calculation_jobs SET status='timed_out',
error='{"code":"QUEUE_TIMEOUT"}'::jsonb,finished_at=clock_timestamp(),updated_at=clock_timestamp()
WHERE project_id=%s AND status='queued' AND queue_expires_at<=clock_timestamp()""",
(self.project_id,),
)
await self.connection.execute(
"""UPDATE workbench.calculation_jobs SET result=NULL
WHERE project_id=%s AND result IS NOT NULL AND result_expires_at<=clock_timestamp()""",
@@ -32,14 +26,14 @@ class CalculationJobRepository:
)
async def enqueue(self, *, owner_user_id, request_key, tool, parameters, binding, timeout_seconds,
queue_wait_seconds=3600, max_active_jobs=100):
max_active_jobs=100):
async with self.connection.transaction():
return await self._enqueue(owner_user_id=owner_user_id, request_key=request_key,
tool=tool, parameters=parameters, binding=binding, timeout_seconds=timeout_seconds,
queue_wait_seconds=queue_wait_seconds, max_active_jobs=max_active_jobs)
max_active_jobs=max_active_jobs)
async def _enqueue(self, *, owner_user_id, request_key, tool, parameters, binding,
timeout_seconds, queue_wait_seconds, max_active_jobs):
timeout_seconds, max_active_jobs):
# Serialize admission within the caller's transaction, across every API process.
await self.connection.execute(
"SELECT pg_advisory_xact_lock(hashtextextended(%s, 1748531781))", (self.project_id,))
@@ -64,11 +58,11 @@ class CalculationJobRepository:
raise CalculationCapacityExceeded()
cursor = await self.connection.execute(
f"""INSERT INTO workbench.calculation_jobs
(job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,queue_expires_at)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,clock_timestamp()+%s*interval '1 second')
(job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s)
RETURNING {JOB_COLUMNS}""",
(uuid4(), self.project_id, owner_user_id, request_key, tool,
Jsonb(parameters), Jsonb(binding), timeout_seconds, queue_wait_seconds),
Jsonb(parameters), Jsonb(binding), timeout_seconds),
)
return dict(await cursor.fetchone())
@@ -118,7 +112,7 @@ class CalculationJobRepository:
lease_generation=j.lease_generation+1,
lease_expires_at=clock_timestamp()+%s*interval '1 second',updated_at=clock_timestamp()
WHERE j.job_id=(SELECT job_id FROM workbench.calculation_jobs
WHERE project_id=%s AND status='queued' AND queue_expires_at>clock_timestamp() ORDER BY created_at,job_id
WHERE project_id=%s AND status='queued' ORDER BY created_at,job_id
FOR UPDATE SKIP LOCKED LIMIT 1) RETURNING j.*""",
(owner, lease_seconds, self.project_id),
)
+18
View File
@@ -6,6 +6,24 @@ from psycopg import AsyncConnection
class NetworkAssetRepository:
@staticmethod
async def get_isolation_model(conn: AsyncConnection) -> list[dict[str, Any]]:
"""Read source nodes, link state and pipe lengths in one model snapshot."""
from app.infra.db.postgresql.read_queries import read_rows
return await read_rows(conn, """
SELECT l.id, l.link_type::text AS kind,
l.start_node_id, l.end_node_id, p.length, p.diameter,
COALESCE(s.status::text, p.status::text, '') AS status
FROM network.links l
LEFT JOIN network.pipes p ON p.link_id = l.id
LEFT JOIN network.link_initial_settings s ON s.link_id = l.id
UNION ALL
SELECT n.id, n.node_type::text, NULL, NULL, NULL, NULL, ''
FROM network.nodes n
WHERE n.node_type::text IN ('reservoir', 'tank')
""")
@staticmethod
async def get_pipes_by_ids(
conn: AsyncConnection,
+28 -2
View File
@@ -264,6 +264,7 @@ class OperationalTaskRepository:
async def list_runs(
self, *, status: str | None, limit: int, cursor_created_at=None, cursor_run_id=None,
task_type: str | None = None, task_created_from: datetime | None = None,
execution_scope: str = "production",
) -> list[dict]:
result = await self.connection.execute(
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
@@ -282,18 +283,20 @@ class OperationalTaskRepository:
AND t.task_id=operational_task_runs.task_id
AND (%s::text IS NULL OR t.task_type=%s::text)
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
AND t.document->>'execution_scope'=%s
)
ORDER BY created_at,run_id LIMIT %s""",
(
self.project_id, status, status, cursor_created_at,
cursor_created_at, cursor_run_id,
task_type, task_type, task_created_from, task_created_from, limit,
task_type, task_type, task_created_from, task_created_from, execution_scope, limit,
),
)
return [dict(row) for row in await result.fetchall()]
async def claim_expired_runs(
self, limit: int, *, task_type: str | None = None, task_created_from: datetime | None = None,
execution_scope: str = "production",
) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
@@ -310,9 +313,10 @@ class OperationalTaskRepository:
AND t.task_id=operational_task_runs.task_id
AND (%s::text IS NULL OR t.task_type=%s::text)
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
AND t.document->>'execution_scope'=%s
)
ORDER BY lease_expires_at,run_id FOR UPDATE SKIP LOCKED LIMIT %s""",
(self.project_id, task_type, task_type, task_created_from, task_created_from, limit),
(self.project_id, task_type, task_type, task_created_from, task_created_from, execution_scope, limit),
)
return [dict(row) for row in await cursor.fetchall()]
@@ -711,6 +715,28 @@ class OperationalTaskRepository:
row = await cursor.fetchone()
return dict(row) if row else None
async def recovery_candidates(self, limit: int, created_from=None, execution_scope: str = "production") -> list[dict]:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.operational_tasks
WHERE project_id=%s AND task_type='event_investigation'
AND status NOT IN ('cancelled','failed')
AND document#>>'{handling_context,phase}'='waiting_scada'
AND document->>'execution_scope'=%s
AND (%s::timestamptz IS NULL OR created_at>=%s)
ORDER BY COALESCE((document#>>'{handling_context,last_checked_at}')::timestamptz,created_at),task_id LIMIT %s""",
(self.project_id, execution_scope, created_from, created_from, limit),
)
return [row["document"] for row in await cursor.fetchall()]
async def list_investigation_alerts(self, task_id: str) -> list[dict]:
cursor = await self.connection.execute(
"""SELECT document FROM workbench.alerts WHERE project_id=%s
AND status IN ('active','acknowledged')
AND document->>'condition_id'=%s ORDER BY alert_id""",
(self.project_id, f"operational-task:{task_id}"),
)
return [row["document"] for row in await cursor.fetchall()]
async def upsert_feedback(self, task_id: str, feedback: dict, accepted_at) -> None:
await self.connection.execute(
"""INSERT INTO workbench.operational_task_external_feedback
+1
View File
@@ -312,6 +312,7 @@ async def task_context_summary(conn: AsyncConnection, *, project_id: UUID, task_
'current_report_revision',t.current_report_revision,
'title_preview',left(t.document->>'title',160),'preview_only',true,
'detail',jsonb_build_object('section','task')),
'handling_context',COALESCE(t.document->'handling_context','{}'::jsonb),
'latest_report',(SELECT REPORT_SUMMARY FROM workbench.operational_task_reports r
WHERE r.project_id=t.project_id AND r.task_id=t.task_id ORDER BY r.revision DESC LIMIT 1),
'selected_report',CASE WHEN %s::int IS NULL THEN NULL ELSE
+207
View File
@@ -0,0 +1,207 @@
"""Short-transaction storage for replay control and resumable batches."""
from contextlib import asynccontextmanager
from datetime import datetime, timezone
from fastapi import HTTPException
from psycopg.types.json import Jsonb
class ScadaReplayRepository:
def __init__(self, connection, project_id):
self.connection, self.project_id = connection, str(project_id)
self.lock_key = f"scada-replay:{self.project_id}"
async def available(self) -> bool:
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT to_regclass('peripheral_sample.scada_replays') IS NOT NULL "
"AND to_regclass('peripheral_sample.scada_replay_batches') IS NOT NULL AS ready"
)
return (await cur.fetchone())["ready"]
@asynccontextmanager
async def control(self):
async with self.connection.transaction():
await self.connection.execute("SELECT pg_advisory_xact_lock(hashtextextended(%s,0))", (self.lock_key,))
yield
@asynccontextmanager
async def processing(self):
# Session lock outlives short transactions, never an open SQL transaction
# across Timescale IO. A dead connection automatically releases ownership.
async with self.connection.transaction():
cur = await self.connection.execute("SELECT pg_try_advisory_lock(hashtextextended(%s,0)) AS owned", (self.lock_key,))
owned = (await cur.fetchone())["owned"]
try:
yield owned
finally:
if owned:
async with self.connection.transaction():
await self.connection.execute("SELECT pg_advisory_unlock(hashtextextended(%s,0))", (self.lock_key,))
async def get(self, replay_id) -> dict:
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT document FROM peripheral_sample.scada_replays WHERE project_id=%s AND replay_id=%s",
(self.project_id, str(replay_id)),
)
row = await cur.fetchone()
if not row:
raise HTTPException(404, "当前项目不存在此 SCADA 回放")
return row["document"]
async def current(self) -> dict | None:
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT document FROM peripheral_sample.scada_replays WHERE project_id=%s"
" ORDER BY created_at DESC LIMIT 1", (self.project_id,),
)
row = await cur.fetchone()
return row["document"] if row else None
async def by_request(self, request_id) -> dict | None:
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT document FROM peripheral_sample.scada_replays WHERE project_id=%s AND request_id=%s",
(self.project_id, str(request_id)),
)
row = await cur.fetchone()
return row["document"] if row else None
async def active_other(self, replay_id) -> bool:
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT 1 FROM peripheral_sample.scada_replays WHERE project_id=%s AND replay_id<>%s "
"AND status IN ('ready','paused') LIMIT 1",
(self.project_id, str(replay_id)),
)
return await cur.fetchone() is not None
async def insert(self, document: dict):
await self.connection.execute(
"INSERT INTO peripheral_sample.scada_replays(project_id,replay_id,request_id,status,document) "
"VALUES(%s,%s,%s,%s,%s)",
(self.project_id, document["replay_id"], document["config"]["request_id"], document["status"], Jsonb(document)),
)
async def save(self, document: dict):
document["updated_at"] = datetime.now(timezone.utc).isoformat()
await self.connection.execute(
"UPDATE peripheral_sample.scada_replays SET status=%s,document=%s,updated_at=now() "
"WHERE project_id=%s AND replay_id=%s",
(document["status"], Jsonb(document), self.project_id, document["replay_id"]),
)
async def pending_batch(self, replay_id) -> dict | None:
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT document FROM peripheral_sample.scada_replay_batches "
"WHERE project_id=%s AND replay_id=%s AND sequence>0 AND stage<>'detected' ORDER BY sequence LIMIT 1",
(self.project_id, str(replay_id)),
)
row = await cur.fetchone()
return row["document"] if row else None
async def save_batch(self, replay_id, batch: dict):
await self.connection.execute(
"INSERT INTO peripheral_sample.scada_replay_batches(project_id,replay_id,sequence,stage,document) "
"VALUES(%s,%s,%s,%s,%s) ON CONFLICT(project_id,replay_id,sequence) "
"DO UPDATE SET stage=excluded.stage,document=excluded.document",
(self.project_id, str(replay_id), batch["sequence"], batch["stage"], Jsonb(batch)),
)
async def clear_unpublished_baseline(self, replay_id) -> None:
"""Discard only an incomplete attempt, retaining every published snapshot."""
await self.connection.execute(
"DELETE FROM peripheral_sample.scada_replay_batches "
"WHERE project_id=%s AND replay_id=%s AND sequence<0 "
"AND stage='clean_prepared' AND document->>'kind'='baseline'",
(self.project_id, str(replay_id)),
)
async def publish_baseline(self, replay_id) -> None:
"""Publish with the replay's baseline metadata in the caller's transaction."""
await self.connection.execute(
"UPDATE peripheral_sample.scada_replay_batches "
"SET stage='detected',document=jsonb_set(document,'{stage}','\"detected\"'::jsonb) "
"WHERE project_id=%s AND replay_id=%s AND sequence<0 "
"AND stage='clean_prepared' AND document->>'kind'='baseline'",
(self.project_id, str(replay_id)),
)
async def events(self, replay_id, offset=0, limit=100) -> list[dict]:
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT e.value AS event FROM peripheral_sample.scada_replay_batches b "
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'events','[]'::jsonb)) e "
"WHERE b.project_id=%s AND b.replay_id=%s ORDER BY b.sequence DESC LIMIT %s OFFSET %s",
(self.project_id, str(replay_id), limit, offset),
)
return [r["event"] for r in await cur.fetchall()]
async def monitoring_outputs(
self,
replay_id,
device_ids: list[str],
start: datetime,
end: datetime,
before_sequence: int,
) -> list[dict]:
"""Read only committed replay outputs available before the current batch."""
if not device_ids:
return []
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT r.value AS reading FROM peripheral_sample.scada_replay_batches b "
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'outputs','[]'::jsonb)) r "
"WHERE b.project_id=%s AND b.replay_id=%s AND b.stage='detected' AND b.sequence<%s "
"AND r.value->>'device_id'=ANY(%s) "
"AND (r.value->>'time')::timestamptz>=%s AND (r.value->>'time')::timestamptz<=%s "
"ORDER BY (r.value->>'time')::timestamptz,r.value->>'device_id'",
(self.project_id, str(replay_id), before_sequence, device_ids, start, end),
)
return [row["reading"] for row in await cur.fetchall()]
async def detected_outputs(
self,
replay_id,
device_ids: list[str],
start: datetime,
end: datetime,
before_sequence: int,
*,
limit: int | None = None,
offset: int = 0,
) -> list[dict]:
"""Read immutable detector outputs available to a bounded replay investigation."""
statement = (
"SELECT r.value AS reading FROM peripheral_sample.scada_replay_batches b "
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'outputs','[]'::jsonb)) r "
"WHERE b.project_id=%s AND b.replay_id=%s AND b.sequence<=%s AND b.stage='detected' "
"AND r.value->>'device_id'=ANY(%s) "
"AND (r.value->>'time')::timestamptz>=%s AND (r.value->>'time')::timestamptz<=%s "
"ORDER BY (r.value->>'time')::timestamptz,r.value->>'device_id'"
)
params: tuple = (
self.project_id, str(replay_id), before_sequence, device_ids, start, end,
)
if limit is not None:
statement += " LIMIT %s OFFSET %s"
params += (limit, offset)
async with self.connection.transaction():
cur = await self.connection.execute(statement, params)
return [row["reading"] for row in await cur.fetchall()]
async def latest_outputs(self, replay_id, through: datetime, limit: int = 500) -> list[dict]:
"""Return the latest committed cleaned output for each replay device."""
async with self.connection.transaction():
cur = await self.connection.execute(
"SELECT DISTINCT ON(r.value->>'device_id') r.value AS reading "
"FROM peripheral_sample.scada_replay_batches b "
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'outputs','[]'::jsonb)) r "
"WHERE b.project_id=%s AND b.replay_id=%s AND b.sequence>0 AND b.stage IN ('cleaned','detected') "
"AND (r.value->>'time')::timestamptz<=%s "
"ORDER BY r.value->>'device_id',(r.value->>'time')::timestamptz DESC LIMIT %s",
(self.project_id, str(replay_id), through, limit),
)
return [row["reading"] for row in await cur.fetchall()]
+109 -1
View File
@@ -1,5 +1,5 @@
from typing import List, Any
from datetime import datetime
from datetime import datetime, timedelta
from collections import defaultdict
from psycopg import AsyncConnection, Connection, sql
from psycopg.rows import dict_row
@@ -9,6 +9,91 @@ class ScadaRepository:
VALUE_FIELDS = frozenset({"monitored_value", "cleaned_value"})
RESULT_COLUMNS = "time, device_id, monitored_value, cleaned_value"
@staticmethod
async def begin_replay_baseline_snapshot(conn: AsyncConnection) -> None:
"""Set the caller's source transaction to a repeatable read-only snapshot."""
await conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
@staticmethod
async def get_warmup_values(
conn: AsyncConnection,
device_ids: list[str],
before: datetime,
per_device_limit: int,
) -> list[dict]:
"""Return the latest raw values before a replay starts for detector warmup."""
async with conn.cursor() as cur:
await cur.execute(
"SELECT d.device_id,h.monitored_value FROM unnest(%s::text[]) d(device_id) "
"CROSS JOIN LATERAL (SELECT monitored_value,time FROM scada.measurements "
"WHERE device_id=d.device_id AND time<%s ORDER BY time DESC LIMIT %s) h "
"ORDER BY d.device_id,h.time",
(device_ids, before, per_device_limit),
)
return [dict(row) for row in await cur.fetchall()]
@staticmethod
async def get_raw_page(
conn: AsyncConnection, device_ids: list[str], start: datetime,
end: datetime, through: datetime, limit: int,
after: tuple[datetime, str] | None = None, *, frame: bool = False,
) -> list[dict]:
"""Read raw only, with a stable composite key and a bounded frame/page."""
predicate = "device_id=ANY(%s) AND time>=%s AND time<%s AND time<=%s"
params: tuple = (device_ids, start, end, through)
if after is not None:
predicate += " AND (time,device_id)>(%s,%s)"
params += after
if frame:
statement = (
"WITH next_time AS (SELECT min(time) AS time FROM scada.measurements WHERE "
+ predicate + ") SELECT m.time,m.device_id,m.monitored_value,"
"md5(COALESCE(m.monitored_value::text,'NULL')) AS source_hash "
"FROM scada.measurements m WHERE m.device_id=ANY(%s) "
"AND m.time=(SELECT time FROM next_time) ORDER BY m.time,m.device_id LIMIT %s"
)
params += (device_ids, limit)
else:
statement = (
"SELECT time,device_id,monitored_value,"
"md5(COALESCE(monitored_value::text,'NULL')) AS source_hash "
"FROM scada.measurements WHERE " + predicate
+ " ORDER BY time,device_id LIMIT %s"
)
params += (limit,)
async with conn.cursor() as cur:
await cur.execute(statement, params)
return [dict(row) for row in await cur.fetchall()]
@staticmethod
async def apply_replay_cleaning(conn: AsyncConnection, rows: list[dict]) -> None:
"""Guard every raw version; update corrected values without inserting raw."""
if not rows:
return
async with conn.transaction():
async with conn.cursor() as cur:
await cur.execute(
"SELECT m.time,m.device_id,md5(COALESCE(m.monitored_value::text,'NULL')) AS source_hash "
"FROM scada.measurements m JOIN unnest(%s::timestamptz[],%s::text[]) "
"b(time,device_id) USING(time,device_id) FOR UPDATE OF m",
([r["time"] for r in rows], [r["device_id"] for r in rows]),
)
existing = {(r["time"].isoformat(), r["device_id"]): r["source_hash"]
for r in await cur.fetchall()}
if len(existing) != len(rows) or any(
existing.get((datetime.fromisoformat(r["time"]).isoformat(), r["device_id"]))
!= r["source_hash"] for r in rows
):
raise ValueError("SOURCE_CHANGED: SCADA 原始值在处理期间发生变化")
await cur.execute(
"UPDATE scada.measurements m SET cleaned_value=b.value "
"FROM unnest(%s::timestamptz[],%s::text[],%s::double precision[]) "
"b(time,device_id,value) WHERE m.time=b.time AND m.device_id=b.device_id "
"AND m.cleaned_value IS DISTINCT FROM b.value",
([r["time"] for r in rows], [r["device_id"] for r in rows],
[r["cleaned_value"] for r in rows]),
)
@staticmethod
async def insert_scada_batch(conn: AsyncConnection, data: List[dict]):
if not data:
@@ -46,6 +131,29 @@ class ScadaRepository:
)
return await cur.fetchall()
@staticmethod
async def get_recovery_scada_readings(
conn: AsyncConnection, device_ids: list[str], incident_at: datetime,
returned_at: datetime, through: datetime, *, baseline_limit: int = 100_001,
) -> list[dict]:
"""Read the causal reference and exactly the first stored frame after T."""
if not device_ids or through <= returned_at:
return []
async with conn.cursor() as cur:
await cur.execute(
"WITH next_frame AS (SELECT min(time) AS time FROM scada.measurements "
"WHERE device_id=ANY(%s) AND time>%s AND time<=%s), "
"reference AS (SELECT time,device_id,monitored_value,cleaned_value "
"FROM scada.measurements WHERE device_id=ANY(%s) AND time>=%s AND time<%s "
"ORDER BY time,device_id LIMIT %s) "
"SELECT time,device_id,monitored_value,cleaned_value FROM reference UNION ALL "
"SELECT m.time,m.device_id,m.monitored_value,m.cleaned_value FROM scada.measurements m "
"JOIN next_frame f ON m.time=f.time WHERE m.device_id=ANY(%s) ORDER BY time,device_id",
(device_ids, returned_at, through, device_ids, incident_at - timedelta(days=15),
incident_at, baseline_limit, device_ids),
)
return [dict(row) for row in await cur.fetchall()]
@staticmethod
async def get_latest_scada_readings(
conn: AsyncConnection,
@@ -194,6 +194,7 @@ class HttpOperationalActionDispatcher:
raise ValueError("invalid external project or timestamp")
receipt = ActionReceipt(
adapter="peripheral_sample", receipt_id=str(order.work_order_id),
# This is the creation action's completion, not field repair.
submitted_at=order.created_at, completed_at=order.created_at,
external_status=order.status,
detail="外围工单已创建,现场工作进度由外围系统继续同步",
+47 -17
View File
@@ -4,7 +4,7 @@ from datetime import datetime, timedelta
from typing import Annotated, Literal
from uuid import UUID
from pydantic import Field, model_validator
from pydantic import BaseModel, ConfigDict, Field, model_validator
from app.mcp.models import Id, StrictInput
@@ -15,12 +15,42 @@ MAX_REQUEST_TIMEOUT_SECONDS = 240
class CalculationInput(StrictInput):
timeout_seconds: int = Field(
default=120, ge=1, le=MAX_REQUEST_TIMEOUT_SECONDS, strict=True,
description="本次HTTP请求响应等待预算,秒,默认120秒;不是后台计算执行时限,受工具目录中的请求上限约束;后台任务通过状态和结果工具查询",
description="本次HTTP请求响应等待预算,秒,默认120秒;后台执行时限由作业系统管理,受工具目录中的请求上限约束;后台任务通过状态和结果工具查询",
)
class NodeCoverageGroup(BaseModel):
model_config = ConfigDict(extra="forbid")
group_id: Id = Field(max_length=80)
title: str = Field(min_length=1, max_length=200)
node_ids: list[Id] = Field(min_length=1, max_length=10000)
class NodeCoverageInput(CalculationInput):
groups: list[NodeCoverageGroup] = Field(min_length=1, max_length=8,
description="已核实的模型节点集合;各组为需要比较的业务或空间集合,可重叠,位置通过模型节点关联获取")
concave_ratio: float = Field(default=0.25, ge=0, le=1, allow_inf_nan=False,
description="凹包比例;越接近1越平滑接近凸包,参数只决定示意形状")
buffer_m: float = Field(default=25, gt=0, le=1000, allow_inf_nan=False,
description="展示缓冲距离,米;用于几何展示")
split_by_connectivity: bool = Field(default=False, strict=True,
description="需要按模型连接分组时设true;地理分布比较默认false,供水关系按拓扑核验")
excluded_link_ids: list[Id] = Field(default_factory=list, max_length=200,
description="连通分组时额外视为断开的管段/阀门,例如已有隔离建议中的关阀ID;仅影响绘图分组")
@model_validator(mode="after")
def bounded_groups(self):
if len({group.group_id for group in self.groups}) != len(self.groups):
raise ValueError("coverage group IDs must be distinct")
if len({node for group in self.groups for node in group.node_ids}) > 10000:
raise ValueError("coverage supports at most 10000 distinct nodes")
if self.excluded_link_ids and not self.split_by_connectivity:
raise ValueError("excluded_link_ids requires split_by_connectivity")
return self
class BurstDetectionInput(CalculationInput):
target_time: datetime = Field(description="带时区的侦测时刻;读取此前15天同周期压力数据,不是单小时定位")
target_time: datetime = Field(description="带时区的侦测时刻;读取此前15天同周期压力数据,返回压力异常诊断")
sensor_nodes: list[Id] = Field(min_length=5, max_length=20, description="已核实的压力监测节点ID,至少5个完整传感器")
sampling_interval_minutes: int = Field(default=15, ge=1, le=60)
@@ -37,14 +67,14 @@ class BurstDetectionInput(CalculationInput):
class BurstLocationInput(CalculationInput):
data_source: Literal["monitoring", "simulation"] = "monitoring"
simulation_run_id: UUID | None = Field(default=None, description="已有分析运行UUID,不是方案名称;simulation模式只读取已有异常结果及模型基线")
burst_leakage: float = Field(gt=0, allow_inf_nan=False, description="当前爆管定位算法必需的假定漏水流量,WNTR SI单位m³/s;可用有数据和方法依据的估计或明确给定的情景假设,说明来源与不确定性,不要求现场实测确认;不能任填数值")
pressure_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="压力监测设备关联的模型节点ID,不是设备查询编号;省略时自动读取当前项目SCADA配置中的压力测点,不需要猜测节点ID")
flow_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="流量监测设备关联的模型要素ID,不是设备查询编号")
simulation_run_id: UUID | None = Field(default=None, description="已有分析运行UUID;simulation模式只读取已有异常结果及模型基线")
burst_leakage: float | None = Field(default=None, gt=0, allow_inf_nan=False, description="水力定位必需的假定漏水流量,WNTR SI单位m³/s;使用有依据的估计或明确给定的情景假设,保留估计方法与输入来源。仅 Server 明确开放的候选定位路径可省略,该路径直接返回候选管段与相似度;其他项目和时段省略将被拒绝")
pressure_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="压力监测设备关联的模型节点ID,查询编号通过设备映射转换;省略时读取项目配置的压力测点")
flow_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="流量监测设备关联的模型要素ID,由设备映射取得")
scada_burst_start: datetime
scada_burst_end: datetime
scada_normal_start: datetime | None = Field(default=None, description="monitoring模式正常对照窗开始,须与结束时间成对提供;均省略时用爆管窗前一天相同时段,不代表该时段已确认正常")
scada_normal_end: datetime | None = Field(default=None, description="monitoring模式正常对照窗结束,须与开始时间成对提供;simulation模式不填写,使用同时间窗模型基线")
scada_normal_start: datetime | None = Field(default=None, description="monitoring模式正常对照窗开始,须与结束时间成对提供;均省略时用爆管窗前一天相同时段,使用前须核对正常对照有效性")
scada_normal_end: datetime | None = Field(default=None, description="monitoring模式正常对照窗结束,须与开始时间成对提供;simulation模式省略此项,使用同时间窗模型基线")
@model_validator(mode="after")
def valid_windows(self):
@@ -70,14 +100,14 @@ class BurstLocationInput(CalculationInput):
class ValveIsolationInput(CalculationInput):
accident_element: list[Id] = Field(min_length=1, max_length=20, description="经核实或明确作为候选的事故管段/节点ID")
accident_element_type: Literal["auto", "link", "node"] = Field(default="auto", description="事故要素类型;auto优先识别管段,再识别节点;节点与管段重号时,节点分析须指定node,纯管段分析可指定link")
disabled_valves: list[Id] = Field(default_factory=list, max_length=200, description="已知无法关闭的阀门;未提供不代表现场确认全部可用")
disabled_valves: list[Id] = Field(default_factory=list, max_length=200, description="已知无法关闭的阀门;现场可操作性按执行前核验记录确认")
class LeakageIdentificationInput(CalculationInput):
scada_start: datetime
scada_end: datetime
sensor_nodes: list[Id] = Field(min_length=1, max_length=20)
q_sum: float = Field(gt=0, allow_inf_nan=False, description="有依据的总漏损流量约束,m³/s,不得使用无依据默认值")
q_sum: float = Field(gt=0, allow_inf_nan=False, description="有依据的总漏损流量约束,m³/s,保留数据来源与估算方法")
start_time: float = Field(ge=0, lt=24, allow_inf_nan=False, description="模型起算小时;需与SCADA时间窗及模型时钟对齐")
timestep: int = Field(default=15, ge=5, le=60, description="计算步长,分钟;需与观测采样对应")
dma_count: int = Field(ge=1, le=20)
@@ -99,7 +129,7 @@ class LeakageIdentificationInput(CalculationInput):
class ScenarioInput(CalculationInput):
start_time: datetime
duration: int = Field(default=900, ge=60, le=86400, description="模拟时长,秒;首次优先短时间窗")
output_node_ids: list[Id] = Field(min_length=1, max_length=20, description="需要返回结果的节点;不把局部结果解释为全网无影响")
output_node_ids: list[Id] = Field(min_length=1, max_length=20, description="需要返回结果的节点;结论范围限定为返回对象")
output_link_ids: list[Id] = Field(default_factory=list, max_length=20)
@model_validator(mode="after")
@@ -110,18 +140,18 @@ class ScenarioInput(CalculationInput):
class ContaminantSimulationInput(ScenarioInput):
source: Id = Field(description="假设污染源节点;这是正向传播模拟,不是自动反演污染源")
source: Id = Field(description="假设污染源节点;这是正向传播模拟,结果按给定源解释")
concentration: float = Field(gt=0, allow_inf_nan=False, description="假设源浓度,mg/L")
pattern: Id | None = Field(default=None, description="已存在的污染源时间模式ID;不填为持续恒定源")
pattern: Id | None = Field(default=None, description="已存在的污染源时间模式ID;省略时采用持续恒定源")
class ValveClosureInput(ScenarioInput):
valves: list[Id] = Field(min_length=1, max_length=20, description="在临时模型中关闭的阀门,不下发现场控制")
valves: list[Id] = Field(min_length=1, max_length=20, description="在临时模型中关闭的阀门,现场操作另走审批流程")
class FlushingInput(ScenarioInput):
drainage_node_id: Id
flushing_flow_m3h: float = Field(gt=0, allow_inf_nan=False, description="明确的冲洗流量,m³/h;不使用无依据的自动排放量")
flushing_flow_m3h: float = Field(gt=0, allow_inf_nan=False, description="明确的冲洗流量,m³/h;提供流量来源与估算依据")
closed_valves: list[Id] = Field(default_factory=list, max_length=20)
@@ -130,7 +160,7 @@ PumpSeries = Annotated[list[NonnegativeValue], Field(min_length=1, max_length=96
class PressureRegulationInput(ScenarioInput):
pump_control: dict[Id, PumpSeries] = Field(min_length=1, max_length=20, description="已登记泵ID及运行序列;工频值为0/1,变频值为频率Hz,每项对应模型水力步长,不自动优化")
pump_control: dict[Id, PumpSeries] = Field(min_length=1, max_length=20, description="已登记泵ID及运行序列;工频值为0/1,变频值为频率Hz,每项对应模型水力步长,按给定序列评估")
tank_init_level: dict[Id, Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(default_factory=dict, max_length=20, description="水箱初始水位,须在模型允许范围内")
+20
View File
@@ -71,6 +71,10 @@ def _calculate(name: str, parameters: dict[str, Any], network: str, username: st
from app.services.valve_isolation import analyze_valve_isolation
return analyze_valve_isolation(network=network, **parameters)
if name == "generate_node_coverage":
from app.services.node_coverage import generate_node_coverage
return generate_node_coverage(network=network, **parameters)
raise ValueError("Calculation is not registered")
finally:
_slots.release()
@@ -94,3 +98,19 @@ async def bounded_calculation(function, *args, **kwargs):
finally:
_slots.release()
return await asyncio.to_thread(run)
async def calculate_replay_burst_location(
parameters: dict[str, Any], *, network: str, username: str,
) -> dict[str, Any]:
"""Run location with Server-built frozen replay series and no live SCADA query."""
from app.services.burst_location import run_burst_location_by_network
return await bounded_calculation(
run_burst_location_by_network,
network=network,
username=username,
scheme_name=None,
final_candidates_csv_path=None,
**parameters,
)
+20 -3
View File
@@ -32,6 +32,8 @@ from app.mcp.catalog import TOOLS
from app.mcp.config import McpSettings
from app.mcp.contexts import resolve_context, verify_task
from app.mcp.models import QueryResult
from app.mcp.display_time import display_query_result, display_time_text
from app.mcp.scada_catalog import compact_scada_catalog
from app.mcp.errors import CalculationCapacityExceeded
from app.mcp.scope import project_scope
from app.mcp.query_service import execute_query, task_service, authorize_query
@@ -43,6 +45,7 @@ logger = logging.getLogger(__name__)
def tool_error(code, message):
message = display_time_text(message)
return types.CallToolResult(
isError=True,
content=[types.TextContent(type="text", text=message)],
@@ -358,7 +361,8 @@ class McpQueryApplication:
result = QueryResult.model_validate(previous["result"])
else:
from app.mcp.calculation_jobs import should_queue, submit_calculation
if should_queue(tool, args):
queued = should_queue(tool, args, project_code=getattr(scope.project, "project_code", None))
if queued:
result = await submit_calculation(
scope, binding, tool, args, call_id,
job_timeout_seconds=self.settings.max_calculation_timeout_seconds,
@@ -377,7 +381,7 @@ class McpQueryApplication:
"RESULT_TOO_LARGE",
"Narrow the query or request a smaller page",
)
if binding.task_id and not should_queue(tool, args) and name not in {
if binding.task_id and not queued and name not in {
"get_calculation_job", "get_calculation_result", "cancel_calculation_job", "list_calculation_jobs",
"get_task_context",
"get_task_evidence",
@@ -422,11 +426,20 @@ class McpQueryApplication:
parameters,
result.model_dump(mode="json"),
)
output = result.model_dump(mode="json")
output = display_query_result(result.model_dump(mode="json"))
content = []
from app.mcp.extended_service import ARTIFACT_TOOLS
# Complete SCADA inputs must reach the model intact, not as a first-page preview.
# The shared 8 MiB result ceiling above still applies before persistence and output.
complete_scada = name == "query_scada_readings" and args.device_ids is None
if (not include_full and name == "list_scada_devices"
and len(json.dumps(output, ensure_ascii=False).encode()) > 12000):
preview = compact_scada_catalog(output, budget=12000)
if preview is not None:
output = preview
if (
not include_full
and not complete_scada
and name not in ARTIFACT_TOOLS
and len(json.dumps(output, ensure_ascii=False).encode()) > 12000
):
@@ -441,6 +454,10 @@ class McpQueryApplication:
"resource_uri": uri,
"preview": "The complete query snapshot is saved in task evidence",
}
from app.mcp.presentation_preview import saved_presentation_preview
presentation = saved_presentation_preview(result)
if presentation is not None:
output["data"]["presentation"] = display_query_result(presentation)
output["truncated"] = True
output["data_gaps"] = [
*result.data_gaps,
+6 -2
View File
@@ -29,7 +29,11 @@ def restore_binding(document: dict) -> BoundContext:
return BoundContext.model_validate({**document, "context_handle": "durable-calculation", "expires_at": datetime.max.replace(tzinfo=timezone.utc)})
def should_queue(tool, args) -> bool:
def should_queue(tool, args, *, project_code: str | None = None) -> bool:
if tool.name == "run_burst_location":
from app.services.burst_location_demo import is_location_demo
if is_location_demo(project_code, args):
return False
return tool.name in QUEUED_TOOLS and (not tool.mutating or not args.dry_run)
@@ -39,7 +43,7 @@ def envelope(source: str, data, *, job_id="list", page=None) -> QueryResult:
def public_job(row: dict) -> dict:
fields = {"job_id", "status", "tool", "created_at", "started_at", "finished_at", "updated_at", "error", "queue_expires_at", "result_expires_at"}
fields = {"job_id", "status", "tool", "created_at", "started_at", "finished_at", "updated_at", "error", "result_expires_at"}
from fastapi.encoders import jsonable_encoder
expiry = row.get("result_expires_at")
available = row.get("status") == "succeeded" and (expiry is None or expiry > datetime.now(timezone.utc))
+46 -34
View File
@@ -6,7 +6,7 @@ from mcp import types
from app.mcp import models as m
from app.mcp import calculation_models as jobs
from app.mcp import extended_models as e
from app.mcp.analysis_models import BurstDetectionInput, BurstLocationInput, ValveIsolationInput, LeakageIdentificationInput
from app.mcp.analysis_models import BurstDetectionInput, BurstLocationInput, ValveIsolationInput, LeakageIdentificationInput, NodeCoverageInput
from app.mcp.analysis_models import ScenarioInput, ContaminantSimulationInput, ValveClosureInput, FlushingInput, PressureRegulationInput, SensorPlacementInput
@@ -23,22 +23,33 @@ class QueryTool:
def protocol_tool(self) -> types.Tool:
if self.mutating:
suffix = "。默认dry_run只预览;执行必须携带预览摘要、相同request_id并通过宿主原生许可。仅项目普通聊天可用,后台调查和reader禁止;失败或超时先核实状态,不自动重试。"
suffix = "。默认dry_run只预览;执行必须携带预览摘要、相同request_id并通过宿主原生许可。由项目成员的普通聊天执行写入;失败或超时先核实状态,再按原请求接续。"
elif self.name == "run_simulation":
suffix = "。任务调用复用证据留存。"
elif self.computation:
suffix = "。仅计算建议,不保存方案、不控制设备;任务调用复用证据留存。"
suffix = "。返回分析建议并留存任务证据;正式方案与现场操作按各自业务流程执行。"
elif self.name == "query_nearby_elements":
suffix = "。仅查询当前已授权项目的模型管网,按距离和元素ID稳定排序;按需分页,不自动遍历全量;空间邻近不代表管网连通或供水关系。"
suffix = "。仅查询当前已授权项目的模型管网,按距离和元素ID稳定排序;按核查目标分页;连接与供水关系由拓扑分析核验。"
elif self.name == "list_scada_devices":
suffix = (
"。默认每页100个,最多500个;page.has_more=true时继续递增offset读取后续页面。"
"大目录内联保留本页全部设备的编号、类型和单位,模型映射及坐标用get_scada_device核验。"
)
elif self.name == "query_scada_readings":
suffix = (
"。首次调查省略device_ids,Server读取完整设备属性并自动翻页读取窗口内全部授权观测,"
"返回devices、readings和coverage,包括缺测及冻结范围外设备;全量结果完整内联。"
"显式device_ids仅用于后续局部复核,不能替代全量核查。"
)
else:
suffix = (
"。先限定对象和时间范围,再读取明细;集合默认20条,按需逐页扩大,"
"不要自动遍历全量;截断或非零offset不能证明完整范围无异常。"
"按当前目标读取必要页面,并说明截断和分页覆盖范围。"
)
return types.Tool(
name=self.name,
description=self.description + suffix + (
"。执行返回持久job_id;排队回执不是分析证据,使用get_calculation_job查询状态、get_calculation_result分页读取成功结果,不重复提交。"
"。返回queued/running和job_id时,使用get_calculation_job查询状态、get_calculation_result读取成功结果,排队回执用于跟踪;直接返回分析结果时立即读取。"
if self.name in {"run_burst_location", "run_leakage_identification", "run_burst_detection", "run_simulation", "run_burst_simulation", "run_contaminant_simulation", "analyze_valve_closure", "analyze_flushing", "analyze_pressure_regulation", "optimize_sensor_placement", "save_simulation_scenario", "create_sensor_placement"}
else ""
),
@@ -51,16 +62,17 @@ class QueryTool:
_specs = [
("run_simulation", "按水力步长在在线业务模型上执行指定时间段的实时模拟,不复制数据库;更新该时段的模型条件及实时基线,返回指定节点和管段结果;duration单位为秒,不创建方案或正式分析运行,结果不是实测", ScenarioInput, "simulation.run", "analysis", False, True),
("run_contaminant_simulation", "在隔离模型中模拟假设污染源的浓度传播;返回指定节点/管段结果,不反演污染源,不代表实测污染范围", ContaminantSimulationInput, "simulation.run", "analysis", False, True),
("analyze_valve_closure", "在隔离模型中模拟关阀的水力影响,返回指定节点/管段结果;不操作真实阀门", ValveClosureInput, "simulation.run", "analysis", False, True),
("analyze_flushing", "在隔离模型中评估指定排放节点、冲洗流量和关阀组合的水力结果;不是现场冲洗或消毒合格证明", FlushingInput, "simulation.run", "analysis", False, True),
("analyze_pressure_regulation", "在隔离模型中评估给定调泵序列和水箱初始水位的压力影响;不自动优化调度,不实际控制泵", PressureRegulationInput, "simulation.run", "analysis", False, True),
("optimize_sensor_placement", "使用灵敏度或K-means算法推荐压力监测点;仅返回候选节点,不保存正式方案,不保证污染溯源可观测性", SensorPlacementInput, "optimization.run", "analysis", False, True),
("run_leakage_identification", "基于SCADA压力与模型进行DMA漏损分配识别,需有依据的总漏损流量和时间对齐;返回区域估计,不是已确认漏点。内部执行遗传算法与水力计算", LeakageIdentificationInput, "burst.run", "analysis", False, True),
("run_burst_detection", "爆管异常侦测:使用至少5个完整压力传感器此前15天数据;只提示压力异常,不定位、不证明爆管", BurstDetectionInput, "burst.run", "analysis", False, True),
("run_burst_location", "爆管候选定位:对比异常与正常压力,可辅助流量;burst_leakage必需,可使用有依据的估计或明确给定的情景假设,不要求现场确诊;本工具不反演总漏水量。压力节点可省略,自动读取项目SCADA压力测点;monitoring正常窗可成对省略,默认前一天同时间段,须说明该对照假设。simulation须有已有分析运行UUID并读取同窗模型基线,不是实测;定位内部会执行水力计算,不创建新的事件方案", BurstLocationInput, "burst.run", "analysis", False, True),
("analyze_valve_isolation", "按事故候选要素分析拓扑隔离边界、建议关闭阀门和受影响节点;候选分析无需等待现场确诊。输出不是污染扩散或水力影响范围;现场操作前须核实阀门映射及可用性,本工具不执行关阀", ValveIsolationInput, "simulation.run", "analysis", False, True),
("run_simulation", "按水力步长在在线业务模型上执行指定时间段的实时模拟;更新该时段的模型条件及实时基线,返回指定节点和管段结果;duration单位为秒,输出为在线模型结果,正式分析成果通过方案工具保存", ScenarioInput, "simulation.run", "analysis", False, True),
("run_contaminant_simulation", "在隔离模型中模拟假设污染源的浓度传播;返回指定节点/管段结果,结果标注假设来源及模型预测范围", ContaminantSimulationInput, "simulation.run", "analysis", False, True),
("analyze_valve_closure", "在隔离模型中模拟关阀的水力影响,返回指定节点/管段结果;现场阀门操作按业务审批执行", ValveClosureInput, "simulation.run", "analysis", False, True),
("analyze_flushing", "在隔离模型中评估指定排放节点、冲洗流量和关阀组合的水力结果;现场效果由执行回执和复测核验", FlushingInput, "simulation.run", "analysis", False, True),
("analyze_pressure_regulation", "在隔离模型中评估给定调泵序列和水箱初始水位的压力影响;输出为给定方案评估,现场调度按审批执行", PressureRegulationInput, "simulation.run", "analysis", False, True),
("optimize_sensor_placement", "使用灵敏度或K-means算法推荐压力监测点;返回候选节点,正式部署另行保存并核验观测能力", SensorPlacementInput, "optimization.run", "analysis", False, True),
("run_leakage_identification", "基于SCADA压力与模型进行DMA漏损分配识别,需有依据的总漏损流量和时间对齐;返回待现场核验的区域估计。内部执行遗传算法与水力计算", LeakageIdentificationInput, "burst.run", "analysis", False, True),
("run_burst_detection", "爆管异常侦测:使用至少5个完整压力传感器此前15天数据;返回压力异常信号,候选管段由定位工具分析", BurstDetectionInput, "burst.run", "analysis", False, True),
("run_burst_location", "爆管候选定位:对比异常与正常压力,可辅助流量;依据现有异常形成候选,水力算法使用有依据的burst_leakage作为输入。任务上下文location_capability.available=true时可省略漏量,Server直接返回候选管段与相似度,随后继续地图展示、隔离边界与影响分析。压力节点可省略,自动读取项目配置;monitoring正常窗省略时默认前一天同时间段,须说明假设;回放必须显式提供冻结正常窗。simulation须有已有分析运行UUID及同窗模型基线,结果标注模型来源", BurstLocationInput, "burst.run", "analysis", False, True),
("analyze_valve_isolation", "按事故候选要素分析拓扑隔离边界、建议关闭阀门和受影响节点;候选分析无需等待现场确诊。输出为拓扑隔离与供水失源范围;水力影响另用关阀模拟评估,现场操作前核实阀门映射及可用性", ValveIsolationInput, "simulation.run", "analysis", False, True),
("generate_node_coverage", "为当前项目已核实的模型节点集合生成空间覆盖示意,适用于候选分布、监测覆盖比较、区域成员、隔离或模拟结果节点等,支持各类项目分析。支持最多8组、合计10000个唯一节点,从真实GIS查询坐标;监测设备、投诉或地址须先关联模型节点。可按连接分组,记录凹包/缓冲参数、缺失节点和display_only;结果为节点空间覆盖示意,保留分组及几何覆盖范围。返回result.map_payload及coverage,任务中用set_overlay引用本次evidence_id、省略data展示;地图通过展示工具发布", NodeCoverageInput, "webgis.view", "analysis", False, True),
(
"get_element_properties",
"读取当前项目的单个管网元素属性",
@@ -70,14 +82,14 @@ _specs = [
),
(
"list_elements",
"分页读取管网元素,不自动读取全网",
"按查询目标分页读取管网元素",
m.ElementPageInput,
"webgis.view",
"asset",
),
(
"query_nearby_elements",
"查询坐标附近的模型元素,默认pipe;center为EPSG:4326 [经度,纬度],可直接使用geocode.center;radius_m默认200米、最大10000米,limit默认5、最大100,支持offset。按点到完整几何的WGS84椭球最短距离筛选并排序,返回element_id、distance_m和属性;不查询原始GIS管线、不自动扩大半径、不解析道路交叉点",
"查询坐标附近的模型元素,默认pipe;center为EPSG:4326 [经度,纬度],可直接使用geocode.center;radius_m默认200米、最大10000米,limit默认5、最大100,支持offset。按点到完整几何的WGS84椭球最短距离筛选并排序,返回element_id、distance_m和属性;查询范围为当前模型管网;原始GIS、范围调整与歧义地点按各自入口核验",
m.NearbyElementsInput,
"webgis.view",
"asset",
@@ -99,7 +111,7 @@ _specs = [
(
"list_scada_devices",
"分页读取SCADA设备和物理单位",
m.PageInput,
m.ScadaDeviceListInput,
"scada.view",
"asset",
),
@@ -113,7 +125,7 @@ _specs = [
),
(
"query_model_baseline_series",
"读取最多20个元素的模型基线,不能称为实测",
"读取最多20个元素的模型基线,结果标注模型来源",
m.BaselineSeriesInput,
"simulation.view",
"model_baseline",
@@ -229,40 +241,40 @@ _specs = [
]
_specs += [
("list_regions", "分页查询当前项目区域/DMA列表", m.PageInput, "webgis.view", "asset"),
("get_region", "查询当前项目区域边界,返回EPSG:4326;不等于供水拓扑", e.RegionInput, "webgis.view", "asset"),
("get_region", "查询当前项目区域边界,返回EPSG:4326几何;供水关系按拓扑核验", e.RegionInput, "webgis.view", "asset"),
("get_region_nodes", "分页查询区域配置的成员节点,未配置时使用区域几何筛选", e.RegionInput, "webgis.view", "asset"),
("get_node_connections", "分页查询节点直接连接的管线,不自动遍历全网", e.NodeConnectionsInput, "webgis.view", "asset"),
("query_network_extent", "按EPSG:4326矩形范围分页查询当前项目模型几何;无GIS配置不跨项目回退", e.ExtentInput, "webgis.view", "asset"),
("get_node_connections", "按目标节点分页查询直接连接的管线", e.NodeConnectionsInput, "webgis.view", "asset"),
("query_network_extent", "按EPSG:4326矩形范围分页查询当前项目模型几何;GIS缺失时返回当前项目的具体缺口", e.ExtentInput, "webgis.view", "asset"),
("list_major_nodes", "按关联管道最小管径查询骨干节点,经纬度EPSG:4326", e.MajorNodesInput, "webgis.view", "asset"),
("get_model_configuration", "读取模式、曲线、控制规则、需水量、水质及反应等模型配置;集合按页返回", e.ModelConfigurationInput, "webgis.view", "asset"),
("list_model_configurations", "分页读取模式或曲线ID", e.ConfigurationListInput, "webgis.view", "asset"),
("search_project_places", "按项目配置的行政区搜索地名,最多10个候选,依赖天地图配置", e.PlaceSearchInput, "webgis.view", "asset"),
("reverse_geocode", "把EPSG:4326地图取点转为项目范围内地名,保留输入坐标", e.ReverseGeocodeInput, "webgis.view", "asset"),
("get_project_info", "读取当前项目概况与地图范围,不返回数据库路由或凭据", m.StrictInput, "webgis.view", "asset"),
("get_project_info", "读取当前项目概况与地图范围,公开字段限于项目业务概况", m.StrictInput, "webgis.view", "asset"),
("export_project_model", "从当前项目数据库导出INP或ChangeSet压缩文件,由宿主保存到会话文件", e.ProjectExportInput, "webgis.view", "asset"),
("get_sensor_placement_candidate", "读取监测点候选节点工程属性", e.SensorCandidateInput, "optimization.run", "asset"),
("export_sensor_placement", "导出已有监测点方案的Excel工程清单,由宿主保存到会话文件", e.SensorExportInput, "optimization.run", "analysis"),
("list_task_schedules", "分页读取当前项目定时任务配置,不启动调度", m.PageInput, "operational_task.view", "task"),
("list_task_schedules", "分页读取当前项目定时任务配置,执行状态由调度系统管理", m.PageInput, "operational_task.view", "task"),
("get_task_run_history", "分页读取指定项目任务的运行历史", e.TaskHistoryInput, "operational_task.view", "task"),
("get_mutation_status", "读取当前身份和会话内的写入执行状态;started只表示已启动,超时后须核实目标数据,不自动重试", e.MutationStatusInput, "webgis.view", "task"),
("calculate_demand_allocation", "按管长权重分配总需水量到节点、区域或全网,只计算不修改模型,按页返回", e.DemandAllocationInput, "webgis.view", "analysis", False, True),
("run_burst_simulation", "在隔离模型中模拟假设管道破口面积的水力影响;不是爆管侦测或定位,不保存正式方案", e.BurstSimulationInput, "simulation.run", "analysis", False, True),
("edit_model_element", "预览或增删改模型节点、管段、泵阀、水池和区域;仅修改模型,不控制现场设备", e.ModelElementMutationInput, "webgis.edit", "asset", False, False, True),
("get_mutation_status", "读取当前身份和会话内的写入执行状态;started只表示已启动,超时后先核实目标数据,再按原请求接续", e.MutationStatusInput, "webgis.view", "task"),
("calculate_demand_allocation", "按管长权重分配总需水量到节点、区域或全网,返回分配建议,模型变更通过业务写入执行,按页返回", e.DemandAllocationInput, "webgis.view", "analysis", False, True),
("run_burst_simulation", "在隔离模型中模拟假设管道破口面积的水力影响;返回给定情景的模拟结果,正式方案通过方案工具保存", e.BurstSimulationInput, "simulation.run", "analysis", False, True),
("edit_model_element", "预览或增删改模型节点、管段、泵阀、水池和区域;变更范围为业务模型,现场设备操作按对应流程执行", e.ModelElementMutationInput, "webgis.edit", "asset", False, False, True),
("edit_model_configuration", "预览或修改模型需水量、模式、曲线、控制规则、水质及模拟选项", e.ConfigurationMutationInput, "webgis.edit", "asset", False, False, True),
("clean_scada_readings", "预览或清洗明确设备与24小时内时间窗的数据,写入cleaned_value", e.ScadaCleaningInput, "scada.clean", "observed", False, False, True),
("write_scada_readings", "预览或补录/修正SCADA数据,要求真实来源依据;不把模拟当实测", e.ScadaWriteInput, "scada.clean", "observed", False, False, True),
("write_scada_readings", "预览或补录/修正SCADA数据,要求真实来源依据;观测来源按真实记录核验", e.ScadaWriteInput, "scada.clean", "observed", False, False, True),
("delete_scada_readings", "预览或删除明确设备与24小时内时间窗的SCADA记录", e.ScadaDeleteInput, "scada.clean", "observed", False, False, True),
("save_simulation_scenario", "预览或重新计算并保存正式隔离场景成果;有真实计算开销,不保存模型生成的观测数组", e.SavedScenarioInput, "simulation.run", "analysis", False, False, True),
("save_simulation_scenario", "预览或重新计算并保存正式隔离场景成果;有真实计算开销,观测数据按真实来源记录", e.SavedScenarioInput, "simulation.run", "analysis", False, False, True),
("create_sensor_placement", "预览或计算并保存正式压力监测点选址方案", e.SensorCreateInput, "optimization.run", "analysis", False, False, True),
("update_sensor_placement", "预览或修改本人有权编辑的监测点方案,保留旧点位乐观校验", e.SensorUpdateInput, "optimization.run", "analysis", False, False, True),
("create_operational_task", "预览或创建项目任务,不伪造Agent运行完成", e.TaskCreateInput, "operational_task.write", "task", False, False, True),
("create_operational_task", "预览或创建项目任务,Agent运行终态由执行系统管理", e.TaskCreateInput, "operational_task.write", "task", False, False, True),
("create_task_schedule", "预览或创建定时任务配置", e.ScheduleCreateInput, "operational_task.write", "task", False, False, True),
("update_task_schedule", "预览或启停定时任务配置,校验版本", e.ScheduleUpdateInput, "operational_task.write", "task", False, False, True),
("retry_operational_task", "预览或重新入队已终止任务;不自动启动新Agent会话", e.TaskRetryInput, "operational_task.write", "task", False, False, True),
("retry_operational_task", "预览或重新入队已终止任务;后续会话由任务调度系统创建", e.TaskRetryInput, "operational_task.write", "task", False, False, True),
("cancel_task_run", "预览或取消指定任务运行,校验运行版本", e.TaskCancelInput, "operational_task.write", "task", False, False, True),
]
_specs += [
("get_calculation_job", "读取当前身份与会话的持久计算状态;排队回执不是分析证据", jobs.CalculationJobInput, "webgis.view", "task"),
("get_calculation_job", "读取当前身份与会话的持久计算状态;排队回执用于跟踪,分析依据读取成功结果", jobs.CalculationJobInput, "webgis.view", "task"),
("get_calculation_result", "读取成功计算的原始结果;data_path选择嵌套集合,limit/offset按需分页", jobs.CalculationResultInput, "webgis.view", "task"),
("cancel_calculation_job", "请求取消本人当前会话的计算;运行中取消由独立计算进程处理,须继续核实最终状态", jobs.CalculationJobInput, "webgis.view", "task"),
("list_calculation_jobs", "分页读取本人当前会话及任务范围的计算状态", jobs.CalculationJobsInput, "webgis.view", "task"),
+4
View File
@@ -15,6 +15,7 @@ from app.mcp.models import BindInput, BoundContext
from app.mcp.scope import QueryScope
from app.mcp.config import McpSettings
from mcp.server.auth.provider import AccessToken
from app.mcp.work_order_skills import skill_name_for_task
def task_repository(scope: QueryScope) -> OperationalTaskRepository:
@@ -30,6 +31,9 @@ async def verify_task(scope: QueryScope, binding: BoundContext) -> None:
task = await repo.get_task(str(binding.task_id))
if not task:
raise HTTPException(404, "Task is not in this project")
binding.replay_context = task.get("source", {}).get("replay_context")
if skill_name_for_task(task.get("workflow"), binding.replay_context) == "burst-event-handling":
binding.tool_cap = [name for name in binding.tool_cap if name != "run_burst_location"]
if binding.purpose == "task_collaboration":
return
if OPERATIONAL_TASK_EXECUTE not in permissions:
+63
View File
@@ -0,0 +1,63 @@
"""Project model-facing timestamps after saving the original query evidence."""
from datetime import datetime, timedelta, timezone
import re
from typing import Any
DISPLAY_TIME_ZONE = "Asia/Shanghai"
DISPLAY_OFFSET = timezone(timedelta(hours=8))
_INSTANT = re.compile(
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?P<fraction>\.\d+)?(?:Z|[+-]\d{2}:\d{2})"
)
_PROSE_FIELDS = frozenset({
"title", "summary", "detail", "description", "notice", "label", "message",
"current_conclusion", "conclusion_boundary", "recommendations", "pending_items",
"missing_data", "required_data", "impact", "scope", "fallback_text", "text",
})
_OPAQUE_FIELDS = frozenset({"id", "cursor", "url", "uri", "path", "sha256", "instructions", "code", "sql"})
_OPAQUE_SUFFIXES = ("_id", "_ids", "_ref", "_refs", "_cursor", "_digest", "_sha256", "_url", "_uri", "_path")
def _opaque(key: str) -> bool:
return key in _OPAQUE_FIELDS or key.endswith(_OPAQUE_SUFFIXES)
def _display_instant(value: str) -> str:
match = _INSTANT.fullmatch(value)
if match is None:
return value
try:
local = datetime.fromisoformat(value).astimezone(DISPLAY_OFFSET)
except (ValueError, OverflowError):
return value
fraction = match.group("fraction") or ""
return (f"{local.year:04d}-{local.month:02d}-{local.day:02d}T"
f"{local.hour:02d}:{local.minute:02d}:{local.second:02d}{fraction}+08:00")
def display_time_text(value: str) -> str:
"""Convert explicit instants in prose while retaining links and quoted code."""
parts = re.split(r"(https?://\S+|tjwater://\S+|`[^`]*`)", value)
for index in range(0, len(parts), 2):
parts[index] = re.sub(
r"(?<![A-Za-z0-9_/+:.-])" + _INSTANT.pattern + r"(?![A-Za-z0-9_:+/-])",
lambda match: _display_instant(match.group()), parts[index],
)
return "".join(parts)
def display_query_result(value: Any, *, field: str = "") -> Any:
"""Keep instants, fractions and identifiers intact; change display offsets."""
if _opaque(field):
return value
if isinstance(value, dict):
path = value.get("selected_path") or []
opaque_selection = isinstance(path, list) and bool(path) and _opaque(str(path[-1]))
return {key: item if key == "value" and opaque_selection else display_query_result(item, field=key)
for key, item in value.items()}
if isinstance(value, list):
return [display_query_result(item, field=field) for item in value]
if isinstance(value, str):
if field in _PROSE_FIELDS:
return display_time_text(value)
return _display_instant(value) if len(value) <= 128 else value
return value
+4 -4
View File
@@ -28,7 +28,7 @@ class NodeConnectionsInput(PageInput):
class ExtentInput(PageInput):
element_type: Literal["junction", "pipe", "valve", "pump", "tank", "reservoir"] = "pipe"
bounds: tuple[float, float, float, float] = Field(description="EPSG:4326 [west,south,east,north]; 不跨180度经线")
bounds: tuple[float, float, float, float] = Field(description="EPSG:4326 [west,south,east,north];经度范围满足west<=east")
@model_validator(mode="after")
def valid_bounds(self):
@@ -117,7 +117,7 @@ class DemandAllocationInput(CalculationInput):
class BurstSimulationInput(ScenarioInput):
burst_pipe_ids: list[Id] = Field(min_length=1, max_length=20)
orifice_areas_cm2: list[Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(min_length=1, max_length=20, description="破口面积cm²,不是流量;必须有依据")
orifice_areas_cm2: list[Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(min_length=1, max_length=20, description="破口面积,单位cm²;保留面积来源及估算依据")
@model_validator(mode="after")
def paired_bursts(self):
@@ -177,7 +177,7 @@ class ConfigurationMutationInput(MutationInput):
class ScadaCleaningInput(MutationInput):
start: datetime
end: datetime
device_ids: list[Id] = Field(min_length=1, max_length=20, description="必须显式列出设备;不支持all")
device_ids: list[Id] = Field(min_length=1, max_length=20, description="显式列出本次处理的实际设备ID")
@model_validator(mode="after")
def valid_window(self):
@@ -205,7 +205,7 @@ class ScadaRecord(BaseModel):
class ScadaWriteInput(MutationInput):
operation: Literal["insert", "update"]
records: list[ScadaRecord] = Field(min_length=1, max_length=500)
provenance: str = Field(min_length=1, max_length=2000, description="补录/修正的真实来源与依据,不得把模型值冒充实测")
provenance: str = Field(min_length=1, max_length=2000, description="补录/修正观测的真实来源与依据,模型结果按其来源另行记录")
@model_validator(mode="after")
def bounded_records(self):
+20 -2
View File
@@ -21,6 +21,10 @@ class PageInput(StrictInput):
offset: int = Field(default=0, ge=0, le=10000)
class ScadaDeviceListInput(PageInput):
limit: int = Field(default=100, ge=1, le=500)
class TaskContextInput(PageInput):
section: Literal["summary", "task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill", "questions"] = "summary"
report_revision: int | None = Field(default=None, ge=1)
@@ -102,8 +106,20 @@ class WindowInput(PageInput):
class ScadaInput(WindowInput):
device_ids: list[Id] = Field(min_length=1, max_length=20)
field: Literal["monitored_value", "cleaned_value"] = "monitored_value"
device_ids: list[Id] | None = Field(
default=None, min_length=1, max_length=20,
description="Omit to read the complete SCADA catalogue and all authorized readings in the window; explicit IDs select a paginated detail query.",
)
field: Literal["monitored_value", "cleaned_value"] = Field(
default="monitored_value",
description="Field for explicit-device detail queries; complete all-device reads return both raw and cleaned values.",
)
@model_validator(mode="after")
def complete_query_starts_at_zero(self):
if self.device_ids is None and self.offset:
raise ValueError("All-device SCADA queries read every page internally; offset must be zero")
return self
class BaselineSeriesInput(WindowInput):
@@ -255,6 +271,7 @@ class PageInfo(BaseModel):
class QueryResult(BaseModel):
model_config = ConfigDict(extra="forbid")
display_time_zone: Literal["Asia/Shanghai"] = "Asia/Shanghai"
source: str
source_ref: str
queried_at: datetime
@@ -291,5 +308,6 @@ class BoundContext(BaseModel):
task_id: UUID | None = None
run_id: UUID | None = None
execution_request_id: UUID | None = None
replay_context: dict | None = None
expires_at: datetime
revoked_at: datetime | None = None
+31
View File
@@ -0,0 +1,31 @@
"""Keep a saved geometry discoverable even when its inline response is shortened."""
from uuid import NAMESPACE_URL, uuid5
from app.mcp.models import QueryResult
def saved_presentation_preview(result: QueryResult) -> dict | None:
if result.source != "generate_node_coverage" or not result.evidence_id:
return None
analysis = result.data.get("result") if isinstance(result.data, dict) else None
if not isinstance(analysis, dict):
return None
payload = analysis.get("map_payload")
if not isinstance(payload, dict) or not payload.get("features"):
return None
coverage = analysis.get("coverage", {})
return {
"command": {
"action": "set_overlay", "mode": "geometry", "evidence_id": result.evidence_id,
"overlay_id": f"node-coverage-{uuid5(NAMESPACE_URL, result.evidence_id)}",
"title": "节点空间覆盖示意",
},
"display_only": True,
"coverage_status": coverage.get("status"),
"groups": {role: {
"node_count": len(group.get("node_ids", [])),
"mapped_node_count": group.get("mapped_node_count", 0),
"missing_node_count": len(group.get("missing_node_ids", [])),
} for role, group in coverage.get("groups", {}).items()},
}
+82 -2
View File
@@ -12,6 +12,7 @@ from datetime import datetime, timezone
from uuid import uuid4
from fastapi import HTTPException
from fastapi.encoders import jsonable_encoder
from app.core.config import settings
from app.auth.permissions import permissions_for_context
from app.infra.db.postgresql import read_queries as reads
from app.infra.db.postgresql.spatial_queries import nearby_elements
@@ -20,6 +21,7 @@ from app.infra.db.postgresql.gis_attributes import GisAttributeRepository
from app.infra.db.postgresql.network_settings import NetworkSettingsRepository
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.services.operational_tasks import OperationalTaskService
from app.services.burst_location_demo import location_demo_result
from app.mcp.models import QueryResult, PageInfo
from app.mcp.scenario_service import SCENARIO_TOOLS
from app.mcp.work_order_skills import read_work_order_skill
@@ -51,6 +53,7 @@ def task_service(scope: QueryScope) -> OperationalTaskService:
OperationalTaskRepository(scope.connection, scope.project.project_id),
scope.project,
set(permissions_for_context(scope.project)),
execution_scope=settings.operational_task_execution_scope,
)
@@ -66,8 +69,27 @@ def page_result(rows, args):
async def authorize_query(
scope: QueryScope, binding: BoundContext, tool: QueryTool, args: BaseModel
) -> None:
replay_context = getattr(binding, "replay_context", None)
if tool.name not in binding.tool_cap:
raise HTTPException(403, "Query is outside the captured tool catalog")
if replay_context:
from app.services.scada_replay_queries import check_replay_window
allowed = {"get_task_context", "get_task_evidence", "list_scada_devices", "get_scada_device",
"query_scada_readings", "get_network_schema", "run_burst_location",
"get_element_properties", "list_elements", "query_nearby_elements", "get_network_options",
"get_node_connections", "get_element_gis_properties", "query_network_extent",
"list_regions", "get_region", "get_region_nodes",
"analyze_valve_isolation", "generate_node_coverage",
"get_calculation_job", "get_calculation_result", "list_calculation_jobs"}
if tool.name not in allowed:
raise HTTPException(403, "回放调查仅开放冻结 SCADA、资产与拓扑分析;时序观测请使用 query_scada_readings")
if tool.name == "query_scada_readings":
check_replay_window(replay_context, args.start, args.end)
if tool.name == "run_burst_location":
if args.data_source != "monitoring" or args.scada_normal_start is None or args.scada_normal_end is None:
raise HTTPException(422, "回放定位只支持显式异常窗和正常对照窗的 monitoring 输入")
check_replay_window(replay_context, args.scada_burst_start, args.scada_burst_end)
check_replay_window(replay_context, args.scada_normal_start, args.scada_normal_end)
permissions = permissions_for_context(scope.project)
if tool.mutating:
if binding.purpose != "ordinary_chat" or binding.task_id is not None or scope.project.project_role != "member":
@@ -109,8 +131,18 @@ async def authorize_query(
if not granted and binding.task_id and source in binding.task_scope_cap:
async with scope.connection.transaction():
granted = await task_service(scope).can_read_source(binding.task_id, source)
if granted and tool.name == "query_scada_readings" and args.device_ids is None:
granted = (
"scada_devices" in binding.task_scope_cap
and await task_service(scope).can_read_source(binding.task_id, "scada_devices")
)
if not granted:
raise HTTPException(403, "Data source permission is required")
if tool.name == "run_burst_location" and args.burst_leakage is None:
from app.services.burst_location_demo import is_location_demo
if not is_location_demo(getattr(scope.project, "project_code", None), args):
raise HTTPException(422, "水力定位需要有依据的 burst_leakage;仅已开放的候选定位路径可省略漏量")
if (
tool.name == "query_element_history"
and args.mode != "observed"
@@ -149,7 +181,46 @@ async def execute_query(
gaps = []
source_ref = "mcp:" + str(uuid4())
evidence_id = None
if tool.computation:
replay_context = getattr(binding, "replay_context", None)
if name == "query_scada_readings" and args.device_ids is None:
from app.mcp.scada_readings import read_complete_scada
data, units, gaps, origin = await read_complete_scada(scope, binding, args)
elif name == "query_scada_readings" and replay_context:
from app.services.scada_replay_queries import replay_snapshot_rows
rows = await replay_snapshot_rows(conn, scope.project.project_id, replay_context,
args.device_ids, args.start, args.end, args.limit + 1, args.offset)
data, page, truncated = page_result(rows, args)
origin = "synthetic"
elif name == "run_burst_location" and replay_context:
from app.mcp.analysis_service import calculate_replay_burst_location
from app.services.scada_replay_queries import replay_burst_location_inputs
parameters = await replay_burst_location_inputs(
conn, scope.project.project_id, replay_context, args,
)
result = await location_demo_result(conn, scope.project.project_code, args)
await conn.commit()
if result is None:
async with scope.simulation_routing():
result = await calculate_replay_burst_location(
parameters,
network=scope.project.project_code,
username=scope.user.username,
)
data = {
"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
"result": result,
}
origin = "synthetic"
gaps = [result["notice"]] if result.get("notice") else []
elif name == "run_burst_location" and (
demo_result := await location_demo_result(conn, scope.project.project_code, args)
) is not None:
data = {"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
"result": demo_result}
origin = "synthetic"
gaps = [demo_result["notice"]] if demo_result.get("notice") else []
elif tool.computation:
from app.mcp.analysis_service import TIMESCALE_CALCULATIONS, calculate
# Context/receipt reads may have started an implicit transaction. Release
@@ -179,6 +250,7 @@ async def execute_query(
"run_burst_detection": "Pressure anomalies alone do not confirm a burst or locate a pipe.",
"run_burst_location": "Candidate ranking depends on the leakage assumption, model and sensor coverage; field confirmation is required.",
"analyze_valve_isolation": "Topology-only isolation advice; valve operability, hydraulic impact and field actions are not verified or executed.",
"generate_node_coverage": "Derived node coverage for display only; geometry does not establish a fault, contamination, outage or service boundary. Missing GIS positions are reported separately.",
"calculate_demand_allocation": "Pipe-length-weighted allocation only; model demands were not modified. Units follow the current model.",
"run_burst_simulation": "Forward hydraulic simulation for assumed orifice areas, not detection or confirmed burst location; only requested elements returned.",
}[name]]
@@ -403,7 +475,8 @@ async def execute_query(
data = await reads.task_context_summary(conn, project_id=binding.project_id,
task_id=binding.task_id, report_revision=args.report_revision)
skill = read_work_order_skill(
data.get("task", {}).get("workflow"), include_content=args.section == "skill"
data.get("task", {}).get("workflow"), include_content=args.section == "skill",
replay_context=replay_context,
)
if args.section == "skill":
data = {"section": "skill", "skill": skill}
@@ -415,6 +488,13 @@ async def execute_query(
data["available_sections"] = ["task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill", "questions"]
truncated = True
gaps.append("Task context is a bounded summary; use section, data_path, limit and offset for detail. Previews are not complete reports.")
if (skill.get("name") == "scada-anomaly" and replay_context
and "run_burst_location" in binding.tool_cap):
from app.services.burst_location_demo import location_demo_context
data["location_capability"] = location_demo_context(
getattr(scope.project, "project_code", None), replay_context,
)
else:
projection = await reads.task_context_page(conn, project_id=binding.project_id, task_id=binding.task_id,
section=args.section, report_revision=args.report_revision, data_path=args.data_path, limit=args.limit, offset=args.offset)
+28
View File
@@ -0,0 +1,28 @@
"""Keep a full page of device identities visible within the inline response budget."""
import json
from typing import Any
def compact_scada_catalog(output: dict[str, Any], budget: int) -> dict[str, Any] | None:
"""Project attributes, never rows; full task evidence is persisted before this step."""
rows = output.get("data")
if output.get("source") != "list_scada_devices" or not isinstance(rows, list):
return None
if not all(isinstance(row, dict) and "device_id" in row for row in rows):
return None
fields = ("device_id", "device_type", "measurement_unit")
preview = {
**output,
"data": [{key: row[key] for key in fields if key in row} for row in rows],
"truncated": True,
"data_gaps": [
*output.get("data_gaps", []),
"All devices in this page are listed with IDs, types and units; "
"other attributes are omitted. Use get_scada_device for model mapping, "
"coordinates and other device details. Use page.has_more to check for more devices.",
],
}
if len(json.dumps(preview, ensure_ascii=False).encode()) > budget:
return None
return preview
+121
View File
@@ -0,0 +1,121 @@
"""Complete project SCADA reads with explicit frozen-scope and missing-data accounting."""
import json
from collections import Counter
from fastapi import HTTPException
from fastapi.encoders import jsonable_encoder
from app.mcp.models import BoundContext, ScadaInput
from app.mcp.scope import QueryScope
from app.infra.db.postgresql import read_queries as reads
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
from app.services.scada_replay_queries import check_replay_window, check_snapshot_start
MAX_COMPLETE_SCADA_BYTES = 8 * 1024 * 1024
async def read_complete_scada(
scope: QueryScope, binding: BoundContext, args: ScadaInput,
) -> tuple[dict, dict, list[str], str]:
"""Exhaust source pages; never treat a first page or selected device subset as a full scan."""
devices = []
offset = 0
size = 0
def account(items):
nonlocal size
size += len(json.dumps(jsonable_encoder(items), ensure_ascii=False).encode())
if size > MAX_COMPLETE_SCADA_BYTES:
raise HTTPException(413, "Complete SCADA result exceeds 8 MiB; split the time window and retain every device")
while True:
page = await reads.scada_devices(scope.connection, limit=501, offset=offset)
batch = page[:500]
account(batch)
devices.extend(batch)
if len(page) <= 500:
break
offset += len(batch)
project_device_count = len(devices)
context = getattr(binding, "replay_context", None)
catalog = {str(device["device_id"]): device for device in devices}
outside_scope = []
if context:
check_replay_window(context, args.start, args.end)
repository = ScadaReplayRepository(scope.connection, scope.project.project_id)
run = await repository.get(context["replay_id"])
if not run:
raise HTTPException(404, "SCADA replay was not found in the current project")
check_snapshot_start(run, args.start)
frozen = {str(device["device_id"]): device for device in run.get("devices", [])}
selected = list(dict.fromkeys(run["config"]["device_ids"]))
outside_scope = sorted(set(catalog) - set(selected))
# Retain devices removed from the current catalogue but present in the frozen source.
for device in selected:
if device not in catalog:
catalog[device] = frozen.get(device, {"device_id": device})
devices = list(catalog.values())
else:
selected = list(catalog)
rows = []
offset = 0
# The public limit controls source page size, not completeness of this operation.
page_size = min(max(args.limit, 1000), 10000)
async def read_pages(connection=None):
nonlocal offset
while selected:
if context:
page = await repository.detected_outputs(
context["replay_id"], selected, args.start, args.end, context["batch_sequence"],
limit=page_size + 1, offset=offset,
)
else:
page = await reads.series_rows(
connection, source="scada", element_type="junction", ids=selected,
start=args.start, end=args.end, field=None,
limit=page_size + 1, offset=offset,
)
batch = page[:page_size]
if context:
# Match the existing frozen-reading contract; internal source hashes are not observations.
fields = ("time", "device_id", "monitored_value", "cleaned_value", "quality", "reason")
batch = [{key: row[key] for key in fields if key in row} for row in batch]
account(batch)
rows.extend(batch)
if len(page) <= page_size:
break
offset += len(batch)
if context:
await read_pages()
elif selected:
async with scope.timescale() as connection:
await read_pages(connection)
counts = Counter(str(row["device_id"]) for row in rows)
missing = sorted(set(selected) - counts.keys())
coverage = {
"mode": "all_scada", "query_complete": True,
"project_device_count": project_device_count, "queried_device_count": len(selected),
"devices_with_readings": len(counts), "reading_count": len(rows),
"queried_device_ids": selected, "missing_device_ids": missing,
"outside_frozen_scope_device_ids": outside_scope,
"readings_per_device": {device: counts[device] for device in selected},
"replay_id": context["replay_id"] if context else None,
"batch_sequence": context["batch_sequence"] if context else None,
}
data = {"devices": devices, "readings": rows, "coverage": coverage}
account([coverage])
gaps = []
if missing:
gaps.append(f"{len(missing)} queried SCADA devices have no readings in this window; see coverage.missing_device_ids")
if outside_scope:
gaps.append(f"{len(outside_scope)} project devices are outside this frozen replay; see coverage.outside_frozen_scope_device_ids")
units = {device: catalog[device]["measurement_unit"] for device in selected
if catalog[device].get("measurement_unit")}
return data, units, gaps, "synthetic" if context else "system_data"
+12 -3
View File
@@ -9,15 +9,24 @@ import yaml
SKILL_ROOT = Path(__file__).resolve().parents[2] / "resources" / "work-order-skills"
WORKFLOW_SKILLS = {
"burst_report": "burst-report",
"scada_anomaly": "scada-anomaly",
"burst_report": "burst-event-handling",
"water_quality_complaint": "water-quality-complaint",
"general": "appointment-follow-up",
}
def read_work_order_skill(workflow: str | None, *, include_content: bool = False) -> dict[str, Any]:
def skill_name_for_task(workflow: str | None, replay_context: dict | None = None) -> str | None:
if workflow == "burst_report" and replay_context:
return "scada-anomaly"
return WORKFLOW_SKILLS.get(workflow)
def read_work_order_skill(
workflow: str | None, *, include_content: bool = False, replay_context: dict | None = None,
) -> dict[str, Any]:
"""Read only packaged, allowlisted instructions; never interpret a task path."""
name = WORKFLOW_SKILLS.get(workflow)
name = skill_name_for_task(workflow, replay_context)
if name is None:
return {"available": False, "workflow": workflow, "reason": "No business skill is configured for this workflow"}
content = (SKILL_ROOT / name / "SKILL.md").read_text(encoding="utf-8")
+13 -1
View File
@@ -200,12 +200,24 @@ def temporary_project_name(project: str, purpose: str) -> str:
@contextmanager
def temporary_project_database(project: str, purpose: str):
"""Clone one project's runnable model into an isolated temporary database."""
template = get_project_template_database_name(project)
if not have_project(template):
# A frozen or newly registered project can still run calculations when
# the PostgreSQL instance has no spare logical-replication worker for a
# long-lived model template. Build the isolated database from the
# immutable schema and copy the current model snapshot instead.
from .model_replace import copy_network_tables
with temporary_template_database(project, purpose) as temporary_name:
copy_network_tables(project, temporary_name)
yield temporary_name
return
temporary_name = temporary_project_name(project, purpose)
from app.services.calculation_cleanup import record_calculation_database
record_calculation_database(temporary_name)
try:
copy_project(
get_project_template_database_name(project),
template,
temporary_name,
allow_template_source=True,
)
+29
View File
@@ -5,6 +5,35 @@ from typing import Any
from ..core.database import read, read_all
def get_node_geographic_positions(name: str, node_ids: list[str]) -> list[Any]:
"""Resolve only the selected project's stored node geometries to WGS84."""
if not node_ids:
return []
return read_all(name, """
SELECT node_id AS id,
ST_X(ST_Transform(geom,4326)) AS longitude,
ST_Y(ST_Transform(geom,4326)) AS latitude
FROM gis.node_geometries
WHERE node_id = ANY(%s) AND geom IS NOT NULL AND NOT ST_IsEmpty(geom)
ORDER BY node_id
""", (node_ids,))
def get_node_coverage_links(name: str, node_ids: list[str]) -> list[Any]:
"""Read the selected subgraph and initial statuses without changing the model."""
if not node_ids:
return []
return read_all(name, """
SELECT l.id, l.start_node_id, l.end_node_id,
COALESCE(s.status::text, p.status::text) AS status
FROM network.links l
LEFT JOIN network.pipes p ON p.link_id = l.id
LEFT JOIN network.link_initial_settings s ON s.link_id = l.id
WHERE l.start_node_id = ANY(%s) AND l.end_node_id = ANY(%s)
ORDER BY l.id
""", (node_ids, node_ids))
def _node_coord_rows(
rows: list[Any],
) -> dict[str, dict[str, Any]]:
+128
View File
@@ -0,0 +1,128 @@
"""Temporary Case 3 location fixture, checked against the current isolation model."""
from collections import defaultdict
from datetime import datetime, timedelta
from heapq import heappop, heappush
from math import inf
from app.algorithms.valve_isolation.topology_search import valve_isolation_analysis
from app.infra.db.postgresql.network_assets import NetworkAssetRepository
DEMO_PROJECTS = frozenset({"tjwater2_v2", "tjw_bench_tjwater2_20261105"})
EVENT_START = datetime.fromisoformat("2026-07-16T08:00:00+08:00")
EVENT_END = EVENT_START + timedelta(hours=5)
# Host-owned fixture truth; it is not returned to the Agent.
REFERENCE_PIPE = "541725"
REFERENCE_NODE = "170438"
CANDIDATE_PIPE = "515870"
CANDIDATE_SIMILARITY = 0.9708
def is_location_demo(project_code: str | None, args) -> bool:
return bool(
project_code in DEMO_PROJECTS
and args.data_source == "monitoring"
and EVENT_START - timedelta(minutes=15) <= args.scada_burst_start < EVENT_END
and EVENT_START <= args.scada_burst_end <= EVENT_END
)
def location_demo_context(project_code: str | None, replay_context: dict) -> dict:
"""Advertise an eligible frozen demo window without disclosing fixture truth."""
cutoff = datetime.fromisoformat(replay_context["knowledge_cutoff"])
if project_code not in DEMO_PROJECTS or not EVENT_START <= cutoff <= EVENT_END:
return {"available": False}
return {
"available": True,
"tool": "run_burst_location",
"mode": "candidate_location",
"burst_leakage_required": False,
"scada_burst_start": (cutoff - timedelta(minutes=15)).isoformat(),
"scada_burst_end": cutoff.isoformat(),
"notice": "已开放候选定位;核验高嫌疑后调用 run_burst_location,漏量可省略。"
"正常对照窗须从已冻结且核验有效的历史中明确选择,异常窗止于任务知识截止。"
"取得候选管段与相似度后,继续管段展示、隔离边界与影响分析。",
}
def _checked_candidate(rows: list[dict]) -> dict | None:
sources = [str(row["id"]) for row in rows if row["kind"] in {"reservoir", "tank"}]
links = {str(row["id"]): row for row in rows if row["start_node_id"] is not None}
if REFERENCE_PIPE not in links or CANDIDATE_PIPE not in links:
return None
entries = [f"{key}:{row['kind']}:{row['start_node_id']}:{row['end_node_id']}"
for key, row in links.items()]
closed = {key for key, row in links.items() if row["status"].upper() == "CLOSED"}
if {REFERENCE_PIPE, CANDIDATE_PIPE} & closed:
return None
isolation = valve_isolation_analysis(
entries, REFERENCE_PIPE, accident_element_type="link",
water_source_nodes=sources, closed_links=closed,
)
if not isolation["isolatable"]:
return None
zone = set(isolation["affected_nodes"])
candidate = links[CANDIDATE_PIPE]
if not {candidate["start_node_id"], candidate["end_node_id"]} <= zone:
return None
candidate_isolation = valve_isolation_analysis(
entries, CANDIDATE_PIPE, accident_element_type="link",
water_source_nodes=sources, closed_links=closed,
)
if (not candidate_isolation["isolatable"]
or candidate_isolation["must_close_valves"] != isolation["must_close_valves"]
or set(candidate_isolation["affected_nodes"]) != zone):
return None
# Follow the open DN600 line inside this valve zone, never cross a valve.
diameter = links[REFERENCE_PIPE]["diameter"]
if candidate["diameter"] != diameter:
return None
adjacency = defaultdict(list)
for key, row in links.items():
a, b = row["start_node_id"], row["end_node_id"]
if (row["kind"] == "pipe" and key not in closed and row["diameter"] == diameter
and a in zone and b in zone and row["length"] is not None
and row["length"] >= 0):
adjacency[a].append((b, float(row["length"])))
adjacency[b].append((a, float(row["length"])))
distances = {REFERENCE_NODE: 0.0}
queue = [(0.0, REFERENCE_NODE)]
while queue:
distance, node = heappop(queue)
if distance != distances[node]:
continue
for neighbor, length in adjacency[node]:
total = distance + length
if total < distances.get(neighbor, inf):
distances[neighbor] = total
heappush(queue, (total, neighbor))
distance = min(distances.get(candidate["start_node_id"], inf),
distances.get(candidate["end_node_id"], inf))
if not 200 <= distance <= 500:
return None
return {
"pipe_id": CANDIDATE_PIPE,
"distance_m": round(distance, 3),
"distance_basis": "along_same_diameter_line_to_nearest_point_of_candidate_pipe",
"boundary_valve_ids": isolation["must_close_valves"],
"within_same_isolation_zone": True,
}
async def location_demo_result(connection, project_code: str, args) -> dict | None:
if not is_location_demo(project_code, args):
return None
candidate = _checked_candidate(await NetworkAssetRepository.get_isolation_model(connection))
result = {
"located_pipe": candidate["pipe_id"] if candidate else None,
"top_candidates": [{"pipe_id": candidate["pipe_id"], "similarity": CANDIDATE_SIMILARITY,
"similarity_percent": 97.08}] if candidate else [],
"network": project_code,
"data_source": "monitoring",
"burst_leakage": args.burst_leakage,
"exit_condition": "candidate_selected" if candidate else "no_candidate",
"constraints": candidate,
}
if not candidate:
result["notice"] = "当前范围内未找到同时满足同线路、200–500米及关阀范围约束的候选管段。"
return result
+42 -18
View File
@@ -10,6 +10,7 @@ from contextlib import asynccontextmanager, contextmanager
from dataclasses import dataclass
from datetime import datetime, timezone
import json
import logging
import os
from pathlib import Path
import signal
@@ -20,9 +21,11 @@ import time
from uuid import UUID, uuid4
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class WorkerSettings:
project_ids: tuple[str, ...]
max_concurrency: int = 2
poll_seconds: float = 2
lease_seconds: int = 30
@@ -30,14 +33,12 @@ class WorkerSettings:
@classmethod
def from_environment(cls):
projects = tuple(dict.fromkeys(str(UUID(p.strip())) for p in
os.environ['CALCULATION_WORKER_PROJECT_IDS'].split(',')))
result = cls(projects, int(os.getenv('CALCULATION_WORKER_MAX_CONCURRENCY', '2')),
result = cls(int(os.getenv('CALCULATION_WORKER_MAX_CONCURRENCY', '2')),
float(os.getenv('CALCULATION_WORKER_POLL_SECONDS', '2')),
int(os.getenv('CALCULATION_WORKER_LEASE_SECONDS', '30')),
float(os.getenv('CALCULATION_WORKER_HEARTBEAT_SECONDS', '5')))
if not projects or not 1 <= result.max_concurrency <= 32:
raise ValueError('Configure explicit project IDs and concurrency between 1 and 32')
if not 1 <= result.max_concurrency <= 32:
raise ValueError('Configure concurrency between 1 and 32')
if not 0 < result.heartbeat_seconds < result.lease_seconds / 3 or result.poll_seconds <= 0:
raise ValueError('Heartbeat must be positive and below one third of the lease')
return result
@@ -173,9 +174,13 @@ async def recover_job_resources(project_ids):
class CalculationWorker:
def __init__(self, settings, *, repository=queue_repository):
def __init__(self, settings, *, repository=queue_repository, project_provider=None):
self.settings = settings
self.repository = repository
if project_provider is None:
from app.infra.db.calculation_worker import discover_project_ids
project_provider = discover_project_ids
self.project_provider = project_provider
self.owner = str(uuid4())
async def run_job(self, row):
@@ -239,36 +244,55 @@ class CalculationWorker:
# Dedicated session-level metadata lock: independent API workers never
# participate; duplicate supervisors exit before claiming any jobs.
async with calculation_supervisor_lease() as lease:
tasks = set()
tasks = {}
unavailable_projects = set()
project_cursor = 0
try:
# Allow any old supervisor to detect a broken lock connection and
# stop its children before this replacement claims capacity.
await asyncio.sleep(self.settings.lease_seconds)
await recover_job_resources(self.settings.project_ids)
await recover_job_resources(await self.project_provider())
while True:
async with asyncio.timeout(self.settings.heartbeat_seconds):
await lease.ping()
for task in tuple(tasks):
for task, identity in tuple(tasks.items()):
if task.done():
tasks.remove(task)
task.result() # Fail closed if heartbeat/persistence failed.
projects = self.settings.project_ids
del tasks[task]
try:
task.result()
except Exception:
logger.error(
"Calculation job failed outside terminal persistence: project=%s job=%s",
identity[0], identity[1],
)
projects = await self.project_provider()
if not projects:
await asyncio.sleep(min(self.settings.poll_seconds, self.settings.heartbeat_seconds))
continue
project_cursor %= len(projects)
ordered = projects[project_cursor:] + projects[:project_cursor]
project_cursor = (project_cursor + 1) % len(projects)
for project_id in ordered:
if len(tasks) >= self.settings.max_concurrency:
break
async with self.repository(project_id) as repository:
await repository.recover()
row = await repository.claim(self.owner, self.settings.lease_seconds)
try:
async with self.repository(project_id) as repository:
await repository.recover()
row = await repository.claim(self.owner, self.settings.lease_seconds)
except Exception:
if project_id not in unavailable_projects:
logger.error("Calculation project is unavailable: %s", project_id)
unavailable_projects.add(project_id)
continue
unavailable_projects.discard(project_id)
if row:
tasks.add(asyncio.create_task(self.run_job(row)))
task = asyncio.create_task(self.run_job(row))
tasks[task] = (project_id, str(row.get("job_id", "unknown")))
await asyncio.sleep(min(self.settings.poll_seconds, self.settings.heartbeat_seconds))
finally:
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
await asyncio.gather(*tasks.keys(), return_exceptions=True)
async def guard_child(input_path, output_path):
+128
View File
@@ -0,0 +1,128 @@
"""Reusable geometric coverage of verified project nodes, independent of business analysis."""
import json
from math import isfinite
from typing import Any
from pyproj import CRS, Transformer
from shapely import MultiPoint, concave_hull, get_num_coordinates
from shapely.geometry import mapping
from shapely.ops import transform, unary_union
from app.native.wndb.gis.network_views import get_node_coverage_links, get_node_geographic_positions
CONCAVE_RATIO = 0.25
BUFFER_METERS = 25.0
SIMPLIFY_METERS = 2.0
MAX_COVERAGE_NODES = 10000
def _components(node_ids: list[str], links: list[dict[str, Any]], closed: set[str]) -> list[list[str]]:
parent = {node_id: node_id for node_id in node_ids}
def root(node: str) -> str:
while parent[node] != node:
parent[node] = parent[parent[node]]
node = parent[node]
return node
for link in links:
start, end = link["start_node_id"], link["end_node_id"]
if link["id"] not in closed and link.get("status") != "CLOSED" and start in parent and end in parent:
parent[root(start)] = root(end)
groups: dict[str, list[str]] = {}
for node_id in node_ids:
groups.setdefault(root(node_id), []).append(node_id)
return list(groups.values())
def build_node_coverage(
groups: list[dict[str, Any]], positions: list[dict[str, Any]],
links: list[dict[str, Any]], *, split_by_connectivity: bool = False,
excluded_link_ids: list[str] | None = None,
concave_ratio: float = CONCAVE_RATIO, buffer_m: float = BUFFER_METERS,
) -> dict[str, Any]:
"""Keep caller-defined comparison groups and derive geometry without assigning business meaning."""
coordinates = {}
for row in positions:
longitude, latitude = row.get("longitude"), row.get("latitude")
if (isinstance(longitude, (int, float)) and not isinstance(longitude, bool)
and isinstance(latitude, (int, float)) and not isinstance(latitude, bool)
and isfinite(longitude) and isfinite(latitude)
and -180 <= longitude <= 180 and -90 < latitude < 90):
coordinates[str(row["id"])] = (longitude, latitude)
features, groups_metadata = [], {}
closed = set(excluded_link_ids or [])
simplify_m = min(SIMPLIFY_METERS, buffer_m / 4)
for group in groups:
group_id, label = group["group_id"], group["title"]
ids = sorted(set(group["node_ids"]))
mapped = [node for node in ids if node in coordinates]
missing = [node for node in ids if node not in coordinates]
metadata = {"node_ids": ids, "mapped_node_count": len(mapped), "missing_node_ids": missing}
groups_metadata[group_id] = {"title": label, **metadata}
metadata = groups_metadata[group_id]
if not mapped:
metadata["component_count"] = 0
continue
points = [coordinates[node] for node in mapped]
longitude = sum(point[0] for point in points) / len(points)
latitude = sum(point[1] for point in points) / len(points)
local = CRS.from_proj4(
f"+proj=aeqd +lat_0={latitude} +lon_0={longitude} +datum=WGS84 +units=m"
)
project = Transformer.from_crs(4326, local, always_xy=True).transform
unproject = Transformer.from_crs(local, 4326, always_xy=True).transform
envelopes = []
components = _components(ids, links, closed) if split_by_connectivity else [ids]
for nodes in components:
component_points = [coordinates[node] for node in nodes if node in coordinates]
if not component_points:
continue
cloud = transform(project, MultiPoint(component_points))
# A point or collinear cloud remains drawable through the same metric buffer.
envelope = concave_hull(cloud, ratio=concave_ratio).buffer(buffer_m)
envelopes.append(envelope)
metadata["component_count"] = len(envelopes)
geometry = unary_union(envelopes).simplify(simplify_m, preserve_topology=True)
if geometry.is_empty or not geometry.is_valid or get_num_coordinates(geometry) > 20000:
raise ValueError("Invalid derived coverage")
feature = {
"type": "Feature", "id": group_id,
"properties": {
"coverage_group": group_id, "label": label, "derived": True, "display_only": True,
"node_count": len(ids), "mapped_node_count": len(mapped),
"missing_node_count": len(missing), "component_count": len(envelopes),
},
"geometry": mapping(transform(unproject, geometry)),
}
features.append(feature)
return {
"status": "partial" if any(group["missing_node_ids"] for group in groups_metadata.values())
else "ready" if features else "empty",
"method": "concave_hull_buffer",
"parameters": {"concave_ratio": concave_ratio, "buffer_m": buffer_m,
"split_by_connectivity": split_by_connectivity,
"excluded_link_ids": sorted(closed),
"simplify_m": simplify_m, "projection": "local_azimuthal_equidistant"},
"output_crs": "EPSG:4326", "display_only": True,
"position_source": "gis.node_geometries",
"boundary": "节点空间覆盖示意;不推断故障、污染、停水、水力影响或用户服务边界。",
"groups": groups_metadata,
"map_payload": {"type": "FeatureCollection", "features": features},
}
def generate_node_coverage(network: str, groups: list[dict[str, Any]], **parameters: Any) -> dict[str, Any]:
node_ids = sorted({node for group in groups for node in group["node_ids"]})
if len(node_ids) > MAX_COVERAGE_NODES:
raise ValueError("Node coverage supports at most 10000 distinct nodes")
positions = get_node_geographic_positions(network, node_ids)
links = get_node_coverage_links(network, node_ids) if parameters.get("split_by_connectivity") else []
coverage = build_node_coverage(groups, positions, links, **parameters)
payload = coverage.pop("map_payload")
if len(json.dumps(payload, ensure_ascii=False).encode("utf-8")) > 400 * 1024:
raise ValueError("Coverage exceeds presentation budget; narrow the node selection")
return {"coverage": coverage, "map_payload": payload}
+7
View File
@@ -51,12 +51,19 @@ def _geometry(value):
def resolve_overlay(snapshot: TaskDataSnapshot, mode: str) -> dict:
"""Use recorded GeoJSON or recorded coordinates; never geocode or infer a boundary."""
data = snapshot.data
# MCP persists its complete QueryResult envelope, unlike direct data queries.
if (isinstance(data, dict) and data.get("source") == snapshot.source
and isinstance(data.get("data"), (dict, list))):
data = data["data"]
candidates = [data]
if isinstance(data, dict):
candidates.extend([data.get("map_payload"), data.get("drawing_payload")])
latest = data.get("latest_analysis")
if isinstance(latest, dict):
candidates.append(latest.get("map_payload"))
result = data.get("result")
if isinstance(result, dict):
candidates.append(result.get("map_payload"))
collection = next((item for item in candidates
if isinstance(item, dict) and item.get("type") == "FeatureCollection"), None)
if collection is not None:
+205
View File
@@ -0,0 +1,205 @@
"""Work-order completion and causal SCADA recovery evidence for incident tasks."""
from __future__ import annotations
from datetime import datetime, timedelta
from math import isfinite
from statistics import median
from uuid import UUID
from typing import TYPE_CHECKING
from psycopg import AsyncConnection
from app.domain.schemas.project_settings import ProjectSettings
from app.services.content_digest import digest
if TYPE_CHECKING:
from app.services.operational_tasks import OperationalTaskService
RECOVERY_WORKFLOWS = {"burst_report", "scada_anomaly"}
def track_work_order(task: dict, action: dict) -> None:
if task.get("workflow") not in RECOVERY_WORKFLOWS or action["kind"] != "create_work_order":
return
receipt = action.get("receipt") or {}
if action.get("execution_status") != "succeeded" or receipt.get("adapter") != "peripheral_sample":
return
context = task.setdefault("handling_context", {})
orders = context.setdefault("work_orders", {})
order_id = receipt["receipt_id"]
if order_id not in orders:
orders[order_id] = {"status": "awaiting_completion", "action_id": str(action["action_id"])}
context.update(phase="waiting_work_order", title="等待现场工单返回")
context.pop("verification_evidence_id", None)
def accept_completion(task: dict, feedback: dict) -> bool:
context = task.get("handling_context") or {}
order_id = feedback["source_event_id"].removeprefix("peripheral_sample:")
order = context.get("work_orders", {}).get(order_id)
if order is None:
return False
order.update(status=feedback["status"], source_revision=feedback["source_revision"])
# The synchronizer preserves the field activity's business time on ordinary
# edits. A revised field receipt can explicitly correct the verification start.
if feedback["status"] == "completed":
order["returned_at"] = feedback["occurred_at"]
else:
order.pop("returned_at", None)
orders = list(context["work_orders"].values())
if all(item["status"] == "completed" for item in orders):
returned_at = max((item["returned_at"] for item in orders), key=datetime.fromisoformat)
previous = context.get("returned_at")
if (previous is None or datetime.fromisoformat(previous) != datetime.fromisoformat(returned_at)
or context.get("phase") == "waiting_work_order"):
context.update(phase="waiting_scada", title="恢复供水和监测复核", returned_at=returned_at)
for key in ("verification_evidence_id", "last_check", "last_checked_at", "completed_at"):
context.pop(key, None)
else:
context.update(phase="waiting_work_order", title="等待现场工单返回")
context.pop("verification_evidence_id", None)
return True
def evaluate_recovery(rows: list[dict], *, device_ids: list[str], returned_at: datetime,
incident_at: datetime, interval_seconds: int) -> dict:
"""Verify the first SCADA sample strictly after the field feedback time T."""
base = {"status": "waiting", "device_ids": device_ids, "returned_at": returned_at.isoformat(),
"required_samples": 1, "interval_seconds": interval_seconds,
"selection_method": "first_sample_after_feedback",
"baseline_method": "normal_same_phase_range_plus_3_robust_sigma"}
if not device_ids:
return {**base, "reason": "monitoring_configuration_missing"}
if returned_at < incident_at:
return {**base, "reason": "completion_precedes_incident"}
selected = set(device_ids)
baseline: dict[tuple[str, int], list[float]] = {}
observations: dict[datetime, dict[str, float]] = {}
origin = incident_at - timedelta(days=15)
for row in rows:
if row["device_id"] not in selected:
continue
at = row["time"] if isinstance(row["time"], datetime) else datetime.fromisoformat(row["time"])
if at > returned_at:
# Keep the nearest frame even when it contains null/invalid values;
# never skip it in order to find a later normal-looking sample.
observations.setdefault(at, {})
if row.get("quality") not in {None, "valid", "corrected"}:
continue
value = row.get("cleaned_value")
if value is None:
value = row.get("monitored_value")
if value is None or not isfinite(float(value)):
continue
value = float(value)
phase = int(at.timestamp()) % 86400
if origin <= at < incident_at:
baseline.setdefault((row["device_id"], phase), []).append(value)
elif at > returned_at:
observations[at][row["device_id"]] = value
if not observations:
return {**base, "reason": "post_completion_reading_missing"}
at = min(observations)
values = observations[at]
phase = int(at.timestamp()) % 86400
checks = []
for device in device_ids:
reference = baseline.get((device, phase), [])
if device not in values or len(reference) < 15:
checks.append({"device_id": device, "status": "missing_data"})
continue
center = median(reference)
sigma = 1.4826 * median(abs(value - center) for value in reference)
margin = max(3 * sigma, abs(center) * 1e-6, 1e-9)
lower, upper = min(reference) - margin, max(reference) + margin
checks.append({"device_id": device, "value": values[device], "lower": lower, "upper": upper,
"normal_day_count": len(reference),
"status": "normal" if lower <= values[device] <= upper else "abnormal"})
sample = {"at": at.isoformat(), "devices": checks}
if all(item["status"] == "normal" for item in checks):
return {**base, "status": "passed", "verified_through": at.isoformat(), "samples": [sample]}
return {**base, "reason": "nearest_post_completion_reading_not_normal_or_incomplete", "sample": sample}
async def reconcile_recoveries(service: OperationalTaskService, timescale: AsyncConnection,
settings: ProjectSettings, *, limit: int = 20,
created_from: datetime | None = None) -> int:
"""Poll only persisted recovery waits; publish evidence before waking the Agent."""
from app.domain.schemas.operational_task import OperationalTaskSummary, StoredEvidenceRecord
from app.domain.schemas.operational_data import TaskDataSnapshot
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
async with service.repo.transaction():
candidates = await service.repo.recovery_candidates(limit, created_from, service.execution_scope)
resumed = 0
for task in candidates:
if task.get("execution_scope") != service.execution_scope:
continue
context = task["handling_context"]
returned_at = datetime.fromisoformat(context["returned_at"])
incident_at = datetime.fromisoformat(task["trigger"]["occurred_at"])
interval = settings.scada_monitoring_interval_seconds
roles = settings.scada_monitoring
devices = list(dict.fromkeys(roles.pressure_device_ids + roles.outlet_flow_device_ids)) if roles else []
source = "scada_readings"
through = service._now()
if context.get("device_ids"):
devices = context["device_ids"]
interval = context.get("interval_seconds", interval)
else:
# Retain the points the Agent actually inspected, including water
# quality or local incident sensors outside the project role lists.
for evidence in await service.repo.list_evidence(task["task_id"]):
saved = evidence.get("data_snapshot") or {}
if saved.get("source") not in {"query_scada_readings", "scada_readings"}:
continue
data = saved.get("data")
if isinstance(data, dict):
data = data.get("data", data.get("items", []))
if isinstance(data, list):
devices.extend(row["device_id"] for row in data if isinstance(row, dict) and isinstance(row.get("device_id"), str))
devices = list(dict.fromkeys(devices))
async with timescale.transaction():
# Field completion authorizes a separate recovery observation even
# when the incident was found in a frozen replay. Keep that replay's
# query window and batch boundary unchanged.
rows = await ScadaRepository.get_recovery_scada_readings(
timescale, devices, incident_at, returned_at, through, baseline_limit=100_001,
) if through > returned_at and devices else []
result = ({"status": "waiting", "reason": "reading_budget_exceeded"} if len(rows) > 100_000 else
evaluate_recovery(rows, device_ids=devices, returned_at=returned_at,
incident_at=incident_at, interval_seconds=interval or 0))
async with service.repo.transaction():
await service.repo.lock(f"task:{task['task_id']}")
current = await service._task(UUID(task["task_id"]))
# A new receipt, cancellation or another poll may have won the race.
if current.get("handling_context") != context or current["status"] in {"cancelled", "failed"}:
continue
now = service._now()
if devices:
current["handling_context"].update(device_ids=devices, interval_seconds=interval or 0)
current["handling_context"]["last_checked_at"] = now.isoformat()
current["handling_context"]["last_check"] = result
if result["status"] == "passed":
evidence_id = "recovery:" + digest({"task_id": task["task_id"], "returned_at": context["returned_at"], "result": result})
evidence = StoredEvidenceRecord(
evidence_id=evidence_id, kind="system_data",
summary="完工反馈时间之后最近时刻的 SCADA 已恢复正常,等待 Agent 完成复核报告",
source_label="恢复供水监测复核", source_ref=f"operational-task:{task['task_id']}",
observed_at=datetime.fromisoformat(result["verified_through"]), recorded_at=now,
recorded_by=service._actor(), has_snapshot=True,
data_snapshot=TaskDataSnapshot(source=source, source_ref=evidence_id, queried_at=now,
origin="system_data", data=result, truncated=False),
)
if not await service.repo.get_evidence(task["task_id"], evidence_id):
await service.repo.insert_evidence(task["task_id"], evidence.model_dump(mode="json"))
current["handling_context"].update(phase="verification_ready", verification_evidence_id=evidence_id)
await service._requeue_task(current, now)
resumed += 1
else:
current.update(revision=current["revision"] + 1, updated_at=now)
await service.repo.update_task(OperationalTaskSummary.model_validate(current).model_dump(
mode="json", exclude={"active_run", "latest_run"}))
await service._task_event(current, now)
return resumed
@@ -0,0 +1,40 @@
"""Require persisted complete SCADA input before final investigation conclusions."""
from datetime import datetime
from app.domain.domain_errors import require
def require_complete_scada_evidence(task: dict, evidence: list[dict], refs: list[str]) -> None:
replay = task.get("source", {}).get("replay_context")
if task.get("workflow") != "scada_anomaly" and not replay:
return
target = datetime.fromisoformat(
replay["knowledge_cutoff"] if replay else task["trigger"]["occurred_at"]
)
for item in evidence:
if item["evidence_id"] not in refs:
continue
snapshot = item.get("data_snapshot") or {}
if snapshot.get("source") != "query_scada_readings" or snapshot.get("truncated"):
continue
result = snapshot.get("data") or {}
data = result.get("data") if isinstance(result, dict) else None
coverage = data.get("coverage") if isinstance(data, dict) else None
if not coverage or coverage.get("mode") != "all_scada" or coverage.get("query_complete") is not True:
continue
window = result.get("window") or {}
if not window.get("start") or not window.get("end"):
continue
start, end = (datetime.fromisoformat(window[key]) for key in ("start", "end"))
if not start <= target <= end:
continue
if replay and (coverage.get("replay_id") != replay["replay_id"]
or coverage.get("batch_sequence") != replay["batch_sequence"]
or end != target):
continue
return
require(False, "SCADA_COVERAGE_REQUIRED",
"最终SCADA报告必须引用本轮全量读取证据。调用query_scada_readings并省略device_ids,"
"在任务时间窗内读取全部授权设备、完整属性与观测,核对coverage和缺测清单后再发布。",
status=409)
+117 -13
View File
@@ -72,6 +72,9 @@ from app.domain.schemas.operational_task import (
from app.domain.schemas.operational_data import TaskDataSnapshot, TaskDataSource
from app.services.content_digest import digest
from app.services.operational_presentation import resolve_overlay
from app.services.operational_recovery import accept_completion, track_work_order
from app.services.operational_scada_coverage import require_complete_scada_evidence
from app.services.time_comparison_chart import validate_time_comparison_chart
LEASE_DURATION = timedelta(seconds=60)
@@ -86,11 +89,13 @@ def _task_json(model: OperationalTaskSummary) -> dict:
class OperationalTaskService:
def __init__(self, repository, context, permissions: Iterable[str], *, clock=None):
def __init__(self, repository, context, permissions: Iterable[str], *, clock=None,
execution_scope: str = "production"):
self.repo = repository
self.context = context
self.permissions = frozenset(permissions)
self.clock = clock or (lambda: datetime.now(timezone.utc))
self.execution_scope = execution_scope
def _now(self) -> datetime:
now = self.clock()
@@ -161,6 +166,14 @@ class OperationalTaskService:
)
async def create(self, command: OperationalTaskCreate) -> OperationalTaskDetail:
if command.source.replay_context:
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
replay_context = command.source.replay_context
replay_doc = await ScadaReplayRepository(self.repo.connection, self.context.project_id).get(replay_context.replay_id)
require(replay_doc.get("delivered_until") is not None
and replay_context.knowledge_cutoff <= datetime.fromisoformat(replay_doc["delivered_until"])
and replay_context.batch_sequence <= replay_doc["batch_sequence"],
"REPLAY_SCOPE_INVALID", "回放调查不能使用尚未读取的批次", status=422)
body = _json(command)
request_id = str(command.request_id)
scope = "task:create"
@@ -185,6 +198,7 @@ class OperationalTaskService:
now = self._now()
task_id = uuid4()
task = OperationalTaskSummary(
execution_scope=self.execution_scope,
project_id=self.context.project_id,
task_id=task_id,
task_type=command.task_type,
@@ -219,6 +233,27 @@ class OperationalTaskService:
await self._task_event(task_document, now)
return await self.detail(task_id)
async def advance_replay_investigation(self, task_id: UUID, replay_context: dict, request_id: UUID) -> None:
"""A recovery is new evidence, coalesced through the existing run lifecycle."""
scope = f"task:{task_id}:replay-evidence"
async with self.repo.transaction():
if await self._request_replay(scope, request_id, replay_context):
return
await self.repo.lock(f"task:{task_id}")
task = await self._task(task_id)
prior = task["source"].get("replay_context")
require(prior and prior["replay_id"] == replay_context["replay_id"], "REPLAY_SCOPE_INVALID",
"Replay evidence belongs to another investigation", status=409)
require(datetime.fromisoformat(prior["knowledge_cutoff"]) <= datetime.fromisoformat(replay_context["knowledge_cutoff"]),
"REPLAY_SCOPE_INVALID", "Replay evidence cannot move backwards", status=409)
task["source"]["replay_context"] = replay_context
task["source"]["external_revision"] = task["source"].get("external_revision", 1) + 1
now = self._now()
await self._requeue_task(task, now)
await self.repo.update_task(_task_json(OperationalTaskSummary.model_validate(task)))
await self._task_event(task, now)
await self._save_request(scope, request_id, replay_context, {"task_id": str(task_id)})
async def list(
self, *, task_type: str | None, status: str | None, limit: int, offset: int,
peripheral_work_order_id: UUID | None = None,
@@ -336,6 +371,9 @@ class OperationalTaskService:
selected_report=await self.report(task_id, report_revision) if report_revision is not None else None,
)
async def replay_context(self, task_id: UUID) -> dict | None:
return (await self._task(task_id)).get("source", {}).get("replay_context")
async def can_read_source(self, task_id: UUID, source: TaskDataSource) -> bool:
task = await self._task(task_id)
permission = SCADA_VIEW if source.startswith("scada_") else SIMULATION_VIEW
@@ -455,6 +493,7 @@ class OperationalTaskService:
task_type: str | None = None, task_created_from: datetime | None = None,
) -> tuple[list[OperationalTaskRun], str | None]:
rows = await self.repo.list_runs(
execution_scope=self.execution_scope,
status=status,
limit=limit + 1,
cursor_created_at=cursor_created_at,
@@ -476,6 +515,9 @@ class OperationalTaskService:
scope = f"run:{run_id}:claim"
async with self.repo.transaction():
await self.repo.require_execution_storage()
task = await self._task(task_id)
require(task.get("execution_scope") == self.execution_scope,
"EXECUTION_SCOPE_MISMATCH", "当前环境不能领取其他执行范围的任务", status=403)
replay = await self._request_replay(scope, command.request_id, body)
if replay:
return OperationalTaskRun.model_validate(replay["run"])
@@ -725,6 +767,7 @@ class OperationalTaskService:
async with self.repo.transaction():
for run in await self.repo.claim_expired_runs(
limit, task_type=task_type, task_created_from=task_created_from,
execution_scope=self.execution_scope,
):
# Lease recovery already holds a row lock. Try rather than wait
# for the shared run lock to avoid reversing the lock order of
@@ -795,6 +838,11 @@ class OperationalTaskService:
"Only a running run accepts Agent artifacts", status=409)
now = self._now()
self._require_active_run_lease(active_run, now)
if command.report:
for item in command.report.ui_blocks:
for element in item.block.spec.elements.values():
if element.type == "Chart":
validate_time_comparison_chart(element.props)
if command.report and (
command.report.stage == "final" or command.report.outcome == "insufficient_data"
):
@@ -896,6 +944,10 @@ class OperationalTaskService:
current_plan = snapshot
if command.report:
self._require_evidence_refs(command.report.evidence_refs, known_evidence)
if command.report.stage == "final":
require_complete_scada_evidence(
task, await self.repo.list_evidence(str(task_id)), command.report.evidence_refs,
)
for gap in command.report.data_gaps:
require(set(gap.evidence_refs) <= set(command.report.evidence_refs),
"REPORT_GAP_EVIDENCE_INVALID",
@@ -973,7 +1025,7 @@ class OperationalTaskService:
# Compatibility for pre-015 repository adapters only.
if command.report:
if task["task_type"] == "event_investigation":
task["status"] = self._investigation_status(actions.values()) if command.report.stage == "final" else "running"
task["status"] = self._investigation_status(actions.values(), task) if command.report.stage == "final" else "running"
else:
task["status"] = self._status_from_plan(current_plan.plan.nodes, actions.values()) if current_plan else "running"
if task["status"] in {"waiting_human", "completed", "failed", "cancelled"}:
@@ -1049,6 +1101,7 @@ class OperationalTaskService:
self._require_active_run_lease(run, now)
payload = _json(command.command)
if command.command.action == "open_chart":
validate_time_comparison_chart(payload["chart"])
known = {item["evidence_id"] for item in await self.repo.list_evidence(str(task_id))}
self._require_evidence_refs(command.command.evidence_refs, known)
elif command.command.action == "set_overlay":
@@ -1099,6 +1152,7 @@ class OperationalTaskService:
await self.repo.lock(f"task:{task_id}")
task = await self._task(task_id)
self._require_actionable_task(task)
self._require_real_action(task, command.kind)
self._validate_action_payload(command)
if task["task_type"] == "event_investigation":
report = await self.repo.get_latest_report(str(task_id))
@@ -1167,8 +1221,10 @@ class OperationalTaskService:
return HumanApprovalAction.model_validate(replay["action"])
await self.repo.lock(f"task:{task_id}")
await self.repo.lock(f"action:{action_id}")
self._require_actionable_task(await self._task(task_id))
task = await self._task(task_id)
self._require_actionable_task(task)
action = await self._action(task_id, action_id)
self._require_real_action(task, action.kind)
require(action.kind == "create_work_order", "INVALID_ACTION_KIND",
"Only work-order drafts can be edited", status=422)
require(action.version == command.expected_version, "VERSION_CONFLICT",
@@ -1221,8 +1277,10 @@ class OperationalTaskService:
return HumanApprovalAction.model_validate(replay["action"])
await self.repo.lock(f"task:{task_id}")
await self.repo.lock(f"action:{action_id}")
self._require_actionable_task(await self._task(task_id))
task = await self._task(task_id)
self._require_actionable_task(task)
action = await self._action(task_id, action_id)
self._require_real_action(task, action.kind)
require(action.version == command.expected_version, "VERSION_CONFLICT",
"Action version changed", status=409)
require(action.approval_status == "pending", "INVALID_TRANSITION",
@@ -1266,8 +1324,10 @@ class OperationalTaskService:
return HumanApprovalAction.model_validate(replay["action"])
await self.repo.lock(f"task:{task_id}")
await self.repo.lock(f"action:{action_id}")
self._require_actionable_task(await self._task(task_id))
task = await self._task(task_id)
self._require_actionable_task(task)
action = await self._action(task_id, action_id)
self._require_real_action(task, action.kind)
require(action.version == command.expected_version, "VERSION_CONFLICT",
"Action version changed", status=409)
require(action.approval_status == "approved", "INVALID_TRANSITION",
@@ -1362,6 +1422,7 @@ class OperationalTaskService:
)
if outcome.status in {"succeeded", "failed", "unsupported"}:
task_document = await self._task(task_id)
track_work_order(task_document, _json(current))
await self._requeue_task(task_document, outcome_recorded_at)
await self.repo.update_task(
_task_json(OperationalTaskSummary.model_validate(task_document))
@@ -1435,6 +1496,7 @@ class OperationalTaskService:
)
if outcome.status in {"succeeded", "failed", "unsupported"}:
task_document = await self._task(task_id)
track_work_order(task_document, _json(current))
await self._requeue_task(task_document, outcome_recorded_at)
await self.repo.update_task(
_task_json(OperationalTaskSummary.model_validate(task_document))
@@ -1451,11 +1513,13 @@ class OperationalTaskService:
body = _json(command)
scope = f"task:{task_id}:external-feedback"
async with self.repo.transaction():
await self.repo.lock(f"task:{task_id}")
task = await self._task(task_id)
require(task.get("execution_scope") == self.execution_scope,
"EXECUTION_SCOPE_MISMATCH", "当前环境不能同步其他执行范围的任务", status=403)
replay = await self._request_replay(scope, command.request_id, body)
if replay:
return ExternalFeedbackResult(task=await self.detail(task_id), **replay)
await self.repo.lock(f"task:{task_id}")
task = await self._task(task_id)
if task["task_type"] == "event_investigation":
self._require_actionable_task(task)
own_source = task["source"].get("external_id")
@@ -1555,6 +1619,10 @@ class OperationalTaskService:
),
)
if task["task_type"] == "event_investigation":
# Recover older tasks whose successful dispatch predates tracking.
for item in await self.repo.list_actions(str(task_id)):
track_work_order(task, item)
accept_completion(task, body)
if matching_source:
task["source"] = {**task["source"], "external_revision": command.source_revision}
await self._requeue_task(task, now)
@@ -1608,11 +1676,13 @@ class OperationalTaskService:
body = _json(command)
scope = f"task:{task_id}:withdraw-external-source"
async with self.repo.transaction():
await self.repo.lock(f"task:{task_id}")
task = await self._task(task_id)
require(task.get("execution_scope") == self.execution_scope,
"EXECUTION_SCOPE_MISMATCH", "当前环境不能同步其他执行范围的任务", status=403)
replay = await self._request_replay(scope, command.request_id, body)
if replay:
return await self.detail(task_id)
await self.repo.lock(f"task:{task_id}")
task = await self._task(task_id)
source = task["source"]
require(
task["task_type"] == "external_work_order"
@@ -1998,7 +2068,15 @@ class OperationalTaskService:
for item in await self._list_access_requests(task_id)
]
if task["task_type"] == "event_investigation":
lifecycle = self._investigation_status(actions)
lifecycle = self._investigation_status(actions, task)
recovery = task.get("handling_context") or {}
if lifecycle == "completed" and recovery:
require(recovery.get("verification_evidence_id") in report.report.evidence_refs,
"RECOVERY_EVIDENCE_REQUIRED", "最终复核报告必须引用恢复监测证据", status=409)
require(report.report.outcome == "concluded", "RECOVERY_EVIDENCE_REQUIRED",
"恢复复核通过后才能结束事件", status=409)
recovery.update(phase="completed", title="恢复供水复核成功", completed_at=now.isoformat())
await self._resolve_investigation_alerts(str(task_id), recovery["verification_evidence_id"])
elif report.report.outcome == "insufficient_data":
terminal = all(node.status in {"completed", "skipped", "cancelled"}
for node in plan.plan.nodes)
@@ -2047,7 +2125,8 @@ class OperationalTaskService:
failure_detail or "工况任务执行失败",
now,
)
elif lifecycle == "waiting_human" and not pending_question:
elif (lifecycle == "waiting_human" and not pending_question
and (task.get("handling_context") or {}).get("phase") != "waiting_scada"):
pending_actions = any(
action.approval_status == "pending" for action in actions
)
@@ -2074,11 +2153,33 @@ class OperationalTaskService:
)
return run
async def _resolve_investigation_alerts(self, task_id: str, evidence_id: str) -> None:
from app.domain.schemas.task_alert import TaskAlertEvent
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
from app.services.task_alerts import TaskAlertService
if not hasattr(self.repo, "list_investigation_alerts"):
return
for alert in await self.repo.list_investigation_alerts(task_id):
command = TaskAlertEvent.model_validate({
**{key: value for key, value in alert.items() if key not in {"status", "updated_at", "revision"}},
"event_id": uuid5(NAMESPACE_URL, f"recovery:{task_id}:{evidence_id}:{alert['alert_id']}"),
"event_type": "resolved",
})
await TaskAlertService(TaskAlertRepository(self.repo.connection, self.repo.project_id), None,
internal_producer_id=command.producer_id).push(command)
async def _task(self, task_id: UUID) -> dict:
task = await self.repo.get_task(str(task_id))
require(task is not None, "TASK_NOT_FOUND", "Operational task not found", status=404)
return task
@staticmethod
def _require_real_action(task: dict, kind: str) -> None:
require(kind == "create_work_order" or not (task.get("source") or {}).get("replay_context"),
"REPLAY_ACTION_DISABLED",
"回放任务不支持设备命令;可生成工单草稿,由有权限的用户确认发布", status=409)
@staticmethod
def _require_actionable_task(task: dict) -> None:
source = task.get("source") or {}
@@ -2142,8 +2243,10 @@ class OperationalTaskService:
return "running"
@staticmethod
def _investigation_status(actions: Iterable[HumanApprovalAction]) -> str:
"""Only recorded business actions can keep an investigation waiting."""
def _investigation_status(actions: Iterable[HumanApprovalAction], task: dict | None = None) -> str:
"""Field completion and recovery evidence precede incident closure."""
if (task or {}).get("handling_context", {}).get("phase") in {"waiting_work_order", "waiting_scada"}:
return "waiting_human"
return "waiting_human" if any(
action.approval_status == "pending" or (
action.approval_status == "approved"
@@ -2432,6 +2535,7 @@ class OperationalTaskService:
due_at: datetime, now: datetime,
) -> OperationalTaskSummary:
return OperationalTaskSummary(
execution_scope=self.execution_scope,
project_id=self.context.project_id, task_id=task_id,
task_type="scheduled_flow", workflow=schedule.schedule_key,
lifecycle_status="open", status="queued", priority=schedule.priority, revision=1,
@@ -0,0 +1,141 @@
"""Fit causal replay thresholds in bounded numeric arrays, before publishing history."""
from datetime import datetime, timedelta
import numpy as np
import pandas as pd
from app.domain.schemas.project_settings import ScadaMonitoringSettings
from app.domain.schemas.scada_replay import ScadaReplayCreate
from app.services.scada_normal_profile import CALIBRATION_VERSION, DIAGNOSTIC_POLICY
def adaptive_score_threshold(scores: list[float]) -> dict:
"""Keep the cutoff below the normal lower tail, with a fixed noise margin."""
values = np.asarray(scores, dtype=float)
if len(values) < 5 or not np.isfinite(values).all():
return {"status": "insufficient_data", "reason": "normal_score_history_incomplete"}
center = float(np.median(values))
sigma = float(1.4826 * np.median(np.abs(values - center)))
lower_tail = float(np.quantile(values, 0.05))
return {
"status": "ready",
"threshold": min(-0.04, center - 3 * sigma, lower_tail - 0.02),
"normal_score_median": center,
"normal_score_min": float(values.min()),
"normal_score_max": float(values.max()),
"normal_score_robust_sigma": sigma,
"normal_score_lower_quantile": lower_tail,
"normal_score_count": len(values),
"method": "min(-0.04, median-3*robust_sigma, normal_q05-0.02)",
}
class MonitoringCalibrationBuilder:
"""Retain bounded numeric series, never full raw pages or device batches."""
def __init__(self, document: dict, config: ScadaReplayCreate):
self.settings = ScadaMonitoringSettings.model_validate(document["monitoring_config"])
self.config = config
self.cutoff = config.start_time
self.start = self.cutoff - timedelta(days=15)
self.interval = int(document["monitoring_interval_seconds"])
self.supported = self.interval % 60 == 0 and 86400 % self.interval == 0
self.count = 15 * (86400 // self.interval) if self.supported else 0
ids = dict.fromkeys(self.settings.pressure_device_ids + self.settings.outlet_flow_device_ids
+ [device for rule in config.rules if rule.kind == "relative_change"
for device in rule.device_ids])
self.values = {device: np.full(self.count, np.nan) for device in ids}
def add(self, rows: list[dict]) -> None:
for row in rows:
device = row["device_id"]
if device not in self.values:
continue
at = datetime.fromisoformat(row["time"])
if at >= self.cutoff:
raise ValueError("动态阈值拟合不能读取回放开始时刻及之后的观测")
offset = (at - self.start).total_seconds()
if (not self.supported or offset < 0 or offset % self.interval
or row.get("quality") not in {"valid", "corrected"}
or row.get("cleaned_value") is None):
continue
index = int(offset // self.interval)
if np.isfinite(self.values[device][index]):
raise ValueError("动态阈值拟合存在重复的设备时间点")
self.values[device][index] = float(row["cleaned_value"])
def fit(self) -> dict:
base = {
"version": CALIBRATION_VERSION,
"diagnostic_policy": DIAGNOSTIC_POLICY,
"knowledge_cutoff": self.cutoff.isoformat(),
"history_start": self.start.isoformat(),
"required_samples_per_device": self.count,
"pressure": {}, "outlet_flow": {}, "diagnostic": {},
}
if not self.supported:
return {**base, "status": "insufficient_data", "reason": "unsupported_sampling_interval"}
available = {device: int(np.isfinite(values).sum()) for device, values in self.values.items()}
# A regular diurnal rise/reversal can be a huge percentage of a small
# preceding flow. Judge its absolute value against causal normal readings
# at the same sampling phase before treating that percentage as abnormal.
points_per_day = 86400 // self.interval
for device, values in self.values.items():
if available[device] != self.count:
continue
days = values.reshape(15, points_per_day)
center = np.median(days, axis=0)
sigma = 1.4826 * np.median(np.abs(days - center), axis=0)
margin = np.maximum(3 * sigma, np.maximum(np.abs(center) * 1e-6, 1e-9))
base["diagnostic"][device] = {
"phase_origin": self.start.isoformat(), "interval_seconds": self.interval,
"lower": (days.min(axis=0) - margin).tolist(),
"upper": (days.max(axis=0) + margin).tolist(),
"method": "normal_same_phase_range_plus_3_robust_sigma", "normal_day_count": 15,
}
for kind, ids, direction, floor in (
("pressure", self.settings.pressure_device_ids, -1, 3.0),
("outlet_flow", self.settings.outlet_flow_device_ids, 1, 5.0),
):
for device in ids:
if available[device] != self.count:
continue
rule = next((rule for rule in self.config.rules
if rule.kind == "relative_change" and device in rule.device_ids), None)
if rule is None:
continue
series = pd.Series(self.values[device])
reference = series.rolling(rule.window_samples, min_periods=rule.minimum_samples).median().shift(1)
changes = (direction * 100 * (series - reference) / reference.abs()).to_numpy()
changes = changes[np.isfinite(changes)]
if not len(changes):
continue
center = float(np.median(changes))
sigma = float(1.4826 * np.median(np.abs(changes - center)))
base[kind][device] = {
"threshold_percent": max(floor, center + 3 * sigma),
"normal_change_median": center,
"normal_change_robust_sigma": sigma,
"normal_change_count": int(len(changes)),
"method": "normal_recent_window_change_median_plus_3_robust_sigma",
"normal_profile": base["diagnostic"].get(device),
}
pressure_ids = self.settings.pressure_device_ids
incomplete = {device: available[device] for device in pressure_ids if available[device] != self.count}
if len(pressure_ids) < 5 or incomplete:
return {**base, "status": "insufficient_data", "reason": "pressure_history_incomplete",
"available_samples": available}
from app.services import scada_replay_monitoring as monitoring
index = pd.date_range(start=self.start, periods=self.count, freq=pd.Timedelta(seconds=self.interval))
frame = pd.DataFrame({device: self.values[device] for device in pressure_ids}, index=index)
result = monitoring.run_burst_detection(
network="scada-replay-baseline", username="scada-replay",
observed_pressure_data=frame, points_per_day=86400 // self.interval,
mu=monitoring.TARGET_MU,
iforest_params={"n_estimators": monitoring.TARGET_N_ESTIMATORS,
"random_state": monitoring.TARGET_RANDOM_STATE, "contamination": "auto"},
sensor_nodes=pressure_ids,
)
score = adaptive_score_threshold([float(row["Score"]) for row in result["rows"]])
return {**base, "status": score["status"], "burst_detection": score}
@@ -0,0 +1,98 @@
"""Persist one accident across monitoring periods without moving its evidence cutoff."""
from datetime import datetime
from math import isfinite
from uuid import NAMESPACE_URL, uuid5
def _anchors(monitoring: dict, indicator: str = "pressure") -> dict:
return {
item["device_id"]: {"baseline": item["baseline"], "threshold_percent": item["threshold_percent"]}
for item in monitoring.get(indicator, {}).get("devices", [])
if item.get("hit") and item.get("baseline", 0) > 0
and 0 < item.get("threshold_percent", 0) < 100
}
def update_incident(document: dict, monitoring: dict, sequence: int) -> tuple[dict | None, str | None]:
incident = document.get("monitoring_incident")
# Adopt the last task from an older replay document on its first new step.
previous = document.get("last_monitoring") or {}
if incident:
incident.setdefault("burst_diagnosed", previous.get("status") == "suspected_burst")
if not incident and previous.get("task_id"):
incident = {
"incident_id": str(uuid5(NAMESPACE_URL, f"scada_replay:{document['replay_id']}:monitor:"
f"{previous['period_end']}:{previous['rule_version']}")),
"started_at": previous["period_end"], "last_observed_at": previous["period_end"],
"status": "active", "task_id": previous["task_id"], "recovery_count": 0,
"pressure_anchors": _anchors(previous), "alert": None,
"flow_anchors": _anchors(previous, "outlet_flow"),
"burst_diagnosed": previous.get("status") == "suspected_burst",
}
document["monitoring_incident"] = incident
if incident and incident.get("last_monitoring_id") == monitoring.get("monitoring_id") and monitoring.get("monitoring_id"):
return incident, incident["last_action"]
anomaly = (monitoring["status"] in {"attention", "suspected_burst"}
or bool(monitoring.get("anomaly_signals"))
or any(monitoring.get(key, {}).get("hit") is True
for key in ("pressure", "outlet_flow", "burst_detection")))
if anomaly and (not incident or incident["status"] == "resolved"):
external = (f"scada_replay:{document['replay_id']}:monitor:"
f"{monitoring['period_end']}:{monitoring['rule_version']}")
incident = {
"incident_id": str(uuid5(NAMESPACE_URL, external)), "external_id": external,
"started_at": monitoring["period_end"], "status": "active", "task_id": None,
"recovery_count": 0, "pressure_anchors": _anchors(monitoring), "alert": None,
"flow_anchors": _anchors(monitoring, "outlet_flow"),
"batch_sequence": sequence,
}
document["monitoring_incident"] = incident
action = "raised"
elif incident and incident["status"] == "active":
# A later diagnostic stage may add pressure/flow evidence to the same task.
# Freeze each newly abnormal device once; retain earlier references.
for indicator, field in (("pressure", "pressure_anchors"), ("outlet_flow", "flow_anchors")):
for device, anchor in _anchors(monitoring, indicator).items():
incident.setdefault(field, {}).setdefault(device, anchor)
current = {item["device_id"]: item for item in monitoring.get("pressure", {}).get("devices", [])}
anchors = incident["pressure_anchors"]
flow = {item["device_id"]: item for item in monitoring.get("outlet_flow", {}).get("devices", [])}
# Rolling relative changes can become normal while absolute pressure stays low.
# Confirm recovery against the frozen pre-accident references as well.
recovered = monitoring["status"] == "normal" and not anomaly and all(
monitoring.get(key, {}).get("status") == "ready"
and monitoring[key].get("hit") is False for key in ("pressure", "burst_detection")
) and all(
device in current and isfinite(current[device]["value"])
and current[device].get("observed_at") is not None
and datetime.fromisoformat(current[device]["observed_at"]) > datetime.fromisoformat(incident["last_observed_at"])
and current[device]["value"] >= anchor["baseline"] * (1 - anchor["threshold_percent"] / 100)
for device, anchor in anchors.items()
) and all(
device in flow and isfinite(flow[device]["value"])
and flow[device].get("observed_at") is not None
and datetime.fromisoformat(flow[device]["observed_at"]) > datetime.fromisoformat(incident["last_observed_at"])
and flow[device]["value"] <= anchor["baseline"] * (1 + anchor["threshold_percent"] / 100)
for device, anchor in incident.get("flow_anchors", {}).items()
)
contiguous = bool(monitoring.get("period_start")) and (
datetime.fromisoformat(monitoring["period_start"]) == datetime.fromisoformat(incident["last_observed_at"])
)
incident["recovery_count"] = (incident["recovery_count"] + 1 if contiguous else 1) if recovered else 0
action = "resolved" if incident["recovery_count"] >= 2 else "ongoing"
if action == "resolved":
incident["status"] = "resolved"
else:
return None, None
incident.update(last_observed_at=monitoring["period_end"],
last_monitoring_id=monitoring.get("monitoring_id"), last_action=action)
return incident, action
def attach_incident(monitoring: dict, incident: dict, action: str) -> None:
monitoring["task_id"] = incident["task_id"]
monitoring["incident"] = {
key: incident[key] for key in (
"incident_id", "status", "started_at", "last_observed_at", "recovery_count",
)
} | {"action": action}
+48
View File
@@ -0,0 +1,48 @@
"""Clock-phase normal envelopes frozen before replay; never inspect future readings."""
from datetime import datetime
from math import isfinite
CALIBRATION_VERSION = "scada-monitor-calibration@2"
DIAGNOSTIC_POLICY = "normal-phase-envelope@1"
def weighted_diagnosis(document: dict) -> bool:
return (document.get("monitoring_config") or {}).get("decision_policy") == "weighted"
def current_calibration(document: dict, start: datetime) -> dict | None:
calibration = document.get("monitoring_calibration") or {}
if (calibration.get("version") == CALIBRATION_VERSION
and calibration.get("diagnostic_policy") == DIAGNOSTIC_POLICY
and calibration.get("knowledge_cutoff") == start.isoformat()):
return calibration
return None
def diagnostic_signal(document: dict, event: dict) -> bool:
"""Never revive a persisted fixed-percentage event as new diagnosis evidence."""
if event.get("category") == "data_quality":
return False
if not weighted_diagnosis(document):
return True
rules = (document.get("config") or {}).get("rules", [])
rule = next((item for item in rules if item["rule_id"] == event["rule_id"]), None)
return bool(rule and (rule["kind"] == "bounds" or event.get("diagnostic_policy") == DIAGNOSTIC_POLICY))
def normal_profile_bounds(profile: dict | None, at: datetime) -> tuple[float, float] | None:
if not profile:
return None
interval = profile.get("interval_seconds", 0)
if not profile.get("phase_origin"):
return None
origin = datetime.fromisoformat(profile["phase_origin"])
lower, upper = profile.get("lower", []), profile.get("upper", [])
offset = (at - origin).total_seconds()
if interval <= 0 or offset < 0 or offset % interval or not lower or len(lower) != len(upper):
return None
slot = int(offset // interval) % len(lower)
low, high = lower[slot], upper[slot]
if low is None or high is None or not isfinite(low) or not isfinite(high):
return None
return float(low), float(high)
+66
View File
@@ -0,0 +1,66 @@
"""Read-only chronological SCADA paging shared by REST and replay processing."""
from __future__ import annotations
import base64
import hashlib
import json
import math
from datetime import datetime
from app.domain.schemas.scada import ScadaRawPage, ScadaRawReading
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
def validate_window(start: datetime, end: datetime) -> None:
if start.tzinfo is None or end.tzinfo is None or start >= end:
raise ValueError("需要带时区、结束晚于开始的 SCADA 时间区间")
def raw_reading(row: dict) -> ScadaRawReading:
value = row["monitored_value"]
invalid = value is not None and not math.isfinite(float(value))
return ScadaRawReading(
time=row["time"], device_id=row["device_id"],
monitored_value=None if invalid else value,
quality="invalid" if invalid else "missing" if value is None else "unvalidated",
)
async def read_raw_page(
conn, *, project_id: str, device_ids: list[str], start: datetime,
end: datetime, limit: int = 500, cursor: str | None = None,
through: datetime | None = None,
) -> ScadaRawPage:
validate_window(start, end)
ids = sorted(set(device_ids))
if not ids or len(ids) > 500 or not 1 <= limit <= 500:
raise ValueError("请选择 1 至 500 个设备,每页最多 500 条")
cap = min(through or end, end)
if cap.tzinfo is None or cap < start:
raise ValueError("模拟时间不在可读取区间内")
signature = hashlib.sha256(json.dumps(
[project_id, ids, start.isoformat(), end.isoformat()], separators=(",", ":")
).encode()).hexdigest()
after = None
if cursor:
try:
data = json.loads(base64.urlsafe_b64decode(cursor.encode()))
frozen = datetime.fromisoformat(data["through"])
last = datetime.fromisoformat(data["time"])
if (data["query"] != signature or frozen.tzinfo is None or last.tzinfo is None
or frozen > cap or not start <= last <= frozen or data["device"] not in ids):
raise ValueError()
cap, after = frozen, (last, data["device"])
except (ValueError, KeyError, TypeError, UnicodeError) as exc:
raise ValueError("SCADA 分页游标无效或查询范围已改变") from exc
rows = await ScadaRepository.get_raw_page(conn, ids, start, end, cap, limit + 1, after)
selected = rows[:limit]
next_cursor = None
if len(rows) > limit:
last = selected[-1]
next_cursor = base64.urlsafe_b64encode(json.dumps({
"query": signature, "through": cap.isoformat(),
"time": last["time"].isoformat(), "device": last["device_id"],
}, separators=(",", ":")).encode()).decode()
return ScadaRawPage(items=[raw_reading(row) for row in selected],
next_cursor=next_cursor, scanned_through=cap)
+604
View File
@@ -0,0 +1,604 @@
"""Durable, project-scoped SCADA playback; raw is never inserted or rewritten."""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
import json
from uuid import NAMESPACE_URL, uuid4, uuid5
from fastapi import HTTPException
from app.core.config import settings
from app.auth.permissions import permissions_for_context
from app.domain.schemas.project_settings import ScadaMonitoringSettings
from app.domain.schemas.scada_replay import ScadaReplayCreate, ScadaReplayCommand, ScadaReplaySnapshot
from app.infra.db.postgresql.scada import ScadaInfoRepository
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
from app.services.content_digest import digest
from app.services.scada_replay_processing import prepare_cleaning, detect_frame
from app.services.scada_replay_monitoring import evaluate_monitoring_period
from app.services.scada_normal_profile import current_calibration, diagnostic_signal, weighted_diagnosis
# 500 devices at 15-minute sampling need 720,000 rows for fifteen days.
# Leave headroom without allowing dense sources to make preparation unbounded.
BASELINE_PAGE_SIZE = 2000
BASELINE_MAX_ROWS = 1_000_000
BASELINE_MAX_BYTES = 512 * 1024 * 1024
class BaselineBudgetExceeded(RuntimeError):
"""The unpublished baseline exceeded its fixed preparation budget."""
def parse_time(value: str) -> datetime:
return datetime.fromisoformat(value)
def snapshot(document: dict) -> ScadaReplaySnapshot:
return ScadaReplaySnapshot(
**{key: document[key] for key in (
"replay_id", "project_id", "revision", "status", "start_time", "end_time",
"virtual_now", "created_at", "updated_at",
)},
delivered_until=document.get("delivered_until"), processed_until=document.get("processed_until"),
processed_count=document.get("processed_count", 0), anomaly_count=document.get("anomaly_count", 0),
devices=document["devices"], rules=document["config"]["rules"],
latest=list(document.get("latest", {}).values()),
active_events=[state["event"] for state in document.get("rule_states", {}).values() if state.get("event")],
error=document.get("error"),
monitoring_interval_seconds=document.get("monitoring_interval_seconds"),
step_pending=document.get("step_pending", False),
monitoring_count=document.get("monitoring_count", 0),
last_monitoring=document.get("last_monitoring"),
# Old persisted documents without a policy still use three-signal detection.
monitoring_config=({"decision_policy": "all_signals", **document["monitoring_config"]}
if document.get("monitoring_config") else None),
monitoring_calibration=document.get("monitoring_calibration"),
)
class ScadaReplayService:
def __init__(self, repository, context, timescale=None, *, clock=None):
self.repo, self.context, self.timescale = repository, context, timescale
self.clock = clock or (lambda: datetime.now(timezone.utc))
async def ensure_storage(self):
if not await self.repo.available():
raise HTTPException(503, "SCADA 回放存储未配置,需要应用 031_scada_replays.sql")
async def create(
self,
command: ScadaReplayCreate,
*,
monitoring_interval_seconds: int | None,
monitoring_config: ScadaMonitoringSettings | None = None,
):
await self.ensure_storage()
async with self.repo.control():
previous = await self.repo.by_request(command.request_id)
config = command.model_dump(mode="json")
if previous:
if previous["config"] != config:
raise HTTPException(409, "创建请求号已用于不同回放配置")
return snapshot(previous)
if monitoring_interval_seconds is None:
raise HTTPException(422, "请先配置项目 settings.scada_monitoring_interval_seconds,再创建演示")
replay_id = str(uuid4())
if await self.repo.active_other(replay_id):
raise HTTPException(409, "当前项目已有回放,请先停止原回放再新建")
metadata = {row["device_id"]: row for row in await ScadaInfoRepository.get_scadas(self.repo.connection)}
if not set(command.device_ids) <= metadata.keys():
raise HTTPException(422, "设备不属于当前项目")
for rule in command.rules:
if any(metadata[device]["measurement_unit"] != rule.unit for device in rule.device_ids):
raise HTTPException(422, "规则单位与设备单位不一致,请分组配置")
if any(metadata[device]["device_type"] == "quality" for device in rule.device_ids) and not rule.metric_name.strip():
raise HTTPException(422, "水质规则必须明确指标名称,不能仅凭单位解释读数")
if monitoring_config:
role_ids = set(monitoring_config.pressure_device_ids) | set(monitoring_config.outlet_flow_device_ids)
if not role_ids <= metadata.keys():
raise HTTPException(422, "项目 SCADA 监测角色包含不存在的设备")
if any(metadata[device]["device_type"] != "pressure" for device in monitoring_config.pressure_device_ids):
raise HTTPException(422, "压力监测角色必须引用 pressure 设备")
if any(metadata[device]["device_type"] not in {"pipe_flow", "flow"}
for device in monitoring_config.outlet_flow_device_ids):
raise HTTPException(422, "出厂边界流量角色必须引用流量设备")
now = self.clock().isoformat()
document = {
"replay_id": replay_id, "project_id": str(self.context.project_id),
"created_by": str(self.context.user_id), "config": config,
"revision": 1, "status": "ready", "start_time": config["start_time"],
"end_time": config["end_time"], "virtual_now": config["start_time"],
"created_at": now, "updated_at": now,
"monitoring_interval_seconds": monitoring_interval_seconds,
"monitoring_config": monitoring_config.model_dump(mode="json") if monitoring_config else None,
"devices": [metadata[device] for device in command.device_ids],
"batch_sequence": 0, "processed_count": 0, "anomaly_count": 0,
"monitoring_count": 0, "latest": {}, "histories": {}, "rule_states": {}, "receipts": {},
}
await self.repo.insert(document)
return snapshot(document)
async def command(self, replay_id, command: ScadaReplayCommand):
await self.ensure_storage()
async with self.repo.control():
document = await self.repo.get(replay_id)
key, body = str(command.request_id), command.model_dump(mode="json", exclude_none=True)
receipt = document.setdefault("receipts", {}).get(key)
if receipt:
if receipt != body:
raise HTTPException(409, "命令请求号已用于不同内容")
return snapshot(document)
if command.expected_revision != document["revision"]:
raise HTTPException(409, "回放控制状态已更新,请刷新后重试")
action, status = command.action, document["status"]
if (action in {"step", "monitor", "retry"} and document.get("warmed")
and weighted_diagnosis(document)
and current_calibration(document, parse_time(document["start_time"])) is None):
raise HTTPException(409, "旧回放缺少新版历史诊断校准,请停止后新建演示")
if document.get("step_pending") and action in {"step", "monitor"}:
raise HTTPException(409, "当前时间步正在处理,请完成后继续")
if action in {"step", "monitor", "retry"} and await self.repo.active_other(replay_id):
raise HTTPException(409, "当前项目已有另一条活动回放")
if action == "step" and status in {"ready", "paused"}:
interval = document.get("monitoring_interval_seconds")
if interval is None:
raise HTTPException(409, "旧回放未固定监测周期,请停止后新建演示")
start = parse_time(document["start_time"])
elapsed = (parse_time(document["virtual_now"]) - start).total_seconds()
target = start + timedelta(seconds=(int(elapsed // interval) + 1) * interval)
document["step_start"] = document["virtual_now"]
document["step_anomaly_signals"] = []
document["step_target"] = min(parse_time(document["end_time"]), target).isoformat()
document["status"], document["step_pending"] = "paused", True
elif action == "monitor" and status in {"ready", "paused", "completed"}:
target = command.at_time
if target is None:
raise HTTPException(422, "选定时刻监测需要提供 at_time")
start = parse_time(document["start_time"])
current = parse_time(document["virtual_now"])
if not start <= target <= parse_time(document["end_time"]):
raise HTTPException(422, "监测时刻必须在本轮演示区间内")
if target < current:
raise HTTPException(409, "监测历史时刻需要新建一轮回放,避免使用后续状态")
interval = document.get("monitoring_interval_seconds")
if interval is None:
raise HTTPException(409, "本轮未固定监测周期,请停止后新建演示")
previous = document.get("last_monitoring") or {}
# Reusing an immutable result must not count as another normal
# period, resolve an incident, or create another investigation.
if not previous.get("period_end") or parse_time(previous["period_end"]) != target:
document["step_start"] = (current if target > current
else max(start, target - timedelta(seconds=interval))).isoformat()
document["step_target"] = target.isoformat()
document["step_anomaly_signals"] = []
document["status"], document["step_pending"] = "paused", True
elif action == "stop" and status not in {"completed", "stopped"}:
document["status"], document["step_pending"] = "stopped", False
document.pop("step_target", None)
document.pop("step_start", None)
elif action == "retry" and status == "failed":
if "SOURCE_CHANGED" in (document.get("error") or ""):
raise HTTPException(409, "原始数据发生变化,需要停止并新建回放")
document["status"], document["error"] = "paused", None
document["step_pending"] = bool(document.get("step_target"))
else:
raise HTTPException(409, "当前回放状态不支持此操作")
document["revision"] += 1
document["receipts"][key] = body
if len(document["receipts"]) > 256:
del document["receipts"][next(iter(document["receipts"]))]
await self.repo.save(document)
return snapshot(document)
async def process_pending_step(self, replay_id):
await self.ensure_storage()
async with self.repo.processing() as owned:
if not owned:
raise HTTPException(409, "当前时间步正在由另一请求处理")
while True:
document = await self.repo.get(replay_id)
if not document.get("step_pending"):
return snapshot(document)
try:
await self._advance(document)
except Exception as exc:
# Failure never discards the prepared batch; retry resumes it.
document = await self.repo.get(replay_id)
document["status"], document["step_pending"] = "failed", False
if isinstance(exc, HTTPException):
document["error"] = str(exc.detail)
elif isinstance(exc, BaselineBudgetExceeded):
document["error"] = str(exc)
else:
document["error"] = (
"SOURCE_CHANGED: 原始观测已改变,需要新建回放"
if "SOURCE_CHANGED" in str(exc)
else "SCADA 处理失败,已保留批次与进度;请检查服务日志后重试"
)
async with self.repo.connection.transaction():
await self.repo.save(document)
raise
async def _advance(self, document):
config = ScadaReplayCreate.model_validate(document["config"])
replay_id = document["replay_id"]
if not document.get("warmed"):
await self._freeze_baseline(document, config)
if weighted_diagnosis(document) and current_calibration(document, config.start_time) is None:
raise HTTPException(409, "旧回放缺少新版历史诊断校准,请停止后新建演示")
batch = await self.repo.pending_batch(replay_id)
if batch is None:
step = document.get("step_pending", False)
virtual = parse_time(document["virtual_now"])
through = parse_time(document["step_target"]) if step else virtual
after = ((parse_time(document["cursor"][0]), document["cursor"][1])
if document.get("cursor") else None)
async with self.timescale.transaction():
rows = await ScadaRepository.get_raw_page(
self.timescale, config.device_ids, config.start_time, config.end_time,
through, 501, after, frame=True,
)
if len(rows) > 500:
raise ValueError("SCADA frame exceeds selected device bound")
at = rows[0]["time"] if rows else through
if step:
virtual = through
document["virtual_now"] = virtual.isoformat()
if not rows and document.get("processed_until") == at.isoformat() and not step:
async with self.repo.connection.transaction():
await self.repo.save(document)
return
sequence = document["batch_sequence"] + 1
refs = [{"time": row["time"].isoformat(), "device_id": row["device_id"],
"source_hash": row["source_hash"]} for row in rows]
batch = {"sequence": sequence, "stage": "selected", "at": at.isoformat(),
"refs": refs, "input_digest": digest(refs), "events": []}
async with self.repo.connection.transaction():
document["batch_sequence"] = sequence
document["delivered_until"] = at.isoformat()
await self.repo.save_batch(replay_id, batch)
await self.repo.save(document)
else:
rows = None
if batch["stage"] == "selected":
if rows is None:
# Only crash recovery reloads the selected raw frame.
at = parse_time(batch["at"])
async with self.timescale.transaction():
rows = await ScadaRepository.get_raw_page(
self.timescale, config.device_ids, at, config.end_time, at, 501,
) if batch["refs"] else []
actual = [{"time": row["time"].isoformat(), "device_id": row["device_id"],
"source_hash": row["source_hash"]} for row in rows]
if digest(actual) != batch["input_digest"]:
raise ValueError("SOURCE_CHANGED: selected raw frame differs")
batch["outputs"] = prepare_cleaning(rows, config)
batch["stage"] = "clean_prepared"
async with self.repo.connection.transaction():
await self.repo.save_batch(replay_id, batch)
if batch["stage"] == "clean_prepared":
await ScadaRepository.apply_replay_cleaning(self.timescale, batch["outputs"])
batch["stage"] = "cleaned"
async with self.repo.connection.transaction():
await self.repo.save_batch(replay_id, batch)
if batch["stage"] == "cleaned":
result, changes = detect_frame(document, batch["outputs"], config, parse_time(batch["at"]))
result["processed_until"] = batch["at"]
target = result.get("step_target")
was_step = bool(target)
result["step_pending"] = bool(target and parse_time(batch["at"]) < parse_time(target))
if was_step:
signals = {item["event_id"]: item for item in result.get("step_anomaly_signals", [])
if diagnostic_signal(result, item)}
for event in changes:
if event["change"] == "raised" and diagnostic_signal(result, event):
signals[event["event_id"]] = {key: event[key] for key in (
"event_id", "rule_id", "device_ids", "detail", "diagnostic_policy"
) if key in event}
result["step_anomaly_signals"] = list(signals.values())
monitoring = None
if was_step and not result["step_pending"]:
period_start = parse_time(result.get("step_start", result["start_time"]))
period_end = parse_time(target)
role = result.get("monitoring_config") or {}
role_ids = list(dict.fromkeys(
role.get("pressure_device_ids", []) + role.get("outlet_flow_device_ids", [])
))
previous = await self.repo.monitoring_outputs(
replay_id, role_ids, period_end - timedelta(days=15),
period_end, batch["sequence"],
) if role_ids else []
from app.mcp.analysis_service import bounded_calculation
monitoring = await bounded_calculation(
evaluate_monitoring_period,
result,
previous + batch["outputs"],
period_start=period_start,
period_end=period_end,
batch_sequence=batch["sequence"],
)
# Any configured measurement rule can enter diagnosis, including
# indicators outside the pressure/outlet roles used for burst scoring.
signals = {item["event_id"]: item for item in result.get("step_anomaly_signals", [])
if diagnostic_signal(result, item)}
for state in result.get("rule_states", {}).values():
event = state.get("event")
if event and diagnostic_signal(result, event):
signals[event["event_id"]] = {key: event[key] for key in (
"event_id", "rule_id", "device_ids", "detail", "diagnostic_policy"
) if key in event}
monitoring["anomaly_signals"] = list(signals.values())
if signals and monitoring["status"] == "normal":
monitoring["status"] = "attention"
monitoring["detail"] = "本周期曾检出测点异常,进入异常调查与爆管诊断"
result["monitoring_count"] = result.get("monitoring_count", 0) + 1
if not result["step_pending"]:
result.pop("step_target", None)
result.pop("step_start", None)
result.pop("step_anomaly_signals", None)
if batch["refs"]:
last = batch["refs"][-1]
result["cursor"] = [last["time"], last["device_id"]]
if parse_time(batch["at"]) >= config.end_time:
result["status"] = "completed"
batch["stage"], batch["events"] = "detected", changes
if monitoring is not None:
batch["monitoring"] = monitoring
async with self.repo.connection.transaction():
await self._create_investigations(result, changes, batch, create_tasks=not was_step)
if monitoring is not None:
await self._create_monitoring_task(result, monitoring, batch)
batch["monitoring"] = monitoring
result["last_monitoring"] = monitoring
await self.repo.save_batch(replay_id, batch)
await self.repo.save(result)
async def _freeze_baseline(self, document, config):
"""Stage bounded pages, then atomically publish one causal source snapshot."""
if document.get("warmed"):
return
start = config.start_time - timedelta(days=15)
batch_count, row_count, byte_count, after = 0, 0, 0, None
counts = {device_id: 0 for device_id in config.device_ids}
valid_counts = dict(counts)
histories = {device_id: [] for device_id in config.device_ids}
window = max(rule.window_samples for rule in config.rules)
calibration = None
if (document.get("monitoring_config") or {}).get("decision_policy") == "weighted":
from app.services.scada_monitoring_calibration import MonitoringCalibrationBuilder
calibration = MonitoringCalibrationBuilder(document, config)
async with self.repo.connection.transaction():
await self.repo.clear_unpublished_baseline(document["replay_id"])
# Keep only the current page in memory. Each BizDB write commits before
# the next source read; clean_prepared baseline pages remain invisible.
async with self.timescale.transaction():
await ScadaRepository.begin_replay_baseline_snapshot(self.timescale)
while True:
rows = await ScadaRepository.get_raw_page(
self.timescale, config.device_ids, start, config.start_time,
config.start_time, min(BASELINE_PAGE_SIZE, BASELINE_MAX_ROWS - row_count + 1), after,
)
if not rows:
break
if row_count + len(rows) > BASELINE_MAX_ROWS:
raise BaselineBudgetExceeded(
f"BASELINE_BUDGET_EXCEEDED: 十五天历史基线超过 {BASELINE_MAX_ROWS:,} 条准备预算,"
"尚未发布;请减少回放设备或核查历史采样密度后重试"
)
outputs = prepare_cleaning(rows, config)
if calibration is not None:
calibration.add(outputs)
batch = {
"sequence": -batch_count - 1, "stage": "clean_prepared", "kind": "baseline",
"at": outputs[-1]["time"], "outputs": outputs, "events": [],
}
batch_bytes = len(json.dumps(batch, ensure_ascii=False, allow_nan=False,
separators=(",", ":")).encode("utf-8"))
if byte_count + batch_bytes > BASELINE_MAX_BYTES:
raise BaselineBudgetExceeded(
f"BASELINE_BUDGET_EXCEEDED: 十五天历史基线超过 {BASELINE_MAX_BYTES // (1024 * 1024)} MiB 准备预算,"
"尚未发布;请减少回放设备或核查历史采样密度后重试"
)
async with self.repo.connection.transaction():
await self.repo.save_batch(document["replay_id"], batch)
batch_count += 1
row_count += len(outputs)
byte_count += batch_bytes
for row in outputs:
device_id = row["device_id"]
counts[device_id] += 1
if row["quality"] == "valid":
valid_counts[device_id] += 1
histories[device_id].append(row["cleaned_value"])
histories[device_id] = histories[device_id][-window:]
last = rows[-1]
after = (last["time"], last["device_id"])
document["baseline"] = {
"start_time": start.isoformat(), "end_time": config.start_time.isoformat(),
"end_exclusive": True, "sample_counts": counts, "valid_sample_counts": valid_counts,
"row_count": row_count, "batch_count": batch_count, "byte_count": byte_count,
}
document["histories"] = histories
if calibration is not None:
from app.mcp.analysis_service import bounded_calculation
document["monitoring_calibration"] = await bounded_calculation(calibration.fit)
document["warmed"] = True
async with self.repo.connection.transaction():
await self.repo.publish_baseline(document["replay_id"])
await self.repo.save(document)
async def _create_investigations(self, document, changes, batch, *, create_tasks=True):
from app.domain.schemas.operational_task import OperationalTaskCreate, TaskSource, TaskTrigger
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.services.operational_tasks import OperationalTaskService
from app.domain.schemas.task_alert import TaskAlertEvent
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
from app.services.task_alerts import TaskAlertService
from uuid import UUID
producer = "scada-replay:" + document["replay_id"]
alerts = TaskAlertService(TaskAlertRepository(self.repo.connection, self.context.project_id),
self.context, internal_producer_id=producer)
tasks = OperationalTaskService(OperationalTaskRepository(self.repo.connection, self.context.project_id),
self.context, permissions_for_context(self.context),
execution_scope=settings.operational_task_execution_scope)
for event in changes:
# Period diagnosis owns measurement notifications during manual steps.
# Retain rule events as evidence; quality alerts remain independent.
if (not create_tasks and event.get("category") != "data_quality"
and (event["change"] == "raised" or event.get("notification_suppressed"))):
if event["change"] == "raised":
event["batch_sequence"] = batch["sequence"]
event["notification_suppressed"] = True
document["rule_states"][event.get("state_key", event["rule_id"])]["event"].update(
batch_sequence=batch["sequence"], notification_suppressed=True,
)
continue
if event["change"] == "raised":
event["batch_sequence"] = batch["sequence"]
document["rule_states"][event.get("state_key", event["rule_id"])]["event"]["batch_sequence"] = batch["sequence"]
await alerts.push(TaskAlertEvent(
event_id=uuid5(NAMESPACE_URL, event["event_id"] + ":" + event["change"]),
alert_id=event["event_id"], producer_id=producer, event_type=event["change"],
occurred_at=event["occurred_at"], condition_id=None,
source={"kind": "scada_replay", "replay_id": document["replay_id"],
"rule_id": event["rule_id"], "batch_sequence": event["batch_sequence"]},
title=f"SCADA 回放异常 · {event['rule_id']}", description=event["detail"], severity="warning",
))
if event["change"] != "raised":
if event.get("task_id"):
await tasks.advance_replay_investigation(UUID(event["task_id"]), {
"replay_id": document["replay_id"], "knowledge_cutoff": batch["at"],
"batch_sequence": batch["sequence"],
}, uuid5(NAMESPACE_URL, event["event_id"] + ":recovery"))
continue
if event.get("category") == "data_quality":
continue
if not create_tasks:
continue
external = f"scada_replay:{document['replay_id']}:event:{event['event_id']}"
command = OperationalTaskCreate(
request_id=uuid5(NAMESPACE_URL, external), task_type="event_investigation",
workflow="scada_anomaly", title=f"SCADA 回放调查 · {event['rule_id']}",
summary=event["detail"], source=TaskSource(
kind="external_system", external_id=external, external_revision=1,
event_kind="scada_anomaly",
replay_context={"replay_id": document["replay_id"],
"knowledge_cutoff": batch["at"], "batch_sequence": batch["sequence"]},
),
trigger=TaskTrigger(kind="event", occurred_at=event["occurred_at"], detail="已存 SCADA 历史的回放检测信号,原因尚待调查"),
)
task = await tasks.create(command)
event["task_id"] = str(task.task_id)
document["rule_states"][event.get("state_key", event["rule_id"])]["event"]["task_id"] = event["task_id"]
async def _create_monitoring_task(self, document, monitoring, batch):
from app.services.scada_monitoring_incidents import attach_incident, update_incident
incident, action = update_incident(document, monitoring, batch["sequence"])
if incident is None:
return
if incident.get("task_id"):
if monitoring["status"] == "suspected_burst" and not incident.get("burst_diagnosed"):
from uuid import UUID
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.services.operational_tasks import OperationalTaskService
tasks = OperationalTaskService(
OperationalTaskRepository(self.repo.connection, self.context.project_id),
self.context, permissions_for_context(self.context),
execution_scope=settings.operational_task_execution_scope,
)
await tasks.advance_replay_investigation(UUID(incident["task_id"]), {
"replay_id": document["replay_id"], "knowledge_cutoff": monitoring["knowledge_cutoff"],
"batch_sequence": batch["sequence"],
}, uuid5(NAMESPACE_URL, incident["incident_id"] + ":burst-evidence"))
incident["burst_diagnosed"] = True
if action == "resolved" and incident.get("alert"):
await self._publish_monitoring_alert(document, incident, "resolved")
attach_incident(monitoring, incident, action)
return
from app.domain.schemas.operational_task import OperationalTaskCreate, TaskSource, TaskTrigger
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
from app.services.operational_tasks import OperationalTaskService
tasks = OperationalTaskService(
OperationalTaskRepository(self.repo.connection, self.context.project_id),
self.context,
permissions_for_context(self.context),
execution_scope=settings.operational_task_execution_scope,
)
external = incident["external_id"]
pressure = monitoring["pressure"]
flow = monitoring["outlet_flow"]
detector = monitoring["burst_detection"]
score = detector.get("target_score")
score_text = f"{score:.4f}" if score is not None else "数据不足"
summary = (
f"SCADA 数据诊断发现指标异常,进入异常调查与爆管诊断。"
f"爆管侦测得分 {score_text};"
f"压力下降测点 {len(pressure.get('hit_device_ids', []))} 个;"
f"出厂边界流量增加测点 {len(flow.get('hit_device_ids', []))} 个。"
"任一指标异常即启动调查,加权分用于评估爆管嫌疑;达到门槛后继续候选定位、隔离与影响分析。"
)
if monitoring.get("anomaly_signals"):
signals = monitoring["anomaly_signals"]
devices = {device for item in signals for device in item.get("device_ids", [])}
summary += f" 单项异常 {len(signals)} 项,涉及 {len(devices)} 个测点;明细通过冻结 SCADA 查询核验。"
if monitoring.get("weighted"):
weighted = monitoring["weighted"]
weights = weighted["weights"]
summary += (
f" 综合评分 {weighted['score']:.2f},爆管嫌疑门槛 {weighted['threshold']:.2f}"
f"(侦测 {weights['burst_detection']:.0%}、压力 {weights['pressure']:.0%}、"
f"流量 {weights['outlet_flow']:.0%})。"
)
if detector.get("score_threshold") is not None:
summary += f" 侦测动态阈值 {detector['score_threshold']:.4f}。"
command = OperationalTaskCreate(
request_id=uuid5(NAMESPACE_URL, external),
task_type="event_investigation",
workflow="scada_anomaly",
title=f"SCADA 异常调查 · {monitoring['period_end']}",
summary=summary,
source=TaskSource(
kind="external_system",
external_id=external,
external_revision=1,
event_kind="scada_anomaly",
replay_context={
"replay_id": document["replay_id"],
"knowledge_cutoff": monitoring["knowledge_cutoff"],
"batch_sequence": batch["sequence"],
},
),
trigger=TaskTrigger(
kind="event",
occurred_at=monitoring["period_end"],
detail="SCADA 任一指标异常启动调查,核验数据后进行爆管诊断;异常不等于已确诊爆管",
),
)
task = await tasks.create(command)
incident["task_id"] = str(task.task_id)
incident["burst_diagnosed"] = monitoring["status"] == "suspected_burst"
incident["alert"] = {
"alert_id": incident["incident_id"],
"producer_id": "scada-replay:" + document["replay_id"],
"occurred_at": incident["started_at"],
"condition_id": f"operational-task:{task.task_id}",
"source": {"kind": "scada_replay", "replay_id": document["replay_id"],
"rule_id": "scada_diagnosis", "batch_sequence": batch["sequence"]},
"title": "SCADA 指标异常", "description": "已启动调查,详细诊断见调查报告。", "severity": "warning",
}
await self._publish_monitoring_alert(document, incident, "raised")
attach_incident(monitoring, incident, action)
async def _publish_monitoring_alert(self, document, incident, change):
from app.domain.schemas.task_alert import TaskAlertEvent
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
from app.services.task_alerts import TaskAlertService
alerts = TaskAlertService(
TaskAlertRepository(self.repo.connection, self.context.project_id), self.context,
internal_producer_id="scada-replay:" + document["replay_id"],
)
await alerts.push(TaskAlertEvent(
**incident["alert"], event_type=change,
event_id=uuid5(NAMESPACE_URL, incident["incident_id"] + ":" + change),
))
+287
View File
@@ -0,0 +1,287 @@
"""Deterministic monitoring summaries built only from frozen replay outputs."""
from __future__ import annotations
from datetime import datetime, timedelta
from statistics import median
from typing import Any
from uuid import NAMESPACE_URL, uuid5
import pandas as pd
from app.domain.schemas.scada_replay import ScadaReplayCreate
from app.domain.schemas.project_settings import ScadaMonitoringSettings
from app.services.scada_normal_profile import current_calibration, normal_profile_bounds
from app.services.burst_detection import (
TARGET_MU,
TARGET_N_ESTIMATORS,
TARGET_RANDOM_STATE,
TARGET_SCORE_THRESHOLD,
run_burst_detection,
)
def _time(value: str | datetime) -> datetime:
return value if isinstance(value, datetime) else datetime.fromisoformat(value)
def _valid_rows(rows: list[dict], device_id: str) -> list[dict]:
return sorted(
(
row for row in rows
if row.get("device_id") == device_id
and row.get("quality") in {"valid", "corrected"}
and row.get("cleaned_value") is not None
),
key=lambda row: _time(row["time"]),
)
def _relative_indicator(
rows: list[dict],
config: ScadaReplayCreate,
device_ids: list[str],
period_start: datetime,
period_end: datetime,
*,
direction: str,
calibration: dict | None = None,
) -> dict[str, Any]:
details: list[dict[str, Any]] = []
gaps: list[dict[str, str]] = []
for device_id in device_ids:
rule = next((
item for item in config.rules
if item.kind == "relative_change" and device_id in item.device_ids
and item.threshold is not None
), None)
if rule is None:
gaps.append({"device_id": device_id, "reason": "relative_change_rule_missing"})
continue
profile = calibration.get(device_id) if calibration is not None else None
if calibration is not None and profile is None:
gaps.append({"device_id": device_id, "reason": "adaptive_history_not_ready"})
continue
threshold = float(profile["threshold_percent"]) if profile else rule.threshold
valid = _valid_rows(rows, device_id)
current = [row for row in valid if period_start < _time(row["time"]) <= period_end]
latest = current[-1] if current else None
if not current:
gaps.append({"device_id": device_id, "reason": "period_reading_missing"})
continue
baseline = ([row for row in valid if _time(row["time"]) < _time(latest["time"])]
[-rule.window_samples:]) if latest else []
if len(baseline) < rule.minimum_samples:
gaps.append({"device_id": device_id, "reason": "baseline_not_ready"})
continue
reference = median(float(row["cleaned_value"]) for row in baseline)
if reference == 0:
gaps.append({"device_id": device_id, "reason": "zero_baseline"})
continue
value = float(latest["cleaned_value"])
change = (value - reference) / abs(reference) * 100
hit = change <= -threshold if direction == "decrease" else change >= threshold
bounds = normal_profile_bounds(profile.get("normal_profile"), _time(latest["time"])) if profile else None
if profile and bounds is None:
gaps.append({"device_id": device_id, "reason": "normal_phase_history_not_ready"})
continue
if bounds is not None:
hit = hit and not bounds[0] <= value <= bounds[1]
details.append({
"device_id": device_id,
"observed_at": _time(latest["time"]).isoformat(),
"baseline": reference,
"baseline_method": "recent_samples_median",
"baseline_start": _time(baseline[0]["time"]).isoformat(),
"baseline_end": _time(baseline[-1]["time"]).isoformat(),
"baseline_sample_count": len(baseline),
"value": value,
"value_change": value - reference,
"unit": rule.unit,
"change_percent": change,
"threshold_percent": threshold,
**({"normal_phase_lower": bounds[0], "normal_phase_upper": bounds[1]} if bounds else {}),
**({"threshold_method": profile["method"],
"configured_threshold_percent": rule.threshold} if profile else {}),
"hit": hit,
})
return {
"status": "ready" if not gaps else "insufficient_data",
"hit": any(item["hit"] for item in details),
"hit_device_ids": [item["device_id"] for item in details if item["hit"]],
"devices": details,
"gaps": gaps,
}
def _burst_detection(
rows: list[dict],
pressure_device_ids: list[str],
interval_seconds: int,
period_end: datetime,
*,
score_threshold: float = TARGET_SCORE_THRESHOLD,
) -> dict[str, Any]:
if len(pressure_device_ids) < 5:
return {"status": "sensor_coverage_insufficient", "hit": False,
"required_sensors": 5, "available_sensors": len(pressure_device_ids)}
if interval_seconds % 60 or 86400 % interval_seconds:
return {"status": "unsupported_sampling_interval", "hit": False,
"interval_seconds": interval_seconds}
points_per_day = 86400 // interval_seconds
count = 15 * points_per_day
series_by_device: dict[str, dict[datetime, float]] = {}
common_times: set[datetime] | None = None
for device_id in pressure_device_ids:
values = {_time(row["time"]): float(row["cleaned_value"])
for row in _valid_rows(rows, device_id) if _time(row["time"]) <= period_end}
series_by_device[device_id] = values
common_times = set(values) if common_times is None else common_times & set(values)
if not common_times:
return {"status": "baseline_not_ready", "hit": False, "required_samples": count}
target = period_end
if target not in common_times:
return {"status": "current_reading_missing", "hit": False,
"target_time": target.isoformat(), "latest_common_time": max(common_times).isoformat()}
expected = [target - timedelta(seconds=interval_seconds * offset)
for offset in range(count - 1, -1, -1)]
incomplete = [device_id for device_id, values in series_by_device.items()
if any(at not in values for at in expected)]
if incomplete:
return {"status": "baseline_not_ready", "hit": False, "required_samples": count,
"incomplete_device_ids": incomplete}
frame = pd.DataFrame({device_id: [series_by_device[device_id][at] for at in expected]
for device_id in pressure_device_ids}, index=expected)
try:
result = run_burst_detection(
network="scada-replay",
username="scada-replay",
observed_pressure_data=frame,
points_per_day=points_per_day,
mu=TARGET_MU,
iforest_params={
"n_estimators": TARGET_N_ESTIMATORS,
"random_state": TARGET_RANDOM_STATE,
"contamination": "auto",
},
sensor_nodes=pressure_device_ids,
)
except (TypeError, ValueError) as exc:
return {"status": "analysis_failed", "hit": False, "reason": str(exc)[:300]}
latest = result["rows"][-1]
score = float(latest["Score"])
return {
"status": "ready",
"hit": score <= score_threshold,
"target_time": target.isoformat(),
"target_score": score,
"score_interpretation": "lower_is_more_anomalous_not_probability",
"score_threshold": score_threshold,
"sensor_count": len(pressure_device_ids),
"reference_day_count": 14,
"latest_sensor_rankings": result["summary"].get("latest_sensor_rankings", []),
}
def evaluate_monitoring_period(
document: dict,
rows: list[dict],
*,
period_start: datetime,
period_end: datetime,
batch_sequence: int,
) -> dict[str, Any]:
config = ScadaReplayCreate.model_validate(document["config"])
monitoring = document.get("monitoring_config")
rule_version = (monitoring or {}).get("rule_version", "unconfigured")
monitoring_id = str(uuid5(
NAMESPACE_URL,
f"{document['replay_id']}:{period_end.isoformat()}:{rule_version}",
))
base = {
"monitoring_id": monitoring_id,
"period_start": period_start.isoformat(),
"period_end": period_end.isoformat(),
"knowledge_cutoff": period_end.isoformat(),
"batch_sequence": batch_sequence,
"rule_version": rule_version,
"task_id": None,
}
if not monitoring:
return {**base, "status": "insufficient_data", "detail": "项目未配置 SCADA 监测设备角色",
"pressure": {"status": "unconfigured", "hit": False},
"outlet_flow": {"status": "unconfigured", "hit": False},
"burst_detection": {"status": "not_run", "hit": False}}
pressure_ids = monitoring.get("pressure_device_ids", [])
flow_ids = monitoring.get("outlet_flow_device_ids", [])
selected = set(config.device_ids)
role_gaps = sorted((set(pressure_ids) | set(flow_ids)) - selected)
weighted_policy = monitoring.get("decision_policy") == "weighted"
if not weighted_policy and (not pressure_ids or not flow_ids or role_gaps):
return {**base, "status": "insufficient_data", "detail": "监测角色为空或不属于本轮回放设备",
"role_gaps": role_gaps,
"pressure": {"status": "unconfigured", "hit": False},
"outlet_flow": {"status": "unconfigured", "hit": False},
"burst_detection": {"status": "not_run", "hit": False}}
calibration = current_calibration(document, config.start_time) if weighted_policy else None
if config.start_time > period_end:
calibration = None
base["role_gaps"] = role_gaps
pressure = _relative_indicator(
rows, config, pressure_ids, period_start, period_end, direction="decrease",
calibration=(calibration or {}).get("pressure", {}) if weighted_policy else None,
)
outlet_flow = _relative_indicator(
rows, config, flow_ids, period_start, period_end, direction="increase",
calibration=(calibration or {}).get("outlet_flow", {}) if weighted_policy else None,
) if flow_ids else {"status": "unconfigured", "hit": False, "hit_device_ids": [], "devices": [], "gaps": []}
detector = {"status": "baseline_not_ready", "hit": False} if weighted_policy and (
not calibration or calibration.get("status") != "ready"
) else _burst_detection(
rows, pressure_ids, int(document["monitoring_interval_seconds"]), period_end,
score_threshold=calibration["burst_detection"]["threshold"] if calibration else TARGET_SCORE_THRESHOLD,
)
if weighted_policy:
settings = ScadaMonitoringSettings.model_validate(monitoring)
weights = settings.weights.model_dump()
indicators = {"burst_detection": detector, "pressure": pressure, "outlet_flow": outlet_flow}
contributions = {key: weights[key] if indicator["status"] == "ready" and indicator["hit"] else 0.0
for key, indicator in indicators.items()}
score = round(sum(contributions.values()), 10)
complete = pressure["status"] == detector["status"] == "ready"
diagnostic_missing = sorted({device for rule in config.rules if rule.kind == "relative_change"
for device in rule.device_ids}
- set((calibration or {}).get("diagnostic", {})))
if not complete:
status, detail = "insufficient_data", (
"旧回放缺少新版历史诊断校准,请停止后新建演示" if calibration is None
else "压力数据或爆管侦测基线不足,尚不能作出组合判定"
)
elif score >= settings.alert_threshold:
status, detail = "suspected_burst", "爆管相关指标加权达到疑似爆管门槛"
elif any(indicator["hit"] for indicator in indicators.values()):
status, detail = "attention", "部分指标异常,进入调查;综合结果尚未达到爆管嫌疑门槛"
elif diagnostic_missing:
status, detail = "insufficient_data", "部分测点缺少完整历史同期基线,不能判定全部指标正常"
else:
status, detail = "normal", "本周期爆管相关指标均未命中"
detail += f"(综合 {score:.2f} / 门槛 {settings.alert_threshold:.2f},流量权重 {weights['outlet_flow']:.0%})"
return {**base, "status": status, "detail": detail, "pressure": pressure,
"outlet_flow": outlet_flow, "burst_detection": detector,
"weighted": {"score": score, "threshold": settings.alert_threshold,
"weights": weights, "contributions": contributions,
"unavailable_indicators": [key for key, value in indicators.items() if value["status"] != "ready"],
"calibration_cutoff": calibration["knowledge_cutoff"] if calibration else None,
"diagnostic_unavailable_device_ids": diagnostic_missing,
"missing_weight_policy": "zero_contribution_without_renormalization"}}
complete = pressure["status"] == outlet_flow["status"] == detector["status"] == "ready"
hits = (bool(detector["hit"]), bool(pressure["hit"]), bool(outlet_flow["hit"]))
if not complete:
status, detail = "insufficient_data", "监测数据、设备角色或爆管侦测基线不足"
elif all(hits):
status, detail = "suspected_burst", "爆管侦测、压力下降与出厂边界流量增加相互支持"
elif any(hits):
status, detail = "attention", "仅部分爆管相关指标命中,需要继续观察或人工核查"
else:
status, detail = "normal", "本周期爆管相关指标均未命中"
return {**base, "status": status, "detail": detail, "pressure": pressure,
"outlet_flow": outlet_flow, "burst_detection": detector}
+160
View File
@@ -0,0 +1,160 @@
"""Causal batch quality checks and grouped anomaly episodes, no database IO."""
from copy import deepcopy
from datetime import datetime
from math import isfinite
from statistics import median
from uuid import NAMESPACE_URL, uuid5
import re
from app.domain.schemas.scada_replay import ScadaReplayCreate
from app.services.scada_normal_profile import (
DIAGNOSTIC_POLICY, current_calibration, normal_profile_bounds, weighted_diagnosis,
)
def prepare_cleaning(rows: list[dict], config: ScadaReplayCreate) -> list[dict]:
result = []
for row in rows:
value = row["monitored_value"]
quality, reason = "valid", "原始值通过数值与已配置量程校验"
if value is None:
quality, reason = "missing", "原始读数缺失"
elif not isfinite(float(value)):
quality, reason = "invalid", "原始读数不是有限数值"
else:
bounds = config.quality_ranges.get(row["device_id"])
if bounds and ((bounds.lower is not None and value < bounds.lower)
or (bounds.upper is not None and value > bounds.upper)):
quality, reason = "invalid", "超出本次明确配置的仪表量程"
result.append({
"time": row["time"].isoformat() if isinstance(row["time"], datetime) else row["time"],
"device_id": row["device_id"], "source_hash": row["source_hash"],
"monitored_value": value if value is not None and isfinite(float(value)) else None,
"cleaned_value": value if quality == "valid" else None,
"quality": quality, "reason": reason,
})
return result
def detect_frame(document: dict, rows: list[dict], config: ScadaReplayCreate, at: datetime) -> tuple[dict, list[dict]]:
"""Consume a complete timestamp; never treat missing data as recovery."""
result = deepcopy(document)
histories = result.setdefault("histories", {})
states = result.setdefault("rule_states", {})
latest = result.setdefault("latest", {})
for row in rows:
latest[row["device_id"]] = row
by_id = {row["device_id"]: row for row in rows}
changes = []
devices = {device["device_id"]: device for device in result.get("devices", [])}
weighted = weighted_diagnosis(result)
calibration = current_calibration(result, config.start_time) if weighted else None
for device_id in config.device_ids:
row = by_id.get(device_id)
invalid = row is not None and row["quality"] != "valid"
previous = latest.get(device_id)
frequency = str(devices.get(device_id, {}).get("transmission_frequency", ""))
match = re.fullmatch(r"(\d+):(\d+):(\d+)", frequency)
if match:
seconds = int(match[1]) * 3600 + int(match[2]) * 60 + int(match[3])
else:
minutes = re.fullmatch(r"(\d+)\s*(?:min|分钟)", frequency)
seconds = int(minutes[1]) * 60 if minutes else 0
prior_time = datetime.fromisoformat(previous["time"]) if previous else config.start_time
missing = not row and seconds > 0 and (at - prior_time).total_seconds() >= seconds * 2
key = "quality-" + device_id
state = states.setdefault(key, {"event": None, "recoveries": 0})
if invalid or missing:
state["recoveries"] = 0
if state["event"] is None:
event = {"event_id": str(uuid5(NAMESPACE_URL, f"{result['replay_id']}:{key}:{at.isoformat()}")),
"rule_id": key, "device_ids": [device_id], "occurred_at": at.isoformat(),
"last_observed_at": at.isoformat(), "status": "active", "category": "data_quality",
"detail": row["reason"] if invalid else "连续两个设备周期未读取到观测",
"task_id": None}
state["event"] = event
changes.append({**event, "change": "raised"})
result["anomaly_count"] = result.get("anomaly_count", 0) + 1
elif row and row["quality"] == "valid" and state["event"]:
state["recoveries"] += 1
if state["recoveries"] >= 2:
changes.append({**state["event"], "status": "resolved", "resolved_at": at.isoformat(), "change": "resolved"})
state["event"] = None
for rule in config.rules:
state_key = f"{rule.rule_id}:{DIAGNOSTIC_POLICY}" if weighted and rule.kind == "relative_change" else rule.rule_id
state = states.setdefault(state_key, {"hits": 0, "recoveries": 0, "event": None})
hits, usable, reasons = [], [], []
for device_id in rule.device_ids:
row = by_id.get(device_id)
if not row or row["quality"] != "valid":
continue
value = row["cleaned_value"]
if rule.kind == "bounds":
triggered = ((rule.lower is not None and value < rule.lower)
or (rule.upper is not None and value > rule.upper))
reason = f"{device_id}={value} {rule.unit},超出配置范围"
elif weighted:
# The percentage in the compatibility rule is not a diagnosis
# threshold. Learn each device's normal same-phase range first.
profile = (calibration or {}).get("diagnostic", {}).get(device_id)
bounds = normal_profile_bounds(profile, at)
if bounds is None:
continue
low, high = bounds
triggered = (value < low or value > high if rule.direction == "both"
else value > high if rule.direction == "increase" else value < low)
reason = f"{device_id}={value} {rule.unit},超出冻结正常同时段范围 [{low:.4f}, {high:.4f}] {rule.unit}"
else:
previous = histories.get(device_id, [])[-rule.window_samples:]
if len(previous) < rule.minimum_samples:
continue
baseline = median(previous)
if baseline == 0:
continue
change = (value - baseline) / abs(baseline) * 100
triggered = (abs(change) >= rule.threshold if rule.direction == "both"
else change >= rule.threshold if rule.direction == "increase"
else change <= -rule.threshold)
reason = f"{device_id} 较历史中位值变化 {change:.1f}%,阈值 {rule.threshold}%"
usable.append(device_id)
if triggered:
hits.append(device_id)
reasons.append(reason)
if len(hits) >= rule.minimum_devices:
state["hits"] += 1
state["recoveries"] = 0
if state["event"] is None and state["hits"] >= rule.consecutive_samples:
event_id = str(uuid5(NAMESPACE_URL, f"{result['replay_id']}:{state_key}:{at.isoformat()}"))
event = {"event_id": event_id, "rule_id": rule.rule_id, "device_ids": hits,
"occurred_at": at.isoformat(), "last_observed_at": at.isoformat(),
"status": "active", "detail": ";".join(reasons)[:2000], "task_id": None}
if weighted and rule.kind == "relative_change":
event.update(diagnostic_policy=DIAGNOSTIC_POLICY, state_key=state_key)
state["event"] = event
changes.append({**event, "change": "raised"})
result["anomaly_count"] = result.get("anomaly_count", 0) + 1
elif state["event"]:
state["event"]["last_observed_at"] = at.isoformat()
elif len(usable) == len(rule.device_ids):
state["hits"] = 0
if state["event"]:
state["recoveries"] += 1
if state["recoveries"] >= rule.recovery_samples:
event = {**state["event"], "status": "resolved", "resolved_at": at.isoformat(),
"change": "resolved"}
changes.append(event)
state["event"] = None
state["recoveries"] = 0
# Incomplete frames cannot count as a consecutive hit/recovery.
elif not usable and any(device in by_id for device in rule.device_ids):
state["hits"] = state["recoveries"] = 0
frozen = {device for rule in config.rules if states[
f"{rule.rule_id}:{DIAGNOSTIC_POLICY}" if weighted and rule.kind == "relative_change" else rule.rule_id
]["event"]
for device in rule.device_ids}
for row in rows:
if row["quality"] == "valid" and row["device_id"] not in frozen:
history = histories.setdefault(row["device_id"], [])
history.append(row["cleaned_value"])
histories[row["device_id"]] = history[-96:]
result["processed_count"] = result.get("processed_count", 0) + len(rows)
return result, changes
+102
View File
@@ -0,0 +1,102 @@
"""Server-enforced replay knowledge bounds and immutable corrected snapshots."""
from datetime import datetime
from fastapi import HTTPException
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
def check_replay_window(context, start, end):
cap = datetime.fromisoformat(context["knowledge_cutoff"])
if start is None or end is None or start.tzinfo is None or end.tzinfo is None or start > end or end > cap:
raise HTTPException(422, f"回放查询需要明确时间范围,结束不得晚于 {cap.isoformat()}")
def check_snapshot_start(run, start):
replay_start = datetime.fromisoformat(run["start_time"])
if start < replay_start:
baseline = run.get("baseline")
if not baseline:
raise HTTPException(422, "本轮回放未冻结历史对照,请新建回放后查询历史基线")
if start < datetime.fromisoformat(baseline["start_time"]):
raise HTTPException(422, "查询起点早于本轮已冻结的历史基线")
async def replay_snapshot_rows(connection, project_id, context, device_ids, start, end, limit, offset=0):
check_replay_window(context, start, end)
repository = ScadaReplayRepository(connection, project_id)
run = await repository.get(context["replay_id"])
if not set(device_ids) <= set(run["config"]["device_ids"]):
raise HTTPException(403, "设备或回放不属于当前调查")
check_snapshot_start(run, start)
rows = await repository.detected_outputs(
context["replay_id"], device_ids, start, end, context["batch_sequence"],
limit=limit, offset=offset,
)
return [{key: row[key] for key in (
"time", "device_id", "monitored_value", "cleaned_value", "quality", "reason"
)} for row in rows]
async def replay_burst_location_inputs(connection, project_id, context, args):
"""Build location series from immutable replay outputs, never live TSDB rows."""
if args.data_source != "monitoring":
raise HTTPException(422, "回放定位只支持 monitoring 数据源")
if args.scada_normal_start is None or args.scada_normal_end is None:
raise HTTPException(422, "回放定位必须显式提供已处理范围内的正常对照窗口")
check_replay_window(context, args.scada_burst_start, args.scada_burst_end)
check_replay_window(context, args.scada_normal_start, args.scada_normal_end)
repository = ScadaReplayRepository(connection, project_id)
run = await repository.get(context["replay_id"])
if run:
check_snapshot_start(run, min(args.scada_burst_start, args.scada_normal_start))
devices = {item["device_id"]: item for item in run.get("devices", [])}
monitoring = run.get("monitoring_config") or {}
pressure_devices = monitoring.get("pressure_device_ids", [])
pressure_map = {str(devices[item]["node_id"]): item for item in pressure_devices
if item in devices and devices[item].get("node_id")}
requested_pressure = args.pressure_scada_ids or list(pressure_map)
if not requested_pressure or not set(requested_pressure) <= set(pressure_map):
raise HTTPException(422, "压力节点必须来自本轮已配置的回放监测角色")
flow_devices = monitoring.get("outlet_flow_device_ids", [])
flow_map = {str(devices[item]["link_id"]): item for item in flow_devices
if item in devices and devices[item].get("link_id")}
requested_flow = args.flow_scada_ids or []
if not set(requested_flow) <= set(flow_map):
raise HTTPException(422, "流量要素必须来自本轮已配置的出厂边界流量角色")
selected = [pressure_map[item] for item in requested_pressure] + [flow_map[item] for item in requested_flow]
rows = await repository.detected_outputs(
context["replay_id"], selected,
min(args.scada_burst_start, args.scada_normal_start),
max(args.scada_burst_end, args.scada_normal_end),
context["batch_sequence"],
)
def averages(mapping, start, end):
result = {}
for element_id, device_id in mapping.items():
values = [float(row["cleaned_value"]) for row in rows
if row.get("device_id") == device_id
and row.get("quality") in {"valid", "corrected"}
and row.get("cleaned_value") is not None
and start <= datetime.fromisoformat(row["time"]) <= end]
if not values:
raise HTTPException(422, f"回放窗口缺少 {device_id} 的有效清洗观测")
result[element_id] = sum(values) / len(values)
return result
pressure_mapping = {item: pressure_map[item] for item in requested_pressure}
flow_mapping = {item: flow_map[item] for item in requested_flow}
return {
"data_source": "monitoring",
"burst_leakage": args.burst_leakage,
"pressure_scada_ids": requested_pressure,
"burst_pressure": averages(pressure_mapping, args.scada_burst_start, args.scada_burst_end),
"normal_pressure": averages(pressure_mapping, args.scada_normal_start, args.scada_normal_end),
**({
"flow_scada_ids": requested_flow,
"burst_flow": averages(flow_mapping, args.scada_burst_start, args.scada_burst_end),
"normal_flow": averages(flow_mapping, args.scada_normal_start, args.scada_normal_end),
"use_scada_flow": True,
} if requested_flow else {}),
}
+78
View File
@@ -0,0 +1,78 @@
"""Publication-only validation for same-time-of-day line/scatter comparisons."""
from __future__ import annotations
import re
from collections.abc import Mapping
from datetime import datetime, timedelta, timezone
from typing import Any
from app.domain.domain_errors import require
DISPLAY_TIMEZONE = timezone(timedelta(hours=8))
DAY_SECONDS = 86400
def validate_time_comparison_chart(chart: Mapping[str, Any]) -> None:
"""Validate new publications; persisted historical charts keep their read contract."""
chart_type = chart["chart_type"]
series = chart["series"]
kinds = {item.get("type") or chart_type for item in series}
if chart_type != "time_comparison":
require(not {"line", "scatter"} <= kinds, "TIME_COMPARISON_INVALID",
"曲线与历史散点复合图须使用 chart_type=time_comparison", status=422)
return
def check(condition: bool, message: str) -> None:
require(condition, "TIME_COMPARISON_INVALID", message, status=422)
kinds = {item.get("type") or "line" for item in series}
check(kinds == {"line", "scatter"},
"time_comparison 的 series 必须包含当前 line 和历史 scatter,不能混入 bar")
labels = chart["x_data"]
check(all(re.fullmatch(r"(?:[01][0-9]|2[0-3]):[0-5][0-9](?::[0-5][0-9])?", label)
for label in labels),
"time_comparison.x_data 只接受 HH:mm 或 HH:mm:ss 时刻;历史日期保留在 sample_times,不能追加到横轴")
check(len({len(label) for label in labels}) == 1, "x_data 必须使用一致的时刻精度")
seconds = [sum(int(part) * factor for part, factor in zip(label.split(":"), (3600, 60, 1)))
for label in labels]
offsets = [0]
for index in range(1, len(seconds)):
step = (seconds[index] - seconds[index - 1]) % DAY_SECONDS
check(step > 0, f"x_data[{index}] 重复了上一时刻;历史点应复用当前槽位")
offsets.append(offsets[-1] + step)
check(offsets[-1] < DAY_SECONDS, "x_data 必须按时间递增且小于一天,不能追加多天历史槽位")
current_bases: set[datetime] = set()
historical_bases: list[datetime] = []
for series_index, item in enumerate(series):
times = item.get("sample_times")
path = f"series[{series_index}]"
check(times is not None and len(times) == len(labels), f"{path}.sample_times 必须提供并与 x_data 等长")
bases: set[datetime] = set()
for index, (value, at) in enumerate(zip(item["data"], times, strict=True)):
check((value is None) == (at is None),
f"{path}.sample_times[{index}]:每个有效值必须保留真实带时区时间,缺测值和时间均填 null")
if at is None:
continue
try:
local = datetime.fromisoformat(at).astimezone(DISPLAY_TIMEZONE)
base = local - timedelta(seconds=offsets[index])
except OverflowError:
check(False, f"{path}.sample_times[{index}] 超出同期对比可表达的日期范围")
label = labels[index]
expected = local.strftime("%H:%M:%S" if len(label) == 8 else "%H:%M")
check(expected == label and local.microsecond == 0 and (len(label) == 8 or local.second == 0),
f"{path}.sample_times[{index}] 换算为 UTC+8 后须与 x_data[{index}]={label} 对齐")
bases.add(base)
check(len(bases) <= 1, f"{path}.sample_times 必须按同一窗口对齐,不能将不同日期拼入同一组")
if (item.get("type") or "line") == "line":
current_bases.update(bases)
else:
historical_bases.extend(bases)
check(len(current_bases) == 1, "当前 line 必须包含真实采样时间,且所有当前曲线使用同一窗口")
current = next(iter(current_bases))
for historical in historical_bases:
delta = (current - historical).total_seconds()
check(delta > 0 and delta % DAY_SECONDS == 0,
"历史 scatter 必须来自当前窗口之前的对应日期,不能使用当天或未来采样")
+44 -2
View File
@@ -310,9 +310,18 @@
"type": "string",
"enum": [
"line",
"bar"
"bar",
"time_comparison"
]
},
"x_axis_name": {
"type": "string",
"maxLength": 128
},
"y_axis_name": {
"type": "string",
"maxLength": 128
},
"x_data": {
"type": "array",
"items": {
@@ -331,10 +340,43 @@
"type": "string",
"maxLength": 200
},
"type": {
"type": "string",
"enum": [
"line",
"bar",
"scatter"
]
},
"data": {
"type": "array",
"items": {
"type": "number"
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
]
},
"minItems": 1,
"maxItems": 100
},
"sample_times": {
"type": "array",
"items": {
"anyOf": [
{
"type": "string",
"maxLength": 64,
"format": "date-time",
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
},
{
"type": "null"
}
]
},
"minItems": 1,
"maxItems": 100
+1 -1
View File
@@ -3,7 +3,7 @@
"contracts": {
"server": {
"file": "server-v1.openapi.json",
"sha256": "fa5b186d0f1e4257352a812e578cb9901baf47beaca33c388e876f1a7b4c42ae"
"sha256": "426b8d9170f253aa7132e9376031a7da8c04ac6f6e693948890d6f343420af1b"
}
}
}
+12
View File
@@ -1687,6 +1687,18 @@
],
"title": "Location"
},
"occurred_at": {
"anyOf": [
{
"format": "date-time",
"type": "string"
},
{
"type": "null"
}
],
"title": "Occurred At"
},
"priority": {
"anyOf": [
{
File diff suppressed because it is too large Load Diff
+44 -2
View File
@@ -211,9 +211,18 @@
"type": "string",
"enum": [
"line",
"bar"
"bar",
"time_comparison"
]
},
"x_axis_name": {
"type": "string",
"maxLength": 128
},
"y_axis_name": {
"type": "string",
"maxLength": 128
},
"x_data": {
"type": "array",
"items": {
@@ -232,10 +241,43 @@
"type": "string",
"maxLength": 128
},
"type": {
"type": "string",
"enum": [
"line",
"bar",
"scatter"
]
},
"data": {
"type": "array",
"items": {
"type": "number"
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
]
},
"minItems": 1,
"maxItems": 100
},
"sample_times": {
"type": "array",
"items": {
"anyOf": [
{
"type": "string",
"maxLength": 64,
"format": "date-time",
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
},
{
"type": "null"
}
]
},
"minItems": 1,
"maxItems": 100
+65
View File
@@ -0,0 +1,65 @@
<svg xmlns="http://www.w3.org/2000/svg" width="720" height="362" viewBox="0 0 720 362" role="img" aria-labelledby="title description">
<title id="title">SCADA 演示时间轴 FM 调频盘样式</title>
<desc id="description">静态设计示意。选定区间为十月八日八点至十四点,当前时间十点十五分。半透明玻璃刻度窗内细密时间刻度横向排列,中央红色指示线固定,控制栏包含暂停、下一批和六十倍速。</desc>
<defs>
<linearGradient id="glass" x1="0" y1="0" x2="0" y2="1">
<stop offset="0" stop-color="#fff" stop-opacity=".72"/>
<stop offset=".5" stop-color="#f7faff" stop-opacity=".4"/>
<stop offset="1" stop-color="#fff" stop-opacity=".64"/>
</linearGradient>
<linearGradient id="edge" x1="0" y1="0" x2="1" y2="0">
<stop offset="0" stop-color="#f7f9fc"/>
<stop offset=".12" stop-color="#f7f9fc" stop-opacity="0"/>
<stop offset=".88" stop-color="#f7f9fc" stop-opacity="0"/>
<stop offset="1" stop-color="#f7f9fc"/>
</linearGradient>
<filter id="blur"><feGaussianBlur stdDeviation="16"/></filter>
<filter id="shadow" x="-.1" y="-.3" width="1.2" height="1.6">
<feDropShadow dx="0" dy="2" stdDeviation="4" flood-color="#243852" flood-opacity=".1"/>
</filter>
<clipPath id="window"><rect x="48" y="130" width="624" height="110" rx="12"/></clipPath>
</defs>
<rect width="720" height="362" rx="18" fill="#e9edf2"/>
<rect x="24" y="22" width="672" height="318" rx="16" fill="#fafbfd" stroke="#d7dee7"/>
<g font-family="'Noto Sans SC Variable', -apple-system, BlinkMacSystemFont, 'Segoe UI', 'PingFang SC', 'Noto Sans CJK SC', sans-serif" fill="#263347">
<text x="48" y="54" font-size="13" font-weight="600">SCADA 时段演示</text>
<text x="672" y="54" text-anchor="end" font-size="11" fill="#778398">静态样式示意</text>
<text x="48" y="79" font-size="12" fill="#68758a">演示区间</text>
<text x="114" y="79" font-size="12">10 月 8 日 08:00 — 14:00</text>
<text x="360" y="118" text-anchor="middle" font-size="28" font-weight="500" font-variant-numeric="tabular-nums" letter-spacing="1">10:15:00</text>
<rect x="48" y="130" width="624" height="110" rx="12" fill="#edf2f7" filter="url(#shadow)"/>
<g clip-path="url(#window)">
<g filter="url(#blur)" opacity=".6">
<path d="M12 234 L280 132 L423 247 L749 142" fill="none" stroke="#c7d8e7" stroke-width="35"/>
<path d="M63 134 L221 219 L438 128 L647 212" fill="none" stroke="#fff" stroke-width="38"/>
</g>
<rect x="48" y="130" width="624" height="110" fill="url(#glass)"/>
<path d="M65 131 H655" stroke="#fff" stroke-width="2" opacity=".9"/>
<path d="M49 138 V231 M671 138 V231" stroke="#fff" opacity=".6"/>
<path d="M60 237 H660" stroke="#a8b6c8" opacity=".25"/>
<g fill="none" stroke="#67778b" stroke-width="1">
<path d="M90 166 V197 M180 166 V197 M270 166 V197 M360 166 V197 M450 166 V197 M540 166 V197 M630 166 V197"/>
<path d="M60 179 V196 M120 179 V196 M150 179 V196 M210 179 V196 M240 179 V196 M300 179 V196 M330 179 V196 M390 179 V196 M420 179 V196 M480 179 V196 M510 179 V196 M570 179 V196 M600 179 V196 M660 179 V196" opacity=".65"/>
</g>
<g font-size="11" text-anchor="middle" font-variant-numeric="tabular-nums">
<text x="90" y="216">09:30</text><text x="180" y="216">09:45</text>
<text x="270" y="216">10:00</text><text x="360" y="216" font-weight="600">10:15</text>
<g fill="#7b889a"><text x="450" y="216">10:30</text><text x="540" y="216">10:45</text><text x="630" y="216">11:00</text></g>
</g>
<rect x="48" y="131" width="624" height="108" fill="url(#edge)"/>
</g>
<rect x="48.5" y="130.5" width="623" height="109" rx="11.5" fill="none" stroke="#fff" stroke-opacity=".8"/>
<path d="M356 141 H364 L360 147 Z" fill="#cf453f"/>
<path d="M360 147 V201" stroke="#cf453f" stroke-width="2"/>
<rect x="48" y="256" width="88" height="38" rx="8" fill="#4662ba"/>
<path d="M65 269 V281 M70 269 V281" stroke="#fff" stroke-width="2"/>
<text x="98" y="280" text-anchor="middle" fill="#fff" font-size="12">暂停</text>
<rect x="148" y="256" width="88" height="38" rx="8" fill="#fff" stroke="#d7dee7"/>
<text x="192" y="280" text-anchor="middle" font-size="12" fill="#7b8799">下一批</text>
<rect x="578" y="256" width="94" height="38" rx="8" fill="#fff" stroke="#d7dee7"/>
<text x="601" y="280" font-size="12">60×</text>
<path d="M651 273 L655 277 L659 273" fill="none" stroke="#64738a" stroke-width="1.5"/>
<text x="48" y="319" font-size="11" fill="#68758a">已读取 10:15 · 已检测 10:10</text>
<text x="672" y="319" text-anchor="end" font-size="11" fill="#68758a">红线对应当前时间 · 横向拨动可回看</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 4.7 KiB

+43 -225
View File
@@ -1,248 +1,66 @@
# 从爆管上报到处置闭环
# 从爆管工单到事件处置闭环
本文定义爆管调查的完整业务路线,供维护者审阅和验收。运行时指令位于 [burst-report/SKILL.md](../resources/work-order-skills/burst-report/SKILL.md),由 Server 根据绑定任务的 `workflow=burst_report` 经 MCP 分发。
工单触发的 `workflow=burst_report` 使用 [爆管事件处理 Skill](../resources/work-order-skills/burst-event-handling/SKILL.md)。Server 按持久化来源选择流程,Agent 接收同一任务的核验上下文;SCADA 异常和历史回放使用 [SCADA Skill](../resources/work-order-skills/scada-anomaly/SKILL.md)。
重点成果是有依据的事故候选位置、隔离阀门清单和影响范围,以及经审批执行、反馈核验后的闭环记录。Agent 根据证据选择、跳过或重做分支,不按图逐节点机械调用工具。工程图覆盖多轮调查和人工业务;每轮发布的进度图只标注实际进行的工作,不复制整图冒充执行记录。
## 完整工程图
## 固定处理路线
```mermaid
flowchart TB
subgraph intake[工单进入与后台任务]
order[用户或外围系统提交爆管上报工单]
source[保存原工单及来源版本、发生时间、位置和描述]
sync[同步器按来源关联调查及工单投影]
queue[Server 保存调查任务和 queued run]
claim[Worker 投递,Agent 领取并维持租约]
context[读取任务摘要、业务 Skill、历史报告和既有动作]
order --> source --> sync --> queue --> claim --> context
end
subgraph investigation[Agent 调查与工程分析]
scope[确定事件时刻、地点、调查目标和真实或演练来源]
location[核对已有资产编号;必要时解析地点和查询附近管段]
monitor[核对相关测点映射、异常窗口、正常对照和数据质量]
cover{查询是否实际覆盖事件与对照窗口}
more[缩小对象和窗口,按需补页,核对缺测及零值]
hypothesis{是否具备定位所需观测和漏水量依据}
locate[计算爆管候选排序,保留假设、覆盖范围与证据]
candidate[整合上报、资产关联、算法与现场反馈得到候选集合]
usable{是否存在可用于隔离评估的候选资产}
isolation[逐候选分析拓扑隔离边界、必关阀及受影响节点]
isolateok{可隔离且阀门可用性有可核验依据}
alternate[纳入已知不可关闭阀门,重新评估替代隔离方案]
hydraulic[按需模拟候选关阀组合的压力和流量影响]
impact[区分拓扑停水区、水力影响区和有资料支持的用户范围]
findings[形成候选位置、阀门清单、影响对比和建议方案]
fieldbasis{是否已有足以支持具体现场核验的结论}
fielddraft[形成有位置、核验目标及回执要求的现场核验建议]
gap[真实关键数据缺失:发布 final insufficient_data 并结束本轮]
error[接口、权限或配置异常:记录具体失败并由宿主结束失败运行]
scope --> location --> candidate
scope --> monitor --> cover
cover -->|未覆盖且可补齐| more --> monitor
cover -->|已覆盖或确认实际缺失| hypothesis
hypothesis -->|具备依据| locate --> candidate
hypothesis -->|条件不具备,明确原因| candidate
candidate --> usable
usable -->|候选有效,标明待现场确认| isolation --> isolateok
isolateok -->|有可评估方案| hydraulic --> impact --> findings
isolateok -->|已知存在不可关阀| alternate --> isolation
isolateok -->|无法形成方案| fieldbasis
usable -->|尚无候选| fieldbasis
fieldbasis -->|可形成具体核验建议| fielddraft
fieldbasis -->|当前判断所需证据确实缺失| gap
monitor -->|不可恢复的查询异常| error
location -->|不可恢复的必要查询异常| error
end
subgraph approval[报告、审批与现场处置]
interim[保存本轮 interim concluded 阶段报告]
proposals[创建具体待审批草稿,关联证据和执行前置条件]
finalround[保存本轮 final 报告,由宿主结束本轮 run]
review[有权限人员审核当前草稿版本]
decision{审批结果}
revise[驳回理由保存并触发原调查新一轮]
submit[Server 通过配置的适配器幂等提交]
receipt{外部提交结果}
reconcile[使用同一执行请求号核对回执,不重复下发]
unsupported[未配置或明确失败:记录原因并重新分析]
workkind{本次获批的是核验还是处置}
fieldcheck[现场核验准确位置、漏水迹象及资产,回传实际记录]
notice[人工实施获批的通知、供水保障和应急安排]
isolatefield[现场核验阀门及实际隔离结果]
repair[现场定位、止水和抢修,回传实际故障管段及修复记录]
clean[按现场风险及获批方案清洗、消毒或冲洗]
restore[核验修复与适用复测,按批准方案恢复供水]
follow[记录恢复后压力流量、漏水复查及用户回访]
findings --> interim
fielddraft --> interim
interim --> proposals --> finalround --> review --> decision
decision -->|驳回| revise
decision -->|批准| submit --> receipt
receipt -->|结果不明或仍在执行| reconcile --> receipt
receipt -->|明确失败或不支持| unsupported
receipt -->|工单已创建或设备动作有回执| workkind
workkind -->|现场核验事项| fieldcheck
workkind -->|有依据的处置事项| notice
notice --> isolatefield --> repair --> clean --> restore --> follow
end
subgraph closure[现场反馈、复核与原事件闭环]
feedback[外围系统保存关联工单的新反馈和单调版本]
requeue[Server 校验来源并为原调查排队新 generation]
reread[重新读取现场事实、执行回执和最新监测证据]
verify[对照预测与现场结果,复核修复、恢复和风险]
faultstate{复核结果支持故障、误报还是尚待核验}
falsealarm[核对排除故障的证据及人工确认]
closed{处置及闭环证据是否齐全}
remaining[列出未完成事项及对应已有或待审批动作]
sourceclosed{原上报工单已结案且版本已核对}
readyreport[发布具备结案条件报告并结束本轮]
closedreport[发布闭环复核 final 报告并结束本轮]
closeowner[有权限人员在原工单系统核对并完成原事件结案]
closedcase[报告与来源事实一致,确认业务闭环]
follow --> feedback --> requeue
fieldcheck --> feedback
reread --> verify --> faultstate
faultstate -->|已确认故障| closed
faultstate -->|有依据排除故障| falsealarm --> sourceclosed
faultstate -->|尚需核验| remaining
closed -->|尚未齐全或复测异常| remaining --> interim
closed -->|证据齐全| sourceclosed
sourceclosed -->|是| closedreport --> closedcase
sourceclosed -->|否| readyreport --> closeowner --> feedback
reread -->|现场位置或阀门条件改变| candidate
end
subgraph presentation[贯穿调查的地图、图表与数据整理]
progress[发布真实调查进展和 Mermaid 状态]
map[高亮候选管段、闪烁建议关阀、展示已有证据覆盖层]
chart[打开异常与对照曲线,展示候选及影响对比表]
filepermit{会话是否已有文件写入权限}
jsonl[按权限整理有来源引用的 JSONL 调查数据]
reportui[通过报告 ui_blocks 发布 json-render 结构化界面]
sourceview[展示保持证据来源,失败不阻断业务调查]
progress --> map --> chart --> filepermit
filepermit -->|已获准且需要导出| jsonl --> reportui
filepermit -->|未获准或无需导出| reportui
reportui --> sourceview
end
context -->|首次或需要重新分析| scope
context -->|已有新反馈待复核| reread
revise --> requeue
unsupported --> requeue
requeue --> queue
gap -->|补充事实被 Server 接受后才重开| requeue
findings -->|无须新增动作| finalonly[无需新增动作时保存 final 报告并结束本轮]
scope -.-> progress
candidate -.-> map
impact -.-> chart
verify -.-> reportui
flowchart TD
order[爆管上报工单] --> context[读取原工单与当前任务]
context --> asset[核对地点并关联管网资产]
asset --> isolation[分析隔离边界与阀门方案]
isolation --> impact[说明事件与隔离方案影响]
impact --> report[保存阶段报告与处置建议]
report --> approval[人工审批与现场执行]
approval --> feedback[接收实际处置与恢复反馈]
feedback --> verify[复核影响与结案条件]
verify --> close[来源系统确认事件闭环]
verify -->|新增事实或遗留事项| asset
```
图中的现场处置由实际审批、工单与回执推进;顺序按现场紧迫性和获批方案调整。必要的紧急通知、现场核验和供水保障不应等待可选的完整模型计算。`clean` 为按风险适用的步骤,不能把每次爆管都判断为污染。`hydraulic` 在缺少模型、时序输入或权限时记录具体限制,仍可保留已完成的拓扑隔离结果。
1. **接收核对**:保留来源工单ID、单调版本、发生时间、位置、描述、现场反馈和已有动作;后续轮次从新增事实接续。
2. **资产关联**:优先核对上报资产编号。坐标或地址通过地理编码、附近资产、属性和连接关系形成有依据的关联;位置歧义列为具体待确认项。
3. **隔离方案**:用 analyze_valve_isolation 分析关联资产,保留必关阀、可选阀、直接隔离与供水失源节点;已知失效阀门用于替代方案分析,现场可操作性列为执行条件。
4. **影响说明**:先展示拓扑范围,再按可用SCADA、水力结果和用户关联资料解释事件当前影响、方案预计影响和保障需求。
5. **报告与处置**:保存阶段报告后提出核验、通知、保障、抢修和恢复等动作草稿;关联现场任务采用 general 工单。审批、派发和现场完成分别核对回执。
6. **反馈复核**:已派发的关联工单保持 `waiting_work_order`,全部收到有效 `completed` 反馈后进入 `waiting_scada`。以现场反馈业务时间为 T,取严格晚于 T 的最近一个 SCADA 采样时刻进行复核;正常范围使用事件前15天同采样相位的正常包络。数据未到或仍异常时保持等待,后台工作进程定期复核;通过后持久保存恢复证据并接续 Agent。Agent 最终报告必须引用该证据,运行完成后才能完成任务并解除对应告警。原事件由有权限人员在来源系统确认行政结案。
`requeue` 表示 Server 接受了新业务事实后分配下一轮,不是 Agent 复活旧会话或自行循环等待。原工单结案由原系统有权限人员处理,后台 Agent 目前没有直接关闭原工单的工具。`completed`、`concluded`、动作 `succeeded` 和“业务闭环”分别核对,不能互相推导。
SCADA作为事件影响的辅助证据,按目标测点和事件/对照窗口读取。具备水力输入且工具已授权时,使用 analyze_valve_closure 评估给定阀门方案;各步骤记录实际结果、适用范围或具体待补项。
## 将业务步骤映射为现有能力
## 报告固定内容
以下产品查询与计算均使用 `mcp_tjwater_` 前缀。参数以本轮实际发现的工具 schema 为准。
报告包含事件概况、地点与资产关联、隔离方案、影响说明、处置建议和待办。摘要以150字以内为目标,资产和测点明细放表格,完整依据引用证据。
| 业务动作 | 已有入口 | 必须形成的结果 |
| --- | --- | --- |
| 接续原任务 | `get_task_context`,依次读取 summary、skill 及所需分节;`get_task_run_history` | 原工单来源、事件时刻、调查代次、流程版本、既有动作与现场反馈 |
| 查找地点和资产 | `geocode` / `search_project_places`、`query_nearby_elements`、`get_element_properties`、`get_node_connections` | 实际候选管段或节点、位置来源、匹配理由;邻近不代表故障确认 |
| 核对监测 | `list_scada_devices`、`get_scada_device`、`query_scada_readings`、`query_element_history` | 设备和模型节点的映射、异常及对照窗口、单位、缺测、分页覆盖 |
| 压力异常侦测 | `run_burst_detection` | 至少 5 个完整压力测点及此前 15 天数据满足时才调用;不把侦测当作定位 |
| 爆管候选定位 | `run_burst_location` | 候选排序、算法实际返回的评价值、观测范围、漏水量假设及来源 |
| 逐候选隔离 | `analyze_valve_isolation` | `accident_elements`、`isolatable`、`must_close_valves`、`optional_valves`、`affected_nodes`、`affected_node_count` 及已知 `disabled_valves` |
| 水力影响 | `analyze_valve_closure` | 指定关阀组合、时间窗、输出对象和模拟压力流量;参数不足时保留明确原因 |
| 用户影响 | 已授权资产属性、区域及外围资料查询 | 受影响对象、关联依据和覆盖限制;没有用户资料不估造户数 |
| 发布和提议 | `publish_task_progress`、`publish_task_report`、`propose_task_action` | 持久化报告、证据引用和待审批动作;动作不是执行事实 |
| 反馈接续 | Server `external-feedback` / `human-evidence` 及 worker | 同一任务的新轮次、真实来源版本、修复和复测证据 |
“影响说明”逐项保留时间、对象、单位、范围、来源与实际或预计的性质:
定位的 `burst_leakage` 必须有依据,单位为 m³/s;`pressure_scada_ids` 是模型节点 ID,不能填设备查询编号,也可省略并使用服务的项目配置。monitoring 模式正常窗可成对省略,使用前一天同窗,但这只是一项对照假设,仍要核对数据有效性。simulation 模式必须引用已有分析运行 UUID,不把在线重算当成已有分析运行。已知事故管段可直接做候选隔离;不必先完成一次没有必要的定位计算,也不能把已知答案作为算法独立定位成功的证明。
| 影响项 | 依据与输出 |
| --- | --- |
| 事件当前影响 | 已查询测点的压力、流量、持续时间和恢复情况 |
| 直接隔离范围 | affected_nodes及对应计数、地图覆盖 |
| 新增供水失源范围 | supply_affected_nodes及对应计数、地图覆盖 |
| 水力影响 | 给定阀门组合、起算时间、模型条件下的压力和流量结果 |
| 用户与保障影响 | 有资料关联的用户、重点设施、保障需求;缺少关联时报告节点与区域范围 |
隔离工具支持明确标注的候选管段,无须等到现场确认才评估方案。多个候选的比较优先分开调用;把多个 ID 一次传入表示联合事故场景,不等同于逐个比较。`disabled_valves` 仅填已有事实支持的不可关阀;空数组不证明现场阀门均可用。`isolatable=false` 不能解释为零影响,即使 `affected_nodes` 为空也须解释返回的计数和限制。
变化量基于同口径对象、单位、时间与模型条件计算;现场实际、模型预测和空间示意分别标注。缺少可选模拟参数时,继续形成已有依据的隔离与保障建议。
## 像调查员一样查询、查看、比较和复核
## 工具与入口约束
每一步先明确待回答的问题,再查小范围资料,观察实际结果后决定下一步。例如:核实地址,查附近管段并查看属性,关联压力点,比较事件前后及正常对照,检查候选,评估关阀影响。只把必要的查询、事实和判断依据写入进度及报告,不保存模型内部推理。
工单事件从上报位置开展资产关联。Agent会话策略隐藏 run_burst_location,Server在绑定与后续解析时同步移除其调用能力;普通聊天和SCADA候选定位沿用各自策略。持久计算在执行前重新校验任务绑定。
- 不满足于“查过 SCADA”:必须确认返回的时间实际覆盖事件。查询 24 小时而只返回当天前 45 分钟时,应缩到事件前后的小窗,按需补页,不直接进入最终判断。
- 正常基线、实测、模型模拟、外围演练和人工反馈分别标注;缺测不是零值,异常不是爆管确认,候选评分不是校准后的概率。
- 查不到事故管段时先检查地址解析、附近资产与上下游关联;能查询的字段先查询,不能直接全部交给现场。
- 既有证据通过 `get_task_evidence` 的有界分页读取,不重复拉取同一批事实。失败、权限拒绝和存储未配置不当作“无异常”或“无数据”。
- 可以用最多两个前台 `investigation-reader` 并行核对无依赖的问题。读取、计算、发布和文件权限仍按角色执行;reader 只返回查询事实及真实证据编号。
历史 burst_report 任务携带 replay_context 时,Server分发scada-anomaly,时序查询保持 knowledge_cutoff 与 batch_sequence,demo能力提示按冻结窗口返回。
## 地图与图表操作
业务阶段和工单、返回时间、复核回执保存在任务文档 `handling_context` JSON 中,不增加数据库列。外围样板更新工单时可提供带时区的 `occurred_at`,同时必须填写现场反馈;该时间写入活动记录,系统的 `updated_at` 仍是实际记录时间。同步器将现场反馈时间传给调查,避免历史演示误用电脑当前时间。
后台调查统一调用 `publish_task_presentation`,每个不同命令使用不同 UUID `request_id`,相同命令重试复用原编号。以下为工具已有能力,不需要浏览器自动化。
预约时间 `scheduled_at` 可在派单前的工单草稿和已创建工单的编辑表单中调整。实际完工反馈时间与预约安排分别记录;Agent 以实际反馈时间为恢复复核起点。新的有效现场反馈修订完工时间后,旧复核证据保留为历史记录,当前复核起点更新,重新读取严格晚于新反馈时间的最近一个 SCADA 采样时刻。只修改预约安排或其他工单信息,不改写已有现场反馈时间。
| 调查进展 | command.action | 取值及依据 |
| --- | --- | --- |
| 找到实际候选资产 | `highlight_features` | `layer=pipes` / `junctions`,`feature_ids` 为已核实的地图要素编号,`effect=steady` |
| 正在核查候选或建议关阀 | `highlight_features` | 对应 pipes 或 valves,`effect=pulse`;闪烁只表示调查关注,不代表设备正在操作 |
| 查看异常与对照 | `open_chart` | 有限长度的时间和值,引用真实 `evidence_refs`;不同日期的对照对齐方式在标题或来源中说明 |
| 查看已有空间结果 | `set_overlay` | 提供 `overlay_id/title/mode/evidence_id`,Server 从证据提取几何,不接收手绘影响区 |
| 撤下旧的临时展示 | `close_chart` / `clear_overlay` / `clear_highlights` | 清理自己记录的临时展示;清理失败不修改业务结果 |
`POST /operational-tasks/recovery/reconcile` 由现有 operational worker 每轮调用,按项目及任务创建边界处理等待复核的任务;SCADA 读取失败不阻断其他调查派发。复核直接读取项目时序库中完工反馈时间之后最近时刻的 SCADA,不依赖回放状态或推进回放时间。
支持的图层为 junctions、pipes、valves、pumps、tanks、reservoirs、scada。先核实业务 ID 与地图 feature ID 的映射,不能把设备编号、节点编号与 GIS 编号混用。每次高亮最多 100 个要素,需要更多时说明所显示的子集;不要假定多次命令会累积成完整范围。没有几何时可以按已知 ID 高亮、用表格解释结果,不画包络多边形冒充影响范围。
这条恢复链路同样跟踪带 `replay_context` 的调查工单。原异常快照的 `knowledge_cutoff` 和 `batch_sequence` 保留,普通回放查询继续受其限制;工单完工反馈单独触发 Server 的 `ScadaRepository.get_recovery_scada_readings()`,以实际完工业务时间为 T,读取严格晚于 T 的最近一帧。例如 2026-07-16 13:00 完工,15 分钟采样时复核 13:15,而非原异常截止的 08:00。Agent 从 `handling_context.verification_evidence_id` 读取保存的恢复证据,并与原异常证据共同引用,不需放开回放查询或重建复测工单。
图表最多 100 个横轴值和 8 组数值序列,单位须在系列名称或标题中明确。不能把 null 缺测填零以通过 schema;可以按有数据的区间分图,并保留缺测说明。`recorded` 仅表示请求保存。浏览器未开 Dev 跟随时仍继续调查,不能等待地图显示回执。后台接口目前没有摄像机缩放、平移、鼠标点击或任意图层开关命令;不要调用仅供聊天的 `locate_features`、`zoom_to_map` 或原生浏览器工具替代。
已有调查在收到新的有效完工反馈时会补登记成功派发的关联工单。旧回报误用了电脑当前时间时,需在来源工单更正现场反馈业务时间并提交新版本;系统不猜测完工时刻,也不改写原报告或旧证据。
## JSONL 调查数据与 json-render 展示
## 验证
JSONL 用于整理已经取得的数据与分析结果,不补造监测、阀门、现场反馈或审批。推荐每行一个对象,记录 `record_type/origin/source_evidence_refs`,并按对象保留时间窗、单位、模型或假设、覆盖限制和实际结果。`origin` 沿用来源分类;本地派生记录引用输入证据,并说明转换方法。不要把 JSONL 文件自身当作 Server 已保存的业务证据。
检查Skill名称、版本、摘要与正文digest一致、内联体积、角色权限、工单工具隔离和SCADA定位回归。Agent类型检查与会话工具策略测试共同覆盖运行时入口。发布后分别验收工单事件处理报告和SCADA候选定位,原有报告按既有修订保留。
按内容选择 `event_context`、`monitoring_coverage`、`candidate`、`isolation`、`hydraulic_impact`、`field_feedback`、`closure_check`,无需为没有结果的类别制造空壳文件。候选记录写实际排序及评价值;隔离记录保留建议必关阀、可选阀、已知不可关阀及受影响节点;闭环记录逐项保留已核验、待核验或不适用及原因。JSONL 是导出约定,不新增 MCP 输入字段或服务端数据协议。
以下两行只演示文件格式,`synthetic`、`example_only` 与空证据引用表示它们不可用于实际报告或审批,不包含真实分析结果:
```jsonl
{"record_type":"candidate","origin":"synthetic","example_only":true,"source_evidence_refs":[],"element_id":"DEMO-PIPE-01","rank":null,"evaluation_value":null,"status":"not_computed","limitations":["仅说明字段,未执行定位"]}
{"record_type":"closure_check","origin":"synthetic","example_only":true,"source_evidence_refs":[],"item":"repair_verified","status":"not_verified","observed_at":null,"limitations":["无现场反馈,不能声称闭环"]}
```
如果当前会话已获准使用原生文件写入,则在当前 workspace 下写 `files/burst-investigation-<generation>.jsonl`,保留原始证据不变;通过允许的读取工具逐行复核 JSON、行数、对象 ID、时间和单位。只引用宿主绑定的 generation,不能从用户文本自选。不得读取兄弟会话、业务数据库、宿主源码或凭据;需要本地转换时只使用已获准的原生工具及沙箱 Shell,不引入 HTTP、SQL 或自由命令查询桥。
当前没有专用的免审批 JSONL 导出工具。文件权限未授予时,不通过 Shell 绕过、不让后台无限等待批准、不声称已生成文件;继续完成报告内的结构化数据,并如实说明文件未保存。恢复写入授权须由有权限的人或宿主处理,Skill 不扩大权限。
图表、表格和指标使用 `publish_task_report.report.ui_blocks`,每块为 `answer-ui@1`,包含独立 `block_id`、`fallback_text` 和 json-render 的 `spec.root/elements`,外层附真实 `evidence_refs`。最多 8 块、总计 400 KB,单块还受现有 schema 限制。优先组织以下内容:
| 展示 | 建议组件 | 数据来源 |
| --- | --- | --- |
| 候选管段及理由 | Table、Evidence、MapLocation | 地点、资产与定位证据;明确候选和已确认的区别 |
| 异常及正常对照 | Chart | 同单位的实际序列,说明时间对齐及缺测 |
| 关阀与隔离范围 | Table、Metric、MapLocation | 隔离证据、阀门属性及候选场景 |
| 水力与用户影响 | Table、Chart | 指定输出节点的模拟结果及真实用户关联资料 |
| 处置与闭环核对 | Table、Evidence | 动作回执、现场反馈、恢复监测和原工单版本 |
只用工具公布的组件属性。JSONL 可作为整理中间格式,但目前没有“把任意 JSONL 直接传给 json-render”的通道;需转换为已注册组件的 JSON 属性。后台不用 `present_answer`,不把原始 JSONL 输出到聊天正文,也不把表单提交当成已保存现场证据。小数据和简单结论可直接文字表达。
## 本轮成果与爆管闭环的验收
报告至少回答下列四类问题,逐项给出实际结果或经过查询证实的具体限制;无需固定排版或创建固定数量的动作。
1. **位置**:事故已确认还是仅有候选?每个候选的模型/地图编号、匹配或定位依据、排名及未确认事项是什么?
2. **隔离**:每个候选场景是否可隔离?应关、可选和已知不可关的阀门分别是什么?有无替代方案及现场前置核验?
3. **影响**:拓扑受影响节点、水力压降或失供范围、用户关联分别有哪些实际结果?未覆盖什么?没有水力或用户资料时分别说明,不能推定为零。
4. **处置与闭环**:当前可审批的工作是什么?已有哪些动作、审批、提交回执和现场事实?还缺什么反馈才具备结案条件?
有具体结论和动作建议时,先保存 `interim/concluded`,再 `propose_task_action`,最后保存 `final`。人工执行事项使用 `work_order_type=general`,保留原爆管事件。工单描述写位置、任务内容、依赖、执行前核验、需回传的事实和责任要求;`propose_task_action` 当前 payload 没有独立的 dependencies 或 feedback_requirements 字段,不发明字段。没有客户通知适配器时,将通知作为人工工单事项,不宣称已发送。
闭环复核至少核对实际故障资产及原因、隔离/阀门实施记录、抢修结果、适用的清洗及水质复测、恢复供水记录和恢复后监测、用户通知及回访、未决动作与遗留风险、原上报工单结案状态。不适用事项必须有事件依据;未收到结果写待核验,不把记录创建时间当现场完成时间。只齐备处置证据而原工单未结案时,结论为“具备结案条件,待原系统确认”,而非“已闭环”。误报或未发现故障的结案也要有支持排除的实际核查与人工确认。
必要查询失败、权限拒绝和存储未配置属于执行异常,按 schema 修正可纠正问题;仍无法取得当前判断必需的资料时交宿主记录失败。已通过有效查询确认关键数据确实缺失且无法补齐,才发布 `final/insufficient_data`,提供 blocking 缺口和查询依据,建议与待办为空,不在该退出分支再生成补数据工单。能够明确形成一项现场核验结论时可提议具体核验动作,但不能把“尚未开展定位”包装为“已完成定位调查”。
## 能力边界与验证
本次交付为工程图和运行时 Skill 文本,使用现有 MCP、发布、审批与反馈能力;不新增自动关单、鼠标自动化、摄像机操作、JSONL 导出接口或数据库字段。Server 目前没有按 Skill 检查定位/隔离成果的强制验收器,本文的业务验收要求仍由 Agent 执行并由人复核。
当前 Skill 仅通过 `section=skill` 整份分发,没有相对路径 reference 读取协议,因此运行时指令保持自包含,不能要求 Agent 读取本工程文档。内容需控制在既有内联预算内。Server 更新资源后,应按原发布方式让实际 Server 实例取得新文件;Agent 读取时核对 `name/version/content_sha256`,不把本地文件更新当成运行中实例已生效。
验证包含 Mermaid 语法解析、本地路径和工具 ID 核对、Skill 分发/权限/体积测试及 `git diff --check`。后续真实行为验收应分别覆盖:有效观测和漏水量依据下的定位;已知候选但缺漏水量时的候选隔离;不可关阀替代方案;SCADA 首屏未覆盖事件时继续读取;接口失败明确终止;文件权限缺失不绕过;无浏览器时照常调查;实际反馈触发新轮及最终闭环复核。没有真实模型计算或现场回执,不报告定位准确性或业务闭环已通过。
关联契约:[分析 MCP](mcp-analysis-tools.md)、[调查展示](investigation-progress-and-presentation.md)、[报告契约](investigation-report-contract.md)、[任务与多轮反馈](operational-tasks.md)、[后台工作进程](operational-task-workers.md)。
关联契约:[分析 MCP](mcp-analysis-tools.md)、[调查展示](investigation-progress-and-presentation.md)、[报告契约](investigation-report-contract.md)、[任务与多轮反馈](operational-tasks.md)。
@@ -77,6 +77,38 @@ Server 校验当前 run、绑定会话、有效租约和幂等身份,记录 `o
没有坐标的地址、空数据、非法几何、模型自绘区域均不能生成覆盖层。该接口不做地址解析、
范围推算或污染传播计算。展示事件总大小上限为 512 KiB,超限返回 422。
`generate_node_coverage` 是独立的节点空间覆盖工具,适用于普通项目聊天和后台调查,
不绑定爆管、阀门隔离或固定执行顺序。调用者提供需要比较的节点分组,Server 从当前项目
`gis.node_geometries` 读取真实位置。监测设备、投诉、地址等须先关联模型节点,
不接受模型自由生成的坐标。具体适用范围和参数见[节点覆盖工具](node-coverage-tool.md)。
计算先投影到局部米制坐标,取凹包(默认 ratio=0.25)、加展示缓冲(默认 25 m),
再以 2 m 容差简化并转回 WGS84;凹包比例与缓冲距离可在有界参数内选择。
默认展示各组的地理包络,不判断连接关系;需要时可按模型连接拆分,额外指定视为断开的连接。
少量点、重合点和共线点同样通过缓冲形成面。多组结果一起保存在 `result.map_payload`,
避免连续 `set_overlay` 覆盖前一组;`result.coverage` 保存来源节点、映射完整性及算法参数。
该面不推断故障、污染、实际停水、水力影响或用户服务边界,缓冲距离也不是供水服务半径。
缺失几何时标记 `partial`,没有可用位置时不生成面。输入最多 8 组、合计 10000 个不同节点;
几何输出遵守覆盖层顶点和大小限制,超限要求缩小范围,不静默抽样或改变原分析结果。
MCP 保存完整查询响应作为不可变证据,展示解析器同时支持响应信封中的
`data.result.map_payload`。Agent 使用 `set_overlay` 引用本次 `evidence_id` 并省略 `data`。
超过内联返回预算时,响应的 `data.presentation.command` 仍保留可发布的展示命令,
`data.presentation.groups` 保留覆盖完整性计数,
完整几何留在证据中,不能把缩略返回误认为没有空间成果。调查按实际步骤同步发布展示:
地址得到明确坐标后缩放定位,取得候选资产后定位候选,检查影响节点后生成并展示覆盖示意,
形成或调整关阀方案后定位阀门,读取监测对照后展示点位及序列。
图表同样随结果返回发布:监测趋势与事件对照用折线图,可比候选评价或方案用柱状图,
水力及污染传播等模拟按实际返回的时刻、对象和指标画图。压力、流量、浓度等不同单位分图,
注明来源、范围与实测或模型属性;没有同口径基准时不计算或绘制改善量,缺测不补零。
过程图表通过 `open_chart` 提供 `chart_id/chart/evidence_refs`,每图最多100个横轴值、8组序列,
超限时分图或明确说明抽样、聚合范围;单值或无可比数据用文字或表格。
关键趋势与比较结论保留到报告中,最终报告的图表不替代过程发布。
业务 Skill 说明本步骤应展示什么,调查指令统一说明发布方法、非空几何检查、去重和失败处理。
同一阶段批量展示,数据或方案没有变化时不重复请求;不为地图动作增加无关业务分析。
报告的 `map_targets` 仅为手动定位入口。关键步骤的数据缺失或展示失败需记录具体限制,
没有浏览器连接也不阻止分析或报告收尾。
`GET /api/v1/operational-tasks/{task_id}/presentation-cursor` 在同一事务中读取
`task_id/execution_generation/active_run/event_cursor`,其中活跃 run 仅返回 `run_id/generation/status`。
浏览器打开 Dev 联动、切换任务或断线重连时,以此建立新基线,只消费之后的展示事件。
+11 -3
View File
@@ -14,9 +14,15 @@ Agent 根据401 challenge 或路径/根 well-known 发现受保护资源元数
调查数据返回前,在短事务内追加现有任务证据并保存调用回执。同一主体、会话、任务、run 和原生 call ID 的重试复用证据;不同参数或失效范围明确报错。七类原有访问申请 scope 保持原契约,MCP 的新增来源只扩充快照来源枚举。没有新增报告发布、审批、控制、Shell 或递归委派权限。
普通查询默认20条;目录类最多500条,时序最多10000条、20个标识、24小时;offset 最大10000。Agent 首次优先单对象、必要的短时间窗和小页,仅在证据不足时按需扩大,不自动遍历全网。SQL 有 limit/offset 和 statement_timeout。历史对比在同一条SQL中按序列、时间对数据点分页,返回 page.has_more;非零offset也标明覆盖不完整。历史曲线展示调用初始每页500点,`RESULT_TOO_LARGE` 时原位减半重试,最多读取1000条、发出16次请求,共用120秒超时(单次请求同时受该剩余总预算约束),图表最多100个有效点;这不改变模型首次查询默认值或时序接口上限。
普通查询默认20条;目录类最多500条,显式指定设备的时序查询最多10000条、20个标识、24小时;offset 最大10000。SCADA调查首次省略 `query_scada_readings.device_ids`,由Server读取项目完整设备目录及窗口内全部授权观测,内部自动翻页,`offset` 必须为0。其余查询按问题选定对象和窗口。SQL 有 limit/offset 和 statement_timeout。历史对比在同一条SQL中按序列、时间对数据点分页,返回 page.has_more;非零offset也标明覆盖不完整。历史曲线展示调用初始每页500点,`RESULT_TOO_LARGE` 时原位减半重试,最多读取1000条、发出16次请求,共用120秒超时(单次请求同时受该剩余总预算约束),图表最多100个有效点;这不改变模型首次查询默认值或时序接口上限。
单份结果超过8MiB报错;超过12KB的调查工具消息只返回证据资源引用,完整快照仍保存,Server 的 `resources/read` 可以读取它。Agent 本地 MCP 只提供工具,将上游资源引用转为 `evidence_read` 中的 `mcp_tjwater_get_task_evidence` 参数,不向 OpenCode 泄露不可在本地读取的链接。`get_task_evidence` 在数据库中按 data_path、limit、offset 选页,嵌套集合和超过256字符的字符串返回路径描述;字符串按 Unicode 字符分页。页和完整资源均在数据库返回前检查8MiB上限,保留原证据编号和合成来源;普通查询要求缩小范围。其他结果仍保留执行后的整体8MiB检查,不能把它理解为所有查询均有数据库端字节限制。原始/清洗实测、模型基线、分析结果、比较和 synthetic 均明确标记。非有限数值转为 null 并记录数据缺口。
单份结果超过8MiB报错;超过12KB的调查工具消息默认返回证据资源引用,设备目录的精简路径见下段。完整快照仍保存,Server 的 `resources/read` 可以读取它。Agent 本地 MCP 只提供工具,将上游资源引用转为 `evidence_read` 中的 `mcp_tjwater_get_task_evidence` 参数,不向 OpenCode 泄露不可在本地读取的链接。`get_task_evidence` 在数据库中按 data_path、limit、offset 选页,嵌套集合和超过256字符的字符串返回路径描述;字符串按 Unicode 字符分页。页和完整资源均在数据库返回前检查8MiB上限,保留原证据编号和合成来源;普通查询要求缩小范围。其他结果仍保留执行后的整体8MiB检查,不能把它理解为所有查询均有数据库端字节限制。原始/清洗实测、模型基线、分析结果、比较和 synthetic 均明确标记。非有限数值转为 null 并记录数据缺口。
`list_scada_devices` 默认每页100个、最多500个,其他集合仍沿用各自的分页默认值。设备目录超过12,000字节时,先内联本页全部设备的 `device_id`、`device_type` 和 `measurement_unit`,省略其他属性并明确标注;模型映射、坐标等通过 `get_scada_device` 核验,任务证据和幂等回执保留完整属性。此时 `truncated` 表示属性省略,设备是否还有后续页以 `page.has_more` 为准,按本页实际设备数递增 `offset`。精简目录仍超预算时沿用证据引用或缩页路径。
全量查询同时要求设备目录与读数权限;通过任务授权读取时,捕获范围及当前批准状态都必须同时包含 `scada_devices` 和 `scada_readings`,显式设备读数查询仍沿用读数授权。
全量 `query_scada_readings` 返回 `data.devices` 的完整属性、`data.readings` 的原始和清洗值,以及 `data.coverage` 的查询设备、读数数量和缺测清单;该模式保留完整内联数据,不经过12KB缩略,仍执行8MiB整份结果上限。回放仅读取指定批次之前的冻结观测,项目中未纳入该回放的设备单列 `outside_frozen_scope_device_ids`。`query_complete=true` 表示授权范围内的查询已读完,不表示设备均有有效数据。SCADA及历史回放任务发布final时,必须引用窗口覆盖本轮时刻、且回放批次匹配的全量证据;单设备、局部集合、未引用或旧批次证据不能通过此校验。既有报告不回写,后续重跑适用新约束。
`get_task_context` 默认 section=summary,只查固定元数据、160字符任务标题及200字符结论预览,并返回 available_sections;preview_only 和 truncated 明确说明不是完整报告。按 section=task/report/evidence/actions/progress/access_requests/plan/run 读取详情,report_revision 保留选择旧报告的语义。data_path 选择嵌套值,limit 默认20、最多500,offset 最大10000;证据、动作和授权申请集合以稳定 ID 为键,按 ID 字典序分页。字符串按字符分页。详情在数据库返回前检查8KB预算,超限缩页或选更深路径。缺少可选进度或授权申请表时返回 metadata.available=false,不冒充空事实。绑定权限范围只读取七类既有 scope,不再加载整个任务上下文。不增加工具数量、数据库表或迁移,也不改变原 REST 完整上下文契约。
@@ -26,7 +32,9 @@ Agent 根据401 challenge 或路径/根 well-known 发现受保护资源元数
### Server 分发业务处置 Skill
业务流程源文件位于 `resources/work-order-skills/*/SKILL.md`,随 Server 镜像分发。`get_task_context` 摘要提供 `handling_skill`(适用 workflow、名称、版本和 SHA-256)及 `display_time_zone=Asia/Shanghai`;使用 `section="skill"` 读取完整正文。流程由服务端持久化任务的 workflow 选择,调用方不能传入文件路径或其他任务类型;未知 workflow 返回 `available=false`,不自动套用其他流程。
业务流程源文件位于 `resources/work-order-skills/*/SKILL.md`,随 Server 镜像分发。`get_task_context` 摘要提供 `handling_skill`(适用 workflow、名称、版本和 SHA-256)及 `display_time_zone=Asia/Shanghai`;使用 `section="skill"` 读取完整正文。流程由持久化任务的 workflow 与 replay_context 选择:工单 burst_report 使用 burst-event-handling,scada_anomaly 及带回放来源的历史 burst_report 使用 scada-anomaly;未知 workflow 返回 available=false。工单事件处理的绑定工具范围移除 run_burst_location,保留资产关联、隔离和影响分析。
报告阅读时区统一为 UTC+8(Asia/Shanghai)。MCP 在原始证据和回执保存后,将模型可见结果中的显式带时区时间转换为 `+08:00`,保留同一时刻和源小数秒精度;存储、查询参数及冻结窗口沿用原始时刻,标识符与引用保持原文。前端对报告正文、交互表格、证据卡片、图表提示及数据表采用同一阅读时区,并兼容历史正文中的 `Z`、`UTC` 和 `UTC0` 标记。历史正文中省略时区的时间按读者默认 UTC+8 展示。
当前覆盖爆管上报、水质投诉与预约跟进。业务 Skill 描述风险核查、隔离停水、用户通知、现场处置和恢复条件;Agent 本地 Skill 维护通用执行方法。权限、审批和执行仍由 Server 代码控制。流程说明没有 evidence_id,不记录成事件证据。具体草稿由现有 `propose_task_action` 保存并等待人工审批;人工通知等步骤通过关联预约工单承接,不能当作通知已发送或现场工作已完成。
+43
View File
@@ -0,0 +1,43 @@
# 节点空间覆盖工具
`generate_node_coverage` 增加 Agent 可选择的空间表达能力,将已取得的模型节点集合转为
可解释、可复核的覆盖示意。它可以在项目聊天、任务协作和后台调查中使用,
不要求爆管任务、隔离成功或特定报告。生成、发布展示和业务分析是三个独立操作。
## 适用范围
| 问题 | 输入依据 | 输出含义 |
| --- | --- | --- |
| 候选对象分布在哪里 | 已核实的候选模型节点 | 候选节点空间包络 |
| 两组监测点覆盖有何不同 | 已核对的设备到模型节点映射 | 已接入节点分布比较,不证明监测可观测性 |
| 区域成员分布在哪里 | 已查询的区域成员节点 | 成员节点包络,不替代已有行政区或 DMA 边界 |
| 哪些节点属于模型影响集合 | 已有拓扑或模拟结果中的节点集合 | 结果节点的空间示意,不重新计算影响 |
| 投诉关联的节点如何分布 | 已解析并核对的投诉到模型节点关联 | 已关联节点分布,不证明投诉或污染真实边界 |
只有地址或未关联设备时,先查询位置和资产关联;已有权威区域几何时优先展示原始边界。
本工具不接收任意坐标、HTML、SQL、脚本或调用者指定的项目身份,不执行关阀、模拟或方案保存。
## 参数和结果
- `groups`:最多 8 组,每组提供唯一 `group_id`、`title` 和 `node_ids`;组之间可重叠,
合计最多 10000 个不同节点。不按组名自动赋予隔离、污染或服务覆盖语义。
- `concave_ratio`:0–1,默认 0.25;越接近 1 越接近凸包。分散的点取单一包络可能包含较大空白,
应按实际比较目标分组,并明确示意含义。
- `buffer_m`:大于 0、最多 1000 m,默认 25 m,仅是展示缓冲。
- `split_by_connectivity`:默认 false。true 时按所选节点的模型连接拆分,并排除初始 CLOSED 连接。
`excluded_link_ids` 可额外指定最多 200 个断开连接,适用于已有关阀建议等有依据的绘图情景。
这不会修改模型,也不能替代完整拓扑分析。
`result.map_payload` 是多组共存的 GeoJSON;`result.coverage` 记录算法参数、节点 ID、
已映射数量、缺失几何节点、分组数量和 `display_only` 边界。缺少部分位置时返回 `partial`,
不补造坐标。单点、重合点或共线点通过米制缓冲生成可绘制面。
## 与展示工具配合
生成工具不会移动地图。项目聊天可用 `present_view` 展示本次返回的几何;后台调查用
`publish_task_presentation` 的 `set_overlay`,引用生成结果自己的 `evidence_id` 并省略 `data`,
Server 从不可变证据解析完整几何。大结果内联缩略时,`data.presentation.command` 保留可发布命令。
`recorded` 仍只表示请求已记录,浏览器显示和后台调查完成各自独立。
本次爆管报告只是一个验证样例:可将 24 个直接隔离节点、295 个供水影响节点作为两组,
按已有三阀建议设置连通分组;不将这种调用固定进 `analyze_valve_isolation`。
+26
View File
@@ -190,3 +190,29 @@ OPERATIONAL_TASKS_STORAGE_TEST=1 conda run -n server python -m pytest tests/inte
```
第二条需要当前配置的 PostgreSQL 可连接及测试 DDL 权限。测试在强制回滚事务中创建表结构和隔离项目数据,结束后不保留这些修改;不代替正式迁移。外部工单调用仍使用隔离 HTTP transport,不会操作实际工单系统。该测试覆盖持久化报告、反馈重新排队、旧会话拒绝、人工确认与丢响应回执核对,而非真实模型推理质量。
### 共用数据库时的执行范围
任务创建时,Server 将 `OPERATIONAL_TASK_EXECUTION_SCOPE`(留空则使用
`ENVIRONMENT`)写入 `workbench.operational_tasks.document.execution_scope`。
本地配置 `development`,线上配置 `production`;浏览器不提交此字段,
重试保留原范围。`execution_scope` 为必填字段,运行时代码不为缺失值兜底。
现有任务通过 `resources/sql/034_operational_task_execution_scope.sql` 一次性补为
`production`;已有明确范围的任务保持原值。此回填是历史任务的迁移策略,
不代表通过旧记录推断出了创建环境。
迁移需要与新版本切换协调执行:暂停任务写入,执行每个业务库的 SQL,
启动新版本并恢复任务写入。旧版本的严格模型不接受新增 JSON 字段,
新版本也拒绝未迁移数据;不能在旧版本仍服务任务请求时单独回填。
SQL 同时增加必填范围约束,不设置数据库默认值。回滚版本前需同步处理该约束
与 JSON 字段,不得直接切回旧镜像。
Worker 应连接所属环境的 `OPERATIONAL_WORKER_SERVER_URL` 和
`OPERATIONAL_WORKER_AGENT_URL`。Server 在运行列表、领取、租约回收与
SCADA recovery 候选查询上限制执行范围;直接领取其他范围返回
`403 EXECUTION_SCOPE_MISMATCH`。外部工单反馈与来源撤回也必须匹配任务的执行范围,
并在幂等重放前校验;开发环境同步器不能更新生产任务或重置其恢复监测起点。
这属于配置隔离,不能替代服务凭据授权:
不要让本地 Worker 通过线上 Server 领取任务。普通任务与历史记录仍共用存储。
修改范围配置需要重启相应 Server。暂停的本地 Worker 不会因代码更新恢复。
+3 -1
View File
@@ -39,7 +39,9 @@ Forward Hostname 后追加 `/api/sample/v1`。本机开发和预览模式也将
- `/api/sample/v1/monitoring/points`、`observations`、`alerts`
- `/api/sample/v1/demo-data`
演示数据通过 `scripts/seed_peripheral_sample.py` 写入 `tjwater_v2` 的 `peripheral_sample` schema。当前固定窗口为 2026-09-17,所有记录标记为 `synthetic`,并使用 `DEMO-` 前缀。
SCADA 按月数据集、5 分钟时间定位、主动周期监测、质量校验、爆管调查与定位边界见 [SCADA 外围系统时段回放、主动监测与爆管调查设计](scada-peripheral-replay-design.md)。部署回放存储需在项目 BizDB 执行 `031_scada_replays.sql`;项目通用配置需在 MetaDB 执行 `033_metadata_project_settings.sql`。监测频率保存于 `projects.settings.scada_monitoring_interval_seconds`,压力与出厂边界流量角色保存在类型化 `projects.settings.scada_monitoring` 中;本阶段不注册定时任务,也不部署 SCADA 回放 worker。代码和结构迁移完成不等于月度数据或 Agent 联调已经验收。
独立监测样本和测试数据通过 `scripts/seed_peripheral_sample.py` 写入 `tjwater_v2` 的 `peripheral_sample` schema。固定窗口为 2026-09-17,样本标记为 `synthetic` 并使用 `DEMO-` 前缀;业务工单保持 `system_data` 来源。
## 工单事件调查
+598
View File
@@ -0,0 +1,598 @@
# SCADA 外围系统时段回放、主动监测与爆管调查设计
状态:时段回放、主动周期监测、条件式 `burst_report` 调查及回放安全的爆管定位适配已实现。单步处理由用户操作触发并在当前 API 请求内完成,不部署 SCADA 回放 worker。`tjwater_v2` 与 `tjwater2_v2` 尚未确认出厂边界流量角色;仍待完整月度数据和真实 Agent 联调。更新日期:2026-11-03。下面明确区分已实现范围、后续目标与验收要求。
本次将外围 SCADA 定义为一个模拟外部数据源:本地 TSDB 已保存原始观测,通过只读 API 按选定时间区间和后台时钟分批读取 `scada.measurements.monitored_value`,将这一批直接交给清洗流水线,保存 `cleaned_value` 后执行异常检测及后续 Agent 调查。原始值、修正值、真实工况异常和处理结论分别保留。浏览器负责控制和展示,关闭页面不停止后台回放。
演示主路径为“TSDB 已存 raw → 用户点击下一步并监测 → 顺序读取一个步长内的观测 → 清洗并更新修正值 → 在步长边界执行多指标监测 → 满足条件时进入爆管调查与定位”。真实外部系统新增观测时,仍先通过入库接口保存 raw,再进入相同的清洗检测流程。演示读取已有数据不再次调用原始值插入接口,也不把同一批数据从一个内部服务读出后再写回原表。
演示操作以用户选定的时间区间为中心:一个月是可选择的数据范围,单次演示可以只覆盖其中几小时或几天。时间轴采用 FM 调频盘样式,玻璃刻度窗内有固定红色指示线,时间刻度带随拨动横向移动。时间轴使用 5 分钟时间戳定位;项目监测频率只决定“下一步并监测”覆盖的业务周期,两者互不替代。
## 首版实现范围与限制
已实现只读 raw 分页、持久回放控制、按项目监测频率推进、逐时刻清洗检测、质量告警与运行异常事件、周期监测回执、组合判定及调查任务。主动单步只有组合结果为 `suspected_burst` 时才创建 `burst_report`。前端改为后端状态投影,主界面只保留选定区间、5 分钟 FM 时间定位、主动周期控制和本周期监测结论,不重复展示 SCADA 设备目录与逐测点读数表。主要代码为 [回放服务](../app/services/scada_replay.py)、[组合监测](../app/services/scada_replay_monitoring.py)、[质量与检测](../app/services/scada_replay_processing.py)、[持久状态](../app/infra/db/postgresql/scada_replays.py) 和 [前端组件](../../next-tjwater-frontend/src/features/workbench/peripheral-sample/scada-replay-section.tsx)。
本版采用每项目最多 500 个设备、一个完整时间戳一批、至多一个未完成批次的串行处理,不实现 20 批并发队列或大于 500 个设备的分片接收。处理变慢时逐批追赶且不跳过中间观测;后台停机期间最多补算两秒现实时间,不一次性追赶全部停机时长。数据质量以有限数值和显式仪表量程为依据,不自动插值或平滑;缺测判断使用设备档案的采样周期。分组规则的恢复需要该组设备在同一帧都有可信观测,不同采样周期或采样相位应拆成规则。
本版冻结设备、区间、规则及批次输出,按批次原始值摘要校验源变更。完整月度数据清单、全月源版本摘要、独立事件真值和检测效果评估仍是后续数据准备工作;能力接口仅判断回放存储是否存在与当前操作权限,不声明月度数据已经就绪。现有 14 天数据不能当作一个月验收。
Agent 查询本轮已检测的固定批次快照,受任务的 replay_id、knowledge_cutoff、batch_sequence 和设备集合约束。新回放首次监测以只读一致快照冻结开始前 15 天历史,负序号静默基线按最多 2,000 行分块暂存为 `clean_prepared`;全部读取完成后,在一个 BizDB 短事务中把批次转为 `detected` 并原子发布基线与初始化标记,历史查询和定位对照才能读取。
关联现场工单完工后的恢复复核采用独立证据链:Server 以现场反馈业务时间 T 读取项目时序库中严格晚于 T 的最近一帧及事件前对照,保存恢复监测证据后接续原调查。此路径不依赖回放推进,也不修改原异常快照的知识上限或批次;Agent 读取保存的恢复证据,不通过普通回放查询越界取数。详见[恢复复核流程](burst-response-workflow.md)。
读取各页期间不持有 BizDB 事务,内存只保留当前页和设备短历史。固定准备预算为 1,000,000 行、512 MiB 序列化批次,覆盖 500 设备十五天每 15 分钟一条所需的 720,000 行并留有余量。超限不保存超限页,状态明确返回 `BASELINE_BUDGET_EXCEEDED` 及中文原因;未发布批次不进入观测、监测或待处理批次。失败重试只清理当前项目、本轮回放的未发布基线,从新的一致源快照重新准备,已发布基线保持不变。
旧回放没有冻结基线时明确拒绝回放前查询,不回落到实时 TSDB。爆管定位支持受回放截止约束的冻结输入适配,其他未适配计算工具继续拒绝。[SCADA 调查 Skill](../resources/work-order-skills/scada-anomaly/SKILL.md) 要求据实说明证据限制。运行异常自动建立调查,数据质量异常仅建立告警;恢复时推进调查代次。真实模型推理与报告效果尚未联调。
控制进度和事件状态使用有界轮询,沿用告警流发布业务告警;本版未新增 scada.replay.changed SSE 主题。演示面板只显示最近一个周期的组合结论,逐测点输出和完整事件历史由分页 API 保留给诊断与调查入口,不在主面板重复铺开。
## Case 3 临时定位 mock(2026-11-07)
`tjwater2_v2` 及其隔离演示项目 `tjw_bench_tjwater2_20261105` 中,2026-07-16
08:00–13:00(北京时间)的 monitoring 定位暂用快速 mock。异常窗起点可提前 15 分钟,
终点必须已经到达事件时刻且不超过事件结束;其他项目、时间和 simulation 输入仍运行原算法。
mock 直接返回并沿用 MCP 权限、回放冻结证据校验、调查证据与幂等回执,不提交慢计算队列。
预设候选为同一 DN600 线路的 `515870`。每次读取当前拓扑,验证候选与事件位于完全相同的
可隔离分区、所需边界阀一致,且不跨阀的同径管线路径距离为 200–500 m。距离定义为实际
事件节点沿管线至候选管段最近点的距离;当前为 495.368 m,不以两条管段中点计算。
无法满足任一条件时返回空候选,不扩大隔离区,也不回落到耗时定位。
候选结果固定返回 `similarity=0.9708` 和 `similarity_percent=97.08`,展示为97.08%。
同一案例在 T 与 T+15 分钟返回相同候选与排序。Agent可见结果不再附加 `mock`、
`demo_mock` 说明、空相似度或 `simulation_times=0` 等阻断提示,也不虚构水力仿真次数。
MCP 回放证据仍保留 `origin=synthetic`;固定相似度是该演示路径的预设返回值,未运行水力定位。
取得候选后继续地图展示、隔离边界与影响分析。
事件真值仅在服务端 fixture 中用于约束校验,
不返回给 Agent。撤回时移除 `app/services/burst_location_demo.py` 的两项目匹配即可恢复原路径;
不修改观测、管网或关阀数据。本项不代表自动监测建单或真实 Agent 报告已完成验收。
## 已确定的设计边界
- 数据集覆盖一个月,包含多次独立异常及各次事件发生、持续、恢复的过程。
- 项目配置一个 SCADA 监测频率,只用于一次主动监测所覆盖的周期;时间轴独立使用 5 分钟时间戳,它不关联设备或设备类型,本阶段不据此创建定时任务。
- 设备采样周期仍属于设备档案。回放实际传入时间以原始观测时间戳为准,不使用项目监测任务间隔生成、重采样或判断设备缺测。
- 复用 `scada.measurements.monitored_value` 与 `cleaned_value`;不另建一套正式 SCADA 原始值和修正值表。
- 回放读取 API 只返回原始观测,修正值由本地清洗流水线计算和保存;GET 不更新游标、不清洗、不创建检测任务。
- 真实压力、流量、水质异常保留在历史序列中。事件恢复或调查结束不删除、不抹平历史异常。
- 数据处理顺序为原始观测持久化、数据质量校验、必要的修正、业务异常检测、事件与调查。
- 步长内的数据批次按原始时间驱动清洗和基础规则检测;到达步长边界后,用户触发的本周期监测汇总多项指标。正常周期不启动 Agent。
- 时间范围、设备、数据集版本和检测规则在一次回放创建后固定;改变这些条件需要新建回放。
- 页面首先选择演示起止时间,再建立回放。月度数据集、设备配置和规则作为来源与设置,不占据主播放区。
- 本次演示对已存 raw 只读,清洗阶段仅更新同一条观测的 cleaned_value;不导出整月副本、不重复插入原始值。每次回放的质量结论、检测结果和调查证据按回放身份隔离。
### 项目通用配置与监测频率
MetaDB 的 `projects.settings` 为项目级通用 JSONB 对象,默认 `{}`,用于保存可扩展的业务参数,不存放密钥。新增参数可直接增加 JSON 键,无需为每项配置新增数据库列;后端为已知参数提供类型校验,并保留其他 JSON 键。
其中 `settings.scada_monitoring_interval_seconds` 表示项目的 SCADA 监测频率,单位为秒。本阶段只用于主动处理边界和监测周期标识,不控制时间轴刻度,不注册 cron、后台 schedule 或周期性 Agent 任务。它与设备目录、设备类型、设备采样周期和页面刷新周期均无关联。省略或设为 `null` 表示未配置;有效范围为 60 秒至 365 天。
管理员可通过 `PATCH /api/v1/admin/projects/{project_id}` 配置:
```json
{
"settings": {
"scada_monitoring_interval_seconds": 900
}
}
```
更新时按 `settings` 顶层键合并:未传入的键保留,传入的嵌套对象整体替换,`settings: {}` 不清空已有配置,`settings: null` 会被拒绝。读取项目详情和管理员项目列表均返回 `settings`。
组合监测仍复用该 JSON 对象,不新增固定数据库列。后端类型化嵌套对象显式声明项目实际具备的监测角色:
```json
{
"settings": {
"scada_monitoring_interval_seconds": 900,
"scada_monitoring": {
"rule_version": "burst-monitor-v2",
"pressure_device_ids": ["明确的压力设备 ID"],
"outlet_flow_device_ids": ["明确的出厂边界流量设备 ID"]
}
}
}
```
当前代码已类型校验并在创建回放时冻结 `scada_monitoring`。不足 5 个完整压力点时保持缺项,能力接口说明原因,监测回执返回 `insufficient_data`。加权模式允许出厂流量缺项,其贡献为零且不重分配权重;`all_signals` 模式仍要求出厂流量就绪。不能把普通流量点冒充出厂流量或虚构设备。`tjwater_v2` 未补写未经核实的监测角色。
部署前需在 MetaDB 执行 `resources/sql/033_metadata_project_settings.sql`。迁移会将实验版本可能存在的独立监测间隔列搬入 `settings`,核对冲突后移除旧列;重复执行不丢失已有配置。当前 `tjwater_v2` 与 `tjwater2_v2` 均配置 900 秒,即 15 分钟的主动监测周期。该参数不改变 5 分钟时间轴定位,本阶段明确不实现通用定时任务调度。
### 动态阈值与加权判定(2026-11-07)
新建回放默认使用 `burst-monitor-v2`,配置仍位于项目 `settings.scada_monitoring`:
```json
{
"rule_version": "burst-monitor-v2",
"decision_policy": "weighted",
"pressure_device_ids": ["已核实的至少5个压力设备"],
"outlet_flow_device_ids": ["已核实的出厂流量设备"],
"weights": {"burst_detection": 0.5, "pressure": 0.4, "outlet_flow": 0.1},
"alert_threshold": 0.8
}
```
三项权重均为 0–1 且总和必须为 1。每项达到自身阈值时贡献其固定权重,加权和达到
`alert_threshold` 才判为疑似爆管。默认侦测与压力共同命中可得 0.90,出厂流量仅提供
0.10 辅助支持。缺数或未配置的流量贡献为零,不把剩余权重重新归一化;压力或侦测
数据不足时仍返回数据不足。综合分不是概率,也不代表现场确诊。
首步只用回放开始前完整 15 天的冻结历史学习阈值,在历史发布事务中一起保存
`monitoring_calibration`;随后固定使用,事件发生后的数据不会参与更新阈值。准备时
为角色测点与参与相对变化规则的测点保留紧凑数值序列,最多 500 个测点;
15 分钟周期下数值数组上限约 5.8 MB。继续逐页保存基线,不重新聚合全部设备的 raw。
15 天历史不完整就声明不足,不插值或借用未来数据。
压力与流量按设备计算此前若干样本中位值对应的历史变化率,阈值取正常变化中位值加
3 倍稳健波动尺度,并分别保留 3%、5% 下限。侦测阈值从冻结正常历史的得分估计,取
`min(-0.04, 正常得分中位值-3×稳健波动尺度, 正常得分5%分位值-0.02)`。
稳健波动尺度在本实现中定义为 `1.4826×MAD`。保留真实侦测分数,调整判定阈值;
不按已知事件日期制造异常分数。周期回执保存实际阈值、权重、各项贡献和校准截止时间。
加权流程的测点诊断采用 `normal-phase-envelope@1`:按前 15 天同一日采样相位的正常值
建立 `[历史最小值−3×稳健波动尺度, 历史最大值+3×稳健波动尺度]` 包络,保留数值精度余量。
普通测点越过所配置方向的同期包络即可启动调查,不再用固定 30% 门槛筛选。
零基准、日常反向流量、近零分母的百分比变化不再单独制造异常。压力/出厂流量的
爆管参考贡献还需越过其历史动态变化阈值;只命中同期包络的有效单点异常仍可进入调查。
显式水质上下限保留。角色参考的滚动基线严格取目标观测之前的样本,与校准窗口对齐;
本周期必须有新观测,模型必须具备周期终点读数,不能复用上一时刻的得分。
本轮校准版本为 `scada-monitor-calibration@2`,并保存 `diagnostic_policy`、各测点包络与
截止时间。缺少完整历史的测点不回退到固定百分比,并在监测回执的缺项中列出。
已预热的加权回放缺少新版校准时,推进/重试返回 409,明确要求停止后新建;
旧相对规则事件不进入新诊断信号,已有历史事件和工单不删除。新事件采用独立规则状态和策略标识。
请求 schema 中的 `relative_change.threshold` 仅为既有协议兼容保留,在加权测点诊断路径
不参与触发;同一规则的窗口参数仍用于角色指标的动态变化计算。
既有冻结回放没有 `decision_policy` 时沿用三项同时命中的旧判定;新加权配置需新建
回放才生效,不更改旧回放的历史结论。显式 `decision_policy=all_signals` 仍为旧规则兼容路径,
不用于当前 Case3 加权演示,也不是新版校准缺失时的回退路径。
本项修改无需数据库迁移。
## 实施前基础与原设计差距
下表记录实施前基础,用于说明改动缘由;本次实际实现以首版范围及后文接口表为准。设备目录采样周期为本轮讨论中 2026-11-02 的只读核查快照,与项目监测任务间隔是两个概念。
| 位置 | 当前能力 | 本次需要补齐 |
| --- | --- | --- |
| [外围 API](../samples/peripheral/api.py)、[存储路由](../samples/peripheral/routing.py) | 挂载在 `/peripheral`,使用 Bearer 与 `X-Project-Id`,按已授权项目解析 BizDB 和 TimescaleDB | 回放控制、源批次、回放状态与能力声明 |
| [历史观测接口](../samples/peripheral/store.py) | 单点、最多 7 天、最多返回 1000 条并声明截断;当前使用 `api_query_id` 查历史 | 回放读取使用规范 `device_id`,按时间和设备联合游标分页;不能直接拼接现有单点接口来装载一个月 |
| [SCADA API](../app/api/v1/endpoints/timeseries/scada.py)、[仓储](../app/infra/db/timescaledb/repositories/scada.py) | 范围读取、最新值读取、批量插入;当前范围读取返回 raw 和 corrected,缺少回放游标及完整流水线 | 增加只读 raw 分页契约;复用存储查询,补齐处理检查点、清洗与检测调度 |
| [清洗服务](../app/services/timeseries_analysis.py) | 从原始值计算并更新 `cleaned_value`;目前支持压力和流量 | 在线质量标记、因果修正及选择值依据;水质不能直接套用现有清洗 |
| [压力清洗](../app/algorithms/scada_cleaning/pressure_series.py)、[流量清洗](../app/algorithms/scada_cleaning/flow_series.py) | 包含居中窗口、后向补值、Kalman 平滑;流量路径还会补为 1 分钟序列 | 在线路径只使用已到达的数据,保留设备频率,不整月预清洗后再模拟检测 |
| [外围工单事件同步](peripheral-work-order-events.md) | 持久事件、通知唤醒、后台同步和定期对账 | 复用可靠处理方式,SCADA 的批次有自己的持久状态 |
| [调查 worker](../scripts/run_operational_worker.py) | 可靠投递 `event_investigation`,恢复执行租约,管理 Agent 执行代次 | 接收由 SCADA 异常事件建立的调查;不承担每条观测的检测 |
| [任务告警](../app/domain/schemas/task_alert.py) | 支持 raised、acknowledged、resolved;当前来源必须携带任务和执行标识 | 增加真实的回放检测来源,不能为调查前告警伪造 `task_id`、`run_id` |
| [MCP 上下文](../app/mcp/contexts.py)、[SCADA 查询](../app/mcp/query_service.py) | 校验任务、项目、执行代次和会话,保存查询证据 | 在所有回放调查读取入口执行统一的时间与数据版本约束 |
| [业务 Skill 映射](../app/mcp/work_order_skills.py) | 目前映射爆管上报、水质投诉、预约跟进 | 增加 SCADA 回放调查流程,不将检测信号直接等同于已确认爆管或污染 |
| [前端 SCADA 区域](../../next-tjwater-frontend/src/features/workbench/peripheral-sample/scada-system-section.tsx) | 旧版同时展示设备表、逐点读数、重复概览和手工合成读数 | 改为服务端状态投影;主界面只承担时间定位、主动单步和周期结论,详细数据留在诊断与调查接口 |
实施前只读核查中,`tjwater_v2` 有压力、流量和水质监测类型,其他项目的类型集合与历史周期并不相同。项目通用配置保存一个监测任务周期,不与设备类型绑定,也不把 5/15 分钟写成枚举;设备原有采样信息仍用于缺测判断。
现有 [演示记录](../../present.md) 描述的是 14 天合成观测,不能作为一个月数据已经准备好的证明。一个月数据准备与本设计实现分别验收。
## 运行链路与职责
```text
本地 TSDB 已存月度原始观测;项目 BizDB 保存设备目录
|
v
只读 raw API / 同一读取服务 <--- 持久回放状态 <--- 页面“下一步并监测”
|
| 返回一批原始观测,不再写入 raw
v
后台固定本批记录引用、摘要和处理身份
|
v
后台质量校验与清洗
|
v
本地 TSDB:仅更新 cleaned_value
|
v
用同批清洗结果执行基础规则检测
|
v
到达步长边界,汇总爆管侦测、压力下降、出厂流量变化
|
v
疑似爆管且定位输入具备 ---> burst_report 任务与 MCP Skill
|
v
工作台告警、调查进度与结果
```
SCADA 单步命令在当前 API 请求内按原始时间顺序完成 raw 读取、清洗、基础规则检测和周期组合监测。每个阶段仍写入持久批次,网络中断后的同一幂等请求可以继续未完成步骤,但页面关闭后不会自动推进时间,也不需要单独的 SCADA 回放进程。
现有调查 worker 只处理已经创建的调查执行,计算 worker 继续处理长计算。当前阶段不增加消息中间件,不扩展通用 schedule API,也不注册 SCADA 定时任务。页面的明确单步命令写入持久 `step_target`,API 只完成这次已授权的周期处理;不能用“每隔几分钟创建一条 Agent 任务”替代。
最小可交付版本必须包含主动周期处理、批次可靠保存、至少一种明确配置的检测规则和结果展示;只移动页面时间轴不能代表数据已经处理。
## 一个自然月的数据集
数据集用带时区的半开区间定义。例如北京时间 `[2026-10-01 00:00, 2026-11-01 00:00)` 是 31 天,不能固定按 30 天计算。内部统一保存带时区时间,界面按 Asia/Shanghai 展示。
准备数据时冻结清单:`dataset_id`、`dataset_revision`、源项目、观测区间、可用历史基线区间、设备规范 ID、类型、单位、设备采样周期、采样相位、指标含义、raw 分片摘要和规则配置版本。由服务端读取项目路由,页面不传物理库名、SQL 或连接信息。摘要只覆盖原始观测与必要元数据,不覆盖会被本轮清洗更新的 cleaned_value。月度摘要由数据集准备流程预先生成,创建回放的 HTTP 请求只校验清单,不同步扫描和加载整个月。
首版演示直接读取已准备在本地 TSDB 的 raw,不另导出一套完整原始数据源。月份数据如果尚未齐备,由单独授权的数据准备过程补齐并生成版本清单;创建或播放回放本身不生成、删除、搬移或重复插入 SCADA 原始观测。演示批次是对已有观测的读取与处理身份,不能在统计中称为本轮新插入条数。
设备档案采样周期与实际观测间隔不一致时,记录配置差异和缺测情况,仍按实际时间戳读取。不会将 15 分钟数据插值成 5 分钟观测,也不会把没有记录的时刻当作正常值。项目监测任务间隔不参与这项判断。
多次异常应同时覆盖:真实压力异常、真实流量变化、真实水质变化、仪表或通信故障、正常调度引起的变化,以及恢复后的再次异常。数据集真值清单独立保存,用于事后计算命中、误报、漏报、重复事件数和发现延迟。真值不得出现在检测输入、Agent 上下文、预告型时间轴标记或普通业务报告中。
水质必须声明具体指标。当前演示中的化学示踪物浓度不能直接解释为余氯、浊度,也不能只凭 `mg/L` 选择检测阈值。流量保留方向,压力确认 m、kPa、MPa 等单位后再应用规则。
数据集准备负责人需要在运行前提交版本化设备清单、规则参数和独立真值。这是新数据集的输入要求,不表示本次已选定新的 5 分钟设备分组或生成了观测。
## 时间轴和读取语义
### 时间与进度
| 名称 | 含义 |
| --- | --- |
| `observed_at` | 原始观测时间,不随回放改写 |
| `virtual_now` | 已完成主动监测的周期边界时间 |
| `delivered_until` | 本轮源区间已读完,且所有分片的记录引用已登记为待处理的时间上限;表示模拟传入进度,不是原始值新插入时间 |
| `processed_until` | 质量校验和检测已完整提交的时间上限 |
| `read_at`、`cleaned_at`、`processed_at` | 实际批次读取、清洗提交和检测完成时间,用于延迟、故障与审计 |
始终满足 `processed_until <= delivered_until <= virtual_now <= end_at`。还未处理任何批次时,读取和处理进度为空,不能伪装成起点已经处理。GET 返回数据不自动推进任何进度;后台确认本批处理身份后才更新读取进度。表格可显示本轮已读取但仍在清洗的 raw 并标明状态,检测只能消费已完成清洗阶段的批次。主界面的次级进度使用“已读取”和“已检测”。
后台仅在播放状态累计模拟时间。暂停期间不推进;恢复时重新锚定现实时间。进程重启从持久进度重新锚定,不把停机时长一次性折算成需要跳过的数据。
### 顺序读取
1. 整次回放选取 `[start_at, end_at)`;首批包含 `start_at` 的观测。
2. 时间推进到 T 时,候选记录必须不晚于 T、早于 `end_at`,且在已成功接收的联合游标 `(observed_at, device_id)` 之后。
3. 所有选中设备合并后按 `observed_at, device_id` 排序;优先按同一观测时刻成批,不按设备分别跑独立时钟。
4. 每个传输片段最多 500 条、序列化正文最多 2 MiB;超限继续分片。记录片段序号与该时刻是否完整,不在半个时刻接收成功时提前推进时间上限;同一时刻的设备分组检测等待该时刻所有分片收齐。
5. 后台将本批记录引用和原始摘要登记成功后,才推进读取游标。重试同一批使用相同编号和摘要;清洗或检测失败从该阶段恢复,不重新选择一批、不重新插入 raw。正常路径复用本次读取的 raw 对象计算清洗结果,清洗提交后直接将同批结果用于检测,不再重新查询该批 raw 或 corrected。
6. 没有观测的时间段按设备档案采样周期与采样相位识别应到时刻。后台提交记录数为零的区间检查批次,推进覆盖进度并触发缺测判断;它不生成伪造观测。不能仅依赖“下一条数据到来”唤醒,否则连续断流永远不会被发现,终点也无法排空完成。
例如设备档案中的压力采样周期为 5 分钟、流量采样周期为 15 分钟。时间轴到 09:10 时,压力设备可传入 09:00、09:05、09:10,流量设备只有 09:00;到 09:15 时,再传入两类设备对应的 09:15 记录。
首版至多保留一个未完成批次,清洗检测提交后才选择下一批。单次请求不跳过周期内的中间观测、质量检查、异常开始和恢复,也不提供播放倍率。
### 页面操作
| 操作 | 约定 |
| --- | --- |
| 创建 | 固定数据集、范围、设备和规则,进入 ready |
| 播放 | ready 或 paused 进入 running,由后台推进 |
| 暂停 | 停止释放新批次;已持久接收的批次继续处理。返回实际生效进度,在途接收与暂停命令通过同一回放版本串行裁决 |
| 下一步并监测 | 在 ready 或 paused 状态推进到下一个项目监测周期边界;周期内的所有观测按原始时间顺序处理,到边界后提交一次本周期监测回执并保持暂停,尾步可短于完整周期 |
| 调速 | 原子更新速度和时间锚点,不改变观测顺序 |
| 拖动时间轴 | 只查看已释放的数据与已产生的事件;未来区间不预览读数,不触发事件 |
| 从指定时刻重播 | 新建 `replay_id`,独立检测与调查;新起点之前仅作为有声明的历史基线,跳过段不声称已经检测 |
| 到达终点 | 停止新批次,进入 draining;待已接收批次完成后 completed。Agent 调查是否完成另行展示 |
| 停止 | 终止继续传入,保留已接收批次、事件和证据;不自动撤销已经建立的调查 |
| 重试故障 | 修复依赖后校验固定配置,failed 进入 paused,重试原未完成批次;源版本或批次内容冲突需要新建回放,不能靠重试忽略 |
生命周期为 ready、running、paused、draining、completed、stopped、failed。队列等待、基线不足和依赖暂不可用是处理状态或原因字段,不再增加一套并行生命周期。
首次运行遇到可重试投递错误,按现实时间退避重试同一批次;权限错误、数据集版本变化、同批编号内容冲突和结构缺失进入 failed,明确原因。用户恢复前必须解决原因;原批次事实继续保留。
### 选定演示区间
用户先在月度数据可用范围内选择起止时间。创建回放后,将该区间固定为当前播放范围;进度比例与剩余时长都相对该区间计算。例如选择 10 月 8 日 08:00 至 14:00,时间轴只覆盖这 6 小时,不能将月内 8 日的位置作为本轮进度。
主要界面常驻显示选定区间和当前监测时间。月份、数据版本、设备范围及规则收进来源与设置区域;需要校验的配置差异仍在开始前明确显示。不存在上一次选择时不自动处理整个月,等待用户确定区间。
演示建立后改变时间区间需要停止并创建新回放;保留旧回放的事件和证据,不把旧回放时间游标直接重设。已完成区间需要再次演示时,新建独立回放;不自动循环处理和重复触发调查。
选定区间之前的可用历史用于已声明的基线初始化及冻结历史对照,不占用主时间轴、不计入当前处理量、不预触发此前事件,也不回写 TSDB 的 cleaned_value。基线不足时保留实际行数和模型数据缺口,不能悄悄扩大演示范围或读取区间之后的数据。
处理到选定终点后停在终点,显示数据处理是否完成及剩余调查数量;不会自动继续处理该月剩余数据。空区间明确显示没有观测,不能通过控件动画假装已经收到 SCADA 数据。
### 5 分钟时间定位与项目监测周期
时间轴以 5 分钟时间戳组织刻度、标签和选择值,只负责定位和回看,不读取 `projects.settings.scada_monitoring_interval_seconds`。方向键按 5 分钟移动;鼠标和触摸的拖动轨迹保持连续调频手感,选定值仍按最接近的 5 分钟时间戳归档。标签密度可以随面板宽度合并,不会改变拖动手感或时间选择精度。
能力接口另行返回当前项目监测周期;创建回放时由服务端固定到回放文档及快照的 `monitoring_interval_seconds`,页面不能提交不同周期。项目配置后续调整只影响新回放,当前演示继续使用固定值;未配置时提示设置。
| 交互 | 行为 |
| --- | --- |
| 方向键 | 前后移动一个 5 分钟时间戳 |
| PageUp / PageDown | 移动一个当前可见的主刻度,主刻度始终为 5 分钟的整数倍 |
| 拖动回看 | 拖动轨迹连续无缝,选择值按 5 分钟量化;只读定位不推进后台 |
| 主刻度 | 宽屏可每 5 分钟标注,窄屏按 10、15 分钟等密度显示,细刻线保留 5 分钟定位 |
| 下一步并监测 | 不读取时间轴位置;如后台已监测至 08:07,15 分钟周期处理到 08:22 |
“下一步并监测”通过持久化 `step_target` 与 `step_pending` 管理。单步开始后,API 按原观测时刻读取、清洗和检测到目标边界,完成前禁用重复单步。若周期内有 5 分钟观测,15 分钟监测周期仍处理其中每个时刻;没有观测则提交空区间检查,不补造数据。尾步到达 end_time 时,仍遵守原始区间 `[start_time, end_time)`。
时间轴以 5 分钟时间戳定位,不能控制或触发周期处理;“下一步并监测”按项目监测频率推进后台周期,不将原始观测重采样。旧回放没有固定周期时,页面要求停止后重新创建,不能把后来修改的项目周期追补为历史事实。
### 步长边界主动监测与爆管流程
一次主动监测的身份为 `(replay_id, period_end, monitor_rule_version)`。同一步骤重试复用原回执,不能重复创建告警、调查或计算。监测回执保存在该步最后一个回放批次的 `monitoring` 中,并将最近一次结果投影到回放快照;内容包含周期起止、知识截止、输入批次上限、数据完整性、各指标状态、组合结论和关联任务。它不是 `scheduled_flow`,也不需要通用定时器。
步长边界按以下顺序执行:
1. 确认本周期内所有已到达观测均已完成质量校验和清洗;缺测、无效值和单位冲突进入数据完整性结果,不能按正常值处理。
2. 汇总压力下降幅度。只使用映射明确、单位一致、质量合格的压力测点,并同时保留异常测点数、覆盖测点数、基线窗口和最大或中位下降幅度。
3. 汇总出厂流量变化。设备必须在项目配置或设备档案中明确标注为“出厂边界流量”,不能把任意 `pipe_flow` 测点当作出厂流量;保留方向、单位、基线和增量。
4. 条件满足时运行爆管侦测。现有侦测要求至少 5 个完整压力点和目标时刻之前 15 天同周期数据;不满足时返回 `baseline_not_ready` 或 `sensor_coverage_insufficient`,不能将“未运行”解释为“无异常”。
5. SCADA 数据诊断发现任一已配置指标异常,就创建 `event_investigation`,使用现有 `scada_anomaly` 工作流并绑定本轮 `replay_context`。调查先核验数据,再进行爆管诊断;综合分用于判断爆管嫌疑,不能作为启动调查的前置门槛。没有异常的正常周期只保存回执。数据不足但已有有效异常信号时仍发起调查并列出缺口。
6. `scada_anomaly` Skill 核对异常证据并进行爆管诊断,输入具备时决定是否调用爆管定位。定位必须有 m³/s 的有依据漏量估计或明确情景假设,以及模型节点映射、异常窗和正常对照窗;输入不具备时保持“疑似爆管、定位待补充”,不能填默认漏量。
组合判定采用可版本化策略,首版演示建议如下:
| 监测结果 | 判定条件 | 后续动作 |
| --- | --- | --- |
| `insufficient_data` | 必需点位缺测、质量无效、单位或角色未配置,或爆管侦测基线不满足 | 保存缺口,不声明正常,不启动定位 |
| `normal` | 数据完整,爆管侦测未命中,压力与出厂流量规则均未命中 | 保存周期回执,不创建 Agent 任务 |
| `attention` | 单项指标命中、指标方向不一致,或周期内曾出现测点异常 | 创建或复用 `scada_anomaly` 调查,进入爆管诊断 |
| `suspected_burst` | 加权分达到门槛,或旧版联合策略的三项指标共同命中 | 同一份 `scada_anomaly` 调查标记爆管嫌疑;定位输入具备后进入计算 |
爆管侦测和压力下降存在相关性,因此回执不能把两者描述成完全独立的两票。出厂流量也会受泵站调度、储水设施和正常用水变化影响。组合结论用于评估爆管嫌疑,不是调查启动门槛或现场确诊。
回放中的 MCP 定位已走 Server 持有的安全适配层:按 `replay_id`、`knowledge_cutoff` 和 `batch_sequence` 从已固化清洗输出组装压力与可选流量序列,再调用现有计算队列。回放只允许 monitoring 模式,并要求显式异常窗、正常对照窗和有依据的 `burst_leakage`;不能直接查询 TSDB 中尚未回放的未来记录。步长爆管侦测同样只读取最近 15 天已固化输出。计算结果保存输入、算法与证据引用,任务恢复或重试仍使用相同知识上限。
### FM 调频盘样式
用户明确选择电台调频盘的刻度、红色指示线和玻璃质感。这里的 FM 表示调谐外观,不增加电台、音频、频段切换或拟物旋钮功能。
视觉方向:工作台内一块冷白半透明的玻璃刻度窗,红色细线作为唯一醒目的时间定位标记;文字、控件与下方数据沿用现有工作台主题。内容顺序为选定区间、当前时间、刻度窗、控制栏、传入与检测进度。交互特征为固定指示线与横向移动的刻度带,手动拨动直接跟手。
![FM 调频式时间轴静态样式示意](assets/scada-fm-timeline.svg)
图中时间和状态仅为静态样式示意,不是运行结果。玻璃下的底色用于说明透光与边缘高光,运行时仍需验证实际工作台背景下的对比度。
| 元素 | 设计约定 |
| --- | --- |
| 演示区间 | 常驻显示完整起止日期和时间;更换区间是次要入口,月度来源收在设置中 |
| 当前时间 | 刻度窗上方突出显示红线对应的时间,数字等宽;跨日时同时显示日期 |
| 时间刻度 | 主刻度有时间标签,次刻度细密且高低有别;实际密度按区间与容器宽度计算,窄屏优先减少标签而非缩小文字 |
| 红色指示线 | 固定在刻度窗水平中央,1 至 2px 宽,带小型顶端指示;这是时间定位色,不代表告警严重度 |
| 玻璃刻度窗 | 局部半透明、背景柔化、细边缘高光和轻微内阴影,玻璃层不影响数字、刻度和红线的清晰度 |
| 控制栏 | 播放或暂停是唯一主操作,下一批与倍速为辅助;回看时才出现回到当前 |
| 次级进度 | 单行显示已传入与已检测时间,积压时显示等待处理;不增加三块同等醒目的仪表 |
玻璃效果是本次按用户要求对 SCADA 调频刻度窗的局部材质例外,外围面板外壳、表格、表单、菜单和 Agent 面板仍沿用现有 shadcn/ui 实体设计。新增时间指示颜色使用独立语义变量,不能让正常播放状态套用危险告警颜色。关闭透明效果或不支持背景模糊时,回退到清晰的实体刻度窗。
固定红线表示当前定位时间,刻度带按 5 分钟时间戳移动;只在选定区间内标注时间,两端之外留白,不带出整个月的刻度。该窗是选定区间的只读定位视窗,完整起止范围始终由上方文字明确表达;拨动不改变传入或检测进度。
手动拖动和触屏横拨改变查看位置,不立即改写后台时间。拨动到已释放时间时显示历史快照并标明回看;拨动到尚未释放时间时只显示待传入时间,不展示未来读数,若需从该位置开始则明确新建回放。回到当前只恢复跟随,不补发旧数据、告警或调查。
时间拨动按 slider 语义提供中文名称、时间范围、当前时间文本和键盘操作:方向键按 5 分钟刻度调整,PageUp/PageDown 调整一个主刻度,Home/End 到选定范围两端。键盘操作即时生效;鼠标和触摸拖动轨迹连续无缝、不带惯性,选定时间按 5 分钟量化。减少动态效果时关闭装饰性过渡,保留真实时间定位。状态不用颜色单独表达。
## 数据质量与业务异常
真实外部系统新增观测时,先通过现有入库边界将 raw 保存到 TSDB。本次演示使用已经入库的原始值,读取 API 校验身份、项目、设备与时间范围后返回 raw,直接进入质量校验和清洗;不为了模拟传入而重复执行原始值 INSERT。清洗只写回同一条观测的 cleaned_value,monitored_value 始终保留。
清洗完成后才启动业务异常检测。valid 可将 cleaned_value 设为原始值,corrected 保存有依据的修正值,无法可靠修正的 invalid 或 missing 保持空值并保存质量结论。suspect 保留 raw 与不确定性,不能自动替换为正常值。是否已经清洗以阶段回执和质量结论为准,不能仅用 cleaned_value 是否为空判断。
| 质量结论 | 选择值与后续处理 |
| --- | --- |
| valid | 使用 raw,包括可靠采集的突降、突升、低压和浓度变化 |
| corrected | 使用有依据的修正值,同时保留 raw、修正值、规则版本、原因和可用时间 |
| suspect | 保留 raw 并声明不确定性,结合其他测点和设备状态复核;不自动改成正常值 |
| invalid 或 missing | 保留原始记录或缺测事实,退出要求可信观测的规则输入,形成数据质量事件 |
质量校验依据结构、有限数值、单位、设备身份、观测顺序、重复记录、量程与设备状态;仅偏离日常水平不能直接判 invalid。负流量、零浓度、低压均需按设备和业务含义解释。缺少仪表状态时明确证据不足,不能凭一次尖峰确认仪表损坏。
当前正式表中的 `cleaned_value` 可以保留,但不等于本轮已经完成清洗。读取 raw 时不清空已有 corrected,也不将旧 corrected 当作清洗输入。清洗阶段使用本批 raw 和本轮已知历史重新判断,提交本轮清洗版本并保存处理结果;检测只能使用该版本已提交的结果,不能直接取一个来源和时间不明的旧 corrected 当作已校验输入。
在线修正只使用当前已到达数据。当前居中窗口、后向补值、整段 Kalman 平滑和整段统计阈值保留为离线分析能力;不能一次性清洗整个月后将结果送入时间轴。在线清洗更新目标 TSDB 的修正值投影,原始值不变;原修正版本、当前修正版本和依据保留在处理证据中,旧回放仍读取自己的固定版本。
每条参与判断的数据记录其 `value_source`、质量结论、选用值、原始记录引用、规则版本和计算时间。回放过程不为未知错误强制生成一个修正数值。
这一区分对应 [EPA 在线水质监测指南第 6.1、6.2 节](https://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=P100UJYF.txt) 的“先校验、再异常检测,并保留原始值”原则;历史修订留痕可参考 [OPC UA Historical Access 第 4.4 节](https://reference.opcfoundation.org/specs/OPC-10000-11/4)。本项目的具体质量判定仍由指标和设备证据决定。
## 检测规则和多次事件
首版实时检测使用有版本的类型化规则:压力变化与上下限、流量变化与方向、水质具体指标的上下限或变化,以及数据质量规则。规则输入包含质量结论,输出只表示异常信号及依据,不确认爆管位置或污染原因。
每项启用的规则必须声明:`rule_id`、指标、方向、单位、阈值、使用的历史窗口、最少有效样本、连续命中点数、恢复阈值、连续恢复点数,以及事件组和最少同时命中的设备数。参数在创建回放前校验并冻结,至少需要一种已配置规则才可创建检测回放。缺失参数时该规则为 not_configured,不能借用统一 30% 或把零到非零统一解释为 100% 污染风险。
同一个事件组由明确的设备集合组成,不使用预设异常真值自动分组。同组、同规则、同一持续异常归并为一个事件;不同事件组可以同时产生不同事件。恢复需要可信的新观测满足恢复条件,缺测不构成恢复。恢复后的再次命中建立新事件。
事件保存首次异常、检测触发、最后异常、恢复时刻、证据引用和调查结论。传感器恢复正常只表示观测恢复,不代表现场维修、污染处置或业务调查已经完成。
事件身份由 `replay_id + rule_id + event_group_id + first_trigger_at` 确定;同一事件更新不创建多条调查。批次重试不重复创建事件,新一次回放的事件与旧回放独立。
现有 [爆管侦测服务](../app/services/burst_detection.py) 可作为满足条件后的进一步分析工具,不能直接等同于逐批在线检测器。当前 target 模式需要 15 天数据及至少 5 个完整压力测点。回放前没有足够历史时,展示 baseline_not_ready;支持短历史的规则仍可运行,不能用未来几天补足参考窗口。
正常基线只使用当时已知、质量合格且适用于该规则的历史。已确认异常可以从正常基线中排除,原始记录仍保留。事后确认或重新标注产生新版本,不改写此前检测“当时知道了什么”。
## 持久化和事务
观测数据以项目目标 TSDB 的 `scada.measurements` 为权威存储,沿用 time、device_id、monitored_value 和 cleaned_value。业务库不替代 TSDB 保存正式观测。
建议在 BizDB 增加以下处理对象:
| 对象 | 权威信息 |
| --- | --- |
| `peripheral_sample.scada_replays` | 回放身份、项目、创建者、冻结配置、生命周期、版本、模拟时间锚点、读取与处理进度、工作租约和恢复检查点 |
| `peripheral_sample.scada_replay_batches` | 本批 TSDB 记录键、raw 摘要、固定规则版本、清洗输出、质量结论、检测结果、处理阶段和重试信息 |
两个对象分别具有控制生命周期与批次处理生命周期。现有 test_data 是手工单条演练 CRUD,缺少顺序、批次和消费语义;任务表表示业务调查,也不能存正常观测批次。这里保存记录引用和处理结果,不复制一整套 raw;需要固化异常证据或修正依据时,只保存必要的原始片段。由于本次不重新摄入 raw,不新增 TSDB 原始值收件表。表和列同时写清数据库注释。
事件及当前告警复用 `workbench.alert_events`、`workbench.alerts`,调查及其证据复用现有 operational 表。告警来源扩展为判别联合:保留现有任务来源,新增 `kind=scada_replay`、`replay_id`、`dataset_revision`、规则组与触发批次,避免伪造调查身份。质量故障也用同一事件存储,按类别区分。
### 批次处理与提交
不使用一个长期跨库事务。批次按以下阶段单调推进,重试保持原批次身份:
1. `selected`:后台读取 raw 后,BizDB 固定记录键、输入摘要、规则与知识截止时间,并在同一事务登记处理身份和读取检查点。GET 本身不做此写入。
2. `clean_prepared`:直接对已读入的这一批 raw 和已加载的历史状态计算质量与修正结果,先将确定的输出及输入摘要保存在 BizDB。正常路径不调用会重新查询同批 raw 的整段清洗入口。
3. `cleaned`:TSDB 用一个有界事务按记录键更新 cleaned_value,并校验 raw 仍与本批输入一致;不更新 monitored_value。只有 TSDB 提交成功且 BizDB 记录已提交阶段后,才允许检测。单行结果相同时不必重写字段。
4. 若 TSDB 已提交但 BizDB 阶段尚未更新,恢复时核验预存的输入与输出;已经匹配则补记 cleaned,未写入则幂等应用预存结果,raw 已变则停止该批次。只在这种恢复或校验冲突路径增加读取,不为每批正常处理增加一次重复查询。
5. `detected`:直接使用本批已提交的清洗结果做检测,BizDB 在同一事务保存检测结果、事件状态、调查意图、检查点和 processed_until,然后发布业务事件。检测失败只重试本阶段,不重新清洗或重写 raw。
以上阶段保存在批次中。页面继续突出模拟时间、已读取进度和已检测进度,清洗中或失败作为当前批次状态展示。正常路径对新批次读取一次 raw、必要时批量更新一次修正值,再复用这批处理结果完成检测;界面后续查看与故障恢复属于独立的必要读取。
BizDB 批次状态是持久待处理队列,通知只负责唤醒。后台启动、通知中断与定期对账均扫描未结束批次,按已提交阶段恢复;同库状态与事件在同一事务保存,不能假定两个数据库天然原子。[PostgreSQL NOTIFY](https://www.postgresql.org/docs/15/sql-notify.html) 和 [SELECT 锁定语义](https://www.postgresql.org/docs/15/sql-select.html) 提供通知及领取基础。
调用慢算法和外部服务前结束数据库事务,不持有事务锁等待 HTTP。清洗结果未提交,不执行异常检测;中断后从已有检查点恢复,不丢弃批次身份重新处理一遍。
### 重复读取与重复回放
读取 API 的同一次游标请求可以安全重试,但相同数据不能被当作新的处理批次。后台以项目、回放、批次序号和分片序号登记幂等身份,记录级使用规范设备 ID 与原始时间;同键同摘要复用现有处理状态,不同摘要返回冲突。
新一轮演示按新的 replay_id 建立独立处理状态,可以重新读取和分析选定区间,但仍不插入 raw。界面统计已读取条数和已处理条数,不能称为新写入 SCADA 条数。
首版同一项目只允许一条活动的读取清洗流水线,避免不同回放同时改写同一 cleaned_value。历史结果仍可同时查看。修正字段表示最近一次已发布清洗结果,旧版结果和依据保留在批次输出及必要的原始证据中;旧 Agent 执行和历史回看不直接读取被后续处理替换的修正投影。
工作台事件流只发布回放控制状态变化、故障与业务告警;不为每个测点值发布 SSE,也不把全部原始数据放入事件日志。未来若新增 `scada.replay.changed` 主题,只用于控制状态,并需同步主题模型、约束和消费契约;当前控制进度由轮询读取。
未读取数据所在 raw 分片摘要变化时停止该回放,要求新数据集版本;本轮 cleaned_value 更新不计作源版本漂移。空检查批次也需要保存处理阶段,记录数为零不代表可以绕过缺测判断。
每个回放同一时刻只有一个有效处理租约,采用数据库会话锁阻止两个请求重复推进同一周期。每个批次提交后再读取下一原始时间戳,不持久保存进程计时器值。
源数据和相关回放证据在本次演示中不自动过期;一个月是数据覆盖范围,不是保存期限。事件结束、暂停或重播不清理已有证据。生产保留策略单独制定。
## Agent 调查与时间隔离
主动单步在边界形成组合监测回执:任一有效指标异常进入 `scada_anomaly` 调查;缺少爆管判别资料不能阻止已知异常进入调查。SCADA 演示请求只负责建立持久调查任务,不要求先生成一条合成 test_data,也不在请求内执行 Agent。
告警事实与“待建立调查”意图在事件事务中保存。创建调查失败时后台对账重试;组合监测调查的来源稳定标识使用 `scada_replay:<replay_id>:monitor:<period_end>:<rule_version>`,一般事件继续使用 `scada_replay:<replay_id>:event:<event_id>`,均沿用现有外部来源幂等约束。事件不因 Agent 不可用而丢失。
已新增 `scada_anomaly` 工作流与对应 Skill,要求先核验数据质量、点位和单位,再读取事件窗口及正常参考。疑似爆管复用现有 `burst_report` Skill;它会区分侦测、定位候选和现场确诊,并要求漏量依据。`run_burst_location` 在回放任务中通过冻结输出适配执行,其他未适配计算继续拒绝;不能仅修改 Skill 文本或 MCP allowlist 绕过知识截止。重计算沿用 [后台计算队列](calculation-queue.md),保留外层 HTTP 限制。
新增 Server 持有的回放查询上下文:`replay_id`、数据集版本、知识截止时刻 `knowledge_cutoff`、已接收批次上限、处理版本和来源事件。它与任务及执行代次绑定,不能由模型或客户端自行扩大。
每代调查开始时冻结其可用批次与时间上限;同一执行中后续回放推进不会偷偷改变证据。确有新的重要证据时,按现有 [调查轮次规则](operational-tasks.md) 创建或合并后续调查轮次,而不是每批次追加一次模型执行。
时间隔离必须覆盖 MCP 的 SCADA 查询、要素历史、任务数据快照、外围观测,以及检测算法内部读取。raw 源读取 API 只允许读取不晚于模拟时钟的观测;检测只使用已完成本轮清洗的批次结果,原始表格可以显示本轮已读取且尚在清洗的 raw。历史回看和旧代调查读取对应固定版本或原始记录引用,异常证据使用已经固化的必要片段。即使 TSDB 存有整个月记录,也必须按本轮读取检查点、设备集合和知识截止时间过滤,不能只在提示词中要求遵守。
所有回放调查证据保留真实来源:合成数据仍为 synthetic,历史回放有明确标识,不能因为读取的是正式表就改称现场实时观测。普通历史数据管理仍可按原权限查看数据集,但不能作为回放会话绕过截止时间的途径。
回放调查只生成演示分析、计划和报告;真实外围工单和现场控制不属于自动回放动作。Server 动作执行边界必须识别回放上下文,不能仅靠 Agent 提示阻止将演示事件发成真实业务工单。
## 接口与界面
以下为已经加入 Server OpenAPI 的路径,统一使用 Bearer 与 X-Project-Id。保留原有 `/peripheral/api/sample/v1` 工单与手工演练接口,回放不增加一层外围代理。
| 接口 | 语义 |
| --- | --- |
| `GET /api/v1/scada-replays/capabilities` | 回放表可用性、写权限、监测频率、爆管组合监测角色是否就绪及最近一轮状态;不代表完整月度数据就绪 |
| `POST /api/v1/scada-replays` | 固定区间、设备、规则和质量量程,返回 ready;携带稳定 request_id |
| `GET /api/v1/scada-replays/{replay_id}` | 权威控制、时间、计数、设备与固定规则、当前输出及故障 |
| `POST /api/v1/scada-replays/{replay_id}/commands` | step、monitor、stop、retry;request_id + expected_revision,monitor 另需带时区的 at_time |
| `GET /api/v1/scada-replays/{replay_id}/observations?through=...` | 仅本轮已读取时间范围内的历史快照,每设备最近一条、最多 500 个 |
| `GET /api/v1/timeseries/scada-readings/raw` | 当前项目 TSDB 原始值联合游标分页;不插入、不隐式清洗 |
| `GET /api/v1/scada-replays/{replay_id}/events?offset=0&limit=100` | 检测变化与调查关联,返回 items、next_offset |
raw API 允许参数为 start_time、end_time、device_ids、cursor、limit;回放请求携带经服务端验证的 replay_id,任务请求由已有绑定解析回放身份。有效范围为所选区间与模拟时间上限的交集,不能因省略 replay_id 绕过已绑定的任务限制。返回 items(time、device_id、monitored_value、quality)、next_cursor 和 scanned_through;本版不返回尚未准备的 dataset_revision,不返回 corrected 作为源输入。
next_cursor 绑定项目、设备、查询区间并固定本轮分页时间上限;本版不为整个查询提供 MVCC 源版本快照,按 time、device_id 排序,每页最多 500 条。读取窗口未结束但当前页为空时,scanned_through 表示已经确认没有观测的区间上限,供后台登记空检查批次。分页、刷新和网络重试均是无副作用读取。
查询和检查点更新使用类型化服务方法及已验证项目上下文。清洗函数接受已经读入的批次,不再调用当前会重新查询 raw 的整段清洗入口;检测函数接受这批已提交清洗结果。页面的单步 POST 命令同步完成一个监测周期,GET 预览和回看不启动任务。
回放读取要求 scada.view,创建及控制要求 scada.clean;普通 raw API 沿用 timeseries SCADA 路由的读权限。API 不自动创建表,不回退默认项目,也不隐式枚举全部活跃项目。
前端 SCADA 页签按以下顺序组织:
1. 选定演示区间:开始时间、结束时间和项目监测步长;设备范围、规则及基线收纳在折叠的检测参数区域。
2. FM 调频式时间轴:局部玻璃刻度窗、固定红色指示线、两侧渐暗及中央放大;按 5 分钟时间戳定位,不读取逐测点列表,也不触发监测。
3. 主动监测操作:以“开始并监测”或“下一步并监测”为唯一主操作,按项目频率同步处理一个周期。
4. 本周期结论:显示正常、关注、数据不足或疑似爆管,以及是否已经进入爆管调查与定位流程。
设备目录、raw 与修正值明细、完整事件列表和手工单点异常录入不在演示主面板展示。底层数据与 API 继续保留,供诊断、追溯和调查任务按权限读取。
沿用现有 workbench-shadcn-surface、日期控件和实体表面,调频刻度窗采用上文明确的局部玻璃例外;时间轴支持键盘操作和中文可访问名称。拖动回看不触发后台写操作。
页面可见时每 5 秒读取一次权威控制快照;单步请求执行期间由操作本身返回最终快照。页面隐藏时停止界面采样。演示面板不轮询完整事件列表,调查入口需要时再按分页 API 查询。
## 实施范围与验证
这是跨 Server、供水前端和 Agent 契约的变更,不新增后台进程。数据、存储、身份及网络复用现有部署。
| 仓库 | 主要落点 |
| --- | --- |
| Server | `app/api/v1/endpoints/scada_replay.py` 的回放控制与能力;`app/services/` 和 `app/infra/db/` 的批次、质量和事件;现有 task_alert、operational_task、MCP 上下文、查询和 Skill |
| Server 契约与存储 | `resources/sql/` 的 BizDB 两张回放表迁移;`contracts/` 的 raw 读取、Server OpenAPI 和清单;外围协议不变;TSDB 沿用已有观测表 |
| 前端 | `src/features/workbench/peripheral-sample/` 的回放控制与状态 hook;SCADA 客户端与生成契约;告警来源和事件解析 |
| Agent | `src/product/` 的生成契约及任务上下文适配,保留已有原生 MCP 与执行权限边界 |
验收用例必须覆盖:
- 5/15 分钟设备混合、乱序源记录、同一时间戳跨页、空片段、结束边界和多次异常。
- 重复批次、相同编号不同内容、响应丢失、请求中断和恢复重试;不漏读、不重复事件。
- GET 重试、分页重复请求、清洗输出预存后中断、TSDB 更新提交而 BizDB 阶段未更新、检测失败;逐阶段恢复,不提前检测。
- 正常处理每批只读一次 raw,清洗后检测复用结果;回放不执行 raw INSERT,不在读取时清空旧 corrected,空数据不伪造观测。
- 周期内多批次单步、重复单步拒绝、并发请求互斥和尾步完成状态。
- 5 分钟时间定位与 15 分钟监测周期相互独立、非整点起点、周期配置变更不影响当前回放、尾步不足完整周期和未配置时禁止开始。
- 每个步长边界只产生一份幂等监测回执;正常、数据不足、单指标关注和疑似爆管分别处理,不为正常周期创建 Agent。
- 爆管侦测基线不足、压力下降、出厂流量变化、设备角色错误、指标方向冲突和多指标同时命中;单指标、部分资料不足及疑似爆管均复用 `scada_anomaly` 调查入口,正常且无异常信号时不建单。
- 回放 MCP 侦测与定位只能读取固定批次和知识截止前数据;定位缺少漏量依据时停止在待补充,不调用算法。
- 选定小时区间、跨日区间和再次演示;刻度与进度只对应选定区间,基线初始化不释放旧事件,到终点不自动播放整月。
- FM 红线固定、刻度带移动、拖动回看不重复投递、未来位置不泄露读数;桌面与 375px、320px 窄屏下的长日期、键盘、减少动态效果及玻璃回退。
- 真实突降不被当作采集误差修掉;负流量、零水质值、空值、错误单位和缺测恢复分别处理。
- 首个异常、持续异常、可信恢复、再次异常及两个事件组并发;不会逐点创建调查。
- 缺少基线、缺少水质指标配置、历史 corrected 无可用时间,以及 source_revision 变化。
- MCP、任务数据接口及算法内部的未来查询均被拒绝;下一代调查才能使用新增证据。
- 合成来源、历史回放来源、事件真值隔离;回放不发布真实现场动作。
- 合成测试链路、真实项目 API、常驻后台、真实 Agent 执行和浏览器分别记录验证结果。
后续实现的验证命令从各仓库执行:Server 运行新增回放、批次、质量、事件与查询边界的聚焦 pytest,并保留现有 SCADA/告警/调查回归;前端运行 `pnpm lint`、`pnpm typecheck`、`pnpm test` 和聚焦 `pnpm test:browser`;Agent 修改后运行 `bun run check` 及对应聚焦测试。具体已执行结果见下方,不把待部署链路计为通过。
数据库应用遵守 [工作台数据库变更门禁](workbench-database-rollout.md):先核对目标项目,备份和回滚演练,再按明确范围应用及验收;不得默认向全部项目写入。本地 MetaDB 已对明确的演示项目应用并复验 `033_metadata_project_settings.sql`,其他项目保持原值;`tjwater2_v2` BizDB 已应用 `031_scada_replays.sql`,不据此推断其他项目已迁移。本次没有生成一个月观测或启动常驻进程。
回退应用前暂停新批次读取,核对 clean_prepared、cleaned 与 detected 阶段,保留原始观测、已提交修正值、回放表、告警与调查证据;升级恢复后从原阶段继续。BizDB 流程记录和 TSDB 修正值不在同一事务,回滚应用不能假定另一库结果也被撤回。不能通过删除事件、重置时间游标或清空 SCADA 原始表来伪装成从未处理。若需要撤回数据库变更,另按备份与新增业务写入情况制定恢复步骤,不在仍有写入时覆盖数据库。
本方案依赖的关键前提是源历史能提供稳定设备身份、可信时间戳和固定数据版本。缺少完整月度观测时,能力接口只能声明实际可用范围;缺少修正来源时采用 raw;缺少检测参数时该类检测不启用。任何这些缺口都不能由时间轴、插值或预设事件答案掩盖。
## 部署准备与验证记录
迁移脚本为 [031_scada_replays.sql](../resources/sql/031_scada_replays.sql),显式限制物理数据库 `tjwater_v2` 与 `tjwater2_v2`,依赖既有 peripheral_sample schema、告警及调查结构。仅新增两张 BizDB 表,不新增 TSDB 观测表。`tjwater2_v2` 已应用该迁移并配置 `settings.scada_monitoring_interval_seconds=900`。
真实数据库验证使用 `SCADA_REPLAY_STORAGE_TEST=1 conda run -n server python -m pytest tests/integration/test_scada_replay_storage.py -q`。该测试在项目路由选出的 BizDB 与 TSDB 双回滚事务中临时建表、写测试观测,覆盖真实 SQL 的 cleaned 更新、事件持久化和主动周期推进,退出全部回滚;它不证明身份认证 HTTP 和真实 Agent 推理已经验收。浏览器测试以 mock Server 验证交互与窄屏布局,不能作为现场检测效果证明。
本轮验证结果(2026-11-03,当前工作区):
| 层级 | 结果与边界 |
| --- | --- |
| Server | 主动监测、条件式爆管调查、冻结定位适配、MCP、项目配置、raw、处理及 OpenAPI 聚焦回归共 120 项通过 |
| 数据库与契约 | 本地 MetaDB 的 033 迁移已单项目应用并重复执行验证;OpenAPI 279 条路径与清单摘要一致;本轮未运行 031 的真实双库集成测试 |
| 前端 | lint、typecheck、全量 148 文件共 884 项 Vitest 通过;设备列表、手工单点演练及对应触发适配已移除 |
| 浏览器 | 1 项 mock Server 场景通过,包含 5 分钟刻度、选择值与键盘步进、连续拖动轨迹、15 分钟主动监测、中央放大、两侧渐暗、固定红针及 1280/375/320px 布局;未作为真实检测效果证明 |
| Agent | 同步 Server OpenAPI 并生成 server-api.d.ts,bun run check 通过;无真实模型推理 |
| 仓库 | Server、供水前端、Agent 的 git diff --check 通过;未提交、推送或部署 |
生产联调仍需核验真实 HTTP 接口、调查执行链路和 Agent 报告,并单独准备完整月度数据及事件真值。不能将以上局部测试解释为这些环节已经完成。
### 同一事故的告警与调查去重(2026-11-07)
同一轮回放首次检出任一有效指标异常时,持久保存 `monitoring_incident`,发送一次 SCADA 异常告警并创建一份 `scada_anomaly` 调查工单,先核验数据再进行爆管诊断。T+15 等后续周期复用事故 ID 和工单,回执 `incident.action=ongoing`;缺数、部分命中或仍处低压时也保持该事故。每个周期的检测结果仍独立保存,原工单的证据截止时间不随每个周期滚动;若单项异常首次升级为疑似爆管,使用既有 `advance_replay_investigation` 推进同一工单的调查代次,为新代次冻结新增证据,仍不重复报警或建单。
只有连续两个相邻的正常监测周期,且首次命中的压力测点全部有新的有效读数、达到事故开始时冻结基准的恢复范围,才将事故标记为恢复并解除同一条告警。恢复范围使用首次压力基准与该点下降阈值,不能因滚动基线适应低压就解除告警;缺数、过期读数或跳过周期会中断连续恢复确认。确认恢复后再次检出指标异常,才新建事故和调查。流量异常的恢复也需核对其冻结基准,不能仅依赖已适应高流量的滚动基线。
主动步进中,压力、流量变化与水质上下限规则保留检测事件作为证据,测量指标的异常通知统一由周期诊断发出;数据质量通知继续独立。周期内已恢复的短暂异常仍保存为本周期调查依据,不因结束时读数恢复而丢失。状态、告警、工单及批次在原有处理锁和同一业务事务内保存,失败重试沿用确定的请求标识。旧回放首次升级时可承接上一周期已有工单,不批量合并历史重复工单。主动创建新一轮回放仍属于独立演示,不跨回放合并事故。
精简演示面板保留时间区间与 FM 时间轴,集中显示最新结论、监测区间、综合分与门槛;指标权重和命中情况按需展开。持续事故明确显示“沿用已有调查工单,本周期未重复报警”,下一周期处理期间保留上一周期结论并标明处理进度。
### 面板规则与实际流程(2026-11-07)
面板以“SCADA 数据诊断 → 任一指标异常 → 爆管诊断 → 报告”为入口,不以爆管综合分筛掉单指标异常。`/capabilities.monitoring_config` 返回新一轮采用的项目规则;回放快照的 `monitoring_config` 和 `monitoring_calibration` 返回本轮实际冻结的配置及阈值。进行中的回放不套用后来修改的项目配置,旧回放未保存判定策略时按原有 `all_signals` 展示。
规则区显示实际三项权重、爆管嫌疑门槛、动态阈值学习状态与历史区间。基线就绪后显示实际侦测阈值及压力/出厂流量的逐测点阈值范围;基线不足不展示虚构阈值。加权角色测点采用实际动态阈值,不再套用固定 30% 输入;“采样范围与基础规则”配置区已移除,面板只读展示实际诊断规则。已具备有效流量基线时,即使压力模型基线不足也保留该流量异常,不以组合资料缺口屏蔽单项诊断。`anomaly_signals` 保存三项爆管参考指标之外的有效测点异常,数据不足与单项命中也能创建同一诊断工单。当前阶段仍需单独验收真实 Agent 推理与报告。
演示新建默认采用 Case3 的北京时间 2026-07-16 07:30–08:30,事件起点为 08:00,即事件前后各 30 分钟。存在其他时段的活动回放时,提供“切换到 Case3”,先停止旧回放再设定该区间,不自动推进或创建调查;后续监测按钮按项目周期推进。停止或完成的旧回放不再覆盖新建演示默认区间,也不将旧结果显示为 Case3 的结论。
### 自动播放与简短通知(2026-11-07)
调查任务仍按 `replay_context.knowledge_cutoff` 与 `batch_sequence` 截断知识,后续播放不自动扩大当前调查窗口。任务摘要只保留指标与异常数量,逐测点明细通过冻结 SCADA 证据查询。`scada-anomaly` 要求高嫌疑诊断继续候选定位;当前 Case3 的任务摘要和 skill 响应返回 `location_capability` 能力提示。提示可用时 `run_burst_location` 允许省略 `burst_leakage`,仍先核验冻结异常窗、正常窗及压力设备,再调用固定候选方法;水力算法仍要求漏量依据。候选输出不再含 mock 提示,MCP回放证据保留 `origin=synthetic`,不向 Agent 暴露事件真值。
Agent可见能力字段统一为 `location_capability`,不再输出旧 `location_demo` 字段。
公共指令与 SCADA Skill 明确候选及相似度返回后继续地图展示、隔离、节点覆盖、影响分析与报告,
来源字段本身不构成停止条件。有效 SCADA 查询不再将来源说明列为 `data_gaps`;实际缺测、
窗口与设备校验仍按原规则处理。现场工单与设备动作维持服务端权限限制,动作拒绝文案明确
允许继续已授权分析。旧工具回执和历史消息不改写;既有 OpenCode 进程的 system 配置副本
需刷新后才使用新的公共指令。
回放调查开放项目资产属性、空间查询、拓扑隔离和节点覆盖,沿用原有业务权限。它们依据当前模型,不代表历史现场状态。实时 TSDB、在线模拟等未接入冻结输入的路径仍不可用;候选定位不以出厂流量命中或现场确诊为前提。
点击“自动播放”后,浏览器以 180 倍速推进演示时钟,一小时区间约用 20 秒播放,后端处理耗时另计。到达项目监测周期边界时调用原有 `step` 接口,每次只允许一个监测请求,等待持久回执后再继续。播放时钟与已处理时间分别展示;时间轴的手动浏览仍按 5 分钟定位,手动拨动先暂停自动播放,不直接触发处理。暂停、关闭面板、切换页签或页面转到后台后停止自动推进;接口失败暂停,终点自动结束,不自动重试或新建下一轮。此能力属于前台演示,不引入通用后台定时任务或新的 SCADA worker。
通知只展示异常状态、调查已启动或沿用原调查及时间,通过“查看调查”打开已有调查与报告。指标、评分、基线、定位和处置依据保留在调查摘要与报告中,Server 告警描述不再复制完整任务摘要。首次实际监测检出新的异常时,局部图标以 1.4 秒单次扩散和轻微缩放强调;同一异常下一周期及旧回执刷新不重播,减少动态效果模式保留文字并关闭动画。异常事件的持久去重与冻结证据规则继续有效。
### 指针选定时刻监测
前端“回到开始时间”只把指针定位到所选区间起点并暂停播放,不清空已处理进度,也不自动调用监测。显式点击“监测选定时刻”才提交 `POST /api/v1/scada-replays/{replay_id}/commands`,动作 `monitor`,携带带时区的 `at_time`、`request_id` 和 `expected_revision`。服务端复用逐帧读取、清洗、检测、历史基线和调查流程,知识截止与最终处理时间精确停在 `at_time`,不自动推进到下一个监测周期。选定起点也可以监测;同一已完成时刻的重复请求复用已有回执,不重复计次、解除告警或创建调查。
`at_time` 必须在本轮区间内,不能替换进行中的处理步;直接要求本轮回到已经处理过的更早时刻返回 409。前端在该场景下提前说明将建立新一轮,停止旧轮后使用原区间配置新建回放,再处理至选定时刻;原回放、告警和调查保留,历史判断不携带旧轮的后续状态。`step` 继续按冻结的项目周期从本轮起点对齐,选定 08:05 后下一步为 08:15,而不是 08:20。此变更新增命令字段,不需要数据库迁移;需同步更新 Server、前端 API 客户端与 OpenAPI 契约。
+101
View File
@@ -0,0 +1,101 @@
# tjwater_v2 爆管 Agent Benchmark
状态:场景框架与只读数据预检已准备;隔离执行环境、真实触发链、Agent 轨迹采集和正式评分器尚未实现。2026-11-03。
## 目标与边界
本套 benchmark 用固定的管网状态、触发条件和隐藏真值,比较提示词、Skill、MCP 工具或模型配置变更对爆管处置的影响。首版限于 `tjwater_v2` 爆管,不扩展水体污染。已有数据库是数据来源;每次正式执行必须在独立评测数据库和运行时完成,原项目数据库不承担工单、SCADA 清洗、告警、任务、计算或证据写入。
```text
原 MetaDB / BizDB / TimescaleDB
| 仅 SELECT、只读事务
v
版本化私有数据快照 -> 隔离评测 BizDB + TimescaleDB -> Server/MCP/worker -> Agent Host
| |
+----轨迹与业务结果----+
|
评分报告 + 版本归档
```
评测 Agent 只能连接隔离环境。单靠客户端 `READ ONLY` 事务不足以保证未来服务不误连原库;运行配置必须使用隔离数据库路由,并由原库侧评测账号权限拒绝写入。现有 `temporary_project_database`/模型模板复制路径不能直接用于原项目,因为它会暂时关闭连接并终止源库会话。应在独立实例先生成逻辑快照,再把可写副本作为评测环境。
## 场景矩阵
配置文件 `resources/benchmarks/tjwater-v2-burst.json` 定义四个候选物理工况:
| 场景 | 管段 | 管径 | 发生时间(北京时间) | 当前观测准备状态 |
|---|---:|---:|---|---|
| B01 | 540672 | DN315 | 2026-10-08 02:00 | 需在隔离集生成事件观测 |
| B02 | 540672 | DN315 | 2026-10-08 12:00 | 历史回放声明待核对当前窗口 |
| B03 | 537407 | DN600 | 2026-10-08 02:00 | 需在隔离集生成事件观测 |
| B04 | 537407 | DN600 | 2026-10-08 12:00 | 需在隔离集生成事件观测 |
每个工况各有工单上报(WO)和 SCADA 监测(SCADA)入口,共 8 个候选用例。另设无爆管正常控制作为误报检查,不算第三类业务。管段、时间和历史分析名称仅是候选定义;准备报告须检查观测窗口、参考工况和触发是否吻合。未满足条件的用例标为 `not_ready`,不能作为 Agent 失败,也不能从统计中静默删除。
工单路径使用 `burst_report` 来源,经既有外围工单同步、操作任务 worker、Agent Host 和 MCP 完成调查。SCADA 路径从隔离副本的正常数据起步,再释放事件数据,由真实 replay worker/规则创建通用异常调查;不能把目标管段或爆管类型预先告诉检测器。纯规则预览只作准备检查,不能代替这条端到端路径。
工单位置由冻结的网络 GIS 几何转成稳定的人类可读位置,结合近似范围和现象描述,不把真实 link ID 直接写成答案。Agent/Server 业务结果是首版主分数;前端呈现单独做 Playwright 结果回放检查,不能用工作台 mock 结果代替 Agent 评测,也不混入业务能力分。
首版使用单一持续爆管,不模拟修复恢复;预计结束状态是有证据的调查报告,可以合理等待人工确认或审批。现有模型只含 2 个阀门,目标管段即使移除全部模型阀门仍与水源连通,因此成功关闭隔离不作为成功标准。Agent 应说明拓扑限制和现场核查需求,不得编造阀门方案。
## 真值、可见信息与评分
真值包在运行前冻结并与 Agent 输入分开保存,至少包含:管段及端点、泄漏参数和单位、观测窗口、传感器映射、独立拓扑参考、允许的等价候选集合、定位是否具备有效流量输入。工单只给近似位置、时间和现象;SCADA 检测器只能看到已释放观测及设备元数据。隐藏真值不得进入 Skill、提示词、任务说明、MCP 上下文、可读工作区、文件名或执行轨迹。
由于当前仿真将 emitter 放在管段端点,邻近管段可能不可区分。定位适用的用例按冻结的等价候选集合和排名评估(Hit@1/3/10),不强求唯一 ID。爆管定位需要有效的 `burst_leakage`(m³/s);孔口面积(cm²)不能直接代替流量。缺少输入时,评分应奖励明确说明限制和所需数据,不鼓励猜测。
总分 100,结果与过程分开报告:
| 维度 | 权重 | 评分依据 |
|---|---:|---|
| 业务结果 | 60 | 触发来源与结论置信度 10;证据、时间窗、单位和数据质量 15;定位 Hit@1/3/10 或正确 abstain 15;影响与不可隔离判断 10;报告和人工后续建议 10 |
| 工具执行 | 20 | 场景清单定义的唯一必需操作有效完成率 12;有效参数、时间/单位范围及结果证据引用 8;同一操作重复调用不增加分数 |
| 计划推进 | 15 | 依赖顺序与证据覆盖 6;根据新证据调整计划 5;报告终态、待人工确认和停止条件 4;不评分模型私有思维链或固定调用顺序 |
| 效率 | 5 | 仅在业务结果至少 45/60 且无关键错误陈述时计分;以已接受基线的 Token、Agent wall time、有效调用数中位数对比,三项等权,单项最高 1.67 分 |
定位分在“输入足以定位”的真值用例中按 Hit@1=15、Hit@3=12、Hit@10=8、未命中=0;输入不足的用例不要求猜管段,明确限制和所需数据得 15,猜测得 0。工具有效完成率按每个场景冻结的唯一必需操作计分;单个操作重复尝试只取首次有效终态,额外调用不扩大分母或分子。
结构化事实、调用结果、时间/单位、候选排名、计划依赖、任务终态由确定性 scorer 判定。报告措辞与计划是否有证据支撑由同一固定版本盲评器独立评分两次,输出结构化子分和引用依据;每次保留评审器模型、提示词版本与原始评分。黄金样例由爆管业务负责人标注。双评差异超过任一子项 2 分、评审器无证据引用或关键结论不确定时转人工复核,复核值和原因进入报告。效率基线建立后,单项得分为 `1.67 × min(1, 基线中位值 / 当前值)`,三项合计最高 5 分;效率基线建立前,该项标为 `unscored`,报告总分上限 95,不伪造完整 100 分。
报告必须同时给出总分、业务通过率、工具调用成功率、定位 Hit@k、误报/漏报、延迟、分项分数和失败原因。队列 accepted 不算计算成功;MCP `isError`、超时和业务拒绝分开统计。系统错误记为 `infra_error`,保留尝试并有限重试幂等步骤;模型业务失败按实际结果计分。不做 retry-until-pass。
初始分项数值映射需要用一个人工审阅的黄金样例和重复运行校准后冻结为 scorer 版本;不能在第一次运行后按期望结果回调规则。首轮建议每个就绪用例重复 3 次,并对同版本做 A/A 重跑,估计模型随机波动;小样本只报告原始分布,不声称统计显著。
## 自动执行与复现
正式 runner 的单个入口按以下检查点运行:
1. `prepare`:校验套件、来源版本、传感器/数据窗口、触发可检出性、模型拓扑和真值包完整性;输出明确就绪/阻塞清单。
2. `snapshot`:以只读导出生成 BizDB、TimescaleDB 及必要文件的不可变版本包和哈希;所有评测写入只落在该版本的可写副本。
3. `trigger`:每例建立唯一 attempt ID,按 WO 或逐步 SCADA 事件路径触发。SCADA 没触发时记录触发失败,不补发 Agent 任务。
4. `investigate`:通过真实 Agent Host 和 Server/MCP 执行,脚本化反馈只提供当时可知事实;模型不得读取未来观测或真值。
5. `collect`:记录 case/attempt、task/run/execution/session、任务和报告修订、计算 job、MCP receipt、工具调用、计划版本、Token/耗时与终态。续跑先查已有业务 ID,禁止重复建单或重复触发。
6. `score/archive`:确定性 scorer 读取隐藏真值和收集轨迹,生成逐例 JSON、汇总报告、HTML/CSV 可选视图及所有代码/配置/数据/scorer 指纹。基线升级由人工明确选择。
每个 case 串行运行,直到能证明不同可写副本完全隔离后再并发。事件时间使用虚拟时钟;JWT、worker lease、heartbeat 和 HTTP timeout 使用现实时间。Agent 输入的查询统一受 `knowledge_cutoff` 约束。每次运行保留所有尝试和原始错误,不保存密钥、访问令牌或完整 DSN。结果进入 Git 忽略的私有工件目录或受控 CI 存储,不提交业务观测和轨迹。
每轮实验固定记录:Agent/Server/frontend 代码内容摘要、提示词与 Skill 摘要、模型及参数、MCP 工具 schema/版本、数据快照 ID/哈希、suite/scorer 版本、基线版本、case、重复次数、运行预算和实际服务指纹。只存 Git SHA 不足以标识当前含未提交改动的运行版本。
## 当前准备状态与执行门槛
当前已有:四个候选场景清单、严格字段校验、分库只读查询与规则预览函数、独立拓扑参考。只读预检已在 2026-11-03 执行:四个场景均有 83 个完整短窗压力测点,但没有一个触发当前短窗 SCADA 规则;15 天检测窗口也没有满足连续 1440 点的压力节点。B02 的工单路径是唯一有现成数据候选的路径,B02 的 SCADA 路径仍需隔离生成能触发的事件数据。该结果只说明历史窗口/当前规则的匹配情况,不证明实际 worker 或 Agent 行为。刚加入的 `scripts/prepare_burst_benchmark.py` 只生成私有预检 JSON,不会建任务、写 SCADA 修正值、运行 Agent 或声称数据集已冻结。
完成自动评测前仍需依次补齐并验收:
1. 已执行只读预检;下一步由评测维护者确认是否保留 B02-WO 为首个可用试跑,并在隔离副本生成 SCADA 触发与 B01/B03/B04 所需观测。
2. 爆管业务/水力分析负责人审阅四个用例的数值参考、有效泄漏流量和等价候选;评测维护者据此固定触发规则与控制组期望。
3. Server 数据维护者形成带模型/观测版本哈希的快照导出和隔离 BizDB/TimescaleDB 路由;通过受限角色、连接路由检查和数据库审计证明 Agent/worker 无法写回源库。源库其他日常服务可能同时写入,因此验收证明 benchmark 没有源写入,不承诺整库前后字节完全相同。
4. Agent 与 Server 维护者实现可恢复 runner 和全链路轨迹导出;先执行一例 WO 与一例 SCADA 的闭环试跑。
5. 评测维护者与爆管业务负责人共同冻结 scorer,在首个通过门槛的用例跑 A/A;再扩到 8 例 × 3 次 × 候选版本。
若源数据在准备期间被日常业务改变,重新读取后产生不同内容指纹,应建立新快照版本,不覆盖旧报告。预检报告只证明来源库读取与候选数据检查完成,不是 Agent 评测结果或上线验收。
## 命令
只读检查命令(在 Server 仓库根目录):
```bash
conda run -n server python scripts/prepare_burst_benchmark.py
```
只检查单个场景可追加 `--case B02`。预检报告写入 `data/benchmark/preflight/`,该目录在 `.gitignore` 中。此命令只读源数据库;正式评测 runner 尚未提供。
+185
View File
@@ -0,0 +1,185 @@
# tjwater2_v2 多距离、弱信号爆管场景
状态(2026-11-07):完成四区域、三档破口、四时段的 48 组本地水力校准与固定阈值敏感性分析,并从中选出 4 个演示 case。4 个爆管方案已保存,压力、流量和正常水质数据已写入隔离项目与 `tjwater2_v2` 的 IoT 库。动态阈值尚未接入真实监测 worker,正式 benchmark Runner、真实工单和 SCADA Agent 链路、评分及归档仍待实现。
同日管网 ID 修复后,R1 管段当前 ID 为 `564345`,流量设备 `542121` 的绑定管段当前 ID 为 `570995`。管段属性、几何、拓扑和 SCADA 测量值未变化。隔离 benchmark 的 BizDB、852,960 条正常实时结果,以及清单直接引用的 2,016 条方案管段结果已同步。以下清单统一使用新 ID。
48 个历史方案保存了全管网结果,其中未被当前 benchmark 清单引用的长管段仍属于迁移前的校准归档,不进入 Agent 输入或当前 scorer。完整历史结果的全量迁移因压缩块重写耗时过长而回滚;如需把旧方案用于全管网地图展示,应重新生成方案或另行离线迁移,不能混用新模型 ID 与旧结果 ID。
## 当前演示数据集
演示数据使用隔离项目 `tjw_bench_tjwater2_20261105`,项目 UUID 为 `66ce6b1a-4415-4cd0-b68f-bf1505576c79`。`tjwater2_v2` 与隔离项目的 IoT 库均写入相同数据,便于在完整系统中演示,也便于 benchmark 指定隔离数据源运行。后续自动评测默认使用隔离项目,避免业务操作、清洗结果或回放状态互相影响。
每个月写入 1 日 00:00 至 18 日 00:00 前的数据,共 17 天、1,632 个 15 分钟时刻。设备包括 83 台压力设备、35 台流量设备和 50 台水质设备,共 168 台。每台设备每月 1,632 点,每月 274,176 行,四个演示窗口合计 1,096,704 行。`cleaned_value` 初始为空,由真实清洗和回放流程产生。
每个事件前都有精确的 15 天同时间点历史,即 1,440 个时刻。按全部 168 台设备计算,每个 case 的事前基线为 241,920 行。检测算法使用事件时刻之前 15 天的同时间点数据,不要求自然月内其余日期全部正常或完整。
### 4 个演示 case
时间均为北京时间(UTC+8)。事件持续 5 小时,包含起止点共 21 帧;事件结束后的下一个 15 分钟时刻恢复正常数据。
| Case | 事件时间 | 入口 | 管段 | 破口面积 | 等效圆孔直径 |
| --- | --- | --- | --- | ---: | ---: |
| `B26-severe` | 2026-05-16 08:00 至 13:00 | SCADA 自动发现 | `564345` | 22.089323 cm² | 53.03 mm |
| `B28-severe` | 2026-06-16 20:00 至 06-17 01:00 | SCADA 自动发现,跨日 | `564345` | 22.089323 cm² | 53.03 mm |
| `B30-severe` | 2026-07-16 08:00 至 13:00 | 工单在事件发生 60 分钟后上报 | `541725` | 353.429174 cm² | 212.13 mm |
| `B35-severe` | 2026-08-16 12:00 至 17:00 | 工单在事件发生 120 分钟后上报 | `538540` | 245.436926 cm² | 176.78 mm |
四个 case 均使用管道截面积的 1/8,比例不是管径的 1/8。B26 和 B28 复用同一物理位置,用来比较早间与晚间跨日条件。B30、B35 使用更大口径和更远压力测点,用来覆盖区域、信号强度和工单发现延迟的差异。
| Case | 破口坐标 WGS84 | 管径 / 长度 | 最近压力设备 / 沿网距离 |
| --- | --- | --- | --- |
| B26 / B28 | 121.471119, 31.005631 | DN150 / 249.176 m | `92724` / 约 0.205 m |
| B30 | 121.569785, 30.956880 | DN600 / 495.368 m | `170486` / 约 1.19 km |
| B35 | 121.487510, 30.823888 | DN500 / 121.485 m | `102837` / 约 2.67 km |
### 泄漏真值与 SCADA 流量响应
破口泄漏量来自水力模拟,是隐藏评测真值,不能直接进入 Agent 的提示词、任务上下文或可见工具结果。SCADA 流量是 Agent 可见证据,单位为 m³/h,已完成 L/s 到 m³/h 的换算并加入确定性仪表噪声。
| Case | 泄漏量 L/s | 泄漏量 m³/h | 代表性 SCADA 流量变化 |
| --- | ---: | ---: | --- |
| B26 | 21.60 至 23.49 | 77.76 至 84.58 | `449417` +37.32 m³/h;`516234` +15.63 m³/h |
| B28 | 21.99 至 25.81 | 79.15 至 92.90 | `449417` +49.42 m³/h;`542121` +102.68 m³/h |
| B30 | 424.15 至 446.33 | 1,526.95 至 1,606.79 | `524529` +296.26 m³/h;`541061` +241.34 m³/h |
| B35 | 236.97 至 261.11 | 853.08 至 940.01 | `508453` +926.08 m³/h;`527865` +46.96 m³/h |
三个主要水厂计量设备的 5 小时最大变化如下。百分比按同时间正常值计算;小流量管段的基线接近 0 时,百分比会失真,评分必须同时看绝对变化量和流向。
| Case | `542017` | `486033` | `542121` |
| --- | ---: | ---: | ---: |
| B26 | +50.11 m³/h,1.23% | -82.44 m³/h,1.27% | +32.15 m³/h,3.47% |
| B28 | -81.51 m³/h,3.37% | +129.10 m³/h,3.65% | +102.68 m³/h,19.10% |
| B30 | +713.91 m³/h,19.13% | +293.30 m³/h,4.75% | +258.24 m³/h,27.95% |
| B35 | +212.19 m³/h,5.43% | +491.69 m³/h,8.01% | +102.87 m³/h,11.64% |
### 正常水质数据
50 台水质设备使用 EPANET 化学水质模拟生成正常余氯数据,没有用固定常数补值。13 个水库和 15 个正常入流水源设置为 0.50 mg/L,全网初始余氯为 0.35 mg/L,体相衰减为 -0.05/天,管壁衰减为 -0.02/天。模型预热 6 天后取第 7 天的正常日曲线,按 15 分钟间隔重复到各 case 窗口。
仪表噪声使用固定种子,标准差为读数的 0.5%,最低标准差为 0.001 mg/L,并截断在正负 3 倍标准差内。最终数据约为 0.071 至 0.507 mg/L,没有零值、负值或求解警告。当前四个爆管 case 内的水质保持正常,只用于保证设备读数完整,不作为爆管响应或判分指标。
## 管段与监测距离
保留一个近测点强信号对照,其余改用中远距离候选。距离取爆管节点到全部压力设备中的最近沿网距离,不是到任意挑选的某个测点的距离。按管道模型长度累计,阀门连接按近零长度处理,表中取近似值。
| 组别 | 管段 ID | 管径 / 长度 | 最近压力设备 / 沿网距离 | 破口经纬度 WGS84 | 边界阀 / 分区节点 |
| --- | --- | --- | --- | --- | --- |
| R1 近点对照 | `564345` | DN150 / 249.176 m | `92724` / 约 0.205 m | 121.471119, 31.005631 | 5 / 11 |
| R2 中距干管 | `541725` | DN600 / 495.368 m | `170486` / 约 1.19 km | 121.569785, 30.956880 | 4 / 64 |
| R3 远距多点响应 | `538540` | DN500 / 121.485 m | `102837` / 约 2.67 km | 121.487510, 30.823888 | 2 / 10 |
| R4 远距弱信号 | `541116` | DN200 / 228.424 m | `406444` / 约 2.91 km | 121.441641, 30.982725 | 4 / 9 |
四处均经过后端隔离算法和水源连通校验,关闭所列边界阀后可在模型拓扑上切断水源。分区节点数不等于用户数、实际停水范围或现场阀门可操作性。
上轮贴近测点的 DN110、DN315、DN500 候选及其结果仍保留,不能因本轮换管段而重用其 case ID。
## 破口面积与发生时间
比例均指管道截面积,不是管径;不再只选一个最容易触发的面积。
| 管段 | 1/32 面积 cm² | 1/16 面积 cm² | 1/8 面积 cm² |
| --- | ---: | ---: | ---: |
| R1 DN150 | 5.522331 | 11.044662 | 22.089323 |
| R2 DN600 | 88.357293 | 176.714587 | 353.429174 |
| R3 DN500 | 61.359232 | 122.718463 | 245.436926 |
| R4 DN200 | 9.817477 | 19.634954 | 39.269908 |
发生时间统一为北京时间 2026-10-08 的 02:00、08:00、12:00、20:00,分别覆盖夜间、早间、日间、晚间。每个事件持续泄漏 5 小时,15 分钟一步,共 21 帧;20:00 场景结束于次日 01:00。3/4 小时视图按前缀截取。
R1 至 R4 分别对应 B25 至 B28、B29 至 B32、B33 至 B36、B37 至 B40;三档面积作为 case 的独立维度,例如 `B29-medium`。共 4×3×4=48 个物理场景,后续再派生 SCADA 与 WO-30/60/120/180 等入口 case,不能把这些派生用例当成不同物理事件。
所有场景有相同起始时间、模型状态、需求模式和 PDA 参数的无爆管对照。三个主要水厂出口设备 `542017`、`486033`、`542121` 均保留;压力单位 m,流量结果 L/s 映射 SCADA 时转为 m³/h。
## 观测结果与动态阈值
各场景在 5 小时内的最大监测压降,以下取四个发生时段的范围。这是相同时间正常/事故模拟的差异,不是运行告警直接可用的未来对照。
| 管段 | 1/32 | 1/16 | 1/8 |
| --- | ---: | ---: | ---: |
| R1 | 11.58% 至 12.69% | 26.37% 至 28.08% | 52.84% 至 54.65% |
| R2 | 4.48% 至 5.57% | 9.92% 至 11.90% | 22.05% 至 25.14% |
| R3 | 3.95% 至 6.24% | 7.91% 至 12.21% | 14.65% 至 21.58% |
| R4 | 0.10% 至 0.13% | 0.19% 至 0.26% | 0.37% 至 0.50% |
48 组均完成求解,未出现求解警告、指定监测点负压或结果缺失。R2 至 R4 因未达到旧 30% 门槛而被标为 `weak_signal_candidate`,仍然是有效爆管正例;不当成无爆管样本,也不为让所有 case 通过而增大破口。
固定阈值复用实际清洗/回放相对变化规则,使用四帧预热和连续两帧,未加入绝对压降门槛。16 个正常对照按“管段×时间”去重:
| 固定压降阈值 | 触发的爆管场景 / 48 | 误报的正常对照 / 16 |
| --- | ---: | ---: |
| 3% | 47 | 13 |
| 5% | 41 | 8 |
| 10% | 25 | 3 |
| 20% | 11 | 0 |
| 30% | 4 | 0 |
这里的“触发”只是规则预览,不是 Agent 识别成功;极低阈值下的高触发率不能忽略正常误报。
动态规则采用独立版本,初始实验方案如下:
1. 每个测点仅用事件前 15 天完整正常数据拟合,15 分钟间隔要求 1440 点;有缺点时报告历史未就绪。
2. 按日内时段建立正常压力中位数参考,避免将正常早晚变化当成事故;用去除日内模式后的 `1.4826×MAD` 表示相对波动。
3. 测点相对阈值取 `max(3%, 3×相对波动)`,与绝对压降至少 0.5 m、连续两帧同时成立。3%、0.5 m 是首轮实验参数,需结合测点精度和正常误报结果校准。
4. 在事件前冻结参考、阈值和规则版本;拟合函数拒绝事件时刻及之后的观测。评估遇到采样中断会重置连续计数。
5. 压力异常后仍需流量或独立压力证据交叉验证。实验函数目前只覆盖压力层,未接入生产监测 worker。
本次水力包只有四帧事件前预热,因此尚不能给出真实项目动态规则的检测率。当前新增的学习与评估函数仅通过合成正常历史验证;不能把上表固定阈值结果说成动态规则已验收。
## 复现与数据边界
清单:[tjwater2-v2-burst-balanced.json](../resources/benchmarks/tjwater2-v2-burst-balanced.json)。原有强信号清单保持不变。
```bash
conda run -n server python scripts/check_burst_simulation_design.py \
--design resources/benchmarks/tjwater2-v2-burst-balanced.json
conda run -n server python scripts/calibrate_burst_benchmark.py \
--design resources/benchmarks/tjwater2-v2-burst-balanced.json --all-severities
conda run -n server python -m pytest \
tests/unit/test_burst_benchmark_calibration.py \
tests/unit/test_regional_burst_selection.py \
tests/unit/test_burst_pressure_thresholds.py -q
```
`scripts/analyze_burst_thresholds.py --calibration-dir <某次校准目录>` 复用已冻结结果做阈值敏感性分析,无需重跑水力。`scripts/prepare_regional_burst_design.py` 输出的 GeoJSON 包含管段、实际破口点和边界阀,位于被忽略的本地数据目录。
源管网模型在准备阶段保持只读。当前已建立隔离 BizDB、IoT 库和模型模板,并通过正常模拟与保存的爆管方案按设备绑定生成最终 `scada.measurements`。为支持完整系统演示,同一份 SCADA 数据也写入 `tjwater2_v2` 的 IoT 库。动态拟合和 Agent 评测只能读取各自知识截止时刻以前的数据;模拟真值及方案信息不进入 Agent 可见上下文。
## 正式生成优先本地后端方法
本地执行优先复用 `app.services.simulation_scenarios.burst_analysis()`。MCP 的 `run_burst_simulation` 适配器和 `save_simulation_scenario` 保存入口也调用这个函数,分别设置 `persist_results=False` 和 `True`。固定模型、SCADA 边界输入、时间、破口与单位后,两条入口共用水力计算实现;MCP 额外承担权限、上下文、任务和审计等流程。
此前直接修改 INP 的结果只用于选管校准。正式 4 个演示 case 已在隔离路由下保存方案,保留正式 `run_id`、模型和结果摘要,再按设备绑定写入 SCADA。Agent 评测仍需使用真实工具链,数据生成成功不能替代 MCP 工具成功率、计划过程和最终业务效果评分。
## 方案结果到 SCADA 的映射
先分配日期窗口,再按该时间运行正常模拟和爆管方案;默认沿用计算结果的真实时间戳写入 SCADA,不在写入时悄悄平移日期。
| SCADA 设备类型 | 绑定字段 | 正常数据源 | 事故数据源 | 写入值 |
| --- | --- | --- | --- | --- |
| 压力 | `asset.scada_devices.node_id` | `realtime.node_results.pressure` | 指定 `run_id` 的 `analysis.node_results.pressure` | 单位核验后写 m |
| 管道流量 | `asset.scada_devices.link_id` | `realtime.link_results.flow` | 指定 `run_id` 的 `analysis.link_results.flow` | 本模型 L/s × 3.6,写 m³/h |
最终键使用 SCADA 的 `device_id`,不能使用模型节点 ID 代替;写入 `scada.measurements.monitored_value`,`cleaned_value` 初始为空,由真实清洗流程计算。压力和管道流量来自正常与爆管水力计算,水质来自独立的正常化学水质模拟。设备列表、绑定、单位和排除原因在生成前冻结,不为了补齐数据自动移动或改绑设备。
`analysis` 的 `run_id` 能区分多个方案,SCADA 的主键只有 `(time, device_id)`,不能区分同一设备同一时刻的多个场景。因此,同管段的不同破口面积也必须分配独立日期,不能只给不同管段分日期。
每个物理 case 的保留窗口覆盖事前 15 天正常历史、5 小时事故、末点采样和恢复缓冲。当前 4 个演示 case 分配在 2026 年 5 月至 8 月,每月一个 case,基线不会混入其他事故。
当前日期映射已冻结在 `resources/benchmarks/tjwater2-v2-burst-demo.json`。B26、B28、B30、B35 分别使用 2026 年 5 月、6 月、7 月、8 月窗口。工单时间、发现检查点和知识截止随日期统一生成。更换提示词、工具或 MCP 版本时复用同一冻结数据,不重新分配日期或重新生成噪声。
5 小时事故结果包含起止点,共 21 帧;当前正常在线模拟为右端不含,直接运行 5 小时仅有 20 帧。若保留事故末点,正常对照须额外生成该时刻,不能将缺失的正常值补零或与上一时刻配对。SCADA 记录使用带时区时间,按北京时间规划、按 UTC 存储和比较;回放查询的右端不含时,查询终点需覆盖最后一帧。
写入前检查整个保留窗口和 `(time, device_id)` 冲突;发现已有记录时停止并重新分配日期,不使用覆盖式 upsert 或 `ON CONFLICT DO NOTHING` 隐藏碰撞。写入采用事务和生成器之间的排他锁,唯一约束兜底。重复执行依据数据集清单和内容摘要验证幂等,不重复插入。
提交后已使用新连接按设备、日期、行数、单位和数值摘要读回校验。生成状态保存在 Git 忽略目录 `data/benchmark-generation/tjwater2-20261105/`,包括 `demo-scada-state.json` 和 `normal-quality-state.json`。这些文件记录 `case_key`、方案 `run_id`、实际日期、单位、行数与指纹,属于本地生成证据,不提交 Git,也不向 Agent 暴露方案真值。
## Benchmark 执行边界
当前数据集已经满足 4 个演示 case 的数据输入要求,但还不能据此声称 Agent benchmark 已完成。后续自动化执行需要按 case 恢复初始业务状态,设置知识截止和入口触发时间,运行真实工单或 SCADA 发现链路,记录计划、工具调用、MCP 调用、任务结果和展示结果,再统一评分和归档。
建议将评分拆成四部分:事件识别与时间得分、定位与影响分析效果得分、工具调用成功率与参数正确性得分、计划和证据链完整性得分。泄漏量、真实管段、方案 `run_id` 和无噪声模拟结果只供 scorer 使用。Agent 可见输入限于当时可查询的 SCADA、工单和正常业务配置。
当前仍有两个运行配置缺口:`tjwater2_v2` 只设置了 `scada_monitoring_interval_seconds=900`,尚未配置 `scada_monitoring.pressure_device_ids` 和 `outlet_flow_device_ids`,联合爆管监测会报告角色未配置;新的 4 个 case 尚未执行完整 SCADA 回放和真实 Agent 端到端验收。
+117
View File
@@ -0,0 +1,117 @@
# tjwater2_v2 爆管模拟校准设计
状态(2026-11-07):本文记录早期强信号管段的校准过程。后续已完成多距离管段校准、4 个演示方案保存,以及压力、流量和正常水质 SCADA 数据生成。正式 Agent benchmark Runner、评分和端到端验收尚未执行。目标项目为 `tjwater2_v2`(演示项目 2(新版))。
当前可执行数据集、4 个演示 case、流量响应、水质数据和 benchmark 边界见[多距离、弱信号与动态阈值设计](tjwater2-v2-burst-balanced-design.md)。本文保留为较早的强信号校准记录,30% 不再作为全部有效爆管用例的统一淘汰条件。
2026-11-07 管网 ID 修复后,本文主候选的当前管段 ID 为 `564345`,流量设备 `542121` 的当前绑定管段 ID 为 `570995`。物理管段、模型参数和校准数值未变化,清单及下文引用均使用新 ID。
## 目标
先用现有管网、阀门和监测点选择可观测的事故管段,通过只读导出的 INP 文件校准破口面积,再在隔离评测数据库保存正式方案。首版使用管道截面积的 1/32、1/16、1/8;1/8 是上限。比例指面积,等效圆孔直径分别为管径的约 17.68%、25%、35.36%。若 1/8 仍不能形成明确降压信号,该管段退出 SCADA 爆管用例,不继续扩大破口。
模拟调用链为:
```text
只读源数据快照 -> 隔离 BizDB/模型模板 -> 正常对照模拟
-> 爆管三档校准 -> 选择最小通过面积
-> 保存正式分析结果 -> 映射 SCADA 数据
```
当前 `temporary_project_database` 会从项目模型模板创建临时数据库,同时短暂禁止模板连接并终止模板会话。因此即使 `run_burst_simulation` 不保存结果,也不能直接针对原项目执行。正式运行必须把只读逻辑快照恢复到独立评测数据库及其模板,Server、worker 和 Agent 全部使用评测路由。
当前校准入口使用 `REPEATABLE READ, READ ONLY` 导出源模型,再修改本地 INP 中的参数,调用仓库自带原生 EPANET;不调用数据库克隆接口,不写原项目或模板,不使用旧 `realtime` 结果代替重算。正常与事故对照均采用 PDA、`REQUIRED PRESSURE=10 m`,避免把 DDA/PDA 的差异算成爆管效果,并保留源配置的 `HEADERROR`、`FLOWCHANGE`、最低压力和压力指数。已确认当前模型无水池、水泵、控制与规则;若后续新增,入口会停止,待补齐状态继承实现。
## 方法参数
`run_burst_simulation` 接收带时区的 `start_time`、60 至 86400 秒的 `duration`、1 至 20 个输出节点、至多 20 条输出管段、1 至 20 条爆管管段,以及与管段一一对应的正数孔口面积 `orifice_areas_cm2`。本次每个调用只模拟一条爆管管段,等待预算使用 240 秒;排队成功只表示已接受,仍需查询计算任务和最终结果。
`tjwater2_v2` 使用 LPS 流量单位、米制压力、0.5 emitter 指数,水力和报告步长均为 15 分钟。压力设备值使用 m,流量设备值使用 m³/h。3600 秒模拟必须产生 0、15、30、45、60 分钟五个结果时刻;缺少任一时刻即失败。爆管服务把 emitter 放在管段末端 junction,切换临时模型为 PDA,并设置 `REQUIRED PRESSURE=10 m`。
孔口面积是输入,不是漏水流量。服务的 emitter 系数为:
```text
coefficient[L/s/sqrt(m)] = 0.65 * area[cm²] / 10000 * sqrt(19.6) * 1000
```
模拟真值的泄漏量按 `coefficient * sqrt(max(事故节点压力, 0))` 计算,并从 L/s 除以 1000 转成 m³/s。PDA 下节点实际需水量会随压力改变,事故与正常的 demand 差值不能直接当成精确泄漏量。该模拟真值留在隐藏评测包,Agent 定位仍需从其可见证据获得泄漏量。流量监测设备单位为 m³/h,模型流量结果为 L/s,映射 SCADA 时乘以 3.6。
## 管段与场景
原 B01/B02 管段 `541720`、B03/B04 管段 `520323` 在恢复水源支路后进行了 1 小时最大面积校准,监测点最大降压分别仅约 2.03%、2.87%,不满足门槛,已退出当前候选。失败结果及当时的完整清单保存在本地报告,不能复用旧 case ID 表示其他管段。
重新从压力点沿网距 400 m 内、DN150 至 DN800、长度至少 50 m 的管段中筛查,按 `length / diameter²` 作为试算排序启发式,优先覆盖不同测点;它不代表水力计算结论。首批试算 24 条,筛出两条进入昼夜完整校准。
当前主候选 `564345` 为 DN150、长 249.176 m,起点 `92686`、破口节点 `7427148338069072993`。该节点距压力点 `92724` 沿网约 0.205 m。关闭全部模型阀门后的所在分区含 11 个节点、5 座边界阀;这只是拓扑参考,尚未验证逐阀关闭的水力影响。三档面积为 5.522331、11.044662、22.089323 cm²。
次候选 `540173` 为 DN200、长 162.128 m,起点 `500819`、破口节点 `517120`,距压力点 `535381` 沿网约 269.125 m。所在分区含 23 个节点、6 座边界阀。三档面积为 9.817477、19.634954、39.269908 cm²。其夜间校准未通过,目前保留为失败对照,不能随主候选一起生成正向 SCADA 用例。
清单同时保存 SCADA 设备 ID 与其实际水力模型对象 ID。例如压力设备 `430299` 绑定节点 `3186958412475415759`;流量设备 `542121` 绑定管段 `570995`。模拟输出使用模型对象 ID,得分与数据映射使用设备 ID。
`542017` 已依据 `tjwater_v2` 恢复缺失的水源 `551778` 和出口管段,并将 SCADA 绑定恢复到 `542017`;原始空间坐标保持不变。当前 BizDB 与模板均为 92393 个节点、91050 条管道、4489 座阀门、13 个水源。三个主要水厂设备 `542017`、`486033`、`542121` 的实际出口绑定均进入两条候选的输出。待核实的 `547063`、`DEMO-WQ-551571` 不参与本轮判分,重筛也排除 `547063`,不得自动移动设备或猜测改绑对象。
两条候选分别在北京时间 2026-10-08 02:00 和 12:00 模拟,使用新 case ID B05/B06、B07/B08。每个场景均有同时间、同持续时长、相同模型状态的正常对照。严重档先验证,双时段均通过后再执行中度和轻度,选择双时段都通过的最小面积;严重档有一个时段失败即退出。正常对照按时间与模型版本复用。
## 通过条件
一个面积只有同时满足以下条件才可冻结:
1. 至少一个指定压力点连续两个 15 分钟时刻较正常对照下降 30%。
2. 同一连续窗口内,另一独立压力点下降至少 10%,或同一指定流量点变化至少 20%。零流量基线不能提供相对变化佐证。
3. 事件前四个 15 分钟正常样本预热后,生产回放清洗/相对变化规则对事故触发、对正常对照不触发。校准预览配置为降压 30%、连续两帧、至少一台压力设备,不等同于完整监测 worker、15 天检测算法或 Agent 已通过。
4. 指定监测节点不出现负压,模拟正常收敛,所有请求输出完整。
5. 同一管段的夜间和白天场景使用同一面积;两者都通过时才能进入正式 benchmark。
压力与流量变化都按相同时间戳的正常对照计算:`abs(事故值 - 正常值) / max(abs(正常值), 1e-6)`。压力规则只取下降方向,流量规则取绝对变化。不得用其他时段均值代替同时间戳对照。
压力下降、流量变化、实际泄漏量、计算状态和缺失输出都保存到校准报告。失败的档位保留,不重试到通过。若严重档失败,该管段标记为不适用;若较小档位失败而较大档位通过,选择最小通过档位。
## 校验与后续执行
清单保存在 `resources/benchmarks/tjwater2-v2-burst-simulation.json`。以下命令使用本地占位 context handle,把全部 12 组计划参数送入当前 `BurstSimulationInput` 做契约校验,再只读检查 Metadata 项目身份、当前 BizDB、模型模板、管段属性、模型单位、步长、监测点、设备到绑定对象的距离、边界阀和既有 emitter;它不创建运行上下文,也不调用模拟工具:
```bash
conda run -n server python scripts/check_burst_simulation_design.py
```
隔离评测数据库准备完成后,runner 先按清单调用非持久 `run_burst_simulation` 校准。面积冻结后,通过 `save_simulation_scenario` 在评测数据库保存全量节点和管段结果,生成固定的运行 ID 和数据摘要。模型分析结果与模拟 SCADA 保留明确来源,不标记为实测数据。
## 自动校准入口与本次结果
```bash
# 1 小时面积校准;输出结果和每次试算的 INP、报告、原生输出
conda run -n server python scripts/calibrate_burst_benchmark.py
# 5 小时完整事件窗口,可派生 3/4/5 小时视图
conda run -n server python scripts/calibrate_burst_benchmark.py --duration-seconds 18000
# 对一个已有校准快照筛查其他管段;每条只算白天单时刻,不代表通过
conda run -n server python scripts/screen_burst_benchmark_pipes.py \
--calibration-dir data/benchmark-calibration/20261104T101304Z-029cc055 --limit 24
conda run -n server python -m pytest tests/unit/test_burst_benchmark_calibration.py -q
```
生成文件仅保存在 Git 忽略的 `data/benchmark-calibration/` 私有目录,清单、源 INP 与求解器均记录 SHA256,每完成一个 case 原子更新报告。源库访问均只读;本入口不创建 `analysis.runs`,不写 `realtime.*` 或 `scada.measurements`。退出码 0 表示校准流程完成,具体管段是否合格须读取报告的 `selection`,不能把运行完成当成所有 case 通过。当前支持进度留存,尚不支持从中断处自动续跑。
2026-11-04 的 5 小时结果(21 帧/场景)为:
| 场景 | 管段 | 面积 cm² | 主要测点压降范围 | 相对变化规则首次触发 | 判定 |
| --- | --- | --- | --- | --- | --- |
| B05 夜间 | `564345` | 22.089323 | 46.08% 至 54.17% | 02:15(+15 min) | 通过 |
| B06 白天 | 同上 | 22.089323 | 49.81% 至 52.84% | 12:15(+15 min) | 通过 |
| B07 夜间 | `540173` | 39.269908 | 26.27% 至 33.96% | 未触发 | 失败 |
| B08 白天 | `540173` | 39.269908 | 30.83% 至 33.92% | 12:15(+15 min) | 通过,但该管段双时段不合格 |
主候选的 1/16 与 1/32 面积在昼夜均未通过,最小通过档位为 1/8 截面积(等效圆孔直径约 53.03 mm)。主压力点 `92724` 的事故压力约 12.21 至 17.87 m,隐藏泄漏真值约 22.21 至 26.86 L/s,流量点 `449417`、`516234` 提供同步佐证。全部正常对照未触发,全部求解无警告,指定监测点无负压。
本地证据目录:
- 原管段失败报告:`data/benchmark-calibration/20261104T101304Z-029cc055/calibration.json`。
- 首批 24 条重筛:该目录下 `screening-df8b9408/screening.json`(明确排除待核实流量点 `547063`)。
- 新候选 1 小时校准:`data/benchmark-calibration/20261104T101859Z-83f0c1a2/calibration.json`。
- 新候选 5 小时验证:`data/benchmark-calibration/20261104T101949Z-3704704b/calibration.json`。
本次 5 小时报告的源 INP 摘要为 `dfb20e54356c2c96d36c97d96516361da7fb1201ce37da4ffc39f339aa8e3e0f`,清单摘要为 `bbcdbf7ce502264ffd826a0bb2c7188c619d6a14c42f82cdbb9320c9d254dd92`。当模型、绑定或参数变化时应重新校准并生成新摘要。
后续多距离设计已从校准结果中选择 B26、B28、B30、B35 四个严重档演示 case,保存到隔离库并按绑定生成 SCADA,同时补齐事件前 15 天历史。中小破口与被淘汰管段仍作为难例或阴性材料,必须另行定义预期,不能沿用正向通过预期。下一步是验证工单和监测真实入口,再执行 Agent 的工具、计划与效果评分。
@@ -0,0 +1,38 @@
{
"version": "burst-benchmark@1",
"suite_id": "tjwater-v2-burst",
"project_code": "tjwater_v2",
"project_id": "324ff7ee-c2d4-5184-a84b-48fb14a2933e",
"sampling_interval_minutes": 15,
"event_minutes": 60,
"detector_days": 15,
"trigger_rule": {
"pressure_drop_percent": 30,
"baseline_samples": 4,
"consecutive_samples": 2,
"minimum_devices": 2
},
"score_weights": {"outcome": 60, "tools": 20, "plan": 15, "efficiency": 5},
"cases": [
{
"case_id": "B01", "pipe_id": "540672", "occurred_at": "2026-10-08T02:00:00+08:00",
"reference_analysis": "test_burst_20260614", "observation_provenance": "requires_generation",
"provenance_note": "夜间爆管尚无已核实的对应异常观测;不能平移中午曲线或将正常观测当作本事件。"
},
{
"case_id": "B02", "pipe_id": "540672", "occurred_at": "2026-10-08T12:00:00+08:00",
"reference_analysis": "test_burst_20260614", "observation_provenance": "declared_historical_replay",
"provenance_note": "present.md 记录该窗口由 test_burst_20260614 生成;需核查当前观测及参考方案,历史声明本身不等于数值验收。"
},
{
"case_id": "B03", "pipe_id": "537407", "occurred_at": "2026-10-08T02:00:00+08:00",
"reference_analysis": "burst_20260610_dn600", "observation_provenance": "requires_generation",
"provenance_note": "存在历史模拟参考,但未核实本时间窗口的该管段异常 SCADA;需在隔离环境生成。"
},
{
"case_id": "B04", "pipe_id": "537407", "occurred_at": "2026-10-08T12:00:00+08:00",
"reference_analysis": "burst_20260610_dn600", "observation_provenance": "requires_generation",
"provenance_note": "本窗口已有其他管段的爆管演示,不能复用为 537407 的真值。"
}
]
}
@@ -0,0 +1,485 @@
{
"version": "burst-simulation-design@1",
"suite_id": "tjwater2-v2-burst-calibration",
"project_code": "tjwater2_v2",
"project_id": "871d7538-589c-47de-a6d5-3185ccb9f0e1",
"model_expectations": {
"node_count": 92393,
"pipe_count": 91050,
"valve_count": 4489,
"flow_unit": "LPS",
"pressure_unit": "METERS",
"pressure_measurement_unit": "m",
"flow_measurement_unit": "m3/h",
"emitter_exponent": 0.5,
"hydraulic_timestep_seconds": 900,
"report_timestep_seconds": 900
},
"duration_seconds": 18000,
"event_times": [
"2026-10-08T02:00:00+08:00",
"2026-10-08T08:00:00+08:00",
"2026-10-08T12:00:00+08:00",
"2026-10-08T20:00:00+08:00"
],
"severity_levels": [
{
"level": "light",
"cross_section_fraction": 0.03125
},
{
"level": "medium",
"cross_section_fraction": 0.0625
},
{
"level": "severe",
"cross_section_fraction": 0.125
}
],
"calibration_order": [
"severe",
"medium",
"light"
],
"selection_rule": {
"primary_pressure_drop_percent": 30.0,
"corroborating_pressure_drop_percent": 10.0,
"flow_change_percent": 20.0,
"consecutive_samples": 2,
"expected_sample_count": 21,
"relative_change_epsilon": 1e-06,
"normal_control_must_not_trigger": true,
"negative_monitor_pressure_allowed": false,
"choose_smallest_passing_area": true
},
"binding_validation": {
"max_pressure_distance_m": 1.0,
"max_flow_distance_m": 1.0
},
"cases": [
{
"case_id": "B25",
"pipe_id": "564345",
"occurred_at": "2026-10-08T02:00:00+08:00"
},
{
"case_id": "B26",
"pipe_id": "564345",
"occurred_at": "2026-10-08T08:00:00+08:00"
},
{
"case_id": "B27",
"pipe_id": "564345",
"occurred_at": "2026-10-08T12:00:00+08:00"
},
{
"case_id": "B28",
"pipe_id": "564345",
"occurred_at": "2026-10-08T20:00:00+08:00"
},
{
"case_id": "B29",
"pipe_id": "541725",
"occurred_at": "2026-10-08T02:00:00+08:00"
},
{
"case_id": "B30",
"pipe_id": "541725",
"occurred_at": "2026-10-08T08:00:00+08:00"
},
{
"case_id": "B31",
"pipe_id": "541725",
"occurred_at": "2026-10-08T12:00:00+08:00"
},
{
"case_id": "B32",
"pipe_id": "541725",
"occurred_at": "2026-10-08T20:00:00+08:00"
},
{
"case_id": "B33",
"pipe_id": "538540",
"occurred_at": "2026-10-08T02:00:00+08:00"
},
{
"case_id": "B34",
"pipe_id": "538540",
"occurred_at": "2026-10-08T08:00:00+08:00"
},
{
"case_id": "B35",
"pipe_id": "538540",
"occurred_at": "2026-10-08T12:00:00+08:00"
},
{
"case_id": "B36",
"pipe_id": "538540",
"occurred_at": "2026-10-08T20:00:00+08:00"
},
{
"case_id": "B37",
"pipe_id": "541116",
"occurred_at": "2026-10-08T02:00:00+08:00"
},
{
"case_id": "B38",
"pipe_id": "541116",
"occurred_at": "2026-10-08T08:00:00+08:00"
},
{
"case_id": "B39",
"pipe_id": "541116",
"occurred_at": "2026-10-08T12:00:00+08:00"
},
{
"case_id": "B40",
"pipe_id": "541116",
"occurred_at": "2026-10-08T20:00:00+08:00"
}
],
"pipes": [
{
"role": "primary",
"pipe_id": "564345",
"diameter_mm": 150.0,
"length_m": 249.176,
"start_node_id": "92686",
"emitter_node_id": "7427148338069072993",
"longitude": 121.47111853069737,
"latitude": 31.005630555976175,
"candidate_orifice_areas_cm2": {
"light": 5.522331,
"medium": 11.044662,
"severe": 22.089323
},
"pressure_bindings": [
{
"device_id": "92724",
"node_id": "92724"
},
{
"device_id": "100026",
"node_id": "100026"
},
{
"device_id": "115039",
"node_id": "115039"
},
{
"device_id": "107672",
"node_id": "107672"
},
{
"device_id": "107651",
"node_id": "107651"
}
],
"flow_bindings": [
{
"device_id": "542017",
"link_id": "542017"
},
{
"device_id": "486033",
"link_id": "486033"
},
{
"device_id": "542121",
"link_id": "570995"
},
{
"device_id": "449417",
"link_id": "449417"
},
{
"device_id": "516234",
"link_id": "516234"
}
],
"boundary_valve_ids": [
"valve-12b1e80612f9432da0e64485",
"valve-880f34624a444f5497faf2ca",
"valve-b6cc8d68e38f43c9b3389e04",
"valve-f189692cccbd40a69dc26cca",
"valve-fa0583a6ac134aeb9513ada5"
],
"output_node_ids": [
"92686",
"7427148338069072993",
"92724",
"100026",
"115039",
"107672",
"107651"
],
"output_link_ids": [
"449417",
"486033",
"516234",
"542017",
"564345",
"570995",
"valve-12b1e80612f9432da0e64485",
"valve-880f34624a444f5497faf2ca",
"valve-b6cc8d68e38f43c9b3389e04",
"valve-f189692cccbd40a69dc26cca",
"valve-fa0583a6ac134aeb9513ada5"
]
},
{
"role": "secondary",
"pipe_id": "541725",
"diameter_mm": 600.0,
"length_m": 495.368,
"start_node_id": "170464",
"emitter_node_id": "170438",
"longitude": 121.56978490259476,
"latitude": 30.956880413066592,
"candidate_orifice_areas_cm2": {
"light": 88.357293,
"medium": 176.714587,
"severe": 353.429174
},
"pressure_bindings": [
{
"device_id": "170486",
"node_id": "170486"
},
{
"device_id": "175618",
"node_id": "175618"
},
{
"device_id": "159673",
"node_id": "159673"
},
{
"device_id": "135160",
"node_id": "135160"
},
{
"device_id": "174754",
"node_id": "174754"
}
],
"flow_bindings": [
{
"device_id": "542017",
"link_id": "542017"
},
{
"device_id": "486033",
"link_id": "486033"
},
{
"device_id": "542121",
"link_id": "570995"
},
{
"device_id": "524529",
"link_id": "524529"
},
{
"device_id": "541061",
"link_id": "541061"
}
],
"boundary_valve_ids": [
"valve-316b3cfa3f40446e90f8c4c4",
"valve-35cccbe86fdf480884e196b5",
"valve-75f76a9f944d4f9aa1941a1b",
"valve-e3238a6bd5784e8d86b9b25e"
],
"output_node_ids": [
"135160",
"159673",
"170438",
"170464",
"170486",
"174754",
"175618"
],
"output_link_ids": [
"486033",
"524529",
"541061",
"541725",
"542017",
"570995",
"valve-316b3cfa3f40446e90f8c4c4",
"valve-35cccbe86fdf480884e196b5",
"valve-75f76a9f944d4f9aa1941a1b",
"valve-e3238a6bd5784e8d86b9b25e"
]
},
{
"role": "secondary",
"pipe_id": "538540",
"diameter_mm": 500.0,
"length_m": 121.485,
"start_node_id": "101686",
"emitter_node_id": "101680",
"longitude": 121.48750960472518,
"latitude": 30.82388769570228,
"candidate_orifice_areas_cm2": {
"light": 61.359232,
"medium": 122.718463,
"severe": 245.436926
},
"pressure_bindings": [
{
"device_id": "121456",
"node_id": "121456"
},
{
"device_id": "102837",
"node_id": "102837"
},
{
"device_id": "84776",
"node_id": "84776"
},
{
"device_id": "112240",
"node_id": "112240"
},
{
"device_id": "87082",
"node_id": "87082"
}
],
"flow_bindings": [
{
"device_id": "542017",
"link_id": "542017"
},
{
"device_id": "486033",
"link_id": "486033"
},
{
"device_id": "542121",
"link_id": "570995"
},
{
"device_id": "508453",
"link_id": "508453"
},
{
"device_id": "527865",
"link_id": "527865"
}
],
"boundary_valve_ids": [
"valve-beab0b66708a4ae99b5e7f5f",
"valve-db75ca893cd246d498786475"
],
"output_node_ids": [
"101680",
"101686",
"102837",
"112240",
"121456",
"84776",
"87082"
],
"output_link_ids": [
"486033",
"508453",
"527865",
"538540",
"542017",
"570995",
"valve-beab0b66708a4ae99b5e7f5f",
"valve-db75ca893cd246d498786475"
]
},
{
"role": "secondary",
"pipe_id": "541116",
"diameter_mm": 200.0,
"length_m": 228.424,
"start_node_id": "40239",
"emitter_node_id": "50835",
"longitude": 121.44164078044408,
"latitude": 30.982724814068543,
"candidate_orifice_areas_cm2": {
"light": 9.817477,
"medium": 19.634954,
"severe": 39.269908
},
"pressure_bindings": [
{
"device_id": "32070",
"node_id": "32070"
},
{
"device_id": "406444",
"node_id": "406444"
},
{
"device_id": "49186",
"node_id": "49186"
},
{
"device_id": "28671",
"node_id": "28671"
},
{
"device_id": "31346",
"node_id": "31346"
}
],
"flow_bindings": [
{
"device_id": "542017",
"link_id": "542017"
},
{
"device_id": "486033",
"link_id": "486033"
},
{
"device_id": "542121",
"link_id": "570995"
},
{
"device_id": "527865",
"link_id": "527865"
},
{
"device_id": "489082",
"link_id": "489082"
}
],
"boundary_valve_ids": [
"valve-89687cf53f95423c8f768d52",
"valve-9781a936671f41a9bcde8c61",
"valve-9e1f308e13df43a398f2e0d5",
"valve-a29aa553a66e490f96b443de"
],
"output_node_ids": [
"28671",
"31346",
"32070",
"40239",
"406444",
"49186",
"50835"
],
"output_link_ids": [
"486033",
"489082",
"527865",
"541116",
"542017",
"570995",
"valve-89687cf53f95423c8f768d52",
"valve-9781a936671f41a9bcde8c61",
"valve-9e1f308e13df43a398f2e0d5",
"valve-a29aa553a66e490f96b443de"
]
}
]
}
@@ -0,0 +1,37 @@
{
"version": "burst-demo-scada@1",
"project_code": "tjw_bench_tjwater2_20261105",
"normal_source_start": "2026-05-01T00:00:00+08:00",
"day_count": 17,
"noise_seed": "tjwater2-normal-scada-v1",
"cases": [
{
"demo_id": "DEMO-202605-B26-severe",
"case_key": "B26-severe",
"target_start": "2026-05-01T00:00:00+08:00",
"event_start": "2026-05-16T08:00:00+08:00",
"entry_mode": "scada_auto"
},
{
"demo_id": "DEMO-202606-B28-severe",
"case_key": "B28-severe",
"target_start": "2026-06-01T00:00:00+08:00",
"event_start": "2026-06-16T20:00:00+08:00",
"entry_mode": "scada_auto"
},
{
"demo_id": "DEMO-202607-B30-severe",
"case_key": "B30-severe",
"target_start": "2026-07-01T00:00:00+08:00",
"event_start": "2026-07-16T08:00:00+08:00",
"entry_mode": "work_order_60"
},
{
"demo_id": "DEMO-202608-B35-severe",
"case_key": "B35-severe",
"target_start": "2026-08-01T00:00:00+08:00",
"event_start": "2026-08-16T12:00:00+08:00",
"entry_mode": "work_order_120"
}
]
}
@@ -0,0 +1,140 @@
{
"version": "burst-simulation-design@1",
"suite_id": "tjwater2-v2-burst-calibration",
"project_code": "tjwater2_v2",
"project_id": "871d7538-589c-47de-a6d5-3185ccb9f0e1",
"model_expectations": {
"node_count": 92393,
"pipe_count": 91050,
"valve_count": 4489,
"flow_unit": "LPS",
"pressure_unit": "METERS",
"pressure_measurement_unit": "m",
"flow_measurement_unit": "m3/h",
"emitter_exponent": 0.5,
"hydraulic_timestep_seconds": 900,
"report_timestep_seconds": 900
},
"duration_seconds": 3600,
"event_times": [
"2026-10-08T02:00:00+08:00",
"2026-10-08T12:00:00+08:00"
],
"severity_levels": [
{"level": "light", "cross_section_fraction": 0.03125},
{"level": "medium", "cross_section_fraction": 0.0625},
{"level": "severe", "cross_section_fraction": 0.125}
],
"calibration_order": ["severe", "medium", "light"],
"selection_rule": {
"primary_pressure_drop_percent": 30,
"corroborating_pressure_drop_percent": 10,
"flow_change_percent": 20,
"consecutive_samples": 2,
"expected_sample_count": 5,
"relative_change_epsilon": 0.000001,
"normal_control_must_not_trigger": true,
"negative_monitor_pressure_allowed": false,
"choose_smallest_passing_area": true
},
"binding_validation": {
"max_pressure_distance_m": 1.0,
"max_flow_distance_m": 1.0
},
"cases": [
{"case_id": "B05", "pipe_id": "564345", "occurred_at": "2026-10-08T02:00:00+08:00"},
{"case_id": "B06", "pipe_id": "564345", "occurred_at": "2026-10-08T12:00:00+08:00"},
{"case_id": "B07", "pipe_id": "540173", "occurred_at": "2026-10-08T02:00:00+08:00"},
{"case_id": "B08", "pipe_id": "540173", "occurred_at": "2026-10-08T12:00:00+08:00"}
],
"pipes": [
{
"role": "primary",
"pipe_id": "564345",
"diameter_mm": 150,
"length_m": 249.176,
"start_node_id": "92686",
"emitter_node_id": "7427148338069072993",
"longitude": 121.469870,
"latitude": 31.005517,
"candidate_orifice_areas_cm2": {
"light": 5.522331,
"medium": 11.044662,
"severe": 22.089323
},
"pressure_bindings": [
{"device_id": "92724", "node_id": "92724"},
{"device_id": "100026", "node_id": "100026"},
{"device_id": "115039", "node_id": "115039"},
{"device_id": "107672", "node_id": "107672"},
{"device_id": "107651", "node_id": "107651"}
],
"flow_bindings": [
{"device_id": "542017", "link_id": "542017"},
{"device_id": "486033", "link_id": "486033"},
{"device_id": "542121", "link_id": "570995"},
{"device_id": "449417", "link_id": "449417"},
{"device_id": "516234", "link_id": "516234"}
],
"boundary_valve_ids": [
"valve-12b1e80612f9432da0e64485",
"valve-b6cc8d68e38f43c9b3389e04",
"valve-fa0583a6ac134aeb9513ada5",
"valve-f189692cccbd40a69dc26cca",
"valve-880f34624a444f5497faf2ca"
],
"output_node_ids": [
"92686", "7427148338069072993", "92724", "100026", "115039", "107672", "107651"
],
"output_link_ids": [
"564345", "542017", "486033", "570995", "449417", "516234",
"valve-12b1e80612f9432da0e64485", "valve-b6cc8d68e38f43c9b3389e04",
"valve-fa0583a6ac134aeb9513ada5", "valve-f189692cccbd40a69dc26cca", "valve-880f34624a444f5497faf2ca"
]
},
{
"role": "secondary",
"pipe_id": "540173",
"diameter_mm": 200,
"length_m": 162.128,
"start_node_id": "500819",
"emitter_node_id": "517120",
"longitude": 121.503495,
"latitude": 30.895889,
"candidate_orifice_areas_cm2": {
"light": 9.817477,
"medium": 19.634954,
"severe": 39.269908
},
"pressure_bindings": [
{"device_id": "535381", "node_id": "535381"},
{"device_id": "142377", "node_id": "142377"},
{"device_id": "430299", "node_id": "3186958412475415759"},
{"device_id": "121456", "node_id": "121456"},
{"device_id": "87082", "node_id": "87082"}
],
"flow_bindings": [
{"device_id": "542017", "link_id": "542017"},
{"device_id": "486033", "link_id": "486033"},
{"device_id": "542121", "link_id": "570995"},
{"device_id": "491909", "link_id": "491909"},
{"device_id": "524529", "link_id": "524529"}
],
"boundary_valve_ids": [
"valve-fd82099a578046b3bde41d63",
"valve-c9f6343550784ec9b3e97f2b",
"valve-1efb02be70174163948ab7ca",
"valve-4686c52b4d7e4d149c1823de",
"valve-aa89ee883fb4461b91eacc3d",
"valve-5e6f00e167804055830f18d8"
],
"output_node_ids": ["500819", "517120", "535381", "142377", "3186958412475415759", "121456", "87082"],
"output_link_ids": [
"540173", "542017", "486033", "570995", "491909", "524529",
"valve-fd82099a578046b3bde41d63", "valve-c9f6343550784ec9b3e97f2b",
"valve-1efb02be70174163948ab7ca", "valve-4686c52b4d7e4d149c1823de",
"valve-aa89ee883fb4461b91eacc3d", "valve-5e6f00e167804055830f18d8"
]
}
]
}
@@ -19,13 +19,18 @@ CREATE TABLE IF NOT EXISTS projects (
gs_workspace VARCHAR(100) UNIQUE NOT NULL,
map_extent JSONB,
map_config JSONB NOT NULL DEFAULT '{}'::jsonb,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
status VARCHAR(20) DEFAULT 'active' NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
CONSTRAINT projects_status_check CHECK (status IN ('active', 'inactive', 'archived'))
CONSTRAINT projects_status_check CHECK (status IN ('active', 'inactive', 'archived')),
CONSTRAINT projects_settings_object_check CHECK (jsonb_typeof(settings) = 'object')
);
ALTER TABLE projects ADD COLUMN IF NOT EXISTS map_config JSONB NOT NULL DEFAULT '{}'::jsonb;
ALTER TABLE projects ADD COLUMN IF NOT EXISTS settings JSONB NOT NULL DEFAULT '{}'::jsonb;
COMMENT ON COLUMN projects.settings IS
'项目级通用配置(JSON 对象,不存放密钥);scada_monitoring_interval_seconds 为监测任务间隔秒数。';
COMMENT ON COLUMN projects.map_config IS
'项目级地图配置(JSON);place_search 保存地名搜索启用状态、服务商、行政区编码及区域名称;保留既有图例与坐标参考配置。';
@@ -0,0 +1,4 @@
-- Queued calculations remain durable until claimed or explicitly cancelled.
DROP INDEX IF EXISTS workbench.calculation_jobs_queue_expiry;
ALTER TABLE workbench.calculation_jobs
DROP COLUMN IF EXISTS queue_expires_at;
+46
View File
@@ -0,0 +1,46 @@
BEGIN;
DO $$ BEGIN
IF current_database() NOT IN ('tjwater_v2', 'tjwater2_v2') THEN
RAISE EXCEPTION '031_scada_replays requires an explicitly verified SCADA demo project';
END IF;
END $$;
CREATE TABLE IF NOT EXISTS peripheral_sample.scada_replays (
project_id uuid NOT NULL,
replay_id uuid NOT NULL,
request_id uuid NOT NULL,
status text NOT NULL CHECK (status IN ('ready','paused','completed','stopped','failed')),
document jsonb NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (project_id,replay_id),
UNIQUE (project_id,request_id)
);
CREATE INDEX IF NOT EXISTS scada_replays_status_idx
ON peripheral_sample.scada_replays(project_id,status,created_at DESC);
CREATE TABLE IF NOT EXISTS peripheral_sample.scada_replay_batches (
project_id uuid NOT NULL,
replay_id uuid NOT NULL,
sequence bigint NOT NULL,
stage text NOT NULL CHECK(stage IN ('selected','clean_prepared','cleaned','detected')),
document jsonb NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY(project_id,replay_id,sequence),
FOREIGN KEY(project_id,replay_id) REFERENCES peripheral_sample.scada_replays(project_id,replay_id)
);
COMMENT ON TABLE peripheral_sample.scada_replays IS 'SCADA 演示回放的权威控制状态与处理检查点,不保存正式原始观测';
COMMENT ON COLUMN peripheral_sample.scada_replays.project_id IS '经认证与路由校验的所属项目';
COMMENT ON COLUMN peripheral_sample.scada_replays.replay_id IS '独立回放身份,不等同 Agent 执行编号';
COMMENT ON COLUMN peripheral_sample.scada_replays.request_id IS '创建请求的稳定幂等编号';
COMMENT ON COLUMN peripheral_sample.scada_replays.status IS '供领取和约束使用的 document.status 查询投影';
COMMENT ON COLUMN peripheral_sample.scada_replays.document IS '冻结配置、设备快照、时钟、检查点、规则状态和命令回执';
COMMENT ON COLUMN peripheral_sample.scada_replays.created_at IS '实际创建时间';
COMMENT ON COLUMN peripheral_sample.scada_replays.updated_at IS '实际控制或进度更新时间';
COMMENT ON TABLE peripheral_sample.scada_replay_batches IS 'SCADA raw 引用、清洗输出及检测阶段的恢复凭证,禁止重写 raw';
COMMENT ON COLUMN peripheral_sample.scada_replay_batches.project_id IS '所属项目';
COMMENT ON COLUMN peripheral_sample.scada_replay_batches.replay_id IS '所属回放';
COMMENT ON COLUMN peripheral_sample.scada_replay_batches.sequence IS '回放内单调批次序号';
COMMENT ON COLUMN peripheral_sample.scada_replay_batches.stage IS '已提交处理阶段';
COMMENT ON COLUMN peripheral_sample.scada_replay_batches.document IS '观测键、raw 摘要、确定的清洗结果与检测事件';
COMMENT ON COLUMN peripheral_sample.scada_replay_batches.created_at IS '实际登记批次时间';
COMMIT;
@@ -0,0 +1,54 @@
BEGIN;
DO $$
BEGIN
IF to_regclass('public.project_databases') IS NULL
OR to_regclass('public.user_project_membership') IS NULL THEN
RAISE EXCEPTION '033_metadata_project_settings must run in the project metadata database';
END IF;
END $$;
ALTER TABLE public.projects
ADD COLUMN IF NOT EXISTS settings jsonb NOT NULL DEFAULT '{}'::jsonb;
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint
WHERE conrelid = 'public.projects'::regclass
AND conname = 'projects_settings_object_check'
) THEN
ALTER TABLE public.projects
ADD CONSTRAINT projects_settings_object_check
CHECK (jsonb_typeof(settings) = 'object');
END IF;
IF EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'projects'
AND column_name = 'scada_monitoring_interval_seconds'
) THEN
IF EXISTS (
SELECT 1 FROM public.projects
WHERE scada_monitoring_interval_seconds IS NOT NULL
AND settings ? 'scada_monitoring_interval_seconds'
AND settings->'scada_monitoring_interval_seconds'
IS DISTINCT FROM to_jsonb(scada_monitoring_interval_seconds)
) THEN
RAISE EXCEPTION 'Conflicting SCADA monitoring intervals; resolve before migration';
END IF;
UPDATE public.projects
SET settings = settings || jsonb_build_object(
'scada_monitoring_interval_seconds', scada_monitoring_interval_seconds
)
WHERE scada_monitoring_interval_seconds IS NOT NULL;
ALTER TABLE public.projects DROP COLUMN scada_monitoring_interval_seconds;
END IF;
END $$;
COMMENT ON COLUMN public.projects.settings IS
'项目级通用配置(JSON 对象,不存放密钥);scada_monitoring_interval_seconds 为监测任务间隔秒数。';
COMMIT;
@@ -0,0 +1,26 @@
-- Run against each project business database during the coordinated Server cutover.
-- Historical tasks are explicitly assigned to production; retain existing scopes.
BEGIN;
SET LOCAL lock_timeout = '3s';
SET LOCAL statement_timeout = '30s';
UPDATE workbench.operational_tasks
SET document = jsonb_set(document, '{execution_scope}', '"production"'::jsonb)
WHERE NOT (document ? 'execution_scope');
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint
WHERE conrelid = 'workbench.operational_tasks'::regclass
AND conname = 'operational_tasks_execution_scope_required'
) THEN
ALTER TABLE workbench.operational_tasks
ADD CONSTRAINT operational_tasks_execution_scope_required CHECK (
(jsonb_typeof(document->'execution_scope') = 'string'
AND length(btrim(document->>'execution_scope')) > 0) IS TRUE
);
END IF;
END $$;
COMMIT;
@@ -2,7 +2,7 @@
name: appointment-follow-up
description: 核查关联预约工单的审批、提交、现场执行和反馈,分别判断预约履约与原事件闭环。
metadata:
version: "2.0.2"
version: "2.0.4"
---
# 预约任务跟进
@@ -18,13 +18,15 @@ metadata:
1. 将要求与来源工单最新版本逐项对照,区分预约、派发、到场、完成和原事件发生时间,不用创建或更新时间替代现场时间,也不因预约已过推定已经执行或失约。
2. 核对动作的审批、提交、执行状态和回执。`approved` 只表示允许提交,`submitted` 只表示已接收,`succeeded` 的建单动作只证明工单创建成功;这些状态均不证明采样、通知、抢修、冲洗或复测完成。`unknown` 由 Server 复核原执行请求,不重复下发。
3. 用 `query_peripheral_data` 查询外围工单、点位和现场观察,按来源 ID、关联动作及版本核对;目录首屏不能证明没有反馈。区分缺失、重复、旧版本和相互冲突的记录。
4. 按原目标查询位置、资产和工作前后的监测。核实设备与模型要素、时间覆盖、单位和数据质量;缺测不是零值,局部正常不证明整项工作完成。
4. 按原目标查询位置、资产和工作前后的监测。地点和资产明确后展示正在核查的工作位置;取得监测对照后展示核查点位,并画出实际取得的工作前后趋势或对比图,核对现场发生时间与数据覆盖,不能把计划时间当实际执行分界;已有影响结果时展示其真实几何或节点覆盖示意。核实设备与模型要素、时间覆盖、单位和数据质量;缺测不是零值,局部正常不证明整项工作完成。
5. 将现场描述、来源系统附件或照片说明、监测变化和原工作要求交叉核对。没有图像查看能力时不声称已检查照片内容。每项要求标为已核验、待核验或不适用并给出依据;状态与证据冲突时明确列出。
## 结果与后续
报告分别说明原预约要求、审批及派发事实、已核实的现场工作和效果、冲突或缺口、当前责任事项,以及它与原事件的关系。等待回执不等于调查完成,更不等于现场工作完成。
位置、资产、监测和已有影响结果随核查步骤同步展示,遵循后台地图与图表规则,不用最终报告代替过程展示;单个数值或无可比数据时用文字、指标或表格,不为凑图新增业务分析。无数据或展示失败时说明具体限制,调查不等待浏览器。
只有证据支持新的具体工作时才提议动作。先核对已有有效动作和关联工单,优先沿原工单的反馈与人工修改流程推进;不要因缺少一项反馈就新建催办或复查工单。新增人工工作使用关联的 `work_order_type=general` 草稿,写明地点、内容、依赖、执行前核验和回执要求。
本流程不自动发起隔离停水、污染水排放或设备命令。发现新的爆管或水质风险时,列出证据、紧迫事项和建议,由原事件的合法调查继续专业分析;不要改写当前任务类型或递归创建同类事件。
@@ -0,0 +1,53 @@
---
name: burst-event-handling
description: 根据爆管上报工单处理事件,完成资产关联、隔离方案、影响说明、处置建议与反馈复核。
metadata:
version: "1.1.1"
---
# 爆管事件处理
适用于工单触发的 `workflow=burst_report`。以工单上报的事件位置和资产线索开展处置,依次完成下列步骤;每一步给出实际结果或具体待补项,已有证据可直接复用。SCADA 异常发现使用独立的 scada-anomaly 流程。
正文、表格、图表及引用时间统一按Asia/Shanghai(UTC+8)阅读。工具返回的+08:00时间保持原值;UTC来源转换后同步更新日期与窗口,例如2026-07-16T00:00:00Z对应7月16日08:00。结构化查询与sample_times保留明确时区,跨日按转换后的日期展示。
## 1. 接收与核对
读取当前任务、原工单及版本、发生时间、上报位置、描述、现场反馈、既有报告与动作。工单按 system_data 读取,区分上报内容与已核实事实,分析证据按各自来源说明。后续轮次优先处理新增事实和上轮待办。
## 2. 地点核验与资产关联
已有管段或节点编号时,用 get_element_properties 核对资产;已有坐标时直接使用,文字地址通过 geocode 解析后用 query_nearby_elements 获取附近资产。结合属性、道路、连接关系和现场记录确定处理对象,使用 get_node_connections 补充连接依据。
在地图展示上报位置和关联资产,标明关联依据及现场确认状态。存在多个合理对象时保留候选及选择理由;关键位置需要人工补充时,先完成独立工作,再用 ask_task_user 提问。
## 3. 隔离方案
对关联资产调用 analyze_valve_isolation,保留 isolatable、must_close_valves、optional_valves、affected_nodes、supply_affected_nodes 及计数。单个候选分别分析;联合事故按真实联合场景提交。已知失效阀门通过 disabled_valves 评估替代边界;现场可操作性列为执行前核验条件。
展示方案阀门和直接隔离、供水失源两类节点;节点覆盖使用 generate_node_coverage,再通过 publish_task_presentation 发布地图请求。取得初步隔离方案后继续影响分析和处置建议。
## 4. 影响说明
报告固定包含“影响说明”,逐项列出对象、范围、时间、单位、依据和实际或预计的性质:
- **事件当前影响**:核查相关测点在事件窗与正常对照窗的压力、流量变化,给出已有数据覆盖的异常对象、持续时间和恢复情况。SCADA 用于核验事件影响,数据缺口与已完成的资产、拓扑结果分别记录。
- **隔离方案影响**:分别说明 affected_nodes 对应的直接隔离范围,以及 supply_affected_nodes 对应的新增失去水源连接范围,保留各自计数和地图覆盖。
- **水力影响**:输入具备且工具已授权时,用 analyze_valve_closure 评估关阀后的压力、流量和低压范围。起算时间、阀门组合、模型边界及输出对象明确可比时给出前后差异;其余情况说明当前模型结果和待补条件。已有可靠破口参数时可按需补充事件情景模拟。
- **用户与保障影响**:有用户、设施或区域关联资料时,列出受影响用户、重要设施和保障需求。仅有模型节点资料时报告节点与区域范围,将用户数量列为待核实。
现场实际停水、模型预计低压、拓扑失源和展示覆盖分别标注。漏量、累计漏水量使用有来源的方法与输入,列明单位及覆盖;其余情况记录待补依据。缺少可选模拟参数时,继续形成已有依据的隔离和保障建议。
## 5. 处置建议与报告
先保存 interim/concluded 阶段报告,再按需要提出核验、通知、供水保障、隔离与抢修、恢复复测等动作草稿。关联现场工作采用 work_order_type=general,保留原爆管上报工单作为事件来源。草稿标题写“地点+动作”,正文以三条概括任务、执行条件和回执要求;审批与实际执行按 Server 流程完成。
报告固定给出事件概况、资产关联、隔离方案、影响说明、处置建议和待办。摘要以150字以内为目标,指标和资产明细放表格,证据展开保留来源。待办应指向具体事实、参数或动作。已完成本轮可执行分析且依赖结果已读取时发布 final,现场处理进展由后续反馈接续。
## 6. 反馈与闭环复核
读取摘要中的 handling_context。waiting_work_order 表示已派单、等待有效完工反馈,创建回执的 completed_at 只证明创建动作完成。现场处理和恢复供水以关联工单的 completed 反馈为准;returned_at 是现场反馈的业务时间,演示时不得换成电脑当前时间。
waiting_scada 表示恢复供水和监测复核阶段,以 returned_at 为反馈时刻 T,读取严格晚于 T 的最近一个 SCADA 采样时刻作为复核来源,说明测点、实际采样时间、读数、单位和正常范围;不等待额外周期,不使用 T 之前、T 当刻或更晚的正常数据替代。后台按事件前15天同期正常包络核验该时刻。最近时刻缺测或异常时保留实际问题,不宣称复核成功。保存本轮阶段成果,等待后续数据接续,不反复派发同一工单。
verification_ready 时读取 verification_evidence_id 对应的证据与快照,核对工单反馈、T、最近采样时刻和测点,展示该时刻读数与正常范围;具备真实多时刻数据时再展示曲线,发布 final/concluded 报告并引用该证据。运行完成后 Server 才完成任务并解除关联告警。原事件在来源系统的行政结案仍由有权限人员确认。
@@ -1,41 +1,15 @@
---
name: burst-report
description: 调查供水爆管上报,形成有依据的候选位置、隔离阀门、影响范围和后续处置建议。
description: 爆管业务流程的历史入口索引,运行时按事件来源选择独立 Skill。
metadata:
version: "2.0.3"
version: "2.0.11"
---
# 爆管上报处置
# 爆管业务流程入口
目标是形成有依据的候选位置、隔离方案、影响判断和可审批处置,并在后续轮次根据真实反馈复核事件闭环。区分候选方案、现场确诊和事件闭环;后两者不能提前阻断当前可完成的分析。演练数据可以用于比较,但必须标注 `synthetic`,不能证明生产故障。
Server 根据绑定任务的 workflow 和 replay_context 分发当前流程:
## 接续与证据
- 工单触发的 burst_report:使用 burst-event-handling,依次完成接收核对、地点与资产关联、隔离方案、影响说明、处置建议和反馈复核。
- SCADA 异常及带 replay_context 的历史回放任务:使用 scada-anomaly,完成冻结数据核验、诊断、候选定位与隔离影响分析。
读取当前任务、事件时刻、地点、来源、已有报告、证据、动作和反馈。新一轮优先检查新增事实和上轮未决项,按时间、来源、对象及输入比较新旧结果,不按 ID 推断先后,也不因原工单未更新就断言没有新证据。保留本流程名称和版本。
## 定位与影响分析
根据现有线索自主选择必要步骤:
1. 核对已有管段或节点。只有地点描述时,先解析位置,再查询附近资产、属性和连接。空间邻近只支持候选,不能单独证明故障。
2. 核对 SCADA 设备与模型节点映射,用 `query_scada_readings` 或 `query_element_history` 查询事件前后及正常对照。先用少量对象和合适时间窗,确认返回时间实际覆盖事件;必要时缩窗或翻页。保留单位、缺测、真实零值和覆盖限制。
3. 有异常观测时评估爆管定位输入。`run_burst_location` 必须有 `burst_leakage`(m³/s);先查询已有估计及方法依据,或结合边界流量、用水和储量变化判断能否估计。不能把单表增量、压力下降或任意数值直接当漏量。
4. 有依据的漏量估计或明确的情景假设可用于条件定位,须说明来源、方法和不确定性。`pressure_scada_ids` 使用模型节点 ID;监测模式读取 SCADA,模拟模式须引用已有分析运行。压力点和历史数据满足工具要求时可选 `run_burst_detection`,但侦测不代替定位,评价值也不表述为概率。缺少定位参数时说明已查来源和影响,继续其他可独立完成的分析。
5. 对已确认或有依据的候选资产使用 `analyze_valve_isolation` 评估阀门隔离,无需等待现场确诊。逐候选比较;多个事故 ID 表示联合场景。保留可隔离性、必关阀、可选阀及受影响节点。未知阀门状态不能写成可用,不可隔离或空结果也不能解释为零影响;已知阀门失效时评估替代边界。
6. 输入具备时用 `analyze_valve_closure` 分析关阀水力影响,核对阀门组合、起算时间、时长和输出对象。区分拓扑范围、水力影响和有依据的用户影响,不虚构户数,不用局部结果代表全网。
查询错误、权限拒绝或接口故障不是缺数证据。必要分析无法执行时,说明失败步骤和已经完成的部分,不借有限摘要宣布调查完成。
## 处置与报告
报告分别给出位置候选及依据、隔离阀门及可用性、替代方案、拓扑与水力影响、用户影响边界,以及处置和闭环缺口。只有初步结果时明确范围,不宣称已经定位、确诊或闭环。按需展示候选、阀门、监测和影响,但展示不替代证据。
形成具体建议后,先保存阶段结论,再提议必要动作。人工核验、通知、定位隔离、抢修、适用的清洗复测和恢复回访使用关联的 `work_order_type=general` 草稿;原爆管事件保持独立。草稿写明位置、任务、依赖、执行前核验和回执要求。没有通知适配器时明确由人工实施。设备命令仅用于真实可控目标,审批和执行由 Server 控制。
抢修或复测尚未发生不阻止有边界的初步方案。现场阀门状态应写为执行前核验条件;工单创建、审批或派发均不证明现场完成。
## 闭环复核
后续轮次核对实际故障资产和原因、隔离实施、抢修记录、适用的清洗与水质复测、恢复供水及恢复后监测、用户通知回访、未决动作和遗留风险。每项标明已核验、待核验或不适用及依据。
证据齐全时报告具备结案条件,由来源系统的有权限人员确认关闭。原工单未结案时写明待确认,读取其新版本后才能认定闭环;误报结案也须有足够的排除证据和人工确认。
通过 get_task_context 的 section="skill" 读取实际流程,保留返回的 name、version、content_sha256。工具范围由绑定任务的服务端策略及 Agent 会话策略共同确定。
@@ -0,0 +1,52 @@
---
name: scada-anomaly
description: 从 SCADA 指标异常开展数据核验、爆管诊断、候选定位与影响分析。
metadata:
version: "1.0.13"
---
# SCADA 数据异常调查
读取任务与 replay_context,核对设备、单位、异常时刻、knowledge_cutoff 和 batch_sequence。使用本轮已提交的 SCADA 观测开展调查,同一持续事件沿用已有调查。证据来源字段用于记录数据取得方式,不构成停止分析的条件。
报告与图表的阅读时区统一为Asia/Shanghai(UTC+8)。工具返回的+08:00时间按原值使用;Z时间换算后同步更新日期、历史系列名和横轴标签,例如2026-07-16T00:00:00Z对应7月16日08:00。sample_times记录相同真实时间点并使用+08:00偏移,查询与冻结边界按带时区的时间点比较。图表标题、横轴和系列名只写日期、时刻与单位,不写“上海时间”“Asia/Shanghai”或“UTC+8”;横轴命名为“时间”。
## 数据核验与诊断
任一已配置指标异常即进入调查,包含压力、流量、水质规则和压力模型信号。先核验数据质量与测点,再判别原因;模型或组合指标有缺口时,继续核查已发现的单项异常。
通过 query_scada_readings 省略device_ids读取冻结异常窗内全部SCADA设备的完整属性和授权观测,覆盖压力、流量、水质等全部类型;正常对照也采用全量读取。Server自动读完所有设备页和观测页,完整返回devices、readings、coverage,结果仍受8MiB整份上限约束,超限明确报错后按时间分段,不能丢弃设备。query_complete表示查询已读完,缺测与无有效值另行判断。查询结束时间止于knowledge_cutoff,数据批次止于batch_sequence;冻结范围外设备单列,不回退实时数据。按需核查开始前15天基线,复用已有完整监测回执。结合monitored_value、cleaned_value和质量标记判断变化,可靠采集的低压与水质变化保留为业务异常。
全量响应中的devices包含完整目录属性,readings包含实际读数,coverage列出全部查询设备、各设备读数数量、缺测和冻结范围外设备。不能只读取已标记、异常或代表设备后推断全网状态。单独读取list_scada_devices时,默认每页100个,持续翻页到page.has_more=false;属性缩略不等于已读取完整设备信息。全量读取后可显式指定device_ids做局部复核,局部读数仍按page.has_more接续,不能替代全量输入。最终报告引用覆盖本轮knowledge_cutoff和batch_sequence的全量证据,Server会拒绝仅有局部或旧批次证据的final。
分别说明压力异常模型得分、压力绝对下降及变化率、已配置出厂边界流量变化,再说明联合证据支持程度。使用实际权重、嫌疑门槛、动态阈值、参与测点与各自基线。综合分用于爆管嫌疑判断;得分越低越异常,目标告警与参考日分别解释。水质等其他异常按实际证据选择原因分支。
## 候选定位与隔离
综合分达到嫌疑门槛,或压力与模型证据支持高嫌疑时,继续候选定位。流量变化和现场确认作为证据维度,定位输入按当前工具要求准备。
检查摘要或 skill 响应顶层的 location_capability。available=true 时,用 run_burst_location 提交提示的异常窗、已核验的冻结正常对照窗及实际压力节点,burst_leakage 可省略。读取返回的候选管段与相似度,优先按 similarity_percent 展示百分比;只有 similarity 时,按0–1小数换算。取得候选后继续地图展示、隔离边界、节点覆盖与影响分析,完成报告后再收尾。其他定位路径使用有依据的漏量估计或明确情景假设。缺少定位输入时记录待补项,继续独立的资产、空间与拓扑核查。
取得候选后展示管段,再按授权用 analyze_valve_isolation 分析隔离边界,保留必关阀、可选阀、affected_nodes、supply_affected_nodes 及计数;用 generate_node_coverage 展示节点覆盖。候选与拓扑建议标注为待现场核验方案。最终报告前核对定位调用及返回结果,调用失败记录具体错误与输入缺口。
## 对比展示与影响
先查询并发布截至本轮knowledge_cutoff或异常时刻t的[t-3小时,t]当前趋势总览,覆盖本轮全部已标识SCADA设备,包括地图定位、等级标记及表格列出的目标设备。按指标和单位分图,同组各设备各占一条当前序列,系列名带device_id;每图最多8组,超过时分组发布,各组使用不同且稳定的chart_id。缺测填null,完全无读数的设备明确列为缺测并说明查询窗口,不能静默省略。压力和流量分别发布图表,包含正常与异常测点;某指标无设备或无有效读数时明确说明。每条当前序列保留三小时窗口内全部已取得采样点,不能用用于判断突变的[t-采样周期,t]相邻两点代替三小时趋势。15分钟数据正常应有13个时刻;缺测保留对应空槽并说明实际覆盖。收尾前逐项核对目标设备、查询覆盖、已发布图表和缺测清单,不用少量代表设备替代全量总览。图表“数据”视图沿用同一完整序列,报告按指标附测点、单位、窗口、有效点数和缺测说明表格。
定位输入具备后继续定位。压力、流量按设备另发七天同期复合图,不占用或替代全量当前总览。chart_type=time_comparison:x_data是UTC+8的HH:mm或HH:mm:ss共享槽,不拼接历史日期;当前[t-3小时,t]用series.type=line,前7天每天t-30分钟、t-15分钟、t用scatter。15分钟数据13槽,历史七组仅末3槽有值。每组sample_times与data等长,保留真实带时区日期;有效值与对应槽的本地时刻一致,缺测值和时间均null。当前曲线共用窗口,历史每组取此前某天对应窗口,跨午夜顺序排列;普通图用line/bar,坐标轴标单位。七天按t的本地日期回退,不含当天。按实际周期保留缺口,按需翻页并报告参考点数;基线未冻结、读取失败或缺测时说明并保留当前趋势,不退化为前一天或相邻两点。
影响说明列出异常前参考、异常期间和已有恢复观测中的压力、流量或水质变化、持续时间、涉及测点及已核实资产。可比对象、单位、时段与运行条件下给出变化量,分别标明实际观测、模型预测和拓扑隔离。停水范围与漏量引用相应拓扑、水力、流量平衡或现场依据;恢复观测、用户关联或现场证据缺失时列为具体待核实项。
## 范围与报告
本轮时序分析保持任务冻结窗口,后续播放形成的观测由相应轮次处理。资产属性、空间查询和拓扑隔离使用项目当前模型,注明模型时态;水力工具按已开放的冻结输入能力调用。定位调用执行权限、设备与冻结观测校验;基线或覆盖缺口按实际返回记录。
报告给出观测事实、规则信号、原因候选、定位与隔离结果、影响说明和建议。摘要以150字以内为目标,测点明细用表格、完整依据留在证据。候选、相似度和拓扑结果具备时,完成相应分析、展示与报告后再收尾;现场核验、审批和维修列为后续待办。现场工单与设备操作遵守服务端动作权限,本轮分析完成与现场事件闭环分别说明。
需要现场核验时,先发布本轮outcome=concluded的调查报告,再调用propose_task_action提出kind=create_work_order、work_order_type=general的关联预约工单草稿,引用候选与影响证据,写明核验对象、内容和回执要求。带replay_context的任务同样可提出草稿,描述中注明历史异常时刻及回放来源;草稿须由有权限的用户编辑、确认后发布,后续派发与反馈进入预约任务。草稿待确认时说明等待人工处理,不将调查报告完成写成现场处置完成。回放任务不支持设备命令。
## 完工后的恢复复核
收到关联工单的completed反馈后,先读取get_task_context摘要中的handling_context,核对实际完工业务时间returned_at,不以工单创建、预约或电脑当前时间代替。Server通过现有恢复监测读取项目时序库中严格晚于returned_at的最近一帧及事件前15天同相位对照,独立保存恢复证据;例如13:00完工、15分钟采样时核对13:15。这条恢复链路适用于回放来源工单,不要求推进回放,原异常knowledge_cutoff与batch_sequence保持不变。
phase=waiting_work_order时等待现场完工;phase=waiting_scada时查看last_check,分别说明尚无后续采样、基线或测点缺项、实际仍异常,等待监测复核,不将原回放查询越界解释为恢复数据不存在。phase=verification_ready时,用get_task_evidence读取verification_evidence_id的完整恢复证据,比较实际样本、正常范围与现场回报,最终报告同时引用原异常全量证据与当前恢复证据。恢复数据已由Server保存,不用query_scada_readings越过原冻结上限重复查询,也不因该上限再建复测工单;确有独立现场工作时才提议新动作。完工时间修订后使用新返回的证据编号,旧证据仅作历史。正常读数不替代阀位、漏点照片等现场事实,报告分别说明监测恢复、现场待核验项与事件闭环。
@@ -2,12 +2,12 @@
name: water-quality-complaint
description: 调查供水水质投诉,核查投诉与监测覆盖,评估候选原因、影响和采样处置建议。
metadata:
version: "2.0.2"
version: "2.0.4"
---
# 水质投诉处置
目标是说明投诉和异常的实际覆盖、可能原因及依据、候选影响范围、采样或处置建议和后续复核条件。区分感官投诉、实测异常、污染源假设和已确认问题;不预设异常来自上游或二次供水。演练结果须标注 `synthetic`,不能证明生产风险。
目标是说明投诉和异常的实际覆盖、可能原因及依据、候选影响范围、采样或处置建议和后续复核条件。区分感官投诉、实测异常、污染源假设和已确认问题;不预设异常来自上游或二次供水。工单按 `system_data` 读取,分析证据按各自来源说明。
## 接续与证据
@@ -18,17 +18,17 @@ metadata:
根据当前证据选择必要步骤:
1. 核实投诉位置及相近时段的同类投诉,用 `query_peripheral_data` 按实际时间、位置和来源标识核对外围工单或观察。目录首屏不能证明没有其他投诉;未知点位先查目录,不猜编号或把演示点当真实资产。
2. 解析未知地址,查询附近资产、属性和连接,区分上游、管网、末梢、二次供水和用户端的实际覆盖。拓扑连接不等于实际流向,方向判断需要同窗模型或监测依据。
3. 核对 SCADA 指标、单位和模型映射,用 `query_scada_readings` 或 `query_element_history` 查询投诉前后及对照,检查时间覆盖、在线状态、漂移、长期缺测和质量标识。缺测不是零值,上游正常不能排除末梢问题,压力正常不能证明水质合格。
2. 解析未知地址,查询附近资产、属性和连接,区分上游、管网、末梢、二次供水和用户端的实际覆盖。地点明确后展示投诉位置,取得关联资产后展示正在核查的管段或节点。拓扑连接不等于实际流向,方向判断需要同窗模型或监测依据。
3. 核对 SCADA 指标、单位和模型映射,用 `query_scada_readings` 或 `query_element_history` 查询投诉前后及对照,检查时间覆盖、在线状态、漂移、长期缺测和质量标识。同步展示核查点位,将实际取得的水质、压力等指标画成趋势或对比图,不同单位分图。缺测不是零值,上游正常不能排除末梢问题,压力正常不能证明水质合格。
4. 综合投诉分布、监测、供水关联和实际采样,列出候选原因、支持证据、反证和区分原因所需的核验。未证实污染源时保留假设,不把感官描述换算成浓度。水质限值只能引用已取得且适用的依据,不能自行承诺安全饮用。
5. 有合理目标管段或节点时,可用 `analyze_valve_isolation` 和 `analyze_valve_closure` 评估候选隔离及关阀影响,无需等待污染确诊,但不能据此直接执行停水。逐候选区分可隔离性、必关阀、可选阀、受影响节点和未知阀门状态。拓扑隔离区、水力影响、污染范围和用户影响分别说明;无用户资料时不虚构户数。
6. 只有假设源节点、浓度及必要模型输入有依据时,才用 `run_contaminant_simulation` 正向比较传播;它不是源头反演,未模拟范围不能宣布安全。具备实际排放节点、流量和前置条件时才用 `analyze_flushing` 评估冲洗。冲洗模拟不证明消毒合格,排放位置和承接条件仍需现场核验与审批。
5. 有合理目标管段或节点时,可用 `analyze_valve_isolation` 和 `analyze_valve_closure` 评估候选隔离及关阀影响,无需等待污染确诊,但不能据此直接执行停水。逐候选区分可隔离性、必关阀、可选阀、受影响节点和未知阀门状态。取得影响节点后展示覆盖示意,形成或调整关阀方案时展示对应阀门。拓扑隔离区、水力影响、污染范围和用户影响分别说明;节点覆盖面不能当作污染传播边界,无用户资料时不虚构户数。
6. 只有假设源节点、浓度及必要模型输入有依据时,才用 `run_contaminant_simulation` 正向比较传播;它不是源头反演,未模拟范围不能宣布安全。具备实际排放节点、流量和前置条件时才用 `analyze_flushing` 评估冲洗。取得模拟结果后,按实际返回的对象、时刻和指标展示浓度、压力或流量曲线;同口径的隔离、冲洗方案可画对比图,标明情景假设和局部覆盖。冲洗模拟不证明消毒合格,排放位置和承接条件仍需现场核验与审批。
缺少源参数不阻止已有证据支持的采样、候选隔离或人工告知建议。查询错误、权限拒绝或接口故障属于执行异常,不能写成无数据或未见异常。
## 处置与报告
报告分别说明投诉与监测覆盖、候选原因和未排除路径、隔离及影响、通知保障、采样或污染水处置,以及复测与恢复条件。每项给出实际结果或经查询确认的限制;区分当前风险判断与尚未发生的未来处置。按需展示投诉、监测、候选隔离和影响,但展示不替代证据。
报告分别说明投诉与监测覆盖、候选原因和未排除路径、隔离及影响、通知保障、采样或污染水处置,以及复测与恢复条件。每项给出实际结果或经查询确认的限制;区分当前风险判断与尚未发生的未来处置。投诉位置、监测、候选隔离和影响按实际步骤同步展示,遵循后台展示规则,不用报告定位链接代替过程展示。无数据或展示失败时说明具体限制;展示不替代证据,也不要求等待浏览器。
形成具体建议后,先保存阶段结论,再提议关联的 `work_order_type=general` 草稿。写明实际位置、采样或处置目标、时间要求、依赖、现场待核验条件和回执要求。通知、采样、清洗冲洗与复测按依赖合理组织,不机械逐节点建单,也不重复已有有效动作。没有客户通知适配器时明确由人工实施;调查消息不能当作用户通知。设备命令仅针对真实可控目标,审批与执行由 Server 控制。
+4
View File
@@ -161,9 +161,13 @@ class UpdateWorkOrder(Model):
clear_scenario_code: StrictBool = False
feedback_requirements: list[Text] | None = None
progress_note: Text | None = None
occurred_at: WireDateTime | None = None
@model_validator(mode="after")
def valid_update(self):
if self.occurred_at is not None:
if self.occurred_at.tzinfo is None or not self.progress_note:
raise ValueError("occurred_at requires a timezone and field feedback")
changed = set(self.model_fields_set) - {"request_id", "expected_revision"}
if not changed:
raise ValueError("at least one work order field must be updated")
+5 -5
View File
@@ -362,7 +362,7 @@ class PostgresStore:
with self.connection(readonly=True) as conn:
self._available(conn, who)
return self._managed_document(conn, who, "work_orders", "work_order_id", work_order_id,
"WORK_ORDER_NOT_FOUND", "测试工单不存在")
"WORK_ORDER_NOT_FOUND", "工单不存在")
def create_work_order(self, who: Identity, command: CreateWorkOrder) -> dict[str, Any]:
def write(conn, _):
@@ -382,11 +382,11 @@ class PostgresStore:
command: UpdateWorkOrder) -> dict[str, Any]:
def write(conn, _):
document = self._managed_document(conn, who, "work_orders", "work_order_id", work_order_id,
"WORK_ORDER_NOT_FOUND", "测试工单不存在")
"WORK_ORDER_NOT_FOUND", "工单不存在")
ensure(document["revision"] == command.expected_revision, "REVISION_CONFLICT", "工单版本已变化,请重新读取", 409)
changes = command.model_dump(mode="json", exclude_unset=True,
exclude={"request_id", "expected_revision", "clear_assignee", "clear_scheduled_at",
"clear_location", "clear_scenario_code", "progress_note"})
"clear_location", "clear_scenario_code", "progress_note", "occurred_at"})
if command.clear_assignee:
changes["assignee"] = None
if command.clear_scheduled_at:
@@ -409,7 +409,7 @@ class PostgresStore:
) + "。", "updated"
else:
title, detail, kind = "工单信息确认", "工单信息已确认,未记录新的现场进展。", "updated"
document["activity"].append({"sequence": revision, "occurred_at": now,
document["activity"].append({"sequence": revision, "occurred_at": command.occurred_at.isoformat() if command.occurred_at else now,
"actor": who.user_id, "kind": kind,
"title": title, "detail": detail})
document.update(revision=revision, updated_at=now,
@@ -421,7 +421,7 @@ class PostgresStore:
command: DeleteResource) -> dict[str, Any]:
def write(conn, _):
document = self._managed_document(conn, who, "work_orders", "work_order_id", work_order_id,
"WORK_ORDER_NOT_FOUND", "测试工单不存在")
"WORK_ORDER_NOT_FOUND", "工单不存在")
ensure(document["revision"] == command.expected_revision, "REVISION_CONFLICT", "工单版本已变化,请重新读取", 409)
revision = document["revision"] + 1
deleted_at = datetime.now(timezone.utc).isoformat()
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env python3
"""Compare replay-rule thresholds against frozen hydraulic outputs without rerunning hydraulics."""
from __future__ import annotations
import argparse
import json
import os
import sys
from datetime import datetime
from pathlib import Path
from types import SimpleNamespace
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--calibration-dir", required=True, type=Path)
parser.add_argument("--thresholds", nargs="+", type=float, default=[3, 5, 10, 20, 30])
args = parser.parse_args()
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
os.umask(0o077)
from scripts.benchmark.calibration import digest, pressure_preview, save_json
from scripts.benchmark.pressure_thresholds import learn_pressure_thresholds
from scripts.benchmark.simulation_design import BurstSimulationDesign
folder = args.calibration_dir.resolve()
source = folder / "calibration.json"
report = json.loads(source.read_text())
if report.get("status") != "completed" or any(not 0 < value <= 100 for value in args.thresholds):
raise ValueError("a completed calibration and thresholds in (0,100] are required")
design = BurstSimulationDesign.model_validate(report["design"])
pipes = {pipe.pipe_id: pipe for pipe in design.pipes}
cases = {case.case_id: case for case in design.cases}
index_by_time = {at: index for index, at in enumerate(design.event_times)}
output = {"version": "burst-threshold-sensitivity@1", "calibration_sha256": digest(source),
"scope": "pure cleaner/replay-rule sensitivity, not Agent or worker acceptance",
"thresholds_percent": args.thresholds, "rows": [], "summary": [],
"adaptive_fit": learn_pressure_thresholds([], ["calibration-history"], design.event_times[0]),
"adaptive_note": "this package has four warmup samples; fitting needs 1440 pre-event normal samples per device",
"absolute_drop_gate_applied": False}
cache = {}
controls = {}
for entry in report["cases"]:
case = cases[entry["case_id"]]
pipe = pipes[case.pipe_id]
index = index_by_time[case.occurred_at]
if index not in cache:
cache[index] = (json.loads((folder / f"normal-{index}.json").read_text()),
json.loads((folder / f"warmup-{index}.json").read_text()))
normal, warmup = cache[index]
burst = json.loads((folder / f"{case.case_id}-{entry['severity']}.json").read_text())
for threshold in args.thresholds:
rule = SimpleNamespace(primary_pressure_drop_percent=threshold,
consecutive_samples=design.selection_rule.consecutive_samples)
key = (pipe.pipe_id, index, threshold)
if key not in controls:
controls[key] = pressure_preview(pipe, case.occurred_at, report["duration_seconds"], 900,
warmup["nodes"], normal["nodes"], rule)
event = pressure_preview(pipe, case.occurred_at, report["duration_seconds"], 900,
warmup["nodes"], burst["nodes"], rule)
at = event["first_signal_at"]
output["rows"].append({"case_id": case.case_id, "pipe_id": pipe.pipe_id,
"severity": entry["severity"], "threshold_percent": threshold,
"event_signal": event["raised"], "first_signal_at": at,
"delay_minutes": (datetime.fromisoformat(at) - case.occurred_at).total_seconds() / 60 if at else None,
"normal_false_positive": controls[key]["raised"]})
for threshold in args.thresholds:
rows = [row for row in output["rows"] if row["threshold_percent"] == threshold]
normal = [value for key, value in controls.items() if key[2] == threshold]
output["summary"].append({"threshold_percent": threshold, "event_cases": len(rows),
"event_signals": sum(row["event_signal"] for row in rows),
"unique_normal_controls": len(normal),
"normal_false_positives": sum(row["raised"] for row in normal)})
path = folder / "threshold-sensitivity.json"
save_json(path, output)
for summary in output["summary"]:
print(summary)
print(f"threshold sensitivity complete: {path}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+1
View File
@@ -0,0 +1 @@
"""Offline benchmark preparation; never imported by the application runtime."""
+226
View File
@@ -0,0 +1,226 @@
"""File-isolated hydraulic calibration; database connections are read-only."""
from __future__ import annotations
import ctypes
import hashlib
import json
import math
import os
import re
import subprocess
from datetime import datetime, timedelta
from pathlib import Path
from uuid import NAMESPACE_URL, uuid5
from zoneinfo import ZoneInfo
from scripts.benchmark.simulation_design import PipeCalibration, SelectionRule
def digest(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def save_json(path: Path, value: dict) -> None:
temporary = path.with_suffix(".pending")
with temporary.open("x", encoding="utf-8") as output:
os.chmod(temporary, 0o600)
json.dump(value, output, ensure_ascii=False, indent=2, allow_nan=False, default=str)
output.write("\n")
temporary.replace(path)
def select_calibrated_area(
pipe: PipeCalibration, passed_levels: list[str], case_results: list[dict], retain_weak: bool,
) -> dict:
selected = passed_levels[-1] if passed_levels else None
soft_blockers = {"insufficient_consecutive_pressure_and_corroboration", "event_rule_did_not_trigger"}
physically_valid = bool(case_results) and all(
not set(case["blockers"]) - soft_blockers for case in case_results
)
status = "calibrated_candidate" if selected else (
"weak_signal_candidate" if retain_weak and physically_valid else "unsuitable"
)
return {"status": status, "severity": selected,
"area_cm2": pipe.candidate_orifice_areas_cm2[selected] if selected else None}
def configure_inp(
source: str,
start: datetime,
duration: int,
*,
emitter: tuple[str, float] | None = None,
pda_options: dict[str, str] | None = None,
) -> str:
"""Apply the burst service's options to a file, including the zero-leak control."""
if start.tzinfo is None:
raise ValueError("simulation start must include a timezone")
start = start.astimezone(ZoneInfo("Asia/Shanghai"))
updates = {
"TIMES": {"DURATION": f"{duration // 3600}:{duration % 3600 // 60:02d}:00",
"PATTERN START": start.strftime("%H:%M:%S"),
"REPORT START": "0:00:00", "STATISTIC": "NONE"},
"OPTIONS": {**(pda_options or {}), "DEMAND MODEL": "PDA", "REQUIRED PRESSURE": "10.0000"},
"REPORT": {"NODES": "NONE", "LINKS": "NONE"},
}
section = None
output = []
pending = {}
for line in source.splitlines():
match = re.fullmatch(r"\s*\[([^]]+)\]\s*", line)
if match:
output.extend(f"{key} {value}" for key, value in pending.items())
section = match[1].upper()
pending = dict(updates.get(section, {}))
output.append(line)
if section == "EMITTERS" and emitter:
node, area = emitter
coefficient = 0.65 * area / 10000 * math.sqrt(19.6) * 1000
output.append(f"{node} {coefficient:.12g}")
continue
tokens = line.split(";", 1)[0].upper().split()
if any(tokens[:len(key.split())] == key.split() for key in updates.get(section, {})):
continue
output.append(line)
output.extend(f"{key} {value}" for key, value in pending.items())
if "[EMITTERS]" not in source.upper() and emitter:
raise ValueError("exported model has no emitter section")
return "\n".join(output) + "\n"
def run_file(inp: Path, node_ids: list[str], link_ids: list[str], *, timeout: int) -> dict:
from app.infra.epanet.epanet import Output
executable = Path(__file__).resolve().parents[2] / "app/infra/epanet/linux/runepanet"
env = os.environ.copy()
env["LD_LIBRARY_PATH"] = f"{executable.parent}:{env.get('LD_LIBRARY_PATH', '')}"
report_path, binary = inp.with_suffix(".rpt"), inp.with_suffix(".bin")
result = subprocess.run(
[str(executable), str(inp), str(report_path), str(binary)],
env=env, capture_output=True, text=True, timeout=timeout, check=False,
)
report = report_path.read_text(encoding="utf-8", errors="replace") if report_path.exists() else ""
warnings = [line.strip() for line in report.splitlines()
if re.search(r"^\s*(?:warning\b|error\s+\d+\b)|system\s+(?:is\s+)?unbalanced|cannot solve hydraulic",
line, re.IGNORECASE)]
summary = {"exit_code": result.returncode, "warnings": warnings,
"inp_sha256": digest(inp), "report_sha256": digest(report_path) if report_path.exists() else None}
if result.returncode or not binary.exists():
return {**summary, "status": "failed"}
reader = Output(str(binary))
names = reader.element_name()
summary.update(status="completed", units=reader.units(), times=reader.times(), nodes={}, links={})
for kind, selected, attributes, function in (
("nodes", node_ids, ["demand", "head", "pressure", "quality"], reader._lib.ENR_getNodeResult),
("links", link_ids, ["flow", "velocity", "headloss", "quality", "status", "setting", "reaction", "friction"],
reader._lib.ENR_getLinkResult),
):
indices = {name: index + 1 for index, name in enumerate(names[kind])}
for name in selected:
index = indices.get(name)
if index is None:
raise ValueError("requested hydraulic output is missing")
series = []
for period in range(summary["times"]["num_periods"]):
values = ctypes.POINTER(ctypes.c_float)()
length = ctypes.c_int()
reader._check(function(reader._handle, period, index, ctypes.byref(values), ctypes.byref(length)))
try:
if length.value != len(attributes):
raise ValueError("unexpected hydraulic result attributes")
row = {key: float(values[i]) for i, key in enumerate(attributes)}
if not all(math.isfinite(value) for value in row.values()):
raise ValueError("non-finite hydraulic result")
series.append(row)
finally:
reader._lib.ENR_free(ctypes.byref(values))
summary[kind][name] = series
return summary
def pressure_preview(
pipe: PipeCalibration, start: datetime, duration: int, step: int,
history: dict, current: dict, rule: SelectionRule,
) -> dict:
"""Exercise the actual replay cleaner/detector without database mutations."""
from app.domain.schemas.scada_replay import ScadaReplayCreate
from app.services.scada_replay_processing import detect_frame, prepare_cleaning
ids = [binding.device_id for binding in pipe.pressure_bindings]
config = ScadaReplayCreate(
request_id=uuid5(NAMESPACE_URL, f"calibration:{pipe.pipe_id}:{start.isoformat()}"),
start_time=start, end_time=start + timedelta(seconds=duration + step), device_ids=ids,
rules=[{"rule_id": "calibration-pressure", "device_ids": ids,
"kind": "relative_change", "direction": "decrease", "unit": "m",
"threshold": rule.primary_pressure_drop_percent,
"window_samples": 4, "minimum_samples": 4,
"consecutive_samples": rule.consecutive_samples, "minimum_devices": 1}],
)
document = {"replay_id": str(config.request_id), "devices": [
{"device_id": key, "transmission_frequency": "0:15:00"} for key in ids],
"histories": {binding.device_id: [row["pressure"] for row in history[binding.node_id]]
for binding in pipe.pressure_bindings}}
raised = []
for index in range(duration // step + 1):
at = start + timedelta(seconds=index * step)
rows = [{"device_id": binding.device_id, "time": at,
"monitored_value": current[binding.node_id][index]["pressure"],
"source_hash": hashlib.sha256(f"{binding.device_id}:{at}".encode()).hexdigest()}
for binding in pipe.pressure_bindings]
document, changes = detect_frame(document, prepare_cleaning(rows, config), config, at)
raised.extend(change for change in changes
if change["change"] == "raised" and change.get("category") != "data_quality")
return {"raised": bool(raised), "first_signal_at": raised[0]["occurred_at"] if raised else None,
"scope": "pure replay rule preview; not monitoring worker or Agent acceptance"}
def assess(
pipe: PipeCalibration, normal: dict, burst: dict, rule: SelectionRule,
*, duration: int, step: int,
) -> dict:
expected_times = {"report_start": 0, "report_step": step,
"sim_duration": duration, "num_periods": duration // step + 1}
blockers = []
for label, result in (("normal", normal), ("burst", burst)):
if result.get("status") != "completed" or result.get("times") != expected_times:
blockers.append(f"{label}_incomplete_or_failed")
if result.get("warnings"):
blockers.append(f"{label}_solver_warning")
if result.get("units") != {"flow": "LPS", "pressure": "MTR", "quality": "NONE"}:
blockers.append(f"{label}_unexpected_units")
if blockers:
return {"passed": False, "blockers": blockers}
pressures, flows = {}, {}
for binding in pipe.pressure_bindings:
before = [row["pressure"] for row in normal["nodes"][binding.node_id]]
after = [row["pressure"] for row in burst["nodes"][binding.node_id]]
if min(before) <= rule.relative_change_epsilon:
blockers.append(f"nonpositive_normal_pressure:{binding.device_id}")
if min(after) < 0:
blockers.append(f"negative_monitor_pressure:{binding.device_id}")
pressures[binding.device_id] = [100 * (a - b) / max(abs(a), rule.relative_change_epsilon)
for a, b in zip(before, after)]
for binding in pipe.flow_bindings:
flows[binding.device_id] = [100 * abs(a["flow"] - b["flow"]) /
max(abs(a["flow"]), rule.relative_change_epsilon)
for a, b in zip(normal["links"][binding.link_id], burst["links"][binding.link_id])]
witnesses = []
count = duration // step + 1
for primary, drops in pressures.items():
for start in range(count - rule.consecutive_samples + 1):
window = range(start, start + rule.consecutive_samples)
if not all(drops[i] >= rule.primary_pressure_drop_percent for i in window):
continue
second = [key for key, values in pressures.items() if key != primary and
all(values[i] >= rule.corroborating_pressure_drop_percent for i in window)]
flow = [key for key, values in flows.items() if
all(values[i] >= rule.flow_change_percent for i in window) and
all(abs(normal["links"][next(b.link_id for b in pipe.flow_bindings if b.device_id == key)][i]["flow"])
> rule.relative_change_epsilon for i in window)]
if second or flow:
witnesses.append({"primary_device": primary, "second_pressure_devices": second,
"flow_devices": flow, "start_offset_seconds": start * step})
if not witnesses:
blockers.append("insufficient_consecutive_pressure_and_corroboration")
return {"passed": not blockers, "blockers": blockers, "witnesses": witnesses,
"pressure_drop_percent": pressures, "flow_change_percent": flows}

Some files were not shown because too many files have changed in this diff Show More