Compare commits
19
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
471ad92a47 | ||
|
|
1892b1d1f0 | ||
|
|
2e8fe86f3f | ||
|
|
b853b61a0b | ||
|
|
e47c61c121 | ||
|
|
6a4478bfe4 | ||
|
|
5197a93163 | ||
|
|
549019507e | ||
|
|
acc8f8f7cb | ||
|
|
b8359ba83c | ||
|
|
c0dea2aacf | ||
|
|
33df25f22b | ||
|
|
2a7b96f813 | ||
|
|
06d8539aa8 | ||
|
|
2552a9b40d | ||
|
|
01bad116c1 | ||
|
|
bf8ea45d40 | ||
|
|
dc83cf5c88 | ||
|
|
ff8aba4964 |
+5
-4
@@ -6,6 +6,8 @@
|
||||
# 基础运行
|
||||
PROJECT_NAME="TJWater Server"
|
||||
ENVIRONMENT="production"
|
||||
# 任务领取范围;留空时沿用 ENVIRONMENT。历史任务需先执行 034 迁移。
|
||||
OPERATIONAL_TASK_EXECUTION_SCOPE=""
|
||||
API_V1_STR="/api/v1"
|
||||
NETWORK_NAME="tjwater"
|
||||
|
||||
@@ -150,10 +152,9 @@ TJWATER_MCP_MAX_REQUEST_TIMEOUT_SECONDS=240
|
||||
# 后台计算独立执行时限为 120–86400 秒,不受工具 timeout_seconds 控制。
|
||||
TJWATER_MCP_MAX_CALCULATION_TIMEOUT_SECONDS=3600
|
||||
|
||||
# 独立计算队列消费者;显式项目 UUID,以逗号分隔。不会在 API lifespan 内启动。
|
||||
# 先在每个目标项目 BizDB 依次应用 026_calculation_jobs.sql 和 027_calculation_job_lifecycle.sql,再启动计算服务。
|
||||
CALCULATION_WORKER_PROJECT_IDS=""
|
||||
# 所有配置项目合计并发,独立于 Uvicorn --workers 4;单实例锁防止重复消费者扩容。
|
||||
# 独立计算队列消费者;从 MetaDB 动态发现活动项目,不会在 API lifespan 内启动。
|
||||
# 先在每个项目 BizDB 应用 026、027 和 030 迁移,再启动计算服务。
|
||||
# 所有活动项目合计并发,独立于 Uvicorn --workers 4;单实例锁防止重复消费者扩容。
|
||||
CALCULATION_WORKER_MAX_CONCURRENCY=2
|
||||
CALCULATION_WORKER_POLL_SECONDS=2
|
||||
CALCULATION_WORKER_LEASE_SECONDS=30
|
||||
|
||||
@@ -12,7 +12,7 @@ Use the existing conda environment when available:
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests/unit tests/auth -q
|
||||
conda run -n server uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
conda run -n server python scripts/run_server.py --reload
|
||||
docker build -t tjwater-server:local .
|
||||
docker compose -f infra/docker/docker-compose.yml config
|
||||
```
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
`next-tjwater-server` 是 NextTJWater 供水系统的 FastAPI 后端,提供认证、项目、管网、模拟、爆管、漏损、SCADA、地图服务、工况任务和受控 MCP 能力。业务数据和时序数据按已认证项目路由,后台工作进程也必须使用明确的项目范围。
|
||||
|
||||
协议、工作进程、数据库、SCADA 与历史资料索引见 [文档目录](docs/README.md)。
|
||||
|
||||
## 技术栈
|
||||
|
||||
- Python 3.12
|
||||
@@ -42,7 +44,7 @@ cp .env.example .env
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests/unit tests/auth -q
|
||||
conda run -n server uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
conda run -n server python scripts/run_server.py --reload
|
||||
```
|
||||
|
||||
如需要进入环境:
|
||||
@@ -84,11 +86,12 @@ CORS 匹配浏览器页面来源,不是 Nginx 连接 IP 或域名解析结果
|
||||
|
||||
## 业务 Skill 与闭环
|
||||
|
||||
[爆管处置工程图](docs/burst-response-workflow.md)覆盖上报、候选定位、隔离阀门、影响分析、地图与数据展示、审批处置和反馈闭环。Server 通过任务 MCP 按 workflow 分发以下业务 Skill,当前版本均为 `2.0.0`:
|
||||
[爆管事件处理流程](docs/burst-response-workflow.md)覆盖工单接收、资产关联、隔离阀门、影响说明、审批处置和反馈闭环。Server 按任务 workflow 与回放来源分发独立 Skill,版本以实际返回为准:
|
||||
|
||||
| workflow | Skill | 调查重点 |
|
||||
| --- | --- | --- |
|
||||
| burst_report | [爆管处置](resources/work-order-skills/burst-report/SKILL.md) | 候选定位、隔离阀门、影响分析、抢修及恢复闭环 |
|
||||
| burst_report(工单) | [爆管事件处理](resources/work-order-skills/burst-event-handling/SKILL.md) | 资产关联、隔离方案、影响说明、处置及恢复闭环 |
|
||||
| scada_anomaly/历史回放 | [SCADA 异常调查](resources/work-order-skills/scada-anomaly/SKILL.md) | 冻结数据诊断、候选定位与隔离影响分析 |
|
||||
| water_quality_complaint | [水质投诉](resources/work-order-skills/water-quality-complaint/SKILL.md) | 投诉关联、监测与采样、候选影响、处置及复测回访 |
|
||||
| general | [预约跟进](resources/work-order-skills/appointment-follow-up/SKILL.md) | 审批与回执、现场履约、反馈复核及与原事件的闭环区别 |
|
||||
|
||||
@@ -97,7 +100,7 @@ JSONL 整理遵守会话文件权限,后台图表和表格通过既有报告 `
|
||||
## 常用命令
|
||||
|
||||
外围接入样板作为 `/peripheral` 子应用随主 API 进程启动,复用 PostgreSQL 且不自动建表;授权边界和契约见 [外围样板说明](docs/peripheral-sample.md)。
|
||||
外围样板提供 synthetic 测试工单和测试数据的版本化 CRUD;工单经同步器投影为 `workbench.operational_tasks` 中的外围来源任务。固定演示时窗和造数方式见 [合成样本说明](docs/peripheral-generated-data.md),运行态读取边界见 [工作台运行态说明](docs/workbench-runtime.md)。这些数据不代表现场事实,也不会触发设备控制。
|
||||
外围系统提供业务工单和反馈的版本化 CRUD,工单来源固定为 `system_data`,经同步器投影为 `workbench.operational_tasks` 中的外围来源任务。系统上报与已核实事实分别记录,现场处置通过审批流程执行。独立测试数据和监测演示的固定时窗及造数方式见 [合成样本说明](docs/peripheral-generated-data.md),运行态读取边界见 [工作台运行态说明](docs/workbench-runtime.md)。
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests -q
|
||||
@@ -111,6 +114,14 @@ docker compose -f infra/docker/docker-compose.yml config
|
||||
- `docker build`:构建后端镜像。
|
||||
- `docker compose config`:检查 compose 配置和变量展开。
|
||||
|
||||
开发时使用 `python scripts/run_server.py --reload` 启用进程交接式重载:父进程持有监听端口,
|
||||
候选进程完成 ASGI 初始化、通过元数据库就绪检查并接管监听后,才通知旧进程停止接收新连接。
|
||||
旧 HTTP 请求和 SSE 连接继续由旧进程完成,期间新请求由新进程响应;不迁移已建立的 TCP 连接,
|
||||
也不通过强制取消旧请求来加快重载。候选启动失败或超出 `--startup-timeout`(默认 60 秒)时,
|
||||
保留旧进程继续服务。快速连续修改最多保留两个正在排空的旧进程,其余修改合并后再重载。
|
||||
默认监视 `app/` 和 `samples/peripheral/` 下的 Python 文件,可重复传入 `--reload-dir` 指定目录。
|
||||
直接执行 `uvicorn ... --reload` 仍使用其原有先停后启流程;已经运行的旧启动命令不会自动切换到新机制。
|
||||
|
||||
## 开发规范
|
||||
|
||||
- Python 文件、函数、变量、Pydantic 字段、JSON body 字段和 query 参数使用 `snake_case`。
|
||||
@@ -204,7 +215,7 @@ Gitea Actions 工作流位于 `.gitea/workflows/package.yml`,可手动触发
|
||||
|
||||
## 产品 MCP 查询
|
||||
|
||||
Server 可选启用 MCP 2026-07-28,默认关闭。当前目录包含 67 项类型化工具,其中 42 项查询、12 项计算、13 项预览式写入。MCP 使用专用 OAuth resource Bearer、宿主绑定上下文和现有项目或任务权限,不提供自由命令执行。
|
||||
Server 可选启用 MCP 2026-07-28,默认关闭。目录包含查询、分析计算、预览式写入和计算任务管理工具;可用项以 `app/mcp/catalog.py` 及当前授权目录为准。MCP 使用专用 OAuth resource Bearer、宿主绑定上下文和现有项目或任务权限,不提供自由命令执行。
|
||||
|
||||
写入工具默认只执行 `dry_run`。真实写入必须使用同一 `request_id`,带上预览返回的 `expected_digest`,并通过宿主原生许可和项目权限检查。超时或结果不明时先查询操作状态,不盲目重试。实施、启用和授权边界见 [MCP 实施与启用](docs/mcp-query-implementation.md) 和 [协议与旧 CLI 能力对照](docs/mcp-query-design.md)。
|
||||
|
||||
|
||||
@@ -7,7 +7,9 @@ from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Query, status
|
||||
from psycopg import AsyncConnection
|
||||
from app.auth.metadata_dependencies import get_metadata_repository
|
||||
|
||||
from app.core.config import settings
|
||||
from app.auth.permissions import (
|
||||
OPERATIONAL_TASK_EXECUTE,
|
||||
OPERATIONAL_TASK_EXTERNAL_SYNC,
|
||||
@@ -22,6 +24,7 @@ from app.auth.project_dependencies import (
|
||||
ProjectContext,
|
||||
get_project_context,
|
||||
get_project_pg_connection,
|
||||
get_project_timescale_connection,
|
||||
)
|
||||
from app.domain.domain_errors import DomainError
|
||||
from app.domain.schemas.workbench_runtime import WorkbenchDateTime
|
||||
@@ -90,11 +93,37 @@ from app.services.operational_tasks import OperationalTaskService
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/operational-tasks/recovery/reconcile")
|
||||
async def reconcile_task_recoveries(
|
||||
command: RecoverExpiredLeasesCommand,
|
||||
ctx: ProjectContext = Depends(require_permission(OPERATIONAL_TASK_EXECUTE)),
|
||||
conn: AsyncConnection = Depends(get_project_pg_connection),
|
||||
timescale=Depends(get_project_timescale_connection),
|
||||
metadata=Depends(get_metadata_repository),
|
||||
) -> dict[str, int]:
|
||||
from app.domain.schemas.project_settings import ProjectSettings
|
||||
from app.services.operational_recovery import reconcile_recoveries
|
||||
|
||||
if SCADA_VIEW not in permissions_for_context(ctx):
|
||||
raise HTTPException(403, "恢复监测需要 SCADA 查看权限")
|
||||
project = await metadata.get_project_by_id(ctx.project_id)
|
||||
if project is None:
|
||||
raise HTTPException(404, "当前项目不存在")
|
||||
try:
|
||||
resumed = await reconcile_recoveries(_service(ctx, conn), timescale,
|
||||
ProjectSettings.model_validate(project.settings or {}), limit=command.limit,
|
||||
created_from=command.task_created_from)
|
||||
return {"resumed": resumed}
|
||||
except DomainError as exc:
|
||||
_raise_domain(exc)
|
||||
|
||||
|
||||
def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskService:
|
||||
return OperationalTaskService(
|
||||
OperationalTaskRepository(conn, ctx.project_id),
|
||||
ctx,
|
||||
_task_permissions(ctx),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -44,12 +44,12 @@ async def replay_capabilities(repo=Depends(repository), ctx=Depends(require_perm
|
||||
monitoring_ready = bool(
|
||||
monitoring
|
||||
and len(monitoring.pressure_device_ids) >= 5
|
||||
and monitoring.outlet_flow_device_ids
|
||||
and (monitoring.decision_policy == "weighted" or monitoring.outlet_flow_device_ids)
|
||||
)
|
||||
if monitoring_ready:
|
||||
monitoring_reason = None
|
||||
elif monitoring is None:
|
||||
monitoring_reason = "项目尚未配置压力与出厂边界流量监测角色"
|
||||
monitoring_reason = "项目尚未配置压力监测角色及组合判定规则"
|
||||
elif len(monitoring.pressure_device_ids) < 5:
|
||||
monitoring_reason = "爆管侦测至少需要 5 个压力监测设备"
|
||||
else:
|
||||
@@ -60,6 +60,7 @@ async def replay_capabilities(repo=Depends(repository), ctx=Depends(require_perm
|
||||
monitoring_interval_seconds=settings.scada_monitoring_interval_seconds,
|
||||
burst_monitoring_ready=monitoring_ready,
|
||||
burst_monitoring_reason=monitoring_reason,
|
||||
monitoring_config=monitoring,
|
||||
current=snapshot(current) if current else None,
|
||||
)
|
||||
|
||||
@@ -86,7 +87,7 @@ async def replay_command(replay_id: UUID, command: ScadaReplayCommand, repo=Depe
|
||||
timescale=Depends(get_project_timescale_connection)):
|
||||
service = ScadaReplayService(repo, ctx, timescale)
|
||||
result = await service.command(replay_id, command)
|
||||
if command.action in {"step", "retry"} and result.step_pending:
|
||||
if command.action in {"step", "monitor", "retry"} and result.step_pending:
|
||||
return await service.process_pending_step(replay_id)
|
||||
return result
|
||||
|
||||
|
||||
@@ -36,6 +36,7 @@ def _service(ctx: ProjectContext, conn: AsyncConnection) -> OperationalTaskServi
|
||||
OperationalTaskRepository(conn, ctx.project_id),
|
||||
ctx,
|
||||
permissions_for_context(ctx),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -8,6 +8,12 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
class Settings(BaseSettings):
|
||||
PROJECT_NAME: str = "TJWater Server"
|
||||
ENVIRONMENT: str = "production"
|
||||
OPERATIONAL_TASK_EXECUTION_SCOPE: str = ""
|
||||
|
||||
@property
|
||||
def operational_task_execution_scope(self) -> str:
|
||||
return self.OPERATIONAL_TASK_EXECUTION_SCOPE.strip() or self.ENVIRONMENT
|
||||
|
||||
API_V1_STR: str = "/api/v1"
|
||||
NETWORK_NAME: str = "default_network"
|
||||
# 敏感配置加密密钥 (Fernet)
|
||||
|
||||
@@ -3,12 +3,28 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from jsonschema import Draft7Validator
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_serializer, model_validator
|
||||
|
||||
CHART_SAMPLE_TIME_PATTERN = (
|
||||
r"^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9]"
|
||||
r"(?:\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
)
|
||||
|
||||
|
||||
def validate_chart_sample_time(value: str) -> None:
|
||||
"""Match the bounded frontend RFC3339 spelling and an actual calendar date."""
|
||||
if len(value) > 64 or re.fullmatch(CHART_SAMPLE_TIME_PATTERN, value) is None:
|
||||
raise ValueError("Chart sample times require RFC3339 seconds and a timezone")
|
||||
if datetime.fromisoformat(value).tzinfo is None:
|
||||
raise ValueError("Chart sample times require a timezone")
|
||||
|
||||
|
||||
_catalog = json.loads(
|
||||
(Path(__file__).resolve().parents[3] / "contracts/answer-ui-catalog.json").read_text()
|
||||
)
|
||||
@@ -71,8 +87,18 @@ class AnswerUiBlock(UiModel):
|
||||
if element.type == "Chart":
|
||||
if any(len(series["data"]) != len(props["x_data"]) for series in props["series"]):
|
||||
raise ValueError("Chart series must match the axis")
|
||||
if any(not math.isfinite(value) for series in props["series"] for value in series["data"]):
|
||||
if any(value is not None and not math.isfinite(value) for series in props["series"] for value in series["data"]):
|
||||
raise ValueError("Chart values must be finite")
|
||||
for series in props["series"]:
|
||||
times = series.get("sample_times")
|
||||
if times is not None:
|
||||
if len(times) != len(series["data"]):
|
||||
raise ValueError("Chart sample times must match the axis")
|
||||
for value, at in zip(series["data"], times, strict=True):
|
||||
if at is not None:
|
||||
if value is None:
|
||||
raise ValueError("Chart sample times require an observed value")
|
||||
validate_chart_sample_time(at)
|
||||
if element.type == "Table" and any(len(row) != len(props["columns"]) for row in props["rows"]):
|
||||
raise ValueError("Table row width must match its columns")
|
||||
if element.type == "FollowUpForm":
|
||||
|
||||
@@ -5,8 +5,10 @@ import re
|
||||
from typing import Annotated, Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import Field, JsonValue, model_validator
|
||||
from app.domain.schemas.answer_ui import ReportUiBlock, validate_report_ui_blocks
|
||||
from pydantic import Field, JsonValue, field_validator, model_serializer, model_validator
|
||||
from app.domain.schemas.answer_ui import (
|
||||
CHART_SAMPLE_TIME_PATTERN, ReportUiBlock, validate_chart_sample_time, validate_report_ui_blocks,
|
||||
)
|
||||
|
||||
from app.domain.schemas.task_question import TaskQuestionRequest
|
||||
from app.domain.schemas.shared import NonEmptyStr, PositiveInt, StrictBool, StrictModel
|
||||
@@ -220,25 +222,61 @@ class InvestigationMapTarget(StrictModel):
|
||||
feature_id: TaskFeatureId
|
||||
|
||||
|
||||
class InvestigationDiagramNodeRef(StrictModel):
|
||||
InvestigationNodeAnnotation = Literal["reference", "recorded", "needs_input", "failed", "not_taken"]
|
||||
LEGACY_INVESTIGATION_NODE_ANNOTATIONS = {
|
||||
"pending": "reference", "running": "reference", "completed": "recorded",
|
||||
"waiting": "needs_input", "waiting_human": "needs_input",
|
||||
"skipped": "not_taken", "cancelled": "not_taken",
|
||||
}
|
||||
|
||||
|
||||
class _InvestigationDiagramNode(StrictModel):
|
||||
node_id: Annotated[str, Field(pattern=r"^[A-Za-z_][A-Za-z0-9_-]{0,79}$")]
|
||||
label: NonEmptyStr
|
||||
detail: Annotated[str, Field(min_length=1, max_length=2000)] | None = None
|
||||
evidence_refs: list[NonEmptyStr] = Field(default_factory=list)
|
||||
action_refs: list[UUID] = Field(default_factory=list)
|
||||
status: Literal[
|
||||
"pending", "running", "completed", "waiting", "waiting_human", "failed", "skipped", "cancelled"
|
||||
] | None = Field(default=None, description=(
|
||||
"Evidence-confirmed node state; waiting means prerequisites/external/system result, "
|
||||
"waiting_human requires manual authorization/approval/feedback. "
|
||||
"Unknown stays omitted/null; host controls colors."
|
||||
))
|
||||
map_targets: list[InvestigationMapTarget] = Field(default_factory=list, max_length=100)
|
||||
|
||||
|
||||
class InvestigationDiagramNodeRef(_InvestigationDiagramNode):
|
||||
status: InvestigationNodeAnnotation | None = Field(default=None, description=(
|
||||
"A saved process-reference annotation, not live execution state. "
|
||||
"recorded means a finding exists, not that a task or field action completed."
|
||||
))
|
||||
|
||||
@field_validator("status", mode="before")
|
||||
@classmethod
|
||||
def read_legacy_annotation(cls, value: object) -> object:
|
||||
return LEGACY_INVESTIGATION_NODE_ANNOTATIONS.get(value, value) if isinstance(value, str) else value
|
||||
|
||||
|
||||
class InvestigationGraphNodeUpdate(_InvestigationDiagramNode):
|
||||
status: InvestigationNodeAnnotation | None = Field(default=None, description=(
|
||||
"Optional reference annotation: reference, recorded finding, needs_input, failed attempt, or not_taken. "
|
||||
"No live node/task binding or per-step synchronization is required."
|
||||
))
|
||||
|
||||
|
||||
class InvestigationGraphEdge(StrictModel):
|
||||
source: Annotated[str, Field(pattern=r"^[A-Za-z_][A-Za-z0-9_-]{0,79}$")]
|
||||
target: Annotated[str, Field(pattern=r"^[A-Za-z_][A-Za-z0-9_-]{0,79}$")]
|
||||
label: Annotated[str, Field(min_length=1, max_length=200)] | None = None
|
||||
|
||||
|
||||
class InvestigationGraphUpdate(StrictModel):
|
||||
title: Annotated[str, Field(min_length=1, max_length=200)] | None = None
|
||||
nodes: list[InvestigationGraphNodeUpdate] = Field(default_factory=list, max_length=100)
|
||||
edges: list[InvestigationGraphEdge] = Field(default_factory=list, max_length=200)
|
||||
|
||||
|
||||
class InvestigationDiagram(StrictModel):
|
||||
title: NonEmptyStr
|
||||
mermaid: Annotated[str, Field(min_length=1, max_length=20000)]
|
||||
node_refs: list[InvestigationDiagramNodeRef] = Field(default_factory=list, max_length=100)
|
||||
# Present only on a graph maintained by the host. Legacy Mermaid stays readable.
|
||||
edges: list[InvestigationGraphEdge] | None = Field(default=None, max_length=200)
|
||||
source_mermaid: Annotated[str, Field(min_length=1, max_length=20000)] | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_description(self):
|
||||
@@ -282,6 +320,7 @@ class InvestigationReport(StrictModel):
|
||||
data_gaps: list[InvestigationDataGap] | None = None
|
||||
diagram: InvestigationDiagram | None = None
|
||||
ui_blocks: list["ReportUiBlock"] = Field(default_factory=list, max_length=8)
|
||||
scada_device_ids: list[TaskFeatureId] | None = Field(default=None, max_length=1000)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_result(self):
|
||||
@@ -321,6 +360,16 @@ class InvestigationProgress(StrictModel):
|
||||
diagram: InvestigationDiagram | None = None
|
||||
|
||||
|
||||
class InvestigationProgressInput(InvestigationProgress):
|
||||
graph_update: InvestigationGraphUpdate | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_graph_input(self):
|
||||
if self.graph_update is not None and self.diagram is not None:
|
||||
raise ValueError("use graph_update without a generated Mermaid diagram")
|
||||
return self
|
||||
|
||||
|
||||
class InvestigationProgressSnapshot(StrictModel):
|
||||
revision: PositiveInt
|
||||
run_id: UUID
|
||||
@@ -365,7 +414,7 @@ class AgentArtifactsCommand(StrictModel):
|
||||
evidence: list[EvidenceInput] = Field(default_factory=list, max_length=100)
|
||||
plan: ExecutionPlanInput | None = None
|
||||
report: InvestigationReportInput | None = None
|
||||
progress: InvestigationProgress | None = None
|
||||
progress: InvestigationProgressInput | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_artifacts(self):
|
||||
@@ -375,6 +424,8 @@ class AgentArtifactsCommand(StrictModel):
|
||||
self.report.diagram if self.report is not None else None,
|
||||
self.progress.diagram if self.progress is not None else None,
|
||||
):
|
||||
if diagram is not None and (diagram.edges is not None or diagram.source_mermaid is not None):
|
||||
raise ValueError("structured graph metadata is host-owned; use progress.graph_update")
|
||||
missing = _undeclared_diagram_node_refs(diagram)
|
||||
if missing:
|
||||
raise ValueError(
|
||||
@@ -572,6 +623,7 @@ class OperationalTaskRun(StrictModel):
|
||||
|
||||
|
||||
class OperationalTaskSummary(StrictModel):
|
||||
execution_scope: NonEmptyStr
|
||||
project_id: UUID
|
||||
task_id: UUID
|
||||
task_type: TaskType
|
||||
@@ -588,6 +640,7 @@ class OperationalTaskSummary(StrictModel):
|
||||
execution_request_id: UUID
|
||||
title: NonEmptyStr
|
||||
summary: NonEmptyStr
|
||||
handling_context: dict[str, JsonValue] = Field(default_factory=dict)
|
||||
source: TaskSource
|
||||
trigger: TaskTrigger
|
||||
scheduled_for: WorkbenchDateTime | None
|
||||
@@ -690,17 +743,64 @@ class AgentArtifactsResult(StrictModel):
|
||||
progress_revision: PositiveInt | None = None
|
||||
|
||||
|
||||
class ScadaChartGap(StrictModel):
|
||||
device_id: TaskFeatureId
|
||||
day_offset: Annotated[int, Field(ge=0, le=7)]
|
||||
missing_count: Annotated[int, Field(ge=1, le=100)]
|
||||
reason: Literal["no_readings", "outside_frozen_scope", "outside_frozen_window", "no_scheduled_samples"]
|
||||
|
||||
|
||||
class ScadaChartContext(StrictModel):
|
||||
role: Literal["current", "comparison"]
|
||||
metric: Literal["pressure", "flow"]
|
||||
unit: Annotated[str, Field(min_length=1, max_length=128)]
|
||||
window_start: WorkbenchDateTime
|
||||
window_end: WorkbenchDateTime
|
||||
value_field: Literal["monitored_value", "cleaned_value"] = "monitored_value"
|
||||
gaps: list[ScadaChartGap] = Field(default_factory=list, max_length=8)
|
||||
|
||||
|
||||
class TaskSafeChartSeries(StrictModel):
|
||||
name: Annotated[str, Field(max_length=128)]
|
||||
data: Annotated[list[Annotated[float, Field(strict=True, allow_inf_nan=False)]], Field(min_length=1, max_length=100)]
|
||||
device_id: TaskFeatureId | None = None
|
||||
day_offset: Annotated[int, Field(ge=0, le=7)] | None = None
|
||||
type: Literal["line", "bar", "scatter"] | None = None
|
||||
data: Annotated[list[Annotated[float, Field(strict=True, allow_inf_nan=False)] | None], Field(min_length=1, max_length=100)]
|
||||
sample_times: Annotated[list[Annotated[str, Field(
|
||||
strict=True, max_length=64,
|
||||
pattern=CHART_SAMPLE_TIME_PATTERN,
|
||||
)] | None], Field(min_length=1, max_length=100)] | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def serialize_optional_fields(self, handler):
|
||||
return {key: value for key, value in handler(self).items() if value is not None}
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_sample_times(self):
|
||||
if self.sample_times is not None:
|
||||
if len(self.sample_times) != len(self.data):
|
||||
raise ValueError("chart sample_times must align with data")
|
||||
for value, at in zip(self.data, self.sample_times, strict=True):
|
||||
if at is not None:
|
||||
if value is None:
|
||||
raise ValueError("chart sample_times require an observed value")
|
||||
validate_chart_sample_time(at)
|
||||
return self
|
||||
|
||||
|
||||
class TaskSafeChart(StrictModel):
|
||||
title: Annotated[str, Field(max_length=200)]
|
||||
chart_type: Literal["line", "bar"]
|
||||
scada: ScadaChartContext | None = None
|
||||
chart_type: Literal["line", "bar", "time_comparison"]
|
||||
x_axis_name: Annotated[str, Field(max_length=128)] | None = None
|
||||
y_axis_name: Annotated[str, Field(max_length=128)] | None = None
|
||||
x_data: Annotated[list[Annotated[str, Field(max_length=128)]], Field(min_length=1, max_length=100)]
|
||||
series: Annotated[list[TaskSafeChartSeries], Field(min_length=1, max_length=8)]
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def serialize_optional_fields(self, handler):
|
||||
return {key: value for key, value in handler(self).items() if value is not None}
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_alignment(self):
|
||||
if any(len(series.data) != len(self.x_data) for series in self.series):
|
||||
|
||||
@@ -1,19 +1,39 @@
|
||||
from math import isclose
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, JsonValue, model_validator
|
||||
|
||||
|
||||
class ScadaMonitoringWeights(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
burst_detection: float = Field(default=0.5, ge=0, le=1)
|
||||
pressure: float = Field(default=0.4, ge=0, le=1)
|
||||
outlet_flow: float = Field(default=0.1, ge=0, le=1)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def normalized(self):
|
||||
if not isclose(self.burst_detection + self.pressure + self.outlet_flow, 1.0, abs_tol=1e-9):
|
||||
raise ValueError("SCADA 三项监测权重之和必须为 1")
|
||||
return self
|
||||
|
||||
|
||||
class ScadaMonitoringSettings(BaseModel):
|
||||
"""Project-owned device roles used by the manual SCADA monitoring demo."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
rule_version: str = Field(
|
||||
default="burst-monitor-v1",
|
||||
default="burst-monitor-v2",
|
||||
min_length=1,
|
||||
max_length=80,
|
||||
pattern=r"^[a-zA-Z0-9_.-]+$",
|
||||
)
|
||||
pressure_device_ids: list[str] = Field(default_factory=list, max_length=20)
|
||||
outlet_flow_device_ids: list[str] = Field(default_factory=list, max_length=20)
|
||||
decision_policy: Literal["all_signals", "weighted"] = "weighted"
|
||||
weights: ScadaMonitoringWeights = Field(default_factory=ScadaMonitoringWeights)
|
||||
alert_threshold: float = Field(default=0.8, gt=0, le=1)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def distinct_roles(self):
|
||||
|
||||
@@ -5,6 +5,8 @@ from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
|
||||
|
||||
class ReplayModel(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
@@ -78,7 +80,26 @@ class ScadaReplayCreate(ReplayModel):
|
||||
class ScadaReplayCommand(ReplayModel):
|
||||
request_id: UUID
|
||||
expected_revision: int = Field(ge=1)
|
||||
action: Literal["step", "stop", "retry"]
|
||||
action: Literal["step", "monitor", "stop", "retry"]
|
||||
at_time: datetime | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def monitoring_time(self):
|
||||
if self.action == "monitor":
|
||||
if self.at_time is None or self.at_time.tzinfo is None:
|
||||
raise ValueError("选定时刻监测需要提供带时区的 at_time")
|
||||
elif self.at_time is not None:
|
||||
raise ValueError("只有选定时刻监测可以指定 at_time")
|
||||
return self
|
||||
|
||||
|
||||
class ScadaMonitoringIncident(ReplayModel):
|
||||
incident_id: UUID
|
||||
status: Literal["active", "resolved"]
|
||||
action: Literal["raised", "ongoing", "resolved"]
|
||||
started_at: datetime
|
||||
last_observed_at: datetime
|
||||
recovery_count: int = Field(ge=0, le=2)
|
||||
|
||||
|
||||
class ScadaMonitoringResult(ReplayModel):
|
||||
@@ -94,7 +115,10 @@ class ScadaMonitoringResult(ReplayModel):
|
||||
pressure: dict
|
||||
outlet_flow: dict
|
||||
burst_detection: dict
|
||||
weighted: dict | None = None
|
||||
incident: ScadaMonitoringIncident | None = None
|
||||
role_gaps: list[str] = Field(default_factory=list)
|
||||
anomaly_signals: list[dict] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ScadaReplaySnapshot(ReplayModel):
|
||||
@@ -108,6 +132,8 @@ class ScadaReplaySnapshot(ReplayModel):
|
||||
step_pending: bool = False
|
||||
monitoring_count: int = 0
|
||||
last_monitoring: ScadaMonitoringResult | None = None
|
||||
monitoring_config: ScadaMonitoringSettings | None = None
|
||||
monitoring_calibration: dict | None = None
|
||||
virtual_now: datetime
|
||||
delivered_until: datetime | None = None
|
||||
processed_until: datetime | None = None
|
||||
@@ -129,6 +155,7 @@ class ScadaReplayCapabilities(ReplayModel):
|
||||
monitoring_interval_seconds: int | None = Field(None, ge=60, le=31_536_000)
|
||||
burst_monitoring_ready: bool = False
|
||||
burst_monitoring_reason: str | None = None
|
||||
monitoring_config: ScadaMonitoringSettings | None = None
|
||||
current: ScadaReplaySnapshot | None = None
|
||||
|
||||
|
||||
|
||||
@@ -7,6 +7,29 @@ from uuid import UUID
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
async def discover_project_ids() -> tuple[str, ...]:
|
||||
"""Return every active PostgreSQL-backed project from authoritative metadata."""
|
||||
from sqlalchemy import select
|
||||
from app.infra.db.metadb.database import SessionLocal
|
||||
from app.infra.db.metadb import models
|
||||
|
||||
async with SessionLocal() as session:
|
||||
rows = await session.scalars(
|
||||
select(models.Project.id)
|
||||
.join(
|
||||
models.ProjectDatabase,
|
||||
models.ProjectDatabase.project_id == models.Project.id,
|
||||
)
|
||||
.where(
|
||||
models.Project.status == "active",
|
||||
models.ProjectDatabase.db_role == "biz_data",
|
||||
models.ProjectDatabase.db_type == "postgresql",
|
||||
)
|
||||
.order_by(models.Project.id)
|
||||
)
|
||||
return tuple(str(project_id) for project_id in rows.all())
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def queue_repository(project_id):
|
||||
"""Resolve the current project route and open its calculation queue."""
|
||||
|
||||
@@ -7,10 +7,10 @@ from app.mcp.errors import CalculationCapacityExceeded
|
||||
|
||||
|
||||
# Explicit metadata projection prevents status polling from decoding large JSON results.
|
||||
JOB_COLUMNS = "job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,status,error,lease_generation,lease_owner,lease_expires_at,created_at,updated_at,started_at,finished_at,queue_expires_at,result_expires_at"
|
||||
JOB_COLUMNS = "job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,status,error,lease_generation,lease_owner,lease_expires_at,created_at,updated_at,started_at,finished_at,result_expires_at"
|
||||
|
||||
|
||||
PUBLIC_JOB_COLUMNS = "job_id,status,tool,error,created_at,updated_at,started_at,finished_at,queue_expires_at,result_expires_at"
|
||||
PUBLIC_JOB_COLUMNS = "job_id,status,tool,error,created_at,updated_at,started_at,finished_at,result_expires_at"
|
||||
|
||||
|
||||
class CalculationJobRepository:
|
||||
@@ -19,12 +19,6 @@ class CalculationJobRepository:
|
||||
self.project_id = str(project_id)
|
||||
|
||||
async def maintain(self):
|
||||
await self.connection.execute(
|
||||
"""UPDATE workbench.calculation_jobs SET status='timed_out',
|
||||
error='{"code":"QUEUE_TIMEOUT"}'::jsonb,finished_at=clock_timestamp(),updated_at=clock_timestamp()
|
||||
WHERE project_id=%s AND status='queued' AND queue_expires_at<=clock_timestamp()""",
|
||||
(self.project_id,),
|
||||
)
|
||||
await self.connection.execute(
|
||||
"""UPDATE workbench.calculation_jobs SET result=NULL
|
||||
WHERE project_id=%s AND result IS NOT NULL AND result_expires_at<=clock_timestamp()""",
|
||||
@@ -32,14 +26,14 @@ class CalculationJobRepository:
|
||||
)
|
||||
|
||||
async def enqueue(self, *, owner_user_id, request_key, tool, parameters, binding, timeout_seconds,
|
||||
queue_wait_seconds=3600, max_active_jobs=100):
|
||||
max_active_jobs=100):
|
||||
async with self.connection.transaction():
|
||||
return await self._enqueue(owner_user_id=owner_user_id, request_key=request_key,
|
||||
tool=tool, parameters=parameters, binding=binding, timeout_seconds=timeout_seconds,
|
||||
queue_wait_seconds=queue_wait_seconds, max_active_jobs=max_active_jobs)
|
||||
max_active_jobs=max_active_jobs)
|
||||
|
||||
async def _enqueue(self, *, owner_user_id, request_key, tool, parameters, binding,
|
||||
timeout_seconds, queue_wait_seconds, max_active_jobs):
|
||||
timeout_seconds, max_active_jobs):
|
||||
# Serialize admission within the caller's transaction, across every API process.
|
||||
await self.connection.execute(
|
||||
"SELECT pg_advisory_xact_lock(hashtextextended(%s, 1748531781))", (self.project_id,))
|
||||
@@ -64,11 +58,11 @@ class CalculationJobRepository:
|
||||
raise CalculationCapacityExceeded()
|
||||
cursor = await self.connection.execute(
|
||||
f"""INSERT INTO workbench.calculation_jobs
|
||||
(job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds,queue_expires_at)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,clock_timestamp()+%s*interval '1 second')
|
||||
(job_id,project_id,owner_user_id,request_key,tool,parameters,binding,timeout_seconds)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s)
|
||||
RETURNING {JOB_COLUMNS}""",
|
||||
(uuid4(), self.project_id, owner_user_id, request_key, tool,
|
||||
Jsonb(parameters), Jsonb(binding), timeout_seconds, queue_wait_seconds),
|
||||
Jsonb(parameters), Jsonb(binding), timeout_seconds),
|
||||
)
|
||||
return dict(await cursor.fetchone())
|
||||
|
||||
@@ -118,7 +112,7 @@ class CalculationJobRepository:
|
||||
lease_generation=j.lease_generation+1,
|
||||
lease_expires_at=clock_timestamp()+%s*interval '1 second',updated_at=clock_timestamp()
|
||||
WHERE j.job_id=(SELECT job_id FROM workbench.calculation_jobs
|
||||
WHERE project_id=%s AND status='queued' AND queue_expires_at>clock_timestamp() ORDER BY created_at,job_id
|
||||
WHERE project_id=%s AND status='queued' ORDER BY created_at,job_id
|
||||
FOR UPDATE SKIP LOCKED LIMIT 1) RETURNING j.*""",
|
||||
(owner, lease_seconds, self.project_id),
|
||||
)
|
||||
|
||||
@@ -6,6 +6,24 @@ from psycopg import AsyncConnection
|
||||
|
||||
|
||||
class NetworkAssetRepository:
|
||||
@staticmethod
|
||||
async def get_isolation_model(conn: AsyncConnection) -> list[dict[str, Any]]:
|
||||
"""Read source nodes, link state and pipe lengths in one model snapshot."""
|
||||
from app.infra.db.postgresql.read_queries import read_rows
|
||||
|
||||
return await read_rows(conn, """
|
||||
SELECT l.id, l.link_type::text AS kind,
|
||||
l.start_node_id, l.end_node_id, p.length, p.diameter,
|
||||
COALESCE(s.status::text, p.status::text, '') AS status
|
||||
FROM network.links l
|
||||
LEFT JOIN network.pipes p ON p.link_id = l.id
|
||||
LEFT JOIN network.link_initial_settings s ON s.link_id = l.id
|
||||
UNION ALL
|
||||
SELECT n.id, n.node_type::text, NULL, NULL, NULL, NULL, ''
|
||||
FROM network.nodes n
|
||||
WHERE n.node_type::text IN ('reservoir', 'tank')
|
||||
""")
|
||||
|
||||
@staticmethod
|
||||
async def get_pipes_by_ids(
|
||||
conn: AsyncConnection,
|
||||
|
||||
@@ -164,7 +164,7 @@ class OperationalTaskRepository:
|
||||
ORDER BY generation DESC LIMIT %s OFFSET %s""",
|
||||
(self.project_id, task_id, limit, offset),
|
||||
)
|
||||
return [{key: value for key, value in row.items() if key != "project_id"}
|
||||
return [{key: value for key, value in row.items() if key not in {"project_id", "artifact_manifest"}}
|
||||
for row in await cursor.fetchall()]
|
||||
|
||||
async def get_latest_run(self, task_id: str) -> dict | None:
|
||||
@@ -264,6 +264,7 @@ class OperationalTaskRepository:
|
||||
async def list_runs(
|
||||
self, *, status: str | None, limit: int, cursor_created_at=None, cursor_run_id=None,
|
||||
task_type: str | None = None, task_created_from: datetime | None = None,
|
||||
execution_scope: str = "production",
|
||||
) -> list[dict]:
|
||||
result = await self.connection.execute(
|
||||
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
|
||||
@@ -282,18 +283,20 @@ class OperationalTaskRepository:
|
||||
AND t.task_id=operational_task_runs.task_id
|
||||
AND (%s::text IS NULL OR t.task_type=%s::text)
|
||||
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
|
||||
AND t.document->>'execution_scope'=%s
|
||||
)
|
||||
ORDER BY created_at,run_id LIMIT %s""",
|
||||
(
|
||||
self.project_id, status, status, cursor_created_at,
|
||||
cursor_created_at, cursor_run_id,
|
||||
task_type, task_type, task_created_from, task_created_from, limit,
|
||||
task_type, task_type, task_created_from, task_created_from, execution_scope, limit,
|
||||
),
|
||||
)
|
||||
return [dict(row) for row in await result.fetchall()]
|
||||
|
||||
async def claim_expired_runs(
|
||||
self, limit: int, *, task_type: str | None = None, task_created_from: datetime | None = None,
|
||||
execution_scope: str = "production",
|
||||
) -> list[dict]:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT run_id,task_id,generation,execution_request_id,status,revision,
|
||||
@@ -310,9 +313,10 @@ class OperationalTaskRepository:
|
||||
AND t.task_id=operational_task_runs.task_id
|
||||
AND (%s::text IS NULL OR t.task_type=%s::text)
|
||||
AND (%s::timestamptz IS NULL OR t.created_at>=%s::timestamptz)
|
||||
AND t.document->>'execution_scope'=%s
|
||||
)
|
||||
ORDER BY lease_expires_at,run_id FOR UPDATE SKIP LOCKED LIMIT %s""",
|
||||
(self.project_id, task_type, task_type, task_created_from, task_created_from, limit),
|
||||
(self.project_id, task_type, task_type, task_created_from, task_created_from, execution_scope, limit),
|
||||
)
|
||||
return [dict(row) for row in await cursor.fetchall()]
|
||||
|
||||
@@ -367,6 +371,34 @@ class OperationalTaskRepository:
|
||||
)
|
||||
return [dict(row) for row in await cursor.fetchall()]
|
||||
|
||||
async def get_scada_presentation(self, task_id: str, run_id: str) -> dict | None:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT EXISTS (SELECT 1 FROM pg_attribute
|
||||
WHERE attrelid=to_regclass('workbench.operational_task_runs')
|
||||
AND attname='artifact_manifest' AND NOT attisdropped) AS available"""
|
||||
)
|
||||
if not (await cursor.fetchone())["available"]:
|
||||
raise DomainError("DEPENDENCY_UNAVAILABLE",
|
||||
"SCADA display coverage requires resources/sql/035_scada_presentation_coverage.sql in this project database",
|
||||
status=503)
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT artifact_manifest->'scada_presentation' AS document FROM workbench.operational_task_runs
|
||||
WHERE project_id=%s AND task_id=%s AND run_id=%s""",
|
||||
(self.project_id, task_id, run_id),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def save_scada_presentation(self, task_id: str, run_id: str, document: dict, now: datetime) -> None:
|
||||
cursor = await self.connection.execute(
|
||||
"""UPDATE workbench.operational_task_runs
|
||||
SET artifact_manifest=jsonb_set(artifact_manifest,'{scada_presentation}',%s::jsonb,true)
|
||||
WHERE project_id=%s AND task_id=%s AND run_id=%s""",
|
||||
(Jsonb({**document, "updated_at": now.isoformat()}), self.project_id, task_id, run_id),
|
||||
)
|
||||
if cursor.rowcount != 1:
|
||||
raise DomainError("RUN_NOT_FOUND", "SCADA display coverage targets an unknown run", status=409)
|
||||
|
||||
async def get_task_by_create_request(self, request_id: str) -> dict | None:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.operational_tasks
|
||||
@@ -401,9 +433,9 @@ class OperationalTaskRepository:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT task.document,
|
||||
CASE WHEN active_run.run_id IS NULL THEN NULL
|
||||
ELSE to_jsonb(active_run) - 'project_id' END AS active_run,
|
||||
ELSE to_jsonb(active_run) - 'project_id' - 'artifact_manifest' END AS active_run,
|
||||
CASE WHEN latest_run.run_id IS NULL THEN NULL
|
||||
ELSE to_jsonb(latest_run) - 'project_id' END AS latest_run
|
||||
ELSE to_jsonb(latest_run) - 'project_id' - 'artifact_manifest' END AS latest_run
|
||||
FROM workbench.operational_tasks AS task
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT run.* FROM workbench.operational_task_runs AS run
|
||||
@@ -624,6 +656,37 @@ class OperationalTaskRepository:
|
||||
row = await cursor.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def get_latest_graph_progress(self, task_id: str) -> dict | None:
|
||||
availability = await self.connection.execute(
|
||||
"SELECT to_regclass('workbench.operational_task_progress') AS relation"
|
||||
)
|
||||
if not (await availability.fetchone())["relation"]:
|
||||
return None
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.operational_task_progress
|
||||
WHERE project_id=%s AND task_id=%s
|
||||
AND jsonb_typeof(document#>'{progress,diagram,edges}')='array'
|
||||
ORDER BY (document->>'execution_generation')::bigint DESC, revision DESC LIMIT 1""",
|
||||
(self.project_id, task_id),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return row["document"] if row else None
|
||||
|
||||
async def get_latest_diagram(self, task_id: str) -> dict | None:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT diagram FROM (
|
||||
SELECT document#>'{progress,diagram}' AS diagram, published_at AS occurred_at
|
||||
FROM workbench.operational_task_progress WHERE project_id=%s AND task_id=%s
|
||||
UNION ALL
|
||||
SELECT document#>'{report,diagram}', created_at
|
||||
FROM workbench.operational_task_reports WHERE project_id=%s AND task_id=%s
|
||||
) AS diagrams WHERE jsonb_typeof(diagram)='object'
|
||||
ORDER BY occurred_at DESC LIMIT 1""",
|
||||
(self.project_id, task_id, self.project_id, task_id),
|
||||
)
|
||||
row = await cursor.fetchone()
|
||||
return row["diagram"] if row else None
|
||||
|
||||
async def save_progress(self, task_id: str, snapshot: dict) -> None:
|
||||
cursor = await self.connection.execute(
|
||||
"""INSERT INTO workbench.operational_task_progress
|
||||
@@ -711,6 +774,28 @@ class OperationalTaskRepository:
|
||||
row = await cursor.fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
async def recovery_candidates(self, limit: int, created_from=None, execution_scope: str = "production") -> list[dict]:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.operational_tasks
|
||||
WHERE project_id=%s AND task_type='event_investigation'
|
||||
AND status NOT IN ('cancelled','failed')
|
||||
AND document#>>'{handling_context,phase}'='waiting_scada'
|
||||
AND document->>'execution_scope'=%s
|
||||
AND (%s::timestamptz IS NULL OR created_at>=%s)
|
||||
ORDER BY COALESCE((document#>>'{handling_context,last_checked_at}')::timestamptz,created_at),task_id LIMIT %s""",
|
||||
(self.project_id, execution_scope, created_from, created_from, limit),
|
||||
)
|
||||
return [row["document"] for row in await cursor.fetchall()]
|
||||
|
||||
async def list_investigation_alerts(self, task_id: str) -> list[dict]:
|
||||
cursor = await self.connection.execute(
|
||||
"""SELECT document FROM workbench.alerts WHERE project_id=%s
|
||||
AND status IN ('active','acknowledged')
|
||||
AND document->>'condition_id'=%s ORDER BY alert_id""",
|
||||
(self.project_id, f"operational-task:{task_id}"),
|
||||
)
|
||||
return [row["document"] for row in await cursor.fetchall()]
|
||||
|
||||
async def upsert_feedback(self, task_id: str, feedback: dict, accepted_at) -> None:
|
||||
await self.connection.execute(
|
||||
"""INSERT INTO workbench.operational_task_external_feedback
|
||||
|
||||
@@ -7,7 +7,9 @@ from uuid import UUID
|
||||
from psycopg import AsyncConnection
|
||||
from psycopg import sql
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from fastapi import HTTPException
|
||||
from app.domain.schemas.operational_task import LEGACY_INVESTIGATION_NODE_ANNOTATIONS
|
||||
from app.infra.db.postgresql.scada import _SCADA_VIEW_SELECT, _device
|
||||
from app.infra.db.timescaledb.repositories.realtime import RealtimeRepository
|
||||
from app.infra.db.timescaledb.repositories.analysis import AnalysisResultsRepository
|
||||
@@ -312,6 +314,7 @@ async def task_context_summary(conn: AsyncConnection, *, project_id: UUID, task_
|
||||
'current_report_revision',t.current_report_revision,
|
||||
'title_preview',left(t.document->>'title',160),'preview_only',true,
|
||||
'detail',jsonb_build_object('section','task')),
|
||||
'handling_context',COALESCE(t.document->'handling_context','{}'::jsonb),
|
||||
'latest_report',(SELECT REPORT_SUMMARY FROM workbench.operational_task_reports r
|
||||
WHERE r.project_id=t.project_id AND r.task_id=t.task_id ORDER BY r.revision DESC LIMIT 1),
|
||||
'selected_report',CASE WHEN %s::int IS NULL THEN NULL ELSE
|
||||
@@ -328,6 +331,29 @@ async def task_context_summary(conn: AsyncConnection, *, project_id: UUID, task_
|
||||
return result
|
||||
|
||||
|
||||
def _reference_diagram_projection(stored_sql: str, stored_params: tuple, section: str) -> tuple[str, tuple]:
|
||||
"""Normalize only saved diagram annotations before any data_path or page projection."""
|
||||
if section not in {"progress", "report"}:
|
||||
raise ValueError("reference diagrams only belong to progress or report sections")
|
||||
path = "{data," + section + ",diagram,node_refs}"
|
||||
statement = f"""
|
||||
WITH reference_documents AS (
|
||||
{stored_sql}
|
||||
), reference_aliases AS (
|
||||
SELECT %s::jsonb AS annotations
|
||||
)
|
||||
SELECT CASE WHEN jsonb_typeof(snapshot#>'{path}')='array' THEN
|
||||
jsonb_set(snapshot,'{path}',COALESCE((
|
||||
SELECT jsonb_agg(CASE WHEN annotations ? (node->>'status')
|
||||
THEN jsonb_set(node,'{{status}}',annotations->(node->>'status'))
|
||||
ELSE node END ORDER BY position)
|
||||
FROM jsonb_array_elements(snapshot#>'{path}') WITH ORDINALITY AS nodes(node,position)
|
||||
),'[]'::jsonb)) ELSE snapshot END AS snapshot
|
||||
FROM reference_documents CROSS JOIN reference_aliases
|
||||
"""
|
||||
return statement, (*stored_params, Jsonb(LEGACY_INVESTIGATION_NODE_ANNOTATIONS))
|
||||
|
||||
|
||||
async def task_context_page(
|
||||
conn: AsyncConnection, *, project_id: UUID, task_id: UUID, section: str,
|
||||
report_revision: int | None, data_path: list[str], limit: int, offset: int,
|
||||
@@ -378,18 +404,23 @@ async def task_context_page(
|
||||
where = " ORDER BY revision DESC LIMIT 1"
|
||||
stored = f"SELECT jsonb_build_object('section',%s::text,'data',document) AS snapshot FROM {relation} WHERE project_id=%s AND task_id=%s{where}"
|
||||
elif section == "run":
|
||||
stored = """SELECT jsonb_build_object('section','run','data',to_jsonb(r)-'project_id') AS snapshot
|
||||
stored = """SELECT jsonb_build_object('section','run','data',to_jsonb(r)-'project_id'-'artifact_manifest') AS snapshot
|
||||
FROM workbench.operational_task_runs r WHERE project_id=%s AND task_id=%s ORDER BY generation DESC LIMIT 1"""
|
||||
params = (project_id, task_id)
|
||||
elif section == "progress":
|
||||
stored = """SELECT jsonb_build_object('section','progress','data',p.document) AS snapshot
|
||||
FROM workbench.operational_task_progress p
|
||||
JOIN workbench.operational_task_runs r ON r.project_id=p.project_id AND r.task_id=p.task_id AND r.run_id=p.run_id
|
||||
JOIN workbench.operational_tasks t ON t.project_id=p.project_id AND t.task_id=p.task_id AND t.execution_generation=r.generation
|
||||
WHERE p.project_id=%s AND p.task_id=%s"""
|
||||
JOIN workbench.operational_tasks t ON t.project_id=p.project_id AND t.task_id=p.task_id
|
||||
WHERE p.project_id=%s AND p.task_id=%s AND r.generation<=t.execution_generation
|
||||
AND (t.execution_generation=r.generation
|
||||
OR jsonb_typeof(p.document#>'{progress,diagram,edges}')='array')
|
||||
ORDER BY (t.execution_generation=r.generation) DESC,r.generation DESC,p.revision DESC LIMIT 1"""
|
||||
params = (project_id, task_id)
|
||||
else:
|
||||
raise HTTPException(422, "Unknown task context section")
|
||||
if section in {"progress", "report"}:
|
||||
stored, params = _reference_diagram_projection(stored, params, section)
|
||||
return await _snapshot_page(conn, stored_sql=stored, stored_params=params,
|
||||
data_path=data_path, limit=limit, offset=projection_offset, mcp_sources=[], max_bytes=8000)
|
||||
|
||||
|
||||
@@ -96,7 +96,7 @@ class ScadaReplayRepository:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
"SELECT document FROM peripheral_sample.scada_replay_batches "
|
||||
"WHERE project_id=%s AND replay_id=%s AND stage<>'detected' ORDER BY sequence LIMIT 1",
|
||||
"WHERE project_id=%s AND replay_id=%s AND sequence>0 AND stage<>'detected' ORDER BY sequence LIMIT 1",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
row = await cur.fetchone()
|
||||
@@ -110,6 +110,25 @@ class ScadaReplayRepository:
|
||||
(self.project_id, str(replay_id), batch["sequence"], batch["stage"], Jsonb(batch)),
|
||||
)
|
||||
|
||||
async def clear_unpublished_baseline(self, replay_id) -> None:
|
||||
"""Discard only an incomplete attempt, retaining every published snapshot."""
|
||||
await self.connection.execute(
|
||||
"DELETE FROM peripheral_sample.scada_replay_batches "
|
||||
"WHERE project_id=%s AND replay_id=%s AND sequence<0 "
|
||||
"AND stage='clean_prepared' AND document->>'kind'='baseline'",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
|
||||
async def publish_baseline(self, replay_id) -> None:
|
||||
"""Publish with the replay's baseline metadata in the caller's transaction."""
|
||||
await self.connection.execute(
|
||||
"UPDATE peripheral_sample.scada_replay_batches "
|
||||
"SET stage='detected',document=jsonb_set(document,'{stage}','\"detected\"'::jsonb) "
|
||||
"WHERE project_id=%s AND replay_id=%s AND sequence<0 "
|
||||
"AND stage='clean_prepared' AND document->>'kind'='baseline'",
|
||||
(self.project_id, str(replay_id)),
|
||||
)
|
||||
|
||||
async def events(self, replay_id, offset=0, limit=100) -> list[dict]:
|
||||
async with self.connection.transaction():
|
||||
cur = await self.connection.execute(
|
||||
@@ -180,7 +199,7 @@ class ScadaReplayRepository:
|
||||
"SELECT DISTINCT ON(r.value->>'device_id') r.value AS reading "
|
||||
"FROM peripheral_sample.scada_replay_batches b "
|
||||
"CROSS JOIN LATERAL jsonb_array_elements(COALESCE(b.document->'outputs','[]'::jsonb)) r "
|
||||
"WHERE b.project_id=%s AND b.replay_id=%s AND b.stage IN ('cleaned','detected') "
|
||||
"WHERE b.project_id=%s AND b.replay_id=%s AND b.sequence>0 AND b.stage IN ('cleaned','detected') "
|
||||
"AND (r.value->>'time')::timestamptz<=%s "
|
||||
"ORDER BY r.value->>'device_id',(r.value->>'time')::timestamptz DESC LIMIT %s",
|
||||
(self.project_id, str(replay_id), through, limit),
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from typing import List, Any
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timedelta
|
||||
from collections import defaultdict
|
||||
from psycopg import AsyncConnection, Connection, sql
|
||||
from psycopg.rows import dict_row
|
||||
@@ -9,6 +9,11 @@ class ScadaRepository:
|
||||
VALUE_FIELDS = frozenset({"monitored_value", "cleaned_value"})
|
||||
RESULT_COLUMNS = "time, device_id, monitored_value, cleaned_value"
|
||||
|
||||
@staticmethod
|
||||
async def begin_replay_baseline_snapshot(conn: AsyncConnection) -> None:
|
||||
"""Set the caller's source transaction to a repeatable read-only snapshot."""
|
||||
await conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
|
||||
|
||||
@staticmethod
|
||||
async def get_warmup_values(
|
||||
conn: AsyncConnection,
|
||||
@@ -126,6 +131,29 @@ class ScadaRepository:
|
||||
)
|
||||
return await cur.fetchall()
|
||||
|
||||
@staticmethod
|
||||
async def get_recovery_scada_readings(
|
||||
conn: AsyncConnection, device_ids: list[str], incident_at: datetime,
|
||||
returned_at: datetime, through: datetime, *, baseline_limit: int = 100_001,
|
||||
) -> list[dict]:
|
||||
"""Read the causal reference and exactly the first stored frame after T."""
|
||||
if not device_ids or through <= returned_at:
|
||||
return []
|
||||
async with conn.cursor() as cur:
|
||||
await cur.execute(
|
||||
"WITH next_frame AS (SELECT min(time) AS time FROM scada.measurements "
|
||||
"WHERE device_id=ANY(%s) AND time>%s AND time<=%s), "
|
||||
"reference AS (SELECT time,device_id,monitored_value,cleaned_value "
|
||||
"FROM scada.measurements WHERE device_id=ANY(%s) AND time>=%s AND time<%s "
|
||||
"ORDER BY time,device_id LIMIT %s) "
|
||||
"SELECT time,device_id,monitored_value,cleaned_value FROM reference UNION ALL "
|
||||
"SELECT m.time,m.device_id,m.monitored_value,m.cleaned_value FROM scada.measurements m "
|
||||
"JOIN next_frame f ON m.time=f.time WHERE m.device_id=ANY(%s) ORDER BY time,device_id",
|
||||
(device_ids, returned_at, through, device_ids, incident_at - timedelta(days=15),
|
||||
incident_at, baseline_limit, device_ids),
|
||||
)
|
||||
return [dict(row) for row in await cur.fetchall()]
|
||||
|
||||
@staticmethod
|
||||
async def get_latest_scada_readings(
|
||||
conn: AsyncConnection,
|
||||
|
||||
@@ -194,6 +194,7 @@ class HttpOperationalActionDispatcher:
|
||||
raise ValueError("invalid external project or timestamp")
|
||||
receipt = ActionReceipt(
|
||||
adapter="peripheral_sample", receipt_id=str(order.work_order_id),
|
||||
# This is the creation action's completion, not field repair.
|
||||
submitted_at=order.created_at, completed_at=order.created_at,
|
||||
external_status=order.status,
|
||||
detail="外围工单已创建,现场工作进度由外围系统继续同步",
|
||||
|
||||
+19
-19
@@ -15,7 +15,7 @@ MAX_REQUEST_TIMEOUT_SECONDS = 240
|
||||
class CalculationInput(StrictInput):
|
||||
timeout_seconds: int = Field(
|
||||
default=120, ge=1, le=MAX_REQUEST_TIMEOUT_SECONDS, strict=True,
|
||||
description="本次HTTP请求响应等待预算,秒,默认120秒;不是后台计算执行时限,受工具目录中的请求上限约束;后台任务通过状态和结果工具查询",
|
||||
description="本次HTTP请求响应等待预算,秒,默认120秒;后台执行时限由作业系统管理,受工具目录中的请求上限约束;后台任务通过状态和结果工具查询",
|
||||
)
|
||||
|
||||
|
||||
@@ -28,13 +28,13 @@ class NodeCoverageGroup(BaseModel):
|
||||
|
||||
class NodeCoverageInput(CalculationInput):
|
||||
groups: list[NodeCoverageGroup] = Field(min_length=1, max_length=8,
|
||||
description="已核实的模型节点集合;各组为需要比较的业务或空间集合,可重叠,不接受任意坐标")
|
||||
description="已核实的模型节点集合;各组为需要比较的业务或空间集合,可重叠,位置通过模型节点关联获取")
|
||||
concave_ratio: float = Field(default=0.25, ge=0, le=1, allow_inf_nan=False,
|
||||
description="凹包比例;越接近1越平滑接近凸包,参数只决定示意形状")
|
||||
buffer_m: float = Field(default=25, gt=0, le=1000, allow_inf_nan=False,
|
||||
description="展示缓冲距离,米;不是服务半径或影响距离")
|
||||
description="展示缓冲距离,米;用于几何展示")
|
||||
split_by_connectivity: bool = Field(default=False, strict=True,
|
||||
description="需要按模型连接分组时设true;地理分布比较默认false,不能据此认定实际供水关系")
|
||||
description="需要按模型连接分组时设true;地理分布比较默认false,供水关系按拓扑核验")
|
||||
excluded_link_ids: list[Id] = Field(default_factory=list, max_length=200,
|
||||
description="连通分组时额外视为断开的管段/阀门,例如已有隔离建议中的关阀ID;仅影响绘图分组")
|
||||
|
||||
@@ -50,7 +50,7 @@ class NodeCoverageInput(CalculationInput):
|
||||
|
||||
|
||||
class BurstDetectionInput(CalculationInput):
|
||||
target_time: datetime = Field(description="带时区的侦测时刻;读取此前15天同周期压力数据,不是单小时定位")
|
||||
target_time: datetime = Field(description="带时区的侦测时刻;读取此前15天同周期压力数据,返回压力异常诊断")
|
||||
sensor_nodes: list[Id] = Field(min_length=5, max_length=20, description="已核实的压力监测节点ID,至少5个完整传感器")
|
||||
sampling_interval_minutes: int = Field(default=15, ge=1, le=60)
|
||||
|
||||
@@ -67,14 +67,14 @@ class BurstDetectionInput(CalculationInput):
|
||||
|
||||
class BurstLocationInput(CalculationInput):
|
||||
data_source: Literal["monitoring", "simulation"] = "monitoring"
|
||||
simulation_run_id: UUID | None = Field(default=None, description="已有分析运行UUID,不是方案名称;simulation模式只读取已有异常结果及模型基线")
|
||||
burst_leakage: float = Field(gt=0, allow_inf_nan=False, description="当前爆管定位算法必需的假定漏水流量,WNTR SI单位m³/s;可用有数据和方法依据的估计或明确给定的情景假设,说明来源与不确定性,不要求现场实测确认;不能任填数值")
|
||||
pressure_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="压力监测设备关联的模型节点ID,不是设备查询编号;省略时自动读取当前项目SCADA配置中的压力测点,不需要猜测节点ID")
|
||||
flow_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="流量监测设备关联的模型要素ID,不是设备查询编号")
|
||||
simulation_run_id: UUID | None = Field(default=None, description="已有分析运行UUID;simulation模式只读取已有异常结果及模型基线")
|
||||
burst_leakage: float | None = Field(default=None, gt=0, allow_inf_nan=False, description="水力定位必需的假定漏水流量,WNTR SI单位m³/s;使用有依据的估计或明确给定的情景假设,保留估计方法与输入来源。仅 Server 明确开放的候选定位路径可省略,该路径直接返回候选管段与相似度;其他项目和时段省略将被拒绝")
|
||||
pressure_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="压力监测设备关联的模型节点ID,查询编号通过设备映射转换;省略时读取项目配置的压力测点")
|
||||
flow_scada_ids: list[Id] | None = Field(default=None, min_length=1, max_length=20, description="流量监测设备关联的模型要素ID,由设备映射取得")
|
||||
scada_burst_start: datetime
|
||||
scada_burst_end: datetime
|
||||
scada_normal_start: datetime | None = Field(default=None, description="monitoring模式正常对照窗开始,须与结束时间成对提供;均省略时用爆管窗前一天相同时段,不代表该时段已确认正常")
|
||||
scada_normal_end: datetime | None = Field(default=None, description="monitoring模式正常对照窗结束,须与开始时间成对提供;simulation模式不填写,使用同时间窗模型基线")
|
||||
scada_normal_start: datetime | None = Field(default=None, description="monitoring模式正常对照窗开始,须与结束时间成对提供;均省略时用爆管窗前一天相同时段,使用前须核对正常对照有效性")
|
||||
scada_normal_end: datetime | None = Field(default=None, description="monitoring模式正常对照窗结束,须与开始时间成对提供;simulation模式省略此项,使用同时间窗模型基线")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_windows(self):
|
||||
@@ -100,14 +100,14 @@ class BurstLocationInput(CalculationInput):
|
||||
class ValveIsolationInput(CalculationInput):
|
||||
accident_element: list[Id] = Field(min_length=1, max_length=20, description="经核实或明确作为候选的事故管段/节点ID")
|
||||
accident_element_type: Literal["auto", "link", "node"] = Field(default="auto", description="事故要素类型;auto优先识别管段,再识别节点;节点与管段重号时,节点分析须指定node,纯管段分析可指定link")
|
||||
disabled_valves: list[Id] = Field(default_factory=list, max_length=200, description="已知无法关闭的阀门;未提供不代表现场确认全部可用")
|
||||
disabled_valves: list[Id] = Field(default_factory=list, max_length=200, description="已知无法关闭的阀门;现场可操作性按执行前核验记录确认")
|
||||
|
||||
|
||||
class LeakageIdentificationInput(CalculationInput):
|
||||
scada_start: datetime
|
||||
scada_end: datetime
|
||||
sensor_nodes: list[Id] = Field(min_length=1, max_length=20)
|
||||
q_sum: float = Field(gt=0, allow_inf_nan=False, description="有依据的总漏损流量约束,m³/s,不得使用无依据默认值")
|
||||
q_sum: float = Field(gt=0, allow_inf_nan=False, description="有依据的总漏损流量约束,m³/s,保留数据来源与估算方法")
|
||||
start_time: float = Field(ge=0, lt=24, allow_inf_nan=False, description="模型起算小时;需与SCADA时间窗及模型时钟对齐")
|
||||
timestep: int = Field(default=15, ge=5, le=60, description="计算步长,分钟;需与观测采样对应")
|
||||
dma_count: int = Field(ge=1, le=20)
|
||||
@@ -129,7 +129,7 @@ class LeakageIdentificationInput(CalculationInput):
|
||||
class ScenarioInput(CalculationInput):
|
||||
start_time: datetime
|
||||
duration: int = Field(default=900, ge=60, le=86400, description="模拟时长,秒;首次优先短时间窗")
|
||||
output_node_ids: list[Id] = Field(min_length=1, max_length=20, description="需要返回结果的节点;不把局部结果解释为全网无影响")
|
||||
output_node_ids: list[Id] = Field(min_length=1, max_length=20, description="需要返回结果的节点;结论范围限定为返回对象")
|
||||
output_link_ids: list[Id] = Field(default_factory=list, max_length=20)
|
||||
|
||||
@model_validator(mode="after")
|
||||
@@ -140,18 +140,18 @@ class ScenarioInput(CalculationInput):
|
||||
|
||||
|
||||
class ContaminantSimulationInput(ScenarioInput):
|
||||
source: Id = Field(description="假设污染源节点;这是正向传播模拟,不是自动反演污染源")
|
||||
source: Id = Field(description="假设污染源节点;这是正向传播模拟,结果按给定源解释")
|
||||
concentration: float = Field(gt=0, allow_inf_nan=False, description="假设源浓度,mg/L")
|
||||
pattern: Id | None = Field(default=None, description="已存在的污染源时间模式ID;不填为持续恒定源")
|
||||
pattern: Id | None = Field(default=None, description="已存在的污染源时间模式ID;省略时采用持续恒定源")
|
||||
|
||||
|
||||
class ValveClosureInput(ScenarioInput):
|
||||
valves: list[Id] = Field(min_length=1, max_length=20, description="在临时模型中关闭的阀门,不下发现场控制")
|
||||
valves: list[Id] = Field(min_length=1, max_length=20, description="在临时模型中关闭的阀门,现场操作另走审批流程")
|
||||
|
||||
|
||||
class FlushingInput(ScenarioInput):
|
||||
drainage_node_id: Id
|
||||
flushing_flow_m3h: float = Field(gt=0, allow_inf_nan=False, description="明确的冲洗流量,m³/h;不使用无依据的自动排放量")
|
||||
flushing_flow_m3h: float = Field(gt=0, allow_inf_nan=False, description="明确的冲洗流量,m³/h;提供流量来源与估算依据")
|
||||
closed_valves: list[Id] = Field(default_factory=list, max_length=20)
|
||||
|
||||
|
||||
@@ -160,7 +160,7 @@ PumpSeries = Annotated[list[NonnegativeValue], Field(min_length=1, max_length=96
|
||||
|
||||
|
||||
class PressureRegulationInput(ScenarioInput):
|
||||
pump_control: dict[Id, PumpSeries] = Field(min_length=1, max_length=20, description="已登记泵ID及运行序列;工频值为0/1,变频值为频率Hz,每项对应模型水力步长,不自动优化")
|
||||
pump_control: dict[Id, PumpSeries] = Field(min_length=1, max_length=20, description="已登记泵ID及运行序列;工频值为0/1,变频值为频率Hz,每项对应模型水力步长,按给定序列评估")
|
||||
tank_init_level: dict[Id, Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(default_factory=dict, max_length=20, description="水箱初始水位,须在模型允许范围内")
|
||||
|
||||
|
||||
|
||||
+17
-4
@@ -32,6 +32,8 @@ from app.mcp.catalog import TOOLS
|
||||
from app.mcp.config import McpSettings
|
||||
from app.mcp.contexts import resolve_context, verify_task
|
||||
from app.mcp.models import QueryResult
|
||||
from app.mcp.display_time import display_query_result, display_time_text
|
||||
from app.mcp.scada_catalog import compact_scada_catalog
|
||||
from app.mcp.errors import CalculationCapacityExceeded
|
||||
from app.mcp.scope import project_scope
|
||||
from app.mcp.query_service import execute_query, task_service, authorize_query
|
||||
@@ -43,6 +45,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def tool_error(code, message):
|
||||
message = display_time_text(message)
|
||||
return types.CallToolResult(
|
||||
isError=True,
|
||||
content=[types.TextContent(type="text", text=message)],
|
||||
@@ -358,7 +361,8 @@ class McpQueryApplication:
|
||||
result = QueryResult.model_validate(previous["result"])
|
||||
else:
|
||||
from app.mcp.calculation_jobs import should_queue, submit_calculation
|
||||
if should_queue(tool, args):
|
||||
queued = should_queue(tool, args, project_code=getattr(scope.project, "project_code", None))
|
||||
if queued:
|
||||
result = await submit_calculation(
|
||||
scope, binding, tool, args, call_id,
|
||||
job_timeout_seconds=self.settings.max_calculation_timeout_seconds,
|
||||
@@ -377,7 +381,7 @@ class McpQueryApplication:
|
||||
"RESULT_TOO_LARGE",
|
||||
"Narrow the query or request a smaller page",
|
||||
)
|
||||
if binding.task_id and not should_queue(tool, args) and name not in {
|
||||
if binding.task_id and not queued and name not in {
|
||||
"get_calculation_job", "get_calculation_result", "cancel_calculation_job", "list_calculation_jobs",
|
||||
"get_task_context",
|
||||
"get_task_evidence",
|
||||
@@ -422,11 +426,20 @@ class McpQueryApplication:
|
||||
parameters,
|
||||
result.model_dump(mode="json"),
|
||||
)
|
||||
output = result.model_dump(mode="json")
|
||||
output = display_query_result(result.model_dump(mode="json"))
|
||||
content = []
|
||||
from app.mcp.extended_service import ARTIFACT_TOOLS
|
||||
# Complete SCADA inputs must reach the model intact, not as a first-page preview.
|
||||
# The shared 8 MiB result ceiling above still applies before persistence and output.
|
||||
complete_scada = name == "query_scada_readings" and args.device_ids is None
|
||||
if (not include_full and name == "list_scada_devices"
|
||||
and len(json.dumps(output, ensure_ascii=False).encode()) > 12000):
|
||||
preview = compact_scada_catalog(output, budget=12000)
|
||||
if preview is not None:
|
||||
output = preview
|
||||
if (
|
||||
not include_full
|
||||
and not complete_scada
|
||||
and name not in ARTIFACT_TOOLS
|
||||
and len(json.dumps(output, ensure_ascii=False).encode()) > 12000
|
||||
):
|
||||
@@ -444,7 +457,7 @@ class McpQueryApplication:
|
||||
from app.mcp.presentation_preview import saved_presentation_preview
|
||||
presentation = saved_presentation_preview(result)
|
||||
if presentation is not None:
|
||||
output["data"]["presentation"] = presentation
|
||||
output["data"]["presentation"] = display_query_result(presentation)
|
||||
output["truncated"] = True
|
||||
output["data_gaps"] = [
|
||||
*result.data_gaps,
|
||||
|
||||
@@ -29,7 +29,11 @@ def restore_binding(document: dict) -> BoundContext:
|
||||
return BoundContext.model_validate({**document, "context_handle": "durable-calculation", "expires_at": datetime.max.replace(tzinfo=timezone.utc)})
|
||||
|
||||
|
||||
def should_queue(tool, args) -> bool:
|
||||
def should_queue(tool, args, *, project_code: str | None = None) -> bool:
|
||||
if tool.name == "run_burst_location":
|
||||
from app.services.burst_location_demo import is_location_demo
|
||||
if is_location_demo(project_code, args):
|
||||
return False
|
||||
return tool.name in QUEUED_TOOLS and (not tool.mutating or not args.dry_run)
|
||||
|
||||
|
||||
@@ -39,7 +43,7 @@ def envelope(source: str, data, *, job_id="list", page=None) -> QueryResult:
|
||||
|
||||
|
||||
def public_job(row: dict) -> dict:
|
||||
fields = {"job_id", "status", "tool", "created_at", "started_at", "finished_at", "updated_at", "error", "queue_expires_at", "result_expires_at"}
|
||||
fields = {"job_id", "status", "tool", "created_at", "started_at", "finished_at", "updated_at", "error", "result_expires_at"}
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
expiry = row.get("result_expires_at")
|
||||
available = row.get("status") == "succeeded" and (expiry is None or expiry > datetime.now(timezone.utc))
|
||||
|
||||
+46
-35
@@ -23,22 +23,33 @@ class QueryTool:
|
||||
|
||||
def protocol_tool(self) -> types.Tool:
|
||||
if self.mutating:
|
||||
suffix = "。默认dry_run只预览;执行必须携带预览摘要、相同request_id并通过宿主原生许可。仅项目普通聊天可用,后台调查和reader禁止;失败或超时先核实状态,不自动重试。"
|
||||
suffix = "。默认dry_run只预览;执行必须携带预览摘要、相同request_id并通过宿主原生许可。由项目成员的普通聊天执行写入;失败或超时先核实状态,再按原请求接续。"
|
||||
elif self.name == "run_simulation":
|
||||
suffix = "。任务调用复用证据留存。"
|
||||
elif self.computation:
|
||||
suffix = "。仅计算建议,不保存方案、不控制设备;任务调用复用证据留存。"
|
||||
suffix = "。返回分析建议并留存任务证据;正式方案与现场操作按各自业务流程执行。"
|
||||
elif self.name == "query_nearby_elements":
|
||||
suffix = "。仅查询当前已授权项目的模型管网,按距离和元素ID稳定排序;按需分页,不自动遍历全量;空间邻近不代表管网连通或供水关系。"
|
||||
suffix = "。仅查询当前已授权项目的模型管网,按距离和元素ID稳定排序;按核查目标分页;连接与供水关系由拓扑分析核验。"
|
||||
elif self.name == "list_scada_devices":
|
||||
suffix = (
|
||||
"。默认每页100个,最多500个;page.has_more=true时继续递增offset读取后续页面。"
|
||||
"大目录内联保留本页全部设备的编号、类型和单位,模型映射及坐标用get_scada_device核验。"
|
||||
)
|
||||
elif self.name == "query_scada_readings":
|
||||
suffix = (
|
||||
"。首次调查省略device_ids,Server读取完整设备属性并自动翻页读取窗口内全部授权观测,"
|
||||
"返回devices、readings和coverage,包括缺测及冻结范围外设备;全量结果完整内联。"
|
||||
"显式device_ids仅用于后续局部复核,不能替代全量核查。"
|
||||
)
|
||||
else:
|
||||
suffix = (
|
||||
"。先限定对象和时间范围,再读取明细;集合默认20条,按需逐页扩大,"
|
||||
"不要自动遍历全量;截断或非零offset不能证明完整范围无异常。"
|
||||
"按当前目标读取必要页面,并说明截断和分页覆盖范围。"
|
||||
)
|
||||
return types.Tool(
|
||||
name=self.name,
|
||||
description=self.description + suffix + (
|
||||
"。执行返回持久job_id;排队回执不是分析证据,使用get_calculation_job查询状态、get_calculation_result分页读取成功结果,不重复提交。"
|
||||
"。返回queued/running和job_id时,使用get_calculation_job查询状态、get_calculation_result读取成功结果,排队回执用于跟踪;直接返回分析结果时立即读取。"
|
||||
if self.name in {"run_burst_location", "run_leakage_identification", "run_burst_detection", "run_simulation", "run_burst_simulation", "run_contaminant_simulation", "analyze_valve_closure", "analyze_flushing", "analyze_pressure_regulation", "optimize_sensor_placement", "save_simulation_scenario", "create_sensor_placement"}
|
||||
else ""
|
||||
),
|
||||
@@ -51,17 +62,17 @@ class QueryTool:
|
||||
|
||||
|
||||
_specs = [
|
||||
("run_simulation", "按水力步长在在线业务模型上执行指定时间段的实时模拟,不复制数据库;更新该时段的模型条件及实时基线,返回指定节点和管段结果;duration单位为秒,不创建方案或正式分析运行,结果不是实测", ScenarioInput, "simulation.run", "analysis", False, True),
|
||||
("run_contaminant_simulation", "在隔离模型中模拟假设污染源的浓度传播;返回指定节点/管段结果,不反演污染源,不代表实测污染范围", ContaminantSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_valve_closure", "在隔离模型中模拟关阀的水力影响,返回指定节点/管段结果;不操作真实阀门", ValveClosureInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_flushing", "在隔离模型中评估指定排放节点、冲洗流量和关阀组合的水力结果;不是现场冲洗或消毒合格证明", FlushingInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_pressure_regulation", "在隔离模型中评估给定调泵序列和水箱初始水位的压力影响;不自动优化调度,不实际控制泵", PressureRegulationInput, "simulation.run", "analysis", False, True),
|
||||
("optimize_sensor_placement", "使用灵敏度或K-means算法推荐压力监测点;仅返回候选节点,不保存正式方案,不保证污染溯源可观测性", SensorPlacementInput, "optimization.run", "analysis", False, True),
|
||||
("run_leakage_identification", "基于SCADA压力与模型进行DMA漏损分配识别,需有依据的总漏损流量和时间对齐;返回区域估计,不是已确认漏点。内部执行遗传算法与水力计算", LeakageIdentificationInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_detection", "爆管异常侦测:使用至少5个完整压力传感器此前15天数据;只提示压力异常,不定位、不证明爆管", BurstDetectionInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_location", "爆管候选定位:对比异常与正常压力,可辅助流量;burst_leakage必需,可使用有依据的估计或明确给定的情景假设,不要求现场确诊;本工具不反演总漏水量。压力节点可省略,自动读取项目SCADA压力测点;monitoring正常窗可成对省略,默认前一天同时间段,须说明该对照假设。simulation须有已有分析运行UUID并读取同窗模型基线,不是实测;定位内部会执行水力计算,不创建新的事件方案", BurstLocationInput, "burst.run", "analysis", False, True),
|
||||
("analyze_valve_isolation", "按事故候选要素分析拓扑隔离边界、建议关闭阀门和受影响节点;候选分析无需等待现场确诊。输出不是污染扩散或水力影响范围;现场操作前须核实阀门映射及可用性,本工具不执行关阀", ValveIsolationInput, "simulation.run", "analysis", False, True),
|
||||
("generate_node_coverage", "为当前项目已核实的模型节点集合生成空间覆盖示意,适用于候选分布、监测覆盖比较、区域成员、隔离或模拟结果节点等,不限调查任务。支持最多8组、合计10000个不同节点,从真实GIS查询坐标;监测设备、投诉或地址须先关联模型节点。可按连接分组,记录凹包/缓冲参数、缺失节点和display_only;不计算故障、污染、实际停水或服务边界。返回result.map_payload及coverage,任务中用set_overlay引用本次evidence_id、省略data展示;生成不自动操作地图", NodeCoverageInput, "webgis.view", "analysis", False, True),
|
||||
("run_simulation", "按水力步长在在线业务模型上执行指定时间段的实时模拟;更新该时段的模型条件及实时基线,返回指定节点和管段结果;duration单位为秒,输出为在线模型结果,正式分析成果通过方案工具保存", ScenarioInput, "simulation.run", "analysis", False, True),
|
||||
("run_contaminant_simulation", "在隔离模型中模拟假设污染源的浓度传播;返回指定节点/管段结果,结果标注假设来源及模型预测范围", ContaminantSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_valve_closure", "在隔离模型中模拟关阀的水力影响,返回指定节点/管段结果;现场阀门操作按业务审批执行", ValveClosureInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_flushing", "在隔离模型中评估指定排放节点、冲洗流量和关阀组合的水力结果;现场效果由执行回执和复测核验", FlushingInput, "simulation.run", "analysis", False, True),
|
||||
("analyze_pressure_regulation", "在隔离模型中评估给定调泵序列和水箱初始水位的压力影响;输出为给定方案评估,现场调度按审批执行", PressureRegulationInput, "simulation.run", "analysis", False, True),
|
||||
("optimize_sensor_placement", "使用灵敏度或K-means算法推荐压力监测点;返回候选节点,正式部署另行保存并核验观测能力", SensorPlacementInput, "optimization.run", "analysis", False, True),
|
||||
("run_leakage_identification", "基于SCADA压力与模型进行DMA漏损分配识别,需有依据的总漏损流量和时间对齐;返回待现场核验的区域估计。内部执行遗传算法与水力计算", LeakageIdentificationInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_detection", "爆管异常侦测:使用至少5个完整压力传感器此前15天数据;返回压力异常信号,候选管段由定位工具分析", BurstDetectionInput, "burst.run", "analysis", False, True),
|
||||
("run_burst_location", "爆管候选定位:对比异常与正常压力,可辅助流量;依据现有异常形成候选,水力算法使用有依据的burst_leakage作为输入。任务上下文location_capability.available=true时可省略漏量,Server直接返回候选管段与相似度,随后继续地图展示、隔离边界与影响分析。压力节点可省略,自动读取项目配置;monitoring正常窗省略时默认前一天同时间段,须说明假设;回放必须显式提供冻结正常窗。simulation须有已有分析运行UUID及同窗模型基线,结果标注模型来源", BurstLocationInput, "burst.run", "analysis", False, True),
|
||||
("analyze_valve_isolation", "按事故候选要素分析拓扑隔离边界、建议关闭阀门和受影响节点;候选分析无需等待现场确诊。输出为拓扑隔离与供水失源范围;水力影响另用关阀模拟评估,现场操作前核实阀门映射及可用性", ValveIsolationInput, "simulation.run", "analysis", False, True),
|
||||
("generate_node_coverage", "为当前项目已核实的模型节点集合生成空间覆盖示意,适用于候选分布、监测覆盖比较、区域成员、隔离或模拟结果节点等,支持各类项目分析。支持最多8组、合计10000个唯一节点,从真实GIS查询坐标;监测设备、投诉或地址须先关联模型节点。可按连接分组,记录凹包/缓冲参数、缺失节点和display_only;结果为节点空间覆盖示意,保留分组及几何覆盖范围。返回result.map_payload及coverage,任务中用set_overlay引用本次evidence_id、省略data展示;地图通过展示工具发布", NodeCoverageInput, "webgis.view", "analysis", False, True),
|
||||
(
|
||||
"get_element_properties",
|
||||
"读取当前项目的单个管网元素属性",
|
||||
@@ -71,14 +82,14 @@ _specs = [
|
||||
),
|
||||
(
|
||||
"list_elements",
|
||||
"分页读取管网元素,不自动读取全网",
|
||||
"按查询目标分页读取管网元素",
|
||||
m.ElementPageInput,
|
||||
"webgis.view",
|
||||
"asset",
|
||||
),
|
||||
(
|
||||
"query_nearby_elements",
|
||||
"查询坐标附近的模型元素,默认pipe;center为EPSG:4326 [经度,纬度],可直接使用geocode.center;radius_m默认200米、最大10000米,limit默认5、最大100,支持offset。按点到完整几何的WGS84椭球最短距离筛选并排序,返回element_id、distance_m和属性;不查询原始GIS管线、不自动扩大半径、不解析道路交叉点",
|
||||
"查询坐标附近的模型元素,默认pipe;center为EPSG:4326 [经度,纬度],可直接使用geocode.center;radius_m默认200米、最大10000米,limit默认5、最大100,支持offset。按点到完整几何的WGS84椭球最短距离筛选并排序,返回element_id、distance_m和属性;查询范围为当前模型管网;原始GIS、范围调整与歧义地点按各自入口核验",
|
||||
m.NearbyElementsInput,
|
||||
"webgis.view",
|
||||
"asset",
|
||||
@@ -100,7 +111,7 @@ _specs = [
|
||||
(
|
||||
"list_scada_devices",
|
||||
"分页读取SCADA设备和物理单位",
|
||||
m.PageInput,
|
||||
m.ScadaDeviceListInput,
|
||||
"scada.view",
|
||||
"asset",
|
||||
),
|
||||
@@ -114,7 +125,7 @@ _specs = [
|
||||
),
|
||||
(
|
||||
"query_model_baseline_series",
|
||||
"读取最多20个元素的模型基线,不能称为实测",
|
||||
"读取最多20个元素的模型基线,结果标注模型来源",
|
||||
m.BaselineSeriesInput,
|
||||
"simulation.view",
|
||||
"model_baseline",
|
||||
@@ -206,7 +217,7 @@ _specs = [
|
||||
),
|
||||
(
|
||||
"get_task_context",
|
||||
"读取绑定任务的有界摘要及handling_skill元数据;section=skill读取Server按任务workflow分发的业务处置Skill正文与版本,属于流程指引而非事件证据;其他section选择task/report/evidence/actions/progress/access_requests/plan/run,data_path按返回路径读取嵌套字段,limit/offset分页;report_revision保留选定报告版本",
|
||||
"读取绑定任务的有界摘要及handling_skill元数据;section=skill读取Server按任务workflow分发的业务处置Skill正文与版本,属于流程指引而非事件证据;其他section选择task/report/evidence/actions/progress/access_requests/plan/run,data_path按返回路径读取嵌套字段,limit/offset分页;progress/report图节点仅为流程参考标记,旧执行态兼容转换,不代表实时进度;实际执行状态读run、动作状态读actions;report_revision保留选定报告版本",
|
||||
m.TaskContextInput,
|
||||
"operational_task.view",
|
||||
"task",
|
||||
@@ -230,40 +241,40 @@ _specs = [
|
||||
]
|
||||
_specs += [
|
||||
("list_regions", "分页查询当前项目区域/DMA列表", m.PageInput, "webgis.view", "asset"),
|
||||
("get_region", "查询当前项目区域边界,返回EPSG:4326;不等于供水拓扑", e.RegionInput, "webgis.view", "asset"),
|
||||
("get_region", "查询当前项目区域边界,返回EPSG:4326几何;供水关系按拓扑核验", e.RegionInput, "webgis.view", "asset"),
|
||||
("get_region_nodes", "分页查询区域配置的成员节点,未配置时使用区域几何筛选", e.RegionInput, "webgis.view", "asset"),
|
||||
("get_node_connections", "分页查询节点直接连接的管线,不自动遍历全网", e.NodeConnectionsInput, "webgis.view", "asset"),
|
||||
("query_network_extent", "按EPSG:4326矩形范围分页查询当前项目模型几何;无GIS配置不跨项目回退", e.ExtentInput, "webgis.view", "asset"),
|
||||
("get_node_connections", "按目标节点分页查询直接连接的管线", e.NodeConnectionsInput, "webgis.view", "asset"),
|
||||
("query_network_extent", "按EPSG:4326矩形范围分页查询当前项目模型几何;GIS缺失时返回当前项目的具体缺口", e.ExtentInput, "webgis.view", "asset"),
|
||||
("list_major_nodes", "按关联管道最小管径查询骨干节点,经纬度EPSG:4326", e.MajorNodesInput, "webgis.view", "asset"),
|
||||
("get_model_configuration", "读取模式、曲线、控制规则、需水量、水质及反应等模型配置;集合按页返回", e.ModelConfigurationInput, "webgis.view", "asset"),
|
||||
("list_model_configurations", "分页读取模式或曲线ID", e.ConfigurationListInput, "webgis.view", "asset"),
|
||||
("search_project_places", "按项目配置的行政区搜索地名,最多10个候选,依赖天地图配置", e.PlaceSearchInput, "webgis.view", "asset"),
|
||||
("reverse_geocode", "把EPSG:4326地图取点转为项目范围内地名,保留输入坐标", e.ReverseGeocodeInput, "webgis.view", "asset"),
|
||||
("get_project_info", "读取当前项目概况与地图范围,不返回数据库路由或凭据", m.StrictInput, "webgis.view", "asset"),
|
||||
("get_project_info", "读取当前项目概况与地图范围,公开字段限于项目业务概况", m.StrictInput, "webgis.view", "asset"),
|
||||
("export_project_model", "从当前项目数据库导出INP或ChangeSet压缩文件,由宿主保存到会话文件", e.ProjectExportInput, "webgis.view", "asset"),
|
||||
("get_sensor_placement_candidate", "读取监测点候选节点工程属性", e.SensorCandidateInput, "optimization.run", "asset"),
|
||||
("export_sensor_placement", "导出已有监测点方案的Excel工程清单,由宿主保存到会话文件", e.SensorExportInput, "optimization.run", "analysis"),
|
||||
("list_task_schedules", "分页读取当前项目定时任务配置,不启动调度", m.PageInput, "operational_task.view", "task"),
|
||||
("list_task_schedules", "分页读取当前项目定时任务配置,执行状态由调度系统管理", m.PageInput, "operational_task.view", "task"),
|
||||
("get_task_run_history", "分页读取指定项目任务的运行历史", e.TaskHistoryInput, "operational_task.view", "task"),
|
||||
("get_mutation_status", "读取当前身份和会话内的写入执行状态;started只表示已启动,超时后须核实目标数据,不自动重试", e.MutationStatusInput, "webgis.view", "task"),
|
||||
("calculate_demand_allocation", "按管长权重分配总需水量到节点、区域或全网,只计算不修改模型,按页返回", e.DemandAllocationInput, "webgis.view", "analysis", False, True),
|
||||
("run_burst_simulation", "在隔离模型中模拟假设管道破口面积的水力影响;不是爆管侦测或定位,不保存正式方案", e.BurstSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("edit_model_element", "预览或增删改模型节点、管段、泵阀、水池和区域;仅修改模型,不控制现场设备", e.ModelElementMutationInput, "webgis.edit", "asset", False, False, True),
|
||||
("get_mutation_status", "读取当前身份和会话内的写入执行状态;started只表示已启动,超时后先核实目标数据,再按原请求接续", e.MutationStatusInput, "webgis.view", "task"),
|
||||
("calculate_demand_allocation", "按管长权重分配总需水量到节点、区域或全网,返回分配建议,模型变更通过业务写入执行,按页返回", e.DemandAllocationInput, "webgis.view", "analysis", False, True),
|
||||
("run_burst_simulation", "在隔离模型中模拟假设管道破口面积的水力影响;返回给定情景的模拟结果,正式方案通过方案工具保存", e.BurstSimulationInput, "simulation.run", "analysis", False, True),
|
||||
("edit_model_element", "预览或增删改模型节点、管段、泵阀、水池和区域;变更范围为业务模型,现场设备操作按对应流程执行", e.ModelElementMutationInput, "webgis.edit", "asset", False, False, True),
|
||||
("edit_model_configuration", "预览或修改模型需水量、模式、曲线、控制规则、水质及模拟选项", e.ConfigurationMutationInput, "webgis.edit", "asset", False, False, True),
|
||||
("clean_scada_readings", "预览或清洗明确设备与24小时内时间窗的数据,写入cleaned_value", e.ScadaCleaningInput, "scada.clean", "observed", False, False, True),
|
||||
("write_scada_readings", "预览或补录/修正SCADA数据,要求真实来源依据;不把模拟当实测", e.ScadaWriteInput, "scada.clean", "observed", False, False, True),
|
||||
("write_scada_readings", "预览或补录/修正SCADA数据,要求真实来源依据;观测来源按真实记录核验", e.ScadaWriteInput, "scada.clean", "observed", False, False, True),
|
||||
("delete_scada_readings", "预览或删除明确设备与24小时内时间窗的SCADA记录", e.ScadaDeleteInput, "scada.clean", "observed", False, False, True),
|
||||
("save_simulation_scenario", "预览或重新计算并保存正式隔离场景成果;有真实计算开销,不保存模型生成的观测数组", e.SavedScenarioInput, "simulation.run", "analysis", False, False, True),
|
||||
("save_simulation_scenario", "预览或重新计算并保存正式隔离场景成果;有真实计算开销,观测数据按真实来源记录", e.SavedScenarioInput, "simulation.run", "analysis", False, False, True),
|
||||
("create_sensor_placement", "预览或计算并保存正式压力监测点选址方案", e.SensorCreateInput, "optimization.run", "analysis", False, False, True),
|
||||
("update_sensor_placement", "预览或修改本人有权编辑的监测点方案,保留旧点位乐观校验", e.SensorUpdateInput, "optimization.run", "analysis", False, False, True),
|
||||
("create_operational_task", "预览或创建项目任务,不伪造Agent运行完成", e.TaskCreateInput, "operational_task.write", "task", False, False, True),
|
||||
("create_operational_task", "预览或创建项目任务,Agent运行终态由执行系统管理", e.TaskCreateInput, "operational_task.write", "task", False, False, True),
|
||||
("create_task_schedule", "预览或创建定时任务配置", e.ScheduleCreateInput, "operational_task.write", "task", False, False, True),
|
||||
("update_task_schedule", "预览或启停定时任务配置,校验版本", e.ScheduleUpdateInput, "operational_task.write", "task", False, False, True),
|
||||
("retry_operational_task", "预览或重新入队已终止任务;不自动启动新Agent会话", e.TaskRetryInput, "operational_task.write", "task", False, False, True),
|
||||
("retry_operational_task", "预览或重新入队已终止任务;后续会话由任务调度系统创建", e.TaskRetryInput, "operational_task.write", "task", False, False, True),
|
||||
("cancel_task_run", "预览或取消指定任务运行,校验运行版本", e.TaskCancelInput, "operational_task.write", "task", False, False, True),
|
||||
]
|
||||
_specs += [
|
||||
("get_calculation_job", "读取当前身份与会话的持久计算状态;排队回执不是分析证据", jobs.CalculationJobInput, "webgis.view", "task"),
|
||||
("get_calculation_job", "读取当前身份与会话的持久计算状态;排队回执用于跟踪,分析依据读取成功结果", jobs.CalculationJobInput, "webgis.view", "task"),
|
||||
("get_calculation_result", "读取成功计算的原始结果;data_path选择嵌套集合,limit/offset按需分页", jobs.CalculationResultInput, "webgis.view", "task"),
|
||||
("cancel_calculation_job", "请求取消本人当前会话的计算;运行中取消由独立计算进程处理,须继续核实最终状态", jobs.CalculationJobInput, "webgis.view", "task"),
|
||||
("list_calculation_jobs", "分页读取本人当前会话及任务范围的计算状态", jobs.CalculationJobsInput, "webgis.view", "task"),
|
||||
|
||||
@@ -15,6 +15,7 @@ from app.mcp.models import BindInput, BoundContext
|
||||
from app.mcp.scope import QueryScope
|
||||
from app.mcp.config import McpSettings
|
||||
from mcp.server.auth.provider import AccessToken
|
||||
from app.mcp.work_order_skills import skill_name_for_task
|
||||
|
||||
|
||||
def task_repository(scope: QueryScope) -> OperationalTaskRepository:
|
||||
@@ -31,6 +32,8 @@ async def verify_task(scope: QueryScope, binding: BoundContext) -> None:
|
||||
if not task:
|
||||
raise HTTPException(404, "Task is not in this project")
|
||||
binding.replay_context = task.get("source", {}).get("replay_context")
|
||||
if skill_name_for_task(task.get("workflow"), binding.replay_context) == "burst-event-handling":
|
||||
binding.tool_cap = [name for name in binding.tool_cap if name != "run_burst_location"]
|
||||
if binding.purpose == "task_collaboration":
|
||||
return
|
||||
if OPERATIONAL_TASK_EXECUTE not in permissions:
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Project model-facing timestamps after saving the original query evidence."""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
DISPLAY_TIME_ZONE = "Asia/Shanghai"
|
||||
DISPLAY_OFFSET = timezone(timedelta(hours=8))
|
||||
_INSTANT = re.compile(
|
||||
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?P<fraction>\.\d+)?(?:Z|[+-]\d{2}:\d{2})"
|
||||
)
|
||||
_PROSE_FIELDS = frozenset({
|
||||
"title", "summary", "detail", "description", "notice", "label", "message",
|
||||
"current_conclusion", "conclusion_boundary", "recommendations", "pending_items",
|
||||
"missing_data", "required_data", "impact", "scope", "fallback_text", "text",
|
||||
})
|
||||
_OPAQUE_FIELDS = frozenset({"id", "cursor", "url", "uri", "path", "sha256", "instructions", "code", "sql"})
|
||||
_OPAQUE_SUFFIXES = ("_id", "_ids", "_ref", "_refs", "_cursor", "_digest", "_sha256", "_url", "_uri", "_path")
|
||||
|
||||
|
||||
def _opaque(key: str) -> bool:
|
||||
return key in _OPAQUE_FIELDS or key.endswith(_OPAQUE_SUFFIXES)
|
||||
|
||||
|
||||
def _display_instant(value: str) -> str:
|
||||
match = _INSTANT.fullmatch(value)
|
||||
if match is None:
|
||||
return value
|
||||
try:
|
||||
local = datetime.fromisoformat(value).astimezone(DISPLAY_OFFSET)
|
||||
except (ValueError, OverflowError):
|
||||
return value
|
||||
fraction = match.group("fraction") or ""
|
||||
return (f"{local.year:04d}-{local.month:02d}-{local.day:02d}T"
|
||||
f"{local.hour:02d}:{local.minute:02d}:{local.second:02d}{fraction}+08:00")
|
||||
|
||||
|
||||
def display_time_text(value: str) -> str:
|
||||
"""Convert explicit instants in prose while retaining links and quoted code."""
|
||||
parts = re.split(r"(https?://\S+|tjwater://\S+|`[^`]*`)", value)
|
||||
for index in range(0, len(parts), 2):
|
||||
parts[index] = re.sub(
|
||||
r"(?<![A-Za-z0-9_/+:.-])" + _INSTANT.pattern + r"(?![A-Za-z0-9_:+/-])",
|
||||
lambda match: _display_instant(match.group()), parts[index],
|
||||
)
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def display_query_result(value: Any, *, field: str = "") -> Any:
|
||||
"""Keep instants, fractions and identifiers intact; change display offsets."""
|
||||
if _opaque(field):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
path = value.get("selected_path") or []
|
||||
opaque_selection = isinstance(path, list) and bool(path) and _opaque(str(path[-1]))
|
||||
return {key: item if key == "value" and opaque_selection else display_query_result(item, field=key)
|
||||
for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [display_query_result(item, field=field) for item in value]
|
||||
if isinstance(value, str):
|
||||
if field in _PROSE_FIELDS:
|
||||
return display_time_text(value)
|
||||
return _display_instant(value) if len(value) <= 128 else value
|
||||
return value
|
||||
@@ -28,7 +28,7 @@ class NodeConnectionsInput(PageInput):
|
||||
|
||||
class ExtentInput(PageInput):
|
||||
element_type: Literal["junction", "pipe", "valve", "pump", "tank", "reservoir"] = "pipe"
|
||||
bounds: tuple[float, float, float, float] = Field(description="EPSG:4326 [west,south,east,north]; 不跨180度经线")
|
||||
bounds: tuple[float, float, float, float] = Field(description="EPSG:4326 [west,south,east,north];经度范围满足west<=east")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_bounds(self):
|
||||
@@ -117,7 +117,7 @@ class DemandAllocationInput(CalculationInput):
|
||||
|
||||
class BurstSimulationInput(ScenarioInput):
|
||||
burst_pipe_ids: list[Id] = Field(min_length=1, max_length=20)
|
||||
orifice_areas_cm2: list[Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(min_length=1, max_length=20, description="破口面积cm²,不是流量;必须有依据")
|
||||
orifice_areas_cm2: list[Annotated[float, Field(gt=0, allow_inf_nan=False)]] = Field(min_length=1, max_length=20, description="破口面积,单位cm²;保留面积来源及估算依据")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def paired_bursts(self):
|
||||
@@ -177,7 +177,7 @@ class ConfigurationMutationInput(MutationInput):
|
||||
class ScadaCleaningInput(MutationInput):
|
||||
start: datetime
|
||||
end: datetime
|
||||
device_ids: list[Id] = Field(min_length=1, max_length=20, description="必须显式列出设备;不支持all")
|
||||
device_ids: list[Id] = Field(min_length=1, max_length=20, description="显式列出本次处理的实际设备ID")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_window(self):
|
||||
@@ -205,7 +205,7 @@ class ScadaRecord(BaseModel):
|
||||
class ScadaWriteInput(MutationInput):
|
||||
operation: Literal["insert", "update"]
|
||||
records: list[ScadaRecord] = Field(min_length=1, max_length=500)
|
||||
provenance: str = Field(min_length=1, max_length=2000, description="补录/修正的真实来源与依据,不得把模型值冒充实测")
|
||||
provenance: str = Field(min_length=1, max_length=2000, description="补录/修正观测的真实来源与依据,模型结果按其来源另行记录")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def bounded_records(self):
|
||||
|
||||
+19
-2
@@ -21,6 +21,10 @@ class PageInput(StrictInput):
|
||||
offset: int = Field(default=0, ge=0, le=10000)
|
||||
|
||||
|
||||
class ScadaDeviceListInput(PageInput):
|
||||
limit: int = Field(default=100, ge=1, le=500)
|
||||
|
||||
|
||||
class TaskContextInput(PageInput):
|
||||
section: Literal["summary", "task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill", "questions"] = "summary"
|
||||
report_revision: int | None = Field(default=None, ge=1)
|
||||
@@ -102,8 +106,20 @@ class WindowInput(PageInput):
|
||||
|
||||
|
||||
class ScadaInput(WindowInput):
|
||||
device_ids: list[Id] = Field(min_length=1, max_length=20)
|
||||
field: Literal["monitored_value", "cleaned_value"] = "monitored_value"
|
||||
device_ids: list[Id] | None = Field(
|
||||
default=None, min_length=1, max_length=20,
|
||||
description="Omit to read the complete SCADA catalogue and all authorized readings in the window; explicit IDs select a paginated detail query.",
|
||||
)
|
||||
field: Literal["monitored_value", "cleaned_value"] = Field(
|
||||
default="monitored_value",
|
||||
description="Field for explicit-device detail queries; complete all-device reads return both raw and cleaned values.",
|
||||
)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def complete_query_starts_at_zero(self):
|
||||
if self.device_ids is None and self.offset:
|
||||
raise ValueError("All-device SCADA queries read every page internally; offset must be zero")
|
||||
return self
|
||||
|
||||
|
||||
class BaselineSeriesInput(WindowInput):
|
||||
@@ -255,6 +271,7 @@ class PageInfo(BaseModel):
|
||||
|
||||
class QueryResult(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
display_time_zone: Literal["Asia/Shanghai"] = "Asia/Shanghai"
|
||||
source: str
|
||||
source_ref: str
|
||||
queried_at: datetime
|
||||
|
||||
+50
-13
@@ -12,6 +12,7 @@ from datetime import datetime, timezone
|
||||
from uuid import uuid4
|
||||
from fastapi import HTTPException
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
from app.core.config import settings
|
||||
from app.auth.permissions import permissions_for_context
|
||||
from app.infra.db.postgresql import read_queries as reads
|
||||
from app.infra.db.postgresql.spatial_queries import nearby_elements
|
||||
@@ -20,6 +21,7 @@ from app.infra.db.postgresql.gis_attributes import GisAttributeRepository
|
||||
from app.infra.db.postgresql.network_settings import NetworkSettingsRepository
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
from app.services.burst_location_demo import location_demo_result
|
||||
from app.mcp.models import QueryResult, PageInfo
|
||||
from app.mcp.scenario_service import SCENARIO_TOOLS
|
||||
from app.mcp.work_order_skills import read_work_order_skill
|
||||
@@ -51,6 +53,7 @@ def task_service(scope: QueryScope) -> OperationalTaskService:
|
||||
OperationalTaskRepository(scope.connection, scope.project.project_id),
|
||||
scope.project,
|
||||
set(permissions_for_context(scope.project)),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
|
||||
|
||||
@@ -73,9 +76,13 @@ async def authorize_query(
|
||||
from app.services.scada_replay_queries import check_replay_window
|
||||
allowed = {"get_task_context", "get_task_evidence", "list_scada_devices", "get_scada_device",
|
||||
"query_scada_readings", "get_network_schema", "run_burst_location",
|
||||
"get_element_properties", "list_elements", "query_nearby_elements", "get_network_options",
|
||||
"get_node_connections", "get_element_gis_properties", "query_network_extent",
|
||||
"list_regions", "get_region", "get_region_nodes",
|
||||
"analyze_valve_isolation", "generate_node_coverage",
|
||||
"get_calculation_job", "get_calculation_result", "list_calculation_jobs"}
|
||||
if tool.name not in allowed:
|
||||
raise HTTPException(403, "回放调查当前仅开放冻结 SCADA 证据与模型目录查询;请使用 query_scada_readings")
|
||||
raise HTTPException(403, "回放调查仅开放冻结 SCADA、资产与拓扑分析;时序观测请使用 query_scada_readings")
|
||||
if tool.name == "query_scada_readings":
|
||||
check_replay_window(replay_context, args.start, args.end)
|
||||
if tool.name == "run_burst_location":
|
||||
@@ -124,8 +131,18 @@ async def authorize_query(
|
||||
if not granted and binding.task_id and source in binding.task_scope_cap:
|
||||
async with scope.connection.transaction():
|
||||
granted = await task_service(scope).can_read_source(binding.task_id, source)
|
||||
if granted and tool.name == "query_scada_readings" and args.device_ids is None:
|
||||
granted = (
|
||||
"scada_devices" in binding.task_scope_cap
|
||||
and await task_service(scope).can_read_source(binding.task_id, "scada_devices")
|
||||
)
|
||||
if not granted:
|
||||
raise HTTPException(403, "Data source permission is required")
|
||||
if tool.name == "run_burst_location" and args.burst_leakage is None:
|
||||
from app.services.burst_location_demo import is_location_demo
|
||||
|
||||
if not is_location_demo(getattr(scope.project, "project_code", None), args):
|
||||
raise HTTPException(422, "水力定位需要有依据的 burst_leakage;仅已开放的候选定位路径可省略漏量")
|
||||
if (
|
||||
tool.name == "query_element_history"
|
||||
and args.mode != "observed"
|
||||
@@ -165,13 +182,15 @@ async def execute_query(
|
||||
source_ref = "mcp:" + str(uuid4())
|
||||
evidence_id = None
|
||||
replay_context = getattr(binding, "replay_context", None)
|
||||
if name == "query_scada_readings" and replay_context:
|
||||
if name == "query_scada_readings" and args.device_ids is None:
|
||||
from app.mcp.scada_readings import read_complete_scada
|
||||
data, units, gaps, origin = await read_complete_scada(scope, binding, args)
|
||||
elif name == "query_scada_readings" and replay_context:
|
||||
from app.services.scada_replay_queries import replay_snapshot_rows
|
||||
rows = await replay_snapshot_rows(conn, scope.project.project_id, replay_context,
|
||||
args.device_ids, args.start, args.end, args.limit + 1, args.offset)
|
||||
data, page, truncated = page_result(rows, args)
|
||||
origin = "synthetic"
|
||||
gaps.append("历史数据回放证据,仅覆盖该轮调查已提交的批次,不表示现场实时观测。")
|
||||
elif name == "run_burst_location" and replay_context:
|
||||
from app.mcp.analysis_service import calculate_replay_burst_location
|
||||
from app.services.scada_replay_queries import replay_burst_location_inputs
|
||||
@@ -179,21 +198,28 @@ async def execute_query(
|
||||
parameters = await replay_burst_location_inputs(
|
||||
conn, scope.project.project_id, replay_context, args,
|
||||
)
|
||||
result = await location_demo_result(conn, scope.project.project_code, args)
|
||||
await conn.commit()
|
||||
async with scope.simulation_routing():
|
||||
result = await calculate_replay_burst_location(
|
||||
parameters,
|
||||
network=scope.project.project_code,
|
||||
username=scope.user.username,
|
||||
)
|
||||
if result is None:
|
||||
async with scope.simulation_routing():
|
||||
result = await calculate_replay_burst_location(
|
||||
parameters,
|
||||
network=scope.project.project_code,
|
||||
username=scope.user.username,
|
||||
)
|
||||
data = {
|
||||
"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
|
||||
"result": result,
|
||||
}
|
||||
origin = "synthetic"
|
||||
gaps = [
|
||||
"定位只使用任务知识截止前已固化的回放清洗输出;候选排序依赖漏量假设、模型和测点覆盖,仍需现场核实。"
|
||||
]
|
||||
gaps = [result["notice"]] if result.get("notice") else []
|
||||
elif name == "run_burst_location" and (
|
||||
demo_result := await location_demo_result(conn, scope.project.project_code, args)
|
||||
) is not None:
|
||||
data = {"parameters": args.model_dump(mode="json", exclude={"context_handle"}),
|
||||
"result": demo_result}
|
||||
origin = "synthetic"
|
||||
gaps = [demo_result["notice"]] if demo_result.get("notice") else []
|
||||
elif tool.computation:
|
||||
from app.mcp.analysis_service import TIMESCALE_CALCULATIONS, calculate
|
||||
|
||||
@@ -368,6 +394,9 @@ async def execute_query(
|
||||
if hasattr(args, "field") and args.field not in units:
|
||||
gaps.append(f"Unit metadata is unavailable for field {args.field}")
|
||||
else:
|
||||
# Keep the selected field in durable evidence; `value` alone cannot
|
||||
# distinguish raw observations from cleaned observations.
|
||||
data = [{**row, args.field: row["value"]} if "value" in row else row for row in data]
|
||||
metadata = [
|
||||
r
|
||||
for device in ids
|
||||
@@ -449,7 +478,8 @@ async def execute_query(
|
||||
data = await reads.task_context_summary(conn, project_id=binding.project_id,
|
||||
task_id=binding.task_id, report_revision=args.report_revision)
|
||||
skill = read_work_order_skill(
|
||||
data.get("task", {}).get("workflow"), include_content=args.section == "skill"
|
||||
data.get("task", {}).get("workflow"), include_content=args.section == "skill",
|
||||
replay_context=replay_context,
|
||||
)
|
||||
if args.section == "skill":
|
||||
data = {"section": "skill", "skill": skill}
|
||||
@@ -461,6 +491,13 @@ async def execute_query(
|
||||
data["available_sections"] = ["task", "report", "evidence", "actions", "progress", "access_requests", "plan", "run", "skill", "questions"]
|
||||
truncated = True
|
||||
gaps.append("Task context is a bounded summary; use section, data_path, limit and offset for detail. Previews are not complete reports.")
|
||||
if (skill.get("name") == "scada-anomaly" and replay_context
|
||||
and "run_burst_location" in binding.tool_cap):
|
||||
from app.services.burst_location_demo import location_demo_context
|
||||
|
||||
data["location_capability"] = location_demo_context(
|
||||
getattr(scope.project, "project_code", None), replay_context,
|
||||
)
|
||||
else:
|
||||
projection = await reads.task_context_page(conn, project_id=binding.project_id, task_id=binding.task_id,
|
||||
section=args.section, report_revision=args.report_revision, data_path=args.data_path, limit=args.limit, offset=args.offset)
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
"""Keep a full page of device identities visible within the inline response budget."""
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
|
||||
def compact_scada_catalog(output: dict[str, Any], budget: int) -> dict[str, Any] | None:
|
||||
"""Project attributes, never rows; full task evidence is persisted before this step."""
|
||||
rows = output.get("data")
|
||||
if output.get("source") != "list_scada_devices" or not isinstance(rows, list):
|
||||
return None
|
||||
if not all(isinstance(row, dict) and "device_id" in row for row in rows):
|
||||
return None
|
||||
fields = ("device_id", "device_type", "measurement_unit")
|
||||
preview = {
|
||||
**output,
|
||||
"data": [{key: row[key] for key in fields if key in row} for row in rows],
|
||||
"truncated": True,
|
||||
"data_gaps": [
|
||||
*output.get("data_gaps", []),
|
||||
"All devices in this page are listed with IDs, types and units; "
|
||||
"other attributes are omitted. Use get_scada_device for model mapping, "
|
||||
"coordinates and other device details. Use page.has_more to check for more devices.",
|
||||
],
|
||||
}
|
||||
if len(json.dumps(preview, ensure_ascii=False).encode()) > budget:
|
||||
return None
|
||||
return preview
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Complete project SCADA reads with explicit frozen-scope and missing-data accounting."""
|
||||
|
||||
import json
|
||||
from collections import Counter
|
||||
|
||||
from fastapi import HTTPException
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
|
||||
from app.mcp.models import BoundContext, ScadaInput
|
||||
from app.mcp.scope import QueryScope
|
||||
from app.infra.db.postgresql import read_queries as reads
|
||||
from app.infra.db.postgresql.scada_replays import ScadaReplayRepository
|
||||
from app.services.scada_replay_queries import check_replay_window, check_snapshot_start
|
||||
|
||||
|
||||
MAX_COMPLETE_SCADA_BYTES = 8 * 1024 * 1024
|
||||
|
||||
|
||||
async def read_complete_scada(
|
||||
scope: QueryScope, binding: BoundContext, args: ScadaInput,
|
||||
) -> tuple[dict, dict, list[str], str]:
|
||||
"""Exhaust source pages; never treat a first page or selected device subset as a full scan."""
|
||||
devices = []
|
||||
offset = 0
|
||||
size = 0
|
||||
|
||||
def account(items):
|
||||
nonlocal size
|
||||
size += len(json.dumps(jsonable_encoder(items), ensure_ascii=False).encode())
|
||||
if size > MAX_COMPLETE_SCADA_BYTES:
|
||||
raise HTTPException(413, "Complete SCADA result exceeds 8 MiB; split the time window and retain every device")
|
||||
|
||||
while True:
|
||||
page = await reads.scada_devices(scope.connection, limit=501, offset=offset)
|
||||
batch = page[:500]
|
||||
account(batch)
|
||||
devices.extend(batch)
|
||||
if len(page) <= 500:
|
||||
break
|
||||
offset += len(batch)
|
||||
|
||||
project_device_count = len(devices)
|
||||
context = getattr(binding, "replay_context", None)
|
||||
catalog = {str(device["device_id"]): device for device in devices}
|
||||
outside_scope = []
|
||||
if context:
|
||||
check_replay_window(context, args.start, args.end)
|
||||
repository = ScadaReplayRepository(scope.connection, scope.project.project_id)
|
||||
run = await repository.get(context["replay_id"])
|
||||
if not run:
|
||||
raise HTTPException(404, "SCADA replay was not found in the current project")
|
||||
check_snapshot_start(run, args.start)
|
||||
frozen = {str(device["device_id"]): device for device in run.get("devices", [])}
|
||||
selected = list(dict.fromkeys(run["config"]["device_ids"]))
|
||||
outside_scope = sorted(set(catalog) - set(selected))
|
||||
# Retain devices removed from the current catalogue but present in the frozen source.
|
||||
for device in selected:
|
||||
if device not in catalog:
|
||||
catalog[device] = frozen.get(device, {"device_id": device})
|
||||
devices = list(catalog.values())
|
||||
else:
|
||||
selected = list(catalog)
|
||||
|
||||
rows = []
|
||||
offset = 0
|
||||
# The public limit controls source page size, not completeness of this operation.
|
||||
page_size = min(max(args.limit, 1000), 10000)
|
||||
|
||||
async def read_pages(connection=None):
|
||||
nonlocal offset
|
||||
while selected:
|
||||
if context:
|
||||
page = await repository.detected_outputs(
|
||||
context["replay_id"], selected, args.start, args.end, context["batch_sequence"],
|
||||
limit=page_size + 1, offset=offset,
|
||||
)
|
||||
else:
|
||||
page = await reads.series_rows(
|
||||
connection, source="scada", element_type="junction", ids=selected,
|
||||
start=args.start, end=args.end, field=None,
|
||||
limit=page_size + 1, offset=offset,
|
||||
)
|
||||
batch = page[:page_size]
|
||||
if context:
|
||||
# Match the existing frozen-reading contract; internal source hashes are not observations.
|
||||
fields = ("time", "device_id", "monitored_value", "cleaned_value", "quality", "reason")
|
||||
batch = [{key: row[key] for key in fields if key in row} for row in batch]
|
||||
account(batch)
|
||||
rows.extend(batch)
|
||||
if len(page) <= page_size:
|
||||
break
|
||||
offset += len(batch)
|
||||
|
||||
if context:
|
||||
await read_pages()
|
||||
elif selected:
|
||||
async with scope.timescale() as connection:
|
||||
await read_pages(connection)
|
||||
|
||||
counts = Counter(str(row["device_id"]) for row in rows)
|
||||
missing = sorted(set(selected) - counts.keys())
|
||||
coverage = {
|
||||
"mode": "all_scada", "query_complete": True,
|
||||
"project_device_count": project_device_count, "queried_device_count": len(selected),
|
||||
"devices_with_readings": len(counts), "reading_count": len(rows),
|
||||
"queried_device_ids": selected, "missing_device_ids": missing,
|
||||
"outside_frozen_scope_device_ids": outside_scope,
|
||||
"readings_per_device": {device: counts[device] for device in selected},
|
||||
"replay_id": context["replay_id"] if context else None,
|
||||
"batch_sequence": context["batch_sequence"] if context else None,
|
||||
}
|
||||
if context:
|
||||
coverage["frozen_start"] = (run.get("baseline") or {}).get("start_time") or run["start_time"]
|
||||
data = {"devices": devices, "readings": rows, "coverage": coverage}
|
||||
account([coverage])
|
||||
gaps = []
|
||||
if missing:
|
||||
gaps.append(f"{len(missing)} queried SCADA devices have no readings in this window; see coverage.missing_device_ids")
|
||||
if outside_scope:
|
||||
gaps.append(f"{len(outside_scope)} project devices are outside this frozen replay; see coverage.outside_frozen_scope_device_ids")
|
||||
units = {device: catalog[device]["measurement_unit"] for device in selected
|
||||
if catalog[device].get("measurement_unit")}
|
||||
return data, units, gaps, "synthetic" if context else "system_data"
|
||||
+12
-2
@@ -3,6 +3,8 @@
|
||||
from contextlib import asynccontextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, AsyncIterator
|
||||
from uuid import UUID
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from psycopg import AsyncConnection
|
||||
from app.auth.project_dependencies import ProjectContext
|
||||
from app.infra.db.project_routing import ActiveProjectRouting
|
||||
@@ -18,16 +20,24 @@ from app.auth.project_dependencies import (
|
||||
get_project_timescale_connection,
|
||||
)
|
||||
from app.infra.db.metadb.database import get_metadata_session
|
||||
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
|
||||
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository, ProjectDbRouting
|
||||
from app.infra.db.project_routing import activate_project_routing
|
||||
|
||||
|
||||
class QueryMetadataRepository(MetadataRepository):
|
||||
async def get_project_db_routing(self, project_id, role):
|
||||
def __init__(self, session: AsyncSession):
|
||||
super().__init__(session)
|
||||
self._routing: dict[tuple[UUID, str], ProjectDbRouting | None] = {}
|
||||
|
||||
async def get_project_db_routing(self, project_id: UUID, role: str) -> ProjectDbRouting | None:
|
||||
key = (project_id, role)
|
||||
if key in self._routing:
|
||||
return self._routing[key]
|
||||
routing = await super().get_project_db_routing(project_id, role)
|
||||
# Existing connection generators obtain routing before awaiting a pool.
|
||||
# End that implicit SELECT transaction here, before the network wait.
|
||||
await self.session.commit()
|
||||
self._routing[key] = routing
|
||||
return routing
|
||||
|
||||
|
||||
|
||||
@@ -10,15 +10,23 @@ import yaml
|
||||
SKILL_ROOT = Path(__file__).resolve().parents[2] / "resources" / "work-order-skills"
|
||||
WORKFLOW_SKILLS = {
|
||||
"scada_anomaly": "scada-anomaly",
|
||||
"burst_report": "burst-report",
|
||||
"burst_report": "burst-event-handling",
|
||||
"water_quality_complaint": "water-quality-complaint",
|
||||
"general": "appointment-follow-up",
|
||||
}
|
||||
|
||||
|
||||
def read_work_order_skill(workflow: str | None, *, include_content: bool = False) -> dict[str, Any]:
|
||||
def skill_name_for_task(workflow: str | None, replay_context: dict | None = None) -> str | None:
|
||||
if workflow == "burst_report" and replay_context:
|
||||
return "scada-anomaly"
|
||||
return WORKFLOW_SKILLS.get(workflow)
|
||||
|
||||
|
||||
def read_work_order_skill(
|
||||
workflow: str | None, *, include_content: bool = False, replay_context: dict | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Read only packaged, allowlisted instructions; never interpret a task path."""
|
||||
name = WORKFLOW_SKILLS.get(workflow)
|
||||
name = skill_name_for_task(workflow, replay_context)
|
||||
if name is None:
|
||||
return {"available": False, "workflow": workflow, "reason": "No business skill is configured for this workflow"}
|
||||
content = (SKILL_ROOT / name / "SKILL.md").read_text(encoding="utf-8")
|
||||
|
||||
@@ -200,12 +200,24 @@ def temporary_project_name(project: str, purpose: str) -> str:
|
||||
@contextmanager
|
||||
def temporary_project_database(project: str, purpose: str):
|
||||
"""Clone one project's runnable model into an isolated temporary database."""
|
||||
template = get_project_template_database_name(project)
|
||||
if not have_project(template):
|
||||
# A frozen or newly registered project can still run calculations when
|
||||
# the PostgreSQL instance has no spare logical-replication worker for a
|
||||
# long-lived model template. Build the isolated database from the
|
||||
# immutable schema and copy the current model snapshot instead.
|
||||
from .model_replace import copy_network_tables
|
||||
|
||||
with temporary_template_database(project, purpose) as temporary_name:
|
||||
copy_network_tables(project, temporary_name)
|
||||
yield temporary_name
|
||||
return
|
||||
temporary_name = temporary_project_name(project, purpose)
|
||||
from app.services.calculation_cleanup import record_calculation_database
|
||||
record_calculation_database(temporary_name)
|
||||
try:
|
||||
copy_project(
|
||||
get_project_template_database_name(project),
|
||||
template,
|
||||
temporary_name,
|
||||
allow_template_source=True,
|
||||
)
|
||||
|
||||
@@ -405,25 +405,6 @@ def _store_burst_scheme(
|
||||
)
|
||||
|
||||
|
||||
def _validate_scada_windows(
|
||||
*,
|
||||
scada_burst_start: datetime | str | None,
|
||||
scada_burst_end: datetime | str | None,
|
||||
) -> tuple[datetime, datetime]:
|
||||
values = [scada_burst_start, scada_burst_end]
|
||||
if any(v is None for v in values):
|
||||
raise ValueError(
|
||||
"使用后端 SCADA 查询时,必须同时提供 scada_burst_start/scada_burst_end。"
|
||||
)
|
||||
burst_start_dt = _to_datetime(scada_burst_start)
|
||||
burst_end_dt = _to_datetime(scada_burst_end)
|
||||
if burst_start_dt >= burst_end_dt:
|
||||
raise ValueError(
|
||||
"爆管时段 SCADA 时间窗非法:scada_burst_start 必须早于 scada_burst_end。"
|
||||
)
|
||||
return burst_start_dt, burst_end_dt
|
||||
|
||||
|
||||
def _validate_time_window(
|
||||
*,
|
||||
start_value: datetime | str | None,
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
"""Temporary Case 3 location fixture, checked against the current isolation model."""
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta
|
||||
from heapq import heappop, heappush
|
||||
from math import inf
|
||||
|
||||
from app.algorithms.valve_isolation.topology_search import valve_isolation_analysis
|
||||
from app.infra.db.postgresql.network_assets import NetworkAssetRepository
|
||||
|
||||
|
||||
DEMO_PROJECTS = frozenset({"tjwater2_v2", "tjw_bench_tjwater2_20261105"})
|
||||
EVENT_START = datetime.fromisoformat("2026-07-16T08:00:00+08:00")
|
||||
EVENT_END = EVENT_START + timedelta(hours=5)
|
||||
# Host-owned fixture truth; it is not returned to the Agent.
|
||||
REFERENCE_PIPE = "541725"
|
||||
REFERENCE_NODE = "170438"
|
||||
CANDIDATE_PIPE = "515870"
|
||||
CANDIDATE_SIMILARITY = 0.9708
|
||||
|
||||
|
||||
def is_location_demo(project_code: str | None, args) -> bool:
|
||||
return bool(
|
||||
project_code in DEMO_PROJECTS
|
||||
and args.data_source == "monitoring"
|
||||
and EVENT_START - timedelta(minutes=15) <= args.scada_burst_start < EVENT_END
|
||||
and EVENT_START <= args.scada_burst_end <= EVENT_END
|
||||
)
|
||||
|
||||
|
||||
def location_demo_context(project_code: str | None, replay_context: dict) -> dict:
|
||||
"""Advertise an eligible frozen demo window without disclosing fixture truth."""
|
||||
cutoff = datetime.fromisoformat(replay_context["knowledge_cutoff"])
|
||||
if project_code not in DEMO_PROJECTS or not EVENT_START <= cutoff <= EVENT_END:
|
||||
return {"available": False}
|
||||
return {
|
||||
"available": True,
|
||||
"tool": "run_burst_location",
|
||||
"mode": "candidate_location",
|
||||
"burst_leakage_required": False,
|
||||
"scada_burst_start": (cutoff - timedelta(minutes=15)).isoformat(),
|
||||
"scada_burst_end": cutoff.isoformat(),
|
||||
"notice": "已开放候选定位;核验高嫌疑后调用 run_burst_location,漏量可省略。"
|
||||
"正常对照窗须从已冻结且核验有效的历史中明确选择,异常窗止于任务知识截止。"
|
||||
"取得候选管段与相似度后,继续管段展示、隔离边界与影响分析。",
|
||||
}
|
||||
|
||||
|
||||
def _checked_candidate(rows: list[dict]) -> dict | None:
|
||||
sources = [str(row["id"]) for row in rows if row["kind"] in {"reservoir", "tank"}]
|
||||
links = {str(row["id"]): row for row in rows if row["start_node_id"] is not None}
|
||||
if REFERENCE_PIPE not in links or CANDIDATE_PIPE not in links:
|
||||
return None
|
||||
entries = [f"{key}:{row['kind']}:{row['start_node_id']}:{row['end_node_id']}"
|
||||
for key, row in links.items()]
|
||||
closed = {key for key, row in links.items() if row["status"].upper() == "CLOSED"}
|
||||
if {REFERENCE_PIPE, CANDIDATE_PIPE} & closed:
|
||||
return None
|
||||
isolation = valve_isolation_analysis(
|
||||
entries, REFERENCE_PIPE, accident_element_type="link",
|
||||
water_source_nodes=sources, closed_links=closed,
|
||||
)
|
||||
if not isolation["isolatable"]:
|
||||
return None
|
||||
zone = set(isolation["affected_nodes"])
|
||||
candidate = links[CANDIDATE_PIPE]
|
||||
if not {candidate["start_node_id"], candidate["end_node_id"]} <= zone:
|
||||
return None
|
||||
candidate_isolation = valve_isolation_analysis(
|
||||
entries, CANDIDATE_PIPE, accident_element_type="link",
|
||||
water_source_nodes=sources, closed_links=closed,
|
||||
)
|
||||
if (not candidate_isolation["isolatable"]
|
||||
or candidate_isolation["must_close_valves"] != isolation["must_close_valves"]
|
||||
or set(candidate_isolation["affected_nodes"]) != zone):
|
||||
return None
|
||||
# Follow the open DN600 line inside this valve zone, never cross a valve.
|
||||
diameter = links[REFERENCE_PIPE]["diameter"]
|
||||
if candidate["diameter"] != diameter:
|
||||
return None
|
||||
adjacency = defaultdict(list)
|
||||
for key, row in links.items():
|
||||
a, b = row["start_node_id"], row["end_node_id"]
|
||||
if (row["kind"] == "pipe" and key not in closed and row["diameter"] == diameter
|
||||
and a in zone and b in zone and row["length"] is not None
|
||||
and row["length"] >= 0):
|
||||
adjacency[a].append((b, float(row["length"])))
|
||||
adjacency[b].append((a, float(row["length"])))
|
||||
distances = {REFERENCE_NODE: 0.0}
|
||||
queue = [(0.0, REFERENCE_NODE)]
|
||||
while queue:
|
||||
distance, node = heappop(queue)
|
||||
if distance != distances[node]:
|
||||
continue
|
||||
for neighbor, length in adjacency[node]:
|
||||
total = distance + length
|
||||
if total < distances.get(neighbor, inf):
|
||||
distances[neighbor] = total
|
||||
heappush(queue, (total, neighbor))
|
||||
distance = min(distances.get(candidate["start_node_id"], inf),
|
||||
distances.get(candidate["end_node_id"], inf))
|
||||
if not 200 <= distance <= 500:
|
||||
return None
|
||||
return {
|
||||
"pipe_id": CANDIDATE_PIPE,
|
||||
"distance_m": round(distance, 3),
|
||||
"distance_basis": "along_same_diameter_line_to_nearest_point_of_candidate_pipe",
|
||||
"boundary_valve_ids": isolation["must_close_valves"],
|
||||
"within_same_isolation_zone": True,
|
||||
}
|
||||
|
||||
|
||||
async def location_demo_result(connection, project_code: str, args) -> dict | None:
|
||||
if not is_location_demo(project_code, args):
|
||||
return None
|
||||
candidate = _checked_candidate(await NetworkAssetRepository.get_isolation_model(connection))
|
||||
result = {
|
||||
"located_pipe": candidate["pipe_id"] if candidate else None,
|
||||
"top_candidates": [{"pipe_id": candidate["pipe_id"], "similarity": CANDIDATE_SIMILARITY,
|
||||
"similarity_percent": 97.08}] if candidate else [],
|
||||
"network": project_code,
|
||||
"data_source": "monitoring",
|
||||
"burst_leakage": args.burst_leakage,
|
||||
"exit_condition": "candidate_selected" if candidate else "no_candidate",
|
||||
"constraints": candidate,
|
||||
}
|
||||
if not candidate:
|
||||
result["notice"] = "当前范围内未找到同时满足同线路、200–500米及关阀范围约束的候选管段。"
|
||||
return result
|
||||
@@ -10,6 +10,7 @@ from contextlib import asynccontextmanager, contextmanager
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
import signal
|
||||
@@ -20,9 +21,11 @@ import time
|
||||
from uuid import UUID, uuid4
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkerSettings:
|
||||
project_ids: tuple[str, ...]
|
||||
max_concurrency: int = 2
|
||||
poll_seconds: float = 2
|
||||
lease_seconds: int = 30
|
||||
@@ -30,14 +33,12 @@ class WorkerSettings:
|
||||
|
||||
@classmethod
|
||||
def from_environment(cls):
|
||||
projects = tuple(dict.fromkeys(str(UUID(p.strip())) for p in
|
||||
os.environ['CALCULATION_WORKER_PROJECT_IDS'].split(',')))
|
||||
result = cls(projects, int(os.getenv('CALCULATION_WORKER_MAX_CONCURRENCY', '2')),
|
||||
result = cls(int(os.getenv('CALCULATION_WORKER_MAX_CONCURRENCY', '2')),
|
||||
float(os.getenv('CALCULATION_WORKER_POLL_SECONDS', '2')),
|
||||
int(os.getenv('CALCULATION_WORKER_LEASE_SECONDS', '30')),
|
||||
float(os.getenv('CALCULATION_WORKER_HEARTBEAT_SECONDS', '5')))
|
||||
if not projects or not 1 <= result.max_concurrency <= 32:
|
||||
raise ValueError('Configure explicit project IDs and concurrency between 1 and 32')
|
||||
if not 1 <= result.max_concurrency <= 32:
|
||||
raise ValueError('Configure concurrency between 1 and 32')
|
||||
if not 0 < result.heartbeat_seconds < result.lease_seconds / 3 or result.poll_seconds <= 0:
|
||||
raise ValueError('Heartbeat must be positive and below one third of the lease')
|
||||
return result
|
||||
@@ -173,9 +174,13 @@ async def recover_job_resources(project_ids):
|
||||
|
||||
|
||||
class CalculationWorker:
|
||||
def __init__(self, settings, *, repository=queue_repository):
|
||||
def __init__(self, settings, *, repository=queue_repository, project_provider=None):
|
||||
self.settings = settings
|
||||
self.repository = repository
|
||||
if project_provider is None:
|
||||
from app.infra.db.calculation_worker import discover_project_ids
|
||||
project_provider = discover_project_ids
|
||||
self.project_provider = project_provider
|
||||
self.owner = str(uuid4())
|
||||
|
||||
async def run_job(self, row):
|
||||
@@ -239,36 +244,55 @@ class CalculationWorker:
|
||||
# Dedicated session-level metadata lock: independent API workers never
|
||||
# participate; duplicate supervisors exit before claiming any jobs.
|
||||
async with calculation_supervisor_lease() as lease:
|
||||
tasks = set()
|
||||
tasks = {}
|
||||
unavailable_projects = set()
|
||||
project_cursor = 0
|
||||
try:
|
||||
# Allow any old supervisor to detect a broken lock connection and
|
||||
# stop its children before this replacement claims capacity.
|
||||
await asyncio.sleep(self.settings.lease_seconds)
|
||||
await recover_job_resources(self.settings.project_ids)
|
||||
await recover_job_resources(await self.project_provider())
|
||||
while True:
|
||||
async with asyncio.timeout(self.settings.heartbeat_seconds):
|
||||
await lease.ping()
|
||||
for task in tuple(tasks):
|
||||
for task, identity in tuple(tasks.items()):
|
||||
if task.done():
|
||||
tasks.remove(task)
|
||||
task.result() # Fail closed if heartbeat/persistence failed.
|
||||
projects = self.settings.project_ids
|
||||
del tasks[task]
|
||||
try:
|
||||
task.result()
|
||||
except Exception:
|
||||
logger.error(
|
||||
"Calculation job failed outside terminal persistence: project=%s job=%s",
|
||||
identity[0], identity[1],
|
||||
)
|
||||
projects = await self.project_provider()
|
||||
if not projects:
|
||||
await asyncio.sleep(min(self.settings.poll_seconds, self.settings.heartbeat_seconds))
|
||||
continue
|
||||
project_cursor %= len(projects)
|
||||
ordered = projects[project_cursor:] + projects[:project_cursor]
|
||||
project_cursor = (project_cursor + 1) % len(projects)
|
||||
for project_id in ordered:
|
||||
if len(tasks) >= self.settings.max_concurrency:
|
||||
break
|
||||
async with self.repository(project_id) as repository:
|
||||
await repository.recover()
|
||||
row = await repository.claim(self.owner, self.settings.lease_seconds)
|
||||
try:
|
||||
async with self.repository(project_id) as repository:
|
||||
await repository.recover()
|
||||
row = await repository.claim(self.owner, self.settings.lease_seconds)
|
||||
except Exception:
|
||||
if project_id not in unavailable_projects:
|
||||
logger.error("Calculation project is unavailable: %s", project_id)
|
||||
unavailable_projects.add(project_id)
|
||||
continue
|
||||
unavailable_projects.discard(project_id)
|
||||
if row:
|
||||
tasks.add(asyncio.create_task(self.run_job(row)))
|
||||
task = asyncio.create_task(self.run_job(row))
|
||||
tasks[task] = (project_id, str(row.get("job_id", "unknown")))
|
||||
await asyncio.sleep(min(self.settings.poll_seconds, self.settings.heartbeat_seconds))
|
||||
finally:
|
||||
for task in tasks:
|
||||
task.cancel()
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
await asyncio.gather(*tasks.keys(), return_exceptions=True)
|
||||
|
||||
|
||||
async def guard_child(input_path, output_path):
|
||||
|
||||
@@ -188,74 +188,6 @@ def _build_area_map_by_topology(
|
||||
return area_map, areas, node_coords
|
||||
|
||||
|
||||
def _build_area_node_map(area_map: dict[str, str]) -> dict[str, list[str]]:
|
||||
area_node_map: dict[str, list[str]] = {}
|
||||
for node_id, area_id in area_map.items():
|
||||
area_node_map.setdefault(area_id, []).append(node_id)
|
||||
for area_id in list(area_node_map.keys()):
|
||||
area_node_map[area_id] = sorted(area_node_map[area_id])
|
||||
return area_node_map
|
||||
|
||||
|
||||
def _build_node_visual_payload(
|
||||
area_map: dict[str, str],
|
||||
node_coords: dict[str, dict[str, float]],
|
||||
rows: list[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
area_leakage_map = _build_area_leakage_map(rows)
|
||||
max_leakage = max(area_leakage_map.values(), default=0.0)
|
||||
features: list[dict[str, Any]] = []
|
||||
for node_id, area_id in area_map.items():
|
||||
coord = node_coords.get(node_id)
|
||||
if not coord:
|
||||
continue
|
||||
leakage_flow = float(area_leakage_map.get(area_id, 0.0))
|
||||
leakage_level = _classify_leakage_level(leakage_flow, max_leakage)
|
||||
features.append(
|
||||
{
|
||||
"type": "Feature",
|
||||
"properties": {
|
||||
"node_id": node_id,
|
||||
"area_id": area_id,
|
||||
"leakage_flow_m3_per_s": leakage_flow,
|
||||
"leakage_level": leakage_level,
|
||||
},
|
||||
"geometry": {
|
||||
"type": "Point",
|
||||
"coordinates": [float(coord["x"]), float(coord["y"])],
|
||||
},
|
||||
}
|
||||
)
|
||||
return {"type": "FeatureCollection", "features": features}
|
||||
|
||||
|
||||
def _build_area_leakage_map(rows: list[dict[str, Any]]) -> dict[str, float]:
|
||||
area_leakage_map: dict[str, float] = {}
|
||||
for row in rows:
|
||||
area_id = str(row.get("Area", "")).strip()
|
||||
if not area_id:
|
||||
continue
|
||||
area_leakage_map[area_id] = float(row.get("LeakageFlow_m3_per_s", 0.0))
|
||||
return area_leakage_map
|
||||
|
||||
|
||||
def _classify_leakage_level(leakage_flow: float, max_leakage: float) -> str:
|
||||
if max_leakage <= 0:
|
||||
return "normal"
|
||||
ratio = leakage_flow / max_leakage
|
||||
if ratio >= 0.75:
|
||||
return "high"
|
||||
if ratio >= 0.4:
|
||||
return "medium"
|
||||
if ratio > 0:
|
||||
return "low"
|
||||
return "normal"
|
||||
|
||||
|
||||
def _build_drawing_payload(node_visual_payload: dict[str, Any]) -> dict[str, Any]:
|
||||
return node_visual_payload
|
||||
|
||||
|
||||
def _build_observed_pressure_from_scada(
|
||||
network: str,
|
||||
sensor_nodes: list[str],
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
"""Accumulate explanatory investigation paths and generate Mermaid deterministically."""
|
||||
import re
|
||||
from app.domain.domain_errors import require
|
||||
from app.domain.schemas.operational_task import (
|
||||
InvestigationDiagram, InvestigationDiagramNodeRef, InvestigationGraphUpdate,
|
||||
)
|
||||
|
||||
|
||||
def _label(value: str) -> str:
|
||||
# Encode Mermaid delimiters and HTML before placing untrusted text in quotes.
|
||||
replacements = {"#": "#35;", "&": "#38;", '"': "#quot;", "<": "#lt;",
|
||||
">": "#gt;", "`": "#96;", "%": "#37;", "|": "#124;", "\\": "#92;"}
|
||||
return "".join(replacements.get(c, " " if c in "\r\n" else c) for c in value)
|
||||
|
||||
|
||||
def adopt_legacy_investigation_graph(diagram: InvestigationDiagram) -> InvestigationDiagram:
|
||||
"""Make a saved legacy graph cumulative without parsing or changing its source."""
|
||||
if diagram.edges is not None:
|
||||
return diagram
|
||||
return diagram.model_copy(deep=True, update={"edges": [], "source_mermaid": diagram.mermaid})
|
||||
|
||||
|
||||
def update_investigation_graph(
|
||||
previous: InvestigationDiagram | None, update: InvestigationGraphUpdate,
|
||||
) -> InvestigationDiagram:
|
||||
"""Upsert nodes and append explicit edges; never delete a path or erase a failed attempt.
|
||||
|
||||
A retry uses a new node id. Relationships describe activity for readers and
|
||||
impose no ordering or completion requirements on the Agent.
|
||||
"""
|
||||
if previous is not None:
|
||||
previous = adopt_legacy_investigation_graph(previous)
|
||||
source = previous.source_mermaid if previous else None
|
||||
def in_source(node_id: str) -> bool:
|
||||
return bool(source and re.search(
|
||||
rf"(?<![A-Za-z0-9_-]){re.escape(node_id)}(?=\s*(?:[\[({{@;]|:::|--|==|-\.|$))",
|
||||
source, re.MULTILINE,
|
||||
))
|
||||
|
||||
nodes = {node.node_id: node.model_copy(deep=True) for node in previous.node_refs} if previous else {}
|
||||
require(len({node.node_id for node in update.nodes}) == len(update.nodes),
|
||||
"GRAPH_NODE_DUPLICATE", "Graph update contains duplicate node ids", status=422)
|
||||
for node in update.nodes:
|
||||
require(node.node_id.lower() not in {"end", "graph", "flowchart", "subgraph"},
|
||||
"GRAPH_NODE_ID_INVALID", "Use a descriptive graph node id", status=422)
|
||||
existing = nodes.get(node.node_id)
|
||||
if existing is None:
|
||||
require(not in_source(node.node_id), "GRAPH_LEGACY_NODE_CONFLICT",
|
||||
"This id belongs to the original Mermaid; use a new node id", status=409)
|
||||
nodes[node.node_id] = InvestigationDiagramNodeRef.model_validate(node.model_dump(exclude_unset=True))
|
||||
continue
|
||||
require(not in_source(node.node_id) or existing.label == node.label,
|
||||
"GRAPH_LEGACY_NODE_CONFLICT", "Keep the original node label; use a new node for a new step", status=409)
|
||||
require(existing.status != "failed" or (
|
||||
("status" not in node.model_fields_set or node.status == "failed")
|
||||
and existing.label == node.label
|
||||
), "GRAPH_FAILED_ATTEMPT_IMMUTABLE", "Keep the failed attempt; use a new node for a retry", status=409)
|
||||
changes = node.model_dump(exclude_unset=True)
|
||||
if existing.status == "failed" and existing.detail and not changes.get("detail"):
|
||||
changes.pop("detail", None)
|
||||
for field in ("evidence_refs", "action_refs", "map_targets"):
|
||||
combined = list(getattr(existing, field))
|
||||
for value in getattr(node, field):
|
||||
if value not in combined:
|
||||
combined.append(value)
|
||||
changes[field] = combined
|
||||
nodes[node.node_id] = InvestigationDiagramNodeRef.model_validate({
|
||||
**existing.model_dump(), **changes,
|
||||
})
|
||||
require(bool(nodes), "GRAPH_NODE_REQUIRED", "Add a node before publishing a graph", status=422)
|
||||
edges = list(previous.edges or []) if previous else []
|
||||
for edge in update.edges:
|
||||
require(edge.source in nodes and edge.target in nodes, "GRAPH_EDGE_NODE_NOT_FOUND",
|
||||
"Every graph edge must reference saved or newly added nodes", status=422)
|
||||
if edge not in edges:
|
||||
edges.append(edge.model_copy(deep=True))
|
||||
require(len(nodes) <= 100 and len(edges) <= 200, "GRAPH_CAPACITY_EXCEEDED",
|
||||
"Investigation graph exceeds 100 nodes or 200 edges; existing paths were retained", status=422)
|
||||
lines = [source or "flowchart TD"]
|
||||
for node in nodes.values():
|
||||
if in_source(node.node_id):
|
||||
continue # Preserve original branches, shapes and labels byte for byte.
|
||||
label = node.label + ("(失败)" if node.status == "failed" else "")
|
||||
lines.append(f' {node.node_id}["{_label(label)}"]')
|
||||
for edge in edges:
|
||||
connector = f' -->|"{_label(edge.label)}"| ' if edge.label else " --> "
|
||||
lines.append(f" {edge.source}{connector}{edge.target}")
|
||||
mermaid = "\n".join(lines)
|
||||
require(len(mermaid) <= 20000, "GRAPH_CAPACITY_EXCEEDED",
|
||||
"Investigation Mermaid exceeds 20000 characters; existing paths were retained", status=422)
|
||||
return InvestigationDiagram(
|
||||
title=update.title or (previous.title if previous else "调查脉络"),
|
||||
mermaid=mermaid, node_refs=list(nodes.values()), edges=edges, source_mermaid=source,
|
||||
)
|
||||
@@ -0,0 +1,222 @@
|
||||
"""Work-order completion and causal SCADA recovery evidence for incident tasks."""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from math import isfinite
|
||||
from statistics import median
|
||||
from uuid import UUID
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from psycopg import AsyncConnection
|
||||
from app.domain.schemas.project_settings import ProjectSettings
|
||||
|
||||
from app.services.content_digest import digest
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
|
||||
|
||||
RECOVERY_WORKFLOWS = {"burst_report", "scada_anomaly"}
|
||||
|
||||
|
||||
def track_work_order(task: dict, action: dict) -> bool:
|
||||
if task.get("workflow") not in RECOVERY_WORKFLOWS or action["kind"] != "create_work_order":
|
||||
return False
|
||||
receipt = action.get("receipt") or {}
|
||||
if action.get("execution_status") != "succeeded" or receipt.get("adapter") != "peripheral_sample":
|
||||
return False
|
||||
context = task.setdefault("handling_context", {})
|
||||
orders = context.setdefault("work_orders", {})
|
||||
order_id = receipt["receipt_id"]
|
||||
if order_id not in orders:
|
||||
proposed_at = action.get("proposed_at")
|
||||
for prior in orders.values():
|
||||
terminal_at = prior.get("terminal_recorded_at")
|
||||
# Only a new decision after the failed field attempt replaces it.
|
||||
# A delayed receipt for an older independent dispatch does not.
|
||||
if (not prior.get("superseded_by") and prior["status"] in {"failed", "cancelled"}
|
||||
and proposed_at and terminal_at
|
||||
and datetime.fromisoformat(proposed_at) > datetime.fromisoformat(terminal_at)):
|
||||
prior["superseded_by"] = order_id
|
||||
orders[order_id] = {"status": "awaiting_completion", "action_id": str(action["action_id"])}
|
||||
context.update(phase="waiting_work_order", title="等待现场工单返回")
|
||||
context.pop("verification_evidence_id", None)
|
||||
return True
|
||||
|
||||
|
||||
def accept_completion(task: dict, feedback: dict, *, recorded_at: datetime | None = None) -> bool:
|
||||
context = task.get("handling_context") or {}
|
||||
order_id = feedback["source_event_id"].removeprefix("peripheral_sample:")
|
||||
order = context.get("work_orders", {}).get(order_id)
|
||||
if order is None:
|
||||
return False
|
||||
order.update(status=feedback["status"], source_revision=feedback["source_revision"])
|
||||
if feedback["status"] in {"failed", "cancelled"}:
|
||||
if recorded_at is not None and not order.get("terminal_recorded_at"):
|
||||
order["terminal_recorded_at"] = recorded_at.isoformat()
|
||||
else:
|
||||
order.pop("terminal_recorded_at", None)
|
||||
# The synchronizer preserves the field activity's business time on ordinary
|
||||
# edits. A revised field receipt can explicitly correct the verification start.
|
||||
if feedback["status"] == "completed":
|
||||
order["returned_at"] = feedback["occurred_at"]
|
||||
else:
|
||||
order.pop("returned_at", None)
|
||||
if order.get("superseded_by"):
|
||||
return True
|
||||
orders = [item for item in context["work_orders"].values() if not item.get("superseded_by")]
|
||||
if orders and all(item["status"] == "completed" for item in orders):
|
||||
returned_at = max((item["returned_at"] for item in orders), key=datetime.fromisoformat)
|
||||
previous = context.get("returned_at")
|
||||
if (previous is None or datetime.fromisoformat(previous) != datetime.fromisoformat(returned_at)
|
||||
or context.get("phase") == "waiting_work_order"):
|
||||
context.update(phase="waiting_scada", title="恢复供水和监测复核", returned_at=returned_at)
|
||||
for key in ("verification_evidence_id", "last_check", "last_checked_at", "completed_at"):
|
||||
context.pop(key, None)
|
||||
else:
|
||||
context.update(phase="waiting_work_order", title="等待现场工单返回")
|
||||
context.pop("verification_evidence_id", None)
|
||||
return True
|
||||
|
||||
|
||||
def evaluate_recovery(rows: list[dict], *, device_ids: list[str], returned_at: datetime,
|
||||
incident_at: datetime, interval_seconds: int) -> dict:
|
||||
"""Verify the first SCADA sample strictly after the field feedback time T."""
|
||||
base = {"status": "waiting", "device_ids": device_ids, "returned_at": returned_at.isoformat(),
|
||||
"required_samples": 1, "interval_seconds": interval_seconds,
|
||||
"selection_method": "first_sample_after_feedback",
|
||||
"baseline_method": "normal_same_phase_range_plus_3_robust_sigma"}
|
||||
if not device_ids:
|
||||
return {**base, "reason": "monitoring_configuration_missing"}
|
||||
if returned_at < incident_at:
|
||||
return {**base, "reason": "completion_precedes_incident"}
|
||||
selected = set(device_ids)
|
||||
baseline: dict[tuple[str, int], list[float]] = {}
|
||||
observations: dict[datetime, dict[str, float]] = {}
|
||||
origin = incident_at - timedelta(days=15)
|
||||
for row in rows:
|
||||
if row["device_id"] not in selected:
|
||||
continue
|
||||
at = row["time"] if isinstance(row["time"], datetime) else datetime.fromisoformat(row["time"])
|
||||
if at > returned_at:
|
||||
# Keep the nearest frame even when it contains null/invalid values;
|
||||
# never skip it in order to find a later normal-looking sample.
|
||||
observations.setdefault(at, {})
|
||||
if row.get("quality") not in {None, "valid", "corrected"}:
|
||||
continue
|
||||
value = row.get("cleaned_value")
|
||||
if value is None:
|
||||
value = row.get("monitored_value")
|
||||
if value is None or not isfinite(float(value)):
|
||||
continue
|
||||
value = float(value)
|
||||
phase = int(at.timestamp()) % 86400
|
||||
if origin <= at < incident_at:
|
||||
baseline.setdefault((row["device_id"], phase), []).append(value)
|
||||
elif at > returned_at:
|
||||
observations[at][row["device_id"]] = value
|
||||
if not observations:
|
||||
return {**base, "reason": "post_completion_reading_missing"}
|
||||
at = min(observations)
|
||||
values = observations[at]
|
||||
phase = int(at.timestamp()) % 86400
|
||||
checks = []
|
||||
for device in device_ids:
|
||||
reference = baseline.get((device, phase), [])
|
||||
if device not in values or len(reference) < 15:
|
||||
checks.append({"device_id": device, "status": "missing_data"})
|
||||
continue
|
||||
center = median(reference)
|
||||
sigma = 1.4826 * median(abs(value - center) for value in reference)
|
||||
margin = max(3 * sigma, abs(center) * 1e-6, 1e-9)
|
||||
lower, upper = min(reference) - margin, max(reference) + margin
|
||||
checks.append({"device_id": device, "value": values[device], "lower": lower, "upper": upper,
|
||||
"normal_day_count": len(reference),
|
||||
"status": "normal" if lower <= values[device] <= upper else "abnormal"})
|
||||
sample = {"at": at.isoformat(), "devices": checks}
|
||||
if all(item["status"] == "normal" for item in checks):
|
||||
return {**base, "status": "passed", "verified_through": at.isoformat(), "samples": [sample]}
|
||||
return {**base, "reason": "nearest_post_completion_reading_not_normal_or_incomplete", "sample": sample}
|
||||
|
||||
|
||||
async def reconcile_recoveries(service: OperationalTaskService, timescale: AsyncConnection,
|
||||
settings: ProjectSettings, *, limit: int = 20,
|
||||
created_from: datetime | None = None) -> int:
|
||||
"""Poll only persisted recovery waits; publish evidence before waking the Agent."""
|
||||
from app.domain.schemas.operational_task import OperationalTaskSummary, StoredEvidenceRecord
|
||||
from app.domain.schemas.operational_data import TaskDataSnapshot
|
||||
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
|
||||
async with service.repo.transaction():
|
||||
candidates = await service.repo.recovery_candidates(limit, created_from, service.execution_scope)
|
||||
resumed = 0
|
||||
for task in candidates:
|
||||
if task.get("execution_scope") != service.execution_scope:
|
||||
continue
|
||||
context = task["handling_context"]
|
||||
returned_at = datetime.fromisoformat(context["returned_at"])
|
||||
incident_at = datetime.fromisoformat(task["trigger"]["occurred_at"])
|
||||
interval = settings.scada_monitoring_interval_seconds
|
||||
roles = settings.scada_monitoring
|
||||
devices = list(dict.fromkeys(roles.pressure_device_ids + roles.outlet_flow_device_ids)) if roles else []
|
||||
source = "scada_readings"
|
||||
through = service._now()
|
||||
if context.get("device_ids"):
|
||||
devices = context["device_ids"]
|
||||
interval = context.get("interval_seconds", interval)
|
||||
else:
|
||||
# Retain the points the Agent actually inspected, including water
|
||||
# quality or local incident sensors outside the project role lists.
|
||||
for evidence in await service.repo.list_evidence(task["task_id"]):
|
||||
saved = evidence.get("data_snapshot") or {}
|
||||
if saved.get("source") not in {"query_scada_readings", "scada_readings"}:
|
||||
continue
|
||||
data = saved.get("data")
|
||||
if isinstance(data, dict):
|
||||
data = data.get("data", data.get("items", []))
|
||||
if isinstance(data, list):
|
||||
devices.extend(row["device_id"] for row in data if isinstance(row, dict) and isinstance(row.get("device_id"), str))
|
||||
devices = list(dict.fromkeys(devices))
|
||||
async with timescale.transaction():
|
||||
# Field completion authorizes a separate recovery observation even
|
||||
# when the incident was found in a frozen replay. Keep that replay's
|
||||
# query window and batch boundary unchanged.
|
||||
rows = await ScadaRepository.get_recovery_scada_readings(
|
||||
timescale, devices, incident_at, returned_at, through, baseline_limit=100_001,
|
||||
) if through > returned_at and devices else []
|
||||
result = ({"status": "waiting", "reason": "reading_budget_exceeded"} if len(rows) > 100_000 else
|
||||
evaluate_recovery(rows, device_ids=devices, returned_at=returned_at,
|
||||
incident_at=incident_at, interval_seconds=interval or 0))
|
||||
async with service.repo.transaction():
|
||||
await service.repo.lock(f"task:{task['task_id']}")
|
||||
current = await service._task(UUID(task["task_id"]))
|
||||
# A new receipt, cancellation or another poll may have won the race.
|
||||
if current.get("handling_context") != context or current["status"] in {"cancelled", "failed"}:
|
||||
continue
|
||||
now = service._now()
|
||||
if devices:
|
||||
current["handling_context"].update(device_ids=devices, interval_seconds=interval or 0)
|
||||
current["handling_context"]["last_checked_at"] = now.isoformat()
|
||||
current["handling_context"]["last_check"] = result
|
||||
if result["status"] == "passed":
|
||||
evidence_id = "recovery:" + digest({"task_id": task["task_id"], "returned_at": context["returned_at"], "result": result})
|
||||
evidence = StoredEvidenceRecord(
|
||||
evidence_id=evidence_id, kind="system_data",
|
||||
summary="完工反馈时间之后最近时刻的 SCADA 已恢复正常,等待 Agent 完成复核报告",
|
||||
source_label="恢复供水监测复核", source_ref=f"operational-task:{task['task_id']}",
|
||||
observed_at=datetime.fromisoformat(result["verified_through"]), recorded_at=now,
|
||||
recorded_by=service._actor(), has_snapshot=True,
|
||||
data_snapshot=TaskDataSnapshot(source=source, source_ref=evidence_id, queried_at=now,
|
||||
origin="system_data", data=result, truncated=False),
|
||||
)
|
||||
if not await service.repo.get_evidence(task["task_id"], evidence_id):
|
||||
await service.repo.insert_evidence(task["task_id"], evidence.model_dump(mode="json"))
|
||||
current["handling_context"].update(phase="verification_ready", verification_evidence_id=evidence_id)
|
||||
await service._requeue_task(current, now)
|
||||
resumed += 1
|
||||
else:
|
||||
current.update(revision=current["revision"] + 1, updated_at=now)
|
||||
await service.repo.update_task(OperationalTaskSummary.model_validate(current).model_dump(
|
||||
mode="json", exclude={"active_run", "latest_run"}))
|
||||
await service._task_event(current, now)
|
||||
return resumed
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Require persisted complete SCADA input before final investigation conclusions."""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from app.domain.domain_errors import require
|
||||
|
||||
|
||||
def require_complete_scada_evidence(task: dict, evidence: list[dict], refs: list[str]) -> None:
|
||||
replay = task.get("source", {}).get("replay_context")
|
||||
if task.get("workflow") != "scada_anomaly" and not replay:
|
||||
return
|
||||
target = datetime.fromisoformat(
|
||||
replay["knowledge_cutoff"] if replay else task["trigger"]["occurred_at"]
|
||||
)
|
||||
for item in evidence:
|
||||
if item["evidence_id"] not in refs:
|
||||
continue
|
||||
snapshot = item.get("data_snapshot") or {}
|
||||
if snapshot.get("source") != "query_scada_readings" or snapshot.get("truncated"):
|
||||
continue
|
||||
result = snapshot.get("data") or {}
|
||||
data = result.get("data") if isinstance(result, dict) else None
|
||||
coverage = data.get("coverage") if isinstance(data, dict) else None
|
||||
if not coverage or coverage.get("mode") != "all_scada" or coverage.get("query_complete") is not True:
|
||||
continue
|
||||
window = result.get("window") or {}
|
||||
if not window.get("start") or not window.get("end"):
|
||||
continue
|
||||
start, end = (datetime.fromisoformat(window[key]) for key in ("start", "end"))
|
||||
if not start <= target <= end:
|
||||
continue
|
||||
if replay and (coverage.get("replay_id") != replay["replay_id"]
|
||||
or coverage.get("batch_sequence") != replay["batch_sequence"]
|
||||
or end != target):
|
||||
continue
|
||||
return
|
||||
require(False, "SCADA_COVERAGE_REQUIRED",
|
||||
"最终SCADA报告必须引用本轮全量读取证据。调用query_scada_readings并省略device_ids,"
|
||||
"在任务时间窗内读取全部授权设备、完整属性与观测,核对coverage和缺测清单后再发布。",
|
||||
status=409)
|
||||
+318
-150
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Iterable
|
||||
from typing import TYPE_CHECKING, Iterable
|
||||
from uuid import NAMESPACE_URL, UUID, uuid4, uuid5
|
||||
|
||||
from app.auth.permissions import (
|
||||
@@ -49,6 +49,8 @@ from app.domain.schemas.operational_task import (
|
||||
InvestigationReportRevision,
|
||||
InvestigationReportReference,
|
||||
InvestigationProgressSnapshot,
|
||||
InvestigationProgress,
|
||||
InvestigationDiagram,
|
||||
IssueCommandPayload,
|
||||
NotificationAcknowledgeCommand,
|
||||
OperationalTaskCreate,
|
||||
@@ -71,7 +73,16 @@ from app.domain.schemas.operational_task import (
|
||||
)
|
||||
from app.domain.schemas.operational_data import TaskDataSnapshot, TaskDataSource
|
||||
from app.services.content_digest import digest
|
||||
from app.services.investigation_graph import adopt_legacy_investigation_graph, update_investigation_graph
|
||||
from app.services.operational_presentation import resolve_overlay
|
||||
from app.services.operational_recovery import accept_completion, track_work_order
|
||||
from app.services.operational_scada_coverage import require_complete_scada_evidence
|
||||
from app.services import scada_presentation_coverage
|
||||
from app.services.time_comparison_chart import validate_time_comparison_chart
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
|
||||
|
||||
LEASE_DURATION = timedelta(seconds=60)
|
||||
@@ -86,11 +97,13 @@ def _task_json(model: OperationalTaskSummary) -> dict:
|
||||
|
||||
|
||||
class OperationalTaskService:
|
||||
def __init__(self, repository, context, permissions: Iterable[str], *, clock=None):
|
||||
def __init__(self, repository: OperationalTaskRepository, context, permissions: Iterable[str], *, clock=None,
|
||||
execution_scope: str = "production"):
|
||||
self.repo = repository
|
||||
self.context = context
|
||||
self.permissions = frozenset(permissions)
|
||||
self.clock = clock or (lambda: datetime.now(timezone.utc))
|
||||
self.execution_scope = execution_scope
|
||||
|
||||
def _now(self) -> datetime:
|
||||
now = self.clock()
|
||||
@@ -109,42 +122,14 @@ class OperationalTaskService:
|
||||
require(owner == command.agent_instance_id,
|
||||
"AGENT_INSTANCE_MISMATCH", "Agent instance does not own this run", status=409)
|
||||
|
||||
@property
|
||||
def _supports_runs(self) -> bool:
|
||||
return hasattr(self.repo, "get_active_run") and hasattr(self.repo, "update_run")
|
||||
|
||||
async def _active_run(self, task: dict) -> dict | None:
|
||||
if self._supports_runs:
|
||||
return await self.repo.get_active_run(str(task["task_id"]))
|
||||
if task.get("status") not in {"queued", "running"}:
|
||||
return None
|
||||
now = self._now()
|
||||
return _json(OperationalTaskRun(
|
||||
run_id=uuid5(NAMESPACE_URL, f"legacy-run:{task['task_id']}:{task['execution_generation']}"),
|
||||
task_id=task["task_id"],
|
||||
generation=task["execution_generation"],
|
||||
execution_request_id=task["execution_request_id"],
|
||||
status="running" if task["status"] == "running" else "queued",
|
||||
revision=task["revision"],
|
||||
agent_session_id=task.get("agent_session_id"),
|
||||
lease_expires_at=task.get("agent_lease_expires_at"),
|
||||
heartbeat_at=task.get("updated_at") if task["status"] == "running" else None,
|
||||
started_at=task.get("created_at") if task["status"] == "running" else None,
|
||||
dispatch_attempts=0,
|
||||
next_dispatch_at=task.get("created_at") or now,
|
||||
created_at=task.get("created_at") or now,
|
||||
updated_at=task.get("updated_at") or now,
|
||||
))
|
||||
return await self.repo.get_active_run(str(task["task_id"]))
|
||||
|
||||
async def _append_event(self, **kwargs) -> int | None:
|
||||
append_event = getattr(self.repo, "append_event", None)
|
||||
return await append_event(**kwargs) if append_event is not None else None
|
||||
async def _append_event(self, **kwargs) -> int:
|
||||
return await self.repo.append_event(**kwargs)
|
||||
|
||||
async def _list_access_requests(self, task_id: UUID | str) -> list[dict]:
|
||||
list_access_requests = getattr(self.repo, "list_access_requests", None)
|
||||
if list_access_requests is None:
|
||||
return []
|
||||
return await list_access_requests(str(task_id))
|
||||
return await self.repo.list_access_requests(str(task_id))
|
||||
|
||||
def _permission_view(self) -> OperationalTaskPermissions:
|
||||
permissions = self.permissions
|
||||
@@ -193,6 +178,7 @@ class OperationalTaskService:
|
||||
now = self._now()
|
||||
task_id = uuid4()
|
||||
task = OperationalTaskSummary(
|
||||
execution_scope=self.execution_scope,
|
||||
project_id=self.context.project_id,
|
||||
task_id=task_id,
|
||||
task_type=command.task_type,
|
||||
@@ -280,9 +266,10 @@ class OperationalTaskService:
|
||||
actions = await self.repo.list_actions(str(task_id))
|
||||
access_requests = await self._list_access_requests(task_id)
|
||||
active_run = await self._active_run(task)
|
||||
latest_run = (await self.repo.get_latest_run(str(task_id))
|
||||
if hasattr(self.repo, "get_latest_run") else active_run)
|
||||
latest_run = await self.repo.get_latest_run(str(task_id))
|
||||
progress = await self.repo.get_latest_progress(str(task_id), task["execution_generation"])
|
||||
if progress is None:
|
||||
progress = await self.repo.get_latest_graph_progress(str(task_id))
|
||||
work_order_snapshot = None
|
||||
if task["task_type"] == "external_work_order" and task["source"].get("external_id"):
|
||||
feedback = await self.repo.get_feedback(
|
||||
@@ -391,9 +378,8 @@ class OperationalTaskService:
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return TaskAccessRequest.model_validate(replay["access_request"])
|
||||
availability = getattr(self.repo, "access_requests_available", None)
|
||||
require(
|
||||
availability is None or await availability(),
|
||||
await self.repo.access_requests_available(),
|
||||
"DEPENDENCY_UNAVAILABLE",
|
||||
"Task data access requests are not enabled for this project database",
|
||||
status=503,
|
||||
@@ -487,6 +473,7 @@ class OperationalTaskService:
|
||||
task_type: str | None = None, task_created_from: datetime | None = None,
|
||||
) -> tuple[list[OperationalTaskRun], str | None]:
|
||||
rows = await self.repo.list_runs(
|
||||
execution_scope=self.execution_scope,
|
||||
status=status,
|
||||
limit=limit + 1,
|
||||
cursor_created_at=cursor_created_at,
|
||||
@@ -508,6 +495,9 @@ class OperationalTaskService:
|
||||
scope = f"run:{run_id}:claim"
|
||||
async with self.repo.transaction():
|
||||
await self.repo.require_execution_storage()
|
||||
task = await self._task(task_id)
|
||||
require(task.get("execution_scope") == self.execution_scope,
|
||||
"EXECUTION_SCOPE_MISMATCH", "当前环境不能领取其他执行范围的任务", status=403)
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return OperationalTaskRun.model_validate(replay["run"])
|
||||
@@ -570,6 +560,8 @@ class OperationalTaskService:
|
||||
return OperationalTaskRun.model_validate(replay["run"])
|
||||
await self.repo.lock(f"run:{run_id}")
|
||||
run = await self._run(task_id, run_id)
|
||||
require(run.get("failure_code") != "SUPERSEDED_BY_REQUEUE", "RUN_SUPERSEDED",
|
||||
"New task input superseded this run; the replacement run will continue", status=409)
|
||||
require(run["revision"] == command.expected_revision, "VERSION_CONFLICT",
|
||||
"Run revision changed", status=409)
|
||||
require(run["status"] in {"claimed", "running"}, "INVALID_TRANSITION",
|
||||
@@ -606,8 +598,7 @@ class OperationalTaskService:
|
||||
return OperationalTaskRun.model_validate(await self._run(task_id, run_id))
|
||||
|
||||
async def _fail_run_messages(self, run_id, reason: str, now: datetime) -> None:
|
||||
if hasattr(self.repo, "fail_run_messages"):
|
||||
await self.repo.fail_run_messages(str(run_id), reason, now)
|
||||
await self.repo.fail_run_messages(str(run_id), reason, now)
|
||||
|
||||
async def fail_run(
|
||||
self, task_id: UUID, run_id: UUID, command: AgentRunFailureCommand
|
||||
@@ -757,11 +748,12 @@ class OperationalTaskService:
|
||||
async with self.repo.transaction():
|
||||
for run in await self.repo.claim_expired_runs(
|
||||
limit, task_type=task_type, task_created_from=task_created_from,
|
||||
execution_scope=self.execution_scope,
|
||||
):
|
||||
# Lease recovery already holds a row lock. Try rather than wait
|
||||
# for the shared run lock to avoid reversing the lock order of
|
||||
# message/finish transactions; a busy run is checked next tick.
|
||||
if hasattr(self.repo, "try_lock") and not await self.repo.try_lock(f"run:{run['run_id']}"):
|
||||
if not await self.repo.try_lock(f"run:{run['run_id']}"):
|
||||
continue
|
||||
task_id = UUID(str(run["task_id"]))
|
||||
expected = run["revision"]
|
||||
@@ -773,6 +765,7 @@ class OperationalTaskService:
|
||||
timed_out = OperationalTaskRun.model_validate(run)
|
||||
await self.repo.update_run(_json(timed_out), expected)
|
||||
await self._fail_run_messages(run["run_id"], "Run lease expired", now)
|
||||
await task_questions.cancel_pending(self, task_id, now)
|
||||
await self._run_event(timed_out, now)
|
||||
task = await self._task(task_id)
|
||||
timeout_count = await self.repo.consecutive_timed_out_runs(str(task_id))
|
||||
@@ -827,6 +820,11 @@ class OperationalTaskService:
|
||||
"Only a running run accepts Agent artifacts", status=409)
|
||||
now = self._now()
|
||||
self._require_active_run_lease(active_run, now)
|
||||
if command.report:
|
||||
for item in command.report.ui_blocks:
|
||||
for element in item.block.spec.elements.values():
|
||||
if element.type == "Chart":
|
||||
validate_time_comparison_chart(element.props)
|
||||
if command.report and (
|
||||
command.report.stage == "final" or command.report.outcome == "insufficient_data"
|
||||
):
|
||||
@@ -860,11 +858,27 @@ class OperationalTaskService:
|
||||
for item in await self.repo.list_actions(str(task_id))
|
||||
}
|
||||
progress_revision = None
|
||||
saved_diagram = None
|
||||
if command.progress is not None or command.report is not None:
|
||||
stored = await self.repo.get_latest_graph_progress(str(task_id))
|
||||
if stored is not None:
|
||||
saved_diagram = InvestigationProgressSnapshot.model_validate(stored).progress.diagram
|
||||
elif command.progress is not None:
|
||||
legacy = await self.repo.get_latest_diagram(str(task_id))
|
||||
saved_diagram = InvestigationDiagram.model_validate(legacy) if legacy else None
|
||||
if command.progress is not None:
|
||||
require(command.run_id is not None, "RUN_REQUIRED",
|
||||
"Progress must identify its current execution run", status=422)
|
||||
if command.progress.diagram:
|
||||
for node in command.progress.diagram.node_refs:
|
||||
diagram = command.progress.diagram
|
||||
if command.progress.graph_update is not None:
|
||||
diagram = update_investigation_graph(saved_diagram, command.progress.graph_update)
|
||||
elif saved_diagram is not None:
|
||||
require(diagram is None or saved_diagram.edges is None, "GRAPH_UPDATE_REQUIRED",
|
||||
"Use graph_update to extend the saved investigation graph", status=422)
|
||||
if diagram is None:
|
||||
diagram = adopt_legacy_investigation_graph(saved_diagram)
|
||||
if diagram:
|
||||
for node in diagram.node_refs:
|
||||
self._require_evidence_refs(node.evidence_refs, known_evidence)
|
||||
require(all(str(ref) in actions for ref in node.action_refs),
|
||||
"ACTION_NOT_FOUND", "Progress references an unknown action", status=422)
|
||||
@@ -873,9 +887,11 @@ class OperationalTaskService:
|
||||
snapshot = InvestigationProgressSnapshot(
|
||||
revision=progress_revision, run_id=active_run["run_id"],
|
||||
execution_generation=task["execution_generation"],
|
||||
source_session_id=command.source_session_id, published_at=now, progress=command.progress,
|
||||
source_session_id=command.source_session_id, published_at=now,
|
||||
progress=InvestigationProgress(summary=command.progress.summary, diagram=diagram),
|
||||
)
|
||||
await self.repo.save_progress(str(task_id), _json(snapshot))
|
||||
saved_diagram = diagram
|
||||
await self._append_event(
|
||||
topic="operational.progress.changed", aggregate_type="operational_task_progress",
|
||||
aggregate_id=str(task_id), aggregate_revision=progress_revision, occurred_at=now,
|
||||
@@ -927,17 +943,32 @@ class OperationalTaskService:
|
||||
await self.repo.insert_plan(str(task_id), _json(snapshot))
|
||||
current_plan = snapshot
|
||||
if command.report:
|
||||
self._require_evidence_refs(command.report.evidence_refs, known_evidence)
|
||||
for gap in command.report.data_gaps:
|
||||
require(set(gap.evidence_refs) <= set(command.report.evidence_refs),
|
||||
report_input = command.report
|
||||
if saved_diagram is not None:
|
||||
diagram = adopt_legacy_investigation_graph(saved_diagram)
|
||||
report_input = report_input.model_copy(update={
|
||||
"diagram": diagram,
|
||||
"evidence_refs": list(dict.fromkeys([
|
||||
*report_input.evidence_refs,
|
||||
*(ref for node in diagram.node_refs for ref in node.evidence_refs),
|
||||
])),
|
||||
})
|
||||
self._require_evidence_refs(report_input.evidence_refs, known_evidence)
|
||||
if report_input.stage == "final":
|
||||
require_complete_scada_evidence(
|
||||
task, await self.repo.list_evidence(str(task_id)), report_input.evidence_refs,
|
||||
)
|
||||
await self._verify_scada_report_presentation(task, active_run, _json(report_input), now)
|
||||
for gap in report_input.data_gaps:
|
||||
require(set(gap.evidence_refs) <= set(report_input.evidence_refs),
|
||||
"REPORT_GAP_EVIDENCE_INVALID",
|
||||
"Gap evidence must be included in report evidence", status=422)
|
||||
if command.report.diagram:
|
||||
for node in command.report.diagram.node_refs:
|
||||
self._require_evidence_refs(node.evidence_refs, set(command.report.evidence_refs))
|
||||
if report_input.diagram:
|
||||
for node in report_input.diagram.node_refs:
|
||||
self._require_evidence_refs(node.evidence_refs, set(report_input.evidence_refs))
|
||||
require(all(str(ref) in actions for ref in node.action_refs),
|
||||
"ACTION_NOT_FOUND", "Diagram references an unknown action", status=422)
|
||||
if command.report.outcome == "insufficient_data":
|
||||
if report_input.outcome == "insufficient_data":
|
||||
require(not any(action.approval_status == "pending" or (
|
||||
action.approval_status == "approved" and action.execution_status in {"not_submitted", "submitted", "unknown"}
|
||||
) for action in actions.values()), "REPORT_ACTION_PENDING",
|
||||
@@ -947,7 +978,7 @@ class OperationalTaskService:
|
||||
revision=report_revision,
|
||||
execution_generation=task["execution_generation"],
|
||||
source_session_id=command.source_session_id,
|
||||
published_at=now, published_by=self._actor(), report=command.report,
|
||||
published_at=now, published_by=self._actor(), report=report_input,
|
||||
)
|
||||
report = await self._report_with_progress_diagram(task_id, report)
|
||||
await self.repo.insert_report(str(task_id), _json(report))
|
||||
@@ -960,20 +991,21 @@ class OperationalTaskService:
|
||||
metadata={"task_id": str(task_id), "run_id": str(active_run["run_id"])},
|
||||
)
|
||||
previous_result = previous_report["report"] if previous_report else None
|
||||
if command.report.stage == "final" and (
|
||||
effective_report = _json(report.report)
|
||||
if report.report.stage == "final" and (
|
||||
previous_report is None
|
||||
or previous_report.get("execution_generation") != task["execution_generation"]
|
||||
or previous_result.get("stage") != "final"
|
||||
or any(previous_result.get(key, [] if key == "ui_blocks" else None) != _json(command.report).get(key)
|
||||
or any(previous_result.get(key, [] if key == "ui_blocks" else None) != effective_report.get(key)
|
||||
for key in ("current_conclusion", "conclusion_boundary", "outcome", "stop_reason",
|
||||
"evidence_refs", "recommendations", "pending_items", "data_gaps", "ui_blocks"))
|
||||
"evidence_refs", "recommendations", "pending_items", "data_gaps", "ui_blocks", "scada_device_ids"))
|
||||
):
|
||||
await self._resolve_notifications(
|
||||
task_id, kinds=["report_updated"], subject_type="task", subject_id=task_id,
|
||||
now=now, reason="report_superseded",
|
||||
)
|
||||
await self._notify(
|
||||
task_id, "report_updated", task["title"], command.report.current_conclusion, now,
|
||||
task_id, "report_updated", task["title"], report.report.current_conclusion, now,
|
||||
report_reference=InvestigationReportReference(
|
||||
execution_generation=task["execution_generation"], revision=report_revision,
|
||||
),
|
||||
@@ -997,25 +1029,9 @@ class OperationalTaskService:
|
||||
lease_expires_at=now + LEASE_DURATION,
|
||||
updated_at=now,
|
||||
)
|
||||
if self._supports_runs:
|
||||
await self.repo.update_run(
|
||||
_json(OperationalTaskRun.model_validate(active_run)), run_expected
|
||||
)
|
||||
else:
|
||||
# Compatibility for pre-015 repository adapters only.
|
||||
if command.report:
|
||||
if task["task_type"] == "event_investigation":
|
||||
task["status"] = self._investigation_status(actions.values()) if command.report.stage == "final" else "running"
|
||||
else:
|
||||
task["status"] = self._status_from_plan(current_plan.plan.nodes, actions.values()) if current_plan else "running"
|
||||
if task["status"] in {"waiting_human", "completed", "failed", "cancelled"}:
|
||||
task["lifecycle_status"] = task["status"]
|
||||
task["agent_lease_expires_at"] = (
|
||||
now + LEASE_DURATION if task["status"] == "running" else None
|
||||
)
|
||||
await self.repo.update_task(
|
||||
_task_json(OperationalTaskSummary.model_validate(task))
|
||||
)
|
||||
await self.repo.update_run(
|
||||
_json(OperationalTaskRun.model_validate(active_run)), run_expected
|
||||
)
|
||||
await self._task_event(task, now)
|
||||
response = {"plan_revision": plan_revision, "report_revision": report_revision}
|
||||
if progress_revision is not None:
|
||||
@@ -1069,8 +1085,6 @@ class OperationalTaskService:
|
||||
await self.repo.lock(f"run:{command.run_id}")
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
require(not task.get("source", {}).get("replay_context"), "REPLAY_ACTION_DISABLED",
|
||||
"SCADA 回放调查仅生成分析与报告,不执行真实工单或设备动作", status=409)
|
||||
run = await self._active_run(task)
|
||||
require(run is not None and str(run["run_id"]) == str(command.run_id)
|
||||
and run["generation"] == task["execution_generation"],
|
||||
@@ -1082,7 +1096,11 @@ class OperationalTaskService:
|
||||
now = self._now()
|
||||
self._require_active_run_lease(run, now)
|
||||
payload = _json(command.command)
|
||||
await self._record_scada_presentation(task, run, payload, now)
|
||||
if command.command.action == "open_chart":
|
||||
validate_time_comparison_chart(payload["chart"], allow_missing_current=bool(
|
||||
payload["chart"].get("scada") and scada_presentation_coverage.applies(task),
|
||||
))
|
||||
known = {item["evidence_id"] for item in await self.repo.list_evidence(str(task_id))}
|
||||
self._require_evidence_refs(command.command.evidence_refs, known)
|
||||
elif command.command.action == "set_overlay":
|
||||
@@ -1120,6 +1138,61 @@ class OperationalTaskService:
|
||||
await self._save_request(scope, command.request_id, body, _json(result))
|
||||
return result
|
||||
|
||||
async def _record_scada_presentation(self, task: dict, run: dict, payload: dict, now: datetime) -> None:
|
||||
if not scada_presentation_coverage.applies(task):
|
||||
return
|
||||
action = payload["action"]
|
||||
targets = []
|
||||
if action == "render_scada_analysis":
|
||||
targets = [item["device_id"] for item in payload["items"]]
|
||||
elif action in {"locate_features", "highlight_features"} and payload["layer"] == "scada":
|
||||
targets = payload["feature_ids"]
|
||||
elif action != "open_chart":
|
||||
return
|
||||
evidence = await self.repo.list_evidence(task["task_id"])
|
||||
observations = scada_presentation_coverage.ScadaObservations(task, evidence)
|
||||
document = await self.repo.get_scada_presentation(task["task_id"], str(run["run_id"]))
|
||||
if action == "open_chart":
|
||||
if payload["chart"].get("scada"):
|
||||
known = {item["evidence_id"] for item in evidence}
|
||||
self._require_evidence_refs(payload["evidence_refs"], known)
|
||||
chart = scada_presentation_coverage.verify_chart(task, payload["chart"], evidence, payload["evidence_refs"])
|
||||
targets = chart["device_ids"]
|
||||
duplicates = [chart_id for chart_id, stored in (document or {}).get("charts", {}).items()
|
||||
if chart_id != payload["chart_id"] and stored["role"] == chart["role"] == "comparison"
|
||||
and any(set(stored.get("coverage", {}).get(device, {}).get("covered", []))
|
||||
& set(chart["coverage"][device]["covered"])
|
||||
for device in set(stored["device_ids"]) & set(targets))]
|
||||
require(not duplicates, "SCADA_CHART_DUPLICATE_DEVICE",
|
||||
f"设备同期图已存在,请更新原chart_id:{', '.join(duplicates)}", refs=duplicates)
|
||||
document = scada_presentation_coverage.add_targets(document, targets, observations)
|
||||
document["charts"][payload["chart_id"]] = chart
|
||||
elif document and payload["chart_id"] in document["charts"]:
|
||||
# Replacing a typed chart with a generic one removes its coverage,
|
||||
# while cumulative targets prevent shrinking the required set.
|
||||
del document["charts"][payload["chart_id"]]
|
||||
else:
|
||||
return
|
||||
else:
|
||||
document = scada_presentation_coverage.add_targets(document, targets, observations)
|
||||
await self.repo.save_scada_presentation(task["task_id"], str(run["run_id"]), document, now)
|
||||
|
||||
async def _verify_scada_report_presentation(self, task: dict, run: dict, report: dict, now: datetime) -> None:
|
||||
if not scada_presentation_coverage.applies(task):
|
||||
return
|
||||
if report.get("stage") != "final" and not report.get("scada_device_ids"):
|
||||
return
|
||||
evidence = await self.repo.list_evidence(task["task_id"])
|
||||
observations = scada_presentation_coverage.ScadaObservations(task, evidence)
|
||||
document = await self.repo.get_scada_presentation(task["task_id"], str(run["run_id"]))
|
||||
targets = list(report.get("scada_device_ids") or [])
|
||||
if targets:
|
||||
document = scada_presentation_coverage.add_targets(document, targets, observations)
|
||||
if report.get("stage") == "final":
|
||||
scada_presentation_coverage.require_complete(task, document, observations)
|
||||
if document:
|
||||
await self.repo.save_scada_presentation(task["task_id"], str(run["run_id"]), document, now)
|
||||
|
||||
async def propose_action(
|
||||
self, task_id: UUID, command: ActionProposalCommand
|
||||
) -> HumanApprovalAction:
|
||||
@@ -1133,6 +1206,7 @@ class OperationalTaskService:
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
self._require_real_action(task, command.kind)
|
||||
self._validate_action_payload(command)
|
||||
if task["task_type"] == "event_investigation":
|
||||
report = await self.repo.get_latest_report(str(task_id))
|
||||
@@ -1201,8 +1275,10 @@ class OperationalTaskService:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
self._require_actionable_task(await self._task(task_id))
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
action = await self._action(task_id, action_id)
|
||||
self._require_real_action(task, action.kind)
|
||||
require(action.kind == "create_work_order", "INVALID_ACTION_KIND",
|
||||
"Only work-order drafts can be edited", status=422)
|
||||
require(action.version == command.expected_version, "VERSION_CONFLICT",
|
||||
@@ -1255,8 +1331,10 @@ class OperationalTaskService:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
self._require_actionable_task(await self._task(task_id))
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
action = await self._action(task_id, action_id)
|
||||
self._require_real_action(task, action.kind)
|
||||
require(action.version == command.expected_version, "VERSION_CONFLICT",
|
||||
"Action version changed", status=409)
|
||||
require(action.approval_status == "pending", "INVALID_TRANSITION",
|
||||
@@ -1300,8 +1378,10 @@ class OperationalTaskService:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
self._require_actionable_task(await self._task(task_id))
|
||||
task = await self._task(task_id)
|
||||
self._require_actionable_task(task)
|
||||
action = await self._action(task_id, action_id)
|
||||
self._require_real_action(task, action.kind)
|
||||
require(action.version == command.expected_version, "VERSION_CONFLICT",
|
||||
"Action version changed", status=409)
|
||||
require(action.approval_status == "approved", "INVALID_TRANSITION",
|
||||
@@ -1361,6 +1441,7 @@ class OperationalTaskService:
|
||||
|
||||
async with self.repo.transaction():
|
||||
await self.repo.lock(f"request:{scope}:{command.request_id}")
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
current = await self._action(task_id, action_id)
|
||||
require(current.version == action.version and current.execution_status == "unknown",
|
||||
@@ -1396,7 +1477,12 @@ class OperationalTaskService:
|
||||
)
|
||||
if outcome.status in {"succeeded", "failed", "unsupported"}:
|
||||
task_document = await self._task(task_id)
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
tracked = track_work_order(task_document, _json(current))
|
||||
if (tracked or task_document["status"] == "cancelled"
|
||||
or (task_document.get("source") or {}).get("deleted_at")):
|
||||
await self._keep_work_order_wait(task_document, outcome_recorded_at)
|
||||
else:
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
await self.repo.update_task(
|
||||
_task_json(OperationalTaskSummary.model_validate(task_document))
|
||||
)
|
||||
@@ -1432,6 +1518,7 @@ class OperationalTaskService:
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return HumanApprovalAction.model_validate(replay["action"])
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
await self.repo.lock(f"action:{action_id}")
|
||||
current = await self._action(task_id, action_id)
|
||||
require(current.version == action.version, "VERSION_CONFLICT",
|
||||
@@ -1469,7 +1556,12 @@ class OperationalTaskService:
|
||||
)
|
||||
if outcome.status in {"succeeded", "failed", "unsupported"}:
|
||||
task_document = await self._task(task_id)
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
tracked = track_work_order(task_document, _json(current))
|
||||
if (tracked or task_document["status"] == "cancelled"
|
||||
or (task_document.get("source") or {}).get("deleted_at")):
|
||||
await self._keep_work_order_wait(task_document, outcome_recorded_at)
|
||||
else:
|
||||
await self._requeue_task(task_document, outcome_recorded_at)
|
||||
await self.repo.update_task(
|
||||
_task_json(OperationalTaskSummary.model_validate(task_document))
|
||||
)
|
||||
@@ -1485,11 +1577,13 @@ class OperationalTaskService:
|
||||
body = _json(command)
|
||||
scope = f"task:{task_id}:external-feedback"
|
||||
async with self.repo.transaction():
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
require(task.get("execution_scope") == self.execution_scope,
|
||||
"EXECUTION_SCOPE_MISMATCH", "当前环境不能同步其他执行范围的任务", status=403)
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return ExternalFeedbackResult(task=await self.detail(task_id), **replay)
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
if task["task_type"] == "event_investigation":
|
||||
self._require_actionable_task(task)
|
||||
own_source = task["source"].get("external_id")
|
||||
@@ -1571,8 +1665,8 @@ class OperationalTaskService:
|
||||
"EVIDENCE_CONFLICT", "Evidence ids are immutable", status=409)
|
||||
else:
|
||||
await self.repo.insert_evidence(str(task_id), _json(evidence))
|
||||
self._apply_feedback_status(task, command.status)
|
||||
if task["task_type"] == "external_work_order":
|
||||
self._apply_feedback_status(task, command.status)
|
||||
task["source"] = {
|
||||
**task["source"],
|
||||
"external_revision": command.source_revision,
|
||||
@@ -1589,9 +1683,30 @@ class OperationalTaskService:
|
||||
),
|
||||
)
|
||||
if task["task_type"] == "event_investigation":
|
||||
# Recover older tasks whose successful dispatch predates tracking.
|
||||
for item in await self.repo.list_actions(str(task_id)):
|
||||
track_work_order(task, item)
|
||||
linked_work_order = accept_completion(task, body, recorded_at=now)
|
||||
if matching_source:
|
||||
task["source"] = {**task["source"], "external_revision": command.source_revision}
|
||||
await self._requeue_task(task, now)
|
||||
feedback_order = (task.get("handling_context") or {}).get("work_orders", {}).get(
|
||||
command.source_event_id.removeprefix("peripheral_sample:"), {})
|
||||
if linked_work_order and (feedback_order.get("superseded_by")
|
||||
or task["handling_context"].get("phase") == "completed"):
|
||||
# A receipt edit with the same completion time does not
|
||||
# reopen an incident that already passed recovery review.
|
||||
task.update(revision=task["revision"] + 1, updated_at=now)
|
||||
elif (linked_work_order and command.status not in {"failed", "cancelled"}
|
||||
and task["handling_context"].get("phase") in {"waiting_work_order", "waiting_scada"}):
|
||||
# Progress is persisted, not a reason to restart analysis.
|
||||
# Completed field work waits for causal SCADA evidence;
|
||||
# reconcile_recoveries wakes the same task when it is ready.
|
||||
await self._keep_work_order_wait(task, now)
|
||||
elif (linked_work_order and task["handling_context"].get("phase") == "verification_ready"
|
||||
and await self.repo.get_active_run(str(task_id)) is not None):
|
||||
await self._keep_work_order_wait(task, now)
|
||||
else:
|
||||
await self._requeue_task(task, now)
|
||||
else:
|
||||
task.update(revision=task["revision"] + 1, updated_at=now)
|
||||
await self.repo.update_task(_task_json(OperationalTaskSummary.model_validate(task)))
|
||||
@@ -1642,11 +1757,13 @@ class OperationalTaskService:
|
||||
body = _json(command)
|
||||
scope = f"task:{task_id}:withdraw-external-source"
|
||||
async with self.repo.transaction():
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
require(task.get("execution_scope") == self.execution_scope,
|
||||
"EXECUTION_SCOPE_MISMATCH", "当前环境不能同步其他执行范围的任务", status=403)
|
||||
replay = await self._request_replay(scope, command.request_id, body)
|
||||
if replay:
|
||||
return await self.detail(task_id)
|
||||
await self.repo.lock(f"task:{task_id}")
|
||||
task = await self._task(task_id)
|
||||
source = task["source"]
|
||||
require(
|
||||
task["task_type"] == "external_work_order"
|
||||
@@ -1952,13 +2069,12 @@ class OperationalTaskService:
|
||||
return DispatchDueSchedulesResult(task_ids=task_ids, dispatched_at=now)
|
||||
|
||||
async def _require_no_pending_calculations(self, task_id: UUID, run: dict) -> None:
|
||||
if hasattr(self.repo, "pending_calculation_jobs"):
|
||||
pending = await self.repo.pending_calculation_jobs(
|
||||
str(task_id), str(run["run_id"]), str(run["execution_request_id"]),
|
||||
)
|
||||
require(not pending, "PENDING_CALCULATIONS",
|
||||
"Wait for submitted calculations before publishing a final report or completing the run",
|
||||
refs=pending, status=409, recovery="retry_same_request")
|
||||
pending = await self.repo.pending_calculation_jobs(
|
||||
str(task_id), str(run["run_id"]), str(run["execution_request_id"]),
|
||||
)
|
||||
require(not pending, "PENDING_CALCULATIONS",
|
||||
"Wait for submitted calculations before publishing a final report or completing the run",
|
||||
refs=pending, status=409, recovery="retry_same_request")
|
||||
|
||||
async def _finish_run(
|
||||
self, task_id: UUID, run_id: UUID, command: AgentRunCommand,
|
||||
@@ -1988,11 +2104,10 @@ class OperationalTaskService:
|
||||
actions = []
|
||||
if outcome == "succeeded":
|
||||
await self._require_no_pending_calculations(task_id, run)
|
||||
if hasattr(self.repo, "pending_run_messages"):
|
||||
pending_messages = await self.repo.pending_run_messages(str(run_id))
|
||||
require(not pending_messages, "PENDING_AGENT_MESSAGES",
|
||||
"Run has messages not yet entered into Agent context",
|
||||
status=409, recovery="retry_same_request", refs=pending_messages)
|
||||
pending_messages = await self.repo.pending_run_messages(str(run_id))
|
||||
require(not pending_messages, "PENDING_AGENT_MESSAGES",
|
||||
"Run has messages not yet entered into Agent context",
|
||||
status=409, recovery="retry_same_request", refs=pending_messages)
|
||||
plan_document = await self.repo.get_latest_plan(str(task_id))
|
||||
report_document = await self.repo.get_latest_report(str(task_id))
|
||||
plan = (
|
||||
@@ -2013,8 +2128,22 @@ class OperationalTaskService:
|
||||
and plan.source_session_id == command.source_session_id
|
||||
))
|
||||
)
|
||||
actions = [HumanApprovalAction.model_validate(item)
|
||||
for item in await self.repo.list_actions(str(task_id))]
|
||||
work_order_wait = task["task_type"] == "event_investigation" and (
|
||||
(task.get("handling_context") or {}).get("phase") in {"waiting_work_order", "waiting_scada"}
|
||||
or any(action.kind == "create_work_order" and (
|
||||
action.approval_status == "pending" or (
|
||||
action.approval_status == "approved"
|
||||
and action.execution_status in {"not_submitted", "submitted", "unknown"}
|
||||
)) for action in actions)
|
||||
)
|
||||
if pending_question:
|
||||
lifecycle = "waiting_human"
|
||||
elif same_generation and work_order_wait and report.report.stage == "interim":
|
||||
# Ending a run is distinct from closing an incident. Keep
|
||||
# the current stage report and release the execution lease.
|
||||
lifecycle = "waiting_human"
|
||||
elif not same_generation or report.report.stage != "final":
|
||||
final_outcome = "failed"
|
||||
lifecycle = "failed"
|
||||
@@ -2023,16 +2152,21 @@ class OperationalTaskService:
|
||||
"Run requires a final report from the same generation"
|
||||
)
|
||||
else:
|
||||
actions = [
|
||||
HumanApprovalAction.model_validate(item)
|
||||
for item in await self.repo.list_actions(str(task_id))
|
||||
]
|
||||
await self._verify_scada_report_presentation(task, run, _json(report.report), now)
|
||||
access_requests = [
|
||||
TaskAccessRequest.model_validate(item)
|
||||
for item in await self._list_access_requests(task_id)
|
||||
]
|
||||
if task["task_type"] == "event_investigation":
|
||||
lifecycle = self._investigation_status(actions)
|
||||
lifecycle = self._investigation_status(actions, task)
|
||||
recovery = task.get("handling_context") or {}
|
||||
if lifecycle == "completed" and recovery:
|
||||
require(recovery.get("verification_evidence_id") in report.report.evidence_refs,
|
||||
"RECOVERY_EVIDENCE_REQUIRED", "最终复核报告必须引用恢复监测证据", status=409)
|
||||
require(report.report.outcome == "concluded", "RECOVERY_EVIDENCE_REQUIRED",
|
||||
"恢复复核通过后才能结束事件", status=409)
|
||||
recovery.update(phase="completed", title="恢复供水复核成功", completed_at=now.isoformat())
|
||||
await self._resolve_investigation_alerts(str(task_id), recovery["verification_evidence_id"])
|
||||
elif report.report.outcome == "insufficient_data":
|
||||
terminal = all(node.status in {"completed", "skipped", "cancelled"}
|
||||
for node in plan.plan.nodes)
|
||||
@@ -2081,7 +2215,8 @@ class OperationalTaskService:
|
||||
failure_detail or "工况任务执行失败",
|
||||
now,
|
||||
)
|
||||
elif lifecycle == "waiting_human" and not pending_question:
|
||||
elif (lifecycle == "waiting_human" and not pending_question
|
||||
and (task.get("handling_context") or {}).get("phase") not in {"waiting_work_order", "waiting_scada"}):
|
||||
pending_actions = any(
|
||||
action.approval_status == "pending" for action in actions
|
||||
)
|
||||
@@ -2108,11 +2243,31 @@ class OperationalTaskService:
|
||||
)
|
||||
return run
|
||||
|
||||
async def _resolve_investigation_alerts(self, task_id: str, evidence_id: str) -> None:
|
||||
from app.domain.schemas.task_alert import TaskAlertEvent
|
||||
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
|
||||
from app.services.task_alerts import TaskAlertService
|
||||
|
||||
for alert in await self.repo.list_investigation_alerts(task_id):
|
||||
command = TaskAlertEvent.model_validate({
|
||||
**{key: value for key, value in alert.items() if key not in {"status", "updated_at", "revision"}},
|
||||
"event_id": uuid5(NAMESPACE_URL, f"recovery:{task_id}:{evidence_id}:{alert['alert_id']}"),
|
||||
"event_type": "resolved",
|
||||
})
|
||||
await TaskAlertService(TaskAlertRepository(self.repo.connection, self.repo.project_id), None,
|
||||
internal_producer_id=command.producer_id).push(command)
|
||||
|
||||
async def _task(self, task_id: UUID) -> dict:
|
||||
task = await self.repo.get_task(str(task_id))
|
||||
require(task is not None, "TASK_NOT_FOUND", "Operational task not found", status=404)
|
||||
return task
|
||||
|
||||
@staticmethod
|
||||
def _require_real_action(task: dict, kind: str) -> None:
|
||||
require(kind == "create_work_order" or not (task.get("source") or {}).get("replay_context"),
|
||||
"REPLAY_ACTION_DISABLED",
|
||||
"回放任务不支持设备命令;可生成工单草稿,由有权限的用户确认发布", status=409)
|
||||
|
||||
@staticmethod
|
||||
def _require_actionable_task(task: dict) -> None:
|
||||
source = task.get("source") or {}
|
||||
@@ -2176,8 +2331,10 @@ class OperationalTaskService:
|
||||
return "running"
|
||||
|
||||
@staticmethod
|
||||
def _investigation_status(actions: Iterable[HumanApprovalAction]) -> str:
|
||||
"""Only recorded business actions can keep an investigation waiting."""
|
||||
def _investigation_status(actions: Iterable[HumanApprovalAction], task: dict | None = None) -> str:
|
||||
"""Field completion and recovery evidence precede incident closure."""
|
||||
if (task or {}).get("handling_context", {}).get("phase") in {"waiting_work_order", "waiting_scada"}:
|
||||
return "waiting_human"
|
||||
return "waiting_human" if any(
|
||||
action.approval_status == "pending" or (
|
||||
action.approval_status == "approved"
|
||||
@@ -2293,44 +2450,58 @@ class OperationalTaskService:
|
||||
}[status]
|
||||
task["agent_lease_expires_at"] = None
|
||||
|
||||
async def _keep_work_order_wait(self, task: dict, now: datetime) -> None:
|
||||
"""Persist a field wait without revoking a run or allocating a generation."""
|
||||
if task["status"] != "cancelled" and not (task.get("source") or {}).get("deleted_at"):
|
||||
active = await self.repo.get_active_run(str(task["task_id"]))
|
||||
if active is not None:
|
||||
task.update(
|
||||
lifecycle_status="open", status="queued" if active["status"] == "queued" else "running",
|
||||
agent_session_id=active.get("agent_session_id"),
|
||||
agent_lease_expires_at=active.get("lease_expires_at"),
|
||||
)
|
||||
else:
|
||||
task.update(lifecycle_status="waiting_human", status="waiting_human",
|
||||
agent_session_id=None, agent_lease_expires_at=None)
|
||||
task.update(revision=task["revision"] + 1, updated_at=now)
|
||||
|
||||
async def _requeue_task(self, task: dict, now: datetime) -> None:
|
||||
await task_questions.cancel_pending(self, UUID(str(task["task_id"])), now)
|
||||
if self._supports_runs:
|
||||
active_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
while active_run is not None:
|
||||
await self.repo.lock(f"run:{active_run['run_id']}")
|
||||
# Completion may have won the run lock after our first read.
|
||||
refreshed_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
if refreshed_run is None or str(refreshed_run["run_id"]) == str(active_run["run_id"]):
|
||||
active_run = refreshed_run
|
||||
break
|
||||
active_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
while active_run is not None:
|
||||
await self.repo.lock(f"run:{active_run['run_id']}")
|
||||
# Completion may have won the run lock after our first read.
|
||||
refreshed_run = await self.repo.get_active_run(str(task["task_id"]))
|
||||
if refreshed_run is None or str(refreshed_run["run_id"]) == str(active_run["run_id"]):
|
||||
active_run = refreshed_run
|
||||
if active_run is not None and active_run["status"] == "queued":
|
||||
# Multiple inputs can arrive before the worker claims the next
|
||||
# generation. They are all visible when that queued run starts,
|
||||
# so coalesce them instead of violating the one-active-run rule.
|
||||
task.update(
|
||||
lifecycle_status="open", status="queued",
|
||||
revision=task["revision"] + 1,
|
||||
execution_generation=active_run["generation"],
|
||||
execution_request_id=str(active_run["execution_request_id"]),
|
||||
agent_session_id=None, agent_lease_expires_at=None,
|
||||
updated_at=now,
|
||||
)
|
||||
return
|
||||
if active_run is not None:
|
||||
expected = active_run["revision"]
|
||||
active_run.update(
|
||||
status="cancelled", revision=expected + 1,
|
||||
lease_expires_at=None, finished_at=now,
|
||||
failure_code="SUPERSEDED_BY_REQUEUE",
|
||||
failure_detail="New task input superseded the active run",
|
||||
updated_at=now,
|
||||
)
|
||||
cancelled = OperationalTaskRun.model_validate(active_run)
|
||||
await self.repo.update_run(_json(cancelled), expected)
|
||||
await self._fail_run_messages(active_run["run_id"], "Run superseded by requeue", now)
|
||||
await self._run_event(cancelled, now)
|
||||
break
|
||||
active_run = refreshed_run
|
||||
if active_run is not None and active_run["status"] == "queued":
|
||||
# Multiple inputs can arrive before the worker claims the next
|
||||
# generation. They are all visible when that queued run starts,
|
||||
# so coalesce them instead of violating the one-active-run rule.
|
||||
task.update(
|
||||
lifecycle_status="open", status="queued",
|
||||
revision=task["revision"] + 1,
|
||||
execution_generation=active_run["generation"],
|
||||
execution_request_id=str(active_run["execution_request_id"]),
|
||||
agent_session_id=None, agent_lease_expires_at=None,
|
||||
updated_at=now,
|
||||
)
|
||||
return
|
||||
if active_run is not None:
|
||||
expected = active_run["revision"]
|
||||
active_run.update(
|
||||
status="cancelled", revision=expected + 1,
|
||||
lease_expires_at=None, finished_at=now,
|
||||
failure_code="SUPERSEDED_BY_REQUEUE",
|
||||
failure_detail="New task input superseded the active run",
|
||||
updated_at=now,
|
||||
)
|
||||
cancelled = OperationalTaskRun.model_validate(active_run)
|
||||
await self.repo.update_run(_json(cancelled), expected)
|
||||
await self._fail_run_messages(active_run["run_id"], "Run superseded by requeue", now)
|
||||
await self._run_event(cancelled, now)
|
||||
generation = task["execution_generation"] + 1
|
||||
task_id = UUID(task["task_id"])
|
||||
task.update(
|
||||
@@ -2340,9 +2511,8 @@ class OperationalTaskService:
|
||||
agent_session_id=None, agent_lease_expires_at=None, updated_at=now,
|
||||
)
|
||||
run = self._new_run(task_id, generation, now)
|
||||
if self._supports_runs:
|
||||
await self.repo.insert_run(_json(run))
|
||||
await self._run_event(run, now, queued=True)
|
||||
await self.repo.insert_run(_json(run))
|
||||
await self._run_event(run, now, queued=True)
|
||||
|
||||
async def _notify(
|
||||
self, task_id: UUID, kind: str, title: str, description: str, now: datetime,
|
||||
@@ -2378,10 +2548,7 @@ class OperationalTaskService:
|
||||
now: datetime,
|
||||
reason: str,
|
||||
) -> None:
|
||||
resolve = getattr(self.repo, "resolve_notifications", None)
|
||||
if resolve is None:
|
||||
return
|
||||
notifications = await resolve(
|
||||
notifications = await self.repo.resolve_notifications(
|
||||
str(task_id),
|
||||
kinds=kinds,
|
||||
subject_type=subject_type,
|
||||
@@ -2466,6 +2633,7 @@ class OperationalTaskService:
|
||||
due_at: datetime, now: datetime,
|
||||
) -> OperationalTaskSummary:
|
||||
return OperationalTaskSummary(
|
||||
execution_scope=self.execution_scope,
|
||||
project_id=self.context.project_id, task_id=task_id,
|
||||
task_type="scheduled_flow", workflow=schedule.schedule_key,
|
||||
lifecycle_status="open", status="queued", priority=schedule.priority, revision=1,
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Fit causal replay thresholds in bounded numeric arrays, before publishing history."""
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.services.scada_normal_profile import CALIBRATION_VERSION, DIAGNOSTIC_POLICY
|
||||
|
||||
|
||||
def adaptive_score_threshold(scores: list[float]) -> dict:
|
||||
"""Keep the cutoff below the normal lower tail, with a fixed noise margin."""
|
||||
values = np.asarray(scores, dtype=float)
|
||||
if len(values) < 5 or not np.isfinite(values).all():
|
||||
return {"status": "insufficient_data", "reason": "normal_score_history_incomplete"}
|
||||
center = float(np.median(values))
|
||||
sigma = float(1.4826 * np.median(np.abs(values - center)))
|
||||
lower_tail = float(np.quantile(values, 0.05))
|
||||
return {
|
||||
"status": "ready",
|
||||
"threshold": min(-0.04, center - 3 * sigma, lower_tail - 0.02),
|
||||
"normal_score_median": center,
|
||||
"normal_score_min": float(values.min()),
|
||||
"normal_score_max": float(values.max()),
|
||||
"normal_score_robust_sigma": sigma,
|
||||
"normal_score_lower_quantile": lower_tail,
|
||||
"normal_score_count": len(values),
|
||||
"method": "min(-0.04, median-3*robust_sigma, normal_q05-0.02)",
|
||||
}
|
||||
|
||||
|
||||
class MonitoringCalibrationBuilder:
|
||||
"""Retain bounded numeric series, never full raw pages or device batches."""
|
||||
|
||||
def __init__(self, document: dict, config: ScadaReplayCreate):
|
||||
self.settings = ScadaMonitoringSettings.model_validate(document["monitoring_config"])
|
||||
self.config = config
|
||||
self.cutoff = config.start_time
|
||||
self.start = self.cutoff - timedelta(days=15)
|
||||
self.interval = int(document["monitoring_interval_seconds"])
|
||||
self.supported = self.interval % 60 == 0 and 86400 % self.interval == 0
|
||||
self.count = 15 * (86400 // self.interval) if self.supported else 0
|
||||
ids = dict.fromkeys(self.settings.pressure_device_ids + self.settings.outlet_flow_device_ids
|
||||
+ [device for rule in config.rules if rule.kind == "relative_change"
|
||||
for device in rule.device_ids])
|
||||
self.values = {device: np.full(self.count, np.nan) for device in ids}
|
||||
|
||||
def add(self, rows: list[dict]) -> None:
|
||||
for row in rows:
|
||||
device = row["device_id"]
|
||||
if device not in self.values:
|
||||
continue
|
||||
at = datetime.fromisoformat(row["time"])
|
||||
if at >= self.cutoff:
|
||||
raise ValueError("动态阈值拟合不能读取回放开始时刻及之后的观测")
|
||||
offset = (at - self.start).total_seconds()
|
||||
if (not self.supported or offset < 0 or offset % self.interval
|
||||
or row.get("quality") not in {"valid", "corrected"}
|
||||
or row.get("cleaned_value") is None):
|
||||
continue
|
||||
index = int(offset // self.interval)
|
||||
if np.isfinite(self.values[device][index]):
|
||||
raise ValueError("动态阈值拟合存在重复的设备时间点")
|
||||
self.values[device][index] = float(row["cleaned_value"])
|
||||
|
||||
def fit(self) -> dict:
|
||||
base = {
|
||||
"version": CALIBRATION_VERSION,
|
||||
"diagnostic_policy": DIAGNOSTIC_POLICY,
|
||||
"knowledge_cutoff": self.cutoff.isoformat(),
|
||||
"history_start": self.start.isoformat(),
|
||||
"required_samples_per_device": self.count,
|
||||
"pressure": {}, "outlet_flow": {}, "diagnostic": {},
|
||||
}
|
||||
if not self.supported:
|
||||
return {**base, "status": "insufficient_data", "reason": "unsupported_sampling_interval"}
|
||||
available = {device: int(np.isfinite(values).sum()) for device, values in self.values.items()}
|
||||
# A regular diurnal rise/reversal can be a huge percentage of a small
|
||||
# preceding flow. Judge its absolute value against causal normal readings
|
||||
# at the same sampling phase before treating that percentage as abnormal.
|
||||
points_per_day = 86400 // self.interval
|
||||
for device, values in self.values.items():
|
||||
if available[device] != self.count:
|
||||
continue
|
||||
days = values.reshape(15, points_per_day)
|
||||
center = np.median(days, axis=0)
|
||||
sigma = 1.4826 * np.median(np.abs(days - center), axis=0)
|
||||
margin = np.maximum(3 * sigma, np.maximum(np.abs(center) * 1e-6, 1e-9))
|
||||
base["diagnostic"][device] = {
|
||||
"phase_origin": self.start.isoformat(), "interval_seconds": self.interval,
|
||||
"lower": (days.min(axis=0) - margin).tolist(),
|
||||
"upper": (days.max(axis=0) + margin).tolist(),
|
||||
"method": "normal_same_phase_range_plus_3_robust_sigma", "normal_day_count": 15,
|
||||
}
|
||||
for kind, ids, direction, floor in (
|
||||
("pressure", self.settings.pressure_device_ids, -1, 3.0),
|
||||
("outlet_flow", self.settings.outlet_flow_device_ids, 1, 5.0),
|
||||
):
|
||||
for device in ids:
|
||||
if available[device] != self.count:
|
||||
continue
|
||||
rule = next((rule for rule in self.config.rules
|
||||
if rule.kind == "relative_change" and device in rule.device_ids), None)
|
||||
if rule is None:
|
||||
continue
|
||||
series = pd.Series(self.values[device])
|
||||
reference = series.rolling(rule.window_samples, min_periods=rule.minimum_samples).median().shift(1)
|
||||
changes = (direction * 100 * (series - reference) / reference.abs()).to_numpy()
|
||||
changes = changes[np.isfinite(changes)]
|
||||
if not len(changes):
|
||||
continue
|
||||
center = float(np.median(changes))
|
||||
sigma = float(1.4826 * np.median(np.abs(changes - center)))
|
||||
base[kind][device] = {
|
||||
"threshold_percent": max(floor, center + 3 * sigma),
|
||||
"normal_change_median": center,
|
||||
"normal_change_robust_sigma": sigma,
|
||||
"normal_change_count": int(len(changes)),
|
||||
"method": "normal_recent_window_change_median_plus_3_robust_sigma",
|
||||
"normal_profile": base["diagnostic"].get(device),
|
||||
}
|
||||
pressure_ids = self.settings.pressure_device_ids
|
||||
incomplete = {device: available[device] for device in pressure_ids if available[device] != self.count}
|
||||
if len(pressure_ids) < 5 or incomplete:
|
||||
return {**base, "status": "insufficient_data", "reason": "pressure_history_incomplete",
|
||||
"available_samples": available}
|
||||
from app.services import scada_replay_monitoring as monitoring
|
||||
|
||||
index = pd.date_range(start=self.start, periods=self.count, freq=pd.Timedelta(seconds=self.interval))
|
||||
frame = pd.DataFrame({device: self.values[device] for device in pressure_ids}, index=index)
|
||||
result = monitoring.run_burst_detection(
|
||||
network="scada-replay-baseline", username="scada-replay",
|
||||
observed_pressure_data=frame, points_per_day=86400 // self.interval,
|
||||
mu=monitoring.TARGET_MU,
|
||||
iforest_params={"n_estimators": monitoring.TARGET_N_ESTIMATORS,
|
||||
"random_state": monitoring.TARGET_RANDOM_STATE, "contamination": "auto"},
|
||||
sensor_nodes=pressure_ids,
|
||||
)
|
||||
score = adaptive_score_threshold([float(row["Score"]) for row in result["rows"]])
|
||||
return {**base, "status": score["status"], "burst_detection": score}
|
||||
@@ -0,0 +1,98 @@
|
||||
"""Persist one accident across monitoring periods without moving its evidence cutoff."""
|
||||
from datetime import datetime
|
||||
from math import isfinite
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
|
||||
def _anchors(monitoring: dict, indicator: str = "pressure") -> dict:
|
||||
return {
|
||||
item["device_id"]: {"baseline": item["baseline"], "threshold_percent": item["threshold_percent"]}
|
||||
for item in monitoring.get(indicator, {}).get("devices", [])
|
||||
if item.get("hit") and item.get("baseline", 0) > 0
|
||||
and 0 < item.get("threshold_percent", 0) < 100
|
||||
}
|
||||
|
||||
|
||||
def update_incident(document: dict, monitoring: dict, sequence: int) -> tuple[dict | None, str | None]:
|
||||
incident = document.get("monitoring_incident")
|
||||
# Adopt the last task from an older replay document on its first new step.
|
||||
previous = document.get("last_monitoring") or {}
|
||||
if incident:
|
||||
incident.setdefault("burst_diagnosed", previous.get("status") == "suspected_burst")
|
||||
if not incident and previous.get("task_id"):
|
||||
incident = {
|
||||
"incident_id": str(uuid5(NAMESPACE_URL, f"scada_replay:{document['replay_id']}:monitor:"
|
||||
f"{previous['period_end']}:{previous['rule_version']}")),
|
||||
"started_at": previous["period_end"], "last_observed_at": previous["period_end"],
|
||||
"status": "active", "task_id": previous["task_id"], "recovery_count": 0,
|
||||
"pressure_anchors": _anchors(previous), "alert": None,
|
||||
"flow_anchors": _anchors(previous, "outlet_flow"),
|
||||
"burst_diagnosed": previous.get("status") == "suspected_burst",
|
||||
}
|
||||
document["monitoring_incident"] = incident
|
||||
if incident and incident.get("last_monitoring_id") == monitoring.get("monitoring_id") and monitoring.get("monitoring_id"):
|
||||
return incident, incident["last_action"]
|
||||
anomaly = (monitoring["status"] in {"attention", "suspected_burst"}
|
||||
or bool(monitoring.get("anomaly_signals"))
|
||||
or any(monitoring.get(key, {}).get("hit") is True
|
||||
for key in ("pressure", "outlet_flow", "burst_detection")))
|
||||
if anomaly and (not incident or incident["status"] == "resolved"):
|
||||
external = (f"scada_replay:{document['replay_id']}:monitor:"
|
||||
f"{monitoring['period_end']}:{monitoring['rule_version']}")
|
||||
incident = {
|
||||
"incident_id": str(uuid5(NAMESPACE_URL, external)), "external_id": external,
|
||||
"started_at": monitoring["period_end"], "status": "active", "task_id": None,
|
||||
"recovery_count": 0, "pressure_anchors": _anchors(monitoring), "alert": None,
|
||||
"flow_anchors": _anchors(monitoring, "outlet_flow"),
|
||||
"batch_sequence": sequence,
|
||||
}
|
||||
document["monitoring_incident"] = incident
|
||||
action = "raised"
|
||||
elif incident and incident["status"] == "active":
|
||||
# A later diagnostic stage may add pressure/flow evidence to the same task.
|
||||
# Freeze each newly abnormal device once; retain earlier references.
|
||||
for indicator, field in (("pressure", "pressure_anchors"), ("outlet_flow", "flow_anchors")):
|
||||
for device, anchor in _anchors(monitoring, indicator).items():
|
||||
incident.setdefault(field, {}).setdefault(device, anchor)
|
||||
current = {item["device_id"]: item for item in monitoring.get("pressure", {}).get("devices", [])}
|
||||
anchors = incident["pressure_anchors"]
|
||||
flow = {item["device_id"]: item for item in monitoring.get("outlet_flow", {}).get("devices", [])}
|
||||
# Rolling relative changes can become normal while absolute pressure stays low.
|
||||
# Confirm recovery against the frozen pre-accident references as well.
|
||||
recovered = monitoring["status"] == "normal" and not anomaly and all(
|
||||
monitoring.get(key, {}).get("status") == "ready"
|
||||
and monitoring[key].get("hit") is False for key in ("pressure", "burst_detection")
|
||||
) and all(
|
||||
device in current and isfinite(current[device]["value"])
|
||||
and current[device].get("observed_at") is not None
|
||||
and datetime.fromisoformat(current[device]["observed_at"]) > datetime.fromisoformat(incident["last_observed_at"])
|
||||
and current[device]["value"] >= anchor["baseline"] * (1 - anchor["threshold_percent"] / 100)
|
||||
for device, anchor in anchors.items()
|
||||
) and all(
|
||||
device in flow and isfinite(flow[device]["value"])
|
||||
and flow[device].get("observed_at") is not None
|
||||
and datetime.fromisoformat(flow[device]["observed_at"]) > datetime.fromisoformat(incident["last_observed_at"])
|
||||
and flow[device]["value"] <= anchor["baseline"] * (1 + anchor["threshold_percent"] / 100)
|
||||
for device, anchor in incident.get("flow_anchors", {}).items()
|
||||
)
|
||||
contiguous = bool(monitoring.get("period_start")) and (
|
||||
datetime.fromisoformat(monitoring["period_start"]) == datetime.fromisoformat(incident["last_observed_at"])
|
||||
)
|
||||
incident["recovery_count"] = (incident["recovery_count"] + 1 if contiguous else 1) if recovered else 0
|
||||
action = "resolved" if incident["recovery_count"] >= 2 else "ongoing"
|
||||
if action == "resolved":
|
||||
incident["status"] = "resolved"
|
||||
else:
|
||||
return None, None
|
||||
incident.update(last_observed_at=monitoring["period_end"],
|
||||
last_monitoring_id=monitoring.get("monitoring_id"), last_action=action)
|
||||
return incident, action
|
||||
|
||||
|
||||
def attach_incident(monitoring: dict, incident: dict, action: str) -> None:
|
||||
monitoring["task_id"] = incident["task_id"]
|
||||
monitoring["incident"] = {
|
||||
key: incident[key] for key in (
|
||||
"incident_id", "status", "started_at", "last_observed_at", "recovery_count",
|
||||
)
|
||||
} | {"action": action}
|
||||
@@ -0,0 +1,48 @@
|
||||
"""Clock-phase normal envelopes frozen before replay; never inspect future readings."""
|
||||
from datetime import datetime
|
||||
from math import isfinite
|
||||
|
||||
CALIBRATION_VERSION = "scada-monitor-calibration@2"
|
||||
DIAGNOSTIC_POLICY = "normal-phase-envelope@1"
|
||||
|
||||
|
||||
def weighted_diagnosis(document: dict) -> bool:
|
||||
return (document.get("monitoring_config") or {}).get("decision_policy") == "weighted"
|
||||
|
||||
|
||||
def current_calibration(document: dict, start: datetime) -> dict | None:
|
||||
calibration = document.get("monitoring_calibration") or {}
|
||||
if (calibration.get("version") == CALIBRATION_VERSION
|
||||
and calibration.get("diagnostic_policy") == DIAGNOSTIC_POLICY
|
||||
and calibration.get("knowledge_cutoff") == start.isoformat()):
|
||||
return calibration
|
||||
return None
|
||||
|
||||
|
||||
def diagnostic_signal(document: dict, event: dict) -> bool:
|
||||
"""Never revive a persisted fixed-percentage event as new diagnosis evidence."""
|
||||
if event.get("category") == "data_quality":
|
||||
return False
|
||||
if not weighted_diagnosis(document):
|
||||
return True
|
||||
rules = (document.get("config") or {}).get("rules", [])
|
||||
rule = next((item for item in rules if item["rule_id"] == event["rule_id"]), None)
|
||||
return bool(rule and (rule["kind"] == "bounds" or event.get("diagnostic_policy") == DIAGNOSTIC_POLICY))
|
||||
|
||||
|
||||
def normal_profile_bounds(profile: dict | None, at: datetime) -> tuple[float, float] | None:
|
||||
if not profile:
|
||||
return None
|
||||
interval = profile.get("interval_seconds", 0)
|
||||
if not profile.get("phase_origin"):
|
||||
return None
|
||||
origin = datetime.fromisoformat(profile["phase_origin"])
|
||||
lower, upper = profile.get("lower", []), profile.get("upper", [])
|
||||
offset = (at - origin).total_seconds()
|
||||
if interval <= 0 or offset < 0 or offset % interval or not lower or len(lower) != len(upper):
|
||||
return None
|
||||
slot = int(offset // interval) % len(lower)
|
||||
low, high = lower[slot], upper[slot]
|
||||
if low is None or high is None or not isfinite(low) or not isfinite(high):
|
||||
return None
|
||||
return float(low), float(high)
|
||||
@@ -0,0 +1,268 @@
|
||||
"""Verify monitoring charts against observations and require run-local device coverage."""
|
||||
from __future__ import annotations
|
||||
|
||||
from copy import deepcopy
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from math import isclose
|
||||
import re
|
||||
|
||||
from app.domain.domain_errors import require
|
||||
|
||||
|
||||
DISPLAY_TIMEZONE = timezone(timedelta(hours=8))
|
||||
METRICS = {"pressure": "pressure", "pipe_flow": "flow", "flow": "flow"}
|
||||
|
||||
|
||||
def applies(task: dict) -> bool:
|
||||
if task.get("workflow") != "scada_anomaly" and not task.get("source", {}).get("replay_context"):
|
||||
return False
|
||||
# Recovery has its own Server-verified post-repair evidence and finalization gate.
|
||||
return (task.get("handling_context") or {}).get("phase") not in {"verification_ready", "completed"}
|
||||
|
||||
|
||||
def cutoff(task: dict) -> datetime:
|
||||
replay = task.get("source", {}).get("replay_context") or {}
|
||||
return datetime.fromisoformat(replay.get("knowledge_cutoff") or task["trigger"]["occurred_at"])
|
||||
|
||||
|
||||
def _unit(value: str) -> str:
|
||||
return value.replace("³", "3").replace(" ", "").lower()
|
||||
|
||||
|
||||
class ScadaObservations:
|
||||
def __init__(self, task: dict, evidence: list[dict], refs: list[str] | None = None):
|
||||
self.devices: dict[str, dict] = {}
|
||||
self.values: dict[tuple[str, datetime], dict] = {}
|
||||
self.windows: list[tuple[set[str], set[str], datetime, datetime]] = []
|
||||
self.frozen_limits: list[tuple[set[str], datetime]] = []
|
||||
target = cutoff(task)
|
||||
replay = task.get("source", {}).get("replay_context") or {}
|
||||
for item in evidence:
|
||||
snapshot = item.get("data_snapshot") or {}
|
||||
result = snapshot.get("data") or {}
|
||||
if snapshot.get("source") != "query_scada_readings" or not isinstance(result, dict):
|
||||
continue
|
||||
data = result.get("data")
|
||||
coverage = data.get("coverage", {}) if isinstance(data, dict) else {}
|
||||
if replay and coverage and (coverage.get("replay_id") != replay["replay_id"]
|
||||
or coverage.get("batch_sequence") != replay["batch_sequence"]):
|
||||
continue
|
||||
window = result.get("window") or {}
|
||||
if not window.get("start") or not window.get("end"):
|
||||
continue
|
||||
start, end = (datetime.fromisoformat(window[key]) for key in ("start", "end"))
|
||||
if end > target:
|
||||
continue
|
||||
devices = data.get("devices", []) if isinstance(data, dict) else []
|
||||
for device in devices:
|
||||
self.devices[str(device["device_id"])] = device
|
||||
if refs is not None and item["evidence_id"] not in refs:
|
||||
continue
|
||||
rows = data.get("readings", []) if isinstance(data, dict) else data
|
||||
if not isinstance(rows, list):
|
||||
continue
|
||||
ids = {str(row["device_id"]) for row in rows if "device_id" in row}
|
||||
outside = set(coverage.get("outside_frozen_scope_device_ids", []))
|
||||
if coverage.get("query_complete") is True:
|
||||
ids.update(coverage.get("queried_device_ids", []))
|
||||
ids.update(str(device["device_id"]) for device in devices)
|
||||
page = result.get("page") or {}
|
||||
complete = not (snapshot.get("truncated") or result.get("truncated")
|
||||
or page.get("has_more") or page.get("offset", 0))
|
||||
if complete:
|
||||
self.windows.append((ids, outside, start, end))
|
||||
if replay and coverage.get("query_complete") and coverage.get("frozen_start"):
|
||||
self.frozen_limits.append((ids, datetime.fromisoformat(coverage["frozen_start"])))
|
||||
for row in rows:
|
||||
if "time" in row and "device_id" in row:
|
||||
at = datetime.fromisoformat(row["time"])
|
||||
if start <= at <= end:
|
||||
observed = self.values.setdefault((str(row["device_id"]), at), {})
|
||||
observed.update({field: row[field] for field in ("monitored_value", "cleaned_value") if field in row})
|
||||
|
||||
def absent_reason(self, device: str, at: datetime) -> str | None:
|
||||
for ids, outside, start, end in self.windows:
|
||||
if device in ids and start <= at <= end:
|
||||
return "outside_frozen_scope" if device in outside else "no_readings"
|
||||
for ids, frozen_start in self.frozen_limits:
|
||||
if device in ids and at < frozen_start:
|
||||
return "outside_frozen_window"
|
||||
return None
|
||||
|
||||
def window_is_empty(self, device: str, start: datetime, end: datetime) -> bool:
|
||||
if any(key == device and start <= at <= end for key, at in self.values):
|
||||
return False
|
||||
intervals = [(left, right) for ids, _, left, right in self.windows if device in ids]
|
||||
intervals.extend((start, frozen_start) for ids, frozen_start in self.frozen_limits if device in ids)
|
||||
through = start
|
||||
for left, right in sorted(intervals):
|
||||
if left > through:
|
||||
break
|
||||
through = max(through, right)
|
||||
if through >= end:
|
||||
return True
|
||||
return False
|
||||
|
||||
def period(self, device: str, start: datetime, end: datetime) -> timedelta:
|
||||
frequency = str(self.devices[device].get("transmission_frequency") or "")
|
||||
match = re.fullmatch(r"(\d+):([0-5]\d):([0-5]\d)", frequency)
|
||||
if match:
|
||||
hours, minutes, seconds = map(int, match.groups())
|
||||
period = timedelta(hours=hours, minutes=minutes, seconds=seconds)
|
||||
else:
|
||||
minutes = re.fullmatch(r"(\d+)\s*(?:min|分钟)", frequency)
|
||||
times = sorted(at for key, at in self.values if key == device and start <= at <= end)
|
||||
period = (timedelta(minutes=int(minutes[1])) if minutes else
|
||||
min((right - left for left, right in zip(times, times[1:])), default=timedelta(0)))
|
||||
require(period > timedelta(0), "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 缺少可核实的采样周期,请读取完整设备属性", refs=[device])
|
||||
return period
|
||||
|
||||
def slots(self, device: str, start: datetime, end: datetime) -> list[datetime]:
|
||||
"""Keep the business bounds exact, aligning slots to the observed sampling phase."""
|
||||
period = self.period(device, start, end)
|
||||
observed = sorted(at for key, at in self.values if key == device and start <= at <= end)
|
||||
historical = sorted(at for key, at in self.values if key == device and at < start)
|
||||
# Referenced historical observations preserve phase even if current data is absent.
|
||||
anchor = (observed[0] if observed else historical[-1] if historical else
|
||||
start.astimezone(DISPLAY_TIMEZONE).replace(hour=0, minute=0, second=0, microsecond=0))
|
||||
first = start + (anchor - start) % period
|
||||
return [first + period * index for index in range(int((end - first) // period) + 1)]
|
||||
|
||||
|
||||
def add_targets(document: dict | None, devices: list[str], observations: ScadaObservations) -> dict:
|
||||
result = deepcopy(document) if document else {"version": 1, "targets": [], "charts": {}}
|
||||
targets = set(result["targets"])
|
||||
for device in devices:
|
||||
require(device in observations.devices, "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 未出现在本轮截止时间内的完整设备证据中", refs=[device])
|
||||
if observations.devices[device].get("device_type") in METRICS:
|
||||
targets.add(device)
|
||||
result["targets"] = sorted(targets)
|
||||
return result
|
||||
|
||||
|
||||
def verify_chart(task: dict, chart: dict, evidence: list[dict], refs: list[str]) -> dict:
|
||||
"""Return verified coverage; mutate only the outgoing chart's Server-derived gaps."""
|
||||
context = chart["scada"]
|
||||
end = cutoff(task)
|
||||
start = end - timedelta(hours=3)
|
||||
require(datetime.fromisoformat(context["window_start"]) == start
|
||||
and datetime.fromisoformat(context["window_end"]) == end,
|
||||
"SCADA_CHART_WINDOW_INVALID", "SCADA图表必须使用本轮截止时刻之前的完整三小时窗口")
|
||||
observations = ScadaObservations(task, evidence, refs)
|
||||
series = chart["series"]
|
||||
ids = {item.get("device_id") for item in series}
|
||||
require(None not in ids and all(device in observations.devices for device in ids),
|
||||
"SCADA_CHART_EVIDENCE_REQUIRED", "每条SCADA序列须绑定已核实的 device_id", refs=sorted(ids - {None}))
|
||||
comparison = context["role"] == "comparison"
|
||||
require(chart["chart_type"] == ("time_comparison" if comparison else "line"),
|
||||
"SCADA_CHART_INVALID", "SCADA总览须使用line,同期图须使用time_comparison")
|
||||
bindings = [(item["device_id"], item.get("day_offset")) for item in series]
|
||||
require(len(bindings) == len(set(bindings)), "SCADA_CHART_INVALID", "SCADA序列的设备与日期绑定不能重复")
|
||||
require((len(ids) == 1 and {day for _, day in bindings} == set(range(8))) if comparison
|
||||
else all(day == 0 for _, day in bindings),
|
||||
"SCADA_CHART_INVALID", "总览每个设备绑定day_offset=0;同期图每个设备须有当天及此前七天共8组序列")
|
||||
field = context.get("value_field", "monitored_value")
|
||||
gaps = []
|
||||
coverage = {}
|
||||
for item in series:
|
||||
device, day = item["device_id"], item["day_offset"]
|
||||
metadata = observations.devices[device]
|
||||
require(METRICS.get(metadata.get("device_type")) == context["metric"]
|
||||
and _unit(metadata.get("measurement_unit") or "") == _unit(context["unit"]),
|
||||
"SCADA_CHART_INVALID", f"设备 {device} 的指标或单位与完整设备证据不一致", refs=[device])
|
||||
period = observations.period(device, start, end)
|
||||
full_slots = observations.slots(device, start, end)
|
||||
no_schedule = not full_slots
|
||||
if no_schedule:
|
||||
require(observations.window_is_empty(device, start - timedelta(days=day), end - timedelta(days=day)),
|
||||
"SCADA_CHART_EVIDENCE_REQUIRED", f"设备 {device} 无计划采样槽,须完整读取对应三小时窗口并证明无读数", refs=[device])
|
||||
# This is an empty display entry, never an observation or a missed scheduled sample.
|
||||
full_slots = [end]
|
||||
labels = [at.astimezone(DISPLAY_TIMEZONE).strftime("%H:%M:%S" if period.total_seconds() % 60 else "%H:%M")
|
||||
for at in full_slots]
|
||||
first = labels.index(chart["x_data"][0]) if chart["x_data"][0] in labels else -1
|
||||
count = len(chart["x_data"])
|
||||
require(first >= 0 and chart["x_data"] == labels[first:first + count]
|
||||
and len(item.get("sample_times") or []) == count,
|
||||
"SCADA_CHART_INVALID", f"设备 {device} 须保留连续真实采样槽和sample_times;超限按时间分图", refs=[device])
|
||||
slots = full_slots[first:first + count]
|
||||
coverage[device] = {"covered": [at.isoformat() for at in slots]}
|
||||
require((item.get("type") or "line") == ("scatter" if day else "line"),
|
||||
"SCADA_CHART_INVALID", "当前序列须为line,历史序列须为scatter")
|
||||
missing = 0
|
||||
reason = "no_readings"
|
||||
expected = {at - timedelta(days=day) for at in (full_slots[-3:] if day else full_slots)}
|
||||
# Off-grid observations cannot be silently discarded by a regular-looking chart.
|
||||
observed = {at for key, at in observations.values if key == device
|
||||
and min(expected) <= at <= max(expected)}
|
||||
require(observed <= expected, "SCADA_CHART_INVALID",
|
||||
f"设备 {device} 含非规则采样时刻,须明确处理,不能静默丢点", refs=[device])
|
||||
for index, slot in enumerate(slots):
|
||||
at = slot - timedelta(days=day)
|
||||
value, sample_time = item["data"][index], item["sample_times"][index]
|
||||
if at not in expected:
|
||||
require(value is None and sample_time is None, "SCADA_CHART_INVALID",
|
||||
"历史散点只保留每日末三个对应时刻")
|
||||
continue
|
||||
record = observations.values.get((device, at))
|
||||
require(record is None or field in record, "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 的证据未标明 {field},请重新读取明确的监测字段", refs=[device])
|
||||
actual = record.get(field) if record is not None else None
|
||||
if actual is None:
|
||||
absent = observations.absent_reason(device, at)
|
||||
require(record is not None or absent is not None, "SCADA_CHART_EVIDENCE_REQUIRED",
|
||||
f"设备 {device} 的 {at.isoformat()} 未被完整读取,不能将未查询当作缺测", refs=[device])
|
||||
require(value is None and sample_time is None, "SCADA_CHART_DATA_MISMATCH",
|
||||
f"设备 {device} 缺测槽必须保留null,不能补零或推算", refs=[device])
|
||||
missing += 1
|
||||
if absent in {"outside_frozen_scope", "outside_frozen_window"}:
|
||||
reason = absent
|
||||
elif no_schedule:
|
||||
reason = "no_scheduled_samples"
|
||||
else:
|
||||
require(value is not None and sample_time is not None
|
||||
and datetime.fromisoformat(sample_time) == at
|
||||
and (isclose(value, actual, rel_tol=1e-9, abs_tol=1e-9)
|
||||
or any(isclose(value, round(actual, digits), rel_tol=0, abs_tol=1e-9)
|
||||
for digits in range(1, 7))),
|
||||
"SCADA_CHART_DATA_MISMATCH",
|
||||
f"设备 {device} 的 {at.isoformat()} 图表值或采样时间与证据不符", refs=[device])
|
||||
if missing:
|
||||
gaps.append({"device_id": device, "day_offset": day, "missing_count": missing, "reason": reason})
|
||||
context["gaps"] = gaps
|
||||
return {"role": context["role"], "metric": context["metric"], "device_ids": sorted(ids),
|
||||
"window_start": context["window_start"], "window_end": context["window_end"], "gaps": gaps,
|
||||
"coverage": coverage}
|
||||
|
||||
|
||||
def require_complete(task: dict, document: dict | None, observations: ScadaObservations) -> None:
|
||||
targets = set((document or {}).get("targets", []))
|
||||
supported = {key for key, device in observations.devices.items() if device.get("device_type") in METRICS}
|
||||
if not supported and not targets:
|
||||
return
|
||||
require(bool(targets), "SCADA_PRESENTATION_INCOMPLETE",
|
||||
"请声明本轮SCADA目标设备并发布当前总览与七天同期图;旧格式图表不计入设备覆盖", status=409)
|
||||
end = cutoff(task)
|
||||
start = end - timedelta(hours=3)
|
||||
covered = {"current": {}, "comparison": {}}
|
||||
for chart in document["charts"].values():
|
||||
if (datetime.fromisoformat(chart["window_start"]) == start
|
||||
and datetime.fromisoformat(chart["window_end"]) == end):
|
||||
for device, slots in chart.get("coverage", {}).items():
|
||||
if device not in chart["device_ids"]:
|
||||
continue
|
||||
covered[chart["role"]].setdefault(device, set()).update(
|
||||
datetime.fromisoformat(at) for at in slots["covered"])
|
||||
missing = {}
|
||||
for role in covered:
|
||||
missing[role] = sorted(device for device in targets
|
||||
if not covered[role].get(device) or not set(observations.slots(device, start, end) or [end])
|
||||
<= covered[role][device])
|
||||
missing_current, missing_comparison = missing["current"], missing["comparison"]
|
||||
require(not missing_current and not missing_comparison, "SCADA_PRESENTATION_INCOMPLETE",
|
||||
f"本轮SCADA目标共{len(targets)}个;缺少当前总览设备:{', '.join(missing_current) or '无'};"
|
||||
f"缺少七天同期对比设备:{', '.join(missing_comparison) or '无'}。请补图后重试;仅作参考不能豁免展示。",
|
||||
refs=sorted(set(missing_current + missing_comparison)), status=409,
|
||||
recovery="retry_same_request")
|
||||
+250
-43
@@ -2,9 +2,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import json
|
||||
from uuid import NAMESPACE_URL, uuid4, uuid5
|
||||
|
||||
from fastapi import HTTPException
|
||||
from app.core.config import settings
|
||||
from app.auth.permissions import permissions_for_context
|
||||
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
@@ -14,6 +16,18 @@ from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
from app.services.content_digest import digest
|
||||
from app.services.scada_replay_processing import prepare_cleaning, detect_frame
|
||||
from app.services.scada_replay_monitoring import evaluate_monitoring_period
|
||||
from app.services.scada_normal_profile import current_calibration, diagnostic_signal, weighted_diagnosis
|
||||
|
||||
|
||||
# 500 devices at 15-minute sampling need 720,000 rows for fifteen days.
|
||||
# Leave headroom without allowing dense sources to make preparation unbounded.
|
||||
BASELINE_PAGE_SIZE = 2000
|
||||
BASELINE_MAX_ROWS = 1_000_000
|
||||
BASELINE_MAX_BYTES = 512 * 1024 * 1024
|
||||
|
||||
|
||||
class BaselineBudgetExceeded(RuntimeError):
|
||||
"""The unpublished baseline exceeded its fixed preparation budget."""
|
||||
|
||||
|
||||
def parse_time(value: str) -> datetime:
|
||||
@@ -36,6 +50,10 @@ def snapshot(document: dict) -> ScadaReplaySnapshot:
|
||||
step_pending=document.get("step_pending", False),
|
||||
monitoring_count=document.get("monitoring_count", 0),
|
||||
last_monitoring=document.get("last_monitoring"),
|
||||
# Old persisted documents without a policy still use three-signal detection.
|
||||
monitoring_config=({"decision_policy": "all_signals", **document["monitoring_config"]}
|
||||
if document.get("monitoring_config") else None),
|
||||
monitoring_calibration=document.get("monitoring_calibration"),
|
||||
)
|
||||
|
||||
|
||||
@@ -105,7 +123,7 @@ class ScadaReplayService:
|
||||
await self.ensure_storage()
|
||||
async with self.repo.control():
|
||||
document = await self.repo.get(replay_id)
|
||||
key, body = str(command.request_id), command.model_dump(mode="json")
|
||||
key, body = str(command.request_id), command.model_dump(mode="json", exclude_none=True)
|
||||
receipt = document.setdefault("receipts", {}).get(key)
|
||||
if receipt:
|
||||
if receipt != body:
|
||||
@@ -114,9 +132,13 @@ class ScadaReplayService:
|
||||
if command.expected_revision != document["revision"]:
|
||||
raise HTTPException(409, "回放控制状态已更新,请刷新后重试")
|
||||
action, status = command.action, document["status"]
|
||||
if document.get("step_pending") and action == "step":
|
||||
if (action in {"step", "monitor", "retry"} and document.get("warmed")
|
||||
and weighted_diagnosis(document)
|
||||
and current_calibration(document, parse_time(document["start_time"])) is None):
|
||||
raise HTTPException(409, "旧回放缺少新版历史诊断校准,请停止后新建演示")
|
||||
if document.get("step_pending") and action in {"step", "monitor"}:
|
||||
raise HTTPException(409, "当前时间步正在处理,请完成后继续")
|
||||
if action in {"step", "retry"} and await self.repo.active_other(replay_id):
|
||||
if action in {"step", "monitor", "retry"} and await self.repo.active_other(replay_id):
|
||||
raise HTTPException(409, "当前项目已有另一条活动回放")
|
||||
if action == "step" and status in {"ready", "paused"}:
|
||||
interval = document.get("monitoring_interval_seconds")
|
||||
@@ -126,8 +148,31 @@ class ScadaReplayService:
|
||||
elapsed = (parse_time(document["virtual_now"]) - start).total_seconds()
|
||||
target = start + timedelta(seconds=(int(elapsed // interval) + 1) * interval)
|
||||
document["step_start"] = document["virtual_now"]
|
||||
document["step_anomaly_signals"] = []
|
||||
document["step_target"] = min(parse_time(document["end_time"]), target).isoformat()
|
||||
document["status"], document["step_pending"] = "paused", True
|
||||
elif action == "monitor" and status in {"ready", "paused", "completed"}:
|
||||
target = command.at_time
|
||||
if target is None:
|
||||
raise HTTPException(422, "选定时刻监测需要提供 at_time")
|
||||
start = parse_time(document["start_time"])
|
||||
current = parse_time(document["virtual_now"])
|
||||
if not start <= target <= parse_time(document["end_time"]):
|
||||
raise HTTPException(422, "监测时刻必须在本轮演示区间内")
|
||||
if target < current:
|
||||
raise HTTPException(409, "监测历史时刻需要新建一轮回放,避免使用后续状态")
|
||||
interval = document.get("monitoring_interval_seconds")
|
||||
if interval is None:
|
||||
raise HTTPException(409, "本轮未固定监测周期,请停止后新建演示")
|
||||
previous = document.get("last_monitoring") or {}
|
||||
# Reusing an immutable result must not count as another normal
|
||||
# period, resolve an incident, or create another investigation.
|
||||
if not previous.get("period_end") or parse_time(previous["period_end"]) != target:
|
||||
document["step_start"] = (current if target > current
|
||||
else max(start, target - timedelta(seconds=interval))).isoformat()
|
||||
document["step_target"] = target.isoformat()
|
||||
document["step_anomaly_signals"] = []
|
||||
document["status"], document["step_pending"] = "paused", True
|
||||
elif action == "stop" and status not in {"completed", "stopped"}:
|
||||
document["status"], document["step_pending"] = "stopped", False
|
||||
document.pop("step_target", None)
|
||||
@@ -161,11 +206,16 @@ class ScadaReplayService:
|
||||
# Failure never discards the prepared batch; retry resumes it.
|
||||
document = await self.repo.get(replay_id)
|
||||
document["status"], document["step_pending"] = "failed", False
|
||||
document["error"] = (
|
||||
"SOURCE_CHANGED: 原始观测已改变,需要新建回放"
|
||||
if "SOURCE_CHANGED" in str(exc)
|
||||
else "SCADA 处理失败,已保留批次与进度;请检查服务日志后重试"
|
||||
)
|
||||
if isinstance(exc, HTTPException):
|
||||
document["error"] = str(exc.detail)
|
||||
elif isinstance(exc, BaselineBudgetExceeded):
|
||||
document["error"] = str(exc)
|
||||
else:
|
||||
document["error"] = (
|
||||
"SOURCE_CHANGED: 原始观测已改变,需要新建回放"
|
||||
if "SOURCE_CHANGED" in str(exc)
|
||||
else "SCADA 处理失败,已保留批次与进度;请检查服务日志后重试"
|
||||
)
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save(document)
|
||||
raise
|
||||
@@ -174,23 +224,9 @@ class ScadaReplayService:
|
||||
config = ScadaReplayCreate.model_validate(document["config"])
|
||||
replay_id = document["replay_id"]
|
||||
if not document.get("warmed"):
|
||||
import math
|
||||
window = max(rule.window_samples for rule in config.rules)
|
||||
async with self.timescale.transaction():
|
||||
warmup = await ScadaRepository.get_warmup_values(
|
||||
self.timescale, config.device_ids, config.start_time, window,
|
||||
)
|
||||
for row in warmup:
|
||||
value = row["monitored_value"]
|
||||
bounds = config.quality_ranges.get(row["device_id"])
|
||||
if value is not None and math.isfinite(value) and not (bounds and (
|
||||
bounds.lower is not None and value < bounds.lower
|
||||
or bounds.upper is not None and value > bounds.upper
|
||||
)):
|
||||
document["histories"].setdefault(row["device_id"], []).append(value)
|
||||
document["warmed"] = True
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save(document)
|
||||
await self._freeze_baseline(document, config)
|
||||
if weighted_diagnosis(document) and current_calibration(document, config.start_time) is None:
|
||||
raise HTTPException(409, "旧回放缺少新版历史诊断校准,请停止后新建演示")
|
||||
batch = await self.repo.pending_batch(replay_id)
|
||||
if batch is None:
|
||||
step = document.get("step_pending", False)
|
||||
@@ -252,6 +288,15 @@ class ScadaReplayService:
|
||||
target = result.get("step_target")
|
||||
was_step = bool(target)
|
||||
result["step_pending"] = bool(target and parse_time(batch["at"]) < parse_time(target))
|
||||
if was_step:
|
||||
signals = {item["event_id"]: item for item in result.get("step_anomaly_signals", [])
|
||||
if diagnostic_signal(result, item)}
|
||||
for event in changes:
|
||||
if event["change"] == "raised" and diagnostic_signal(result, event):
|
||||
signals[event["event_id"]] = {key: event[key] for key in (
|
||||
"event_id", "rule_id", "device_ids", "detail", "diagnostic_policy"
|
||||
) if key in event}
|
||||
result["step_anomaly_signals"] = list(signals.values())
|
||||
monitoring = None
|
||||
if was_step and not result["step_pending"]:
|
||||
period_start = parse_time(result.get("step_start", result["start_time"]))
|
||||
@@ -261,7 +306,7 @@ class ScadaReplayService:
|
||||
role.get("pressure_device_ids", []) + role.get("outlet_flow_device_ids", [])
|
||||
))
|
||||
previous = await self.repo.monitoring_outputs(
|
||||
replay_id, role_ids, max(config.start_time, period_end - timedelta(days=15)),
|
||||
replay_id, role_ids, period_end - timedelta(days=15),
|
||||
period_end, batch["sequence"],
|
||||
) if role_ids else []
|
||||
from app.mcp.analysis_service import bounded_calculation
|
||||
@@ -273,11 +318,25 @@ class ScadaReplayService:
|
||||
period_end=period_end,
|
||||
batch_sequence=batch["sequence"],
|
||||
)
|
||||
result["last_monitoring"] = monitoring
|
||||
# Any configured measurement rule can enter diagnosis, including
|
||||
# indicators outside the pressure/outlet roles used for burst scoring.
|
||||
signals = {item["event_id"]: item for item in result.get("step_anomaly_signals", [])
|
||||
if diagnostic_signal(result, item)}
|
||||
for state in result.get("rule_states", {}).values():
|
||||
event = state.get("event")
|
||||
if event and diagnostic_signal(result, event):
|
||||
signals[event["event_id"]] = {key: event[key] for key in (
|
||||
"event_id", "rule_id", "device_ids", "detail", "diagnostic_policy"
|
||||
) if key in event}
|
||||
monitoring["anomaly_signals"] = list(signals.values())
|
||||
if signals and monitoring["status"] == "normal":
|
||||
monitoring["status"] = "attention"
|
||||
monitoring["detail"] = "本周期曾检出测点异常,进入异常调查与爆管诊断"
|
||||
result["monitoring_count"] = result.get("monitoring_count", 0) + 1
|
||||
if not result["step_pending"]:
|
||||
result.pop("step_target", None)
|
||||
result.pop("step_start", None)
|
||||
result.pop("step_anomaly_signals", None)
|
||||
if batch["refs"]:
|
||||
last = batch["refs"][-1]
|
||||
result["cursor"] = [last["time"], last["device_id"]]
|
||||
@@ -295,6 +354,80 @@ class ScadaReplayService:
|
||||
await self.repo.save_batch(replay_id, batch)
|
||||
await self.repo.save(result)
|
||||
|
||||
async def _freeze_baseline(self, document, config):
|
||||
"""Stage bounded pages, then atomically publish one causal source snapshot."""
|
||||
if document.get("warmed"):
|
||||
return
|
||||
start = config.start_time - timedelta(days=15)
|
||||
batch_count, row_count, byte_count, after = 0, 0, 0, None
|
||||
counts = {device_id: 0 for device_id in config.device_ids}
|
||||
valid_counts = dict(counts)
|
||||
histories = {device_id: [] for device_id in config.device_ids}
|
||||
window = max(rule.window_samples for rule in config.rules)
|
||||
calibration = None
|
||||
if (document.get("monitoring_config") or {}).get("decision_policy") == "weighted":
|
||||
from app.services.scada_monitoring_calibration import MonitoringCalibrationBuilder
|
||||
calibration = MonitoringCalibrationBuilder(document, config)
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.clear_unpublished_baseline(document["replay_id"])
|
||||
# Keep only the current page in memory. Each BizDB write commits before
|
||||
# the next source read; clean_prepared baseline pages remain invisible.
|
||||
async with self.timescale.transaction():
|
||||
await ScadaRepository.begin_replay_baseline_snapshot(self.timescale)
|
||||
while True:
|
||||
rows = await ScadaRepository.get_raw_page(
|
||||
self.timescale, config.device_ids, start, config.start_time,
|
||||
config.start_time, min(BASELINE_PAGE_SIZE, BASELINE_MAX_ROWS - row_count + 1), after,
|
||||
)
|
||||
if not rows:
|
||||
break
|
||||
if row_count + len(rows) > BASELINE_MAX_ROWS:
|
||||
raise BaselineBudgetExceeded(
|
||||
f"BASELINE_BUDGET_EXCEEDED: 十五天历史基线超过 {BASELINE_MAX_ROWS:,} 条准备预算,"
|
||||
"尚未发布;请减少回放设备或核查历史采样密度后重试"
|
||||
)
|
||||
outputs = prepare_cleaning(rows, config)
|
||||
if calibration is not None:
|
||||
calibration.add(outputs)
|
||||
batch = {
|
||||
"sequence": -batch_count - 1, "stage": "clean_prepared", "kind": "baseline",
|
||||
"at": outputs[-1]["time"], "outputs": outputs, "events": [],
|
||||
}
|
||||
batch_bytes = len(json.dumps(batch, ensure_ascii=False, allow_nan=False,
|
||||
separators=(",", ":")).encode("utf-8"))
|
||||
if byte_count + batch_bytes > BASELINE_MAX_BYTES:
|
||||
raise BaselineBudgetExceeded(
|
||||
f"BASELINE_BUDGET_EXCEEDED: 十五天历史基线超过 {BASELINE_MAX_BYTES // (1024 * 1024)} MiB 准备预算,"
|
||||
"尚未发布;请减少回放设备或核查历史采样密度后重试"
|
||||
)
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.save_batch(document["replay_id"], batch)
|
||||
batch_count += 1
|
||||
row_count += len(outputs)
|
||||
byte_count += batch_bytes
|
||||
for row in outputs:
|
||||
device_id = row["device_id"]
|
||||
counts[device_id] += 1
|
||||
if row["quality"] == "valid":
|
||||
valid_counts[device_id] += 1
|
||||
histories[device_id].append(row["cleaned_value"])
|
||||
histories[device_id] = histories[device_id][-window:]
|
||||
last = rows[-1]
|
||||
after = (last["time"], last["device_id"])
|
||||
document["baseline"] = {
|
||||
"start_time": start.isoformat(), "end_time": config.start_time.isoformat(),
|
||||
"end_exclusive": True, "sample_counts": counts, "valid_sample_counts": valid_counts,
|
||||
"row_count": row_count, "batch_count": batch_count, "byte_count": byte_count,
|
||||
}
|
||||
document["histories"] = histories
|
||||
if calibration is not None:
|
||||
from app.mcp.analysis_service import bounded_calculation
|
||||
document["monitoring_calibration"] = await bounded_calculation(calibration.fit)
|
||||
document["warmed"] = True
|
||||
async with self.repo.connection.transaction():
|
||||
await self.repo.publish_baseline(document["replay_id"])
|
||||
await self.repo.save(document)
|
||||
|
||||
async def _create_investigations(self, document, changes, batch, *, create_tasks=True):
|
||||
from app.domain.schemas.operational_task import OperationalTaskCreate, TaskSource, TaskTrigger
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
@@ -307,11 +440,23 @@ class ScadaReplayService:
|
||||
alerts = TaskAlertService(TaskAlertRepository(self.repo.connection, self.context.project_id),
|
||||
self.context, internal_producer_id=producer)
|
||||
tasks = OperationalTaskService(OperationalTaskRepository(self.repo.connection, self.context.project_id),
|
||||
self.context, permissions_for_context(self.context))
|
||||
self.context, permissions_for_context(self.context),
|
||||
execution_scope=settings.operational_task_execution_scope)
|
||||
for event in changes:
|
||||
# Period diagnosis owns measurement notifications during manual steps.
|
||||
# Retain rule events as evidence; quality alerts remain independent.
|
||||
if (not create_tasks and event.get("category") != "data_quality"
|
||||
and (event["change"] == "raised" or event.get("notification_suppressed"))):
|
||||
if event["change"] == "raised":
|
||||
event["batch_sequence"] = batch["sequence"]
|
||||
event["notification_suppressed"] = True
|
||||
document["rule_states"][event.get("state_key", event["rule_id"])]["event"].update(
|
||||
batch_sequence=batch["sequence"], notification_suppressed=True,
|
||||
)
|
||||
continue
|
||||
if event["change"] == "raised":
|
||||
event["batch_sequence"] = batch["sequence"]
|
||||
document["rule_states"][event["rule_id"]]["event"]["batch_sequence"] = batch["sequence"]
|
||||
document["rule_states"][event.get("state_key", event["rule_id"])]["event"]["batch_sequence"] = batch["sequence"]
|
||||
await alerts.push(TaskAlertEvent(
|
||||
event_id=uuid5(NAMESPACE_URL, event["event_id"] + ":" + event["change"]),
|
||||
alert_id=event["event_id"], producer_id=producer, event_type=event["change"],
|
||||
@@ -345,10 +490,31 @@ class ScadaReplayService:
|
||||
)
|
||||
task = await tasks.create(command)
|
||||
event["task_id"] = str(task.task_id)
|
||||
document["rule_states"][event["rule_id"]]["event"]["task_id"] = event["task_id"]
|
||||
document["rule_states"][event.get("state_key", event["rule_id"])]["event"]["task_id"] = event["task_id"]
|
||||
|
||||
async def _create_monitoring_task(self, document, monitoring, batch):
|
||||
if monitoring["status"] != "suspected_burst":
|
||||
from app.services.scada_monitoring_incidents import attach_incident, update_incident
|
||||
incident, action = update_incident(document, monitoring, batch["sequence"])
|
||||
if incident is None:
|
||||
return
|
||||
if incident.get("task_id"):
|
||||
if monitoring["status"] == "suspected_burst" and not incident.get("burst_diagnosed"):
|
||||
from uuid import UUID
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
from app.services.operational_tasks import OperationalTaskService
|
||||
tasks = OperationalTaskService(
|
||||
OperationalTaskRepository(self.repo.connection, self.context.project_id),
|
||||
self.context, permissions_for_context(self.context),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
await tasks.advance_replay_investigation(UUID(incident["task_id"]), {
|
||||
"replay_id": document["replay_id"], "knowledge_cutoff": monitoring["knowledge_cutoff"],
|
||||
"batch_sequence": batch["sequence"],
|
||||
}, uuid5(NAMESPACE_URL, incident["incident_id"] + ":burst-evidence"))
|
||||
incident["burst_diagnosed"] = True
|
||||
if action == "resolved" and incident.get("alert"):
|
||||
await self._publish_monitoring_alert(document, incident, "resolved")
|
||||
attach_incident(monitoring, incident, action)
|
||||
return
|
||||
from app.domain.schemas.operational_task import OperationalTaskCreate, TaskSource, TaskTrigger
|
||||
from app.infra.db.postgresql.operational_tasks import OperationalTaskRepository
|
||||
@@ -357,30 +523,46 @@ class ScadaReplayService:
|
||||
OperationalTaskRepository(self.repo.connection, self.context.project_id),
|
||||
self.context,
|
||||
permissions_for_context(self.context),
|
||||
execution_scope=settings.operational_task_execution_scope,
|
||||
)
|
||||
external = (
|
||||
f"scada_replay:{document['replay_id']}:monitor:"
|
||||
f"{monitoring['period_end']}:{monitoring['rule_version']}"
|
||||
)
|
||||
external = incident["external_id"]
|
||||
pressure = monitoring["pressure"]
|
||||
flow = monitoring["outlet_flow"]
|
||||
detector = monitoring["burst_detection"]
|
||||
score = detector.get("target_score")
|
||||
score_text = f"{score:.4f}" if score is not None else "数据不足"
|
||||
summary = (
|
||||
f"步长监测疑似爆管:爆管侦测得分 {detector['target_score']:.4f};"
|
||||
f"压力下降测点 {len(pressure['hit_device_ids'])} 个;"
|
||||
f"出厂边界流量增加测点 {len(flow['hit_device_ids'])} 个。"
|
||||
f"SCADA 数据诊断发现指标异常,进入异常调查与爆管诊断。"
|
||||
f"爆管侦测得分 {score_text};"
|
||||
f"压力下降测点 {len(pressure.get('hit_device_ids', []))} 个;"
|
||||
f"出厂边界流量增加测点 {len(flow.get('hit_device_ids', []))} 个。"
|
||||
"任一指标异常即启动调查,加权分用于评估爆管嫌疑;达到门槛后继续候选定位、隔离与影响分析。"
|
||||
)
|
||||
if monitoring.get("anomaly_signals"):
|
||||
signals = monitoring["anomaly_signals"]
|
||||
devices = {device for item in signals for device in item.get("device_ids", [])}
|
||||
summary += f" 单项异常 {len(signals)} 项,涉及 {len(devices)} 个测点;明细通过冻结 SCADA 查询核验。"
|
||||
if monitoring.get("weighted"):
|
||||
weighted = monitoring["weighted"]
|
||||
weights = weighted["weights"]
|
||||
summary += (
|
||||
f" 综合评分 {weighted['score']:.2f},爆管嫌疑门槛 {weighted['threshold']:.2f}"
|
||||
f"(侦测 {weights['burst_detection']:.0%}、压力 {weights['pressure']:.0%}、"
|
||||
f"流量 {weights['outlet_flow']:.0%})。"
|
||||
)
|
||||
if detector.get("score_threshold") is not None:
|
||||
summary += f" 侦测动态阈值 {detector['score_threshold']:.4f}。"
|
||||
command = OperationalTaskCreate(
|
||||
request_id=uuid5(NAMESPACE_URL, external),
|
||||
task_type="event_investigation",
|
||||
workflow="burst_report",
|
||||
title=f"SCADA 回放疑似爆管 · {monitoring['period_end']}",
|
||||
workflow="scada_anomaly",
|
||||
title=f"SCADA 异常调查 · {monitoring['period_end']}",
|
||||
summary=summary,
|
||||
source=TaskSource(
|
||||
kind="external_system",
|
||||
external_id=external,
|
||||
external_revision=1,
|
||||
event_kind="scada_burst_suspected",
|
||||
event_kind="scada_anomaly",
|
||||
replay_context={
|
||||
"replay_id": document["replay_id"],
|
||||
"knowledge_cutoff": monitoring["knowledge_cutoff"],
|
||||
@@ -390,8 +572,33 @@ class ScadaReplayService:
|
||||
trigger=TaskTrigger(
|
||||
kind="event",
|
||||
occurred_at=monitoring["period_end"],
|
||||
detail="主动步长监测的多项指标相互支持,进入爆管调查;尚未现场确诊",
|
||||
detail="SCADA 任一指标异常启动调查,核验数据后进行爆管诊断;异常不等于已确诊爆管",
|
||||
),
|
||||
)
|
||||
task = await tasks.create(command)
|
||||
monitoring["task_id"] = str(task.task_id)
|
||||
incident["task_id"] = str(task.task_id)
|
||||
incident["burst_diagnosed"] = monitoring["status"] == "suspected_burst"
|
||||
incident["alert"] = {
|
||||
"alert_id": incident["incident_id"],
|
||||
"producer_id": "scada-replay:" + document["replay_id"],
|
||||
"occurred_at": incident["started_at"],
|
||||
"condition_id": f"operational-task:{task.task_id}",
|
||||
"source": {"kind": "scada_replay", "replay_id": document["replay_id"],
|
||||
"rule_id": "scada_diagnosis", "batch_sequence": batch["sequence"]},
|
||||
"title": "SCADA 指标异常", "description": "已启动调查,详细诊断见调查报告。", "severity": "warning",
|
||||
}
|
||||
await self._publish_monitoring_alert(document, incident, "raised")
|
||||
attach_incident(monitoring, incident, action)
|
||||
|
||||
async def _publish_monitoring_alert(self, document, incident, change):
|
||||
from app.domain.schemas.task_alert import TaskAlertEvent
|
||||
from app.infra.db.postgresql.task_alerts import TaskAlertRepository
|
||||
from app.services.task_alerts import TaskAlertService
|
||||
alerts = TaskAlertService(
|
||||
TaskAlertRepository(self.repo.connection, self.context.project_id), self.context,
|
||||
internal_producer_id="scada-replay:" + document["replay_id"],
|
||||
)
|
||||
await alerts.push(TaskAlertEvent(
|
||||
**incident["alert"], event_type=change,
|
||||
event_id=uuid5(NAMESPACE_URL, incident["incident_id"] + ":" + change),
|
||||
))
|
||||
|
||||
@@ -9,6 +9,8 @@ from uuid import NAMESPACE_URL, uuid5
|
||||
import pandas as pd
|
||||
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.domain.schemas.project_settings import ScadaMonitoringSettings
|
||||
from app.services.scada_normal_profile import current_calibration, normal_profile_bounds
|
||||
from app.services.burst_detection import (
|
||||
TARGET_MU,
|
||||
TARGET_N_ESTIMATORS,
|
||||
@@ -42,6 +44,7 @@ def _relative_indicator(
|
||||
period_end: datetime,
|
||||
*,
|
||||
direction: str,
|
||||
calibration: dict | None = None,
|
||||
) -> dict[str, Any]:
|
||||
details: list[dict[str, Any]] = []
|
||||
gaps: list[dict[str, str]] = []
|
||||
@@ -54,30 +57,51 @@ def _relative_indicator(
|
||||
if rule is None:
|
||||
gaps.append({"device_id": device_id, "reason": "relative_change_rule_missing"})
|
||||
continue
|
||||
valid = _valid_rows(rows, device_id)
|
||||
baseline = [row for row in valid if _time(row["time"]) < period_start][-rule.window_samples:]
|
||||
current = [row for row in valid if period_start <= _time(row["time"]) <= period_end]
|
||||
if len(baseline) < rule.minimum_samples:
|
||||
gaps.append({"device_id": device_id, "reason": "baseline_not_ready"})
|
||||
profile = calibration.get(device_id) if calibration is not None else None
|
||||
if calibration is not None and profile is None:
|
||||
gaps.append({"device_id": device_id, "reason": "adaptive_history_not_ready"})
|
||||
continue
|
||||
threshold = float(profile["threshold_percent"]) if profile else rule.threshold
|
||||
valid = _valid_rows(rows, device_id)
|
||||
current = [row for row in valid if period_start < _time(row["time"]) <= period_end]
|
||||
latest = current[-1] if current else None
|
||||
if not current:
|
||||
gaps.append({"device_id": device_id, "reason": "period_reading_missing"})
|
||||
continue
|
||||
baseline = ([row for row in valid if _time(row["time"]) < _time(latest["time"])]
|
||||
[-rule.window_samples:]) if latest else []
|
||||
if len(baseline) < rule.minimum_samples:
|
||||
gaps.append({"device_id": device_id, "reason": "baseline_not_ready"})
|
||||
continue
|
||||
reference = median(float(row["cleaned_value"]) for row in baseline)
|
||||
if reference == 0:
|
||||
gaps.append({"device_id": device_id, "reason": "zero_baseline"})
|
||||
continue
|
||||
latest = current[-1]
|
||||
value = float(latest["cleaned_value"])
|
||||
change = (value - reference) / abs(reference) * 100
|
||||
hit = change <= -rule.threshold if direction == "decrease" else change >= rule.threshold
|
||||
hit = change <= -threshold if direction == "decrease" else change >= threshold
|
||||
bounds = normal_profile_bounds(profile.get("normal_profile"), _time(latest["time"])) if profile else None
|
||||
if profile and bounds is None:
|
||||
gaps.append({"device_id": device_id, "reason": "normal_phase_history_not_ready"})
|
||||
continue
|
||||
if bounds is not None:
|
||||
hit = hit and not bounds[0] <= value <= bounds[1]
|
||||
details.append({
|
||||
"device_id": device_id,
|
||||
"observed_at": _time(latest["time"]).isoformat(),
|
||||
"baseline": reference,
|
||||
"baseline_method": "recent_samples_median",
|
||||
"baseline_start": _time(baseline[0]["time"]).isoformat(),
|
||||
"baseline_end": _time(baseline[-1]["time"]).isoformat(),
|
||||
"baseline_sample_count": len(baseline),
|
||||
"value": value,
|
||||
"value_change": value - reference,
|
||||
"unit": rule.unit,
|
||||
"change_percent": change,
|
||||
"threshold_percent": rule.threshold,
|
||||
"threshold_percent": threshold,
|
||||
**({"normal_phase_lower": bounds[0], "normal_phase_upper": bounds[1]} if bounds else {}),
|
||||
**({"threshold_method": profile["method"],
|
||||
"configured_threshold_percent": rule.threshold} if profile else {}),
|
||||
"hit": hit,
|
||||
})
|
||||
return {
|
||||
@@ -94,6 +118,8 @@ def _burst_detection(
|
||||
pressure_device_ids: list[str],
|
||||
interval_seconds: int,
|
||||
period_end: datetime,
|
||||
*,
|
||||
score_threshold: float = TARGET_SCORE_THRESHOLD,
|
||||
) -> dict[str, Any]:
|
||||
if len(pressure_device_ids) < 5:
|
||||
return {"status": "sensor_coverage_insufficient", "hit": False,
|
||||
@@ -112,7 +138,10 @@ def _burst_detection(
|
||||
common_times = set(values) if common_times is None else common_times & set(values)
|
||||
if not common_times:
|
||||
return {"status": "baseline_not_ready", "hit": False, "required_samples": count}
|
||||
target = max(common_times)
|
||||
target = period_end
|
||||
if target not in common_times:
|
||||
return {"status": "current_reading_missing", "hit": False,
|
||||
"target_time": target.isoformat(), "latest_common_time": max(common_times).isoformat()}
|
||||
expected = [target - timedelta(seconds=interval_seconds * offset)
|
||||
for offset in range(count - 1, -1, -1)]
|
||||
incomplete = [device_id for device_id, values in series_by_device.items()
|
||||
@@ -142,10 +171,11 @@ def _burst_detection(
|
||||
score = float(latest["Score"])
|
||||
return {
|
||||
"status": "ready",
|
||||
"hit": score <= TARGET_SCORE_THRESHOLD,
|
||||
"hit": score <= score_threshold,
|
||||
"target_time": target.isoformat(),
|
||||
"target_score": score,
|
||||
"score_threshold": TARGET_SCORE_THRESHOLD,
|
||||
"score_interpretation": "lower_is_more_anomalous_not_probability",
|
||||
"score_threshold": score_threshold,
|
||||
"sensor_count": len(pressure_device_ids),
|
||||
"reference_day_count": 14,
|
||||
"latest_sensor_rankings": result["summary"].get("latest_sensor_rankings", []),
|
||||
@@ -185,17 +215,64 @@ def evaluate_monitoring_period(
|
||||
flow_ids = monitoring.get("outlet_flow_device_ids", [])
|
||||
selected = set(config.device_ids)
|
||||
role_gaps = sorted((set(pressure_ids) | set(flow_ids)) - selected)
|
||||
if not pressure_ids or not flow_ids or role_gaps:
|
||||
weighted_policy = monitoring.get("decision_policy") == "weighted"
|
||||
if not weighted_policy and (not pressure_ids or not flow_ids or role_gaps):
|
||||
return {**base, "status": "insufficient_data", "detail": "监测角色为空或不属于本轮回放设备",
|
||||
"role_gaps": role_gaps,
|
||||
"pressure": {"status": "unconfigured", "hit": False},
|
||||
"outlet_flow": {"status": "unconfigured", "hit": False},
|
||||
"burst_detection": {"status": "not_run", "hit": False}}
|
||||
pressure = _relative_indicator(rows, config, pressure_ids, period_start, period_end, direction="decrease")
|
||||
outlet_flow = _relative_indicator(rows, config, flow_ids, period_start, period_end, direction="increase")
|
||||
detector = _burst_detection(
|
||||
rows, pressure_ids, int(document["monitoring_interval_seconds"]), period_end,
|
||||
calibration = current_calibration(document, config.start_time) if weighted_policy else None
|
||||
if config.start_time > period_end:
|
||||
calibration = None
|
||||
base["role_gaps"] = role_gaps
|
||||
pressure = _relative_indicator(
|
||||
rows, config, pressure_ids, period_start, period_end, direction="decrease",
|
||||
calibration=(calibration or {}).get("pressure", {}) if weighted_policy else None,
|
||||
)
|
||||
outlet_flow = _relative_indicator(
|
||||
rows, config, flow_ids, period_start, period_end, direction="increase",
|
||||
calibration=(calibration or {}).get("outlet_flow", {}) if weighted_policy else None,
|
||||
) if flow_ids else {"status": "unconfigured", "hit": False, "hit_device_ids": [], "devices": [], "gaps": []}
|
||||
detector = {"status": "baseline_not_ready", "hit": False} if weighted_policy and (
|
||||
not calibration or calibration.get("status") != "ready"
|
||||
) else _burst_detection(
|
||||
rows, pressure_ids, int(document["monitoring_interval_seconds"]), period_end,
|
||||
score_threshold=calibration["burst_detection"]["threshold"] if calibration else TARGET_SCORE_THRESHOLD,
|
||||
)
|
||||
if weighted_policy:
|
||||
settings = ScadaMonitoringSettings.model_validate(monitoring)
|
||||
weights = settings.weights.model_dump()
|
||||
indicators = {"burst_detection": detector, "pressure": pressure, "outlet_flow": outlet_flow}
|
||||
contributions = {key: weights[key] if indicator["status"] == "ready" and indicator["hit"] else 0.0
|
||||
for key, indicator in indicators.items()}
|
||||
score = round(sum(contributions.values()), 10)
|
||||
complete = pressure["status"] == detector["status"] == "ready"
|
||||
diagnostic_missing = sorted({device for rule in config.rules if rule.kind == "relative_change"
|
||||
for device in rule.device_ids}
|
||||
- set((calibration or {}).get("diagnostic", {})))
|
||||
if not complete:
|
||||
status, detail = "insufficient_data", (
|
||||
"旧回放缺少新版历史诊断校准,请停止后新建演示" if calibration is None
|
||||
else "压力数据或爆管侦测基线不足,尚不能作出组合判定"
|
||||
)
|
||||
elif score >= settings.alert_threshold:
|
||||
status, detail = "suspected_burst", "爆管相关指标加权达到疑似爆管门槛"
|
||||
elif any(indicator["hit"] for indicator in indicators.values()):
|
||||
status, detail = "attention", "部分指标异常,进入调查;综合结果尚未达到爆管嫌疑门槛"
|
||||
elif diagnostic_missing:
|
||||
status, detail = "insufficient_data", "部分测点缺少完整历史同期基线,不能判定全部指标正常"
|
||||
else:
|
||||
status, detail = "normal", "本周期爆管相关指标均未命中"
|
||||
detail += f"(综合 {score:.2f} / 门槛 {settings.alert_threshold:.2f},流量权重 {weights['outlet_flow']:.0%})"
|
||||
return {**base, "status": status, "detail": detail, "pressure": pressure,
|
||||
"outlet_flow": outlet_flow, "burst_detection": detector,
|
||||
"weighted": {"score": score, "threshold": settings.alert_threshold,
|
||||
"weights": weights, "contributions": contributions,
|
||||
"unavailable_indicators": [key for key, value in indicators.items() if value["status"] != "ready"],
|
||||
"calibration_cutoff": calibration["knowledge_cutoff"] if calibration else None,
|
||||
"diagnostic_unavailable_device_ids": diagnostic_missing,
|
||||
"missing_weight_policy": "zero_contribution_without_renormalization"}}
|
||||
complete = pressure["status"] == outlet_flow["status"] == detector["status"] == "ready"
|
||||
hits = (bool(detector["hit"]), bool(pressure["hit"]), bool(outlet_flow["hit"]))
|
||||
if not complete:
|
||||
|
||||
@@ -6,6 +6,9 @@ from statistics import median
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
import re
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.services.scada_normal_profile import (
|
||||
DIAGNOSTIC_POLICY, current_calibration, normal_profile_bounds, weighted_diagnosis,
|
||||
)
|
||||
|
||||
|
||||
def prepare_cleaning(rows: list[dict], config: ScadaReplayCreate) -> list[dict]:
|
||||
@@ -43,6 +46,8 @@ def detect_frame(document: dict, rows: list[dict], config: ScadaReplayCreate, at
|
||||
by_id = {row["device_id"]: row for row in rows}
|
||||
changes = []
|
||||
devices = {device["device_id"]: device for device in result.get("devices", [])}
|
||||
weighted = weighted_diagnosis(result)
|
||||
calibration = current_calibration(result, config.start_time) if weighted else None
|
||||
for device_id in config.device_ids:
|
||||
row = by_id.get(device_id)
|
||||
invalid = row is not None and row["quality"] != "valid"
|
||||
@@ -75,7 +80,8 @@ def detect_frame(document: dict, rows: list[dict], config: ScadaReplayCreate, at
|
||||
changes.append({**state["event"], "status": "resolved", "resolved_at": at.isoformat(), "change": "resolved"})
|
||||
state["event"] = None
|
||||
for rule in config.rules:
|
||||
state = states.setdefault(rule.rule_id, {"hits": 0, "recoveries": 0, "event": None})
|
||||
state_key = f"{rule.rule_id}:{DIAGNOSTIC_POLICY}" if weighted and rule.kind == "relative_change" else rule.rule_id
|
||||
state = states.setdefault(state_key, {"hits": 0, "recoveries": 0, "event": None})
|
||||
hits, usable, reasons = [], [], []
|
||||
for device_id in rule.device_ids:
|
||||
row = by_id.get(device_id)
|
||||
@@ -86,6 +92,17 @@ def detect_frame(document: dict, rows: list[dict], config: ScadaReplayCreate, at
|
||||
triggered = ((rule.lower is not None and value < rule.lower)
|
||||
or (rule.upper is not None and value > rule.upper))
|
||||
reason = f"{device_id}={value} {rule.unit},超出配置范围"
|
||||
elif weighted:
|
||||
# The percentage in the compatibility rule is not a diagnosis
|
||||
# threshold. Learn each device's normal same-phase range first.
|
||||
profile = (calibration or {}).get("diagnostic", {}).get(device_id)
|
||||
bounds = normal_profile_bounds(profile, at)
|
||||
if bounds is None:
|
||||
continue
|
||||
low, high = bounds
|
||||
triggered = (value < low or value > high if rule.direction == "both"
|
||||
else value > high if rule.direction == "increase" else value < low)
|
||||
reason = f"{device_id}={value} {rule.unit},超出冻结正常同时段范围 [{low:.4f}, {high:.4f}] {rule.unit}"
|
||||
else:
|
||||
previous = histories.get(device_id, [])[-rule.window_samples:]
|
||||
if len(previous) < rule.minimum_samples:
|
||||
@@ -106,10 +123,12 @@ def detect_frame(document: dict, rows: list[dict], config: ScadaReplayCreate, at
|
||||
state["hits"] += 1
|
||||
state["recoveries"] = 0
|
||||
if state["event"] is None and state["hits"] >= rule.consecutive_samples:
|
||||
event_id = str(uuid5(NAMESPACE_URL, f"{result['replay_id']}:{rule.rule_id}:{at.isoformat()}"))
|
||||
event_id = str(uuid5(NAMESPACE_URL, f"{result['replay_id']}:{state_key}:{at.isoformat()}"))
|
||||
event = {"event_id": event_id, "rule_id": rule.rule_id, "device_ids": hits,
|
||||
"occurred_at": at.isoformat(), "last_observed_at": at.isoformat(),
|
||||
"status": "active", "detail": ";".join(reasons)[:2000], "task_id": None}
|
||||
if weighted and rule.kind == "relative_change":
|
||||
event.update(diagnostic_policy=DIAGNOSTIC_POLICY, state_key=state_key)
|
||||
state["event"] = event
|
||||
changes.append({**event, "change": "raised"})
|
||||
result["anomaly_count"] = result.get("anomaly_count", 0) + 1
|
||||
@@ -128,7 +147,9 @@ def detect_frame(document: dict, rows: list[dict], config: ScadaReplayCreate, at
|
||||
# Incomplete frames cannot count as a consecutive hit/recovery.
|
||||
elif not usable and any(device in by_id for device in rule.device_ids):
|
||||
state["hits"] = state["recoveries"] = 0
|
||||
frozen = {device for rule in config.rules if states[rule.rule_id]["event"]
|
||||
frozen = {device for rule in config.rules if states[
|
||||
f"{rule.rule_id}:{DIAGNOSTIC_POLICY}" if weighted and rule.kind == "relative_change" else rule.rule_id
|
||||
]["event"]
|
||||
for device in rule.device_ids}
|
||||
for row in rows:
|
||||
if row["quality"] == "valid" and row["device_id"] not in frozen:
|
||||
|
||||
@@ -12,14 +12,23 @@ def check_replay_window(context, start, end):
|
||||
raise HTTPException(422, f"回放查询需要明确时间范围,结束不得晚于 {cap.isoformat()}")
|
||||
|
||||
|
||||
def check_snapshot_start(run, start):
|
||||
replay_start = datetime.fromisoformat(run["start_time"])
|
||||
if start < replay_start:
|
||||
baseline = run.get("baseline")
|
||||
if not baseline:
|
||||
raise HTTPException(422, "本轮回放未冻结历史对照,请新建回放后查询历史基线")
|
||||
if start < datetime.fromisoformat(baseline["start_time"]):
|
||||
raise HTTPException(422, "查询起点早于本轮已冻结的历史基线")
|
||||
|
||||
|
||||
async def replay_snapshot_rows(connection, project_id, context, device_ids, start, end, limit, offset=0):
|
||||
check_replay_window(context, start, end)
|
||||
repository = ScadaReplayRepository(connection, project_id)
|
||||
run = await repository.get(context["replay_id"])
|
||||
if not set(device_ids) <= set(run["config"]["device_ids"]):
|
||||
raise HTTPException(403, "设备或回放不属于当前调查")
|
||||
if start < datetime.fromisoformat(run["start_time"]):
|
||||
raise HTTPException(422, "当前调查证据接口仅返回本轮已处理观测,回放前基线已用于后台初始化")
|
||||
check_snapshot_start(run, start)
|
||||
rows = await repository.detected_outputs(
|
||||
context["replay_id"], device_ids, start, end, context["batch_sequence"],
|
||||
limit=limit, offset=offset,
|
||||
@@ -40,9 +49,7 @@ async def replay_burst_location_inputs(connection, project_id, context, args):
|
||||
repository = ScadaReplayRepository(connection, project_id)
|
||||
run = await repository.get(context["replay_id"])
|
||||
if run:
|
||||
replay_start = datetime.fromisoformat(run["start_time"])
|
||||
if args.scada_burst_start < replay_start or args.scada_normal_start < replay_start:
|
||||
raise HTTPException(422, "定位窗口必须位于本轮已处理回放范围内")
|
||||
check_snapshot_start(run, min(args.scada_burst_start, args.scada_normal_start))
|
||||
devices = {item["device_id"]: item for item in run.get("devices", [])}
|
||||
monitoring = run.get("monitoring_config") or {}
|
||||
pressure_devices = monitoring.get("pressure_device_ids", [])
|
||||
|
||||
@@ -13,9 +13,7 @@ if TYPE_CHECKING:
|
||||
|
||||
|
||||
async def list_questions(service: "OperationalTaskService", task_id: UUID) -> list[dict]:
|
||||
# Legacy test repositories do not implement question persistence.
|
||||
reader = getattr(service.repo, "list_questions", None)
|
||||
return await reader(str(task_id)) if reader else []
|
||||
return await service.repo.list_questions(str(task_id))
|
||||
|
||||
|
||||
async def cancel_pending(service: "OperationalTaskService", task_id: UUID, now: datetime) -> None:
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
"""Publication-only validation for same-time-of-day line/scatter comparisons."""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
|
||||
from app.domain.domain_errors import require
|
||||
|
||||
|
||||
DISPLAY_TIMEZONE = timezone(timedelta(hours=8))
|
||||
DAY_SECONDS = 86400
|
||||
|
||||
|
||||
def validate_time_comparison_chart(chart: Mapping[str, Any], *, allow_missing_current: bool = False) -> None:
|
||||
"""Validate new publications; persisted historical charts keep their read contract."""
|
||||
chart_type = chart["chart_type"]
|
||||
series = chart["series"]
|
||||
kinds = {item.get("type") or chart_type for item in series}
|
||||
if chart_type != "time_comparison":
|
||||
require(not {"line", "scatter"} <= kinds, "TIME_COMPARISON_INVALID",
|
||||
"曲线与历史散点复合图须使用 chart_type=time_comparison", status=422)
|
||||
return
|
||||
|
||||
def check(condition: bool, message: str) -> None:
|
||||
require(condition, "TIME_COMPARISON_INVALID", message, status=422)
|
||||
|
||||
kinds = {item.get("type") or "line" for item in series}
|
||||
check(kinds == {"line", "scatter"},
|
||||
"time_comparison 的 series 必须包含当前 line 和历史 scatter,不能混入 bar")
|
||||
labels = chart["x_data"]
|
||||
check(all(re.fullmatch(r"(?:[01][0-9]|2[0-3]):[0-5][0-9](?::[0-5][0-9])?", label)
|
||||
for label in labels),
|
||||
"time_comparison.x_data 只接受 HH:mm 或 HH:mm:ss 时刻;历史日期保留在 sample_times,不能追加到横轴")
|
||||
check(len({len(label) for label in labels}) == 1, "x_data 必须使用一致的时刻精度")
|
||||
seconds = [sum(int(part) * factor for part, factor in zip(label.split(":"), (3600, 60, 1)))
|
||||
for label in labels]
|
||||
offsets = [0]
|
||||
for index in range(1, len(seconds)):
|
||||
step = (seconds[index] - seconds[index - 1]) % DAY_SECONDS
|
||||
check(step > 0, f"x_data[{index}] 重复了上一时刻;历史点应复用当前槽位")
|
||||
offsets.append(offsets[-1] + step)
|
||||
check(offsets[-1] < DAY_SECONDS, "x_data 必须按时间递增且小于一天,不能追加多天历史槽位")
|
||||
|
||||
current_bases: set[datetime] = set()
|
||||
historical_bases: list[datetime] = []
|
||||
for series_index, item in enumerate(series):
|
||||
times = item.get("sample_times")
|
||||
path = f"series[{series_index}]"
|
||||
check(times is not None and len(times) == len(labels), f"{path}.sample_times 必须提供并与 x_data 等长")
|
||||
bases: set[datetime] = set()
|
||||
for index, (value, at) in enumerate(zip(item["data"], times, strict=True)):
|
||||
check((value is None) == (at is None),
|
||||
f"{path}.sample_times[{index}]:每个有效值必须保留真实带时区时间,缺测值和时间均填 null")
|
||||
if at is None:
|
||||
continue
|
||||
try:
|
||||
local = datetime.fromisoformat(at).astimezone(DISPLAY_TIMEZONE)
|
||||
base = local - timedelta(seconds=offsets[index])
|
||||
except OverflowError:
|
||||
check(False, f"{path}.sample_times[{index}] 超出同期对比可表达的日期范围")
|
||||
label = labels[index]
|
||||
expected = local.strftime("%H:%M:%S" if len(label) == 8 else "%H:%M")
|
||||
check(expected == label and local.microsecond == 0 and (len(label) == 8 or local.second == 0),
|
||||
f"{path}.sample_times[{index}] 换算为 UTC+8 后须与 x_data[{index}]={label} 对齐")
|
||||
bases.add(base)
|
||||
check(len(bases) <= 1, f"{path}.sample_times 必须按同一窗口对齐,不能将不同日期拼入同一组")
|
||||
if (item.get("type") or "line") == "line":
|
||||
current_bases.update(bases)
|
||||
else:
|
||||
historical_bases.extend(bases)
|
||||
scada = chart.get("scada")
|
||||
empty_scada = allow_missing_current and not current_bases and scada and scada["role"] == "comparison"
|
||||
check(len(current_bases) == 1 or bool(empty_scada),
|
||||
"当前 line 必须包含真实采样时间,且所有当前曲线使用同一窗口")
|
||||
# SCADA publication separately verifies every null against persisted reads.
|
||||
if empty_scada:
|
||||
window_start = datetime.fromisoformat(scada["window_start"]).astimezone(DISPLAY_TIMEZONE)
|
||||
start_seconds = window_start.hour * 3600 + window_start.minute * 60 + window_start.second
|
||||
current = window_start.replace(microsecond=0) + timedelta(seconds=(seconds[0] - start_seconds) % DAY_SECONDS)
|
||||
else:
|
||||
current = next(iter(current_bases))
|
||||
for historical in historical_bases:
|
||||
delta = (current - historical).total_seconds()
|
||||
check(delta > 0 and delta % DAY_SECONDS == 0,
|
||||
"历史 scatter 必须来自当前窗口之前的对应日期,不能使用当天或未来采样")
|
||||
@@ -1,11 +1,10 @@
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
from typing import Any, Dict, List
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from psycopg import AsyncConnection
|
||||
from uuid import UUID
|
||||
|
||||
from app.algorithms.pipe_health_prediction.survival_predictor import (
|
||||
PipeHealthSurvivalPredictor,
|
||||
@@ -15,14 +14,11 @@ from app.algorithms.scada_cleaning.pressure_series import clean_pressure_data_df
|
||||
from app.infra.db.postgresql.network_assets import NetworkAssetRepository
|
||||
from app.infra.db.postgresql.scada import ScadaInfoRepository
|
||||
from app.infra.db.timescaledb.repositories.realtime import RealtimeRepository
|
||||
from app.infra.db.timescaledb.repositories.analysis import AnalysisResultsRepository
|
||||
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
|
||||
|
||||
|
||||
class TimeseriesAnalysisService:
|
||||
"""
|
||||
复合查询类,提供跨表查询功能
|
||||
"""
|
||||
"""SCADA 清洗与管道健康预测;历史查询由 TimeseriesHistoryService 负责。"""
|
||||
|
||||
@staticmethod
|
||||
async def _get_project_scada_index(
|
||||
@@ -31,294 +27,6 @@ class TimeseriesAnalysisService:
|
||||
scadas = await ScadaInfoRepository.get_scadas(postgres_conn)
|
||||
return {scada["device_id"]: scada for scada in scadas}
|
||||
|
||||
@staticmethod
|
||||
async def get_scada_associated_realtime_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
postgres_conn: AsyncConnection,
|
||||
device_ids: List[str],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 SCADA 关联的 link/node 模拟值
|
||||
|
||||
根据传入的 SCADA device_ids,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
postgres_conn: PostgreSQL 异步连接
|
||||
device_ids: SCADA 设备ID列表
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 device_id 为键,值为数据列表,每个数据包含 time, value 和 scada_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当 SCADA 设备未找到或字段无效时
|
||||
"""
|
||||
scada_by_id = await TimeseriesAnalysisService._get_project_scada_index(postgres_conn)
|
||||
link_devices: dict[str, str] = {}
|
||||
node_devices: dict[str, str] = {}
|
||||
for device_id in device_ids:
|
||||
target_scada = scada_by_id.get(device_id)
|
||||
if not target_scada:
|
||||
raise ValueError(f"SCADA device {device_id} not found")
|
||||
scada_type = target_scada["device_type"]
|
||||
if scada_type in {"pipe_flow", "flow"}:
|
||||
link_devices[device_id] = target_scada["link_id"]
|
||||
elif scada_type == "pressure":
|
||||
node_devices[device_id] = target_scada["node_id"]
|
||||
else:
|
||||
raise ValueError(f"Unknown SCADA type: {scada_type}")
|
||||
|
||||
link_series = await RealtimeRepository.get_link_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time,
|
||||
list(dict.fromkeys(link_devices.values())), "flow",
|
||||
)
|
||||
node_series = await RealtimeRepository.get_node_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time,
|
||||
list(dict.fromkeys(node_devices.values())), "pressure",
|
||||
)
|
||||
return {
|
||||
device_id: [
|
||||
{**item, "scada_id": device_id}
|
||||
for item in (
|
||||
link_series.get(element_id, [])
|
||||
if device_id in link_devices
|
||||
else node_series.get(element_id, [])
|
||||
)
|
||||
]
|
||||
for device_id, element_id in (link_devices | node_devices).items()
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_scada_associated_analysis_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
postgres_conn: AsyncConnection,
|
||||
device_ids: List[str],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
run_id: UUID,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 SCADA 关联的 link/node 分析模拟值
|
||||
|
||||
根据传入的 SCADA device_ids,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
postgres_conn: PostgreSQL 异步连接
|
||||
device_ids: SCADA 设备ID列表
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 device_id 为键,值为数据列表,每个数据包含 time, value 和 scada_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当 SCADA 设备未找到或字段无效时
|
||||
"""
|
||||
scada_by_id = await TimeseriesAnalysisService._get_project_scada_index(postgres_conn)
|
||||
link_devices: dict[str, str] = {}
|
||||
node_devices: dict[str, str] = {}
|
||||
for device_id in device_ids:
|
||||
target_scada = scada_by_id.get(device_id)
|
||||
if not target_scada:
|
||||
raise ValueError(f"SCADA device {device_id} not found")
|
||||
|
||||
scada_type = target_scada["device_type"]
|
||||
if scada_type in {"pipe_flow", "flow"}:
|
||||
link_devices[device_id] = target_scada["link_id"]
|
||||
elif scada_type == "pressure":
|
||||
node_devices[device_id] = target_scada["node_id"]
|
||||
else:
|
||||
raise ValueError(f"Unknown SCADA type: {scada_type}")
|
||||
|
||||
link_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "link",
|
||||
list(dict.fromkeys(link_devices.values())), start_time, end_time, "flow",
|
||||
)
|
||||
node_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "node",
|
||||
list(dict.fromkeys(node_devices.values())), start_time, end_time, "pressure",
|
||||
)
|
||||
return {
|
||||
device_id: [
|
||||
{**item, "scada_id": device_id}
|
||||
for item in (
|
||||
link_series.get(element_id, [])
|
||||
if device_id in link_devices
|
||||
else node_series.get(element_id, [])
|
||||
)
|
||||
]
|
||||
for device_id, element_id in (link_devices | node_devices).items()
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_realtime_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
feature_infos: List[Tuple[str, str]],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 link/node 模拟值
|
||||
|
||||
根据传入的 feature_infos,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
feature_infos: 传入的 feature 信息列表,包含 (element_id, type)
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 feature_id 为键,值为数据列表,每个数据包含 time, value 和 feature_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当 SCADA 设备未找到或字段无效时
|
||||
"""
|
||||
pipe_ids: list[str] = []
|
||||
junction_ids: list[str] = []
|
||||
for feature_id, feature_type in feature_infos:
|
||||
if feature_type.lower() == "pipe":
|
||||
pipe_ids.append(feature_id)
|
||||
elif feature_type.lower() == "junction":
|
||||
junction_ids.append(feature_id)
|
||||
else:
|
||||
raise ValueError(f"Unknown type: {feature_type}")
|
||||
link_series = await RealtimeRepository.get_link_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time, list(dict.fromkeys(pipe_ids)), "flow"
|
||||
)
|
||||
node_series = await RealtimeRepository.get_node_fields_by_ids_time_range(
|
||||
timescale_conn, start_time, end_time,
|
||||
list(dict.fromkeys(junction_ids)), "pressure",
|
||||
)
|
||||
return {
|
||||
feature_id: [
|
||||
{**item, "feature_id": feature_id}
|
||||
for item in (
|
||||
link_series.get(feature_id, [])
|
||||
if feature_type.lower() == "pipe"
|
||||
else node_series.get(feature_id, [])
|
||||
)
|
||||
]
|
||||
for feature_id, feature_type in feature_infos
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_analysis_simulation_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
feature_infos: List[Tuple[str, str]],
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
run_id: UUID,
|
||||
) -> Dict[str, List[Dict[str, Any]]]:
|
||||
"""
|
||||
获取 link/node 分析模拟值
|
||||
|
||||
根据传入的 feature_infos,找到关联的 link/node,
|
||||
并根据对应的 type,查询对应的模拟数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
feature_infos: 传入的 feature 信息列表,包含 (element_id, type)
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
run_id: 分析运行 ID
|
||||
|
||||
Returns:
|
||||
模拟数据字典,以 feature_id 为键,值为数据列表,每个数据包含 time, value 和 feature_id
|
||||
|
||||
Raises:
|
||||
ValueError: 当类型无效时
|
||||
"""
|
||||
pipe_ids: list[str] = []
|
||||
junction_ids: list[str] = []
|
||||
for feature_id, feature_type in feature_infos:
|
||||
if feature_type.lower() == "pipe":
|
||||
pipe_ids.append(feature_id)
|
||||
elif feature_type.lower() == "junction":
|
||||
junction_ids.append(feature_id)
|
||||
else:
|
||||
raise ValueError(f"Unknown type: {feature_type}")
|
||||
link_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "link", list(dict.fromkeys(pipe_ids)),
|
||||
start_time, end_time, "flow",
|
||||
)
|
||||
node_series = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn, run_id, "node", list(dict.fromkeys(junction_ids)),
|
||||
start_time, end_time, "pressure",
|
||||
)
|
||||
return {
|
||||
feature_id: [
|
||||
{**item, "feature_id": feature_id}
|
||||
for item in (
|
||||
link_series.get(feature_id, [])
|
||||
if feature_type.lower() == "pipe"
|
||||
else node_series.get(feature_id, [])
|
||||
)
|
||||
]
|
||||
for feature_id, feature_type in feature_infos
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
async def get_element_associated_scada_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
postgres_conn: AsyncConnection,
|
||||
element_id: str,
|
||||
start_time: datetime,
|
||||
end_time: datetime,
|
||||
use_cleaned: bool = False,
|
||||
) -> Optional[Any]:
|
||||
"""
|
||||
获取 link/node 关联的 SCADA 监测值
|
||||
|
||||
根据传入的 link/node id,匹配 SCADA 信息,
|
||||
如果存在关联的 SCADA device_id,获取实际的监测数据
|
||||
|
||||
Args:
|
||||
timescale_conn: TimescaleDB 异步连接
|
||||
postgres_conn: PostgreSQL 异步连接
|
||||
element_id: link 或 node 的 ID
|
||||
start_time: 开始时间
|
||||
end_time: 结束时间
|
||||
use_cleaned: 是否使用清洗后的数据 (True: "cleaned_value", False: "monitored_value")
|
||||
|
||||
Returns:
|
||||
SCADA 监测数据值,如果没有找到则返回 None
|
||||
|
||||
Raises:
|
||||
ValueError: 当元素类型无效时
|
||||
"""
|
||||
|
||||
scada_by_id = await TimeseriesAnalysisService._get_project_scada_index(postgres_conn)
|
||||
associated_scada = next(
|
||||
(
|
||||
scada
|
||||
for scada in scada_by_id.values()
|
||||
if (scada.get("node_id") or scada.get("link_id")) == element_id
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
if not associated_scada:
|
||||
return None
|
||||
|
||||
device_id = associated_scada["device_id"]
|
||||
|
||||
data_field = "cleaned_value" if use_cleaned else "monitored_value"
|
||||
|
||||
res = await ScadaRepository.get_scada_field_by_id_time_range(
|
||||
timescale_conn, [device_id], start_time, end_time, data_field
|
||||
)
|
||||
|
||||
return {element_id: res.get(device_id, [])}
|
||||
|
||||
@staticmethod
|
||||
async def clean_scada_data(
|
||||
timescale_conn: AsyncConnection,
|
||||
|
||||
@@ -57,6 +57,33 @@ def _points(
|
||||
]
|
||||
|
||||
|
||||
def _history_series(
|
||||
*, element_type: HistoryElementType, element_id: str,
|
||||
source: HistorySource, source_unit: str, device_id: str | None = None,
|
||||
unit_inferred: bool = False, points: list[HistoryPoint] | None = None,
|
||||
) -> ElementHistorySeries:
|
||||
metric = _target_metric(element_type)
|
||||
return ElementHistorySeries(
|
||||
element_id=element_id, element_type=element_type, device_id=device_id,
|
||||
metric=metric, source=source, source_unit=source_unit,
|
||||
display_unit=display_unit_for_metric(metric.value),
|
||||
unit_inferred=unit_inferred, points=points if points is not None else [],
|
||||
)
|
||||
|
||||
|
||||
async def _resolve_result_units(
|
||||
postgres_conn: AsyncConnection, query: ElementHistoryQuery,
|
||||
) -> tuple[dict[str, Any], bool]:
|
||||
units = await NetworkSettingsRepository.get_result_units(postgres_conn)
|
||||
if query.mode.value != "analysis_comparison":
|
||||
return units, False
|
||||
run = await AnalysisRepository.get_run(postgres_conn, query.run_id)
|
||||
if run is None:
|
||||
raise ValueError(f"analysis run does not exist: {query.run_id}")
|
||||
recorded = (run.get("parameters") or {}).get("result_units")
|
||||
return (recorded, False) if isinstance(recorded, dict) else (units, True)
|
||||
|
||||
|
||||
async def _query_page(timescale_conn, postgres_conn, query, selected_devices, limit, offset):
|
||||
descriptors = []
|
||||
series = []
|
||||
@@ -66,16 +93,14 @@ async def _query_page(timescale_conn, postgres_conn, query, selected_devices, li
|
||||
"source": source, "element_type": element_type,
|
||||
"id": device_id if source == "scada" else element_id, "field": field,
|
||||
})
|
||||
metric = _target_metric(element_type)
|
||||
series.append(ElementHistorySeries(
|
||||
series.append(_history_series(
|
||||
element_id=element_id, element_type=element_type, device_id=device_id,
|
||||
metric=metric, source=(
|
||||
source=(
|
||||
HistorySource.SCADA_RAW if field == "monitored_value" else HistorySource.SCADA_CLEANED
|
||||
) if source == "scada" else (
|
||||
HistorySource.ANALYSIS_SIMULATION if source == "analysis" else HistorySource.REALTIME_SIMULATION
|
||||
),
|
||||
source_unit=unit, display_unit=display_unit_for_metric(metric.value),
|
||||
unit_inferred=unit_inferred, points=[],
|
||||
source_unit=unit, unit_inferred=unit_inferred,
|
||||
))
|
||||
|
||||
# Stable series ordering makes a page crossing raw/cleaned/model boundaries repeatable.
|
||||
@@ -84,17 +109,8 @@ async def _query_page(timescale_conn, postgres_conn, query, selected_devices, li
|
||||
add(source="scada", element_type=element_type, element_id=element_id,
|
||||
device_id=device_id, field=field, unit=device["measurement_unit"])
|
||||
if query.mode.value != "observed":
|
||||
units = await NetworkSettingsRepository.get_result_units(postgres_conn)
|
||||
units, inferred = await _resolve_result_units(postgres_conn, query)
|
||||
source = "analysis" if query.mode.value == "analysis_comparison" else "realtime"
|
||||
inferred = False
|
||||
if source == "analysis":
|
||||
run = await AnalysisRepository.get_run(postgres_conn, query.run_id)
|
||||
if run is None:
|
||||
raise ValueError(f"analysis run does not exist: {query.run_id}")
|
||||
recorded = (run.get("parameters") or {}).get("result_units")
|
||||
inferred = not isinstance(recorded, dict)
|
||||
if not inferred:
|
||||
units = recorded
|
||||
targets = {(t.element_type, t.element_id) for t in query.elements}
|
||||
for element_type, element_id in sorted(targets):
|
||||
metric = _target_metric(element_type)
|
||||
@@ -202,21 +218,18 @@ class TimeseriesHistoryService:
|
||||
rows = scada_by_device.get(device_id, [])
|
||||
if not rows:
|
||||
continue
|
||||
metric = _target_metric(element_type)
|
||||
unit = device["measurement_unit"]
|
||||
for source, field in (
|
||||
(HistorySource.SCADA_RAW, "monitored_value"),
|
||||
(HistorySource.SCADA_CLEANED, "cleaned_value"),
|
||||
):
|
||||
series.append(
|
||||
ElementHistorySeries(
|
||||
_history_series(
|
||||
element_id=element_id,
|
||||
element_type=element_type,
|
||||
device_id=device_id,
|
||||
metric=metric,
|
||||
source=source,
|
||||
source_unit=unit,
|
||||
display_unit=display_unit_for_metric(metric.value),
|
||||
points=_points(
|
||||
rows,
|
||||
value_field=field,
|
||||
@@ -227,15 +240,8 @@ class TimeseriesHistoryService:
|
||||
if query.mode.value == "observed":
|
||||
return ElementHistoryResponse(series=series)
|
||||
|
||||
current_units = await NetworkSettingsRepository.get_result_units(postgres_conn)
|
||||
result_units, unit_inferred = await _resolve_result_units(postgres_conn, query)
|
||||
if query.mode.value == "analysis_comparison":
|
||||
run = await AnalysisRepository.get_run(postgres_conn, query.run_id)
|
||||
if run is None:
|
||||
raise ValueError(f"analysis run does not exist: {query.run_id}")
|
||||
parameters = run.get("parameters") or {}
|
||||
recorded_units = parameters.get("result_units")
|
||||
unit_inferred = not isinstance(recorded_units, dict)
|
||||
result_units = recorded_units if not unit_inferred else current_units
|
||||
link_values = await AnalysisResultsRepository.get_series_by_ids(
|
||||
timescale_conn,
|
||||
query.run_id,
|
||||
@@ -256,8 +262,6 @@ class TimeseriesHistoryService:
|
||||
)
|
||||
source = HistorySource.ANALYSIS_SIMULATION
|
||||
else:
|
||||
unit_inferred = False
|
||||
result_units = current_units
|
||||
link_values = await RealtimeRepository.get_link_fields_by_ids_time_range(
|
||||
timescale_conn,
|
||||
query.start_time,
|
||||
@@ -285,13 +289,11 @@ class TimeseriesHistoryService:
|
||||
continue
|
||||
source_unit = str(result_units.get(metric.value, "")).strip()
|
||||
series.append(
|
||||
ElementHistorySeries(
|
||||
_history_series(
|
||||
element_id=target.element_id,
|
||||
element_type=target.element_type,
|
||||
metric=metric,
|
||||
source=source,
|
||||
source_unit=source_unit,
|
||||
display_unit=display_unit_for_metric(metric.value),
|
||||
unit_inferred=unit_inferred,
|
||||
points=_points(
|
||||
rows,
|
||||
|
||||
@@ -310,9 +310,18 @@
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar"
|
||||
"bar",
|
||||
"time_comparison"
|
||||
]
|
||||
},
|
||||
"x_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"y_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"x_data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
@@ -331,10 +340,43 @@
|
||||
"type": "string",
|
||||
"maxLength": 200
|
||||
},
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar",
|
||||
"scatter"
|
||||
]
|
||||
},
|
||||
"data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "number"
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
},
|
||||
"sample_times": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"contracts": {
|
||||
"server": {
|
||||
"file": "server-v1.openapi.json",
|
||||
"sha256": "b039042be13243a88691309cb74eee105a3b71765c342edb9222374e41c0458e"
|
||||
"sha256": "d96df154b86b797d0db913d1e69cf335d92d34dd119f800c9346b90c6119c9bb"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1687,6 +1687,18 @@
|
||||
],
|
||||
"title": "Location"
|
||||
},
|
||||
"occurred_at": {
|
||||
"anyOf": [
|
||||
{
|
||||
"format": "date-time",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"title": "Occurred At"
|
||||
},
|
||||
"priority": {
|
||||
"anyOf": [
|
||||
{
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -207,13 +207,113 @@
|
||||
"type": "string",
|
||||
"maxLength": 200
|
||||
},
|
||||
"scada": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"role": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"current",
|
||||
"comparison"
|
||||
]
|
||||
},
|
||||
"metric": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"pressure",
|
||||
"flow"
|
||||
]
|
||||
},
|
||||
"unit": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128
|
||||
},
|
||||
"window_start": {
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
"window_end": {
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
"value_field": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"monitored_value",
|
||||
"cleaned_value"
|
||||
]
|
||||
},
|
||||
"gaps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"device_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128
|
||||
},
|
||||
"day_offset": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 7
|
||||
},
|
||||
"missing_count": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 100
|
||||
},
|
||||
"reason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"no_readings",
|
||||
"outside_frozen_scope",
|
||||
"outside_frozen_window",
|
||||
"no_scheduled_samples"
|
||||
]
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"device_id",
|
||||
"day_offset",
|
||||
"missing_count",
|
||||
"reason"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"maxItems": 8
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"role",
|
||||
"metric",
|
||||
"unit",
|
||||
"window_start",
|
||||
"window_end"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"chart_type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar"
|
||||
"bar",
|
||||
"time_comparison"
|
||||
]
|
||||
},
|
||||
"x_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"y_axis_name": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"x_data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
@@ -232,10 +332,53 @@
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
},
|
||||
"device_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 128
|
||||
},
|
||||
"day_offset": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 7
|
||||
},
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"line",
|
||||
"bar",
|
||||
"scatter"
|
||||
]
|
||||
},
|
||||
"data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "number"
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
},
|
||||
"sample_times": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"maxLength": 64,
|
||||
"format": "date-time",
|
||||
"pattern": "^(?:[1-9][0-9]{3}|0[1-9][0-9]{2}|00[1-9][0-9]|000[1-9])-[0-9]{2}-[0-9]{2}T(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 100
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# 供水 Server 文档目录
|
||||
|
||||
环境、启动和发布命令见 [仓库 README](../README.md),开发规则见 [AGENTS.md](../AGENTS.md)。整理日期:2026-11-10。下列文档按职责组织,不将历史迁移或本地验证当作现有服务状态。
|
||||
|
||||
## API、认证与数据
|
||||
|
||||
| 主题 | 维护入口 |
|
||||
| --- | --- |
|
||||
| API 命名、字段和生成契约 | [API 风格](api-style.md) · [OpenAPI](../contracts/server-v1.openapi.json) |
|
||||
| 身份、项目成员、管理接口与元数据 | [认证与用户管理](../AUTHENTICATION_AND_USER_MANAGEMENT.md) |
|
||||
| MCP 设计依据、启用与认证配置 | [查询设计](mcp-query-design.md) · [实施与启用](mcp-query-implementation.md) · [Keycloak 配置](mcp-keycloak-setup.md) |
|
||||
| MCP 分析、项目服务和后台计算 | [分析工具](mcp-analysis-tools.md) · [项目服务](mcp-project-services.md) · [计算队列](calculation-queue.md) |
|
||||
| 项目地名与空间覆盖 | [地名搜索](project-place-search.md) · [节点覆盖](node-coverage-tool.md) |
|
||||
| 项目 BizDB 与时序模型 | [数据库架构](../resources/db_v2/DATABASE_ARCHITECTURE.md) · [管网结构](../resources/db_v2/WNDB_STRUCTURE.md) |
|
||||
|
||||
工具目录以 `app/mcp/catalog.py` 为准;启用范围、项目权限和会话用途决定实际可用项,不以文档中的历史工具数量推定。
|
||||
|
||||
## 任务、调查与工作台
|
||||
|
||||
| 主题 | 维护入口 |
|
||||
| --- | --- |
|
||||
| 任务类型、调查轮次与协作 | [工况任务契约](operational-tasks.md) |
|
||||
| Worker、租约、持久消息、外部动作和部署 | [工况工作进程](operational-task-workers.md) |
|
||||
| 进展、展示事件与人工反馈续调 | [调查进展与展示](investigation-progress-and-presentation.md) |
|
||||
| 正式报告与稳定引用 | [调查报告契约](investigation-report-contract.md) |
|
||||
| 爆管工单到处置闭环 | [爆管处置路线](burst-response-workflow.md) |
|
||||
| 工作台运行态、外围工单与告警存储 | [运行态与存储](workbench-runtime.md) |
|
||||
| 事件顺序、SSE 与读取去重 | [工作台事件链路](workbench-event-refinement.md) |
|
||||
| 通知进入、退出与生命周期 | [通知路由](operational-notification-routing.md) |
|
||||
| 迁移门禁与逐项目验证 | [数据库变更门禁](workbench-database-rollout.md) |
|
||||
| 外围工单来源、项目路由和主动同步 | [外围服务](peripheral-sample.md) |
|
||||
|
||||
[旧动态计划协议](dynamic-investigation-plan.md) 用于兼容历史计划,当前任务和报告规则以上述专门契约为准。[15173 UI Keycloak 客户端](integration-ui-keycloak-client.md) 是特定联调环境说明,不能作为全部部署环境的通用配置。
|
||||
|
||||
## SCADA、回放与数据集
|
||||
|
||||
| 主题 | 维护入口 |
|
||||
| --- | --- |
|
||||
| 回放、时间轴、周期监测与调查边界 | [SCADA 回放设计与实现](scada-peripheral-replay-design.md) |
|
||||
| 外围测试样本与数据再现 | [合成样本](peripheral-generated-data.md) |
|
||||
| 凤阳历史爆管模拟 | [凤阳 SCADA 复制记录](fengyang-burst-scada.md) |
|
||||
| 奉贤独立 Agent benchmark | [tjwater_v2 benchmark](tjwater-v2-burst-benchmark.md) |
|
||||
| tjwater2 场景选取与观测 | [多距离与弱信号方案](tjwater2-v2-burst-balanced-design.md) |
|
||||
| tjwater2 水力校准方法 | [爆管模拟校准](tjwater2-v2-burst-simulation-design.md) |
|
||||
| 监测点算法与资源压力取舍 | [灵敏度优化](sensor-sensitivity-optimization.md) |
|
||||
|
||||
这些数据文档分别描述数据准备、算法设计或历史试算,不能互相替代,也不证明目标数据库已经加载或真实 Agent 已完成验收。
|
||||
|
||||
## 合并与历史资料
|
||||
|
||||
- `workbench-operations-storage.md` 已并入运行态文档的“工单与告警存储”。
|
||||
- `peripheral-work-order-events.md` 已并入外围服务文档的“工单事件主动同步”,旧固定库限制已按项目路由实现修订。
|
||||
- [历史命名审查](history/backend-naming-audit.md) 与 [失效接口排查](history/retired-api-investigation.md) 保留原始审查依据,不能作为当前 API 目录。
|
||||
- `scripts/missing_api_endpoints.md` 是历史能力盘点,新增能力前需重新核对当前契约与调用方。
|
||||
|
||||
设计与部署说明保留分工:设计解释选择与边界,运行文档给出可执行配置;不同数据库、不同数据集和不同生命周期的业务对象不为减少文件数而合并。
|
||||
+2
-1
@@ -1,7 +1,8 @@
|
||||
# TJWater REST API v1
|
||||
|
||||
This contract is the public API contract for coordinated TJWater
|
||||
Server, Agent, Frontend, and CLI releases.
|
||||
Server, Agent, and Frontend releases. Legacy CLI material is a historical
|
||||
capability reference, not a current runtime integration.
|
||||
|
||||
- Paths use lowercase kebab-case, have no trailing slash, and identify
|
||||
resources rather than handler actions.
|
||||
|
||||
+43
-225
@@ -1,248 +1,66 @@
|
||||
# 从爆管上报到处置闭环
|
||||
# 从爆管工单到事件处置闭环
|
||||
|
||||
本文定义爆管调查的完整业务路线,供维护者审阅和验收。运行时指令位于 [burst-report/SKILL.md](../resources/work-order-skills/burst-report/SKILL.md),由 Server 根据绑定任务的 `workflow=burst_report` 经 MCP 分发。
|
||||
工单触发的 `workflow=burst_report` 使用 [爆管事件处理 Skill](../resources/work-order-skills/burst-event-handling/SKILL.md)。Server 按持久化来源选择流程,Agent 接收同一任务的核验上下文;SCADA 异常和历史回放使用 [SCADA Skill](../resources/work-order-skills/scada-anomaly/SKILL.md)。
|
||||
|
||||
重点成果是有依据的事故候选位置、隔离阀门清单和影响范围,以及经审批执行、反馈核验后的闭环记录。Agent 根据证据选择、跳过或重做分支,不按图逐节点机械调用工具。工程图覆盖多轮调查和人工业务;每轮发布的进度图只标注实际进行的工作,不复制整图冒充执行记录。
|
||||
|
||||
## 完整工程图
|
||||
## 固定处理路线
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph intake[工单进入与后台任务]
|
||||
order[用户或外围系统提交爆管上报工单]
|
||||
source[保存原工单及来源版本、发生时间、位置和描述]
|
||||
sync[同步器按来源关联调查及工单投影]
|
||||
queue[Server 保存调查任务和 queued run]
|
||||
claim[Worker 投递,Agent 领取并维持租约]
|
||||
context[读取任务摘要、业务 Skill、历史报告和既有动作]
|
||||
order --> source --> sync --> queue --> claim --> context
|
||||
end
|
||||
|
||||
subgraph investigation[Agent 调查与工程分析]
|
||||
scope[确定事件时刻、地点、调查目标和真实或演练来源]
|
||||
location[核对已有资产编号;必要时解析地点和查询附近管段]
|
||||
monitor[核对相关测点映射、异常窗口、正常对照和数据质量]
|
||||
cover{查询是否实际覆盖事件与对照窗口}
|
||||
more[缩小对象和窗口,按需补页,核对缺测及零值]
|
||||
hypothesis{是否具备定位所需观测和漏水量依据}
|
||||
locate[计算爆管候选排序,保留假设、覆盖范围与证据]
|
||||
candidate[整合上报、资产关联、算法与现场反馈得到候选集合]
|
||||
usable{是否存在可用于隔离评估的候选资产}
|
||||
isolation[逐候选分析拓扑隔离边界、必关阀及受影响节点]
|
||||
isolateok{可隔离且阀门可用性有可核验依据}
|
||||
alternate[纳入已知不可关闭阀门,重新评估替代隔离方案]
|
||||
hydraulic[按需模拟候选关阀组合的压力和流量影响]
|
||||
impact[区分拓扑停水区、水力影响区和有资料支持的用户范围]
|
||||
findings[形成候选位置、阀门清单、影响对比和建议方案]
|
||||
fieldbasis{是否已有足以支持具体现场核验的结论}
|
||||
fielddraft[形成有位置、核验目标及回执要求的现场核验建议]
|
||||
gap[真实关键数据缺失:发布 final insufficient_data 并结束本轮]
|
||||
error[接口、权限或配置异常:记录具体失败并由宿主结束失败运行]
|
||||
scope --> location --> candidate
|
||||
scope --> monitor --> cover
|
||||
cover -->|未覆盖且可补齐| more --> monitor
|
||||
cover -->|已覆盖或确认实际缺失| hypothesis
|
||||
hypothesis -->|具备依据| locate --> candidate
|
||||
hypothesis -->|条件不具备,明确原因| candidate
|
||||
candidate --> usable
|
||||
usable -->|候选有效,标明待现场确认| isolation --> isolateok
|
||||
isolateok -->|有可评估方案| hydraulic --> impact --> findings
|
||||
isolateok -->|已知存在不可关阀| alternate --> isolation
|
||||
isolateok -->|无法形成方案| fieldbasis
|
||||
usable -->|尚无候选| fieldbasis
|
||||
fieldbasis -->|可形成具体核验建议| fielddraft
|
||||
fieldbasis -->|当前判断所需证据确实缺失| gap
|
||||
monitor -->|不可恢复的查询异常| error
|
||||
location -->|不可恢复的必要查询异常| error
|
||||
end
|
||||
|
||||
subgraph approval[报告、审批与现场处置]
|
||||
interim[保存本轮 interim concluded 阶段报告]
|
||||
proposals[创建具体待审批草稿,关联证据和执行前置条件]
|
||||
finalround[保存本轮 final 报告,由宿主结束本轮 run]
|
||||
review[有权限人员审核当前草稿版本]
|
||||
decision{审批结果}
|
||||
revise[驳回理由保存并触发原调查新一轮]
|
||||
submit[Server 通过配置的适配器幂等提交]
|
||||
receipt{外部提交结果}
|
||||
reconcile[使用同一执行请求号核对回执,不重复下发]
|
||||
unsupported[未配置或明确失败:记录原因并重新分析]
|
||||
workkind{本次获批的是核验还是处置}
|
||||
fieldcheck[现场核验准确位置、漏水迹象及资产,回传实际记录]
|
||||
notice[人工实施获批的通知、供水保障和应急安排]
|
||||
isolatefield[现场核验阀门及实际隔离结果]
|
||||
repair[现场定位、止水和抢修,回传实际故障管段及修复记录]
|
||||
clean[按现场风险及获批方案清洗、消毒或冲洗]
|
||||
restore[核验修复与适用复测,按批准方案恢复供水]
|
||||
follow[记录恢复后压力流量、漏水复查及用户回访]
|
||||
findings --> interim
|
||||
fielddraft --> interim
|
||||
interim --> proposals --> finalround --> review --> decision
|
||||
decision -->|驳回| revise
|
||||
decision -->|批准| submit --> receipt
|
||||
receipt -->|结果不明或仍在执行| reconcile --> receipt
|
||||
receipt -->|明确失败或不支持| unsupported
|
||||
receipt -->|工单已创建或设备动作有回执| workkind
|
||||
workkind -->|现场核验事项| fieldcheck
|
||||
workkind -->|有依据的处置事项| notice
|
||||
notice --> isolatefield --> repair --> clean --> restore --> follow
|
||||
end
|
||||
|
||||
subgraph closure[现场反馈、复核与原事件闭环]
|
||||
feedback[外围系统保存关联工单的新反馈和单调版本]
|
||||
requeue[Server 校验来源并为原调查排队新 generation]
|
||||
reread[重新读取现场事实、执行回执和最新监测证据]
|
||||
verify[对照预测与现场结果,复核修复、恢复和风险]
|
||||
faultstate{复核结果支持故障、误报还是尚待核验}
|
||||
falsealarm[核对排除故障的证据及人工确认]
|
||||
closed{处置及闭环证据是否齐全}
|
||||
remaining[列出未完成事项及对应已有或待审批动作]
|
||||
sourceclosed{原上报工单已结案且版本已核对}
|
||||
readyreport[发布具备结案条件报告并结束本轮]
|
||||
closedreport[发布闭环复核 final 报告并结束本轮]
|
||||
closeowner[有权限人员在原工单系统核对并完成原事件结案]
|
||||
closedcase[报告与来源事实一致,确认业务闭环]
|
||||
follow --> feedback --> requeue
|
||||
fieldcheck --> feedback
|
||||
reread --> verify --> faultstate
|
||||
faultstate -->|已确认故障| closed
|
||||
faultstate -->|有依据排除故障| falsealarm --> sourceclosed
|
||||
faultstate -->|尚需核验| remaining
|
||||
closed -->|尚未齐全或复测异常| remaining --> interim
|
||||
closed -->|证据齐全| sourceclosed
|
||||
sourceclosed -->|是| closedreport --> closedcase
|
||||
sourceclosed -->|否| readyreport --> closeowner --> feedback
|
||||
reread -->|现场位置或阀门条件改变| candidate
|
||||
end
|
||||
|
||||
subgraph presentation[贯穿调查的地图、图表与数据整理]
|
||||
progress[发布真实调查进展和 Mermaid 状态]
|
||||
map[高亮候选管段、闪烁建议关阀、展示已有证据覆盖层]
|
||||
chart[打开异常与对照曲线,展示候选及影响对比表]
|
||||
filepermit{会话是否已有文件写入权限}
|
||||
jsonl[按权限整理有来源引用的 JSONL 调查数据]
|
||||
reportui[通过报告 ui_blocks 发布 json-render 结构化界面]
|
||||
sourceview[展示保持证据来源,失败不阻断业务调查]
|
||||
progress --> map --> chart --> filepermit
|
||||
filepermit -->|已获准且需要导出| jsonl --> reportui
|
||||
filepermit -->|未获准或无需导出| reportui
|
||||
reportui --> sourceview
|
||||
end
|
||||
|
||||
context -->|首次或需要重新分析| scope
|
||||
context -->|已有新反馈待复核| reread
|
||||
revise --> requeue
|
||||
unsupported --> requeue
|
||||
requeue --> queue
|
||||
gap -->|补充事实被 Server 接受后才重开| requeue
|
||||
findings -->|无须新增动作| finalonly[无需新增动作时保存 final 报告并结束本轮]
|
||||
scope -.-> progress
|
||||
candidate -.-> map
|
||||
impact -.-> chart
|
||||
verify -.-> reportui
|
||||
flowchart TD
|
||||
order[爆管上报工单] --> context[读取原工单与当前任务]
|
||||
context --> asset[核对地点并关联管网资产]
|
||||
asset --> isolation[分析隔离边界与阀门方案]
|
||||
isolation --> impact[说明事件与隔离方案影响]
|
||||
impact --> report[保存阶段报告与处置建议]
|
||||
report --> approval[人工审批与现场执行]
|
||||
approval --> feedback[接收实际处置与恢复反馈]
|
||||
feedback --> verify[复核影响与结案条件]
|
||||
verify --> close[Server 完成任务并解除告警]
|
||||
verify -->|仍异常或有未完成处置| asset
|
||||
```
|
||||
|
||||
图中的现场处置由实际审批、工单与回执推进;顺序按现场紧迫性和获批方案调整。必要的紧急通知、现场核验和供水保障不应等待可选的完整模型计算。`clean` 为按风险适用的步骤,不能把每次爆管都判断为污染。`hydraulic` 在缺少模型、时序输入或权限时记录具体限制,仍可保留已完成的拓扑隔离结果。
|
||||
1. **接收核对**:保留来源工单ID、单调版本、发生时间、位置、描述、现场反馈和已有动作;后续轮次从新增事实接续。
|
||||
2. **资产关联**:优先核对上报资产编号。坐标或地址通过地理编码、附近资产、属性和连接关系形成有依据的关联;位置歧义列为具体待确认项。
|
||||
3. **隔离方案**:用 analyze_valve_isolation 分析关联资产,保留必关阀、可选阀、直接隔离与供水失源节点;已知失效阀门用于替代方案分析,现场可操作性列为执行条件。
|
||||
4. **影响说明**:先展示拓扑范围,再按可用SCADA、水力结果和用户关联资料解释事件当前影响、方案预计影响和保障需求。
|
||||
5. **报告与处置**:保存阶段报告后提出核验、通知、保障、抢修和恢复等动作草稿;关联现场任务采用 general 工单。审批、派发和现场完成分别核对回执。
|
||||
6. **反馈复核**:已派发的关联工单保持 `waiting_work_order`,全部收到有效 `completed` 反馈后进入 `waiting_scada`。以现场反馈业务时间为 T,取严格晚于 T 的最近一个 SCADA 采样时刻进行复核;正常范围使用事件前15天同采样相位的正常包络。数据未到或仍异常时保持等待,后台工作进程定期复核;通过后持久保存恢复证据并接续 Agent。Agent 最终报告必须引用该证据,运行完成后才能完成任务并解除对应告警。现场资料缺项仅限定未核实细节,不新增本任务的关闭门槛。来源系统另有明确行政结案流程时,按其实际状态另记。
|
||||
|
||||
`requeue` 表示 Server 接受了新业务事实后分配下一轮,不是 Agent 复活旧会话或自行循环等待。原工单结案由原系统有权限人员处理,后台 Agent 目前没有直接关闭原工单的工具。`completed`、`concluded`、动作 `succeeded` 和“业务闭环”分别核对,不能互相推导。
|
||||
SCADA作为事件影响的辅助证据,按目标测点和事件/对照窗口读取。具备水力输入且工具已授权时,使用 analyze_valve_closure 评估给定阀门方案;各步骤记录实际结果、适用范围或具体待补项。
|
||||
|
||||
## 将业务步骤映射为现有能力
|
||||
## 报告固定内容
|
||||
|
||||
以下产品查询与计算均使用 `mcp_tjwater_` 前缀。参数以本轮实际发现的工具 schema 为准。
|
||||
报告包含事件概况、地点与资产关联、隔离方案、影响说明、处置建议和待办。摘要以150字以内为目标,资产和测点明细放表格,完整依据引用证据。
|
||||
|
||||
| 业务动作 | 已有入口 | 必须形成的结果 |
|
||||
| --- | --- | --- |
|
||||
| 接续原任务 | `get_task_context`,依次读取 summary、skill 及所需分节;`get_task_run_history` | 原工单来源、事件时刻、调查代次、流程版本、既有动作与现场反馈 |
|
||||
| 查找地点和资产 | `geocode` / `search_project_places`、`query_nearby_elements`、`get_element_properties`、`get_node_connections` | 实际候选管段或节点、位置来源、匹配理由;邻近不代表故障确认 |
|
||||
| 核对监测 | `list_scada_devices`、`get_scada_device`、`query_scada_readings`、`query_element_history` | 设备和模型节点的映射、异常及对照窗口、单位、缺测、分页覆盖 |
|
||||
| 压力异常侦测 | `run_burst_detection` | 至少 5 个完整压力测点及此前 15 天数据满足时才调用;不把侦测当作定位 |
|
||||
| 爆管候选定位 | `run_burst_location` | 候选排序、算法实际返回的评价值、观测范围、漏水量假设及来源 |
|
||||
| 逐候选隔离 | `analyze_valve_isolation` | `accident_elements`、`isolatable`、`must_close_valves`、`optional_valves`、`affected_nodes`、`affected_node_count` 及已知 `disabled_valves` |
|
||||
| 水力影响 | `analyze_valve_closure` | 指定关阀组合、时间窗、输出对象和模拟压力流量;参数不足时保留明确原因 |
|
||||
| 用户影响 | 已授权资产属性、区域及外围资料查询 | 受影响对象、关联依据和覆盖限制;没有用户资料不估造户数 |
|
||||
| 发布和提议 | `publish_task_progress`、`publish_task_report`、`propose_task_action` | 持久化报告、证据引用和待审批动作;动作不是执行事实 |
|
||||
| 反馈接续 | Server `external-feedback` / `human-evidence` 及 worker | 同一任务的新轮次、真实来源版本、修复和复测证据 |
|
||||
“影响说明”逐项保留时间、对象、单位、范围、来源与实际或预计的性质:
|
||||
|
||||
定位的 `burst_leakage` 必须有依据,单位为 m³/s;`pressure_scada_ids` 是模型节点 ID,不能填设备查询编号,也可省略并使用服务的项目配置。monitoring 模式正常窗可成对省略,使用前一天同窗,但这只是一项对照假设,仍要核对数据有效性。simulation 模式必须引用已有分析运行 UUID,不把在线重算当成已有分析运行。已知事故管段可直接做候选隔离;不必先完成一次没有必要的定位计算,也不能把已知答案作为算法独立定位成功的证明。
|
||||
| 影响项 | 依据与输出 |
|
||||
| --- | --- |
|
||||
| 事件当前影响 | 已查询测点的压力、流量、持续时间和恢复情况 |
|
||||
| 直接隔离范围 | affected_nodes及对应计数、地图覆盖 |
|
||||
| 新增供水失源范围 | supply_affected_nodes及对应计数、地图覆盖 |
|
||||
| 水力影响 | 给定阀门组合、起算时间、模型条件下的压力和流量结果 |
|
||||
| 用户与保障影响 | 有资料关联的用户、重点设施、保障需求;缺少关联时报告节点与区域范围 |
|
||||
|
||||
隔离工具支持明确标注的候选管段,无须等到现场确认才评估方案。多个候选的比较优先分开调用;把多个 ID 一次传入表示联合事故场景,不等同于逐个比较。`disabled_valves` 仅填已有事实支持的不可关阀;空数组不证明现场阀门均可用。`isolatable=false` 不能解释为零影响,即使 `affected_nodes` 为空也须解释返回的计数和限制。
|
||||
变化量基于同口径对象、单位、时间与模型条件计算;现场实际、模型预测和空间示意分别标注。缺少可选模拟参数时,继续形成已有依据的隔离与保障建议。
|
||||
|
||||
## 像调查员一样查询、查看、比较和复核
|
||||
## 工具与入口约束
|
||||
|
||||
每一步先明确待回答的问题,再查小范围资料,观察实际结果后决定下一步。例如:核实地址,查附近管段并查看属性,关联压力点,比较事件前后及正常对照,检查候选,评估关阀影响。只把必要的查询、事实和判断依据写入进度及报告,不保存模型内部推理。
|
||||
工单事件从上报位置开展资产关联。Agent会话策略隐藏 run_burst_location,Server在绑定与后续解析时同步移除其调用能力;普通聊天和SCADA候选定位沿用各自策略。持久计算在执行前重新校验任务绑定。
|
||||
|
||||
- 不满足于“查过 SCADA”:必须确认返回的时间实际覆盖事件。查询 24 小时而只返回当天前 45 分钟时,应缩到事件前后的小窗,按需补页,不直接进入最终判断。
|
||||
- 正常基线、实测、模型模拟、外围演练和人工反馈分别标注;缺测不是零值,异常不是爆管确认,候选评分不是校准后的概率。
|
||||
- 查不到事故管段时先检查地址解析、附近资产与上下游关联;能查询的字段先查询,不能直接全部交给现场。
|
||||
- 既有证据通过 `get_task_evidence` 的有界分页读取,不重复拉取同一批事实。失败、权限拒绝和存储未配置不当作“无异常”或“无数据”。
|
||||
- 可以用最多两个前台 `investigation-reader` 并行核对无依赖的问题。读取、计算、发布和文件权限仍按角色执行;reader 只返回查询事实及真实证据编号。
|
||||
历史 burst_report 任务携带 replay_context 时,Server分发scada-anomaly,时序查询保持 knowledge_cutoff 与 batch_sequence,demo能力提示按冻结窗口返回。
|
||||
|
||||
## 地图与图表操作
|
||||
业务阶段和工单、返回时间、复核回执保存在任务文档 `handling_context` JSON 中,不增加数据库列。外围样板更新工单时可提供带时区的 `occurred_at`,同时必须填写现场反馈;该时间写入活动记录,系统的 `updated_at` 仍是实际记录时间。同步器将现场反馈时间传给调查,避免历史演示误用电脑当前时间。
|
||||
|
||||
后台调查统一调用 `publish_task_presentation`,每个不同命令使用不同 UUID `request_id`,相同命令重试复用原编号。以下为工具已有能力,不需要浏览器自动化。
|
||||
预约时间 `scheduled_at` 可在派单前的工单草稿和已创建工单的编辑表单中调整。实际完工反馈时间与预约安排分别记录;Agent 以实际反馈时间为恢复复核起点。新的有效现场反馈修订完工时间后,旧复核证据保留为历史记录,当前复核起点更新,重新读取严格晚于新反馈时间的最近一个 SCADA 采样时刻。只修改预约安排或其他工单信息,不改写已有现场反馈时间。
|
||||
|
||||
| 调查进展 | command.action | 取值及依据 |
|
||||
| --- | --- | --- |
|
||||
| 找到实际候选资产 | `highlight_features` | `layer=pipes` / `junctions`,`feature_ids` 为已核实的地图要素编号,`effect=steady` |
|
||||
| 正在核查候选或建议关阀 | `highlight_features` | 对应 pipes 或 valves,`effect=pulse`;闪烁只表示调查关注,不代表设备正在操作 |
|
||||
| 查看异常与对照 | `open_chart` | 有限长度的时间和值,引用真实 `evidence_refs`;不同日期的对照对齐方式在标题或来源中说明 |
|
||||
| 查看已有空间结果 | `set_overlay` | 提供 `overlay_id/title/mode/evidence_id`,Server 从证据提取几何,不接收手绘影响区 |
|
||||
| 撤下旧的临时展示 | `close_chart` / `clear_overlay` / `clear_highlights` | 清理自己记录的临时展示;清理失败不修改业务结果 |
|
||||
`POST /operational-tasks/recovery/reconcile` 由现有 operational worker 每轮调用,按项目及任务创建边界处理等待复核的任务;SCADA 读取失败不阻断其他调查派发。复核直接读取项目时序库中完工反馈时间之后最近时刻的 SCADA,不依赖回放状态或推进回放时间。
|
||||
|
||||
支持的图层为 junctions、pipes、valves、pumps、tanks、reservoirs、scada。先核实业务 ID 与地图 feature ID 的映射,不能把设备编号、节点编号与 GIS 编号混用。每次高亮最多 100 个要素,需要更多时说明所显示的子集;不要假定多次命令会累积成完整范围。没有几何时可以按已知 ID 高亮、用表格解释结果,不画包络多边形冒充影响范围。
|
||||
这条恢复链路同样跟踪带 `replay_context` 的调查工单。原异常快照的 `knowledge_cutoff` 和 `batch_sequence` 保留,普通回放查询继续受其限制;工单完工反馈单独触发 Server 的 `ScadaRepository.get_recovery_scada_readings()`,以实际完工业务时间为 T,读取严格晚于 T 的最近一帧。例如 2026-07-16 13:00 完工,15 分钟采样时复核 13:15,而非原异常截止的 08:00。Agent 从 `handling_context.verification_evidence_id` 读取保存的恢复证据,并与原异常证据共同引用,不需放开回放查询或重建复测工单。
|
||||
|
||||
图表最多 100 个横轴值和 8 组数值序列,单位须在系列名称或标题中明确。不能把 null 缺测填零以通过 schema;可以按有数据的区间分图,并保留缺测说明。`recorded` 仅表示请求保存。浏览器未开 Dev 跟随时仍继续调查,不能等待地图显示回执。后台接口目前没有摄像机缩放、平移、鼠标点击或任意图层开关命令;不要调用仅供聊天的 `locate_features`、`zoom_to_map` 或原生浏览器工具替代。
|
||||
已有调查在收到新的有效完工反馈时会补登记成功派发的关联工单。旧回报误用了电脑当前时间时,需在来源工单更正现场反馈业务时间并提交新版本;系统不猜测完工时刻,也不改写原报告或旧证据。
|
||||
|
||||
## JSONL 调查数据与 json-render 展示
|
||||
## 验证
|
||||
|
||||
JSONL 用于整理已经取得的数据与分析结果,不补造监测、阀门、现场反馈或审批。推荐每行一个对象,记录 `record_type/origin/source_evidence_refs`,并按对象保留时间窗、单位、模型或假设、覆盖限制和实际结果。`origin` 沿用来源分类;本地派生记录引用输入证据,并说明转换方法。不要把 JSONL 文件自身当作 Server 已保存的业务证据。
|
||||
检查Skill名称、版本、摘要与正文digest一致、内联体积、角色权限、工单工具隔离和SCADA定位回归。Agent类型检查与会话工具策略测试共同覆盖运行时入口。发布后分别验收工单事件处理报告和SCADA候选定位,原有报告按既有修订保留。
|
||||
|
||||
按内容选择 `event_context`、`monitoring_coverage`、`candidate`、`isolation`、`hydraulic_impact`、`field_feedback`、`closure_check`,无需为没有结果的类别制造空壳文件。候选记录写实际排序及评价值;隔离记录保留建议必关阀、可选阀、已知不可关阀及受影响节点;闭环记录逐项保留已核验、待核验或不适用及原因。JSONL 是导出约定,不新增 MCP 输入字段或服务端数据协议。
|
||||
|
||||
以下两行只演示文件格式,`synthetic`、`example_only` 与空证据引用表示它们不可用于实际报告或审批,不包含真实分析结果:
|
||||
|
||||
```jsonl
|
||||
{"record_type":"candidate","origin":"synthetic","example_only":true,"source_evidence_refs":[],"element_id":"DEMO-PIPE-01","rank":null,"evaluation_value":null,"status":"not_computed","limitations":["仅说明字段,未执行定位"]}
|
||||
{"record_type":"closure_check","origin":"synthetic","example_only":true,"source_evidence_refs":[],"item":"repair_verified","status":"not_verified","observed_at":null,"limitations":["无现场反馈,不能声称闭环"]}
|
||||
```
|
||||
|
||||
如果当前会话已获准使用原生文件写入,则在当前 workspace 下写 `files/burst-investigation-<generation>.jsonl`,保留原始证据不变;通过允许的读取工具逐行复核 JSON、行数、对象 ID、时间和单位。只引用宿主绑定的 generation,不能从用户文本自选。不得读取兄弟会话、业务数据库、宿主源码或凭据;需要本地转换时只使用已获准的原生工具及沙箱 Shell,不引入 HTTP、SQL 或自由命令查询桥。
|
||||
|
||||
当前没有专用的免审批 JSONL 导出工具。文件权限未授予时,不通过 Shell 绕过、不让后台无限等待批准、不声称已生成文件;继续完成报告内的结构化数据,并如实说明文件未保存。恢复写入授权须由有权限的人或宿主处理,Skill 不扩大权限。
|
||||
|
||||
图表、表格和指标使用 `publish_task_report.report.ui_blocks`,每块为 `answer-ui@1`,包含独立 `block_id`、`fallback_text` 和 json-render 的 `spec.root/elements`,外层附真实 `evidence_refs`。最多 8 块、总计 400 KB,单块还受现有 schema 限制。优先组织以下内容:
|
||||
|
||||
| 展示 | 建议组件 | 数据来源 |
|
||||
| --- | --- | --- |
|
||||
| 候选管段及理由 | Table、Evidence、MapLocation | 地点、资产与定位证据;明确候选和已确认的区别 |
|
||||
| 异常及正常对照 | Chart | 同单位的实际序列,说明时间对齐及缺测 |
|
||||
| 关阀与隔离范围 | Table、Metric、MapLocation | 隔离证据、阀门属性及候选场景 |
|
||||
| 水力与用户影响 | Table、Chart | 指定输出节点的模拟结果及真实用户关联资料 |
|
||||
| 处置与闭环核对 | Table、Evidence | 动作回执、现场反馈、恢复监测和原工单版本 |
|
||||
|
||||
只用工具公布的组件属性。JSONL 可作为整理中间格式,但目前没有“把任意 JSONL 直接传给 json-render”的通道;需转换为已注册组件的 JSON 属性。后台不用 `present_answer`,不把原始 JSONL 输出到聊天正文,也不把表单提交当成已保存现场证据。小数据和简单结论可直接文字表达。
|
||||
|
||||
## 本轮成果与爆管闭环的验收
|
||||
|
||||
报告至少回答下列四类问题,逐项给出实际结果或经过查询证实的具体限制;无需固定排版或创建固定数量的动作。
|
||||
|
||||
1. **位置**:事故已确认还是仅有候选?每个候选的模型/地图编号、匹配或定位依据、排名及未确认事项是什么?
|
||||
2. **隔离**:每个候选场景是否可隔离?应关、可选和已知不可关的阀门分别是什么?有无替代方案及现场前置核验?
|
||||
3. **影响**:拓扑受影响节点、水力压降或失供范围、用户关联分别有哪些实际结果?未覆盖什么?没有水力或用户资料时分别说明,不能推定为零。
|
||||
4. **处置与闭环**:当前可审批的工作是什么?已有哪些动作、审批、提交回执和现场事实?还缺什么反馈才具备结案条件?
|
||||
|
||||
有具体结论和动作建议时,先保存 `interim/concluded`,再 `propose_task_action`,最后保存 `final`。人工执行事项使用 `work_order_type=general`,保留原爆管事件。工单描述写位置、任务内容、依赖、执行前核验、需回传的事实和责任要求;`propose_task_action` 当前 payload 没有独立的 dependencies 或 feedback_requirements 字段,不发明字段。没有客户通知适配器时,将通知作为人工工单事项,不宣称已发送。
|
||||
|
||||
闭环复核至少核对实际故障资产及原因、隔离/阀门实施记录、抢修结果、适用的清洗及水质复测、恢复供水记录和恢复后监测、用户通知及回访、未决动作与遗留风险、原上报工单结案状态。不适用事项必须有事件依据;未收到结果写待核验,不把记录创建时间当现场完成时间。只齐备处置证据而原工单未结案时,结论为“具备结案条件,待原系统确认”,而非“已闭环”。误报或未发现故障的结案也要有支持排除的实际核查与人工确认。
|
||||
|
||||
必要查询失败、权限拒绝和存储未配置属于执行异常,按 schema 修正可纠正问题;仍无法取得当前判断必需的资料时交宿主记录失败。已通过有效查询确认关键数据确实缺失且无法补齐,才发布 `final/insufficient_data`,提供 blocking 缺口和查询依据,建议与待办为空,不在该退出分支再生成补数据工单。能够明确形成一项现场核验结论时可提议具体核验动作,但不能把“尚未开展定位”包装为“已完成定位调查”。
|
||||
|
||||
## 能力边界与验证
|
||||
|
||||
本次交付为工程图和运行时 Skill 文本,使用现有 MCP、发布、审批与反馈能力;不新增自动关单、鼠标自动化、摄像机操作、JSONL 导出接口或数据库字段。Server 目前没有按 Skill 检查定位/隔离成果的强制验收器,本文的业务验收要求仍由 Agent 执行并由人复核。
|
||||
|
||||
当前 Skill 仅通过 `section=skill` 整份分发,没有相对路径 reference 读取协议,因此运行时指令保持自包含,不能要求 Agent 读取本工程文档。内容需控制在既有内联预算内。Server 更新资源后,应按原发布方式让实际 Server 实例取得新文件;Agent 读取时核对 `name/version/content_sha256`,不把本地文件更新当成运行中实例已生效。
|
||||
|
||||
验证包含 Mermaid 语法解析、本地路径和工具 ID 核对、Skill 分发/权限/体积测试及 `git diff --check`。后续真实行为验收应分别覆盖:有效观测和漏水量依据下的定位;已知候选但缺漏水量时的候选隔离;不可关阀替代方案;SCADA 首屏未覆盖事件时继续读取;接口失败明确终止;文件权限缺失不绕过;无浏览器时照常调查;实际反馈触发新轮及最终闭环复核。没有真实模型计算或现场回执,不报告定位准确性或业务闭环已通过。
|
||||
|
||||
关联契约:[分析 MCP](mcp-analysis-tools.md)、[调查展示](investigation-progress-and-presentation.md)、[报告契约](investigation-report-contract.md)、[任务与多轮反馈](operational-tasks.md)、[后台工作进程](operational-task-workers.md)。
|
||||
关联契约:[分析 MCP](mcp-analysis-tools.md)、[调查展示](investigation-progress-and-presentation.md)、[报告契约](investigation-report-contract.md)、[任务与多轮反馈](operational-tasks.md)。
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# 动态调查计划数据契约
|
||||
# 旧动态调查计划兼容契约
|
||||
|
||||
调查任务从工单投诉内容或人工请求创建。Agent 在本次执行中生成 `execution-plan@2`,根据新证据追加修订;数据库保存不可变计划快照的 JSONB 文档,并不保存固定流程模板。任务、Run、证据、访问申请、审核动作和报告各有独立生命周期。计划中的 `gate_ref` 只引用已经持久化的访问申请或动作,不能把审批结果写成模型自行判断的文本。
|
||||
> 历史兼容文档:字段和示例用于读取已有 `execution-plan@1/@2` 快照,不要求当前调查生成或执行这些计划。当前进展与累计图见 [调查进展](investigation-progress-and-presentation.md),Agent 生命周期见 [任务执行](../../next-tjwater-agent/docs/execution-plan.md)。
|
||||
|
||||
早期设计由 Agent 生成 `execution-plan@2` 并按证据追加修订;数据库保存不可变计划快照的 JSONB 文档,不保存固定流程模板。任务、Run、证据、访问申请、审核动作和报告各有独立生命周期。计划中的 `gate_ref` 只引用已经持久化的访问申请或动作,不能把审批结果写成模型自行判断的文本。
|
||||
|
||||
## `execution-plan@2`
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Backend Naming Audit
|
||||
|
||||
> 历史接口盘点,保留旧名称与修正依据。本文中的旧仓库名、URL 和替代建议不作为当前可用接口清单;现行规范见 [API 风格](../api-style.md),实际契约见 [Server OpenAPI](../../contracts/server-v1.openapi.json)。
|
||||
|
||||
DOC-003 audit for the internal `TJWaterServerBinary` backend.
|
||||
|
||||
## Scope
|
||||
@@ -1,5 +1,7 @@
|
||||
# `app/api/v1/endpoints/` 失效 API 排查与修正
|
||||
|
||||
> 历史接口盘点,保留旧名称与修正依据。本文中的旧仓库名、URL 和替代建议不作为当前可用接口清单;现行规范见 [API 风格](../api-style.md),实际契约见 [Server OpenAPI](../../contracts/server-v1.openapi.json)。
|
||||
|
||||
排查范围:`app/api/v1/endpoints/`
|
||||
|
||||
结论:本次共确认 5 个问题接口,处理结果如下:
|
||||
@@ -67,7 +69,7 @@ DMA 计算请使用:
|
||||
|
||||
服务区计算请使用:
|
||||
|
||||
- `/api/v1/calculateservicearea/`
|
||||
- `/api/v1/calculateservicearea/`
|
||||
现在返回全部时间步结果,不再接收 `time_index`
|
||||
|
||||
## 变更文件
|
||||
@@ -1,8 +1,9 @@
|
||||
# 调查进展与实时展示
|
||||
|
||||
调查执行中可持续发布正文和完整 Mermaid 图,尚未形成结论时也能发布。
|
||||
调查执行中可按需整理正文和流程参考图,Server 累计保留路径并生成 Mermaid,尚未形成结论时也能发布。
|
||||
Server 将阅读进展、业务报告和浏览器展示请求分别保存。运行结束仍要求同轮最终报告,
|
||||
进展中的节点状态不参与调度、审批或结束判断。
|
||||
图节点不绑定实际执行任务,标记只解释最近整理的路线、结果和限制;不参与调度、审批、等待或结束判断。
|
||||
路线有实质调整、形成重要结果、需要保留失败尝试或整理报告时更新即可,不要求逐步发布、频繁刷新或及时同步。
|
||||
|
||||
## 进展快照
|
||||
|
||||
@@ -14,37 +15,73 @@ Server 将阅读进展、业务报告和浏览器展示请求分别保存。运
|
||||
"run_id": "<current-run-uuid>",
|
||||
"source_session_id": "<claimed-session-id>",
|
||||
"progress": {
|
||||
"summary": "正在核对投诉时段的监测记录,尚未形成结论。",
|
||||
"diagram": {
|
||||
"title": "当前调查进展",
|
||||
"mermaid": "flowchart LR\nread[核查监测记录] --> judge[判断证据范围]",
|
||||
"node_refs": [{
|
||||
"summary": "整理投诉时段的监测核查路线,尚未形成结论。",
|
||||
"graph_update": {
|
||||
"title": "调查流程参考",
|
||||
"nodes": [{
|
||||
"node_id": "read",
|
||||
"label": "核查监测记录",
|
||||
"status": "running",
|
||||
"status": "reference",
|
||||
"evidence_refs": [],
|
||||
"action_refs": [],
|
||||
"map_targets": [{"source_id": "pipes", "feature_id": "<known-pipe-id>"}]
|
||||
}]
|
||||
}],
|
||||
"edges": []
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
示例中的编号由实际任务和资产替换。正文最长 20,000 字符,`diagram` 可空。
|
||||
示例中的编号由实际任务和资产替换。正文最长 20,000 字符,`graph_update` 可省略。
|
||||
图示沿用 [调查报告契约](investigation-report-contract.md) 的 Mermaid 安全限制。
|
||||
节点状态可空,或为 `pending/running/completed/waiting_human/failed/skipped/cancelled`。
|
||||
`map_targets` 只接受 `junctions/pipes/valves/pumps/tanks/reservoirs/scada` 七类来源和要素编号。
|
||||
图中节点引用在本次发布内唯一,证据和动作引用必须属于该任务。跨修订可以修改节点编号、
|
||||
分支和文字,Server 不验证依赖执行顺序或节点完工。
|
||||
节点标记可省略或为 null,均显示为参考步骤;新增量更新只接受以下五种:
|
||||
|
||||
每次传入 `progress` 都完整替换该 run 的快照,传入 `diagram: null` 清除上一版图示。
|
||||
省略 `progress` 保留原快照。同一个请求编号重试只发布一次。
|
||||
| status | 显示 | 含义 |
|
||||
| --- | --- | --- |
|
||||
| reference | 参考步骤 | 路线或分支说明,不断言是否执行 |
|
||||
| recorded | 已有结果 | 已整理出结果或依据,不表示任务、计算或现场动作完成 |
|
||||
| needs_input | 待补充条件 | 记录缺少的资料、权限或人工事实,不控制任务等待 |
|
||||
| failed | 尝试失败 | 保留一次失败尝试及原因,后续路线另建节点 |
|
||||
| not_taken | 未采用路线 | 未采用、不适用或改走其他分支 |
|
||||
|
||||
旧图读取兼容:pending/running 转为 reference,completed 转为 recorded,waiting/waiting_human 转为 needs_input,
|
||||
skipped/cancelled 转为 not_taken;failed 保留。仅规范化读取结果,不改写历史文档,不再显示“正在执行”或“记录时执行中”。
|
||||
REST 与 MCP 使用相同的旧标记映射。MCP 的 progress/report 图示在分页及 data_path 投影之前完成转换,
|
||||
读取整份说明、节点列表或单个 status 时含义一致;实际 task/run/action 状态和旧执行计划状态保持各自语义。
|
||||
`map_targets` 只接受 `junctions/pipes/valves/pumps/tanks/reservoirs/scada` 七类来源和要素编号。
|
||||
图中节点引用唯一,证据和动作引用必须属于该任务。新 Agent 使用下述增量更新,节点编号在任务各轮保持稳定。
|
||||
Server 不验证依赖执行顺序或推断节点完工。
|
||||
|
||||
`progress.graph_update` 包含可选 `title`、`nodes` 和 `edges`。节点沿用 `node_id/label/status/evidence_refs/action_refs/map_targets`,
|
||||
可用 `detail` 记录实际结果或失败原因;连线为 `{source,target,label?}`。仅提交新增或变化的内容,
|
||||
`update_investigation_graph` 合并节点、追加明确连线并生成 Mermaid,保留已有路径和引用。
|
||||
尝试失败时标记 `failed`,重试或替代路线使用新节点及连线;原失败节点不能改写为成功。
|
||||
仅更新正文或传入 `diagram: null` 均保留累计图,原始 Mermaid 不再由新 Agent 生成。
|
||||
首次增量更新沿用最近保存的旧图,将原源码保存在 `source_mermaid`,保留原分支和形状后追加内容;
|
||||
旧图内节点不得改名或重用为其他步骤。已被旧实现覆盖且未存入报告的内容无法还原。
|
||||
尚未迁移的旧图仍接受旧客户端显式提交的完整图;仅更新正文或传入 `diagram: null` 时自动接管旧图,
|
||||
保留原源码并转为累计图。任务启用累计图后拒绝旧客户端重画覆盖,不提供通过空值清图的入口。
|
||||
省略 `progress` 保留原快照,同一个请求编号重试只发布一次。
|
||||
`workbench.operational_task_progress` 每个任务和 run 保存最新一份快照,独立于七天事件清理。
|
||||
任务详情和 Agent 上下文返回 `latest_progress`,包含 `revision/run_id/execution_generation/`
|
||||
`published_at/source_session_id/progress`。只读取当前执行代次,新轮次尚未发布时返回空。
|
||||
`published_at/source_session_id/progress`。优先返回当前执行代次;新轮次尚未发布时保留最近的累计图及其原始轮次。
|
||||
`get_task_context(section="progress")` 同样允许读取上轮累计图以接续,摘要和图示不证明新轮次已执行。
|
||||
报告发布时由 Server 存入累计图及其已有依据,历史报告保持独立快照;最多保留 100 个节点、200 条连线,超限显式报错,不静默裁剪。
|
||||
快照更新发出 `operational.progress.changed`,前端据此读取最新内容。
|
||||
|
||||
### 旧任务图迁移
|
||||
|
||||
在 Server 仓库运行 `python -m scripts.migrate_investigation_graphs --plan /受保护目录/plan.json --project-id <项目UUID>`;
|
||||
多个项目重复指定 `--project-id`。预览只读数据库,计划文件以私有权限保存原始进度文档和迁移后的完整文档。
|
||||
核对项目、业务库、任务和图数量后,运行 `python -m scripts.migrate_investigation_graphs --apply /受保护目录/plan.json`。
|
||||
连接使用 MetaDB 路由,计划中不保存连接凭据。迁移逐项目事务执行,与正常发布使用相同的事件锁和任务锁,
|
||||
预览后仍为旧图但已发生变化的行会使该项目回滚;重复执行跳过已经接管的图。
|
||||
|
||||
迁移仅更新已有进度图及其 revision,保留原 summary、发布时间、run、会话、节点说明和引用;
|
||||
原 Mermaid 字节保存在 `source_mermaid`,旧节点标记转换为参考标记,不解析或猜测原始连线。
|
||||
历史报告保持原样,通过读取兼容显示新标记。迁移逐项目校验任务、run 和报告整行摘要未变,
|
||||
并发出进度变更事件及写入独立执行回执;不产生报告更新通知。备份和回执不得提交到仓库。
|
||||
|
||||
## 浏览器实时联动
|
||||
|
||||
`POST /api/v1/operational-tasks/{task_id}/presentation-events` 接收
|
||||
@@ -68,7 +105,7 @@ Server 校验当前 run、绑定会话、有效租约和幂等身份,记录 `o
|
||||
|
||||
以上十种动作与 Chat 的 `present_view`、Dev JSONL 共用扁平 `command: {action, ...字段}`。Chat 使用 `frontend-action@2` 和执行回执;任务仍只记录展示请求,Dev 只做本地预览。三者使用独立的地图源、SCADA 图层和图表状态。Dev 的动作数据行必须为 `{"record_type":"presentation","command":{...}}`;界面块行使用 `record_type: ui_block`,新示例明确带 `phase: final`。普通调查数据保持来源字段,不执行其中的 action。仓库示例见[前端统一展示夹具](../../next-tjwater-frontend/tests/fixtures/unified-presentation.jsonl),其中编号来自测试夹具。
|
||||
|
||||
图表限定折线和柱形,包含 `title/chart_type/x_data/series`。最多 100 个横轴值和 8 组有限数值序列,
|
||||
图表使用普通折线 line、柱状 bar,或当前曲线加历史散点的 time_comparison,包含 `title/chart_type/x_data/series`。最多 100 个横轴值和 8 组有限数值序列,
|
||||
序列长度与横轴对齐,引用证据必须属于当前任务。地图高亮来源沿用上述七类。
|
||||
覆盖层的权威数据来自不可变证据。省略 `data` 时,Server 从证据快照提取现有 GeoJSON,
|
||||
或使用已记录的 `longitude/latitude` 点位,将 `FeatureCollection` 加入事件中的 `command.data`。
|
||||
@@ -103,7 +140,53 @@ MCP 保存完整查询响应作为不可变证据,展示解析器同时支持
|
||||
注明来源、范围与实测或模型属性;没有同口径基准时不计算或绘制改善量,缺测不补零。
|
||||
过程图表通过 `open_chart` 提供 `chart_id/chart/evidence_refs`,每图最多100个横轴值、8组序列,
|
||||
超限时分图或明确说明抽样、聚合范围;单值或无可比数据用文字或表格。
|
||||
同指标、单位和窗口的同期复合图按对象分别发布稳定 chart_id,标题为“对象ID 共同标题”,每条序列名以“对象ID ”开头。共同标题、chart_type、轴名、x_data、当前序列去掉对象前缀后的名称与 sample_times 完全一致时,浏览器将对象数据集聚合为一个子图表:右侧按钮切换对象,下方图例控制序列;窄屏对象列表移到下方。普通趋势总览、柱状图和结构化 Chart 保持独立,不新增 objects 字段。原 chart_id 的 close_chart 只移除对应对象,关闭聚合卡片移除组内对象。
|
||||
|
||||
浏览器的总图表窗口统一管理过程图组与报告 Chart,子图表仍可手动独立弹窗放大。只有正在跟随的运行中任务新收到 open_chart 时可自动弹出总窗口;后续命令更新内容,不自动打开子窗口,用户关闭后不重弹。任务打开、切换、刷新、已完成任务载入、SSE 重连历史补载及历史导航只恢复数据,不弹窗。窗口意图是浏览器临时状态,不写入事件或历史快照,不新增协议字段;open_chart 的内容加载与窗口是否弹出是两个不同状态,Server 的 recorded 也不表示浏览器已经打开窗口。
|
||||
|
||||
time_comparison 的 x_data 使用精度一致、按窗口排列的 UTC+8 HH:mm 或 HH:mm:ss 共享槽,允许跨午夜但短于一天,不追加历史日期。series 同时包含当前 line 与历史 scatter,不混入 bar;sample_times 与 data 等长,有效值和真实带时区时间成对保留,缺测值与时间均为 null。时间转换后须与横轴槽一致,当前线使用同一窗口,每条历史序列来自同一历史日的对应窗口,不接受当天或未来日期。新任务展示与报告发布不满足这些规则时返回 TIME_COMPARISON_INVALID,旧保存内容保持结构读取。普通 line/bar 跳过同期语义,但不能将 line/scatter 复合图标为 line。
|
||||
|
||||
前端维护的脚本 scripts/export-view-command-contract.mjs 同步 Agent/Server 的结构协议及 tests/fixtures 下的统一 JSONL 和 chart-publication-cases.json;JSON Schema 不代替跨字段语义校验。Server 的独立测试用相同正反例校验 Pydantic 边界和 validate_time_comparison_chart;Agent 主机、原生工具及 Dev 的新图表输入也执行对应规则。
|
||||
|
||||
关键趋势与比较结论保留到报告中,最终报告的图表不替代过程发布。
|
||||
|
||||
### SCADA 设备展示覆盖
|
||||
|
||||
SCADA 异常初次调查的监测过程图使用可选的 `chart.scada` 结构:
|
||||
`role=current|comparison`、`metric=pressure|flow`、`unit`、带时区的 `window_start/window_end`,
|
||||
窗口固定为本轮截止时刻之前三小时;采样槽保持真实采样相位,截止非整刻不平移。超过100槽按连续时间分图,使用不同稳定chart_id,元数据仍声明完整三小时窗口;Server逐点核实后合并覆盖,缺少任一槽仍拒绝收尾。同期历史只取完整窗口末三个采样槽,同设备同期分图不得重叠。低频设备窗口无计划采样槽时,完整读取对应当天/历史三小时并核实无读数后,以截止时刻的单个null保留设备入口,标记no_scheduled_samples,不计作漏采。`value_field` 默认为 `monitored_value`,可显式选 `cleaned_value`。
|
||||
每条序列绑定 `device_id` 和 `day_offset`(当天 0,此前七天 1 至 7)。
|
||||
同期图每个设备包含当天 line 和七组 scatter,历史仅末三个对应采样槽有值。
|
||||
Server 校验设备属性、单位、完整采样槽、真实时间与引用快照的对应值,允许数值四舍五入保留 1 至 6 位小数,
|
||||
保留负流量;缺测必须有完整查询或明确空值记录证明,未查询不能视为缺测。
|
||||
`chart.scada.gaps` 由 Server 生成并存入展示事件,前端保留设备入口并展示缺测点数。
|
||||
回放全量查询的 `coverage.frozen_start` 记录服务端已冻结的最早时刻;历史早于该边界时,
|
||||
以本轮匹配 replay/batch 的完整回执证明 `outside_frozen_window`,无需虚构一次成功的历史读取。
|
||||
|
||||
地图的 SCADA 定位/标记、已验证图表和最终报告 `scada_device_ids` 累计本轮目标;
|
||||
监测明细表使用该字段声明完整设备集合。按指标和单位分图,跨多张总览取设备并集,
|
||||
同一 `chart_id` 更新只采用最新覆盖,累计目标不缩小。关闭图表及清空地图只影响浏览器,
|
||||
不撤销产物覆盖。最终报告及成功结束均拒绝遗漏,返回 `409 SCADA_PRESENTATION_INCOMPLETE`,
|
||||
`related_refs` 列出待补设备;Agent 补图后可重试原请求。普通进度和 interim 报告可继续发布。
|
||||
“仅作参考”不豁免展示;确有缺测时仍发布有空槽的图表并保留设备,
|
||||
完整查询无法完成时保留 interim 和具体失败原因,不伪造缺测来通过 final。
|
||||
|
||||
覆盖事实保存在 `workbench.operational_task_runs.artifact_manifest.scada_presentation`,
|
||||
JSON 包含累计 `targets`、按 `chart_id` 保存的最新验证 `charts` 及 `updated_at`。
|
||||
它与 Run 一起保留,独立于七天展示事件清理;不依赖浏览器在线或回执。
|
||||
不同 Run、项目和截止窗口不能抵数。恢复复核阶段沿用恢复证据门禁,不要求重画异常轮次图表。
|
||||
历史读取不要求新字段,已完成报告不回写;旧 Agent 的无绑定图表仍可读,但不能通过新轮次的覆盖门禁。
|
||||
|
||||
部署按[工作台数据库变更门禁](workbench-database-rollout.md)核实目标 BizDB、备份和回滚演练。
|
||||
先部署已兼容内部清单读取的 Server,再暂停目标项目的任务写入并在维护窗口应用
|
||||
[035 迁移](../resources/sql/035_scada_presentation_coverage.sql),核对读取与存储就绪后发布 Agent 和前端。
|
||||
不要在旧 Server 仍处理请求时新增该列:旧版整行 Run 投影会将内部字段传入严格响应模型。
|
||||
新 Server 在迁移前可读取历史任务,SCADA 写入明确返回 503;完成迁移后再恢复目标项目写入。
|
||||
迁移只为 Run 新增通用 JSON 产物清单,默认空对象;不会自动回填旧图或自动迁移其他项目。
|
||||
未应用迁移的 SCADA 写入与收尾返回明确的 503,历史读取保持可用。
|
||||
Run 读取投影显式排除内部清单,REST 与 MCP 原有 Run 结构保持一致。
|
||||
若回退到本次读取兼容修复之前的 Server,须先备份清单再移除新增列;旧代码的整行读取不能直接带着新列回退。
|
||||
本次实现不代表迁移或上线已执行。
|
||||
业务 Skill 说明本步骤应展示什么,调查指令统一说明发布方法、非空几何检查、去重和失败处理。
|
||||
同一阶段批量展示,数据或方案没有变化时不重复请求;不为地图动作增加无关业务分析。
|
||||
报告的 `map_targets` 仅为手动定位入口。关键步骤的数据缺失或展示失败需记录具体限制,
|
||||
|
||||
@@ -47,6 +47,9 @@ node_id 为最多 80 字符的 ASCII 标识符,必须以字母或下划线开
|
||||
并与 Mermaid 源码中实际节点使用完全相同的 ID;禁止为同一节点另造引用 ID;
|
||||
最多 100 条引用。证据必须来自报告 evidence_refs,动作必须属于当前任务。
|
||||
节点可以说明候选方向与条件关系;图中的分支、状态文字不能证明实际并发、动作执行或现场完成。
|
||||
可选 status 仅为参考标记:reference(参考步骤)、recorded(已有结果)、needs_input(待补充条件)、
|
||||
failed(尝试失败)、not_taken(未采用路线)。recorded 不证明业务完成,标记不与任务或工具运行状态绑定。
|
||||
旧运行状态按[兼容映射](investigation-progress-and-presentation.md)读取,未提供标记时保持中性的参考步骤。
|
||||
图的颜色、字体和交互由前端统一配置。正文不包含 SVG 或 HTML。
|
||||
|
||||
## 业务约束
|
||||
@@ -150,7 +153,8 @@ failure_code 为 ARTIFACT_INCOMPLETE;调用方须读取返回的业务状态
|
||||
|
||||
## 发布、阅读与协作
|
||||
|
||||
执行中尚未形成结论的正文和 Mermaid 使用独立的 `progress` 快照持续更新,
|
||||
执行中尚未形成结论的正文和 Mermaid 使用独立的 `progress` 快照按需整理,
|
||||
在路线调整、重要结果或报告归纳时更新即可,不要求逐步更新或及时同步。
|
||||
见 [调查进展与实时展示](investigation-progress-and-presentation.md)。
|
||||
报告的 interim/final 与 concluded/insufficient_data 结果契约保持不变。
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# NextTJWater MCP 查询与认证设计
|
||||
|
||||
日期:2026-10-15。状态:查询与认证代码已在本地实现,认证发现、resource 参数和版本头体校验已补齐,默认关闭,未部署;真实 Keycloak 与项目数据库验收待完成。
|
||||
初稿日期:2026-10-15。本文保留 MCP 查询与认证的设计依据,后续工具扩展及当前启用步骤见 [实施与启用](mcp-query-implementation.md)。早期本地实现、默认配置和验证快照不代表当前线上启用或部署状态;需核对目标环境。
|
||||
|
||||
本设计将现有后端查询能力通过 MCP 开放给普通项目 Agent、调查主 Agent 和调查只读子 Agent。采用 MCP 2026-07-28、Streamable HTTP、现有 Python Server 内挂载服务、可信 Agent 宿主按请求代理认证。旧 CLI 用于查找功能和接口线索,业务实现继续由当前 Server 负责。
|
||||
|
||||
第一版完成查询、权限、取消、分页、调查证据和凭据续期的完整闭环。已有报告发布、工单审批、设备控制、调查运行生命周期继续使用当前受控工具;不通过 MCP 自动开放旧 CLI 中的分析执行命令。
|
||||
本文第一版设计聚焦查询、权限、取消、分页、调查证据和凭据续期。后续已扩展受控分析、预览式写入及后台计算队列,详见实施说明;报告发布、审批、设备控制和调查运行仍遵循各自业务边界,不从旧 CLI 清单自动开放能力。
|
||||
|
||||
## 当前系统与功能核对
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 产品 MCP 查询:本地实施与启用
|
||||
|
||||
实现使用 MCP 2026-07-28、Python mcp 2.3.0、TypeScript @modelcontextprotocol/client 与 @modelcontextprotocol/server 2.3.1。Server 提供 `/mcp` 和71项工具(42项原查询、12项计算、13项预览式写入、4项计算任务管理);Agent 将其投影合并到本地 MCP,由 OpenCode 原生发现为 `mcp_tjwater_*`,普通项目会话、调查主 Agent 和已验证的直接只读子 Agent 共享42项查询;12项计算仅向非reader会话开放,13项写入仅向项目成员的普通聊天开放。个人会话不能获取项目凭据。旧 CLI 只作为能力清单,没有运行时 CLI 桥接。
|
||||
实现使用 MCP 2026-07-28、Python mcp 2.3.0、TypeScript @modelcontextprotocol/client 与 @modelcontextprotocol/server 2.3.1。Server 通过 `/mcp` 提供查询、分析计算、预览式写入和计算任务管理,具体目录以 `app/mcp/catalog.py` 为准;Agent 投影到本地 MCP,由 OpenCode 原生发现为 `mcp_tjwater_*`。普通项目会话、调查主 Agent 和已验证的直接只读子 Agent 在各自授权范围内查询;计算仅向非 reader 会话开放,写入仅向项目成员的普通聊天开放。个人会话不能获取项目凭据。旧 CLI 只作为能力清单,没有运行时 CLI 桥接。
|
||||
|
||||
`run_simulation` 是在线重算,复用 REST 在线计算服务而不复制数据库;按水力步长更新模型条件和实时基线,不创建正式方案。MCP 时长为秒,原 REST 时长仍为分钟。其 annotation 为 `readOnlyHint=false`、`destructiveHint=true`;只读 reader 不能调用,其他方案模拟继续使用隔离临时模型。详细边界见 [MCP 分析工具说明](mcp-analysis-tools.md)。
|
||||
|
||||
@@ -14,9 +14,15 @@ Agent 根据401 challenge 或路径/根 well-known 发现受保护资源元数
|
||||
|
||||
调查数据返回前,在短事务内追加现有任务证据并保存调用回执。同一主体、会话、任务、run 和原生 call ID 的重试复用证据;不同参数或失效范围明确报错。七类原有访问申请 scope 保持原契约,MCP 的新增来源只扩充快照来源枚举。没有新增报告发布、审批、控制、Shell 或递归委派权限。
|
||||
|
||||
普通查询默认20条;目录类最多500条,时序最多10000条、20个标识、24小时;offset 最大10000。Agent 首次优先单对象、必要的短时间窗和小页,仅在证据不足时按需扩大,不自动遍历全网。SQL 有 limit/offset 和 statement_timeout。历史对比在同一条SQL中按序列、时间对数据点分页,返回 page.has_more;非零offset也标明覆盖不完整。历史曲线展示调用初始每页500点,`RESULT_TOO_LARGE` 时原位减半重试,最多读取1000条、发出16次请求,共用120秒超时(单次请求同时受该剩余总预算约束),图表最多100个有效点;这不改变模型首次查询默认值或时序接口上限。
|
||||
普通查询默认20条;目录类最多500条,显式指定设备的时序查询最多10000条、20个标识、24小时;offset 最大10000。SCADA调查首次省略 `query_scada_readings.device_ids`,由Server读取项目完整设备目录及窗口内全部授权观测,内部自动翻页,`offset` 必须为0。其余查询按问题选定对象和窗口。SQL 有 limit/offset 和 statement_timeout。历史对比在同一条SQL中按序列、时间对数据点分页,返回 page.has_more;非零offset也标明覆盖不完整。历史曲线展示调用初始每页500点,`RESULT_TOO_LARGE` 时原位减半重试,最多读取1000条、发出16次请求,共用120秒超时(单次请求同时受该剩余总预算约束),图表最多100个有效点;这不改变模型首次查询默认值或时序接口上限。
|
||||
|
||||
单份结果超过8MiB报错;超过12KB的调查工具消息只返回证据资源引用,完整快照仍保存,Server 的 `resources/read` 可以读取它。Agent 本地 MCP 只提供工具,将上游资源引用转为 `evidence_read` 中的 `mcp_tjwater_get_task_evidence` 参数,不向 OpenCode 泄露不可在本地读取的链接。`get_task_evidence` 在数据库中按 data_path、limit、offset 选页,嵌套集合和超过256字符的字符串返回路径描述;字符串按 Unicode 字符分页。页和完整资源均在数据库返回前检查8MiB上限,保留原证据编号和合成来源;普通查询要求缩小范围。其他结果仍保留执行后的整体8MiB检查,不能把它理解为所有查询均有数据库端字节限制。原始/清洗实测、模型基线、分析结果、比较和 synthetic 均明确标记。非有限数值转为 null 并记录数据缺口。
|
||||
单份结果超过8MiB报错;超过12KB的调查工具消息默认返回证据资源引用,设备目录的精简路径见下段。完整快照仍保存,Server 的 `resources/read` 可以读取它。Agent 本地 MCP 只提供工具,将上游资源引用转为 `evidence_read` 中的 `mcp_tjwater_get_task_evidence` 参数,不向 OpenCode 泄露不可在本地读取的链接。`get_task_evidence` 在数据库中按 data_path、limit、offset 选页,嵌套集合和超过256字符的字符串返回路径描述;字符串按 Unicode 字符分页。页和完整资源均在数据库返回前检查8MiB上限,保留原证据编号和合成来源;普通查询要求缩小范围。其他结果仍保留执行后的整体8MiB检查,不能把它理解为所有查询均有数据库端字节限制。原始/清洗实测、模型基线、分析结果、比较和 synthetic 均明确标记。非有限数值转为 null 并记录数据缺口。
|
||||
|
||||
`list_scada_devices` 默认每页100个、最多500个,其他集合仍沿用各自的分页默认值。设备目录超过12,000字节时,先内联本页全部设备的 `device_id`、`device_type` 和 `measurement_unit`,省略其他属性并明确标注;模型映射、坐标等通过 `get_scada_device` 核验,任务证据和幂等回执保留完整属性。此时 `truncated` 表示属性省略,设备是否还有后续页以 `page.has_more` 为准,按本页实际设备数递增 `offset`。精简目录仍超预算时沿用证据引用或缩页路径。
|
||||
|
||||
全量查询同时要求设备目录与读数权限;通过任务授权读取时,捕获范围及当前批准状态都必须同时包含 `scada_devices` 和 `scada_readings`,显式设备读数查询仍沿用读数授权。
|
||||
|
||||
全量 `query_scada_readings` 返回 `data.devices` 的完整属性、`data.readings` 的原始和清洗值,以及 `data.coverage` 的查询设备、读数数量和缺测清单;该模式保留完整内联数据,不经过12KB缩略,仍执行8MiB整份结果上限。回放仅读取指定批次之前的冻结观测,项目中未纳入该回放的设备单列 `outside_frozen_scope_device_ids`。`query_complete=true` 表示授权范围内的查询已读完,不表示设备均有有效数据。SCADA及历史回放任务发布final时,必须引用窗口覆盖本轮时刻、且回放批次匹配的全量证据;单设备、局部集合、未引用或旧批次证据不能通过此校验。既有报告不回写,后续重跑适用新约束。
|
||||
|
||||
`get_task_context` 默认 section=summary,只查固定元数据、160字符任务标题及200字符结论预览,并返回 available_sections;preview_only 和 truncated 明确说明不是完整报告。按 section=task/report/evidence/actions/progress/access_requests/plan/run 读取详情,report_revision 保留选择旧报告的语义。data_path 选择嵌套值,limit 默认20、最多500,offset 最大10000;证据、动作和授权申请集合以稳定 ID 为键,按 ID 字典序分页。字符串按字符分页。详情在数据库返回前检查8KB预算,超限缩页或选更深路径。缺少可选进度或授权申请表时返回 metadata.available=false,不冒充空事实。绑定权限范围只读取七类既有 scope,不再加载整个任务上下文。不增加工具数量、数据库表或迁移,也不改变原 REST 完整上下文契约。
|
||||
|
||||
@@ -26,7 +32,9 @@ Agent 根据401 challenge 或路径/根 well-known 发现受保护资源元数
|
||||
|
||||
### Server 分发业务处置 Skill
|
||||
|
||||
业务流程源文件位于 `resources/work-order-skills/*/SKILL.md`,随 Server 镜像分发。`get_task_context` 摘要提供 `handling_skill`(适用 workflow、名称、版本和 SHA-256)及 `display_time_zone=Asia/Shanghai`;使用 `section="skill"` 读取完整正文。流程由服务端持久化任务的 workflow 选择,调用方不能传入文件路径或其他任务类型;未知 workflow 返回 `available=false`,不自动套用其他流程。
|
||||
业务流程源文件位于 `resources/work-order-skills/*/SKILL.md`,随 Server 镜像分发。`get_task_context` 摘要提供 `handling_skill`(适用 workflow、名称、版本和 SHA-256)及 `display_time_zone=Asia/Shanghai`;使用 `section="skill"` 读取完整正文。流程由持久化任务的 workflow 与 replay_context 选择:工单 burst_report 使用 burst-event-handling,scada_anomaly 及带回放来源的历史 burst_report 使用 scada-anomaly;未知 workflow 返回 available=false。工单事件处理的绑定工具范围移除 run_burst_location,保留资产关联、隔离和影响分析。
|
||||
|
||||
报告阅读时区统一为 UTC+8(Asia/Shanghai)。MCP 在原始证据和回执保存后,将模型可见结果中的显式带时区时间转换为 `+08:00`,保留同一时刻和源小数秒精度;存储、查询参数及冻结窗口沿用原始时刻,标识符与引用保持原文。前端对报告正文、交互表格、证据卡片、图表提示及数据表采用同一阅读时区,并兼容历史正文中的 `Z`、`UTC` 和 `UTC0` 标记。历史正文中省略时区的时间按读者默认 UTC+8 展示。
|
||||
|
||||
当前覆盖爆管上报、水质投诉与预约跟进。业务 Skill 描述风险核查、隔离停水、用户通知、现场处置和恢复条件;Agent 本地 Skill 维护通用执行方法。权限、审批和执行仍由 Server 代码控制。流程说明没有 evidence_id,不记录成事件证据。具体草稿由现有 `propose_task_action` 保存并等待人工审批;人工通知等步骤通过关联预约工单承接,不能当作通知已发送或现场工作已完成。
|
||||
|
||||
@@ -99,10 +107,10 @@ Agent:
|
||||
|
||||
```bash
|
||||
bun run check
|
||||
bun test tests/product/mcpAuthDiscovery.test.ts tests/product/mcpQueryClient.test.ts tests/product/credentialRefresh.test.ts tests/runtime/task-agent-policy.test.ts tests/runtime/opencodeConfig.test.ts tests/opencode/mcpPolicy.test.ts tests/opencode/productMcp.test.ts
|
||||
bun test tests/product/mcpAuthDiscovery.test.ts tests/product/mcpQueryClient.test.ts tests/product/credentialRefresh.test.ts tests/runtime/task-agent-policy.test.ts tests/runtime/opencodeConfig.test.ts tests/opencode/mcpPolicy.test.ts tests/opencode/queryMcp.test.ts tests/mcp/queryServer.test.ts
|
||||
MCP_TEST_PYTHON=<隔离Server环境Python> MCP_TEST_SERVER_ROOT=<Server仓库绝对路径> bun tests/integration/mcp-query-smoke.ts
|
||||
```
|
||||
|
||||
HTTP smoke 只启动自己的 loopback fixture,使用真实两端 SDK,无生产数据库、Keycloak或模型推理。它验证新版发现、真实资源元数据 HTTP 读取、授权服务器元数据及带 resource 的两种 grant(后两者为桩)、67项工具、主机绑定、查询及撤销;真实环境验收记录见上方实际运行环境一节,不能用 fixture 结果代替。前端严格解析已接受这些 MCP 快照来源,未加入浏览器 MCP 凭据或新 UI。
|
||||
HTTP smoke 只启动自己的 loopback fixture,使用真实两端 SDK,无生产数据库、Keycloak或模型推理。它验证新版发现、真实资源元数据 HTTP 读取、授权服务器元数据及带 resource 的两种 grant(后两者为桩)、对应源码版本的工具目录、主机绑定、查询及撤销;真实环境验收记录见上方实际运行环境一节,不能用 fixture 结果代替。前端严格解析已接受这些 MCP 快照来源,未加入浏览器 MCP 凭据或新 UI。
|
||||
|
||||
查询限制回归使用 `python -m pytest tests/unit/test_mcp_query_limits.py tests/unit/test_mcp_query.py tests/unit/test_timeseries_history.py tests/unit/test_timeseries_history_schema.py -q`。实际 PostgreSQL SQL 验证可设置 `MCP_QUERY_TEST_DSN` 后运行 `python -m pytest tests/integration/test_mcp_query_limits_postgres.py -q`;该 DSN 必须指向空的、可丢弃的独立数据库,测试创建临时业务同名 schema 并在事务结束时回滚。它验证数据点跨序列翻页、证据路径/Unicode分页、任务摘要与指定报告、章节稳定路径、项目/任务隔离、既有scope边界、字节门槛和合成摘要,不代替真实项目验收。
|
||||
|
||||
@@ -150,7 +150,7 @@ conda run -n server python scripts/sync_peripheral_work_orders.py --once
|
||||
conda run -n server python scripts/sync_peripheral_work_orders.py
|
||||
```
|
||||
|
||||
使用相同工作身份,并配置 `OPERATIONAL_ACTION_PERIPHERAL_URL`。同步器订阅源工单变更事件,收到消息即读取分页工单,将外围版本按系统数据投影到 `external_work_order` 任务;默认每 300 秒兜底对账。工单完成只表示该外部任务完成,不代表其他调查自动完成。事件协议、`tjwater_v2` 范围与启用步骤见 [外围工单主动同步](peripheral-work-order-events.md)。
|
||||
使用相同工作身份,并配置 `OPERATIONAL_ACTION_PERIPHERAL_URL`。同步器订阅源工单变更事件,收到源工单事件后按 ID 定向读取工单,将外围版本按系统数据投影到 `external_work_order` 任务;默认每 300 秒兜底对账。工单完成只表示该外部任务完成,不代表其他调查自动完成。事件协议、项目存储范围与启用步骤见 [外围工单主动同步](peripheral-sample.md#工单事件主动同步)。
|
||||
|
||||
工况任务面板的刷新按钮只重新读取当前项目的工作台快照,不直接触发外围系统全量对账。外围工单由独立同步器使用服务身份响应后端事件,避免多标签页或普通用户刷新放大全量扫描,也避免浏览器项目与同步器固定项目不一致。
|
||||
|
||||
@@ -190,3 +190,29 @@ OPERATIONAL_TASKS_STORAGE_TEST=1 conda run -n server python -m pytest tests/inte
|
||||
```
|
||||
|
||||
第二条需要当前配置的 PostgreSQL 可连接及测试 DDL 权限。测试在强制回滚事务中创建表结构和隔离项目数据,结束后不保留这些修改;不代替正式迁移。外部工单调用仍使用隔离 HTTP transport,不会操作实际工单系统。该测试覆盖持久化报告、反馈重新排队、旧会话拒绝、人工确认与丢响应回执核对,而非真实模型推理质量。
|
||||
|
||||
### 共用数据库时的执行范围
|
||||
|
||||
任务创建时,Server 将 `OPERATIONAL_TASK_EXECUTION_SCOPE`(留空则使用
|
||||
`ENVIRONMENT`)写入 `workbench.operational_tasks.document.execution_scope`。
|
||||
本地配置 `development`,线上配置 `production`;浏览器不提交此字段,
|
||||
重试保留原范围。`execution_scope` 为必填字段,运行时代码不为缺失值兜底。
|
||||
现有任务通过 `resources/sql/034_operational_task_execution_scope.sql` 一次性补为
|
||||
`production`;已有明确范围的任务保持原值。此回填是历史任务的迁移策略,
|
||||
不代表通过旧记录推断出了创建环境。
|
||||
|
||||
迁移需要与新版本切换协调执行:暂停任务写入,执行每个业务库的 SQL,
|
||||
启动新版本并恢复任务写入。旧版本的严格模型不接受新增 JSON 字段,
|
||||
新版本也拒绝未迁移数据;不能在旧版本仍服务任务请求时单独回填。
|
||||
SQL 同时增加必填范围约束,不设置数据库默认值。回滚版本前需同步处理该约束
|
||||
与 JSON 字段,不得直接切回旧镜像。
|
||||
|
||||
Worker 应连接所属环境的 `OPERATIONAL_WORKER_SERVER_URL` 和
|
||||
`OPERATIONAL_WORKER_AGENT_URL`。Server 在运行列表、领取、租约回收与
|
||||
SCADA recovery 候选查询上限制执行范围;直接领取其他范围返回
|
||||
`403 EXECUTION_SCOPE_MISMATCH`。外部工单反馈与来源撤回也必须匹配任务的执行范围,
|
||||
并在幂等重放前校验;开发环境同步器不能更新生产任务或重置其恢复监测起点。
|
||||
这属于配置隔离,不能替代服务凭据授权:
|
||||
不要让本地 Worker 通过线上 Server 领取任务。普通任务与历史记录仍共用存储。
|
||||
|
||||
修改范围配置需要重启相应 Server。暂停的本地 Worker 不会因代码更新恢复。
|
||||
|
||||
@@ -52,7 +52,7 @@ conda run -n server python scripts/seed_peripheral_sample.py --project-id <proje
|
||||
|
||||
外围面板的“历史监测回放”读取现有项目历史监测源,与上述 `water-demo-v1` 合成数据集是两个独立区域。前端可通过 `TJWATER_PERIPHERAL_SAMPLE_TIME_MODE=demo` 默认使用已知有数据的固定窗口,并用 `TJWATER_PERIPHERAL_SAMPLE_DEMO_START`、`TJWATER_PERIPHERAL_SAMPLE_DEMO_END` 配置开始和结束时间;切到 `recent` 时按 `TJWATER_PERIPHERAL_SAMPLE_RECENT_HOURS` 查询最近时段。界面仍允许在运行时切换演示、最近和自定义窗口。
|
||||
|
||||
本地默认演示窗口为 UTC 2026-04-30 08:00 至 16:00。它只控制历史监测查询,不改变 `water-demo-v1` 自身的 UTC 2026-09-17 00:00 至 12:00 固定内容。
|
||||
前端源码默认窗口为 UTC 2026-09-17 00:00 至 12:00,可由运行时配置覆盖,见 [前端配置定义](../../next-tjwater-frontend/src/shared/config/env.ts)。它只控制历史监测查询,不改变 `water-demo-v1` 自身的固定内容;部署环境的实际配置与数据覆盖需现场核对。
|
||||
|
||||
## 接口与页面
|
||||
|
||||
@@ -62,5 +62,4 @@ conda run -n server python scripts/seed_peripheral_sample.py --project-id <proje
|
||||
|
||||
前端在“外围样板”中单独读取合成样本,展示版本、种子和摘要,以及读数、设备状态、告警和运行历史。加载失败不会清空或阻止既有核查任务及历史回放。
|
||||
|
||||
核查任务、反馈仍需用户依据已保存来源显式提交。生成器不会伪造 Next 来源、现场反馈或业务审批。
|
||||
已审核反馈现在可经人工确认[导入独立新证据](field-feedback-import.md),再另行确认建立[人工复核草稿修订](feedback-review-revisions.md)。专业重分析不在本次已确认的样板反馈范围内;生成器不会自动导入或修订成果。
|
||||
外围业务工单及现场反馈使用 `system_data` 来源,合成生成器不伪造业务上报、人工证据或审批。旧的“反馈导入后建立人工复核草稿”不再作为当前流程;人工事实、外部反馈与调查新轮次的边界见 [工况任务](operational-tasks.md#调查轮次与协作边界),工单来源与事件同步见 [外围服务](peripheral-sample.md)。
|
||||
|
||||
@@ -41,10 +41,63 @@ Forward Hostname 后追加 `/api/sample/v1`。本机开发和预览模式也将
|
||||
|
||||
SCADA 按月数据集、5 分钟时间定位、主动周期监测、质量校验、爆管调查与定位边界见 [SCADA 外围系统时段回放、主动监测与爆管调查设计](scada-peripheral-replay-design.md)。部署回放存储需在项目 BizDB 执行 `031_scada_replays.sql`;项目通用配置需在 MetaDB 执行 `033_metadata_project_settings.sql`。监测频率保存于 `projects.settings.scada_monitoring_interval_seconds`,压力与出厂边界流量角色保存在类型化 `projects.settings.scada_monitoring` 中;本阶段不注册定时任务,也不部署 SCADA 回放 worker。代码和结构迁移完成不等于月度数据或 Agent 联调已经验收。
|
||||
|
||||
演示数据通过 `scripts/seed_peripheral_sample.py` 写入 `tjwater_v2` 的 `peripheral_sample` schema。当前固定窗口为 2026-09-17,所有记录标记为 `synthetic`,并使用 `DEMO-` 前缀。
|
||||
独立监测样本和测试数据通过 `scripts/seed_peripheral_sample.py` 写入 `tjwater_v2` 的 `peripheral_sample` schema。固定窗口为 2026-09-17,样本标记为 `synthetic` 并使用 `DEMO-` 前缀;业务工单保持 `system_data` 来源。
|
||||
|
||||
## 工单事件调查
|
||||
|
||||
`POST /api/sample/v1/work-orders` 可创建 `work_order_type=water_quality_complaint` 的水质投诉事件,或 `work_order_type=burst_report` 的爆管上报,均默认使用业务来源。二者由外围同步器转换为统一事件调查任务,再由后端 worker 可靠交给 Agent;浏览器不再负责启动调查。事件和执行请求 ID 均由稳定业务身份确定,回执恢复或重试不会重复创建任务。普通业务录入无需绑定演练场景码,也不会向 Agent 注入预设监测结果或邻区投诉情况。
|
||||
|
||||
外围服务只保存工单,不直接调用 Agent。`scripts/sync_peripheral_work_orders.py` 使用受认证的后台身份同步外部版本并创建调查任务,`scripts/run_operational_worker.py` 再将持久化的待执行任务交给 Agent,因此浏览器关闭后流程仍可继续。
|
||||
|
||||
## 工单事件主动同步
|
||||
|
||||
工单的创建、编辑、现场反馈和删除改为由来源事件触发同步。浏览器继续使用已有工作台 SSE,不需要新增连接或定时工单轮询。
|
||||
|
||||
```text
|
||||
外围工单写入
|
||||
→ 同一 PostgreSQL 事务保存源记录、历史版本和变更事件
|
||||
→ COMMIT 后 NOTIFY
|
||||
→ Server 工作台 SSE 唤醒外围同步器
|
||||
→ 同步器通过授权 API 更新任务投影
|
||||
→ task/action/notification 事件推送给浏览器
|
||||
```
|
||||
|
||||
### 项目与存储范围
|
||||
|
||||
当前 HTTP 请求按已授权项目的 Metadata 路由访问其 BizDB,源变更事件复用 `workbench.domain_event_log` 的 `workbench.work_order.changed` topic。聚合类型为 `peripheral_work_order`,元数据只包含 `source=peripheral_sample`、工单 ID 和 `created|updated|deleted`,不包含正文。
|
||||
|
||||
源记录、历史版本及变更事件在同一事务写入,提交后通过 PostgreSQL NOTIFY 唤醒监听;事务失败整体回滚,幂等重放不再次产生事件。外围服务只保存源事实,任务投影、调查创建和通知仍经 Server 状态机与授权 API 处理。
|
||||
|
||||
代码仅在当前项目已有事件表时写事件,不自动创建表、不回退其他项目数据库;表缺失时跳过事件发布,已有表写入失败则使源变更事务失败。HTTP 项目路由、数据库结构是否部署、后台同步器是否配置该项目是三个条件,需要分别核验。早期测试和迁移仅在 `tjwater_v2` 执行的记录不再作为运行时代码的固定库限制。扩展遵循 [数据库变更门禁](workbench-database-rollout.md)。
|
||||
|
||||
### 同步器
|
||||
|
||||
从 Server 仓库运行已有命令:
|
||||
|
||||
```bash
|
||||
conda run -n server python scripts/sync_peripheral_work_orders.py
|
||||
conda run -n server python scripts/sync_peripheral_work_orders.py --once
|
||||
```
|
||||
|
||||
继续复用 `OPERATIONAL_WORKER_*` OAuth 工作身份及 `OPERATIONAL_ACTION_PERIPHERAL_URL`,不需要新账号或密钥。启动和游标过期时先读取事件游标,再进行全量对账;扫描期间的事件从该游标继续补读。相同源版本仍由原幂等键去重。源事件按工单 ID 唤醒同步器;动作回执事件只补查该任务关联的工单,避免源创建早于回执保存时漏同步。任务投影事件不再触发全量扫描。
|
||||
|
||||
正常状态收到源事件即处理对应 ID:`GET /api/v1/operational-tasks?peripheral_work_order_id=<uuid>` 只返回同项目的来源任务和动作回执关联任务;外围接口只读取该工单,不再分页遍历全部工单和全部调查详情。重复 ID 合并;目标工单的明确删除只撤回该工单投影,不影响其他任务。
|
||||
|
||||
内存待处理 ID 最多 200 个,超过上限合并为一次全量对账,不持续扩张队列。这是突发积压保护,不增加公共配置项。
|
||||
|
||||
默认每 300 秒仍进行一次全量对账,处理漏通知、断线、旧批量脚本直接写库及重启。持续收到事件不会推迟全量对账期限。可用 `--reconcile-seconds` 调整这一兜底周期(最小 1 秒),不再复用工况 worker 的 30 秒租约检查周期。同步失败按 5、10、20、40、60 秒退避重试;事件连接断开每 5 秒重试,保留最后游标。扫描开始前取出待处理 ID,扫描期间新到达的 ID 留给下一轮;失败的 ID 会放回队列,重试不依赖新消息。
|
||||
|
||||
浏览器收到源事件时不会提前刷新业务快照,而是等待任务投影事件后读取对应任务。工况手动刷新仍只读 Server 权威快照。工单事件不等于 Header 通知;只有需要处理的业务待办或告警才进入通知列表。
|
||||
|
||||
游标顺序、共享 Hub 查询与兼容边界见 [工作台事件链路](workbench-event-refinement.md)。新增查询参数已从 Server OpenAPI 同步生成 Agent 类型,不另建手写契约。
|
||||
|
||||
### 验证与启用
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests/unit/test_peripheral_push_sync.py tests/unit/test_peripheral_work_order_events.py tests/unit/test_peripheral_task_sync.py -q
|
||||
PERIPHERAL_SAMPLE_LIVE_TEST=1 conda run -n server python -m pytest tests/integration/test_peripheral_work_order_events.py -q
|
||||
```
|
||||
|
||||
数据库测试强制限定 `tjwater_v2`,使用随机项目键和回滚事务,不保留测试工单或事件。覆盖创建、修改、反馈、删除、幂等重放及回滚无事件。HTTP 隔离测试覆盖源事件即时投影、游标重置、断线续传、失败重试和监听取消。
|
||||
|
||||
发布时外围服务随主 Server 镜像更新,同步器是独立常驻进程;需按维护者授权的发布流程分别更新 Server、同步器及配套前端。提交代码不证明现有进程已重载。回退应用代码不涉及数据结构回滚,已有事件仍可正常补读;回退到旧同步器后恢复周期对账。
|
||||
|
||||
@@ -1,50 +0,0 @@
|
||||
# 外围工单主动同步
|
||||
|
||||
工单的创建、编辑、现场反馈和删除改为由来源事件触发同步。浏览器继续使用已有工作台 SSE,不需要新增连接或定时工单轮询。
|
||||
|
||||
```text
|
||||
外围工单写入
|
||||
→ 同一 PostgreSQL 事务保存源记录、历史版本和变更事件
|
||||
→ COMMIT 后 NOTIFY
|
||||
→ Server 工作台 SSE 唤醒外围同步器
|
||||
→ 同步器通过授权 API 更新任务投影
|
||||
→ task/action/notification 事件推送给浏览器
|
||||
```
|
||||
|
||||
## 数据库范围
|
||||
|
||||
本次仅启用 `tjwater_v2`。没有新增表、列或迁移;复用 `workbench.domain_event_log` 的 `workbench.work_order.changed` topic。源事件的聚合类型为 `peripheral_work_order`,元数据只包含 `source=peripheral_sample`、工单 ID 和 `created|updated|deleted`,不包含工单正文。事务失败时,源记录、历史、事件及通知一起回滚;同一请求的幂等重放不再产生事件。
|
||||
|
||||
外围服务只写源记录及源变更事实,不直接改业务任务。任务投影、调查创建和通知仍通过 Server 状态机和现有授权 API 处理。其他数据库不写入该源事件;扩展前遵循 [数据库变更门禁](workbench-database-rollout.md)。事件表缺失或无写权限时,`tjwater_v2` 的工单写入返回存储错误,不会假装成功。
|
||||
|
||||
## 同步器
|
||||
|
||||
从 Server 仓库运行已有命令:
|
||||
|
||||
```bash
|
||||
conda run -n server python scripts/sync_peripheral_work_orders.py
|
||||
conda run -n server python scripts/sync_peripheral_work_orders.py --once
|
||||
```
|
||||
|
||||
继续复用 `OPERATIONAL_WORKER_*` OAuth 工作身份及 `OPERATIONAL_ACTION_PERIPHERAL_URL`,不需要新账号或密钥。启动和游标过期时先读取事件游标,再进行全量对账;扫描期间的事件从该游标继续补读。相同源版本仍由原幂等键去重。源事件按工单 ID 唤醒同步器;动作回执事件只补查该任务关联的工单,避免源创建早于回执保存时漏同步。任务投影事件不再触发全量扫描。
|
||||
|
||||
正常状态收到源事件即处理对应 ID:`GET /api/v1/operational-tasks?peripheral_work_order_id=<uuid>` 只返回同项目的来源任务和动作回执关联任务;外围接口只读取该工单,不再分页遍历全部工单和全部调查详情。重复 ID 合并;目标工单的明确删除只撤回该工单投影,不影响其他任务。
|
||||
|
||||
内存待处理 ID 最多 200 个,超过上限合并为一次全量对账,不持续扩张队列。这是突发积压保护,不增加公共配置项。
|
||||
|
||||
默认每 300 秒仍进行一次全量对账,处理漏通知、断线、旧批量脚本直接写库及重启。持续收到事件不会推迟全量对账期限。可用 `--reconcile-seconds` 调整这一兜底周期(最小 1 秒),不再复用工况 worker 的 30 秒租约检查周期。同步失败按 5、10、20、40、60 秒退避重试;事件连接断开每 5 秒重试,保留最后游标。扫描开始前取出待处理 ID,扫描期间新到达的 ID 留给下一轮;失败的 ID 会放回队列,重试不依赖新消息。
|
||||
|
||||
浏览器收到源事件时不会提前刷新业务快照,而是等待任务投影事件后读取对应任务。工况手动刷新仍只读 Server 权威快照。工单事件不等于 Header 通知;只有需要处理的业务待办或告警才进入通知列表。
|
||||
|
||||
游标顺序、共享 Hub 查询与兼容边界见 [事件链路收敛](workbench-event-refinement.md)。新增查询参数已从 Server OpenAPI 同步生成 Agent 类型,不另建手写契约。
|
||||
|
||||
## 验证与启用
|
||||
|
||||
```bash
|
||||
conda run -n server python -m pytest tests/unit/test_peripheral_push_sync.py tests/unit/test_peripheral_work_order_events.py tests/unit/test_peripheral_task_sync.py -q
|
||||
PERIPHERAL_SAMPLE_LIVE_TEST=1 conda run -n server python -m pytest tests/integration/test_peripheral_work_order_events.py -q
|
||||
```
|
||||
|
||||
数据库测试强制限定 `tjwater_v2`,使用随机项目键和回滚事务,不保留测试工单或事件。覆盖创建、修改、反馈、删除、幂等重放及回滚无事件。HTTP 隔离测试覆盖源事件即时投影、游标重置、断线续传、失败重试和监听取消。
|
||||
|
||||
上线时更新外围服务、Server 同步器及前端,再按既有发布流程重启对应进程。提交代码不会自动重启现有服务;旧同步器进程仍按旧的 30 秒周期运行。回退应用代码不涉及数据结构回滚,已有事件仍可正常补读;回退到旧同步器后恢复周期对账。
|
||||
@@ -16,10 +16,37 @@
|
||||
|
||||
本版冻结设备、区间、规则及批次输出,按批次原始值摘要校验源变更。完整月度数据清单、全月源版本摘要、独立事件真值和检测效果评估仍是后续数据准备工作;能力接口仅判断回放存储是否存在与当前操作权限,不声明月度数据已经就绪。现有 14 天数据不能当作一个月验收。
|
||||
|
||||
Agent 查询本轮已检测的固定批次快照,受任务的 replay_id、knowledge_cutoff、batch_sequence 约束。开始前历史只供后台基线初始化,本版调查查询不返回该部分;爆管定位已支持受回放截止约束的冻结输入适配,其他未适配计算工具继续拒绝。新 [SCADA 调查 Skill](../resources/work-order-skills/scada-anomaly/SKILL.md) 要求据实说明这些证据限制。运行异常自动建立调查,数据质量异常仅建立告警,不触发模型;恢复时推进调查代次。真实模型推理与报告效果尚未联调。
|
||||
Agent 查询本轮已检测的固定批次快照,受任务的 replay_id、knowledge_cutoff、batch_sequence 和设备集合约束。新回放首次监测以只读一致快照冻结开始前 15 天历史,负序号静默基线按最多 2,000 行分块暂存为 `clean_prepared`;全部读取完成后,在一个 BizDB 短事务中把批次转为 `detected` 并原子发布基线与初始化标记,历史查询和定位对照才能读取。
|
||||
|
||||
关联现场工单完工后的恢复复核采用独立证据链:Server 以现场反馈业务时间 T 读取项目时序库中严格晚于 T 的最近一帧及事件前对照,保存恢复监测证据后接续原调查。此路径不依赖回放推进,也不修改原异常快照的知识上限或批次;Agent 读取保存的恢复证据,不通过普通回放查询越界取数。详见[恢复复核流程](burst-response-workflow.md)。
|
||||
|
||||
读取各页期间不持有 BizDB 事务,内存只保留当前页和设备短历史。固定准备预算为 1,000,000 行、512 MiB 序列化批次,覆盖 500 设备十五天每 15 分钟一条所需的 720,000 行并留有余量。超限不保存超限页,状态明确返回 `BASELINE_BUDGET_EXCEEDED` 及中文原因;未发布批次不进入观测、监测或待处理批次。失败重试只清理当前项目、本轮回放的未发布基线,从新的一致源快照重新准备,已发布基线保持不变。
|
||||
|
||||
旧回放没有冻结基线时明确拒绝回放前查询,不回落到实时 TSDB。爆管定位支持受回放截止约束的冻结输入适配,其他未适配计算工具继续拒绝。[SCADA 调查 Skill](../resources/work-order-skills/scada-anomaly/SKILL.md) 要求据实说明证据限制。运行异常自动建立调查,数据质量异常仅建立告警;恢复时推进调查代次。真实模型推理与报告效果尚未联调。
|
||||
|
||||
控制进度和事件状态使用有界轮询,沿用告警流发布业务告警;本版未新增 scada.replay.changed SSE 主题。演示面板只显示最近一个周期的组合结论,逐测点输出和完整事件历史由分页 API 保留给诊断与调查入口,不在主面板重复铺开。
|
||||
|
||||
## Case 3 临时定位 mock(2026-11-07)
|
||||
|
||||
`tjwater2_v2` 及其隔离演示项目 `tjw_bench_tjwater2_20261105` 中,2026-07-16
|
||||
08:00–13:00(北京时间)的 monitoring 定位暂用快速 mock。异常窗起点可提前 15 分钟,
|
||||
终点必须已经到达事件时刻且不超过事件结束;其他项目、时间和 simulation 输入仍运行原算法。
|
||||
mock 直接返回并沿用 MCP 权限、回放冻结证据校验、调查证据与幂等回执,不提交慢计算队列。
|
||||
|
||||
预设候选为同一 DN600 线路的 `515870`。每次读取当前拓扑,验证候选与事件位于完全相同的
|
||||
可隔离分区、所需边界阀一致,且不跨阀的同径管线路径距离为 200–500 m。距离定义为实际
|
||||
事件节点沿管线至候选管段最近点的距离;当前为 495.368 m,不以两条管段中点计算。
|
||||
无法满足任一条件时返回空候选,不扩大隔离区,也不回落到耗时定位。
|
||||
|
||||
候选结果固定返回 `similarity=0.9708` 和 `similarity_percent=97.08`,展示为97.08%。
|
||||
同一案例在 T 与 T+15 分钟返回相同候选与排序。Agent可见结果不再附加 `mock`、
|
||||
`demo_mock` 说明、空相似度或 `simulation_times=0` 等阻断提示,也不虚构水力仿真次数。
|
||||
MCP 回放证据仍保留 `origin=synthetic`;固定相似度是该演示路径的预设返回值,未运行水力定位。
|
||||
取得候选后继续地图展示、隔离边界与影响分析。
|
||||
事件真值仅在服务端 fixture 中用于约束校验,
|
||||
不返回给 Agent。撤回时移除 `app/services/burst_location_demo.py` 的两项目匹配即可恢复原路径;
|
||||
不修改观测、管网或关阀数据。本项不代表自动监测建单或真实 Agent 报告已完成验收。
|
||||
|
||||
## 已确定的设计边界
|
||||
|
||||
- 数据集覆盖一个月,包含多次独立异常及各次事件发生、持续、恢复的过程。
|
||||
@@ -59,7 +86,7 @@ MetaDB 的 `projects.settings` 为项目级通用 JSONB 对象,默认 `{}`,
|
||||
"settings": {
|
||||
"scada_monitoring_interval_seconds": 900,
|
||||
"scada_monitoring": {
|
||||
"rule_version": "burst-monitor-v1",
|
||||
"rule_version": "burst-monitor-v2",
|
||||
"pressure_device_ids": ["明确的压力设备 ID"],
|
||||
"outlet_flow_device_ids": ["明确的出厂边界流量设备 ID"]
|
||||
}
|
||||
@@ -67,10 +94,62 @@ MetaDB 的 `projects.settings` 为项目级通用 JSONB 对象,默认 `{}`,
|
||||
}
|
||||
```
|
||||
|
||||
当前代码已类型校验并在创建回放时冻结 `scada_monitoring`。项目没有出厂流量或不足 5 个完整压力点时保持缺项,能力接口说明原因,监测回执返回 `insufficient_data`;不能为了让演示通过而把其他流量点冒充出厂流量或虚构设备。`tjwater_v2` 现有目录只有通用 `pipe_flow` 类型及模型管段映射,没有足以证明“出厂边界”的角色字段,因此本轮未写入该嵌套配置。
|
||||
当前代码已类型校验并在创建回放时冻结 `scada_monitoring`。不足 5 个完整压力点时保持缺项,能力接口说明原因,监测回执返回 `insufficient_data`。加权模式允许出厂流量缺项,其贡献为零且不重分配权重;`all_signals` 模式仍要求出厂流量就绪。不能把普通流量点冒充出厂流量或虚构设备。`tjwater_v2` 未补写未经核实的监测角色。
|
||||
|
||||
部署前需在 MetaDB 执行 `resources/sql/033_metadata_project_settings.sql`。迁移会将实验版本可能存在的独立监测间隔列搬入 `settings`,核对冲突后移除旧列;重复执行不丢失已有配置。当前 `tjwater_v2` 与 `tjwater2_v2` 均配置 900 秒,即 15 分钟的主动监测周期。该参数不改变 5 分钟时间轴定位,本阶段明确不实现通用定时任务调度。
|
||||
|
||||
### 动态阈值与加权判定(2026-11-07)
|
||||
|
||||
新建回放默认使用 `burst-monitor-v2`,配置仍位于项目 `settings.scada_monitoring`:
|
||||
|
||||
```json
|
||||
{
|
||||
"rule_version": "burst-monitor-v2",
|
||||
"decision_policy": "weighted",
|
||||
"pressure_device_ids": ["已核实的至少5个压力设备"],
|
||||
"outlet_flow_device_ids": ["已核实的出厂流量设备"],
|
||||
"weights": {"burst_detection": 0.5, "pressure": 0.4, "outlet_flow": 0.1},
|
||||
"alert_threshold": 0.8
|
||||
}
|
||||
```
|
||||
|
||||
三项权重均为 0–1 且总和必须为 1。每项达到自身阈值时贡献其固定权重,加权和达到
|
||||
`alert_threshold` 才判为疑似爆管。默认侦测与压力共同命中可得 0.90,出厂流量仅提供
|
||||
0.10 辅助支持。缺数或未配置的流量贡献为零,不把剩余权重重新归一化;压力或侦测
|
||||
数据不足时仍返回数据不足。综合分不是概率,也不代表现场确诊。
|
||||
|
||||
首步只用回放开始前完整 15 天的冻结历史学习阈值,在历史发布事务中一起保存
|
||||
`monitoring_calibration`;随后固定使用,事件发生后的数据不会参与更新阈值。准备时
|
||||
为角色测点与参与相对变化规则的测点保留紧凑数值序列,最多 500 个测点;
|
||||
15 分钟周期下数值数组上限约 5.8 MB。继续逐页保存基线,不重新聚合全部设备的 raw。
|
||||
15 天历史不完整就声明不足,不插值或借用未来数据。
|
||||
|
||||
压力与流量按设备计算此前若干样本中位值对应的历史变化率,阈值取正常变化中位值加
|
||||
3 倍稳健波动尺度,并分别保留 3%、5% 下限。侦测阈值从冻结正常历史的得分估计,取
|
||||
`min(-0.04, 正常得分中位值-3×稳健波动尺度, 正常得分5%分位值-0.02)`。
|
||||
稳健波动尺度在本实现中定义为 `1.4826×MAD`。保留真实侦测分数,调整判定阈值;
|
||||
不按已知事件日期制造异常分数。周期回执保存实际阈值、权重、各项贡献和校准截止时间。
|
||||
|
||||
加权流程的测点诊断采用 `normal-phase-envelope@1`:按前 15 天同一日采样相位的正常值
|
||||
建立 `[历史最小值−3×稳健波动尺度, 历史最大值+3×稳健波动尺度]` 包络,保留数值精度余量。
|
||||
普通测点越过所配置方向的同期包络即可启动调查,不再用固定 30% 门槛筛选。
|
||||
零基准、日常反向流量、近零分母的百分比变化不再单独制造异常。压力/出厂流量的
|
||||
爆管参考贡献还需越过其历史动态变化阈值;只命中同期包络的有效单点异常仍可进入调查。
|
||||
显式水质上下限保留。角色参考的滚动基线严格取目标观测之前的样本,与校准窗口对齐;
|
||||
本周期必须有新观测,模型必须具备周期终点读数,不能复用上一时刻的得分。
|
||||
|
||||
本轮校准版本为 `scada-monitor-calibration@2`,并保存 `diagnostic_policy`、各测点包络与
|
||||
截止时间。缺少完整历史的测点不回退到固定百分比,并在监测回执的缺项中列出。
|
||||
已预热的加权回放缺少新版校准时,推进/重试返回 409,明确要求停止后新建;
|
||||
旧相对规则事件不进入新诊断信号,已有历史事件和工单不删除。新事件采用独立规则状态和策略标识。
|
||||
请求 schema 中的 `relative_change.threshold` 仅为既有协议兼容保留,在加权测点诊断路径
|
||||
不参与触发;同一规则的窗口参数仍用于角色指标的动态变化计算。
|
||||
|
||||
既有冻结回放没有 `decision_policy` 时沿用三项同时命中的旧判定;新加权配置需新建
|
||||
回放才生效,不更改旧回放的历史结论。显式 `decision_policy=all_signals` 仍为旧规则兼容路径,
|
||||
不用于当前 Case3 加权演示,也不是新版校准缺失时的回退路径。
|
||||
本项修改无需数据库迁移。
|
||||
|
||||
## 实施前基础与原设计差距
|
||||
|
||||
下表记录实施前基础,用于说明改动缘由;本次实际实现以首版范围及后文接口表为准。设备目录采样周期为本轮讨论中 2026-11-02 的只读核查快照,与项目监测任务间隔是两个概念。
|
||||
@@ -82,7 +161,7 @@ MetaDB 的 `projects.settings` 为项目级通用 JSONB 对象,默认 `{}`,
|
||||
| [SCADA API](../app/api/v1/endpoints/timeseries/scada.py)、[仓储](../app/infra/db/timescaledb/repositories/scada.py) | 范围读取、最新值读取、批量插入;当前范围读取返回 raw 和 corrected,缺少回放游标及完整流水线 | 增加只读 raw 分页契约;复用存储查询,补齐处理检查点、清洗与检测调度 |
|
||||
| [清洗服务](../app/services/timeseries_analysis.py) | 从原始值计算并更新 `cleaned_value`;目前支持压力和流量 | 在线质量标记、因果修正及选择值依据;水质不能直接套用现有清洗 |
|
||||
| [压力清洗](../app/algorithms/scada_cleaning/pressure_series.py)、[流量清洗](../app/algorithms/scada_cleaning/flow_series.py) | 包含居中窗口、后向补值、Kalman 平滑;流量路径还会补为 1 分钟序列 | 在线路径只使用已到达的数据,保留设备频率,不整月预清洗后再模拟检测 |
|
||||
| [外围工单事件同步](peripheral-work-order-events.md) | 持久事件、通知唤醒、后台同步和定期对账 | 复用可靠处理方式,SCADA 的批次有自己的持久状态 |
|
||||
| [外围工单事件同步](peripheral-sample.md#工单事件主动同步) | 持久事件、通知唤醒、后台同步和定期对账 | 复用可靠处理方式,SCADA 的批次有自己的持久状态 |
|
||||
| [调查 worker](../scripts/run_operational_worker.py) | 可靠投递 `event_investigation`,恢复执行租约,管理 Agent 执行代次 | 接收由 SCADA 异常事件建立的调查;不承担每条观测的检测 |
|
||||
| [任务告警](../app/domain/schemas/task_alert.py) | 支持 raised、acknowledged、resolved;当前来源必须携带任务和执行标识 | 增加真实的回放检测来源,不能为调查前告警伪造 `task_id`、`run_id` |
|
||||
| [MCP 上下文](../app/mcp/contexts.py)、[SCADA 查询](../app/mcp/query_service.py) | 校验任务、项目、执行代次和会话,保存查询证据 | 在所有回放调查读取入口执行统一的时间与数据版本约束 |
|
||||
@@ -202,7 +281,7 @@ SCADA 单步命令在当前 API 请求内按原始时间顺序完成 raw 读取
|
||||
|
||||
演示建立后改变时间区间需要停止并创建新回放;保留旧回放的事件和证据,不把旧回放时间游标直接重设。已完成区间需要再次演示时,新建独立回放;不自动循环处理和重复触发调查。
|
||||
|
||||
选定区间之前的可用历史仅用于已声明的基线初始化,不占用主时间轴、不作为新到达批次、不预触发此前事件。基线不足时在当前区间显示具体可用性,不能悄悄扩大演示范围或读取区间之后的数据。
|
||||
选定区间之前的可用历史用于已声明的基线初始化及冻结历史对照,不占用主时间轴、不计入当前处理量、不预触发此前事件,也不回写 TSDB 的 cleaned_value。基线不足时保留实际行数和模型数据缺口,不能悄悄扩大演示范围或读取区间之后的数据。
|
||||
|
||||
处理到选定终点后停在终点,显示数据处理是否完成及剩余调查数量;不会自动继续处理该月剩余数据。空区间明确显示没有观测,不能通过控件动画假装已经收到 SCADA 数据。
|
||||
|
||||
@@ -234,8 +313,8 @@ SCADA 单步命令在当前 API 请求内按原始时间顺序完成 raw 读取
|
||||
2. 汇总压力下降幅度。只使用映射明确、单位一致、质量合格的压力测点,并同时保留异常测点数、覆盖测点数、基线窗口和最大或中位下降幅度。
|
||||
3. 汇总出厂流量变化。设备必须在项目配置或设备档案中明确标注为“出厂边界流量”,不能把任意 `pipe_flow` 测点当作出厂流量;保留方向、单位、基线和增量。
|
||||
4. 条件满足时运行爆管侦测。现有侦测要求至少 5 个完整压力点和目标时刻之前 15 天同周期数据;不满足时返回 `baseline_not_ready` 或 `sensor_coverage_insufficient`,不能将“未运行”解释为“无异常”。
|
||||
5. 组合上述证据形成监测结论;只有疑似爆管才创建 `event_investigation`,工作流使用现有 `burst_report`,并绑定本轮 `replay_context`。正常周期只保存回执,不启动 Agent;数据不足周期保存缺口并在界面展示。
|
||||
6. `burst_report` Skill 核对证据后决定是否调用爆管定位。定位必须有 m³/s 的有依据漏量估计或明确情景假设,以及模型节点映射、异常窗和正常对照窗;输入不具备时保持“疑似爆管、定位待补充”,不能填默认漏量。
|
||||
5. SCADA 数据诊断发现任一已配置指标异常,就创建 `event_investigation`,使用现有 `scada_anomaly` 工作流并绑定本轮 `replay_context`。调查先核验数据,再进行爆管诊断;综合分用于判断爆管嫌疑,不能作为启动调查的前置门槛。没有异常的正常周期只保存回执。数据不足但已有有效异常信号时仍发起调查并列出缺口。
|
||||
6. `scada_anomaly` Skill 核对异常证据并进行爆管诊断,输入具备时决定是否调用爆管定位。定位必须有 m³/s 的有依据漏量估计或明确情景假设,以及模型节点映射、异常窗和正常对照窗;输入不具备时保持“疑似爆管、定位待补充”,不能填默认漏量。
|
||||
|
||||
组合判定采用可版本化策略,首版演示建议如下:
|
||||
|
||||
@@ -243,10 +322,10 @@ SCADA 单步命令在当前 API 请求内按原始时间顺序完成 raw 读取
|
||||
| --- | --- | --- |
|
||||
| `insufficient_data` | 必需点位缺测、质量无效、单位或角色未配置,或爆管侦测基线不满足 | 保存缺口,不声明正常,不启动定位 |
|
||||
| `normal` | 数据完整,爆管侦测未命中,压力与出厂流量规则均未命中 | 保存周期回执,不创建 Agent 任务 |
|
||||
| `attention` | 只有单项指标命中,或指标间方向不一致 | 保存观察结果;下一步继续监测,必要时由人发起调查 |
|
||||
| `suspected_burst` | 爆管侦测命中,且压力下降与出厂流量增加在同一业务时间窗内相互支持 | 创建一次 `burst_report` 调查;定位输入具备后进入计算 |
|
||||
| `attention` | 单项指标命中、指标方向不一致,或周期内曾出现测点异常 | 创建或复用 `scada_anomaly` 调查,进入爆管诊断 |
|
||||
| `suspected_burst` | 加权分达到门槛,或旧版联合策略的三项指标共同命中 | 同一份 `scada_anomaly` 调查标记爆管嫌疑;定位输入具备后进入计算 |
|
||||
|
||||
爆管侦测和压力下降存在相关性,因此回执不能把两者描述成完全独立的两票。出厂流量也会受泵站调度、储水设施和正常用水变化影响。组合结论只是调查触发条件,不是现场确诊。
|
||||
爆管侦测和压力下降存在相关性,因此回执不能把两者描述成完全独立的两票。出厂流量也会受泵站调度、储水设施和正常用水变化影响。组合结论用于评估爆管嫌疑,不是调查启动门槛或现场确诊。
|
||||
|
||||
回放中的 MCP 定位已走 Server 持有的安全适配层:按 `replay_id`、`knowledge_cutoff` 和 `batch_sequence` 从已固化清洗输出组装压力与可选流量序列,再调用现有计算队列。回放只允许 monitoring 模式,并要求显式异常窗、正常对照窗和有依据的 `burst_leakage`;不能直接查询 TSDB 中尚未回放的未来记录。步长爆管侦测同样只读取最近 15 天已固化输出。计算结果保存输入、算法与证据引用,任务恢复或重试仍使用相同知识上限。
|
||||
|
||||
@@ -366,7 +445,7 @@ BizDB 批次状态是持久待处理队列,通知只负责唤醒。后台启
|
||||
|
||||
## Agent 调查与时间隔离
|
||||
|
||||
主动单步在边界形成组合监测回执:一般关注和数据不足只保存回执,只有 `suspected_burst` 进入 `burst_report`。SCADA 演示请求只负责建立持久调查任务,不要求先生成一条合成 test_data,也不在请求内执行 Agent。
|
||||
主动单步在边界形成组合监测回执:任一有效指标异常进入 `scada_anomaly` 调查;缺少爆管判别资料不能阻止已知异常进入调查。SCADA 演示请求只负责建立持久调查任务,不要求先生成一条合成 test_data,也不在请求内执行 Agent。
|
||||
|
||||
告警事实与“待建立调查”意图在事件事务中保存。创建调查失败时后台对账重试;组合监测调查的来源稳定标识使用 `scada_replay:<replay_id>:monitor:<period_end>:<rule_version>`,一般事件继续使用 `scada_replay:<replay_id>:event:<event_id>`,均沿用现有外部来源幂等约束。事件不因 Agent 不可用而丢失。
|
||||
|
||||
@@ -391,7 +470,7 @@ BizDB 批次状态是持久待处理队列,通知只负责唤醒。后台启
|
||||
| `GET /api/v1/scada-replays/capabilities` | 回放表可用性、写权限、监测频率、爆管组合监测角色是否就绪及最近一轮状态;不代表完整月度数据就绪 |
|
||||
| `POST /api/v1/scada-replays` | 固定区间、设备、规则和质量量程,返回 ready;携带稳定 request_id |
|
||||
| `GET /api/v1/scada-replays/{replay_id}` | 权威控制、时间、计数、设备与固定规则、当前输出及故障 |
|
||||
| `POST /api/v1/scada-replays/{replay_id}/commands` | play、pause、step、set_speed、stop、retry;request_id + expected_revision |
|
||||
| `POST /api/v1/scada-replays/{replay_id}/commands` | step、monitor、stop、retry;request_id + expected_revision,monitor 另需带时区的 at_time |
|
||||
| `GET /api/v1/scada-replays/{replay_id}/observations?through=...` | 仅本轮已读取时间范围内的历史快照,每设备最近一条、最多 500 个 |
|
||||
| `GET /api/v1/timeseries/scada-readings/raw` | 当前项目 TSDB 原始值联合游标分页;不插入、不隐式清洗 |
|
||||
| `GET /api/v1/scada-replays/{replay_id}/events?offset=0&limit=100` | 检测变化与调查关联,返回 items、next_offset |
|
||||
@@ -437,7 +516,7 @@ next_cursor 绑定项目、设备、查询区间并固定本轮分页时间上
|
||||
- 周期内多批次单步、重复单步拒绝、并发请求互斥和尾步完成状态。
|
||||
- 5 分钟时间定位与 15 分钟监测周期相互独立、非整点起点、周期配置变更不影响当前回放、尾步不足完整周期和未配置时禁止开始。
|
||||
- 每个步长边界只产生一份幂等监测回执;正常、数据不足、单指标关注和疑似爆管分别处理,不为正常周期创建 Agent。
|
||||
- 爆管侦测基线不足、压力下降、出厂流量变化、设备角色错误、指标方向冲突和多指标同时命中;只有疑似爆管建立 `burst_report`。
|
||||
- 爆管侦测基线不足、压力下降、出厂流量变化、设备角色错误、指标方向冲突和多指标同时命中;单指标、部分资料不足及疑似爆管均复用 `scada_anomaly` 调查入口,正常且无异常信号时不建单。
|
||||
- 回放 MCP 侦测与定位只能读取固定批次和知识截止前数据;定位缺少漏量依据时停止在待补充,不调用算法。
|
||||
- 选定小时区间、跨日区间和再次演示;刻度与进度只对应选定区间,基线初始化不释放旧事件,到终点不自动播放整月。
|
||||
- FM 红线固定、刻度带移动、拖动回看不重复投递、未来位置不泄露读数;桌面与 375px、320px 窄屏下的长日期、键盘、减少动态效果及玻璃回退。
|
||||
@@ -474,3 +553,46 @@ next_cursor 绑定项目、设备、查询区间并固定本轮分页时间上
|
||||
| 仓库 | Server、供水前端、Agent 的 git diff --check 通过;未提交、推送或部署 |
|
||||
|
||||
生产联调仍需核验真实 HTTP 接口、调查执行链路和 Agent 报告,并单独准备完整月度数据及事件真值。不能将以上局部测试解释为这些环节已经完成。
|
||||
|
||||
### 同一事故的告警与调查去重(2026-11-07)
|
||||
|
||||
同一轮回放首次检出任一有效指标异常时,持久保存 `monitoring_incident`,发送一次 SCADA 异常告警并创建一份 `scada_anomaly` 调查工单,先核验数据再进行爆管诊断。T+15 等后续周期复用事故 ID 和工单,回执 `incident.action=ongoing`;缺数、部分命中或仍处低压时也保持该事故。每个周期的检测结果仍独立保存,原工单的证据截止时间不随每个周期滚动;若单项异常首次升级为疑似爆管,使用既有 `advance_replay_investigation` 推进同一工单的调查代次,为新代次冻结新增证据,仍不重复报警或建单。
|
||||
|
||||
只有连续两个相邻的正常监测周期,且首次命中的压力测点全部有新的有效读数、达到事故开始时冻结基准的恢复范围,才将事故标记为恢复并解除同一条告警。恢复范围使用首次压力基准与该点下降阈值,不能因滚动基线适应低压就解除告警;缺数、过期读数或跳过周期会中断连续恢复确认。确认恢复后再次检出指标异常,才新建事故和调查。流量异常的恢复也需核对其冻结基准,不能仅依赖已适应高流量的滚动基线。
|
||||
|
||||
主动步进中,压力、流量变化与水质上下限规则保留检测事件作为证据,测量指标的异常通知统一由周期诊断发出;数据质量通知继续独立。周期内已恢复的短暂异常仍保存为本周期调查依据,不因结束时读数恢复而丢失。状态、告警、工单及批次在原有处理锁和同一业务事务内保存,失败重试沿用确定的请求标识。旧回放首次升级时可承接上一周期已有工单,不批量合并历史重复工单。主动创建新一轮回放仍属于独立演示,不跨回放合并事故。
|
||||
|
||||
精简演示面板保留时间区间与 FM 时间轴,集中显示最新结论、监测区间、综合分与门槛;指标权重和命中情况按需展开。持续事故明确显示“沿用已有调查工单,本周期未重复报警”,下一周期处理期间保留上一周期结论并标明处理进度。
|
||||
|
||||
|
||||
### 面板规则与实际流程(2026-11-07)
|
||||
|
||||
面板以“SCADA 数据诊断 → 任一指标异常 → 爆管诊断 → 报告”为入口,不以爆管综合分筛掉单指标异常。`/capabilities.monitoring_config` 返回新一轮采用的项目规则;回放快照的 `monitoring_config` 和 `monitoring_calibration` 返回本轮实际冻结的配置及阈值。进行中的回放不套用后来修改的项目配置,旧回放未保存判定策略时按原有 `all_signals` 展示。
|
||||
|
||||
规则区显示实际三项权重、爆管嫌疑门槛、动态阈值学习状态与历史区间。基线就绪后显示实际侦测阈值及压力/出厂流量的逐测点阈值范围;基线不足不展示虚构阈值。加权角色测点采用实际动态阈值,不再套用固定 30% 输入;“采样范围与基础规则”配置区已移除,面板只读展示实际诊断规则。已具备有效流量基线时,即使压力模型基线不足也保留该流量异常,不以组合资料缺口屏蔽单项诊断。`anomaly_signals` 保存三项爆管参考指标之外的有效测点异常,数据不足与单项命中也能创建同一诊断工单。当前阶段仍需单独验收真实 Agent 推理与报告。
|
||||
|
||||
演示新建默认采用 Case3 的北京时间 2026-07-16 07:30–08:30,事件起点为 08:00,即事件前后各 30 分钟。存在其他时段的活动回放时,提供“切换到 Case3”,先停止旧回放再设定该区间,不自动推进或创建调查;后续监测按钮按项目周期推进。停止或完成的旧回放不再覆盖新建演示默认区间,也不将旧结果显示为 Case3 的结论。
|
||||
|
||||
### 自动播放与简短通知(2026-11-07)
|
||||
|
||||
调查任务仍按 `replay_context.knowledge_cutoff` 与 `batch_sequence` 截断知识,后续播放不自动扩大当前调查窗口。任务摘要只保留指标与异常数量,逐测点明细通过冻结 SCADA 证据查询。`scada-anomaly` 要求高嫌疑诊断继续候选定位;当前 Case3 的任务摘要和 skill 响应返回 `location_capability` 能力提示。提示可用时 `run_burst_location` 允许省略 `burst_leakage`,仍先核验冻结异常窗、正常窗及压力设备,再调用固定候选方法;水力算法仍要求漏量依据。候选输出不再含 mock 提示,MCP回放证据保留 `origin=synthetic`,不向 Agent 暴露事件真值。
|
||||
|
||||
Agent可见能力字段统一为 `location_capability`,不再输出旧 `location_demo` 字段。
|
||||
公共指令与 SCADA Skill 明确候选及相似度返回后继续地图展示、隔离、节点覆盖、影响分析与报告,
|
||||
来源字段本身不构成停止条件。有效 SCADA 查询不再将来源说明列为 `data_gaps`;实际缺测、
|
||||
窗口与设备校验仍按原规则处理。现场工单与设备动作维持服务端权限限制,动作拒绝文案明确
|
||||
允许继续已授权分析。旧工具回执和历史消息不改写;既有 OpenCode 进程的 system 配置副本
|
||||
需刷新后才使用新的公共指令。
|
||||
|
||||
回放调查开放项目资产属性、空间查询、拓扑隔离和节点覆盖,沿用原有业务权限。它们依据当前模型,不代表历史现场状态。实时 TSDB、在线模拟等未接入冻结输入的路径仍不可用;候选定位不以出厂流量命中或现场确诊为前提。
|
||||
|
||||
点击“自动播放”后,浏览器以 180 倍速推进演示时钟,一小时区间约用 20 秒播放,后端处理耗时另计。到达项目监测周期边界时调用原有 `step` 接口,每次只允许一个监测请求,等待持久回执后再继续。播放时钟与已处理时间分别展示;时间轴的手动浏览仍按 5 分钟定位,手动拨动先暂停自动播放,不直接触发处理。暂停、关闭面板、切换页签或页面转到后台后停止自动推进;接口失败暂停,终点自动结束,不自动重试或新建下一轮。此能力属于前台演示,不引入通用后台定时任务或新的 SCADA worker。
|
||||
|
||||
通知只展示异常状态、调查已启动或沿用原调查及时间,通过“查看调查”打开已有调查与报告。指标、评分、基线、定位和处置依据保留在调查摘要与报告中,Server 告警描述不再复制完整任务摘要。首次实际监测检出新的异常时,局部图标以 1.4 秒单次扩散和轻微缩放强调;同一异常下一周期及旧回执刷新不重播,减少动态效果模式保留文字并关闭动画。异常事件的持久去重与冻结证据规则继续有效。
|
||||
|
||||
|
||||
### 指针选定时刻监测
|
||||
|
||||
前端“回到开始时间”只把指针定位到所选区间起点并暂停播放,不清空已处理进度,也不自动调用监测。显式点击“监测选定时刻”才提交 `POST /api/v1/scada-replays/{replay_id}/commands`,动作 `monitor`,携带带时区的 `at_time`、`request_id` 和 `expected_revision`。服务端复用逐帧读取、清洗、检测、历史基线和调查流程,知识截止与最终处理时间精确停在 `at_time`,不自动推进到下一个监测周期。选定起点也可以监测;同一已完成时刻的重复请求复用已有回执,不重复计次、解除告警或创建调查。
|
||||
|
||||
`at_time` 必须在本轮区间内,不能替换进行中的处理步;直接要求本轮回到已经处理过的更早时刻返回 409。前端在该场景下提前说明将建立新一轮,停止旧轮后使用原区间配置新建回放,再处理至选定时刻;原回放、告警和调查保留,历史判断不携带旧轮的后续状态。`step` 继续按冻结的项目周期从本轮起点对齐,选定 08:05 后下一步为 08:15,而不是 08:20。此变更新增命令字段,不需要数据库迁移;需同步更新 Server、前端 API 客户端与 OpenAPI 契约。
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# tjwater_v2 爆管 Agent Benchmark
|
||||
|
||||
状态:场景框架与只读数据预检已准备;隔离执行环境、真实触发链、Agent 轨迹采集和正式评分器尚未实现。2026-11-03。
|
||||
|
||||
## 目标与边界
|
||||
|
||||
本套 benchmark 用固定的管网状态、触发条件和隐藏真值,比较提示词、Skill、MCP 工具或模型配置变更对爆管处置的影响。首版限于 `tjwater_v2` 爆管,不扩展水体污染。已有数据库是数据来源;每次正式执行必须在独立评测数据库和运行时完成,原项目数据库不承担工单、SCADA 清洗、告警、任务、计算或证据写入。
|
||||
|
||||
```text
|
||||
原 MetaDB / BizDB / TimescaleDB
|
||||
| 仅 SELECT、只读事务
|
||||
v
|
||||
版本化私有数据快照 -> 隔离评测 BizDB + TimescaleDB -> Server/MCP/worker -> Agent Host
|
||||
| |
|
||||
+----轨迹与业务结果----+
|
||||
|
|
||||
评分报告 + 版本归档
|
||||
```
|
||||
|
||||
评测 Agent 只能连接隔离环境。单靠客户端 `READ ONLY` 事务不足以保证未来服务不误连原库;运行配置必须使用隔离数据库路由,并由原库侧评测账号权限拒绝写入。现有 `temporary_project_database`/模型模板复制路径不能直接用于原项目,因为它会暂时关闭连接并终止源库会话。应在独立实例先生成逻辑快照,再把可写副本作为评测环境。
|
||||
|
||||
## 场景矩阵
|
||||
|
||||
配置文件 `resources/benchmarks/tjwater-v2-burst.json` 定义四个候选物理工况:
|
||||
|
||||
| 场景 | 管段 | 管径 | 发生时间(北京时间) | 当前观测准备状态 |
|
||||
|---|---:|---:|---|---|
|
||||
| B01 | 540672 | DN315 | 2026-10-08 02:00 | 需在隔离集生成事件观测 |
|
||||
| B02 | 540672 | DN315 | 2026-10-08 12:00 | 历史回放声明待核对当前窗口 |
|
||||
| B03 | 537407 | DN600 | 2026-10-08 02:00 | 需在隔离集生成事件观测 |
|
||||
| B04 | 537407 | DN600 | 2026-10-08 12:00 | 需在隔离集生成事件观测 |
|
||||
|
||||
每个工况各有工单上报(WO)和 SCADA 监测(SCADA)入口,共 8 个候选用例。另设无爆管正常控制作为误报检查,不算第三类业务。管段、时间和历史分析名称仅是候选定义;准备报告须检查观测窗口、参考工况和触发是否吻合。未满足条件的用例标为 `not_ready`,不能作为 Agent 失败,也不能从统计中静默删除。
|
||||
|
||||
工单路径使用 `burst_report` 来源,经既有外围工单同步、操作任务 worker、Agent Host 和 MCP 完成调查。SCADA 路径从隔离副本的正常数据起步,再释放事件数据,由真实 replay worker/规则创建通用异常调查;不能把目标管段或爆管类型预先告诉检测器。纯规则预览只作准备检查,不能代替这条端到端路径。
|
||||
|
||||
工单位置由冻结的网络 GIS 几何转成稳定的人类可读位置,结合近似范围和现象描述,不把真实 link ID 直接写成答案。Agent/Server 业务结果是首版主分数;前端呈现单独做 Playwright 结果回放检查,不能用工作台 mock 结果代替 Agent 评测,也不混入业务能力分。
|
||||
|
||||
首版使用单一持续爆管,不模拟修复恢复;预计结束状态是有证据的调查报告,可以合理等待人工确认或审批。现有模型只含 2 个阀门,目标管段即使移除全部模型阀门仍与水源连通,因此成功关闭隔离不作为成功标准。Agent 应说明拓扑限制和现场核查需求,不得编造阀门方案。
|
||||
|
||||
## 真值、可见信息与评分
|
||||
|
||||
真值包在运行前冻结并与 Agent 输入分开保存,至少包含:管段及端点、泄漏参数和单位、观测窗口、传感器映射、独立拓扑参考、允许的等价候选集合、定位是否具备有效流量输入。工单只给近似位置、时间和现象;SCADA 检测器只能看到已释放观测及设备元数据。隐藏真值不得进入 Skill、提示词、任务说明、MCP 上下文、可读工作区、文件名或执行轨迹。
|
||||
|
||||
由于当前仿真将 emitter 放在管段端点,邻近管段可能不可区分。定位适用的用例按冻结的等价候选集合和排名评估(Hit@1/3/10),不强求唯一 ID。爆管定位需要有效的 `burst_leakage`(m³/s);孔口面积(cm²)不能直接代替流量。缺少输入时,评分应奖励明确说明限制和所需数据,不鼓励猜测。
|
||||
|
||||
总分 100,结果与过程分开报告:
|
||||
|
||||
| 维度 | 权重 | 评分依据 |
|
||||
|---|---:|---|
|
||||
| 业务结果 | 60 | 触发来源与结论置信度 10;证据、时间窗、单位和数据质量 15;定位 Hit@1/3/10 或正确 abstain 15;影响与不可隔离判断 10;报告和人工后续建议 10 |
|
||||
| 工具执行 | 20 | 场景清单定义的唯一必需操作有效完成率 12;有效参数、时间/单位范围及结果证据引用 8;同一操作重复调用不增加分数 |
|
||||
| 计划推进 | 15 | 依赖顺序与证据覆盖 6;根据新证据调整计划 5;报告终态、待人工确认和停止条件 4;不评分模型私有思维链或固定调用顺序 |
|
||||
| 效率 | 5 | 仅在业务结果至少 45/60 且无关键错误陈述时计分;以已接受基线的 Token、Agent wall time、有效调用数中位数对比,三项等权,单项最高 1.67 分 |
|
||||
|
||||
定位分在“输入足以定位”的真值用例中按 Hit@1=15、Hit@3=12、Hit@10=8、未命中=0;输入不足的用例不要求猜管段,明确限制和所需数据得 15,猜测得 0。工具有效完成率按每个场景冻结的唯一必需操作计分;单个操作重复尝试只取首次有效终态,额外调用不扩大分母或分子。
|
||||
|
||||
结构化事实、调用结果、时间/单位、候选排名、计划依赖、任务终态由确定性 scorer 判定。报告措辞与计划是否有证据支撑由同一固定版本盲评器独立评分两次,输出结构化子分和引用依据;每次保留评审器模型、提示词版本与原始评分。黄金样例由爆管业务负责人标注。双评差异超过任一子项 2 分、评审器无证据引用或关键结论不确定时转人工复核,复核值和原因进入报告。效率基线建立后,单项得分为 `1.67 × min(1, 基线中位值 / 当前值)`,三项合计最高 5 分;效率基线建立前,该项标为 `unscored`,报告总分上限 95,不伪造完整 100 分。
|
||||
|
||||
报告必须同时给出总分、业务通过率、工具调用成功率、定位 Hit@k、误报/漏报、延迟、分项分数和失败原因。队列 accepted 不算计算成功;MCP `isError`、超时和业务拒绝分开统计。系统错误记为 `infra_error`,保留尝试并有限重试幂等步骤;模型业务失败按实际结果计分。不做 retry-until-pass。
|
||||
|
||||
初始分项数值映射需要用一个人工审阅的黄金样例和重复运行校准后冻结为 scorer 版本;不能在第一次运行后按期望结果回调规则。首轮建议每个就绪用例重复 3 次,并对同版本做 A/A 重跑,估计模型随机波动;小样本只报告原始分布,不声称统计显著。
|
||||
|
||||
## 自动执行与复现
|
||||
|
||||
正式 runner 的单个入口按以下检查点运行:
|
||||
|
||||
1. `prepare`:校验套件、来源版本、传感器/数据窗口、触发可检出性、模型拓扑和真值包完整性;输出明确就绪/阻塞清单。
|
||||
2. `snapshot`:以只读导出生成 BizDB、TimescaleDB 及必要文件的不可变版本包和哈希;所有评测写入只落在该版本的可写副本。
|
||||
3. `trigger`:每例建立唯一 attempt ID,按 WO 或逐步 SCADA 事件路径触发。SCADA 没触发时记录触发失败,不补发 Agent 任务。
|
||||
4. `investigate`:通过真实 Agent Host 和 Server/MCP 执行,脚本化反馈只提供当时可知事实;模型不得读取未来观测或真值。
|
||||
5. `collect`:记录 case/attempt、task/run/execution/session、任务和报告修订、计算 job、MCP receipt、工具调用、计划版本、Token/耗时与终态。续跑先查已有业务 ID,禁止重复建单或重复触发。
|
||||
6. `score/archive`:确定性 scorer 读取隐藏真值和收集轨迹,生成逐例 JSON、汇总报告、HTML/CSV 可选视图及所有代码/配置/数据/scorer 指纹。基线升级由人工明确选择。
|
||||
|
||||
每个 case 串行运行,直到能证明不同可写副本完全隔离后再并发。事件时间使用虚拟时钟;JWT、worker lease、heartbeat 和 HTTP timeout 使用现实时间。Agent 输入的查询统一受 `knowledge_cutoff` 约束。每次运行保留所有尝试和原始错误,不保存密钥、访问令牌或完整 DSN。结果进入 Git 忽略的私有工件目录或受控 CI 存储,不提交业务观测和轨迹。
|
||||
|
||||
每轮实验固定记录:Agent/Server/frontend 代码内容摘要、提示词与 Skill 摘要、模型及参数、MCP 工具 schema/版本、数据快照 ID/哈希、suite/scorer 版本、基线版本、case、重复次数、运行预算和实际服务指纹。只存 Git SHA 不足以标识当前含未提交改动的运行版本。
|
||||
|
||||
## 当前准备状态与执行门槛
|
||||
|
||||
当前已有:四个候选场景清单、严格字段校验、分库只读查询与规则预览函数、独立拓扑参考。只读预检已在 2026-11-03 执行:四个场景均有 83 个完整短窗压力测点,但没有一个触发当前短窗 SCADA 规则;15 天检测窗口也没有满足连续 1440 点的压力节点。B02 的工单路径是唯一有现成数据候选的路径,B02 的 SCADA 路径仍需隔离生成能触发的事件数据。该结果只说明历史窗口/当前规则的匹配情况,不证明实际 worker 或 Agent 行为。刚加入的 `scripts/prepare_burst_benchmark.py` 只生成私有预检 JSON,不会建任务、写 SCADA 修正值、运行 Agent 或声称数据集已冻结。
|
||||
|
||||
完成自动评测前仍需依次补齐并验收:
|
||||
|
||||
1. 已执行只读预检;下一步由评测维护者确认是否保留 B02-WO 为首个可用试跑,并在隔离副本生成 SCADA 触发与 B01/B03/B04 所需观测。
|
||||
2. 爆管业务/水力分析负责人审阅四个用例的数值参考、有效泄漏流量和等价候选;评测维护者据此固定触发规则与控制组期望。
|
||||
3. Server 数据维护者形成带模型/观测版本哈希的快照导出和隔离 BizDB/TimescaleDB 路由;通过受限角色、连接路由检查和数据库审计证明 Agent/worker 无法写回源库。源库其他日常服务可能同时写入,因此验收证明 benchmark 没有源写入,不承诺整库前后字节完全相同。
|
||||
4. Agent 与 Server 维护者实现可恢复 runner 和全链路轨迹导出;先执行一例 WO 与一例 SCADA 的闭环试跑。
|
||||
5. 评测维护者与爆管业务负责人共同冻结 scorer,在首个通过门槛的用例跑 A/A;再扩到 8 例 × 3 次 × 候选版本。
|
||||
|
||||
若源数据在准备期间被日常业务改变,重新读取后产生不同内容指纹,应建立新快照版本,不覆盖旧报告。预检报告只证明来源库读取与候选数据检查完成,不是 Agent 评测结果或上线验收。
|
||||
|
||||
## 命令
|
||||
|
||||
只读检查命令(在 Server 仓库根目录):
|
||||
|
||||
```bash
|
||||
conda run -n server python scripts/prepare_burst_benchmark.py
|
||||
```
|
||||
|
||||
只检查单个场景可追加 `--case B02`。预检报告写入 `data/benchmark/preflight/`,该目录在 `.gitignore` 中。此命令只读源数据库;正式评测 runner 尚未提供。
|
||||
@@ -0,0 +1,185 @@
|
||||
# tjwater2_v2 多距离、弱信号爆管场景
|
||||
|
||||
状态(2026-11-07):完成四区域、三档破口、四时段的 48 组本地水力校准与固定阈值敏感性分析,并从中选出 4 个演示 case。4 个爆管方案已保存,压力、流量和正常水质数据已写入隔离项目与 `tjwater2_v2` 的 IoT 库。动态阈值尚未接入真实监测 worker,正式 benchmark Runner、真实工单和 SCADA Agent 链路、评分及归档仍待实现。
|
||||
|
||||
同日管网 ID 修复后,R1 管段当前 ID 为 `564345`,流量设备 `542121` 的绑定管段当前 ID 为 `570995`。管段属性、几何、拓扑和 SCADA 测量值未变化。隔离 benchmark 的 BizDB、852,960 条正常实时结果,以及清单直接引用的 2,016 条方案管段结果已同步。以下清单统一使用新 ID。
|
||||
|
||||
48 个历史方案保存了全管网结果,其中未被当前 benchmark 清单引用的长管段仍属于迁移前的校准归档,不进入 Agent 输入或当前 scorer。完整历史结果的全量迁移因压缩块重写耗时过长而回滚;如需把旧方案用于全管网地图展示,应重新生成方案或另行离线迁移,不能混用新模型 ID 与旧结果 ID。
|
||||
|
||||
## 当前演示数据集
|
||||
|
||||
演示数据使用隔离项目 `tjw_bench_tjwater2_20261105`,项目 UUID 为 `66ce6b1a-4415-4cd0-b68f-bf1505576c79`。`tjwater2_v2` 与隔离项目的 IoT 库均写入相同数据,便于在完整系统中演示,也便于 benchmark 指定隔离数据源运行。后续自动评测默认使用隔离项目,避免业务操作、清洗结果或回放状态互相影响。
|
||||
|
||||
每个月写入 1 日 00:00 至 18 日 00:00 前的数据,共 17 天、1,632 个 15 分钟时刻。设备包括 83 台压力设备、35 台流量设备和 50 台水质设备,共 168 台。每台设备每月 1,632 点,每月 274,176 行,四个演示窗口合计 1,096,704 行。`cleaned_value` 初始为空,由真实清洗和回放流程产生。
|
||||
|
||||
每个事件前都有精确的 15 天同时间点历史,即 1,440 个时刻。按全部 168 台设备计算,每个 case 的事前基线为 241,920 行。检测算法使用事件时刻之前 15 天的同时间点数据,不要求自然月内其余日期全部正常或完整。
|
||||
|
||||
### 4 个演示 case
|
||||
|
||||
时间均为北京时间(UTC+8)。事件持续 5 小时,包含起止点共 21 帧;事件结束后的下一个 15 分钟时刻恢复正常数据。
|
||||
|
||||
| Case | 事件时间 | 入口 | 管段 | 破口面积 | 等效圆孔直径 |
|
||||
| --- | --- | --- | --- | ---: | ---: |
|
||||
| `B26-severe` | 2026-05-16 08:00 至 13:00 | SCADA 自动发现 | `564345` | 22.089323 cm² | 53.03 mm |
|
||||
| `B28-severe` | 2026-06-16 20:00 至 06-17 01:00 | SCADA 自动发现,跨日 | `564345` | 22.089323 cm² | 53.03 mm |
|
||||
| `B30-severe` | 2026-07-16 08:00 至 13:00 | 工单在事件发生 60 分钟后上报 | `541725` | 353.429174 cm² | 212.13 mm |
|
||||
| `B35-severe` | 2026-08-16 12:00 至 17:00 | 工单在事件发生 120 分钟后上报 | `538540` | 245.436926 cm² | 176.78 mm |
|
||||
|
||||
四个 case 均使用管道截面积的 1/8,比例不是管径的 1/8。B26 和 B28 复用同一物理位置,用来比较早间与晚间跨日条件。B30、B35 使用更大口径和更远压力测点,用来覆盖区域、信号强度和工单发现延迟的差异。
|
||||
|
||||
| Case | 破口坐标 WGS84 | 管径 / 长度 | 最近压力设备 / 沿网距离 |
|
||||
| --- | --- | --- | --- |
|
||||
| B26 / B28 | 121.471119, 31.005631 | DN150 / 249.176 m | `92724` / 约 0.205 m |
|
||||
| B30 | 121.569785, 30.956880 | DN600 / 495.368 m | `170486` / 约 1.19 km |
|
||||
| B35 | 121.487510, 30.823888 | DN500 / 121.485 m | `102837` / 约 2.67 km |
|
||||
|
||||
### 泄漏真值与 SCADA 流量响应
|
||||
|
||||
破口泄漏量来自水力模拟,是隐藏评测真值,不能直接进入 Agent 的提示词、任务上下文或可见工具结果。SCADA 流量是 Agent 可见证据,单位为 m³/h,已完成 L/s 到 m³/h 的换算并加入确定性仪表噪声。
|
||||
|
||||
| Case | 泄漏量 L/s | 泄漏量 m³/h | 代表性 SCADA 流量变化 |
|
||||
| --- | ---: | ---: | --- |
|
||||
| B26 | 21.60 至 23.49 | 77.76 至 84.58 | `449417` +37.32 m³/h;`516234` +15.63 m³/h |
|
||||
| B28 | 21.99 至 25.81 | 79.15 至 92.90 | `449417` +49.42 m³/h;`542121` +102.68 m³/h |
|
||||
| B30 | 424.15 至 446.33 | 1,526.95 至 1,606.79 | `524529` +296.26 m³/h;`541061` +241.34 m³/h |
|
||||
| B35 | 236.97 至 261.11 | 853.08 至 940.01 | `508453` +926.08 m³/h;`527865` +46.96 m³/h |
|
||||
|
||||
三个主要水厂计量设备的 5 小时最大变化如下。百分比按同时间正常值计算;小流量管段的基线接近 0 时,百分比会失真,评分必须同时看绝对变化量和流向。
|
||||
|
||||
| Case | `542017` | `486033` | `542121` |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| B26 | +50.11 m³/h,1.23% | -82.44 m³/h,1.27% | +32.15 m³/h,3.47% |
|
||||
| B28 | -81.51 m³/h,3.37% | +129.10 m³/h,3.65% | +102.68 m³/h,19.10% |
|
||||
| B30 | +713.91 m³/h,19.13% | +293.30 m³/h,4.75% | +258.24 m³/h,27.95% |
|
||||
| B35 | +212.19 m³/h,5.43% | +491.69 m³/h,8.01% | +102.87 m³/h,11.64% |
|
||||
|
||||
### 正常水质数据
|
||||
|
||||
50 台水质设备使用 EPANET 化学水质模拟生成正常余氯数据,没有用固定常数补值。13 个水库和 15 个正常入流水源设置为 0.50 mg/L,全网初始余氯为 0.35 mg/L,体相衰减为 -0.05/天,管壁衰减为 -0.02/天。模型预热 6 天后取第 7 天的正常日曲线,按 15 分钟间隔重复到各 case 窗口。
|
||||
|
||||
仪表噪声使用固定种子,标准差为读数的 0.5%,最低标准差为 0.001 mg/L,并截断在正负 3 倍标准差内。最终数据约为 0.071 至 0.507 mg/L,没有零值、负值或求解警告。当前四个爆管 case 内的水质保持正常,只用于保证设备读数完整,不作为爆管响应或判分指标。
|
||||
|
||||
## 管段与监测距离
|
||||
|
||||
保留一个近测点强信号对照,其余改用中远距离候选。距离取爆管节点到全部压力设备中的最近沿网距离,不是到任意挑选的某个测点的距离。按管道模型长度累计,阀门连接按近零长度处理,表中取近似值。
|
||||
|
||||
| 组别 | 管段 ID | 管径 / 长度 | 最近压力设备 / 沿网距离 | 破口经纬度 WGS84 | 边界阀 / 分区节点 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| R1 近点对照 | `564345` | DN150 / 249.176 m | `92724` / 约 0.205 m | 121.471119, 31.005631 | 5 / 11 |
|
||||
| R2 中距干管 | `541725` | DN600 / 495.368 m | `170486` / 约 1.19 km | 121.569785, 30.956880 | 4 / 64 |
|
||||
| R3 远距多点响应 | `538540` | DN500 / 121.485 m | `102837` / 约 2.67 km | 121.487510, 30.823888 | 2 / 10 |
|
||||
| R4 远距弱信号 | `541116` | DN200 / 228.424 m | `406444` / 约 2.91 km | 121.441641, 30.982725 | 4 / 9 |
|
||||
|
||||
四处均经过后端隔离算法和水源连通校验,关闭所列边界阀后可在模型拓扑上切断水源。分区节点数不等于用户数、实际停水范围或现场阀门可操作性。
|
||||
|
||||
上轮贴近测点的 DN110、DN315、DN500 候选及其结果仍保留,不能因本轮换管段而重用其 case ID。
|
||||
|
||||
## 破口面积与发生时间
|
||||
|
||||
比例均指管道截面积,不是管径;不再只选一个最容易触发的面积。
|
||||
|
||||
| 管段 | 1/32 面积 cm² | 1/16 面积 cm² | 1/8 面积 cm² |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| R1 DN150 | 5.522331 | 11.044662 | 22.089323 |
|
||||
| R2 DN600 | 88.357293 | 176.714587 | 353.429174 |
|
||||
| R3 DN500 | 61.359232 | 122.718463 | 245.436926 |
|
||||
| R4 DN200 | 9.817477 | 19.634954 | 39.269908 |
|
||||
|
||||
发生时间统一为北京时间 2026-10-08 的 02:00、08:00、12:00、20:00,分别覆盖夜间、早间、日间、晚间。每个事件持续泄漏 5 小时,15 分钟一步,共 21 帧;20:00 场景结束于次日 01:00。3/4 小时视图按前缀截取。
|
||||
|
||||
R1 至 R4 分别对应 B25 至 B28、B29 至 B32、B33 至 B36、B37 至 B40;三档面积作为 case 的独立维度,例如 `B29-medium`。共 4×3×4=48 个物理场景,后续再派生 SCADA 与 WO-30/60/120/180 等入口 case,不能把这些派生用例当成不同物理事件。
|
||||
|
||||
所有场景有相同起始时间、模型状态、需求模式和 PDA 参数的无爆管对照。三个主要水厂出口设备 `542017`、`486033`、`542121` 均保留;压力单位 m,流量结果 L/s 映射 SCADA 时转为 m³/h。
|
||||
|
||||
## 观测结果与动态阈值
|
||||
|
||||
各场景在 5 小时内的最大监测压降,以下取四个发生时段的范围。这是相同时间正常/事故模拟的差异,不是运行告警直接可用的未来对照。
|
||||
|
||||
| 管段 | 1/32 | 1/16 | 1/8 |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| R1 | 11.58% 至 12.69% | 26.37% 至 28.08% | 52.84% 至 54.65% |
|
||||
| R2 | 4.48% 至 5.57% | 9.92% 至 11.90% | 22.05% 至 25.14% |
|
||||
| R3 | 3.95% 至 6.24% | 7.91% 至 12.21% | 14.65% 至 21.58% |
|
||||
| R4 | 0.10% 至 0.13% | 0.19% 至 0.26% | 0.37% 至 0.50% |
|
||||
|
||||
48 组均完成求解,未出现求解警告、指定监测点负压或结果缺失。R2 至 R4 因未达到旧 30% 门槛而被标为 `weak_signal_candidate`,仍然是有效爆管正例;不当成无爆管样本,也不为让所有 case 通过而增大破口。
|
||||
|
||||
固定阈值复用实际清洗/回放相对变化规则,使用四帧预热和连续两帧,未加入绝对压降门槛。16 个正常对照按“管段×时间”去重:
|
||||
|
||||
| 固定压降阈值 | 触发的爆管场景 / 48 | 误报的正常对照 / 16 |
|
||||
| --- | ---: | ---: |
|
||||
| 3% | 47 | 13 |
|
||||
| 5% | 41 | 8 |
|
||||
| 10% | 25 | 3 |
|
||||
| 20% | 11 | 0 |
|
||||
| 30% | 4 | 0 |
|
||||
|
||||
这里的“触发”只是规则预览,不是 Agent 识别成功;极低阈值下的高触发率不能忽略正常误报。
|
||||
|
||||
动态规则采用独立版本,初始实验方案如下:
|
||||
|
||||
1. 每个测点仅用事件前 15 天完整正常数据拟合,15 分钟间隔要求 1440 点;有缺点时报告历史未就绪。
|
||||
2. 按日内时段建立正常压力中位数参考,避免将正常早晚变化当成事故;用去除日内模式后的 `1.4826×MAD` 表示相对波动。
|
||||
3. 测点相对阈值取 `max(3%, 3×相对波动)`,与绝对压降至少 0.5 m、连续两帧同时成立。3%、0.5 m 是首轮实验参数,需结合测点精度和正常误报结果校准。
|
||||
4. 在事件前冻结参考、阈值和规则版本;拟合函数拒绝事件时刻及之后的观测。评估遇到采样中断会重置连续计数。
|
||||
5. 压力异常后仍需流量或独立压力证据交叉验证。实验函数目前只覆盖压力层,未接入生产监测 worker。
|
||||
|
||||
本次水力包只有四帧事件前预热,因此尚不能给出真实项目动态规则的检测率。当前新增的学习与评估函数仅通过合成正常历史验证;不能把上表固定阈值结果说成动态规则已验收。
|
||||
|
||||
## 复现与数据边界
|
||||
|
||||
清单:[tjwater2-v2-burst-balanced.json](../resources/benchmarks/tjwater2-v2-burst-balanced.json)。原有强信号清单保持不变。
|
||||
|
||||
```bash
|
||||
conda run -n server python scripts/check_burst_simulation_design.py \
|
||||
--design resources/benchmarks/tjwater2-v2-burst-balanced.json
|
||||
|
||||
conda run -n server python scripts/calibrate_burst_benchmark.py \
|
||||
--design resources/benchmarks/tjwater2-v2-burst-balanced.json --all-severities
|
||||
|
||||
conda run -n server python -m pytest \
|
||||
tests/unit/test_burst_benchmark_calibration.py \
|
||||
tests/unit/test_regional_burst_selection.py \
|
||||
tests/unit/test_burst_pressure_thresholds.py -q
|
||||
```
|
||||
|
||||
`scripts/analyze_burst_thresholds.py --calibration-dir <某次校准目录>` 复用已冻结结果做阈值敏感性分析,无需重跑水力。`scripts/prepare_regional_burst_design.py` 输出的 GeoJSON 包含管段、实际破口点和边界阀,位于被忽略的本地数据目录。
|
||||
|
||||
源管网模型在准备阶段保持只读。当前已建立隔离 BizDB、IoT 库和模型模板,并通过正常模拟与保存的爆管方案按设备绑定生成最终 `scada.measurements`。为支持完整系统演示,同一份 SCADA 数据也写入 `tjwater2_v2` 的 IoT 库。动态拟合和 Agent 评测只能读取各自知识截止时刻以前的数据;模拟真值及方案信息不进入 Agent 可见上下文。
|
||||
|
||||
## 正式生成优先本地后端方法
|
||||
|
||||
本地执行优先复用 `app.services.simulation_scenarios.burst_analysis()`。MCP 的 `run_burst_simulation` 适配器和 `save_simulation_scenario` 保存入口也调用这个函数,分别设置 `persist_results=False` 和 `True`。固定模型、SCADA 边界输入、时间、破口与单位后,两条入口共用水力计算实现;MCP 额外承担权限、上下文、任务和审计等流程。
|
||||
|
||||
此前直接修改 INP 的结果只用于选管校准。正式 4 个演示 case 已在隔离路由下保存方案,保留正式 `run_id`、模型和结果摘要,再按设备绑定写入 SCADA。Agent 评测仍需使用真实工具链,数据生成成功不能替代 MCP 工具成功率、计划过程和最终业务效果评分。
|
||||
|
||||
## 方案结果到 SCADA 的映射
|
||||
|
||||
先分配日期窗口,再按该时间运行正常模拟和爆管方案;默认沿用计算结果的真实时间戳写入 SCADA,不在写入时悄悄平移日期。
|
||||
|
||||
| SCADA 设备类型 | 绑定字段 | 正常数据源 | 事故数据源 | 写入值 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 压力 | `asset.scada_devices.node_id` | `realtime.node_results.pressure` | 指定 `run_id` 的 `analysis.node_results.pressure` | 单位核验后写 m |
|
||||
| 管道流量 | `asset.scada_devices.link_id` | `realtime.link_results.flow` | 指定 `run_id` 的 `analysis.link_results.flow` | 本模型 L/s × 3.6,写 m³/h |
|
||||
|
||||
最终键使用 SCADA 的 `device_id`,不能使用模型节点 ID 代替;写入 `scada.measurements.monitored_value`,`cleaned_value` 初始为空,由真实清洗流程计算。压力和管道流量来自正常与爆管水力计算,水质来自独立的正常化学水质模拟。设备列表、绑定、单位和排除原因在生成前冻结,不为了补齐数据自动移动或改绑设备。
|
||||
|
||||
`analysis` 的 `run_id` 能区分多个方案,SCADA 的主键只有 `(time, device_id)`,不能区分同一设备同一时刻的多个场景。因此,同管段的不同破口面积也必须分配独立日期,不能只给不同管段分日期。
|
||||
|
||||
每个物理 case 的保留窗口覆盖事前 15 天正常历史、5 小时事故、末点采样和恢复缓冲。当前 4 个演示 case 分配在 2026 年 5 月至 8 月,每月一个 case,基线不会混入其他事故。
|
||||
|
||||
当前日期映射已冻结在 `resources/benchmarks/tjwater2-v2-burst-demo.json`。B26、B28、B30、B35 分别使用 2026 年 5 月、6 月、7 月、8 月窗口。工单时间、发现检查点和知识截止随日期统一生成。更换提示词、工具或 MCP 版本时复用同一冻结数据,不重新分配日期或重新生成噪声。
|
||||
|
||||
5 小时事故结果包含起止点,共 21 帧;当前正常在线模拟为右端不含,直接运行 5 小时仅有 20 帧。若保留事故末点,正常对照须额外生成该时刻,不能将缺失的正常值补零或与上一时刻配对。SCADA 记录使用带时区时间,按北京时间规划、按 UTC 存储和比较;回放查询的右端不含时,查询终点需覆盖最后一帧。
|
||||
|
||||
写入前检查整个保留窗口和 `(time, device_id)` 冲突;发现已有记录时停止并重新分配日期,不使用覆盖式 upsert 或 `ON CONFLICT DO NOTHING` 隐藏碰撞。写入采用事务和生成器之间的排他锁,唯一约束兜底。重复执行依据数据集清单和内容摘要验证幂等,不重复插入。
|
||||
|
||||
提交后已使用新连接按设备、日期、行数、单位和数值摘要读回校验。生成状态保存在 Git 忽略目录 `data/benchmark-generation/tjwater2-20261105/`,包括 `demo-scada-state.json` 和 `normal-quality-state.json`。这些文件记录 `case_key`、方案 `run_id`、实际日期、单位、行数与指纹,属于本地生成证据,不提交 Git,也不向 Agent 暴露方案真值。
|
||||
|
||||
## Benchmark 执行边界
|
||||
|
||||
当前数据集已经满足 4 个演示 case 的数据输入要求,但还不能据此声称 Agent benchmark 已完成。后续自动化执行需要按 case 恢复初始业务状态,设置知识截止和入口触发时间,运行真实工单或 SCADA 发现链路,记录计划、工具调用、MCP 调用、任务结果和展示结果,再统一评分和归档。
|
||||
|
||||
建议将评分拆成四部分:事件识别与时间得分、定位与影响分析效果得分、工具调用成功率与参数正确性得分、计划和证据链完整性得分。泄漏量、真实管段、方案 `run_id` 和无噪声模拟结果只供 scorer 使用。Agent 可见输入限于当时可查询的 SCADA、工单和正常业务配置。
|
||||
|
||||
当前仍有两个运行配置缺口:`tjwater2_v2` 只设置了 `scada_monitoring_interval_seconds=900`,尚未配置 `scada_monitoring.pressure_device_ids` 和 `outlet_flow_device_ids`,联合爆管监测会报告角色未配置;新的 4 个 case 尚未执行完整 SCADA 回放和真实 Agent 端到端验收。
|
||||
@@ -1,10 +1,14 @@
|
||||
# tjwater2_v2 爆管模拟校准设计
|
||||
|
||||
状态:参数与只读校验已准备,尚未运行水力模拟。目标项目为 `tjwater2_v2`(演示项目2(新版)),不是 `tjwater_v2`。
|
||||
状态(2026-11-07):本文记录早期强信号管段的校准过程。后续已完成多距离管段校准、4 个演示方案保存,以及压力、流量和正常水质 SCADA 数据生成。正式 Agent benchmark Runner、评分和端到端验收尚未执行。目标项目为 `tjwater2_v2`(演示项目 2(新版))。
|
||||
|
||||
当前可执行数据集、4 个演示 case、流量响应、水质数据和 benchmark 边界见[多距离、弱信号与动态阈值设计](tjwater2-v2-burst-balanced-design.md)。本文保留为较早的强信号校准记录,30% 不再作为全部有效爆管用例的统一淘汰条件。
|
||||
|
||||
2026-11-07 管网 ID 修复后,本文主候选的当前管段 ID 为 `564345`,流量设备 `542121` 的当前绑定管段 ID 为 `570995`。物理管段、模型参数和校准数值未变化,清单及下文引用均使用新 ID。
|
||||
|
||||
## 目标
|
||||
|
||||
先用现有管网、阀门和监测点选择可观测的事故管段,再在隔离评测数据库中校准破口面积。首版使用管道截面积的 1/32、1/16、1/8;1/8 是上限。若 1/8 仍不能形成明确降压信号,该管段退出 SCADA 爆管用例,不继续扩大破口。
|
||||
先用现有管网、阀门和监测点选择可观测的事故管段,通过只读导出的 INP 文件校准破口面积,再在隔离评测数据库保存正式方案。首版使用管道截面积的 1/32、1/16、1/8;1/8 是上限。比例指面积,等效圆孔直径分别为管径的约 17.68%、25%、35.36%。若 1/8 仍不能形成明确降压信号,该管段退出 SCADA 爆管用例,不继续扩大破口。
|
||||
|
||||
模拟调用链为:
|
||||
|
||||
@@ -16,6 +20,8 @@
|
||||
|
||||
当前 `temporary_project_database` 会从项目模型模板创建临时数据库,同时短暂禁止模板连接并终止模板会话。因此即使 `run_burst_simulation` 不保存结果,也不能直接针对原项目执行。正式运行必须把只读逻辑快照恢复到独立评测数据库及其模板,Server、worker 和 Agent 全部使用评测路由。
|
||||
|
||||
当前校准入口使用 `REPEATABLE READ, READ ONLY` 导出源模型,再修改本地 INP 中的参数,调用仓库自带原生 EPANET;不调用数据库克隆接口,不写原项目或模板,不使用旧 `realtime` 结果代替重算。正常与事故对照均采用 PDA、`REQUIRED PRESSURE=10 m`,避免把 DDA/PDA 的差异算成爆管效果,并保留源配置的 `HEADERROR`、`FLOWCHANGE`、最低压力和压力指数。已确认当前模型无水池、水泵、控制与规则;若后续新增,入口会停止,待补齐状态继承实现。
|
||||
|
||||
## 方法参数
|
||||
|
||||
`run_burst_simulation` 接收带时区的 `start_time`、60 至 86400 秒的 `duration`、1 至 20 个输出节点、至多 20 条输出管段、1 至 20 条爆管管段,以及与管段一一对应的正数孔口面积 `orifice_areas_cm2`。本次每个调用只模拟一条爆管管段,等待预算使用 240 秒;排队成功只表示已接受,仍需查询计算任务和最终结果。
|
||||
@@ -28,27 +34,31 @@
|
||||
coefficient[L/s/sqrt(m)] = 0.65 * area[cm²] / 10000 * sqrt(19.6) * 1000
|
||||
```
|
||||
|
||||
正式运行后,定位使用的泄漏量按事故节点相对正常对照的 demand 增量计算,并从 L/s 除以 1000 转成 m³/s。流量监测设备单位为 m³/h,模型流量结果为 L/s,映射 SCADA 时乘以 3.6。
|
||||
模拟真值的泄漏量按 `coefficient * sqrt(max(事故节点压力, 0))` 计算,并从 L/s 除以 1000 转成 m³/s。PDA 下节点实际需水量会随压力改变,事故与正常的 demand 差值不能直接当成精确泄漏量。该模拟真值留在隐藏评测包,Agent 定位仍需从其可见证据获得泄漏量。流量监测设备单位为 m³/h,模型流量结果为 L/s,映射 SCADA 时乘以 3.6。
|
||||
|
||||
## 管段与场景
|
||||
|
||||
主场景管段 `541720` 为 DN800、长 483.038 m,破口节点 `502046`。附近两个压力点沿管网约 1.2 km,最近流量点约 1.2 km;两座边界阀可隔离 11 个节点。其破口面积为 157.079633、314.159265、628.318531 cm²。
|
||||
原 B01/B02 管段 `541720`、B03/B04 管段 `520323` 在恢复水源支路后进行了 1 小时最大面积校准,监测点最大降压分别仅约 2.03%、2.87%,不满足门槛,已退出当前候选。失败结果及当时的完整清单保存在本地报告,不能复用旧 case ID 表示其他管段。
|
||||
|
||||
清单同时保存 SCADA 设备 ID 与其实际水力模型对象 ID,不能假定两者相同。例如压力设备 `447382`、`444985` 分别绑定节点 `7543252226207002012`、`14141146287589625673`,流量设备 `542121` 绑定管段 `pipe-eaa5c512bd8f4649b37bdbe3d`。模拟输出使用模型对象 ID,得分与数据映射使用设备 ID。
|
||||
重新从压力点沿网距 400 m 内、DN150 至 DN800、长度至少 50 m 的管段中筛查,按 `length / diameter²` 作为试算排序启发式,优先覆盖不同测点;它不代表水力计算结论。首批试算 24 条,筛出两条进入昼夜完整校准。
|
||||
|
||||
启动校验还要求所选压力设备到绑定节点、流量设备到绑定管段的本地坐标距离均不超过 1 m。全量复核中待人工确认的 `542017`、`547063` 和 `DEMO-WQ-551571` 不进入本套爆管 benchmark;不得为通过校验而自动吸附设备坐标或猜测改绑对象。
|
||||
当前主候选 `564345` 为 DN150、长 249.176 m,起点 `92686`、破口节点 `7427148338069072993`。该节点距压力点 `92724` 沿网约 0.205 m。关闭全部模型阀门后的所在分区含 11 个节点、5 座边界阀;这只是拓扑参考,尚未验证逐阀关闭的水力影响。三档面积为 5.522331、11.044662、22.089323 cm²。
|
||||
|
||||
次级场景管段 `520323` 为 DN300、长 33.5951 m,破口节点 `91055`。最近压力点沿管网约 262 m,五座边界阀可隔离 19 个节点。其破口面积为 22.089323、44.178647、88.357293 cm²。
|
||||
次候选 `540173` 为 DN200、长 162.128 m,起点 `500819`、破口节点 `517120`,距压力点 `535381` 沿网约 269.125 m。所在分区含 23 个节点、6 座边界阀。三档面积为 9.817477、19.634954、39.269908 cm²。其夜间校准未通过,目前保留为失败对照,不能随主候选一起生成正向 SCADA 用例。
|
||||
|
||||
两条管段分别在北京时间 2026-10-08 02:00 和 12:00 模拟,形成 B01 至 B04 四个物理场景。每个场景都必须有同时间、同持续时长、相同模型状态的无爆管正常对照。严重档先验证是否可观测;通过后依次测试中度和轻度,最终选择满足要求的最小面积。最坏情况下执行 12 次爆管校准模拟,正常对照按项目、时间和模型版本复用。
|
||||
清单同时保存 SCADA 设备 ID 与其实际水力模型对象 ID。例如压力设备 `430299` 绑定节点 `3186958412475415759`;流量设备 `542121` 绑定管段 `570995`。模拟输出使用模型对象 ID,得分与数据映射使用设备 ID。
|
||||
|
||||
`542017` 已依据 `tjwater_v2` 恢复缺失的水源 `551778` 和出口管段,并将 SCADA 绑定恢复到 `542017`;原始空间坐标保持不变。当前 BizDB 与模板均为 92393 个节点、91050 条管道、4489 座阀门、13 个水源。三个主要水厂设备 `542017`、`486033`、`542121` 的实际出口绑定均进入两条候选的输出。待核实的 `547063`、`DEMO-WQ-551571` 不参与本轮判分,重筛也排除 `547063`,不得自动移动设备或猜测改绑对象。
|
||||
|
||||
两条候选分别在北京时间 2026-10-08 02:00 和 12:00 模拟,使用新 case ID B05/B06、B07/B08。每个场景均有同时间、同持续时长、相同模型状态的正常对照。严重档先验证,双时段均通过后再执行中度和轻度,选择双时段都通过的最小面积;严重档有一个时段失败即退出。正常对照按时间与模型版本复用。
|
||||
|
||||
## 通过条件
|
||||
|
||||
一个面积只有同时满足以下条件才可冻结:
|
||||
|
||||
1. 至少一个指定压力点连续两个 15 分钟时刻较正常对照下降 30%。
|
||||
2. 第二压力点下降至少 10%,或指定流量点相对正常对照变化至少 20%。
|
||||
3. 正常对照不触发相同规则。
|
||||
2. 同一连续窗口内,另一独立压力点下降至少 10%,或同一指定流量点变化至少 20%。零流量基线不能提供相对变化佐证。
|
||||
3. 事件前四个 15 分钟正常样本预热后,生产回放清洗/相对变化规则对事故触发、对正常对照不触发。校准预览配置为降压 30%、连续两帧、至少一台压力设备,不等同于完整监测 worker、15 天检测算法或 Agent 已通过。
|
||||
4. 指定监测节点不出现负压,模拟正常收敛,所有请求输出完整。
|
||||
5. 同一管段的夜间和白天场景使用同一面积;两者都通过时才能进入正式 benchmark。
|
||||
|
||||
@@ -65,3 +75,43 @@ conda run -n server python scripts/check_burst_simulation_design.py
|
||||
```
|
||||
|
||||
隔离评测数据库准备完成后,runner 先按清单调用非持久 `run_burst_simulation` 校准。面积冻结后,通过 `save_simulation_scenario` 在评测数据库保存全量节点和管段结果,生成固定的运行 ID 和数据摘要。模型分析结果与模拟 SCADA 保留明确来源,不标记为实测数据。
|
||||
|
||||
## 自动校准入口与本次结果
|
||||
|
||||
```bash
|
||||
# 1 小时面积校准;输出结果和每次试算的 INP、报告、原生输出
|
||||
conda run -n server python scripts/calibrate_burst_benchmark.py
|
||||
|
||||
# 5 小时完整事件窗口,可派生 3/4/5 小时视图
|
||||
conda run -n server python scripts/calibrate_burst_benchmark.py --duration-seconds 18000
|
||||
|
||||
# 对一个已有校准快照筛查其他管段;每条只算白天单时刻,不代表通过
|
||||
conda run -n server python scripts/screen_burst_benchmark_pipes.py \
|
||||
--calibration-dir data/benchmark-calibration/20261104T101304Z-029cc055 --limit 24
|
||||
|
||||
conda run -n server python -m pytest tests/unit/test_burst_benchmark_calibration.py -q
|
||||
```
|
||||
|
||||
生成文件仅保存在 Git 忽略的 `data/benchmark-calibration/` 私有目录,清单、源 INP 与求解器均记录 SHA256,每完成一个 case 原子更新报告。源库访问均只读;本入口不创建 `analysis.runs`,不写 `realtime.*` 或 `scada.measurements`。退出码 0 表示校准流程完成,具体管段是否合格须读取报告的 `selection`,不能把运行完成当成所有 case 通过。当前支持进度留存,尚不支持从中断处自动续跑。
|
||||
|
||||
2026-11-04 的 5 小时结果(21 帧/场景)为:
|
||||
|
||||
| 场景 | 管段 | 面积 cm² | 主要测点压降范围 | 相对变化规则首次触发 | 判定 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| B05 夜间 | `564345` | 22.089323 | 46.08% 至 54.17% | 02:15(+15 min) | 通过 |
|
||||
| B06 白天 | 同上 | 22.089323 | 49.81% 至 52.84% | 12:15(+15 min) | 通过 |
|
||||
| B07 夜间 | `540173` | 39.269908 | 26.27% 至 33.96% | 未触发 | 失败 |
|
||||
| B08 白天 | `540173` | 39.269908 | 30.83% 至 33.92% | 12:15(+15 min) | 通过,但该管段双时段不合格 |
|
||||
|
||||
主候选的 1/16 与 1/32 面积在昼夜均未通过,最小通过档位为 1/8 截面积(等效圆孔直径约 53.03 mm)。主压力点 `92724` 的事故压力约 12.21 至 17.87 m,隐藏泄漏真值约 22.21 至 26.86 L/s,流量点 `449417`、`516234` 提供同步佐证。全部正常对照未触发,全部求解无警告,指定监测点无负压。
|
||||
|
||||
本地证据目录:
|
||||
|
||||
- 原管段失败报告:`data/benchmark-calibration/20261104T101304Z-029cc055/calibration.json`。
|
||||
- 首批 24 条重筛:该目录下 `screening-df8b9408/screening.json`(明确排除待核实流量点 `547063`)。
|
||||
- 新候选 1 小时校准:`data/benchmark-calibration/20261104T101859Z-83f0c1a2/calibration.json`。
|
||||
- 新候选 5 小时验证:`data/benchmark-calibration/20261104T101949Z-3704704b/calibration.json`。
|
||||
|
||||
本次 5 小时报告的源 INP 摘要为 `dfb20e54356c2c96d36c97d96516361da7fb1201ce37da4ffc39f339aa8e3e0f`,清单摘要为 `bbcdbf7ce502264ffd826a0bb2c7188c619d6a14c42f82cdbb9320c9d254dd92`。当模型、绑定或参数变化时应重新校准并生成新摘要。
|
||||
|
||||
后续多距离设计已从校准结果中选择 B26、B28、B30、B35 四个严重档演示 case,保存到隔离库并按绑定生成 SCADA,同时补齐事件前 15 天历史。中小破口与被淘汰管段仍作为难例或阴性材料,必须另行定义预期,不能沿用正向通过预期。下一步是验证工单和监测真实入口,再执行 Agent 的工具、计划与效果评分。
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
- 这个锁会串行化事件写事务,任务仓储的短命令事务(包括心跳)也使用它。这是当前低频业务规模下的明确取舍,不适用于高频遥测或长事务。扩大业务事件量前应实际测量写入等待时间,不把增加锁误称为写吞吐优化。
|
||||
- 每个进程/项目仍只有一条专用 LISTEN 连接,不长期占用普通请求池。Hub 按唤醒代次共享同游标读取;最多保留 32 个游标批次,每批最多 200 条,慢客户端不会形成无界缓存。事件通知和每 30 秒一次的共享对账使缓存失效;每个浏览器的 15 秒 keepalive 不再触发独立 SQL 查询。无订阅者 60 秒后仍关闭 Hub。
|
||||
- `reset_required` 允许权威快照把客户端游标从较大值重置为 0;普通快照仍禁止回退,过时并发响应仍不能覆盖新响应。重置失败会在重连时重试。只有 SSE 请求成功打开后才显示连接正常,HTTP 失败按原退避策略重连;离开或重置时释放流读取器。
|
||||
- 工单源事件按 ID 定向同步。任务投影的 `operational.task.changed` 负责更新详情,其伴随的 `workbench.work_order.changed/operational_task` 不再重复拉取整份快照。告警和分析事件仍合并后刷新运行态;SCADA/流量采样通道不变。全量对账见 [外围工单主动同步](peripheral-work-order-events.md)。
|
||||
- 工单源事件按 ID 定向同步。任务投影的 `operational.task.changed` 负责更新详情,其伴随的 `workbench.work_order.changed/operational_task` 不再重复拉取整份快照。告警和分析事件仍合并后刷新运行态;SCADA/流量采样通道不变。全量对账见 [外围工单主动同步](peripheral-sample.md#工单事件主动同步)。
|
||||
- 项目查询按同一身份提供器共享,缓存 5 分钟。每个调用仍重新获取 token,各调用可独立取消;共享查询有 15 秒超时,失败后清除缓存。此缓存不授予权限,Server 每次请求和每次 SSE 重建仍鉴权。
|
||||
- 事件接收游标与数据读取成功分开处理:增量详情、通知或事件触发的运行态快照失败时,只对失败读取按 1、2、4 秒递增至 30 秒重试,成功后才确认失效事件。被并发游标守卫拒绝的快照不能当作刷新成功;仅在更新的成功快照已覆盖失效游标时停止重试。正常读取成功后不安排定时请求;依赖变化、重置或离开页面会取消旧请求及重试定时器。这不是新增正常轮询通道,也不扩大初始化列表的裁剪/重试语义。
|
||||
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
# 工作台工单与告警存储契约
|
||||
|
||||
工作台聚合外围来源工况任务与任务主动推送的告警。浏览器不推导告警,也不直接写工况任务。
|
||||
|
||||
## 任务告警
|
||||
|
||||
DDL 位于 `resources/sql/008_workbench_alert_events.sql`。任务通过 `POST /api/v1/workbench/task-alert-events` 推送 `raised`、`acknowledged` 或 `resolved` 事件。来源包含 `task_type`、`task_id` 和 `run_id`,生产者由认证项目身份绑定。`event_id` 在项目和生产者范围内幂等,告警事实在首次 raised 后不可改写。
|
||||
|
||||
## 预约工单
|
||||
|
||||
外围样板的 `peripheral_sample.work_orders` 是演示工单的管理来源;`activity` 只记录实际发生的创建、修改和现场反馈,不由总状态补造阶段。同步器将有效工单写入 `workbench.operational_tasks` 的 `external_work_order` 投影。原有三张独立演示工单也已迁入外围样板,并由同一面板管理。
|
||||
|
||||
`resources/sql/011_workbench_work_orders.sql` 和 `017_workbench_schema_comments.sql` 属于历史基线;`021_retire_workbench_work_orders.sql` 在验证三个迁入目标及其任务投影后,仅在 `tjwater_v2` 删除旧表。不要重放 `011` 重建它。`GET /api/v1/workbench/runtime` 暂保留空的 `work_orders` 字段作为接口兼容。其他项目数据库没有旧表,不需要应用 `021`。
|
||||
|
||||
告警事件表需要 `SELECT, INSERT`,告警状态表需要 `SELECT, INSERT, UPDATE`。迁移及扩展门禁见 [`workbench-database-rollout.md`](workbench-database-rollout.md)。
|
||||
@@ -7,4 +7,20 @@
|
||||
- `workbench.alert_events` 保存任务显式推送的事件,`workbench.alerts` 保存当前状态。分析结果、测点值和缺测状态不会自动变成告警。
|
||||
- 每个可选分区独立读取并返回 `ready`、`not_configured` 或 `degraded`,一个分区失败不会污染其他分区。
|
||||
|
||||
原有三张独立演示工单已迁入 `peripheral_sample.work_orders`,仍由外围样板面板管理,并由同步器投影为 `external_work_order` 工况任务。旧表退役前的完整数据库备份由部署记录保存,不能重新运行历史 `011_workbench_work_orders.sql` 或旧种子脚本重建双份数据。
|
||||
历史上原有三张独立演示工单已迁入 `peripheral_sample.work_orders`,仍由外围样板面板管理,并由同步器投影为 `external_work_order` 工况任务。旧表退役前的完整数据库备份由部署记录保存,不能重新运行历史 `011_workbench_work_orders.sql` 或旧种子脚本重建双份数据。
|
||||
|
||||
## 工单与告警存储
|
||||
|
||||
### 任务告警
|
||||
|
||||
DDL 位于 `resources/sql/008_workbench_alert_events.sql`。任务通过 `POST /api/v1/workbench/task-alert-events` 推送 `raised`、`acknowledged` 或 `resolved` 事件。来源包含 `task_type`、`task_id` 和 `run_id`,生产者由认证项目身份绑定。`event_id` 在项目和生产者范围内幂等,告警事实在首次 raised 后不可改写。
|
||||
|
||||
### 外围工单
|
||||
|
||||
外围服务的 `peripheral_sample.work_orders` 是业务工单的管理来源,当前 `origin` 固定为 `system_data`;`activity` 只记录实际发生的创建、修改和现场反馈,不由总状态补造阶段。同步器将有效工单写入 `workbench.operational_tasks` 的 `external_work_order` 投影。原有三张独立演示工单也已迁入外围样板,并由同一面板管理。
|
||||
|
||||
`resources/sql/011_workbench_work_orders.sql` 和 `017_workbench_schema_comments.sql` 属于历史基线;`021_retire_workbench_work_orders.sql` 在验证三个迁入目标及其任务投影后,仅在 `tjwater_v2` 删除旧表。不要重放 `011` 重建它。`GET /api/v1/workbench/runtime` 暂保留空的 `work_orders` 字段作为接口兼容。其他项目数据库没有旧表,不需要应用 `021`。
|
||||
|
||||
告警事件表需要 `SELECT, INSERT`,告警状态表需要 `SELECT, INSERT, UPDATE`。迁移及扩展门禁见 [`workbench-database-rollout.md`](workbench-database-rollout.md)。
|
||||
|
||||
当前外围工单来源与按项目路由见 [外围服务](peripheral-sample.md),其中 `sample` 只保留兼容名称。上述旧表迁移的历史数据库范围不表示其他项目已应用迁移,也不限制当前 API 的项目路由。
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"version": "burst-benchmark@1",
|
||||
"suite_id": "tjwater-v2-burst",
|
||||
"project_code": "tjwater_v2",
|
||||
"project_id": "324ff7ee-c2d4-5184-a84b-48fb14a2933e",
|
||||
"sampling_interval_minutes": 15,
|
||||
"event_minutes": 60,
|
||||
"detector_days": 15,
|
||||
"trigger_rule": {
|
||||
"pressure_drop_percent": 30,
|
||||
"baseline_samples": 4,
|
||||
"consecutive_samples": 2,
|
||||
"minimum_devices": 2
|
||||
},
|
||||
"score_weights": {"outcome": 60, "tools": 20, "plan": 15, "efficiency": 5},
|
||||
"cases": [
|
||||
{
|
||||
"case_id": "B01", "pipe_id": "540672", "occurred_at": "2026-10-08T02:00:00+08:00",
|
||||
"reference_analysis": "test_burst_20260614", "observation_provenance": "requires_generation",
|
||||
"provenance_note": "夜间爆管尚无已核实的对应异常观测;不能平移中午曲线或将正常观测当作本事件。"
|
||||
},
|
||||
{
|
||||
"case_id": "B02", "pipe_id": "540672", "occurred_at": "2026-10-08T12:00:00+08:00",
|
||||
"reference_analysis": "test_burst_20260614", "observation_provenance": "declared_historical_replay",
|
||||
"provenance_note": "present.md 记录该窗口由 test_burst_20260614 生成;需核查当前观测及参考方案,历史声明本身不等于数值验收。"
|
||||
},
|
||||
{
|
||||
"case_id": "B03", "pipe_id": "537407", "occurred_at": "2026-10-08T02:00:00+08:00",
|
||||
"reference_analysis": "burst_20260610_dn600", "observation_provenance": "requires_generation",
|
||||
"provenance_note": "存在历史模拟参考,但未核实本时间窗口的该管段异常 SCADA;需在隔离环境生成。"
|
||||
},
|
||||
{
|
||||
"case_id": "B04", "pipe_id": "537407", "occurred_at": "2026-10-08T12:00:00+08:00",
|
||||
"reference_analysis": "burst_20260610_dn600", "observation_provenance": "requires_generation",
|
||||
"provenance_note": "本窗口已有其他管段的爆管演示,不能复用为 537407 的真值。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,485 @@
|
||||
{
|
||||
"version": "burst-simulation-design@1",
|
||||
"suite_id": "tjwater2-v2-burst-calibration",
|
||||
"project_code": "tjwater2_v2",
|
||||
"project_id": "871d7538-589c-47de-a6d5-3185ccb9f0e1",
|
||||
"model_expectations": {
|
||||
"node_count": 92393,
|
||||
"pipe_count": 91050,
|
||||
"valve_count": 4489,
|
||||
"flow_unit": "LPS",
|
||||
"pressure_unit": "METERS",
|
||||
"pressure_measurement_unit": "m",
|
||||
"flow_measurement_unit": "m3/h",
|
||||
"emitter_exponent": 0.5,
|
||||
"hydraulic_timestep_seconds": 900,
|
||||
"report_timestep_seconds": 900
|
||||
},
|
||||
"duration_seconds": 18000,
|
||||
"event_times": [
|
||||
"2026-10-08T02:00:00+08:00",
|
||||
"2026-10-08T08:00:00+08:00",
|
||||
"2026-10-08T12:00:00+08:00",
|
||||
"2026-10-08T20:00:00+08:00"
|
||||
],
|
||||
"severity_levels": [
|
||||
{
|
||||
"level": "light",
|
||||
"cross_section_fraction": 0.03125
|
||||
},
|
||||
{
|
||||
"level": "medium",
|
||||
"cross_section_fraction": 0.0625
|
||||
},
|
||||
{
|
||||
"level": "severe",
|
||||
"cross_section_fraction": 0.125
|
||||
}
|
||||
],
|
||||
"calibration_order": [
|
||||
"severe",
|
||||
"medium",
|
||||
"light"
|
||||
],
|
||||
"selection_rule": {
|
||||
"primary_pressure_drop_percent": 30.0,
|
||||
"corroborating_pressure_drop_percent": 10.0,
|
||||
"flow_change_percent": 20.0,
|
||||
"consecutive_samples": 2,
|
||||
"expected_sample_count": 21,
|
||||
"relative_change_epsilon": 1e-06,
|
||||
"normal_control_must_not_trigger": true,
|
||||
"negative_monitor_pressure_allowed": false,
|
||||
"choose_smallest_passing_area": true
|
||||
},
|
||||
"binding_validation": {
|
||||
"max_pressure_distance_m": 1.0,
|
||||
"max_flow_distance_m": 1.0
|
||||
},
|
||||
"cases": [
|
||||
{
|
||||
"case_id": "B25",
|
||||
"pipe_id": "564345",
|
||||
"occurred_at": "2026-10-08T02:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B26",
|
||||
"pipe_id": "564345",
|
||||
"occurred_at": "2026-10-08T08:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B27",
|
||||
"pipe_id": "564345",
|
||||
"occurred_at": "2026-10-08T12:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B28",
|
||||
"pipe_id": "564345",
|
||||
"occurred_at": "2026-10-08T20:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B29",
|
||||
"pipe_id": "541725",
|
||||
"occurred_at": "2026-10-08T02:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B30",
|
||||
"pipe_id": "541725",
|
||||
"occurred_at": "2026-10-08T08:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B31",
|
||||
"pipe_id": "541725",
|
||||
"occurred_at": "2026-10-08T12:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B32",
|
||||
"pipe_id": "541725",
|
||||
"occurred_at": "2026-10-08T20:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B33",
|
||||
"pipe_id": "538540",
|
||||
"occurred_at": "2026-10-08T02:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B34",
|
||||
"pipe_id": "538540",
|
||||
"occurred_at": "2026-10-08T08:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B35",
|
||||
"pipe_id": "538540",
|
||||
"occurred_at": "2026-10-08T12:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B36",
|
||||
"pipe_id": "538540",
|
||||
"occurred_at": "2026-10-08T20:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B37",
|
||||
"pipe_id": "541116",
|
||||
"occurred_at": "2026-10-08T02:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B38",
|
||||
"pipe_id": "541116",
|
||||
"occurred_at": "2026-10-08T08:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B39",
|
||||
"pipe_id": "541116",
|
||||
"occurred_at": "2026-10-08T12:00:00+08:00"
|
||||
},
|
||||
{
|
||||
"case_id": "B40",
|
||||
"pipe_id": "541116",
|
||||
"occurred_at": "2026-10-08T20:00:00+08:00"
|
||||
}
|
||||
],
|
||||
"pipes": [
|
||||
{
|
||||
"role": "primary",
|
||||
"pipe_id": "564345",
|
||||
"diameter_mm": 150.0,
|
||||
"length_m": 249.176,
|
||||
"start_node_id": "92686",
|
||||
"emitter_node_id": "7427148338069072993",
|
||||
"longitude": 121.47111853069737,
|
||||
"latitude": 31.005630555976175,
|
||||
"candidate_orifice_areas_cm2": {
|
||||
"light": 5.522331,
|
||||
"medium": 11.044662,
|
||||
"severe": 22.089323
|
||||
},
|
||||
"pressure_bindings": [
|
||||
{
|
||||
"device_id": "92724",
|
||||
"node_id": "92724"
|
||||
},
|
||||
{
|
||||
"device_id": "100026",
|
||||
"node_id": "100026"
|
||||
},
|
||||
{
|
||||
"device_id": "115039",
|
||||
"node_id": "115039"
|
||||
},
|
||||
{
|
||||
"device_id": "107672",
|
||||
"node_id": "107672"
|
||||
},
|
||||
{
|
||||
"device_id": "107651",
|
||||
"node_id": "107651"
|
||||
}
|
||||
],
|
||||
"flow_bindings": [
|
||||
{
|
||||
"device_id": "542017",
|
||||
"link_id": "542017"
|
||||
},
|
||||
{
|
||||
"device_id": "486033",
|
||||
"link_id": "486033"
|
||||
},
|
||||
{
|
||||
"device_id": "542121",
|
||||
"link_id": "570995"
|
||||
},
|
||||
{
|
||||
"device_id": "449417",
|
||||
"link_id": "449417"
|
||||
},
|
||||
{
|
||||
"device_id": "516234",
|
||||
"link_id": "516234"
|
||||
}
|
||||
],
|
||||
"boundary_valve_ids": [
|
||||
"valve-12b1e80612f9432da0e64485",
|
||||
"valve-880f34624a444f5497faf2ca",
|
||||
"valve-b6cc8d68e38f43c9b3389e04",
|
||||
"valve-f189692cccbd40a69dc26cca",
|
||||
"valve-fa0583a6ac134aeb9513ada5"
|
||||
],
|
||||
"output_node_ids": [
|
||||
"92686",
|
||||
"7427148338069072993",
|
||||
"92724",
|
||||
"100026",
|
||||
"115039",
|
||||
"107672",
|
||||
"107651"
|
||||
],
|
||||
"output_link_ids": [
|
||||
"449417",
|
||||
"486033",
|
||||
"516234",
|
||||
"542017",
|
||||
"564345",
|
||||
"570995",
|
||||
"valve-12b1e80612f9432da0e64485",
|
||||
"valve-880f34624a444f5497faf2ca",
|
||||
"valve-b6cc8d68e38f43c9b3389e04",
|
||||
"valve-f189692cccbd40a69dc26cca",
|
||||
"valve-fa0583a6ac134aeb9513ada5"
|
||||
]
|
||||
},
|
||||
{
|
||||
"role": "secondary",
|
||||
"pipe_id": "541725",
|
||||
"diameter_mm": 600.0,
|
||||
"length_m": 495.368,
|
||||
"start_node_id": "170464",
|
||||
"emitter_node_id": "170438",
|
||||
"longitude": 121.56978490259476,
|
||||
"latitude": 30.956880413066592,
|
||||
"candidate_orifice_areas_cm2": {
|
||||
"light": 88.357293,
|
||||
"medium": 176.714587,
|
||||
"severe": 353.429174
|
||||
},
|
||||
"pressure_bindings": [
|
||||
{
|
||||
"device_id": "170486",
|
||||
"node_id": "170486"
|
||||
},
|
||||
{
|
||||
"device_id": "175618",
|
||||
"node_id": "175618"
|
||||
},
|
||||
{
|
||||
"device_id": "159673",
|
||||
"node_id": "159673"
|
||||
},
|
||||
{
|
||||
"device_id": "135160",
|
||||
"node_id": "135160"
|
||||
},
|
||||
{
|
||||
"device_id": "174754",
|
||||
"node_id": "174754"
|
||||
}
|
||||
],
|
||||
"flow_bindings": [
|
||||
{
|
||||
"device_id": "542017",
|
||||
"link_id": "542017"
|
||||
},
|
||||
{
|
||||
"device_id": "486033",
|
||||
"link_id": "486033"
|
||||
},
|
||||
{
|
||||
"device_id": "542121",
|
||||
"link_id": "570995"
|
||||
},
|
||||
{
|
||||
"device_id": "524529",
|
||||
"link_id": "524529"
|
||||
},
|
||||
{
|
||||
"device_id": "541061",
|
||||
"link_id": "541061"
|
||||
}
|
||||
],
|
||||
"boundary_valve_ids": [
|
||||
"valve-316b3cfa3f40446e90f8c4c4",
|
||||
"valve-35cccbe86fdf480884e196b5",
|
||||
"valve-75f76a9f944d4f9aa1941a1b",
|
||||
"valve-e3238a6bd5784e8d86b9b25e"
|
||||
],
|
||||
"output_node_ids": [
|
||||
"135160",
|
||||
"159673",
|
||||
"170438",
|
||||
"170464",
|
||||
"170486",
|
||||
"174754",
|
||||
"175618"
|
||||
],
|
||||
"output_link_ids": [
|
||||
"486033",
|
||||
"524529",
|
||||
"541061",
|
||||
"541725",
|
||||
"542017",
|
||||
"570995",
|
||||
"valve-316b3cfa3f40446e90f8c4c4",
|
||||
"valve-35cccbe86fdf480884e196b5",
|
||||
"valve-75f76a9f944d4f9aa1941a1b",
|
||||
"valve-e3238a6bd5784e8d86b9b25e"
|
||||
]
|
||||
},
|
||||
{
|
||||
"role": "secondary",
|
||||
"pipe_id": "538540",
|
||||
"diameter_mm": 500.0,
|
||||
"length_m": 121.485,
|
||||
"start_node_id": "101686",
|
||||
"emitter_node_id": "101680",
|
||||
"longitude": 121.48750960472518,
|
||||
"latitude": 30.82388769570228,
|
||||
"candidate_orifice_areas_cm2": {
|
||||
"light": 61.359232,
|
||||
"medium": 122.718463,
|
||||
"severe": 245.436926
|
||||
},
|
||||
"pressure_bindings": [
|
||||
{
|
||||
"device_id": "121456",
|
||||
"node_id": "121456"
|
||||
},
|
||||
{
|
||||
"device_id": "102837",
|
||||
"node_id": "102837"
|
||||
},
|
||||
{
|
||||
"device_id": "84776",
|
||||
"node_id": "84776"
|
||||
},
|
||||
{
|
||||
"device_id": "112240",
|
||||
"node_id": "112240"
|
||||
},
|
||||
{
|
||||
"device_id": "87082",
|
||||
"node_id": "87082"
|
||||
}
|
||||
],
|
||||
"flow_bindings": [
|
||||
{
|
||||
"device_id": "542017",
|
||||
"link_id": "542017"
|
||||
},
|
||||
{
|
||||
"device_id": "486033",
|
||||
"link_id": "486033"
|
||||
},
|
||||
{
|
||||
"device_id": "542121",
|
||||
"link_id": "570995"
|
||||
},
|
||||
{
|
||||
"device_id": "508453",
|
||||
"link_id": "508453"
|
||||
},
|
||||
{
|
||||
"device_id": "527865",
|
||||
"link_id": "527865"
|
||||
}
|
||||
],
|
||||
"boundary_valve_ids": [
|
||||
"valve-beab0b66708a4ae99b5e7f5f",
|
||||
"valve-db75ca893cd246d498786475"
|
||||
],
|
||||
"output_node_ids": [
|
||||
"101680",
|
||||
"101686",
|
||||
"102837",
|
||||
"112240",
|
||||
"121456",
|
||||
"84776",
|
||||
"87082"
|
||||
],
|
||||
"output_link_ids": [
|
||||
"486033",
|
||||
"508453",
|
||||
"527865",
|
||||
"538540",
|
||||
"542017",
|
||||
"570995",
|
||||
"valve-beab0b66708a4ae99b5e7f5f",
|
||||
"valve-db75ca893cd246d498786475"
|
||||
]
|
||||
},
|
||||
{
|
||||
"role": "secondary",
|
||||
"pipe_id": "541116",
|
||||
"diameter_mm": 200.0,
|
||||
"length_m": 228.424,
|
||||
"start_node_id": "40239",
|
||||
"emitter_node_id": "50835",
|
||||
"longitude": 121.44164078044408,
|
||||
"latitude": 30.982724814068543,
|
||||
"candidate_orifice_areas_cm2": {
|
||||
"light": 9.817477,
|
||||
"medium": 19.634954,
|
||||
"severe": 39.269908
|
||||
},
|
||||
"pressure_bindings": [
|
||||
{
|
||||
"device_id": "32070",
|
||||
"node_id": "32070"
|
||||
},
|
||||
{
|
||||
"device_id": "406444",
|
||||
"node_id": "406444"
|
||||
},
|
||||
{
|
||||
"device_id": "49186",
|
||||
"node_id": "49186"
|
||||
},
|
||||
{
|
||||
"device_id": "28671",
|
||||
"node_id": "28671"
|
||||
},
|
||||
{
|
||||
"device_id": "31346",
|
||||
"node_id": "31346"
|
||||
}
|
||||
],
|
||||
"flow_bindings": [
|
||||
{
|
||||
"device_id": "542017",
|
||||
"link_id": "542017"
|
||||
},
|
||||
{
|
||||
"device_id": "486033",
|
||||
"link_id": "486033"
|
||||
},
|
||||
{
|
||||
"device_id": "542121",
|
||||
"link_id": "570995"
|
||||
},
|
||||
{
|
||||
"device_id": "527865",
|
||||
"link_id": "527865"
|
||||
},
|
||||
{
|
||||
"device_id": "489082",
|
||||
"link_id": "489082"
|
||||
}
|
||||
],
|
||||
"boundary_valve_ids": [
|
||||
"valve-89687cf53f95423c8f768d52",
|
||||
"valve-9781a936671f41a9bcde8c61",
|
||||
"valve-9e1f308e13df43a398f2e0d5",
|
||||
"valve-a29aa553a66e490f96b443de"
|
||||
],
|
||||
"output_node_ids": [
|
||||
"28671",
|
||||
"31346",
|
||||
"32070",
|
||||
"40239",
|
||||
"406444",
|
||||
"49186",
|
||||
"50835"
|
||||
],
|
||||
"output_link_ids": [
|
||||
"486033",
|
||||
"489082",
|
||||
"527865",
|
||||
"541116",
|
||||
"542017",
|
||||
"570995",
|
||||
"valve-89687cf53f95423c8f768d52",
|
||||
"valve-9781a936671f41a9bcde8c61",
|
||||
"valve-9e1f308e13df43a398f2e0d5",
|
||||
"valve-a29aa553a66e490f96b443de"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"version": "burst-demo-scada@1",
|
||||
"project_code": "tjw_bench_tjwater2_20261105",
|
||||
"normal_source_start": "2026-05-01T00:00:00+08:00",
|
||||
"day_count": 17,
|
||||
"noise_seed": "tjwater2-normal-scada-v1",
|
||||
"cases": [
|
||||
{
|
||||
"demo_id": "DEMO-202605-B26-severe",
|
||||
"case_key": "B26-severe",
|
||||
"target_start": "2026-05-01T00:00:00+08:00",
|
||||
"event_start": "2026-05-16T08:00:00+08:00",
|
||||
"entry_mode": "scada_auto"
|
||||
},
|
||||
{
|
||||
"demo_id": "DEMO-202606-B28-severe",
|
||||
"case_key": "B28-severe",
|
||||
"target_start": "2026-06-01T00:00:00+08:00",
|
||||
"event_start": "2026-06-16T20:00:00+08:00",
|
||||
"entry_mode": "scada_auto"
|
||||
},
|
||||
{
|
||||
"demo_id": "DEMO-202607-B30-severe",
|
||||
"case_key": "B30-severe",
|
||||
"target_start": "2026-07-01T00:00:00+08:00",
|
||||
"event_start": "2026-07-16T08:00:00+08:00",
|
||||
"entry_mode": "work_order_60"
|
||||
},
|
||||
{
|
||||
"demo_id": "DEMO-202608-B35-severe",
|
||||
"case_key": "B35-severe",
|
||||
"target_start": "2026-08-01T00:00:00+08:00",
|
||||
"event_start": "2026-08-16T12:00:00+08:00",
|
||||
"entry_mode": "work_order_120"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -4,8 +4,8 @@
|
||||
"project_code": "tjwater2_v2",
|
||||
"project_id": "871d7538-589c-47de-a6d5-3185ccb9f0e1",
|
||||
"model_expectations": {
|
||||
"node_count": 92392,
|
||||
"pipe_count": 91049,
|
||||
"node_count": 92393,
|
||||
"pipe_count": 91050,
|
||||
"valve_count": 4489,
|
||||
"flow_unit": "LPS",
|
||||
"pressure_unit": "METERS",
|
||||
@@ -42,95 +42,98 @@
|
||||
"max_flow_distance_m": 1.0
|
||||
},
|
||||
"cases": [
|
||||
{"case_id": "B01", "pipe_id": "541720", "occurred_at": "2026-10-08T02:00:00+08:00"},
|
||||
{"case_id": "B02", "pipe_id": "541720", "occurred_at": "2026-10-08T12:00:00+08:00"},
|
||||
{"case_id": "B03", "pipe_id": "520323", "occurred_at": "2026-10-08T02:00:00+08:00"},
|
||||
{"case_id": "B04", "pipe_id": "520323", "occurred_at": "2026-10-08T12:00:00+08:00"}
|
||||
{"case_id": "B05", "pipe_id": "564345", "occurred_at": "2026-10-08T02:00:00+08:00"},
|
||||
{"case_id": "B06", "pipe_id": "564345", "occurred_at": "2026-10-08T12:00:00+08:00"},
|
||||
{"case_id": "B07", "pipe_id": "540173", "occurred_at": "2026-10-08T02:00:00+08:00"},
|
||||
{"case_id": "B08", "pipe_id": "540173", "occurred_at": "2026-10-08T12:00:00+08:00"}
|
||||
],
|
||||
"pipes": [
|
||||
{
|
||||
"role": "primary",
|
||||
"pipe_id": "541720",
|
||||
"diameter_mm": 800,
|
||||
"length_m": 483.038,
|
||||
"start_node_id": "532586",
|
||||
"emitter_node_id": "502046",
|
||||
"longitude": 121.483586,
|
||||
"latitude": 30.996219,
|
||||
"pipe_id": "564345",
|
||||
"diameter_mm": 150,
|
||||
"length_m": 249.176,
|
||||
"start_node_id": "92686",
|
||||
"emitter_node_id": "7427148338069072993",
|
||||
"longitude": 121.469870,
|
||||
"latitude": 31.005517,
|
||||
"candidate_orifice_areas_cm2": {
|
||||
"light": 157.079633,
|
||||
"medium": 314.159265,
|
||||
"severe": 628.318531
|
||||
"light": 5.522331,
|
||||
"medium": 11.044662,
|
||||
"severe": 22.089323
|
||||
},
|
||||
"pressure_bindings": [
|
||||
{"device_id": "447382", "node_id": "7543252226207002012"},
|
||||
{"device_id": "444985", "node_id": "14141146287589625673"},
|
||||
{"device_id": "92724", "node_id": "92724"},
|
||||
{"device_id": "100026", "node_id": "100026"},
|
||||
{"device_id": "115039", "node_id": "115039"},
|
||||
{"device_id": "107672", "node_id": "107672"}
|
||||
{"device_id": "107672", "node_id": "107672"},
|
||||
{"device_id": "107651", "node_id": "107651"}
|
||||
],
|
||||
"flow_bindings": [
|
||||
{"device_id": "542121", "link_id": "pipe-eaa5c512bd8f4649b37bdbe3d"},
|
||||
{"device_id": "542017", "link_id": "542017"},
|
||||
{"device_id": "486033", "link_id": "486033"},
|
||||
{"device_id": "542121", "link_id": "570995"},
|
||||
{"device_id": "449417", "link_id": "449417"},
|
||||
{"device_id": "516234", "link_id": "516234"}
|
||||
],
|
||||
"boundary_valve_ids": [
|
||||
"valve-6fd58ab5b5714fbab738a544",
|
||||
"valve-8aea6a1876d8475b9f35a845"
|
||||
"valve-12b1e80612f9432da0e64485",
|
||||
"valve-b6cc8d68e38f43c9b3389e04",
|
||||
"valve-fa0583a6ac134aeb9513ada5",
|
||||
"valve-f189692cccbd40a69dc26cca",
|
||||
"valve-880f34624a444f5497faf2ca"
|
||||
],
|
||||
"output_node_ids": [
|
||||
"12290448554375617026", "1371402016460607520", "400946", "413270",
|
||||
"414851", "465564", "466633", "482531", "493166", "502046",
|
||||
"532586", "7543252226207002012", "14141146287589625673",
|
||||
"100026", "115039", "107672"
|
||||
"92686", "7427148338069072993", "92724", "100026", "115039", "107672", "107651"
|
||||
],
|
||||
"output_link_ids": [
|
||||
"541720", "pipe-eaa5c512bd8f4649b37bdbe3d", "449417", "516234",
|
||||
"valve-6fd58ab5b5714fbab738a544",
|
||||
"valve-8aea6a1876d8475b9f35a845"
|
||||
"564345", "542017", "486033", "570995", "449417", "516234",
|
||||
"valve-12b1e80612f9432da0e64485", "valve-b6cc8d68e38f43c9b3389e04",
|
||||
"valve-fa0583a6ac134aeb9513ada5", "valve-f189692cccbd40a69dc26cca", "valve-880f34624a444f5497faf2ca"
|
||||
]
|
||||
},
|
||||
{
|
||||
"role": "secondary",
|
||||
"pipe_id": "520323",
|
||||
"diameter_mm": 300,
|
||||
"length_m": 33.5951,
|
||||
"start_node_id": "91054",
|
||||
"emitter_node_id": "91055",
|
||||
"longitude": 121.468305,
|
||||
"latitude": 30.932363,
|
||||
"pipe_id": "540173",
|
||||
"diameter_mm": 200,
|
||||
"length_m": 162.128,
|
||||
"start_node_id": "500819",
|
||||
"emitter_node_id": "517120",
|
||||
"longitude": 121.503495,
|
||||
"latitude": 30.895889,
|
||||
"candidate_orifice_areas_cm2": {
|
||||
"light": 22.089323,
|
||||
"medium": 44.178647,
|
||||
"severe": 88.357293
|
||||
"light": 9.817477,
|
||||
"medium": 19.634954,
|
||||
"severe": 39.269908
|
||||
},
|
||||
"pressure_bindings": [
|
||||
{"device_id": "91100", "node_id": "91100"},
|
||||
{"device_id": "98367", "node_id": "98367"},
|
||||
{"device_id": "69693", "node_id": "69693"},
|
||||
{"device_id": "78085", "node_id": "78085"},
|
||||
{"device_id": "105833", "node_id": "105833"}
|
||||
{"device_id": "535381", "node_id": "535381"},
|
||||
{"device_id": "142377", "node_id": "142377"},
|
||||
{"device_id": "430299", "node_id": "3186958412475415759"},
|
||||
{"device_id": "121456", "node_id": "121456"},
|
||||
{"device_id": "87082", "node_id": "87082"}
|
||||
],
|
||||
"flow_bindings": [
|
||||
{"device_id": "527865", "link_id": "527865"},
|
||||
{"device_id": "489082", "link_id": "489082"},
|
||||
{"device_id": "531564", "link_id": "531564"}
|
||||
{"device_id": "542017", "link_id": "542017"},
|
||||
{"device_id": "486033", "link_id": "486033"},
|
||||
{"device_id": "542121", "link_id": "570995"},
|
||||
{"device_id": "491909", "link_id": "491909"},
|
||||
{"device_id": "524529", "link_id": "524529"}
|
||||
],
|
||||
"boundary_valve_ids": [
|
||||
"valve-18b95b02a7564e7dac737102",
|
||||
"valve-3a8fd38228d74583a08b00d9",
|
||||
"valve-3b453e4e901049d1a1346689",
|
||||
"valve-ad39de2f15f24ade9d18c836",
|
||||
"valve-fc1164434a954f418a6f11d6"
|
||||
"valve-fd82099a578046b3bde41d63",
|
||||
"valve-c9f6343550784ec9b3e97f2b",
|
||||
"valve-1efb02be70174163948ab7ca",
|
||||
"valve-4686c52b4d7e4d149c1823de",
|
||||
"valve-aa89ee883fb4461b91eacc3d",
|
||||
"valve-5e6f00e167804055830f18d8"
|
||||
],
|
||||
"output_node_ids": ["91054", "91055", "91100", "98367", "69693", "78085", "105833"],
|
||||
"output_node_ids": ["500819", "517120", "535381", "142377", "3186958412475415759", "121456", "87082"],
|
||||
"output_link_ids": [
|
||||
"520323", "527865", "489082", "531564",
|
||||
"valve-18b95b02a7564e7dac737102",
|
||||
"valve-3a8fd38228d74583a08b00d9",
|
||||
"valve-3b453e4e901049d1a1346689",
|
||||
"valve-ad39de2f15f24ade9d18c836",
|
||||
"valve-fc1164434a954f418a6f11d6"
|
||||
"540173", "542017", "486033", "570995", "491909", "524529",
|
||||
"valve-fd82099a578046b3bde41d63", "valve-c9f6343550784ec9b3e97f2b",
|
||||
"valve-1efb02be70174163948ab7ca", "valve-4686c52b4d7e4d149c1823de",
|
||||
"valve-aa89ee883fb4461b91eacc3d", "valve-5e6f00e167804055830f18d8"
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
-- Queued calculations remain durable until claimed or explicitly cancelled.
|
||||
DROP INDEX IF EXISTS workbench.calculation_jobs_queue_expiry;
|
||||
ALTER TABLE workbench.calculation_jobs
|
||||
DROP COLUMN IF EXISTS queue_expires_at;
|
||||
@@ -0,0 +1,26 @@
|
||||
-- Run against each project business database during the coordinated Server cutover.
|
||||
-- Historical tasks are explicitly assigned to production; retain existing scopes.
|
||||
BEGIN;
|
||||
SET LOCAL lock_timeout = '3s';
|
||||
SET LOCAL statement_timeout = '30s';
|
||||
|
||||
UPDATE workbench.operational_tasks
|
||||
SET document = jsonb_set(document, '{execution_scope}', '"production"'::jsonb)
|
||||
WHERE NOT (document ? 'execution_scope');
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM pg_constraint
|
||||
WHERE conrelid = 'workbench.operational_tasks'::regclass
|
||||
AND conname = 'operational_tasks_execution_scope_required'
|
||||
) THEN
|
||||
ALTER TABLE workbench.operational_tasks
|
||||
ADD CONSTRAINT operational_tasks_execution_scope_required CHECK (
|
||||
(jsonb_typeof(document->'execution_scope') = 'string'
|
||||
AND length(btrim(document->>'execution_scope')) > 0) IS TRUE
|
||||
);
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
COMMIT;
|
||||
@@ -0,0 +1,14 @@
|
||||
-- Run-scoped artifact facts survive domain-event retention without a new lifecycle table.
|
||||
BEGIN;
|
||||
SET LOCAL lock_timeout = '5s';
|
||||
|
||||
ALTER TABLE workbench.operational_task_runs
|
||||
ADD COLUMN IF NOT EXISTS artifact_manifest jsonb NOT NULL DEFAULT '{}'::jsonb;
|
||||
ALTER TABLE workbench.operational_task_runs
|
||||
DROP CONSTRAINT IF EXISTS operational_task_runs_artifact_manifest_object;
|
||||
ALTER TABLE workbench.operational_task_runs
|
||||
ADD CONSTRAINT operational_task_runs_artifact_manifest_object
|
||||
CHECK (jsonb_typeof(artifact_manifest) = 'object');
|
||||
COMMENT ON COLUMN workbench.operational_task_runs.artifact_manifest IS
|
||||
'Server-verified per-run artifact facts. scada_presentation holds cumulative targets and latest chart coverage, independent of browser receipts and event retention; not run status.';
|
||||
COMMIT;
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
name: burst-event-handling
|
||||
description: 根据爆管上报工单处理事件,完成资产关联、隔离方案、影响说明、处置建议与反馈复核。
|
||||
metadata:
|
||||
version: "1.1.3"
|
||||
---
|
||||
|
||||
# 爆管事件处理
|
||||
|
||||
适用于工单触发的 `workflow=burst_report`。以工单上报的事件位置和资产线索开展处置,依次完成下列步骤;每一步给出实际结果或具体待补项,已有证据可直接复用。SCADA 异常发现使用独立的 scada-anomaly 流程。
|
||||
|
||||
先按handling_context接续:等待阶段按第6节保存本轮阶段报告;verification_ready复用已有异常分析与资产、隔离证据,仅补反馈和恢复复核,不重跑初次调查步骤。真正新增的独立事实另行核查。
|
||||
|
||||
正文、表格、图表及引用时间统一按Asia/Shanghai(UTC+8)阅读。工具返回的+08:00时间保持原值;UTC来源转换后同步更新日期与窗口,例如2026-07-16T00:00:00Z对应7月16日08:00。结构化查询与sample_times保留明确时区,跨日按转换后的日期展示。
|
||||
|
||||
## 1. 接收与核对
|
||||
|
||||
读取当前任务、原工单及版本、发生时间、上报位置、描述、现场反馈、既有报告与动作。工单按 system_data 读取,区分上报内容与已核实事实,分析证据按各自来源说明。后续轮次优先处理新增事实和上轮待办。
|
||||
|
||||
## 2. 地点核验与资产关联
|
||||
|
||||
已有管段或节点编号时,用 get_element_properties 核对资产;已有坐标时直接使用,文字地址通过 geocode 解析后用 query_nearby_elements 获取附近资产。结合属性、道路、连接关系和现场记录确定处理对象,使用 get_node_connections 补充连接依据。
|
||||
|
||||
在地图展示上报位置和关联资产,标明关联依据及现场确认状态。存在多个合理对象时保留候选及选择理由;关键位置需要人工补充时,先完成独立工作,再用 ask_task_user 提问。
|
||||
|
||||
## 3. 隔离方案
|
||||
|
||||
对关联资产调用 analyze_valve_isolation,保留 isolatable、must_close_valves、optional_valves、affected_nodes、supply_affected_nodes 及计数。单个候选分别分析;联合事故按真实联合场景提交。已知失效阀门通过 disabled_valves 评估替代边界;现场可操作性列为执行前核验条件。
|
||||
|
||||
展示方案阀门和直接隔离、供水失源两类节点;节点覆盖使用 generate_node_coverage,再通过 publish_task_presentation 发布地图请求。取得初步隔离方案后继续影响分析和处置建议。
|
||||
|
||||
## 4. 影响说明
|
||||
|
||||
报告固定包含“影响说明”,逐项列出对象、范围、时间、单位、依据和实际或预计的性质:
|
||||
|
||||
- **事件当前影响**:核查相关测点在事件窗与正常对照窗的压力、流量变化,给出已有数据覆盖的异常对象、持续时间和恢复情况。SCADA 用于核验事件影响,数据缺口与已完成的资产、拓扑结果分别记录。
|
||||
- **隔离方案影响**:分别说明 affected_nodes 对应的直接隔离范围,以及 supply_affected_nodes 对应的新增失去水源连接范围,保留各自计数和地图覆盖。
|
||||
- **水力影响**:输入具备且工具已授权时,用 analyze_valve_closure 评估关阀后的压力、流量和低压范围。起算时间、阀门组合、模型边界及输出对象明确可比时给出前后差异;其余情况说明当前模型结果和待补条件。已有可靠破口参数时可按需补充事件情景模拟。
|
||||
- **用户与保障影响**:有用户、设施或区域关联资料时,列出受影响用户、重要设施和保障需求。仅有模型节点资料时报告节点与区域范围,将用户数量列为待核实。
|
||||
|
||||
现场实际停水、模型预计低压、拓扑失源和展示覆盖分别标注。漏量、累计漏水量使用有来源的方法与输入,列明单位及覆盖;其余情况记录待补依据。缺少可选模拟参数时,继续形成已有依据的隔离和保障建议。
|
||||
|
||||
## 5. 处置建议与报告
|
||||
|
||||
先保存 interim/concluded 阶段报告,再按需要提出核验、通知、供水保障、隔离与抢修、恢复复测等动作草稿。关联现场工作采用 work_order_type=general,保留原爆管上报工单作为事件来源。草稿标题写“地点+动作”,正文以三条概括任务、执行条件和回执要求;审批与实际执行按 Server 流程完成。
|
||||
|
||||
报告固定给出事件概况、资产关联、隔离方案、影响说明、处置建议和待办。摘要以150字以内为目标,指标和资产明细放表格,证据展开保留来源。待办应指向具体事实、参数或动作。本轮可执行分析完成且依赖结果已读取后,若仍待审批、现场工单或恢复监测,按第6节以interim结束本轮,恢复闭环后发布final;无此依赖时按后台调查停止条件发布final。
|
||||
|
||||
## 6. 反馈与闭环复核
|
||||
|
||||
读取摘要中的 handling_context。waiting_work_order 表示已派单、等待有效完工反馈,创建回执的 completed_at 只证明创建动作完成。现场处理和恢复供水以关联工单的 completed 反馈为准;returned_at 是现场反馈的业务时间,演示时不得换成电脑当前时间。
|
||||
|
||||
草稿待审批、waiting_work_order或waiting_scada且当前无独立可执行工作时,保存本轮interim/concluded阶段报告并关联真实action_id,然后结束本轮执行,任务保持等待并释放租约。不能只更新progress或沿用旧轮报告,不循环查询工单或重复异常分析。创建成功及普通进度更新不启动新一轮;工单失败或取消时重新决策,完工后由Server等待恢复证据就绪,再唤醒同一任务。
|
||||
|
||||
waiting_scada 表示恢复供水和监测复核阶段,以 returned_at 为反馈时刻 T,读取严格晚于 T 的最近一个 SCADA 采样时刻作为复核来源,说明测点、实际采样时间、读数、单位和正常范围;不等待额外周期,不使用 T 之前、T 当刻或更晚的正常数据替代。后台按事件前15天同期正常包络核验该时刻。最近时刻缺测或异常时保留实际问题,不宣称复核成功。保存本轮阶段成果,等待后续数据接续,不反复派发同一工单。
|
||||
|
||||
verification_ready 时读取 verification_evidence_id 对应的证据与快照,核对工单反馈、T、最近采样时刻和测点,展示该时刻读数与正常范围;具备真实多时刻数据时再展示曲线,工单 completed 且恢复证据 status=passed 时发布 final/concluded 恢复闭环报告并引用该证据。Server 完成运行后关闭任务并解除关联告警,phase=completed 表示已闭环。照片、阀位或复测记录未提供时仅记为资料缺项,不增设关闭前置条件。来源系统若另有明确行政结案流程,按其实际状态另记,不推断为当前任务尚未闭环。
|
||||
@@ -1,53 +1,15 @@
|
||||
---
|
||||
name: burst-report
|
||||
description: 调查供水爆管上报,形成有依据的候选位置、隔离阀门、影响范围和后续处置建议。
|
||||
description: 爆管业务流程的历史入口索引,运行时按事件来源选择独立 Skill。
|
||||
metadata:
|
||||
version: "2.0.8"
|
||||
version: "2.0.11"
|
||||
---
|
||||
|
||||
# 爆管上报处置
|
||||
# 爆管业务流程入口
|
||||
|
||||
目标是形成有依据的候选位置、隔离方案、影响判断和可审批处置,并在后续轮次根据真实反馈复核事件闭环。区分候选方案、现场确诊和事件闭环;后两者不能提前阻断当前可完成的分析。演练数据可以用于比较,但必须标注 `synthetic`,不能证明生产故障。
|
||||
Server 根据绑定任务的 workflow 和 replay_context 分发当前流程:
|
||||
|
||||
## 接续与证据
|
||||
- 工单触发的 burst_report:使用 burst-event-handling,依次完成接收核对、地点与资产关联、隔离方案、影响说明、处置建议和反馈复核。
|
||||
- SCADA 异常及带 replay_context 的历史回放任务:使用 scada-anomaly,完成冻结数据核验、诊断、候选定位与隔离影响分析。
|
||||
|
||||
读取当前任务、事件时刻、地点、来源、已有报告、证据、动作和反馈。新一轮优先检查新增事实和上轮未决项,按时间、来源、对象及输入比较新旧结果,不按 ID 推断先后,也不因原工单未更新就断言没有新证据。保留本流程名称和版本。
|
||||
|
||||
若任务来源带有 `replay_context`,这是 SCADA 历史回放中的疑似爆管演示。先读取任务摘要中的步长监测结论,再用 `query_scada_readings` 核对冻结批次;不得查询知识截止之后的数据。回放中的 `run_burst_location` 只接受 monitoring 模式,并要求显式提供本轮已处理范围内的异常窗和正常对照窗;Server 会从冻结清洗输出构造计算输入。缺少有依据的 `burst_leakage` 时保留“定位待补充”,不得用出厂流量增量、压力下降或默认值替代。回放结果保持 synthetic,不发布真实工单或设备动作。
|
||||
|
||||
## 定位与影响分析
|
||||
|
||||
根据现有线索自主选择必要步骤:
|
||||
|
||||
1. 核对已有管段或节点。只有地点描述时,先解析位置,再查询附近资产、属性和连接。地点明确后展示调查位置,候选资产取得后展示正在核查的管段或节点。空间邻近只支持候选,不能单独证明故障。
|
||||
2. 核对 SCADA 设备与模型节点映射,用 `query_scada_readings` 或 `query_element_history` 查询事件前后及正常对照。先用少量对象和合适时间窗,确认返回时间实际覆盖事件;必要时缩窗或翻页。同步展示核查点位,并将取得的事件、对照压力或流量序列画成过程曲线,分清指标和单位;保留缺测、真实零值和覆盖限制。
|
||||
3. 有异常观测时评估爆管定位输入。`run_burst_location` 必须有 `burst_leakage`(m³/s);先查询已有估计及方法依据,或结合边界流量、用水和储量变化判断能否估计。不能把单表增量、压力下降或任意数值直接当漏量。
|
||||
4. 有依据的漏量估计或明确的情景假设可用于条件定位,须说明来源、方法和不确定性。`pressure_scada_ids` 使用模型节点 ID;监测模式读取 SCADA,模拟模式须引用已有分析运行。取得多个可比候选的评价值后,用柱状图展示实际排序及评价含义。压力点和历史数据满足工具要求时可选 `run_burst_detection`,但侦测不代替定位,评价值也不表述为概率。缺少定位参数时说明已查来源和影响,继续其他可独立完成的分析。
|
||||
5. 对已确认或有依据的候选资产使用 `analyze_valve_isolation` 评估阀门隔离,无需等待现场确诊。逐候选比较;多个事故 ID 表示联合场景。保留可隔离性、必关阀、可选阀及受影响节点。结果取得后展示直接隔离与供水影响节点的覆盖示意,形成或调整关阀方案时展示对应阀门;区分两种影响及必关、可选阀。未知阀门状态不能写成可用,不可隔离或空结果也不能解释为零影响;已知阀门失效时评估替代边界。
|
||||
6. 输入具备时用 `analyze_valve_closure` 分析关阀水力影响,核对阀门组合、起算时间、时长和输出对象。结果返回后,用曲线展示请求节点的压力、请求管段的流量;有同口径方案或基准时再画对比图,不自造前后变化。不同单位分图,标明局部模型结果。区分拓扑范围、水力影响和有依据的用户影响,不虚构户数,不用局部结果代表全网。
|
||||
|
||||
查询错误、权限拒绝或接口故障不是缺数证据。必要分析无法执行时,说明失败步骤和已经完成的部分,不借有限摘要宣布调查完成。
|
||||
|
||||
## 事件前后影响说明
|
||||
|
||||
每轮按实际证据说明以下阶段的影响,给出结果或具体限制;尚未发生的处置列为预测和待验证项:
|
||||
|
||||
1. **事件前与爆管后、隔离前**:核对正常基线和异常窗口,说明压力、流量变化及已覆盖的异常点位、资产和区域。候选管段及破口面积有依据时,可用 `run_burst_simulation` 补充情景水力影响,标明假设。漏量估计须说明边界流量、正常用水及储量变化等依据;有可靠漏量序列和时间覆盖时可累计漏水量,保留单位、方法和不确定性。定位工具的 `burst_leakage` 是输入,不能当成反演所得漏量。
|
||||
2. **关阀隔离方案**:说明止漏目标、直接隔离节点和关阀后新增失去水源连接的节点,分别保留 `affected_nodes` 与 `supply_affected_nodes` 及其计数。结合关阀水力结果说明预计停水、低压或供水不足的范围和覆盖限制;有用户关联资料时说明受影响用户、重要设施及保障需求。拓扑失源、模型预测和现场实际停水分别标注,不把节点数当户数。
|
||||
3. **实际隔离与恢复供水后**:后续轮次按实际关阀、抢修和恢复时间读取前后监测与现场反馈,对照预测核查止漏效果、实际影响范围和恢复情况,说明持续低压、异常流量、再次异常及尚待验证的影响。核对正常基线、观察窗口和判定依据,工单状态或单个正常读数不足以证明影响消除。
|
||||
|
||||
比较须核对对象、单位、时间对齐、模型版本、需水与水源边界、泵阀状态及计算设置;区分实测变化与模型方案差异。关阀工具不会自动继承爆管情景,条件不同的独立计算不能直接作为同一事故前后效果。可比时给出变化量和方案差异,缺少可比基准时说明当前结果及缺口,不补造改善比例。必要的紧急核验、通知和供水保障不等待可选的完整模拟。
|
||||
|
||||
## 处置与报告
|
||||
|
||||
报告分别给出位置候选及依据、隔离阀门及可用性、替代方案、事件前后分阶段的拓扑与水力影响、用户影响边界,以及处置和闭环缺口。影响说明保留时间窗、对象、证据、实测或预测属性及限制。只有初步结果时明确范围,不宣称已经定位、确诊或闭环。位置、候选、监测、影响和阀门方案按实际调查步骤同步展示,遵循后台展示规则,不用报告定位链接代替过程展示。覆盖面保留分组、缺失几何和示意边界,不能当作实际停水、水力影响或用户服务边界。无数据或展示失败时说明具体限制;展示不替代证据,也不要求等待浏览器。
|
||||
|
||||
形成具体建议后,先保存阶段结论,再提议必要动作。人工核验、通知、定位隔离、抢修、适用的清洗复测和恢复回访使用关联的 `work_order_type=general` 草稿;原爆管事件保持独立。草稿写明位置、任务、依赖、执行前核验和回执要求。没有通知适配器时明确由人工实施。设备命令仅用于真实可控目标,审批和执行由 Server 控制。
|
||||
|
||||
抢修或复测尚未发生不阻止有边界的初步方案。现场阀门状态应写为执行前核验条件;工单创建、审批或派发均不证明现场完成。
|
||||
|
||||
## 闭环复核
|
||||
|
||||
后续轮次核对实际故障资产和原因、隔离实施、抢修记录、适用的清洗与水质复测、恢复供水及恢复后监测、用户通知回访、未决动作和遗留风险。每项标明已核验、待核验或不适用及依据。
|
||||
|
||||
证据齐全时报告具备结案条件,由来源系统的有权限人员确认关闭。原工单未结案时写明待确认,读取其新版本后才能认定闭环;误报结案也须有足够的排除证据和人工确认。
|
||||
通过 get_task_context 的 section="skill" 读取实际流程,保留返回的 name、version、content_sha256。工具范围由绑定任务的服务端策略及 Agent 会话策略共同确定。
|
||||
|
||||
@@ -1,18 +1,58 @@
|
||||
---
|
||||
name: scada-anomaly
|
||||
description: 调查 SCADA 回放中的异常信号,区分读数质量与运行异常,形成可追溯的演示报告。
|
||||
description: 从 SCADA 指标异常开展数据核验、爆管诊断、候选定位与影响分析。
|
||||
metadata:
|
||||
version: "1.0.1"
|
||||
version: "1.0.16"
|
||||
---
|
||||
|
||||
# SCADA 回放异常调查
|
||||
# SCADA 数据异常调查
|
||||
|
||||
先读取任务上下文和来源中的 replay_context,核对规则、设备、单位、异常时刻及知识截止时间。数据来自已存 SCADA 的演示回放,不代表新采集的现场故障。
|
||||
读取任务与 replay_context,核对设备、单位、异常时刻、knowledge_cutoff 和 batch_sequence。使用本轮已提交的 SCADA 观测开展调查,同一持续事件沿用已有调查。证据来源字段用于记录数据取得方式,不构成停止分析的条件。
|
||||
|
||||
通过 query_scada_readings 读取事件前后本轮已处理的观测;查询结束时间不得晚于 knowledge_cutoff。结合 monitored_value 与 cleaned_value 判断数值变化,不将被可靠采集的低压或水质变化当作数据错误。缺测、无效值和未完成清洗应明确记录。
|
||||
先按handling_context接续:待审批/工单/监测且无独立工作时,保存本轮interim/concluded及action_id后退出,不轮询。verification_ready复用原异常证据,只补恢复复核;初次分析按下文。
|
||||
|
||||
报告说明事件前后影响:在已开放且已处理的观测范围内,核对异常前参考、异常期间及已有恢复观测,列出压力、流量或水质变化、持续时间、涉及测点和已核实的资产关联。只有对象、单位、时间与运行条件可比时才给出变化量;说明参考窗口、证据和覆盖限制。尚无恢复观测时列为待验证,读数恢复不能证明现场修复或用户影响消除。停水范围、关阀影响及泄漏量需要相应拓扑、水力、流量平衡或现场依据;当前证据无法支持时明确待分析,不由单点低压推定停水区、受影响户数或漏量。
|
||||
报告与图表的阅读时区统一为Asia/Shanghai(UTC+8)。工具返回的+08:00时间按原值使用;Z时间换算后同步更新日期、历史系列名和横轴标签,例如2026-07-16T00:00:00Z对应7月16日08:00。sample_times记录相同真实时间点并使用+08:00偏移,查询与冻结边界按带时区的时间点比较。图表标题、横轴和系列名只写日期、时刻与单位,不写“上海时间”“Asia/Shanghai”或“UTC+8”;横轴命名为“时间”。
|
||||
|
||||
当前回放调查仅开放冻结 SCADA 证据、设备与模型目录;不支持原生计算或绕过回放上限的其他数据入口。回放前历史用于后台基线初始化,当前证据接口不返回该部分。需要更多历史或进一步计算时,报告具体缺口,不能读取未来或虚构分析结果。
|
||||
## 数据核验与诊断
|
||||
|
||||
报告列出观测事实、规则信号、质量问题、尚未确认的原因和建议核查。压力异常不等于已确认爆管,浓度变化不等于已确认污染。回放只生成演示分析与报告,不创建真实现场工单、不下发设备命令。
|
||||
任一已配置指标异常即进入调查,包含压力、流量、水质规则和压力模型信号。先核验数据质量与测点,再判别原因;模型或组合指标有缺口时,继续核查已发现的单项异常。
|
||||
|
||||
通过 query_scada_readings 省略device_ids读取冻结异常窗内全部SCADA设备的完整属性和授权观测,覆盖压力、流量、水质等全部类型;正常对照也采用全量读取。Server自动读完所有设备页和观测页,完整返回devices、readings、coverage,结果仍受8MiB整份上限约束,超限明确报错后按时间分段,不能丢弃设备。query_complete表示查询已读完,缺测与无有效值另行判断。查询结束时间止于knowledge_cutoff,数据批次止于batch_sequence;冻结范围外设备单列,不回退实时数据。按需核查开始前15天基线,复用已有完整监测回执。结合monitored_value、cleaned_value和质量标记判断变化,可靠采集的低压与水质变化保留为业务异常。
|
||||
|
||||
全量devices含完整目录,readings含观测,coverage含各设备读数数量、缺测及冻结范围外清单。不得由代表测点推断全网。单查list_scada_devices默认每页100,读至page.has_more=false,属性缩略不代表完整信息。全量后可按device_ids局部复核并持续翻页,不能替代全量输入。final引用本轮knowledge_cutoff及batch_sequence的全量证据;Server拒绝局部或旧批次证据。
|
||||
|
||||
分别说明压力异常模型得分、压力绝对下降及变化率、已配置出厂边界流量变化,再说明联合证据支持程度。使用实际权重、嫌疑门槛、动态阈值、参与测点与各自基线。综合分用于爆管嫌疑判断;得分越低越异常,目标告警与参考日分别解释。水质等其他异常按实际证据选择原因分支。
|
||||
|
||||
## 候选定位与隔离
|
||||
|
||||
综合分达到嫌疑门槛,或压力与模型证据支持高嫌疑时,继续候选定位。流量变化和现场确认作为证据维度,定位输入按当前工具要求准备。
|
||||
|
||||
检查摘要或 skill 响应顶层的 location_capability。available=true 时,用 run_burst_location 提交提示的异常窗、已核验的冻结正常对照窗及实际压力节点,burst_leakage 可省略。读取返回的候选管段与相似度,优先按 similarity_percent 展示百分比;只有 similarity 时,按0–1小数换算。取得候选后继续地图展示、隔离边界、节点覆盖与影响分析,完成报告后再收尾。其他定位路径使用有依据的漏量估计或明确情景假设。缺少定位输入时记录待补项,继续独立的资产、空间与拓扑核查。
|
||||
|
||||
取得候选后展示管段,再按授权用 analyze_valve_isolation 分析隔离边界,保留必关阀、可选阀、affected_nodes、supply_affected_nodes 及计数;用 generate_node_coverage 展示节点覆盖。候选与拓扑建议标注为待现场核验方案。最终报告前核对定位调用及返回结果,调用失败记录具体错误与输入缺口。
|
||||
|
||||
## 对比展示与影响
|
||||
|
||||
监测图必填chart.scada:role=current/comparison、metric=pressure/flow、unit、window_start/window_end(三小时);series绑定device_id、day_offset=0..7。evidence_refs包含所有采样来源。报告scada_device_ids声明表格目标。Server在final及成功结束校验本轮总览和同期覆盖,SCADA_PRESENTATION_INCOMPLETE须补图;仅作参考不豁免。缺测须完整查询证明,保留null和设备图;scada.gaps由Server生成。恢复复核沿用恢复门禁。
|
||||
|
||||
先查询并发布以本轮knowledge_cutoff或异常时刻t为终点的[t-3小时,t]当前总览,覆盖地图定位、等级标记和表格列出的全部SCADA目标,含正常与异常测点。压力、流量按指标及单位分图,每设备一条序列,name带device_id;每图最多8组,超限按设备分图并用不同稳定chart_id。保留三小时内真实采样网格;截止非整刻不平移采样。超过100槽按连续时间分图,各用稳定chart_id,scada窗口仍填完整三小时;Server合并采样槽覆盖。低频无计划采样槽时,完整查询当天及历史窗口,以截止时刻单个null保留入口。缺测保留设备;“数据”视图沿用全部序列,报告附设备、单位、窗口、有效点数与缺测表。
|
||||
|
||||
定位输入具备后继续定位。压力、流量按设备另发七天同期复合图,不占用或替代全量当前总览。chart_type=time_comparison:x_data是UTC+8的HH:mm或HH:mm:ss共享槽,不拼接历史日期;当前[t-3小时,t]用series.type=line,前7天每天t-30分钟、t-15分钟、t用scatter。15分钟数据13槽,历史七组仅末3槽有值。每组sample_times与data等长,保留真实带时区日期;有效值与对应槽的本地时刻一致,缺测值和时间均null。当前曲线共用窗口,历史每组取此前某天对应窗口,跨午夜顺序排列;普通图用line/bar,坐标轴标单位。七天按t的本地日期回退,不含当天。按实际周期保留缺口,按需翻页并报告参考点数;基线未冻结、读取失败或缺测时说明并保留当前趋势,不退化为前一天或相邻两点。
|
||||
|
||||
影响说明列出异常前参考、异常期间和已有恢复观测中的压力、流量或水质变化、持续时间、涉及测点及已核实资产。可比对象、单位、时段与运行条件下给出变化量,分别标明实际观测、模型预测和拓扑隔离。停水范围与漏量引用相应拓扑、水力、流量平衡或现场依据;恢复观测、用户关联或现场证据缺失时列为具体待核实项。
|
||||
|
||||
## 范围与报告
|
||||
|
||||
本轮时序分析保持任务冻结窗口,后续播放形成的观测由相应轮次处理。资产属性、空间查询和拓扑隔离使用项目当前模型,注明模型时态;水力工具按已开放的冻结输入能力调用。定位调用执行权限、设备与冻结观测校验;基线或覆盖缺口按实际返回记录。
|
||||
|
||||
报告给出观测事实、规则信号、原因候选、定位与隔离结果、影响说明和建议。摘要以150字以内为目标,测点明细用表格、完整依据留在证据。候选、相似度和拓扑结果具备时,完成相应分析、展示与报告后再收尾;现场核验、审批和维修按实际进度列为待办,已完工事项采用回执状态。现场工单与设备操作遵守服务端动作权限,恢复复核通过后按下节闭环。
|
||||
|
||||
需要现场核验时,先发布本轮outcome=concluded的调查报告,再调用propose_task_action提出kind=create_work_order、work_order_type=general的关联预约工单草稿,引用候选与影响证据,写明核验对象、内容和回执要求。带replay_context的任务同样可提出草稿,描述中注明历史异常时刻及回放来源;草稿须由有权限的用户编辑、确认后发布,后续派发与反馈进入预约任务。草稿待确认时说明等待人工处理,不将调查报告完成写成现场处置完成。回放任务不支持设备命令。
|
||||
|
||||
## 完工后的恢复复核
|
||||
|
||||
关联工单completed后,以handling_context.returned_at为实际完工业务时间,不用创建、预约或电脑当前时间代替。Server读取严格晚于该时刻的最近一帧及事件前15天同相位对照,独立保存恢复证据。回放来源工单同样适用,不推进回放,原knowledge_cutoff与batch_sequence不变。
|
||||
|
||||
waiting_work_order等待完工;waiting_scada按last_check说明尚无后续采样、数据缺项或仍异常,等待监测复核。verification_ready时用get_task_evidence读取verification_evidence_id完整快照;工单completed且恢复证据status=passed时,发布final/concluded恢复闭环报告,同时引用原异常全量证据与当前恢复证据。Server完成运行后关闭任务并解除关联告警,phase=completed表示已闭环。阀位、漏点照片、现场复测数值等未提供时仅记为资料缺项,不新增现场核验、人工确认或更多采样的闭环条件。
|
||||
|
||||
恢复观测由Server独立保存,原冻结上限不阻断此证据的使用;query_scada_readings仍遵守冻结窗口。完工时间修订后使用新证据编号,旧证据仅作历史。确有独立工作时再提议新动作。
|
||||
|
||||
@@ -7,7 +7,7 @@ metadata:
|
||||
|
||||
# 水质投诉处置
|
||||
|
||||
目标是说明投诉和异常的实际覆盖、可能原因及依据、候选影响范围、采样或处置建议和后续复核条件。区分感官投诉、实测异常、污染源假设和已确认问题;不预设异常来自上游或二次供水。演练结果须标注 `synthetic`,不能证明生产风险。
|
||||
目标是说明投诉和异常的实际覆盖、可能原因及依据、候选影响范围、采样或处置建议和后续复核条件。区分感官投诉、实测异常、污染源假设和已确认问题;不预设异常来自上游或二次供水。工单按 `system_data` 读取,分析证据按各自来源说明。
|
||||
|
||||
## 接续与证据
|
||||
|
||||
|
||||
@@ -161,9 +161,13 @@ class UpdateWorkOrder(Model):
|
||||
clear_scenario_code: StrictBool = False
|
||||
feedback_requirements: list[Text] | None = None
|
||||
progress_note: Text | None = None
|
||||
occurred_at: WireDateTime | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_update(self):
|
||||
if self.occurred_at is not None:
|
||||
if self.occurred_at.tzinfo is None or not self.progress_note:
|
||||
raise ValueError("occurred_at requires a timezone and field feedback")
|
||||
changed = set(self.model_fields_set) - {"request_id", "expected_revision"}
|
||||
if not changed:
|
||||
raise ValueError("at least one work order field must be updated")
|
||||
|
||||
@@ -362,7 +362,7 @@ class PostgresStore:
|
||||
with self.connection(readonly=True) as conn:
|
||||
self._available(conn, who)
|
||||
return self._managed_document(conn, who, "work_orders", "work_order_id", work_order_id,
|
||||
"WORK_ORDER_NOT_FOUND", "测试工单不存在")
|
||||
"WORK_ORDER_NOT_FOUND", "工单不存在")
|
||||
|
||||
def create_work_order(self, who: Identity, command: CreateWorkOrder) -> dict[str, Any]:
|
||||
def write(conn, _):
|
||||
@@ -382,11 +382,11 @@ class PostgresStore:
|
||||
command: UpdateWorkOrder) -> dict[str, Any]:
|
||||
def write(conn, _):
|
||||
document = self._managed_document(conn, who, "work_orders", "work_order_id", work_order_id,
|
||||
"WORK_ORDER_NOT_FOUND", "测试工单不存在")
|
||||
"WORK_ORDER_NOT_FOUND", "工单不存在")
|
||||
ensure(document["revision"] == command.expected_revision, "REVISION_CONFLICT", "工单版本已变化,请重新读取", 409)
|
||||
changes = command.model_dump(mode="json", exclude_unset=True,
|
||||
exclude={"request_id", "expected_revision", "clear_assignee", "clear_scheduled_at",
|
||||
"clear_location", "clear_scenario_code", "progress_note"})
|
||||
"clear_location", "clear_scenario_code", "progress_note", "occurred_at"})
|
||||
if command.clear_assignee:
|
||||
changes["assignee"] = None
|
||||
if command.clear_scheduled_at:
|
||||
@@ -409,7 +409,7 @@ class PostgresStore:
|
||||
) + "。", "updated"
|
||||
else:
|
||||
title, detail, kind = "工单信息确认", "工单信息已确认,未记录新的现场进展。", "updated"
|
||||
document["activity"].append({"sequence": revision, "occurred_at": now,
|
||||
document["activity"].append({"sequence": revision, "occurred_at": command.occurred_at.isoformat() if command.occurred_at else now,
|
||||
"actor": who.user_id, "kind": kind,
|
||||
"title": title, "detail": detail})
|
||||
document.update(revision=revision, updated_at=now,
|
||||
@@ -421,7 +421,7 @@ class PostgresStore:
|
||||
command: DeleteResource) -> dict[str, Any]:
|
||||
def write(conn, _):
|
||||
document = self._managed_document(conn, who, "work_orders", "work_order_id", work_order_id,
|
||||
"WORK_ORDER_NOT_FOUND", "测试工单不存在")
|
||||
"WORK_ORDER_NOT_FOUND", "工单不存在")
|
||||
ensure(document["revision"] == command.expected_revision, "REVISION_CONFLICT", "工单版本已变化,请重新读取", 409)
|
||||
revision = document["revision"] + 1
|
||||
deleted_at = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compare replay-rule thresholds against frozen hydraulic outputs without rerunning hydraulics."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--calibration-dir", required=True, type=Path)
|
||||
parser.add_argument("--thresholds", nargs="+", type=float, default=[3, 5, 10, 20, 30])
|
||||
args = parser.parse_args()
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
os.umask(0o077)
|
||||
from scripts.benchmark.calibration import digest, pressure_preview, save_json
|
||||
from scripts.benchmark.pressure_thresholds import learn_pressure_thresholds
|
||||
from scripts.benchmark.simulation_design import BurstSimulationDesign
|
||||
|
||||
folder = args.calibration_dir.resolve()
|
||||
source = folder / "calibration.json"
|
||||
report = json.loads(source.read_text())
|
||||
if report.get("status") != "completed" or any(not 0 < value <= 100 for value in args.thresholds):
|
||||
raise ValueError("a completed calibration and thresholds in (0,100] are required")
|
||||
design = BurstSimulationDesign.model_validate(report["design"])
|
||||
pipes = {pipe.pipe_id: pipe for pipe in design.pipes}
|
||||
cases = {case.case_id: case for case in design.cases}
|
||||
index_by_time = {at: index for index, at in enumerate(design.event_times)}
|
||||
output = {"version": "burst-threshold-sensitivity@1", "calibration_sha256": digest(source),
|
||||
"scope": "pure cleaner/replay-rule sensitivity, not Agent or worker acceptance",
|
||||
"thresholds_percent": args.thresholds, "rows": [], "summary": [],
|
||||
"adaptive_fit": learn_pressure_thresholds([], ["calibration-history"], design.event_times[0]),
|
||||
"adaptive_note": "this package has four warmup samples; fitting needs 1440 pre-event normal samples per device",
|
||||
"absolute_drop_gate_applied": False}
|
||||
cache = {}
|
||||
controls = {}
|
||||
for entry in report["cases"]:
|
||||
case = cases[entry["case_id"]]
|
||||
pipe = pipes[case.pipe_id]
|
||||
index = index_by_time[case.occurred_at]
|
||||
if index not in cache:
|
||||
cache[index] = (json.loads((folder / f"normal-{index}.json").read_text()),
|
||||
json.loads((folder / f"warmup-{index}.json").read_text()))
|
||||
normal, warmup = cache[index]
|
||||
burst = json.loads((folder / f"{case.case_id}-{entry['severity']}.json").read_text())
|
||||
for threshold in args.thresholds:
|
||||
rule = SimpleNamespace(primary_pressure_drop_percent=threshold,
|
||||
consecutive_samples=design.selection_rule.consecutive_samples)
|
||||
key = (pipe.pipe_id, index, threshold)
|
||||
if key not in controls:
|
||||
controls[key] = pressure_preview(pipe, case.occurred_at, report["duration_seconds"], 900,
|
||||
warmup["nodes"], normal["nodes"], rule)
|
||||
event = pressure_preview(pipe, case.occurred_at, report["duration_seconds"], 900,
|
||||
warmup["nodes"], burst["nodes"], rule)
|
||||
at = event["first_signal_at"]
|
||||
output["rows"].append({"case_id": case.case_id, "pipe_id": pipe.pipe_id,
|
||||
"severity": entry["severity"], "threshold_percent": threshold,
|
||||
"event_signal": event["raised"], "first_signal_at": at,
|
||||
"delay_minutes": (datetime.fromisoformat(at) - case.occurred_at).total_seconds() / 60 if at else None,
|
||||
"normal_false_positive": controls[key]["raised"]})
|
||||
for threshold in args.thresholds:
|
||||
rows = [row for row in output["rows"] if row["threshold_percent"] == threshold]
|
||||
normal = [value for key, value in controls.items() if key[2] == threshold]
|
||||
output["summary"].append({"threshold_percent": threshold, "event_cases": len(rows),
|
||||
"event_signals": sum(row["event_signal"] for row in rows),
|
||||
"unique_normal_controls": len(normal),
|
||||
"normal_false_positives": sum(row["raised"] for row in normal)})
|
||||
path = folder / "threshold-sensitivity.json"
|
||||
save_json(path, output)
|
||||
for summary in output["summary"]:
|
||||
print(summary)
|
||||
print(f"threshold sensitivity complete: {path}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1 @@
|
||||
"""Offline benchmark preparation; never imported by the application runtime."""
|
||||
@@ -0,0 +1,226 @@
|
||||
"""File-isolated hydraulic calibration; database connections are read-only."""
|
||||
from __future__ import annotations
|
||||
|
||||
import ctypes
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from scripts.benchmark.simulation_design import PipeCalibration, SelectionRule
|
||||
|
||||
|
||||
def digest(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def save_json(path: Path, value: dict) -> None:
|
||||
temporary = path.with_suffix(".pending")
|
||||
with temporary.open("x", encoding="utf-8") as output:
|
||||
os.chmod(temporary, 0o600)
|
||||
json.dump(value, output, ensure_ascii=False, indent=2, allow_nan=False, default=str)
|
||||
output.write("\n")
|
||||
temporary.replace(path)
|
||||
|
||||
|
||||
def select_calibrated_area(
|
||||
pipe: PipeCalibration, passed_levels: list[str], case_results: list[dict], retain_weak: bool,
|
||||
) -> dict:
|
||||
selected = passed_levels[-1] if passed_levels else None
|
||||
soft_blockers = {"insufficient_consecutive_pressure_and_corroboration", "event_rule_did_not_trigger"}
|
||||
physically_valid = bool(case_results) and all(
|
||||
not set(case["blockers"]) - soft_blockers for case in case_results
|
||||
)
|
||||
status = "calibrated_candidate" if selected else (
|
||||
"weak_signal_candidate" if retain_weak and physically_valid else "unsuitable"
|
||||
)
|
||||
return {"status": status, "severity": selected,
|
||||
"area_cm2": pipe.candidate_orifice_areas_cm2[selected] if selected else None}
|
||||
|
||||
|
||||
def configure_inp(
|
||||
source: str,
|
||||
start: datetime,
|
||||
duration: int,
|
||||
*,
|
||||
emitter: tuple[str, float] | None = None,
|
||||
pda_options: dict[str, str] | None = None,
|
||||
) -> str:
|
||||
"""Apply the burst service's options to a file, including the zero-leak control."""
|
||||
if start.tzinfo is None:
|
||||
raise ValueError("simulation start must include a timezone")
|
||||
start = start.astimezone(ZoneInfo("Asia/Shanghai"))
|
||||
updates = {
|
||||
"TIMES": {"DURATION": f"{duration // 3600}:{duration % 3600 // 60:02d}:00",
|
||||
"PATTERN START": start.strftime("%H:%M:%S"),
|
||||
"REPORT START": "0:00:00", "STATISTIC": "NONE"},
|
||||
"OPTIONS": {**(pda_options or {}), "DEMAND MODEL": "PDA", "REQUIRED PRESSURE": "10.0000"},
|
||||
"REPORT": {"NODES": "NONE", "LINKS": "NONE"},
|
||||
}
|
||||
section = None
|
||||
output = []
|
||||
pending = {}
|
||||
for line in source.splitlines():
|
||||
match = re.fullmatch(r"\s*\[([^]]+)\]\s*", line)
|
||||
if match:
|
||||
output.extend(f"{key} {value}" for key, value in pending.items())
|
||||
section = match[1].upper()
|
||||
pending = dict(updates.get(section, {}))
|
||||
output.append(line)
|
||||
if section == "EMITTERS" and emitter:
|
||||
node, area = emitter
|
||||
coefficient = 0.65 * area / 10000 * math.sqrt(19.6) * 1000
|
||||
output.append(f"{node} {coefficient:.12g}")
|
||||
continue
|
||||
tokens = line.split(";", 1)[0].upper().split()
|
||||
if any(tokens[:len(key.split())] == key.split() for key in updates.get(section, {})):
|
||||
continue
|
||||
output.append(line)
|
||||
output.extend(f"{key} {value}" for key, value in pending.items())
|
||||
if "[EMITTERS]" not in source.upper() and emitter:
|
||||
raise ValueError("exported model has no emitter section")
|
||||
return "\n".join(output) + "\n"
|
||||
|
||||
|
||||
def run_file(inp: Path, node_ids: list[str], link_ids: list[str], *, timeout: int) -> dict:
|
||||
from app.infra.epanet.epanet import Output
|
||||
|
||||
executable = Path(__file__).resolve().parents[2] / "app/infra/epanet/linux/runepanet"
|
||||
env = os.environ.copy()
|
||||
env["LD_LIBRARY_PATH"] = f"{executable.parent}:{env.get('LD_LIBRARY_PATH', '')}"
|
||||
report_path, binary = inp.with_suffix(".rpt"), inp.with_suffix(".bin")
|
||||
result = subprocess.run(
|
||||
[str(executable), str(inp), str(report_path), str(binary)],
|
||||
env=env, capture_output=True, text=True, timeout=timeout, check=False,
|
||||
)
|
||||
report = report_path.read_text(encoding="utf-8", errors="replace") if report_path.exists() else ""
|
||||
warnings = [line.strip() for line in report.splitlines()
|
||||
if re.search(r"^\s*(?:warning\b|error\s+\d+\b)|system\s+(?:is\s+)?unbalanced|cannot solve hydraulic",
|
||||
line, re.IGNORECASE)]
|
||||
summary = {"exit_code": result.returncode, "warnings": warnings,
|
||||
"inp_sha256": digest(inp), "report_sha256": digest(report_path) if report_path.exists() else None}
|
||||
if result.returncode or not binary.exists():
|
||||
return {**summary, "status": "failed"}
|
||||
reader = Output(str(binary))
|
||||
names = reader.element_name()
|
||||
summary.update(status="completed", units=reader.units(), times=reader.times(), nodes={}, links={})
|
||||
for kind, selected, attributes, function in (
|
||||
("nodes", node_ids, ["demand", "head", "pressure", "quality"], reader._lib.ENR_getNodeResult),
|
||||
("links", link_ids, ["flow", "velocity", "headloss", "quality", "status", "setting", "reaction", "friction"],
|
||||
reader._lib.ENR_getLinkResult),
|
||||
):
|
||||
indices = {name: index + 1 for index, name in enumerate(names[kind])}
|
||||
for name in selected:
|
||||
index = indices.get(name)
|
||||
if index is None:
|
||||
raise ValueError("requested hydraulic output is missing")
|
||||
series = []
|
||||
for period in range(summary["times"]["num_periods"]):
|
||||
values = ctypes.POINTER(ctypes.c_float)()
|
||||
length = ctypes.c_int()
|
||||
reader._check(function(reader._handle, period, index, ctypes.byref(values), ctypes.byref(length)))
|
||||
try:
|
||||
if length.value != len(attributes):
|
||||
raise ValueError("unexpected hydraulic result attributes")
|
||||
row = {key: float(values[i]) for i, key in enumerate(attributes)}
|
||||
if not all(math.isfinite(value) for value in row.values()):
|
||||
raise ValueError("non-finite hydraulic result")
|
||||
series.append(row)
|
||||
finally:
|
||||
reader._lib.ENR_free(ctypes.byref(values))
|
||||
summary[kind][name] = series
|
||||
return summary
|
||||
|
||||
|
||||
def pressure_preview(
|
||||
pipe: PipeCalibration, start: datetime, duration: int, step: int,
|
||||
history: dict, current: dict, rule: SelectionRule,
|
||||
) -> dict:
|
||||
"""Exercise the actual replay cleaner/detector without database mutations."""
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.services.scada_replay_processing import detect_frame, prepare_cleaning
|
||||
|
||||
ids = [binding.device_id for binding in pipe.pressure_bindings]
|
||||
config = ScadaReplayCreate(
|
||||
request_id=uuid5(NAMESPACE_URL, f"calibration:{pipe.pipe_id}:{start.isoformat()}"),
|
||||
start_time=start, end_time=start + timedelta(seconds=duration + step), device_ids=ids,
|
||||
rules=[{"rule_id": "calibration-pressure", "device_ids": ids,
|
||||
"kind": "relative_change", "direction": "decrease", "unit": "m",
|
||||
"threshold": rule.primary_pressure_drop_percent,
|
||||
"window_samples": 4, "minimum_samples": 4,
|
||||
"consecutive_samples": rule.consecutive_samples, "minimum_devices": 1}],
|
||||
)
|
||||
document = {"replay_id": str(config.request_id), "devices": [
|
||||
{"device_id": key, "transmission_frequency": "0:15:00"} for key in ids],
|
||||
"histories": {binding.device_id: [row["pressure"] for row in history[binding.node_id]]
|
||||
for binding in pipe.pressure_bindings}}
|
||||
raised = []
|
||||
for index in range(duration // step + 1):
|
||||
at = start + timedelta(seconds=index * step)
|
||||
rows = [{"device_id": binding.device_id, "time": at,
|
||||
"monitored_value": current[binding.node_id][index]["pressure"],
|
||||
"source_hash": hashlib.sha256(f"{binding.device_id}:{at}".encode()).hexdigest()}
|
||||
for binding in pipe.pressure_bindings]
|
||||
document, changes = detect_frame(document, prepare_cleaning(rows, config), config, at)
|
||||
raised.extend(change for change in changes
|
||||
if change["change"] == "raised" and change.get("category") != "data_quality")
|
||||
return {"raised": bool(raised), "first_signal_at": raised[0]["occurred_at"] if raised else None,
|
||||
"scope": "pure replay rule preview; not monitoring worker or Agent acceptance"}
|
||||
|
||||
|
||||
def assess(
|
||||
pipe: PipeCalibration, normal: dict, burst: dict, rule: SelectionRule,
|
||||
*, duration: int, step: int,
|
||||
) -> dict:
|
||||
expected_times = {"report_start": 0, "report_step": step,
|
||||
"sim_duration": duration, "num_periods": duration // step + 1}
|
||||
blockers = []
|
||||
for label, result in (("normal", normal), ("burst", burst)):
|
||||
if result.get("status") != "completed" or result.get("times") != expected_times:
|
||||
blockers.append(f"{label}_incomplete_or_failed")
|
||||
if result.get("warnings"):
|
||||
blockers.append(f"{label}_solver_warning")
|
||||
if result.get("units") != {"flow": "LPS", "pressure": "MTR", "quality": "NONE"}:
|
||||
blockers.append(f"{label}_unexpected_units")
|
||||
if blockers:
|
||||
return {"passed": False, "blockers": blockers}
|
||||
pressures, flows = {}, {}
|
||||
for binding in pipe.pressure_bindings:
|
||||
before = [row["pressure"] for row in normal["nodes"][binding.node_id]]
|
||||
after = [row["pressure"] for row in burst["nodes"][binding.node_id]]
|
||||
if min(before) <= rule.relative_change_epsilon:
|
||||
blockers.append(f"nonpositive_normal_pressure:{binding.device_id}")
|
||||
if min(after) < 0:
|
||||
blockers.append(f"negative_monitor_pressure:{binding.device_id}")
|
||||
pressures[binding.device_id] = [100 * (a - b) / max(abs(a), rule.relative_change_epsilon)
|
||||
for a, b in zip(before, after)]
|
||||
for binding in pipe.flow_bindings:
|
||||
flows[binding.device_id] = [100 * abs(a["flow"] - b["flow"]) /
|
||||
max(abs(a["flow"]), rule.relative_change_epsilon)
|
||||
for a, b in zip(normal["links"][binding.link_id], burst["links"][binding.link_id])]
|
||||
witnesses = []
|
||||
count = duration // step + 1
|
||||
for primary, drops in pressures.items():
|
||||
for start in range(count - rule.consecutive_samples + 1):
|
||||
window = range(start, start + rule.consecutive_samples)
|
||||
if not all(drops[i] >= rule.primary_pressure_drop_percent for i in window):
|
||||
continue
|
||||
second = [key for key, values in pressures.items() if key != primary and
|
||||
all(values[i] >= rule.corroborating_pressure_drop_percent for i in window)]
|
||||
flow = [key for key, values in flows.items() if
|
||||
all(values[i] >= rule.flow_change_percent for i in window) and
|
||||
all(abs(normal["links"][next(b.link_id for b in pipe.flow_bindings if b.device_id == key)][i]["flow"])
|
||||
> rule.relative_change_epsilon for i in window)]
|
||||
if second or flow:
|
||||
witnesses.append({"primary_device": primary, "second_pressure_devices": second,
|
||||
"flow_devices": flow, "start_offset_seconds": start * step})
|
||||
if not witnesses:
|
||||
blockers.append("insufficient_consecutive_pressure_and_corroboration")
|
||||
return {"passed": not blockers, "blockers": blockers, "witnesses": witnesses,
|
||||
"pressure_drop_percent": pressures, "flow_change_percent": flows}
|
||||
@@ -0,0 +1,145 @@
|
||||
"""Versioned burst cases and deterministic preparation checks."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from collections import defaultdict, deque
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
|
||||
class StrictModel(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid", allow_inf_nan=False)
|
||||
|
||||
|
||||
class BurstCase(StrictModel):
|
||||
case_id: str = Field(pattern=r"^B[0-9]{2}$")
|
||||
pipe_id: str = Field(pattern=r"^[0-9]+$")
|
||||
occurred_at: datetime
|
||||
reference_analysis: str = Field(min_length=1, max_length=200)
|
||||
observation_provenance: Literal["declared_historical_replay", "requires_generation"]
|
||||
provenance_note: str = Field(min_length=1)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def aware_time(self):
|
||||
if self.occurred_at.tzinfo is None:
|
||||
raise ValueError("case time must include a timezone")
|
||||
return self
|
||||
|
||||
|
||||
class TriggerRule(StrictModel):
|
||||
pressure_drop_percent: float = Field(gt=0, le=100)
|
||||
baseline_samples: int = Field(ge=3, le=96)
|
||||
consecutive_samples: int = Field(ge=1, le=20)
|
||||
minimum_devices: int = Field(ge=1, le=500)
|
||||
|
||||
|
||||
class BurstSuite(StrictModel):
|
||||
version: Literal["burst-benchmark@1"]
|
||||
suite_id: Literal["tjwater-v2-burst"]
|
||||
project_code: Literal["tjwater_v2"]
|
||||
project_id: UUID
|
||||
sampling_interval_minutes: int = Field(ge=1, le=60)
|
||||
event_minutes: int = Field(ge=15, le=1440)
|
||||
detector_days: Literal[15]
|
||||
trigger_rule: TriggerRule
|
||||
score_weights: dict[str, int]
|
||||
cases: list[BurstCase] = Field(min_length=1, max_length=100)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def valid_suite(self):
|
||||
if 1440 % self.sampling_interval_minutes or self.event_minutes % self.sampling_interval_minutes:
|
||||
raise ValueError("sample interval must divide a day and the event window")
|
||||
if len({case.case_id for case in self.cases}) != len(self.cases):
|
||||
raise ValueError("case ids must be unique")
|
||||
if set(self.score_weights) != {"outcome", "tools", "plan", "efficiency"}:
|
||||
raise ValueError("all four score dimensions are required")
|
||||
if any(value <= 0 for value in self.score_weights.values()) or sum(self.score_weights.values()) != 100:
|
||||
raise ValueError("positive score weights must sum to 100")
|
||||
for case in self.cases:
|
||||
if case.occurred_at.timestamp() % (60 * self.sampling_interval_minutes):
|
||||
raise ValueError("event times must align with the sampling grid")
|
||||
return self
|
||||
|
||||
def select(self, case_ids: list[str]) -> list[BurstCase]:
|
||||
if not case_ids:
|
||||
return self.cases
|
||||
unknown = set(case_ids) - {case.case_id for case in self.cases}
|
||||
if unknown:
|
||||
raise ValueError("unknown case ids: " + ", ".join(sorted(unknown)))
|
||||
return [case for case in self.cases if case.case_id in case_ids]
|
||||
|
||||
|
||||
def load_suite(path: Path) -> BurstSuite:
|
||||
return BurstSuite.model_validate_json(path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def canonical_json(value) -> str:
|
||||
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"),
|
||||
allow_nan=False, default=str)
|
||||
|
||||
|
||||
def fingerprint(value) -> str:
|
||||
return hashlib.sha256(canonical_json(value).encode()).hexdigest()
|
||||
|
||||
|
||||
def complete_devices(rows: list[dict], start: datetime, end: datetime, interval: int) -> list[str]:
|
||||
"""Counts alone are insufficient: require finite values on the exact grid."""
|
||||
step = timedelta(minutes=interval)
|
||||
expected = {start + step * index for index in range(int((end - start) / step) + 1)}
|
||||
values: dict[str, dict[datetime, float]] = defaultdict(dict)
|
||||
invalid: set[str] = set()
|
||||
for row in rows:
|
||||
device = row["device_id"]
|
||||
at = row["time"]
|
||||
if not start <= at <= end:
|
||||
continue
|
||||
value = row["monitored_value"]
|
||||
if at not in expected or at in values[device] or value is None or not math.isfinite(float(value)):
|
||||
invalid.add(device)
|
||||
else:
|
||||
values[device][at] = float(value)
|
||||
return sorted(device for device, series in values.items()
|
||||
if device not in invalid and set(series) == expected)
|
||||
|
||||
|
||||
def isolation_reference(links: list[dict], source_nodes: list[str], pipe_id: str) -> dict:
|
||||
"""Independent graph check, separate from the production isolation algorithm."""
|
||||
graph: dict[str, list[str]] = defaultdict(list)
|
||||
target = None
|
||||
valves = []
|
||||
for link in links:
|
||||
if link["id"] == pipe_id:
|
||||
target = (link["start_node_id"], link["end_node_id"])
|
||||
if link["kind"].lower() == "valve":
|
||||
valves.append(link["id"])
|
||||
continue
|
||||
if str(link["status"]).lower() == "closed":
|
||||
continue
|
||||
left, right = link["start_node_id"], link["end_node_id"]
|
||||
graph[left].append(right)
|
||||
graph[right].append(left)
|
||||
if target is None:
|
||||
return {"status": "target_missing"}
|
||||
if not source_nodes:
|
||||
return {"status": "sources_missing"}
|
||||
reached = set(source_nodes)
|
||||
pending = deque(source_nodes)
|
||||
while pending:
|
||||
for node in graph[pending.popleft()]:
|
||||
if node not in reached:
|
||||
reached.add(node)
|
||||
pending.append(node)
|
||||
connected = any(node in reached for node in target)
|
||||
return {
|
||||
"status": "not_isolatable" if connected else "requires_boundary_verification",
|
||||
"model_valve_count": len(valves),
|
||||
"source_count": len(source_nodes),
|
||||
"source_reachable_after_closing_all_model_valves": connected,
|
||||
"scope": "undirected model topology; not hydraulic or field acceptance",
|
||||
}
|
||||
@@ -0,0 +1,576 @@
|
||||
"""Isolated database preparation and deterministic burst-dataset scheduling."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from uuid import UUID
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from psycopg import connect, sql
|
||||
from psycopg.conninfo import make_conninfo
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
|
||||
def canonical_hash(value) -> str:
|
||||
return hashlib.sha256(json.dumps(value, sort_keys=True, default=str, separators=(",", ":")).encode()).hexdigest()
|
||||
|
||||
|
||||
def deterministic_standard_normal(*, seed: str, at: datetime, device_id: str, metric: str) -> float:
|
||||
"""Return order-independent deterministic Gaussian noise for one observation."""
|
||||
material = f"{seed}\0{at.isoformat()}\0{device_id}\0{metric}".encode()
|
||||
digest = hashlib.sha256(material).digest()
|
||||
first = (int.from_bytes(digest[:8], "big") + 1) / (2**64 + 1)
|
||||
second = (int.from_bytes(digest[8:16], "big") + 1) / (2**64 + 1)
|
||||
return math.sqrt(-2.0 * math.log(first)) * math.cos(2.0 * math.pi * second)
|
||||
|
||||
|
||||
def noisy_scada_observation(
|
||||
*,
|
||||
raw_value: float,
|
||||
device_type: str,
|
||||
at: datetime,
|
||||
device_id: str,
|
||||
seed: str,
|
||||
pressure_sigma_percent: float = 0.2,
|
||||
pressure_sigma_floor_m: float = 0.02,
|
||||
flow_sigma_percent: float = 0.5,
|
||||
clip_sigma: float = 3.0,
|
||||
) -> float:
|
||||
"""Convert model units and add reproducible zero-mean telemetry noise."""
|
||||
if not math.isfinite(raw_value):
|
||||
raise ValueError("model observation must be finite")
|
||||
if min(pressure_sigma_percent, pressure_sigma_floor_m, flow_sigma_percent, clip_sigma) < 0:
|
||||
raise ValueError("noise parameters must be non-negative")
|
||||
z = max(-clip_sigma, min(clip_sigma, deterministic_standard_normal(
|
||||
seed=seed, at=at, device_id=device_id, metric=device_type,
|
||||
)))
|
||||
if device_type == "pressure":
|
||||
converted = raw_value
|
||||
sigma = max(abs(converted) * pressure_sigma_percent / 100.0, pressure_sigma_floor_m)
|
||||
return max(0.0, converted + z * sigma)
|
||||
if device_type == "pipe_flow":
|
||||
converted = raw_value * 3.6 # L/s -> m3/h
|
||||
sigma = abs(converted) * flow_sigma_percent / 100.0
|
||||
return converted + z * sigma
|
||||
raise ValueError(f"unsupported SCADA device type: {device_type}")
|
||||
|
||||
|
||||
def write_realtime_scada_day(
|
||||
*,
|
||||
business_dsn: str,
|
||||
timescale_dsn: str,
|
||||
start: datetime,
|
||||
interval_seconds: int = 900,
|
||||
duration_seconds: int = 86400,
|
||||
noise_seed: str,
|
||||
pressure_sigma_percent: float = 0.2,
|
||||
pressure_sigma_floor_m: float = 0.02,
|
||||
flow_sigma_percent: float = 0.5,
|
||||
clip_sigma: float = 3.0,
|
||||
) -> dict:
|
||||
"""Map one online-model day into noisy SCADA observations."""
|
||||
if start.tzinfo is None or duration_seconds <= 0 or interval_seconds <= 0:
|
||||
raise ValueError("an aware start and positive duration/interval are required")
|
||||
if duration_seconds % interval_seconds:
|
||||
raise ValueError("duration must be divisible by the sampling interval")
|
||||
expected_times = [
|
||||
start + timedelta(seconds=offset)
|
||||
for offset in range(0, duration_seconds, interval_seconds)
|
||||
]
|
||||
end = start + timedelta(seconds=duration_seconds)
|
||||
with connect(business_dsn, row_factory=dict_row) as business:
|
||||
devices = business.execute(
|
||||
"""SELECT device_id,device_type,node_id,link_id,measurement_unit
|
||||
FROM asset.scada_devices
|
||||
WHERE device_type IN ('pressure','pipe_flow')
|
||||
ORDER BY device_id"""
|
||||
).fetchall()
|
||||
if not devices:
|
||||
raise ValueError("no pressure or pipe-flow SCADA bindings are available")
|
||||
records: list[tuple] = []
|
||||
raw_fingerprint_rows: list[tuple] = []
|
||||
with connect(timescale_dsn, row_factory=dict_row) as timescale:
|
||||
for device in devices:
|
||||
if device["device_type"] == "pressure":
|
||||
if not device["node_id"] or device["measurement_unit"] != "m":
|
||||
raise ValueError("pressure devices must bind nodes and use metres")
|
||||
rows = timescale.execute(
|
||||
"""SELECT time,pressure AS value FROM realtime.node_results
|
||||
WHERE node_id=%s AND time>=%s AND time<%s ORDER BY time""",
|
||||
(device["node_id"], start, end),
|
||||
).fetchall()
|
||||
else:
|
||||
if not device["link_id"] or device["measurement_unit"] != "m3/h":
|
||||
raise ValueError("flow devices must bind links and use m3/h")
|
||||
rows = timescale.execute(
|
||||
"""SELECT time,flow AS value FROM realtime.link_results
|
||||
WHERE link_id=%s AND time>=%s AND time<%s ORDER BY time""",
|
||||
(device["link_id"], start, end),
|
||||
).fetchall()
|
||||
actual_times = [row["time"] for row in rows]
|
||||
if actual_times != expected_times:
|
||||
raise ValueError(f"incomplete realtime samples for device {device['device_id']}")
|
||||
for row in rows:
|
||||
raw_value = float(row["value"])
|
||||
value = noisy_scada_observation(
|
||||
raw_value=raw_value,
|
||||
device_type=device["device_type"],
|
||||
at=row["time"],
|
||||
device_id=device["device_id"],
|
||||
seed=noise_seed,
|
||||
pressure_sigma_percent=pressure_sigma_percent,
|
||||
pressure_sigma_floor_m=pressure_sigma_floor_m,
|
||||
flow_sigma_percent=flow_sigma_percent,
|
||||
clip_sigma=clip_sigma,
|
||||
)
|
||||
raw_fingerprint_rows.append((row["time"].isoformat(), device["device_id"], raw_value))
|
||||
records.append((row["time"], device["device_id"], value, None))
|
||||
collision = timescale.execute(
|
||||
"SELECT count(*) count FROM scada.measurements WHERE time>=%s AND time<%s",
|
||||
(start, end),
|
||||
).fetchone()["count"]
|
||||
if collision:
|
||||
raise ValueError("SCADA target day is not empty")
|
||||
with timescale.cursor().copy(
|
||||
"COPY scada.measurements (time,device_id,monitored_value,cleaned_value) FROM STDIN"
|
||||
) as copy:
|
||||
for record in records:
|
||||
copy.write_row(record)
|
||||
timescale.commit()
|
||||
persisted = timescale.execute(
|
||||
"SELECT count(*) count FROM scada.measurements WHERE time>=%s AND time<%s",
|
||||
(start, end),
|
||||
).fetchone()["count"]
|
||||
if persisted != len(records):
|
||||
raise ValueError("SCADA normal-day read-back count does not match generated records")
|
||||
return {
|
||||
"start": start.isoformat(),
|
||||
"end_exclusive": end.isoformat(),
|
||||
"devices": len(devices),
|
||||
"samples_per_device": len(expected_times),
|
||||
"rows": persisted,
|
||||
"units": {"pressure": "m", "pipe_flow": "m3/h", "model_flow": "L/s"},
|
||||
"noise": {
|
||||
"seed": noise_seed,
|
||||
"distribution": "deterministic_clipped_gaussian",
|
||||
"pressure_sigma_percent": pressure_sigma_percent,
|
||||
"pressure_sigma_floor_m": pressure_sigma_floor_m,
|
||||
"flow_sigma_percent": flow_sigma_percent,
|
||||
"clip_sigma": clip_sigma,
|
||||
},
|
||||
"raw_sha256": canonical_hash(raw_fingerprint_rows),
|
||||
"value_sha256": canonical_hash(
|
||||
[(time.isoformat(), device_id, value) for time, device_id, value, _ in records]
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def write_demo_scada_window(
|
||||
*,
|
||||
business_dsn: str,
|
||||
timescale_dsn: str,
|
||||
normal_source_start: datetime,
|
||||
target_start: datetime,
|
||||
day_count: int,
|
||||
event_start: datetime,
|
||||
event_run_id: str,
|
||||
noise_seed: str,
|
||||
interval_seconds: int = 900,
|
||||
) -> dict:
|
||||
"""Repeat one normal online day and replace one five-hour event window."""
|
||||
if any(value.tzinfo is None for value in (normal_source_start, target_start, event_start)):
|
||||
raise ValueError("normal, target, and event times must be timezone-aware")
|
||||
if day_count < 15 or interval_seconds <= 0 or 86400 % interval_seconds:
|
||||
raise ValueError("demo windows require at least 15 complete aligned days")
|
||||
target_end = target_start + timedelta(days=day_count)
|
||||
if not target_start <= event_start < target_end:
|
||||
raise ValueError("event must fall inside the demo window")
|
||||
run_uuid = UUID(str(event_run_id))
|
||||
with connect(business_dsn, row_factory=dict_row) as business:
|
||||
devices = business.execute(
|
||||
"""SELECT device_id,device_type,node_id,link_id,measurement_unit
|
||||
FROM asset.scada_devices
|
||||
WHERE device_type IN ('pressure','pipe_flow') ORDER BY device_id"""
|
||||
).fetchall()
|
||||
pressure_devices = [row for row in devices if row["device_type"] == "pressure"]
|
||||
flow_devices = [row for row in devices if row["device_type"] == "pipe_flow"]
|
||||
if any(not row["node_id"] or row["measurement_unit"] != "m" for row in pressure_devices):
|
||||
raise ValueError("pressure devices must bind nodes and use metres")
|
||||
if any(not row["link_id"] or row["measurement_unit"] != "m3/h" for row in flow_devices):
|
||||
raise ValueError("flow devices must bind links and use m3/h")
|
||||
normal_end = normal_source_start + timedelta(days=1)
|
||||
expected_normal_times = [
|
||||
normal_source_start + timedelta(seconds=offset)
|
||||
for offset in range(0, 86400, interval_seconds)
|
||||
]
|
||||
with connect(timescale_dsn, row_factory=dict_row) as timescale:
|
||||
normal_nodes = timescale.execute(
|
||||
"""SELECT time,node_id,pressure AS value FROM realtime.node_results
|
||||
WHERE time>=%s AND time<%s AND node_id=ANY(%s) ORDER BY time,node_id""",
|
||||
(normal_source_start, normal_end, [row["node_id"] for row in pressure_devices]),
|
||||
).fetchall()
|
||||
normal_links = timescale.execute(
|
||||
"""SELECT time,link_id,flow AS value FROM realtime.link_results
|
||||
WHERE time>=%s AND time<%s AND link_id=ANY(%s) ORDER BY time,link_id""",
|
||||
(normal_source_start, normal_end, [row["link_id"] for row in flow_devices]),
|
||||
).fetchall()
|
||||
event_nodes = timescale.execute(
|
||||
"""SELECT time,node_id,pressure AS value FROM analysis.node_results
|
||||
WHERE run_id=%s AND node_id=ANY(%s) ORDER BY time,node_id""",
|
||||
(run_uuid, [row["node_id"] for row in pressure_devices]),
|
||||
).fetchall()
|
||||
event_links = timescale.execute(
|
||||
"""SELECT time,link_id,flow AS value FROM analysis.link_results
|
||||
WHERE run_id=%s AND link_id=ANY(%s) ORDER BY time,link_id""",
|
||||
(run_uuid, [row["link_id"] for row in flow_devices]),
|
||||
).fetchall()
|
||||
|
||||
normal_node_values = {(row["time"], row["node_id"]): float(row["value"]) for row in normal_nodes}
|
||||
normal_link_values = {(row["time"], row["link_id"]): float(row["value"]) for row in normal_links}
|
||||
for device in devices:
|
||||
values = normal_node_values if device["device_type"] == "pressure" else normal_link_values
|
||||
element = device["node_id"] if device["device_type"] == "pressure" else device["link_id"]
|
||||
if any((at, element) not in values for at in expected_normal_times):
|
||||
raise ValueError(f"normal source day is incomplete for device {device['device_id']}")
|
||||
|
||||
records: dict[tuple[datetime, str], tuple] = {}
|
||||
for day_offset in range(day_count):
|
||||
day_start = target_start + timedelta(days=day_offset)
|
||||
for sample_offset, source_at in enumerate(expected_normal_times):
|
||||
target_at = day_start + timedelta(seconds=sample_offset * interval_seconds)
|
||||
for device in pressure_devices:
|
||||
raw = normal_node_values[(source_at, device["node_id"])]
|
||||
value = noisy_scada_observation(
|
||||
raw_value=raw, device_type="pressure", at=target_at,
|
||||
device_id=device["device_id"], seed=noise_seed,
|
||||
)
|
||||
records[(target_at, device["device_id"])] = (
|
||||
target_at, device["device_id"], value, None,
|
||||
)
|
||||
for device in flow_devices:
|
||||
raw = normal_link_values[(source_at, device["link_id"])]
|
||||
value = noisy_scada_observation(
|
||||
raw_value=raw, device_type="pipe_flow", at=target_at,
|
||||
device_id=device["device_id"], seed=noise_seed,
|
||||
)
|
||||
records[(target_at, device["device_id"])] = (
|
||||
target_at, device["device_id"], value, None,
|
||||
)
|
||||
|
||||
if not event_nodes or not event_links:
|
||||
raise ValueError("event analysis has no bound pressure or flow results")
|
||||
source_event_start = min(row["time"] for row in event_nodes + event_links)
|
||||
event_times = sorted({row["time"] for row in event_nodes + event_links})
|
||||
if len(event_times) != 21 or any(
|
||||
right - left != timedelta(seconds=interval_seconds)
|
||||
for left, right in zip(event_times, event_times[1:])
|
||||
):
|
||||
raise ValueError("event analysis must contain 21 aligned fifteen-minute frames")
|
||||
event_node_values = {(row["time"], row["node_id"]): float(row["value"]) for row in event_nodes}
|
||||
event_link_values = {(row["time"], row["link_id"]): float(row["value"]) for row in event_links}
|
||||
for source_at in event_times:
|
||||
target_at = event_start + (source_at - source_event_start)
|
||||
if target_at >= target_end:
|
||||
raise ValueError("event results extend beyond the demo window")
|
||||
for device in pressure_devices:
|
||||
key = (source_at, device["node_id"])
|
||||
if key not in event_node_values:
|
||||
raise ValueError(f"event pressure is incomplete for device {device['device_id']}")
|
||||
value = noisy_scada_observation(
|
||||
raw_value=event_node_values[key], device_type="pressure", at=target_at,
|
||||
device_id=device["device_id"], seed=noise_seed,
|
||||
)
|
||||
records[(target_at, device["device_id"])] = (
|
||||
target_at, device["device_id"], value, None,
|
||||
)
|
||||
for device in flow_devices:
|
||||
key = (source_at, device["link_id"])
|
||||
if key not in event_link_values:
|
||||
raise ValueError(f"event flow is incomplete for device {device['device_id']}")
|
||||
value = noisy_scada_observation(
|
||||
raw_value=event_link_values[key], device_type="pipe_flow", at=target_at,
|
||||
device_id=device["device_id"], seed=noise_seed,
|
||||
)
|
||||
records[(target_at, device["device_id"])] = (
|
||||
target_at, device["device_id"], value, None,
|
||||
)
|
||||
|
||||
ordered = [records[key] for key in sorted(records)]
|
||||
expected_count = day_count * len(expected_normal_times) * len(devices)
|
||||
if len(ordered) != expected_count:
|
||||
raise ValueError("demo SCADA window has an unexpected observation count")
|
||||
timescale.execute(
|
||||
"DELETE FROM scada.measurements WHERE time>=%s AND time<%s",
|
||||
(target_start, target_end),
|
||||
)
|
||||
with timescale.cursor().copy(
|
||||
"COPY scada.measurements (time,device_id,monitored_value,cleaned_value) FROM STDIN"
|
||||
) as copy:
|
||||
for record in ordered:
|
||||
copy.write_row(record)
|
||||
timescale.commit()
|
||||
persisted = timescale.execute(
|
||||
"""SELECT count(*) count,count(DISTINCT device_id) devices,
|
||||
count(DISTINCT time) frames,count(cleaned_value) cleaned
|
||||
FROM scada.measurements WHERE time>=%s AND time<%s""",
|
||||
(target_start, target_end),
|
||||
).fetchone()
|
||||
if persisted != {
|
||||
"count": expected_count,
|
||||
"devices": len(devices),
|
||||
"frames": day_count * len(expected_normal_times),
|
||||
"cleaned": 0,
|
||||
}:
|
||||
raise ValueError("demo SCADA read-back verification failed")
|
||||
return {
|
||||
"start": target_start.isoformat(),
|
||||
"end_exclusive": target_end.isoformat(),
|
||||
"day_count": day_count,
|
||||
"event_start": event_start.isoformat(),
|
||||
"event_end": (event_start + timedelta(seconds=20 * interval_seconds)).isoformat(),
|
||||
"event_run_id": str(run_uuid),
|
||||
"devices": len(devices),
|
||||
"frames": day_count * len(expected_normal_times),
|
||||
"rows": expected_count,
|
||||
"cleaned_rows": 0,
|
||||
"noise_seed": noise_seed,
|
||||
"value_sha256": canonical_hash(
|
||||
[(time.isoformat(), device_id, value) for time, device_id, value, _ in ordered]
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def create_empty_database(source_dsn: str, target: str, schema_template: str) -> None:
|
||||
if not target.startswith("tjw_bench_") or target.endswith("_template"):
|
||||
raise ValueError("benchmark targets must use the owned tjw_bench_ prefix")
|
||||
with connect(make_conninfo(source_dsn, dbname="postgres"), autocommit=True, row_factory=dict_row) as connection:
|
||||
if connection.execute("SELECT 1 FROM pg_database WHERE datname=%s", (target,)).fetchone():
|
||||
raise ValueError("target database already exists; explicit state is required for resumption")
|
||||
# Templates are intentionally connection-disabled; clone without altering that state.
|
||||
connection.execute(sql.SQL("CREATE DATABASE {} TEMPLATE {}").format(sql.Identifier(target), sql.Identifier(schema_template)))
|
||||
|
||||
|
||||
def copy_project_inputs(source_dsn: str, target_dsn: str) -> dict:
|
||||
from scripts.check_burst_simulation_design import readonly
|
||||
from app.native.wndb.core.model_replace import _copy_order, _copy_table, _model_tables, _table_columns
|
||||
|
||||
summaries = {}
|
||||
with readonly(source_dsn) as source, connect(target_dsn, row_factory=dict_row) as target:
|
||||
source.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
|
||||
tables = _model_tables(source)
|
||||
ordered = _copy_order(source, tables)
|
||||
for schema, table in reversed(ordered):
|
||||
target.execute(sql.SQL("DELETE FROM {}").format(sql.Identifier(schema, table)))
|
||||
for schema, table in ordered + [("asset", "scada_devices")]:
|
||||
columns = _table_columns(source, schema, table)
|
||||
if _table_columns(target, schema, table) != columns:
|
||||
raise ValueError(f"schema differs for {schema}.{table}")
|
||||
_copy_table(source, target, schema, table, columns)
|
||||
statement = sql.SQL("SELECT count(*) count FROM {}").format(sql.Identifier(schema, table))
|
||||
before, after = source.execute(statement).fetchone()["count"], target.execute(statement).fetchone()["count"]
|
||||
if before != after:
|
||||
raise ValueError("isolated model copy count mismatch")
|
||||
summaries[f"{schema}.{table}"] = before
|
||||
target.commit()
|
||||
return summaries
|
||||
|
||||
|
||||
def apply_project_runtime_migrations(target_dsn: str, repository_root: Path) -> list[str]:
|
||||
"""Install the additive MCP and calculation queue tables in an owned target."""
|
||||
migrations = (
|
||||
"025_mcp_query_contexts.sql",
|
||||
"026_calculation_jobs.sql",
|
||||
"027_calculation_job_lifecycle.sql",
|
||||
"030_remove_calculation_queue_expiry.sql",
|
||||
)
|
||||
applied = []
|
||||
with connect(target_dsn) as connection:
|
||||
for filename in migrations:
|
||||
path = repository_root / "resources" / "sql" / filename
|
||||
connection.execute(path.read_text(encoding="utf-8"))
|
||||
applied.append(filename)
|
||||
connection.commit()
|
||||
return applied
|
||||
|
||||
|
||||
async def register_benchmark_project(
|
||||
*,
|
||||
source_project_id: str,
|
||||
target: str,
|
||||
target_biz_dsn: str,
|
||||
target_iot_dsn: str,
|
||||
):
|
||||
"""Register owned databases and mirror source membership without role expansion."""
|
||||
from uuid import UUID
|
||||
|
||||
from app.infra.db.metadb.database import SessionLocal
|
||||
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
|
||||
|
||||
source_id = UUID(str(source_project_id))
|
||||
async with SessionLocal() as session:
|
||||
repository = MetadataRepository(session)
|
||||
source = await repository.get_project_by_id(source_id)
|
||||
if source is None:
|
||||
raise ValueError("source project metadata is unavailable")
|
||||
project = await repository.get_project_by_code(target)
|
||||
marker = {
|
||||
"benchmark_source_project_id": str(source_id),
|
||||
"benchmark_dataset": "burst-v1",
|
||||
}
|
||||
if project is None:
|
||||
project = await repository.create_project(
|
||||
name=f"{source.name} 爆管 Benchmark",
|
||||
code=target,
|
||||
description="隔离的爆管 benchmark 模拟与 SCADA 回放数据集",
|
||||
gs_workspace=target,
|
||||
map_extent=source.map_extent,
|
||||
status="active",
|
||||
map_config=source.map_config or {},
|
||||
settings={**(source.settings or {}), **marker},
|
||||
)
|
||||
elif any((project.settings or {}).get(key) != value for key, value in marker.items()):
|
||||
raise ValueError("existing target project is not owned by this benchmark dataset")
|
||||
await repository.upsert_project_database_config(
|
||||
project.id,
|
||||
db_role="biz_data",
|
||||
db_type="postgresql",
|
||||
dsn=target_biz_dsn,
|
||||
pool_min_size=1,
|
||||
pool_max_size=4,
|
||||
)
|
||||
await repository.upsert_project_database_config(
|
||||
project.id,
|
||||
db_role="iot_data",
|
||||
db_type="timescaledb",
|
||||
dsn=target_iot_dsn,
|
||||
pool_min_size=1,
|
||||
pool_max_size=4,
|
||||
)
|
||||
existing = {member.user_id for member in await repository.list_project_members(project.id)}
|
||||
for member in await repository.list_project_members(source_id):
|
||||
if member.user_id not in existing:
|
||||
await repository.add_project_member(
|
||||
project.id, member.user_id, member.project_role
|
||||
)
|
||||
return project
|
||||
|
||||
|
||||
def write_analysis_scada(
|
||||
*,
|
||||
business_dsn: str,
|
||||
timescale_dsn: str,
|
||||
run_id: str,
|
||||
) -> dict:
|
||||
"""Materialize one completed hydraulic run as bound SCADA observations."""
|
||||
run_uuid = UUID(str(run_id))
|
||||
with connect(business_dsn, row_factory=dict_row) as business:
|
||||
devices = business.execute(
|
||||
"""SELECT device_id,device_type,node_id,link_id,measurement_unit
|
||||
FROM asset.scada_devices
|
||||
WHERE device_type IN ('pressure','pipe_flow')
|
||||
ORDER BY device_id"""
|
||||
).fetchall()
|
||||
pressure = [row for row in devices if row["device_type"] == "pressure"]
|
||||
flows = [row for row in devices if row["device_type"] == "pipe_flow"]
|
||||
if any(not row["node_id"] or row["measurement_unit"] != "m" for row in pressure):
|
||||
raise ValueError("pressure devices must bind nodes and use metres")
|
||||
if any(not row["link_id"] or row["measurement_unit"] != "m3/h" for row in flows):
|
||||
raise ValueError("flow devices must bind links and use m3/h")
|
||||
|
||||
pressure_by_node: dict[str, list[str]] = {}
|
||||
flow_by_link: dict[str, list[str]] = {}
|
||||
for row in pressure:
|
||||
pressure_by_node.setdefault(row["node_id"], []).append(row["device_id"])
|
||||
for row in flows:
|
||||
flow_by_link.setdefault(row["link_id"], []).append(row["device_id"])
|
||||
|
||||
records: list[tuple] = []
|
||||
with connect(timescale_dsn, row_factory=dict_row) as timescale:
|
||||
node_rows = timescale.execute(
|
||||
"""SELECT time,node_id,pressure FROM analysis.node_results
|
||||
WHERE run_id=%s AND node_id=ANY(%s) ORDER BY time,node_id""",
|
||||
(run_uuid, list(pressure_by_node)),
|
||||
).fetchall()
|
||||
link_rows = timescale.execute(
|
||||
"""SELECT time,link_id,flow FROM analysis.link_results
|
||||
WHERE run_id=%s AND link_id=ANY(%s) ORDER BY time,link_id""",
|
||||
(run_uuid, list(flow_by_link)),
|
||||
).fetchall()
|
||||
for row in node_rows:
|
||||
records.extend(
|
||||
(row["time"], device_id, float(row["pressure"]), None)
|
||||
for device_id in pressure_by_node[row["node_id"]]
|
||||
)
|
||||
for row in link_rows:
|
||||
records.extend(
|
||||
(row["time"], device_id, float(row["flow"]) * 3.6, None)
|
||||
for device_id in flow_by_link[row["link_id"]]
|
||||
)
|
||||
if not records:
|
||||
raise ValueError("analysis run has no bound SCADA results")
|
||||
start = min(row[0] for row in records)
|
||||
end = max(row[0] for row in records)
|
||||
collision = timescale.execute(
|
||||
"SELECT count(*) count FROM scada.measurements WHERE time BETWEEN %s AND %s",
|
||||
(start, end),
|
||||
).fetchone()["count"]
|
||||
if collision:
|
||||
raise ValueError("SCADA target interval is not empty")
|
||||
with timescale.cursor().copy(
|
||||
"COPY scada.measurements (time,device_id,monitored_value,cleaned_value) FROM STDIN"
|
||||
) as copy:
|
||||
for record in sorted(records):
|
||||
copy.write_row(record)
|
||||
timescale.commit()
|
||||
persisted = timescale.execute(
|
||||
"SELECT count(*) count FROM scada.measurements WHERE time BETWEEN %s AND %s",
|
||||
(start, end),
|
||||
).fetchone()["count"]
|
||||
if persisted != len(records):
|
||||
raise ValueError("SCADA read-back count does not match generated records")
|
||||
samples_per_device: dict[str, int] = {}
|
||||
for _, device_id, _, _ in records:
|
||||
samples_per_device[device_id] = samples_per_device.get(device_id, 0) + 1
|
||||
if len(set(samples_per_device.values())) != 1:
|
||||
raise ValueError("bound SCADA devices have inconsistent sample counts")
|
||||
return {
|
||||
"run_id": str(run_uuid),
|
||||
"start": start.isoformat(),
|
||||
"end": end.isoformat(),
|
||||
"devices": len(samples_per_device),
|
||||
"samples_per_device": next(iter(samples_per_device.values())),
|
||||
"rows": persisted,
|
||||
"value_sha256": canonical_hash(
|
||||
[(time.isoformat(), device_id, value) for time, device_id, value, _ in sorted(records)]
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def make_schedule(design) -> list[dict]:
|
||||
"""Seventeen-day slots keep fifteen clean training days plus a recovery buffer."""
|
||||
local = ZoneInfo("Asia/Shanghai")
|
||||
first_date = datetime(2024, 1, 16, tzinfo=local)
|
||||
rows = []
|
||||
pipes = {pipe.pipe_id: pipe for pipe in design.pipes}
|
||||
for case in design.cases:
|
||||
for level in ("light", "medium", "severe"):
|
||||
source = case.occurred_at.astimezone(local)
|
||||
at = first_date + timedelta(days=17 * len(rows), hours=source.hour, minutes=source.minute)
|
||||
rows.append({"case_key": f"{case.case_id}-{level}", "case_id": case.case_id,
|
||||
"severity": level, "pipe_id": case.pipe_id,
|
||||
"area_cm2": pipes[case.pipe_id].candidate_orifice_areas_cm2[level],
|
||||
"event_at": at.isoformat(), "history_start": (at - timedelta(days=15)).isoformat(),
|
||||
"event_end": (at + timedelta(seconds=design.duration_seconds)).isoformat(),
|
||||
"reserved_end": (at + timedelta(seconds=design.duration_seconds, days=1)).isoformat()})
|
||||
previous = None
|
||||
for row in rows:
|
||||
start = datetime.fromisoformat(row["history_start"])
|
||||
if previous is not None and start <= previous:
|
||||
raise ValueError("dataset date reservations overlap")
|
||||
previous = datetime.fromisoformat(row["reserved_end"])
|
||||
if previous is not None and previous >= datetime.now(timezone.utc):
|
||||
raise ValueError("benchmark slots must remain historical")
|
||||
return rows
|
||||
@@ -0,0 +1,99 @@
|
||||
"""Learn pressure thresholds from pre-event normal history, never burst truth."""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from datetime import datetime, timedelta
|
||||
from statistics import median
|
||||
|
||||
|
||||
def learn_pressure_thresholds(
|
||||
rows: list[dict], device_ids: list[str], cutoff: datetime, *,
|
||||
history_days: int = 15, interval_seconds: int = 900,
|
||||
relative_floor_percent: float = 3.0, noise_multiplier: float = 3.0,
|
||||
absolute_floor_m: float = 0.5,
|
||||
) -> dict:
|
||||
"""Estimate noise after removing each daily slot's median operating pattern."""
|
||||
if (cutoff.tzinfo is None or history_days < 1 or interval_seconds <= 0
|
||||
or 86400 % interval_seconds or not device_ids or len(set(device_ids)) != len(device_ids)):
|
||||
raise ValueError("an aware cutoff and whole-day sampling grid are required")
|
||||
if not all(math.isfinite(value) and value > 0 for value in (relative_floor_percent, noise_multiplier, absolute_floor_m)):
|
||||
raise ValueError("threshold floors and noise multiplier must be positive")
|
||||
count = history_days * 86400 // interval_seconds
|
||||
start = cutoff - timedelta(days=history_days)
|
||||
expected = {start + timedelta(seconds=interval_seconds * index) for index in range(count)}
|
||||
data = {device: {} for device in device_ids}
|
||||
for row in rows:
|
||||
device = row["device_id"]
|
||||
if device not in data:
|
||||
continue
|
||||
at = row["time"]
|
||||
if at.tzinfo is None or at >= cutoff:
|
||||
raise ValueError("adaptive fitting must not consume event or future observations")
|
||||
if at < start:
|
||||
continue
|
||||
value = row["monitored_value"]
|
||||
if at not in expected or at in data[device] or value is None or not math.isfinite(float(value)) or float(value) <= 0:
|
||||
raise ValueError("adaptive fitting requires distinct positive finite pressure samples on the grid")
|
||||
data[device][at] = float(value)
|
||||
insufficient = {device: len(values) for device, values in data.items() if set(values) != expected}
|
||||
if insufficient:
|
||||
return {"status": "history_not_ready", "required_samples_per_device": count,
|
||||
"available_samples": insufficient, "knowledge_cutoff": cutoff.isoformat(), "devices": {}}
|
||||
fitted = {}
|
||||
for device, values in data.items():
|
||||
slots = {}
|
||||
for at, value in values.items():
|
||||
slot = int((at - start).total_seconds()) // interval_seconds % (86400 // interval_seconds)
|
||||
slots.setdefault(slot, []).append(value)
|
||||
references = {slot: median(group) for slot, group in slots.items()}
|
||||
residuals = []
|
||||
for at, value in values.items():
|
||||
slot = int((at - start).total_seconds()) // interval_seconds % (86400 // interval_seconds)
|
||||
residuals.append(100 * (value / references[slot] - 1))
|
||||
center = median(residuals)
|
||||
robust_sigma = 1.4826 * median(abs(value - center) for value in residuals)
|
||||
fitted[device] = {"relative_drop_percent": max(relative_floor_percent, noise_multiplier * robust_sigma),
|
||||
"absolute_drop_m": absolute_floor_m, "robust_noise_percent": robust_sigma,
|
||||
"reference_pressure_m_by_slot": {str(slot): value for slot, value in references.items()}}
|
||||
return {"status": "ready", "required_samples_per_device": count,
|
||||
"knowledge_cutoff": cutoff.isoformat(), "devices": fitted,
|
||||
"reference_grid_start": start.isoformat(), "interval_seconds": interval_seconds,
|
||||
"method": "daily-slot median removal; 1.4826*MAD; thresholds frozen before event"}
|
||||
|
||||
|
||||
def evaluate_pressure_thresholds(rows: list[dict], fitted: dict, *, consecutive_samples: int = 2) -> dict:
|
||||
"""Experimental adaptive evaluator; this is not the production replay worker."""
|
||||
if fitted.get("status") != "ready" or consecutive_samples < 1:
|
||||
raise ValueError("a fitted rule and positive consecutive sample count are required")
|
||||
start = datetime.fromisoformat(fitted["reference_grid_start"])
|
||||
cutoff = datetime.fromisoformat(fitted["knowledge_cutoff"])
|
||||
step = fitted["interval_seconds"]
|
||||
states = {}
|
||||
signals = []
|
||||
for row in sorted(rows, key=lambda item: (item["time"], item["device_id"])):
|
||||
device = row["device_id"]
|
||||
if device not in fitted["devices"]:
|
||||
continue
|
||||
at, value = row["time"], row["monitored_value"]
|
||||
if at.tzinfo is None or at < cutoff or (at - start).total_seconds() % step:
|
||||
raise ValueError("evaluation samples must follow the fit cutoff on the sampling grid")
|
||||
previous, hits, signaled = states.get(device, (None, 0, False))
|
||||
if previous is not None and at <= previous:
|
||||
raise ValueError("evaluation samples must be distinct")
|
||||
if previous is None or at - previous != timedelta(seconds=step):
|
||||
hits = 0
|
||||
rule = fitted["devices"][device]
|
||||
slot = int((at - start).total_seconds()) // step % (86400 // step)
|
||||
expected = rule["reference_pressure_m_by_slot"][str(slot)]
|
||||
valid = value is not None and math.isfinite(float(value)) and float(value) > 0
|
||||
drop = expected - float(value) if valid else 0
|
||||
hit = valid and drop >= rule["absolute_drop_m"] and 100 * drop / expected >= rule["relative_drop_percent"]
|
||||
hits = hits + 1 if hit else 0
|
||||
if hits >= consecutive_samples and not signaled:
|
||||
signals.append({"device_id": device, "first_signal_at": at.isoformat(),
|
||||
"reference_pressure_m": expected, "absolute_drop_m": drop,
|
||||
"relative_drop_percent": 100 * drop / expected})
|
||||
signaled = True
|
||||
states[device] = (at, hits, signaled)
|
||||
return {"raised": bool(signals), "signals": signals,
|
||||
"scope": "experimental adaptive pressure rule only; not a worker or Agent result"}
|
||||
@@ -0,0 +1,263 @@
|
||||
"""Read project data through metadata routing, in repeatable read-only transactions."""
|
||||
from __future__ import annotations
|
||||
|
||||
import gzip
|
||||
import hashlib
|
||||
import itertools
|
||||
import json
|
||||
import math
|
||||
from contextlib import contextmanager
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
import psycopg
|
||||
from psycopg.conninfo import conninfo_to_dict, make_conninfo
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
from scripts.benchmark.contracts import (
|
||||
BurstCase, BurstSuite, canonical_json, complete_devices, fingerprint, isolation_reference,
|
||||
)
|
||||
|
||||
|
||||
async def resolve_sources(suite: BurstSuite) -> tuple[dict, dict[str, str]]:
|
||||
# Import lazily: validating a suite must not load environment credentials.
|
||||
from sqlalchemy import text
|
||||
from app.infra.db.metadb.database import SessionLocal, engine
|
||||
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
|
||||
|
||||
try:
|
||||
async with SessionLocal() as session:
|
||||
await session.execute(text("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY"))
|
||||
repo = MetadataRepository(session)
|
||||
project = await repo.get_project_by_code(suite.project_code)
|
||||
if project is None or project.id != suite.project_id or project.status != "active":
|
||||
raise ValueError("benchmark project identity or active status does not match")
|
||||
routes = {}
|
||||
for role in ("biz_data", "iot_data"):
|
||||
route = await repo.get_project_db_routing(project.id, role)
|
||||
if route is None:
|
||||
raise ValueError("benchmark project database routing is incomplete")
|
||||
routes[role] = route.dsn
|
||||
identity = {"project_id": str(project.id), "project_code": project.code, "name": project.name}
|
||||
await session.rollback()
|
||||
return identity, routes
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def readonly_source(dsn: str):
|
||||
# Replace inherited options: never rely on application credentials being read-only.
|
||||
safe_dsn = make_conninfo(dsn, options=(
|
||||
"-c default_transaction_read_only=on -c statement_timeout=20000 "
|
||||
"-c lock_timeout=2000 -c idle_in_transaction_session_timeout=600000"
|
||||
), application_name="tjwater-benchmark-source-readonly", connect_timeout="5")
|
||||
with psycopg.connect(safe_dsn, row_factory=dict_row) as connection:
|
||||
connection.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
|
||||
if connection.execute("SHOW transaction_read_only").fetchone()["transaction_read_only"] != "on":
|
||||
raise ValueError("source transaction is not read-only")
|
||||
try:
|
||||
yield connection
|
||||
finally:
|
||||
connection.rollback()
|
||||
|
||||
|
||||
def source_identity(connection, dsn: str) -> dict:
|
||||
values = conninfo_to_dict(dsn)
|
||||
row = connection.execute(
|
||||
"SELECT current_database() AS database, current_setting('server_version_num') AS server_version, "
|
||||
"transaction_timestamp() AS observed_at"
|
||||
).fetchone()
|
||||
if row["database"] != values.get("dbname"):
|
||||
raise ValueError("connected source differs from the metadata route")
|
||||
return {
|
||||
**row,
|
||||
"read_only": True,
|
||||
"isolation": "repeatable read",
|
||||
# Equal database names on different servers are not equal routes.
|
||||
"route_sha256": fingerprint({key: values.get(key) for key in ("host", "port", "dbname")}),
|
||||
}
|
||||
|
||||
|
||||
def read_model(connection, cases: list[BurstCase]) -> dict:
|
||||
links = connection.execute(
|
||||
"SELECT l.id, l.link_type::text AS kind, l.start_node_id, l.end_node_id, "
|
||||
"COALESCE(s.status::text,p.status::text) AS status "
|
||||
"FROM network.links l LEFT JOIN network.pipes p ON p.link_id=l.id "
|
||||
"LEFT JOIN network.link_initial_settings s ON s.link_id=l.id ORDER BY l.id"
|
||||
).fetchall()
|
||||
for link in links:
|
||||
for key in ("id", "start_node_id", "end_node_id"):
|
||||
link[key] = str(link[key])
|
||||
sources = [str(row["id"]) for row in connection.execute(
|
||||
"SELECT id FROM network.nodes WHERE node_type::text IN ('reservoir','tank') ORDER BY id"
|
||||
).fetchall()]
|
||||
devices = connection.execute(
|
||||
"SELECT device_id,device_type,node_id,link_id,api_query_id,measurement_unit, "
|
||||
"transmission_frequency::text AS transmission_frequency FROM asset.scada_devices "
|
||||
"WHERE device_type::text IN ('pressure','pipe_flow','flow') ORDER BY device_id"
|
||||
).fetchall()
|
||||
for device in devices:
|
||||
for key in ("device_id", "node_id", "link_id"):
|
||||
if device[key] is not None:
|
||||
device[key] = str(device[key])
|
||||
analyses = connection.execute(
|
||||
"SELECT run_id,name,run_type,status,parameters FROM analysis.runs WHERE name=ANY(%s) "
|
||||
"ORDER BY name,created_at,run_id", ([case.reference_analysis for case in cases],),
|
||||
).fetchall()
|
||||
return {"links": links, "source_nodes": sources, "devices": devices, "reference_analyses": analyses}
|
||||
|
||||
|
||||
def readings(connection, device_ids: list[str], start: datetime, end: datetime) -> list[dict]:
|
||||
return connection.execute(
|
||||
"SELECT time,device_id,monitored_value FROM scada.measurements "
|
||||
"WHERE device_id=ANY(%s) AND time>=%s AND time<=%s ORDER BY time,device_id",
|
||||
(device_ids, start, end),
|
||||
).fetchall()
|
||||
|
||||
|
||||
def long_history_capability(connection, suite: BurstSuite, case: BurstCase, devices: list[dict]) -> dict:
|
||||
interval = suite.sampling_interval_minutes
|
||||
expected = suite.detector_days * 1440 // interval
|
||||
start = case.occurred_at - timedelta(days=suite.detector_days) + timedelta(minutes=interval)
|
||||
mapped = {device["device_id"]: device["node_id"] for device in devices
|
||||
if device["device_type"] == "pressure" and device["node_id"] and device["api_query_id"]}
|
||||
rows = connection.execute(
|
||||
"SELECT device_id,count(*) AS count,count(DISTINCT time) AS unique_count, "
|
||||
"count(*) FILTER (WHERE monitored_value IS NOT NULL "
|
||||
"AND monitored_value::text NOT IN ('NaN','Infinity','-Infinity') "
|
||||
"AND mod(extract(epoch FROM (time-%s::timestamptz)),%s)=0) AS valid_count "
|
||||
"FROM scada.measurements WHERE device_id=ANY(%s) AND time>=%s AND time<=%s "
|
||||
"GROUP BY device_id ORDER BY device_id",
|
||||
(case.occurred_at, interval * 60, list(mapped), start, case.occurred_at),
|
||||
).fetchall()
|
||||
eligible = sorted({mapped[row["device_id"]] for row in rows
|
||||
if row["count"] == row["unique_count"] == row["valid_count"] == expected})
|
||||
return {"available": len(eligible) >= 5, "expected_samples_per_device": expected,
|
||||
"complete_pressure_nodes": eligible, "tool_node_limit": 20,
|
||||
"required_for": "optional run_burst_detection, not the short-window SCADA trigger"}
|
||||
|
||||
|
||||
def trigger_preview(suite: BurstSuite, case: BurstCase, rows: list[dict], devices: list[dict], *, normal=False) -> dict:
|
||||
"""Exercise the current pure detector; do not tick replay, clean TSDB or create tasks."""
|
||||
from app.domain.schemas.scada_replay import ScadaReplayCreate
|
||||
from app.services.scada_replay_processing import detect_frame, prepare_cleaning
|
||||
|
||||
start = case.occurred_at - (timedelta(days=1) if normal else timedelta())
|
||||
pressure = [device for device in devices
|
||||
if device["device_type"] == "pressure" and device["measurement_unit"] == "m"]
|
||||
ids = {device["device_id"] for device in pressure}
|
||||
rule = suite.trigger_rule
|
||||
if len(ids) < rule.minimum_devices:
|
||||
return {"raised": False, "reason": "insufficient pressure devices with unit m"}
|
||||
config = ScadaReplayCreate(
|
||||
request_id=uuid5(NAMESPACE_URL, f"benchmark:{case.case_id}:{normal}"),
|
||||
start_time=start, end_time=start + timedelta(minutes=suite.event_minutes + suite.sampling_interval_minutes),
|
||||
device_ids=sorted(ids), rules=[{
|
||||
"rule_id": "benchmark-pressure-drop", "device_ids": sorted(ids),
|
||||
"kind": "relative_change", "direction": "decrease", "unit": "m",
|
||||
"threshold": rule.pressure_drop_percent, "window_samples": rule.baseline_samples,
|
||||
"minimum_samples": rule.baseline_samples, "consecutive_samples": rule.consecutive_samples,
|
||||
"minimum_devices": rule.minimum_devices,
|
||||
}],
|
||||
)
|
||||
history: dict[str, list[float]] = {device: [] for device in ids}
|
||||
for row in rows:
|
||||
value = row["monitored_value"]
|
||||
if row["device_id"] in ids and row["time"] < start and value is not None and math.isfinite(float(value)):
|
||||
history[row["device_id"]].append(float(value))
|
||||
document = {"replay_id": str(config.request_id), "devices": pressure,
|
||||
"histories": {key: values[-rule.baseline_samples:] for key, values in history.items()}}
|
||||
changes = []
|
||||
for at, group in itertools.groupby(rows, key=lambda row: row["time"]):
|
||||
if at < start:
|
||||
continue
|
||||
frame = [{**row, "source_hash": fingerprint([row["time"], row["device_id"], row["monitored_value"]])}
|
||||
for row in group if row["device_id"] in ids]
|
||||
document, events = detect_frame(document, prepare_cleaning(frame, config), config, at)
|
||||
changes.extend(event for event in events if event.get("category") != "data_quality")
|
||||
raised = [event for event in changes if event["change"] == "raised"]
|
||||
return {"raised": bool(raised), "first_signal_at": raised[0]["occurred_at"] if raised else None,
|
||||
"event_count": len(raised), "warm_pressure_devices": sum(len(v) >= rule.baseline_samples for v in history.values()),
|
||||
"scope": "pure short-window rule preview; not worker/Agent end-to-end acceptance"}
|
||||
|
||||
|
||||
def inspect_case(iot_connection, suite: BurstSuite, case: BurstCase, model: dict) -> dict:
|
||||
devices = model["devices"]
|
||||
ids = [device["device_id"] for device in devices]
|
||||
warmup = timedelta(minutes=suite.trigger_rule.baseline_samples * suite.sampling_interval_minutes)
|
||||
end = case.occurred_at + timedelta(minutes=suite.event_minutes)
|
||||
event_rows = readings(iot_connection, ids, case.occurred_at - warmup, end)
|
||||
normal_rows = readings(iot_connection, ids, case.occurred_at - timedelta(days=1) - warmup,
|
||||
end - timedelta(days=1))
|
||||
complete = set(complete_devices(event_rows, case.occurred_at, end, suite.sampling_interval_minutes))
|
||||
complete &= set(complete_devices(normal_rows, case.occurred_at - timedelta(days=1),
|
||||
end - timedelta(days=1), suite.sampling_interval_minutes))
|
||||
pressure_ids = {device["device_id"] for device in devices if device["device_type"] == "pressure"}
|
||||
matching = [run for run in model["reference_analyses"] if run["name"] == case.reference_analysis
|
||||
and run["run_type"] == "burst_analysis" and run["status"] == "completed"
|
||||
and case.pipe_id in [str(value) for value in run["parameters"].get("burst_ID", [])]]
|
||||
topology = isolation_reference(model["links"], model["source_nodes"], case.pipe_id)
|
||||
blockers = []
|
||||
if case.observation_provenance != "declared_historical_replay":
|
||||
blockers.append("event_observations_require_isolated_generation")
|
||||
if not matching:
|
||||
blockers.append("matching_completed_reference_analysis_missing")
|
||||
if len(complete & pressure_ids) < 5:
|
||||
blockers.append("fewer_than_five_complete_event_and_control_pressure_devices")
|
||||
if topology["status"] in {"target_missing", "sources_missing"}:
|
||||
blockers.append(topology["status"])
|
||||
event_signal = trigger_preview(suite, case, event_rows, devices)
|
||||
normal_signal = trigger_preview(suite, case, normal_rows, devices, normal=True)
|
||||
scada_blockers = list(blockers)
|
||||
if not event_signal["raised"]:
|
||||
scada_blockers.append("short_window_event_trigger_not_raised")
|
||||
if normal_signal["raised"]:
|
||||
scada_blockers.append("normal_control_false_positive")
|
||||
return {
|
||||
"case_id": case.case_id, "pipe_id": case.pipe_id, "occurred_at": case.occurred_at,
|
||||
"observation_provenance": case.observation_provenance,
|
||||
"data_candidate_available": not blockers,
|
||||
"blockers": blockers,
|
||||
"complete_event_and_control_pressure_devices": sorted(complete & pressure_ids),
|
||||
"window_sha256": fingerprint({"event": event_rows, "normal": normal_rows}),
|
||||
"optional_long_history_detection": long_history_capability(iot_connection, suite, case, devices),
|
||||
"event_signal_preview": event_signal, "normal_signal_preview": normal_signal,
|
||||
"topology_reference": topology,
|
||||
"variants": [
|
||||
{"case_id": case.case_id + "-WO", "data_candidate_available": not blockers,
|
||||
"blockers": list(blockers)},
|
||||
{"case_id": case.case_id + "-SCADA", "data_candidate_available": not scada_blockers,
|
||||
"blockers": scada_blockers},
|
||||
],
|
||||
"acceptance_pending": ["numerical_reference_review", "approved_equivalent_candidates",
|
||||
"public_leakage_input_or_declared_location_limitation",
|
||||
"isolated_runtime_and_cutoff_verification"],
|
||||
}
|
||||
|
||||
|
||||
def export_readings(connection, path: Path, suite: BurstSuite, cases: list[BurstCase], devices: list[dict]) -> dict:
|
||||
"""Export only burst-related devices and the selected history/event window."""
|
||||
start = min(case.occurred_at for case in cases) - timedelta(days=suite.detector_days)
|
||||
end = max(case.occurred_at for case in cases) + timedelta(minutes=suite.event_minutes)
|
||||
ids = [device["device_id"] for device in devices]
|
||||
count = 0
|
||||
digest = hashlib.sha256()
|
||||
path.touch(mode=0o600, exist_ok=False)
|
||||
with path.open("wb") as raw, gzip.GzipFile(fileobj=raw, mode="wb", mtime=0) as output:
|
||||
with connection.cursor(name="benchmark_raw_export") as cursor:
|
||||
cursor.execute(
|
||||
"SELECT time,device_id,monitored_value FROM scada.measurements "
|
||||
"WHERE device_id=ANY(%s) AND time>=%s AND time<=%s ORDER BY time,device_id", (ids, start, end),
|
||||
)
|
||||
for row in cursor:
|
||||
count += 1
|
||||
if count > 1_000_000:
|
||||
raise ValueError("selected observation export exceeds one million records")
|
||||
line = (canonical_json(row) + "\n").encode()
|
||||
digest.update(line)
|
||||
output.write(line)
|
||||
return {"file": str(path.name), "records": count, "content_sha256": digest.hexdigest(),
|
||||
"start": start, "end": end, "origin": "existing_database_readonly_export",
|
||||
"visibility": "private; contains future samples and must not be exposed directly to Agent"}
|
||||
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Calibrate a read-only model export using file-isolated native EPANET runs."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import traceback
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--design", type=Path, default=Path("resources/benchmarks/tjwater2-v2-burst-simulation.json"))
|
||||
parser.add_argument("--output-root", type=Path, default=Path("data/benchmark-calibration"))
|
||||
parser.add_argument("--duration-seconds", type=int, help="Override calibration duration, e.g. 18000 for five hours")
|
||||
parser.add_argument("--timeout-seconds", type=int, default=240)
|
||||
parser.add_argument("--all-severities", action="store_true", help="Retain weak-signal sizes even if the severe level fails")
|
||||
args = parser.parse_args()
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
os.umask(0o077)
|
||||
report = None
|
||||
report_path = None
|
||||
try:
|
||||
from psycopg.conninfo import make_conninfo
|
||||
from app.infra.db.project_routing import ActiveProjectRouting, activate_project_routing
|
||||
from app.native.wndb.core.connection import close_all_project_pools, project_transaction
|
||||
from app.native.wndb.inp.exporter import dump_inp
|
||||
from scripts.check_burst_simulation_design import readonly, resolve_project, validate_database, validate_tool_inputs
|
||||
from scripts.benchmark.simulation_design import load_simulation_design
|
||||
from scripts.benchmark.calibration import assess, configure_inp, digest, pressure_preview, run_file, save_json, select_calibrated_area
|
||||
|
||||
design = load_simulation_design(args.design)
|
||||
validate_tool_inputs(design)
|
||||
duration = args.duration_seconds if args.duration_seconds is not None else design.duration_seconds
|
||||
step = design.model_expectations.report_timestep_seconds
|
||||
if not 3600 <= duration <= 18000 or duration % step or not 1 <= args.timeout_seconds <= 1800:
|
||||
raise ValueError("duration must be whole report steps between one and five hours; timeout 1..1800")
|
||||
biz, _ = asyncio.run(resolve_project(design))
|
||||
with readonly(biz) as connection:
|
||||
validate_database(connection, design, template=False)
|
||||
folder = (args.output_root / (datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ") + "-" + uuid4().hex[:8])).resolve()
|
||||
folder.mkdir(parents=True, mode=0o700)
|
||||
snapshot = folder / "source.inp"
|
||||
readonly_dsn = make_conninfo(biz, options="-c default_transaction_read_only=on -c statement_timeout=20000")
|
||||
routing = ActiveProjectRouting(project_code=design.project_code, business_dsn=readonly_dsn)
|
||||
try:
|
||||
with activate_project_routing(routing), project_transaction(design.project_code) as connection:
|
||||
connection.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
|
||||
# Stateless models permit event-aligned runs and independent warmup history.
|
||||
counts = connection.execute("SELECT (SELECT count(*) FROM network.tanks) tanks, "
|
||||
"(SELECT count(*) FROM network.pumps) pumps, "
|
||||
"(SELECT count(*) FROM network.controls) controls, "
|
||||
"(SELECT count(*) FROM network.rules) rules").fetchone()
|
||||
if any(counts.values()):
|
||||
raise ValueError("stateful model requires an explicit warm-start implementation")
|
||||
pda_options = {row["key"]: row["value"] for row in connection.execute(
|
||||
"SELECT key,value FROM network.simulation_settings WHERE engine_version='legacy' "
|
||||
"AND key=ANY(%s)",
|
||||
(["HEADERROR", "FLOWCHANGE", "MINIMUM PRESSURE", "PRESSURE EXPONENT"],),
|
||||
)}
|
||||
dump_inp(design.project_code, str(snapshot), "2")
|
||||
finally:
|
||||
close_all_project_pools()
|
||||
source = snapshot.read_text(encoding="utf-8")
|
||||
node_ids = sorted({node for pipe in design.pipes for node in pipe.output_node_ids})
|
||||
link_ids = sorted({link for pipe in design.pipes for link in pipe.output_link_ids})
|
||||
report = {"version": "burst-calibration@1", "status": "running", "created_at": datetime.now(timezone.utc),
|
||||
"source_effect": "read-only model export; local file simulations only",
|
||||
"model_sha256": digest(snapshot), "design_sha256": digest(args.design),
|
||||
"design": design.model_dump(mode="json"), "duration_seconds": duration,
|
||||
"pda_options": pda_options,
|
||||
"assessment_scope": "strong-signal calibration, not Agent accuracy or absence of a physical burst",
|
||||
"all_severities": args.all_severities,
|
||||
"engine_sha256": digest(Path("app/infra/epanet/linux/runepanet")),
|
||||
"cases": [], "selection": {}, "controls": {},
|
||||
"limitations": ["No analysis scenario or SCADA rows have been persisted",
|
||||
"No worker, MCP, Agent, or 15-day detection has been executed",
|
||||
"The four-sample pressure replay rule is a calibration preview"]}
|
||||
report_path = folder / "calibration.json"
|
||||
save_json(report_path, report)
|
||||
|
||||
def simulate(label, start, seconds, emitter=None):
|
||||
path = folder / f"{label}.inp"
|
||||
path.write_text(configure_inp(source, start, seconds, emitter=emitter, pda_options=pda_options), encoding="utf-8")
|
||||
result = run_file(path, node_ids, link_ids, timeout=args.timeout_seconds)
|
||||
save_json(folder / f"{label}.json", result)
|
||||
print(f"{label}: {result['status']}, solver warnings={len(result['warnings'])}", flush=True)
|
||||
return result
|
||||
|
||||
controls = {}
|
||||
histories = {}
|
||||
for index, at in enumerate(design.event_times):
|
||||
controls[at] = simulate(f"normal-{index}", at, duration)
|
||||
warmup = simulate(f"warmup-{index}", at - timedelta(seconds=4 * step), 3 * step)
|
||||
if warmup.get("status") != "completed" or warmup.get("warnings") or warmup.get("times") != {
|
||||
"report_start": 0, "report_step": step, "sim_duration": 3 * step, "num_periods": 4
|
||||
}:
|
||||
raise ValueError("normal warmup did not produce four converged samples")
|
||||
histories[at] = warmup["nodes"]
|
||||
report["controls"][at.isoformat()] = {"normal_file": f"normal-{index}.json", "warmup_file": f"warmup-{index}.json"}
|
||||
save_json(report_path, report)
|
||||
for pipe in design.pipes:
|
||||
cases = [case for case in design.cases if case.pipe_id == pipe.pipe_id]
|
||||
passed_levels = []
|
||||
for severity in design.calibration_order:
|
||||
level_passed = True
|
||||
for case in cases:
|
||||
area = pipe.candidate_orifice_areas_cm2[severity]
|
||||
burst = simulate(f"{case.case_id}-{severity}", case.occurred_at, duration, (pipe.emitter_node_id, area))
|
||||
normal = controls[case.occurred_at]
|
||||
result = assess(pipe, normal, burst, design.selection_rule, duration=duration, step=step)
|
||||
if normal.get("status") == burst.get("status") == "completed" and not any(
|
||||
reason.endswith("incomplete_or_failed") for reason in result["blockers"]
|
||||
):
|
||||
for key, current in (("normal_trigger", normal), ("event_trigger", burst)):
|
||||
result[key] = pressure_preview(pipe, case.occurred_at, duration, step,
|
||||
histories[case.occurred_at], current["nodes"], design.selection_rule)
|
||||
if result["normal_trigger"]["raised"]:
|
||||
result["blockers"].append("normal_control_false_positive")
|
||||
if not result["event_trigger"]["raised"]:
|
||||
result["blockers"].append("event_rule_did_not_trigger")
|
||||
coefficient = 0.65 * area / 10000 * (19.6 ** 0.5) * 1000
|
||||
# PDA demand can change with pressure: demand delta alone is not emitter flow.
|
||||
result["emitter_flow_m3_s"] = [coefficient * max(row["pressure"], 0) ** 0.5 / 1000
|
||||
for row in burst["nodes"][pipe.emitter_node_id]]
|
||||
result.update(case_id=case.case_id, severity=severity, orifice_area_cm2=area,
|
||||
passed=not result["blockers"])
|
||||
report["cases"].append(result)
|
||||
level_passed &= result["passed"]
|
||||
save_json(report_path, report)
|
||||
print(f"{case.case_id}-{severity}: physical and rule checks {'PASS' if result['passed'] else 'FAIL'}", flush=True)
|
||||
if level_passed:
|
||||
passed_levels.append(severity)
|
||||
if severity == "severe" and not level_passed and not args.all_severities:
|
||||
break
|
||||
case_ids = {case.case_id for case in cases}
|
||||
report["selection"][pipe.pipe_id] = select_calibrated_area(
|
||||
pipe, passed_levels, [case for case in report["cases"] if case["case_id"] in case_ids], args.all_severities,
|
||||
)
|
||||
save_json(report_path, report)
|
||||
report["status"] = "completed"
|
||||
save_json(report_path, report)
|
||||
print(f"calibration complete; report: {report_path}", flush=True)
|
||||
return 0
|
||||
except Exception as exc:
|
||||
frames = traceback.extract_tb(exc.__traceback__)
|
||||
location = frames[-1] if frames else None
|
||||
suffix = f" at {location.filename}:{location.lineno}" if location else ""
|
||||
if report is not None and report_path is not None:
|
||||
report.update(status="failed", failure_type=type(exc).__name__)
|
||||
save_json(report_path, report)
|
||||
print(f"calibration failed ({type(exc).__name__}){suffix}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user