Author SHA1 Message Date
jiang b72e42521c 优化时间范围查询,添加 UTC 时间标准化处理 2026-06-01 16:46:51 +08:00
jiang c2ccb7bc4e 移除实时数据和仿真结果接口,优化代码结构 2026-05-26 18:49:25 +08:00
jiang 88be97ddeb 修正单元测试失败代码 2026-05-25 17:51:45 +08:00
jiang 2317f4d527 新增 API 测试用例,修复失效接口问题 2026-05-21 15:32:12 +08:00
jiang 751950e5b5 调整函数说明 2026-05-20 11:45:01 +08:00
jiang a1dcbd4230 更新 dockerfile,提高打包效率 2026-04-30 13:06:09 +08:00
jiang 3b712ea467 优化传感器布置算法,修复数据库更新逻辑 2026-04-17 17:21:50 +08:00
jiang bf2aaa5ff7 后端统一时区为 UTC 2026-04-14 14:46:51 +08:00
jiang 51b481d174 优化临时文件管理,增强错误日志记录 2026-04-08 11:47:46 +08:00
jiang 644babf77e 将环境设置为生产模式;更新网络名称配置 2026-04-08 10:49:01 +08:00
jiang 6b09c6b20d 删除 Dockerfile 中的临时文件复制指令 2026-04-03 14:53:55 +08:00
jiang 93cbd7e7b3 独立 copilot 服务 2026-03-27 13:52:12 +08:00
jiang 0196206ed3 创建层级化目录的 skills 2026-03-27 13:05:22 +08:00
jiang 88eec2787b 整理 api tags 2026-03-27 12:31:52 +08:00
jiang 621cd9d2f9 删除 router 中多余的tags 2026-03-26 16:09:17 +08:00
jiang 600ddd329c 添加流式 Copilot 请求处理及审计中间件优化 2026-03-24 16:01:22 +08:00
jiang c184610035 添加 Copilot 聊天流式响应接口及测试 2026-03-24 11:22:00 +08:00
jiang 21dd393aee 添加 Copilot 聊天流式响应功能及相关配置 2026-03-23 18:03:00 +08:00
jiang b0acfb21ec DSN 复用已有连接池 2026-03-19 11:16:36 +08:00
jiang 20ec7d9c8d 添加加密文本输入提示示例 2026-03-19 09:43:22 +08:00
jiang 7c44654195 实现多进程 epanet 模拟,不保留临时文件 2026-03-18 16:56:44 +08:00
jiang c5d3075ae2 添加获取项目信息接口及相关数据模型 2026-03-17 18:27:58 +08:00
jiang 2ea5ce14ba 修正 Dockerfile 中 pip 安装命令的参数,使用 uv 加快部署 2026-03-17 15:56:06 +08:00
jiang adb5dc01fb 强制使用 utf-8 存取 2026-03-16 15:59:09 +08:00
jiang fb9f3217e2 添加清理 pycache 和编译扩展的功能 2026-03-16 15:59:09 +08:00
jiang 5e8600a0a7 添加对单个python文件的编译支持 2026-03-16 15:59:09 +08:00
jiang 1dcaf5ae9f 更新引用路径 2026-03-16 15:59:09 +08:00
jiang a792838e80 移除中文注释,避免 Github Action 工作流出错 2026-03-16 15:59:09 +08:00
jiang 3cd76b9b52 优化打包流程,增加编译路径和忽略规则 2026-03-16 15:59:09 +08:00
jiang e6d00e9bc6 更新构建工作流,删除不必要的安全脚本 2026-03-16 15:59:09 +08:00
jiang 68c12cc4eb 添加构建和打包工作流 2026-03-16 15:59:09 +08:00
jiang e0c247f3b2 更新封装路径 2026-03-16 15:59:09 +08:00
jiang c3bf48499b 删除旧文件 2026-03-16 15:59:09 +08:00
jiang 102cfffefe 更改编译代码的文件名 2026-03-16 15:59:09 +08:00
jiang 1a76c89054 更新metadb引用路径 2026-03-16 15:59:09 +08:00
jiang 1673396e1a 重构时序数据库连接逻辑,移除冗余代码 2026-03-16 15:59:09 +08:00
jiang c137adedad 元数据库目录结构变更 2026-03-16 15:59:09 +08:00
jiang 5041922c84 移除未使用的区域相关函数导入 2026-03-16 15:59:09 +08:00
jiang cfe69e581b 更新API请求体,移除不必要的请求参数 2026-03-16 15:59:09 +08:00
jiang b513d05611 优化API文档,添加参数描述和示例 2026-03-16 15:56:37 +08:00
jiang 9a8d851275 删除网络元素相关的API端点空文件 2026-03-16 15:53:18 +08:00
jiang 50a1e78073 移除旧的InternalQueries类,更新管道查询逻辑 2026-03-16 15:53:18 +08:00
jiang 83a6143146 重构SCADA信息获取,移除旧的数据库接口 2026-03-16 15:53:18 +08:00
296 changed files with 3720359 additions and 4583 deletions
-19
View File
@@ -1,19 +0,0 @@
.git
.github
.gitea
__pycache__/
.pytest_cache/
.mypy_cache/
.venv/
venv/
build/
dist/
package/
temp/
data/
# db_inp/
inp/
.env
*.pyc
*.dump
app/algorithms/health/model/my_survival_forest_model_quxi.joblib
+8 -20
View File
@@ -1,16 +1,19 @@
# TJWater Server 环境变量配置模板
# 复制此文件为 .env 并填写实际值
# CI/CD: 将生产 .env 的完整内容保存为 Gitea 仓库密钥 TJWATER_SERVER_ENV。
ENVIRONMENT="production"
NETWORK_NAME="tjwater"
# ============================================
# 敏感配置加密 (必填)
# 安全配置 (必填)
# ============================================
# Fernet 格式,生产环境必须替换为独立密钥
# JWT 密钥 - 用于生成和验证 Token
# 生成方式: openssl rand -hex 32
SECRET_KEY=your-secret-key-here-change-in-production-use-openssl-rand-hex-32
# 数据加密密钥 - 用于敏感数据加密
# 生成方式: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
# 用于项目数据库 DSN、GeoServer 管理密码等敏感配置
DATABASE_ENCRYPTION_KEY="replace-with-generated-fernet-key"
ENCRYPTION_KEY=
DATABASE_ENCRYPTION_KEY="rJC2VqLg4KrlSq+DGJcYm869q4v5KB2dFAeuQTe0I50="
# ============================================
# 数据库配置 (PostgreSQL)
@@ -45,18 +48,3 @@ METADATA_DB_PASSWORD="password"
KEYCLOAK_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----"
KEYCLOAK_ALGORITHM=RS256
KEYCLOAK_AUDIENCE="account"
# ============================================
# Bocha Web Search API
# ============================================
BOCHA_API_KEY="sk-your-bocha-api-key"
BOCHA_WEB_SEARCH_URL="https://api.bochaai.com/v1/web-search"
BOCHA_WEB_SEARCH_TIMEOUT_SECONDS=30
# ============================================
# Tianditu Geocoding API
# ============================================
TIANDITU_GEOCODER_TOKEN="your-tianditu-geocoder-token"
TIANDITU_GEOCODER_URL="https://api.tianditu.gov.cn/geocoder"
TIANDITU_GEOCODER_TIMEOUT_SECONDS=30
+82
View File
@@ -0,0 +1,82 @@
# Copilot Instructions for TJWater Server
This repository contains the backend code for the TJWater Server, a water distribution network management system built with FastAPI.
## High-Level Architecture
The application follows a layered architecture:
- **Entry Point**: `app/main.py` initializes the FastAPI application, database connections (PostgreSQL & TimescaleDB), and middleware.
- **API Layer**: `app/api/v1` contains the route handlers.
- **Service Layer**: `app/services` contains business logic and orchestration.
- **Infrastructure Layer**: `app/infra` handles database connections (`db`), audit logging (`audit`), and external integrations.
- **Domain Layer**: `app/domain` likely contains core domain models.
- **Native/Algorithms**: `app/native` and `app/algorithms` handle specialized water network calculations (possibly using EPANET/WNTR).
## Build, Test, and Run Commands
### Environment Setup
- Dependencies are listed in `requirements.txt`.
- Configuration is managed via environment variables (see `.env.example` if available, or `app/core/config.py`).
- **Important**: Ensure `.env` is configured with correct database credentials for both PostgreSQL and TimescaleDB.
If first time setting up, you may want to create a Conda environment:
```bash
conda create -n server python=3.12
conda activate server
pip install uv
uv pip install -r requirements.txt
conda install -c conda-forge pymetis
```
### Running the Server
The preferred way to run the server locally is using the helper script which sets up the Python path correctly:
```bash
conda activate server
python scripts/run_server.py
```
Alternatively, you can run directly with uvicorn (ensure PYTHONPATH includes the root):
```bash
conda activate server
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
```
### Running Tests
Use `pytest` to run tests. The `tests/conftest.py` handles path setup.
```bash
# Run all tests
pytest
# Run a specific test file
pytest tests/unit/test_specific_file.py
# Run a specific test case
pytest tests/unit/test_specific_file.py::test_function_name
```
### Building (Optional)
The project includes scripts to compile Python modules to `.pyd` files using Cython (see `scripts/build_pyd.py`). This is likely for distribution/performance but not required for standard development.
## Key Conventions
- **Async/Await**: The codebase heavily uses `async` and `await` for I/O operations, especially database interactions.
- **Database Management**:
- Connections are managed globally in `app.infra.db` and initialized in `lifespan` (app/main.py).
- Use `app.infra.db.dynamic_manager` for project-specific database connections (multi-tenancy/dynamic projects).
- **Pydantic**: extensively used for data validation and settings management.
- **Scripts**: The `scripts/` directory contains many utility scripts for maintenance, data processing, and server management. Check there before writing new operational scripts.
- **Water Network Modeling**: Interactions with water network models often involve `epanet` or `wntr` libraries. Be aware of domain-specific terminology (nodes, links, junctions, tanks).
## Code Style
- Follow standard PEP 8 guidelines.
- No specific linter configuration was found, so default to standard Python formatting.
+128
View File
@@ -0,0 +1,128 @@
name: Build And Package
on:
push:
tags:
- "v*"
jobs:
build-package:
runs-on: ${{ matrix.os }}
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
steps:
- name: Checkout source
uses: actions/checkout@v5
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
- name: Install system build tools
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y build-essential
- name: Install compile dependencies
run: |
python -m pip install --upgrade pip
pip install cython setuptools wheel
- name: Run Cython compile
run: |
python scripts/compile.py
- name: Prepare package and archive
run: |
python - <<'PY'
import os
import shutil
import tarfile
import zipfile
import sys
from pathlib import Path
root = Path.cwd()
package_dir = root / "package"
dist_dir = root / "dist"
for d in [package_dir, dist_dir]:
if d.exists():
shutil.rmtree(d)
d.mkdir(parents=True, exist_ok=True)
# Define directories with compiled artifacts
compile_dirs = ["app/services", "app/native/wndb", "app/algorithms"]
# Global ignore list
ignore_names = {
".git",
".github",
"__pycache__",
".pytest_cache",
".mypy_cache",
".venv",
"venv",
"temp",
"tests",
"package",
"dist",
}
def ignore_func(directory, names):
rel_dir = os.path.relpath(directory, root).replace("\\", "/")
is_in_compile_path = any(rel_dir.startswith(d) for d in compile_dirs)
ignored = []
for name in names:
if name in ignore_names or name.endswith(".pyc"):
ignored.append(name)
# Exclude source .py files only in compiled directories
elif is_in_compile_path and name.endswith(".py"):
ignored.append(name)
return ignored
for item in root.iterdir():
if item.name in ignore_names:
continue
target = package_dir / item.name
if item.is_dir():
shutil.copytree(item, target, ignore=ignore_func)
else:
shutil.copy2(item, target)
# Safety guard: ensure no .github directory remains
github_paths = [p for p in package_dir.rglob(".github") if p.is_dir()]
for p in github_paths:
shutil.rmtree(p, ignore_errors=True)
sha = os.environ["GITHUB_SHA"]
run_os = os.environ["RUNNER_OS"].lower()
if run_os == "windows":
archive_path = dist_dir / f"tjwater-server-{run_os}-{sha}.zip"
with zipfile.ZipFile(archive_path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
for f in package_dir.rglob("*"):
if f.is_file():
zf.write(f, f.relative_to(package_dir))
else:
archive_path = dist_dir / f"tjwater-server-{run_os}-{sha}.tar.gz"
with tarfile.open(archive_path, "w:gz") as tf:
tf.add(package_dir, arcname=".")
print(f"Archive created: {archive_path}")
PY
shell: bash
- name: Upload package artifact
uses: actions/upload-artifact@v5
with:
name: tjwater-server-package-${{ runner.os }}
path: dist/*
retention-days: 14
+1
View File
@@ -7,3 +7,4 @@ build/
*.dump
.vscode/
app/algorithms/health/model/my_survival_forest_model_quxi.joblib
inp/
-73
View File
@@ -1,73 +0,0 @@
# Repository Guidelines
## Project Purpose
This repository is the customer-delivery edition of the TJWater backend. Treat it as a deployable delivery package, not as the primary internal development repository. Changes should be limited to customer-facing fixes, deployment compatibility, configuration templates, packaging, and delivery documentation.
Do not introduce internal-only experiments, debug utilities, local data, or source material that is not required for customer operation.
## Source Encapsulation Requirement
Core backend source code must be encapsulated before delivery. The sensitive implementation areas include business services, native integrations, hydraulic/network algorithms, and EPANET-related logic, especially:
- `app/services`
- `app/native`
- `app/algorithms`
- `app/infra/epanet`
Use the existing Cython packaging flow in `scripts/compile.py` for these areas. Do not ship uncompiled core `.py` files in the final customer package when compiled extension modules are expected. Keep public entry points, configuration loading, route wiring, and minimal package files readable only where required for runtime and operations.
Before removing source files with `--delete-source`, verify the build from a clean working tree or disposable copy. The delete mode is destructive by design.
## Workspace Structure
- `app/main.py` is the FastAPI entry point.
- `app/api` contains HTTP route handlers.
- `app/services` contains core business orchestration and must be protected in delivery builds.
- `app/native`, `app/algorithms`, and `app/infra/epanet` contain specialized computation and integration code that must be protected in delivery builds.
- `infra/` and `Dockerfile` contain deployment assets.
- `scripts/` contains operational and packaging helpers.
- `tests/` contains backend tests.
## Common Commands
Run commands from this repository root:
```bash
conda run -n server python -m pytest tests -q
conda run -n server python scripts/run_server.py
conda run -n server python scripts/compile.py
```
Clean compiled extensions when needed:
```bash
conda run -n server python scripts/compile.py --clean
```
Only use source deletion in a prepared delivery copy:
```bash
conda run -n server python scripts/compile.py --delete-source
```
## Change Policy
Keep changes scoped and conservative. Prefer compatibility patches, configuration adjustments, packaging fixes, and customer deployment hardening. If a change requires substantial business logic updates, make it first in the internal backend repository and then port the reviewed result here.
Do not weaken the encapsulation process to make debugging easier. If debugging requires readable source, do it in the internal repository or a non-delivery branch/copy.
## Testing Guidelines
Run the narrowest useful test command for the affected area. For delivery packaging changes, verify both:
- tests still pass before packaging;
- the packaged/compiled runtime can import and start the FastAPI app.
Do not add tests that depend on untracked customer data, local database dumps, or machine-specific files.
## Security & Delivery Rules
Never commit `.env`, production credentials, customer data, logs, generated caches, `node_modules/`, database dumps, or temporary delivery archives. Use `.env.example` or deployment documentation for required configuration.
Review Docker and deployment files carefully before delivery. Customer packages should contain only the files needed to run, operate, and diagnose the deployed service.
-90
View File
@@ -1,90 +0,0 @@
# TJWater Authentication and Metadata Management
## Ownership
Keycloak owns login identity, credentials, token issuance, and token expiry.
TJWater metadata stores only business snapshots and authorization data:
- `users.keycloak_id` is the stable identity binding.
- `users.username`, `users.email`, and `users.last_login_at` are Keycloak claim caches.
- `users.role`, `users.is_active`, and `users.is_superuser` control TJWater system access.
- `user_project_membership.project_role` controls project access.
The backend does not accept passwords, does not issue local JWTs, and does not
trust frontend-supplied user IDs.
## Login Snapshot Refresh
Every authenticated metadata-user resolution validates the Keycloak access token
and reads `sub`, `preferred_username` or `username`, and `email` claims. The
backend finds `users` by `keycloak_id = sub`, rejects inactive or missing users,
then refreshes `username`, `email`, and `last_login_at`.
This keeps local display data current without changing the identity binding.
There is no Keycloak webhook requirement; second-level user or permission sync is
out of scope unless explicitly requested later.
## Admin APIs
All admin APIs require metadata admin access: `users.is_superuser = true` or
`users.role = 'admin'`.
User and membership management:
- `GET /api/v1/admin/me`
- `POST /api/v1/admin/users/sync`
- `POST /api/v1/admin/users/sync/batch`
- `GET /api/v1/admin/users`
- `GET /api/v1/admin/users/{user_id}`
- `PATCH /api/v1/admin/users/{user_id}`
- `GET /api/v1/admin/projects/{project_id}/members`
- `POST /api/v1/admin/projects/{project_id}/members`
- `PATCH /api/v1/admin/projects/{project_id}/members/{user_id}`
- `DELETE /api/v1/admin/projects/{project_id}/members/{user_id}`
Project configuration:
- `GET /api/v1/admin/projects`
- `POST /api/v1/admin/projects`
- `PATCH /api/v1/admin/projects/{project_id}`
- `GET /api/v1/admin/projects/{project_id}/databases`
- `PUT /api/v1/admin/projects/{project_id}/databases`
- `DELETE /api/v1/admin/projects/{project_id}/databases/{db_role}`
- `POST /api/v1/admin/projects/{project_id}/databases/{db_role}/health`
## Secret Handling
Admins submit plaintext DSNs only through HTTPS admin APIs. Operators should not
write encrypted columns manually.
- `project_databases.dsn_encrypted` is encrypted with `DATABASE_ENCRYPTION_KEY`.
- Admin responses return only `has_dsn`.
- Audit logs record whether a secret was updated, but never store plaintext DSNs
or other secrets.
Generate the database encryption key with:
```bash
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
```
Keep keys stable for the lifetime of encrypted metadata. Rotating a key requires
decrypting with the old key and re-encrypting with the new key.
## Metadata Schema Patches
Apply metadata patches in order:
1. `resources/sql/004_metadata_auth_management.sql`
2. `resources/sql/005_metadata_project_configuration.sql`
`004` creates Keycloak-backed metadata users and project memberships. `005`
creates project and project database routing tables with uniqueness, role/type,
and pool-size constraints.
## Frontend System Management
`/system-admin` is shown only after `GET /api/v1/admin/me` confirms metadata
admin access. The page lets admins maintain metadata users, project members,
projects, project database routing for `biz_data` and `iot_data`, connection
health checks. This replaces direct SQL editing for normal project onboarding.
-80
View File
@@ -1,80 +0,0 @@
# Backend Naming Audit
DOC-003 audit for the customer-delivery `TJWaterServerCustomer` backend.
## Scope
Reviewed FastAPI route decorators under `app/api/v1/endpoints`, router prefixes in `app/api/v1/router.py`, and public request/response schema fields in `app/api` and `app/domain`.
The backend is mounted only under `/api/v1` from `app/main.py`; the old no-prefix router include remains commented out.
## Current Good Surface
These newer routes already follow the naming rule for public HTTP paths:
- Audit: `/api/v1/audit/logs`, `/api/v1/audit/logs/count`
- Metadata: `/api/v1/meta/project`, `/api/v1/meta/projects`, `/api/v1/meta/db/health`
- Business APIs: `/api/v1/burst-detection/detect`, `/api/v1/burst-location/locate`, `/api/v1/leakage/identify`
- Time-series APIs: `/api/v1/scada/by-ids-time-range`, `/api/v1/scada/by-ids-field-time-range`, `/api/v1/composite/clean-scada`
- Project data APIs: `/api/v1/scada-info`, `/api/v1/scheme-list`, `/api/v1/burst-locate-result`
- Web integrations: `/api/v1/web-search`, `/api/v1/geocode`
Path template parameters such as `{user_id}`, `{device_id}`, `{scheme_name}`, and `{link_id}` intentionally remain `snake_case`.
## Legacy URL Categories
### Keep With Compatibility
These now have `kebab-case` aliases. The frontend has been migrated to the replacement paths; keep the old paths as deprecated compatibility aliases for Agent planning, tests, customer scripts, or external callers:
| Current URL | Suggested replacement |
| --- | --- |
| `/api/v1/openproject/` | `/api/v1/projects/open` |
| `/api/v1/project_info/` | `/api/v1/project-info` |
| `/api/v1/getallschemes/` | `/api/v1/schemes` |
| `/api/v1/getallsensorplacements/` | `/api/v1/sensor-placement-schemes` |
| `/api/v1/sensorplacementscheme/create` | `/api/v1/sensor-placement-schemes` |
| `/api/v1/burst_analysis/` | `/api/v1/burst-analysis` |
| `/api/v1/valve_isolation_analysis/` | `/api/v1/valve-isolation-analysis` |
| `/api/v1/flushing_analysis/` | `/api/v1/flushing-analysis` |
| `/api/v1/contaminant_simulation/` | `/api/v1/contaminant-simulation` |
| `/api/v1/runsimulationmanuallybydate/` | `/api/v1/simulations/run-by-date` |
### Broad Legacy Surface
These route groups expose many command-style concatenated paths. They should not be copied into new work; replace only when a caller migration is planned:
- Project lifecycle: `listprojects`, `createproject`, `deleteproject`, `isprojectopen`, `closeproject`, `copyproject`, `importinp`, `exportinp`, `readinp`, `dumpinp`, `lockproject`, `unlockproject`
- Network object CRUD: `addjunction`, `getjunctionelevation`, `setpipediameter`, `getvalvesetting`, and similar junction/pipe/pump/tank/reservoir/valve routes
- Region/DMA/VD commands: `calculatedistrictmeteringareaforregion`, `getdistrictmeteringarea`, `generatevirtualdistrict`, and related routes
- SCADA native CRUD: `getscadadevice`, `setscadadevicedata`, `cleanscadaelement`, and related routes
- Snapshot/cache utilities: `takesnapshotforoperation`, `syncwithserver`, `clearrediskey`, `queryredis`
- Advanced simulation endpoints with underscore paths: `pressure_regulation`, `daily_scheduling_analysis`, `network_update`, `pressure_sensor_placement_kmeans`
### Direct Cleanup Candidates
These are likely safe only after confirming no caller uses them:
- `/api/v1/test_dict/`: development/test utility in `misc.py`.
- `/api/v1/takenapshotforcurrentoperation`: typo compatibility path; keep deprecated if any client may still call it.
- `/api/v1/getpumpenergyproperties//` and `/api/v1/setpumpenergyproperties//`: double-slash paths in options endpoints.
## Field Naming
Most public JSON, query, and SSE fields are already `snake_case`, including `user_id`, `scheme_name`, `scheme_type`, `start_time`, `end_time`, `device_ids`, `session_id`, and `request_id`.
Known legacy exception:
- `BurstAnalysis.burst_ID` in `app/api/v1/endpoints/simulation.py` should become `burst_id` on a new API contract. Preserve `burst_ID` only for the legacy body shape.
Headers keep standard HTTP casing:
- `X-Project-Id`
## Internal vs Customer Difference
The customer backend retains local auth/user-management routes under `/api/v1/auth` and `/api/v1/users`; the internal backend has migrated to Keycloak/metadata admin routes. Treat those Customer-only auth routes as delivery compatibility surface, not a source for new internal API naming.
## Recommendation
Do not rename existing legacy routes in place. For each active legacy route, keep the new `kebab-case` alias as the documented path, keep the old route marked deprecated, migrate remaining Agent/customer/script callers, then remove only after a documented compatibility window.
-102
View File
@@ -1,102 +0,0 @@
# tjwater-server Customer Image Packaging Notes
This repository is the customer-delivery backend package. Build delivery images from this repository root.
## Image Build
Build the customer backend image:
```bash
docker build -t tjwater-server:latest .
```
The `Dockerfile` already performs source encapsulation in the builder stage:
```bash
python scripts/compile.py
python scripts/compile.py --delete-source
```
The compiled and source-deleted `app/` directory is then copied into the final runner stage. The source deletion happens inside the Docker builder layer only; it does not delete local workspace files.
## Encapsulation Scope
`scripts/compile.py` defaults to compiling these sensitive areas:
- `app/services`
- `app/native/wndb`
- `app/algorithms`
- `app/infra/epanet/epanet.py`
These areas should not contain uncompiled `.py` source files in the delivered image. Public entry points, API route wiring, schemas, configuration, and operational files may remain readable when required for runtime.
## Verification
After building, verify the image tag:
```bash
docker image ls tjwater-server
```
Verify that core source files were removed and compiled extensions exist:
```bash
docker run --rm --entrypoint sh tjwater-server:latest -c "\
printf 'core_py_count='; \
find /app/app/services /app/app/native/wndb /app/app/algorithms /app/app/infra/epanet/epanet.py -name '*.py' 2>/dev/null | wc -l; \
printf 'core_so_count='; \
find /app/app/services /app/app/native/wndb /app/app/algorithms /app/app/infra/epanet -name '*.so' 2>/dev/null | wc -l; \
python -c 'import app.main; print(\"import_app_main=ok\")'"
```
Expected delivery result:
```text
core_py_count=0
core_so_count=<non-zero>
import_app_main=ok
```
Warnings from third-party packages during import are not necessarily build failures. Treat non-zero exit codes, failed imports, or leftover core `.py` files as blockers.
## Export For Windows Delivery
Export the image tarball to the Windows desktop from WSL:
```bash
docker save -o /mnt/c/Users/admin/Desktop/tjwater-server-latest.tar tjwater-server:latest
```
Adjust the Windows username if needed. To find available desktop paths:
```bash
find /mnt/c/Users -maxdepth 2 -type d \( -name Desktop -o -name 桌面 \) 2>/dev/null
```
On the target machine, import the image with:
```bash
docker load -i tjwater-server-latest.tar
```
## Deployment Caution
The development `infra/docker/docker-compose.yml` bind-mounts local source:
```yaml
volumes:
- ../../app:/app/app
- ../../resources:/app/resources
```
Do not use that source mount for customer delivery of the encapsulated image. Mounting `../../app` over `/app/app` replaces the compiled code inside the image with local source files and defeats the encapsulation. For delivery compose files, use the built image directly and mount only required runtime data/config paths.
## Local Safety
Do not run this destructive command in the normal working tree:
```bash
python scripts/compile.py --delete-source
```
Only use `--delete-source` in Docker builder stages or in a disposable delivery copy. The normal delivery image build already handles this safely.
+5 -24
View File
@@ -1,39 +1,20 @@
FROM condaforge/miniforge3:latest AS runtime-base
FROM condaforge/miniforge3:latest
WORKDIR /app
ENV PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple \
PIP_TRUSTED_HOST=pypi.tuna.tsinghua.edu.cn \
UV_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
# 安装 Python 3.12 和 pymetis (通过 conda-forge 避免编译问题)
RUN mamba install -y python=3.12 pymetis && \
mamba clean -afy
COPY requirements.txt .
RUN pip install --no-cache-dir uv
RUN pip install uv
RUN uv pip install --system --no-cache-dir -r requirements.txt
FROM runtime-base AS builder
RUN mamba install -y c-compiler cxx-compiler && \
mamba clean -afy
COPY app ./app
COPY scripts ./scripts
RUN python scripts/compile.py && \
python scripts/compile.py --delete-source
FROM runtime-base AS runner
# 将代码放入子目录 'app',将数据放入子目录 'db_inp'
# 这样临时文件默认会生成在 /app 下,而代码在 /app/app 下,实现了分离
COPY --from=builder /app/app ./app
RUN python -c "from pathlib import Path; from zipfile import ZipFile; model_dir = Path('app/algorithms/health/model'); zip_path = model_dir / 'my_survival_forest_model_quxi.zip'; joblib_name = 'my_survival_forest_model_quxi.joblib'; joblib_path = model_dir / joblib_name; assert zip_path.exists(), f'Model archive not found: {zip_path}'; archive = ZipFile(zip_path); archive.extract(joblib_name, model_dir); archive.close(); assert joblib_path.exists(), f'Model file not extracted: {joblib_path}'" && \
rm -f app/algorithms/health/model/my_survival_forest_model_quxi.zip
# COPY db_inp ./db_inp
RUN mkdir -p db_inp temp data inp
COPY app ./app
COPY db_inp ./db_inp
COPY .env .
# 设置 PYTHONPATH 以便 uvicorn 找到 app 模块
ENV PYTHONPATH=/app
-107
View File
@@ -1,107 +0,0 @@
# TJWaterServerCustomer 客户版后端
`TJWaterServerCustomer` 是 TJWater 客户交付版 Python 后端。该仓库应被视为可部署交付包,只保留客户运行、配置、部署、诊断和交付说明所需内容。
## 技术栈
- Python 3.12
- FastAPI / Uvicorn
- Pydantic / SQLAlchemy / psycopg
- Redis、PostgreSQL、PostGIS、TimescaleDB
- WNTR、EPANET、Cython、科学计算与空间分析依赖
- pytest
## 目录结构
```text
app/main.py FastAPI 入口
app/api/ HTTP API 路由
app/auth/ 认证和权限上下文
app/core/ 配置、日志和基础设施初始化
app/domain/ 领域模型和 Pydantic schema
app/infra/ 数据库、缓存、EPANET 和外部集成
app/services/ 核心业务服务,交付镜像中必须封装
app/algorithms/ 核心算法,交付镜像中必须封装
app/native/ 本地管网数据读写与转换,交付镜像中必须封装
scripts/compile.py Cython 封装脚本
infra/docker/ Docker Compose 编排
tests/ 后端测试
```
## 本地开发
推荐使用已有 conda 环境:
```bash
conda run -n server python -m pytest tests -q
conda run -n server python scripts/run_server.py
```
本地调试不要在正常工作树执行源码删除命令。
## 客户版镜像打包
交付镜像标签通常为:
```bash
docker build -t tjwater-server:latest .
```
`Dockerfile` 会在 builder 阶段执行:
```bash
python scripts/compile.py
python scripts/compile.py --delete-source
```
源码删除只发生在 Docker 构建层内,不会删除本地工作树源码。详细封装、验证和 Windows 导出说明见:
```text
DELIVERY_PACKAGING_NOTES.md
```
## 封装范围
`scripts/compile.py` 默认封装:
- `app/services`
- `app/native/wndb`
- `app/algorithms`
- `app/infra/epanet/epanet.py`
交付镜像中这些核心目录不应残留未编译的 `.py` 源码,应以 `.so` 扩展模块运行。
## 验证命令
构建后检查镜像:
```bash
docker image ls tjwater-server
```
检查核心源码是否已删除、FastAPI 入口是否可导入:
```bash
docker run --rm --entrypoint sh tjwater-server:latest -c "\
printf 'core_py_count='; \
find /app/app/services /app/app/native/wndb /app/app/algorithms /app/app/infra/epanet/epanet.py -name '*.py' 2>/dev/null | wc -l; \
printf 'core_so_count='; \
find /app/app/services /app/app/native/wndb /app/app/algorithms /app/app/infra/epanet -name '*.so' 2>/dev/null | wc -l; \
python -c 'import app.main; print(\"import_app_main=ok\")'"
```
期望结果:
```text
core_py_count=0
core_so_count=<非零>
import_app_main=ok
```
## 部署注意
客户交付时不要把本地 `../../app` 挂载到容器 `/app/app`,否则会覆盖镜像内已封装代码。交付 compose 文件应使用构建好的镜像,只挂载必要的运行数据和配置。
## 安全规则
不要提交 `.env`、生产凭据、客户数据、数据库 dump、日志、生成缓存、临时交付压缩包或本地运行目录。客户版仓库不应加入内部实验、调试工具或非交付源码材料。
+424
View File
@@ -0,0 +1,424 @@
# Agent CLI 接口范围确认
本文档确认 `app/api/v1/endpoints/` 面向 Agent CLI 的首批封装范围。
## 结论
首批 CLI 采用 **少量顶层入口 + 业务域二级分组 + 只读/分析优先** 的设计。
```text
tjwater auth
tjwater project
tjwater network
tjwater component
tjwater simulation
tjwater analysis
tjwater data
tjwater help
tjwater result
```
首批默认不暴露:
- 会修改 network 的接口:`add*``set*``delete*``generate*`
- 项目生命周期接口:创建、删除、导入、打开、关闭、锁定、解锁、复制
- 数据写入/清理接口:insert、update、delete、clean、clear、batch store
- 用户管理接口:创建、更新、删除、激活、停用
- 快照回滚和批量命令执行接口:undo、redo、pick、batch
## 设计原则
- CLI 不按 HTTP endpoint 一比一映射,而按 Agent 任务组织。
- 首批只暴露 `schema``list``get``exists`、只读计算和分析类能力。
- CLI 输入优先使用显式选项、可重复选项、枚举值和文件路径,尽量不要求用户直接输入 JSON。
- CLI 输出统一使用 JSON;大结果写入 result-ref,只在 stdout 返回摘要、路径和元数据。
- 现有 HTTP 路径的拼写错误、双斜杠、错误方法不继承到 CLI。
- 高频命令可以提供 alias,但文档和 skill 只写规范命令。
## 分级约束
| 顶层命令 | 二级范围 | 说明 |
|---|---|---|
| `auth` | `me``refresh` | 登录态和当前用户 |
| `project` | `list``info``status``export-inp``data` | 项目发现和只读项目数据 |
| `network` | `list``get``schema``exists``geometry``region``tag` | 管网拓扑、元素、几何、分区,只读 |
| `component` | `curve``pattern``option``control``quality``visual` | EPANET 组件类能力 |
| `simulation` | `run``run-inp``output` | 模拟运行和模拟输出 |
| `analysis` | `burst``leakage``valve``flushing``age``sensor-placement``risk` | 任务级分析 |
| `data` | `timeseries``scada``scheme``extension``misc` | 数据查询 |
| `help` | `--json``COMMAND --json` | Agent 能力发现和命令说明 |
| `result` | `show``metadata``export` | `result-ref` 读取和导出 |
命令深度建议:
- 常规命令不超过 3 层:`tjwater component curve list`
- 时序数据允许 4 层:`tjwater data timeseries realtime links`
- `risk` 归入 `analysis risk`
- `scada``scheme``extension` 归入 `data`
## 首批 CLI 范围
### Auth / Project
来源:
```text
app/api/v1/endpoints/auth.py
app/api/v1/endpoints/meta.py
app/api/v1/endpoints/project.py
app/api/v1/endpoints/project_data.py
```
| 命令 | 覆盖接口 | 说明 |
|---|---|---|
| `tjwater auth me` | `GET /auth/me` | 当前登录用户 |
| `tjwater auth refresh` | `POST /auth/refresh` | 仅在 CLI 需要维护登录态时暴露 |
| `tjwater project list` | `GET /meta/projects` | 项目列表 |
| `tjwater project info --project PROJECT` | `GET /meta/project` | 项目信息 |
| `tjwater project db-health --project PROJECT` | `GET /meta/db/health` | 项目数据库健康 |
| `tjwater project export-inp --project PROJECT --out-ref` | `GET /exportinp/``GET /dumpinp/``GET /downloadinp/` | 导出 INP,写 `result-ref` |
| `tjwater project data --project PROJECT --kind scada-info\|scheme-list\|burst-locate-result` | `GET /scada-info``GET /scheme-list``GET /burst-locate-result*` | 项目业务数据 |
暂不暴露:
```text
POST /auth/register
POST /auth/login
POST /auth/login/simple
GET /listprojects/
GET /project_info/
GET /haveproject/
GET /isprojectopen/
GET /isprojectlocked/
GET /isprojectlockedbyme/
POST /createproject/
POST /deleteproject/
POST /openproject/
POST /closeproject/
POST /copyproject/
POST /importinp/
POST /readinp/
POST /lockproject/
POST /unlockproject/
POST /uploadinp/
GET /convertv3tov2/
```
### Network
来源:
```text
app/api/v1/endpoints/network/*.py
```
| 命令 | 覆盖接口 | 说明 |
|---|---|---|
| `tjwater network list --network NET --type nodes\|links` | `GET /getnodes/``GET /getlinks/` | 节点/管线 ID 列表 |
| `tjwater network exists --network NET --type node\|link\|junction\|pipe\|... --id ID` | `GET /isnode/``GET /islink/` 等 | 元素存在性 |
| `tjwater network type --network NET --id ID` | `GET /getnodetype/``GET /getlinktype/``GET /getelementtype/` | 元素类型 |
| `tjwater network get --network NET --id ID` | `GET /getelementproperties/``GET /getnodeproperties/``GET /getlinkproperties/` | 自动识别类型并取属性 |
| `tjwater network get --network NET --type junction\|pipe\|pump\|... --id ID` | 各类 `get*properties` | 指定类型取属性 |
| `tjwater network list-properties --network NET --type junction\|pipe\|pump\|... --out-ref` | 各类 `getall*properties` | 全量属性,写 `result-ref` |
| `tjwater network schema --network NET --type junction\|reservoir\|tank\|pipe\|pump\|valve\|demand\|tag\|region` | 各类 `get*schema` | 属性架构 |
| `tjwater network links-of-node --network NET --node NODE` | `GET /getnodelinks/` | 节点关联管线 |
| `tjwater network geometry --network NET --scope full\|extent\|major-nodes\|major-pipes\|link-nodes --out-ref` | `geometry.py``GET` 接口 | 几何数据 |
| `tjwater network demand-calc --network NET --scope node\|region\|network --out-ref` | `GET /calculatedemandto*/` | 需水量计算 |
| `tjwater network region get\|list\|schema --network NET --kind dma\|service-area\|virtual-district` | `regions.py``GET` 查询接口 | 分区信息 |
| `tjwater network region-calc --network NET --kind dma\|service-area\|virtual-district --out-ref` | `GET /calculate*/` | 分区计算 |
| `tjwater network tag get\|list\|schema --network NET` | `GET /gettag/``GET /gettags/``GET /gettagschema/` | 标签信息 |
暂不暴露:
```text
add*
set*
delete*
generate*
POST /generatedistrictmeteringarea/
POST /generatesubdistrictmeteringarea/
POST /generateservicearea/
POST /generatevirtualdistrict/
```
备注:`GET /settitle/` 语义是修改标题,首批不暴露。
### Component
来源:
```text
app/api/v1/endpoints/components/*.py
```
| 命令 | 覆盖接口 | 说明 |
|---|---|---|
| `tjwater component curve schema\|list\|get\|exists` | `curves.py` 下只读接口 | 曲线 |
| `tjwater component pattern schema\|list\|get\|exists` | `patterns.py` 下只读接口 | 模式 |
| `tjwater component option schema\|get --kind time\|energy\|pump-energy\|general` | `options.py` 下只读接口 | 时间、能耗、泵能耗、通用选项 |
| `tjwater component control schema\|get --kind control\|rule` | `controls.py` 下只读接口 | 控制和规则 |
| `tjwater component quality schema\|get --kind quality\|emitter\|source\|reaction\|pipe-reaction\|tank-reaction\|mixing` | `quality.py` 下只读接口 | 水质相关组件 |
| `tjwater component visual schema\|list\|get --kind vertex\|label\|backdrop\|vertex-links\|vertices` | `visuals.py` 下只读接口 | 图形元素、标签、背景 |
暂不暴露:
```text
POST /addcurve/
POST /setcurveproperties/
POST /deletecurve/
POST /addpattern/
POST /setpatternproperties/
POST /deletepattern/
POST /settimeproperties/
POST /setenergyproperties/
GET /setpumpenergyproperties//
POST /setoptionproperties/
POST /setcontrolproperties/
POST /setruleproperties/
POST /setqualityproperties/
POST /setemitterproperties/
POST /setsource/
POST /addsource/
POST /deletesource/
POST /setreaction/
POST /setpipereaction/
POST /settankreaction/
POST /setmixing/
POST /addmixing/
POST /deletemixing/
POST /setvertexproperties/
POST /addvertex/
POST /deletevertex/
POST /setlabelproperties/
POST /addlabel/
POST /deletelabel/
POST /setbackdropproperties/
```
备注:
- `getsourcechema` 路径拼写疑似错误,CLI 统一使用 `component quality schema --kind source`
- `getallvertexlinks``getallvertices` 当前返回 JSON 字符串,CLI 应输出标准 JSON。
### Simulation / Analysis / Risk
来源:
```text
app/api/v1/endpoints/simulation.py
app/api/v1/endpoints/leakage.py
app/api/v1/endpoints/burst_detection.py
app/api/v1/endpoints/burst_location.py
app/api/v1/endpoints/risk.py
```
| 命令 | 覆盖接口 | 说明 |
|---|---|---|
| `tjwater simulation run --project PROJECT --out-ref` | `GET /runprojectreturndict/` | 运行项目模拟,使用结构化 JSON 返回 |
| `tjwater simulation run-inp --inp PATH --out-ref` | `GET /runinp/` | 运行 INP |
| `tjwater simulation output --project PROJECT --out-ref` | `GET /dumpoutput/` | 导出模拟输出 |
| `tjwater analysis burst --project PROJECT --start-time TIME --duration SEC --burst ID:SIZE --out-ref` | `GET /burst_analysis/` | 爆管分析,`--burst` 可重复 |
| `tjwater analysis valve --project PROJECT --mode close\|isolation --start-time TIME --valve VALVE --out-ref` | `GET /valve_close_analysis/``GET /valve_isolation_analysis/` | 阀门分析,`--valve` 可重复 |
| `tjwater analysis flushing --project PROJECT --start-time TIME --valve VALVE:OPENING --drainage-node NODE --flow FLOW --out-ref` | `GET /flushing_analysis/` | 冲洗分析,`--valve` 可重复 |
| `tjwater analysis age --project PROJECT --start-time TIME --duration SEC --out-ref` | `GET /age_analysis/` | 水龄分析 |
| `tjwater analysis contaminant --project PROJECT --start-time TIME --duration SEC --source NODE:VALUE --out-ref` | `GET /contaminant_simulation/` | 污染物模拟 |
| `tjwater analysis sensor-placement --project PROJECT --method sensitivity\|kmeans --count N --out-ref` | 传感器放置分析接口 | 不包含创建方案 |
| `tjwater analysis leakage identify --scheme SCHEME --start-time TIME --end-time TIME --out-ref` | `POST /leakage/identify/` | 漏损识别 |
| `tjwater analysis leakage schemes list\|get` | `GET /leakage/schemes/``GET /leakage/schemes/{scheme_name}` | 漏损方案查询 |
| `tjwater analysis burst-detection detect --scheme SCHEME --start-time TIME --end-time TIME --out-ref` | `POST /burst-detection/detect/` | 爆管检测 |
| `tjwater analysis burst-detection schemes list\|get` | `GET /burst-detection/schemes/``GET /burst-detection/schemes/{scheme_name}` | 爆管检测方案查询 |
| `tjwater analysis burst-location locate --scheme SCHEME --start-time TIME --end-time TIME --out-ref` | `POST /burst-location/locate/` | 爆管定位 |
| `tjwater analysis burst-location schemes list\|get` | `GET /burst-location/schemes/``GET /burst-location/schemes/{scheme_name}` | 爆管定位方案查询 |
| `tjwater analysis risk pipe --network NET --pipe PIPE --time-range ...` | `risk.py` 下管道风险 `GET` 接口 | 管道风险 |
| `tjwater analysis risk network --network NET --out-ref` | `GET /getnetworkpiperiskprobabilitynow/``GET /getpiperiskprobabilitygeometries/` | 全网风险 |
暂缓或暂不暴露:
```text
POST /network_project/
GET /runproject/
POST /network_update/
POST /project_management/
POST /sensorplacementscheme/create
POST /runsimulationmanuallybydate/
POST /pump_failure/
POST /pressure_regulation/
POST /scheduling_analysis/
POST /daily_scheduling_analysis/
```
### Data
来源:
```text
app/api/v1/endpoints/timeseries/*.py
app/api/v1/endpoints/scada.py
app/api/v1/endpoints/schemes.py
app/api/v1/endpoints/extension.py
app/api/v1/endpoints/misc.py
app/api/v1/endpoints/project_data.py
```
| 命令 | 覆盖接口 | 说明 |
|---|---|---|
| `tjwater data timeseries realtime links --start-time TIME --end-time TIME --out-ref` | `GET /realtime/links` | 实时管道数据 |
| `tjwater data timeseries realtime nodes --start-time TIME --end-time TIME --out-ref` | `GET /realtime/nodes` | 实时节点数据 |
| `tjwater data timeseries realtime simulation --query by-id-time\|by-time-property --id ID --time TIME --property PROPERTY --out-ref` | `GET /realtime/query/*` | 实时模拟查询 |
| `tjwater data timeseries scheme links --scheme SCHEME --start-time TIME --end-time TIME --out-ref` | `GET /scheme/links``GET /scheme/links/{link_id}/field` | 方案管道数据 |
| `tjwater data timeseries scheme node-field --node NODE --field FIELD --out-ref` | `GET /scheme/nodes/{node_id}/field` | 方案节点字段 |
| `tjwater data timeseries scheme simulation --query by-id-time\|by-scheme-time-property --scheme SCHEME --id ID --time TIME --property PROPERTY --out-ref` | `GET /scheme/query/*` | 方案模拟查询 |
| `tjwater data timeseries scada query --device-ids ... --time-range ... --out-ref` | `GET /scada/by-ids-time-range``GET /scada/by-ids-field-time-range` | SCADA 时序 |
| `tjwater data timeseries composite --kind scada-simulation\|element-simulation\|element-scada --feature FEATURE --start-time TIME --end-time TIME --out-ref` | `GET /composite/*` | 复合查询,`--feature` 可重复 |
| `tjwater data timeseries composite pipeline-health --pipe PIPE --start-time TIME --end-time TIME --out-ref` | `GET /composite/pipeline-health-prediction` | 管道健康预测 |
| `tjwater data scada schema --kind device\|device-data\|element\|info` | `GET /getscada*schema/` | `SCADA` 元数据 `schema` |
| `tjwater data scada get\|list --kind device\|device-data\|element\|info` | `scada.py``GET` 查询接口 | `SCADA` 元数据 |
| `tjwater data scheme schema\|get\|list --network NET` | `schemes.py``GET` 接口 | 方案查询 |
| `tjwater data extension keys\|get\|list --network NET` | `extension.py``GET` 查询接口 | 扩展数据查询 |
| `tjwater data misc sensor-placements --network NET --out-ref` | `GET /getallsensorplacements/` | 传感器位置 |
| `tjwater data misc burst-location-results --network NET --out-ref` | `GET /getallburstlocateresults/` | 爆管定位结果 |
暂不暴露:
```text
POST /realtime/*/batch
DELETE /realtime/*
PATCH /realtime/*
POST /realtime/simulation/store
POST /scheme/*/batch
PATCH /scheme/*
DELETE /scheme/*
POST /scheme/simulation/store
POST /scada/batch
PATCH /scada/{device_id}/field
DELETE /scada/by-id-time-range
POST /composite/clean-scada
POST /setscadadevice/
POST /addscadadevice/
POST /deletescadadevice/
POST /cleanscadadevice/
POST /setscadadevicedata/
POST /addscadadevicedata/
POST /deletescadadevicedata/
POST /cleanscadadevicedata/
POST /setscadaelement/
POST /addscadaelement/
POST /deletescadaelement/
POST /cleanscadaelement/
POST /setextensiondata/
POST /test_dict/
GET /getjson/
```
### 不纳入首批 CLI 的运维接口
来源:
```text
app/api/v1/endpoints/snapshots.py
app/api/v1/endpoints/cache.py
app/api/v1/endpoints/audit.py
app/api/v1/endpoints/users.py
app/api/v1/endpoints/user_management.py
```
这些接口不纳入首批 Agent CLI。原因是它们更偏运维、审计、用户管理或状态回滚,不属于 Agent 面向水务业务分析的核心调用范围。
暂不暴露:
```text
GET /getcurrentoperationid/
GET /getsnapshots/
GET /havesnapshot/
GET /havesnapshotforoperation/
GET /havesnapshotforcurrentoperation/
GET /getrestoreoperation/
POST /undo/
POST /redo/
POST /takesnapshot*/
POST /picksnapshot/
POST /pickoperation/
GET /syncwithserver/
POST /batch/
POST /compressedbatch/
POST /setrestoreoperation/
GET /queryredis/
POST /clearrediskey/
POST /clearrediskeys/
POST /clearallredis/
GET /audit/logs
GET /audit/logs/my
GET /audit/logs/count
GET /getuserschema/
GET /getuser/
GET /getallusers/
PUT /users/{user_id}
DELETE /users/{user_id}
POST /users/{user_id}/activate
POST /users/{user_id}/deactivate
```
## Help / Result
这两个模块不直接对应现有 endpoint,但建议作为 Agent CLI 的基础设施。能力发现更适合复用 CLI 的 `help` 语义,而不是新增一个偏内部化的 `capability` 顶层命令。
| 命令 | 说明 |
|---|---|
| `tjwater help --json` | 返回当前 CLI 能力清单,供 Agent 发现可用命令 |
| `tjwater help COMMAND --json` | 返回某个命令的参数、输出、示例和推荐后续命令 |
| `tjwater result show REF` | 读取 `result-ref` 内容,必要时分页或摘要 |
| `tjwater result metadata REF` | 读取 `result-ref` 元数据 |
| `tjwater result export REF --format json\|csv` | 导出结果 |
## 输出规范
成功:
```json
{
"ok": true,
"summary": "读取成功",
"data": {},
"result_ref": null,
"metadata": {},
"next_commands": []
}
```
失败:
```json
{
"ok": false,
"error": "invalid_argument",
"message": "缺少必要参数 --network",
"recoverable": true,
"suggested_command": "tjwater component curve list --network NET"
}
```
大结果:
```json
{
"ok": true,
"summary": "查询完成,结果已写入 result-ref",
"result_ref": "TJWaterAgent/data/result-refs/example.json",
"metadata": {
"schema": "network_properties_v1",
"rows": 1200
}
}
```
## 后续开放条件
如后续要开放写操作,需要单独设计:
- 权限校验
- dry-run / preview
- 显式确认机制
- 审计日志
- 变更快照
- 回滚策略
- Agent 可读的错误恢复建议
+35 -44
View File
@@ -1,45 +1,36 @@
"""Algorithm package with side-effect-free, lazy compatibility exports."""
from app.algorithms.cleaning import flow_data_clean, pressure_data_clean
from app.algorithms.sensor import (
pressure_sensor_placement_sensitivity,
pressure_sensor_placement_kmeans,
)
from app.algorithms.isolation.valve import valve_isolation_analysis
from app.algorithms.leakage import LeakageIdentifier
from app.algorithms.health import PipelineHealthAnalyzer
from app.algorithms.burst_location import run_burst_location
from app.algorithms.simulation.scenarios import (
convert_to_local_unit,
burst_analysis,
valve_close_analysis,
flushing_analysis,
contaminant_simulation,
age_analysis,
pressure_regulation,
)
from importlib import import_module
from typing import Any
_EXPORT_MODULES = {
"flow_data_clean": "app.algorithms.cleaning",
"pressure_data_clean": "app.algorithms.cleaning",
"pressure_sensor_placement_sensitivity": "app.algorithms.sensor",
"pressure_sensor_placement_kmeans": "app.algorithms.sensor",
"valve_isolation_analysis": "app.algorithms.isolation.valve",
"LeakageIdentifier": "app.algorithms.leakage",
"PipelineHealthAnalyzer": "app.algorithms.health",
"run_burst_location": "app.algorithms.burst_location",
**{
name: "app.algorithms.simulation.scenarios"
for name in (
"convert_to_local_unit",
"burst_analysis",
"valve_close_analysis",
"flushing_analysis",
"contaminant_simulation",
"age_analysis",
"pressure_regulation",
)
},
}
__all__ = list(_EXPORT_MODULES)
def __getattr__(name: str) -> Any:
try:
module_name = _EXPORT_MODULES[name]
except KeyError as exc:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}") from exc
value = getattr(import_module(module_name), name)
globals()[name] = value
return value
def __dir__() -> list[str]:
return sorted({*globals(), *__all__})
__all__ = [
"flow_data_clean",
"pressure_data_clean",
"pressure_sensor_placement_sensitivity",
"pressure_sensor_placement_kmeans",
"convert_to_local_unit",
"burst_analysis",
"valve_close_analysis",
"flushing_analysis",
"contaminant_simulation",
"age_analysis",
"pressure_regulation",
"valve_isolation_analysis",
"LeakageIdentifier",
"PipelineHealthAnalyzer",
"run_burst_location",
]
@@ -121,7 +121,7 @@ def run_burst_location(
basic_pressure: float = 10.0,
n_workers: int = DEFAULT_N_WORKERS,
partition_on_full_graph: bool = True,
visualize_partition: bool = False,
visualize_partition: bool = True,
visualize_pause_seconds: float = 0.3,
final_candidates_csv_path: (
str | None
+6 -8
View File
@@ -122,14 +122,12 @@ def _worker_evaluate(raw_ratios: np.ndarray) -> float:
class LeakageIdentifier:
FLOW_UNIT_TO_M3S = {
"m3/s": 1.0,
"m³/s": 1.0,
"m3/h": 1.0 / 3600.0,
"m³/h": 1.0 / 3600.0,
"L/s": 1.0 / 1000.0,
"L/min": 1.0 / 60000.0,
}
FLOW_UNIT_TO_M3S = {
"m3/s": 1.0,
"m3/h": 1.0 / 3600.0,
"L/s": 1.0 / 1000.0,
"L/min": 1.0 / 60000.0,
}
@classmethod
def _flow_to_m3s(cls, value: float, unit: str) -> float:
+12 -3
View File
@@ -29,7 +29,6 @@ import pytz
import requests
import time
import app.services.project_info as project_info
from app.services.time_api import parse_clock_duration_seconds
url_path = 'http://10.101.15.16:9000/loong' # 内网
# url_path = 'http://183.64.62.100:9057/loong' # 外网
@@ -552,11 +551,21 @@ def from_clock_to_seconds (clock: str)->int:
return hr*3600+mnt*60+seconds
def from_clock_to_seconds_2 (clock: str)->int:
return parse_clock_duration_seconds(clock)
str_format="%H:%M:%S"
dt=datetime.strptime(clock,str_format)
hr=dt.hour
mnt=dt.minute
seconds=dt.second
return hr*3600+mnt*60+seconds
def from_clock_to_seconds_3 (clock: str)->int:
return parse_clock_duration_seconds(clock)
str_format = "%H:%M" # 更新时间格式以适应 "小时:分钟" 格式
dt = datetime.strptime(clock,str_format)
hr = dt.hour
mnt = dt.minute
seconds = dt.second
return hr * 3600 + mnt * 60
###convert datetimestring
+4 -16
View File
@@ -72,7 +72,6 @@ def burst_analysis(
modify_variable_pump_pattern: dict[str, list] = None,
modify_valve_opening: dict[str, float] = None,
scheme_name: str = None,
username: str | None = None,
) -> None:
"""
爆管模拟
@@ -87,9 +86,6 @@ def burst_analysis(
:param scheme_name: 方案名称
:return:
"""
if not username:
raise ValueError("username is required when storing burst analysis scheme")
scheme_detail: dict = {
"burst_ID": burst_ID,
"burst_size": burst_size,
@@ -215,7 +211,7 @@ def burst_analysis(
name=name,
scheme_name=scheme_name,
scheme_type="burst_analysis",
username=username,
username="admin",
scheme_start_time=modify_pattern_start_time,
scheme_detail=scheme_detail,
)
@@ -315,7 +311,6 @@ def flushing_analysis(
drainage_node_ID: str = None,
flushing_flow: float = 0,
scheme_name: str = None,
username: str | None = None,
) -> None:
"""
管道冲洗模拟
@@ -328,9 +323,6 @@ def flushing_analysis(
:param scheme_name: 方案名称
:return:
"""
if not username:
raise ValueError("username is required when storing flushing analysis scheme")
scheme_detail: dict = {
"duration": modify_total_duration,
"valve_opening": modify_valve_opening,
@@ -463,7 +455,7 @@ def flushing_analysis(
name=name,
scheme_name=scheme_name,
scheme_type="flushing_analysis",
username=username,
username="admin",
scheme_start_time=modify_pattern_start_time,
scheme_detail=scheme_detail,
)
@@ -481,7 +473,6 @@ def contaminant_simulation(
concentration: float, # 污染源浓度,单位mg/L
scheme_name: str = None,
source_pattern: str = None, # 污染源时间变化模式名称
username: str | None = None,
) -> None:
"""
污染模拟
@@ -495,9 +486,6 @@ def contaminant_simulation(
:param scheme_name: 方案名称
:return:
"""
if not username:
raise ValueError("username is required when storing contaminant analysis scheme")
scheme_detail: dict = {
"source": source,
"concentration": concentration,
@@ -620,7 +608,7 @@ def contaminant_simulation(
name=name,
scheme_name=scheme_name,
scheme_type="contaminant_analysis",
username=username,
username="admin",
scheme_start_time=modify_pattern_start_time,
scheme_detail=scheme_detail,
)
@@ -674,7 +662,7 @@ def age_analysis(
new_name,
"realtime",
modify_pattern_start_time,
duration=modify_total_duration,
modify_total_duration,
downloading_prohibition=True,
)
simulation_result = json.loads(result)
-695
View File
@@ -1,695 +0,0 @@
from typing import List
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Path, Query, Response, status
from sqlalchemy import text
from sqlalchemy.engine.url import make_url
from sqlalchemy.exc import IntegrityError, SQLAlchemyError
from sqlalchemy.ext.asyncio import create_async_engine
from app.auth.metadata_dependencies import (
get_current_metadata_admin,
get_metadata_repository,
)
from app.core.audit import AuditAction, log_audit_event
from app.domain.schemas.admin_metadata import (
AdminProjectCreateRequest,
AdminProjectResponse,
AdminProjectUpdateRequest,
MetadataUsersBatchSyncRequest,
MetadataUserResponse,
MetadataUserSyncRequest,
MetadataUserSyncResult,
MetadataUserUpdateRequest,
ProjectDatabaseHealthResponse,
ProjectDatabaseHealthRequest,
ProjectDatabaseResponse,
ProjectDatabaseUpsertRequest,
ProjectDbRole,
ProjectMemberCreateRequest,
ProjectMemberResponse,
ProjectMemberUpdateRequest,
)
from app.infra.db.metadb import models
from app.infra.db.metadb.repositories.metadata_repository import (
MetadataRepository,
ProjectDbRouting,
)
router = APIRouter()
def _project_response(project: models.Project) -> AdminProjectResponse:
return AdminProjectResponse(
project_id=project.id,
name=project.name,
code=project.code,
description=project.description,
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
status=project.status,
created_at=project.created_at,
updated_at=project.updated_at,
)
def _project_database_response(
record: models.ProjectDatabase,
) -> ProjectDatabaseResponse:
return ProjectDatabaseResponse(
id=record.id,
project_id=record.project_id,
db_role=record.db_role,
db_type=record.db_type,
pool_min_size=record.pool_min_size,
pool_max_size=record.pool_max_size,
has_dsn=bool(record.dsn_encrypted),
)
def _database_audit_payload(payload: ProjectDatabaseUpsertRequest) -> dict:
return {
"db_role": payload.db_role,
"db_type": _db_type_for_role(payload.db_role),
"pool_min_size": payload.pool_min_size,
"pool_max_size": payload.pool_max_size,
"dsn_updated": payload.dsn is not None,
}
def _to_async_sqlalchemy_url(dsn: str) -> str:
parsed = make_url(dsn)
if parsed.drivername in {"postgresql", "postgres"}:
parsed = parsed.set(drivername="postgresql+psycopg")
return parsed.render_as_string(hide_password=False)
def _db_type_for_role(db_role: str) -> str:
if db_role == "iot_data":
return "timescaledb"
return "postgresql"
def _status_for_config_value_error(exc: ValueError) -> int:
if "DATABASE_ENCRYPTION_KEY" in str(exc):
return status.HTTP_503_SERVICE_UNAVAILABLE
return status.HTTP_400_BAD_REQUEST
async def _check_database_connection(routing: ProjectDbRouting) -> None:
engine = create_async_engine(
_to_async_sqlalchemy_url(routing.dsn),
pool_size=1,
max_overflow=0,
pool_pre_ping=True,
)
try:
async with engine.connect() as conn:
await conn.execute(text("SELECT 1"))
finally:
await engine.dispose()
def _database_health_error_detail(exc: Exception) -> str:
message = str(exc)
lower_message = message.lower()
if "password authentication failed" in lower_message:
return "连通性测试失败:用户名或密码错误,请检查 DSN 中的账号密码。"
if "connection refused" in lower_message:
return "连通性测试失败:目标主机或端口拒绝连接,请检查地址、端口和服务状态。"
if "timeout" in lower_message or "timed out" in lower_message:
return "连通性测试失败:连接超时,请检查网络、防火墙和数据库服务状态。"
if "could not translate host name" in lower_message or "name or service not known" in lower_message:
return "连通性测试失败:数据库主机名无法解析,请检查 DSN 中的主机地址。"
first_line = message.splitlines()[0] if message else exc.__class__.__name__
return f"连通性测试失败:{first_line}"
async def _upsert_and_audit_metadata_user(
payload: MetadataUserSyncRequest,
*,
current_user,
metadata_repo: MetadataRepository,
response_status: int,
) -> MetadataUserResponse:
user = await metadata_repo.upsert_user_from_keycloak(
keycloak_id=payload.keycloak_id,
username=payload.username,
email=str(payload.email),
role=payload.role,
is_active=payload.is_active,
)
await log_audit_event(
action=AuditAction.UPDATE,
user_id=current_user.id,
resource_type="metadata_user",
resource_id=str(user.id),
request_data=payload.model_dump(mode="json"),
response_status=response_status,
session=metadata_repo.session,
)
return MetadataUserResponse.model_validate(user)
@router.get("/me", response_model=MetadataUserResponse)
async def get_metadata_admin_me(
current_user=Depends(get_current_metadata_admin),
) -> MetadataUserResponse:
return MetadataUserResponse.model_validate(current_user)
@router.post("/users/sync", response_model=MetadataUserResponse)
async def sync_metadata_user(
payload: MetadataUserSyncRequest,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> MetadataUserResponse:
try:
return await _upsert_and_audit_metadata_user(
payload,
current_user=current_user,
metadata_repo=metadata_repo,
response_status=status.HTTP_200_OK,
)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="User keycloak_id, username, or email conflicts with an existing user",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
@router.post("/users/sync/batch", response_model=List[MetadataUserSyncResult])
async def sync_metadata_users_batch(
payload: MetadataUsersBatchSyncRequest,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[MetadataUserSyncResult]:
results: list[MetadataUserSyncResult] = []
for item in payload.users:
try:
user = await _upsert_and_audit_metadata_user(
item,
current_user=current_user,
metadata_repo=metadata_repo,
response_status=status.HTTP_200_OK,
)
except IntegrityError as exc:
results.append(
MetadataUserSyncResult(
keycloak_id=item.keycloak_id,
success=False,
error="User keycloak_id, username, or email conflicts with an existing user",
)
)
await metadata_repo.session.rollback()
except SQLAlchemyError as exc:
results.append(
MetadataUserSyncResult(
keycloak_id=item.keycloak_id,
success=False,
error=f"Metadata database error: {exc}",
)
)
await metadata_repo.session.rollback()
else:
results.append(
MetadataUserSyncResult(
keycloak_id=item.keycloak_id,
success=True,
user=user,
)
)
return results
@router.get("/users", response_model=List[MetadataUserResponse])
async def list_metadata_users(
skip: int = Query(0, ge=0),
limit: int = Query(100, ge=1, le=1000),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[MetadataUserResponse]:
users = await metadata_repo.list_users(skip=skip, limit=limit)
return [MetadataUserResponse.model_validate(user) for user in users]
@router.get("/projects", response_model=List[AdminProjectResponse])
async def list_admin_projects(
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[AdminProjectResponse]:
projects = await metadata_repo.list_project_records()
return [_project_response(project) for project in projects]
@router.post(
"/projects",
response_model=AdminProjectResponse,
status_code=status.HTTP_201_CREATED,
)
async def create_admin_project(
payload: AdminProjectCreateRequest,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> AdminProjectResponse:
try:
project = await metadata_repo.create_project(
name=payload.name,
code=payload.code,
description=payload.description,
gs_workspace=payload.gs_workspace,
map_extent=payload.map_extent,
status=payload.status,
)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Project code or workspace conflicts with an existing project",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
await log_audit_event(
action=AuditAction.CREATE,
user_id=current_user.id,
project_id=project.id,
resource_type="project",
resource_id=str(project.id),
request_data=payload.model_dump(mode="json"),
response_status=status.HTTP_201_CREATED,
session=metadata_repo.session,
)
return _project_response(project)
@router.patch(
"/projects/{project_id}",
response_model=AdminProjectResponse,
)
async def update_admin_project(
payload: AdminProjectUpdateRequest,
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> AdminProjectResponse:
updates = payload.model_dump(mode="json", exclude_unset=True)
try:
project = await metadata_repo.update_project(project_id, updates=updates)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Project code or workspace conflicts with an existing project",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
await log_audit_event(
action=AuditAction.UPDATE,
user_id=current_user.id,
project_id=project.id,
resource_type="project",
resource_id=str(project.id),
request_data=updates,
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return _project_response(project)
@router.get(
"/projects/{project_id}/databases",
response_model=List[ProjectDatabaseResponse],
)
async def list_project_databases(
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[ProjectDatabaseResponse]:
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
records = await metadata_repo.list_project_databases(project_id)
return [_project_database_response(record) for record in records]
@router.put(
"/projects/{project_id}/databases",
response_model=ProjectDatabaseResponse,
)
async def upsert_project_database(
payload: ProjectDatabaseUpsertRequest,
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectDatabaseResponse:
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
try:
routing = (
ProjectDbRouting(
project_id=project_id,
db_role=payload.db_role,
db_type=_db_type_for_role(payload.db_role),
dsn=payload.dsn,
pool_min_size=payload.pool_min_size,
pool_max_size=payload.pool_max_size,
)
if payload.dsn
else await metadata_repo.get_project_db_routing(project_id, payload.db_role)
)
if routing is None:
raise ValueError("dsn is required when creating project database config")
await _check_database_connection(routing)
except ValueError as exc:
raise HTTPException(
status_code=_status_for_config_value_error(exc),
detail=str(exc),
) from exc
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=_database_health_error_detail(exc),
) from exc
try:
record = await metadata_repo.upsert_project_database_config(
project_id,
db_role=payload.db_role,
db_type=_db_type_for_role(payload.db_role),
dsn=payload.dsn,
pool_min_size=payload.pool_min_size,
pool_max_size=payload.pool_max_size,
)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Project database role conflicts with an existing config",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
await log_audit_event(
action=AuditAction.CONFIG_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_database",
resource_id=payload.db_role,
request_data=_database_audit_payload(payload),
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return _project_database_response(record)
@router.delete(
"/projects/{project_id}/databases/{db_role}",
status_code=status.HTTP_204_NO_CONTENT,
)
async def delete_project_database(
project_id: UUID = Path(...),
db_role: ProjectDbRole = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> None:
removed = await metadata_repo.delete_project_database_config(project_id, db_role)
if not removed:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project database config not found",
)
await log_audit_event(
action=AuditAction.CONFIG_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_database",
resource_id=db_role,
request_data={"deleted": True},
response_status=status.HTTP_204_NO_CONTENT,
session=metadata_repo.session,
)
@router.post(
"/projects/{project_id}/databases/{db_role}/health",
response_model=ProjectDatabaseHealthResponse,
)
async def check_project_database_health(
response: Response,
project_id: UUID = Path(...),
db_role: ProjectDbRole = Path(...),
payload: ProjectDatabaseHealthRequest | None = None,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectDatabaseHealthResponse:
dsn_to_test = payload.dsn if payload and payload.dsn else None
if dsn_to_test:
routing = ProjectDbRouting(
project_id=project_id,
db_role=db_role,
db_type=_db_type_for_role(db_role),
dsn=dsn_to_test,
pool_min_size=1,
pool_max_size=1,
)
else:
try:
routing = await metadata_repo.get_project_db_routing(project_id, db_role)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Project database routing DSN is invalid: {exc}",
) from exc
if routing is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project database config not found",
)
try:
await _check_database_connection(routing)
except Exception as exc: # health endpoint should return diagnostic status
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
return ProjectDatabaseHealthResponse(
project_id=project_id,
db_role=db_role,
db_type=routing.db_type,
ok=False,
detail=_database_health_error_detail(exc),
)
return ProjectDatabaseHealthResponse(
project_id=project_id,
db_role=db_role,
db_type=routing.db_type,
ok=True,
detail="连通性测试通过",
)
@router.get("/users/{user_id}", response_model=MetadataUserResponse)
async def get_metadata_user(
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> MetadataUserResponse:
user = await metadata_repo.get_user_by_id(user_id)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
return MetadataUserResponse.model_validate(user)
@router.patch("/users/{user_id}", response_model=MetadataUserResponse)
async def update_metadata_user(
payload: MetadataUserUpdateRequest,
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> MetadataUserResponse:
updates = payload.model_dump(mode="json", exclude_unset=True)
if user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify themselves",
)
user = await metadata_repo.update_user_admin(
user_id,
updates=updates,
)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
await log_audit_event(
action=AuditAction.UPDATE,
user_id=current_user.id,
resource_type="metadata_user",
resource_id=str(user.id),
request_data=updates,
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return MetadataUserResponse.model_validate(user)
@router.get(
"/projects/{project_id}/members",
response_model=List[ProjectMemberResponse],
)
async def list_project_members(
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[ProjectMemberResponse]:
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project not found"
)
members = await metadata_repo.list_project_members(project_id)
return [ProjectMemberResponse(**member.__dict__) for member in members]
@router.post(
"/projects/{project_id}/members",
response_model=ProjectMemberResponse,
status_code=status.HTTP_201_CREATED,
)
async def add_project_member(
payload: ProjectMemberCreateRequest,
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectMemberResponse:
if payload.user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify their own project membership",
)
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project not found"
)
user = await metadata_repo.get_user_by_id(payload.user_id)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
existing = await metadata_repo.get_project_membership(project_id, payload.user_id)
if existing is not None:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="User is already a project member",
)
membership = await metadata_repo.add_project_member(
project_id, payload.user_id, payload.project_role
)
await log_audit_event(
action=AuditAction.PERMISSION_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_member",
resource_id=str(payload.user_id),
request_data=payload.model_dump(mode="json"),
response_status=status.HTTP_201_CREATED,
session=metadata_repo.session,
)
return ProjectMemberResponse(
id=membership.id,
user_id=membership.user_id,
project_id=membership.project_id,
project_role=membership.project_role,
username=user.username,
email=user.email,
is_active=user.is_active,
)
@router.patch(
"/projects/{project_id}/members/{user_id}",
response_model=ProjectMemberResponse,
)
async def update_project_member(
payload: ProjectMemberUpdateRequest,
project_id: UUID = Path(...),
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectMemberResponse:
if user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify their own project membership",
)
user = await metadata_repo.get_user_by_id(user_id)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
membership = await metadata_repo.update_project_member_role(
project_id, user_id, payload.project_role
)
if membership is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project member not found"
)
await log_audit_event(
action=AuditAction.PERMISSION_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_member",
resource_id=str(user_id),
request_data=payload.model_dump(mode="json"),
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return ProjectMemberResponse(
id=membership.id,
user_id=membership.user_id,
project_id=membership.project_id,
project_role=membership.project_role,
username=user.username,
email=user.email,
is_active=user.is_active,
)
@router.delete("/projects/{project_id}/members/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
async def remove_project_member(
project_id: UUID = Path(...),
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> None:
if user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify their own project membership",
)
removed = await metadata_repo.remove_project_member(project_id, user_id)
if not removed:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project member not found"
)
await log_audit_event(
action=AuditAction.PERMISSION_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_member",
resource_id=str(user_id),
response_status=status.HTTP_204_NO_CONTENT,
session=metadata_repo.session,
)
-50
View File
@@ -1,50 +0,0 @@
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from app.auth.keycloak_dependencies import get_current_keycloak_payload
from app.auth.metadata_dependencies import get_current_metadata_user
from app.auth.project_dependencies import (
ProjectContext,
get_project_context,
)
router = APIRouter()
class AgentAuthContextResponse(BaseModel):
user_id: str
keycloak_sub: str
username: str
role: str
is_superuser: bool
project_id: str
network: str
project_role: str
token_expires_at: str | None = None
@router.get("/agent/auth/context", response_model=AgentAuthContextResponse)
async def get_agent_auth_context(
ctx: ProjectContext = Depends(get_project_context),
current_user=Depends(get_current_metadata_user),
keycloak_payload: dict = Depends(get_current_keycloak_payload),
) -> AgentAuthContextResponse:
exp = keycloak_payload.get("exp")
token_expires_at = (
datetime.fromtimestamp(exp, tz=timezone.utc).isoformat()
if isinstance(exp, int)
else None
)
return AgentAuthContextResponse(
user_id=str(current_user.id),
keycloak_sub=str(current_user.keycloak_id),
username=current_user.username,
role=current_user.role,
is_superuser=current_user.is_superuser,
project_id=str(ctx.project_id),
network=ctx.project_code,
project_role=ctx.project_role,
token_expires_at=token_expires_at,
)
+190
View File
@@ -0,0 +1,190 @@
from typing import Annotated
from datetime import timedelta
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordRequestForm
from app.core.config import settings
from app.core.security import create_access_token, create_refresh_token, verify_password
from app.domain.schemas.user import UserCreate, UserResponse, UserLogin, Token
from app.infra.db.metadb.repositories.user_repository import UserRepository
from app.auth.dependencies import get_user_repository, get_current_active_user
from app.domain.schemas.user import UserInDB
import logging
logger = logging.getLogger(__name__)
router = APIRouter()
@router.post(
"/register", response_model=UserResponse, status_code=status.HTTP_201_CREATED
)
async def register(
user_data: UserCreate, user_repo: UserRepository = Depends(get_user_repository)
) -> UserResponse:
"""
用户注册
创建新用户账号
"""
# 检查用户名和邮箱是否已存在
if await user_repo.user_exists(username=user_data.username):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Username already registered",
)
if await user_repo.user_exists(email=user_data.email):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST, detail="Email already registered"
)
# 创建用户
try:
user = await user_repo.create_user(user_data)
if not user:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to create user",
)
return UserResponse.model_validate(user)
except Exception as e:
logger.error(f"Error during user registration: {e}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Registration failed",
)
@router.post("/login", response_model=Token)
async def login(
form_data: Annotated[OAuth2PasswordRequestForm, Depends()],
user_repo: UserRepository = Depends(get_user_repository),
) -> Token:
"""
用户登录(OAuth2 标准格式)
返回 JWT Access Token 和 Refresh Token
"""
# 验证用户(支持用户名或邮箱登录)
user = await user_repo.get_user_by_username(form_data.username)
if not user:
# 尝试用邮箱登录
user = await user_repo.get_user_by_email(form_data.username)
if not user or not verify_password(form_data.password, user.hashed_password):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect username or password",
headers={"WWW-Authenticate": "Bearer"},
)
if not user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user account"
)
# 生成 Token
access_token = create_access_token(subject=user.username)
refresh_token = create_refresh_token(subject=user.username)
return Token(
access_token=access_token,
refresh_token=refresh_token,
token_type="bearer",
expires_in=settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60,
)
@router.post("/login/simple", response_model=Token)
async def login_simple(
username: str,
password: str,
user_repo: UserRepository = Depends(get_user_repository),
) -> Token:
"""
简化版登录接口(保持向后兼容)
直接使用 username 和 password 参数
"""
# 验证用户
user = await user_repo.get_user_by_username(username)
if not user:
user = await user_repo.get_user_by_email(username)
if not user or not verify_password(password, user.hashed_password):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect username or password",
)
if not user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user account"
)
# 生成 Token
access_token = create_access_token(subject=user.username)
refresh_token = create_refresh_token(subject=user.username)
return Token(
access_token=access_token,
refresh_token=refresh_token,
token_type="bearer",
expires_in=settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60,
)
@router.get("/me", response_model=UserResponse)
async def get_current_user_info(
current_user: UserInDB = Depends(get_current_active_user),
) -> UserResponse:
"""
获取当前登录用户信息
"""
return UserResponse.model_validate(current_user)
@router.post("/refresh", response_model=Token)
async def refresh_token(
refresh_token: str, user_repo: UserRepository = Depends(get_user_repository)
) -> Token:
"""
刷新 Access Token
使用 Refresh Token 获取新的 Access Token
"""
from jose import jwt, JWTError
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate refresh token",
headers={"WWW-Authenticate": "Bearer"},
)
try:
payload = jwt.decode(
refresh_token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM]
)
username: str = payload.get("sub")
token_type: str = payload.get("type")
if username is None or token_type != "refresh":
raise credentials_exception
except JWTError:
raise credentials_exception
# 验证用户仍然存在且激活
user = await user_repo.get_user_by_username(username)
if not user or not user.is_active:
raise credentials_exception
# 生成新的 Access Token
new_access_token = create_access_token(subject=user.username)
return Token(
access_token=new_access_token,
refresh_token=refresh_token, # 保持原 refresh token
token_type="bearer",
expires_in=settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60,
)
+2 -4
View File
@@ -29,10 +29,8 @@ class BurstLocationRequest(BaseModel):
normal_flow: dict[str, float] | list[dict[str, Any]] | None = Field(None, description="正常时的流量数据")
min_dpressure: float = Field(2.0, description="最小压力差(bar")
basic_pressure: float = Field(10.0, description="基准压力(bar")
scada_burst_start: datetime | None = Field(None, description="爆管/模拟方案开始时间")
scada_burst_end: datetime | None = Field(None, description="爆管/模拟方案结束时间")
scada_normal_start: datetime | None = Field(None, description="监测数据正常工况开始时间")
scada_normal_end: datetime | None = Field(None, description="监测数据正常工况结束时间")
scada_burst_start: datetime | None = Field(None, description="SCADA爆管开始时间")
scada_burst_end: datetime | None = Field(None, description="SCADA爆管结束时间")
use_scada_flow: bool = Field(False, description="是否使用SCADA流量数据")
scheme_name: str | None = Field(None, description="方案名称")
simulation_scheme_name: str | None = Field(None, description="模拟方案名称")
-29
View File
@@ -1,29 +0,0 @@
from typing import Any
from fastapi import APIRouter, HTTPException, status
from app.services.geocoding import (
TiandituGeocodeRequest,
TiandituGeocodingAPIError,
TiandituGeocodingConfigError,
geocode_tianditu,
)
router = APIRouter()
@router.post(
"/tianditu/geocode",
summary="Tianditu Geocoding",
description="调用天地图地理编码服务,将结构化地址转换为经纬度",
)
async def tianditu_geocode(request: TiandituGeocodeRequest) -> dict[str, Any]:
try:
return await geocode_tianditu(request)
except TiandituGeocodingConfigError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=str(exc),
) from exc
except TiandituGeocodingAPIError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
+18 -22
View File
@@ -1,6 +1,5 @@
import logging
from fastapi import APIRouter, Depends, HTTPException, status, Query, Path
import psycopg
from psycopg import AsyncConnection
from sqlalchemy import text
from sqlalchemy.exc import SQLAlchemyError
@@ -16,6 +15,7 @@ from app.auth.project_dependencies import (
from app.auth.metadata_dependencies import get_current_metadata_user
from app.core.config import settings
from app.domain.schemas.metadata import (
GeoServerConfigResponse,
ProjectMetaResponse,
ProjectSummaryResponse,
)
@@ -33,22 +33,34 @@ async def get_project_metadata(
"""
获取项目元数据
返回当前项目的完整元数据
返回当前项目的完整元数据,包括项目基本信息和GeoServer配置
"""
project = await metadata_repo.get_project_by_id(ctx.project_id)
if not project:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project not found"
)
geoserver = await metadata_repo.get_geoserver_config(ctx.project_id)
geoserver_payload = (
GeoServerConfigResponse(
gs_base_url=geoserver.gs_base_url,
gs_admin_user=geoserver.gs_admin_user,
gs_datastore_name=geoserver.gs_datastore_name,
default_extent=geoserver.default_extent,
srid=geoserver.srid,
)
if geoserver
else None
)
return ProjectMetaResponse(
project_id=project.id,
name=project.name,
code=project.code,
description=project.description,
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
status=project.status,
project_role=ctx.project_role,
geoserver=geoserver_payload,
)
@@ -98,23 +110,7 @@ async def project_db_health(
检查PostgreSQL和TimescaleDB数据库的连接状态
"""
try:
await pg_session.execute(text("SELECT 1"))
except SQLAlchemyError as exc:
logger.error("Project PostgreSQL health check failed", exc_info=True)
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Project PostgreSQL health check failed: {exc}",
) from exc
try:
async with ts_conn.cursor() as cur:
await cur.execute("SELECT 1")
except psycopg.Error as exc:
logger.error("Project TimescaleDB health check failed", exc_info=True)
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Project TimescaleDB health check failed: {exc}",
) from exc
await pg_session.execute(text("SELECT 1"))
async with ts_conn.cursor() as cur:
await cur.execute("SELECT 1")
return {"postgres": "ok", "timescale": "ok"}
+1 -2
View File
@@ -28,8 +28,7 @@ async def fastapi_get_json():
)
@router.get("/sensor-placement-schemes", summary="获取所有传感器位置", description="获取网络中所有传感器的放置位置信息")
@router.get("/getallsensorplacements/", summary="获取所有传感器位置(旧路径)", description="获取网络中所有传感器的放置位置信息", deprecated=True)
@router.get("/getallsensorplacements/", summary="获取所有传感器位置", description="获取网络中所有传感器的放置位置信息")
async def fastapi_get_all_sensor_placements(network: str = Query(..., description="管网名称(或数据库名称)")) -> list[dict[Any, Any]]:
"""
获取所有传感器位置
+2 -2
View File
@@ -10,7 +10,7 @@ from app.services.tjnetwork import (
get_network_node_coords,
get_node_coord,
)
from app.auth.metadata_dependencies import get_current_metadata_user
from app.auth.dependencies import get_current_user as verify_token
from app.infra.cache.redis_client import redis_client, encode_datetime, decode_datetime
import msgpack
@@ -64,7 +64,7 @@ async def fastapi_get_network_in_extent(
@router.get(
"/getnetworkgeometries/",
dependencies=[Depends(get_current_metadata_user)],
dependencies=[Depends(verify_token)],
summary="获取完整网络几何信息",
description="获取整个水网的所有节点、管线和SCADA点的几何信息(需要身份验证)"
)
+16 -7
View File
@@ -4,7 +4,7 @@ from fastapi.responses import PlainTextResponse
from typing import Any, Dict, List
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
from app.auth.project_dependencies import get_metadata_repository
from app.domain.schemas.metadata import ProjectMetaResponse
from app.domain.schemas.metadata import ProjectMetaResponse, GeoServerConfigResponse
import app.services.project_info as project_info
from app.infra.db.postgresql.database import get_database_instance as get_pg_db
from app.infra.db.timescaledb.database import get_database_instance as get_ts_db
@@ -42,20 +42,29 @@ inpDir = "data/" # Assuming data directory exists or is defined somewhere.
router = APIRouter()
lockedPrjs: Dict[str, str] = {}
@router.get("/project-info", summary="获取项目信息", description="从数据库获取项目的详细信息,包括地图范围等。", response_model=ProjectMetaResponse)
@router.get("/project_info/", summary="获取项目信息(旧路径)", description="从数据库获取项目的详细信息,包括地图范围等。", response_model=ProjectMetaResponse, deprecated=True)
@router.get("/project_info/", summary="获取项目信息", description="从数据库获取项目的详细信息,包括地图范围等。", response_model=ProjectMetaResponse)
async def get_project_info_endpoint(
network: str = Query(..., description="管网名称(或项目代码)"),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
):
"""
获取项目信息
- **network**: 管网名称(或项目代码)
"""
project_detail = await metadata_repo.get_project_detail_by_code(network)
if not project_detail:
raise HTTPException(status_code=404, detail=f"Project {network} not found")
geoserver_payload = None
if project_detail.geoserver:
geoserver_payload = GeoServerConfigResponse(
gs_base_url=project_detail.geoserver.gs_base_url,
gs_admin_user=project_detail.geoserver.gs_admin_user,
gs_datastore_name=project_detail.geoserver.gs_datastore_name,
default_extent=project_detail.geoserver.default_extent,
srid=project_detail.geoserver.srid,
)
return ProjectMetaResponse(
project_id=project_detail.project_id,
@@ -65,7 +74,8 @@ async def get_project_info_endpoint(
gs_workspace=project_detail.gs_workspace,
map_extent=project_detail.map_extent,
status=project_detail.status,
project_role="viewer",
project_role="viewer", # Default role for public access
geoserver=geoserver_payload
)
@router.get("/listprojects/", summary="获取项目列表", description="获取服务器上所有可用的供水管网项目名称列表。")
@@ -123,8 +133,7 @@ async def is_project_open_endpoint(
"""
return is_project_open(network)
@router.post("/projects/open", summary="打开项目", description="将指定项目加载到内存中,并初始化数据库连接池。")
@router.post("/openproject/", summary="打开项目(旧路径)", description="将指定项目加载到内存中,并初始化数据库连接池。", deprecated=True)
@router.post("/openproject/", summary="打开项目", description="将指定项目加载到内存中,并初始化数据库连接池。")
async def open_project_endpoint(
network: str = Query(..., description="管网名称(或数据库名称)")
):
+5 -2
View File
@@ -1,9 +1,10 @@
from fastapi import APIRouter, Depends, HTTPException, Path, Query
from psycopg import AsyncConnection
from app.infra.db.postgresql.scada import ScadaInfoRepository
import app.native.wndb as wndb
from app.infra.db.postgresql.scheme import SchemeRepository
from app.auth.project_dependencies import get_project_pg_connection
from app.services import project_info
router = APIRouter()
@@ -25,7 +26,9 @@ async def get_scada_info_with_connection(
返回项目中所有的SCADA设备信息
"""
try:
scada_data = await ScadaInfoRepository.get_scadas(conn)
_ = conn
network_name = project_info.name
scada_data = wndb.get_all_scada_info(network_name) if network_name else []
return {"success": True, "data": scada_data, "count": len(scada_data)}
except Exception as e:
raise HTTPException(
+1 -2
View File
@@ -22,8 +22,7 @@ async def fastapi_get_scheme(network: str = Query(..., description="管网名称
"""
return get_scheme(network, schema_name)
@router.get("/schemes", summary="获取所有方案", description="获取指定网络的所有方案信息")
@router.get("/getallschemes/", summary="获取所有方案(旧路径)", description="获取指定网络的所有方案信息", deprecated=True)
@router.get("/getallschemes/", summary="获取所有方案", description="获取指定网络的所有方案信息")
async def fastapi_get_all_schemes(network: str = Query(..., description="管网名称(或数据库名称)")) -> list[dict[Any, Any]]:
"""
获取所有方案列表
+39 -59
View File
@@ -4,9 +4,8 @@ import json
import os
import shutil
import threading
from fastapi import APIRouter, Depends, HTTPException, File, UploadFile, Query, Path, Body
from fastapi import APIRouter, HTTPException, File, UploadFile, Query, Path, Body
from fastapi.responses import PlainTextResponse
from app.auth.keycloak_dependencies import get_current_keycloak_username
import app.services.simulation as simulation
import app.services.globals as globals
from app.services.tjnetwork import (
@@ -36,26 +35,16 @@ from app.services.simulation_ops import (
daily_scheduling_simulation,
)
from app.services.valve_isolation import analyze_valve_isolation
from app.services.time_api import (
parse_aware_time,
parse_clock_duration_seconds,
parse_utc_time,
)
from pydantic import BaseModel, Field, field_validator
from pydantic import BaseModel, Field
router = APIRouter()
class RunSimulationManuallyByDate(BaseModel):
name: str = Field(..., description="管网名称(或数据库名称)")
start_time: str = Field(..., description="开始时间 (ISO 8601 / RFC3339,必须显式带时区)")
duration: int = Field(..., gt=0, description="持续时间 (分钟)")
@field_validator("start_time")
@classmethod
def validate_start_time_timezone(cls, value: str) -> str:
parse_aware_time(value, field_name="start_time")
return value
simulation_date: str = Field(..., description="模拟基准日期 (YYYY-MM-DD)")
start_time: str = Field(..., description="开始时间 (HH:MM 或 HH:MM:SS)")
duration: int = Field(..., description="持续时间 (分钟)")
class BurstAnalysis(BaseModel):
@@ -120,25 +109,30 @@ class PressureSensorPlacement(BaseModel):
def run_simulation_manually_by_date(
network_name: str, start_time: datetime, duration: int
network_name: str, base_date: datetime, start_time: str, duration: int
) -> None:
end_datetime = start_time + timedelta(minutes=duration)
time_properties = simulation.get_time(network_name)
hydraulic_step_seconds = parse_clock_duration_seconds(
time_properties["HYDRAULIC TIMESTEP"],
field_name="HYDRAULIC TIMESTEP",
time_parts = list(map(int, start_time.split(":")))
if len(time_parts) == 2:
start_hour, start_minute = time_parts
start_second = 0
elif len(time_parts) == 3:
start_hour, start_minute, start_second = time_parts
else:
raise ValueError("Invalid start_time format. Use HH:MM or HH:MM:SS")
start_datetime = base_date.replace(
hour=start_hour, minute=start_minute, second=start_second
)
if hydraulic_step_seconds <= 0:
raise ValueError("HYDRAULIC TIMESTEP must be greater than 0.")
hydraulic_step = timedelta(seconds=hydraulic_step_seconds)
current_time = start_time
end_datetime = start_datetime + timedelta(minutes=duration)
current_time = start_datetime
while current_time < end_datetime:
iso_time = current_time.strftime("%Y-%m-%dT%H:%M:%S") + "+08:00"
simulation.run_simulation(
name=network_name,
simulation_type="realtime",
modify_pattern_start_time=current_time.isoformat(timespec="seconds"),
modify_pattern_start_time=iso_time,
)
current_time += hydraulic_step
current_time += timedelta(minutes=15)
# 必须用这个PlainTextResponse,不然每个key都有引号
@@ -201,8 +195,7 @@ async def dump_output_endpoint(output: str = Query(..., description="模拟输
# Analysis Endpoints
@router.get("/burst-analysis", summary="爆管分析(高级)", description="高级版本的爆管分析,支持在指定时间点修改泵控制模式和阀门开度,以分析这些改变对爆管影响的作用。支持固定泵和变速泵的独立控制。")
@router.get("/burst_analysis/", summary="爆管分析(高级,旧路径)", description="高级版本的爆管分析,支持在指定时间点修改泵控制模式和阀门开度,以分析这些改变对爆管影响的作用。支持固定泵和变速泵的独立控制。", deprecated=True)
@router.get("/burst_analysis/", summary="爆管分析(高级)", description="高级版本的爆管分析,支持在指定时间点修改泵控制模式和阀门开度,以分析这些改变对爆管影响的作用。支持固定泵和变速泵的独立控制。")
async def fastapi_burst_analysis(
network: str = Query(..., description="管网名称(或数据库名称)"),
modify_pattern_start_time: str = Query(..., description="模式修改开始时间(ISO 8601格式)"),
@@ -210,7 +203,6 @@ async def fastapi_burst_analysis(
burst_size: list[float] = Query(..., description="对应各爆管点的爆管流量大小列表(L/s)"),
modify_total_duration: int = Query(..., description="模拟总时长(秒)"),
scheme_name: str = Query(..., description="分析方案名称"),
username: str = Depends(get_current_keycloak_username),
) -> str:
"""
爆管分析(高级版本)
@@ -231,7 +223,6 @@ async def fastapi_burst_analysis(
burst_size=burst_size,
modify_total_duration=modify_total_duration,
scheme_name=scheme_name,
username=username,
)
return "success"
@@ -242,7 +233,6 @@ async def fastapi_valve_close_analysis(
start_time: str = Query(..., description="阀门关闭开始时间(ISO 8601格式)"),
valves: List[str] = Query(..., description="要关闭的阀门ID列表"),
duration: int | None = Query(None, description="模拟持续时间(秒),默认900秒"),
scheme_name: str = Query(..., description="阀门关闭方案名称"),
) -> str:
"""
阀门关闭分析(高级版本)
@@ -251,7 +241,6 @@ async def fastapi_valve_close_analysis(
- **start_time**: 阀门关闭开始时间
- **valves**: 要关闭的阀门ID列表
- **duration**: 模拟持续时间(秒,可选,默认900)
- **scheme_name**: 阀门关闭方案名称
支持同时关闭多个阀门进行分析。
"""
@@ -260,13 +249,11 @@ async def fastapi_valve_close_analysis(
modify_pattern_start_time=start_time,
modify_total_duration=duration or 900,
modify_valve_opening={valve_id: 0.0 for valve_id in valves},
scheme_name=scheme_name,
)
return result or "success"
@router.get("/valve-isolation-analysis", summary="阀门隔离分析", description="分析当发生突发事件时,通过关闭指定阀门进行隔离,确定哪些阀门必须关闭、哪些可选关闭,以及隔离的可行性。")
@router.get("/valve_isolation_analysis/", summary="阀门隔离分析(旧路径)", description="分析当发生突发事件时,通过关闭指定阀门进行隔离,确定哪些阀门必须关闭、哪些可选关闭,以及隔离的可行性。", deprecated=True)
@router.get("/valve_isolation_analysis/", summary="阀门隔离分析", description="分析当发生突发事件时,通过关闭指定阀门进行隔离,确定哪些阀门必须关闭、哪些可选关闭,以及隔离的可行性。")
async def valve_isolation_endpoint(
network: str = Query(..., description="管网名称(或数据库名称)"),
accident_element: List[str] = Query(..., description="发生事故的管段/节点ID列表"),
@@ -306,8 +293,7 @@ async def valve_isolation_endpoint(
return result
@router.get("/flushing-analysis", response_class=PlainTextResponse, summary="冲洗分析(高级)", description="高级版本的冲洗分析,支持同时开启多个阀门进行冲洗,指定排污节点,并设置固定的冲洗流量。返回纯文本格式的分析结果。")
@router.get("/flushing_analysis/", response_class=PlainTextResponse, summary="冲洗分析(高级,旧路径)", description="高级版本的冲洗分析,支持同时开启多个阀门进行冲洗,指定排污节点,并设置固定的冲洗流量。返回纯文本格式的分析结果。", deprecated=True)
@router.get("/flushing_analysis/", response_class=PlainTextResponse, summary="冲洗分析(高级)", description="高级版本的冲洗分析,支持同时开启多个阀门进行冲洗,指定排污节点,并设置固定的冲洗流量。返回纯文本格式的分析结果。")
async def fastapi_flushing_analysis(
network: str = Query(..., description="管网名称(或数据库名称)"),
start_time: str = Query(..., description="冲洗开始时间(ISO 8601格式)"),
@@ -316,8 +302,7 @@ async def fastapi_flushing_analysis(
drainage_node_ID: str = Query(..., description="排污节点ID"),
flush_flow: float = Query(0, description="冲洗流量(L/s),0表示自动计算"),
duration: int | None = Query(None, description="模拟持续时间(秒),默认900秒"),
scheme_name: str = Query(..., description="冲洗方案名称"),
username: str = Depends(get_current_keycloak_username),
scheme_name: str | None = Query(None, description="冲洗方案名称(可选)"),
) -> str:
"""
冲洗分析(高级版本)
@@ -329,7 +314,7 @@ async def fastapi_flushing_analysis(
- **drainage_node_ID**: 排污节点ID
- **flush_flow**: 冲洗流量(L/s
- **duration**: 模拟持续时间(秒,可选,默认900)
- **scheme_name**: 冲洗方案名称
- **scheme_name**: 冲洗方案名称(可选)
支持多阀联合冲洗操作。
"""
@@ -344,22 +329,19 @@ async def fastapi_flushing_analysis(
drainage_node_ID=drainage_node_ID,
flushing_flow=flush_flow,
scheme_name=scheme_name,
username=username,
)
return result or "success"
@router.get("/contaminant-simulation", response_class=PlainTextResponse, summary="污染物模拟", description="对管网中的污染物扩散进行模拟,评估污染源对管网的影响范围和浓度分布。支持指定污染源位置、污染浓度和扩散模式。")
@router.get("/contaminant_simulation/", response_class=PlainTextResponse, summary="污染物模拟(旧路径)", description="对管网中的污染物扩散进行模拟,评估污染源对管网的影响范围和浓度分布。支持指定污染源位置、污染浓度和扩散模式。", deprecated=True)
@router.get("/contaminant_simulation/", response_class=PlainTextResponse, summary="污染物模拟", description="对管网中的污染物扩散进行模拟,评估污染源对管网的影响范围和浓度分布。支持指定污染源位置、污染浓度和扩散模式。")
async def fastapi_contaminant_simulation(
network: str = Query(..., description="管网名称(或数据库名称)"),
start_time: str = Query(..., description="污染开始时间(ISO 8601格式)"),
source: str = Query(..., description="污染源节点ID"),
concentration: float = Query(..., description="污染浓度(mg/L"),
duration: int = Query(..., description="模拟持续时间(秒)"),
scheme_name: str = Query(..., description="模拟方案名称"),
scheme_name: str | None = Query(None, description="模拟方案名称(可选)"),
pattern: str | None = Query(None, description="污染源模式ID(可选)"),
username: str = Depends(get_current_keycloak_username),
) -> str:
"""
污染物模拟
@@ -369,7 +351,7 @@ async def fastapi_contaminant_simulation(
- **source**: 污染源节点ID
- **concentration**: 污染浓度(mg/L
- **duration**: 模拟持续时间(秒)
- **scheme_name**: 模拟方案名称
- **scheme_name**: 模拟方案名称(可选)
- **pattern**: 污染源模式ID(可选)
用于评估管网中污染物的传播和影响范围。
@@ -382,7 +364,6 @@ async def fastapi_contaminant_simulation(
source=source,
concentration=concentration,
source_pattern=pattern,
username=username,
)
return result or "success"
@@ -738,8 +719,7 @@ async def fastapi_pressure_sensor_placement_kmeans(
)
@router.post("/sensor-placement-schemes", summary="传感器放置方案创建", description="创建新的传感器放置方案,支持灵敏度分析和KMeans聚类两种方法。根据指定的方法自动计算最优的传感器放置位置。")
@router.post("/sensorplacementscheme/create", summary="传感器放置方案创建(旧路径)", description="创建新的传感器放置方案,支持灵敏度分析和KMeans聚类两种方法。根据指定的方法自动计算最优的传感器放置位置。", deprecated=True)
@router.post("/sensorplacementscheme/create", summary="传感器放置方案创建", description="创建新的传感器放置方案,支持灵敏度分析和KMeans聚类两种方法。根据指定的方法自动计算最优的传感器放置位置。")
async def fastapi_pressure_sensor_placement(
network: str = Query(..., description="管网名称(或数据库名称)"),
scheme_name: str = Query(..., description="放置方案名称"),
@@ -787,8 +767,7 @@ async def fastapi_pressure_sensor_placement(
return "success"
@router.post("/simulations/run-by-date", summary="手动运行日期指定模拟", description="根据指定的开始时间和持续时间,手动运行水力模拟。开始时间必须是显式带时区的 ISO 8601 / RFC3339 时间")
@router.post("/runsimulationmanuallybydate/", summary="手动运行日期指定模拟(旧路径)", description="根据指定的开始时间和持续时间,手动运行水力模拟。开始时间必须是显式带时区的 ISO 8601 / RFC3339 时间。", deprecated=True)
@router.post("/runsimulationmanuallybydate/", summary="手动运行日期指定模拟", description="根据指定的日期、开始时间和持续时间,手动运行水力模拟。系统将自动查询管网参数并执行模拟")
async def fastapi_run_simulation_manually_by_date(
data: RunSimulationManuallyByDate = Body(..., description="模拟运行参数"),
) -> dict[str, str]:
@@ -797,13 +776,14 @@ async def fastapi_run_simulation_manually_by_date(
请求体参数:
- **name**: 管网名称(或数据库名称)
- **start_time**: 开始时间(ISO 8601 / RFC3339,必须显式带时区
- **simulation_date**: 模拟基准日期(YYYY-MM-DD格式
- **start_time**: 开始时间(HH:MM或HH:MM:SS格式)
- **duration**: 模拟持续时间(分钟)
系统将从指定时间开始,按15分钟间隔多次运行模拟。
系统将从指定日期和时间开始,按15分钟间隔多次运行模拟。
每次模拟间隔15分钟,直至达到指定的总持续时间。
"""
item = data.model_dump()
item = data.dict()
try:
simulation.query_corresponding_element_id_and_query_id(item["name"])
simulation.query_corresponding_pattern_id_and_query_id(item["name"])
@@ -830,10 +810,10 @@ async def fastapi_run_simulation_manually_by_date(
globals.source_outflow_region_id,
globals.realtime_region_pipe_flow_and_demand_id,
)
start_time = parse_utc_time(item["start_time"], field_name="start_time")
base_date = datetime.strptime(item["simulation_date"], "%Y-%m-%d")
run_simulation_manually_by_date(
item["name"], start_time, item["duration"]
item["name"], base_date, item["start_time"], item["duration"]
)
return {"status": "success"}
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
return {"status": "error", "message": str(exc)}
+215
View File
@@ -0,0 +1,215 @@
"""
用户管理 API 接口
演示权限控制的使用
"""
from typing import List
from fastapi import APIRouter, Depends, HTTPException, status, Path, Query
from app.domain.schemas.user import UserResponse, UserUpdate, UserCreate
from app.domain.models.role import UserRole
from app.domain.schemas.user import UserInDB
from app.infra.db.metadb.repositories.user_repository import UserRepository
from app.auth.dependencies import get_user_repository, get_current_active_user
from app.auth.permissions import get_current_admin, require_role, check_resource_owner
router = APIRouter()
@router.get(
"/",
summary="列出所有用户",
description="获取用户列表(仅管理员)",
response_model=List[UserResponse],
)
async def list_users(
skip: int = Query(0, ge=0, description="跳过的用户数"),
limit: int = Query(100, ge=1, le=1000, description="返回的最大用户数"),
current_user: UserInDB = Depends(require_role(UserRole.ADMIN)),
user_repo: UserRepository = Depends(get_user_repository),
) -> List[UserResponse]:
"""
获取用户列表
获取系统中所有的用户信息(需要管理员权限)
"""
users = await user_repo.get_all_users(skip=skip, limit=limit)
return [UserResponse.model_validate(user) for user in users]
@router.get(
"/{user_id}",
summary="获取用户详情",
description="获取指定用户的详细信息",
response_model=UserResponse,
)
async def get_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_active_user),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
获取用户详情
管理员可查看所有用户,普通用户只能查看自己
"""
# 检查权限
if not check_resource_owner(user_id, current_user):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="You don't have permission to view this user",
)
user = await user_repo.get_user_by_id(user_id)
if not user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return UserResponse.model_validate(user)
@router.put(
"/{user_id}",
summary="更新用户信息",
description="更新指定用户的信息",
response_model=UserResponse,
)
async def update_user(
user_id: int = Path(..., gt=0, description="用户ID"),
user_update: UserUpdate = None,
current_user: UserInDB = Depends(get_current_active_user),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
更新用户信息
管理员可更新所有用户,普通用户只能更新自己(且不能修改角色)
"""
# 检查用户是否存在
target_user = await user_repo.get_user_by_id(user_id)
if not target_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
# 权限检查
is_owner = current_user.id == user_id
is_admin = UserRole(current_user.role).has_permission(UserRole.ADMIN)
if not is_owner and not is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="You don't have permission to update this user",
)
# 非管理员不能修改角色和激活状态
if not is_admin:
if user_update.role is not None:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only admins can change user roles",
)
if user_update.is_active is not None:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only admins can change user active status",
)
# 更新用户
updated_user = await user_repo.update_user(user_id, user_update)
if not updated_user:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to update user",
)
return UserResponse.model_validate(updated_user)
@router.delete("/{user_id}", summary="删除用户", description="删除指定用户(仅管理员)")
async def delete_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_admin),
user_repo: UserRepository = Depends(get_user_repository),
) -> dict:
"""
删除用户
删除指定用户(需要管理员权限,不能删除自己)
"""
# 不能删除自己
if current_user.id == user_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="You cannot delete your own account",
)
success = await user_repo.delete_user(user_id)
if not success:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return {"message": "User deleted successfully"}
@router.post(
"/{user_id}/activate",
summary="激活用户",
description="激活指定用户账户(仅管理员)",
response_model=UserResponse,
)
async def activate_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_admin),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
激活用户
激活指定用户的账户(需要管理员权限)
"""
user_update = UserUpdate(is_active=True)
updated_user = await user_repo.update_user(user_id, user_update)
if not updated_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return UserResponse.model_validate(updated_user)
@router.post(
"/{user_id}/deactivate",
summary="停用用户",
description="停用指定用户账户(仅管理员)",
response_model=UserResponse,
)
async def deactivate_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_admin),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
停用用户
停用指定用户的账户(需要管理员权限,不能停用自己)
"""
# 不能停用自己
if current_user.id == user_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="You cannot deactivate your own account",
)
user_update = UserUpdate(is_active=False)
updated_user = await user_repo.update_user(user_id, user_update)
if not updated_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return UserResponse.model_validate(updated_user)
-29
View File
@@ -1,29 +0,0 @@
from typing import Any
from fastapi import APIRouter, HTTPException, status
from app.services.web_search import (
BochaSearchAPIError,
BochaSearchConfigError,
WebSearchRequest,
search_bocha_web,
)
router = APIRouter()
@router.post(
"/web-search",
summary="Web Search",
description="调用 Bocha Web Search API 获取实时网页搜索结果",
)
async def web_search(request: WebSearchRequest) -> dict[str, Any]:
try:
return await search_bocha_web(request)
except BochaSearchConfigError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=str(exc),
) from exc
except BochaSearchAPIError as exc:
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
+7 -9
View File
@@ -1,12 +1,12 @@
from fastapi import APIRouter
from app.api.v1.endpoints import (
admin_metadata,
agent_auth,
auth,
project,
simulation,
scada,
extension,
snapshots,
# data_query,
users,
schemes,
misc,
@@ -15,10 +15,9 @@ from app.api.v1.endpoints import (
leakage,
burst_detection,
burst_location,
user_management, # 新增:用户管理
audit, # 新增:审计日志
meta,
web_search,
geocoding,
)
from app.api.v1.endpoints.network import (
general,
@@ -53,10 +52,10 @@ from app.api.v1.endpoints.timeseries import (
api_router = APIRouter()
# Core Services
api_router.include_router(agent_auth.router, tags=["Agent Auth"])
api_router.include_router(auth.router, prefix="/auth", tags=["Auth"])
api_router.include_router(
admin_metadata.router, prefix="/admin", tags=["Metadata Admin"]
)
user_management.router, prefix="/users", tags=["User Management"]
) # 新增
api_router.include_router(audit.router, prefix="/audit", tags=["Audit Logs"]) # 新增
api_router.include_router(meta.router, tags=["Metadata"])
api_router.include_router(project.router, tags=["Project"])
@@ -86,6 +85,7 @@ api_router.include_router(visuals.router, tags=["Visuals"])
# Simulation & Data
api_router.include_router(simulation.router, tags=["Simulation Control"])
# api_router.include_router(data_query.router, tags=["Data Query & InfluxDB"])
api_router.include_router(scada.router)
api_router.include_router(snapshots.router, tags=["Snapshots"])
api_router.include_router(users.router, tags=["Users"])
@@ -93,8 +93,6 @@ api_router.include_router(schemes.router, tags=["Schemes"])
api_router.include_router(misc.router, tags=["Misc"])
api_router.include_router(risk.router, tags=["Risk"])
api_router.include_router(cache.router, tags=["Cache"])
api_router.include_router(web_search.router, tags=["Web Search"])
api_router.include_router(geocoding.router, tags=["Geocoding"])
api_router.include_router(leakage.router, prefix="/leakage", tags=["Leakage"])
api_router.include_router(
burst_detection.router, prefix="/burst-detection", tags=["Burst Detection"]
+100
View File
@@ -0,0 +1,100 @@
from typing import Annotated, Optional
from fastapi import Depends, HTTPException, status, Request
from fastapi.security import OAuth2PasswordBearer
from jose import jwt, JWTError
from app.core.config import settings
from app.domain.schemas.user import UserInDB, TokenPayload
from app.infra.db.metadb.repositories.user_repository import UserRepository
from app.infra.db.postgresql.database import Database
oauth2_scheme = OAuth2PasswordBearer(tokenUrl=f"{settings.API_V1_STR}/auth/login")
# 数据库依赖
async def get_db(request: Request) -> Database:
"""
获取数据库实例
从 FastAPI app.state 中获取在启动时初始化的数据库连接
"""
if not hasattr(request.app.state, "db"):
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Database not initialized",
)
return request.app.state.db
async def get_user_repository(db: Database = Depends(get_db)) -> UserRepository:
"""获取用户仓储实例"""
return UserRepository(db)
async def get_current_user(
token: str = Depends(oauth2_scheme),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserInDB:
"""
获取当前登录用户
从 JWT Token 中解析用户信息,并从数据库验证
"""
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate credentials",
headers={"WWW-Authenticate": "Bearer"},
)
try:
payload = jwt.decode(
token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM]
)
username: str = payload.get("sub")
token_type: str = payload.get("type", "access")
if username is None:
raise credentials_exception
if token_type != "access":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token type. Access token required.",
headers={"WWW-Authenticate": "Bearer"},
)
except JWTError:
raise credentials_exception
# 从数据库获取用户
user = await user_repo.get_user_by_username(username)
if user is None:
raise credentials_exception
return user
async def get_current_active_user(
current_user: UserInDB = Depends(get_current_user),
) -> UserInDB:
"""
获取当前活跃用户(必须是激活状态)
"""
if not current_user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user"
)
return current_user
async def get_current_superuser(
current_user: UserInDB = Depends(get_current_user),
) -> UserInDB:
"""
获取当前超级管理员用户
"""
if not current_user.is_superuser:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Not enough privileges. Superuser access required.",
)
return current_user
+43 -26
View File
@@ -8,41 +8,35 @@ from jose import JWTError, jwt
from app.core.config import settings
oauth2_optional = OAuth2PasswordBearer(
tokenUrl="keycloak", auto_error=False
tokenUrl=f"{settings.API_V1_STR}/auth/login", auto_error=False
)
# logger = logging.getLogger(__name__)
def _decode_keycloak_token(token: str) -> dict:
if not settings.KEYCLOAK_PUBLIC_KEY:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Keycloak public key is not configured",
)
key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
return jwt.decode(
token,
key,
algorithms=[settings.KEYCLOAK_ALGORITHM],
audience=settings.KEYCLOAK_AUDIENCE or None,
)
async def get_current_keycloak_payload(
async def get_current_keycloak_sub(
token: str | None = Depends(oauth2_optional),
) -> dict:
) -> UUID:
if not token:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Not authenticated",
headers={"WWW-Authenticate": "Bearer"},
)
if settings.KEYCLOAK_PUBLIC_KEY:
key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
algorithms = [settings.KEYCLOAK_ALGORITHM]
else:
key = settings.SECRET_KEY
algorithms = [settings.ALGORITHM]
try:
return _decode_keycloak_token(token)
payload = jwt.decode(
token,
key,
algorithms=algorithms,
audience=settings.KEYCLOAK_AUDIENCE or None,
)
except JWTError as exc:
# logger.warning("Keycloak token validation failed: %s", exc)
raise HTTPException(
@@ -51,10 +45,6 @@ async def get_current_keycloak_payload(
headers={"WWW-Authenticate": "Bearer"},
) from exc
async def get_current_keycloak_sub(
payload: dict = Depends(get_current_keycloak_payload),
) -> UUID:
sub = payload.get("sub")
if not sub:
raise HTTPException(
@@ -74,8 +64,35 @@ async def get_current_keycloak_sub(
async def get_current_keycloak_username(
payload: dict = Depends(get_current_keycloak_payload),
token: str | None = Depends(oauth2_optional),
) -> str:
if not token:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Not authenticated",
headers={"WWW-Authenticate": "Bearer"},
)
if settings.KEYCLOAK_PUBLIC_KEY:
key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
algorithms = [settings.KEYCLOAK_ALGORITHM]
else:
key = settings.SECRET_KEY
algorithms = [settings.ALGORITHM]
try:
payload = jwt.decode(
token,
key,
algorithms=algorithms,
audience=settings.KEYCLOAK_AUDIENCE or None,
)
except JWTError as exc:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token",
headers={"WWW-Authenticate": "Bearer"},
) from exc
username = payload.get("preferred_username") or payload.get("username")
if not username:
raise HTTPException(
+3 -47
View File
@@ -6,7 +6,8 @@ from fastapi import Depends, HTTPException, status
from sqlalchemy.exc import SQLAlchemyError
from sqlalchemy.ext.asyncio import AsyncSession
from app.auth.keycloak_dependencies import get_current_keycloak_payload
from app.auth.keycloak_dependencies import get_current_keycloak_sub
from app.core.config import settings
from app.infra.db.metadb.database import get_metadata_session
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
@@ -19,40 +20,10 @@ async def get_metadata_repository(
return MetadataRepository(session)
def _keycloak_sub_from_payload(payload: dict) -> UUID:
sub = payload.get("sub")
if not sub:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Missing subject claim",
headers={"WWW-Authenticate": "Bearer"},
)
try:
return UUID(str(sub))
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid subject claim",
headers={"WWW-Authenticate": "Bearer"},
) from exc
def _username_from_payload(payload: dict) -> str | None:
username = payload.get("preferred_username") or payload.get("username")
return str(username) if username else None
def _email_from_payload(payload: dict) -> str | None:
email = payload.get("email")
return str(email) if email else None
async def get_current_metadata_user(
keycloak_payload: dict = Depends(get_current_keycloak_payload),
keycloak_sub: UUID = Depends(get_current_keycloak_sub),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
):
keycloak_sub = _keycloak_sub_from_payload(keycloak_payload)
try:
user = await metadata_repo.get_user_by_keycloak_id(keycloak_sub)
except SQLAlchemyError as exc:
@@ -68,21 +39,6 @@ async def get_current_metadata_user(
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user"
)
try:
user = await metadata_repo.refresh_user_keycloak_snapshot(
user,
username=_username_from_payload(keycloak_payload),
email=_email_from_payload(keycloak_payload),
)
except SQLAlchemyError as exc:
logger.error(
"Metadata DB error while refreshing current user snapshot",
exc_info=True,
)
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
return user
+106
View File
@@ -0,0 +1,106 @@
"""
权限控制依赖项和装饰器
基于角色的访问控制(RBAC
"""
from typing import Callable
from fastapi import Depends, HTTPException, status
from app.domain.models.role import UserRole
from app.domain.schemas.user import UserInDB
from app.auth.dependencies import get_current_active_user
def require_role(required_role: UserRole):
"""
要求特定角色或更高权限
用法:
@router.get("/admin-only")
async def admin_endpoint(user: UserInDB = Depends(require_role(UserRole.ADMIN))):
...
Args:
required_role: 需要的最低角色
Returns:
依赖函数
"""
async def role_checker(
current_user: UserInDB = Depends(get_current_active_user)
) -> UserInDB:
user_role = UserRole(current_user.role)
if not user_role.has_permission(required_role):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f"Insufficient permissions. Required role: {required_role.value}, "
f"Your role: {user_role.value}"
)
return current_user
return role_checker
# 预定义的权限检查依赖
require_admin = require_role(UserRole.ADMIN)
require_operator = require_role(UserRole.OPERATOR)
require_user = require_role(UserRole.USER)
def get_current_admin(
current_user: UserInDB = Depends(require_admin)
) -> UserInDB:
"""
获取当前管理员用户
等同于 Depends(require_role(UserRole.ADMIN))
"""
return current_user
def get_current_operator(
current_user: UserInDB = Depends(require_operator)
) -> UserInDB:
"""
获取当前操作员用户(或更高权限)
等同于 Depends(require_role(UserRole.OPERATOR))
"""
return current_user
def check_resource_owner(user_id: int, current_user: UserInDB) -> bool:
"""
检查是否是资源拥有者或管理员
Args:
user_id: 资源拥有者ID
current_user: 当前用户
Returns:
是否有权限
"""
# 管理员可以访问所有资源
if UserRole(current_user.role).has_permission(UserRole.ADMIN):
return True
# 检查是否是资源拥有者
return current_user.id == user_id
def require_owner_or_admin(user_id: int):
"""
要求是资源拥有者或管理员
Args:
user_id: 资源拥有者ID
Returns:
依赖函数
"""
async def owner_or_admin_checker(
current_user: UserInDB = Depends(get_current_active_user)
) -> UserInDB:
if not check_resource_owner(user_id, current_user):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="You don't have permission to access this resource"
)
return current_user
return owner_or_admin_checker
-2
View File
@@ -25,7 +25,6 @@ logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class ProjectContext:
project_id: UUID
project_code: str
user_id: UUID
project_role: str
@@ -86,7 +85,6 @@ async def get_project_context(
return ProjectContext(
project_id=project.id,
project_code=project.code,
user_id=user.id,
project_role=membership_role,
)
+17 -13
View File
@@ -11,8 +11,17 @@ class Settings(BaseSettings):
NETWORK_NAME: str = "default_network"
# 敏感配置加密密钥 (Fernet)
DATABASE_ENCRYPTION_KEY: str = ""
# JWT 配置
SECRET_KEY: str = (
"your-secret-key-here-change-in-production-use-openssl-rand-hex-32"
)
ALGORITHM: str = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES: int = 30
REFRESH_TOKEN_EXPIRE_DAYS: int = 7
# 数据加密密钥 (使用 Fernet)
ENCRYPTION_KEY: str = "" # 必须从环境变量设置
DATABASE_ENCRYPTION_KEY: str = "" # project_databases.dsn_encrypted 专用
# Database Config (PostgreSQL)
DB_NAME: str = "tjwater"
@@ -27,6 +36,11 @@ class Settings(BaseSettings):
TIMESCALEDB_DB_PORT: str = "5433"
TIMESCALEDB_DB_USER: str = "postgres"
TIMESCALEDB_DB_PASSWORD: str = "password"
# InfluxDB
INFLUXDB_URL: str = "http://localhost:8086"
INFLUXDB_TOKEN: str = "token"
INFLUXDB_ORG: str = "org"
INFLUXDB_BUCKET: str = "bucket"
# Metadata Database Config (PostgreSQL)
METADATA_DB_NAME: str = "system_hub"
@@ -45,21 +59,11 @@ class Settings(BaseSettings):
PROJECT_TS_POOL_MIN_SIZE: int = 1
PROJECT_TS_POOL_MAX_SIZE: int = 10
# Keycloak access token verification
# Keycloak JWT (optional override)
KEYCLOAK_PUBLIC_KEY: str = ""
KEYCLOAK_ALGORITHM: str = "RS256"
KEYCLOAK_AUDIENCE: str = ""
# Bocha Web Search API
BOCHA_API_KEY: str = ""
BOCHA_WEB_SEARCH_URL: str = "https://api.bochaai.com/v1/web-search"
BOCHA_WEB_SEARCH_TIMEOUT_SECONDS: float = 30.0
# Tianditu Geocoding API
TIANDITU_GEOCODER_TOKEN: str = ""
TIANDITU_GEOCODER_URL: str = "https://api.tianditu.gov.cn/geocoder"
TIANDITU_GEOCODER_TIMEOUT_SECONDS: float = 30.0
@property
def SQLALCHEMY_DATABASE_URI(self) -> str:
db_password = quote_plus(self.DB_PASSWORD)
+7 -3
View File
@@ -20,10 +20,10 @@ class Encryptor:
key: 加密密钥,如果为 None 则从环境变量读取
"""
if key is None:
key_str = os.getenv("DATABASE_ENCRYPTION_KEY") or settings.DATABASE_ENCRYPTION_KEY
key_str = os.getenv("ENCRYPTION_KEY") or settings.ENCRYPTION_KEY
if not key_str:
raise ValueError(
"DATABASE_ENCRYPTION_KEY not found in environment variables or .env. "
"ENCRYPTION_KEY not found in environment variables or .env. "
"Generate one using: Encryptor.generate_key()"
)
key = key_str.encode()
@@ -80,13 +80,15 @@ _database_encryptor: Optional[Encryptor] = None
def is_encryption_configured() -> bool:
return is_database_encryption_configured()
return bool(os.getenv("ENCRYPTION_KEY") or settings.ENCRYPTION_KEY)
def is_database_encryption_configured() -> bool:
return bool(
os.getenv("DATABASE_ENCRYPTION_KEY")
or settings.DATABASE_ENCRYPTION_KEY
or os.getenv("ENCRYPTION_KEY")
or settings.ENCRYPTION_KEY
)
@@ -105,6 +107,8 @@ def get_database_encryptor() -> Encryptor:
key_str = (
os.getenv("DATABASE_ENCRYPTION_KEY")
or settings.DATABASE_ENCRYPTION_KEY
or os.getenv("ENCRYPTION_KEY")
or settings.ENCRYPTION_KEY
)
if not key_str:
raise ValueError(
+95
View File
@@ -0,0 +1,95 @@
from datetime import datetime, timedelta, timezone
from typing import Optional, Union, Any
from jose import jwt
from passlib.context import CryptContext
from app.core.config import settings
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
def _utc_now() -> datetime:
return datetime.now(timezone.utc)
def create_access_token(
subject: Union[str, Any], expires_delta: Optional[timedelta] = None
) -> str:
"""
创建 JWT Access Token
Args:
subject: 用户标识(通常是用户名或用户ID)
expires_delta: 过期时间增量
Returns:
JWT token 字符串
"""
if expires_delta:
expire = _utc_now() + expires_delta
else:
expire = _utc_now() + timedelta(
minutes=settings.ACCESS_TOKEN_EXPIRE_MINUTES
)
to_encode = {
"exp": expire,
"sub": str(subject),
"type": "access",
"iat": _utc_now(),
}
encoded_jwt = jwt.encode(
to_encode, settings.SECRET_KEY, algorithm=settings.ALGORITHM
)
return encoded_jwt
def create_refresh_token(subject: Union[str, Any]) -> str:
"""
创建 JWT Refresh Token(长期有效)
Args:
subject: 用户标识
Returns:
JWT refresh token 字符串
"""
expire = _utc_now() + timedelta(days=settings.REFRESH_TOKEN_EXPIRE_DAYS)
to_encode = {
"exp": expire,
"sub": str(subject),
"type": "refresh",
"iat": _utc_now(),
}
encoded_jwt = jwt.encode(
to_encode, settings.SECRET_KEY, algorithm=settings.ALGORITHM
)
return encoded_jwt
def verify_password(plain_password: str, hashed_password: str) -> bool:
"""
验证密码
Args:
plain_password: 明文密码
hashed_password: 密码哈希
Returns:
是否匹配
"""
return pwd_context.verify(plain_password, hashed_password)
def get_password_hash(password: str) -> str:
"""
生成密码哈希
Args:
password: 明文密码
Returns:
bcrypt 哈希字符串
"""
return pwd_context.hash(password)
+1 -3
View File
@@ -1,9 +1,7 @@
"""
This module is reserved for future implementation of advanced cryptographic operations.
Current Fernet encryption helpers are implemented in `app.core.encryption`.
Login credentials are owned by Keycloak; this backend does not hash or store
local passwords.
Current basic encryption (Fernet) and password hashing are implemented in `app.core.encryption` and `app.core.security`.
Future expansion may include:
- Asymmetric encryption (RSA/ECC) for secure communication
- Key management and rotation services
+36
View File
@@ -0,0 +1,36 @@
from enum import Enum
class UserRole(str, Enum):
"""用户角色枚举"""
ADMIN = "ADMIN" # 管理员 - 完全权限
OPERATOR = "OPERATOR" # 操作员 - 可修改数据
USER = "USER" # 普通用户 - 读写权限
VIEWER = "VIEWER" # 观察者 - 仅查询权限
def __str__(self):
return self.value
@classmethod
def get_hierarchy(cls) -> dict:
"""
获取角色层级(数字越大权限越高)
"""
return {
cls.VIEWER: 1,
cls.USER: 2,
cls.OPERATOR: 3,
cls.ADMIN: 4,
}
def has_permission(self, required_role: 'UserRole') -> bool:
"""
检查当前角色是否有足够权限
Args:
required_role: 需要的最低角色
Returns:
True if has permission
"""
hierarchy = self.get_hierarchy()
return hierarchy[self] >= hierarchy[required_role]
-134
View File
@@ -1,134 +0,0 @@
from datetime import datetime
from typing import Literal
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, model_validator
BusinessRole = Literal["admin", "user", "operator", "viewer"]
ProjectRole = Literal["owner", "admin", "member", "viewer"]
ProjectStatus = Literal["active", "inactive", "archived"]
ProjectDbRole = Literal["biz_data", "iot_data"]
class MetadataUserSyncRequest(BaseModel):
keycloak_id: UUID
username: str = Field(..., min_length=1, max_length=50)
email: str = Field(..., min_length=1, max_length=100)
role: BusinessRole = "user"
is_active: bool = True
class MetadataUsersBatchSyncRequest(BaseModel):
users: list[MetadataUserSyncRequest] = Field(..., min_length=1, max_length=500)
class MetadataUserUpdateRequest(BaseModel):
role: BusinessRole | None = None
is_active: bool | None = None
class MetadataUserResponse(BaseModel):
id: UUID
keycloak_id: UUID
username: str
email: str
role: str
is_active: bool
is_superuser: bool
created_at: datetime
updated_at: datetime
last_login_at: datetime | None = None
model_config = ConfigDict(from_attributes=True)
class MetadataUserSyncResult(BaseModel):
keycloak_id: UUID
user: MetadataUserResponse | None = None
success: bool
error: str | None = None
class ProjectMemberCreateRequest(BaseModel):
user_id: UUID
project_role: ProjectRole = "viewer"
class ProjectMemberUpdateRequest(BaseModel):
project_role: ProjectRole
class ProjectMemberResponse(BaseModel):
id: UUID
user_id: UUID
project_id: UUID
project_role: str
username: str
email: str
is_active: bool
class AdminProjectCreateRequest(BaseModel):
name: str = Field(..., min_length=1, max_length=100)
code: str = Field(..., min_length=1, max_length=50)
description: str | None = None
gs_workspace: str = Field(..., min_length=1, max_length=100)
map_extent: dict | None = None
status: ProjectStatus = "active"
class AdminProjectUpdateRequest(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=100)
code: str | None = Field(default=None, min_length=1, max_length=50)
description: str | None = None
gs_workspace: str | None = Field(default=None, min_length=1, max_length=100)
map_extent: dict | None = None
status: ProjectStatus | None = None
class AdminProjectResponse(BaseModel):
project_id: UUID
name: str
code: str
description: str | None = None
gs_workspace: str
map_extent: dict | None = None
status: str
created_at: datetime
updated_at: datetime
class ProjectDatabaseUpsertRequest(BaseModel):
db_role: ProjectDbRole
dsn: str | None = Field(default=None, min_length=1)
pool_min_size: int = Field(default=2, ge=1)
pool_max_size: int = Field(default=10, ge=1)
@model_validator(mode="after")
def validate_pool_bounds(self):
if self.pool_max_size < self.pool_min_size:
raise ValueError("pool_max_size must be greater than or equal to pool_min_size")
return self
class ProjectDatabaseResponse(BaseModel):
id: UUID
project_id: UUID
db_role: str
db_type: str
pool_min_size: int
pool_max_size: int
has_dsn: bool
class ProjectDatabaseHealthRequest(BaseModel):
dsn: str | None = Field(default=None, min_length=1)
class ProjectDatabaseHealthResponse(BaseModel):
project_id: UUID
db_role: str
db_type: str
ok: bool
detail: str
+12 -3
View File
@@ -4,22 +4,31 @@ from uuid import UUID
from pydantic import BaseModel
class GeoServerConfigResponse(BaseModel):
gs_base_url: Optional[str]
gs_admin_user: Optional[str]
gs_datastore_name: str
default_extent: Optional[dict]
srid: int
class ProjectMetaResponse(BaseModel):
project_id: UUID
name: str
code: str
description: Optional[str] = None
description: Optional[str]
gs_workspace: str
map_extent: Optional[dict] = None
map_extent: Optional[dict]
status: str
project_role: str
geoserver: Optional[GeoServerConfigResponse]
class ProjectSummaryResponse(BaseModel):
project_id: UUID
name: str
code: str
description: Optional[str] = None
description: Optional[str]
gs_workspace: str
status: str
project_role: str
+68
View File
@@ -0,0 +1,68 @@
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, EmailStr, Field, ConfigDict
from app.domain.models.role import UserRole
# ============================================
# Request Schemas (输入)
# ============================================
class UserCreate(BaseModel):
"""用户注册"""
username: str = Field(..., min_length=3, max_length=50,
description="用户名,3-50个字符")
email: EmailStr = Field(..., description="邮箱地址")
password: str = Field(..., min_length=6, max_length=100,
description="密码,至少6个字符")
role: UserRole = Field(default=UserRole.USER, description="用户角色")
class UserLogin(BaseModel):
"""用户登录"""
username: str = Field(..., description="用户名或邮箱")
password: str = Field(..., description="密码")
class UserUpdate(BaseModel):
"""用户信息更新"""
email: Optional[EmailStr] = None
password: Optional[str] = Field(None, min_length=6, max_length=100)
role: Optional[UserRole] = None
is_active: Optional[bool] = None
# ============================================
# Response Schemas (输出)
# ============================================
class UserResponse(BaseModel):
"""用户信息响应(不含密码)"""
id: int
username: str
email: str
role: UserRole
is_active: bool
is_superuser: bool
created_at: datetime
updated_at: datetime
model_config = ConfigDict(from_attributes=True)
class UserInDB(UserResponse):
"""数据库中的用户(含密码哈希)"""
hashed_password: str
# ============================================
# Token Schemas
# ============================================
class Token(BaseModel):
"""JWT Token 响应"""
access_token: str
refresh_token: Optional[str] = None
token_type: str = "bearer"
expires_in: int = Field(..., description="过期时间(秒)")
class TokenPayload(BaseModel):
"""JWT Token Payload"""
sub: str = Field(..., description="用户ID或用户名")
exp: Optional[int] = None
iat: Optional[int] = None
type: str = Field(default="access", description="token类型: access 或 refresh")
+14 -6
View File
@@ -33,6 +33,8 @@ class AuditMiddleware(BaseHTTPMiddleware):
# 需要审计的路径前缀
AUDIT_PATHS = [
# "/api/v1/auth/",
# "/api/v1/users/",
# "/api/v1/projects/",
# "/api/v1/networks/",
]
@@ -191,14 +193,20 @@ class AuditMiddleware(BaseHTTPMiddleware):
return None
sub = None
try:
if not settings.KEYCLOAK_PUBLIC_KEY:
return None
key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
key = (
settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
if settings.KEYCLOAK_PUBLIC_KEY
else settings.SECRET_KEY
)
algorithms = (
[settings.KEYCLOAK_ALGORITHM]
if settings.KEYCLOAK_PUBLIC_KEY
else [settings.ALGORITHM]
)
payload = jwt.decode(
token,
key,
algorithms=[settings.KEYCLOAK_ALGORITHM],
algorithms=algorithms,
audience=settings.KEYCLOAK_AUDIENCE or None,
)
sub = payload.get("sub")
@@ -213,7 +221,7 @@ class AuditMiddleware(BaseHTTPMiddleware):
keycloak_id = UUID(sub)
user = await repo.get_user_by_keycloak_id(keycloak_id)
except ValueError:
return None
user = await repo.get_user_by_username(sub)
if user and user.is_active:
return user.id
return None
+2 -2
View File
@@ -54,9 +54,9 @@ class ProjectConnectionManager:
def _normalize_pg_url(self, url: str) -> str:
parsed = make_url(url)
if parsed.drivername in {"postgresql", "postgres"}:
if parsed.drivername == "postgresql":
parsed = parsed.set(drivername="postgresql+psycopg")
return parsed.render_as_string(hide_password=False)
return str(parsed)
async def get_pg_sessionmaker(
self,
View File
File diff suppressed because it is too large Load Diff
+6
View File
@@ -0,0 +1,6 @@
# influxdb数据库连接信息
url = "http://127.0.0.1:8086" # 替换为你的InfluxDB实例地址
token = "kMPX2V5HsbzPpUT2B9HPBu1sTG1Emf-lPlT2UjxYnGAuocpXq_f_0lK4HHs-TbbKyjsZpICkMsyXG_V2D7P7yQ==" # 替换为你的InfluxDB Token
# _ENCODED_TOKEN = "eEdETTVSWnFSSkF1ekFHUy1vdFhVZEMyTkZkWTc1cUpBalJMcUFCNHA1V2NJSUFsSVVwT3BUOF95QTE2QU9IbUpXZXJ3UV8wOGd3Yjg0c3k0MmpuWlE9PQ=="
# token = base64.b64decode(_ENCODED_TOKEN).decode("utf-8")
org = "TJWATERORG" # 替换为你的Organization名称
+33
View File
@@ -0,0 +1,33 @@
from influxdb_client import InfluxDBClient, Point, WriteOptions
from influxdb_client.client.query_api import QueryApi
import influxdb_info
# 配置 InfluxDB 连接
url = influxdb_info.url
token = influxdb_info.token
org = influxdb_info.org
bucket = "SCADA_data"
# 创建 InfluxDB 客户端
client = InfluxDBClient(url=url, token=token, org=org)
# 创建查询 API 对象
query_api = client.query_api()
# 构建查询语句
query = f'''
from(bucket: "{bucket}")
|> range(start: -1h)
'''
# 执行查询
result = query_api.query(query)
print(result)
# 处理查询结果
for table in result:
for record in table.records:
print(f"Time: {record.get_time()}, Value: {record.get_value()}, Measurement: {record.get_measurement()}, Field: {record.get_field()}")
# 关闭客户端连接
client.close()
+20
View File
@@ -64,6 +64,26 @@ class ProjectDatabase(Base):
pool_max_size: Mapped[int] = mapped_column(Integer, default=10)
class ProjectGeoServerConfig(Base):
__tablename__ = "project_geoserver_configs"
id: Mapped[UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True)
project_id: Mapped[UUID] = mapped_column(
PGUUID(as_uuid=True), unique=True, index=True
)
gs_base_url: Mapped[str | None] = mapped_column(Text, nullable=True)
gs_admin_user: Mapped[str | None] = mapped_column(String(50), nullable=True)
gs_admin_password_encrypted: Mapped[str | None] = mapped_column(
Text, nullable=True
)
gs_datastore_name: Mapped[str] = mapped_column(String(100), default="ds_postgis")
default_extent: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
srid: Mapped[int] = mapped_column(Integer, default=4326)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.utcnow
)
class UserProjectMembership(Base):
__tablename__ = "user_project_membership"
@@ -1,15 +1,16 @@
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Optional, List
from uuid import UUID, uuid4
from uuid import UUID
from cryptography.fernet import InvalidToken
from sqlalchemy import delete, select
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.encryption import (
get_database_encryptor,
get_encryptor,
is_database_encryption_configured,
is_encryption_configured,
)
from app.infra.db.metadb import models
@@ -42,6 +43,17 @@ class ProjectDbRouting:
pool_max_size: int
@dataclass(frozen=True)
class ProjectGeoServerInfo:
project_id: UUID
gs_base_url: Optional[str]
gs_admin_user: Optional[str]
gs_admin_password: Optional[str]
gs_datastore_name: str
default_extent: Optional[dict]
srid: int
@dataclass(frozen=True)
class ProjectSummary:
project_id: UUID
@@ -63,27 +75,7 @@ class ProjectDetail:
gs_workspace: str
map_extent: Optional[dict]
status: str
@dataclass(frozen=True)
class ProjectMemberSummary:
id: UUID
user_id: UUID
project_id: UUID
project_role: str
username: str
email: str
is_active: bool
def _utcnow() -> datetime:
return datetime.now(timezone.utc)
def _encrypt_database_secret(value: str) -> str:
if not is_database_encryption_configured():
raise ValueError("DATABASE_ENCRYPTION_KEY is not configured")
return get_database_encryptor().encrypt(value)
geoserver: Optional[ProjectGeoServerInfo]
class MetadataRepository:
@@ -104,86 +96,6 @@ class MetadataRepository:
)
return result.scalar_one_or_none()
async def get_user_by_id(self, user_id: UUID) -> Optional[models.User]:
result = await self.session.execute(
select(models.User).where(models.User.id == user_id)
)
return result.scalar_one_or_none()
async def list_users(self, skip: int = 0, limit: int = 100) -> List[models.User]:
result = await self.session.execute(
select(models.User)
.order_by(models.User.created_at.desc())
.offset(skip)
.limit(limit)
)
return list(result.scalars().all())
async def upsert_user_from_keycloak(
self,
*,
keycloak_id: UUID,
username: str,
email: str,
role: str,
is_active: bool,
) -> models.User:
user = await self.get_user_by_keycloak_id(keycloak_id)
if user is None:
user = models.User(
id=uuid4(),
keycloak_id=keycloak_id,
username=username,
email=email,
role=role,
is_active=is_active,
is_superuser=False,
)
self.session.add(user)
else:
user.username = username
user.email = email
user.role = role
user.is_active = is_active
await self.session.commit()
await self.session.refresh(user)
return user
async def refresh_user_keycloak_snapshot(
self,
user: models.User,
*,
username: str | None,
email: str | None,
last_login_at: datetime | None = None,
) -> models.User:
if username:
user.username = username
if email:
user.email = email
user.last_login_at = last_login_at or _utcnow()
user.updated_at = _utcnow()
await self.session.commit()
await self.session.refresh(user)
return user
async def update_user_admin(
self,
user_id: UUID,
*,
updates: dict,
) -> Optional[models.User]:
user = await self.get_user_by_id(user_id)
if user is None:
return None
if "role" in updates:
user.role = updates["role"]
if "is_active" in updates:
user.is_active = updates["is_active"]
await self.session.commit()
await self.session.refresh(user)
return user
async def get_project_by_id(self, project_id: UUID) -> Optional[models.Project]:
result = await self.session.execute(
select(models.Project).where(models.Project.id == project_id)
@@ -196,66 +108,13 @@ class MetadataRepository:
)
return result.scalar_one_or_none()
async def list_project_records(self) -> List[models.Project]:
result = await self.session.execute(
select(models.Project).order_by(models.Project.name)
)
return list(result.scalars().all())
async def create_project(
self,
*,
name: str,
code: str,
description: str | None,
gs_workspace: str,
map_extent: dict | None,
status: str,
) -> models.Project:
project = models.Project(
id=uuid4(),
name=name,
code=code,
description=description,
gs_workspace=gs_workspace,
map_extent=map_extent,
status=status,
created_at=_utcnow(),
updated_at=_utcnow(),
)
self.session.add(project)
await self.session.commit()
await self.session.refresh(project)
return project
async def update_project(
self,
project_id: UUID,
*,
updates: dict,
) -> Optional[models.Project]:
project = await self.get_project_by_id(project_id)
if project is None:
return None
for field in (
"name",
"code",
"description",
"gs_workspace",
"map_extent",
"status",
):
if field in updates:
setattr(project, field, updates[field])
project.updated_at = _utcnow()
await self.session.commit()
await self.session.refresh(project)
return project
async def get_project_detail_by_code(self, code: str) -> Optional[ProjectDetail]:
project = await self.get_project_by_code(code)
if not project:
return None
geoserver = await self.get_geoserver_config(project.id)
return ProjectDetail(
project_id=project.id,
name=project.name,
@@ -264,6 +123,7 @@ class MetadataRepository:
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
status=project.status,
geoserver=geoserver
)
async def get_membership_role(
@@ -277,142 +137,6 @@ class MetadataRepository:
)
return result.scalar_one_or_none()
async def list_project_members(
self, project_id: UUID
) -> List[ProjectMemberSummary]:
stmt = (
select(models.UserProjectMembership, models.User)
.join(models.User, models.User.id == models.UserProjectMembership.user_id)
.where(models.UserProjectMembership.project_id == project_id)
.order_by(models.User.username)
)
result = await self.session.execute(stmt)
return [
ProjectMemberSummary(
id=membership.id,
user_id=membership.user_id,
project_id=membership.project_id,
project_role=membership.project_role,
username=user.username,
email=user.email,
is_active=user.is_active,
)
for membership, user in result.all()
]
async def get_project_membership(
self, project_id: UUID, user_id: UUID
) -> Optional[models.UserProjectMembership]:
result = await self.session.execute(
select(models.UserProjectMembership).where(
models.UserProjectMembership.project_id == project_id,
models.UserProjectMembership.user_id == user_id,
)
)
return result.scalar_one_or_none()
async def add_project_member(
self, project_id: UUID, user_id: UUID, project_role: str
) -> models.UserProjectMembership:
membership = models.UserProjectMembership(
id=uuid4(),
user_id=user_id,
project_id=project_id,
project_role=project_role,
)
self.session.add(membership)
await self.session.commit()
await self.session.refresh(membership)
return membership
async def update_project_member_role(
self, project_id: UUID, user_id: UUID, project_role: str
) -> Optional[models.UserProjectMembership]:
membership = await self.get_project_membership(project_id, user_id)
if membership is None:
return None
membership.project_role = project_role
await self.session.commit()
await self.session.refresh(membership)
return membership
async def remove_project_member(self, project_id: UUID, user_id: UUID) -> bool:
result = await self.session.execute(
delete(models.UserProjectMembership).where(
models.UserProjectMembership.project_id == project_id,
models.UserProjectMembership.user_id == user_id,
)
)
await self.session.commit()
return bool(result.rowcount)
async def list_project_databases(
self, project_id: UUID
) -> List[models.ProjectDatabase]:
result = await self.session.execute(
select(models.ProjectDatabase)
.where(models.ProjectDatabase.project_id == project_id)
.order_by(models.ProjectDatabase.db_role)
)
return list(result.scalars().all())
async def get_project_database_config(
self, project_id: UUID, db_role: str
) -> Optional[models.ProjectDatabase]:
result = await self.session.execute(
select(models.ProjectDatabase).where(
models.ProjectDatabase.project_id == project_id,
models.ProjectDatabase.db_role == db_role,
)
)
return result.scalar_one_or_none()
async def upsert_project_database_config(
self,
project_id: UUID,
*,
db_role: str,
db_type: str,
dsn: str | None,
pool_min_size: int,
pool_max_size: int,
) -> models.ProjectDatabase:
record = await self.get_project_database_config(project_id, db_role)
if record is None:
if dsn is None:
raise ValueError("dsn is required when creating project database config")
record = models.ProjectDatabase(
id=uuid4(),
project_id=project_id,
db_role=db_role,
db_type=db_type,
dsn_encrypted=_encrypt_database_secret(dsn),
pool_min_size=pool_min_size,
pool_max_size=pool_max_size,
)
self.session.add(record)
else:
record.db_type = db_type
if dsn is not None:
record.dsn_encrypted = _encrypt_database_secret(dsn)
record.pool_min_size = pool_min_size
record.pool_max_size = pool_max_size
await self.session.commit()
await self.session.refresh(record)
return record
async def delete_project_database_config(
self, project_id: UUID, db_role: str
) -> bool:
result = await self.session.execute(
delete(models.ProjectDatabase).where(
models.ProjectDatabase.project_id == project_id,
models.ProjectDatabase.db_role == db_role,
)
)
await self.session.commit()
return bool(result.rowcount)
async def get_project_db_routing(
self, project_id: UUID, db_role: str
) -> Optional[ProjectDbRouting]:
@@ -445,6 +169,35 @@ class MetadataRepository:
pool_max_size=record.pool_max_size,
)
async def get_geoserver_config(
self, project_id: UUID
) -> Optional[ProjectGeoServerInfo]:
result = await self.session.execute(
select(models.ProjectGeoServerConfig).where(
models.ProjectGeoServerConfig.project_id == project_id
)
)
record = result.scalar_one_or_none()
if not record:
return None
if record.gs_admin_password_encrypted:
if is_encryption_configured():
encryptor = get_encryptor()
password = encryptor.decrypt(record.gs_admin_password_encrypted)
else:
password = record.gs_admin_password_encrypted
else:
password = None
return ProjectGeoServerInfo(
project_id=record.project_id,
gs_base_url=record.gs_base_url,
gs_admin_user=record.gs_admin_user,
gs_admin_password=password,
gs_datastore_name=record.gs_datastore_name,
default_extent=record.default_extent,
srid=record.srid,
)
async def list_projects_for_user(self, user_id: UUID) -> List[ProjectSummary]:
stmt = (
select(models.Project, models.UserProjectMembership.project_role)
@@ -0,0 +1,235 @@
from typing import Optional, List
from datetime import datetime
from app.infra.db.postgresql.database import Database
from app.domain.schemas.user import UserCreate, UserUpdate, UserInDB
from app.domain.models.role import UserRole
from app.core.security import get_password_hash
import logging
logger = logging.getLogger(__name__)
class UserRepository:
"""用户数据访问层"""
def __init__(self, db: Database):
self.db = db
async def create_user(self, user: UserCreate) -> Optional[UserInDB]:
"""
创建新用户
Args:
user: 用户创建数据
Returns:
创建的用户对象
"""
hashed_password = get_password_hash(user.password)
query = """
INSERT INTO users (username, email, hashed_password, role, is_active, is_superuser)
VALUES (%(username)s, %(email)s, %(hashed_password)s, %(role)s, TRUE, FALSE)
RETURNING id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
"""
try:
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {
'username': user.username,
'email': user.email,
'hashed_password': hashed_password,
'role': user.role.value
})
row = await cur.fetchone()
if row:
return UserInDB(**row)
except Exception as e:
logger.error(f"Error creating user: {e}")
raise
return None
async def get_user_by_id(self, user_id: int) -> Optional[UserInDB]:
"""根据ID获取用户"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
WHERE id = %(user_id)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'user_id': user_id})
row = await cur.fetchone()
if row:
return UserInDB(**row)
return None
async def get_user_by_username(self, username: str) -> Optional[UserInDB]:
"""根据用户名获取用户"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
WHERE username = %(username)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'username': username})
row = await cur.fetchone()
if row:
return UserInDB(**row)
return None
async def get_user_by_email(self, email: str) -> Optional[UserInDB]:
"""根据邮箱获取用户"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
WHERE email = %(email)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'email': email})
row = await cur.fetchone()
if row:
return UserInDB(**row)
return None
async def get_all_users(self, skip: int = 0, limit: int = 100) -> List[UserInDB]:
"""获取所有用户(分页)"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
ORDER BY created_at DESC
LIMIT %(limit)s OFFSET %(skip)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'skip': skip, 'limit': limit})
rows = await cur.fetchall()
return [UserInDB(**row) for row in rows]
async def update_user(self, user_id: int, user_update: UserUpdate) -> Optional[UserInDB]:
"""
更新用户信息
Args:
user_id: 用户ID
user_update: 更新数据
Returns:
更新后的用户对象
"""
# 构建动态更新语句
update_fields = []
params = {'user_id': user_id}
if user_update.email is not None:
update_fields.append("email = %(email)s")
params['email'] = user_update.email
if user_update.password is not None:
update_fields.append("hashed_password = %(hashed_password)s")
params['hashed_password'] = get_password_hash(user_update.password)
if user_update.role is not None:
update_fields.append("role = %(role)s")
params['role'] = user_update.role.value
if user_update.is_active is not None:
update_fields.append("is_active = %(is_active)s")
params['is_active'] = user_update.is_active
if not update_fields:
return await self.get_user_by_id(user_id)
query = f"""
UPDATE users
SET {', '.join(update_fields)}, updated_at = CURRENT_TIMESTAMP
WHERE id = %(user_id)s
RETURNING id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
"""
try:
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, params)
row = await cur.fetchone()
if row:
return UserInDB(**row)
except Exception as e:
logger.error(f"Error updating user {user_id}: {e}")
raise
return None
async def delete_user(self, user_id: int) -> bool:
"""
删除用户
Args:
user_id: 用户ID
Returns:
是否成功删除
"""
query = "DELETE FROM users WHERE id = %(user_id)s"
try:
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'user_id': user_id})
return cur.rowcount > 0
except Exception as e:
logger.error(f"Error deleting user {user_id}: {e}")
return False
async def user_exists(self, username: str = None, email: str = None) -> bool:
"""
检查用户是否存在
Args:
username: 用户名
email: 邮箱
Returns:
是否存在
"""
conditions = []
params = {}
if username:
conditions.append("username = %(username)s")
params['username'] = username
if email:
conditions.append("email = %(email)s")
params['email'] = email
if not conditions:
return False
query = f"""
SELECT EXISTS(
SELECT 1 FROM users WHERE {' OR '.join(conditions)}
)
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, params)
result = await cur.fetchone()
return result['exists'] if result else False
-51
View File
@@ -1,51 +0,0 @@
from typing import Any
from psycopg import AsyncConnection
def _optional_text(value: Any) -> str | None:
return str(value).strip() if value is not None else None
def _optional_float(value: Any) -> float | None:
return float(value) if value is not None else None
class ScadaInfoRepository:
"""Read SCADA metadata from the current project's business database."""
@staticmethod
async def get_scadas(conn: AsyncConnection) -> list[dict[str, Any]]:
async with conn.cursor() as cur:
await cur.execute(
"""
SELECT id,
type,
associated_element_id,
api_query_id,
transmission_mode,
transmission_frequency,
reliability,
x_coor,
y_coor
FROM public.scada_info
"""
)
records = await cur.fetchall()
return [
{
"id": str(record["id"]).strip(),
"type": str(record["type"]).strip().lower(),
"associated_element_id": _optional_text(
record["associated_element_id"]
),
"api_query_id": record["api_query_id"],
"transmission_mode": record["transmission_mode"],
"transmission_frequency": record["transmission_frequency"],
"reliability": _optional_float(record["reliability"]),
"x": _optional_float(record["x_coor"]),
"y": _optional_float(record["y_coor"]),
}
for record in records
]
+154 -154
View File
@@ -1,19 +1,18 @@
import time
from typing import List, Optional, Any, Dict, Tuple
from datetime import datetime, timedelta
from typing import Any, Dict, List, Optional, Tuple
import numpy as np
import pandas as pd
from psycopg import AsyncConnection
import app.native.wndb as wndb
import pandas as pd
import numpy as np
from app.algorithms.cleaning.flow import clean_flow_data_df_kf
from app.algorithms.cleaning.pressure import clean_pressure_data_df_km
from app.algorithms.health.analyzer import PipelineHealthAnalyzer
from app.infra.db.postgresql.scada import ScadaInfoRepository
import app.native.wndb as wndb
from app.infra.db.timescaledb.repositories.realtime import RealtimeRepository
from app.infra.db.timescaledb.repositories.scheme import SchemeRepository
from app.infra.db.timescaledb.repositories.scada import ScadaRepository
from app.services import project_info
class CompositeQueries:
@@ -21,13 +20,6 @@ class CompositeQueries:
复合查询类提供跨表查询功能
"""
@staticmethod
async def _get_project_scada_index(
postgres_conn: AsyncConnection,
) -> Dict[str, Dict[str, Any]]:
scadas = await ScadaInfoRepository.get_scadas(postgres_conn)
return {scada["id"]: scada for scada in scadas}
@staticmethod
async def get_scada_associated_realtime_simulation_data(
timescale_conn: AsyncConnection,
@@ -56,22 +48,31 @@ class CompositeQueries:
ValueError: SCADA 设备未找到或字段无效时
"""
result = {}
scada_by_id = await CompositeQueries._get_project_scada_index(postgres_conn)
# 1. 查询所有 SCADA 信息
network_name = project_info.name
scada_infos = wndb.get_all_scada_info(network_name) if network_name else []
for device_id in device_ids:
target_scada = scada_by_id.get(device_id)
# 2. 根据 device_id 找到对应的 SCADA 信息
target_scada = None
for scada in scada_infos:
if scada["id"] == device_id:
target_scada = scada
break
if not target_scada:
raise ValueError(f"SCADA device {device_id} not found")
# 3. 根据 type 和 associated_element_id 查询对应的模拟数据
element_id = target_scada["associated_element_id"]
scada_type = target_scada["type"]
if scada_type == "pipe_flow":
if scada_type.lower() == "pipe_flow":
# 查询 link 模拟数据
res = await RealtimeRepository.get_link_field_by_time_range(
timescale_conn, start_time, end_time, element_id, "flow"
)
elif scada_type == "pressure":
elif scada_type.lower() == "pressure":
# 查询 node 模拟数据
res = await RealtimeRepository.get_node_field_by_time_range(
timescale_conn, start_time, end_time, element_id, "pressure"
@@ -114,17 +115,26 @@ class CompositeQueries:
ValueError: SCADA 设备未找到或字段无效时
"""
result = {}
scada_by_id = await CompositeQueries._get_project_scada_index(postgres_conn)
# 1. 查询所有 SCADA 信息
network_name = project_info.name
scada_infos = wndb.get_all_scada_info(network_name) if network_name else []
for device_id in device_ids:
target_scada = scada_by_id.get(device_id)
# 2. 根据 device_id 找到对应的 SCADA 信息
target_scada = None
for scada in scada_infos:
if scada["id"] == device_id:
target_scada = scada
break
if not target_scada:
raise ValueError(f"SCADA device {device_id} not found")
# 3. 根据 type 和 associated_element_id 查询对应的模拟数据
element_id = target_scada["associated_element_id"]
scada_type = target_scada["type"]
if scada_type == "pipe_flow":
if scada_type.lower() == "pipe_flow":
# 查询 link 模拟数据
res = await SchemeRepository.get_link_field_by_scheme_and_time_range(
timescale_conn,
@@ -135,7 +145,7 @@ class CompositeQueries:
element_id,
"flow",
)
elif scada_type == "pressure":
elif scada_type.lower() == "pressure":
# 查询 node 模拟数据
res = await SchemeRepository.get_node_field_by_scheme_and_time_range(
timescale_conn,
@@ -157,19 +167,19 @@ class CompositeQueries:
@staticmethod
async def get_realtime_simulation_data(
timescale_conn: AsyncConnection,
feature_infos: List[Tuple[str, str]],
featureInfos: List[Tuple[str, str]],
start_time: datetime,
end_time: datetime,
) -> Dict[str, List[Dict[str, Any]]]:
"""
获取 link/node 模拟值
根据传入的 feature_infos找到关联的 link/node
根据传入的 featureInfos找到关联的 link/node
并根据对应的 type查询对应的模拟数据
Args:
timescale_conn: TimescaleDB 异步连接
feature_infos: 传入的 feature 信息列表包含 (element_id, type)
featureInfos: 传入的 feature 信息列表包含 (element_id, type)
start_time: 开始时间
end_time: 结束时间
@@ -180,20 +190,20 @@ class CompositeQueries:
ValueError: SCADA 设备未找到或字段无效时
"""
result = {}
for feature_id, feature_type in feature_infos:
for feature_id, type in featureInfos:
if feature_type.lower() == "pipe":
if type.lower() == "pipe":
# 查询 link 模拟数据
res = await RealtimeRepository.get_link_field_by_time_range(
timescale_conn, start_time, end_time, feature_id, "flow"
)
elif feature_type.lower() == "junction":
elif type.lower() == "junction":
# 查询 node 模拟数据
res = await RealtimeRepository.get_node_field_by_time_range(
timescale_conn, start_time, end_time, feature_id, "pressure"
)
else:
raise ValueError(f"Unknown type: {feature_type}")
raise ValueError(f"Unknown type: {type}")
# 添加 scada_id 到每个数据项
for item in res:
item["feature_id"] = feature_id
@@ -203,7 +213,7 @@ class CompositeQueries:
@staticmethod
async def get_scheme_simulation_data(
timescale_conn: AsyncConnection,
feature_infos: List[Tuple[str, str]],
featureInfos: List[Tuple[str, str]],
start_time: datetime,
end_time: datetime,
scheme_type: str,
@@ -212,12 +222,12 @@ class CompositeQueries:
"""
获取 link/node scheme 模拟值
根据传入的 feature_infos找到关联的 link/node
根据传入的 featureInfos找到关联的 link/node
并根据对应的 type查询对应的模拟数据
Args:
timescale_conn: TimescaleDB 异步连接
feature_infos: 传入的 feature 信息列表包含 (element_id, type)
featureInfos: 传入的 feature 信息列表包含 (element_id, type)
start_time: 开始时间
end_time: 结束时间
scheme_type: 工况类型
@@ -230,8 +240,8 @@ class CompositeQueries:
ValueError: 当类型无效时
"""
result = {}
for feature_id, feature_type in feature_infos:
if feature_type.lower() == "pipe":
for feature_id, type in featureInfos:
if type.lower() == "pipe":
# 查询 link 模拟数据
res = await SchemeRepository.get_link_field_by_scheme_and_time_range(
timescale_conn,
@@ -242,7 +252,7 @@ class CompositeQueries:
feature_id,
"flow",
)
elif feature_type.lower() == "junction":
elif type.lower() == "junction":
# 查询 node 模拟数据
res = await SchemeRepository.get_node_field_by_scheme_and_time_range(
timescale_conn,
@@ -254,7 +264,7 @@ class CompositeQueries:
"pressure",
)
else:
raise ValueError(f"Unknown type: {feature_type}")
raise ValueError(f"Unknown type: {type}")
# 添加 feature_id 到每个数据项
for item in res:
item["feature_id"] = feature_id
@@ -291,27 +301,33 @@ class CompositeQueries:
ValueError: 当元素类型无效时
"""
scada_by_id = await CompositeQueries._get_project_scada_index(postgres_conn)
associated_scada = next(
(
scada
for scada in scada_by_id.values()
if scada["associated_element_id"] == element_id
),
None,
)
# 1. 查询所有 SCADA 信息
network_name = project_info.name
scada_infos = wndb.get_all_scada_info(network_name) if network_name else []
# 2. 根据 element_type 和 element_id 找到关联的 SCADA 设备
associated_scada = None
for scada in scada_infos:
if scada["associated_element_id"] == element_id:
associated_scada = scada
break
if not associated_scada:
# 没有找到关联的 SCADA 设备
return None
# 3. 通过 SCADA device_id 获取监测数据
device_id = associated_scada["id"]
# 根据 use_cleaned 参数选择字段
data_field = "cleaned_value" if use_cleaned else "monitored_value"
# 保证 device_id 以列表形式传递
res = await ScadaRepository.get_scada_field_by_id_time_range(
timescale_conn, [device_id], start_time, end_time, data_field
)
# 将 device_id 替换为 element_id 返回
return {element_id: res.get(device_id, [])}
@staticmethod
@@ -335,124 +351,108 @@ class CompositeQueries:
end_time: 结束时间
Returns:
"success"
Raises:
ValueError: 当前项目没有可清洗设备或指定时间范围内没有监测数据
"success" 或错误信息
"""
scada_by_id = await CompositeQueries._get_project_scada_index(postgres_conn)
supported_types = {"pressure", "pipe_flow", "flow"}
try:
# 获取所有 SCADA 信息
network_name = project_info.name
scada_infos = wndb.get_all_scada_info(network_name) if network_name else []
# 将列表转换为字典,以 device_id 为键
scada_device_info_dict = {info["id"]: info for info in scada_infos}
if device_ids:
device_ids = [str(device_id).strip() for device_id in device_ids]
missing_metadata_ids = [
device_id
for device_id in device_ids
if device_id not in scada_by_id
]
if missing_metadata_ids:
raise ValueError(
f"当前项目中有 {len(missing_metadata_ids)} 个 SCADA 设备缺少元数据"
)
# 如果 device_ids 为空,则处理所有 SCADA 设备
if not device_ids:
device_ids = list(scada_device_info_dict.keys())
unsupported_ids = [
device_id
for device_id in device_ids
if scada_by_id[device_id]["type"] not in supported_types
]
if unsupported_ids:
raise ValueError(
f"当前项目中有 {len(unsupported_ids)} 个 SCADA 设备类型不支持清洗"
)
else:
device_ids = [
device_id
for device_id, info in scada_by_id.items()
if info["type"] in supported_types
]
if not device_ids:
raise ValueError("当前项目没有可清洗的 SCADA 设备")
data = await ScadaRepository.get_scada_field_by_id_time_range(
timescale_conn, device_ids, start_time, end_time, "monitored_value"
)
if not data:
raise ValueError("指定时间范围内没有 SCADA 监测数据")
normalized_data = {
str(device_id): records for device_id, records in data.items()
}
missing_data_ids = [
device_id for device_id in device_ids if not normalized_data.get(device_id)
]
if missing_data_ids:
raise ValueError(
f"指定时间范围内有 {len(missing_data_ids)} 个 SCADA 设备没有监测数据"
# 批量查询所有设备的数据
data = await ScadaRepository.get_scada_field_by_id_time_range(
timescale_conn, device_ids, start_time, end_time, "monitored_value"
)
all_records = [
{
"time": record["time"],
"device_id": device_id,
"value": record["value"],
}
for device_id, records in normalized_data.items()
for record in records
]
if not all_records:
raise ValueError("指定时间范围内没有 SCADA 监测数据")
if not data:
return "error: fetch none scada data" # 没有数据,直接返回
df_long = pd.DataFrame(all_records)
df = df_long.pivot(index="time", columns="device_id", values="value")
pressure_ids = [
device_id
for device_id in df.columns
if scada_by_id[device_id]["type"] == "pressure"
]
flow_ids = [
device_id
for device_id in df.columns
if scada_by_id[device_id]["type"] in {"pipe_flow", "flow"}
]
updated_rows = 0
for grouped_ids, cleaning_function in (
(pressure_ids, clean_pressure_data_df_km),
(flow_ids, clean_flow_data_df_kf),
):
if not grouped_ids:
continue
source_df = df[grouped_ids].reset_index()
cleaned_df = cleaning_function(source_df)
time_values = cleaned_df["time"].tolist()
for device_id in grouped_ids:
if device_id not in cleaned_df.columns:
raise ValueError(f"设备 {device_id} 的清洗结果缺少数据列")
cleaned_values = cleaned_df[device_id].tolist()
for time_value, value in zip(time_values, cleaned_values):
time_dt = (
time_value
if isinstance(time_value, datetime)
else datetime.fromisoformat(str(time_value))
# 将嵌套字典转换为 DataFrame,使用 time 作为索引
# data 格式: {device_id: [{"time": "...", "value": ...}, ...]}
all_records = []
for device_id, records in data.items():
for record in records:
all_records.append(
{
"time": record["time"],
"device_id": device_id,
"value": record["value"],
}
)
await ScadaRepository.update_scada_field(
timescale_conn,
time_dt,
device_id,
"cleaned_value",
value,
)
updated_rows += 1
if updated_rows == 0:
raise ValueError("SCADA 数据清洗未产生任何数据库更新")
if not all_records:
return "error: fetch none scada data" # 没有数据,直接返回
return "success"
# 创建 DataFrame 并透视,使 device_id 成为列
df_long = pd.DataFrame(all_records)
df = df_long.pivot(index="time", columns="device_id", values="value")
# 根据type分类设备
pressure_ids = [
id
for id in df.columns
if scada_device_info_dict.get(id, {}).get("type") == "pressure"
]
flow_ids = [
id
for id in df.columns
if scada_device_info_dict.get(id, {}).get("type") == "pipe_flow"
]
# 处理pressure数据
if pressure_ids:
pressure_df = df[pressure_ids]
# 重置索引,将 time 变为普通列
pressure_df = pressure_df.reset_index()
# 调用清洗方法
cleaned_df = clean_pressure_data_df_km(pressure_df)
# 将清洗后的数据写回数据库
for device_id in pressure_ids:
if device_id in cleaned_df.columns:
cleaned_values = cleaned_df[device_id].tolist()
time_values = cleaned_df["time"].tolist()
for i, time_str in enumerate(time_values):
time_dt = datetime.fromisoformat(time_str)
value = cleaned_values[i]
await ScadaRepository.update_scada_field(
timescale_conn,
time_dt,
device_id,
"cleaned_value",
value,
)
# 处理flow数据
if flow_ids:
flow_df = df[flow_ids]
# 重置索引,将 time 变为普通列
flow_df = flow_df.reset_index()
# 调用清洗方法
cleaned_df = clean_flow_data_df_kf(flow_df)
# 将清洗后的数据写回数据库
for device_id in flow_ids:
if device_id in cleaned_df.columns:
cleaned_values = cleaned_df[device_id].tolist()
time_values = cleaned_df["time"].tolist()
for i, time_str in enumerate(time_values):
time_dt = datetime.fromisoformat(time_str)
value = cleaned_values[i]
await ScadaRepository.update_scada_field(
timescale_conn,
time_dt,
device_id,
"cleaned_value",
value,
)
return "success"
except Exception as e:
return f"error: {str(e)}"
@staticmethod
async def predict_pipeline_health(
+9 -19
View File
@@ -50,7 +50,6 @@ class InternalStorage:
link_result_list: List[dict],
result_start_time: str,
num_periods: int = 1,
result_timestep_seconds: int | None = None,
db_name: str = None,
max_retries: int = 3,
):
@@ -71,7 +70,6 @@ class InternalStorage:
link_result_list,
result_start_time,
num_periods,
result_timestep_seconds,
)
break # 成功
except Exception as e:
@@ -231,14 +229,7 @@ class InternalQueries:
scheme_type: str | None = None,
scheme_name: str | None = None,
) -> dict[str, list[dict]]:
normalized_element_ids = list(
dict.fromkeys(
normalized
for normalized in (str(element_id).strip() for element_id in element_ids)
if normalized
)
)
if not normalized_element_ids:
if not element_ids:
return {}
start_dt = parse_utc_time(start_time, field_name="start_time")
@@ -262,9 +253,9 @@ class InternalQueries:
with conn.cursor(row_factory=dict_row) as cur:
if schema_name == "scheme":
query = sql.SQL(
"SELECT btrim(id::text) AS id, time, {} FROM {}.{} "
"SELECT id, time, {} FROM {}.{} "
"WHERE scheme_type = %s AND scheme_name = %s "
"AND time >= %s AND time <= %s AND btrim(id::text) = ANY(%s)"
"AND time >= %s AND time <= %s AND id = ANY(%s)"
).format(
sql.Identifier(field),
sql.Identifier(schema_name),
@@ -277,26 +268,25 @@ class InternalQueries:
scheme_name,
start_dt,
end_dt,
normalized_element_ids,
element_ids,
),
)
else:
query = sql.SQL(
"SELECT btrim(id::text) AS id, time, {} FROM {}.{} "
"WHERE time >= %s AND time <= %s AND btrim(id::text) = ANY(%s)"
"SELECT id, time, {} FROM {}.{} "
"WHERE time >= %s AND time <= %s AND id = ANY(%s)"
).format(
sql.Identifier(field),
sql.Identifier(schema_name),
sql.Identifier(table_name),
)
cur.execute(query, (start_dt, end_dt, normalized_element_ids))
cur.execute(query, (start_dt, end_dt, element_ids))
rows = cur.fetchall()
result: dict[str, list[dict]] = {
element_id: [] for element_id in normalized_element_ids
element_id: [] for element_id in element_ids
}
for row in rows:
element_id = str(row["id"]).strip()
result.setdefault(element_id, []).append(
result.setdefault(row["id"], []).append(
{"time": row["time"].isoformat(), "value": row[field]}
)
for element_id in result:
+21 -31
View File
@@ -3,24 +3,10 @@ from datetime import datetime, timedelta
from collections import defaultdict
from psycopg import AsyncConnection, Connection, sql
import app.services.globals as globals
from app.services.time_api import parse_clock_duration_seconds, parse_utc_time
from app.services.time_api import parse_utc_time
class SchemeRepository:
@staticmethod
def _get_result_timestep(result_timestep_seconds: int | None) -> timedelta:
if result_timestep_seconds is not None:
if result_timestep_seconds <= 0:
raise ValueError("result_timestep_seconds must be greater than 0.")
return timedelta(seconds=result_timestep_seconds)
timestep_seconds = parse_clock_duration_seconds(
globals.hydraulic_timestep,
field_name="HYDRAULIC TIMESTEP",
)
if timestep_seconds <= 0:
raise ValueError("HYDRAULIC TIMESTEP must be greater than 0.")
return timedelta(seconds=timestep_seconds)
# --- Link Simulation ---
@@ -466,7 +452,6 @@ class SchemeRepository:
link_result_list: List[Dict[str, any]],
result_start_time: str,
num_periods: int = 1,
result_timestep_seconds: int | None = None,
):
"""
Store scheme simulation results to TimescaleDB.
@@ -483,16 +468,20 @@ class SchemeRepository:
result_start_time, field_name="result_start_time"
)
timestep = SchemeRepository._get_result_timestep(result_timestep_seconds)
timestep_parts = globals.hydraulic_timestep.split(":")
timestep = timedelta(
hours=int(timestep_parts[0]),
minutes=int(timestep_parts[1]),
seconds=int(timestep_parts[2]),
)
# Prepare node data for batch insert
node_data = []
for node_result in node_result_list:
node_id = node_result.get("node")
result_rows = node_result.get("result", [])
for period_index in range(min(num_periods, len(result_rows))):
for period_index in range(num_periods):
current_time = simulation_time + (timestep * period_index)
data = result_rows[period_index]
data = node_result.get("result", [])[period_index]
node_data.append(
{
"time": current_time,
@@ -510,10 +499,9 @@ class SchemeRepository:
link_data = []
for link_result in link_result_list:
link_id = link_result.get("link")
result_rows = link_result.get("result", [])
for period_index in range(min(num_periods, len(result_rows))):
for period_index in range(num_periods):
current_time = simulation_time + (timestep * period_index)
data = result_rows[period_index]
data = link_result.get("result", [])[period_index]
link_data.append(
{
"time": current_time,
@@ -547,7 +535,6 @@ class SchemeRepository:
link_result_list: List[Dict[str, any]],
result_start_time: str,
num_periods: int = 1,
result_timestep_seconds: int | None = None,
):
"""
Store scheme simulation results to TimescaleDB (sync version).
@@ -564,16 +551,20 @@ class SchemeRepository:
result_start_time, field_name="result_start_time"
)
timestep = SchemeRepository._get_result_timestep(result_timestep_seconds)
timestep_parts = globals.hydraulic_timestep.split(":")
timestep = timedelta(
hours=int(timestep_parts[0]),
minutes=int(timestep_parts[1]),
seconds=int(timestep_parts[2]),
)
# Prepare node data for batch insert
node_data = []
for node_result in node_result_list:
node_id = node_result.get("node")
result_rows = node_result.get("result", [])
for period_index in range(min(num_periods, len(result_rows))):
for period_index in range(num_periods):
current_time = simulation_time + (timestep * period_index)
data = result_rows[period_index]
data = node_result.get("result", [])[period_index]
node_data.append(
{
"time": current_time,
@@ -591,10 +582,9 @@ class SchemeRepository:
link_data = []
for link_result in link_result_list:
link_id = link_result.get("link")
result_rows = link_result.get("result", [])
for period_index in range(min(num_periods, len(result_rows))):
for period_index in range(num_periods):
current_time = simulation_time + (timestep * period_index)
data = result_rows[period_index]
data = link_result.get("result", [])[period_index]
link_data.append(
{
"time": current_time,
+1 -76
View File
@@ -1,78 +1,3 @@
from collections.abc import Iterator
from contextlib import contextmanager
from threading import RLock
import psycopg as pg
from app.core.config import get_pgconn_string
g_conn_dict: dict[str, pg.Connection] = {}
_registry_lock = RLock()
_project_locks: dict[str, RLock] = {}
def _is_closed(connection: pg.Connection) -> bool:
return bool(getattr(connection, "closed", False))
def _close_connection(connection: pg.Connection) -> None:
if not _is_closed(connection):
connection.close()
def _is_healthy(connection: pg.Connection) -> bool:
if _is_closed(connection):
return False
try:
with connection.cursor() as cur:
cur.execute("SELECT 1")
except pg.Error:
return False
return True
def _get_project_lock(name: str) -> RLock:
with _registry_lock:
lock = _project_locks.get(name)
if lock is None:
lock = RLock()
_project_locks[name] = lock
return lock
def open_connection(name: str) -> pg.Connection:
with _get_project_lock(name):
connection = g_conn_dict.get(name)
if connection is None or not _is_healthy(connection):
if connection is not None:
_close_connection(connection)
connection = pg.connect(
conninfo=get_pgconn_string(db_name=name), autocommit=True
)
g_conn_dict[name] = connection
return connection
def is_connection_open(name: str) -> bool:
with _get_project_lock(name):
connection = g_conn_dict.get(name)
if connection is None:
return False
if not _is_healthy(connection):
del g_conn_dict[name]
_close_connection(connection)
return False
return True
def close_connection(name: str) -> None:
with _get_project_lock(name):
connection = g_conn_dict.pop(name, None)
if connection is not None:
_close_connection(connection)
@contextmanager
def project_connection(name: str) -> Iterator[pg.Connection]:
with _get_project_lock(name):
yield open_connection(name)
g_conn_dict : dict[str, pg.Connection] = {}
+15 -19
View File
@@ -1,6 +1,6 @@
from typing import Any
from psycopg.rows import dict_row, Row
from .connection import project_connection
from .connection import g_conn_dict as conn
API_ADD = 'add'
API_UPDATE = 'update'
@@ -83,33 +83,29 @@ class DbChangeSet:
def read(name: str, sql: str) -> Row:
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(sql)
row = cur.fetchone()
if row == None:
raise Exception(sql)
return row
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(sql)
row = cur.fetchone()
if row == None:
raise Exception(sql)
return row
def read_all(name: str, sql: str) -> list[Row]:
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(sql)
return cur.fetchall()
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(sql)
return cur.fetchall()
def try_read(name: str, sql: str) -> Row | None:
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(sql)
return cur.fetchone()
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(sql)
return cur.fetchone()
def write(name: str, sql: str) -> None:
with project_connection(name) as conn:
with conn.cursor() as cur:
cur.execute(sql)
with conn[name].cursor() as cur:
cur.execute(sql)
def get_current_operation(name: str) -> int:
+12 -9
View File
@@ -2,11 +2,7 @@ import os
import psycopg as pg
from psycopg import sql
from psycopg.rows import dict_row
from .connection import (
close_connection,
is_connection_open,
open_connection,
)
from .connection import g_conn_dict as conn
from app.core.config import get_pgconn_string, get_pg_config, get_pg_password
# no undo/redo
@@ -35,7 +31,9 @@ def have_project(name: str) -> bool:
def copy_project(source: str, new: str) -> None:
close_connection(source)
if source in conn:
conn[source].close()
del conn[source]
with pg.connect(
conninfo=get_pgconn_string(db_name="postgres"), autocommit=True
@@ -178,12 +176,17 @@ def clean_project(excluded: list[str] = []) -> None:
def open_project(name: str) -> None:
open_connection(name)
if name not in conn:
conn[name] = pg.connect(
conninfo=get_pgconn_string(db_name=name), autocommit=True
)
def is_project_open(name: str) -> bool:
return is_connection_open(name)
return name in conn
def close_project(name: str) -> None:
close_connection(name)
if name in conn:
conn[name].close()
del conn[name]
+43 -51
View File
@@ -1,5 +1,5 @@
from psycopg.rows import dict_row, Row
from .connection import project_connection
from .connection import g_conn_dict as conn
from .database import read
from typing import Any
@@ -47,10 +47,9 @@ ELEMENT_TYPES : dict[str, int] = {
}
def _get_from(name: str, id: str, base_type: str) -> Row | None:
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select * from {base_type} where id = '{id}'")
return cur.fetchone()
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select * from {base_type} where id = '{id}'")
return cur.fetchone()
def is_node(name: str, id: str) -> bool:
@@ -126,11 +125,10 @@ def is_region(name: str, id: str) -> bool:
def _get_all(name: str, base_type: str) -> list[str]:
ids : list[str] = []
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from {base_type} order by id")
for record in cur:
ids.append(record['id'])
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from {base_type} order by id")
for record in cur:
ids.append(record['id'])
return ids
@@ -140,32 +138,29 @@ def get_nodes(name: str) -> list[str]:
# DingZQ
def _get_nodes_by_type(name: str, type: str) -> list[str]:
ids : list[str] = []
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from {_NODE} where type = '{type}' order by id")
for record in cur:
ids.append(record['id'])
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from {_NODE} where type = '{type}' order by id")
for record in cur:
ids.append(record['id'])
return ids
# DingZQ
def get_nodes_id_and_type(name: str) -> dict[str, str]:
nodes_id_and_type: dict[str, str] = {}
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select id, type from {_NODE} order by id")
for record in cur:
nodes_id_and_type[record['id']] = record['type']
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select id, type from {_NODE} order by id")
for record in cur:
nodes_id_and_type[record['id']] = record['type']
return nodes_id_and_type
# DingZQ 2024-12-31
def get_major_nodes(name: str, diameter: int) -> list[str]:
major_nodes_set = set()
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select node1, node2 from pipes where diameter > {diameter}")
for record in cur:
major_nodes_set.add(record['node1'])
major_nodes_set.add(record['node2'])
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select node1, node2 from pipes where diameter > {diameter}")
for record in cur:
major_nodes_set.add(record['node1'])
major_nodes_set.add(record['node2'])
return list(major_nodes_set)
@@ -188,32 +183,29 @@ def get_links(name: str) -> list[str]:
# DingZQ
def _get_links_by_type(name: str, type: str) -> list[str]:
ids : list[str] = []
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from {_LINK} where type = '{type}' order by id")
for record in cur:
ids.append(record['id'])
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from {_LINK} where type = '{type}' order by id")
for record in cur:
ids.append(record['id'])
return ids
# DingZQ
def get_links_id_and_type(name: str) -> dict[str, str]:
links_id_and_type: dict[str, str] = {}
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select id, type from {_LINK} order by id")
for record in cur:
links_id_and_type[record['id']] = record['type']
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select id, type from {_LINK} order by id")
for record in cur:
links_id_and_type[record['id']] = record['type']
return links_id_and_type
# DingZQ 2024-12-31
# 获取直径大于800的管道
def get_major_pipes(name: str, diameter: int) -> list[str]:
major_pipe_ids: list[str] = []
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from pipes where diameter > {diameter} order by id")
for record in cur:
major_pipe_ids.append(record['id'])
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from pipes where diameter > {diameter} order by id")
for record in cur:
major_pipe_ids.append(record['id'])
return major_pipe_ids
# DingZQ
@@ -240,16 +232,15 @@ def get_regions(name: str) -> list[str]:
return _get_all(name, _REGION)
def get_node_links(name: str, id: str) -> list[str]:
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
links: list[str] = []
for p in cur.execute(f"select id from pipes where node1 = '{id}' or node2 = '{id}'").fetchall():
links.append(p['id'])
for p in cur.execute(f"select id from pumps where node1 = '{id}' or node2 = '{id}'").fetchall():
links.append(p['id'])
for p in cur.execute(f"select id from valves where node1 = '{id}' or node2 = '{id}'").fetchall():
links.append(p['id'])
return links
with conn[name].cursor(row_factory=dict_row) as cur:
links: list[str] = []
for p in cur.execute(f"select id from pipes where node1 = '{id}' or node2 = '{id}'").fetchall():
links.append(p['id'])
for p in cur.execute(f"select id from pumps where node1 = '{id}' or node2 = '{id}'").fetchall():
links.append(p['id'])
for p in cur.execute(f"select id from valves where node1 = '{id}' or node2 = '{id}'").fetchall():
links.append(p['id'])
return links
def get_link_nodes(name: str, id: str) -> list[str]:
@@ -268,3 +259,4 @@ def get_region_type(name: str, id: str)->str:
return type
+4 -7
View File
@@ -1,7 +1,5 @@
from .database import *
from .connection import project_connection
from .s0_base import get_link_nodes
from psycopg.rows import dict_row
def sql_update_coord(node: str, x: float, y: float) -> str:
coord = f"st_geomfromtext('point({x} {y})')"
@@ -51,11 +49,10 @@ def get_links_in_extent(name: str, x1: float, y1: float, x2: float, y2: float) -
node_ids = set([s.split(':')[0] for s in get_nodes_in_extent(name, x1, y1, x2, y2)])
all_link_ids = []
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from pipes")
for record in cur:
all_link_ids.append(record['id'])
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select id from pipes")
for record in cur:
all_link_ids.append(record['id'])
links = []
for link_id in all_link_ids:
+30 -35
View File
@@ -1,7 +1,5 @@
from .database import *
from .connection import project_connection
from .s0_base import *
from psycopg.rows import dict_row
import json
def get_pipe_risk_probability_now(name: str, pipe_id: str) -> dict[str, Any]:
@@ -30,31 +28,29 @@ def get_pipe_risk_probability(name: str, pipe_id: str) -> dict[str, Any]:
def get_network_pipe_risk_probability_now(name: str) -> list[dict[str, Any]]:
pipe_risk_probability_list = []
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select * from pipe_risk_probability")
for record in cur:
#pipe_risk_probability_list.append(record)
t = {}
t['pipeid'] = record['pipeid']
t['pipeage'] = record['pipeage']
t['risk_probability_now'] = record['risk_probability_now']
pipe_risk_probability_list.append(t)
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select * from pipe_risk_probability")
for record in cur:
#pipe_risk_probability_list.append(record)
t = {}
t['pipeid'] = record['pipeid']
t['pipeage'] = record['pipeage']
t['risk_probability_now'] = record['risk_probability_now']
pipe_risk_probability_list.append(t)
return pipe_risk_probability_list
def get_pipes_risk_probability(name: str, pipe_ids: list[str]) -> list[dict[str, Any]]:
pipe_risk_probability_list = []
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select * from pipe_risk_probability")
for record in cur:
if record['pipeid'] in pipe_ids:
t = {}
t['pipeid'] = record['pipeid']
t['x'] = record['x']
t['y'] = record['y']
pipe_risk_probability_list.append(t)
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select * from pipe_risk_probability")
for record in cur:
if record['pipeid'] in pipe_ids:
t = {}
t['pipeid'] = record['pipeid']
t['x'] = record['x']
t['y'] = record['y']
pipe_risk_probability_list.append(t)
return pipe_risk_probability_list
@@ -71,22 +67,21 @@ def get_pipe_risk_probability_geometries(name: str) -> dict[str, Any]:
# key_endnode = '下游节点'
key_geometry = 'geometry'
with project_connection(name) as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(f"select *, ST_AsGeoJSON(geometry) AS {key_geometry} from gis_pipe")
with conn[name].cursor(row_factory=dict_row) as cur:
cur.execute(f"select *, ST_AsGeoJSON(geometry) AS {key_geometry} from gis_pipe")
for record in cur:
id = record[key_pipeId]
geom = json.loads(record[key_geometry])
for record in cur:
id = record[key_pipeId]
geom = json.loads(record[key_geometry])
pipe_risk_probability_geometries[id] = {
'points': geom['coordinates']
}
pipe_risk_probability_geometries[id] = {
'points': geom['coordinates']
}
for col in record:
if col != key_geometry:
pipe_risk_probability_geometries[id][col] = record[col]
for col in record:
if col != key_geometry:
pipe_risk_probability_geometries[id][col] = record[col]
# print(len(pipe_risk_probability_geometries))
return pipe_risk_probability_geometries
return pipe_risk_probability_geometries
+35 -4
View File
@@ -1,5 +1,36 @@
"""Service package.
from app.services.network_import import network_update, submit_scada_info
from app.services.scheme_management import (
create_user,
delete_user,
scheme_name_exists,
store_scheme_info,
delete_scheme_info,
query_scheme_list,
upload_shp_to_pg,
submit_risk_probability_result,
)
from app.services.valve_isolation import analyze_valve_isolation
from app.services.simulation_ops import (
project_management,
scheduling_simulation,
daily_scheduling_simulation,
)
from app.services.leakage_identifier import run_leakage_identification
Keep package initialization lightweight. Import concrete service modules directly,
for example: `from app.services.tjnetwork import open_project`.
"""
__all__ = [
"network_update",
"submit_scada_info",
"create_user",
"delete_user",
"scheme_name_exists",
"store_scheme_info",
"delete_scheme_info",
"query_scheme_list",
"upload_shp_to_pg",
"submit_risk_probability_result",
"project_management",
"scheduling_simulation",
"daily_scheduling_simulation",
"analyze_valve_isolation",
"run_leakage_identification",
]
+47 -223
View File
@@ -1,7 +1,7 @@
from __future__ import annotations
import os
from datetime import datetime, timedelta
from datetime import datetime
from typing import Any
import pandas as pd
@@ -11,7 +11,6 @@ from app.infra.db.timescaledb.internal_queries import InternalQueries
from app.services.scheme_management import (
query_burst_location_scheme_detail,
query_burst_location_schemes,
query_scheme_list,
scheme_name_exists,
store_scheme_info,
)
@@ -41,7 +40,7 @@ def _normalize_series(data: SeriesInput, field_name: str) -> pd.Series:
else:
raise ValueError(f"Unsupported data format for {field_name}.")
series.index = series.index.map(_normalize_identifier)
series.index = series.index.map(str)
return pd.to_numeric(series, errors="raise")
@@ -61,8 +60,6 @@ def run_burst_location_by_network(
basic_pressure: float = 10.0,
scada_burst_start: datetime | str | None = None,
scada_burst_end: datetime | str | None = None,
scada_normal_start: datetime | str | None = None,
scada_normal_end: datetime | str | None = None,
use_scada_flow: bool = False,
scheme_name: str | None = None,
simulation_scheme_name: str | None = None,
@@ -90,43 +87,16 @@ def run_burst_location_by_network(
for value in [
scada_burst_start,
scada_burst_end,
scada_normal_start,
scada_normal_end,
]
)
if use_scada_pressure:
burst_start_dt, burst_end_dt = _validate_time_window(
start_value=scada_burst_start,
end_value=scada_burst_end,
start_field="scada_burst_start",
end_field="scada_burst_end",
label=(
"爆管方案时间窗"
if normalized_data_source == "simulation"
else "爆管时段 SCADA 时间窗"
),
)
normal_start_dt: datetime | None = None
normal_end_dt: datetime | None = None
if scada_normal_start is not None or scada_normal_end is not None:
normal_start_dt, normal_end_dt = _validate_time_window(
start_value=scada_normal_start,
end_value=scada_normal_end,
start_field="scada_normal_start",
end_field="scada_normal_end",
label="正常时段 SCADA 时间窗",
)
normal_pressure_from_payload = (
_normalize_series(normal_pressure, "normal_pressure")
if normal_pressure is not None
else None
burst_start_dt, burst_end_dt = _validate_scada_windows(
scada_burst_start=scada_burst_start,
scada_burst_end=scada_burst_end,
)
if normalized_data_source == "simulation":
if not simulation_scheme_name:
raise ValueError("模拟方案模式必须提供 simulation_scheme_name。")
normal_start_dt = burst_start_dt
normal_end_dt = burst_end_dt
(
burst_pressure_series,
burst_pressure_samples,
@@ -147,8 +117,8 @@ def run_burst_location_by_network(
) = _build_observed_series_from_simulation(
network=network,
sensor_ids=selected_pressure_ids,
start_dt=normal_start_dt,
end_dt=normal_end_dt,
start_dt=burst_start_dt,
end_dt=burst_end_dt,
data_type="pressure",
series_name="normal_pressure",
simulation_source="realtime",
@@ -157,11 +127,6 @@ def run_burst_location_by_network(
)
observed_source = "simulation_scheme_burst_realtime_normal_timerange"
else:
if normal_pressure_from_payload is None and (
normal_start_dt is None or normal_end_dt is None
):
normal_start_dt = burst_start_dt - timedelta(days=1)
normal_end_dt = burst_end_dt - timedelta(days=1)
(
burst_pressure_series,
burst_pressure_samples,
@@ -173,31 +138,21 @@ def run_burst_location_by_network(
data_type="pressure",
series_name="burst_pressure",
)
if normal_pressure_from_payload is None:
(
normal_pressure_series,
normal_pressure_samples,
) = _build_observed_series_from_scada(
network=network,
sensor_ids=selected_pressure_ids,
start_dt=normal_start_dt,
end_dt=normal_end_dt,
data_type="pressure",
series_name="normal_pressure",
)
observed_source = "scada_burst_scada_normal_timerange"
else:
normal_pressure_series = normal_pressure_from_payload
normal_pressure_samples = 1
observed_source = "scada_burst_payload_normal_timerange"
selected_pressure_ids, burst_pressure_series, normal_pressure_series = (
_align_observed_series_pair(
ids=selected_pressure_ids,
burst_series=burst_pressure_series,
normal_series=normal_pressure_series,
data_label="压力数据",
(
normal_pressure_series,
normal_pressure_samples,
) = _build_observed_series_from_simulation(
network=network,
sensor_ids=selected_pressure_ids,
start_dt=burst_start_dt,
end_dt=burst_end_dt,
data_type="pressure",
series_name="normal_pressure",
simulation_source="realtime",
simulation_scheme_name=None,
simulation_scheme_type=resolved_simulation_scheme_type,
)
)
observed_source = "scada_burst_realtime_normal_timerange"
else:
if burst_pressure is None or normal_pressure is None:
raise ValueError(
@@ -224,11 +179,6 @@ def run_burst_location_by_network(
)
if not selected_flow_ids:
raise ValueError("未找到可用流量传感器,无法从 SCADA 查询流量数据。")
normal_flow_from_payload = (
_normalize_series(normal_flow, "normal_flow")
if normal_flow is not None
else None
)
if normalized_data_source == "simulation":
if not simulation_scheme_name:
raise ValueError("模拟方案模式必须提供 simulation_scheme_name。")
@@ -249,8 +199,8 @@ def run_burst_location_by_network(
_build_observed_series_from_simulation(
network=network,
sensor_ids=selected_flow_ids,
start_dt=normal_start_dt,
end_dt=normal_end_dt,
start_dt=burst_start_dt,
end_dt=burst_end_dt,
data_type="flow",
series_name="normal_flow",
simulation_source="realtime",
@@ -259,11 +209,6 @@ def run_burst_location_by_network(
)
)
else:
if normal_flow_from_payload is None and (
normal_start_dt is None or normal_end_dt is None
):
normal_start_dt = burst_start_dt - timedelta(days=1)
normal_end_dt = burst_end_dt - timedelta(days=1)
burst_flow_series, burst_flow_samples = _build_observed_series_from_scada(
network=network,
sensor_ids=selected_flow_ids,
@@ -272,26 +217,17 @@ def run_burst_location_by_network(
data_type="flow",
series_name="burst_flow",
)
if normal_flow_from_payload is None:
normal_flow_series, normal_flow_samples = (
_build_observed_series_from_scada(
network=network,
sensor_ids=selected_flow_ids,
start_dt=normal_start_dt,
end_dt=normal_end_dt,
data_type="flow",
series_name="normal_flow",
)
)
else:
normal_flow_series = normal_flow_from_payload
normal_flow_samples = 1
selected_flow_ids, burst_flow_series, normal_flow_series = (
_align_observed_series_pair(
ids=selected_flow_ids,
burst_series=burst_flow_series,
normal_series=normal_flow_series,
data_label="流量数据",
normal_flow_series, normal_flow_samples = (
_build_observed_series_from_simulation(
network=network,
sensor_ids=selected_flow_ids,
start_dt=burst_start_dt,
end_dt=burst_end_dt,
data_type="flow",
series_name="normal_flow",
simulation_source="realtime",
simulation_scheme_name=None,
simulation_scheme_type=resolved_simulation_scheme_type,
)
)
else:
@@ -322,7 +258,6 @@ def run_burst_location_by_network(
normal_flow=normal_flow_series,
min_dpressure=min_dpressure,
basic_pressure=basic_pressure,
visualize_partition=False,
)
payload: dict[str, Any] = {
@@ -346,23 +281,10 @@ def run_burst_location_by_network(
"burst_start": burst_start_dt.isoformat(),
"burst_end": burst_end_dt.isoformat(),
}
if normal_start_dt is not None and normal_end_dt is not None:
payload["scada_window"].update(
{
"normal_start": normal_start_dt.isoformat(),
"normal_end": normal_end_dt.isoformat(),
}
)
if normalized_data_source == "simulation":
simulation_burst_ids = _get_simulation_scheme_burst_ids(
network=network,
scheme_name=simulation_scheme_name,
scheme_type=resolved_simulation_scheme_type,
)
payload["simulation_scheme"] = {
"name": simulation_scheme_name,
"type": resolved_simulation_scheme_type,
"burst_ids": simulation_burst_ids,
}
if scheme_name:
_store_burst_scheme(
@@ -454,64 +376,6 @@ def _validate_scada_windows(
return burst_start_dt, burst_end_dt
def _validate_time_window(
*,
start_value: datetime | str | None,
end_value: datetime | str | None,
start_field: str,
end_field: str,
label: str,
) -> tuple[datetime, datetime]:
if start_value is None or end_value is None:
raise ValueError(f"{label}必须同时提供 {start_field}/{end_field}")
start_dt = _to_datetime(start_value)
end_dt = _to_datetime(end_value)
if start_dt >= end_dt:
raise ValueError(f"{label}非法:{start_field} 必须早于 {end_field}")
return start_dt, end_dt
def _get_simulation_scheme_burst_ids(
*, network: str, scheme_name: str | None, scheme_type: str
) -> list[str]:
if not scheme_name:
return []
rows = query_scheme_list(network) or []
for row in rows:
if len(row) < 7:
continue
if row[1] != scheme_name or row[2] != scheme_type:
continue
detail = row[6] if isinstance(row[6], dict) else {}
return _normalize_burst_ids(detail.get("burst_ID"))
return []
def _normalize_burst_ids(value: Any) -> list[str]:
if value is None:
return []
if isinstance(value, (list, tuple, set)):
return _dedupe_ids([str(item) for item in value])
return _dedupe_ids([str(value)])
def _align_observed_series_pair(
*,
ids: list[str],
burst_series: pd.Series,
normal_series: pd.Series,
data_label: str,
) -> tuple[list[str], pd.Series, pd.Series]:
common_ids = [
sensor_id
for sensor_id in _dedupe_ids(ids)
if sensor_id in burst_series.index and sensor_id in normal_series.index
]
if not common_ids:
raise ValueError(f"{data_label}没有同时具备爆管时段和正常时段有效数据的点位。")
return common_ids, burst_series.loc[common_ids], normal_series.loc[common_ids]
def _build_observed_series_from_scada(
*,
network: str,
@@ -521,14 +385,13 @@ def _build_observed_series_from_scada(
data_type: str,
series_name: str,
) -> tuple[pd.Series, int]:
sensor_ids = _dedupe_ids(sensor_ids)
scada_mapping = _build_scada_mapping(network=network, data_type=data_type)
missing_ids = [
sensor_id for sensor_id in sensor_ids if sensor_id not in scada_mapping
]
if missing_ids:
preview = ", ".join(missing_ids[:10])
raise ValueError(f"{_series_display_name(series_name)} 缺少可用 SCADA 映射: {preview}")
raise ValueError(f"{series_name} 缺少可用 SCADA 映射: {preview}")
query_ids = [scada_mapping[sensor_id] for sensor_id in sensor_ids]
scada_data = InternalQueries.query_scada_by_ids_timerange(
@@ -537,7 +400,6 @@ def _build_observed_series_from_scada(
start_time=start_dt.isoformat(),
end_time=end_dt.isoformat(),
)
scada_data = _normalize_timeseries_by_id(scada_data)
values: dict[str, float] = {}
sample_counts: list[int] = []
for sensor_id, query_id in zip(sensor_ids, query_ids):
@@ -546,13 +408,9 @@ def _build_observed_series_from_scada(
float(item["value"]) for item in records if item.get("value") is not None
]
if not numeric_values:
continue
raise ValueError(f"{series_name} 在时间窗内无有效数据: {sensor_id}")
values[sensor_id] = float(sum(numeric_values) / len(numeric_values))
sample_counts.append(len(numeric_values))
if not values:
raise ValueError(
f"{_series_display_name(series_name)} 在时间窗内无有效数据: {', '.join(sensor_ids[:10])}"
)
return pd.Series(values, dtype=float), min(sample_counts)
@@ -569,14 +427,13 @@ def _build_observed_series_from_simulation(
simulation_scheme_name: str | None,
simulation_scheme_type: str,
) -> tuple[pd.Series, int]:
sensor_ids = _dedupe_ids(sensor_ids)
sensor_metadata = _build_sensor_metadata(network=network, data_type=data_type)
missing_ids = [
sensor_id for sensor_id in sensor_ids if sensor_id not in sensor_metadata
]
if missing_ids:
preview = ", ".join(missing_ids[:10])
raise ValueError(f"{_series_display_name(series_name)} 缺少可用 SCADA 映射: {preview}")
raise ValueError(f"{series_name} 缺少可用 SCADA 映射: {preview}")
simulation_data = _query_simulation_data_by_sensor_ids(
network=network,
@@ -589,7 +446,6 @@ def _build_observed_series_from_simulation(
simulation_scheme_name=simulation_scheme_name,
simulation_scheme_type=simulation_scheme_type,
)
simulation_data = _normalize_timeseries_by_id(simulation_data)
values: dict[str, float] = {}
sample_counts: list[int] = []
for sensor_id in sensor_ids:
@@ -598,24 +454,13 @@ def _build_observed_series_from_simulation(
float(item["value"]) for item in records if item.get("value") is not None
]
if not numeric_values:
raise ValueError(
f"{_series_display_name(series_name)} 在时间窗内无有效模拟数据: {sensor_id}"
)
raise ValueError(f"{series_name} 在时间窗内无有效模拟数据: {sensor_id}")
values[sensor_id] = float(sum(numeric_values) / len(numeric_values))
sample_counts.append(len(numeric_values))
return pd.Series(values, dtype=float), min(sample_counts)
def _series_display_name(series_name: str) -> str:
return {
"burst_pressure": "爆管压力数据",
"normal_pressure": "正常压力数据",
"burst_flow": "爆管流量数据",
"normal_flow": "正常流量数据",
}.get(series_name, series_name)
def _query_simulation_data_by_sensor_ids(
*,
network: str,
@@ -631,7 +476,6 @@ def _query_simulation_data_by_sensor_ids(
if simulation_source not in {"scheme", "realtime"}:
raise ValueError(f"Unsupported simulation_source: {simulation_source}")
sensor_ids = _dedupe_ids(sensor_ids)
result: dict[str, list[dict[str, Any]]] = {
sensor_id: [] for sensor_id in sensor_ids
}
@@ -712,7 +556,6 @@ def _query_simulation_values(
simulation_scheme_name: str | None,
simulation_scheme_type: str,
) -> dict[str, list[dict[str, Any]]]:
element_ids = _dedupe_ids(element_ids)
if not element_ids:
return {}
if simulation_source == "scheme":
@@ -752,9 +595,14 @@ def _build_sensor_metadata(network: str, data_type: str) -> dict[str, dict[str,
continue
else:
raise ValueError(f"Unsupported data_type: {data_type}")
element_id = _normalize_identifier(item.get("associated_element_id"))
query_id = _normalize_identifier(item.get("api_query_id"))
if element_id and query_id:
element_id = item.get("associated_element_id")
query_id = item.get("api_query_id")
if (
isinstance(element_id, str)
and element_id
and isinstance(query_id, str)
and query_id
):
metadata[element_id] = {"query_id": query_id, "scada_type": scada_type}
return metadata
@@ -790,31 +638,7 @@ def _get_sensor_nodes(network: str, data_type: str) -> list[str]:
def _dedupe_ids(ids: list[str] | None) -> list[str]:
if ids is None:
return []
return list(
dict.fromkeys(
normalized
for normalized in (_normalize_identifier(item) for item in ids)
if normalized
)
)
def _normalize_identifier(value: Any) -> str:
if value is None:
return ""
return str(value).strip()
def _normalize_timeseries_by_id(
data: dict[Any, list[dict[str, Any]]] | None,
) -> dict[str, list[dict[str, Any]]]:
normalized_data: dict[str, list[dict[str, Any]]] = {}
for raw_id, records in (data or {}).items():
normalized_id = _normalize_identifier(raw_id)
if not normalized_id:
continue
normalized_data.setdefault(normalized_id, []).extend(records or [])
return normalized_data
return list(dict.fromkeys([str(item) for item in ids if item]))
def _to_datetime(value: datetime | str) -> datetime:
-76
View File
@@ -1,76 +0,0 @@
import json
from typing import Any
import httpx
from pydantic import AliasChoices, BaseModel, Field
from app.core.config import settings
class TiandituGeocodeRequest(BaseModel):
keyword: str = Field(
...,
min_length=1,
validation_alias=AliasChoices("keyword", "keyWord"),
description="地理编码地址关键字",
)
class TiandituGeocodingConfigError(RuntimeError):
pass
class TiandituGeocodingAPIError(RuntimeError):
def __init__(self, status_code: int, detail: Any):
super().__init__("Tianditu Geocoding API request failed")
self.status_code = status_code
self.detail = detail
async def geocode_tianditu(
request: TiandituGeocodeRequest,
*,
client: httpx.AsyncClient | None = None,
) -> dict[str, Any]:
if not settings.TIANDITU_GEOCODER_TOKEN:
raise TiandituGeocodingConfigError("TIANDITU_GEOCODER_TOKEN is not configured")
params = {
"ds": json.dumps({"keyWord": request.keyword}, ensure_ascii=False),
"tk": settings.TIANDITU_GEOCODER_TOKEN,
}
if client is not None:
response = await client.get(settings.TIANDITU_GEOCODER_URL, params=params)
return _parse_response(response)
async with httpx.AsyncClient(
timeout=settings.TIANDITU_GEOCODER_TIMEOUT_SECONDS
) as managed_client:
response = await managed_client.get(
settings.TIANDITU_GEOCODER_URL,
params=params,
)
return _parse_response(response)
def _parse_response(response: httpx.Response) -> dict[str, Any]:
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
raise TiandituGeocodingAPIError(
exc.response.status_code,
_response_detail(exc.response),
) from exc
data = response.json()
if str(data.get("status")) != "0":
raise TiandituGeocodingAPIError(502, data)
return data
def _response_detail(response: httpx.Response) -> Any:
try:
return response.json()
except ValueError:
return response.text
+2 -2
View File
@@ -23,8 +23,8 @@ non_realtime_region_patterns = {} # 基于source_outflow_region进行区分
realtime_region_pipe_flow_and_demand_id = {} # 基于source_outflow_region搜索该分区中的实时pipe_flow和demand的api_query_id,后续用region的流量 - 实时流量计的流量
realtime_region_pipe_flow_and_demand_patterns = {} # 基于source_outflow_region搜索该分区中的实时pipe_flow和demand的associated_pattern,后续用region的流量 - 实时流量计的流量
# ---------------------------------------------------------
# 历史数据访问相关全局变量
# 全局变量,用于存储不同类型的 realtime api_query_id
# influxdb_api.py中的全局变量
# 全局变量,用于存储不同类型的realtime api_query_id
reservoir_liquid_level_realtime_ids = []
tank_liquid_level_realtime_ids = []
fixed_pump_realtime_ids = []
+11 -18
View File
@@ -28,12 +28,12 @@ import pytz
import requests
import time
from typing import Optional, Tuple
import app.infra.db.influxdb.api as influxdb_api
import typing
import psycopg
import logging
import app.services.globals as globals
import app.services.project_info as project_info
from app.services.time_api import parse_beijing_time, parse_clock_duration_seconds
from app.core.config import get_pgconn_string
from app.infra.db.timescaledb.internal_queries import (
InternalQueries as TimescaleInternalQueries,
@@ -661,14 +661,13 @@ def from_seconds_to_clock(secs: int) -> str:
def convert_time_format(original_time: str) -> str:
"""
格式转换带时区的 ISO 8601 / RFC3339 时间转为北京时间的YYYY-MM-DD HH:MM:SS
:param original_time: str带显式时区的时间
格式转换2024-04-13T08:00:00+08:00"转为“2024-04-13 08:00:00”
:param original_time: str 2024-04-13T08:00:00+08:00"格式的时间
:return: str2024-04-13 08:00:00格式的时间
"""
normalized_time = parse_beijing_time(
original_time, field_name="modify_pattern_start_time"
)
return normalized_time.replace(microsecond=0).strftime("%Y-%m-%d %H:%M:%S")
new_time = original_time.replace("T", " ")
new_time = new_time.replace("+08:00", "")
return new_time
def get_history_pattern_info(project_name, pattern_name):
@@ -756,13 +755,11 @@ def run_simulation(
# 获取水力模拟步长,如’0:15:00‘
globals.hydraulic_timestep = dic_time["HYDRAULIC TIMESTEP"]
# 转换为分钟浮点数,兼容 EPANET 的 H:MM 和 H:MM:SS 写法
globals.PATTERN_TIME_STEP = (
parse_clock_duration_seconds(
globals.hydraulic_timestep,
field_name="HYDRAULIC TIMESTEP",
)
/ 60
# 将时间字符串转换为 timedelta 对象
time_obj = datetime.strptime(globals.hydraulic_timestep, "%H:%M:%S")
# 转换为分钟浮点数
globals.PATTERN_TIME_STEP = float(
time_obj.hour * 60 + time_obj.minute + time_obj.second / 60
)
# 对输入的时间参数进行处理
pattern_start_time = convert_time_format(modify_pattern_start_time)
@@ -1259,9 +1256,6 @@ def run_simulation(
node_result, link_result, modify_pattern_start_time, db_name=db_name
)
elif simulation_type.upper() == "EXTENDED":
result_timestep_seconds = times_info.get("report_step")
if result_timestep_seconds is None:
raise RuntimeError("run_project output missing times.report_step")
TimescaleInternalStorage.store_scheme_simulation(
scheme_type,
scheme_name,
@@ -1269,7 +1263,6 @@ def run_simulation(
link_result,
modify_pattern_start_time,
num_periods_result,
result_timestep_seconds,
db_name=db_name,
)
endtime = time.time()
-35
View File
@@ -89,41 +89,6 @@ def to_time_range(dt: datetime, delta: float) -> tuple[datetime, datetime]:
return (start_time, end_time)
def parse_clock_duration_seconds(clock: str, field_name: str = "duration") -> int:
"""
Parse EPANET-style clock durations into seconds.
Accepted formats include H:MM, HH:MM, H:MM:SS, and HH:MM:SS.
"""
if not isinstance(clock, str):
raise ValueError(f"{field_name} must be a string clock duration.")
parts = clock.strip().split(":")
if len(parts) not in (2, 3):
raise ValueError(
f"{field_name} must use H:MM or H:MM:SS format, got {clock!r}."
)
try:
values = [int(part) for part in parts]
except ValueError as exc:
raise ValueError(
f"{field_name} must contain numeric clock parts, got {clock!r}."
) from exc
if any(value < 0 for value in values):
raise ValueError(f"{field_name} must not contain negative values.")
hours, minutes = values[0], values[1]
seconds = values[2] if len(values) == 3 else 0
if minutes >= 60 or seconds >= 60:
raise ValueError(
f"{field_name} minutes and seconds must be less than 60, got {clock!r}."
)
return hours * 3600 + minutes * 60 + seconds
def parse_beijing_date_range(query_date: str) -> tuple[datetime, datetime]:
'''
将一个日期字符串转换成 start/end 时间段传进来的日期被认为是北京时间
-93
View File
@@ -1,93 +0,0 @@
from typing import Any, Literal
import httpx
from pydantic import BaseModel, Field
from app.core.config import settings
Freshness = Literal["noLimit", "oneDay", "oneWeek", "oneMonth", "oneYear"]
class WebSearchRequest(BaseModel):
query: str = Field(..., min_length=1, description="搜索关键词")
freshness: Freshness | str = Field(
default="noLimit",
description="时间范围:noLimit、oneDay、oneWeek、oneMonth、oneYear 或日期范围",
)
summary: bool = Field(default=True, description="是否返回网页摘要")
count: int = Field(default=10, ge=1, le=50, description="返回结果数量")
include: list[str] | None = Field(default=None, description="限定搜索域名")
exclude: list[str] | None = Field(default=None, description="排除搜索域名")
class BochaSearchConfigError(RuntimeError):
pass
class BochaSearchAPIError(RuntimeError):
def __init__(self, status_code: int, detail: Any):
super().__init__("Bocha Web Search API request failed")
self.status_code = status_code
self.detail = detail
def _build_payload(request: WebSearchRequest) -> dict[str, Any]:
payload = request.model_dump(exclude_none=True)
if request.include:
payload["include"] = ",".join(request.include)
if request.exclude:
payload["exclude"] = ",".join(request.exclude)
return payload
async def search_bocha_web(
request: WebSearchRequest,
*,
client: httpx.AsyncClient | None = None,
) -> dict[str, Any]:
if not settings.BOCHA_API_KEY:
raise BochaSearchConfigError("BOCHA_API_KEY is not configured")
headers = {
"Authorization": f"Bearer {settings.BOCHA_API_KEY}",
"Content-Type": "application/json",
}
payload = _build_payload(request)
if client is not None:
response = await client.post(
settings.BOCHA_WEB_SEARCH_URL,
headers=headers,
json=payload,
)
return _parse_response(response)
async with httpx.AsyncClient(
timeout=settings.BOCHA_WEB_SEARCH_TIMEOUT_SECONDS
) as managed_client:
response = await managed_client.post(
settings.BOCHA_WEB_SEARCH_URL,
headers=headers,
json=payload,
)
return _parse_response(response)
def _parse_response(response: httpx.Response) -> dict[str, Any]:
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
raise BochaSearchAPIError(
exc.response.status_code,
_response_detail(exc.response),
) from exc
return response.json()
def _response_detail(response: httpx.Response) -> Any:
try:
return response.json()
except ValueError:
return response.text
+1 -2
View File
@@ -3,10 +3,9 @@ services:
# Core API Service
# ==========================================
api:
image: ${TJWATER_SERVER_IMAGE:-tjwater-server:local}
build:
context: ../..
dockerfile: Dockerfile
dockerfile: infra/docker/Dockerfile
container_name: tjwater_api
restart: always
ports:
+105205
View File
File diff suppressed because it is too large Load Diff
+26498
View File
File diff suppressed because it is too large Load Diff
+30973
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+111322
View File
File diff suppressed because it is too large Load Diff
+328198
View File
File diff suppressed because it is too large Load Diff
+213
View File
@@ -0,0 +1,213 @@
[TITLE]
Hanoi example by Fujiwara and Khang, Water Resources Research, 1990
[JUNCTIONS]
;ID Elev Demand Pattern
2 0 890 ;
3 0 850 ;
4 0 130 ;
5 0 725 ;
6 0 1005 ;
7 0 1350 ;
8 0 550 ;
9 0 525 ;
10 0 525 ;
11 0 500 ;
12 0 560 ;
13 0 940 ;
14 0 615 ;
15 0 280 ;
16 0 310 ;
17 0 865 ;
18 0 1345 ;
19 0 60 ;
20 0 1275 ;
21 0 930 ;
22 0 485 ;
23 0 1045 ;
24 0 820 ;
25 0 170 ;
26 0 900 ;
27 0 370 ;
28 0 290 ;
29 0 360 ;
30 0 360 ;
31 0 105 ;
32 0 805 ;
[RESERVOIRS]
;ID Head Pattern
1 100.0 ;
[TANKS]
;ID Elevation InitLevel MinLevel MaxLevel Diameter MinVol VolCurve
[PIPES]
;ID Node1 Node2 Length Diameter Roughness MinorLoss Status
1 1 2 100 0.0001 130 0 open ;
2 2 3 1350 0.0001 130 0 open ;
3 3 4 900 0.0001 130 0 open ;
4 4 5 1150 0.0001 130 0 open ;
5 5 6 1450 0.0001 130 0 open ;
6 6 7 450 0.0001 130 0 open ;
7 7 8 850 0.0001 130 0 open ;
8 8 9 850 0.0001 130 0 open ;
9 9 10 800 0.0001 130 0 open ;
10 10 11 950 0.0001 130 0 open ;
11 11 12 1200 0.0001 130 0 open ;
12 12 13 3500 0.0001 130 0 open ;
13 10 14 800 0.0001 130 0 open ;
14 14 15 500 0.0001 130 0 open ;
15 15 16 550 0.0001 130 0 open ;
16 17 16 2730 0.0001 130 0 open ;
17 18 17 1750 0.0001 130 0 open ;
18 19 18 800 0.0001 130 0 open ;
19 3 19 400 0.0001 130 0 open ;
20 3 20 2200 0.0001 130 0 open ;
21 20 21 1500 0.0001 130 0 open ;
22 21 22 500 0.0001 130 0 open ;
23 20 23 2650 0.0001 130 0 open ;
24 23 24 1230 0.0001 130 0 open ;
25 24 25 1300 0.0001 130 0 open ;
26 26 25 850 0.0001 130 0 open ;
27 27 26 300 0.0001 130 0 open ;
28 16 27 750 0.0001 130 0 open ;
29 23 28 1500 0.0001 130 0 open ;
30 28 29 2000 0.0001 130 0 open ;
31 29 30 1600 0.0001 130 0 open ;
32 30 31 150 0.0001 130 0 open ;
33 32 31 860 0.0001 130 0 open ;
34 25 32 950 0.0001 130 0 open ;
[PUMPS]
;ID Node1 Node2 Parameters
[VALVES]
;ID Node1 Node2 Diameter Type Setting MinorLoss
[TAGS]
[DEMANDS]
;Junction Demand Pattern Category
[STATUS]
;ID Status/Setting
[PATTERNS]
;ID Multipliers
[CURVES]
;ID X-Value Y-Value
[CONTROLS]
[RULES]
[ENERGY]
Global Efficiency 75
Global Price 0
Demand Charge 0
[EMITTERS]
;Junction Coefficient
[QUALITY]
;Node InitQual
[SOURCES]
;Node Type Quality Pattern
[REACTIONS]
;Type Pipe/Tank Coefficient
[REACTIONS]
Order Bulk 1
Order Wall 1
Global Bulk 0
Global Wall 0
Limiting Potential 0
Roughness Correlation 0
[MIXING]
;Tank Model
[TIMES]
Duration 0
Hydraulic Timestep 1:00
Quality Timestep 0:05
Pattern Timestep 1:00
Pattern Start 0:00
Report Timestep 1:00
Report Start 0:00
Start ClockTime 12 am
Statistic None
[REPORT]
Status No
Summary No
Page 0
[OPTIONS]
Units CMH
Headloss H-W
Specific Gravity 1
Viscosity 1
Trials 40
Accuracy 0.001
Unbalanced Continue 10
Pattern 1
Demand Multiplier 1.0
Emitter Exponent 0.5
Quality NONE mg/L
Diffusivity 1
Tolerance 0.01
[COORDINATES]
;Node X-Coord Y-Coord
2 5021.20 1582.17
3 5025.20 2585.42
4 5874.22 2588.30
5 6873.11 2588.30
6 8103.51 2585.42
7 8103.51 3234.67
8 8106.66 4179.28
9 8106.66 5133.78
10 7318.64 5133.78
11 7319.94 5831.65
12 7319.94 6671.19
13 5636.76 6676.24
14 6530.63 5133.78
15 5676.02 5133.78
16 5021.20 5133.78
17 5021.20 4412.36
18 5021.20 3868.52
19 5021.20 3191.49
20 3587.87 2588.30
21 3587.87 1300.84
22 3587.87 901.29
23 1978.55 2588.30
24 1975.58 4084.35
25 1980.46 5137.63
26 3077.46 5137.63
27 3933.52 5133.78
28 846.04 2588.20
29 -552.41 2588.20
30 -552.38 4369.06
31 -549.36 5137.63
32 536.45 5137.63
1 5360.71 1354.05
[VERTICES]
;Link X-Coord Y-Coord
[LABELS]
;X-Coord Y-Coord Label & Anchor Node
[BACKDROP]
DIMENSIONS -985.92 612.54 8551.27 6964.99
UNITS None
FILE
OFFSET 0.00 0.00
[END]
+132
View File
@@ -0,0 +1,132 @@
[TITLE]
[JUNCTIONS]
;ID Elev Demand Pattern
1 0 0 ;
2 0 0 ;
3 0 0 ;
4 0 0 ;
5 0 0 ;
[RESERVOIRS]
;ID Head Pattern
[TANKS]
;ID Elevation InitLevel MinLevel MaxLevel Diameter MinVol VolCurve
[PIPES]
;ID Node1 Node2 Length Diameter Roughness MinorLoss Status
1 3 2 1000 12 100 0 Open ;
[PUMPS]
;ID Node1 Node2 Parameters
[VALVES]
;ID Node1 Node2 Diameter Type Setting MinorLoss
[TAGS]
[DEMANDS]
;Junction Demand Pattern Category
[STATUS]
;ID Status/Setting
[PATTERNS]
;ID Multipliers
[CURVES]
;ID X-Value Y-Value
[CONTROLS]
[RULES]
[ENERGY]
Global Efficiency 75
Global Price 0
Demand Charge 0
[EMITTERS]
;Junction Coefficient
[QUALITY]
;Node InitQual
[SOURCES]
;Node Type Quality Pattern
[REACTIONS]
;Type Pipe/Tank Coefficient
[REACTIONS]
Order Bulk 1
Order Tank 1
Order Wall 1
Global Bulk 0
Global Wall 0
Limiting Potential 0
Roughness Correlation 0
[MIXING]
;Tank Model
[TIMES]
Duration 0:00
Hydraulic Timestep 1:00
Quality Timestep 0:05
Pattern Timestep 1:00
Pattern Start 0:00
Report Timestep 1:00
Report Start 0:00
Start ClockTime 12 am
Statistic NONE
[REPORT]
Status No
Summary No
Page 0
[OPTIONS]
Units GPM
Headloss H-W
Specific Gravity 1
Viscosity 1
Trials 40
Accuracy 0.001
CHECKFREQ 2
MAXCHECK 10
DAMPLIMIT 0
Unbalanced Continue 10
Pattern 1
Demand Multiplier 1.0
Emitter Exponent 0.5
Quality None mg/L
Diffusivity 1
Tolerance 0.01
[COORDINATES]
;Node X-Coord Y-Coord
1 455.97 6698.11
2 6022.01 3616.35
3 7374.21 6509.43
4 3128.93 5786.16
5 2122.64 2358.49
[VERTICES]
;Link X-Coord Y-Coord
[LABELS]
;X-Coord Y-Coord Label & Anchor Node
[BACKDROP]
DIMENSIONS 0.00 0.00 10000.00 10000.00
UNITS None
FILE
OFFSET 0.00 0.00
[END]
+213
View File
@@ -0,0 +1,213 @@
[TITLE]
[JUNCTIONS]
;ID Elev Demand Pattern
N23 0 0 ;
N1 18 5 ;
N2 18 10 ;
N3 14 0 ;
N4 12 5 ;
N5 14 30 ;
N24 0 0 ;
N25 0 0 ;
N14 20 5 ;
N13 23 0 ;
N16 10 0 ;
N17 7 0 ;
N19 10 5 ;
N20 7 0 ;
N21 10 0 ;
N22 15 20 ;
N18 8 5 ;
N6 15 10 ;
N7 14.5 0 ;
N8 14 20 ;
N9 14 0 ;
N10 15 5 ;
N11 12 10 ;
N12 15 0 ;
N15 8 20 ;
[RESERVOIRS]
;ID Head Pattern
[TANKS]
;ID Elevation InitLevel MinLevel MaxLevel Diameter MinVol VolCurve
1 0 54.66 54.5 56 1000000 0 ;
2 0 54.60 54.5 55.5 1000000 0 ;
3 0 54.50 54 55.5 1000000 0 ;
[PIPES]
;ID Node1 Node2 Length Diameter Roughness MinorLoss Status
P1 N23 N1 606 457 110 0 Open ;
P9 N1 N2 1930 457 110 0 Open ;
P10 N2 N3 5150 305 10 0 Open ;
P30 N3 N4 326 0.152 100 0 Open ;
P31 N4 N5 844 229 110 0 Open ;
P35 N5 N22 1408 152 100 0 Open ;
P36 N5 N7 500 381 110 0 Open ;
P34 N7 N6 615 381 110 0 Open ;
P32 N6 N3 1274 152 100 0 Open ;
P27 N6 N8 743 381 110 0 Open ;
P26 N8 N9 443 229 90 0 Open ;
P37 N9 N6 300 229 90 0 Open ;
P25 N8 N10 249 305 105 0 Open ;
P5 N24 N10 3383 305 100 0 Open ;
P2 N23 N24 454 457 110 0 Open ;
P3 N24 N14 2782 229 105 0 Open ;
P4 N14 N25 304 381 135 0 Open ;
P6 N24 N13 1767 475 110 0 Open ;
P7 N13 N14 1014 381 135 0 Open ;
P23 N10 N11 542 229 90 0 Open ;
P22 N11 N12 777 229 90 0 Open ;
P24 N8 N12 1600 457 110 0 Open ;
P8 N25 N16 1014 381 6 0 Open ;
P13 N16 N17 822 305 140 0 Open ;
P16 N17 N19 1072 229 90 0 Open ;
P17 N19 N20 864 152 90 0 Open ;
P18 N20 N21 711 152 90 0 Open ;
P14 N17 N18 411 152 100 0 Open ;
P15 N20 N18 701 229 110 0 Open ;
P20 N15 N22 2334 152 100 0 Open ;
P19 N15 N21 832 152 90 0 Open ;
P12 N15 N16 914 229 125 0 Open ;
0 1 N23 1 1000 110 0 Open ;
16 2 N24 1 1000 100 0 Open ;
20 3 N25 1 1000 100 0 Open ;
P11 N13 N12 762 457 110 0 Open ;
P21 N12 N15 1996 0.229 95 0 Open ;
P28 N8 N22 931 229 125 0 Open ;
P29 N21 N22 2689 152 100 0 Open ;
P33 N5 N6 1115 229 90 0 Open ;
[PUMPS]
;ID Node1 Node2 Parameters
[VALVES]
;ID Node1 Node2 Diameter Type Setting MinorLoss
[TAGS]
[DEMANDS]
;Junction Demand Pattern Category
[STATUS]
;ID Status/Setting
[PATTERNS]
;ID Multipliers
[CURVES]
;ID X-Value Y-Value
[CONTROLS]
[RULES]
[ENERGY]
Global Efficiency 75
Global Price 0
Demand Charge 0
[EMITTERS]
;Junction Coefficient
[QUALITY]
;Node InitQual
[SOURCES]
;Node Type Quality Pattern
[REACTIONS]
;Type Pipe/Tank Coefficient
[REACTIONS]
Order Bulk 1
Order Tank 1
Order Wall 1
Global Bulk 0
Global Wall 0
Limiting Potential 0
Roughness Correlation 0
[MIXING]
;Tank Model
[TIMES]
Duration 0
Hydraulic Timestep 1:00
Quality Timestep 0:05
Pattern Timestep 1:00
Pattern Start 0:00
Report Timestep 1:00
Report Start 0:00
Start ClockTime 12 am
Statistic None
[REPORT]
Status No
Summary No
Page 0
[OPTIONS]
Units LPS
Headloss H-W
Specific Gravity 1
Viscosity 1
Trials 40
Accuracy 0.001
Unbalanced Continue 10
Pattern 1
Demand Multiplier 1.0
Emitter Exponent 0.5
Quality None mg/L
Diffusivity 1
Tolerance 0.01
[COORDINATES]
;Node X-Coord Y-Coord
N23 2146.34 8439.02
N1 3268.29 8471.54
N2 4585.37 8455.28
N3 6081.30 8422.76
N4 7121.95 8504.07
N5 8260.16 8520.33
N24 2048.78 6536.59
N25 1918.70 3821.14
N14 2048.78 4796.75
N13 2845.53 5235.77
N16 3154.47 3739.84
N17 4455.28 3804.88
N19 5560.98 3837.40
N20 6601.63 3788.62
N21 7804.88 3788.62
N22 9203.25 3788.62
N18 6162.60 3317.07
N6 6845.53 7268.29
N7 7918.70 7235.77
N8 6357.72 6552.85
N9 6975.61 6552.85
N10 4065.04 6439.02
N11 4048.78 5723.58
N12 5056.91 5040.65
N15 5073.17 4552.85
1 1674.80 8520.33
2 1170.73 6536.59
3 1170.73 4016.26
[VERTICES]
;Link X-Coord Y-Coord
16 1170.73 6308.94
[LABELS]
;X-Coord Y-Coord Label & Anchor Node
[BACKDROP]
DIMENSIONS 0.00 0.00 10000.00 10000.00
UNITS None
FILE
OFFSET 0.00 0.00
[END]
+14197
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+178
View File
@@ -0,0 +1,178 @@
[TITLE]
EPANET Example Network 1
A simple example of modeling chlorine decay. Both bulk and
wall reactions are included.
[JUNCTIONS]
;ID Elev Demand Pattern
10 710 0 ;
11 710 150 ;
12 700 150 ;
13 695 100 ;
21 700 150 ;
22 695 200 ;
23 690 150 ;
31 700 100 ;
32 710 100 ;
[RESERVOIRS]
;ID Head Pattern
9 800 ;
[TANKS]
;ID Elevation InitLevel MinLevel MaxLevel Diameter MinVol VolCurve
2 850 120 100 150 50.5 0 ;
[PIPES]
;ID Node1 Node2 Length Diameter Roughness MinorLoss Status
10 10 11 10530 18 100 0 Open ;
11 11 12 5280 14 100 0 Open ;
12 12 13 5280 10 100 0 Open ;
21 21 22 5280 10 100 0 Open ;
22 22 23 5280 12 100 0 Open ;
31 31 32 5280 6 100 0 Open ;
110 2 12 200 18 100 0 Open ;
111 11 21 5280 10 100 0 Open ;
112 12 22 5280 12 100 0 Open ;
113 13 23 5280 8 100 0 Open ;
121 21 31 5280 8 100 0 Open ;
122 22 32 5280 6 100 0 Open ;
[PUMPS]
;ID Node1 Node2 Parameters
9 9 10 HEAD 1 ;
[VALVES]
;ID Node1 Node2 Diameter Type Setting MinorLoss
[TAGS]
[DEMANDS]
;Junction Demand Pattern Category
[STATUS]
;ID Status/Setting
[PATTERNS]
;ID Multipliers
;Demand Pattern
1 1.0 1.2 1.4 1.6 1.4 1.2
1 1.0 0.8 0.6 0.4 0.6 0.8
[CURVES]
;ID X-Value Y-Value
;PUMP: Pump Curve for Pump 9
1 1500 250
[CONTROLS]
LINK 9 OPEN IF NODE 2 BELOW 110
LINK 9 CLOSED IF NODE 2 ABOVE 140
[RULES]
[ENERGY]
Global Efficiency 75
Global Price 0.0
Demand Charge 0.0
[EMITTERS]
;Junction Coefficient
[QUALITY]
;Node InitQual
10 0.5
11 0.5
12 0.5
13 0.5
21 0.5
22 0.5
23 0.5
31 0.5
32 0.5
9 1.0
2 1.0
[SOURCES]
;Node Type Quality Pattern
[REACTIONS]
;Type Pipe/Tank Coefficient
[REACTIONS]
Order Bulk 1
Order Tank 1
Order Wall 1
Global Bulk -.5
Global Wall -1
Limiting Potential 0.0
Roughness Correlation 0.0
[MIXING]
;Tank Model
[TIMES]
Duration 24:00
Hydraulic Timestep 1:00
Quality Timestep 0:05
Pattern Timestep 2:00
Pattern Start 0:00
Report Timestep 1:00
Report Start 0:00
Start ClockTime 12 am
Statistic None
[REPORT]
Status Yes
Summary No
Page 0
[OPTIONS]
Units GPM
Headloss H-W
Specific Gravity 1.0
Viscosity 1.0
Trials 40
Accuracy 0.001
CHECKFREQ 2
MAXCHECK 10
DAMPLIMIT 0
Unbalanced Continue 10
Pattern 1
Demand Multiplier 1.0
Emitter Exponent 0.5
Quality Chlorine mg/L
Diffusivity 1.0
Tolerance 0.01
[COORDINATES]
;Node X-Coord Y-Coord
10 20.00 70.00
11 30.00 70.00
12 50.00 70.00
13 70.00 70.00
21 30.00 40.00
22 50.00 40.00
23 70.00 40.00
31 30.00 10.00
32 50.00 10.00
9 10.00 70.00
2 50.00 90.00
[VERTICES]
;Link X-Coord Y-Coord
[LABELS]
;X-Coord Y-Coord Label & Anchor Node
6.99 73.63 "Source"
13.48 68.13 "Pump"
43.85 91.21 "Tank"
[BACKDROP]
DIMENSIONS 7.00 6.00 73.00 94.00
UNITS None
FILE
OFFSET 0.00 0.00
[END]
+309
View File
@@ -0,0 +1,309 @@
[TITLE]
EPANET Example Network 2
Example of modeling a 55-hour fluoride tracer study.
Measured fluoride data is contained in the file Net2-FL.dat
and should be registered with the project to produce a
Calibration Report (select Calibration Data from the Project
menu).
[JUNCTIONS]
;ID Elev Demand Pattern
1 50 -694.4 2 ;
2 100 8 ;
3 60 14 ;
4 60 8 ;
5 100 8 ;
6 125 5 ;
7 160 4 ;
8 110 9 ;
9 180 14 ;
10 130 5 ;
11 185 34.78 ;
12 210 16 ;
13 210 2 ;
14 200 2 ;
15 190 2 ;
16 150 20 ;
17 180 20 ;
18 100 20 ;
19 150 5 ;
20 170 19 ;
21 150 16 ;
22 200 10 ;
23 230 8 ;
24 190 11 ;
25 230 6 ;
27 130 8 ;
28 110 0 ;
29 110 7 ;
30 130 3 ;
31 190 17 ;
32 110 17 ;
33 180 1.5 ;
34 190 1.5 ;
35 110 0 ;
36 110 1 ;
[RESERVOIRS]
;ID Head Pattern
[TANKS]
;ID Elevation InitLevel MinLevel MaxLevel Diameter MinVol VolCurve
26 235 56.7 50 70 50 0 ;
[PIPES]
;ID Node1 Node2 Length Diameter Roughness MinorLoss Status
1 1 2 2400 12 100 0 Open ;
2 2 5 800 12 100 0 Open ;
3 2 3 1300 8 100 0 Open ;
4 3 4 1200 8 100 0 Open ;
5 4 5 1000 12 100 0 Open ;
6 5 6 1200 12 100 0 Open ;
7 6 7 2700 12 100 0 Open ;
8 7 8 1200 12 140 0 Open ;
9 7 9 400 12 100 0 Open ;
10 8 10 1000 8 140 0 Open ;
11 9 11 700 12 100 0 Open ;
12 11 12 1900 12 100 0 Open ;
13 12 13 600 12 100 0 Open ;
14 13 14 400 12 100 0 Open ;
15 14 15 300 12 100 0 Open ;
16 13 16 1500 8 100 0 Open ;
17 15 17 1500 8 100 0 Open ;
18 16 17 600 8 100 0 Open ;
19 17 18 700 12 100 0 Open ;
20 18 32 350 12 100 0 Open ;
21 16 19 1400 8 100 0 Open ;
22 14 20 1100 12 100 0 Open ;
23 20 21 1300 8 100 0 Open ;
24 21 22 1300 8 100 0 Open ;
25 20 22 1300 8 100 0 Open ;
26 24 23 600 12 100 0 Open ;
27 15 24 250 12 100 0 Open ;
28 23 25 300 12 100 0 Open ;
29 25 26 200 12 100 0 Open ;
30 25 31 600 12 100 0 Open ;
31 31 27 400 8 100 0 Open ;
32 27 29 400 8 100 0 Open ;
34 29 28 700 8 100 0 Open ;
35 22 33 1000 8 100 0 Open ;
36 33 34 400 8 100 0 Open ;
37 32 19 500 8 100 0 Open ;
38 29 35 500 8 100 0 Open ;
39 35 30 1000 8 100 0 Open ;
40 28 35 700 8 100 0 Open ;
41 28 36 300 8 100 0 Open ;
[PUMPS]
;ID Node1 Node2 Parameters
[VALVES]
;ID Node1 Node2 Diameter Type Setting MinorLoss
[TAGS]
[DEMANDS]
;Junction Demand Pattern Category
[STATUS]
;ID Status/Setting
[PATTERNS]
;ID Multipliers
;Demand Pattern
1 1.26 1.04 .97 .97 .89 1.19
1 1.28 .67 .67 1.34 2.46 .97
1 .92 .68 1.43 .61 .31 .78
1 .37 .67 1.26 1.56 1.19 1.26
1 .6 1.1 1.03 .73 .88 1.06
1 .99 1.72 1.12 1.34 1.12 .97
1 1.04 1.15 .91 .61 .68 .46
1 .51 .74 1.12 1.34 1.26 .97
1 .82 1.37 1.03 .81 .88 .81
1 .81
;Pump Station Outflow Pattern
2 .96 .96 .96 .96 .96 .96
2 .62 0 0 0 0 0
2 .8 1 1 1 1 .15
2 0 0 0 0 0 0
2 .55 .92 .92 .92 .92 .9
2 .9 .45 0 0 0 0
2 0 .7 1 1 1 1
2 .2 0 0 0 0 0
2 0 .74 .92 .92 .92 .92
2 .92
;Pump Station Fluoride Pattern
3 .98 1.02 1.05 .99 .64 .46
3 .35 .35 .35 .35 .35 .35
3 .17 .17 .13 .13 .13 .15
3 .15 .15 .15 .15 .15 .15
3 .15 .12 .1 .08 .11 .09
3 .09 .08 .08 .08 .08 .08
3 .08 .09 .07 .07 .09 .09
3 .09 .09 .09 .09 .09 .09
3 .09 .08 .35 .72 .82 .92
3 1
[CURVES]
;ID X-Value Y-Value
[CONTROLS]
[RULES]
[ENERGY]
Global Efficiency 75
Global Price 0.0
Demand Charge 0.0
[EMITTERS]
;Junction Coefficient
[QUALITY]
;Node InitQual
1 1.0
2 1.0
3 1.0
4 1.0
5 1.0
6 1.0
7 1.0
8 1.0
9 1.0
10 1.0
11 1.0
12 1.0
13 1.0
14 1.0
15 1.0
16 1.0
17 1.0
18 1.0
19 1.0
20 1.0
21 1.0
22 1.0
23 1.0
24 1.0
25 1.0
27 1.0
28 1.0
29 1.0
30 1.0
31 1.0
32 1.0
33 1.0
34 1.0
35 1.0
36 1.0
26 1.0
[SOURCES]
;Node Type Quality Pattern
1 CONCEN 1.0 3
[REACTIONS]
;Type Pipe/Tank Coefficient
[REACTIONS]
Order Bulk 1
Order Tank 1
Order Wall 1
Global Bulk 0.0
Global Wall 0.0
Limiting Potential 0.0
Roughness Correlation 0.0
[MIXING]
;Tank Model
[TIMES]
Duration 55:00
Hydraulic Timestep 1:00
Quality Timestep 0:05
Pattern Timestep 1:00
Pattern Start 0:00
Report Timestep 1:00
Report Start 0:00
Start ClockTime 8 am
Statistic None
[REPORT]
Status No
Summary No
Page 0
[OPTIONS]
Units GPM
Headloss H-W
Specific Gravity 1.0
Viscosity 1.0
Trials 40
Accuracy 0.001
CHECKFREQ 2
MAXCHECK 10
DAMPLIMIT 0
Unbalanced Continue 10
Pattern 1
Demand Multiplier 1.0
Emitter Exponent 0.5
Quality Fluoride mg/L
Diffusivity 1.0
Tolerance 0.01
[COORDINATES]
;Node X-Coord Y-Coord
1 21.00 4.00
2 19.00 20.00
3 11.00 21.00
4 14.00 28.00
5 19.00 25.00
6 28.00 23.00
7 36.00 39.00
8 38.00 30.00
9 36.00 42.00
10 37.00 23.00
11 37.00 49.00
12 39.00 60.00
13 38.00 64.00
14 38.00 66.00
15 37.00 69.00
16 27.00 65.00
17 27.00 69.00
18 23.00 68.00
19 21.00 59.00
20 45.00 68.00
21 51.00 62.00
22 54.00 69.00
23 35.00 74.00
24 37.00 71.00
25 35.00 76.00
27 39.00 87.00
28 49.00 85.00
29 42.00 86.00
30 47.00 80.00
31 37.00 80.00
32 23.00 64.00
33 56.00 73.00
34 56.00 77.00
35 43.00 81.00
36 53.00 87.00
26 33.00 76.00
[VERTICES]
;Link X-Coord Y-Coord
[LABELS]
;X-Coord Y-Coord Label & Anchor Node
24.00 7.00 "Pump"
24.00 4.00 "Station"
26.76 77.42 "Tank"
[BACKDROP]
DIMENSIONS 8.75 -0.15 58.25 91.15
UNITS None
FILE
OFFSET 0.00 0.00
[END]
+147940
View File
File diff suppressed because it is too large Load Diff
+60024
View File
File diff suppressed because it is too large Load Diff
Binary file not shown.
File diff suppressed because it is too large Load Diff
+7878
View File
File diff suppressed because it is too large Load Diff
+252
View File
@@ -0,0 +1,252 @@
[TITLE]
[JUNCTIONS]
;ID Elev Demand Pattern
2 5.80 19.49 1 ;
3 5.80 19.49 1 ;
4 5.80 19.50 1 ;
5 5.80 19.49 1 ;
6 5.80 19.49 1 ;
7 5.80 19.50 1 ;
8 5.80 19.49 1 ;
9 5.80 19.50 1 ;
10 5.80 19.49 1 ;
11 5.80 19.49 1 ;
12 5.80 19.50 1 ;
13 5.80 19.49 1 ;
15 5.80 19.49 1 ;
16 5.80 19.50 1 ;
17 5.80 19.49 1 ;
18 5.80 19.49 1 ;
19 5.80 19.50 1 ;
20 5.80 19.49 1 ;
21 5.80 19.49 1 ;
22 0 136.45 1 ;
23 0 0 1 ;
24 0 0 1 ;
[RESERVOIRS]
;ID Head Pattern
1 3.19 ;
14 7.40 ;
[TANKS]
;ID Elevation InitLevel MinLevel MaxLevel Diameter MinVol VolCurve
[PIPES]
;ID Node1 Node2 Length Diameter Roughness MinorLoss Status
5 2 6 1976 315 120 0 Open ;
6 6 7 1300 250 120 0 Open ;
7 7 8 1051 160 120 0 Open ;
8 3 9 6210 250 120 0 Open ;
9 5 10 2173 200 120 0 Open ;
10 4 11 2984 500 120 0 Open ;
11 11 12 2100 500 120 0 Open ;
12 11 13 2199 250 120 0 Open ;
14 15 16 3100 315 120 0 Open ;
15 15 17 6485 400 120 0 Open ;
16 16 18 2688 200 120 0 Open ;
17 17 19 2306 200 120 0 Open ;
18 17 20 4161 250 120 0 Open ;
19 20 21 2841 200 120 0 Open ;
20 12 22 1 1000 120 0 Open ;
1 23 2 3024 355 100 0 Open ;
2 23 3 1400 400 120 0 Open ;
3 23 4 3149 500 120 0 Open ;
4 23 5 5400 250 120 0 Open ;
13 24 15 3554 400 120 0 Open ;
[PUMPS]
;ID Node1 Node2 Parameters
23 1 23 HEAD 1 ;
22 1 23 HEAD 1 ;
21 1 23 HEAD 1 ;
24 14 24 HEAD 1 ;
25 14 24 HEAD 1 ;
[VALVES]
;ID Node1 Node2 Diameter Type Setting MinorLoss
[TAGS]
[DEMANDS]
;Junction Demand Pattern Category
[STATUS]
;ID Status/Setting
[PATTERNS]
;ID Multipliers
;
1 1
[CURVES]
;ID X-Value Y-Value
;Ë®±Ã:
1 0 56.99
1 33.33 56.74
1 50 56.49
1 66.67 55.74
1 83.33 54.24
1 100 51.98
1 116.67 48.72
1 133.33 43.95
1 150 38.44
[CONTROLS]
[RULES]
[ENERGY]
Global Efficiency 75
Global Price 0
Demand Charge 0
[EMITTERS]
;Junction Coefficient
[QUALITY]
;Node InitQual
1 0.3
14 0.3
[SOURCES]
;Node Type Quality Pattern
[REACTIONS]
;Type Pipe/Tank Coefficient
[REACTIONS]
Order Bulk 1
Order Tank 1
Order Wall 1
Global Bulk -1
Global Wall -0.5
Limiting Potential 0
Roughness Correlation 0
[MIXING]
;Tank Model
[TIMES]
Duration 48:00
Hydraulic Timestep 0:05
Quality Timestep 0:05
Pattern Timestep 0:05
Pattern Start 0:00
Report Timestep 0:05
Report Start 0:00
Start ClockTime 12 am
Statistic None
[REPORT]
Status No
Summary No
Page 0
[OPTIONS]
Units LPS
Headloss H-W
Specific Gravity 1
Viscosity 1
Trials 100
Accuracy 0.01
CHECKFREQ 2
MAXCHECK 10
DAMPLIMIT 0
Unbalanced Continue 10
Pattern 1
Demand Multiplier 1.0
Emitter Exponent 0.5
Quality »¯Ñ§³É·Ö mg/L
Diffusivity 1
Tolerance 0.01
[COORDINATES]
;Node X-Coord Y-Coord
2 2054.14 6695.86
3 2249.20 7663.22
4 3523.09 7328.82
5 3136.94 9211.78
6 1409.24 6787.42
7 1122.61 6819.27
8 800.16 6950.64
9 1357.48 8809.71
10 3531.05 9996.02
11 3654.46 8308.12
12 4353.11 8268.31
13 3710.19 9060.51
15 5467.75 8264.33
16 5467.75 9327.23
17 6769.51 9243.63
18 5220.94 9765.13
19 6359.47 9390.92
20 8055.33 8853.50
21 8970.94 8901.27
22 4353.60 8338.97
23 2590.07 7540.80
24 4405.35 8378.78
1 2900.58 7676.15
14 4309.81 8420.58
[VERTICES]
;Link X-Coord Y-Coord
5 1970.54 6707.80
5 1791.40 6719.75
6 1373.41 6689.89
6 1242.04 6769.51
8 1817.28 7750.80
8 1875.00 8134.95
8 1968.55 8190.68
8 1849.12 8529.06
8 1833.20 8833.60
10 3642.52 7852.31
14 5481.69 8612.66
15 6072.85 8238.46
15 6076.83 8349.92
15 6106.69 8481.29
15 6156.45 8610.67
15 6385.35 8616.64
15 6357.48 8865.45
15 6666.00 8887.34
15 6753.58 9203.82
16 5467.75 9460.59
16 5254.78 9486.46
16 5095.54 9488.46
16 5127.39 9767.12
17 6803.34 9317.28
17 6596.34 9402.87
17 6486.86 9464.57
17 6445.06 9355.10
18 7131.77 9080.41
18 7689.09 8875.40
19 8292.20 8837.58
19 8682.32 8909.24
19 8769.90 8895.30
2 2502.49 7568.67
2 2243.73 7600.52
3 2781.15 7461.19
3 2880.67 7433.32
4 2613.95 7636.35
4 2757.27 7839.37
4 2785.13 8508.16
13 5010.45 8384.75
13 5040.31 8289.21
13 5229.40 8271.30
23 2789.11 7711.98
21 2900.58 7564.69
24 4345.64 8378.78
25 4363.55 8410.63
[LABELS]
;X-Coord Y-Coord Label & Anchor Node
[BACKDROP]
DIMENSIONS 0.00 0.00 10000.00 10000.00
UNITS None
FILE
OFFSET 0.00 0.00
[END]
File diff suppressed because it is too large Load Diff
+109
View File
@@ -0,0 +1,109 @@
[TITLE]
[JUNCTIONS]
[RESERVOIRS]
[TANKS]
[PIPES]
[PUMPS]
[VALVES]
[DEMANDS]
[EMITTERS]
[STATUS]
[PATTERNS]
[CURVES]
[CONTROLS]
[ENERGY]
GLOBAL EFFICIENCY 0.7500
GLOBAL PRICE 0.0000
DEMAND CHARGE 0.0000
[QUALITY]
[SOURCES]
[MIXING]
[REACTIONS]
ORDER BULK 1.0000
ORDER WALL 1.0000
ORDER TANK 1.0000
GLOBAL BULK 0.0000
GLOBAL WALL 0.0000
LIMITING POTENTIAL 0.0000
ROUGHNESS CORRELATION 0.0000
[OPTIONS]
FLOW_UNITS GPM
PRESSURE_UNITS PSI
HEADLOSS_MODEL H-W
SPECIFIC_GRAVITY 1.0000
SPECIFIC_VISCOSITY 1.0000
MAXIMUM_TRIALS 100
HEAD_TOLERANCE 0.0000
FLOW_TOLERANCE 0.0000
FLOW_CHANGE_LIMIT 0.0000
TIME_WEIGHT 0.0000
STEP_SIZING FULL
IF_UNBALANCED STOP
DEMAND_MODEL FIXED
DEMAND_PATTERN
DEMAND_MULTIPLIER 1.0000
MINIMUM_PRESSURE 0.0000
SERVICE_PRESSURE 0.0000
PRESSURE_EXPONENT 0.5000
LEAKAGE_MODEL NONE
LEAKAGE_COEFF1 0.0000
LEAKAGE_COEFF2 0.0000
EMITTER_EXPONENT 0.5000
QUALITY_MODEL NONE
SPECIFIC_DIFFUSIVITY 1.0000
QUALITY_TOLERANCE 0.0100
[TIMES]
TOTAL DURATION 0:00:00
HYDRAULIC TIMESTEP 1:00:00
QUALITY TIMESTEP 0:05:00
RULE TIMESTEP 0:05:00
PATTERN TIMESTEP 1:00:00
PATTERN START 0:00:00
REPORT TIMESTEP 1:00:00
REPORT START 0:00:00
START CLOCKTIME 0:00:00
[REPORT]
SUMMARY YES
[COORDINATES]
[VERTICES]
[REGION]
[BOUND]
[DATA_NODE_OF_REGION]
[DMA]
[VD]
[SA]
[DATA_REGION_OF_NODE]
+38821
View File
File diff suppressed because it is too large Load Diff
+37670
View File
File diff suppressed because it is too large Load Diff
+40290
View File
File diff suppressed because it is too large Load Diff
+73400
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More