9 Commits
Author SHA1 Message Date
jiang 76cf6c32bc fix(agent): expose network context
Server CI/CD / docker-image (push) Successful in 16s
Server CI/CD / deploy-fallback-log (push) Has been skipped
2026-07-08 18:41:56 +08:00
jiang 5a91da0904 fix(burst-location): use correct data sources
Simulation mode reads scheme data for both burst and normal observations. Monitoring mode reuses the burst window when no normal window is provided.
2026-07-08 17:51:08 +08:00
jiang d62bcae85e fix(burst-location): normalize simulation ids 2026-07-08 17:17:41 +08:00
jiang 4c0a4b29e9 refactor(metadata): drop geoserver config refs 2026-06-13 14:57:33 +08:00
jiang 80ca985c28 fix(cli): use renamed backend APIs 2026-06-13 13:56:44 +08:00
jiang 5a55d65002 refactor(api): add kebab-case legacy aliases 2026-06-13 13:07:16 +08:00
jiang d99f4cec6a refactor(admin): remove geoserver config 2026-06-12 15:28:14 +08:00
jiang a6e7a2e75c feat(admin): add project metadata config 2026-06-12 15:08:37 +08:00
jiang 23c008f602 feat(auth): migrate to Keycloak metadata auth 2026-06-12 10:18:41 +08:00
60 changed files with 2835 additions and 1962 deletions
+4 -8
View File
@@ -4,17 +4,13 @@
ENVIRONMENT="production" ENVIRONMENT="production"
NETWORK_NAME="tjwater" NETWORK_NAME="tjwater"
# ============================================ # ============================================
# 安全配置 (必填) # 敏感配置加密 (必填)
# ============================================ # ============================================
# JWT 密钥 - 用于生成和验证 Token # Fernet 格式,生产环境必须替换为独立密钥
# 生成方式: openssl rand -hex 32
SECRET_KEY=your-secret-key-here-change-in-production-use-openssl-rand-hex-32
# 数据加密密钥 - 用于敏感数据加密
# 生成方式: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())" # 生成方式: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
ENCRYPTION_KEY= # 用于项目数据库 DSN、GeoServer 管理密码等敏感配置
DATABASE_ENCRYPTION_KEY="rJC2VqLg4KrlSq+DGJcYm869q4v5KB2dFAeuQTe0I50=" DATABASE_ENCRYPTION_KEY="replace-with-generated-fernet-key"
# ============================================ # ============================================
# 数据库配置 (PostgreSQL) # 数据库配置 (PostgreSQL)
+90
View File
@@ -0,0 +1,90 @@
# TJWater Authentication and Metadata Management
## Ownership
Keycloak owns login identity, credentials, token issuance, and token expiry.
TJWater metadata stores only business snapshots and authorization data:
- `users.keycloak_id` is the stable identity binding.
- `users.username`, `users.email`, and `users.last_login_at` are Keycloak claim caches.
- `users.role`, `users.is_active`, and `users.is_superuser` control TJWater system access.
- `user_project_membership.project_role` controls project access.
The backend does not accept passwords, does not issue local JWTs, and does not
trust frontend-supplied user IDs.
## Login Snapshot Refresh
Every authenticated metadata-user resolution validates the Keycloak access token
and reads `sub`, `preferred_username` or `username`, and `email` claims. The
backend finds `users` by `keycloak_id = sub`, rejects inactive or missing users,
then refreshes `username`, `email`, and `last_login_at`.
This keeps local display data current without changing the identity binding.
There is no Keycloak webhook requirement; second-level user or permission sync is
out of scope unless explicitly requested later.
## Admin APIs
All admin APIs require metadata admin access: `users.is_superuser = true` or
`users.role = 'admin'`.
User and membership management:
- `GET /api/v1/admin/me`
- `POST /api/v1/admin/users/sync`
- `POST /api/v1/admin/users/sync/batch`
- `GET /api/v1/admin/users`
- `GET /api/v1/admin/users/{user_id}`
- `PATCH /api/v1/admin/users/{user_id}`
- `GET /api/v1/admin/projects/{project_id}/members`
- `POST /api/v1/admin/projects/{project_id}/members`
- `PATCH /api/v1/admin/projects/{project_id}/members/{user_id}`
- `DELETE /api/v1/admin/projects/{project_id}/members/{user_id}`
Project configuration:
- `GET /api/v1/admin/projects`
- `POST /api/v1/admin/projects`
- `PATCH /api/v1/admin/projects/{project_id}`
- `GET /api/v1/admin/projects/{project_id}/databases`
- `PUT /api/v1/admin/projects/{project_id}/databases`
- `DELETE /api/v1/admin/projects/{project_id}/databases/{db_role}`
- `POST /api/v1/admin/projects/{project_id}/databases/{db_role}/health`
## Secret Handling
Admins submit plaintext DSNs only through HTTPS admin APIs. Operators should not
write encrypted columns manually.
- `project_databases.dsn_encrypted` is encrypted with `DATABASE_ENCRYPTION_KEY`.
- Admin responses return only `has_dsn`.
- Audit logs record whether a secret was updated, but never store plaintext DSNs
or other secrets.
Generate the database encryption key with:
```bash
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
```
Keep keys stable for the lifetime of encrypted metadata. Rotating a key requires
decrypting with the old key and re-encrypting with the new key.
## Metadata Schema Patches
Apply metadata patches in order:
1. `resources/sql/004_metadata_auth_management.sql`
2. `resources/sql/005_metadata_project_configuration.sql`
`004` creates Keycloak-backed metadata users and project memberships. `005`
creates project and project database routing tables with uniqueness, role/type,
and pool-size constraints.
## Frontend System Management
`/system-admin` is shown only after `GET /api/v1/admin/me` confirms metadata
admin access. The page lets admins maintain metadata users, project members,
projects, project database routing for `biz_data` and `iot_data`, connection
health checks. This replaces direct SQL editing for normal project onboarding.
+77
View File
@@ -0,0 +1,77 @@
# Backend Naming Audit
DOC-003 audit for the internal `TJWaterServerBinary` backend.
## Scope
Reviewed FastAPI route decorators under `app/api/v1/endpoints`, router prefixes in `app/api/v1/router.py`, and public request/response schema fields in `app/api` and `app/domain`.
The backend is mounted only under `/api/v1` from `app/main.py`; the old no-prefix router include remains commented out.
## Current Good Surface
These newer routes already follow the naming rule for public HTTP paths:
- Metadata/admin: `/api/v1/admin/projects`, `/api/v1/admin/users/sync`, `/api/v1/admin/projects/{project_id}/members`
- Audit: `/api/v1/audit/logs`, `/api/v1/audit/logs/count`
- Agent auth: `/api/v1/agent/auth/context`
- Business APIs: `/api/v1/burst-detection/detect`, `/api/v1/burst-location/locate`, `/api/v1/leakage/identify`
- Time-series APIs: `/api/v1/scada/by-ids-time-range`, `/api/v1/scada/by-ids-field-time-range`, `/api/v1/composite/clean-scada`
- Project data APIs: `/api/v1/scada-info`, `/api/v1/scheme-list`, `/api/v1/burst-locate-result`
- Web integrations: `/api/v1/web-search`, `/api/v1/geocode`
Path template parameters such as `{project_id}`, `{user_id}`, `{device_id}`, `{scheme_name}`, and `{link_id}` intentionally remain `snake_case`.
## Legacy URL Categories
### Keep With Compatibility
These now have `kebab-case` aliases. The frontend has been migrated to the replacement paths; keep the old paths as deprecated compatibility aliases for Agent planning, tests, customer scripts, or external callers:
| Current URL | Suggested replacement |
| --- | --- |
| `/api/v1/openproject/` | `/api/v1/projects/open` |
| `/api/v1/project_info/` | `/api/v1/project-info` |
| `/api/v1/getallschemes/` | `/api/v1/schemes` |
| `/api/v1/getallsensorplacements/` | `/api/v1/sensor-placement-schemes` |
| `/api/v1/sensorplacementscheme/create` | `/api/v1/sensor-placement-schemes` |
| `/api/v1/burst_analysis/` | `/api/v1/burst-analysis` |
| `/api/v1/valve_isolation_analysis/` | `/api/v1/valve-isolation-analysis` |
| `/api/v1/flushing_analysis/` | `/api/v1/flushing-analysis` |
| `/api/v1/contaminant_simulation/` | `/api/v1/contaminant-simulation` |
| `/api/v1/runsimulationmanuallybydate/` | `/api/v1/simulations/run-by-date` |
### Broad Legacy Surface
These route groups expose many command-style concatenated paths. They should not be copied into new work; replace only when a caller migration is planned:
- Project lifecycle: `listprojects`, `createproject`, `deleteproject`, `isprojectopen`, `closeproject`, `copyproject`, `importinp`, `exportinp`, `readinp`, `dumpinp`, `lockproject`, `unlockproject`
- Network object CRUD: `addjunction`, `getjunctionelevation`, `setpipediameter`, `getvalvesetting`, and similar junction/pipe/pump/tank/reservoir/valve routes
- Region/DMA/VD commands: `calculatedistrictmeteringareaforregion`, `getdistrictmeteringarea`, `generatevirtualdistrict`, and related routes
- SCADA native CRUD: `getscadadevice`, `setscadadevicedata`, `cleanscadaelement`, and related routes
- Snapshot/cache utilities: `takesnapshotforoperation`, `syncwithserver`, `clearrediskey`, `queryredis`
- Advanced simulation endpoints with underscore paths: `pressure_regulation`, `daily_scheduling_analysis`, `network_update`, `pressure_sensor_placement_kmeans`
### Direct Cleanup Candidates
These are likely safe only after confirming no caller uses them:
- `/api/v1/test_dict/`: development/test utility in `misc.py`.
- `/api/v1/takenapshotforcurrentoperation`: typo compatibility path; keep deprecated if any client may still call it.
- `/api/v1/getpumpenergyproperties//` and `/api/v1/setpumpenergyproperties//`: double-slash paths in options endpoints.
## Field Naming
Most public JSON, query, and SSE fields are already `snake_case`, including `project_id`, `user_id`, `scheme_name`, `scheme_type`, `start_time`, `end_time`, `device_ids`, `session_id`, and `request_id`.
Known legacy exception:
- `BurstAnalysis.burst_ID` in `app/api/v1/endpoints/simulation.py` should become `burst_id` on a new API contract. Preserve `burst_ID` only for the legacy body shape.
Headers keep standard HTTP casing:
- `X-Project-Id`
## Recommendation
Do not rename existing legacy routes in place. For each active legacy route, keep the new `kebab-case` alias as the documented path, keep the old route marked deprecated, migrate remaining Agent/customer/script callers, then remove only after a documented compatibility window.
+695
View File
@@ -0,0 +1,695 @@
from typing import List
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Path, Query, Response, status
from sqlalchemy import text
from sqlalchemy.engine.url import make_url
from sqlalchemy.exc import IntegrityError, SQLAlchemyError
from sqlalchemy.ext.asyncio import create_async_engine
from app.auth.metadata_dependencies import (
get_current_metadata_admin,
get_metadata_repository,
)
from app.core.audit import AuditAction, log_audit_event
from app.domain.schemas.admin_metadata import (
AdminProjectCreateRequest,
AdminProjectResponse,
AdminProjectUpdateRequest,
MetadataUsersBatchSyncRequest,
MetadataUserResponse,
MetadataUserSyncRequest,
MetadataUserSyncResult,
MetadataUserUpdateRequest,
ProjectDatabaseHealthResponse,
ProjectDatabaseHealthRequest,
ProjectDatabaseResponse,
ProjectDatabaseUpsertRequest,
ProjectDbRole,
ProjectMemberCreateRequest,
ProjectMemberResponse,
ProjectMemberUpdateRequest,
)
from app.infra.db.metadb import models
from app.infra.db.metadb.repositories.metadata_repository import (
MetadataRepository,
ProjectDbRouting,
)
router = APIRouter()
def _project_response(project: models.Project) -> AdminProjectResponse:
return AdminProjectResponse(
project_id=project.id,
name=project.name,
code=project.code,
description=project.description,
gs_workspace=project.gs_workspace,
map_extent=project.map_extent,
status=project.status,
created_at=project.created_at,
updated_at=project.updated_at,
)
def _project_database_response(
record: models.ProjectDatabase,
) -> ProjectDatabaseResponse:
return ProjectDatabaseResponse(
id=record.id,
project_id=record.project_id,
db_role=record.db_role,
db_type=record.db_type,
pool_min_size=record.pool_min_size,
pool_max_size=record.pool_max_size,
has_dsn=bool(record.dsn_encrypted),
)
def _database_audit_payload(payload: ProjectDatabaseUpsertRequest) -> dict:
return {
"db_role": payload.db_role,
"db_type": _db_type_for_role(payload.db_role),
"pool_min_size": payload.pool_min_size,
"pool_max_size": payload.pool_max_size,
"dsn_updated": payload.dsn is not None,
}
def _to_async_sqlalchemy_url(dsn: str) -> str:
parsed = make_url(dsn)
if parsed.drivername in {"postgresql", "postgres"}:
parsed = parsed.set(drivername="postgresql+psycopg")
return parsed.render_as_string(hide_password=False)
def _db_type_for_role(db_role: str) -> str:
if db_role == "iot_data":
return "timescaledb"
return "postgresql"
def _status_for_config_value_error(exc: ValueError) -> int:
if "DATABASE_ENCRYPTION_KEY" in str(exc):
return status.HTTP_503_SERVICE_UNAVAILABLE
return status.HTTP_400_BAD_REQUEST
async def _check_database_connection(routing: ProjectDbRouting) -> None:
engine = create_async_engine(
_to_async_sqlalchemy_url(routing.dsn),
pool_size=1,
max_overflow=0,
pool_pre_ping=True,
)
try:
async with engine.connect() as conn:
await conn.execute(text("SELECT 1"))
finally:
await engine.dispose()
def _database_health_error_detail(exc: Exception) -> str:
message = str(exc)
lower_message = message.lower()
if "password authentication failed" in lower_message:
return "连通性测试失败:用户名或密码错误,请检查 DSN 中的账号密码。"
if "connection refused" in lower_message:
return "连通性测试失败:目标主机或端口拒绝连接,请检查地址、端口和服务状态。"
if "timeout" in lower_message or "timed out" in lower_message:
return "连通性测试失败:连接超时,请检查网络、防火墙和数据库服务状态。"
if "could not translate host name" in lower_message or "name or service not known" in lower_message:
return "连通性测试失败:数据库主机名无法解析,请检查 DSN 中的主机地址。"
first_line = message.splitlines()[0] if message else exc.__class__.__name__
return f"连通性测试失败:{first_line}"
async def _upsert_and_audit_metadata_user(
payload: MetadataUserSyncRequest,
*,
current_user,
metadata_repo: MetadataRepository,
response_status: int,
) -> MetadataUserResponse:
user = await metadata_repo.upsert_user_from_keycloak(
keycloak_id=payload.keycloak_id,
username=payload.username,
email=str(payload.email),
role=payload.role,
is_active=payload.is_active,
)
await log_audit_event(
action=AuditAction.UPDATE,
user_id=current_user.id,
resource_type="metadata_user",
resource_id=str(user.id),
request_data=payload.model_dump(mode="json"),
response_status=response_status,
session=metadata_repo.session,
)
return MetadataUserResponse.model_validate(user)
@router.get("/me", response_model=MetadataUserResponse)
async def get_metadata_admin_me(
current_user=Depends(get_current_metadata_admin),
) -> MetadataUserResponse:
return MetadataUserResponse.model_validate(current_user)
@router.post("/users/sync", response_model=MetadataUserResponse)
async def sync_metadata_user(
payload: MetadataUserSyncRequest,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> MetadataUserResponse:
try:
return await _upsert_and_audit_metadata_user(
payload,
current_user=current_user,
metadata_repo=metadata_repo,
response_status=status.HTTP_200_OK,
)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="User keycloak_id, username, or email conflicts with an existing user",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
@router.post("/users/sync/batch", response_model=List[MetadataUserSyncResult])
async def sync_metadata_users_batch(
payload: MetadataUsersBatchSyncRequest,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[MetadataUserSyncResult]:
results: list[MetadataUserSyncResult] = []
for item in payload.users:
try:
user = await _upsert_and_audit_metadata_user(
item,
current_user=current_user,
metadata_repo=metadata_repo,
response_status=status.HTTP_200_OK,
)
except IntegrityError as exc:
results.append(
MetadataUserSyncResult(
keycloak_id=item.keycloak_id,
success=False,
error="User keycloak_id, username, or email conflicts with an existing user",
)
)
await metadata_repo.session.rollback()
except SQLAlchemyError as exc:
results.append(
MetadataUserSyncResult(
keycloak_id=item.keycloak_id,
success=False,
error=f"Metadata database error: {exc}",
)
)
await metadata_repo.session.rollback()
else:
results.append(
MetadataUserSyncResult(
keycloak_id=item.keycloak_id,
success=True,
user=user,
)
)
return results
@router.get("/users", response_model=List[MetadataUserResponse])
async def list_metadata_users(
skip: int = Query(0, ge=0),
limit: int = Query(100, ge=1, le=1000),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[MetadataUserResponse]:
users = await metadata_repo.list_users(skip=skip, limit=limit)
return [MetadataUserResponse.model_validate(user) for user in users]
@router.get("/projects", response_model=List[AdminProjectResponse])
async def list_admin_projects(
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[AdminProjectResponse]:
projects = await metadata_repo.list_project_records()
return [_project_response(project) for project in projects]
@router.post(
"/projects",
response_model=AdminProjectResponse,
status_code=status.HTTP_201_CREATED,
)
async def create_admin_project(
payload: AdminProjectCreateRequest,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> AdminProjectResponse:
try:
project = await metadata_repo.create_project(
name=payload.name,
code=payload.code,
description=payload.description,
gs_workspace=payload.gs_workspace,
map_extent=payload.map_extent,
status=payload.status,
)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Project code or workspace conflicts with an existing project",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
await log_audit_event(
action=AuditAction.CREATE,
user_id=current_user.id,
project_id=project.id,
resource_type="project",
resource_id=str(project.id),
request_data=payload.model_dump(mode="json"),
response_status=status.HTTP_201_CREATED,
session=metadata_repo.session,
)
return _project_response(project)
@router.patch(
"/projects/{project_id}",
response_model=AdminProjectResponse,
)
async def update_admin_project(
payload: AdminProjectUpdateRequest,
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> AdminProjectResponse:
updates = payload.model_dump(mode="json", exclude_unset=True)
try:
project = await metadata_repo.update_project(project_id, updates=updates)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Project code or workspace conflicts with an existing project",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
await log_audit_event(
action=AuditAction.UPDATE,
user_id=current_user.id,
project_id=project.id,
resource_type="project",
resource_id=str(project.id),
request_data=updates,
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return _project_response(project)
@router.get(
"/projects/{project_id}/databases",
response_model=List[ProjectDatabaseResponse],
)
async def list_project_databases(
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[ProjectDatabaseResponse]:
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
records = await metadata_repo.list_project_databases(project_id)
return [_project_database_response(record) for record in records]
@router.put(
"/projects/{project_id}/databases",
response_model=ProjectDatabaseResponse,
)
async def upsert_project_database(
payload: ProjectDatabaseUpsertRequest,
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectDatabaseResponse:
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
try:
routing = (
ProjectDbRouting(
project_id=project_id,
db_role=payload.db_role,
db_type=_db_type_for_role(payload.db_role),
dsn=payload.dsn,
pool_min_size=payload.pool_min_size,
pool_max_size=payload.pool_max_size,
)
if payload.dsn
else await metadata_repo.get_project_db_routing(project_id, payload.db_role)
)
if routing is None:
raise ValueError("dsn is required when creating project database config")
await _check_database_connection(routing)
except ValueError as exc:
raise HTTPException(
status_code=_status_for_config_value_error(exc),
detail=str(exc),
) from exc
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=_database_health_error_detail(exc),
) from exc
try:
record = await metadata_repo.upsert_project_database_config(
project_id,
db_role=payload.db_role,
db_type=_db_type_for_role(payload.db_role),
dsn=payload.dsn,
pool_min_size=payload.pool_min_size,
pool_max_size=payload.pool_max_size,
)
except IntegrityError as exc:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="Project database role conflicts with an existing config",
) from exc
except SQLAlchemyError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
await log_audit_event(
action=AuditAction.CONFIG_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_database",
resource_id=payload.db_role,
request_data=_database_audit_payload(payload),
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return _project_database_response(record)
@router.delete(
"/projects/{project_id}/databases/{db_role}",
status_code=status.HTTP_204_NO_CONTENT,
)
async def delete_project_database(
project_id: UUID = Path(...),
db_role: ProjectDbRole = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> None:
removed = await metadata_repo.delete_project_database_config(project_id, db_role)
if not removed:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project database config not found",
)
await log_audit_event(
action=AuditAction.CONFIG_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_database",
resource_id=db_role,
request_data={"deleted": True},
response_status=status.HTTP_204_NO_CONTENT,
session=metadata_repo.session,
)
@router.post(
"/projects/{project_id}/databases/{db_role}/health",
response_model=ProjectDatabaseHealthResponse,
)
async def check_project_database_health(
response: Response,
project_id: UUID = Path(...),
db_role: ProjectDbRole = Path(...),
payload: ProjectDatabaseHealthRequest | None = None,
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectDatabaseHealthResponse:
dsn_to_test = payload.dsn if payload and payload.dsn else None
if dsn_to_test:
routing = ProjectDbRouting(
project_id=project_id,
db_role=db_role,
db_type=_db_type_for_role(db_role),
dsn=dsn_to_test,
pool_min_size=1,
pool_max_size=1,
)
else:
try:
routing = await metadata_repo.get_project_db_routing(project_id, db_role)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Project database routing DSN is invalid: {exc}",
) from exc
if routing is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project database config not found",
)
try:
await _check_database_connection(routing)
except Exception as exc: # health endpoint should return diagnostic status
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
return ProjectDatabaseHealthResponse(
project_id=project_id,
db_role=db_role,
db_type=routing.db_type,
ok=False,
detail=_database_health_error_detail(exc),
)
return ProjectDatabaseHealthResponse(
project_id=project_id,
db_role=db_role,
db_type=routing.db_type,
ok=True,
detail="连通性测试通过",
)
@router.get("/users/{user_id}", response_model=MetadataUserResponse)
async def get_metadata_user(
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> MetadataUserResponse:
user = await metadata_repo.get_user_by_id(user_id)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
return MetadataUserResponse.model_validate(user)
@router.patch("/users/{user_id}", response_model=MetadataUserResponse)
async def update_metadata_user(
payload: MetadataUserUpdateRequest,
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> MetadataUserResponse:
updates = payload.model_dump(mode="json", exclude_unset=True)
if user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify themselves",
)
user = await metadata_repo.update_user_admin(
user_id,
updates=updates,
)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
await log_audit_event(
action=AuditAction.UPDATE,
user_id=current_user.id,
resource_type="metadata_user",
resource_id=str(user.id),
request_data=updates,
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return MetadataUserResponse.model_validate(user)
@router.get(
"/projects/{project_id}/members",
response_model=List[ProjectMemberResponse],
)
async def list_project_members(
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> List[ProjectMemberResponse]:
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project not found"
)
members = await metadata_repo.list_project_members(project_id)
return [ProjectMemberResponse(**member.__dict__) for member in members]
@router.post(
"/projects/{project_id}/members",
response_model=ProjectMemberResponse,
status_code=status.HTTP_201_CREATED,
)
async def add_project_member(
payload: ProjectMemberCreateRequest,
project_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectMemberResponse:
if payload.user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify their own project membership",
)
project = await metadata_repo.get_project_by_id(project_id)
if project is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project not found"
)
user = await metadata_repo.get_user_by_id(payload.user_id)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
existing = await metadata_repo.get_project_membership(project_id, payload.user_id)
if existing is not None:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="User is already a project member",
)
membership = await metadata_repo.add_project_member(
project_id, payload.user_id, payload.project_role
)
await log_audit_event(
action=AuditAction.PERMISSION_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_member",
resource_id=str(payload.user_id),
request_data=payload.model_dump(mode="json"),
response_status=status.HTTP_201_CREATED,
session=metadata_repo.session,
)
return ProjectMemberResponse(
id=membership.id,
user_id=membership.user_id,
project_id=membership.project_id,
project_role=membership.project_role,
username=user.username,
email=user.email,
is_active=user.is_active,
)
@router.patch(
"/projects/{project_id}/members/{user_id}",
response_model=ProjectMemberResponse,
)
async def update_project_member(
payload: ProjectMemberUpdateRequest,
project_id: UUID = Path(...),
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> ProjectMemberResponse:
if user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify their own project membership",
)
user = await metadata_repo.get_user_by_id(user_id)
if user is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="User not found")
membership = await metadata_repo.update_project_member_role(
project_id, user_id, payload.project_role
)
if membership is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project member not found"
)
await log_audit_event(
action=AuditAction.PERMISSION_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_member",
resource_id=str(user_id),
request_data=payload.model_dump(mode="json"),
response_status=status.HTTP_200_OK,
session=metadata_repo.session,
)
return ProjectMemberResponse(
id=membership.id,
user_id=membership.user_id,
project_id=membership.project_id,
project_role=membership.project_role,
username=user.username,
email=user.email,
is_active=user.is_active,
)
@router.delete("/projects/{project_id}/members/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
async def remove_project_member(
project_id: UUID = Path(...),
user_id: UUID = Path(...),
current_user=Depends(get_current_metadata_admin),
metadata_repo: MetadataRepository = Depends(get_metadata_repository),
) -> None:
if user_id == current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Users cannot modify their own project membership",
)
removed = await metadata_repo.remove_project_member(project_id, user_id)
if not removed:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project member not found"
)
await log_audit_event(
action=AuditAction.PERMISSION_CHANGE,
user_id=current_user.id,
project_id=project_id,
resource_type="project_member",
resource_id=str(user_id),
response_status=status.HTTP_204_NO_CONTENT,
session=metadata_repo.session,
)
+50
View File
@@ -0,0 +1,50 @@
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from app.auth.keycloak_dependencies import get_current_keycloak_payload
from app.auth.metadata_dependencies import get_current_metadata_user
from app.auth.project_dependencies import (
ProjectContext,
get_project_context,
)
router = APIRouter()
class AgentAuthContextResponse(BaseModel):
user_id: str
keycloak_sub: str
username: str
role: str
is_superuser: bool
project_id: str
network: str
project_role: str
token_expires_at: str | None = None
@router.get("/agent/auth/context", response_model=AgentAuthContextResponse)
async def get_agent_auth_context(
ctx: ProjectContext = Depends(get_project_context),
current_user=Depends(get_current_metadata_user),
keycloak_payload: dict = Depends(get_current_keycloak_payload),
) -> AgentAuthContextResponse:
exp = keycloak_payload.get("exp")
token_expires_at = (
datetime.fromtimestamp(exp, tz=timezone.utc).isoformat()
if isinstance(exp, int)
else None
)
return AgentAuthContextResponse(
user_id=str(current_user.id),
keycloak_sub=str(current_user.keycloak_id),
username=current_user.username,
role=current_user.role,
is_superuser=current_user.is_superuser,
project_id=str(ctx.project_id),
network=ctx.project_code,
project_role=ctx.project_role,
token_expires_at=token_expires_at,
)
-190
View File
@@ -1,190 +0,0 @@
from typing import Annotated
from datetime import timedelta
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordRequestForm
from app.core.config import settings
from app.core.security import create_access_token, create_refresh_token, verify_password
from app.domain.schemas.user import UserCreate, UserResponse, UserLogin, Token
from app.infra.db.metadb.repositories.user_repository import UserRepository
from app.auth.dependencies import get_user_repository, get_current_active_user
from app.domain.schemas.user import UserInDB
import logging
logger = logging.getLogger(__name__)
router = APIRouter()
@router.post(
"/register", response_model=UserResponse, status_code=status.HTTP_201_CREATED
)
async def register(
user_data: UserCreate, user_repo: UserRepository = Depends(get_user_repository)
) -> UserResponse:
"""
用户注册
创建新用户账号
"""
# 检查用户名和邮箱是否已存在
if await user_repo.user_exists(username=user_data.username):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Username already registered",
)
if await user_repo.user_exists(email=user_data.email):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST, detail="Email already registered"
)
# 创建用户
try:
user = await user_repo.create_user(user_data)
if not user:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to create user",
)
return UserResponse.model_validate(user)
except Exception as e:
logger.error(f"Error during user registration: {e}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Registration failed",
)
@router.post("/login", response_model=Token)
async def login(
form_data: Annotated[OAuth2PasswordRequestForm, Depends()],
user_repo: UserRepository = Depends(get_user_repository),
) -> Token:
"""
用户登录(OAuth2 标准格式)
返回 JWT Access Token 和 Refresh Token
"""
# 验证用户(支持用户名或邮箱登录)
user = await user_repo.get_user_by_username(form_data.username)
if not user:
# 尝试用邮箱登录
user = await user_repo.get_user_by_email(form_data.username)
if not user or not verify_password(form_data.password, user.hashed_password):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect username or password",
headers={"WWW-Authenticate": "Bearer"},
)
if not user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user account"
)
# 生成 Token
access_token = create_access_token(subject=user.username)
refresh_token = create_refresh_token(subject=user.username)
return Token(
access_token=access_token,
refresh_token=refresh_token,
token_type="bearer",
expires_in=settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60,
)
@router.post("/login/simple", response_model=Token)
async def login_simple(
username: str,
password: str,
user_repo: UserRepository = Depends(get_user_repository),
) -> Token:
"""
简化版登录接口(保持向后兼容)
直接使用 username 和 password 参数
"""
# 验证用户
user = await user_repo.get_user_by_username(username)
if not user:
user = await user_repo.get_user_by_email(username)
if not user or not verify_password(password, user.hashed_password):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect username or password",
)
if not user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user account"
)
# 生成 Token
access_token = create_access_token(subject=user.username)
refresh_token = create_refresh_token(subject=user.username)
return Token(
access_token=access_token,
refresh_token=refresh_token,
token_type="bearer",
expires_in=settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60,
)
@router.get("/me", response_model=UserResponse)
async def get_current_user_info(
current_user: UserInDB = Depends(get_current_active_user),
) -> UserResponse:
"""
获取当前登录用户信息
"""
return UserResponse.model_validate(current_user)
@router.post("/refresh", response_model=Token)
async def refresh_token(
refresh_token: str, user_repo: UserRepository = Depends(get_user_repository)
) -> Token:
"""
刷新 Access Token
使用 Refresh Token 获取新的 Access Token
"""
from jose import jwt, JWTError
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate refresh token",
headers={"WWW-Authenticate": "Bearer"},
)
try:
payload = jwt.decode(
refresh_token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM]
)
username: str = payload.get("sub")
token_type: str = payload.get("type")
if username is None or token_type != "refresh":
raise credentials_exception
except JWTError:
raise credentials_exception
# 验证用户仍然存在且激活
user = await user_repo.get_user_by_username(username)
if not user or not user.is_active:
raise credentials_exception
# 生成新的 Access Token
new_access_token = create_access_token(subject=user.username)
return Token(
access_token=new_access_token,
refresh_token=refresh_token, # 保持原 refresh token
token_type="bearer",
expires_in=settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60,
)
+4 -2
View File
@@ -29,8 +29,10 @@ class BurstLocationRequest(BaseModel):
normal_flow: dict[str, float] | list[dict[str, Any]] | None = Field(None, description="正常时的流量数据") normal_flow: dict[str, float] | list[dict[str, Any]] | None = Field(None, description="正常时的流量数据")
min_dpressure: float = Field(2.0, description="最小压力差(bar") min_dpressure: float = Field(2.0, description="最小压力差(bar")
basic_pressure: float = Field(10.0, description="基准压力(bar") basic_pressure: float = Field(10.0, description="基准压力(bar")
scada_burst_start: datetime | None = Field(None, description="SCADA爆管开始时间") scada_burst_start: datetime | None = Field(None, description="爆管/模拟方案开始时间")
scada_burst_end: datetime | None = Field(None, description="SCADA爆管结束时间") scada_burst_end: datetime | None = Field(None, description="爆管/模拟方案结束时间")
scada_normal_start: datetime | None = Field(None, description="监测数据正常工况开始时间")
scada_normal_end: datetime | None = Field(None, description="监测数据正常工况结束时间")
use_scada_flow: bool = Field(False, description="是否使用SCADA流量数据") use_scada_flow: bool = Field(False, description="是否使用SCADA流量数据")
scheme_name: str | None = Field(None, description="方案名称") scheme_name: str | None = Field(None, description="方案名称")
simulation_scheme_name: str | None = Field(None, description="模拟方案名称") simulation_scheme_name: str | None = Field(None, description="模拟方案名称")
+1 -15
View File
@@ -16,7 +16,6 @@ from app.auth.project_dependencies import (
from app.auth.metadata_dependencies import get_current_metadata_user from app.auth.metadata_dependencies import get_current_metadata_user
from app.core.config import settings from app.core.config import settings
from app.domain.schemas.metadata import ( from app.domain.schemas.metadata import (
GeoServerConfigResponse,
ProjectMetaResponse, ProjectMetaResponse,
ProjectSummaryResponse, ProjectSummaryResponse,
) )
@@ -34,25 +33,13 @@ async def get_project_metadata(
""" """
获取项目元数据 获取项目元数据
返回当前项目的完整元数据,包括项目基本信息和GeoServer配置 返回当前项目的完整元数据,包括项目基本信息和项目权限
""" """
project = await metadata_repo.get_project_by_id(ctx.project_id) project = await metadata_repo.get_project_by_id(ctx.project_id)
if not project: if not project:
raise HTTPException( raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="Project not found" status_code=status.HTTP_404_NOT_FOUND, detail="Project not found"
) )
geoserver = await metadata_repo.get_geoserver_config(ctx.project_id)
geoserver_payload = (
GeoServerConfigResponse(
gs_base_url=geoserver.gs_base_url,
gs_admin_user=geoserver.gs_admin_user,
gs_datastore_name=geoserver.gs_datastore_name,
default_extent=geoserver.default_extent,
srid=geoserver.srid,
)
if geoserver
else None
)
return ProjectMetaResponse( return ProjectMetaResponse(
project_id=project.id, project_id=project.id,
name=project.name, name=project.name,
@@ -62,7 +49,6 @@ async def get_project_metadata(
map_extent=project.map_extent, map_extent=project.map_extent,
status=project.status, status=project.status,
project_role=ctx.project_role, project_role=ctx.project_role,
geoserver=geoserver_payload,
) )
+2 -1
View File
@@ -28,7 +28,8 @@ async def fastapi_get_json():
) )
@router.get("/getallsensorplacements/", summary="获取所有传感器位置", description="获取网络中所有传感器的放置位置信息") @router.get("/sensor-placement-schemes", summary="获取所有传感器位置", description="获取网络中所有传感器的放置位置信息")
@router.get("/getallsensorplacements/", summary="获取所有传感器位置(旧路径)", description="获取网络中所有传感器的放置位置信息", deprecated=True)
async def fastapi_get_all_sensor_placements(network: str = Query(..., description="管网名称(或数据库名称)")) -> list[dict[Any, Any]]: async def fastapi_get_all_sensor_placements(network: str = Query(..., description="管网名称(或数据库名称)")) -> list[dict[Any, Any]]:
""" """
获取所有传感器位置 获取所有传感器位置
+2 -2
View File
@@ -10,7 +10,7 @@ from app.services.tjnetwork import (
get_network_node_coords, get_network_node_coords,
get_node_coord, get_node_coord,
) )
from app.auth.dependencies import get_current_user as verify_token from app.auth.metadata_dependencies import get_current_metadata_user
from app.infra.cache.redis_client import redis_client, encode_datetime, decode_datetime from app.infra.cache.redis_client import redis_client, encode_datetime, decode_datetime
import msgpack import msgpack
@@ -64,7 +64,7 @@ async def fastapi_get_network_in_extent(
@router.get( @router.get(
"/getnetworkgeometries/", "/getnetworkgeometries/",
dependencies=[Depends(verify_token)], dependencies=[Depends(get_current_metadata_user)],
summary="获取完整网络几何信息", summary="获取完整网络几何信息",
description="获取整个水网的所有节点、管线和SCADA点的几何信息(需要身份验证)" description="获取整个水网的所有节点、管线和SCADA点的几何信息(需要身份验证)"
) )
+5 -15
View File
@@ -4,7 +4,7 @@ from fastapi.responses import PlainTextResponse
from typing import Any, Dict, List from typing import Any, Dict, List
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
from app.auth.project_dependencies import get_metadata_repository from app.auth.project_dependencies import get_metadata_repository
from app.domain.schemas.metadata import ProjectMetaResponse, GeoServerConfigResponse from app.domain.schemas.metadata import ProjectMetaResponse
import app.services.project_info as project_info import app.services.project_info as project_info
from app.infra.db.postgresql.database import get_database_instance as get_pg_db from app.infra.db.postgresql.database import get_database_instance as get_pg_db
from app.infra.db.timescaledb.database import get_database_instance as get_ts_db from app.infra.db.timescaledb.database import get_database_instance as get_ts_db
@@ -42,7 +42,8 @@ inpDir = "data/" # Assuming data directory exists or is defined somewhere.
router = APIRouter() router = APIRouter()
lockedPrjs: Dict[str, str] = {} lockedPrjs: Dict[str, str] = {}
@router.get("/project_info/", summary="获取项目信息", description="从数据库获取项目的详细信息,包括地图范围等。", response_model=ProjectMetaResponse) @router.get("/project-info", summary="获取项目信息", description="从数据库获取项目的详细信息,包括地图范围等。", response_model=ProjectMetaResponse)
@router.get("/project_info/", summary="获取项目信息(旧路径)", description="从数据库获取项目的详细信息,包括地图范围等。", response_model=ProjectMetaResponse, deprecated=True)
async def get_project_info_endpoint( async def get_project_info_endpoint(
network: str = Query(..., description="管网名称(或项目代码)"), network: str = Query(..., description="管网名称(或项目代码)"),
metadata_repo: MetadataRepository = Depends(get_metadata_repository), metadata_repo: MetadataRepository = Depends(get_metadata_repository),
@@ -55,17 +56,6 @@ async def get_project_info_endpoint(
project_detail = await metadata_repo.get_project_detail_by_code(network) project_detail = await metadata_repo.get_project_detail_by_code(network)
if not project_detail: if not project_detail:
raise HTTPException(status_code=404, detail=f"Project {network} not found") raise HTTPException(status_code=404, detail=f"Project {network} not found")
geoserver_payload = None
if project_detail.geoserver:
geoserver_payload = GeoServerConfigResponse(
gs_base_url=project_detail.geoserver.gs_base_url,
gs_admin_user=project_detail.geoserver.gs_admin_user,
gs_datastore_name=project_detail.geoserver.gs_datastore_name,
default_extent=project_detail.geoserver.default_extent,
srid=project_detail.geoserver.srid,
)
return ProjectMetaResponse( return ProjectMetaResponse(
project_id=project_detail.project_id, project_id=project_detail.project_id,
name=project_detail.name, name=project_detail.name,
@@ -75,7 +65,6 @@ async def get_project_info_endpoint(
map_extent=project_detail.map_extent, map_extent=project_detail.map_extent,
status=project_detail.status, status=project_detail.status,
project_role="viewer", # Default role for public access project_role="viewer", # Default role for public access
geoserver=geoserver_payload
) )
@router.get("/listprojects/", summary="获取项目列表", description="获取服务器上所有可用的供水管网项目名称列表。") @router.get("/listprojects/", summary="获取项目列表", description="获取服务器上所有可用的供水管网项目名称列表。")
@@ -133,7 +122,8 @@ async def is_project_open_endpoint(
""" """
return is_project_open(network) return is_project_open(network)
@router.post("/openproject/", summary="打开项目", description="将指定项目加载到内存中,并初始化数据库连接池。") @router.post("/projects/open", summary="打开项目", description="将指定项目加载到内存中,并初始化数据库连接池。")
@router.post("/openproject/", summary="打开项目(旧路径)", description="将指定项目加载到内存中,并初始化数据库连接池。", deprecated=True)
async def open_project_endpoint( async def open_project_endpoint(
network: str = Query(..., description="管网名称(或数据库名称)") network: str = Query(..., description="管网名称(或数据库名称)")
): ):
+2 -1
View File
@@ -22,7 +22,8 @@ async def fastapi_get_scheme(network: str = Query(..., description="管网名称
""" """
return get_scheme(network, schema_name) return get_scheme(network, schema_name)
@router.get("/getallschemes/", summary="获取所有方案", description="获取指定网络的所有方案信息") @router.get("/schemes", summary="获取所有方案", description="获取指定网络的所有方案信息")
@router.get("/getallschemes/", summary="获取所有方案(旧路径)", description="获取指定网络的所有方案信息", deprecated=True)
async def fastapi_get_all_schemes(network: str = Query(..., description="管网名称(或数据库名称)")) -> list[dict[Any, Any]]: async def fastapi_get_all_schemes(network: str = Query(..., description="管网名称(或数据库名称)")) -> list[dict[Any, Any]]:
""" """
获取所有方案列表 获取所有方案列表
+12 -6
View File
@@ -188,7 +188,8 @@ async def dump_output_endpoint(output: str = Query(..., description="模拟输
# Analysis Endpoints # Analysis Endpoints
@router.get("/burst_analysis/", summary="爆管分析(高级)", description="高级版本的爆管分析,支持在指定时间点修改泵控制模式和阀门开度,以分析这些改变对爆管影响的作用。支持固定泵和变速泵的独立控制。") @router.get("/burst-analysis", summary="爆管分析(高级)", description="高级版本的爆管分析,支持在指定时间点修改泵控制模式和阀门开度,以分析这些改变对爆管影响的作用。支持固定泵和变速泵的独立控制。")
@router.get("/burst_analysis/", summary="爆管分析(高级,旧路径)", description="高级版本的爆管分析,支持在指定时间点修改泵控制模式和阀门开度,以分析这些改变对爆管影响的作用。支持固定泵和变速泵的独立控制。", deprecated=True)
async def fastapi_burst_analysis( async def fastapi_burst_analysis(
network: str = Query(..., description="管网名称(或数据库名称)"), network: str = Query(..., description="管网名称(或数据库名称)"),
modify_pattern_start_time: str = Query(..., description="模式修改开始时间(ISO 8601格式)"), modify_pattern_start_time: str = Query(..., description="模式修改开始时间(ISO 8601格式)"),
@@ -249,7 +250,8 @@ async def fastapi_valve_close_analysis(
return result or "success" return result or "success"
@router.get("/valve_isolation_analysis/", summary="阀门隔离分析", description="分析当发生突发事件时,通过关闭指定阀门进行隔离,确定哪些阀门必须关闭、哪些可选关闭,以及隔离的可行性。") @router.get("/valve-isolation-analysis", summary="阀门隔离分析", description="分析当发生突发事件时,通过关闭指定阀门进行隔离,确定哪些阀门必须关闭、哪些可选关闭,以及隔离的可行性。")
@router.get("/valve_isolation_analysis/", summary="阀门隔离分析(旧路径)", description="分析当发生突发事件时,通过关闭指定阀门进行隔离,确定哪些阀门必须关闭、哪些可选关闭,以及隔离的可行性。", deprecated=True)
async def valve_isolation_endpoint( async def valve_isolation_endpoint(
network: str = Query(..., description="管网名称(或数据库名称)"), network: str = Query(..., description="管网名称(或数据库名称)"),
accident_element: List[str] = Query(..., description="发生事故的管段/节点ID列表"), accident_element: List[str] = Query(..., description="发生事故的管段/节点ID列表"),
@@ -289,7 +291,8 @@ async def valve_isolation_endpoint(
return result return result
@router.get("/flushing_analysis/", response_class=PlainTextResponse, summary="冲洗分析(高级)", description="高级版本的冲洗分析,支持同时开启多个阀门进行冲洗,指定排污节点,并设置固定的冲洗流量。返回纯文本格式的分析结果。") @router.get("/flushing-analysis", response_class=PlainTextResponse, summary="冲洗分析(高级)", description="高级版本的冲洗分析,支持同时开启多个阀门进行冲洗,指定排污节点,并设置固定的冲洗流量。返回纯文本格式的分析结果。")
@router.get("/flushing_analysis/", response_class=PlainTextResponse, summary="冲洗分析(高级,旧路径)", description="高级版本的冲洗分析,支持同时开启多个阀门进行冲洗,指定排污节点,并设置固定的冲洗流量。返回纯文本格式的分析结果。", deprecated=True)
async def fastapi_flushing_analysis( async def fastapi_flushing_analysis(
network: str = Query(..., description="管网名称(或数据库名称)"), network: str = Query(..., description="管网名称(或数据库名称)"),
start_time: str = Query(..., description="冲洗开始时间(ISO 8601格式)"), start_time: str = Query(..., description="冲洗开始时间(ISO 8601格式)"),
@@ -329,7 +332,8 @@ async def fastapi_flushing_analysis(
return result or "success" return result or "success"
@router.get("/contaminant_simulation/", response_class=PlainTextResponse, summary="污染物模拟", description="对管网中的污染物扩散进行模拟,评估污染源对管网的影响范围和浓度分布。支持指定污染源位置、污染浓度和扩散模式。") @router.get("/contaminant-simulation", response_class=PlainTextResponse, summary="污染物模拟", description="对管网中的污染物扩散进行模拟,评估污染源对管网的影响范围和浓度分布。支持指定污染源位置、污染浓度和扩散模式。")
@router.get("/contaminant_simulation/", response_class=PlainTextResponse, summary="污染物模拟(旧路径)", description="对管网中的污染物扩散进行模拟,评估污染源对管网的影响范围和浓度分布。支持指定污染源位置、污染浓度和扩散模式。", deprecated=True)
async def fastapi_contaminant_simulation( async def fastapi_contaminant_simulation(
network: str = Query(..., description="管网名称(或数据库名称)"), network: str = Query(..., description="管网名称(或数据库名称)"),
start_time: str = Query(..., description="污染开始时间(ISO 8601格式)"), start_time: str = Query(..., description="污染开始时间(ISO 8601格式)"),
@@ -715,7 +719,8 @@ async def fastapi_pressure_sensor_placement_kmeans(
) )
@router.post("/sensorplacementscheme/create", summary="传感器放置方案创建", description="创建新的传感器放置方案,支持灵敏度分析和KMeans聚类两种方法。根据指定的方法自动计算最优的传感器放置位置。") @router.post("/sensor-placement-schemes", summary="传感器放置方案创建", description="创建新的传感器放置方案,支持灵敏度分析和KMeans聚类两种方法。根据指定的方法自动计算最优的传感器放置位置。")
@router.post("/sensorplacementscheme/create", summary="传感器放置方案创建(旧路径)", description="创建新的传感器放置方案,支持灵敏度分析和KMeans聚类两种方法。根据指定的方法自动计算最优的传感器放置位置。", deprecated=True)
async def fastapi_pressure_sensor_placement( async def fastapi_pressure_sensor_placement(
network: str = Query(..., description="管网名称(或数据库名称)"), network: str = Query(..., description="管网名称(或数据库名称)"),
scheme_name: str = Query(..., description="放置方案名称"), scheme_name: str = Query(..., description="放置方案名称"),
@@ -763,7 +768,8 @@ async def fastapi_pressure_sensor_placement(
return "success" return "success"
@router.post("/runsimulationmanuallybydate/", summary="手动运行日期指定模拟", description="根据指定的开始时间和持续时间,手动运行水力模拟。开始时间必须是显式带时区的 ISO 8601 / RFC3339 时间。") @router.post("/simulations/run-by-date", summary="手动运行日期指定模拟", description="根据指定的开始时间和持续时间,手动运行水力模拟。开始时间必须是显式带时区的 ISO 8601 / RFC3339 时间。")
@router.post("/runsimulationmanuallybydate/", summary="手动运行日期指定模拟(旧路径)", description="根据指定的开始时间和持续时间,手动运行水力模拟。开始时间必须是显式带时区的 ISO 8601 / RFC3339 时间。", deprecated=True)
async def fastapi_run_simulation_manually_by_date( async def fastapi_run_simulation_manually_by_date(
data: RunSimulationManuallyByDate = Body(..., description="模拟运行参数"), data: RunSimulationManuallyByDate = Body(..., description="模拟运行参数"),
) -> dict[str, str]: ) -> dict[str, str]:
-215
View File
@@ -1,215 +0,0 @@
"""
用户管理 API 接口
演示权限控制的使用
"""
from typing import List
from fastapi import APIRouter, Depends, HTTPException, status, Path, Query
from app.domain.schemas.user import UserResponse, UserUpdate, UserCreate
from app.domain.models.role import UserRole
from app.domain.schemas.user import UserInDB
from app.infra.db.metadb.repositories.user_repository import UserRepository
from app.auth.dependencies import get_user_repository, get_current_active_user
from app.auth.permissions import get_current_admin, require_role, check_resource_owner
router = APIRouter()
@router.get(
"/",
summary="列出所有用户",
description="获取用户列表(仅管理员)",
response_model=List[UserResponse],
)
async def list_users(
skip: int = Query(0, ge=0, description="跳过的用户数"),
limit: int = Query(100, ge=1, le=1000, description="返回的最大用户数"),
current_user: UserInDB = Depends(require_role(UserRole.ADMIN)),
user_repo: UserRepository = Depends(get_user_repository),
) -> List[UserResponse]:
"""
获取用户列表
获取系统中所有的用户信息(需要管理员权限)
"""
users = await user_repo.get_all_users(skip=skip, limit=limit)
return [UserResponse.model_validate(user) for user in users]
@router.get(
"/{user_id}",
summary="获取用户详情",
description="获取指定用户的详细信息",
response_model=UserResponse,
)
async def get_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_active_user),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
获取用户详情
管理员可查看所有用户,普通用户只能查看自己
"""
# 检查权限
if not check_resource_owner(user_id, current_user):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="You don't have permission to view this user",
)
user = await user_repo.get_user_by_id(user_id)
if not user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return UserResponse.model_validate(user)
@router.put(
"/{user_id}",
summary="更新用户信息",
description="更新指定用户的信息",
response_model=UserResponse,
)
async def update_user(
user_id: int = Path(..., gt=0, description="用户ID"),
user_update: UserUpdate = None,
current_user: UserInDB = Depends(get_current_active_user),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
更新用户信息
管理员可更新所有用户,普通用户只能更新自己(且不能修改角色)
"""
# 检查用户是否存在
target_user = await user_repo.get_user_by_id(user_id)
if not target_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
# 权限检查
is_owner = current_user.id == user_id
is_admin = UserRole(current_user.role).has_permission(UserRole.ADMIN)
if not is_owner and not is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="You don't have permission to update this user",
)
# 非管理员不能修改角色和激活状态
if not is_admin:
if user_update.role is not None:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only admins can change user roles",
)
if user_update.is_active is not None:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only admins can change user active status",
)
# 更新用户
updated_user = await user_repo.update_user(user_id, user_update)
if not updated_user:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to update user",
)
return UserResponse.model_validate(updated_user)
@router.delete("/{user_id}", summary="删除用户", description="删除指定用户(仅管理员)")
async def delete_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_admin),
user_repo: UserRepository = Depends(get_user_repository),
) -> dict:
"""
删除用户
删除指定用户(需要管理员权限,不能删除自己)
"""
# 不能删除自己
if current_user.id == user_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="You cannot delete your own account",
)
success = await user_repo.delete_user(user_id)
if not success:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return {"message": "User deleted successfully"}
@router.post(
"/{user_id}/activate",
summary="激活用户",
description="激活指定用户账户(仅管理员)",
response_model=UserResponse,
)
async def activate_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_admin),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
激活用户
激活指定用户的账户(需要管理员权限)
"""
user_update = UserUpdate(is_active=True)
updated_user = await user_repo.update_user(user_id, user_update)
if not updated_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return UserResponse.model_validate(updated_user)
@router.post(
"/{user_id}/deactivate",
summary="停用用户",
description="停用指定用户账户(仅管理员)",
response_model=UserResponse,
)
async def deactivate_user(
user_id: int = Path(..., gt=0, description="用户ID"),
current_user: UserInDB = Depends(get_current_admin),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserResponse:
"""
停用用户
停用指定用户的账户(需要管理员权限,不能停用自己)
"""
# 不能停用自己
if current_user.id == user_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="You cannot deactivate your own account",
)
user_update = UserUpdate(is_active=False)
updated_user = await user_repo.update_user(user_id, user_update)
if not updated_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND, detail="User not found"
)
return UserResponse.model_validate(updated_user)
+5 -5
View File
@@ -1,6 +1,7 @@
from fastapi import APIRouter from fastapi import APIRouter
from app.api.v1.endpoints import ( from app.api.v1.endpoints import (
auth, admin_metadata,
agent_auth,
project, project,
simulation, simulation,
scada, scada,
@@ -15,7 +16,6 @@ from app.api.v1.endpoints import (
leakage, leakage,
burst_detection, burst_detection,
burst_location, burst_location,
user_management, # 新增:用户管理
audit, # 新增:审计日志 audit, # 新增:审计日志
meta, meta,
web_search, web_search,
@@ -54,10 +54,10 @@ from app.api.v1.endpoints.timeseries import (
api_router = APIRouter() api_router = APIRouter()
# Core Services # Core Services
api_router.include_router(auth.router, prefix="/auth", tags=["Auth"]) api_router.include_router(agent_auth.router, tags=["Agent Auth"])
api_router.include_router( api_router.include_router(
user_management.router, prefix="/users", tags=["User Management"] admin_metadata.router, prefix="/admin", tags=["Metadata Admin"]
) # 新增 )
api_router.include_router(audit.router, prefix="/audit", tags=["Audit Logs"]) # 新增 api_router.include_router(audit.router, prefix="/audit", tags=["Audit Logs"]) # 新增
api_router.include_router(meta.router, tags=["Metadata"]) api_router.include_router(meta.router, tags=["Metadata"])
api_router.include_router(project.router, tags=["Project"]) api_router.include_router(project.router, tags=["Project"])
-100
View File
@@ -1,100 +0,0 @@
from typing import Annotated, Optional
from fastapi import Depends, HTTPException, status, Request
from fastapi.security import OAuth2PasswordBearer
from jose import jwt, JWTError
from app.core.config import settings
from app.domain.schemas.user import UserInDB, TokenPayload
from app.infra.db.metadb.repositories.user_repository import UserRepository
from app.infra.db.postgresql.database import Database
oauth2_scheme = OAuth2PasswordBearer(tokenUrl=f"{settings.API_V1_STR}/auth/login")
# 数据库依赖
async def get_db(request: Request) -> Database:
"""
获取数据库实例
从 FastAPI app.state 中获取在启动时初始化的数据库连接
"""
if not hasattr(request.app.state, "db"):
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Database not initialized",
)
return request.app.state.db
async def get_user_repository(db: Database = Depends(get_db)) -> UserRepository:
"""获取用户仓储实例"""
return UserRepository(db)
async def get_current_user(
token: str = Depends(oauth2_scheme),
user_repo: UserRepository = Depends(get_user_repository),
) -> UserInDB:
"""
获取当前登录用户
从 JWT Token 中解析用户信息,并从数据库验证
"""
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate credentials",
headers={"WWW-Authenticate": "Bearer"},
)
try:
payload = jwt.decode(
token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM]
)
username: str = payload.get("sub")
token_type: str = payload.get("type", "access")
if username is None:
raise credentials_exception
if token_type != "access":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token type. Access token required.",
headers={"WWW-Authenticate": "Bearer"},
)
except JWTError:
raise credentials_exception
# 从数据库获取用户
user = await user_repo.get_user_by_username(username)
if user is None:
raise credentials_exception
return user
async def get_current_active_user(
current_user: UserInDB = Depends(get_current_user),
) -> UserInDB:
"""
获取当前活跃用户(必须是激活状态)
"""
if not current_user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user"
)
return current_user
async def get_current_superuser(
current_user: UserInDB = Depends(get_current_user),
) -> UserInDB:
"""
获取当前超级管理员用户
"""
if not current_user.is_superuser:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Not enough privileges. Superuser access required.",
)
return current_user
+26 -43
View File
@@ -8,35 +8,41 @@ from jose import JWTError, jwt
from app.core.config import settings from app.core.config import settings
oauth2_optional = OAuth2PasswordBearer( oauth2_optional = OAuth2PasswordBearer(
tokenUrl=f"{settings.API_V1_STR}/auth/login", auto_error=False tokenUrl="keycloak", auto_error=False
) )
# logger = logging.getLogger(__name__) # logger = logging.getLogger(__name__)
async def get_current_keycloak_sub( def _decode_keycloak_token(token: str) -> dict:
if not settings.KEYCLOAK_PUBLIC_KEY:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Keycloak public key is not configured",
)
key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
return jwt.decode(
token,
key,
algorithms=[settings.KEYCLOAK_ALGORITHM],
audience=settings.KEYCLOAK_AUDIENCE or None,
)
async def get_current_keycloak_payload(
token: str | None = Depends(oauth2_optional), token: str | None = Depends(oauth2_optional),
) -> UUID: ) -> dict:
if not token: if not token:
raise HTTPException( raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, status_code=status.HTTP_401_UNAUTHORIZED,
detail="Not authenticated", detail="Not authenticated",
headers={"WWW-Authenticate": "Bearer"}, headers={"WWW-Authenticate": "Bearer"},
) )
if settings.KEYCLOAK_PUBLIC_KEY:
key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
algorithms = [settings.KEYCLOAK_ALGORITHM]
else:
key = settings.SECRET_KEY
algorithms = [settings.ALGORITHM]
try: try:
payload = jwt.decode( return _decode_keycloak_token(token)
token,
key,
algorithms=algorithms,
audience=settings.KEYCLOAK_AUDIENCE or None,
)
except JWTError as exc: except JWTError as exc:
# logger.warning("Keycloak token validation failed: %s", exc) # logger.warning("Keycloak token validation failed: %s", exc)
raise HTTPException( raise HTTPException(
@@ -45,6 +51,10 @@ async def get_current_keycloak_sub(
headers={"WWW-Authenticate": "Bearer"}, headers={"WWW-Authenticate": "Bearer"},
) from exc ) from exc
async def get_current_keycloak_sub(
payload: dict = Depends(get_current_keycloak_payload),
) -> UUID:
sub = payload.get("sub") sub = payload.get("sub")
if not sub: if not sub:
raise HTTPException( raise HTTPException(
@@ -64,35 +74,8 @@ async def get_current_keycloak_sub(
async def get_current_keycloak_username( async def get_current_keycloak_username(
token: str | None = Depends(oauth2_optional), payload: dict = Depends(get_current_keycloak_payload),
) -> str: ) -> str:
if not token:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Not authenticated",
headers={"WWW-Authenticate": "Bearer"},
)
if settings.KEYCLOAK_PUBLIC_KEY:
key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
algorithms = [settings.KEYCLOAK_ALGORITHM]
else:
key = settings.SECRET_KEY
algorithms = [settings.ALGORITHM]
try:
payload = jwt.decode(
token,
key,
algorithms=algorithms,
audience=settings.KEYCLOAK_AUDIENCE or None,
)
except JWTError as exc:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token",
headers={"WWW-Authenticate": "Bearer"},
) from exc
username = payload.get("preferred_username") or payload.get("username") username = payload.get("preferred_username") or payload.get("username")
if not username: if not username:
raise HTTPException( raise HTTPException(
+47 -3
View File
@@ -6,8 +6,7 @@ from fastapi import Depends, HTTPException, status
from sqlalchemy.exc import SQLAlchemyError from sqlalchemy.exc import SQLAlchemyError
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.auth.keycloak_dependencies import get_current_keycloak_sub from app.auth.keycloak_dependencies import get_current_keycloak_payload
from app.core.config import settings
from app.infra.db.metadb.database import get_metadata_session from app.infra.db.metadb.database import get_metadata_session
from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository from app.infra.db.metadb.repositories.metadata_repository import MetadataRepository
@@ -20,10 +19,40 @@ async def get_metadata_repository(
return MetadataRepository(session) return MetadataRepository(session)
def _keycloak_sub_from_payload(payload: dict) -> UUID:
sub = payload.get("sub")
if not sub:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Missing subject claim",
headers={"WWW-Authenticate": "Bearer"},
)
try:
return UUID(str(sub))
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid subject claim",
headers={"WWW-Authenticate": "Bearer"},
) from exc
def _username_from_payload(payload: dict) -> str | None:
username = payload.get("preferred_username") or payload.get("username")
return str(username) if username else None
def _email_from_payload(payload: dict) -> str | None:
email = payload.get("email")
return str(email) if email else None
async def get_current_metadata_user( async def get_current_metadata_user(
keycloak_sub: UUID = Depends(get_current_keycloak_sub), keycloak_payload: dict = Depends(get_current_keycloak_payload),
metadata_repo: MetadataRepository = Depends(get_metadata_repository), metadata_repo: MetadataRepository = Depends(get_metadata_repository),
): ):
keycloak_sub = _keycloak_sub_from_payload(keycloak_payload)
try: try:
user = await metadata_repo.get_user_by_keycloak_id(keycloak_sub) user = await metadata_repo.get_user_by_keycloak_id(keycloak_sub)
except SQLAlchemyError as exc: except SQLAlchemyError as exc:
@@ -39,6 +68,21 @@ async def get_current_metadata_user(
raise HTTPException( raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user" status_code=status.HTTP_403_FORBIDDEN, detail="Inactive user"
) )
try:
user = await metadata_repo.refresh_user_keycloak_snapshot(
user,
username=_username_from_payload(keycloak_payload),
email=_email_from_payload(keycloak_payload),
)
except SQLAlchemyError as exc:
logger.error(
"Metadata DB error while refreshing current user snapshot",
exc_info=True,
)
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=f"Metadata database error: {exc}",
) from exc
return user return user
-106
View File
@@ -1,106 +0,0 @@
"""
权限控制依赖项和装饰器
基于角色的访问控制(RBAC
"""
from typing import Callable
from fastapi import Depends, HTTPException, status
from app.domain.models.role import UserRole
from app.domain.schemas.user import UserInDB
from app.auth.dependencies import get_current_active_user
def require_role(required_role: UserRole):
"""
要求特定角色或更高权限
用法:
@router.get("/admin-only")
async def admin_endpoint(user: UserInDB = Depends(require_role(UserRole.ADMIN))):
...
Args:
required_role: 需要的最低角色
Returns:
依赖函数
"""
async def role_checker(
current_user: UserInDB = Depends(get_current_active_user)
) -> UserInDB:
user_role = UserRole(current_user.role)
if not user_role.has_permission(required_role):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f"Insufficient permissions. Required role: {required_role.value}, "
f"Your role: {user_role.value}"
)
return current_user
return role_checker
# 预定义的权限检查依赖
require_admin = require_role(UserRole.ADMIN)
require_operator = require_role(UserRole.OPERATOR)
require_user = require_role(UserRole.USER)
def get_current_admin(
current_user: UserInDB = Depends(require_admin)
) -> UserInDB:
"""
获取当前管理员用户
等同于 Depends(require_role(UserRole.ADMIN))
"""
return current_user
def get_current_operator(
current_user: UserInDB = Depends(require_operator)
) -> UserInDB:
"""
获取当前操作员用户(或更高权限)
等同于 Depends(require_role(UserRole.OPERATOR))
"""
return current_user
def check_resource_owner(user_id: int, current_user: UserInDB) -> bool:
"""
检查是否是资源拥有者或管理员
Args:
user_id: 资源拥有者ID
current_user: 当前用户
Returns:
是否有权限
"""
# 管理员可以访问所有资源
if UserRole(current_user.role).has_permission(UserRole.ADMIN):
return True
# 检查是否是资源拥有者
return current_user.id == user_id
def require_owner_or_admin(user_id: int):
"""
要求是资源拥有者或管理员
Args:
user_id: 资源拥有者ID
Returns:
依赖函数
"""
async def owner_or_admin_checker(
current_user: UserInDB = Depends(get_current_active_user)
) -> UserInDB:
if not check_resource_owner(user_id, current_user):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="You don't have permission to access this resource"
)
return current_user
return owner_or_admin_checker
+2
View File
@@ -25,6 +25,7 @@ logger = logging.getLogger(__name__)
@dataclass(frozen=True) @dataclass(frozen=True)
class ProjectContext: class ProjectContext:
project_id: UUID project_id: UUID
project_code: str
user_id: UUID user_id: UUID
project_role: str project_role: str
@@ -85,6 +86,7 @@ async def get_project_context(
return ProjectContext( return ProjectContext(
project_id=project.id, project_id=project.id,
project_code=project.code,
user_id=user.id, user_id=user.id,
project_role=membership_role, project_role=membership_role,
) )
+3 -12
View File
@@ -11,17 +11,8 @@ class Settings(BaseSettings):
NETWORK_NAME: str = "default_network" NETWORK_NAME: str = "default_network"
# JWT 配置 # 敏感配置加密密钥 (Fernet)
SECRET_KEY: str = ( DATABASE_ENCRYPTION_KEY: str = ""
"your-secret-key-here-change-in-production-use-openssl-rand-hex-32"
)
ALGORITHM: str = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES: int = 30
REFRESH_TOKEN_EXPIRE_DAYS: int = 7
# 数据加密密钥 (使用 Fernet)
ENCRYPTION_KEY: str = "" # 必须从环境变量设置
DATABASE_ENCRYPTION_KEY: str = "" # project_databases.dsn_encrypted 专用
# Database Config (PostgreSQL) # Database Config (PostgreSQL)
DB_NAME: str = "tjwater" DB_NAME: str = "tjwater"
@@ -59,7 +50,7 @@ class Settings(BaseSettings):
PROJECT_TS_POOL_MIN_SIZE: int = 1 PROJECT_TS_POOL_MIN_SIZE: int = 1
PROJECT_TS_POOL_MAX_SIZE: int = 10 PROJECT_TS_POOL_MAX_SIZE: int = 10
# Keycloak JWT (optional override) # Keycloak access token verification
KEYCLOAK_PUBLIC_KEY: str = "" KEYCLOAK_PUBLIC_KEY: str = ""
KEYCLOAK_ALGORITHM: str = "RS256" KEYCLOAK_ALGORITHM: str = "RS256"
KEYCLOAK_AUDIENCE: str = "" KEYCLOAK_AUDIENCE: str = ""
+3 -7
View File
@@ -20,10 +20,10 @@ class Encryptor:
key: 加密密钥,如果为 None 则从环境变量读取 key: 加密密钥,如果为 None 则从环境变量读取
""" """
if key is None: if key is None:
key_str = os.getenv("ENCRYPTION_KEY") or settings.ENCRYPTION_KEY key_str = os.getenv("DATABASE_ENCRYPTION_KEY") or settings.DATABASE_ENCRYPTION_KEY
if not key_str: if not key_str:
raise ValueError( raise ValueError(
"ENCRYPTION_KEY not found in environment variables or .env. " "DATABASE_ENCRYPTION_KEY not found in environment variables or .env. "
"Generate one using: Encryptor.generate_key()" "Generate one using: Encryptor.generate_key()"
) )
key = key_str.encode() key = key_str.encode()
@@ -80,15 +80,13 @@ _database_encryptor: Optional[Encryptor] = None
def is_encryption_configured() -> bool: def is_encryption_configured() -> bool:
return bool(os.getenv("ENCRYPTION_KEY") or settings.ENCRYPTION_KEY) return is_database_encryption_configured()
def is_database_encryption_configured() -> bool: def is_database_encryption_configured() -> bool:
return bool( return bool(
os.getenv("DATABASE_ENCRYPTION_KEY") os.getenv("DATABASE_ENCRYPTION_KEY")
or settings.DATABASE_ENCRYPTION_KEY or settings.DATABASE_ENCRYPTION_KEY
or os.getenv("ENCRYPTION_KEY")
or settings.ENCRYPTION_KEY
) )
@@ -107,8 +105,6 @@ def get_database_encryptor() -> Encryptor:
key_str = ( key_str = (
os.getenv("DATABASE_ENCRYPTION_KEY") os.getenv("DATABASE_ENCRYPTION_KEY")
or settings.DATABASE_ENCRYPTION_KEY or settings.DATABASE_ENCRYPTION_KEY
or os.getenv("ENCRYPTION_KEY")
or settings.ENCRYPTION_KEY
) )
if not key_str: if not key_str:
raise ValueError( raise ValueError(
-95
View File
@@ -1,95 +0,0 @@
from datetime import datetime, timedelta, timezone
from typing import Optional, Union, Any
from jose import jwt
from passlib.context import CryptContext
from app.core.config import settings
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
def _utc_now() -> datetime:
return datetime.now(timezone.utc)
def create_access_token(
subject: Union[str, Any], expires_delta: Optional[timedelta] = None
) -> str:
"""
创建 JWT Access Token
Args:
subject: 用户标识(通常是用户名或用户ID)
expires_delta: 过期时间增量
Returns:
JWT token 字符串
"""
if expires_delta:
expire = _utc_now() + expires_delta
else:
expire = _utc_now() + timedelta(
minutes=settings.ACCESS_TOKEN_EXPIRE_MINUTES
)
to_encode = {
"exp": expire,
"sub": str(subject),
"type": "access",
"iat": _utc_now(),
}
encoded_jwt = jwt.encode(
to_encode, settings.SECRET_KEY, algorithm=settings.ALGORITHM
)
return encoded_jwt
def create_refresh_token(subject: Union[str, Any]) -> str:
"""
创建 JWT Refresh Token(长期有效)
Args:
subject: 用户标识
Returns:
JWT refresh token 字符串
"""
expire = _utc_now() + timedelta(days=settings.REFRESH_TOKEN_EXPIRE_DAYS)
to_encode = {
"exp": expire,
"sub": str(subject),
"type": "refresh",
"iat": _utc_now(),
}
encoded_jwt = jwt.encode(
to_encode, settings.SECRET_KEY, algorithm=settings.ALGORITHM
)
return encoded_jwt
def verify_password(plain_password: str, hashed_password: str) -> bool:
"""
验证密码
Args:
plain_password: 明文密码
hashed_password: 密码哈希
Returns:
是否匹配
"""
return pwd_context.verify(plain_password, hashed_password)
def get_password_hash(password: str) -> str:
"""
生成密码哈希
Args:
password: 明文密码
Returns:
bcrypt 哈希字符串
"""
return pwd_context.hash(password)
-36
View File
@@ -1,36 +0,0 @@
from enum import Enum
class UserRole(str, Enum):
"""用户角色枚举"""
ADMIN = "ADMIN" # 管理员 - 完全权限
OPERATOR = "OPERATOR" # 操作员 - 可修改数据
USER = "USER" # 普通用户 - 读写权限
VIEWER = "VIEWER" # 观察者 - 仅查询权限
def __str__(self):
return self.value
@classmethod
def get_hierarchy(cls) -> dict:
"""
获取角色层级(数字越大权限越高)
"""
return {
cls.VIEWER: 1,
cls.USER: 2,
cls.OPERATOR: 3,
cls.ADMIN: 4,
}
def has_permission(self, required_role: 'UserRole') -> bool:
"""
检查当前角色是否有足够权限
Args:
required_role: 需要的最低角色
Returns:
True if has permission
"""
hierarchy = self.get_hierarchy()
return hierarchy[self] >= hierarchy[required_role]
+134
View File
@@ -0,0 +1,134 @@
from datetime import datetime
from typing import Literal
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, model_validator
BusinessRole = Literal["admin", "user", "operator", "viewer"]
ProjectRole = Literal["owner", "admin", "member", "viewer"]
ProjectStatus = Literal["active", "inactive", "archived"]
ProjectDbRole = Literal["biz_data", "iot_data"]
class MetadataUserSyncRequest(BaseModel):
keycloak_id: UUID
username: str = Field(..., min_length=1, max_length=50)
email: str = Field(..., min_length=1, max_length=100)
role: BusinessRole = "user"
is_active: bool = True
class MetadataUsersBatchSyncRequest(BaseModel):
users: list[MetadataUserSyncRequest] = Field(..., min_length=1, max_length=500)
class MetadataUserUpdateRequest(BaseModel):
role: BusinessRole | None = None
is_active: bool | None = None
class MetadataUserResponse(BaseModel):
id: UUID
keycloak_id: UUID
username: str
email: str
role: str
is_active: bool
is_superuser: bool
created_at: datetime
updated_at: datetime
last_login_at: datetime | None = None
model_config = ConfigDict(from_attributes=True)
class MetadataUserSyncResult(BaseModel):
keycloak_id: UUID
user: MetadataUserResponse | None = None
success: bool
error: str | None = None
class ProjectMemberCreateRequest(BaseModel):
user_id: UUID
project_role: ProjectRole = "viewer"
class ProjectMemberUpdateRequest(BaseModel):
project_role: ProjectRole
class ProjectMemberResponse(BaseModel):
id: UUID
user_id: UUID
project_id: UUID
project_role: str
username: str
email: str
is_active: bool
class AdminProjectCreateRequest(BaseModel):
name: str = Field(..., min_length=1, max_length=100)
code: str = Field(..., min_length=1, max_length=50)
description: str | None = None
gs_workspace: str = Field(..., min_length=1, max_length=100)
map_extent: dict | None = None
status: ProjectStatus = "active"
class AdminProjectUpdateRequest(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=100)
code: str | None = Field(default=None, min_length=1, max_length=50)
description: str | None = None
gs_workspace: str | None = Field(default=None, min_length=1, max_length=100)
map_extent: dict | None = None
status: ProjectStatus | None = None
class AdminProjectResponse(BaseModel):
project_id: UUID
name: str
code: str
description: str | None = None
gs_workspace: str
map_extent: dict | None = None
status: str
created_at: datetime
updated_at: datetime
class ProjectDatabaseUpsertRequest(BaseModel):
db_role: ProjectDbRole
dsn: str | None = Field(default=None, min_length=1)
pool_min_size: int = Field(default=2, ge=1)
pool_max_size: int = Field(default=10, ge=1)
@model_validator(mode="after")
def validate_pool_bounds(self):
if self.pool_max_size < self.pool_min_size:
raise ValueError("pool_max_size must be greater than or equal to pool_min_size")
return self
class ProjectDatabaseResponse(BaseModel):
id: UUID
project_id: UUID
db_role: str
db_type: str
pool_min_size: int
pool_max_size: int
has_dsn: bool
class ProjectDatabaseHealthRequest(BaseModel):
dsn: str | None = Field(default=None, min_length=1)
class ProjectDatabaseHealthResponse(BaseModel):
project_id: UUID
db_role: str
db_type: str
ok: bool
detail: str
-9
View File
@@ -4,14 +4,6 @@ from uuid import UUID
from pydantic import BaseModel from pydantic import BaseModel
class GeoServerConfigResponse(BaseModel):
gs_base_url: Optional[str] = None
gs_admin_user: Optional[str] = None
gs_datastore_name: str
default_extent: Optional[dict] = None
srid: int
class ProjectMetaResponse(BaseModel): class ProjectMetaResponse(BaseModel):
project_id: UUID project_id: UUID
name: str name: str
@@ -21,7 +13,6 @@ class ProjectMetaResponse(BaseModel):
map_extent: Optional[dict] = None map_extent: Optional[dict] = None
status: str status: str
project_role: str project_role: str
geoserver: Optional[GeoServerConfigResponse] = None
class ProjectSummaryResponse(BaseModel): class ProjectSummaryResponse(BaseModel):
-68
View File
@@ -1,68 +0,0 @@
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, EmailStr, Field, ConfigDict
from app.domain.models.role import UserRole
# ============================================
# Request Schemas (输入)
# ============================================
class UserCreate(BaseModel):
"""用户注册"""
username: str = Field(..., min_length=3, max_length=50,
description="用户名,3-50个字符")
email: EmailStr = Field(..., description="邮箱地址")
password: str = Field(..., min_length=6, max_length=100,
description="密码,至少6个字符")
role: UserRole = Field(default=UserRole.USER, description="用户角色")
class UserLogin(BaseModel):
"""用户登录"""
username: str = Field(..., description="用户名或邮箱")
password: str = Field(..., description="密码")
class UserUpdate(BaseModel):
"""用户信息更新"""
email: Optional[EmailStr] = None
password: Optional[str] = Field(None, min_length=6, max_length=100)
role: Optional[UserRole] = None
is_active: Optional[bool] = None
# ============================================
# Response Schemas (输出)
# ============================================
class UserResponse(BaseModel):
"""用户信息响应(不含密码)"""
id: int
username: str
email: str
role: UserRole
is_active: bool
is_superuser: bool
created_at: datetime
updated_at: datetime
model_config = ConfigDict(from_attributes=True)
class UserInDB(UserResponse):
"""数据库中的用户(含密码哈希)"""
hashed_password: str
# ============================================
# Token Schemas
# ============================================
class Token(BaseModel):
"""JWT Token 响应"""
access_token: str
refresh_token: Optional[str] = None
token_type: str = "bearer"
expires_in: int = Field(..., description="过期时间(秒)")
class TokenPayload(BaseModel):
"""JWT Token Payload"""
sub: str = Field(..., description="用户ID或用户名")
exp: Optional[int] = None
iat: Optional[int] = None
type: str = Field(default="access", description="token类型: access 或 refresh")
+6 -14
View File
@@ -33,8 +33,6 @@ class AuditMiddleware(BaseHTTPMiddleware):
# 需要审计的路径前缀 # 需要审计的路径前缀
AUDIT_PATHS = [ AUDIT_PATHS = [
# "/api/v1/auth/",
# "/api/v1/users/",
# "/api/v1/projects/", # "/api/v1/projects/",
# "/api/v1/networks/", # "/api/v1/networks/",
] ]
@@ -193,20 +191,14 @@ class AuditMiddleware(BaseHTTPMiddleware):
return None return None
sub = None sub = None
try: try:
key = ( if not settings.KEYCLOAK_PUBLIC_KEY:
settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n") return None
if settings.KEYCLOAK_PUBLIC_KEY
else settings.SECRET_KEY key = settings.KEYCLOAK_PUBLIC_KEY.replace("\\n", "\n")
)
algorithms = (
[settings.KEYCLOAK_ALGORITHM]
if settings.KEYCLOAK_PUBLIC_KEY
else [settings.ALGORITHM]
)
payload = jwt.decode( payload = jwt.decode(
token, token,
key, key,
algorithms=algorithms, algorithms=[settings.KEYCLOAK_ALGORITHM],
audience=settings.KEYCLOAK_AUDIENCE or None, audience=settings.KEYCLOAK_AUDIENCE or None,
) )
sub = payload.get("sub") sub = payload.get("sub")
@@ -221,7 +213,7 @@ class AuditMiddleware(BaseHTTPMiddleware):
keycloak_id = UUID(sub) keycloak_id = UUID(sub)
user = await repo.get_user_by_keycloak_id(keycloak_id) user = await repo.get_user_by_keycloak_id(keycloak_id)
except ValueError: except ValueError:
user = await repo.get_user_by_username(sub) return None
if user and user.is_active: if user and user.is_active:
return user.id return user.id
return None return None
+2 -2
View File
@@ -54,9 +54,9 @@ class ProjectConnectionManager:
def _normalize_pg_url(self, url: str) -> str: def _normalize_pg_url(self, url: str) -> str:
parsed = make_url(url) parsed = make_url(url)
if parsed.drivername == "postgresql": if parsed.drivername in {"postgresql", "postgres"}:
parsed = parsed.set(drivername="postgresql+psycopg") parsed = parsed.set(drivername="postgresql+psycopg")
return str(parsed) return parsed.render_as_string(hide_password=False)
async def get_pg_sessionmaker( async def get_pg_sessionmaker(
self, self,
-20
View File
@@ -64,26 +64,6 @@ class ProjectDatabase(Base):
pool_max_size: Mapped[int] = mapped_column(Integer, default=10) pool_max_size: Mapped[int] = mapped_column(Integer, default=10)
class ProjectGeoServerConfig(Base):
__tablename__ = "project_geoserver_configs"
id: Mapped[UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True)
project_id: Mapped[UUID] = mapped_column(
PGUUID(as_uuid=True), unique=True, index=True
)
gs_base_url: Mapped[str | None] = mapped_column(Text, nullable=True)
gs_admin_user: Mapped[str | None] = mapped_column(String(50), nullable=True)
gs_admin_password_encrypted: Mapped[str | None] = mapped_column(
Text, nullable=True
)
gs_datastore_name: Mapped[str] = mapped_column(String(100), default="ds_postgis")
default_extent: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
srid: Mapped[int] = mapped_column(Integer, default=4326)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.utcnow
)
class UserProjectMembership(Base): class UserProjectMembership(Base):
__tablename__ = "user_project_membership" __tablename__ = "user_project_membership"
@@ -1,16 +1,15 @@
from dataclasses import dataclass from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Optional, List from typing import Optional, List
from uuid import UUID from uuid import UUID, uuid4
from cryptography.fernet import InvalidToken from cryptography.fernet import InvalidToken
from sqlalchemy import select from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.core.encryption import ( from app.core.encryption import (
get_database_encryptor, get_database_encryptor,
get_encryptor,
is_database_encryption_configured, is_database_encryption_configured,
is_encryption_configured,
) )
from app.infra.db.metadb import models from app.infra.db.metadb import models
@@ -43,17 +42,6 @@ class ProjectDbRouting:
pool_max_size: int pool_max_size: int
@dataclass(frozen=True)
class ProjectGeoServerInfo:
project_id: UUID
gs_base_url: Optional[str]
gs_admin_user: Optional[str]
gs_admin_password: Optional[str]
gs_datastore_name: str
default_extent: Optional[dict]
srid: int
@dataclass(frozen=True) @dataclass(frozen=True)
class ProjectSummary: class ProjectSummary:
project_id: UUID project_id: UUID
@@ -75,7 +63,27 @@ class ProjectDetail:
gs_workspace: str gs_workspace: str
map_extent: Optional[dict] map_extent: Optional[dict]
status: str status: str
geoserver: Optional[ProjectGeoServerInfo]
@dataclass(frozen=True)
class ProjectMemberSummary:
id: UUID
user_id: UUID
project_id: UUID
project_role: str
username: str
email: str
is_active: bool
def _utcnow() -> datetime:
return datetime.now(timezone.utc)
def _encrypt_database_secret(value: str) -> str:
if not is_database_encryption_configured():
raise ValueError("DATABASE_ENCRYPTION_KEY is not configured")
return get_database_encryptor().encrypt(value)
class MetadataRepository: class MetadataRepository:
@@ -96,6 +104,86 @@ class MetadataRepository:
) )
return result.scalar_one_or_none() return result.scalar_one_or_none()
async def get_user_by_id(self, user_id: UUID) -> Optional[models.User]:
result = await self.session.execute(
select(models.User).where(models.User.id == user_id)
)
return result.scalar_one_or_none()
async def list_users(self, skip: int = 0, limit: int = 100) -> List[models.User]:
result = await self.session.execute(
select(models.User)
.order_by(models.User.created_at.desc())
.offset(skip)
.limit(limit)
)
return list(result.scalars().all())
async def upsert_user_from_keycloak(
self,
*,
keycloak_id: UUID,
username: str,
email: str,
role: str,
is_active: bool,
) -> models.User:
user = await self.get_user_by_keycloak_id(keycloak_id)
if user is None:
user = models.User(
id=uuid4(),
keycloak_id=keycloak_id,
username=username,
email=email,
role=role,
is_active=is_active,
is_superuser=False,
)
self.session.add(user)
else:
user.username = username
user.email = email
user.role = role
user.is_active = is_active
await self.session.commit()
await self.session.refresh(user)
return user
async def refresh_user_keycloak_snapshot(
self,
user: models.User,
*,
username: str | None,
email: str | None,
last_login_at: datetime | None = None,
) -> models.User:
if username:
user.username = username
if email:
user.email = email
user.last_login_at = last_login_at or _utcnow()
user.updated_at = _utcnow()
await self.session.commit()
await self.session.refresh(user)
return user
async def update_user_admin(
self,
user_id: UUID,
*,
updates: dict,
) -> Optional[models.User]:
user = await self.get_user_by_id(user_id)
if user is None:
return None
if "role" in updates:
user.role = updates["role"]
if "is_active" in updates:
user.is_active = updates["is_active"]
await self.session.commit()
await self.session.refresh(user)
return user
async def get_project_by_id(self, project_id: UUID) -> Optional[models.Project]: async def get_project_by_id(self, project_id: UUID) -> Optional[models.Project]:
result = await self.session.execute( result = await self.session.execute(
select(models.Project).where(models.Project.id == project_id) select(models.Project).where(models.Project.id == project_id)
@@ -108,13 +196,66 @@ class MetadataRepository:
) )
return result.scalar_one_or_none() return result.scalar_one_or_none()
async def list_project_records(self) -> List[models.Project]:
result = await self.session.execute(
select(models.Project).order_by(models.Project.name)
)
return list(result.scalars().all())
async def create_project(
self,
*,
name: str,
code: str,
description: str | None,
gs_workspace: str,
map_extent: dict | None,
status: str,
) -> models.Project:
project = models.Project(
id=uuid4(),
name=name,
code=code,
description=description,
gs_workspace=gs_workspace,
map_extent=map_extent,
status=status,
created_at=_utcnow(),
updated_at=_utcnow(),
)
self.session.add(project)
await self.session.commit()
await self.session.refresh(project)
return project
async def update_project(
self,
project_id: UUID,
*,
updates: dict,
) -> Optional[models.Project]:
project = await self.get_project_by_id(project_id)
if project is None:
return None
for field in (
"name",
"code",
"description",
"gs_workspace",
"map_extent",
"status",
):
if field in updates:
setattr(project, field, updates[field])
project.updated_at = _utcnow()
await self.session.commit()
await self.session.refresh(project)
return project
async def get_project_detail_by_code(self, code: str) -> Optional[ProjectDetail]: async def get_project_detail_by_code(self, code: str) -> Optional[ProjectDetail]:
project = await self.get_project_by_code(code) project = await self.get_project_by_code(code)
if not project: if not project:
return None return None
geoserver = await self.get_geoserver_config(project.id)
return ProjectDetail( return ProjectDetail(
project_id=project.id, project_id=project.id,
name=project.name, name=project.name,
@@ -123,7 +264,6 @@ class MetadataRepository:
gs_workspace=project.gs_workspace, gs_workspace=project.gs_workspace,
map_extent=project.map_extent, map_extent=project.map_extent,
status=project.status, status=project.status,
geoserver=geoserver
) )
async def get_membership_role( async def get_membership_role(
@@ -137,6 +277,142 @@ class MetadataRepository:
) )
return result.scalar_one_or_none() return result.scalar_one_or_none()
async def list_project_members(
self, project_id: UUID
) -> List[ProjectMemberSummary]:
stmt = (
select(models.UserProjectMembership, models.User)
.join(models.User, models.User.id == models.UserProjectMembership.user_id)
.where(models.UserProjectMembership.project_id == project_id)
.order_by(models.User.username)
)
result = await self.session.execute(stmt)
return [
ProjectMemberSummary(
id=membership.id,
user_id=membership.user_id,
project_id=membership.project_id,
project_role=membership.project_role,
username=user.username,
email=user.email,
is_active=user.is_active,
)
for membership, user in result.all()
]
async def get_project_membership(
self, project_id: UUID, user_id: UUID
) -> Optional[models.UserProjectMembership]:
result = await self.session.execute(
select(models.UserProjectMembership).where(
models.UserProjectMembership.project_id == project_id,
models.UserProjectMembership.user_id == user_id,
)
)
return result.scalar_one_or_none()
async def add_project_member(
self, project_id: UUID, user_id: UUID, project_role: str
) -> models.UserProjectMembership:
membership = models.UserProjectMembership(
id=uuid4(),
user_id=user_id,
project_id=project_id,
project_role=project_role,
)
self.session.add(membership)
await self.session.commit()
await self.session.refresh(membership)
return membership
async def update_project_member_role(
self, project_id: UUID, user_id: UUID, project_role: str
) -> Optional[models.UserProjectMembership]:
membership = await self.get_project_membership(project_id, user_id)
if membership is None:
return None
membership.project_role = project_role
await self.session.commit()
await self.session.refresh(membership)
return membership
async def remove_project_member(self, project_id: UUID, user_id: UUID) -> bool:
result = await self.session.execute(
delete(models.UserProjectMembership).where(
models.UserProjectMembership.project_id == project_id,
models.UserProjectMembership.user_id == user_id,
)
)
await self.session.commit()
return bool(result.rowcount)
async def list_project_databases(
self, project_id: UUID
) -> List[models.ProjectDatabase]:
result = await self.session.execute(
select(models.ProjectDatabase)
.where(models.ProjectDatabase.project_id == project_id)
.order_by(models.ProjectDatabase.db_role)
)
return list(result.scalars().all())
async def get_project_database_config(
self, project_id: UUID, db_role: str
) -> Optional[models.ProjectDatabase]:
result = await self.session.execute(
select(models.ProjectDatabase).where(
models.ProjectDatabase.project_id == project_id,
models.ProjectDatabase.db_role == db_role,
)
)
return result.scalar_one_or_none()
async def upsert_project_database_config(
self,
project_id: UUID,
*,
db_role: str,
db_type: str,
dsn: str | None,
pool_min_size: int,
pool_max_size: int,
) -> models.ProjectDatabase:
record = await self.get_project_database_config(project_id, db_role)
if record is None:
if dsn is None:
raise ValueError("dsn is required when creating project database config")
record = models.ProjectDatabase(
id=uuid4(),
project_id=project_id,
db_role=db_role,
db_type=db_type,
dsn_encrypted=_encrypt_database_secret(dsn),
pool_min_size=pool_min_size,
pool_max_size=pool_max_size,
)
self.session.add(record)
else:
record.db_type = db_type
if dsn is not None:
record.dsn_encrypted = _encrypt_database_secret(dsn)
record.pool_min_size = pool_min_size
record.pool_max_size = pool_max_size
await self.session.commit()
await self.session.refresh(record)
return record
async def delete_project_database_config(
self, project_id: UUID, db_role: str
) -> bool:
result = await self.session.execute(
delete(models.ProjectDatabase).where(
models.ProjectDatabase.project_id == project_id,
models.ProjectDatabase.db_role == db_role,
)
)
await self.session.commit()
return bool(result.rowcount)
async def get_project_db_routing( async def get_project_db_routing(
self, project_id: UUID, db_role: str self, project_id: UUID, db_role: str
) -> Optional[ProjectDbRouting]: ) -> Optional[ProjectDbRouting]:
@@ -169,35 +445,6 @@ class MetadataRepository:
pool_max_size=record.pool_max_size, pool_max_size=record.pool_max_size,
) )
async def get_geoserver_config(
self, project_id: UUID
) -> Optional[ProjectGeoServerInfo]:
result = await self.session.execute(
select(models.ProjectGeoServerConfig).where(
models.ProjectGeoServerConfig.project_id == project_id
)
)
record = result.scalar_one_or_none()
if not record:
return None
if record.gs_admin_password_encrypted:
if is_encryption_configured():
encryptor = get_encryptor()
password = encryptor.decrypt(record.gs_admin_password_encrypted)
else:
password = record.gs_admin_password_encrypted
else:
password = None
return ProjectGeoServerInfo(
project_id=record.project_id,
gs_base_url=record.gs_base_url,
gs_admin_user=record.gs_admin_user,
gs_admin_password=password,
gs_datastore_name=record.gs_datastore_name,
default_extent=record.default_extent,
srid=record.srid,
)
async def list_projects_for_user(self, user_id: UUID) -> List[ProjectSummary]: async def list_projects_for_user(self, user_id: UUID) -> List[ProjectSummary]:
stmt = ( stmt = (
select(models.Project, models.UserProjectMembership.project_role) select(models.Project, models.UserProjectMembership.project_role)
@@ -1,235 +0,0 @@
from typing import Optional, List
from datetime import datetime
from app.infra.db.postgresql.database import Database
from app.domain.schemas.user import UserCreate, UserUpdate, UserInDB
from app.domain.models.role import UserRole
from app.core.security import get_password_hash
import logging
logger = logging.getLogger(__name__)
class UserRepository:
"""用户数据访问层"""
def __init__(self, db: Database):
self.db = db
async def create_user(self, user: UserCreate) -> Optional[UserInDB]:
"""
创建新用户
Args:
user: 用户创建数据
Returns:
创建的用户对象
"""
hashed_password = get_password_hash(user.password)
query = """
INSERT INTO users (username, email, hashed_password, role, is_active, is_superuser)
VALUES (%(username)s, %(email)s, %(hashed_password)s, %(role)s, TRUE, FALSE)
RETURNING id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
"""
try:
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {
'username': user.username,
'email': user.email,
'hashed_password': hashed_password,
'role': user.role.value
})
row = await cur.fetchone()
if row:
return UserInDB(**row)
except Exception as e:
logger.error(f"Error creating user: {e}")
raise
return None
async def get_user_by_id(self, user_id: int) -> Optional[UserInDB]:
"""根据ID获取用户"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
WHERE id = %(user_id)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'user_id': user_id})
row = await cur.fetchone()
if row:
return UserInDB(**row)
return None
async def get_user_by_username(self, username: str) -> Optional[UserInDB]:
"""根据用户名获取用户"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
WHERE username = %(username)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'username': username})
row = await cur.fetchone()
if row:
return UserInDB(**row)
return None
async def get_user_by_email(self, email: str) -> Optional[UserInDB]:
"""根据邮箱获取用户"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
WHERE email = %(email)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'email': email})
row = await cur.fetchone()
if row:
return UserInDB(**row)
return None
async def get_all_users(self, skip: int = 0, limit: int = 100) -> List[UserInDB]:
"""获取所有用户(分页)"""
query = """
SELECT id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
FROM users
ORDER BY created_at DESC
LIMIT %(limit)s OFFSET %(skip)s
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'skip': skip, 'limit': limit})
rows = await cur.fetchall()
return [UserInDB(**row) for row in rows]
async def update_user(self, user_id: int, user_update: UserUpdate) -> Optional[UserInDB]:
"""
更新用户信息
Args:
user_id: 用户ID
user_update: 更新数据
Returns:
更新后的用户对象
"""
# 构建动态更新语句
update_fields = []
params = {'user_id': user_id}
if user_update.email is not None:
update_fields.append("email = %(email)s")
params['email'] = user_update.email
if user_update.password is not None:
update_fields.append("hashed_password = %(hashed_password)s")
params['hashed_password'] = get_password_hash(user_update.password)
if user_update.role is not None:
update_fields.append("role = %(role)s")
params['role'] = user_update.role.value
if user_update.is_active is not None:
update_fields.append("is_active = %(is_active)s")
params['is_active'] = user_update.is_active
if not update_fields:
return await self.get_user_by_id(user_id)
query = f"""
UPDATE users
SET {', '.join(update_fields)}, updated_at = CURRENT_TIMESTAMP
WHERE id = %(user_id)s
RETURNING id, username, email, hashed_password, role, is_active, is_superuser,
created_at, updated_at
"""
try:
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, params)
row = await cur.fetchone()
if row:
return UserInDB(**row)
except Exception as e:
logger.error(f"Error updating user {user_id}: {e}")
raise
return None
async def delete_user(self, user_id: int) -> bool:
"""
删除用户
Args:
user_id: 用户ID
Returns:
是否成功删除
"""
query = "DELETE FROM users WHERE id = %(user_id)s"
try:
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, {'user_id': user_id})
return cur.rowcount > 0
except Exception as e:
logger.error(f"Error deleting user {user_id}: {e}")
return False
async def user_exists(self, username: str = None, email: str = None) -> bool:
"""
检查用户是否存在
Args:
username: 用户名
email: 邮箱
Returns:
是否存在
"""
conditions = []
params = {}
if username:
conditions.append("username = %(username)s")
params['username'] = username
if email:
conditions.append("email = %(email)s")
params['email'] = email
if not conditions:
return False
query = f"""
SELECT EXISTS(
SELECT 1 FROM users WHERE {' OR '.join(conditions)}
)
"""
async with self.db.get_connection() as conn:
async with conn.cursor() as cur:
await cur.execute(query, params)
result = await cur.fetchone()
return result['exists'] if result else False
+17 -9
View File
@@ -229,7 +229,14 @@ class InternalQueries:
scheme_type: str | None = None, scheme_type: str | None = None,
scheme_name: str | None = None, scheme_name: str | None = None,
) -> dict[str, list[dict]]: ) -> dict[str, list[dict]]:
if not element_ids: normalized_element_ids = list(
dict.fromkeys(
normalized
for normalized in (str(element_id).strip() for element_id in element_ids)
if normalized
)
)
if not normalized_element_ids:
return {} return {}
start_dt = parse_utc_time(start_time, field_name="start_time") start_dt = parse_utc_time(start_time, field_name="start_time")
@@ -253,9 +260,9 @@ class InternalQueries:
with conn.cursor(row_factory=dict_row) as cur: with conn.cursor(row_factory=dict_row) as cur:
if schema_name == "scheme": if schema_name == "scheme":
query = sql.SQL( query = sql.SQL(
"SELECT id, time, {} FROM {}.{} " "SELECT btrim(id::text) AS id, time, {} FROM {}.{} "
"WHERE scheme_type = %s AND scheme_name = %s " "WHERE scheme_type = %s AND scheme_name = %s "
"AND time >= %s AND time <= %s AND id = ANY(%s)" "AND time >= %s AND time <= %s AND btrim(id::text) = ANY(%s)"
).format( ).format(
sql.Identifier(field), sql.Identifier(field),
sql.Identifier(schema_name), sql.Identifier(schema_name),
@@ -268,25 +275,26 @@ class InternalQueries:
scheme_name, scheme_name,
start_dt, start_dt,
end_dt, end_dt,
element_ids, normalized_element_ids,
), ),
) )
else: else:
query = sql.SQL( query = sql.SQL(
"SELECT id, time, {} FROM {}.{} " "SELECT btrim(id::text) AS id, time, {} FROM {}.{} "
"WHERE time >= %s AND time <= %s AND id = ANY(%s)" "WHERE time >= %s AND time <= %s AND btrim(id::text) = ANY(%s)"
).format( ).format(
sql.Identifier(field), sql.Identifier(field),
sql.Identifier(schema_name), sql.Identifier(schema_name),
sql.Identifier(table_name), sql.Identifier(table_name),
) )
cur.execute(query, (start_dt, end_dt, element_ids)) cur.execute(query, (start_dt, end_dt, normalized_element_ids))
rows = cur.fetchall() rows = cur.fetchall()
result: dict[str, list[dict]] = { result: dict[str, list[dict]] = {
element_id: [] for element_id in element_ids element_id: [] for element_id in normalized_element_ids
} }
for row in rows: for row in rows:
result.setdefault(row["id"], []).append( element_id = str(row["id"]).strip()
result.setdefault(element_id, []).append(
{"time": row["time"].isoformat(), "value": row[field]} {"time": row["time"].isoformat(), "value": row[field]}
) )
for element_id in result: for element_id in result:
+154 -53
View File
@@ -40,7 +40,7 @@ def _normalize_series(data: SeriesInput, field_name: str) -> pd.Series:
else: else:
raise ValueError(f"Unsupported data format for {field_name}.") raise ValueError(f"Unsupported data format for {field_name}.")
series.index = series.index.map(str) series.index = series.index.map(_normalize_identifier)
return pd.to_numeric(series, errors="raise") return pd.to_numeric(series, errors="raise")
@@ -60,6 +60,8 @@ def run_burst_location_by_network(
basic_pressure: float = 10.0, basic_pressure: float = 10.0,
scada_burst_start: datetime | str | None = None, scada_burst_start: datetime | str | None = None,
scada_burst_end: datetime | str | None = None, scada_burst_end: datetime | str | None = None,
scada_normal_start: datetime | str | None = None,
scada_normal_end: datetime | str | None = None,
use_scada_flow: bool = False, use_scada_flow: bool = False,
scheme_name: str | None = None, scheme_name: str | None = None,
simulation_scheme_name: str | None = None, simulation_scheme_name: str | None = None,
@@ -87,12 +89,37 @@ def run_burst_location_by_network(
for value in [ for value in [
scada_burst_start, scada_burst_start,
scada_burst_end, scada_burst_end,
scada_normal_start,
scada_normal_end,
] ]
) )
if use_scada_pressure: if use_scada_pressure:
burst_start_dt, burst_end_dt = _validate_scada_windows( burst_start_dt, burst_end_dt = _validate_time_window(
scada_burst_start=scada_burst_start, start_value=scada_burst_start,
scada_burst_end=scada_burst_end, end_value=scada_burst_end,
start_field="scada_burst_start",
end_field="scada_burst_end",
label=(
"爆管方案时间窗"
if normalized_data_source == "simulation"
else "爆管时段 SCADA 时间窗"
),
)
normal_start_dt: datetime | None = None
normal_end_dt: datetime | None = None
if scada_normal_start is not None or scada_normal_end is not None:
normal_start_dt, normal_end_dt = _validate_time_window(
start_value=scada_normal_start,
end_value=scada_normal_end,
start_field="scada_normal_start",
end_field="scada_normal_end",
label="正常时段 SCADA 时间窗",
)
normal_pressure_from_payload = (
_normalize_series(normal_pressure, "normal_pressure")
if normal_pressure is not None
else None
) )
if normalized_data_source == "simulation": if normalized_data_source == "simulation":
if not simulation_scheme_name: if not simulation_scheme_name:
@@ -117,15 +144,15 @@ def run_burst_location_by_network(
) = _build_observed_series_from_simulation( ) = _build_observed_series_from_simulation(
network=network, network=network,
sensor_ids=selected_pressure_ids, sensor_ids=selected_pressure_ids,
start_dt=burst_start_dt, start_dt=normal_start_dt or burst_start_dt,
end_dt=burst_end_dt, end_dt=normal_end_dt or burst_end_dt,
data_type="pressure", data_type="pressure",
series_name="normal_pressure", series_name="normal_pressure",
simulation_source="realtime", simulation_source="scheme",
simulation_scheme_name=None, simulation_scheme_name=simulation_scheme_name,
simulation_scheme_type=resolved_simulation_scheme_type, simulation_scheme_type=resolved_simulation_scheme_type,
) )
observed_source = "simulation_scheme_burst_realtime_normal_timerange" observed_source = "simulation_scheme_timerange"
else: else:
( (
burst_pressure_series, burst_pressure_series,
@@ -138,21 +165,27 @@ def run_burst_location_by_network(
data_type="pressure", data_type="pressure",
series_name="burst_pressure", series_name="burst_pressure",
) )
( if normal_pressure_from_payload is None:
normal_pressure_series, (
normal_pressure_samples, normal_pressure_series,
) = _build_observed_series_from_simulation( normal_pressure_samples,
network=network, ) = _build_observed_series_from_scada(
sensor_ids=selected_pressure_ids, network=network,
start_dt=burst_start_dt, sensor_ids=selected_pressure_ids,
end_dt=burst_end_dt, start_dt=normal_start_dt or burst_start_dt,
data_type="pressure", end_dt=normal_end_dt or burst_end_dt,
series_name="normal_pressure", data_type="pressure",
simulation_source="realtime", series_name="normal_pressure",
simulation_scheme_name=None, )
simulation_scheme_type=resolved_simulation_scheme_type, observed_source = (
) "scada_burst_scada_normal_timerange"
observed_source = "scada_burst_realtime_normal_timerange" if normal_start_dt is not None and normal_end_dt is not None
else "scada_timerange"
)
else:
normal_pressure_series = normal_pressure_from_payload
normal_pressure_samples = 1
observed_source = "scada_burst_payload_normal_timerange"
else: else:
if burst_pressure is None or normal_pressure is None: if burst_pressure is None or normal_pressure is None:
raise ValueError( raise ValueError(
@@ -179,6 +212,11 @@ def run_burst_location_by_network(
) )
if not selected_flow_ids: if not selected_flow_ids:
raise ValueError("未找到可用流量传感器,无法从 SCADA 查询流量数据。") raise ValueError("未找到可用流量传感器,无法从 SCADA 查询流量数据。")
normal_flow_from_payload = (
_normalize_series(normal_flow, "normal_flow")
if normal_flow is not None
else None
)
if normalized_data_source == "simulation": if normalized_data_source == "simulation":
if not simulation_scheme_name: if not simulation_scheme_name:
raise ValueError("模拟方案模式必须提供 simulation_scheme_name。") raise ValueError("模拟方案模式必须提供 simulation_scheme_name。")
@@ -199,12 +237,12 @@ def run_burst_location_by_network(
_build_observed_series_from_simulation( _build_observed_series_from_simulation(
network=network, network=network,
sensor_ids=selected_flow_ids, sensor_ids=selected_flow_ids,
start_dt=burst_start_dt, start_dt=normal_start_dt or burst_start_dt,
end_dt=burst_end_dt, end_dt=normal_end_dt or burst_end_dt,
data_type="flow", data_type="flow",
series_name="normal_flow", series_name="normal_flow",
simulation_source="realtime", simulation_source="scheme",
simulation_scheme_name=None, simulation_scheme_name=simulation_scheme_name,
simulation_scheme_type=resolved_simulation_scheme_type, simulation_scheme_type=resolved_simulation_scheme_type,
) )
) )
@@ -217,19 +255,20 @@ def run_burst_location_by_network(
data_type="flow", data_type="flow",
series_name="burst_flow", series_name="burst_flow",
) )
normal_flow_series, normal_flow_samples = ( if normal_flow_from_payload is None:
_build_observed_series_from_simulation( normal_flow_series, normal_flow_samples = (
network=network, _build_observed_series_from_scada(
sensor_ids=selected_flow_ids, network=network,
start_dt=burst_start_dt, sensor_ids=selected_flow_ids,
end_dt=burst_end_dt, start_dt=normal_start_dt or burst_start_dt,
data_type="flow", end_dt=normal_end_dt or burst_end_dt,
series_name="normal_flow", data_type="flow",
simulation_source="realtime", series_name="normal_flow",
simulation_scheme_name=None, )
simulation_scheme_type=resolved_simulation_scheme_type,
) )
) else:
normal_flow_series = normal_flow_from_payload
normal_flow_samples = 1
else: else:
if flow_scada_ids is not None: if flow_scada_ids is not None:
selected_flow_ids = _dedupe_ids(flow_scada_ids) selected_flow_ids = _dedupe_ids(flow_scada_ids)
@@ -281,6 +320,13 @@ def run_burst_location_by_network(
"burst_start": burst_start_dt.isoformat(), "burst_start": burst_start_dt.isoformat(),
"burst_end": burst_end_dt.isoformat(), "burst_end": burst_end_dt.isoformat(),
} }
if normal_start_dt is not None and normal_end_dt is not None:
payload["scada_window"].update(
{
"normal_start": normal_start_dt.isoformat(),
"normal_end": normal_end_dt.isoformat(),
}
)
if normalized_data_source == "simulation": if normalized_data_source == "simulation":
payload["simulation_scheme"] = { payload["simulation_scheme"] = {
"name": simulation_scheme_name, "name": simulation_scheme_name,
@@ -376,6 +422,23 @@ def _validate_scada_windows(
return burst_start_dt, burst_end_dt return burst_start_dt, burst_end_dt
def _validate_time_window(
*,
start_value: datetime | str | None,
end_value: datetime | str | None,
start_field: str,
end_field: str,
label: str,
) -> tuple[datetime, datetime]:
if start_value is None or end_value is None:
raise ValueError(f"{label}必须同时提供 {start_field}/{end_field}")
start_dt = _to_datetime(start_value)
end_dt = _to_datetime(end_value)
if start_dt >= end_dt:
raise ValueError(f"{label}非法:{start_field} 必须早于 {end_field}")
return start_dt, end_dt
def _build_observed_series_from_scada( def _build_observed_series_from_scada(
*, *,
network: str, network: str,
@@ -385,13 +448,14 @@ def _build_observed_series_from_scada(
data_type: str, data_type: str,
series_name: str, series_name: str,
) -> tuple[pd.Series, int]: ) -> tuple[pd.Series, int]:
sensor_ids = _dedupe_ids(sensor_ids)
scada_mapping = _build_scada_mapping(network=network, data_type=data_type) scada_mapping = _build_scada_mapping(network=network, data_type=data_type)
missing_ids = [ missing_ids = [
sensor_id for sensor_id in sensor_ids if sensor_id not in scada_mapping sensor_id for sensor_id in sensor_ids if sensor_id not in scada_mapping
] ]
if missing_ids: if missing_ids:
preview = ", ".join(missing_ids[:10]) preview = ", ".join(missing_ids[:10])
raise ValueError(f"{series_name} 缺少可用 SCADA 映射: {preview}") raise ValueError(f"{_series_display_name(series_name)} 缺少可用 SCADA 映射: {preview}")
query_ids = [scada_mapping[sensor_id] for sensor_id in sensor_ids] query_ids = [scada_mapping[sensor_id] for sensor_id in sensor_ids]
scada_data = InternalQueries.query_scada_by_ids_timerange( scada_data = InternalQueries.query_scada_by_ids_timerange(
@@ -400,6 +464,7 @@ def _build_observed_series_from_scada(
start_time=start_dt.isoformat(), start_time=start_dt.isoformat(),
end_time=end_dt.isoformat(), end_time=end_dt.isoformat(),
) )
scada_data = _normalize_timeseries_by_id(scada_data)
values: dict[str, float] = {} values: dict[str, float] = {}
sample_counts: list[int] = [] sample_counts: list[int] = []
for sensor_id, query_id in zip(sensor_ids, query_ids): for sensor_id, query_id in zip(sensor_ids, query_ids):
@@ -408,7 +473,9 @@ def _build_observed_series_from_scada(
float(item["value"]) for item in records if item.get("value") is not None float(item["value"]) for item in records if item.get("value") is not None
] ]
if not numeric_values: if not numeric_values:
raise ValueError(f"{series_name} 在时间窗内无有效数据: {sensor_id}") raise ValueError(
f"{_series_display_name(series_name)} 在时间窗内无有效数据: {sensor_id}"
)
values[sensor_id] = float(sum(numeric_values) / len(numeric_values)) values[sensor_id] = float(sum(numeric_values) / len(numeric_values))
sample_counts.append(len(numeric_values)) sample_counts.append(len(numeric_values))
@@ -427,13 +494,14 @@ def _build_observed_series_from_simulation(
simulation_scheme_name: str | None, simulation_scheme_name: str | None,
simulation_scheme_type: str, simulation_scheme_type: str,
) -> tuple[pd.Series, int]: ) -> tuple[pd.Series, int]:
sensor_ids = _dedupe_ids(sensor_ids)
sensor_metadata = _build_sensor_metadata(network=network, data_type=data_type) sensor_metadata = _build_sensor_metadata(network=network, data_type=data_type)
missing_ids = [ missing_ids = [
sensor_id for sensor_id in sensor_ids if sensor_id not in sensor_metadata sensor_id for sensor_id in sensor_ids if sensor_id not in sensor_metadata
] ]
if missing_ids: if missing_ids:
preview = ", ".join(missing_ids[:10]) preview = ", ".join(missing_ids[:10])
raise ValueError(f"{series_name} 缺少可用 SCADA 映射: {preview}") raise ValueError(f"{_series_display_name(series_name)} 缺少可用 SCADA 映射: {preview}")
simulation_data = _query_simulation_data_by_sensor_ids( simulation_data = _query_simulation_data_by_sensor_ids(
network=network, network=network,
@@ -446,6 +514,7 @@ def _build_observed_series_from_simulation(
simulation_scheme_name=simulation_scheme_name, simulation_scheme_name=simulation_scheme_name,
simulation_scheme_type=simulation_scheme_type, simulation_scheme_type=simulation_scheme_type,
) )
simulation_data = _normalize_timeseries_by_id(simulation_data)
values: dict[str, float] = {} values: dict[str, float] = {}
sample_counts: list[int] = [] sample_counts: list[int] = []
for sensor_id in sensor_ids: for sensor_id in sensor_ids:
@@ -454,13 +523,24 @@ def _build_observed_series_from_simulation(
float(item["value"]) for item in records if item.get("value") is not None float(item["value"]) for item in records if item.get("value") is not None
] ]
if not numeric_values: if not numeric_values:
raise ValueError(f"{series_name} 在时间窗内无有效模拟数据: {sensor_id}") raise ValueError(
f"{_series_display_name(series_name)} 在时间窗内无有效模拟数据: {sensor_id}"
)
values[sensor_id] = float(sum(numeric_values) / len(numeric_values)) values[sensor_id] = float(sum(numeric_values) / len(numeric_values))
sample_counts.append(len(numeric_values)) sample_counts.append(len(numeric_values))
return pd.Series(values, dtype=float), min(sample_counts) return pd.Series(values, dtype=float), min(sample_counts)
def _series_display_name(series_name: str) -> str:
return {
"burst_pressure": "爆管压力数据",
"normal_pressure": "正常压力数据",
"burst_flow": "爆管流量数据",
"normal_flow": "正常流量数据",
}.get(series_name, series_name)
def _query_simulation_data_by_sensor_ids( def _query_simulation_data_by_sensor_ids(
*, *,
network: str, network: str,
@@ -476,6 +556,7 @@ def _query_simulation_data_by_sensor_ids(
if simulation_source not in {"scheme", "realtime"}: if simulation_source not in {"scheme", "realtime"}:
raise ValueError(f"Unsupported simulation_source: {simulation_source}") raise ValueError(f"Unsupported simulation_source: {simulation_source}")
sensor_ids = _dedupe_ids(sensor_ids)
result: dict[str, list[dict[str, Any]]] = { result: dict[str, list[dict[str, Any]]] = {
sensor_id: [] for sensor_id in sensor_ids sensor_id: [] for sensor_id in sensor_ids
} }
@@ -556,6 +637,7 @@ def _query_simulation_values(
simulation_scheme_name: str | None, simulation_scheme_name: str | None,
simulation_scheme_type: str, simulation_scheme_type: str,
) -> dict[str, list[dict[str, Any]]]: ) -> dict[str, list[dict[str, Any]]]:
element_ids = _dedupe_ids(element_ids)
if not element_ids: if not element_ids:
return {} return {}
if simulation_source == "scheme": if simulation_source == "scheme":
@@ -595,14 +677,9 @@ def _build_sensor_metadata(network: str, data_type: str) -> dict[str, dict[str,
continue continue
else: else:
raise ValueError(f"Unsupported data_type: {data_type}") raise ValueError(f"Unsupported data_type: {data_type}")
element_id = item.get("associated_element_id") element_id = _normalize_identifier(item.get("associated_element_id"))
query_id = item.get("api_query_id") query_id = _normalize_identifier(item.get("api_query_id"))
if ( if element_id and query_id:
isinstance(element_id, str)
and element_id
and isinstance(query_id, str)
and query_id
):
metadata[element_id] = {"query_id": query_id, "scada_type": scada_type} metadata[element_id] = {"query_id": query_id, "scada_type": scada_type}
return metadata return metadata
@@ -638,7 +715,31 @@ def _get_sensor_nodes(network: str, data_type: str) -> list[str]:
def _dedupe_ids(ids: list[str] | None) -> list[str]: def _dedupe_ids(ids: list[str] | None) -> list[str]:
if ids is None: if ids is None:
return [] return []
return list(dict.fromkeys([str(item) for item in ids if item])) return list(
dict.fromkeys(
normalized
for normalized in (_normalize_identifier(item) for item in ids)
if normalized
)
)
def _normalize_identifier(value: Any) -> str:
if value is None:
return ""
return str(value).strip()
def _normalize_timeseries_by_id(
data: dict[Any, list[dict[str, Any]]] | None,
) -> dict[str, list[dict[str, Any]]]:
normalized_data: dict[str, list[dict[str, Any]]] = {}
for raw_id, records in (data or {}).items():
normalized_id = _normalize_identifier(raw_id)
if not normalized_id:
continue
normalized_data.setdefault(normalized_id, []).extend(records or [])
return normalized_data
def _to_datetime(value: datetime | str) -> datetime: def _to_datetime(value: datetime | str) -> datetime:
-3
View File
@@ -32,7 +32,6 @@ def test_load_auth_context_supports_aliases(monkeypatch):
monkeypatch.setenv("TJWATER_SERVER", "http://server") monkeypatch.setenv("TJWATER_SERVER", "http://server")
monkeypatch.setenv("TJWATER_ACCESS_TOKEN", "abc") monkeypatch.setenv("TJWATER_ACCESS_TOKEN", "abc")
monkeypatch.setenv("TJWATER_PROJECT_ID", "p1") monkeypatch.setenv("TJWATER_PROJECT_ID", "p1")
monkeypatch.setenv("TJWATER_USER_ID", "u1")
monkeypatch.setenv("TJWATER_USERNAME", "tester") monkeypatch.setenv("TJWATER_USERNAME", "tester")
monkeypatch.setenv("TJWATER_NETWORK", "net1") monkeypatch.setenv("TJWATER_NETWORK", "net1")
@@ -41,7 +40,6 @@ def test_load_auth_context_supports_aliases(monkeypatch):
assert auth.server == "http://server" assert auth.server == "http://server"
assert auth.access_token == "abc" assert auth.access_token == "abc"
assert auth.project_id == "p1" assert auth.project_id == "p1"
assert auth.user_id == "u1"
assert auth.username == "tester" assert auth.username == "tester"
assert auth.network == "net1" assert auth.network == "net1"
@@ -50,7 +48,6 @@ def test_build_runtime_context_uses_default_server(monkeypatch):
monkeypatch.delenv("TJWATER_SERVER", raising=False) monkeypatch.delenv("TJWATER_SERVER", raising=False)
monkeypatch.delenv("TJWATER_ACCESS_TOKEN", raising=False) monkeypatch.delenv("TJWATER_ACCESS_TOKEN", raising=False)
monkeypatch.delenv("TJWATER_PROJECT_ID", raising=False) monkeypatch.delenv("TJWATER_PROJECT_ID", raising=False)
monkeypatch.delenv("TJWATER_USER_ID", raising=False)
monkeypatch.delenv("TJWATER_USERNAME", raising=False) monkeypatch.delenv("TJWATER_USERNAME", raising=False)
monkeypatch.delenv("TJWATER_NETWORK", raising=False) monkeypatch.delenv("TJWATER_NETWORK", raising=False)
monkeypatch.delenv("TJWATER_EXTRA_HEADERS", raising=False) monkeypatch.delenv("TJWATER_EXTRA_HEADERS", raising=False)
+5 -5
View File
@@ -53,7 +53,7 @@ def simulation_run(
ctx, ctx,
summary="触发模拟成功", summary="触发模拟成功",
method="POST", method="POST",
path="/runsimulationmanuallybydate/", path="/simulations/run-by-date",
json_body=body, json_body=body,
require_auth=True, require_auth=True,
require_network_ctx=True, require_network_ctx=True,
@@ -87,7 +87,7 @@ def analysis_burst(
ctx, ctx,
summary="爆管分析执行成功", summary="爆管分析执行成功",
method="GET", method="GET",
path="/burst_analysis/", path="/burst-analysis",
params=params, params=params,
require_auth=True, require_auth=True,
require_network_ctx=True, require_network_ctx=True,
@@ -151,7 +151,7 @@ def analysis_valve(
ctx, ctx,
summary="阀门隔离分析执行成功", summary="阀门隔离分析执行成功",
method="GET", method="GET",
path="/valve_isolation_analysis/", path="/valve-isolation-analysis",
params=params, params=params,
require_auth=True, require_auth=True,
require_network_ctx=True, require_network_ctx=True,
@@ -186,7 +186,7 @@ def analysis_flushing(
ctx, ctx,
summary="冲洗分析执行成功", summary="冲洗分析执行成功",
method="GET", method="GET",
path="/flushing_analysis/", path="/flushing-analysis",
params=params, params=params,
require_auth=True, require_auth=True,
require_network_ctx=True, require_network_ctx=True,
@@ -240,7 +240,7 @@ def analysis_contaminant(
ctx, ctx,
summary="污染物模拟执行成功", summary="污染物模拟执行成功",
method="GET", method="GET",
path="/contaminant_simulation/", path="/contaminant-simulation",
params=params, params=params,
require_auth=True, require_auth=True,
require_network_ctx=True, require_network_ctx=True,
+1 -1
View File
@@ -501,7 +501,7 @@ def data_scheme_list(ctx: typer.Context) -> None:
ctx, ctx,
summary="读取方案列表成功", summary="读取方案列表成功",
method="GET", method="GET",
path="/getallschemes/", path="/schemes",
params={"network": require_network(runtime)}, params={"network": require_network(runtime)},
require_auth=True, require_auth=True,
require_network_ctx=True, require_network_ctx=True,
+16 -7
View File
@@ -46,7 +46,6 @@ class AuthContext:
server: str | None = None server: str | None = None
access_token: str | None = None access_token: str | None = None
project_id: str | None = None project_id: str | None = None
user_id: str | None = None
username: str | None = None username: str | None = None
network: str | None = None network: str | None = None
headers: dict[str, str] = field(default_factory=dict) headers: dict[str, str] = field(default_factory=dict)
@@ -98,7 +97,6 @@ def load_auth_context(auth_stdin: bool = False) -> AuthContext:
"server": os.getenv("TJWATER_SERVER"), "server": os.getenv("TJWATER_SERVER"),
"access_token": os.getenv("TJWATER_ACCESS_TOKEN"), "access_token": os.getenv("TJWATER_ACCESS_TOKEN"),
"project_id": os.getenv("TJWATER_PROJECT_ID"), "project_id": os.getenv("TJWATER_PROJECT_ID"),
"user_id": os.getenv("TJWATER_USER_ID"),
"username": os.getenv("TJWATER_USERNAME"), "username": os.getenv("TJWATER_USERNAME"),
"network": os.getenv("TJWATER_NETWORK"), "network": os.getenv("TJWATER_NETWORK"),
"headers": json.loads(extra_headers) if extra_headers else {}, "headers": json.loads(extra_headers) if extra_headers else {},
@@ -117,7 +115,6 @@ def load_auth_context(auth_stdin: bool = False) -> AuthContext:
server=_pick(raw, "server", "base_url"), server=_pick(raw, "server", "base_url"),
access_token=_pick(raw, "access_token", "token", "accessToken"), access_token=_pick(raw, "access_token", "token", "accessToken"),
project_id=_pick(raw, "project_id", "projectId", "x_project_id"), project_id=_pick(raw, "project_id", "projectId", "x_project_id"),
user_id=_pick(raw, "user_id", "userId", "x_user_id"),
username=_pick(raw, "username", "preferred_username"), username=_pick(raw, "username", "preferred_username"),
network=_pick(raw, "network", "project_code", "projectCode", "project"), network=_pick(raw, "network", "project_code", "projectCode", "project"),
headers={str(key): str(value) for key, value in headers.items()}, headers={str(key): str(value) for key, value in headers.items()},
@@ -350,8 +347,6 @@ def build_headers(
headers["X-Project-Id"] = require_project_id(ctx) headers["X-Project-Id"] = require_project_id(ctx)
elif ctx.auth.project_id: elif ctx.auth.project_id:
headers["X-Project-Id"] = ctx.auth.project_id headers["X-Project-Id"] = ctx.auth.project_id
if ctx.auth.user_id:
headers["X-User-Id"] = ctx.auth.user_id
return headers return headers
@@ -409,6 +404,14 @@ def _parse_response_body(response: requests.Response) -> Any:
return {} return {}
def _with_network_param(params: dict[str, Any] | None, network: str) -> dict[str, Any]:
params = dict(params or {})
if "network" in params or "network_name" in params or "name" in params:
return params
params["network"] = network
return params
def request_json( def request_json(
ctx: RuntimeContext, ctx: RuntimeContext,
*, *,
@@ -422,10 +425,13 @@ def request_json(
require_username_ctx: bool = False, require_username_ctx: bool = False,
) -> tuple[Any, int]: ) -> tuple[Any, int]:
require_server(ctx) require_server(ctx)
network = None
if require_network_ctx: if require_network_ctx:
require_network(ctx) network = require_network(ctx)
if require_username_ctx: if require_username_ctx:
require_username(ctx) require_username(ctx)
if network and (params is not None or json_body is None):
params = _with_network_param(params, network)
url = f"{require_server(ctx)}/api/v1{path}" url = f"{require_server(ctx)}/api/v1{path}"
headers = build_headers(ctx, require_auth=require_auth, require_project=require_project) headers = build_headers(ctx, require_auth=require_auth, require_project=require_project)
@@ -479,8 +485,11 @@ def request_bytes(
require_network_ctx: bool = False, require_network_ctx: bool = False,
) -> tuple[bytes, int]: ) -> tuple[bytes, int]:
require_server(ctx) require_server(ctx)
network = None
if require_network_ctx: if require_network_ctx:
require_network(ctx) network = require_network(ctx)
if network:
params = _with_network_param(params, network)
url = f"{require_server(ctx)}/api/v1{path}" url = f"{require_server(ctx)}/api/v1{path}"
headers = build_headers(ctx, require_auth=require_auth, require_project=require_project) headers = build_headers(ctx, require_auth=require_auth, require_project=require_project)
+3 -3
View File
@@ -137,7 +137,7 @@ COMMAND_DOCS: dict[tuple[str, ...], CommandDoc] = {
("simulation", "run"): CommandDoc( ("simulation", "run"): CommandDoc(
path=("simulation", "run"), path=("simulation", "run"),
summary="触发指定绝对时间的模拟运行", summary="触发指定绝对时间的模拟运行",
description="把显式带时区的 RFC3339 start-time 直接传给 /runsimulationmanuallybydate/;服务端按带时区时间处理并统一按 UTC 存储结果,实时数据需后续通过 data timeseries 在对应时间段查询。duration 单位为分钟。", description="把显式带时区的 RFC3339 start-time 直接传给 /simulations/run-by-date;服务端按带时区时间处理并统一按 UTC 存储结果,实时数据需后续通过 data timeseries 在对应时间段查询。duration 单位为分钟。",
options=( options=(
CommandOptionDoc("start-time", "显式带时区的开始时间", required=True), CommandOptionDoc("start-time", "显式带时区的开始时间", required=True),
CommandOptionDoc("duration", "持续分钟数", required=True), CommandOptionDoc("duration", "持续分钟数", required=True),
@@ -211,7 +211,7 @@ COMMAND_DOCS: dict[tuple[str, ...], CommandDoc] = {
("analysis", "contaminant"): CommandDoc( ("analysis", "contaminant"): CommandDoc(
path=("analysis", "contaminant"), path=("analysis", "contaminant"),
summary="执行污染物模拟", summary="执行污染物模拟",
description="调用 /contaminant_simulation/。duration 单位为秒。", description="调用 /contaminant-simulation。duration 单位为秒。",
options=( options=(
CommandOptionDoc("start-time", "显式带时区的开始时间", required=True), CommandOptionDoc("start-time", "显式带时区的开始时间", required=True),
CommandOptionDoc("duration", "持续秒数", required=True), CommandOptionDoc("duration", "持续秒数", required=True),
@@ -499,7 +499,7 @@ COMMAND_DOCS: dict[tuple[str, ...], CommandDoc] = {
("data", "scheme", "list"): CommandDoc( ("data", "scheme", "list"): CommandDoc(
path=("data", "scheme", "list"), path=("data", "scheme", "list"),
summary="列出方案", summary="列出方案",
description="调用 /getallschemes/", description="调用 /schemes。",
examples=("tjwater-cli data scheme list",), examples=("tjwater-cli data scheme list",),
), ),
} }
+7 -12
View File
@@ -194,12 +194,12 @@ app/api/v1/endpoints/risk.py
| 命令 | 覆盖接口 | 说明 | | 命令 | 覆盖接口 | 说明 |
|---|---|---| |---|---|---|
| `tjwater-cli simulation run --start-time RFC3339 --duration MINUTES` | `POST /runsimulationmanuallybydate/` | 按指定绝对开始时间触发当前 project 的实时模拟;`start-time` 必须显式带时区,结果写入服务端时序库,后续通过 `tjwater-cli data timeseries realtime *` 查询 | | `tjwater-cli simulation run --start-time RFC3339 --duration MINUTES` | `POST /simulations/run-by-date` | 按指定绝对开始时间触发当前 project 的实时模拟;`start-time` 必须显式带时区,结果写入服务端时序库,后续通过 `tjwater-cli data timeseries realtime *` 查询 |
| `tjwater-cli analysis burst --start-time TIME --duration SEC --scheme SCHEME --burst-file FILE` | `GET /burst_analysis/` | 爆管分析;`FILE` 提供爆管点与流量列表,CLI 负责转换为 `burst_ID[]` / `burst_size[]` | | `tjwater-cli analysis burst --start-time TIME --duration SEC --scheme SCHEME --burst-file FILE` | `GET /burst-analysis` | 爆管分析;`FILE` 提供爆管点与流量列表,CLI 负责转换为 `burst_ID[]` / `burst_size[]` |
| `tjwater-cli analysis valve --mode close\|isolation --start-time TIME --valve VALVE [--scheme SCHEME]` | `GET /valve_close_analysis/``GET /valve_isolation_analysis/` | 阀门分析;close 模式需要 `--scheme``--valve` 可重复 | | `tjwater-cli analysis valve --mode close\|isolation --start-time TIME --valve VALVE [--scheme SCHEME]` | `GET /valve_close_analysis/``GET /valve-isolation-analysis` | 阀门分析;close 模式需要 `--scheme``--valve` 可重复 |
| `tjwater-cli analysis flushing --start-time TIME --valve-setting-file FILE --drainage-node NODE --flow FLOW --scheme SCHEME [--duration SEC]` | `GET /flushing_analysis/` | 冲洗分析;`FILE` 提供阀门与开度列表,CLI 负责转换为 `valves[]` / `valves_k[]` | | `tjwater-cli analysis flushing --start-time TIME --valve-setting-file FILE --drainage-node NODE --flow FLOW --scheme SCHEME [--duration SEC]` | `GET /flushing-analysis` | 冲洗分析;`FILE` 提供阀门与开度列表,CLI 负责转换为 `valves[]` / `valves_k[]` |
| `tjwater-cli analysis age --start-time TIME --duration SEC` | `GET /age_analysis/` | 水龄分析 | | `tjwater-cli analysis age --start-time TIME --duration SEC` | `GET /age_analysis/` | 水龄分析 |
| `tjwater-cli analysis contaminant --start-time TIME --duration SEC --source-node NODE --concentration VALUE --scheme SCHEME [--pattern PATTERN]` | `GET /contaminant_simulation/` | 污染物模拟 | | `tjwater-cli analysis contaminant --start-time TIME --duration SEC --source-node NODE --concentration VALUE --scheme SCHEME [--pattern PATTERN]` | `GET /contaminant-simulation` | 污染物模拟 |
| `tjwater-cli analysis sensor-placement kmeans --count N` | `GET /pressuresensorplacementkmeans/` | 基于 kmeans 的传感器放置分析;不包含创建方案 | | `tjwater-cli analysis sensor-placement kmeans --count N` | `GET /pressuresensorplacementkmeans/` | 基于 kmeans 的传感器放置分析;不包含创建方案 |
| `tjwater-cli analysis leakage identify --scheme SCHEME --start-time TIME --end-time TIME` | `POST /leakage/identify/` | 漏损识别 | | `tjwater-cli analysis leakage identify --scheme SCHEME --start-time TIME --end-time TIME` | `POST /leakage/identify/` | 漏损识别 |
| `tjwater-cli analysis leakage schemes list\|get` | `GET /leakage/schemes/``GET /leakage/schemes/{scheme_name}` | 漏损方案查询 | | `tjwater-cli analysis leakage schemes list\|get` | `GET /leakage/schemes/``GET /leakage/schemes/{scheme_name}` | 漏损方案查询 |
@@ -229,7 +229,7 @@ POST /daily_scheduling_analysis/
- 首批 CLI 统一按同步命令设计,避免引入额外的异步轮询协议。 - 首批 CLI 统一按同步命令设计,避免引入额外的异步轮询协议。
- `simulation run` 不直接回传全量模拟结果;它负责触发服务端模拟,并返回执行摘要、时间窗口和后续查询提示。 - `simulation run` 不直接回传全量模拟结果;它负责触发服务端模拟,并返回执行摘要、时间窗口和后续查询提示。
- 当前 `runsimulationmanuallybydate` 接口会从 `start_time` 指定的绝对时间开始,按 15 分钟步长运行直到达到 `duration`,结果持久化到服务端时序存储。 - 当前 `simulations/run-by-date` 接口会从 `start_time` 指定的绝对时间开始,按 15 分钟步长运行直到达到 `duration`,结果持久化到服务端时序存储。
- `start_time` 必须显式带时区;CLI 推荐直接传 **UTC+8** 时间,服务端统一转换后执行和落库。CLI 文档与帮助信息需要把这条规则写成显式契约,不能把数据库存储时间直接暴露成用户输入语义。 - `start_time` 必须显式带时区;CLI 推荐直接传 **UTC+8** 时间,服务端统一转换后执行和落库。CLI 文档与帮助信息需要把这条规则写成显式契约,不能把数据库存储时间直接暴露成用户输入语义。
- 模拟结果读取统一走 `tjwater-cli data timeseries realtime *`,而不是再单独设计 `simulation output` - 模拟结果读取统一走 `tjwater-cli data timeseries realtime *`,而不是再单独设计 `simulation output`
- `analysis` 相关命令首批也按同步请求处理;若后续服务端真的引入任务队列,再单独设计 `job` 类基础设施能力。 - `analysis` 相关命令首批也按同步请求处理;若后续服务端真的引入任务队列,再单独设计 `job` 类基础设施能力。
@@ -306,10 +306,9 @@ app/api/v1/endpoints/snapshots.py
app/api/v1/endpoints/cache.py app/api/v1/endpoints/cache.py
app/api/v1/endpoints/audit.py app/api/v1/endpoints/audit.py
app/api/v1/endpoints/users.py app/api/v1/endpoints/users.py
app/api/v1/endpoints/user_management.py
``` ```
这些接口不纳入首批 Agent CLI。原因是它们更偏运维、审计、用户管理或状态回滚,不属于 Agent 面向水务业务分析的核心调用范围。 这些接口不纳入首批 Agent CLI。原因是它们更偏运维、审计或状态回滚,不属于 Agent 面向水务业务分析的核心调用范围。
暂不暴露: 暂不暴露:
@@ -339,10 +338,6 @@ GET /audit/logs/count
GET /getuserschema/ GET /getuserschema/
GET /getuser/ GET /getuser/
GET /getallusers/ GET /getallusers/
PUT /users/{user_id}
DELETE /users/{user_id}
POST /users/{user_id}/activate
POST /users/{user_id}/deactivate
``` ```
## Help ## Help
@@ -0,0 +1,62 @@
-- Metadata auth management schema patch.
-- Keycloak owns login credentials; TJWater stores only business identity and access.
CREATE EXTENSION IF NOT EXISTS pgcrypto;
DO $$
DECLARE
users_id_type text;
BEGIN
SELECT data_type INTO users_id_type
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'users'
AND column_name = 'id';
IF users_id_type IS NULL THEN
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
keycloak_id UUID UNIQUE NOT NULL,
username VARCHAR(50) UNIQUE NOT NULL,
email VARCHAR(100) UNIQUE NOT NULL,
role VARCHAR(20) DEFAULT 'user' NOT NULL,
is_active BOOLEAN DEFAULT TRUE NOT NULL,
is_superuser BOOLEAN DEFAULT FALSE NOT NULL,
attributes JSONB,
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
last_login_at TIMESTAMP WITH TIME ZONE
);
ELSIF users_id_type <> 'uuid' THEN
RAISE EXCEPTION
'Existing public.users.id is %, not uuid. Export old local users, create Keycloak accounts, then migrate to metadata UUID users before applying this patch.',
users_id_type;
END IF;
END $$;
ALTER TABLE users
ADD COLUMN IF NOT EXISTS keycloak_id UUID,
ADD COLUMN IF NOT EXISTS attributes JSONB,
ADD COLUMN IF NOT EXISTS last_login_at TIMESTAMP WITH TIME ZONE;
ALTER TABLE users
ALTER COLUMN role SET DEFAULT 'user';
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_keycloak_id ON users(keycloak_id);
CREATE INDEX IF NOT EXISTS idx_users_role ON users(role);
CREATE INDEX IF NOT EXISTS idx_users_is_active ON users(is_active);
CREATE TABLE IF NOT EXISTS user_project_membership (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL,
project_id UUID NOT NULL,
project_role VARCHAR(20) DEFAULT 'viewer' NOT NULL,
CONSTRAINT user_project_membership_role_check
CHECK (project_role IN ('owner', 'admin', 'member', 'viewer')),
CONSTRAINT user_project_membership_unique UNIQUE (user_id, project_id)
);
CREATE INDEX IF NOT EXISTS idx_user_project_membership_user_id
ON user_project_membership(user_id);
CREATE INDEX IF NOT EXISTS idx_user_project_membership_project_id
ON user_project_membership(project_id);
@@ -0,0 +1,55 @@
-- Metadata project configuration schema patch.
-- Admin APIs write these tables; operators should not hand-edit encrypted values.
CREATE EXTENSION IF NOT EXISTS pgcrypto;
CREATE OR REPLACE FUNCTION update_updated_at_column()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = CURRENT_TIMESTAMP;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TABLE IF NOT EXISTS projects (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(100) NOT NULL,
code VARCHAR(50) UNIQUE NOT NULL,
description TEXT,
gs_workspace VARCHAR(100) UNIQUE NOT NULL,
map_extent JSONB,
status VARCHAR(20) DEFAULT 'active' NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
CONSTRAINT projects_status_check CHECK (status IN ('active', 'inactive', 'archived'))
);
CREATE INDEX IF NOT EXISTS idx_projects_status ON projects(status);
CREATE INDEX IF NOT EXISTS idx_projects_code ON projects(code);
DROP TRIGGER IF EXISTS update_projects_updated_at ON projects;
CREATE TRIGGER update_projects_updated_at
BEFORE UPDATE ON projects
FOR EACH ROW
EXECUTE FUNCTION update_updated_at_column();
CREATE TABLE IF NOT EXISTS project_databases (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id UUID NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
db_role VARCHAR(20) NOT NULL,
db_type VARCHAR(20) NOT NULL,
dsn_encrypted TEXT NOT NULL,
pool_min_size INTEGER DEFAULT 2 NOT NULL,
pool_max_size INTEGER DEFAULT 10 NOT NULL,
CONSTRAINT project_databases_unique_role UNIQUE (project_id, db_role),
CONSTRAINT project_databases_role_check CHECK (db_role IN ('biz_data', 'iot_data')),
CONSTRAINT project_databases_type_check CHECK (db_type IN ('postgresql', 'timescaledb')),
CONSTRAINT project_databases_pool_check CHECK (
pool_min_size >= 1 AND pool_max_size >= pool_min_size
)
);
CREATE INDEX IF NOT EXISTS idx_project_databases_project_id
ON project_databases(project_id);
CREATE INDEX IF NOT EXISTS idx_project_databases_role
ON project_databases(db_role);
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""Build metadata user sync payloads from an old-user to Keycloak mapping CSV."""
from __future__ import annotations
import argparse
import csv
import json
from pathlib import Path
REQUIRED_COLUMNS = {"keycloak_id", "username", "email"}
def parse_bool(value: str | None) -> bool:
if value is None or value == "":
return True
return value.strip().lower() not in {"0", "false", "no", "n", "disabled"}
def build_payload(mapping_csv: Path) -> dict:
with mapping_csv.open(newline="", encoding="utf-8") as handle:
reader = csv.DictReader(handle)
missing = REQUIRED_COLUMNS.difference(reader.fieldnames or [])
if missing:
raise SystemExit(f"missing required CSV columns: {', '.join(sorted(missing))}")
users = []
for row in reader:
users.append(
{
"keycloak_id": row["keycloak_id"].strip(),
"username": row["username"].strip(),
"email": row["email"].strip(),
"role": (row.get("role") or "user").strip().lower(),
"is_active": parse_bool(row.get("is_active")),
}
)
return {"users": users}
def main() -> None:
parser = argparse.ArgumentParser(
description=(
"Convert old local user mappings into a JSON body for "
"POST /api/v1/admin/users/sync/batch. Passwords are never migrated."
)
)
parser.add_argument("mapping_csv", type=Path)
parser.add_argument("-o", "--output", type=Path)
args = parser.parse_args()
payload = build_payload(args.mapping_csv)
content = json.dumps(payload, ensure_ascii=False, indent=2)
if args.output:
args.output.write_text(content + "\n", encoding="utf-8")
else:
print(content)
if __name__ == "__main__":
main()
+577
View File
@@ -0,0 +1,577 @@
from datetime import datetime, timezone
from types import SimpleNamespace
from unittest.mock import AsyncMock
from uuid import uuid4
import pytest
from fastapi import HTTPException
from fastapi import Response
from app.auth.metadata_dependencies import get_current_metadata_admin
from app.api.v1.endpoints import admin_metadata
from app.domain.schemas.admin_metadata import (
AdminProjectCreateRequest,
MetadataUsersBatchSyncRequest,
MetadataUserSyncRequest,
MetadataUserUpdateRequest,
ProjectDatabaseUpsertRequest,
ProjectMemberCreateRequest,
ProjectMemberUpdateRequest,
)
from app.infra.db.metadb.repositories.metadata_repository import ProjectDbRouting
@pytest.fixture
def anyio_backend():
return "asyncio"
def _user(**overrides):
data = {
"id": uuid4(),
"keycloak_id": uuid4(),
"username": "alice",
"email": "alice@example.com",
"role": "user",
"is_active": True,
"is_superuser": False,
"created_at": datetime(2026, 1, 1, tzinfo=timezone.utc),
"updated_at": datetime(2026, 1, 1, tzinfo=timezone.utc),
"last_login_at": None,
}
data.update(overrides)
return SimpleNamespace(**data)
def _project(**overrides):
data = {"id": uuid4(), "name": "Demo"}
data.update(overrides)
return SimpleNamespace(**data)
def _membership(**overrides):
data = {
"id": uuid4(),
"user_id": uuid4(),
"project_id": uuid4(),
"project_role": "viewer",
}
data.update(overrides)
return SimpleNamespace(**data)
def _database_config(**overrides):
data = {
"id": uuid4(),
"project_id": uuid4(),
"db_role": "biz_data",
"db_type": "postgresql",
"dsn_encrypted": "encrypted-dsn",
"pool_min_size": 1,
"pool_max_size": 5,
}
data.update(overrides)
return SimpleNamespace(**data)
def test_to_async_sqlalchemy_url_preserves_password():
url = admin_metadata._to_async_sqlalchemy_url(
"postgresql://tjwater:secret@192.168.1.114:5433/tjwater"
)
assert url == "postgresql+psycopg://tjwater:secret@192.168.1.114:5433/tjwater"
assert "***" not in url
@pytest.mark.anyio
async def test_sync_metadata_user_upserts_without_password(monkeypatch):
keycloak_id = uuid4()
synced_user = _user(keycloak_id=keycloak_id, username="new-user")
admin = _user(role="admin", is_superuser=True)
repo = SimpleNamespace(
session=object(),
upsert_user_from_keycloak=AsyncMock(return_value=synced_user),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
response = await admin_metadata.sync_metadata_user(
MetadataUserSyncRequest(
keycloak_id=keycloak_id,
username="new-user",
email="new-user@example.com",
role="user",
is_active=True,
),
current_user=admin,
metadata_repo=repo,
)
repo.upsert_user_from_keycloak.assert_awaited_once()
kwargs = repo.upsert_user_from_keycloak.await_args.kwargs
assert kwargs["keycloak_id"] == keycloak_id
assert "password" not in kwargs
assert response.username == "new-user"
admin_metadata.log_audit_event.assert_awaited_once()
@pytest.mark.anyio
async def test_batch_sync_metadata_users_returns_per_user_results(monkeypatch):
users = [_user(username="alice"), _user(username="bob")]
admin = _user(role="admin", is_superuser=True)
repo = SimpleNamespace(
session=SimpleNamespace(rollback=AsyncMock()),
upsert_user_from_keycloak=AsyncMock(side_effect=users),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
response = await admin_metadata.sync_metadata_users_batch(
MetadataUsersBatchSyncRequest(
users=[
MetadataUserSyncRequest(
keycloak_id=users[0].keycloak_id,
username="alice",
email="alice@example.com",
role="user",
is_active=True,
),
MetadataUserSyncRequest(
keycloak_id=users[1].keycloak_id,
username="bob",
email="bob@example.com",
role="viewer",
is_active=True,
),
]
),
current_user=admin,
metadata_repo=repo,
)
assert [item.success for item in response] == [True, True]
assert [item.user.username for item in response] == ["alice", "bob"]
assert repo.upsert_user_from_keycloak.await_count == 2
assert admin_metadata.log_audit_event.await_count == 2
@pytest.mark.anyio
async def test_update_metadata_user_updates_role_and_active_status(monkeypatch):
user_id = uuid4()
updated = _user(id=user_id, role="operator", is_active=False)
repo = SimpleNamespace(
session=object(),
update_user_admin=AsyncMock(return_value=updated),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
response = await admin_metadata.update_metadata_user(
MetadataUserUpdateRequest(
role="operator",
is_active=False,
),
user_id=user_id,
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
repo.update_user_admin.assert_awaited_once_with(
user_id,
updates={"role": "operator", "is_active": False},
)
assert response.role == "operator"
admin_metadata.log_audit_event.assert_awaited_once()
@pytest.mark.anyio
async def test_update_metadata_user_rejects_self_update(monkeypatch):
current_user = _user(role="admin", is_superuser=True)
repo = SimpleNamespace(
session=object(),
update_user_admin=AsyncMock(),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
with pytest.raises(HTTPException) as exc:
await admin_metadata.update_metadata_user(
MetadataUserUpdateRequest(role="viewer"),
user_id=current_user.id,
current_user=current_user,
metadata_repo=repo,
)
assert exc.value.status_code == 403
repo.update_user_admin.assert_not_called()
admin_metadata.log_audit_event.assert_not_called()
@pytest.mark.anyio
async def test_create_project_audits_metadata_admin_change(monkeypatch):
project = SimpleNamespace(
id=uuid4(),
name="Demo Project",
code="demo",
description="desc",
gs_workspace="demo_ws",
map_extent={"bbox": [1, 2, 3, 4]},
status="active",
created_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
updated_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
repo = SimpleNamespace(
session=object(),
create_project=AsyncMock(return_value=project),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
response = await admin_metadata.create_admin_project(
AdminProjectCreateRequest(
name="Demo Project",
code="demo",
description="desc",
gs_workspace="demo_ws",
map_extent={"bbox": [1, 2, 3, 4]},
status="active",
),
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert response.project_id == project.id
repo.create_project.assert_awaited_once()
admin_metadata.log_audit_event.assert_awaited_once()
@pytest.mark.anyio
async def test_upsert_project_database_hides_dsn_and_audits_without_plaintext(monkeypatch):
project_id = uuid4()
record = _database_config(project_id=project_id)
repo = SimpleNamespace(
session=object(),
get_project_by_id=AsyncMock(return_value=_project(id=project_id)),
upsert_project_database_config=AsyncMock(return_value=record),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
monkeypatch.setattr(admin_metadata, "_check_database_connection", AsyncMock())
response = await admin_metadata.upsert_project_database(
ProjectDatabaseUpsertRequest(
db_role="biz_data",
dsn="postgresql://user:secret@localhost/db",
pool_min_size=1,
pool_max_size=5,
),
project_id=project_id,
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert response.has_dsn is True
assert "dsn" not in response.model_dump()
admin_metadata._check_database_connection.assert_awaited_once()
repo.upsert_project_database_config.assert_awaited_once()
assert repo.upsert_project_database_config.await_args.kwargs["db_type"] == "postgresql"
request_data = admin_metadata.log_audit_event.await_args.kwargs["request_data"]
assert request_data["dsn_updated"] is True
assert request_data["db_type"] == "postgresql"
assert "dsn" not in request_data
assert "postgresql://user:secret@localhost/db" not in str(request_data)
@pytest.mark.anyio
async def test_upsert_project_database_rejects_unhealthy_connection(monkeypatch):
project_id = uuid4()
repo = SimpleNamespace(
session=object(),
get_project_by_id=AsyncMock(return_value=_project(id=project_id)),
upsert_project_database_config=AsyncMock(),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
monkeypatch.setattr(
admin_metadata,
"_check_database_connection",
AsyncMock(
side_effect=Exception(
'FATAL: password authentication failed for user "tjwater"'
)
),
)
with pytest.raises(HTTPException) as exc:
await admin_metadata.upsert_project_database(
ProjectDatabaseUpsertRequest(
db_role="iot_data",
dsn="postgresql://tjwater:bad@192.168.1.114:5433/tjwater",
pool_min_size=1,
pool_max_size=5,
),
project_id=project_id,
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert exc.value.status_code == 400
assert exc.value.detail == "连通性测试失败:用户名或密码错误,请检查 DSN 中的账号密码。"
repo.upsert_project_database_config.assert_not_called()
admin_metadata.log_audit_event.assert_not_called()
@pytest.mark.anyio
async def test_project_database_health_returns_ok(monkeypatch):
project_id = uuid4()
repo = SimpleNamespace(
get_project_db_routing=AsyncMock(
return_value=ProjectDbRouting(
project_id=project_id,
db_role="biz_data",
db_type="postgresql",
dsn="postgresql://user:secret@localhost/db",
pool_min_size=1,
pool_max_size=5,
)
)
)
monkeypatch.setattr(admin_metadata, "_check_database_connection", AsyncMock())
response = await admin_metadata.check_project_database_health(
project_id=project_id,
db_role="biz_data",
response=Response(),
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert response.ok is True
assert response.detail == "连通性测试通过"
admin_metadata._check_database_connection.assert_awaited_once()
@pytest.mark.anyio
async def test_project_database_health_can_test_unsaved_plaintext_dsn(monkeypatch):
project_id = uuid4()
repo = SimpleNamespace(
get_project_db_routing=AsyncMock(),
)
monkeypatch.setattr(admin_metadata, "_check_database_connection", AsyncMock())
response = await admin_metadata.check_project_database_health(
project_id=project_id,
db_role="iot_data",
payload=admin_metadata.ProjectDatabaseHealthRequest(
dsn="postgresql://tjwater:secret@192.168.1.114:5433/tjwater"
),
response=Response(),
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert response.ok is True
assert response.db_type == "timescaledb"
routing = admin_metadata._check_database_connection.await_args.args[0]
assert routing.dsn == "postgresql://tjwater:secret@192.168.1.114:5433/tjwater"
repo.get_project_db_routing.assert_not_called()
@pytest.mark.anyio
async def test_project_database_health_sanitizes_password_failures(monkeypatch):
project_id = uuid4()
repo = SimpleNamespace(
get_project_db_routing=AsyncMock(
return_value=ProjectDbRouting(
project_id=project_id,
db_role="iot_data",
db_type="timescaledb",
dsn="postgresql://tjwater:bad-password@192.168.1.114:5433/db",
pool_min_size=1,
pool_max_size=5,
)
)
)
monkeypatch.setattr(
admin_metadata,
"_check_database_connection",
AsyncMock(
side_effect=Exception(
'(psycopg.OperationalError) connection failed: FATAL: '
'password authentication failed for user "tjwater"'
)
),
)
fastapi_response = Response()
response = await admin_metadata.check_project_database_health(
project_id=project_id,
db_role="iot_data",
response=fastapi_response,
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert fastapi_response.status_code == 503
assert response.ok is False
assert response.db_type == "timescaledb"
assert response.detail == "连通性测试失败:用户名或密码错误,请检查 DSN 中的账号密码。"
assert "psycopg" not in response.detail
@pytest.mark.anyio
async def test_metadata_admin_dependency_rejects_non_admin_user():
with pytest.raises(HTTPException) as exc:
await get_current_metadata_admin(_user(role="user", is_superuser=False))
assert exc.value.status_code == 403
assert exc.value.detail == "Admin access required"
@pytest.mark.anyio
async def test_add_project_member_rejects_duplicate(monkeypatch):
project_id = uuid4()
user_id = uuid4()
repo = SimpleNamespace(
session=object(),
get_project_by_id=AsyncMock(return_value=_project(id=project_id)),
get_user_by_id=AsyncMock(return_value=_user(id=user_id)),
get_project_membership=AsyncMock(return_value=_membership()),
add_project_member=AsyncMock(),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
with pytest.raises(HTTPException) as exc:
await admin_metadata.add_project_member(
ProjectMemberCreateRequest(user_id=user_id, project_role="viewer"),
project_id=project_id,
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert exc.value.status_code == 409
repo.add_project_member.assert_not_called()
@pytest.mark.anyio
async def test_add_project_member_rejects_self_membership_change(monkeypatch):
project_id = uuid4()
current_user = _user(role="admin", is_superuser=True)
repo = SimpleNamespace(
session=object(),
get_project_by_id=AsyncMock(),
add_project_member=AsyncMock(),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
with pytest.raises(HTTPException) as exc:
await admin_metadata.add_project_member(
ProjectMemberCreateRequest(
user_id=current_user.id,
project_role="viewer",
),
project_id=project_id,
current_user=current_user,
metadata_repo=repo,
)
assert exc.value.status_code == 403
repo.get_project_by_id.assert_not_called()
repo.add_project_member.assert_not_called()
admin_metadata.log_audit_event.assert_not_called()
@pytest.mark.anyio
async def test_update_project_member_role_audits_change(monkeypatch):
project_id = uuid4()
user_id = uuid4()
user = _user(id=user_id, username="bob", email="bob@example.com")
membership = _membership(
user_id=user_id,
project_id=project_id,
project_role="admin",
)
repo = SimpleNamespace(
session=object(),
get_user_by_id=AsyncMock(return_value=user),
update_project_member_role=AsyncMock(return_value=membership),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
response = await admin_metadata.update_project_member(
ProjectMemberUpdateRequest(project_role="admin"),
project_id=project_id,
user_id=user_id,
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert response.project_role == "admin"
repo.update_project_member_role.assert_awaited_once_with(
project_id, user_id, "admin"
)
admin_metadata.log_audit_event.assert_awaited_once()
@pytest.mark.anyio
async def test_update_project_member_rejects_self_membership_change(monkeypatch):
project_id = uuid4()
current_user = _user(role="admin", is_superuser=True)
repo = SimpleNamespace(
session=object(),
get_user_by_id=AsyncMock(),
update_project_member_role=AsyncMock(),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
with pytest.raises(HTTPException) as exc:
await admin_metadata.update_project_member(
ProjectMemberUpdateRequest(project_role="admin"),
project_id=project_id,
user_id=current_user.id,
current_user=current_user,
metadata_repo=repo,
)
assert exc.value.status_code == 403
repo.get_user_by_id.assert_not_called()
repo.update_project_member_role.assert_not_called()
admin_metadata.log_audit_event.assert_not_called()
@pytest.mark.anyio
async def test_remove_project_member_audits_change(monkeypatch):
project_id = uuid4()
user_id = uuid4()
repo = SimpleNamespace(
session=object(),
remove_project_member=AsyncMock(return_value=True),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
response = await admin_metadata.remove_project_member(
project_id=project_id,
user_id=user_id,
current_user=_user(role="admin", is_superuser=True),
metadata_repo=repo,
)
assert response is None
repo.remove_project_member.assert_awaited_once_with(project_id, user_id)
admin_metadata.log_audit_event.assert_awaited_once()
@pytest.mark.anyio
async def test_remove_project_member_rejects_self_membership_change(monkeypatch):
project_id = uuid4()
current_user = _user(role="admin", is_superuser=True)
repo = SimpleNamespace(
session=object(),
remove_project_member=AsyncMock(),
)
monkeypatch.setattr(admin_metadata, "log_audit_event", AsyncMock())
with pytest.raises(HTTPException) as exc:
await admin_metadata.remove_project_member(
project_id=project_id,
user_id=current_user.id,
current_user=current_user,
metadata_repo=repo,
)
assert exc.value.status_code == 403
repo.remove_project_member.assert_not_called()
admin_metadata.log_audit_event.assert_not_called()
+82
View File
@@ -0,0 +1,82 @@
from types import SimpleNamespace
from uuid import uuid4
from fastapi import HTTPException, status
from fastapi.testclient import TestClient
from app.api.v1.endpoints import agent_auth as agent_auth_endpoint
from app.auth.keycloak_dependencies import get_current_keycloak_payload
from app.auth.metadata_dependencies import get_current_metadata_user
from app.auth.project_dependencies import ProjectContext, get_project_context
from tests.conftest import build_test_app
def _build_client(*, project_context=None, current_user=None) -> TestClient:
app = build_test_app(agent_auth_endpoint.router, "/api/v1")
if project_context is not None:
app.dependency_overrides[get_project_context] = lambda: project_context
if current_user is not None:
app.dependency_overrides[get_current_metadata_user] = lambda: current_user
app.dependency_overrides[get_current_keycloak_payload] = lambda: {"exp": 1781183400}
return TestClient(app)
def test_agent_auth_context_returns_metadata_user_and_project_context():
user_id = uuid4()
keycloak_sub = uuid4()
project_id = uuid4()
client = _build_client(
project_context=ProjectContext(
project_id=project_id,
project_code="fengyang",
user_id=user_id,
project_role="editor",
),
current_user=SimpleNamespace(
id=user_id,
keycloak_id=keycloak_sub,
username="alice",
role="user",
is_superuser=False,
),
)
response = client.get("/api/v1/agent/auth/context")
assert response.status_code == 200
assert response.json() == {
"user_id": str(user_id),
"keycloak_sub": str(keycloak_sub),
"username": "alice",
"role": "user",
"is_superuser": False,
"project_id": str(project_id),
"network": "fengyang",
"project_role": "editor",
"token_expires_at": "2026-06-11T13:10:00+00:00",
}
def test_agent_auth_context_propagates_project_auth_failures():
def reject_project():
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="No access to project",
)
app = build_test_app(agent_auth_endpoint.router, "/api/v1")
app.dependency_overrides[get_project_context] = reject_project
app.dependency_overrides[get_current_metadata_user] = lambda: SimpleNamespace(
id=uuid4(),
keycloak_id=uuid4(),
username="alice",
role="user",
is_superuser=False,
)
app.dependency_overrides[get_current_keycloak_payload] = lambda: {"exp": 1781183400}
client = TestClient(app)
response = client.get("/api/v1/agent/auth/context")
assert response.status_code == 403
assert response.json()["detail"] == "No access to project"
+9 -10
View File
@@ -2,7 +2,7 @@
""" """
测试新增 API 集成 测试新增 API 集成
验证新的认证用户管理和审计日志接口是否正确集成 验证 Keycloak/metadata 认证和审计日志接口是否正确集成
""" """
import sys import sys
@@ -17,16 +17,15 @@ sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../.
"module_name, desc", "module_name, desc",
[ [
("app.core.encryption", "加密模块"), ("app.core.encryption", "加密模块"),
("app.core.security", "安全模块"),
("app.core.audit", "审计模块"), ("app.core.audit", "审计模块"),
("app.domain.models.role", "角色模型"),
("app.domain.schemas.user", "用户Schema"),
("app.domain.schemas.audit", "审计Schema"), ("app.domain.schemas.audit", "审计Schema"),
("app.auth.permissions", "权限控制"), ("app.auth.keycloak_dependencies", "Keycloak Token 校验"),
("app.api.v1.endpoints.auth", "认证接口"), ("app.auth.metadata_dependencies", "Metadata 用户解析"),
("app.api.v1.endpoints.user_management", "用户管理接口"), ("app.auth.project_dependencies", "项目权限控制"),
("app.api.v1.endpoints.agent_auth", "Agent 认证上下文接口"),
("app.api.v1.endpoints.meta", "Metadata 接口"),
("app.api.v1.endpoints.audit", "审计日志接口"), ("app.api.v1.endpoints.audit", "审计日志接口"),
("app.infra.db.metadb.repositories.user_repository", "用户仓储"), ("app.infra.db.metadb.repositories.metadata_repository", "Metadata 仓储"),
("app.infra.db.metadb.repositories.audit_repository", "审计仓储"), ("app.infra.db.metadb.repositories.audit_repository", "审计仓储"),
("app.infra.audit.middleware", "审计中间件"), ("app.infra.audit.middleware", "审计中间件"),
], ],
@@ -49,8 +48,8 @@ def test_router_configuration():
routes = [r.path for r in api_router.routes if hasattr(r, "path")] routes = [r.path for r in api_router.routes if hasattr(r, "path")]
# 验证基础路径是否存在 # 验证基础路径是否存在
assert any("/auth" in r for r in routes), "缺少认证相关路由 (/auth)" assert any("/agent/auth/context" in r for r in routes), "缺少 Agent 认证上下文路由"
assert any("/users" in r for r in routes), "缺少用户管理路由 (/users)" assert any("/meta" in r for r in routes), "缺少 Metadata 路由"
assert any("/audit" in r for r in routes), "缺少审计日志路由 (/audit)" assert any("/audit" in r for r in routes), "缺少审计日志路由 (/audit)"
except Exception as e: except Exception as e:
-139
View File
@@ -1,139 +0,0 @@
from types import SimpleNamespace
from unittest.mock import AsyncMock
from fastapi.testclient import TestClient
from app.api.v1.endpoints import auth as auth_endpoint
from app.auth.dependencies import get_current_active_user, get_user_repository
from app.core.security import create_access_token, create_refresh_token, get_password_hash
from tests.conftest import build_test_app, make_user
def _build_client(repo, current_user=None) -> TestClient:
app = build_test_app(auth_endpoint.router, "/api/v1/auth")
app.dependency_overrides[get_user_repository] = lambda: repo
if current_user is not None:
app.dependency_overrides[get_current_active_user] = lambda: current_user
return TestClient(app)
def test_register_success():
repo = SimpleNamespace(
user_exists=AsyncMock(side_effect=[False, False]),
create_user=AsyncMock(return_value=make_user()),
)
client = _build_client(repo)
response = client.post(
"/api/v1/auth/register",
json={
"username": "tester",
"email": "tester@example.com",
"password": "secret123",
},
)
assert response.status_code == 201
assert response.json()["username"] == "tester"
def test_register_rejects_duplicate_username():
repo = SimpleNamespace(
user_exists=AsyncMock(side_effect=[True]),
create_user=AsyncMock(),
)
client = _build_client(repo)
response = client.post(
"/api/v1/auth/register",
json={
"username": "tester",
"email": "tester@example.com",
"password": "secret123",
},
)
assert response.status_code == 400
assert response.json()["detail"] == "Username already registered"
repo.create_user.assert_not_awaited()
def test_login_supports_email_lookup():
hashed_password = get_password_hash("secret123")
repo = SimpleNamespace(
get_user_by_username=AsyncMock(return_value=None),
get_user_by_email=AsyncMock(
return_value=make_user(
email="tester@example.com",
hashed_password=hashed_password,
)
),
)
client = _build_client(repo)
response = client.post(
"/api/v1/auth/login",
data={"username": "tester@example.com", "password": "secret123"},
)
assert response.status_code == 200
assert response.json()["token_type"] == "bearer"
repo.get_user_by_email.assert_awaited_once_with("tester@example.com")
def test_login_simple_uses_query_params():
hashed_password = get_password_hash("secret123")
repo = SimpleNamespace(
get_user_by_username=AsyncMock(
return_value=make_user(hashed_password=hashed_password)
),
get_user_by_email=AsyncMock(),
)
client = _build_client(repo)
response = client.post(
"/api/v1/auth/login/simple",
params={"username": "tester", "password": "secret123"},
)
assert response.status_code == 200
assert response.json()["token_type"] == "bearer"
def test_me_returns_current_user_info():
client = _build_client(SimpleNamespace(), current_user=make_user(username="alice"))
response = client.get("/api/v1/auth/me")
assert response.status_code == 200
assert response.json()["username"] == "alice"
def test_refresh_rejects_access_token():
repo = SimpleNamespace(get_user_by_username=AsyncMock())
client = _build_client(repo)
response = client.post(
"/api/v1/auth/refresh",
params={"refresh_token": create_access_token("tester")},
)
assert response.status_code == 401
def test_refresh_success_returns_new_access_token():
repo = SimpleNamespace(
get_user_by_username=AsyncMock(return_value=make_user()),
)
client = _build_client(repo)
refresh_token = create_refresh_token("tester")
response = client.post(
"/api/v1/auth/refresh",
params={"refresh_token": refresh_token},
)
assert response.status_code == 200
payload = response.json()
assert payload["refresh_token"] == refresh_token
assert payload["token_type"] == "bearer"
-5
View File
@@ -36,7 +36,6 @@ def test_meta_project_returns_map_extent(monkeypatch):
project_id = uuid4() project_id = uuid4()
repo = SimpleNamespace( repo = SimpleNamespace(
get_project_by_id=lambda _project_id: None, get_project_by_id=lambda _project_id: None,
get_geoserver_config=lambda _project_id: None,
) )
async def get_project_by_id(_project_id): async def get_project_by_id(_project_id):
@@ -50,11 +49,7 @@ def test_meta_project_returns_map_extent(monkeypatch):
status="active", status="active",
) )
async def get_geoserver_config(_project_id):
return None
repo.get_project_by_id = get_project_by_id repo.get_project_by_id = get_project_by_id
repo.get_geoserver_config = get_geoserver_config
app = build_test_app(module.router, "/api/v1") app = build_test_app(module.router, "/api/v1")
app.dependency_overrides[module.get_project_context] = lambda: SimpleNamespace( app.dependency_overrides[module.get_project_context] = lambda: SimpleNamespace(
+3 -9
View File
@@ -84,7 +84,7 @@ def test_project_info_returns_404_when_missing(monkeypatch):
assert response.json()["detail"] == "Project missing not found" assert response.json()["detail"] == "Project missing not found"
def test_project_info_returns_geoserver_payload(monkeypatch): def test_project_info_returns_project_workspace(monkeypatch):
module = _load_project_module(monkeypatch) module = _load_project_module(monkeypatch)
detail = SimpleNamespace( detail = SimpleNamespace(
project_id=uuid4(), project_id=uuid4(),
@@ -94,13 +94,6 @@ def test_project_info_returns_geoserver_payload(monkeypatch):
gs_workspace="ws", gs_workspace="ws",
map_extent={"xmin": 1, "ymin": 2, "xmax": 3, "ymax": 4}, map_extent={"xmin": 1, "ymin": 2, "xmax": 3, "ymax": 4},
status="active", status="active",
geoserver=SimpleNamespace(
gs_base_url="http://gs",
gs_admin_user="admin",
gs_datastore_name="store",
default_extent={"xmin": 1, "ymin": 2, "xmax": 3, "ymax": 4},
srid=4326,
),
) )
repo = SimpleNamespace(get_project_detail_by_code=AsyncMock(return_value=detail)) repo = SimpleNamespace(get_project_detail_by_code=AsyncMock(return_value=detail))
app = build_test_app(module.router, "/api/v1") app = build_test_app(module.router, "/api/v1")
@@ -112,7 +105,8 @@ def test_project_info_returns_geoserver_payload(monkeypatch):
assert response.status_code == 200 assert response.status_code == 200
payload = response.json() payload = response.json()
assert payload["code"] == "demo" assert payload["code"] == "demo"
assert payload["geoserver"]["gs_base_url"] == "http://gs" assert payload["gs_workspace"] == "ws"
assert "geoserver" not in payload
def test_open_project_returns_network_even_when_db_connection_fails(monkeypatch): def test_open_project_returns_network_even_when_db_connection_fails(monkeypatch):
@@ -1,95 +0,0 @@
from types import SimpleNamespace
from unittest.mock import AsyncMock
from fastapi.testclient import TestClient
from app.api.v1.endpoints import user_management as user_management_endpoint
from app.auth.dependencies import get_current_active_user, get_user_repository
from app.auth.permissions import get_current_admin
from app.domain.models.role import UserRole
from tests.conftest import build_test_app, make_user
def _build_client(repo, *, current_user=None, admin_user=None) -> TestClient:
app = build_test_app(user_management_endpoint.router, "/users")
app.dependency_overrides[get_user_repository] = lambda: repo
if current_user is not None:
app.dependency_overrides[get_current_active_user] = lambda: current_user
if admin_user is not None:
app.dependency_overrides[get_current_admin] = lambda: admin_user
return TestClient(app)
def test_list_users_requires_admin_role():
repo = SimpleNamespace(
get_all_users=AsyncMock(
return_value=[
make_user(id=1, username="admin", role=UserRole.ADMIN),
make_user(id=2, username="user2"),
]
)
)
client = _build_client(
repo,
current_user=make_user(id=1, role=UserRole.ADMIN),
)
response = client.get("/users/", params={"skip": 5, "limit": 2})
assert response.status_code == 200
assert len(response.json()) == 2
repo.get_all_users.assert_awaited_once_with(skip=5, limit=2)
def test_get_user_rejects_non_owner_non_admin():
repo = SimpleNamespace(get_user_by_id=AsyncMock())
client = _build_client(repo, current_user=make_user(id=2, role=UserRole.USER))
response = client.get("/users/3")
assert response.status_code == 403
assert response.json()["detail"] == "You don't have permission to view this user"
repo.get_user_by_id.assert_not_awaited()
def test_update_user_blocks_role_change_for_non_admin():
repo = SimpleNamespace(
get_user_by_id=AsyncMock(return_value=make_user(id=1)),
update_user=AsyncMock(),
)
client = _build_client(repo, current_user=make_user(id=1, role=UserRole.USER))
response = client.put("/users/1", json={"role": "ADMIN"})
assert response.status_code == 403
assert response.json()["detail"] == "Only admins can change user roles"
repo.update_user.assert_not_awaited()
def test_delete_user_blocks_self_delete_for_admin():
admin_user = make_user(id=1, role=UserRole.ADMIN, is_superuser=True)
repo = SimpleNamespace(delete_user=AsyncMock())
client = _build_client(repo, admin_user=admin_user)
response = client.delete("/users/1")
assert response.status_code == 400
assert response.json()["detail"] == "You cannot delete your own account"
repo.delete_user.assert_not_awaited()
def test_activate_user_updates_active_flag():
repo = SimpleNamespace(
update_user=AsyncMock(return_value=make_user(id=2, is_active=True)),
)
client = _build_client(
repo,
admin_user=make_user(id=1, role=UserRole.ADMIN, is_superuser=True),
)
response = client.post("/users/2/activate")
assert response.status_code == 200
assert response.json()["is_active"] is True
user_update = repo.update_user.await_args.args[1]
assert user_update.is_active is True
+83
View File
@@ -0,0 +1,83 @@
from datetime import datetime, timezone
from types import SimpleNamespace
from unittest.mock import AsyncMock
from uuid import uuid4
import pytest
from fastapi import HTTPException
from app.auth import metadata_dependencies
@pytest.fixture
def anyio_backend():
return "asyncio"
def _user(**overrides):
data = {
"id": uuid4(),
"keycloak_id": uuid4(),
"username": "old-name",
"email": "old@example.com",
"role": "user",
"is_active": True,
"is_superuser": False,
"created_at": datetime(2026, 1, 1, tzinfo=timezone.utc),
"updated_at": datetime(2026, 1, 1, tzinfo=timezone.utc),
"last_login_at": None,
}
data.update(overrides)
return SimpleNamespace(**data)
@pytest.mark.anyio
async def test_current_metadata_user_refreshes_keycloak_claim_snapshot():
keycloak_id = uuid4()
user = _user(keycloak_id=keycloak_id)
refreshed = _user(
id=user.id,
keycloak_id=keycloak_id,
username="alice",
email="alice@example.com",
last_login_at=datetime(2026, 6, 12, tzinfo=timezone.utc),
)
repo = SimpleNamespace(
get_user_by_keycloak_id=AsyncMock(return_value=user),
refresh_user_keycloak_snapshot=AsyncMock(return_value=refreshed),
)
response = await metadata_dependencies.get_current_metadata_user(
{
"sub": str(keycloak_id),
"preferred_username": "alice",
"email": "alice@example.com",
},
metadata_repo=repo,
)
assert response.username == "alice"
repo.get_user_by_keycloak_id.assert_awaited_once_with(keycloak_id)
repo.refresh_user_keycloak_snapshot.assert_awaited_once_with(
user,
username="alice",
email="alice@example.com",
)
@pytest.mark.anyio
async def test_current_metadata_user_rejects_invalid_keycloak_sub():
repo = SimpleNamespace(
get_user_by_keycloak_id=AsyncMock(),
refresh_user_keycloak_snapshot=AsyncMock(),
)
with pytest.raises(HTTPException) as exc:
await metadata_dependencies.get_current_metadata_user(
{"sub": "not-a-uuid"},
metadata_repo=repo,
)
assert exc.value.status_code == 401
repo.get_user_by_keycloak_id.assert_not_called()
repo.refresh_user_keycloak_snapshot.assert_not_called()
-36
View File
@@ -1,36 +0,0 @@
from jose import jwt
from app.core.config import settings
from app.core.security import (
create_access_token,
create_refresh_token,
get_password_hash,
verify_password,
)
def test_password_hash_roundtrip():
hashed = get_password_hash("secret123")
assert hashed != "secret123"
assert verify_password("secret123", hashed) is True
assert verify_password("wrong", hashed) is False
def test_create_access_token_sets_access_type():
token = create_access_token("alice")
payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
assert payload["sub"] == "alice"
assert payload["type"] == "access"
assert "exp" in payload
assert "iat" in payload
def test_create_refresh_token_sets_refresh_type():
token = create_refresh_token("alice")
payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
assert payload["sub"] == "alice"
assert payload["type"] == "refresh"
assert "exp" in payload
assert "iat" in payload
-19
View File
@@ -159,25 +159,6 @@ class FakeAsyncSession:
self.refreshed.append(obj) self.refreshed.append(obj)
def make_user(**overrides):
from app.domain.models.role import UserRole
from app.domain.schemas.user import UserInDB
data = {
"id": 1,
"username": "tester",
"email": "tester@example.com",
"hashed_password": "hashed-password",
"role": UserRole.USER,
"is_active": True,
"is_superuser": False,
"created_at": datetime(2025, 1, 1, tzinfo=timezone.utc),
"updated_at": datetime(2025, 1, 1, tzinfo=timezone.utc),
}
data.update(overrides)
return UserInDB(**data)
def make_audit_log(**overrides): def make_audit_log(**overrides):
data = { data = {
"id": uuid4(), "id": uuid4(),
+3 -3
View File
@@ -22,14 +22,14 @@ def test_create_log_adds_commits_and_refreshes(monkeypatch):
result = asyncio.run( result = asyncio.run(
repo.create_log( repo.create_log(
action="LOGIN", action="CREATE_PROJECT",
request_method="POST", request_method="POST",
request_path="/auth/login", request_path="/api/v1/projects",
response_status=200, response_status=200,
) )
) )
assert result.action == "LOGIN" assert result.action == "CREATE_PROJECT"
assert result.request_method == "POST" assert result.request_method == "POST"
assert session.commit_count == 1 assert session.commit_count == 1
assert len(session.added) == 1 assert len(session.added) == 1
-97
View File
@@ -1,97 +0,0 @@
import asyncio
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
from fastapi import HTTPException
from app.auth import dependencies
from app.core.security import create_access_token, create_refresh_token
from tests.conftest import make_user
def test_get_db_returns_app_state_db():
request = SimpleNamespace(app=SimpleNamespace(state=SimpleNamespace(db="db-instance")))
result = asyncio.run(dependencies.get_db(request))
assert result == "db-instance"
def test_get_db_raises_when_database_missing():
request = SimpleNamespace(app=SimpleNamespace(state=SimpleNamespace()))
with pytest.raises(HTTPException) as exc_info:
asyncio.run(dependencies.get_db(request))
assert exc_info.value.status_code == 503
assert exc_info.value.detail == "Database not initialized"
def test_get_current_user_accepts_valid_access_token():
repo = SimpleNamespace(get_user_by_username=AsyncMock(return_value=make_user()))
result = asyncio.run(
dependencies.get_current_user(
token=create_access_token("tester"),
user_repo=repo,
)
)
assert result.username == "tester"
repo.get_user_by_username.assert_awaited_once_with("tester")
def test_get_current_user_rejects_refresh_token():
repo = SimpleNamespace(get_user_by_username=AsyncMock())
with pytest.raises(HTTPException) as exc_info:
asyncio.run(
dependencies.get_current_user(
token=create_refresh_token("tester"),
user_repo=repo,
)
)
assert exc_info.value.status_code == 401
assert exc_info.value.detail == "Invalid token type. Access token required."
repo.get_user_by_username.assert_not_awaited()
def test_get_current_user_rejects_missing_user():
repo = SimpleNamespace(get_user_by_username=AsyncMock(return_value=None))
with pytest.raises(HTTPException) as exc_info:
asyncio.run(
dependencies.get_current_user(
token=create_access_token("ghost"),
user_repo=repo,
)
)
assert exc_info.value.status_code == 401
assert exc_info.value.detail == "Could not validate credentials"
def test_get_current_active_user_rejects_inactive_user():
with pytest.raises(HTTPException) as exc_info:
asyncio.run(
dependencies.get_current_active_user(
current_user=make_user(is_active=False),
)
)
assert exc_info.value.status_code == 403
assert exc_info.value.detail == "Inactive user"
def test_get_current_superuser_rejects_non_superuser():
with pytest.raises(HTTPException) as exc_info:
asyncio.run(
dependencies.get_current_superuser(
current_user=make_user(is_superuser=False),
)
)
assert exc_info.value.status_code == 403
assert exc_info.value.detail == "Not enough privileges. Superuser access required."
+167 -17
View File
@@ -190,7 +190,7 @@ def test_run_burst_location_uses_single_timerange_with_burst_source_split(monkey
use_scada_flow=True, use_scada_flow=True,
) )
assert result["observed_source"] == "simulation_scheme_burst_realtime_normal_timerange" assert result["observed_source"] == "simulation_scheme_timerange"
assert result["simulation_scheme"] == { assert result["simulation_scheme"] == {
"name": "BurstSchemeA", "name": "BurstSchemeA",
"type": "burst_analysis", "type": "burst_analysis",
@@ -199,22 +199,17 @@ def test_run_burst_location_uses_single_timerange_with_burst_source_split(monkey
assert result["flow_samples"] == {"burst": 4, "normal": 4} assert result["flow_samples"] == {"burst": 4, "normal": 4}
assert list(captured["burst_pressure"].index) == ["J1"] assert list(captured["burst_pressure"].index) == ["J1"]
assert captured["burst_pressure"]["J1"] == pytest.approx(15.0) assert captured["burst_pressure"]["J1"] == pytest.approx(15.0)
assert captured["normal_pressure"]["J1"] == pytest.approx(11.0) assert captured["normal_pressure"]["J1"] == pytest.approx(15.0)
assert captured["burst_flow"]["J2"] == pytest.approx(6.0) assert captured["burst_flow"]["J2"] == pytest.approx(6.0)
assert captured["burst_flow"]["P1"] == pytest.approx(8.0) assert captured["burst_flow"]["P1"] == pytest.approx(8.0)
assert captured["normal_flow"]["J2"] == pytest.approx(4.0) assert captured["normal_flow"]["J2"] == pytest.approx(6.0)
assert captured["normal_flow"]["P1"] == pytest.approx(5.0) assert captured["normal_flow"]["P1"] == pytest.approx(8.0)
assert all(call["scheme_name"] == "BurstSchemeA" for call in scheme_calls) assert all(call["scheme_name"] == "BurstSchemeA" for call in scheme_calls)
assert len(scheme_calls) == 3 assert len(scheme_calls) == 6
assert any(call["element_type"] == "node" and call["field"] == "pressure" for call in scheme_calls) assert any(call["element_type"] == "node" and call["field"] == "pressure" for call in scheme_calls)
assert any(call["element_type"] == "link" and call["field"] == "flow" for call in scheme_calls) assert any(call["element_type"] == "link" and call["field"] == "flow" for call in scheme_calls)
assert any(call["element_type"] == "node" and call["field"] == "actual_demand" for call in scheme_calls) assert any(call["element_type"] == "node" and call["field"] == "actual_demand" for call in scheme_calls)
assert len(realtime_calls) == 3 assert realtime_calls == []
assert all(datetime.fromisoformat(call["start_time"]).hour == 0 for call in realtime_calls)
assert all(datetime.fromisoformat(call["end_time"]).hour == 1 for call in realtime_calls)
assert any(call["element_type"] == "node" and call["field"] == "pressure" for call in realtime_calls)
assert any(call["element_type"] == "link" and call["field"] == "flow" for call in realtime_calls)
assert any(call["element_type"] == "node" and call["field"] == "actual_demand" for call in realtime_calls)
assert result["scada_window"] == { assert result["scada_window"] == {
"burst_start": "2025-01-01T00:00:00+00:00", "burst_start": "2025-01-01T00:00:00+00:00",
"burst_end": "2025-01-01T01:00:00+00:00", "burst_end": "2025-01-01T01:00:00+00:00",
@@ -248,7 +243,90 @@ def test_run_burst_location_requires_simulation_scheme_name(monkeypatch, tmp_pat
) )
def test_run_burst_location_monitoring_uses_scada_for_burst_and_realtime_for_normal( def test_build_observed_series_from_simulation_normalizes_result_ids(monkeypatch):
module = _load_burst_location_module()
query_calls = []
monkeypatch.setattr(
module,
"get_all_scada_info",
lambda network: [
{
"type": "pressure",
"associated_element_id": " 100026 ",
"api_query_id": " pressure-query ",
}
],
)
def fake_scheme_query(**kwargs):
query_calls.append(kwargs)
return {
100026: [
{"time": kwargs["start_time"], "value": 10.0},
{"time": kwargs["end_time"], "value": 14.0},
]
}
monkeypatch.setattr(
module.InternalQueries,
"query_scheme_simulation_by_ids_timerange",
staticmethod(fake_scheme_query),
)
series, sample_count = module._build_observed_series_from_simulation(
network="tjwater",
sensor_ids=["100026"],
start_dt=datetime(2025, 1, 1, 0, 0, 0, tzinfo=timezone.utc),
end_dt=datetime(2025, 1, 1, 1, 0, 0, tzinfo=timezone.utc),
data_type="pressure",
series_name="burst_pressure",
simulation_source="scheme",
simulation_scheme_name="BurstSchemeA",
simulation_scheme_type="burst_analysis",
)
assert query_calls[0]["element_ids"] == ["100026"]
assert sample_count == 2
assert series["100026"] == pytest.approx(12.0)
def test_build_observed_series_from_scada_uses_chinese_error_label(monkeypatch):
module = _load_burst_location_module()
monkeypatch.setattr(
module,
"get_all_scada_info",
lambda network: [
{
"type": "pressure",
"associated_element_id": "100026",
"api_query_id": "pressure-query",
}
],
)
monkeypatch.setattr(
module.InternalQueries,
"query_scada_by_ids_timerange",
staticmethod(lambda **kwargs: {"pressure-query": []}),
)
with pytest.raises(ValueError) as exc_info:
module._build_observed_series_from_scada(
network="tjwater",
sensor_ids=["100026"],
start_dt=datetime(2025, 1, 1, 0, 0, 0, tzinfo=timezone.utc),
end_dt=datetime(2025, 1, 1, 1, 0, 0, tzinfo=timezone.utc),
data_type="pressure",
series_name="burst_pressure",
)
message = str(exc_info.value)
assert "爆管压力数据 在时间窗内无有效数据: 100026" in message
assert "burst_pressure" not in message
def test_run_burst_location_monitoring_uses_scada_for_burst_and_normal(
monkeypatch, tmp_path monkeypatch, tmp_path
): ):
module = _load_burst_location_module() module = _load_burst_location_module()
@@ -276,10 +354,14 @@ def test_run_burst_location_monitoring_uses_scada_for_burst_and_realtime_for_nor
def fake_scada_query(**kwargs): def fake_scada_query(**kwargs):
scada_calls.append(kwargs) scada_calls.append(kwargs)
start_hour = datetime.fromisoformat(kwargs["start_time"]).astimezone(
timezone(timedelta(hours=8))
).hour
values = [20.0, 22.0] if start_hour == 8 else [10.0, 12.0]
return { return {
"pressure-query": [ "pressure-query": [
{"time": kwargs["start_time"], "value": 20.0}, {"time": kwargs["start_time"], "value": values[0]},
{"time": kwargs["end_time"], "value": 22.0}, {"time": kwargs["end_time"], "value": values[1]},
] ]
} }
@@ -310,10 +392,78 @@ def test_run_burst_location_monitoring_uses_scada_for_burst_and_realtime_for_nor
burst_leakage=1.0, burst_leakage=1.0,
scada_burst_start=datetime(2025, 1, 1, 8, 0, 0, tzinfo=timezone(timedelta(hours=8))), scada_burst_start=datetime(2025, 1, 1, 8, 0, 0, tzinfo=timezone(timedelta(hours=8))),
scada_burst_end=datetime(2025, 1, 1, 9, 0, 0, tzinfo=timezone(timedelta(hours=8))), scada_burst_end=datetime(2025, 1, 1, 9, 0, 0, tzinfo=timezone(timedelta(hours=8))),
scada_normal_start=datetime(2025, 1, 1, 7, 0, 0, tzinfo=timezone(timedelta(hours=8))),
scada_normal_end=datetime(2025, 1, 1, 8, 0, 0, tzinfo=timezone(timedelta(hours=8))),
) )
assert result["observed_source"] == "scada_burst_realtime_normal_timerange" assert result["observed_source"] == "scada_burst_scada_normal_timerange"
assert len(scada_calls) == 1 assert len(scada_calls) == 2
assert len(realtime_calls) == 1 assert len(realtime_calls) == 0
assert captured["burst_pressure"]["J1"] == pytest.approx(21.0) assert captured["burst_pressure"]["J1"] == pytest.approx(21.0)
assert captured["normal_pressure"]["J1"] == pytest.approx(11.0) assert captured["normal_pressure"]["J1"] == pytest.approx(11.0)
assert result["scada_window"] == {
"burst_start": "2025-01-01T00:00:00+00:00",
"burst_end": "2025-01-01T01:00:00+00:00",
"normal_start": "2024-12-31T23:00:00+00:00",
"normal_end": "2025-01-01T00:00:00+00:00",
}
def test_run_burst_location_monitoring_reuses_burst_window_for_normal(
monkeypatch, tmp_path
):
module = _load_burst_location_module()
captured = {}
scada_calls = []
monkeypatch.setattr(
module,
"get_all_scada_info",
lambda network: [
{
"type": "pressure",
"associated_element_id": "J1",
"api_query_id": "pressure-query",
}
],
)
monkeypatch.setattr(module, "_prepare_burst_inp", lambda network: str(tmp_path / "fake.inp"))
monkeypatch.setattr(
module,
"run_burst_location",
lambda **kwargs: captured.update(kwargs) or {"located_pipe": "Pipe-001"},
)
monkeypatch.setattr(
module.InternalQueries,
"query_scada_by_ids_timerange",
staticmethod(
lambda **kwargs: scada_calls.append(kwargs)
or {
"pressure-query": [
{"time": kwargs["start_time"], "value": 20.0},
{"time": kwargs["end_time"], "value": 22.0},
]
}
),
)
monkeypatch.setattr(
module.InternalQueries,
"query_realtime_simulation_by_ids_timerange",
staticmethod(lambda **kwargs: pytest.fail("monitoring mode must not query realtime simulation")),
)
result = module.run_burst_location_by_network(
network="tjwater",
username="testuser",
data_source="monitoring",
burst_leakage=1.0,
scada_burst_start=datetime(2025, 1, 1, 8, 0, 0, tzinfo=timezone(timedelta(hours=8))),
scada_burst_end=datetime(2025, 1, 1, 9, 0, 0, tzinfo=timezone(timedelta(hours=8))),
)
assert result["observed_source"] == "scada_timerange"
assert len(scada_calls) == 2
assert scada_calls[0]["start_time"] == scada_calls[1]["start_time"]
assert scada_calls[0]["end_time"] == scada_calls[1]["end_time"]
assert captured["burst_pressure"]["J1"] == pytest.approx(21.0)
assert captured["normal_pressure"]["J1"] == pytest.approx(21.0)
+12
View File
@@ -0,0 +1,12 @@
from app.infra.db.dynamic_manager import ProjectConnectionManager
def test_normalize_pg_url_preserves_password():
manager = ProjectConnectionManager()
url = manager._normalize_pg_url(
"postgresql://tjwater:secret@192.168.1.114:5433/tjwater"
)
assert url == "postgresql+psycopg://tjwater:secret@192.168.1.114:5433/tjwater"
assert "***" not in url
@@ -23,6 +23,11 @@ class _DummyEncryptor:
self._raise_invalid_token = raise_invalid_token self._raise_invalid_token = raise_invalid_token
self.encrypted_values = [] self.encrypted_values = []
def encrypt(self, value):
encrypted = f"encrypted::{value}"
self.encrypted_values.append(value)
return encrypted
def decrypt(self, _value): def decrypt(self, _value):
if self._raise_invalid_token: if self._raise_invalid_token:
raise InvalidToken() raise InvalidToken()
@@ -117,3 +122,46 @@ def test_encrypted_dsn_decrypts_without_migration(monkeypatch):
assert routing.dsn == "postgresql://u:p%40ss@host/db" assert routing.dsn == "postgresql://u:p%40ss@host/db"
session.commit.assert_not_awaited() session.commit.assert_not_awaited()
def test_upsert_project_database_config_encrypts_plaintext_dsn(monkeypatch):
project_id = uuid4()
session = SimpleNamespace(
execute=None,
add=None,
commit=None,
refresh=None,
)
added = []
session.execute = AsyncMock(return_value=_DummyResult(None))
session.add = lambda item: added.append(item)
session.commit = AsyncMock()
session.refresh = AsyncMock()
encryptor = _DummyEncryptor()
repo = MetadataRepository(session)
monkeypatch.setattr(
"app.infra.db.metadb.repositories.metadata_repository.is_database_encryption_configured",
lambda: True,
)
monkeypatch.setattr(
"app.infra.db.metadb.repositories.metadata_repository.get_database_encryptor",
lambda: encryptor,
)
record = asyncio.run(
repo.upsert_project_database_config(
project_id,
db_role="biz_data",
db_type="postgresql",
dsn="postgresql://user:secret@localhost/db",
pool_min_size=1,
pool_max_size=5,
)
)
assert encryptor.encrypted_values == ["postgresql://user:secret@localhost/db"]
assert record.dsn_encrypted == "encrypted::postgresql://user:secret@localhost/db"
assert added == [record]
session.commit.assert_awaited_once()
session.refresh.assert_awaited_once_with(record)
-56
View File
@@ -1,56 +0,0 @@
import asyncio
import pytest
from fastapi import HTTPException
from app.auth import permissions
from app.domain.models.role import UserRole
from tests.conftest import make_user
def test_require_role_allows_higher_privilege_user():
checker = permissions.require_role(UserRole.OPERATOR)
result = asyncio.run(checker(current_user=make_user(role=UserRole.ADMIN)))
assert result.role == UserRole.ADMIN
def test_require_role_rejects_insufficient_role():
checker = permissions.require_role(UserRole.ADMIN)
with pytest.raises(HTTPException) as exc_info:
asyncio.run(checker(current_user=make_user(role=UserRole.USER)))
assert exc_info.value.status_code == 403
assert "Required role: ADMIN" in exc_info.value.detail
def test_check_resource_owner_allows_admin():
assert permissions.check_resource_owner(
99,
make_user(id=1, role=UserRole.ADMIN),
) is True
def test_check_resource_owner_allows_owner():
assert permissions.check_resource_owner(
7,
make_user(id=7, role=UserRole.USER),
) is True
def test_check_resource_owner_rejects_other_user():
assert permissions.check_resource_owner(
7,
make_user(id=8, role=UserRole.USER),
) is False
def test_require_owner_or_admin_rejects_other_user():
checker = permissions.require_owner_or_admin(7)
with pytest.raises(HTTPException) as exc_info:
asyncio.run(checker(current_user=make_user(id=8, role=UserRole.USER)))
assert exc_info.value.status_code == 403
assert exc_info.value.detail == "You don't have permission to access this resource"
-124
View File
@@ -1,124 +0,0 @@
import asyncio
from unittest.mock import AsyncMock
import pytest
from app.domain.models.role import UserRole
from app.domain.schemas.user import UserCreate, UserUpdate
from app.infra.db.metadb.repositories.user_repository import UserRepository
from tests.conftest import FakeCursor, FakeDB
def _user_row(**overrides):
base = {
"id": 1,
"username": "tester",
"email": "tester@example.com",
"hashed_password": "hashed-password",
"role": "USER",
"is_active": True,
"is_superuser": False,
"created_at": "2025-01-01T00:00:00+00:00",
"updated_at": "2025-01-01T00:00:00+00:00",
}
base.update(overrides)
return base
def test_create_user_hashes_password_and_returns_model(monkeypatch):
cursor = FakeCursor(fetchone_results=[_user_row()])
repo = UserRepository(FakeDB(cursor))
monkeypatch.setattr(
"app.infra.db.metadb.repositories.user_repository.get_password_hash",
lambda password: f"hashed::{password}",
)
result = asyncio.run(
repo.create_user(
UserCreate(
username="tester",
email="tester@example.com",
password="secret123",
)
)
)
assert result is not None
assert result.username == "tester"
assert cursor.executed[0][1]["hashed_password"] == "hashed::secret123"
def test_update_user_without_fields_returns_existing_user(monkeypatch):
repo = UserRepository(FakeDB(FakeCursor()))
existing_user = AsyncMock(return_value="existing")
monkeypatch.setattr(repo, "get_user_by_id", existing_user)
result = asyncio.run(repo.update_user(1, UserUpdate()))
assert result == "existing"
existing_user.assert_awaited_once_with(1)
def test_update_user_builds_dynamic_query(monkeypatch):
cursor = FakeCursor(fetchone_results=[_user_row(role="ADMIN", email="new@example.com")])
repo = UserRepository(FakeDB(cursor))
monkeypatch.setattr(
"app.infra.db.metadb.repositories.user_repository.get_password_hash",
lambda password: f"hashed::{password}",
)
result = asyncio.run(
repo.update_user(
1,
UserUpdate(
email="new@example.com",
password="new-secret",
role=UserRole.ADMIN,
is_active=False,
),
),
)
assert result is not None
query, params = cursor.executed[0]
assert "email = %(email)s" in query
assert "hashed_password = %(hashed_password)s" in query
assert "role = %(role)s" in query
assert "is_active = %(is_active)s" in query
assert params["hashed_password"] == "hashed::new-secret"
assert params["role"] == "ADMIN"
assert params["is_active"] is False
def test_delete_user_returns_false_when_execute_raises():
cursor = FakeCursor()
cursor.execute = AsyncMock(side_effect=RuntimeError("boom"))
repo = UserRepository(FakeDB(cursor))
result = asyncio.run(repo.delete_user(1))
assert result is False
def test_user_exists_short_circuits_without_filters():
cursor = FakeCursor()
repo = UserRepository(FakeDB(cursor))
result = asyncio.run(repo.user_exists())
assert result is False
assert cursor.executed == []
def test_user_exists_checks_username_or_email():
cursor = FakeCursor(fetchone_results=[{"exists": True}])
repo = UserRepository(FakeDB(cursor))
result = asyncio.run(
repo.user_exists(username="tester", email="tester@example.com")
)
assert result is True
query, params = cursor.executed[0]
assert "username = %(username)s OR email = %(email)s" in query
assert params == {"username": "tester", "email": "tester@example.com"}