TJWaterServerBinary/.github/copilot-instructions.md

TJWater Server - Copilot Instructions

This is a FastAPI-based water network management system (供水管网智能管理系统) that provides hydraulic simulation, SCADA data integration, network element management, and risk analysis capabilities.

Running the Server

# Install dependencies
pip install -r requirements.txt

# Start the server (default: http://0.0.0.0:8000 with 2 workers)
python scripts/run_server.py

# Note: On Windows, the script automatically sets WindowsSelectorEventLoopPolicy

Running Tests

# Run all tests
pytest

# Run a specific test file with verbose output
pytest tests/unit/test_pipeline_health_analyzer.py -v

# Run from conftest helper
python tests/conftest.py

Architecture Overview

Core Components

1. Native Modules (app/native/): Platform-specific compiled extensions (.so for Linux, .pyd for Windows) providing performance-critical functionality including:

   • SCADA device integration
   • Water distribution analysis (WDA)
   • Pipe risk probability calculations
   • Wrapped through app.services.tjnetwork interface

2. Services Layer (app/services/):

   • tjnetwork.py: Main network API wrapper around native modules
   • simulation.py: Hydraulic simulation orchestration (EPANET integration)
   • project_info.py: Project configuration management
   • epanet/: EPANET hydraulic engine integration

3. API Layer (app/api/v1/):

   • Network Elements: Separate endpoint modules for junctions, reservoirs, tanks, pipes, pumps, valves
   • Components: Curves, patterns, controls, options, quality, visuals
   • Network Features: Tags, demands, geometry, regions/DMAs
   • Core Services: Auth, project, simulation, SCADA, data query, snapshots

4. Database Infrastructure (app/infra/db/):

   • PostgreSQL: Primary relational database (users, audit logs, project metadata)
   • TimescaleDB: Time-series extension for historical data
   • InfluxDB: Optional time-series database for high-frequency SCADA data
   • Connection pools initialized in main.py lifespan context
   • Database instance stored in app.state.db for dependency injection

5. Domain Layer (app/domain/):

   • models/: Enums and domain objects (e.g., UserRole)
   • schemas/: Pydantic models for request/response validation

6. Algorithms (app/algorithms/):

   • api_ex/: Analysis algorithms (k-means sensor placement, sensitivity analysis, pipeline health)
   • data_cleaning.py: Data preprocessing utilities
   • simulations.py: Simulation helpers

Security & Authentication

• Authentication: JWT-based with access tokens (30 min) and refresh tokens (7 days)
• Authorization: Role-based access control (RBAC) with 4 roles:
  • VIEWER: Read-only access
  • USER: Read-write access
  • OPERATOR: Modify data
  • ADMIN: Full permissions
• Audit Logging: AuditMiddleware automatically logs POST/PUT/DELETE requests
• Encryption: Fernet symmetric encryption for sensitive data (app.core.encryption)

Default admin accounts:

• admin / admin123
• tjwater / tjwater@123

Key Files

• app/main.py: FastAPI app initialization, lifespan (DB pools), CORS, middleware, router mounting
• app/api/v1/router.py: Central router aggregating all endpoint modules
• app/core/config.py: Settings management using pydantic-settings
• app/auth/dependencies.py: Auth dependencies (get_current_active_user, get_db)
• app/auth/permissions.py: Permission decorators (require_role, get_current_admin)
• .env: Environment configuration (database credentials, JWT secret, encryption key)

Important Conventions

Database Connections

• Database instances are initialized in main.py lifespan and stored in app.state.db
• Access via dependency injection:

  from app.auth.dependencies import get_db
  
  async def endpoint(db = Depends(get_db)):
      # Use db connection
  

Authentication in Endpoints

Use dependency injection for auth requirements:

from app.auth.dependencies import get_current_active_user
from app.auth.permissions import require_role, get_current_admin
from app.domain.models.role import UserRole

# Require any authenticated user
@router.get("/data")
async def get_data(current_user = Depends(get_current_active_user)):
    return data

# Require specific role (USER or higher)
@router.post("/data")
async def create_data(current_user = Depends(require_role(UserRole.USER))):
    return result

# Admin-only access
@router.delete("/data/{id}")
async def delete_data(id: int, current_user = Depends(get_current_admin)):
    return result

API Routing Structure

• All v1 APIs are mounted under /api/v1 prefix via api_router
• Legacy routes without version prefix are also mounted for backward compatibility
• Group related endpoints in separate router modules under app/api/v1/endpoints/
• Use descriptive tags in router.py for OpenAPI documentation grouping

Native Module Integration

• Native modules are pre-compiled for specific platforms
• Always import through app.native.api or app.services.tjnetwork
• The tjnetwork service wraps native APIs with constants like:
  • Element types: JUNCTION, RESERVOIR, TANK, PIPE, PUMP, VALVE
  • Operations: API_ADD, API_UPDATE, API_DELETE
  • ChangeSet for batch operations

Project Initialization

• On startup, main.py automatically loads project from project_info.name if set
• Projects are opened via open_project(name) from tjnetwork service

Audit Logging

Manual audit logging for critical operations:

from app.core.audit import log_audit_event, AuditAction

await log_audit_event(
    action=AuditAction.UPDATE,
    user_id=current_user.id,
    username=current_user.username,
    resource_type="resource_name",
    resource_id=str(resource_id),
    ip_address=request.client.host,
    request_data=data
)

Environment Configuration

• Copy .env.example to .env before first run
• Required environment variables:
  • SECRET_KEY: JWT signing (generate with openssl rand -hex 32)
  • ENCRYPTION_KEY: Data encryption (generate with Fernet)
  • Database credentials for PostgreSQL, TimescaleDB, and optionally InfluxDB

Database Migrations

SQL migration scripts are in migrations/:

• 001_create_users_table.sql: User authentication tables
• 002_create_audit_logs_table.sql: Audit logging tables

Apply with:

psql -U postgres -d tjwater -f migrations/001_create_users_table.sql

API Documentation

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc
• OpenAPI schema: http://localhost:8000/openapi.json

Additional Resources

• SECURITY_README.md: Comprehensive security feature documentation
• DEPLOYMENT.md: Integration guide for security features
• readme.md: Project overview and directory structure (in Chinese)