删除、移动旧文档
This commit is contained in:
+61
-185
@@ -1,206 +1,82 @@
|
||||
# TJWater Server - Copilot Instructions
|
||||
# Copilot Instructions for TJWater Server
|
||||
|
||||
This is a FastAPI-based water network management system (供水管网智能管理系统) that provides hydraulic simulation, SCADA data integration, network element management, and risk analysis capabilities.
|
||||
This repository contains the backend code for the TJWater Server, a water distribution network management system built with FastAPI.
|
||||
|
||||
## Running the Server
|
||||
## High-Level Architecture
|
||||
|
||||
The application follows a layered architecture:
|
||||
|
||||
- **Entry Point**: `app/main.py` initializes the FastAPI application, database connections (PostgreSQL & TimescaleDB), and middleware.
|
||||
- **API Layer**: `app/api/v1` contains the route handlers.
|
||||
- **Service Layer**: `app/services` contains business logic and orchestration.
|
||||
- **Infrastructure Layer**: `app/infra` handles database connections (`db`), audit logging (`audit`), and external integrations.
|
||||
- **Domain Layer**: `app/domain` likely contains core domain models.
|
||||
- **Native/Algorithms**: `app/native` and `app/algorithms` handle specialized water network calculations (possibly using EPANET/WNTR).
|
||||
|
||||
## Build, Test, and Run Commands
|
||||
|
||||
### Environment Setup
|
||||
|
||||
- Dependencies are listed in `requirements.txt`.
|
||||
- Configuration is managed via environment variables (see `.env.example` if available, or `app/core/config.py`).
|
||||
- **Important**: Ensure `.env` is configured with correct database credentials for both PostgreSQL and TimescaleDB.
|
||||
|
||||
If first time setting up, you may want to create a Conda environment:
|
||||
|
||||
```bash
|
||||
# activate the server environment
|
||||
conda create -n server python=3.12
|
||||
conda activate server
|
||||
# Install dependencies
|
||||
<!-- pip install -r requirements.txt -->
|
||||
|
||||
# Start the server (default: http://0.0.0.0:8000 with 2 workers)
|
||||
python scripts/run_server.py
|
||||
|
||||
# Note: On Windows, the script automatically sets WindowsSelectorEventLoopPolicy
|
||||
pip install uv
|
||||
uv pip install -r requirements.txt
|
||||
conda install -c conda-forge pymetis
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
### Running the Server
|
||||
|
||||
The preferred way to run the server locally is using the helper script which sets up the Python path correctly:
|
||||
|
||||
```bash
|
||||
# activate the server environment
|
||||
conda activate server
|
||||
python scripts/run_server.py
|
||||
```
|
||||
|
||||
Alternatively, you can run directly with uvicorn (ensure PYTHONPATH includes the root):
|
||||
|
||||
```bash
|
||||
conda activate server
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
Use `pytest` to run tests. The `tests/conftest.py` handles path setup.
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
pytest
|
||||
|
||||
# Run a specific test file with verbose output
|
||||
pytest tests/unit/test_pipeline_health_analyzer.py -v
|
||||
# Run a specific test file
|
||||
pytest tests/unit/test_specific_file.py
|
||||
|
||||
# Run from conftest helper
|
||||
python tests/conftest.py
|
||||
# Run a specific test case
|
||||
pytest tests/unit/test_specific_file.py::test_function_name
|
||||
```
|
||||
|
||||
## Architecture Overview
|
||||
### Building (Optional)
|
||||
|
||||
### Core Components
|
||||
The project includes scripts to compile Python modules to `.pyd` files using Cython (see `scripts/build_pyd.py`). This is likely for distribution/performance but not required for standard development.
|
||||
|
||||
1. **Native Modules** (`app/native/`): Platform-specific compiled extensions (`.so` for Linux, `.pyd` for Windows) providing performance-critical functionality including:
|
||||
- SCADA device integration
|
||||
- Water distribution analysis (WDA)
|
||||
- Pipe risk probability calculations
|
||||
- Wrapped through `app.services.tjnetwork` interface
|
||||
## Key Conventions
|
||||
|
||||
2. **Services Layer** (`app/services/`):
|
||||
- `tjnetwork.py`: Main network API wrapper around native modules
|
||||
- `simulation.py`: Hydraulic simulation orchestration (EPANET integration)
|
||||
- `project_info.py`: Project configuration management
|
||||
- `epanet/`: EPANET hydraulic engine integration
|
||||
- **Async/Await**: The codebase heavily uses `async` and `await` for I/O operations, especially database interactions.
|
||||
- **Database Management**:
|
||||
- Connections are managed globally in `app.infra.db` and initialized in `lifespan` (app/main.py).
|
||||
- Use `app.infra.db.dynamic_manager` for project-specific database connections (multi-tenancy/dynamic projects).
|
||||
- **Pydantic**: extensively used for data validation and settings management.
|
||||
- **Scripts**: The `scripts/` directory contains many utility scripts for maintenance, data processing, and server management. Check there before writing new operational scripts.
|
||||
- **Water Network Modeling**: Interactions with water network models often involve `epanet` or `wntr` libraries. Be aware of domain-specific terminology (nodes, links, junctions, tanks).
|
||||
|
||||
3. **API Layer** (`app/api/v1/`):
|
||||
- **Network Elements**: Separate endpoint modules for junctions, reservoirs, tanks, pipes, pumps, valves
|
||||
- **Components**: Curves, patterns, controls, options, quality, visuals
|
||||
- **Network Features**: Tags, demands, geometry, regions/DMAs
|
||||
- **Core Services**: Auth, project, simulation, SCADA, data query, snapshots
|
||||
## Code Style
|
||||
|
||||
4. **Database Infrastructure** (`app/infra/db/`):
|
||||
- **PostgreSQL**: Primary relational database (users, audit logs, project metadata)
|
||||
- **TimescaleDB**: Time-series extension for historical data
|
||||
- **InfluxDB**: Optional time-series database for high-frequency SCADA data
|
||||
- Connection pools initialized in `main.py` lifespan context
|
||||
- Database instance stored in `app.state.db` for dependency injection
|
||||
|
||||
5. **Domain Layer** (`app/domain/`):
|
||||
- `models/`: Enums and domain objects (e.g., `UserRole`)
|
||||
- `schemas/`: Pydantic models for request/response validation
|
||||
|
||||
6. **Algorithms** (`app/algorithms/`):
|
||||
- `api_ex/`: Analysis algorithms (k-means sensor placement, sensitivity analysis, pipeline health)
|
||||
- `data_cleaning.py`: Data preprocessing utilities
|
||||
- `simulations.py`: Simulation helpers
|
||||
|
||||
### Security & Authentication
|
||||
|
||||
- **Authentication**: JWT-based with access tokens (30 min) and refresh tokens (7 days)
|
||||
- **Authorization**: Role-based access control (RBAC) with 4 roles:
|
||||
- `VIEWER`: Read-only access
|
||||
- `USER`: Read-write access
|
||||
- `OPERATOR`: Modify data
|
||||
- `ADMIN`: Full permissions
|
||||
- **Audit Logging**: `AuditMiddleware` automatically logs POST/PUT/DELETE requests
|
||||
- **Encryption**: Fernet symmetric encryption for sensitive data (`app.core.encryption`)
|
||||
|
||||
Default admin accounts:
|
||||
|
||||
- `admin` / `admin123`
|
||||
- `tjwater` / `tjwater@123`
|
||||
|
||||
### Key Files
|
||||
|
||||
- `app/main.py`: FastAPI app initialization, lifespan (DB pools), CORS, middleware, router mounting
|
||||
- `app/api/v1/router.py`: Central router aggregating all endpoint modules
|
||||
- `app/core/config.py`: Settings management using `pydantic-settings`
|
||||
- `app/auth/dependencies.py`: Auth dependencies (`get_current_active_user`, `get_db`)
|
||||
- `app/auth/permissions.py`: Permission decorators (`require_role`, `get_current_admin`)
|
||||
- `.env`: Environment configuration (database credentials, JWT secret, encryption key)
|
||||
|
||||
## Important Conventions
|
||||
|
||||
### Database Connections
|
||||
|
||||
- Database instances are initialized in `main.py` lifespan and stored in `app.state.db`
|
||||
- Access via dependency injection:
|
||||
|
||||
```python
|
||||
from app.auth.dependencies import get_db
|
||||
|
||||
async def endpoint(db = Depends(get_db)):
|
||||
# Use db connection
|
||||
```
|
||||
|
||||
### Authentication in Endpoints
|
||||
|
||||
Use dependency injection for auth requirements:
|
||||
|
||||
```python
|
||||
from app.auth.dependencies import get_current_active_user
|
||||
from app.auth.permissions import require_role, get_current_admin
|
||||
from app.domain.models.role import UserRole
|
||||
|
||||
# Require any authenticated user
|
||||
@router.get("/data")
|
||||
async def get_data(current_user = Depends(get_current_active_user)):
|
||||
return data
|
||||
|
||||
# Require specific role (USER or higher)
|
||||
@router.post("/data")
|
||||
async def create_data(current_user = Depends(require_role(UserRole.USER))):
|
||||
return result
|
||||
|
||||
# Admin-only access
|
||||
@router.delete("/data/{id}")
|
||||
async def delete_data(id: int, current_user = Depends(get_current_admin)):
|
||||
return result
|
||||
```
|
||||
|
||||
### API Routing Structure
|
||||
|
||||
- All v1 APIs are mounted under `/api/v1` prefix via `api_router`
|
||||
- Legacy routes without version prefix are also mounted for backward compatibility
|
||||
- Group related endpoints in separate router modules under `app/api/v1/endpoints/`
|
||||
- Use descriptive tags in `router.py` for OpenAPI documentation grouping
|
||||
|
||||
### Native Module Integration
|
||||
|
||||
- Native modules are pre-compiled for specific platforms
|
||||
- Always import through `app.native.api` or `app.services.tjnetwork`
|
||||
- The `tjnetwork` service wraps native APIs with constants like:
|
||||
- Element types: `JUNCTION`, `RESERVOIR`, `TANK`, `PIPE`, `PUMP`, `VALVE`
|
||||
- Operations: `API_ADD`, `API_UPDATE`, `API_DELETE`
|
||||
- `ChangeSet` for batch operations
|
||||
|
||||
### Project Initialization
|
||||
|
||||
- On startup, `main.py` automatically loads project from `project_info.name` if set
|
||||
- Projects are opened via `open_project(name)` from `tjnetwork` service
|
||||
|
||||
### Audit Logging
|
||||
|
||||
Manual audit logging for critical operations:
|
||||
|
||||
```python
|
||||
from app.core.audit import log_audit_event, AuditAction
|
||||
|
||||
await log_audit_event(
|
||||
action=AuditAction.UPDATE,
|
||||
user_id=current_user.id,
|
||||
username=current_user.username,
|
||||
resource_type="resource_name",
|
||||
resource_id=str(resource_id),
|
||||
ip_address=request.client.host,
|
||||
request_data=data
|
||||
)
|
||||
```
|
||||
|
||||
### Environment Configuration
|
||||
|
||||
- Copy `.env.example` to `.env` before first run
|
||||
- Required environment variables:
|
||||
- `SECRET_KEY`: JWT signing (generate with `openssl rand -hex 32`)
|
||||
- `ENCRYPTION_KEY`: Data encryption (generate with Fernet)
|
||||
- Database credentials for PostgreSQL, TimescaleDB, and optionally InfluxDB
|
||||
|
||||
### Database Migrations
|
||||
|
||||
SQL migration scripts are in `migrations/`:
|
||||
|
||||
- `001_create_users_table.sql`: User authentication tables
|
||||
- `002_create_audit_logs_table.sql`: Audit logging tables
|
||||
|
||||
Apply with:
|
||||
|
||||
```bash
|
||||
psql -U postgres -d tjwater -f migrations/001_create_users_table.sql
|
||||
```
|
||||
|
||||
## API Documentation
|
||||
|
||||
- Swagger UI: http://localhost:8000/docs
|
||||
- ReDoc: http://localhost:8000/redoc
|
||||
- OpenAPI schema: http://localhost:8000/openapi.json
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- `SECURITY_README.md`: Comprehensive security feature documentation
|
||||
- `DEPLOYMENT.md`: Integration guide for security features
|
||||
- `readme.md`: Project overview and directory structure (in Chinese)
|
||||
- Follow standard PEP 8 guidelines.
|
||||
- No specific linter configuration was found, so default to standard Python formatting.
|
||||
|
||||
Reference in New Issue
Block a user