# TJWater ServerBinary - Secondary Development Documentation Guide

## Executive Summary

TJWaterServerBinary is a **FastAPI-based water distribution network management system** with integrated EPANET simulation, SCADA data management, and comprehensive security architecture. This guide identifies key components and patterns essential for secondary development.

---

## 1. Project Structure & Key Directories

### Overall Architecture
```
TJWaterServerBinary/
├── app/
│   ├── main.py                 # FastAPI application entry + lifecycle management
│   ├── api/v1/                 # API routes and endpoints
│   ├── algorithms/             # Core algorithms (simulation, leakage detection, etc.)
│   ├── auth/                   # Authentication logic and dependencies
│   ├── core/                   # Core configuration, security, encryption
│   ├── domain/                 # Domain models and Pydantic schemas
│   ├── infra/                  # Infrastructure layer (DB, cache, audit)
│   ├── services/               # Business logic services
│   ├── native/                 # Native module integration (binary/compiled code)
│   └── utils/                  # Utility functions
├── infra/docker/               # Docker and deployment configs
├── resources/sql/              # SQL initialization scripts
├── tests/                       # Unit and integration tests
└── requirements.txt            # Python dependencies
```

### Key Directory Details

| Directory | Purpose | Key Files |
|-----------|---------|-----------|
| **api/v1** | REST API endpoints organized by domain | `router.py` (centralized registration), `endpoints/` (individual controllers) |
| **domain** | Data models and schemas | `models/role.py`, `schemas/user.py`, `schemas/audit.py` |
| **infra** | Database access, caching, audit | `db/postgresql/`, `db/timescaledb/`, `cache/`, `audit/` |
| **auth** | JWT/OAuth2 validation, permissions | `dependencies.py`, `permissions.py`, `keycloak_dependencies.py` |
| **core** | Security, encryption, config | `config.py`, `security.py`, `encryption.py`, `audit.py` |
| **native** | Python-to-binary integration | `api/` module wrapping compiled code |
| **services** | Business logic & orchestration | `tjnetwork.py` (project management), `simulation.py`, etc. |

---

## 2. API Endpoint Addition (Router Registration)

### How to Add New API Endpoints

#### Step 1: Create Endpoint File
Create a new file in `app/api/v1/endpoints/` with your endpoint handlers:

```python
# app/api/v1/endpoints/my_feature.py
from fastapi import APIRouter, Depends, HTTPException, status
from app.auth.dependencies import get_current_active_user
from app.domain.schemas.user import UserInDB
from app.infra.repositories.my_repository import MyRepository
from app.auth.dependencies import get_user_repository

router = APIRouter()

@router.get("/my-feature/")
async def get_my_feature(user: UserInDB = Depends(get_current_active_user)):
    """Get feature data"""
    return {"feature": "data"}

@router.post("/my-feature/")
async def create_my_feature(
    data: dict,
    user: UserInDB = Depends(get_current_active_user),
):
    """Create new feature"""
    return {"status": "created"}
```

#### Step 2: Register Router in Central Router
Edit `app/api/v1/router.py` to include your new router:

```python
# app/api/v1/router.py
from app.api.v1.endpoints import my_feature  # Import your endpoint module

api_router = APIRouter()

# Add your router
api_router.include_router(
    my_feature.router, 
    prefix="/my-feature",          # Optional: URL prefix
    tags=["My Feature"]             # For OpenAPI documentation
)
```

#### Step 3: Authentication & Authorization
- **No Auth Required**: Endpoint is public
- **Authenticated Users**: Use `Depends(get_current_active_user)`
- **Role-Based Access**: Use `Depends(require_admin)` or `Depends(require_role(UserRole.OPERATOR))`

```python
from app.auth.permissions import require_admin, require_role
from app.domain.models.role import UserRole

@router.post("/admin-only/")
async def admin_only(user: UserInDB = Depends(require_admin)):
    return {"admin": user.username}
```

#### Step 4: Response Models
Use Pydantic schemas for validation:

```python
from pydantic import BaseModel
from typing import Optional

class MyFeatureResponse(BaseModel):
    id: int
    name: str
    status: Optional[str] = None

@router.get("/my-feature/{id}", response_model=MyFeatureResponse)
async def get_feature(id: int):
    return MyFeatureResponse(id=id, name="Feature", status="active")
```

### Router Registration Pattern
The central router (`app/api/v1/router.py`) uses `include_router()` to compose all endpoints:

```python
# Current structure (98 lines):
api_router.include_router(auth.router, prefix="/auth", tags=["Auth"])
api_router.include_router(project.router, tags=["Project"])
api_router.include_router(simulation.router, tags=["Simulation Control"])
# ... 30+ more routers
```

---

## 3. Database Interaction Patterns

### ORM & Database Strategy
**TJWater uses psycopg (PostgreSQL async driver) directly, NOT SQLAlchemy ORM**

#### Multiple Database Types
- **PostgreSQL**: Main relational data (users, metadata)
- **TimescaleDB**: Time-series data (SCADA, sensor readings)
- **InfluxDB**: Alternative time-series storage (optional)
- **Metadata DB**: Separate instance (`system_hub` database)

### Database Layer Architecture

#### 1. Database Instances (`app/infra/db/postgresql/database.py`)

```python
class Database:
    """Manages async connection pooling"""
    
    def __init__(self, db_name=None):
        self.pool = None
        self.db_name = db_name
    
    def init_pool(self, db_name=None):
        """Initialize connection pool"""
        self.pool = psycopg_pool.AsyncConnectionPool(...)
    
    async def get_connection(self) -> AsyncGenerator:
        """Yield connection from pool"""
        async with self.pool.connection() as conn:
            yield conn

# Global instances
db = Database()  # Default PostgreSQL
_database_instances = {}  # Per-project caches
```

#### 2. Repository Pattern (`app/infra/repositories/`)

```python
class UserRepository:
    """Data access layer for users"""
    
    def __init__(self, db: Database):
        self.db = db
    
    async def create_user(self, user: UserCreate) -> Optional[UserInDB]:
        query = """
            INSERT INTO users (username, email, hashed_password, role)
            VALUES (%(username)s, %(email)s, %(hashed_password)s, %(role)s)
            RETURNING id, username, email, role, created_at
        """
        async with self.db.get_connection() as conn:
            async with conn.cursor() as cur:
                await cur.execute(query, {
                    'username': user.username,
                    'email': user.email,
                    'hashed_password': get_password_hash(user.password),
                    'role': user.role.value
                })
                row = await cur.fetchone()
                return UserInDB(**row) if row else None
```

#### 3. Query Execution Pattern

```python
# Reading data
async with db.get_connection() as conn:
    async with conn.cursor() as cur:
        await cur.execute("SELECT * FROM users WHERE id = %s", (user_id,))
        row = await cur.fetchone()  # Returns dict_row

# Writing data
async with db.get_connection() as conn:
    async with conn.cursor() as cur:
        await cur.execute(
            "UPDATE users SET email = %s WHERE id = %s",
            (new_email, user_id)
        )
        # conn auto-commits (if autocommit=True in pool config)
```

### Database Configuration

Via `app/core/config.py` and `.env`:

```python
# Main PostgreSQL (users, metadata)
DB_HOST = "localhost"
DB_PORT = "5432"
DB_NAME = "tjwater"
DB_USER = "postgres"
DB_PASSWORD = "password"

# TimescaleDB (time-series)
TIMESCALEDB_DB_HOST = "localhost"
TIMESCALEDB_DB_PORT = "5433"
TIMESCALEDB_DB_NAME = "szh"

# Metadata DB (system info)
METADATA_DB_NAME = "system_hub"
```

### Dynamic Multi-Database Support

Project-specific databases are managed dynamically:

```python
# app/infra/db/dynamic_manager.py
async def get_database_instance(db_name: Optional[str] = None) -> Database:
    """Get or create DB instance for project"""
    if not db_name:
        return db  # Default instance
    
    if db_name not in _database_instances:
        instance = Database(db_name=db_name)
        instance.init_pool()
        await instance.open()
        _database_instances[db_name] = instance
    
    return _database_instances[db_name]
```

### Migrations & Schema Creation

- **Location**: `resources/sql/` contains initialization scripts
- **Naming**: Numbered order (e.g., `002_create_audit_logs_table.sql`)
- **Approach**: SQL scripts are manually executed; no Alembic/Flyway framework
- **Example**: `resources/sql/002_create_audit_logs_table.sql`

---

## 4. Native Module Integration

### Architecture: Python ↔ Binary Code

TJWater integrates compiled C/Go binaries for water network simulation:

```
Python (FastAPI)
    ↓
app/native/wndb/
    ├── project.py              (list, create, delete, open projects)
    ├── s2_junctions.py         (CRUD for junctions)
    ├── s5_pipes.py             (CRUD for pipes)
    └── [40+ schema modules]    (network elements)
    ↓
app/native/api_encap/          (Low-level binary wrappers)
    └── api.so / api.dll        (Compiled binary library)
```

### How Python Calls Native Code

#### 1. Native API Initialization
```python
# app/native/wndb/__init__.py
from app.native.api_encap.api import (
    ChangeSet,  # Change tracking
    API_ADD, API_UPDATE, API_DELETE,  # Operations
    list_project, create_project, open_project, close_project,
    # ... 100+ functions
)
```

#### 2. Example: Project Management
```python
# app/services/tjnetwork.py
from app.native.wndb import list_project, open_project, close_project

def list_project() -> list[str]:
    """List all project databases"""
    # Calls into compiled binary to enumerate DB names
    return api.list_project()

def open_project(name: str) -> None:
    """Open project and load into memory"""
    # Binary loads .inp file or DB data
    api.open_project(name)
```

#### 3. ChangeSet Tracking
```python
# For tracking network modifications
class ChangeSet:
    """Tracks ADD/UPDATE/DELETE operations on network elements"""
    def __init__(self):
        self.adds = []
        self.updates = []
        self.deletes = []

# Usage in services
def update_junction(name: str, demand: float):
    change = api.set_junction(name, {"demand": demand})
    # Returns ChangeSet for auditing/undo-redo
```

#### 4. Network CRUD Operations
```python
# All operations follow this pattern:
from app.native.wndb import get_all_junctions, add_junction, set_junction

# Read
junctions = get_all_junctions()  # Returns list of dicts

# Create
add_junction({"id": "J1", "x": 0.0, "y": 0.0, "demand": 100.0})

# Update
set_junction("J1", {"demand": 150.0})

# Delete
delete_junction("J1")
```

### Data Flow Example: Simulation
```
FastAPI Request (POST /simulation/run)
    ↓
app/api/v1/endpoints/simulation.py
    ↓
app/services/simulation.py (business logic)
    ↓
app/native/wndb/ (calls binary functions)
    ├── open_project("szh")
    ├── set_option("QUALITY", "CHEMICAL")
    ├── run_simulation()
    └── get_result()
    ↓
TimescaleDB/InfluxDB (store results)
    ↓
Response to client
```

### Python Module Constants
Native module exports constants for enumerations:

```python
from app.native.wndb import (
    OPTION_UNITS_LPS,          # Flow units
    OPTION_PRESSURE_KPA,       # Pressure units
    OPTION_QUALITY_CHEMICAL,   # Quality type
    PIPE_STATUS_OPEN,          # Pipe status
    SCADA_DEVICE_TYPE_PRESSURE,
)
```

---

## 5. Authentication & Authorization Mechanisms

### Architecture: JWT + OAuth2 + Role-Based Access Control (RBAC)

### 1. Authentication Flow

#### User Registration
```python
# app/api/v1/endpoints/auth.py
@router.post("/register", response_model=UserResponse)
async def register(user_data: UserCreate):
    # 1. Hash password
    hashed = get_password_hash(user_data.password)
    
    # 2. Store in DB via repository
    user = await user_repo.create_user({
        'username': user_data.username,
        'email': user_data.email,
        'hashed_password': hashed,
        'role': 'USER'
    })
    return UserResponse.model_validate(user)
```

#### User Login & Token Generation
```python
@router.post("/login", response_model=Token)
async def login(form_data: OAuth2PasswordRequestForm = Depends()):
    # 1. Verify credentials
    user = await user_repo.get_user_by_username(form_data.username)
    if not verify_password(form_data.password, user.hashed_password):
        raise HTTPException(status_code=401, detail="Invalid credentials")
    
    # 2. Generate JWT tokens
    access_token = create_access_token(subject=user.username)
    refresh_token = create_refresh_token(subject=user.username)
    
    return Token(
        access_token=access_token,
        refresh_token=refresh_token,
        token_type="bearer",
        expires_in=settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60
    )
```

### 2. JWT Token Configuration
```python
# app/core/config.py
SECRET_KEY: str = "your-secret-key"  # Change in production
ALGORITHM: str = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES: int = 30
REFRESH_TOKEN_EXPIRE_DAYS: int = 7
```

### 3. Token Validation & Current User
```python
# app/auth/dependencies.py
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")

async def get_current_user(token: str = Depends(oauth2_scheme)) -> UserInDB:
    """Validate JWT token and extract user"""
    try:
        payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
        username: str = payload.get("sub")
        token_type: str = payload.get("type", "access")
        
        if username is None or token_type != "access":
            raise HTTPException(status_code=401, detail="Invalid token")
        
        # Fetch user from DB
        user = await user_repo.get_user_by_username(username)
        if user is None:
            raise HTTPException(status_code=401, detail="User not found")
        
        return user
    except JWTError:
        raise HTTPException(status_code=401, detail="Invalid token")
```

### 4. Role-Based Access Control (RBAC)

#### User Roles Enum
```python
# app/domain/models/role.py
class UserRole(str, Enum):
    ADMIN = "ADMIN"          # Full permissions
    OPERATOR = "OPERATOR"    # Modify data
    USER = "USER"            # Read/write
    VIEWER = "VIEWER"        # Read-only

# Hierarchy
get_hierarchy() = {
    VIEWER: 1,
    USER: 2,
    OPERATOR: 3,
    ADMIN: 4,
}
```

#### Permission Decorators
```python
# app/auth/permissions.py

def require_admin(current_user: UserInDB = Depends(get_current_active_user)) -> UserInDB:
    """Restrict to admin users"""
    if current_user.role != UserRole.ADMIN:
        raise HTTPException(status_code=403, detail="Admin access required")
    return current_user

def require_role(role: UserRole):
    """Factory for role-based permission checks"""
    async def permission_checker(
        current_user: UserInDB = Depends(get_current_active_user)
    ) -> UserInDB:
        if not UserRole(current_user.role).has_permission(role):
            raise HTTPException(status_code=403, detail=f"Role {role} required")
        return current_user
    return permission_checker

def require_operator(current_user: UserInDB = Depends(get_current_active_user)) -> UserInDB:
    return current_user if require_role(UserRole.OPERATOR) else None
```

#### Using Permissions in Endpoints
```python
# Require admin
@router.delete("/users/{user_id}")
async def delete_user(
    user_id: int,
    admin: UserInDB = Depends(require_admin)
):
    await user_repo.delete_user(user_id)
    return {"status": "deleted"}

# Require operator or higher
@router.post("/projects/")
async def create_project(
    data: dict,
    operator: UserInDB = Depends(require_operator)
):
    return {"project": "created"}
```

### 5. Optional Keycloak Integration
```python
# app/auth/keycloak_dependencies.py
# Alternative: Validate tokens from external Keycloak server

KEYCLOAK_PUBLIC_KEY: str = ""  # From .env
KEYCLOAK_ALGORITHM: str = "RS256"

# Similar JWT validation but with Keycloak's public key
```

### 6. Token Schema
```python
# app/domain/schemas/user.py
class Token(BaseModel):
    access_token: str
    refresh_token: Optional[str] = None
    token_type: str = "bearer"
    expires_in: int  # Seconds

class UserInDB(BaseModel):
    id: int
    username: str
    email: str
    role: str  # "ADMIN", "USER", etc.
    is_active: bool
    is_superuser: bool
```

---

## 6. Configuration Management

### Configuration Hierarchy
```
Environment Variables (.env file)
    ↓
app/core/config.py (Pydantic Settings)
    ↓
Injected via Depends() in endpoints/services
    ↓
Runtime usage
```

### Config File: `app/core/config.py`
Uses Pydantic v2 `BaseSettings` with `.env` support:

```python
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    # App
    PROJECT_NAME: str = "TJWater Server"
    API_V1_STR: str = "/api/v1"
    
    # Security
    SECRET_KEY: str = "your-secret"
    ALGORITHM: str = "HS256"
    ACCESS_TOKEN_EXPIRE_MINUTES: int = 30
    
    # Encryption (Fernet symmetric)
    ENCRYPTION_KEY: str = ""  # Required from .env
    DATABASE_ENCRYPTION_KEY: str = ""
    
    # PostgreSQL
    DB_NAME: str = "tjwater"
    DB_HOST: str = "localhost"
    DB_PORT: str = "5432"
    DB_USER: str = "postgres"
    DB_PASSWORD: str = "password"
    
    # TimescaleDB
    TIMESCALEDB_DB_NAME: str = "tjwater"
    TIMESCALEDB_DB_HOST: str = "localhost"
    TIMESCALEDB_DB_PORT: str = "5433"
    
    # Metadata DB
    METADATA_DB_NAME: str = "system_hub"
    METADATA_DB_HOST: str = "localhost"
    METADATA_DB_PORT: str = "5432"
    
    # Cache sizing
    PROJECT_PG_CACHE_SIZE: int = 50
    PROJECT_TS_CACHE_SIZE: int = 50
    
    # Connection pooling
    PROJECT_PG_POOL_SIZE: int = 5
    PROJECT_TS_POOL_MIN_SIZE: int = 1
    PROJECT_TS_POOL_MAX_SIZE: int = 10
    
    # InfluxDB (optional)
    INFLUXDB_URL: str = "http://localhost:8086"
    INFLUXDB_TOKEN: str = "token"
    
    @property
    def SQLALCHEMY_DATABASE_URI(self) -> str:
        return f"postgresql://{self.DB_USER}:{self.DB_PASSWORD}@{self.DB_HOST}:{self.DB_PORT}/{self.DB_NAME}"
    
    model_config = SettingsConfigDict(
        env_file=".env",
        extra="ignore",
    )

settings = Settings()
```

### Environment File: `.env.example`
```bash
# Security
SECRET_KEY=your-secret-key-change-in-production
ENCRYPTION_KEY=<generated via Fernet.generate_key()>
DATABASE_ENCRYPTION_KEY=

# PostgreSQL
DB_HOST=localhost
DB_PORT=5432
DB_NAME=tjwater
DB_USER=postgres
DB_PASSWORD=password

# TimescaleDB
TIMESCALEDB_DB_NAME=szh
TIMESCALEDB_DB_HOST=localhost
TIMESCALEDB_DB_PORT=5433
TIMESCALEDB_DB_USER=tjwater
TIMESCALEDB_DB_PASSWORD=Tjwater@123456

# Metadata DB
METADATA_DB_NAME=system_hub
METADATA_DB_HOST=localhost

# Cache/Pool
PROJECT_PG_CACHE_SIZE=50
PROJECT_TS_CACHE_SIZE=50
```

### Using Configuration in Code
```python
from app.core.config import settings

# Direct access
db_url = settings.SQLALCHEMY_DATABASE_URI
secret = settings.SECRET_KEY

# In FastAPI lifespan
@asynccontextmanager
async def lifespan(app: FastAPI):
    # Initialize based on settings
    db.init_pool(settings.DB_NAME)
    yield
    await db.close()
```

### Encryption Configuration
```python
# app/core/encryption.py
from app.core.config import settings

class Encryptor:
    def __init__(self, key: Optional[bytes] = None):
        if key is None:
            key_str = settings.ENCRYPTION_KEY
            if not key_str:
                raise ValueError("ENCRYPTION_KEY not configured")
            key = key_str.encode()
        self.fernet = Fernet(key)
    
    def encrypt(self, data: str) -> str:
        return self.fernet.encrypt(data.encode()).decode()
    
    def decrypt(self, data: str) -> str:
        return self.fernet.decrypt(data.encode()).decode()
    
    @staticmethod
    def generate_key() -> str:
        """Generate new encryption key"""
        return Fernet.generate_key().decode()
```

---

## 7. Testing Setup

### Test Structure
```
tests/
├── conftest.py              # Pytest configuration & fixtures
├── api/                     # API integration tests
│   ├── test_api_integration.py
│   └── test_leakage_endpoints.py
├── unit/                    # Unit tests
│   ├── test_burst_location_service.py
│   ├── test_metadata_repository_dsn_decrypt.py
│   └── test_pipeline_health_analyzer.py
└── auth/                    # Auth tests
    └── test_encryption.py
```

### Configuration: `tests/conftest.py`
```python
import pytest
import sys
import os

# Add project root to path
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))

def run_this_test(test_file):
    """Run a single test file"""
    test_name = os.path.splitext(os.path.basename(test_file))[0]
    pytest.main([test_file, "-v"])
```

### Testing Tools
```python
# requirements.txt includes:
pytest==8.3.5                    # Test runner
# httpx (via FastAPI) for async HTTP client
```

### Example: API Integration Test
```python
# tests/api/test_api_integration.py
import pytest

@pytest.mark.parametrize(
    "module_name, desc",
    [
        ("app.core.encryption", "Encryption"),
        ("app.auth.permissions", "Permissions"),
        ("app.api.v1.endpoints.auth", "Auth endpoints"),
    ],
)
def test_module_imports(module_name, desc):
    """Verify critical modules can be imported"""
    try:
        __import__(module_name)
    except ImportError as e:
        pytest.fail(f"Cannot import {desc}: {e}")

def test_router_configuration():
    """Verify router is properly configured"""
    from app.api.v1 import router
    api_router = router.api_router
    routes = [r.path for r in api_router.routes if hasattr(r, "path")]
    assert len(routes) > 0
```

### Example: Encryption Test
```python
# tests/auth/test_encryption.py
from app.core.encryption import Encryptor

def test_encrypt_decrypt():
    """Test encryption roundtrip"""
    encryptor = Encryptor.generate_key()
    enc = Encryptor(encryptor)
    
    plaintext = "sensitive_data"
    encrypted = enc.encrypt(plaintext)
    decrypted = enc.decrypt(encrypted)
    
    assert decrypted == plaintext
```

### Running Tests
```bash
# Run all tests
pytest tests/ -v

# Run specific test file
pytest tests/api/test_api_integration.py -v

# Run specific test
pytest tests/auth/test_encryption.py::test_encrypt_decrypt -v
```

---

## 8. Audit & Security Features

### Audit Middleware
Automatically logs all critical operations:

```python
# app/infra/audit/middleware.py
class AuditMiddleware(BaseHTTPMiddleware):
    AUDIT_METHODS = ["POST", "PUT", "DELETE", "PATCH"]
    AUDIT_TAGS = ["Audit", "Users", "Project", "Junctions", "Pipes", ...]
    
    async def dispatch(self, request: Request, call_next):
        # 1. Capture request body (for writes)
        # 2. Execute request
        # 3. Log to audit DB if matches criteria
        # 4. Return response
```

### Audit Logging
```python
# app/core/audit.py
async def log_audit_event(
    action: AuditAction,  # "CREATE", "UPDATE", "DELETE", "LOGIN"
    resource_type: str,   # "user", "project", "pipe"
    resource_id: str,
    actor_id: int,        # User ID
    details: dict = None
):
    """Log operation to audit table"""
    # Persists to audit_logs table via AuditRepository
```

### Audit Schema
```python
# app/domain/schemas/audit.py
class AuditLog(BaseModel):
    id: int
    timestamp: datetime
    action: str
    resource_type: str
    resource_id: str
    actor_id: int
    details: Optional[dict] = None
```

---

## 9. Key Dependencies & Tech Stack

### Core Framework
```
FastAPI==0.128.0           # Web framework
uvicorn==0.34.0            # ASGI server
Pydantic==2.10.6           # Data validation
pydantic-settings==2.12.0  # Configuration management
```

### Database
```
psycopg==3.2.5             # PostgreSQL async driver
psycopg-pool==3.3.0        # Connection pooling
GeoAlchemy2==0.17.1        # GIS support
SQLAlchemy==2.0.41         # ORM (optional)
```

### Security & Auth
```
PyJWT==2.10.1              # JWT handling
python-jose==3.5.0         # Token validation
passlib==1.7.4             # Password hashing
cryptography==46.0.3       # Encryption (Fernet)
Authlib==1.6.6             # OAuth2 support
```

### Data Science / Simulation
```
wntr==1.3.2                # Water network toolkit (EPANET)
numpy==1.26.2              # Numerical computing
pandas==2.2.3              # Data frames
scipy==1.15.2              # Scientific computing
scikit-learn==1.6.1        # ML algorithms
```

### Time-Series & Data Storage
```
influxdb-client==1.48.0    # InfluxDB client
redis==5.2.1               # Caching & sessions
```

### Utilities
```
python-dotenv==1.2.1       # .env file support
python-multipart==0.0.20   # File upload handling
email-validator==2.3.0     # Email validation
```

---

## 10. Documentation & Deployment

### Key Documentation Files
- **SECURITY_README.md**: Complete security implementation guide
- **DEPLOYMENT.md**: Production deployment checklist
- **INTEGRATION_CHECKLIST.md**: System integration steps
- **setup_server.md**: Initial server setup
- **setup_influxdb.md**: InfluxDB configuration

### Docker Deployment
```yaml
# infra/docker/docker-compose.yml
services:
  api:
    build: ...
    ports: ["8000:8000"]
    environment:
      - DB_HOST=postgis
      - TIMESCALEDB_HOST=timescaledb
      - REDIS_HOST=redis
  
  postgis:
    image: postgis/postgis:14-3.5
  
  timescaledb:
    image: timescale/timescaledb:latest-pg15
  
  redis:
    image: redis:latest
  
  influxdb:
    image: influxdb:2.7
  
  keycloak:
    image: keycloak/keycloak:latest
  
  grafana:
    image: grafana/grafana:latest
```

---

## 11. Recommended Documentation to Create

### For Secondary Development Team:

1. **API Development Guide**
   - How to add new endpoints (covered in Section 2)
   - Common endpoint patterns
   - Error handling standards
   - Response format conventions

2. **Database Development Guide**
   - Repository pattern usage
   - Query building best practices
   - Migration procedures
   - Multi-database considerations

3. **Native Module Integration Guide**
   - How to add new native functions
   - ChangeSet tracking
   - Binary library updates
   - Debugging native code calls

4. **Authentication & Authorization**
   - Token generation and validation
   - Role-based access control implementation
   - Permission decorator usage
   - Keycloak integration

5. **Testing & CI/CD**
   - Unit test patterns
   - Integration test setup
   - Test database configuration
   - GitHub Actions workflow

6. **Deployment & DevOps**
   - Docker build & deployment
   - Kubernetes manifests
   - Environment variable management
   - Production security checklist

7. **Troubleshooting Guide**
   - Common issues and solutions
   - Database connection problems
   - Authentication debugging
   - Native module errors

---

## Summary Table: Key Components

| Component | Location | Purpose | Key Technology |
|-----------|----------|---------|-----------------|
| **API Routes** | `app/api/v1/endpoints/` | REST endpoints | FastAPI, Pydantic |
| **Router** | `app/api/v1/router.py` | Central route registration | APIRouter |
| **Auth** | `app/auth/` | JWT/OAuth2 validation | PyJWT, passlib |
| **Database** | `app/infra/db/` | Data persistence | psycopg, async pools |
| **Repository** | `app/infra/repositories/` | Data access layer | psycopg cursor |
| **Domain** | `app/domain/` | Models & schemas | Pydantic, Enum |
| **Services** | `app/services/` | Business logic | Python, native APIs |
| **Native API** | `app/native/wndb/` | Binary integration | ctypes/cffi, .so/.dll |
| **Config** | `app/core/config.py` | Settings management | Pydantic Settings |
| **Security** | `app/core/security.py` | Encryption, hashing | cryptography, passlib |
| **Audit** | `app/infra/audit/` | Operation logging | Middleware, Repository |
| **Tests** | `tests/` | Test suite | pytest |

---

*This guide provides the essential patterns and structures needed for secondary development on TJWaterServerBinary. Refer to individual source files for detailed implementation examples.*