stupa-pdf-api/README_NEW_ARCHITECTURE.md

# STUPA PDF API - New Architecture (v3.0)

## Overview

The STUPA PDF API has been completely redesigned to support modern authentication, flexible PDF form handling, and a sophisticated workflow system. This document describes the new architecture and features.

## Key Changes from Previous Version

### 1. **Removed LaTeX Compilation**
- No more embedded LaTeX compilation
- Users upload fillable PDF templates instead
- More flexible and maintainable approach

### 2. **OIDC/OAuth2 Authentication**
- Full integration with Nextcloud via OIDC
- Automatic role mapping from OIDC groups
- Single Sign-On (SSO) support

### 3. **Dual Authentication System**
- **OIDC**: Primary authentication via Nextcloud
- **Email**: Magic link authentication as fallback
- Seamless switching between auth methods

### 4. **Role-Based Access Control (RBAC)**
- Dynamic role assignment from OIDC claims
- Configurable permissions per role
- Special roles for workflow participants

### 5. **PDF Template Management**
- Upload any fillable PDF as a template
- Graphical field mapping interface
- Support for multiple form types

### 6. **Visual Form Designer**
- Drag-and-drop form builder
- Custom styling and themes
- Responsive form layouts

### 7. **Advanced Workflow System**
- Multi-stage approval process
- Voting mechanism for AStA members
- Automatic status transitions

## Architecture Components

### Backend Services

#### Authentication Services
- `auth_oidc.py`: OIDC/OAuth2 authentication with Nextcloud
- `auth_email.py`: Email-based authentication with magic links

#### PDF Services
- `pdf_template.py`: Template upload and field mapping
- `pdf_processor.py`: PDF filling and generation

#### Workflow Services
- `workflow_engine.py`: Application workflow management
- `voting_service.py`: Voting and approval handling

### Database Schema

#### Core Tables
- `users`: User accounts with OIDC integration
- `roles`: Role definitions with permissions
- `sessions`: Active user sessions

#### Form Management
- `form_templates`: PDF template definitions
- `field_mappings`: PDF field to data mappings
- `form_designs`: Visual form configurations

#### Application Management
- `applications`: Main application records
- `application_votes`: Voting records
- `application_history`: Audit trail
- `application_attachments`: File attachments

## Workflow Process

The new application workflow follows this state machine:

```
DRAFT → BEANTRAGT → BEARBEITUNG_GESPERRT → ZU_PRUEFEN → ZUR_ABSTIMMUNG → GENEHMIGT/ABGELEHNT
```

### States Explained

1. **DRAFT**: Initial creation, user can edit
2. **BEANTRAGT**: Submitted by user, awaiting processing
3. **BEARBEITUNG_GESPERRT**: Processing locked, under review
4. **ZU_PRUEFEN**: To be reviewed by Haushaltsbeauftragte & Finanzreferent
5. **ZUR_ABSTIMMUNG**: Open for voting by AStA members (requires 5 votes)
6. **GENEHMIGT**: Approved
7. **ABGELEHNT**: Rejected (can be manually reset to BEANTRAGT)

## Configuration

### Environment Variables

#### OIDC Configuration
```env
OIDC_ENABLED=true
OIDC_ISSUER=https://nextcloud.example.com
OIDC_CLIENT_ID=your_client_id
OIDC_CLIENT_SECRET=your_client_secret
OIDC_REDIRECT_URI=http://localhost:3001/auth/callback
```

#### Role Mapping from OIDC Groups
```env
OIDC_ADMIN_GROUPS=admin,administrators
OIDC_BUDGET_REVIEWER_GROUPS=haushaltsbeauftragte
OIDC_FINANCE_REVIEWER_GROUPS=finanzreferent
OIDC_ASTA_GROUPS=asta,asta_members
```

#### Email Configuration
```env
EMAIL_ENABLED=true
SMTP_HOST=localhost
SMTP_PORT=587
EMAIL_FROM=noreply@example.com
```

#### Workflow Configuration
```env
WORKFLOW_REQUIRED_VOTES=5
WORKFLOW_APPROVAL_THRESHOLD=50.0
WORKFLOW_REVIEW_TIMEOUT_DAYS=14
WORKFLOW_VOTING_TIMEOUT_DAYS=7
```

## API Endpoints

### Authentication
- `POST /api/auth/oidc/authorize` - Initiate OIDC flow
- `POST /api/auth/oidc/callback` - OIDC callback handler
- `POST /api/auth/email/register` - Email registration
- `POST /api/auth/email/verify` - Verify email token
- `POST /api/auth/email/login` - Request magic link
- `GET /api/auth/me` - Get current user

### Form Templates
- `GET /api/templates` - List available templates
- `POST /api/templates` - Upload new template (admin)
- `GET /api/templates/{id}` - Get template details
- `PUT /api/templates/{id}/mappings` - Update field mappings
- `POST /api/templates/{id}/test` - Test field mappings

### Applications
- `GET /api/applications` - List user's applications
- `POST /api/applications` - Create new application
- `GET /api/applications/{id}` - Get application details
- `PUT /api/applications/{id}` - Update application
- `POST /api/applications/{id}/submit` - Submit for review
- `POST /api/applications/{id}/review` - Review application
- `POST /api/applications/{id}/vote` - Vote on application

### Admin
- `GET /api/admin/users` - List all users
- `PUT /api/admin/users/{id}/roles` - Update user roles
- `GET /api/admin/roles` - List all roles
- `POST /api/admin/roles/{id}/mappings` - Configure OIDC mappings

## Frontend Components

### User Dashboard
- View all applications
- Track application status
- Access application history

### Form Designer (Admin)
- Visual form builder interface
- Drag-drop field arrangement
- Field property configuration
- Conditional logic builder

### PDF Field Mapper (Admin)
- Upload PDF templates
- Auto-detect form fields
- Map fields to data model
- Configure validation rules

### Voting Interface (AStA Members)
- Review pending applications
- Cast votes with comments
- View voting history

## Security Features

### Authentication
- JWT-based session management
- Refresh token rotation
- Automatic session expiry

### Authorization
- Role-based permissions
- Resource-level access control
- API rate limiting

### Data Protection
- Encrypted token storage
- Secure password hashing
- CSRF protection
- XSS prevention

## Deployment

### Docker Compose

```bash
# Development
docker-compose up -d

# Production
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```

### Required Services
- MySQL 8.0+
- Redis 7.0+
- SMTP server (for email auth)
- Nextcloud instance (for OIDC)

### Volumes
- `/app/uploads` - Uploaded files
- `/app/templates` - PDF templates
- `/app/attachments` - Application attachments

## Migration from v2

1. **Backup existing data**
   ```bash
   docker-compose exec db mysqldump -u root -p stupa > backup.sql
   ```

2. **Run migrations**
   ```bash
   docker-compose exec api python -m alembic upgrade head
   ```

3. **Configure OIDC**
   - Register application in Nextcloud
   - Configure redirect URIs
   - Map groups to roles

4. **Upload PDF templates**
   - Convert existing LaTeX templates to fillable PDFs
   - Upload via admin interface
   - Configure field mappings

## Development

### Setup Development Environment

```bash
# Clone repository
git clone https://github.com/your-org/stupa-pdf-api.git
cd stupa-pdf-api

# Checkout new architecture branch
git checkout feature/oidc-pdf-upload-redesign

# Copy environment file
cp .env.example .env

# Start services
docker-compose --profile dev up
```

### Testing

```bash
# Run backend tests
docker-compose exec api pytest

# Run frontend tests
docker-compose exec frontend npm test

# Test OIDC integration
docker-compose exec api python -m tests.test_oidc
```

## Troubleshooting

### OIDC Connection Issues
- Verify Nextcloud OAuth2 app is configured
- Check redirect URI matches configuration
- Ensure client ID/secret are correct

### PDF Processing Issues
- Verify PDF has fillable form fields
- Check field names don't contain special characters
- Ensure PDF is not password protected

### Email Delivery Issues
- Check SMTP configuration
- Verify firewall allows SMTP port
- Test with local mail server (MailHog)

## Support

For issues or questions about the new architecture:
1. Check the documentation
2. Review the migration guide
3. Contact the development team

## License

This project is licensed under the MIT License - see the LICENSE file for details.