Skip to main content

ARC // OS — Deployment Guide (For Developers)

Last Updated: 2026-01-21
Audience: Developers and system administrators deploying ARC // OS

Overview​

This guide covers deployment of ARC // OS to production environments. The system can be deployed using Docker Compose (recommended) or manually on bare metal servers.

For user documentation, see USER_GUIDE.md.

Barebones Debian Server Setup​

This section covers setting up ARC // OS on a fresh Debian 13 server from scratch, including Docker installation, Git configuration, and GitHub Actions setup.

Prerequisites​

  • Fresh Debian 13 (Trixie) server
  • Root or sudo access
  • Domain name pointing to server IP (for SSL)
  • GitHub repository with code

Step 1: Initial Server Setup​

1.1 Update System​

# Update package lists
sudo apt update
sudo apt upgrade -y

# Install essential tools
sudo apt install -y curl wget git ufw fail2ban htop

1.2 Create Deployment User​

# Create a non-root user for deployment
sudo adduser deploy
sudo usermod -aG sudo deploy

# Switch to deploy user
su - deploy

1.3 Configure Firewall​

# Allow SSH (adjust port if needed - be careful not to lock yourself out!)
sudo ufw allow 22/tcp

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Enable firewall
sudo ufw enable
sudo ufw status

Important: Make sure SSH access is working before enabling the firewall. If you're connecting via SSH, ensure port 22 is allowed before running ufw enable.

# Edit SSH config
sudo nano /etc/ssh/sshd_config

# Recommended settings:
# PermitRootLogin no
# PasswordAuthentication no (use SSH keys only)
# Port 22 (or change to non-standard port)

# Restart SSH
sudo systemctl restart sshd

Step 2: Install Docker and Docker Compose​

2.1 Install Docker​

# Remove old versions
sudo apt remove -y docker docker-engine docker.io containerd runc

# Install prerequisites
sudo apt install -y ca-certificates gnupg lsb-release

# Add Docker's official GPG key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# Set up repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Add user to docker group
sudo usermod -aG docker deploy

# Verify installation
docker --version
docker compose version

# Log out and back in for group changes to take effect
exit
# SSH back in

2.2 Configure Docker​

# Configure Docker daemon (optional optimizations)
sudo nano /etc/docker/daemon.json

# Add:
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

# Restart Docker
sudo systemctl restart docker
sudo systemctl enable docker

Step 3: Setup Application Directory​

# Create application directory
sudo mkdir -p /opt/arcos
sudo chown deploy:deploy /opt/arcos
cd /opt/arcos

# Create necessary subdirectories
mkdir -p docker ssl env uploads backups

Step 4: Setup Git and GitHub​

4.1 Generate SSH Key for GitHub​

# Generate SSH key
ssh-keygen -t ed25519 -C "deploy@arcos-server" -f ~/.ssh/github_deploy

# Display public key
cat ~/.ssh/github_deploy.pub

4.2 Add SSH Key to GitHub​

  1. Go to GitHub → Settings → SSH and GPG keys
  2. Click "New SSH key"
  3. Paste the public key from above
  4. Save

4.3 Configure Git​

# Configure git
git config --global user.name "ARC OS Deploy"
git config --global user.email "deploy@yourdomain.com"

# Test GitHub connection
ssh -T git@github.com

4.4 Clone Repository (Optional - GitHub Actions will handle deployment)​

# If you want to manually clone (not required for GitHub Actions)
cd /opt/arcos
git clone git@github.com:yourusername/arcos.git .

Step 5: Setup GitHub Actions for Deployment​

5.1 Create GitHub Secrets​

Go to your GitHub repository → Settings → Secrets and variables → Actions

Add the following secrets:

Required Secrets:

  • DEPLOY_SSH_KEY - Private SSH key for server access (content of ~/.ssh/github_deploy from server)
  • DEPLOY_HOST - Server IP or domain (e.g., arc.corgicy.com or 123.456.789.0)
  • DEPLOY_USER - SSH user (e.g., deploy)

Optional Secrets (for production):

  • POSTGRES_PASSWORD - Database password (generate with: openssl rand -base64 24)
  • POSTGRES_USER - Database user (default: arcos, optional)
  • POSTGRES_DB - Database name (default: arcos, optional)
  • JWT_SECRET - JWT signing secret (generate with: openssl rand -base64 32)
  • DATABASE_URL - Only needed if NOT using docker-compose.yml (auto-constructed otherwise)
  • EMAIL_PROVIDER_API_KEY - If using email service
  • AWS_ACCESS_KEY_ID - If using S3 storage
  • AWS_SECRET_ACCESS_KEY - If using S3 storage

5.2 Generate SSH Key for GitHub Actions​

On your local machine (not server):

# Generate key pair
ssh-keygen -t ed25519 -C "github-actions-deploy" -f ~/.ssh/github_actions_deploy

# Copy public key to server
ssh-copy-id -i ~/.ssh/github_actions_deploy.pub deploy@your-server-ip

# Display private key (add to GitHub secret DEPLOY_SSH_KEY)
cat ~/.ssh/github_actions_deploy

Important: The private key goes in GitHub secret DEPLOY_SSH_KEY, the public key should be authorized on the server.

5.3 Authorize GitHub Actions SSH Key on Server​

On the server:

# Add the public key to authorized_keys
mkdir -p ~/.ssh
chmod 700 ~/.ssh
vim ~/.ssh/authorized_keys
# Paste the public key from github_actions_deploy.pub

chmod 600 ~/.ssh/authorized_keys

Step 6: Configure Environment Variables​

6.1 Create Environment File on Server​

cd /opt/arcos

# Copy the example file (from repository)
cp env/.env.example .env

# Edit and fill in the actual values
nano .env

Note: The example file (env/.env.prod.example) is in the repository. Copy it to env/.env.prod and fill in the actual values.

Required values to set:

See env/.env.example in the repository for the complete template.

Minimum required values:

  • POSTGRES_PASSWORD - Generate with: openssl rand -base64 24
  • JWT_SECRET - Generate with: openssl rand -base64 32
  • CORS_ORIGIN - Your production domain (e.g., https://arc.corgicy.com)

All other values have sensible defaults but can be customized as needed.

Important Database Security Notes:

  • The database runs in Docker and is only accessible from within the Docker network (not exposed to the internet)
  • The POSTGRES_PASSWORD is used to:
    1. Initialize the PostgreSQL database container
    2. Automatically construct the DATABASE_URL for server and worker services
  • Never commit passwords to git - use environment variables or secrets management
  • The password should be strong (at least 24 characters, generated randomly)
  • Docker Compose automatically creates the DATABASE_URL from these variables:
    • Format: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
    • Hostname is db (the Docker service name), not postgres or localhost

6.2 Generate Strong Secrets​

# Generate JWT secret (32+ characters recommended)
openssl rand -base64 32

# Generate database password (24+ characters recommended)
openssl rand -base64 24

Step 7: Setup SSL/TLS with Let's Encrypt​

7.1 Install Certbot​

sudo apt install -y certbot python3-certbot-nginx

7.2 Get SSL Certificate​

# Stop nginx if running
sudo systemctl stop nginx

# Get certificate (replace with your domain)
sudo certbot certonly --standalone -d arcos.corgicy.com -d www.arcos.corgicy.com

# Certificates will be in:
# /etc/letsencrypt/live/yourdomain.com/fullchain.pem
# /etc/letsencrypt/live/yourdomain.com/privkey.pem

7.3 Setup Auto-Renewal​

# Test renewal
sudo certbot renew --dry-run

# Certbot auto-renewal is already configured via systemd timer
sudo systemctl status certbot.timer

Step 8: Configure Docker Compose for Production​

The GitHub Actions workflow will copy docker-compose.yml to the server. Ensure your repository has this file configured correctly.

8.1 Verify docker-compose.yml​

Ensure your docker-compose.yml includes:

  • PostgreSQL service
  • Server service
  • Web service
  • Marketing service (optional)
  • Docs service (optional)
  • Nginx reverse proxy

Step 9: First Deployment​

9.1 Manual First Deployment (Before GitHub Actions)​

cd /opt/arcos

# Pull images
docker compose --env-file env/.env.prod -f docker-compose.yml pull

# Run database migrations
docker compose --env-file env/.env.prod -f docker-compose.yml run --rm server pnpm prisma migrate deploy
docker compose --env-file env/.env.prod -f docker-compose.yml run --rm server pnpm prisma generate

# Seed database (optional)
docker compose --env-file env/.env.prod -f docker-compose.yml run --rm server pnpm prisma db seed

# Start services
docker compose --env-file env/.env.prod -f docker-compose.yml up -d

# Check logs
docker compose --env-file env/.env.prod -f docker-compose.yml logs -f

9.2 Verify Deployment​

# Check services are running
docker compose --env-file env/.env.prod -f docker-compose.yml ps

# Check health endpoint
curl http://localhost/health

# Check logs for errors
docker compose --env-file env/.env.prod -f docker-compose.yml logs server
docker compose --env-file env/.env.prod -f docker-compose.yml logs web

Step 10: Enable GitHub Actions Deployment​

Once the server is set up and manually deployed, GitHub Actions will automatically deploy on every push to main branch.

10.1 Verify GitHub Actions Workflow​

  1. Push a commit to main branch
  2. Go to GitHub → Actions tab
  3. Watch the deployment workflow
  4. Check for any errors

10.2 Troubleshooting GitHub Actions​

Common Issues:

  1. SSH Connection Failed

    • Verify DEPLOY_SSH_KEY secret contains the private key
    • Verify DEPLOY_HOST and DEPLOY_USER are correct
    • Test SSH connection manually: ssh deploy@your-server-ip

  2. Docker Login Failed

    • Verify GITHUB_TOKEN is available (automatically provided)
    • Check GitHub Container Registry permissions

  3. Deployment Failed

    • Check server logs: docker compose logs
    • Verify environment variables are set correctly
    • Check disk space: df -h

Step 11: Post-Deployment Setup​

11.1 Setup Monitoring (Optional)​

# Install monitoring tools
sudo apt install -y htop iotop nethogs

# Setup log rotation
sudo nano /etc/logrotate.d/docker-containers

11.2 Setup Backups​

# Create backup script
nano /opt/arcos/backup.sh
chmod +x /opt/arcos/backup.sh

# Add to crontab
crontab -e
# Add: 0 2 * * * /opt/arcos/backup.sh

11.3 Setup Database Backups​

# Create database backup script
nano /opt/arcos/backups/db-backup.sh

Example backup script:

#!/bin/bash
BACKUP_DIR="/opt/arcos/backups"
DATE=$(date +%Y%m%d_%H%M%S)
docker compose --env-file env/.env.prod -f docker-compose.yml exec -T db pg_dump -U arcos arcos > "$BACKUP_DIR/db_backup_$DATE.sql"
# Keep only last 7 days
find "$BACKUP_DIR" -name "db_backup_*.sql" -mtime +7 -delete

Step 12: Maintenance Commands​

# View logs
docker compose --env-file env/.env.prod -f docker-compose.yml logs -f

# Restart services
docker compose --env-file env/.env.prod -f docker-compose.yml restart

# Update and redeploy
docker compose --env-file env/.env.prod -f docker-compose.yml pull
docker compose --env-file env/.env.prod -f docker-compose.yml up -d

# Run database migrations
docker compose --env-file env/.env.prod -f docker-compose.yml run --rm server pnpm prisma migrate deploy

# Access database
docker compose --env-file env/.env.prod -f docker-compose.yml exec postgres psql -U arcos arcos

# Clean up old images
docker image prune -a -f

Security Checklist​

  • Firewall configured (UFW)
  • SSH key authentication only (no passwords)
  • Fail2ban installed and configured
  • Non-root user for deployment
  • Docker group permissions limited
  • SSL/TLS certificates installed
  • Strong passwords for database
  • Environment variables secured
  • Regular backups configured
  • Log rotation configured
  • System updates automated

Prerequisites​

  • Node.js LTS (v20+)
  • PostgreSQL 15+ (or use Docker)
  • Docker & Docker Compose (for containerized deployment)
  • pnpm package manager (⚠️ CRITICAL: This project uses pnpm exclusively. Do NOT use yarn or npm.)
  • Basic server administration knowledge

Why Docker Compose?​

  • Consistent environment - Same setup across dev, staging, production
  • Easy updates - Pull, build, restart
  • Isolated services - Database, backend, frontend separated
  • Simplified deployment - One command to start everything

Step-by-Step Deployment​

1. Clone and Configure​

# Clone repository
git clone <repository-url>
cd arcos

# Copy environment files
cp server/.env.example server/.env
cp web/.env.example web/.env

2. Configure Environment Variables​

Backend (server/.env):

# Database
DATABASE_URL="postgresql://arcos:strong-password@postgres:5432/arcos"

# Authentication
JWT_SECRET="your-strong-random-secret-here-minimum-32-characters"

# Server
PORT=3001
NODE_ENV=production

# CORS (your frontend domain)
CORS_ORIGIN="https://yourdomain.com"

# File Upload
FILE_STORAGE_ADAPTER=disk  # or 's3' for production
FILE_UPLOAD_DIR=./uploads
MAX_FILE_SIZE=10485760  # 10MB

# Rate Limiting
RATE_LIMIT_MAX=1000
RATE_LIMIT_TIME_WINDOW=900000  # 15 minutes
RATE_LIMIT_AUTH_MAX=100
RATE_LIMIT_AUTH_TIME_WINDOW=900000

# Email Service (for test deployment)
EMAIL_PROVIDER=mock  # Use 'mock' for test deployment (saves emails to server/emails directory)
# Future: 'aws-ses' or 'postmark' for production

Frontend (web/.env):

VITE_API_URL=https://api.yourdomain.com

3. Build and Start Services​

# Build and start all services
docker compose up --build -d

# View logs
docker compose logs -f

# Check service status
docker compose ps

4. Initialize Database​

# Run migrations
docker compose exec server pnpm prisma migrate deploy

# Generate Prisma Client
docker compose exec server pnpm prisma generate

# Seed database (required for production)
docker compose exec server pnpm prisma db seed

Note: Database seeding is required for production deployments. See Database Seeding section below for details.

5. Verify Deployment​

# Check backend health
curl http://localhost:3001/health
# Should return: {"status":"ok"}

# Check frontend
curl http://localhost:3000
# Should return HTML

6. Setup Reverse Proxy (Nginx)​

# /etc/nginx/sites-available/arcos
server {
    listen 80;
    server_name yourdomain.com;

    # Frontend
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    # API
    location /api {
        proxy_pass http://localhost:3001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
    }
}

7. Setup SSL/TLS (Let's Encrypt)​

# Install Certbot
sudo apt-get install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d yourdomain.com -d api.yourdomain.com

# Auto-renewal (already configured by Certbot)

Manual Deployment (Bare Metal)​

When to Use Manual Deployment​

  • Custom infrastructure - Specific server requirements
  • Performance optimization - Fine-tuned for your hardware
  • Integration needs - Custom integrations with existing systems
  • Compliance requirements - Specific security or compliance needs

Step-by-Step Deployment​

1. Server Setup​

# Update system
sudo apt-get update && sudo apt-get upgrade -y

# Install Node.js (using nvm)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 20
nvm use 20

# Install PostgreSQL
sudo apt-get install postgresql-15 postgresql-contrib

# Install pnpm
npm install -g pnpm

2. Database Setup​

# Create database user
sudo -u postgres psql
CREATE USER arcos WITH PASSWORD 'strong-password';
CREATE DATABASE arcos OWNER arcos;
GRANT ALL PRIVILEGES ON DATABASE arcos TO arcos;
\q

# Test connection
psql -U arcos -d arcos -h localhost

3. Backend Deployment​

cd server

# Install dependencies
pnpm install --production

# Configure environment
cp .env.example .env
# Edit .env with your settings

# Run migrations
pnpm prisma migrate deploy

# Generate Prisma Client
pnpm prisma generate

# Build TypeScript
pnpm build

# Start with PM2
npm install -g pm2
pm2 start dist/index.js --name arcos-api
pm2 save
pm2 startup  # Setup auto-start on boot

4. Frontend Deployment​

cd web

# Install dependencies
pnpm install

# Build for production
pnpm build

# Serve with Nginx (see Nginx configuration below)

5. Nginx Configuration​

# /etc/nginx/sites-available/arcos
server {
    listen 80;
    server_name yourdomain.com;

    # Frontend
    root /path/to/arcos/web/dist;
    index index.html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    # API proxy
    location /api {
        proxy_pass http://localhost:3001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
    }

    # Static assets caching
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
}
# Enable site
sudo ln -s /etc/nginx/sites-available/arcos /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Database Management​

Database Seeding​

Database seeding ensures that essential global data is available to all users. This includes achievements, shop items, challenges, bounties, opponents, journal questions, and exercise catalog entries.

Important: All seed scripts are idempotent (safe to run multiple times) and only seed global data (userId = null). They do not create users or user-specific data.

Production Seeding:

# Docker deployment
docker compose exec server pnpm prisma db seed

# Manual deployment
cd server
pnpm prisma db seed

What Gets Seeded:

  • ✅ Global Achievements (available to all users)
  • ✅ Global Shop Items (items available in shop)
  • ✅ Global Challenges (challenge templates)
  • ✅ Global Bounties (bounty templates)
  • ✅ Global Opponents (for gamification battles)
  • ✅ Global Journal Questions (default journal questions)
  • ✅ Exercise Catalog (common exercises available to all users)

Development/Test Seeding:

For development environments, you may also want to seed demo users and test data:

# This is typically handled by seed.ts automatically in development
# Demo user creation should be separate from production seeds

Verification:

After seeding, verify that global data is present:

# Check achievements
docker compose exec server pnpm prisma studio
# Navigate to Achievement table, verify global achievements exist (userId = null)

# Or via API (if admin endpoints available)
curl http://localhost:3001/api/gamification/shop
# Should return shop items

For detailed seeding information, see SEEDING_PLAN.md.

Migrations​

Development:

cd server
pnpm prisma migrate dev --name migration_name

Production:

# Always backup first!
docker compose exec server pnpm prisma migrate deploy
# OR
cd server && pnpm prisma migrate deploy

Backups​

Automated Backup Script:

#!/bin/bash
# backup-db.sh
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/backups/arcos"
mkdir -p $BACKUP_DIR

# Docker
docker compose exec -T db pg_dump -U arcos arcos > $BACKUP_DIR/backup_$DATE.sql

# Or manual
pg_dump -U arcos arcos > $BACKUP_DIR/backup_$DATE.sql

# Keep only last 30 days
find $BACKUP_DIR -name "backup_*.sql" -mtime +30 -delete

Restore:

# From backup
psql -U arcos arcos < backup_20260121_120000.sql

# Or Docker
docker compose exec -T db psql -U arcos arcos < backup_20260121_120000.sql

File Storage​

Disk Storage (Development/Test Deployment)​

Configuration:

  • Environment Variable: FILE_STORAGE_ADAPTER=disk (default)
  • Upload Directory: FILE_UPLOAD_DIR=./uploads (default) or /app/uploads (Docker)
  • Storage Location: Files stored in {uploadDir}/{userId}/{fileId}/{filename} structure

Docker Setup:

# In docker-compose.yml
volumes:
  uploads_data:/app/uploads  # Persistent volume for file uploads

# Environment variables
FILE_STORAGE_ADAPTER=disk
FILE_UPLOAD_DIR=/app/uploads

Volume Configuration:

  • Volume Name: uploads_data (named Docker volume)
  • Mount Path: /app/uploads (inside container)
  • Permissions: Files created with default Node.js process permissions (typically readable/writable by container user)
  • Persistence: Volume persists across container restarts and updates

Manual Setup (Non-Docker):

# Create uploads directory
mkdir -p server/uploads
chmod 755 server/uploads

# Ensure sufficient disk space
df -h

File Permissions:

  • Files are created with default Node.js fs.writeFile() permissions
  • Directory structure is created with fs.mkdir(..., { recursive: true })
  • In Docker, files are owned by the container's user (typically the Node.js process user)
  • For production, ensure the uploads directory has proper permissions for the application user

Orphaned File Cleanup:

  • Use the admin endpoint POST /api/v1/admin/files/cleanup to find and remove files that exist on disk but not in the database
  • Dry-run mode: POST /api/v1/admin/files/cleanup?dryRun=true - Reports orphaned files without deleting
  • Delete mode: POST /api/v1/admin/files/cleanup?dryRun=false - Deletes orphaned files
  • Requires admin authentication
  • Returns count of orphaned files found/deleted
  • In dry-run mode, returns array of orphaned file paths

Backup:

# Backup uploads directory
tar -czf uploads_backup_$(date +%Y%m%d).tar.gz server/uploads/

# Docker backup
docker compose exec server tar -czf /tmp/uploads_backup.tar.gz /app/uploads
docker compose cp server:/tmp/uploads_backup.tar.gz ./uploads_backup.tar.gz

Setup:

  1. Create S3 bucket with appropriate permissions
  2. Configure IAM user with S3 access
  3. Set environment variables: 
    FILE_STORAGE_ADAPTER=s3
    AWS_S3_BUCKET=your-bucket-name
    AWS_S3_REGION=us-east-1
    AWS_ACCESS_KEY_ID=your-access-key
    AWS_SECRET_ACCESS_KEY=your-secret-key
  4. Configure bucket policies for public read (if needed)
  5. Setup CDN (CloudFront) for better performance

Email Configuration​

Test Deployment (Mock Email Service)​

For test deployment, ARC // OS uses a mock email service that saves emails to the file system instead of sending them. This allows you to verify email functionality without configuring a real email provider.

Configuration:

# In server/.env
EMAIL_PROVIDER=mock  # Default if not set

How it works:

  • Emails are saved to server/emails/ directory as JSON files
  • Each email is stored with a unique ID and timestamp
  • Emails are also stored in memory for API access
  • View sent emails via API: GET /api/emails (development only)

Verifying email delivery:

# Check emails directory
ls -la server/emails/

# View a specific email file
cat server/emails/<email-id>.json

# Or use the API (if running)
curl http://localhost:3001/api/emails

Docker deployment:

  • Ensure server/emails/ directory is mounted as a volume for persistence
  • Emails will be saved inside the container at /app/emails/
  • Mount a volume: -v ./emails:/app/emails in docker-compose.yml

Production Email Providers (Future)​

Note: Real email providers (AWS SES, Postmark) are not yet implemented. The system is designed to support them via the IEmailService interface.

Planned configuration:

# For AWS SES (future)
EMAIL_PROVIDER=aws-ses
AWS_SES_REGION=us-east-1
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key

# For Postmark (future)
EMAIL_PROVIDER=postmark
POSTMARK_API_KEY=your-api-key
POSTMARK_FROM_EMAIL=noreply@yourdomain.com

Current status:

  • ✅ Mock email service implemented and tested - Complete: Comprehensive integration test (email-delivery.integration.test.ts) verifies email sending, file storage, memory storage, and API access. All 7 tests passing.
  • ✅ Email configuration documented - Complete: Added email configuration section to DEPLOYMENT.md with details on mock email service, how to verify delivery, Docker volume mounting, and future production provider plans. Updated README.md and DOCKER_DEPLOYMENT.md with EMAIL_PROVIDER environment variable documentation.
  • ✅ Email service interface ready for production providers - Complete: Adapter pattern implemented, ready for AWS SES or Postmark integration
  • ⏳ AWS SES adapter (deferred until after testing)
  • ⏳ Postmark adapter (deferred until after testing)

Security Checklist​

Before Going Live​

  • Strong JWT_SECRET - Cryptographically random (minimum 32 characters)
  • HTTPS enabled - All traffic over SSL/TLS
  • Secure cookies - secure: true in production
  • CORS configured - Only allow your frontend domain
  • Database credentials - Strong passwords, not default
  • File upload limits - Reasonable size limits enforced
  • Rate limiting - API rate limiting enabled
  • Input validation - All inputs validated with Zod
  • SQL injection prevention - Using Prisma (parameterized queries)
  • XSS prevention - React automatic escaping
  • CSRF protection - httpOnly cookies
  • Environment variables - Never commit secrets to git
  • Backup strategy - Regular automated backups
  • Update dependencies - Keep dependencies up to date
  • Error handling - Don't expose internal errors to users
  • Firewall rules - Only necessary ports open (22, 80, 443)
  • SSH key authentication - Disable password authentication

Monitoring & Maintenance​

Health Checks​

# Backend health
curl http://localhost:3001/health
# Response: {"status":"ok"}

# Database connectivity
docker compose exec db pg_isready
# OR
pg_isready -h localhost -p 5432

Logging​

Docker:

# View all logs
docker compose logs -f

# View specific service
docker compose logs -f server

# View last 100 lines
docker compose logs --tail=100 server

PM2:

# View logs
pm2 logs arcos-api

# Monitor
pm2 monit

Updates​

Docker:

# Pull latest code
git pull

# Rebuild and restart
docker compose up --build -d

# Run migrations if needed
docker compose exec server pnpm prisma migrate deploy
docker compose exec server pnpm prisma generate

Manual:

# Pull latest code
git pull

# Install dependencies
pnpm install

# Run migrations
pnpm prisma migrate deploy
pnpm prisma generate

# Rebuild
pnpm build

# Restart
pm2 restart arcos-api

Scaling Considerations​

Database​

  • Connection pooling - Prisma handles this automatically
  • Read replicas - For read-heavy workloads
  • Indexes - Ensure proper indexes on foreign keys and search fields
  • Query optimization - Monitor slow queries

Application​

  • Horizontal scaling - Multiple backend instances behind load balancer
  • Stateless design - No session storage in memory (uses database)
  • Caching - Redis for frequently accessed data (future)
  • CDN - For static assets and images

File Storage​

  • S3 - Scalable object storage
  • CDN - CloudFront or similar for image delivery
  • Image optimization - WebP conversion, compression

Rollback Procedures​

Rolling Back a Deployment​

If a deployment causes issues, you can rollback to a previous version:

Docker Deployment:

# 1. Stop current deployment
docker compose down

# 2. Checkout previous version
git checkout <previous-commit-hash>

# 3. Rebuild and restart
docker compose up --build -d

# 4. Rollback database migrations if needed
docker compose exec server pnpm prisma migrate resolve --rolled-back <migration-name>

Manual Deployment:

# 1. Stop application
pm2 stop arcos-api

# 2. Checkout previous version
git checkout <previous-commit-hash>

# 3. Install dependencies and rebuild
pnpm install
pnpm build

# 4. Restart application
pm2 restart arcos-api

# 5. Rollback database migrations if needed
cd server
pnpm prisma migrate resolve --rolled-back <migration-name>

Rolling Back Database Migrations​

Important: Only rollback migrations if absolutely necessary. Always backup first!

# 1. Backup database
pg_dump -U arcos arcos > backup_before_rollback.sql

# 2. Mark migration as rolled back (doesn't actually undo migration)
pnpm prisma migrate resolve --rolled-back <migration-name>

# 3. Manually revert database changes if needed
# This requires manual SQL to undo schema changes

Note: Prisma doesn't automatically rollback migrations. You must manually revert schema changes if needed. Consider using database backups instead.

Emergency Rollback​

For critical issues requiring immediate rollback:

# Quick rollback to previous Docker image
docker compose down
docker compose up -d --no-build

# Or restore from backup
# 1. Stop services
docker compose down

# 2. Restore database
docker compose exec -T db psql -U arcos arcos < backup.sql

# 3. Restart services
docker compose up -d

Troubleshooting​

Database Connection Errors​

# Check database is running
pg_isready -h localhost -p 5432

# Check connection string
echo $DATABASE_URL

# Test connection
psql $DATABASE_URL -c "SELECT 1;"

Port Conflicts​

# Check what's using port 3001
lsof -i :3001

# Kill process
kill -9 <PID>

Migration Errors​

# Check migration status
pnpm prisma migrate status

# Force apply migrations
pnpm prisma migrate deploy --skip-generate

Support​

For deployment issues:

  1. Check logs (docker compose logs or PM2 logs)
  2. Verify environment variables
  3. Check database connectivity
  4. Review health endpoint
  5. Check file permissions and disk space

For development setup, see QUICKSTART.md.