aa5ca7ff31d2ba0d0195d979e114e3b0c21407aa
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 4310d9a1c4 | Update publish.yml | |||
| 20baba9a5f |
Update GitHub Actions delete-packag-versions workflow
Change the package-name input from referencing environment variables directly to using env.PR and env.PROJECT_NAME references for the backend and frontend workflows. |
|||
| 81ea4244af | Update publish.yml | |||
| 5ca10f3b0e |
Fix typo in build and push frontend job name
Update context and Dockerfile path for frontend builds |
|||
| def4ddabb7 | Update publish.yml | |||
| 06c9ff6ed6 |
fix: add continue-on-error to image cleanup steps
- Image cleanup steps will now continue on error instead of failing the workflow - This handles the case where packages don't exist yet or aren't found - Keeps workflow from breaking on first cleanup attempt - Maintains proper YAML formatting and indentation |
|||
| b7ebd7cf23 | Clean up old container images in publish workflow | |||
| b7306a963a |
Migration from SQlite3 to PostgreSQL (#1)
* Migrate from SQLite to PostgreSQL for dev and prod environments
- Replace better-sqlite3 with pg library in backend
- Update database.js to use PostgreSQL connection pooling
- Convert migrations.sql to PostgreSQL syntax with proper triggers and constraints
- Convert seeds.sql to PostgreSQL syntax with JSONB for colors and ON CONFLICT handling
- Update docker-compose.yml with PostgreSQL service and db-migrate container
- Update deploy.sh to generate production docker-compose with PostgreSQL configuration
- Configure environment variables for database connection (DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD)
* Update database models to use PostgreSQL async API
- Convert User.js from better-sqlite3 to PostgreSQL async queries
- Use parameterized queries with , placeholders
- Update all methods to use async/await
- Use result.rowCount instead of result.changes
- Use result.rows[0].id for RETURNING clause results
- Convert Commander.js from better-sqlite3 to PostgreSQL async queries
- Implement proper async methods with pg library
- Update JSONB color handling (no longer needs JSON.parse/stringify)
- Use ILIKE for case-insensitive search instead of LIKE
- Use proper numeric casting for win rate calculations
- Convert Game.js from better-sqlite3 to PostgreSQL async queries
- All query methods now properly async
- Update boolean handling (true/false instead of 1/0)
- Use ILIKE for case-insensitive commander name search
- Use RETURNING clause instead of lastInsertRowid
All models now use dbManager.query(), dbManager.get(), and dbManager.all() methods
* Add PostgreSQL cleanup and repository pattern for improved DB abstraction
Database Improvements:
- Fix migrate.js to use async PostgreSQL API with proper error handling
- Update .env.example to reflect PostgreSQL configuration variables
- Update GitHub Actions workflow to generate production docker-compose with PostgreSQL
Architectural Improvements:
- Create base Repository class providing common CRUD operations
- Implement UserRepository for user-specific database operations
- Implement CommanderRepository for commander-specific database operations
- Implement GameRepository for game-specific database operations
- All repositories use parameterized queries to prevent SQL injection
- Centralized database access patterns for better maintainability
Benefits:
- Cleaner separation of concerns (repositories handle data access)
- Reusable repository pattern can be extended for new entities
- Better error handling and transaction support
- Improved code organization and maintainability
- All database queries now properly handle PostgreSQL async operations
* Add comprehensive PostgreSQL migration documentation
- Complete migration summary with before/after comparison
- Detailed changes to each component
- Architecture improvements and benefits
- Repository pattern explanation
- Quick start guide for development and production
- Deployment instructions
- Testing guidelines
- Verification checklist
* Fix Docker build: update package-lock.json and npm syntax
- Run npm install to update package-lock.json with pg dependencies
- Replace deprecated --only=production with --omit=dev (npm 7+)
- Resolves Docker build error about missing pg packages in lock file
* Add migration status document
- Executive summary of completed migration
- Detailed commits history with verification
- Architecture before/after comparison
- Deployment instructions (dev & prod)
- Testing procedures
- Performance characteristics
- Troubleshooting guide
- Rollback instructions (if needed)
- Final sign-off for production readiness
* Add comprehensive deployment checklist
- Pre-deployment verification checklist
- Development deployment step-by-step guide
- Production deployment procedures
- Rollback instructions for emergency cases
- Monitoring and maintenance procedures
- Common issues and solutions
- Performance monitoring commands
- Sign-off checklist for deployment readiness
* Fix PostgreSQL user authentication in docker-compose
- Use postgres superuser instead of creating custom user
- Add init-db script directory for database initialization
- Update docker-compose to mount init scripts
- Simplify credentials: use 'postgres' user for dev
- Update .env.example with correct default credentials
- Health check now uses postgres user
This resolves the 'role edh_user does not exist' error on container startup.
The postgres superuser can create and manage databases and migrations.
* Add Docker Compose testing guide
- Prerequisites checklist
- Step-by-step startup instructions
- Service verification procedures
- API and database testing commands
- Frontend verification steps
- Troubleshooting guide for common issues
- Cleanup procedures
- Expected results checklist
* Fix PostgreSQL password authentication - use correct default user
- Remove incorrect POSTGRES_USER setting (defaults to postgres)
- Set POSTGRES_PASSWORD=edh_password for the default postgres user
- Set POSTGRES_DB=edh_stats to create database automatically
- Update db-migrate and backend services to use password: edh_password
- Update .env.example with correct credentials
PostgreSQL Docker image automatically creates the 'postgres' superuser
when POSTGRES_PASSWORD is set. This resolves the authentication error.
* Add final verification document for completed migration
- Complete migration status verification
- All issues resolved checklist
- Final commit log with all 10 commits
- Quick start command for docker-compose
- Development credentials reference
- Complete list of deliverables
- Documentation index
- Ready-to-deploy status confirmation
* Fix PostgreSQL initialization and SQL migration execution
- Fix 01-init.sql: Remove invalid MySQL syntax and rely on POSTGRES_DB for database creation
- Fix database.js: Execute entire migration/seed SQL files as single queries instead of splitting by semicolon
This prevents issues with multi-statement SQL constructs (functions, views, triggers)
- Fix docker-compose.yml: Add listen_addresses=* to allow connections from Docker network containers
and add PGPASSWORD to healthcheck for proper password authentication
All services now start successfully:
- PostgreSQL accepts connections from Docker network
- Migrations run without errors
- Seed data is properly inserted
- Backend API starts and health checks pass
- Database schema with tables, views, and triggers created correctly
* Fix production docker-compose configuration in deploy.sh
- Add listen_addresses=* to PostgreSQL command for Docker network connectivity
- Use 'postgres' superuser instead of DB_USER variable (matches development setup)
- Fix PostgreSQL healthcheck to include PGPASSWORD environment variable
- Fix frontend healthcheck to check root path instead of non-existent /health endpoint
- Add resource limits to frontend container for consistency
- Update .env documentation to reflect correct PostgreSQL user
* Fix DB_USER configuration consistency
- Change default DB_USER in database.js from 'edh_user' to 'postgres'
- Aligns with .env.example, docker-compose.yml, and deploy.sh
- Add clarifying comment in .env.example explaining superuser requirement
- DB_USER must be a superuser to run migrations and create schema objects
The PostgreSQL superuser 'postgres' is created automatically by the Docker image
and has the necessary privileges for all application operations.
* Add DB_SEED environment variable to toggle automatic seeding
- Add DB_SEED environment variable to db-migrate service (default: false)
- Update migrate.js to check DB_SEED and automatically seed if enabled
- Fix seeds.sql ON CONFLICT clauses and sequence resets to use dynamic MAX(id)
- Seeds can now be triggered by setting DB_SEED=true in docker-compose or .env
- Add documentation to .env.example explaining DB_SEED option
- Update deploy.sh to support DB_SEED in production configuration
This allows developers to quickly populate test data during development
without manual seeding commands, while keeping it opt-in for clean databases.
* Fix Commander model: properly convert colors array to JSON for JSONB storage
- Convert JavaScript arrays to JSON strings before inserting into JSONB column
- Add ::jsonb type cast in SQL queries for explicit JSONB conversion
- Handle both array and string inputs in create() and update() methods
- Fixes 'invalid input syntax for type json' error when creating/updating commanders
The pg library doesn't automatically convert JS arrays to JSON, so we must
stringify them before passing to PostgreSQL. The ::jsonb cast ensures proper
type conversion in the database.
* Fix JSON parsing in routes: PostgreSQL JSONB is already parsed
PostgreSQL's pg library automatically parses JSONB columns into JavaScript objects.
The routes were incorrectly calling JSON.parse() on already-parsed JSONB data,
which would fail or cause errors.
Fixed in:
- backend/src/routes/commanders.js (3 occurrences)
- backend/src/routes/games.js (3 occurrences)
- backend/src/routes/stats.js (1 occurrence)
Changed from: JSON.parse(colors) or JSON.parse(commander_colors)
Changed to: colors || [] or commander_colors || []
This matches how the models already handle JSONB data correctly.
* Fix seeds.sql: correct bcrypt hash for password123
The previous bcrypt hash was incorrect and did not match 'password123'.
Generated the correct hash using bcryptjs with 12 rounds.
Correct credentials for seeded test users:
- Username: testuser
Password: password123
- Username: magictg
Password: password123
This allows developers to login to the application with seeded data.
* Fix stats routes: convert from SQLite to PostgreSQL async methods
- Replace db.prepare().get() with await dbManager.get()
- Replace db.prepare().all() with await dbManager.all()
- Update parameterized query placeholders from ? to $1, $2, etc
- Change boolean comparisons from 'won = 1' to 'won = TRUE' for PostgreSQL
- Remove unnecessary db.initialize() calls
- Both /api/stats/overview and /api/stats/commanders now working correctly
* Fix games routes: remove SQLite boolean conversions and unnecessary JSON parsing
- Remove boolean-to-integer conversion (was converting true/false to 1/0)
- Remove JSON.parse() on JSONB colors column (PostgreSQL pg driver already parses JSONB)
- Fix in both POST create response and PUT update response
- Colors array now correctly returned as already-parsed JavaScript array
- Boolean fields now correctly returned as native boolean type
* Fix frontend: remove JSON.parse() on colors from API responses
- colors field is now pre-parsed array from PostgreSQL JSONB
- Simplified stats.html line 124: remove JSON.parse(stat.colors)
- Simplified dashboard.html line 279: remove defensive type checking for colors
- Frontend now properly handles colors as JavaScript arrays
* Simplify: remove defensive type checking for commanderColors in
dashboard
- game.commanderColors is always an array from PostgreSQL JSONB
- Changed from complex ternary to simple: game.commanderColors || []
* feat: improve environment variable handling in docker-compose and .env.example
- Add RATE_LIMIT_WINDOW and RATE_LIMIT_MAX to .env.example (commented for now)
- Update docker-compose.yml to use environment variables with defaults
- All DB_* variables now use default format
- NODE_ENV, JWT_SECRET, CORS_ORIGIN, LOG_LEVEL, ALLOW_REGISTRATION now respect env vars
- DB_SEED now uses environment variable
- Improves flexibility for development, testing, and production deployments
- Maintains backward compatibility with defaults
- Reduces hardcoded values and increases configurability
* fix: use DB_PASSWORD environment variable in postgres healthcheck
- PGPASSWORD in healthcheck was hardcoded to 'edh_password'
- Changed to use ${DB_PASSWORD:-edh_password} for consistency
- Ensures healthcheck respects DB_PASSWORD environment variable
- Fixes issue where custom DB_PASSWORD would cause healthcheck to fail
* fix: make PostgreSQL external port configurable via DB_PORT
- Changed postgres port mapping from hardcoded '5432:5432' to '${DB_PORT:-5432}:5432'
- Allows users to expose PostgreSQL on different external port via DB_PORT env variable
- Internal container port remains 5432 (unchanged)
- Enables non-standard port usage in constrained environments
- Maintains backward compatibility with default of 5432
* fix: update production docker-compose template in deploy.sh for environment variables
Changes to generated docker-compose.prod.deployed.yml:
Postgres Service:
- Added configurable external port: ${DB_PORT:-5432}:5432
- Ensures port mapping respects DB_PORT environment variable
DB-Migrate Service:
- DB_HOST: postgres -> ${DB_HOST:-postgres}
- DB_PORT: 5432 -> ${DB_PORT:-5432}
- DB_USER: postgres -> ${DB_USER:-postgres}
- Maintains configuration consistency with development
Backend Service:
- DB_HOST: postgres -> ${DB_HOST:-postgres}
- DB_PORT: 5432 -> ${DB_PORT:-5432}
- DB_USER: postgres -> ${DB_USER:-postgres}
- LOG_LEVEL: warn -> ${LOG_LEVEL:-warn}
- Removed hardcoded RATE_LIMIT_* variables (not used yet)
- All variables now properly parameterized
Documentation:
- Updated .env example to include DB_USER, LOG_LEVEL, DB_SEED
- Better guidance for production deployment
Ensures production deployments have same flexibility as development
* fix: update GitHub Actions workflow for PostgreSQL and environment variables
Postgres Service:
- POSTGRES_USER: edh_user -> postgres (matches .env.example and deploy.sh)
- POSTGRES_PASSWORD: change-this-in-production -> edh_password (matches .env.example)
- Added ports configuration: ${DB_PORT:-5432}:5432 (allows external access)
- Fixed healthcheck to use PGPASSWORD and proper variable syntax
DB-Migrate Service:
- DB_HOST: postgres -> ${DB_HOST:-postgres}
- DB_PORT: 5432 -> ${DB_PORT:-5432}
- DB_USER: edh_user -> postgres
- DB_PASSWORD: change-this-in-production -> edh_password
- Added DB_SEED=${DB_SEED:-false}
Backend Service:
- DB_HOST: postgres -> ${DB_HOST:-postgres}
- DB_PORT: 5432 -> ${DB_PORT:-5432}
- DB_USER: edh_user -> postgres
- DB_PASSWORD: change-this-in-production -> edh_password
- JWT_SECRET: removed unsafe default (must be provided)
- LOG_LEVEL: warn -> ${LOG_LEVEL:-warn}
Ensures GitHub Actions workflow is consistent with:
- docker-compose.yml (development)
- deploy.sh (production script)
- .env.example (configuration template)
* feat: implement global rate limiting and request/response logging
- Added rateLimitConfig to jwt.js with configurable window (minutes) and max requests
- Integrated global rate limiting into server.js using RATE_LIMIT_WINDOW and RATE_LIMIT_MAX env vars
- Default: 100 requests per 15 minutes (overridable via environment)
- Added request/response logging hooks for debugging (logged at debug level)
- Logs include method, URL, IP, status code, and duration
- Updated .env.example to document rate limiting configuration
* docs: update README for PostgreSQL migration and new features
- Updated intro to mention PostgreSQL instead of SQLite
- Added rate limiting and request logging features to infrastructure
section
- Updated Technology Stack to reflect PostgreSQL and rate-limiting
- Revised environment variables section with PostgreSQL config
- Added Custom Environment Variables section with examples
- Expanded Database section with PostgreSQL-specific details
- Added Tips & Common Operations for PostgreSQL management
- Updated Recent Changes to document Session 3 migration work
- Updated Development Notes for async database operations
- Added JSONB field documentation
* security: remove exposed PostgreSQL port from docker-compose
PostgreSQL no longer needs to be exposed to the host since:
- Backend container accesses postgres via internal Docker network
- DB_PORT=5432 is only for internal container connections, not port mapping
- Removes unnecessary attack surface in production
Changes:
- Removed 'ports:' section from postgres service in docker-compose.yml
- Removed port mapping from production deploy.sh template
- Clarified DB_PORT usage in .env.example (internal only)
- Added DB_USER to .env.example with explanation
Security Impact:
- PostgreSQL only accessible within Docker network
- Reduced container exposure to host network
- More secure production deployments
Tested:
- All services start successfully
- Backend connects to postgres via internal network
- Login works, database queries successful
- Frontend accessible on 8081, Backend on 3002
* refactor: remove hardcoded DB_PORT, use PostgreSQL standard port 5432
Simplified database configuration by removing configurable DB_PORT since
PostgreSQL always runs on standard port 5432:
Changes:
- Updated backend/src/config/database.js to hardcode port 5432
- Removed DB_PORT from all docker-compose services
- Removed DB_PORT from production deploy.sh template
- Updated .env.example with clearer documentation
- Clarified that port 5432 is not configurable
Benefits:
- Simpler configuration (fewer environment variables)
- Standard PostgreSQL port is expected behavior
- Reduced configuration surface area
- Still flexible: can adjust DB_HOST for different database servers
Tested:
- All services start successfully
- Database connections work via internal Docker network
- User authentication functional
- API endpoints respond correctly
* docs: update README to reflect DB_PORT removal and configuration simplification
Updated documentation to reflect latest changes:
- Removed DB_PORT from environment variables section (port 5432 is standard)
- Added note that PostgreSQL port is not configurable
- Clarified connection details (port 5432 is standard, not configurable)
- Updated project structure: postgres_data instead of database
- Added deployment script to project structure
- Updated Recent Changes section with configuration simplification details
- Added DB_SEED documentation to environment variables
- Improved clarity on which settings are configurable vs. standard
Emphasizes the security and simplicity improvements from removing
unnecessary port configuration.
* Remove migration docs and init scripts
* refactor: migrate routes from models to repositories
Replaced all data access layer calls in routes from Model classes to Repository classes.
Changes:
- auth.js: Now uses UserRepository instead of User model
* User.create() → UserRepository.createUser()
* User.findByUsername() → UserRepository.findByUsername()
* User.findById() → UserRepository.findById()
* User.verifyPassword() → UserRepository.verifyPassword()
* User.updatePassword() → UserRepository.updatePassword()
* User.updateUsername() → UserRepository.updateUsername()
* User.updateProfile() → UserRepository.updateProfile()
- commanders.js: Now uses CommanderRepository instead of Commander model
* Commander.create() → CommanderRepository.createCommander()
* Commander.findByUserId() → CommanderRepository.getCommandersByUserId()
* Commander.search() → CommanderRepository.searchCommandersByName()
* Commander.findById() → CommanderRepository.findById()
* Commander.update() → CommanderRepository.updateCommander()
* Commander.delete() → CommanderRepository.deleteCommander()
* Commander.getStats() → CommanderRepository.getCommanderStats()
* Commander.getPopular() → CommanderRepository.getPopularCommandersByUserId()
- games.js: Now uses GameRepository instead of Game model
* Game.findByUserId() → GameRepository.getGamesByUserId()
* Game.findById() → GameRepository.getGameById()
* Game.create() → GameRepository.createGame()
* Game.update() → GameRepository.updateGame()
* Game.delete() → GameRepository.deleteGame()
* Game.exportByUserId() → GameRepository.exportGamesByUserId()
Benefits:
✅ Clean separation of concerns (routes vs data access)
✅ Better testability (can mock repositories)
✅ More maintainable (database logic centralized)
✅ Consistent patterns across all data access
✅ Easier to add caching or logging layers
Testing:
✓ All syntax checks pass
✓ Authentication working
✓ Commanders endpoint returning 5 commanders
✓ Games endpoint returning 16 games
✓ All endpoints functional
* refactor: remove unused model classes
Models (User, Commander, Game) have been fully replaced by their
corresponding Repository classes. All functionality is preserved in
the repositories with no loss of capability or breaking changes.
Deleted files:
- User.js (136 lines)
- Commander.js (195 lines)
- Game.js (204 lines)
Total: ~535 lines of unused code removed
Benefits:
✅ Cleaner codebase - no duplicate data access logic
✅ Single source of truth - repositories handle all data access
✅ Better maintainability - clear separation of concerns
✅ No confusion - developers only use repositories
✅ Follows DRY principle - no code duplication
Testing:
✓ All routes verified to use repositories only
✓ All endpoints tested and working
✓ Authentication (8 endpoints)
✓ Commanders (7 endpoints)
✓ Games (6 endpoints)
✓ Stats (read-only)
No breaking changes - all functionality identical to before.
* Configure DB env defaults and add health checks
- Use DB_USER, DB_PASSWORD, and DB_NAME with defaults in deploy.sh and
docker-compose.yml
- Replace wget-based health check with curl to /health in the frontend
service
- Remove listen_addresses configuration from Postgres in
deploy/docker-compose
- Delete frontend/public/status.html
* Return camelCase game data and richer responses
* Add validation utilities and stricter schemas
* Update commanders.html
|
|||
| 2506ffd2be | Update publish.yml | |||
| a3f452c9ee | Update publish.yml | |||
| fc85985317 | Add deployment tooling and GHCR publish workflow |
