Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

engine: river job queue #4135

Merged
merged 27 commits into from
Nov 12, 2024
Merged

engine: river job queue #4135

merged 27 commits into from
Nov 12, 2024

Conversation

mastercactapus
Copy link
Member

@mastercactapus mastercactapus commented Nov 11, 2024

Description:
Adds River job queue

  • local dev only shows the UI, when it's used more maybe this should be always on
  • part of a larger plan to move away from custom 5 sec engine cycles to proper job processing
  • not currently used to keep PR small, signals work will be the first, since that's still experimental
  • it's recommended to re-use the same DB which works well for us, migrations are imported directly (thus treated as "generated" code)

- Remove multiple instances of `sql.DB` and `Connector` usage.
- Switch to using `pgxpool` for database connections consistently.
- Simplify the shutdown process by ensuring proper resource cleanup.
- Implement a new lock querying method for the switchover state.
- Delete unused `connector.go` as it is no longer needed.
- Replace manual retry logic with higher-level config setup
- Remove custom retry driver and connector implementations
- Simplify code by using pgxpool's built-in configuration options
- Implement SWOConnUnlockAll to release all locks
- Configure PGX pool to unlock upon release of connections
- Eliminated import of the unused time package.
- Removed unnecessary pool configuration for health checks and connection idle time to simplify code.
- Updated to sqlc v1.27.0
- Removed sql-migrate dependency
- Enhanced parsing logic for migrations without external library
- Adjusted file mode representation to octal style
- Initialize River client and UI server in the app
- Add River-related error handling
- Update HTTP routing for River UI access
- Enhance navigation with a link to the River UI
- Create database migrations for River job management
KatieMSB
KatieMSB previously approved these changes Nov 11, 2024
m17ch
m17ch previously approved these changes Nov 12, 2024
m17ch
m17ch previously approved these changes Nov 12, 2024
KatieMSB
KatieMSB previously approved these changes Nov 12, 2024
@mastercactapus mastercactapus dismissed stale reviews from KatieMSB and m17ch via 66e432b November 12, 2024 17:08
m17ch
m17ch previously approved these changes Nov 12, 2024
KatieMSB
KatieMSB previously approved these changes Nov 12, 2024
@mastercactapus mastercactapus dismissed stale reviews from KatieMSB and m17ch via af286fa November 12, 2024 17:20
@mastercactapus mastercactapus merged commit ceb3f44 into master Nov 12, 2024
8 checks passed
@mastercactapus mastercactapus deleted the river-job-queue branch November 12, 2024 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants