Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

engine/cleanupmgr: finalize migration to job queue for cleanup operations #4218

Merged
merged 48 commits into from
Dec 26, 2024
Merged

engine/cleanupmgr: finalize migration to job queue for cleanup operations #4218

merged 48 commits into from
Dec 26, 2024

Conversation

mastercactapus
Copy link
Member

Description:
This pull request finalizes the transition to the job queue system for the cleanup manager module.

  • Adds new worker method for handling API keys
  • Adds a daily periodic job to do the cleanup
  • Removes old code & SQL statements

@mastercactapus
cleanup: implement context-aware sleep and cleanup manager functionality
0c3dbd6
@mastercactapus
cleanup: format code for consistency in alert store initialization
9cc35dd
@mastercactapus
cleanup: remove unused ConfigSource field from SetupArgs struct
e5c72d7
@mastercactapus
Merge branch 'master' into cleanup-mgr-alert-jobs
a208406
@mastercactapus
cleanup: refactor alert cleanup logic and add shift cleanup functiona…
7ba52bb 
…lity
@mastercactapus
cleanup: refactor alert cleanup logic to use whileWork for better con…
bb0b63e 
…trol flow
@mastercactapus
Merge branch 'cleanup-mgr-alert-jobs' into cleanup-mgr-shifts
a4cc12c
@mastercactapus
cleanup: add comment
859194a
@mastercactapus
Merge branch 'cleanup-mgr-alert-jobs' into cleanup-mgr-shifts
66b17bf
@mastercactapus
cleanup: remove unused cleanup statements from DB and update logic
be2a5d1
@mastercactapus
test: refactor alert auto-close and cleanup tests for improved reliab…
f3bf6dd 
…ility
@mastercactapus
Merge branch 'cleanup-mgr-alert-jobs' into cleanup-mgr-shifts
7f6a499
@mastercactapus
Merge branch 'master' into cleanup-mgr-shifts
145d1c7 
# Conflicts:
#	engine/cleanupmanager/db.go
#	engine/cleanupmanager/queries.sql
#	gadb/queries.sql.go
@mastercactapus
fix: update ShiftArgs Kind to reflect cleanup-manager-shifts
eeea937
@mastercactapus
Merge branch 'master' into cleanup-mgr-shifts
7976e6c
@mastercactapus
feat: add periodic jobs for schedule data cleanup and update queries
26523b7
@mastercactapus
Merge branch 'master' into cleanup-mgr-schedule-data
ff5d164 
# Conflicts:
#	engine/cleanupmanager/queries.sql
#	engine/cleanupmanager/update.go
@mastercactapus
feat: add workers for cleanup of shifts and schedule data
2a28887
@mastercactapus
fix: update periodic job interval for schedule data cleanup to 24 hours
f8c555d
@mastercactapus
feat: add logging support to cleanup manager and engine initialization
9ee4383
@mastercactapus
refactor: streamline schedule data cleanup by extracting user validat…
8494e91 
…ion and shift trimming logic
@mastercactapus
feat: add logging for schedule data updates in cleanup manager
fb7a782
@mastercactapus
refactor: remove unnecessary checks for empty shifts in user and shif…
fdad66d 
…t trimming functions
@mastercactapus
refactor: enhance schedule data cleanup by improving user validation …
2c9ffe6 
…and logging
@mastercactapus
refactor: improve formatting of schedule data update call in cleanup …
8592ff8 
…manager
@mastercactapus
docs: add comment to clarify cleanupData function purpose in schedule…
86f7b84 
… data management
@mastercactapus
feat: add CleanupAlertLogs function to manage alert log entries for d…
0899d6c 
…eleted alerts
@mastercactapus
refactor: remove unused cleanupAlertLogs statement and related logic …
df8cdcb 
…from cleanup manager
@mastercactapus
feat: implement timeout for CleanupAlertLogs worker to handle longer …
634bd2b 
…job durations
@mastercactapus
refactor: rename cleanupDays to more descriptive staleThresholdDays i…
dff614d 
…n cleanup manager functions
@mastercactapus
docs: enhance comment for CleanupMgrScheduleData to clarify last_clea…
bd31e64 
…nup_at usage
@mastercactapus
Merge branch 'master' into cleanup-mgr-schedule-data
ab61a03
@mastercactapus
Merge branch 'cleanup-mgr-schedule-data' into cleanup-mgr-alertlog-jo…
a056d48 
…b-queue
@mastercactapus
engine/cleanupmgr: refactor schedule data cleanup logic and add job f…
fd5ef14 
…or looking up schedules needing cleanup
@mastercactapus
Merge branch 'cleanup-mgr-sched-data-individual-jobs' into cleanup-mg…
28896c5 
…r-alertlog-job-queue
@mastercactapus
engine/cleanupmgr: implement alert log cleanup job scheduling and ref…
b85bbdb 
…actor related queries
@mastercactapus
engine/initriver: remove unused noopWorker type
aa505b2
@mastercactapus
engine/cleanupmgr: rename LookForWorkArgs types for consistency
4bee028
@mastercactapus
Merge branch 'master' into cleanup-mgr-sched-data-individual-jobs
00155fb
@mastercactapus
Merge branch 'cleanup-mgr-sched-data-individual-jobs' into cleanup-mg…
c4962e9 
…r-alertlog-job-queue
@mastercactapus
Merge branch 'master' into cleanup-mgr-alertlog-job-queue
4158efd 
# Conflicts:
#	engine/cleanupmanager/setup.go
@mastercactapus
feat(cleanupmanager): implement API key cleanup functionality
acc14da
@mastercactapus
feat(cleanupmanager): add periodic job for API key cleanup and remove…
77a229c 
… unused update logic
@mastercactapus
Merge branch 'master' into cleanup-mgr-api-keys
d8e3203 
# Conflicts:
#	engine/cleanupmanager/alertlogs.go
#	engine/cleanupmanager/db.go
#	engine/cleanupmanager/queries.sql
#	engine/cleanupmanager/setup.go
#	engine/cleanupmanager/update.go
#	gadb/queries.sql.go
@mastercactapus
fix merge issue
256dd0f
@mastercactapus
fix merge issue
483ebc9
@mastercactapus
regen
a63d12b
@mastercactapus
fix arg type
75e2825
@mastercactapus mastercactapus marked this pull request as ready for review December 23, 2024 23:21
CameronJHall
CameronJHall approved these changes Dec 26, 2024
View reviewed changes
Copy link
Collaborator

@CameronJHall CameronJHall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for including some of the readability improvements from previous PRs.

@mastercactapus mastercactapus merged commit 1ac513f into master Dec 26, 2024
9 checks passed
@mastercactapus mastercactapus deleted the cleanup-mgr-api-keys branch December 26, 2024 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AlaricWhitney AlaricWhitney AlaricWhitney approved these changes

@CameronJHall CameronJHall CameronJHall approved these changes

Assignees
No one assigned
Labels
size/m
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mastercactapus @CameronJHall @AlaricWhitney