Add Functionality to Stop a Running Job on the Harvest Job Page

[Jin-Sun-tts]

User Story

Implement a feature on the Harvest Job Page that allows users to stop a running job. Upon cancelling, the backend should handle job cancellation by updating the job's status in the HarvestDB job table and associated error table, marking the job as 'error'.

Acceptance Criteria

Frontend:

• Add a "Stop Job" button to the Harvest Job Page.
• The button should be disabled for completed or already stopped jobs.
• A confirmation modal should appear when the "Stop Job" button is clicked, asking the user to confirm the cancellation.

Backend:

• On job cancellation, update the job's status in the harvest_job table to "error".
• Update any relevant entries in the job_error table.
• Ensure that any running processes tied to the job are properly terminated.
• Return appropriate success or error responses to the frontend.

Testing:

• Write unit and integration tests to verify that stopping a job updates both the job and error tables.
• Ensure edge cases are tested (e.g., stopping already completed jobs, handling database errors).

[btylerburton]

We rewrite this to account for the gap created by marking jobs complete when they are in mid-run.

My thinking is instead of doing complicated calculations to update current job progress, we:

• mark the job COMPLETE
• whenever a new job runs it will cleanse the DB of any records that do not have a CKAN_ID, which means they didn't sync.
• This should account for any cases where the job doesn't terminate on its own

We should discuss this as a team

[nickumia]

@btylerburton Your solution sounds like a hack (which it may not be), but I don't think I know the exact motivation for "stop" action. Also don't know what type of "job" this is and how it can be manually stopped otherwise.

Some possible speculations:

• From a user perspective, totally want to stop a job if I hit a button by accident or if something was misconfigured and the current job is meaningless.
• From a devops/admin perspective, totally want to stop a job if it'll waste a bunch of resources and needs to be changed or not run right now.

In both of those cases, they don't necessarily care about the backend of how it works. But as a devops, totally don't want to kill things in a way that will create incomplete or bad data states (i.e. leftover data that needs to be cleaned up at some point).

Skipping the complicated calculations to update current job progress sounds like a development tradeoff, but technical debt doesn't seem worth that decision. I think this ties to a different comment that I made,

• Dealing with large harvest source with high data volatility  #5022 (comment)

Main concern is data duplication that Data.gov has been struggling with for you-know-how-long. If the ADD-DELETE-MODIFIED paradigm still exists, that should drive this implementation too. If data should not be committed anywhere unless it's everywhere it needs to be whereby it can be searched and compared easily.

Probably didn't help, but that's by 🪙🪙

P.S. Love the discuss label which is the only way I found this ticket, you guys should use that label more or create a public label for outsiders to know what they can join in the conversation about (for the once in a blue moon that I check out the board haha...)