Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deletion of (large) projects takes too long #636

Closed
nscuro opened this issue Jun 28, 2023 · 1 comment · Fixed by DependencyTrack/hyades-apiserver#233
Closed

Deletion of (large) projects takes too long #636

nscuro opened this issue Jun 28, 2023 · 1 comment · Fixed by DependencyTrack/hyades-apiserver#233
Assignees
Labels
component/api-server defect Something isn't working p3 Nice-to-have features performance size/M Medium effort

Comments

@nscuro
Copy link
Member

nscuro commented Jun 28, 2023

When deleting a project via REST API, the API server will all related objects, and finally the project itself.

https://github.com/DependencyTrack/hyades-apiserver/blob/afe74d8f89a57a0c1c1f373ede45abf2f25eb64e/src/main/java/org/dependencytrack/persistence/ProjectQueryManager.java#L685-L714

Behind the scenes, the deletion happens via DataNucleus' deletePersistent or deletePersistentAll methods.

In order for DN to maintain its in-memory representation of the database, execute any query hooks etc., these methods will first fetch the objects before issuing a DELETE query to the database. Not only is this wasteful in terms of resources, it also causes the deletion of large projects to take very long. We've been seeing duration in the minute range.

We should investigate on using DataNucleus' Bulk Delete queries instead, which immediately issue a DELETE query to the database.

Similar to #635, it would be even better if the entire deletion would be transactional, so we're not leaving behind broken data in case of a failure mid-way.

@nscuro nscuro added defect Something isn't working p3 Nice-to-have features size/M Medium effort component/api-server performance labels Jun 28, 2023
@nscuro
Copy link
Member Author

nscuro commented Jun 30, 2023

We can work on this after #635

@nscuro nscuro added this to Hyades Jul 6, 2023
@nscuro nscuro moved this to Todo in Hyades Jul 6, 2023
@nscuro nscuro self-assigned this Jul 6, 2023
@nscuro nscuro moved this from Todo to In Progress in Hyades Jul 10, 2023
@nscuro nscuro added the in review Implementation is complete and currently in review label Jul 17, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Done in Hyades Jul 19, 2023
@nscuro nscuro removed the in review Implementation is complete and currently in review label Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/api-server defect Something isn't working p3 Nice-to-have features performance size/M Medium effort
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant