Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: prevent OOM when processing large gradle project with many depencies #12

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: prevent OOM when processing large gradle project with many depencies #12

wants to merge 1 commit into from

Conversation

dteller
Copy link

@dteller dteller commented May 11, 2023

  • [X ] All tests passed. If this feature is not already covered by the tests, I added new
    tests.

We have a gradle project with 195 modules that have many interdependencies.
The plugin uses over 100G of memory and eventually fails.
After making these changes it ran quickly in little memory.

@github-actions
Copy link

github-actions bot commented May 11, 2023 •
edited
Loading

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@dteller
Copy link
Author

dteller commented May 11, 2023

I have read the CLA Document and I hereby sign the CLA

@dteller
Copy link
Author

dteller commented May 11, 2023

recheck

1 similar comment
@dteller
Copy link
Author

dteller commented May 11, 2023

recheck

@dteller
Copy link
Author

dteller commented May 11, 2023

PS: Not sure why it is complaining that I haven't signed the CLA.

@asafgabai
Copy link
Contributor

Hi @dteller, thank you for your contribution!
We're aware of the memory consumption issue that you encountered and have already thought of a solution.
As you already noticed, the issue typically occurs in projects that have numerous repeating dependencies, resulting in high memory consumption and storage usage for the result files.
Although your pull request resolves the problem, it introduces another issue where each dependency only appears once in the tree.
To address both issues, we've decided to flatten the tree into a list of nodes, where each node points to its children. This approach will resolve the memory issue since each node will only appear once, while retaining all the paths from the root to each node.
A pull request for this change will be opened soon.

@asafgabai
Copy link
Contributor

Hi @dteller, the fix for the high memory consumption has been implemented and is now part of version 3.0.0.
Please note that the output structure of the plugin has been changed.
Please let us know if you encounter any further issues or have any additional feedback.

@dteller
Copy link
Author

dteller commented Jun 27, 2023 via email

@asafgabai
Copy link
Contributor

It's going to be incorporated into JFrog IntelliJ IDEA Plugin, which is scheduled for release in the coming days. We plan to include it also in JFrog CLI and Frogbot soon.

@asafgabai
Copy link
Contributor

@dteller I opened a pull request for using this in JFrog CLI.
I'll update here when it's released.

@dteller
Copy link
Author

dteller commented Sep 28, 2023 via email

@asafgabai
Copy link
Contributor

@dteller JFrog CLI version 2.49.0 is now available, incorporating the latest gradle-dep-tree version with enhanced memory efficiency. We value your input and welcome your feedback on this release.

@dteller
Copy link
Author

dteller commented Oct 5, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dteller @asafgabai