Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fixed memory leak and added more profiling middlewares #211

Merged
merged 1 commit into from
Nov 7, 2024

Conversation

TeKrop
Copy link
Owner

@TeKrop TeKrop commented Nov 7, 2024

Summary by Sourcery

Fix memory leak and enhance profiling capabilities by adding support for multiple profiling middlewares. Update project dependencies to include new profiling tools.

Bug Fixes:

  • Fix memory leak by ensuring proper handling of temporary files and memory snapshots in profiling middlewares.

Enhancements:

  • Refactor profiling middleware to support multiple profilers, including Memray, PyInstrument, TraceMalloc, and ObjGraph.

Build:

  • Update project dependencies to include new profiling tools: Memray and ObjGraph.

@TeKrop TeKrop added bug Something isn't working enhancement New feature or request labels Nov 7, 2024
@TeKrop TeKrop self-assigned this Nov 7, 2024
Copy link
Contributor

sourcery-ai bot commented Nov 7, 2024

Reviewer's Guide by Sourcery

This PR addresses a memory leak and enhances profiling capabilities by introducing multiple profiling middlewares. The implementation replaces the single PyInstrument profiler with a more flexible middleware system that supports multiple profiling tools (Memray, PyInstrument, Tracemalloc, and ObjGraph). Additionally, it fixes memory leaks in the player career parser by properly handling BeautifulSoup string extractions.

Sequence diagram for profiling middleware selection

sequenceDiagram
    actor User
    participant App
    participant Logger
    participant Middleware
    User->>App: Send request with profile query
    App->>Logger: Log "Profiling is enabled"
    App->>Middleware: Add selected profiler middleware
    Middleware-->>App: Middleware added
    App-->>User: Processed request with profiling
Loading

ER diagram for Profiler enum changes

erDiagram
    Profiler {
        string MEMRAY
        string PYINSTRUMENT
        string TRACEMALLOC
        string OBJGRAPH
    }
    Settings {
        string profiler
    }
    Profiler ||--o{ Settings : "used by"
Loading

Class diagram for new profiling middlewares

classDiagram
    class OverFastMiddleware {
        +dispatch(Request, Callable) HTMLResponse|JSONResponse
        +_dispatch(Request, Callable) HTMLResponse|JSONResponse
    }
    class MemrayInMemoryMiddleware {
        +_dispatch(Request, Callable) HTMLResponse
        +generate_html_report(Path) str
    }
    class PyInstrumentMiddleware {
        +_dispatch(Request, Callable) HTMLResponse
    }
    class TraceMallocMiddleware {
        +_dispatch(Request, Callable) JSONResponse
    }
    class ObjGraphMiddleware {
        +_dispatch(Request, Callable) JSONResponse
    }
    OverFastMiddleware <|-- MemrayInMemoryMiddleware
    OverFastMiddleware <|-- PyInstrumentMiddleware
    OverFastMiddleware <|-- TraceMallocMiddleware
    OverFastMiddleware <|-- ObjGraphMiddleware
Loading

File-Level Changes

Change Details Files
Implemented a new profiling middleware system with multiple profiler options
  • Created a new base OverFastMiddleware class for common profiling functionality
  • Added support for Memray profiler for memory tracking
  • Added support for PyInstrument profiler for performance profiling
  • Added support for Tracemalloc profiler for memory leak detection
  • Added support for ObjGraph profiler for object tracking
  • Replaced boolean profiling setting with a profiler selection enum
app/middlewares.py
app/main.py
app/enums.py
app/config.py
Fixed memory leaks in BeautifulSoup parsing
  • Modified string extraction to use contents[0] instead of string property
  • Removed SoupStrainer usage in favor of direct tag finding
  • Updated parser to properly handle HTML content extraction
app/players/parsers/player_career_parser.py
app/parsers.py
Updated project dependencies and configuration
  • Added new profiling tool dependencies (memray, objgraph)
  • Bumped version to 3.2.1
  • Updated development dependencies
pyproject.toml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time. You can also use
    this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

sonarqubecloud bot commented Nov 7, 2024

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @TeKrop - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟡 General issues: 3 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟡 Complexity: 1 issue found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

app/middlewares.py Show resolved Hide resolved
app/parsers.py Show resolved Hide resolved
app/middlewares.py Show resolved Hide resolved
app/middlewares.py Show resolved Hide resolved
@TeKrop TeKrop merged commit da6d501 into main Nov 7, 2024
3 checks passed
@TeKrop TeKrop deleted the bugfix/memory-performances branch November 7, 2024 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant