Skip to content

Don't load entire distribution into memory #7

@dstufft

Description

@dstufft

Currently this fetches the distribution from PyPI into a BytesIO object, after doing a requests.get() call (not streaming).

That means that while we're inside of _get_dist, we'll currently be using 2x the file size of the distribution worth of extra RAM, and outside of it we'll be using 1x the file size of extra RAM.

This should probably buffer to a temporary file and use streaming requests so that a large distribution doesn't kill us on memory.

This might just be #5 but I wanted to call it out explicitly since this applies even if we're storing the files somewhere.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions