Skip to content

Conversation

@JaffaKetchup
Copy link
Owner

@JaffaKetchup JaffaKetchup commented Feb 3, 2025

  • Converted import/export operations to use chunking instead of streaming
  • Improved efficiency of import/export operations

Fixes #178.

@JaffaKetchup
Copy link
Owner Author

JaffaKetchup commented Feb 3, 2025

The problem is that using manual chunking seems to grow the database rapidly for some reason, and uses considerably more memory, whereas streaming seems to be slightly slower, but uses much less memory and does not appear to grow the database at all.

Unfortunately, the growth caused by the chunking is quite significant (it seems to be the entire amount of the import stores' tiles), and that might make this unsuitable for production, especially as it cannot be shrunk afterward.

EDIT: I've reduced the memory growth rate significantly by using more transactions - whilst this does mean potentially slightly reduced performance, it appears not to be noticable. This also resolved the size growth of the database file.

@JaffaKetchup JaffaKetchup marked this pull request as ready for review March 9, 2025 18:27
@JaffaKetchup JaffaKetchup merged commit 022bbe0 into main Mar 9, 2025
6 checks passed
@JaffaKetchup JaffaKetchup deleted the import-fix branch March 9, 2025 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Import/export operations causing fatal crash on iOS

2 participants