Dynamically load connection modules for reduced memory usage #418
Conversation
Done in response to powersync-ja#410. Refactors module loading to dynamically import database connection and storage modules (MySQL, PostgreSQL) based on the configuration, rather than registering them all upfront. Signed-off-by: Najam Ahmed Ansari <ahmed.najamansari@gmail.com>
🦋 Changeset detectedLatest commit: b4b1923 The changes in this PR will be included in the next version bump. This PR includes changesets to release 11 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
|
rkistner
left a comment
rkistner
left a comment
There was a problem hiding this comment.
Could you confirm whether this approach actually helps to reduce memory usage?
|
@rkistner Thanks for taking a look! Yes there's definitely a little improvement when running the server against MongoDB in my fix vs the main branch baseline. On my branch:
On the baseline (main branch):
During both sets of tests I spun up fresh DB/BE/UI containers via Docker compose, started powersync-server via entry.ts in unified runner mode (default) and took snapshots at the 10/60/600 second marks. During this time, I opened up the todo app UI in two different browser windows and made small operations in each (creating a list, adding tasks to a list). |
- Split modules between storage and connection. - Cleaner code that now checks if given connection/storage types can be handled by us or not. - Error raised if any connection types cannot be handled or if any imports fail. - Explicit test case that simulates import failure for a module. Signed-off-by: Najam Ahmed Ansari <ahmed.najamansari@gmail.com>
Signed-off-by: Najam Ahmed Ansari <ahmed.najamansari@gmail.com>
4 participants
Fix for #410.
Module loading is moved to
service/src/util/module-loader.ts. This change hasbeen applied to all three runners (unified, API, sync).
Tests for the module loader have been added in
service/test/src/util/module-loader.test.ts.