feat(pglite/sync): Add commitGranularity and onInitialSync parame…

…ters to `syncShapeToTable` (#456) * Add `commitGranularity` parameter in the `syncShapeToTable` * Fix tests * Docs * add tests for commitGranularity * Add test of the commitThrottle option * Fix onInitialSync and add test * doc: export return type for syncShapeToTable (#452) * doc: export return type for syncShapeToTable * feat: expose stream object as of for useShape hook * Changeset --------- Co-authored-by: Sam Willis <[email protected]> * Remove part of test thats flaky on CI * formatting * Fix types * Remove transaction commit granularity until Electric has stabilised on LSN metadata * Post review changes --------- Co-authored-by: Yacine <[email protected]>
electric-sql · Dec 9, 2024 · 46c102c · 46c102c
1 parent 515a155
commit 46c102c
Show file tree

Hide file tree

Showing 4 changed files with 791 additions and 123 deletions.
diff --git a/.changeset/happy-jokes-yawn.md b/.changeset/happy-jokes-yawn.md
@@ -0,0 +1,5 @@
+---
+'@electric-sql/pglite-sync': patch
+---
+
+Add options for the `commitGranularity` parameter in the `syncShapeToTable` function, enabling the user to choose how often the sync should commit.
diff --git a/docs/docs/sync.md b/docs/docs/sync.md
@@ -94,6 +94,21 @@ It takes the following options as an object:
 - `useCopy: boolean`<br>
   Whether to use the `COPY FROM` command to insert the initial data, defaults to `false`. This process may be faster than inserting row by row as it combines the inserts into a CSV to be passed to Postgres.
 
+- `commitGranularity: CommitGranularity`<br>
+  The granularity of the commit operation, defaults to `"up-to-date"`. Note that a commit will always be performed immediately on the `up-to-date` message.
+  Options:
+
+  - `"up-to-date"`: Commit all messages when the `up-to-date` message is received.
+  <!-- - `"transaction"`: Commit all messages within transactions as they were applied to the source Postgres. -->
+  - `"operation"`: Commit each message in its own transaction.
+  - `number`: Commit every N messages.
+
+- `commitThrottle: number`<br>
+  The number of milliseconds to wait between commits, defaults to `0`.
+
+- `onInitialSync: () => void`<br>
+  A callback that is called when the initial sync is complete.
+
 The returned `shape` object from the `syncShapeToTable` call has the following methods:
 
 - `isUpToDate: boolean`<br>