Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ We use the following icons for supported backend databases and SDKs:
- Postgres: `icon="elephant"`
- MongoDB: `icon="leaf"`
- MySQL: `icon="dolphin"`
- SQL Server: `icon="server"`
- Flutter: `icon="flutter"`
- React Native: `icon="react"`
- Web: `icon="js"`
Expand Down
20 changes: 5 additions & 15 deletions architecture/architecture-overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,33 +10,23 @@ description: "The core components of PowerSync are the service and client SDKs"
The [PowerSync Service](/architecture/powersync-service) and client SDK operate in unison to keep client-side SQLite databases in sync with a backend database. Learn about their architecture:

<CardGroup cols="2">
<Card title="PowerSync Service" icon="server" horizontal href="/architecture/powersync-service">

</Card>
<Card title="Client Architecture" icon="microchip" horizontal href="/architecture/client-architecture">

</Card>
<Card title="PowerSync Service" icon="server" horizontal href="/architecture/powersync-service" />
<Card title="Client Architecture" icon="microchip" horizontal href="/architecture/client-architecture" />
</CardGroup>

### Protocol

Learn about the sync protocol used between PowerSync clients and a [PowerSync Service](/architecture/powersync-service):

<CardGroup cols="2">
<Card title="PowerSync Protocol" icon="network-wired" horizontal href="/architecture/powersync-protocol">

</Card>
<Card title="Consistency" icon="scale-balanced" horizontal href="/architecture/consistency">

</Card>
<Card title="PowerSync Protocol" icon="network-wired" horizontal href="/architecture/powersync-protocol" />
<Card title="Consistency" icon="scale-balanced" horizontal href="/architecture/consistency" />
</CardGroup>

### Self-Hosted Architecture

For more details on typical architecture of a production self-hosted deployment, see here:

<CardGroup cols="2">
<Card title="Installation" icon="download" horizontal href="/self-hosting/installation">

</Card>
<Card title="Installation" icon="download" horizontal href="/self-hosting/installation" />
</CardGroup>
2 changes: 1 addition & 1 deletion architecture/powersync-protocol.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,4 @@ Write checkpoints are used to ensure clients have synced their own changes back

Creating a write checkpoint is a separate operation, which is performed by the client after all data has been uploaded. It is important that this happens after the data has been written to the backend source database.

The server then keeps track of the current CDC stream position on the database (LSN in Postgres, resume token in MongoDB, or GTID + Binlog Position in MySQL), and notifies the client when the data has been replicated, as part of checkpoint data in the normal data stream.
The server then keeps track of the current CDC stream position on the database (LSN in Postgres & SQL Server, resume token in MongoDB and GTID + Binlog Position in MySQL), and notifies the client when the data has been replicated, as part of checkpoint data in the normal data stream.
Binary file modified images/architecture/powersync-docs-diagram-architecture-overview.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion installation/app-backend-setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ PowerSync generally assumes that you have some kind of "backend application" as

When you integrate PowerSync into your app project, PowerSync relies on that "backend application" for a few purposes:

1. **Allowing client-side write operations to be uploaded** and [applied](/installation/app-backend-setup/writing-client-changes) to the backend database (Postgres, MongoDB or MySQL). When you write to the client-side SQLite database provided by PowerSync, those writes are also placed into an upload queue. The PowerSync Client SDK manages uploading of those writes to your backend using the `uploadData()` function that you defined in the [Client-Side Setup](/installation/client-side-setup/integrating-with-your-backend) part of the implementation. That `uploadData()` function should call your backend application API to apply the writes to your backend database. The reason why we designed PowerSync this way is to give you full control over things like data validation and authorization of writes, while PowerSync itself requires minimal permissions.
1. **Allowing client-side write operations to be uploaded** and [applied](/installation/app-backend-setup/writing-client-changes) to the backend database (Postgres, MongoDB, MySQL, or SQL Server). When you write to the client-side SQLite database provided by PowerSync, those writes are also placed into an upload queue. The PowerSync Client SDK manages uploading of those writes to your backend using the `uploadData()` function that you defined in the [Client-Side Setup](/installation/client-side-setup/integrating-with-your-backend) part of the implementation. That `uploadData()` function should call your backend application API to apply the writes to your backend database. The reason why we designed PowerSync this way is to give you full control over things like data validation and authorization of writes, while PowerSync itself requires minimal permissions.
2. **Authentication integration:** Your backend is responsible for securely generating the [JWTs](/installation/authentication-setup) used by the PowerSync Client SDK to authenticate with the [PowerSync Service](/architecture/powersync-service).

<Frame caption="An overview of how PowerSync interacts with your backend application.">
Expand Down
4 changes: 2 additions & 2 deletions installation/app-backend-setup/writing-client-changes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ Since you get to define the client-side `uploadData()` function as you wish, you
You can also use any API style you want — e.g. REST, GraphQL, gRPC, etc.

<Warning>
It's important that your API endpoint be blocking/synchronous with underlying writes to the backend database (Postgres, MongoDB or MySQL).
It's important that your API endpoint be blocking/synchronous with underlying writes to the backend database (Postgres, MongoDB, MySQL, or SQL Server).

In other words, don't place writes into something like a queue for processing later — process them immediately. For more details, see the explainer below.
</Warning>

<Accordion title="Why must my write endpoint be synchronous?">
PowerSync uses a server-authoritative architecture with a checkpoint system for conflict resolution and [consistency](/architecture/consistency). The client advances to a new write checkpoint after uploads have been processed, so if the client believes that the server has written changes into your backend database (Postgres, MongoDB or MySQL), but the next checkpoint does not contain your uploaded changes, those changes will be removed from the client. This could manifest as UI glitches for your end-users, where the changes disappear from the device for a few seconds and then re-appear.
PowerSync uses a server-authoritative architecture with a checkpoint system for conflict resolution and [consistency](/architecture/consistency). The client advances to a new write checkpoint after uploads have been processed, so if the client believes that the server has written changes into your backend database (Postgres, MongoDB, MySQL, or SQL Server), but the next checkpoint does not contain your uploaded changes, those changes will be removed from the client. This could manifest as UI glitches for your end-users, where the changes disappear from the device for a few seconds and then re-appear.
</Accordion>

### Write operations recorded on the client
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ After you've [instantiated](/installation/client-side-setup/instantiate-powersyn

| Purpose | Description |
|---------|-------------|
| **Uploading writes to your backend:** | Writes that are made to the client-side SQLite database are uploaded to your backend application, where you control how they're applied to your backend database (Postgres, MongoDB or MySQL). This is how PowerSync achieves bi-directional syncing of data. |
| **Uploading writes to your backend:** | Writes that are made to the client-side SQLite database are uploaded to your backend application, where you control how they're applied to your backend database (Postgres, MongoDB, MySQL, or SQL Server). This is how PowerSync achieves bi-directional syncing of data. |
| **Authentication integration:** | PowerSync uses JWTs for authentication between the Client SDK and PowerSync Service. Your backend application should be able to generate JWTs that the PowerSync Client SDK can retrieve and use for authentication against your [PowerSync Service](/architecture/powersync-service) instance. |

Accordingly, you must pass a _backend connector_ as an argument when you call `connect()` on the client-side PowerSync database. You must define that backend connector, and it must implement two functions/methods:
Expand Down
23 changes: 23 additions & 0 deletions installation/database-connection.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -228,3 +228,26 @@ For other providers and self-hosted databases:
Make sure that your MySQL database allows access to PowerSync's IPs — see [Security and IP Filtering](/installation/database-setup/security-and-ip-filtering)
</Info>

## <Icon icon="server" iconType="solid" size={32}/> SQL Server (Alpha) Specifics

1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connections** view.
2. Click **Connect to Source Database** and ensure the **"SQL Server"** tab is selected.
3. Fill in your SQL Server connection details:
1. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" are required.
2. "**Name**" can be any name for the connection.
3. "**Host**" is the endpoint for your SQL Server instance.
4. "**Port**" is typically 1433 for SQL Server (default port).
5. "**Database name**" is the database where CDC is enabled.
6. "**Username**" and "**Password**" maps to the database user created in [Source Database Setup](/installation/database-setup#sql-server-alpha) (e.g., `powersync_user`).
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.

PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.

<Info>
Make sure that your SQL Server database allows access to PowerSync's IPs — see [Security and IP Filtering](/installation/database-setup/security-and-ip-filtering)
</Info>

Also see:
- [SQL Server Setup](/installation/database-setup#sql-server-alpha)

186 changes: 185 additions & 1 deletion installation/database-setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
sidebarTitle: "Overview"
---

Jump to: [Postgres](#postgres) | [MongoDB](#mongodb) | [MySQL](#mysql-beta)
Jump to: [Postgres](#postgres) | [MongoDB](#mongodb) | [MySQL](#mysql-beta) | [SQL Server](#sql-server-alpha)

import PostgresPowerSyncUser from '/snippets/postgres-powersync-user.mdx';
import PostgresPowerSyncPublication from '/snippets/postgres-powersync-publication.mdx';
Expand Down Expand Up @@ -399,6 +399,190 @@
binlog-ignore-db=user_db
```

## <Icon icon="server" iconType="solid" size={32} /> SQL Server (Alpha)

<Check>
**Version compatibility**: PowerSync requires SQL Server 2022+ or Azure SQL Database.
</Check>

PowerSync replicates data from SQL Server using Change Data Capture (CDC). This process builds up change tables based on changes to tracked tables. The change tables are populated by reading from the SQL Server transaction log on a fixed interval. PowerSync then polls these CDC tables for changes using built-in stored procedures.

For more information about CDC, see:
- [Change Data Capture (SQL Server)](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-data-capture-sql-server) - for SQL Server
- [Change Data Capture (Azure SQL Database)](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql) - for Azure SQL Database

### Supported Editions/Versions

| Database | Edition | Version | Min Service Tier |
|------------------|---------------------------------------------|---------|---------------------------------------------------------------------------------------|
| SQL Server 2022+ | Standard, Enterprise, Developer, Evaluation | 16.0+ | N/A |
| Azure SQL* | Database, Managed instance | N/A | Any service tier on Vcore subscription model. S3 tier and up on DTU purchasing model. |

Check warning on line 419 in installation/database-setup.mdx

View check run for this annotation

Mintlify / Mintlify Validation (powersync) - vale-spellcheck

installation/database-setup.mdx#L419

Did you really mean 'Vcore'?

\* Azure SQL Database is always running on the latest version of the SQL Server DB Engine

### Database Setup Requirements

#### 1. Enable CDC on the Database

Change Data Capture (CDC) needs to be enabled on the database:

```sql
-- Enable CDC on the database if not already enabled
USE <YOUR_DATABASE_NAME>;
IF (SELECT is_cdc_enabled FROM sys.databases WHERE name = '<YOUR_DATABASE_NAME>') = 0
BEGIN
EXEC sys.sp_cdc_enable_db;
END
GO
```

#### 2. Create the PowerSync Checkpoints Table

PowerSync requires a `_powersync_checkpoints` table to generate regular checkpoints. CDC must be enabled for this table:

```sql
-- Create the PowerSync checkpoints table in your schema
IF OBJECT_ID('<YOUR_SCHEMA>._powersync_checkpoints', 'U') IS NULL
BEGIN
CREATE TABLE <YOUR_SCHEMA>._powersync_checkpoints (
id INT IDENTITY PRIMARY KEY,
last_updated DATETIME NOT NULL DEFAULT GETUTCDATE()
);
END

-- Enable CDC for the powersync checkpoints table if not already enabled
-- Note: the cdc_reader role created the first time CDC is enabled on a table
IF NOT EXISTS (SELECT 1 FROM cdc.change_tables WHERE source_object_id = OBJECT_ID(N'<YOUR_SCHEMA>._powersync_checkpoints'))
BEGIN
EXEC sys.sp_cdc_enable_table
@source_schema = N'<YOUR_SCHEMA>',
@source_name = N'_powersync_checkpoints',
@role_name = N'cdc_reader',
@supports_net_changes = 0;
END
GO
```

#### 3. Enable CDC on Tables

CDC must be enabled for all tables that need to be replicated:

```sql
-- Enable CDC for specific tables in your schema if not already enabled
IF NOT EXISTS (SELECT 1 FROM cdc.change_tables WHERE source_object_id = OBJECT_ID(N'<YOUR_SCHEMA>.<YOUR_TABLE_NAME>'))
BEGIN
EXEC sys.sp_cdc_enable_table
@source_schema = N'<YOUR_SCHEMA>',
@source_name = N'<YOUR_TABLE_NAME>',
@role_name = N'cdc_reader',
@supports_net_changes = 0;
END
GO
```

Repeat this for each table you want to replicate. Note that PowerSync does not currently use the net changes functionality so `@supports_net_changes` can be set to `0`.

#### 4. Create Database User

Create a database user for PowerSync with the following permissions:

**Required permissions:**
- Read/Write permissions on the `_powersync_checkpoints` table
- Read permissions on the replicated tables
- `cdc_reader` role (grants access to CDC tables and functions)
- `VIEW DATABASE PERFORMANCE STATE` (SQL Server and Azure SQL)
- `VIEW SERVER PERFORMANCE STATE` (SQL Server only)

```sql
-- Create a SQL login for the powersync_user if missing. Note SQL Logins are created at the server level.
USE [master];
IF NOT EXISTS (SELECT 1 FROM sys.server_principals WHERE name = '<YOUR_DB_USER>')
BEGIN
CREATE LOGIN [<YOUR_DB_USER>] WITH PASSWORD = 'YOUR_DB_USER_PASSWORD', CHECK_POLICY = ON;
END
GO

-- Create a DB user for PowerSync if missing. Note DB users are created at the database level.
USE [<YOUR_DATABASE_NAME>];
IF NOT EXISTS (SELECT 1 FROM sys.database_principals WHERE name = '<YOUR_DB_USER>')
BEGIN
CREATE USER [<YOUR_DB_USER>] FOR LOGIN [<YOUR_DB_USER>];
END
GO

-- Grant read/write access to the PowerSync checkpoints table
GRANT SELECT, INSERT, UPDATE ON <YOUR_SCHEMA>._powersync_checkpoints TO [<YOUR_DB_USER>];
GO

-- Grant read access to all tables in the database using the db_datareader role (Less restrictive)
IF IS_ROLEMEMBER('db_datareader', '<YOUR_DB_USER>') = 0
BEGIN
ALTER ROLE db_datareader ADD MEMBER <YOUR_DB_USER>;
END
GO

-- Grant access to CDC tables and functions using the cdc_reader role
IF IS_ROLEMEMBER('cdc_reader', '<YOUR_DB_USER>') = 0
BEGIN
ALTER ROLE cdc_reader ADD MEMBER <YOUR_DB_USER>;
END
GO

-- Grant performance state permissions

-- Note: For Azure SQL, only VIEW DATABASE PERFORMANCE STATE is required. Granted at the database level.
-- PowerSync uses this to access the sys.dm_db_log_stats DMV and the sys.dm_db_partition_stats DMV
USE [<YOUR_DATABASE_NAME>];
GRANT VIEW DATABASE PERFORMANCE STATE TO [<YOUR_DB_USER>];
GO

-- VIEW SERVER PERFORMANCE STATE is only available on SQL Server (not Azure SQL). Granted at the server level.
-- PowerSync requires this permission to access the sys.dm_db_log_stats DMV on SQL Server.
USE [master];
IF SERVERPROPERTY('EngineEdition') != 5 -- 5 = Azure SQL Database
BEGIN
GRANT VIEW SERVER PERFORMANCE STATE TO [<YOUR_DB_USER>];
END
GO
```

Alternatively, you can grant more granular permissions on specific tables:

```sql
-- Grant SELECT on the specific replicated tables (recommended for more restrictive access)
GRANT SELECT ON <YOUR_SCHEMA>.<YOUR_TABLE_NAME> TO [<YOUR_DB_USER>];
```

<Info>
For Azure SQL Database, the `VIEW SERVER PERFORMANCE STATE` permission is not available and not required. Only `VIEW DATABASE PERFORMANCE STATE` is needed.
</Info>

### CDC Management

Management and performance turning of CDC is left to the developer and is primarily done by modifying the change capture jobs. See [Change Data Capture Jobs (SQL Server)](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/administer-and-monitor-change-data-capture-sql-server?view=sql-server-ver17) for more details.
Capture Job settings of interest to PowerSync:
- Polling interval: The frequency at which the capture job reads changes from the transaction log. Default is every 5 seconds. Can be set to 0 so that there is zero downtime between scans, but this will impact database performance.
- Max Trans: The maximum number of transactions that are processed per scan. Default is 500.
- Max Scans: The maximum number of scans that are performed per capture job scan cycle. Default is 10.

Cleanup Job settings of interest to PowerSync:
- Retention: The retention period before data is expired from the CDC tables. Default is 3 days. If your PowerSync instance is offline for longer than this period, data will need to be fully re-synced. Specified in minutes.

Recommended Capture Job settings:
| Parameter | Recommended Value |
|-------------------------|-------------------|
| maxtrans | 5000 |

Check warning on line 574 in installation/database-setup.mdx

View check run for this annotation

Mintlify / Mintlify Validation (powersync) - vale-spellcheck

installation/database-setup.mdx#L574

Did you really mean 'maxtrans'?
| maxscans | 10 |

Check warning on line 575 in installation/database-setup.mdx

View check run for this annotation

Mintlify / Mintlify Validation (powersync) - vale-spellcheck

installation/database-setup.mdx#L575

Did you really mean 'maxscans'?
| pollinginterval| 1 second |

Check warning on line 576 in installation/database-setup.mdx

View check run for this annotation

Mintlify / Mintlify Validation (powersync) - vale-spellcheck

installation/database-setup.mdx#L576

Did you really mean 'pollinginterval'?

### Limitations

- For Azure SQL Database, the CDC capture and cleanup jobs are managed automatically. Manual configuration is greatly limited.
See: [Azure CDC Customization Limitations](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql#cdc-customization)
The main limitation is that the capture job polling interval cannot be modified and is fixed at 20 seconds. It is, however, still possible to [manually trigger](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql#manual-cdc-control) the capture job on demand.
- CDC can only be enabled on Azure SQL Databases on the vCore purchasing model, OR for database on the DTU purchasing model, only on the S3 tier and above.

Check warning on line 583 in installation/database-setup.mdx

View check run for this annotation

Mintlify / Mintlify Validation (powersync) - vale-spellcheck

installation/database-setup.mdx#L583

Did you really mean 'vCore'?
See: [Azure SQL Database compute requirements](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql#azure-sql-database-compute-requirements)

## Next Step

Next, connect PowerSync to your database:
Expand Down
2 changes: 1 addition & 1 deletion installation/database-setup/private-endpoints.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Do not rely on Private Endpoints as the only form of security. Always use strong

## Current Limitations

1. Private Endpoints are currently only supported for Postgres and MongoDB instances. [Contact us](/resources/contact-us) if you need this for MySQL.
1. Private Endpoints are currently only supported for Postgres and MongoDB instances. [Contact us](/resources/contact-us) if you need this for MySQL or SQL Server.
2. Self-service is not yet available on the PowerSync side — [contact PowerSync support](/resources/contact-us) to configure the instance.
3. Only AWS is supported currently — other cloud providers are not supported yet.
4. The **"Test Connection"** function on the [PowerSync Dashboard](https://dashboard.powersync.com/) is not supported yet - the instance has to be deployed to test the connection.
Expand Down
Loading