Skip to content

Commit

Permalink
Adds authToken support to neon() (#97)
Browse files Browse the repository at this point in the history 
* Adds `authToken` support which can be used to send a JWT in the
`Bearer` of the Authentication HTTP header
* We don't have WebSockets support for this hence no changes to the Pool
class
* The sanity "tests" (`src/index.ts`) run, but have not been modified
because they need to run against an instance of Neon that supports this
(and that's not possible yet)

We won't merge this PR until we can add some tests for it, but the code
review can happen now.

## How to use this?
```typescript
import { neon } from "@neondatabase/serverless";
import { auth, clerkClient } from "@clerk/nextjs/server";

const sql = neon(process.env.DATABASE_URL, {
  authToken: async () => clerkClient.sessions.getToken(auth(), "Neon"),
});

...
```

---------

Co-authored-by: Pedro Figueiredo <[email protected]>
  • Loading branch information
@davidgomes @pffigueiredo
davidgomes and pffigueiredo authored Oct 3, 2024
1 parent 6a9a2d2 commit d90ad3b
Show file tree
Hide file tree
Showing 14 changed files with 14,953 additions and 5,958 deletions.
78 changes: 34 additions & 44 deletions dist/jsr/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,12 @@

`@neondatabase/serverless` is [Neon](https://neon.tech)'s PostgreSQL driver for JavaScript and TypeScript. It's:

* **Low-latency**, thanks to [message pipelining](https://neon.tech/blog/quicker-serverless-postgres) and other optimizations
* **Ideal for serverless/edge** deployment, using https and WebSockets in place of TCP
* **A drop-in replacement** for [node-postgres](https://node-postgres.com/), aka [`pg`](https://www.npmjs.com/package/pg) (on which it's based)

- **Low-latency**, thanks to [message pipelining](https://neon.tech/blog/quicker-serverless-postgres) and other optimizations
- **Ideal for serverless/edge** deployment, using https and WebSockets in place of TCP
- **A drop-in replacement** for [node-postgres](https://node-postgres.com/), aka [`pg`](https://www.npmjs.com/package/pg) (on which it's based)

## Get started


### Install it

Install it with your preferred JavaScript package manager. It's named `@neondatabase/serverless` on npm and `@neon/serverless` on JSR. So, for example:
Expand All @@ -26,7 +24,6 @@ bunx jsr add @neon/serverless

Using TypeScript? No worries: types are included either way.


### Configure it

Get your connection string from the [Neon console](https://console.neon.tech/) and set it as an environment variable. Something like:
Expand All @@ -35,7 +32,6 @@ Get your connection string from the [Neon console](https://console.neon.tech/) a
DATABASE_URL=postgres://username:[email protected]/neondb
```


### Use it

For one-shot queries, use the `neon` function. For instance:
Expand All @@ -50,7 +46,6 @@ const [post] = await sql`SELECT * FROM posts WHERE id = ${postId}`;

Note: interpolating `${postId}` here is [safe from SQL injection](https://neon.tech/blog/sql-template-tags).


### Deploy it

Turn this example into a complete API endpoint deployed on [Vercel Edge Functions](https://vercel.com/docs/concepts/functions/edge-functions) at `https://myapp.vercel.dev/api/post?postId=123` by following two simple steps:
Expand All @@ -71,7 +66,7 @@ export default async (req: Request, ctx: any) => {
if (!post) return new Response('Not found', { status: 404 });

// return the post as JSON
return new Response(JSON.stringify(post), {
return new Response(JSON.stringify(post), {
headers: { 'content-type': 'application/json' }
});
}
Expand All @@ -93,14 +88,12 @@ npx vercel deploy

The `neon` query function has a few [additional options](CONFIG.md).


## Sessions, transactions, and node-postgres compatibility

A query using the `neon` function, as shown above, is carried by an https [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) request.
A query using the `neon` function, as shown above, is carried by an https [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) request.

This should work — and work fast — from any modern JavaScript environment. But you can only send one query at a time this way: sessions and transactions are not supported.


### `transaction()`

Multiple queries can be issued via fetch request within a single, non-interactive transaction by using the `transaction()` function. This is exposed as a property on the query function.
Expand All @@ -120,32 +113,29 @@ const [posts, tags] = await sql.transaction([

There are some [additional options](CONFIG.md) when using `transaction()`.


### `Pool` and `Client`

Use the `Pool` or `Client` constructors, instead of the functions described above, when you need:

* **session or interactive transaction support**, and/or
- **session or interactive transaction support**, and/or

* **compatibility with node-postgres**, which supports query libraries like [Kysely](https://kysely.dev/) or [Zapatos](https://jawj.github.io/zapatos/).
- **compatibility with node-postgres**, which supports query libraries like [Kysely](https://kysely.dev/) or [Zapatos](https://jawj.github.io/zapatos/).

Queries using `Pool` and `Client` are carried by WebSockets. There are **two key things** to know about this:

1. **In Node.js** and some other environments, there's no built-in WebSocket support. In these cases, supply a WebSocket constructor function.

2. **In serverless environments** such as Vercel Edge Functions or Cloudflare Workers, WebSocket connections can't outlive a single request.

That means `Pool` or `Client` objects must be connected, used and closed **within a single request handler**. Don't create them outside a request handler; don't create them in one handler and try to reuse them in another; and to avoid exhausting available connections, don't forget to close them.

These points are demonstrated in the examples below.
2. **In serverless environments** such as Vercel Edge Functions or Cloudflare Workers, WebSocket connections can't outlive a single request.

That means `Pool` or `Client` objects must be connected, used and closed **within a single request handler**. Don't create them outside a request handler; don't create them in one handler and try to reuse them in another; and to avoid exhausting available connections, don't forget to close them.

### API
These points are demonstrated in the examples below.

* **The full API guide** to `Pool` and `Client` can be found in the [node-postgres docs](https://node-postgres.com/).
### API

* There are a few [additional configuration options](CONFIG.md) that apply to `Pool` and `Client` here.
- **The full API guide** to `Pool` and `Client` can be found in the [node-postgres docs](https://node-postgres.com/).

- There are a few [additional configuration options](CONFIG.md) that apply to `Pool` and `Client` here.

## Example: Node.js with `Pool.connect()`

Expand All @@ -155,24 +145,29 @@ In Node.js, it takes two lines to configure WebSocket support. For example:
import { Pool, neonConfig } from '@neondatabase/serverless';

import ws from 'ws';
neonConfig.webSocketConstructor = ws; // <-- this is the key bit
neonConfig.webSocketConstructor = ws; // <-- this is the key bit

const pool = new Pool({ connectionString: process.env.DATABASE_URL });
pool.on('error', err => console.error(err)); // deal with e.g. re-connect
pool.on('error', (err) => console.error(err)); // deal with e.g. re-connect
// ...

const client = await pool.connect();

try {
await client.query('BEGIN');
const { rows: [{ id: postId }] } = await client.query('INSERT INTO posts (title) VALUES ($1) RETURNING id', ['Welcome']);
await client.query('INSERT INTO photos (post_id, url) VALUES ($1, $2)', [postId, 's3.bucket/photo/url']);
const {
rows: [{ id: postId }],
} = await client.query('INSERT INTO posts (title) VALUES ($1) RETURNING id', [
'Welcome',
]);
await client.query('INSERT INTO photos (post_id, url) VALUES ($1, $2)', [
postId,
's3.bucket/photo/url',
]);
await client.query('COMMIT');

} catch (err) {
await client.query('ROLLBACK');
throw err;

} finally {
client.release();
}
Expand All @@ -185,10 +180,9 @@ Other WebSocket libraries are available. For example, you could replace `ws` in

```typescript
import { WebSocket } from 'undici';
neonConfig.webSocketConstructor = WebSocket;
neonConfig.webSocketConstructor = WebSocket;
```


## Example: Vercel Edge Function with `Pool.query()`

We can rewrite the Vercel Edge Function shown above (under the heading 'Deploy it') to use `Pool`, as follows:
Expand All @@ -210,12 +204,12 @@ export default async (req: Request, ctx: any) => {
const [post] = await pool.query('SELECT * FROM posts WHERE id = $1', [postId]);
if (!post) return new Response('Not found', { status: 404 });

// end the `Pool` inside the same request handler
// end the `Pool` inside the same request handler
// (unlike `await`, `ctx.waitUntil` won't hold up the response)
ctx.waitUntil(pool.end());

// return the post as JSON
return new Response(JSON.stringify(post), {
return new Response(JSON.stringify(post), {
headers: { 'content-type': 'application/json' }
});
}
Expand All @@ -228,7 +222,6 @@ export const config = {

Note: we don't actually use the pooling capabilities of `Pool` in this example. But it's slightly briefer than using `Client` and, because `Pool.query` is designed for one-shot queries, we may in future automatically route these queries over https for lower latency.


## Example: Vercel Edge Function with `Client`

Using `Client` instead, the example looks like this:
Expand All @@ -251,12 +244,12 @@ export default async (req: Request, ctx: any) => {
const [post] = await client.query('SELECT * FROM posts WHERE id = $1', [postId]);
if (!post) return new Response('Not found', { status: 404 });

// end the `Client` inside the same request handler
// end the `Client` inside the same request handler
// (unlike `await`, `ctx.waitUntil` won't hold up the response)
ctx.waitUntil(client.end());

// return the post as JSON
return new Response(JSON.stringify(post), {
return new Response(JSON.stringify(post), {
headers: { 'content-type': 'application/json' }
});
}
Expand All @@ -271,23 +264,20 @@ export const config = {

These repos show how to use `@neondatabase/serverless` with a variety of environments and tools:

* [Raw SQL + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-rawsql)
* [Raw SQL via https + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-http)
* [Raw SQL + Cloudflare Workers](https://github.com/neondatabase/serverless-cfworker-demo)
* [Kysely + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-kysely)
* [Zapatos + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-zapatos)

- [Raw SQL + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-rawsql)
- [Raw SQL via https + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-http)
- [Raw SQL + Cloudflare Workers](https://github.com/neondatabase/serverless-cfworker-demo)
- [Kysely + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-kysely)
- [Zapatos + Vercel Edge Functions](https://github.com/neondatabase/neon-vercel-zapatos)

## Bring your own Postgres database

This package comes configured to connect to a Neon database. But you can also use it to connect to your own Postgres instances if you [run your own WebSocket proxy](DEPLOY.md).


## Open-source

This code is released under the [MIT license](LICENSE).


## Feedback and support

Please visit [Neon Community](https://community.neon.tech/) or [Support](https://neon.tech/docs/introduction/support).
46 changes: 17 additions & 29 deletions dist/jsr/index.d.ts
View file
Original file line number Diff line number Diff line change
@@ -1,34 +1,15 @@

// @neondatabase/serverless driver types, mimicking pg

export { DatabaseError } from "pg-protocol";
export {
ClientConfig,
ConnectionConfig,
Defaults,
PoolConfig,
QueryConfig,
CustomTypesConfig,
Submittable,
QueryArrayConfig,
FieldDef,
QueryResultBase,
QueryResultRow,
QueryResult,
QueryArrayResult,
Notification,
ResultBuilder,
QueryParse,
BindConfig,
ExecuteConfig,
MessageConfig,
Connection,
Query,
Events,
types,
defaults,
native,
BindConfig, ClientConfig, Connection, ConnectionConfig, CustomTypesConfig, Defaults, defaults, Events, ExecuteConfig, FieldDef, MessageConfig, native, Notification, PoolConfig, Query, QueryArrayConfig, QueryArrayResult, QueryConfig, QueryParse, QueryResult, QueryResultBase,
QueryResultRow, ResultBuilder, Submittable, types
} from "pg";
export { DatabaseError } from "pg-protocol";

interface FetchEndpointOptions {
jwtAuth?: boolean;
}

export interface NeonConfigGlobalOnly {
/**
Expand All @@ -43,7 +24,7 @@ export interface NeonConfigGlobalOnly {
* Default: `host => 'https://' + host + '/sql'`
*
*/
fetchEndpoint: string | ((host: string, port: number | string) => string);
fetchEndpoint: string | ((host: string, port: number | string, options?: FetchEndpointOptions) => string);

/**
* **Experimentally**, when `poolQueryViaFetch` is `true`, and no listeners
Expand Down Expand Up @@ -178,10 +159,10 @@ export interface NeonConfigGlobalAndClient {
export interface NeonConfig extends NeonConfigGlobalOnly, NeonConfigGlobalAndClient { }

import {
ClientBase as PgClientBase,
Client as PgClient,
PoolClient as PgPoolClient,
ClientBase as PgClientBase,
Pool as PgPool,
PoolClient as PgPoolClient,
} from "pg";

export class ClientBase extends PgClientBase {
Expand Down Expand Up @@ -252,6 +233,13 @@ export interface HTTPQueryOptions<ArrayMode extends boolean, FullResults extends
* options take precedence.
*/
fetchOptions?: Record<string, any>;

/**
* JWT auth token to be passed as the Bearer token in the Authorization header
*
* Default: `undefined`
*/
authToken?: string | (() => Promise<string> | string);
}

export interface HTTPTransactionOptions<ArrayMode extends boolean, FullResults extends boolean> extends HTTPQueryOptions<ArrayMode, FullResults> {
Expand Down
Loading

0 comments on commit d90ad3b

Please sign in to comment.