Epic: Support dblink and postgres_fdw extensions

[vadim2404]

Motivation

dblink and postgres_fdw extensions are quite popular in analytical, migration, and sharding use cases. Therefore we need to install them.

DoD

As a user, I can install these two extensions and use them.

Implementation ideas

Tasks

• Ensure that computes use JWT token while communicating with SKs and PSs
• Add traffic metering in/out Internet #4704
• Enable network policies that allow traffic only for selected pageserver, safekeepers, proxy, and internet traffic
• Enable these extensions
• Check that you cannot connect to localhost as superuser

Other related tasks and Epics

• neondatabase/cloud#5745
• Epic: Provide pgsql-http extension #5631

[vmatt]

Hey, when this can be expected?

[clarkbw]

The postgres_fdw extension is expected to release this quarter, Q4 2024

[acervantes23]

@vadim2404 what is the target ship date for this?

[vadim2404]

I'll forward this question to @ololobus

[ololobus]

It's currently set to the Mid-Nov here, but not sure it's gonna happen. Early Dec is more realistic. I'll keep the roadmap updated as it goes

[acervantes23]

can we keep GH up to date with the target ship date? my assumption is to check issues/epics for the target ship date and not spreadsheets

[ololobus]

issues/epics for the target ship date and not spreadsheets

We haven't had such fields on the Epic/issue-sized tasks, only in company projects. I'm not sure that I want to create a company project for every Epic/issue-sized task. We can probably update team projects to include this dates, but then syncing everything is the mess. It's probably something to discuss with Anna and/or Bryan

[ololobus]

This week:

• @myrrc is working on Add traffic metering in/out Internet #4704, target opening a PR this week and write/run tests (autoscaling repo?)

[ololobus]

This week:

• @myrrc test and pass review on vm-runner: external traffic Prometheus metrics endpoint autoscaling#1153
• Build and test dblink and postgres_fdw extensions

[myrrc]

This week - still waiting for vm-runner review

[myrrc]

Runner merged, waiting for deployment

[ololobus]

This week:

• @myrrc enable traffic metering on staging
• @myrrc test dblink and enable

[acervantes23]

hey @ololobus will we enable this week?

[ololobus]

hey @ololobus will we enable this week?

Def, not this week because it's Fri evening :) I think that we may target merging the PR next week, but cc @myrrc to correct me

[ololobus]

This week:

• @myrrc targets to merge dblink support

[ololobus]

Question from @MMeent

Re: DBLink:
Do we have enough protections for that?
IIRC, we had a local:trust policy (i.e. no password), which is kinda bad when someone tries to connect to localhost with a neon_superuser dbname using e.g. dblink or postgres_fdw.

I.e. how we restrict users from connecting as cloud_admin. Heikki, exts should have a protection from that

@myrrc to double-check

Stas: if it's possible to conenct, we can probably use socket for our conns and restrict tcp localhost access

[acervantes23]

Hows this one looking @ololobus ?