Skip to content

Commit

Permalink
docs: add vercel example to show a portal to private snowflake api
Browse files Browse the repository at this point in the history
  • Loading branch information
snandam authored and mrinalwadhwa committed Dec 17, 2024
1 parent 5277ba0 commit aa81110
Show file tree
Hide file tree
Showing 7 changed files with 322 additions and 0 deletions.
2 changes: 2 additions & 0 deletions examples/command/portals/vercel/example-1/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*.ticket
.vercel
114 changes: 114 additions & 0 deletions examples/command/portals/vercel/example-1/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Call private APIs in Vercel

![Architecture](./diagram.png)


## Snowflake: Setup a private API in Snowpark Container Services

- [Follow](https://quickstarts.snowflake.com/guide/build_a_private_custom_api_in_python/index.html?index=..%2F..index#0) to setup a private API in Snowpark Container Services

- Create a database, table and insert some data

```sql
-- Use the accountadmin role to create the database, table and insert some data
USE ROLE ACCOUNTADMIN;

-- Grant the DATA_API_ROLE the necessary permissions to use the database and schema
GRANT USAGE ON DATABASE TEST_DATABASE TO ROLE DATA_API_ROLE;
GRANT USAGE ON SCHEMA TEST_DATABASE.PUBLIC TO ROLE DATA_API_ROLE;

-- Create the table
CREATE OR REPLACE TABLE TEST_DATABASE.PUBLIC.PRODUCTS (
id INTEGER PRIMARY KEY,
product_name VARCHAR(255),
price NUMBER(10,2)
);

-- Grant the DATA_API_ROLE the necessary permissions to use the table
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE TEST_DATABASE.PUBLIC.PRODUCTS TO ROLE DATA_API_ROLE;


-- Insert sample data
INSERT INTO TEST_DATABASE.PUBLIC.PRODUCTS (id, product_name, price)
VALUES (1, 'Ergonomic Keyboard', 10.99);

INSERT INTO TEST_DATABASE.PUBLIC.PRODUCTS (id, product_name, price)
VALUES (2, 'Wireless Mouse', 20.99);

INSERT INTO TEST_DATABASE.PUBLIC.PRODUCTS (id, product_name, price)
VALUES (3, 'LED Monitor 27"', 30.99);
```

- Create two new endpoints `/projects` and `/products/update`. Update `connector.py` to include the new endpoints.

```python
@connector.route('/products')
def get_products():
sql = '''
SELECT id, product_name, price
FROM TEST_DATABASE.public.products
'''
try:
res = conn.cursor(DictCursor).execute(sql)
return make_response(jsonify(res.fetchall()))
except:
abort(500, "Error reading from Snowflake. Check the logs for details.")

@connector.route('/products/update', methods=['POST'])
def update_products():
sql = '''
UPDATE TEST_DATABASE.public.products
SET price = ROUND(UNIFORM(10, 100, RANDOM()), 2)
'''
try:
conn.cursor().execute(sql)
conn.commit()
return make_response(jsonify({"message": "Products prices updated successfully"}), 200)
except:
conn.rollback()
abort(500, "Error updating prices in Snowflake. Check the logs for details.")
```

- Rebuild and publish the dataapi image (`/api/private/api/dataapi`) with new endpoints. Drop and create `API.PRIVATE.API` service with new image.


## Vercel: Setup Vercel Serverless functions to access the private API.

### Download latest ockam binary

- Download ockam binary (x86_64-unknown-linux-gnu) from the [Ockam](https://github.com/build-trust/ockam/releases) github repository and place it in the `data/linux-x86_64` directory. Rename the binary to `ockam`.

### Create an enrollment ticket for the Vercel function

```sh
# Generate an inlet ticket for the Vercel function.
ockam project ticket --usage-count 100 --expires-in 10h --attribute snowflake-api-service-inlet > vercel-inlet.ticket
```

### Setup a Vercel project and add the inlet ticket as a secret

- Select `Project Settings`, `Environments`, `Production
- Click `Add Environment Variable`
- Select `Sensitive`
- Add `OCKAM_SNOWFLAKE_INLET_ENROLLMENT_TICKET` as the name
- Add the value of `vercel-inlet.ticket` as the value
- Click `Add`

### Deploy the Vercel function

- Setup vercel cli and select the project
- Deploy the function inside the `api` directory

```sh
vercel --prod
```

### Test the Vercel function

- Use the `/api` endpoint to get the products
- Use the `/api/update` endpoint to update the products


## Setup a Next.js frontend deployed to Vercel to access vercel function

- Optionally you can setup a Next.js frontend to access the vercel function via it's public URLs.
184 changes: 184 additions & 0 deletions examples/command/portals/vercel/example-1/api/index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
"""
Simple API handler for accessing Snowflake private API endpoints via Ockam portal
"""

import json
import os
import subprocess
import threading
import time
from http.server import BaseHTTPRequestHandler
import requests

# Constants
OCKAM_BINARY_PATH = '/var/task/data/linux-x86_64/ockam' if os.environ.get('VERCEL_ENV') == 'production' else 'ockam'
MAX_RETRIES = 60
RETRY_DELAY = 1 # seconds
LOCAL_API_ENDPOINT = "http://127.0.0.1:8081"
REQUEST_TIMEOUT = 3 # seconds

# Track if Ockam node creation has been attempted
# Only one Ockam node should run per serverless function runtime - this flag prevents multiple node creation attempts.
NODE_INITIALIZED = False

def is_production() -> bool:
"""Check if the environment is production."""
return os.environ.get('VERCEL_ENV') == 'production'

def get_ockam_version() -> str:
"""
Get the installed Ockam version.
Returns version string or error message.
"""
try:
ockam_path = OCKAM_BINARY_PATH
if is_production() and not os.path.exists(ockam_path):
return f"Error: Ockam binary not found at {os.path.abspath(ockam_path)}"

result = subprocess.run([ockam_path, '--version'],
capture_output=True,
text=True)

return result.stdout.strip() if result.returncode == 0 else f"Error running ockam: {result.stderr}"
except Exception as e:
return f"Error: {str(e)} (CWD: {os.getcwd()})"

class handler(BaseHTTPRequestHandler):
"""Handler for Snowflake API requests via Ockam secure channel."""

def create_ockam_node(self, enrollment_ticket: str) -> bool:
"""
Initialize Ockam node with the provided enrollment ticket.
Returns True if successful or already attempted, False otherwise.
"""
if not is_production():
return False

global NODE_INITIALIZED
request_id = self.headers.get('x-vercel-id', os.urandom(8).hex())

print(f"[{request_id}] Starting with NODE_INITIALIZED = {NODE_INITIALIZED}")

if not NODE_INITIALIZED:
NODE_INITIALIZED = True

def run_ockam_node():
try:
config = '''{
"tcp-inlet": {
"from": "127.0.0.1:8081",
"via": "snowflake-api-service-relay",
"allow": "snowflake-api-service-outlet"
}
}'''

print(f"[{request_id}] Starting ockam node with config: {config}", flush=True)

subprocess.Popen([
OCKAM_BINARY_PATH,
'node',
'create',
'--configuration',
config,
'--enrollment-ticket',
enrollment_ticket.strip()
], env={
**os.environ,
'OCKAM_HOME': '/tmp',
'OCKAM_OPENTELEMETRY_EXPORT': 'false',
'OCKAM_DISABLE_UPGRADE_CHECK': 'true'
})

except Exception as e:
print(f"[{request_id}] Error in node creation: {str(e)}", flush=True)

thread = threading.Thread(target=run_ockam_node)
thread.daemon = True
print(f"[{request_id}] Starting ockam node thread", flush=True)
thread.start()

return True

def handle_select(self, request_id: str) -> dict:
"""Handle GET requests for product data."""
for retry in range(MAX_RETRIES):
try:
response = requests.get(
f"{LOCAL_API_ENDPOINT}/connector/products",
timeout=REQUEST_TIMEOUT
)
response.raise_for_status()
print(f"[{request_id}] Connected successfully after {retry} retries")
return {"status": "success", "data": response.json()}

except Exception as e:
print(f"[{request_id}] Attempt {retry + 1} failed: {str(e)}")
if retry < MAX_RETRIES - 1:
time.sleep(RETRY_DELAY)
else:
return {"status": "error", "message": f"Connection failed: {str(e)}"}

def handle_update(self, request_id: str) -> dict:
"""Handle POST requests for product updates."""
for retry in range(MAX_RETRIES):
try:
response = requests.post(
f"{LOCAL_API_ENDPOINT}/connector/products/update",
timeout=REQUEST_TIMEOUT
)
response.raise_for_status()
print(f"[{request_id}] Update successful after {retry} retries")

updated_values = self.handle_select(request_id)
return {
"status": "success",
"update_result": response.json(),
"current_values": updated_values["data"]
}

except Exception as e:
print(f"[{request_id}] Attempt {retry + 1} failed: {str(e)}")
if retry < MAX_RETRIES - 1:
time.sleep(RETRY_DELAY)
else:
return {"status": "error", "message": f"Update failed: {str(e)}"}

def do_GET(self):
"""Handle incoming GET requests."""
try:
# Verify Ockam installation
version_result = get_ockam_version()
if 'Error' in version_result:
self._send_error(500, version_result)
return

# Verify enrollment ticket
enrollment_ticket = os.environ.get('OCKAM_SNOWFLAKE_INLET_ENROLLMENT_TICKET')
if not enrollment_ticket:
self._send_error(500, 'OCKAM_SNOWFLAKE_INLET_ENROLLMENT_TICKET not configured')
return

# Initialize Ockam node
if not self.create_ockam_node(enrollment_ticket):
self._send_error(500, 'Failed to initialize Ockam node')
return

# Handle the request
request_id = self.headers.get('x-vercel-id', os.urandom(8).hex())
result = self.handle_update(request_id) if self.path == '/api/update' else self.handle_select(request_id)

self._send_response(200, result)

except Exception as e:
self._send_error(500, str(e))

def _send_response(self, status: int, data: dict):
"""Helper method to send JSON responses."""
self.send_response(status)
self.send_header('Content-Type', 'application/json')
self.end_headers()
self.wfile.write(json.dumps(data).encode())

def _send_error(self, status: int, message: str):
"""Helper method to send error responses."""
self._send_response(status, {'error': message})
Empty file.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions examples/command/portals/vercel/example-1/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Flask==3.0.3
psycopg2-binary==2.9.9
requests==2.31.0
19 changes: 19 additions & 0 deletions examples/command/portals/vercel/example-1/vercel.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"redirects": [
{
"source": "/",
"destination": "/api"
}
],
"rewrites": [
{
"source": "/api/update",
"destination": "/api"
}
],
"functions": {
"api/**/*": {
"maxDuration": 300
}
}
}

0 comments on commit aa81110

Please sign in to comment.