Skip to content

Commit

Permalink
Merge pull request supabase#17369 from supabase/or/ai-docs-split
Browse files Browse the repository at this point in the history
Separate python/js ai docs
  • Loading branch information
olirice authored Sep 12, 2023
2 parents b019a51 + 3802d34 commit 440b742
Show file tree
Hide file tree
Showing 8 changed files with 56 additions and 197 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -964,34 +964,9 @@ export const ai = {
url: '/guides/ai/structured-unstructured',
},
{
name: 'Quickstarts',
url: undefined,
items: [
{ name: 'Developing locally with Vecs', url: '/guides/ai/vecs-python-client' },
{ name: 'Creating and managing collections', url: '/guides/ai/quickstarts/hello-world' },
{
name: 'Generate Embeddings',
url: '/guides/ai/quickstarts/generate-text-embeddings',
},
{ name: 'Text Deduplication', url: '/guides/ai/quickstarts/text-deduplication' },
{ name: 'Face similarity search', url: '/guides/ai/quickstarts/face-similarity' },
],
},
{
name: 'Python Client',
url: undefined,
items: [
{ name: 'API', url: '/guides/ai/python/api' },
{ name: 'Collections', url: '/guides/ai/python/collections' },
{ name: 'Indexes', url: '/guides/ai/python/indexes' },
{ name: 'Metadata', url: '/guides/ai/python/metadata' },
],
},
{
name: 'Guides',
name: 'Learn',
url: undefined,
items: [
{ name: 'Managing collections', url: '/guides/ai/managing-collections' },
{ name: 'Vector columns', url: '/guides/ai/vector-columns' },
{ name: 'Vector indexes', url: '/guides/ai/vector-indexes', items: vectorIndexItems },
{ name: 'Engineering for scale', url: '/guides/ai/engineering-for-scale' },
Expand All @@ -1000,25 +975,23 @@ export const ai = {
],
},
{
name: 'Examples',
name: 'JavaScript Examples',
url: undefined,
items: [
{
name: 'OpenAI completions using Edge Functions',
url: '/guides/ai/examples/openai',
},
{
name: 'Image search with OpenAI CLIP',
url: '/guides/ai/examples/image-search-openai-clip',
},

{
name: 'Generate image captions using Hugging Face',
url: '/guides/ai/examples/huggingface-image-captioning',
},
{
name: 'Building ChatGPT Plugins',
url: '/guides/ai/examples/building-chatgpt-plugins',
name: 'Generate Embeddings',
url: '/guides/ai/quickstarts/generate-text-embeddings',
},

{
name: 'Adding generative Q&A to your documentation',
url: '/guides/ai/examples/headless-vector-search',
Expand All @@ -1029,6 +1002,36 @@ export const ai = {
},
],
},
{
name: 'Python Client',
url: undefined,
items: [
{ name: 'Choosing a Client', url: '/guides/ai/python-clients' },
{ name: 'API', url: '/guides/ai/python/api' },
{ name: 'Collections', url: '/guides/ai/python/collections' },
{ name: 'Indexes', url: '/guides/ai/python/indexes' },
{ name: 'Metadata', url: '/guides/ai/python/metadata' },
],
},
{
name: 'Python Examples',
url: undefined,
items: [
{ name: 'Developing locally with Vecs', url: '/guides/ai/vecs-python-client' },
{ name: 'Creating and managing collections', url: '/guides/ai/quickstarts/hello-world' },

{ name: 'Text Deduplication', url: '/guides/ai/quickstarts/text-deduplication' },
{ name: 'Face similarity search', url: '/guides/ai/quickstarts/face-similarity' },
{
name: 'Image search with OpenAI CLIP',
url: '/guides/ai/examples/image-search-openai-clip',
},
{
name: 'Building ChatGPT Plugins',
url: '/guides/ai/examples/building-chatgpt-plugins',
},
],
},
{
name: 'Third-Party Tools',
url: undefined,
Expand Down
156 changes: 0 additions & 156 deletions apps/docs/pages/guides/ai/managing-collections.mdx

This file was deleted.

18 changes: 18 additions & 0 deletions apps/docs/pages/guides/ai/python-clients.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import Layout from '~/layouts/DefaultGuideLayout'

export const meta = {
id: 'ai-python-clients',
title: 'Choosing a Client',
description: 'Learn how to manage vectors using Python',
sidebar_label: 'Choosing a Client',
}

As described in [Structured & Unstructured Embeddings](/docs/guides/ai/structured-unstructured), AI workloads come in many forms.

For data science or ephemeral workloads, the [Supabase Vecs](https://supabase.github.io/vecs/) client gets you started quickly. All you need is a connection string and vecs handles setting up your database to store and query vectors with associated metadata.

For production python applications with version controlled migrations, we recommend adding first class vector support to your toolchain by [registering the vector type with your ORM](https://github.com/pgvector/pgvector-python). pgvector provides bindings for the most commonly used SQL drivers/libraries including Django, SQLAlchemy, SQLModel, psycopg, asyncpg and Peewee.

export const Page = ({ children }) => <Layout meta={meta} children={children} />

export default Page
2 changes: 1 addition & 1 deletion apps/docs/pages/guides/ai/vecs-python-client.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ docs.query(

## Deep Dive

For a more in-depth guide on `vecs` collections, see [Managing collections](/docs/guides/ai/managing-collections).
For a more in-depth guide on `vecs` collections, see [API](/docs/guides/ai/python/api).

## Resources

Expand Down
2 changes: 1 addition & 1 deletion apps/docs/pages/guides/ai/vector-columns.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ export const meta = {
sidebar_label: 'Vector columns',
}

Supabase offers a number of different ways to store and query vectors within Postgres. If you prefer to use Python to store and query your vectors using collections, see [Managing collections](/docs/guides/ai/managing-collections). If you want more control over vectors within your own Postgres tables or would like to interact with them using a different language like JavaScript, keep reading.
Supabase offers a number of different ways to store and query vectors within Postgres. The SQL included in this guide is applicable for clients in all programming languages. If you are a Python user see your [Python client options](/docs/guides/ai/python-clients) after reading the `Learn` section.

Vectors in Supabase are enabled via [pgvector](https://github.com/pgvector/pgvector/), a PostgreSQL extension for storing and querying vectors in Postgres. It can be used to store [embeddings](/docs/guides/ai/concepts#what-are-embeddings).

Expand Down
2 changes: 0 additions & 2 deletions apps/docs/pages/guides/ai/vector-indexes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,6 @@ Indexes can be used to improve performance of nearest neighbor search using vari

Currently vectors with up to 2,000 dimensions can be indexed.

If you are using the `vecs` Python library, follow the instructions in [Managing collections](/docs/guides/ai/managing-collections#create-an-index) to create indexes.

## Resources

Read more about indexing on `pgvector`'s [GitHub page](https://github.com/pgvector/pgvector#indexing).
Expand Down
2 changes: 0 additions & 2 deletions apps/docs/pages/guides/ai/vector-indexes/hnsw-indexes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@ create index on items using hnsw (column_name vector_cosine_ops);

Currently vectors with up to 2,000 dimensions can be indexed.

If you are using the `vecs` Python library, follow the instructions in [Managing collections](/docs/guides/ai/managing-collections#create-an-index) to create indexes.

## How does HNSW work?

HNSW uses proximity graphs (graphs connecting nodes based on distance between them) to approximate nearest-neighbor search. To understand HNSW, we can break it down into 2 parts:
Expand Down
2 changes: 0 additions & 2 deletions apps/docs/pages/guides/ai/vector-indexes/ivf-indexes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,6 @@ create index on items using ivfflat (column_name vector_cosine_ops) with (lists

Currently vectors with up to 2,000 dimensions can be indexed.

If you are using the `vecs` Python library, follow the instructions in [Managing collections](/docs/guides/ai/managing-collections#create-an-index) to create indexes.

## How does IVFFlat work?

IVF stands for 'inverted file indexes'. It works by clustering your vectors in order to reduce the similarity search scope. Rather than comparing a vector to every other vector, the vector is only compared against vectors within the same cell cluster (or nearby clusters, depending on your configuration).
Expand Down

0 comments on commit 440b742

Please sign in to comment.