Skip to content

Commit

Permalink
Preview PR pingcap/docs#18779 and this preview is triggered from commit
Browse files Browse the repository at this point in the history
  • Loading branch information
Docsite Preview Bot committed Oct 21, 2024
1 parent f281abc commit aeadbcf
Show file tree
Hide file tree
Showing 11 changed files with 52 additions and 87 deletions.
4 changes: 4 additions & 0 deletions markdown-pages/en/tidbcloud/master/TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,10 @@
- [Langchain](/vector-search-integrate-with-langchain.md)
- Embedding Models/Services
- [Jina AI](/vector-search-integrate-with-jinaai-embedding.md)
- ORM Libraries
- [SQLAlchemy](/vector-search-integrate-with-sqlalchemy.md)
- [peewee](/vector-search-integrate-with-peewee.md)
- [Django ORM](/vector-search-integrate-with-django-orm.md)
- Reference
- [Vector Data Types](/vector-search-data-types.md)
- [Vector Functions and Operators](/vector-search-functions-and-operators.md)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ This page lists the release notes of [TiDB Cloud](https://www.pingcap.com/tidb-c

- [Vector data types](/vector-search-data-types.md), [vector index](/vector-search-index.md), and [vector functions and operators](/vector-search-functions-and-operators.md).
- Ecosystem integrations with [LangChain](/vector-search-integrate-with-langchain.md), [LlamaIndex](/vector-search-integrate-with-llamaindex.md), and [JinaAI](/vector-search-integrate-with-jinaai-embedding.md).
- Programming language support for Python: [SQLAlchemy](/vector-search-integrate-with-sqlalchemy.md), [Peewee](/vector-search-integrate-with-peewee.md), and [Django ORM](/vector-search-integrate-with-django-orm.md).
- Sample applications and tutorials: perform semantic searches for documents using [Python](/vector-search-get-started-using-python.md) or [SQL](/vector-search-get-started-using-sql.md).

For more information, see [Vector search (beta) overview](/vector-search-overview.md).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down Expand Up @@ -239,34 +239,6 @@ class Document(models.Model):
embedding = VectorField(dimensions=3)
```

#### Define a vector column optimized with index

> **Note**
>
> This section is only applicable to [TiDB Serverless](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-serverless) clusters.
Define a 3-dimensional vector column and optimize it with a [vector search index](/vector-search-index.md) (HNSW index).

```python
class DocumentWithIndex(models.Model):
content = models.TextField()
# Note:
# - Using comment to add hnsw index is a temporary solution. In the future it will use `CREATE INDEX` syntax.
# - Currently the HNSW index cannot be changed after the table has been created.
# - Only Django >= 4.2 supports `db_comment`.
embedding = VectorField(dimensions=3, db_comment="hnsw(distance=cosine)")
```

TiDB will use this index to speed up vector search queries based on the cosine distance function.

### Store documents with embeddings

```python
Document.objects.create(content="dog", embedding=[1, 2, 1])
Document.objects.create(content="fish", embedding=[1, 2, 4])
Document.objects.create(content="tree", embedding=[1, 0, 0])
```

### Search the nearest neighbor documents

TiDB Vector support the following distance functions:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down Expand Up @@ -237,26 +237,6 @@ class Document(Model):
embedding = VectorField(3)
```

#### Define a vector column optimized with index

> **Note**
>
> This section is only applicable to [TiDB Serverless](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-serverless) clusters.
Define a 3-dimensional vector column and optimize it with a [vector search index](/vector-search-index.md) (HNSW index).

```python
class DocumentWithIndex(Model):
class Meta:
database = db
table_name = 'peewee_demo_documents_with_index'

content = TextField()
embedding = VectorField(3, constraints=[SQL("COMMENT 'hnsw(distance=cosine)'")])
```

TiDB will use this index to accelerate vector search queries based on the cosine distance function.

### Store documents with embeddings

```python
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ To complete this tutorial, you need:
**If you don't have a TiDB cluster, you can create one as follows:**

- (Recommended) Follow [Creating a TiDB Cloud Serverless cluster](/develop/dev-guide-build-cluster-in-cloud.md) to create your own TiDB Cloud cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster.
- Follow [Deploy a local test TiDB cluster](https://docs.pingcap.com/tidb/stable/quick-start-with-tidb#deploy-a-local-test-cluster) or [Deploy a production TiDB cluster](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup) to create a local cluster of v8.4.0 or a later version.

</CustomContent>

Expand Down Expand Up @@ -201,34 +201,6 @@ class Document(Base):
embedding = Column(VectorType(3))
```

#### Define a vector column optimized with index

> **Note**
>
> This section is only applicable to [TiDB Serverless](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-serverless) clusters.
Define a 3-dimensional vector column and optimize it with a [vector search index](/vector-search-index.md) (HNSW index).

```python
class DocumentWithIndex(Base):
__tablename__ = 'sqlalchemy_demo_documents_with_index'
id = Column(Integer, primary_key=True)
content = Column(Text)
embedding = Column(VectorType(3), comment="hnsw(distance=cosine)")
```

TiDB will use this index to accelerate vector search queries based on the cosine distance function.

### Store documents with embeddings

```python
with Session(engine) as session:
session.add(Document(content="dog", embedding=[1, 2, 1]))
session.add(Document(content="fish", embedding=[1, 2, 4]))
session.add(Document(content="tree", embedding=[1, 0, 0]))
session.commit()
```

### Search the nearest neighbor documents

Search for the top-3 documents that are semantically closest to the query vector `[1, 2, 3]` based on the cosine distance function.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: Vector Search Integration Overview
summary: An overview of TiDB vector search integration, including supported AI frameworks and embedding models.
summary: An overview of TiDB vector search integration, including supported AI frameworks, embedding models, and ORM libraries.
---

# Vector Search Integration Overview

This document provides an overview of TiDB vector search integration, including supported AI frameworks and embedding models.
This document provides an overview of TiDB vector search integration, including supported AI frameworks, embedding models, and Object Relational Mapping (ORM) libraries.

<CustomContent platform="tidb">

Expand Down Expand Up @@ -40,4 +40,40 @@ The following table lists some mainstream embedding service providers and the co

| Embedding service providers | Tutorial |
|-----------------------------|---------------------------------------------------------------------------------------------------------------------|
| Jina AI | [Integrate Vector Search with Jina AI Embeddings API](/vector-search-integrate-with-jinaai-embedding.md) |
| Jina AI | [Integrate Vector Search with Jina AI Embeddings API](/vector-search-integrate-with-jinaai-embedding.md) |

## Object Relational Mapping (ORM) libraries

You can integrate TiDB Vector Search with your ORM library to interact with the TiDB database.

The following table lists the supported ORM libraries and the corresponding integration tutorials:

<table>
<tr>
<th>Language</th>
<th>ORM/Client</th>
<th>How to install</th>
<th>Tutorial</th>
</tr>
<tr>
<td rowspan="4">Python</td>
<td>TiDB Vector Client</td>
<td><code>pip install tidb-vector[client]</code></td>
<td><a href="/tidbcloud/vector-search-get-started-using-python">Get Started with Vector Search Using Python</a></td>
</tr>
<tr>
<td>SQLAlchemy</td>
<td><code>pip install tidb-vector</code></td>
<td><a href="/tidbcloud/vector-search-integrate-with-sqlalchemy">Integrate TiDB Vector Search with SQLAlchemy</a></td>
</tr>
<tr>
<td>peewee</td>
<td><code>pip install tidb-vector</code></td>
<td><a href="/tidbcloud/vector-search-integrate-with-peewee">Integrate TiDB Vector Search with peewee</a></td>
</tr>
<tr>
<td>Django</td>
<td><code>pip install django-tidb[vector]</code></td>
<td><a href="/tidbcloud/vector-search-integrate-with-django-orm">Integrate TiDB Vector Search with Django</a></td>
</tr>
</table>

0 comments on commit aeadbcf

Please sign in to comment.