Fix readme and api version (#293)

* Fix readme and api version * Address comment * Add empty line
aigc-apps · Dec 2, 2024 · c5aa45c · c5aa45c
1 parent 9775485
commit c5aa45c
Show file tree

Hide file tree

Showing 26 changed files with 635 additions and 254 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -83,13 +83,13 @@ repos:
         exclude: |
           poetry.lock|
           (\/.*?\.[\w:]+)/pyproject.toml|
-          (\/.*?\.[\w:]+)/poetry.lock
+          (\/.*?\.[\w:]+)/poetry.lock|
         args:
           [
             "--exclude",
             "./data/tokenization/qwen.tiktoken",
             "--skip",
-            "*.json",
+            "*.json,*.txt",
             "--ignore-words-list",
             "astroid,gallary,momento,narl,ot,rouge,nin,gere",
           ]

diff --git a/README.md b/README.md
@@ -41,23 +41,111 @@ PAI-RAG is an easy-to-use opensource framework for modular RAG (Retrieval-Augmen
 
 # 🔎 Get Started
 
-## Run in Docker
+You can run PAI-RAG locally using either a Docker environment or directly from the source code.
+
+## Run with Docker
+
+1. Set up the environmental variables.
 
-1. Setup environmental variables.
    ```bash
-   cd docker
+   git clone [email protected]:aigc-apps/PAI-RAG.git
+   cd PAI-RAG/docker
    cp .env.example .env
    ```
-   edit `.env` file if you are using dashscope api or oss store:
-2. Start with docker compose command:
+
+   Edit `.env` file if you are using dashscope api or oss store. See [.env.example](./docker/.env.example) for more details.
+   Note you can also configure these settings from our console ui, but it's more safe to configure from environmental variables.
+
+2. Start the Docker containers with the following command
    ```bash
    docker compose up -d
    ```
-3. Now you can open http://localhost:8000 to check whether it works. The service will need to download the model weights, which may take a while (usually 20 minutes).
+3. Open your web browser and navigate to http://localhost:8000 to verify that the service is running. The service will need to download the model weights, which may take around 20 minutes.
+
+## Run in a Local Environment
+
+If you prefer to run or develop PAI-RAG locally, please refer to [local development guide](./docs/develop/local_develop.md)
+
+## Simple Query Using the Web UI
+
+1. Open http://localhost:8000 in your web browser. Adjust the index and LLM settings to your preferred models
+
+<img src="docs/figures/quick_start/setting.png" width="600px"/>
+
+2. Go to the "Upload" tab and upload the test data: ./example_data/paul_graham/paul_graham_essay.txt.
+
+<img src="docs/figures/quick_start/upload.png" width="600px"/>
+
+3. Once the upload is complete, switch to the "Chat" tab.
+
+<img src="docs/figures/quick_start/query.png" width="600px"/>
+
+## Simple Query Using the RAG API
+
+1. Open http://localhost:8000 in your web browser. Adjust the index and LLM settings to your preferred models
+
+2. Upload data via API:
+   Go to the PAI-RAG base directory
+
+   ```shell
+   cd PAI-RAG
+   ```
 
-## Run in Local Environment
+   **Request**
 
-If you want to start running/developing pai_rag locally, please refer to [local development](./docs/develop/local_develop.md)
+   ```shell
+   curl -X 'POST' http://localhost:8000/api/v1/upload_data \
+      -H 'Content-Type: multipart/form-data' \
+      -F 'files=@example_data/paul_graham/paul_graham_essay.txt'
+   ```
+
+   **Response**
+
+   ```json
+   {
+     "task_id": "1bcea36a1db740d28194df8af40c7226"
+   }
+   ```
+
+3. Check the status of the upload job:
+
+   **Request**
+
+   ```shell
+   curl 'http://localhost:8000/api/v1/get_upload_state?task_id=1bcea36a1db740d28194df8af40c7226'
+   ```
+
+   **Response**
+
+   ```json
+   {
+     "task_id": "1bcea36a1db740d28194df8af40c7226",
+     "status": "completed",
+     "detail": null
+   }
+   ```
+
+4. Perform a RAG query:
+
+   **Request**
+
+   ```shell
+   curl -X 'POST' http://localhost:8000/api/v1/query \
+      -H "Content-Type: application/json" \
+      -d '{"question":"What did the author do growing up?"}'
+   ```
+
+   **Response**
+
+   ```json
+   {
+      "answer":"Growing up, the author worked on writing and programming outside of school. Specifically, he wrote short stories, which he now considers to be awful due to their lack of plot and focus on characters with strong feelings. In terms of programming, he first tried writing programs on an IBM 1401 in 9th grade, using an early version of Fortran. The experience was limited because the only form of input for programs was data stored on punched cards, and he didn't have much data to work with. Later, after getting a TRS-80 microcomputer around 1980, he really started programming by creating simple games, a program to predict the flight height of model rockets, and even a word processor that his father used to write at least one book.",
+      "session_id":"ba245d630f4d44a295514345a05c24a3",
+      "docs":[
+         ...
+      ]
+   }
+   ```
 
 # 📜 Documents
 

diff --git a/README_zh.md b/README_zh.md
@@ -38,18 +38,21 @@ PAI-RAG 是一个易于使用的模块化 RAG（检索增强生成）开源框
 
 ## Docker镜像启动
 
-为了更方便使用，节省较长时间的环境安装问题，我们也提供了直接基于镜像启动的方式。
+您可以通过两种方式在本地运行 PAI-RAG：Docker 环境或直接从源代码运行。
 
-1. 配置环境变量
+1. 设置环境变量
 
    ```bash
-   cd docker
+   git clone [email protected]:aigc-apps/PAI-RAG.git
+   cd PAI-RAG/docker
    cp .env.example .env
    ```
 
-   如果你需要使用dashscope api或者OSS存储，可以根据需要修改.env中的环境变量。
+   如果您需要使用通义千问API或者阿里云OSS存储，请编辑 .env 文件。
+   其中DASHSCOPE_API_KEY获取地址为 https://dashscope.console.aliyun.com/apiKey。
+   当服务启动后您依然可以在WEB UI中配置这些API_KEY信息，但是我们建议您通过环境变量的方式配置。
 
-2. 启动
+2. 使用`docker compose`命令启动服务：
 
    ```bash
    docker-compose up -d
@@ -59,7 +62,89 @@ PAI-RAG 是一个易于使用的模块化 RAG（检索增强生成）开源框
 
 ## 本地启动
 
-如果想在本地启动或者进行代码开发，可以参考文档：[本地运行](./docs/develop/local_develop_zh.md)
+如果想在本地启动或者进行代码开发，可以参考文档：[本地开发指南](./docs/develop/local_develop_zh.md)
+
+## 通过Web UI查询的示例
+
+1. 打开 http://localhost:8000 在浏览器中。根据需要调整索引和LLM设置。
+
+   <img src="docs/figures/quick_start/setting.png" width="600px"/>
+
+2. 访问"上传"页面，上传测试数据：./example_data/paul_graham/paul_graham_essay.txt。
+
+   <img src="docs/figures/quick_start/upload.png" width="600px"/>
+
+3. 切换到"聊天"页面, 进行对话。
+
+   <img src="docs/figures/quick_start/query.png" width="600px"/>
+
+## 通过API接口查询的示例
+
+1. 打开 http://localhost:8000 在浏览器中。根据需要调整索引和LLM设置。
+
+2. 使用API上传数据：
+
+   切换到`PAI-RAG`目录
+
+   ```shell
+   cd PAI-RAG
+   ```
+
+   **请求**
+
+   ```shell
+   curl -X 'POST' http://localhost:8000/api/v1/upload_data \
+   -H 'Content-Type: multipart/form-data' \
+      -F 'files=@example_data/paul_graham/paul_graham_essay.txt'
+   ```
+
+   **响应**
+
+   ```json
+   {
+     "task_id": "1bcea36a1db740d28194df8af40c7226"
+   }
+   ```
+
+3. 检查上传任务的状态：
+
+   **请求**
+
+   ```shell
+   curl 'http://localhost:8000/api/v1/get_upload_state?task_id=1bcea36a1db740d28194df8af40c7226'
+   ```
+
+   **响应**
+
+   ```json
+   {
+     "task_id": "1bcea36a1db740d28194df8af40c7226",
+     "status": "completed",
+     "detail": null
+   }
+   ```
+
+4. Perform a RAG query:
+
+   **请求**
+
+   ```shell
+   curl -X 'POST' http://localhost:8000/api/v1/query \
+      -H "Content-Type: application/json" \
+      -d '{"question":"What did the author do growing up?"}'
+   ```
+
+   **响应**
+
+   ```json
+   {
+      "answer":"Growing up, the author worked on writing and programming outside of school. Specifically, he wrote short stories, which he now considers to be awful due to their lack of plot and focus on characters with strong feelings. In terms of programming, he first tried writing programs on an IBM 1401 in 9th grade, using an early version of Fortran. The experience was limited because the only form of input for programs was data stored on punched cards, and he didn't have much data to work with. Later, after getting a TRS-80 microcomputer around 1980, he really started programming by creating simple games, a program to predict the flight height of model rockets, and even a word processor that his father used to write at least one book.",
+      "session_id":"ba245d630f4d44a295514345a05c24a3",
+      "docs":[
+         ...
+      ]
+   }
+   ```
 
 # 📜 文档
 

diff --git a/docker/.env.example b/docker/.env.example
@@ -4,3 +4,5 @@ DASHSCOPE_API_KEY=
 # OSS AK SK
 OSS_ACCESS_KEY_ID=
 OSS_ACCESS_KEY_SECRET=
+OSS_BUCKET=
+OSS_ENDPOINT=
diff --git a/docker/compose.yaml b/docker/compose.yaml
@@ -8,13 +8,15 @@ services:
       DASHSCOPE_API_KEY: ${DASHSCOPE_API_KEY}
       OSS_ACCESS_KEY_ID: ${OSS_ACCESS_KEY_ID}
       OSS_ACCESS_KEY_SECRET: ${OSS_ACCESS_KEY_SECRET}
+      PAIRAG_RAG__oss_store__bucket: ${OSS_BUCKET}
+      PAIRAG_RAG__oss_store__endpoint: ${OSS_ENDPOINT:-oss-cn-hangzhou.aliyuncs.com}
 
     volumes:
       - ../model_repository:/app/model_repository
       - ./app_data:/app/localdata
     entrypoint: ["pai_rag", "serve"]
     healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8001/health"]
+      test: ["CMD", "curl", "-f", "http://localhost:8001/api/v1/health"]
       interval: 30s
       retries: 40
       start_period: 20s
@@ -24,6 +26,8 @@ services:
     ports:
       - "8002:8002"
     restart: always
+    environment:
+      DASHSCOPE_API_KEY: ${DASHSCOPE_API_KEY}
     depends_on:
       - api
     entrypoint: ["pai_rag", "ui", "-c", "http://api:8001"]

diff --git a/docker/nginx/default.conf b/docker/nginx/default.conf
@@ -33,7 +33,7 @@ server {
         proxy_pass http://127.0.0.1:8001;
     }
 
-    location /health {
+    location /api {
         proxy_set_header Host \$host;
         proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
         proxy_pass http://127.0.0.1:8001;

diff --git a/docker/nginx/default.conf.compose b/docker/nginx/default.conf.compose
@@ -33,7 +33,7 @@ server {
         proxy_pass http://api:8001;
     }
 
-    location /health {
+    location /api {
         proxy_set_header Host \$host;
         proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
         proxy_pass http://api:8001;

diff --git a/docs/agentic_rag.md b/docs/agentic_rag.md
@@ -76,7 +76,7 @@ RAG提供了非常好的知识库查询能力，然而在现实场景中问答
       }
     },
     "required": ["from_city", "to_city", "date"],
-    "url": "http://127.0.0.1:8001/demo/api/flights"
+    "url": "http://localhost:8001/demo/api/flights"
   },
   {
     "content_type": "application/json",
@@ -101,7 +101,7 @@ RAG提供了非常好的知识库查询能力，然而在现实场景中问答
       }
     },
     "required": ["city", "checkin_date", "checkout_date"],
-    "url": "http://127.0.0.1:8001/demo/api/hotels"
+    "url": "http://localhost:8001/demo/api/hotels"
   }
 ]
 ```
@@ -220,7 +220,7 @@ Agentic RAG支持通过API调用的方式，以支持更多复杂的业务场景
 ### RAG - 知识库问答
 
 ```sh
-curl -X POST http://localhost:8001/service/query \
+curl -X POST http://localhost:8001/api/v1/query \
 -H "Authorization: YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{"question": "什么是组件化技术", "with_intent": true, "stream": true}'
@@ -348,7 +348,7 @@ data: {"delta": "", "is_finished": true, "session_id": "671bb4e8ca324a34ac6fe7f1
 ### 使用工具 - 查询时间（内置工具）
 
 ```sh
-curl -X POST http://localhost:8001/service/query \
+curl -X POST http://localhost:8001/api/v1/query \
 -H "Authorization: YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{"question": "今天的日期", "with_intent": true, "stream": true}'
@@ -381,7 +381,7 @@ data: {"delta": "", "is_finished": true}
 ### 使用工具 - 查询天气
 
 ```sh
-curl -X POST http://localhost:8001/service/query \
+curl -X POST http://localhost:8001/api/v1/query \
 -H "Authorization: YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{"question": "杭州的天气", "with_intent": true, "stream": true}'