Skip to content

Latest commit

 

History

History
123 lines (84 loc) · 5.56 KB

README.md

File metadata and controls

123 lines (84 loc) · 5.56 KB
Raw

Chat with PDF 🗣️💬📄

Chat with PDF is a full-stack AI-powered application that lets you to ask questions to PDF documents.

The application is running with server-side rendering on the edge using Cloudflare Pages.

You can deploy it with zero configuration on your Cloudflare account using NuxtHub:

Deploy to NuxtHub

🚀 Key Features

  • Hybrid RAG: Hybrid RAG using Full-Text Search on D1 and Vector Search on Vectorize
  • Streamed Responses: Information is streamed real-time to the UI using Server-Sent Events
  • High-Performance: Deployed on the edge with server-side rendering using Cloudflare Pages

🛠 Tech Stack

  • Nuxt - The Intuitive Vue Framework
  • Nuxt UI - Beautiful UI library with TailwindCSS
  • Drizzle ORM - Powerful modern TypeScript ORM
  • unpdf - Platform-agnostic version of PDF.js for serverless environments
  • NuxtHub Rate Limit - Ratelimiting requests
  • NuxtHub - Build & deploy to your Cloudflare account with zero configuration
    • hubBlob() to store PDFs in Cloudflare R2
    • hubDatabase() to store document chunks and full-text search on Cloudflare D1
    • hubAI() to run Cloudflare AI models for LLM chat and generating text embeddings
    • hubVectorize() to find relevant document context in Cloudflare Vectorize
    • hubKV() for IP ratelimiting
  • npx nuxthub deploy - To deploy the app on your Cloudflare account for free

🏎️ How does it work?

Hybrid Search RAG

This project uses a combination of classical Full Text Search (sparse) against Cloudflare D1 and Hybrid Search with embeddings against Vectorize (dense) to provide the best of both worlds providing the most applicable context to the LLM.

The way it works is this:

  1. We take user input and we rewrite it to 5 different queries using an LLM
  2. We run each of these queries against our both datastores - D1 database using BM25 for full-text search and Vectorize for dense retrieval
  3. We take the results from both datastores and we merge them together using Reciprocal Rank Fusion which provides us with a single list of results
  4. We then take the top 10 results from this list and we pass them to the LLM to generate a response

Credits: https://github.com/RafalWilinski/cloudflare-rag#hybrid-search-rag

🚀 Quick Start

  1. Install dependencies with pnpm 
    pnpm install
  2. Create & link a NuxtHub project to enable running AI models on your Cloudflare account 
    npx nuxthub link
  3. Deploy the application to your Cloudflare account 
    npx nuxthub deploy
  4. Launch the dev server 
    pnpm dev --remote

Visit http://localhost:3000 and start chatting with documents!

🌐 Deploy to the World for Free

Host your Chat with PDF instance on a free Cloudflare account and free NuxtHub account.

Deploy it online in the NuxtHub UI:

Deploy to NuxtHub

Or locally with the NuxtHub CLI:

npx nuxthub deploy

This command will deploy your Chat with PDF instance to your Cloudflare account and provision a Cloudflare R2 bucket. You will also get a free <your-app>.nuxt.dev domain.

What's included in Cloudflare free plan:

  • 100,000 requests/day
  • 10 GB storage on Cloudflare R2
  • 10,000 tokens a day for Workers AI
  • 30 million queried vector dimensions / month
  • 5 million stored vector dimensions

Read more about the pricing on our detailed pricing page.

You can also deploy using Cloudflare Pages CI or GitHub actions.

Remote Storage

Once your project is deployed, you can use NuxtHub Remote Storage to connect to your preview or production Cloudflare R2 bucket in development using the --remote flag:

pnpm dev --remote

🔗 Useful Links

📝 License

Published under the MIT license.

🙋 Credits

  • cloudflare-rag by Rafal Wilinski - Chat with PDF is a port of the cloudflare-rag project to NuxtHub and Nuxt UI. The core logic and functionality are derived from cloudflare-rag, adapted to work with NuxtHub.
  • hub-chat by Rajeev R Sharma - Parts of UI and inspiration for the streaming composable.