Chat securely with your documents using private LLM

Solution Architecture

Tech Stack

1. Python 3.10 or higher
2. Pinecone as Vector Store
3. Huggingface and Transformers for locally deploying LLM (We are using Google's Flan T5 Base model with 248M parameters (3GB) - so that my laptop doesn't die)
4. Langchain for Conversation chain wrapper around the LLM
5. Streamlit for the UI
6. AWS Sagemaker for scaled up version

Running it locally

Create a virtual environment :

virtualenv .venv

Install requirements :

pip3 install requirements.txt

Keep your documents under doc_chat/assets

Indexing the documents to create embeddings and store it in Pinecone :

python3 indexing.py

Chat with the document :

streamlit run main.py

Snapshots

Sample user Q&A :

Fetching query-relevant context from documents :