How can I get the answer without adding memory again?

[water-in-stone]

🐛 Describe the bug

import os
from openai import OpenAI
from mem0 import Memory
from extract_info import extract_structured_info
from extract_info import read_file
from extract_info import get_file_address

os.environ["OPENAI_API_KEY"] = "sk-test"

config = {
    "http_client_proxies"
    "llm": {
        "provider": "openai",
        "config": {
            "model": "gpt-4o",
            "temperature": 0.2,
            "max_tokens": 1500,
        },
    },
    "embedder": {"provider": "openai", "config": {"model": "text-embedding-3-large"}},
    "vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test",
            "embedding_model_dims": 3072,
        },
    },
    "version": "v1.1",
}


class BrowserAIAgent:
    def __init__(self):
        self.client = OpenAI()
        self.memory = Memory.from_config(config)
        self.messages = [
            {
                "role": "system",
                "content": "You are a personal AI Assistant for Browser.",
            }
        ]

    def ask_question(self, question, user_id):
        # Fetch previous related memories
        previous_memories = self.search_memories(question, user_id=user_id)
        prompt = question
        if previous_memories:
            prompt = f"User input: {question}\n Previous memories: {previous_memories}"
        # enable multiple round chat
        self.messages.append({"role": "user", "content": prompt})

        response = self.client.chat.completions.create(
            model="gpt-4o", messages=self.messages
        )
        answer = response.choices[0].message.content
        self.messages.append({"role": "assistant", "content": answer})

        return answer

    def get_memories(self, user_id):
        memories = self.memory.get_all(user_id=user_id)
        return [m["memory"] for m in memories["memories"]]

    def search_memories(self, query, user_id):
        memories = self.memory.search(query, user_id=user_id)
        return [m["memory"] for m in memories["memories"]]

    def add_memory(self, memory, user_id, metadata):
        self.memory.add(memory, user_id=user_id, metadata=metadata)


user_id = "browser"
ai_assistant = BrowserAIAgent()


def add_memories_for_browser():
    add_browser_docs("/Users/browser/docs/document")


def add_browser_docs(directory_path):
    files_address = get_file_address(directory_path)

    for file_address in files_address:
        file_path = os.path.join(directory_path, file_address)
        print("try to analyze ", file_address, ", file path is ", file_path)
        markdown_text = read_file(file_path)
        structured_info = extract_structured_info(markdown_text)

        ai_assistant.add_memory(
            markdown_text, user_id=user_id, metadata=structured_info
        )


def main():
    # I have added some documents for the Browser before, and I don't want to add them again.
    # add_memories_for_browser()

    while True:
        question = input("Question: ")
        if question.lower() in ["q", "exit"]:
            print("Exiting...")
            break

        answer = ai_assistant.ask_question(question, user_id=user_id)
        print(f"Answer: {answer}")
        memories = ai_assistant.get_memories(user_id=user_id)
        print("Memories:")
        for memory in memories:
            print(f"- {memory}")
        print("-----")


if __name__ == "__main__":
    main()

My source code is shown above and stored in a file named 'agent.py'. I can input a question and get the correct answer after I execute python agent.py. However, I cannot get the correct answer when I execute agent.py again with the same question. Is Mem0 only able to store data in memory and not on the hard disk?

[spike-spiegel-21]

Hi @water-in-stone , It looks like you are not persisting your vector store. It means that the vector store configuration that you are using currently:

"vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test",
            "embedding_model_dims": 3072,
        },
    }

will create a temporary collection in tmp/qdrant that you will loose as soon as you close your program agent.py. I recommend you to use docker container configuration by using these commands:

docker pull qdrant/qdrant

docker run -p 6333:6333 -p 6334:6334 \
    -v $(pwd)/qdrant_storage:/qdrant/storage:z \
    qdrant/qdrant

and then use these configurations:

"vector_store": {
        "provider": "qdrant",
        "config": {
            "host": "localhost",
            "port": 6333,
        }
    },

[water-in-stone]

Hi @water-in-stone , It looks like you are not persisting your vector store. It means that the vector store configuration that you are using currently:

"vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test",
            "embedding_model_dims": 3072,
        },
    }

will create a temporary collection in tmp/qdrant that you will loose as soon as you close your program agent.py. I recommend you to use docker container configuration by using these commands:

docker pull qdrant/qdrant

docker run -p 6333:6333 -p 6334:6334 \
    -v $(pwd)/qdrant_storage:/qdrant/storage:z \
    qdrant/qdrant

and then use these configurations:

"vector_store": {
        "provider": "qdrant",
        "config": {
            "host": "localhost",
            "port": 6333,
        }
    },

Good answer! I will use configurations you provided.

[water-in-stone]

@spike-spiegel-21 Due to some complex reasons, I cannot use Docker. How can I use a local Qdrant database without Docker?

[Cirr0e]

Hey there! I've got a solid solution to help you persist your Qdrant vector store memories locally without Docker. 🚀

Based on the issue search and documentation, here's a robust approach to local storage:

1. Modify your configuration to use a persistent local path:

import os

# Create a persistent storage directory
QDRANT_STORAGE_PATH = os.path.join(os.path.dirname(__file__), 'qdrant_storage')
os.makedirs(QDRANT_STORAGE_PATH, exist_ok=True)

config = {
    "llm": {...},  # Your existing LLM configuration
    "embedder": {...},  # Your existing embedder configuration
    "vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test",
            "embedding_model_dims": 3072,
            "path": QDRANT_STORAGE_PATH,  # Persistent local storage
            "host": "localhost",  # Optional: explicitly set host
            "port": 6333,  # Optional: standard Qdrant port
        },
    },
    "version": "v1.1",
}

2. Best Practices for Data Management:

• The path parameter ensures your vector store data is saved locally
• os.makedirs() creates the directory if it doesn't exist
• Using a relative path keeps the solution portable

3. Data Backup and Migration Tip:
   If you ever need to move or backup your data, Qdrant supports snapshots:

import requests

# Create a snapshot (you might want to add this as a method in your class)
def create_qdrant_snapshot(host='localhost', port=6333):
    response = requests.post(f'http://{host}:{port}/snapshots')
    if response.status_code == 200:
        print("Snapshot created successfully!")
    else:
        print("Snapshot creation failed.")

4. Key Considerations:

• This approach keeps everything local
• Memories will persist between program runs
• No Docker required
• Easy to backup and manage

🌟 Pro Tip: Regularly backup your qdrant_storage directory, especially before making significant changes to your system.

Let me know if this works for you or if you need any further customization! Happy coding! 👨‍💻

[water-in-stone]

Hey there! I've got a solid solution to help you persist your Qdrant vector store memories locally without Docker. 🚀

Based on the issue search and documentation, here's a robust approach to local storage:

1. Modify your configuration to use a persistent local path:

import os

# Create a persistent storage directory
QDRANT_STORAGE_PATH = os.path.join(os.path.dirname(__file__), 'qdrant_storage')
os.makedirs(QDRANT_STORAGE_PATH, exist_ok=True)

config = {
    "llm": {...},  # Your existing LLM configuration
    "embedder": {...},  # Your existing embedder configuration
    "vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test",
            "embedding_model_dims": 3072,
            "path": QDRANT_STORAGE_PATH,  # Persistent local storage
            "host": "localhost",  # Optional: explicitly set host
            "port": 6333,  # Optional: standard Qdrant port
        },
    },
    "version": "v1.1",
}

2. Best Practices for Data Management:

• The path parameter ensures your vector store data is saved locally
• os.makedirs() creates the directory if it doesn't exist
• Using a relative path keeps the solution portable

3. Data Backup and Migration Tip:
   If you ever need to move or backup your data, Qdrant supports snapshots:

import requests

# Create a snapshot (you might want to add this as a method in your class)
def create_qdrant_snapshot(host='localhost', port=6333):
    response = requests.post(f'http://{host}:{port}/snapshots')
    if response.status_code == 200:
        print("Snapshot created successfully!")
    else:
        print("Snapshot creation failed.")

4. Key Considerations:

• This approach keeps everything local
• Memories will persist between program runs
• No Docker required
• Easy to backup and manage

🌟 Pro Tip: Regularly backup your qdrant_storage directory, especially before making significant changes to your system.

Let me know if this works for you or if you need any further customization! Happy coding! 👨‍💻

Thanks a lot!