Beyond Small Context: Strategies for Retrieving Detailed Summaries with LLMs?

[RLinnae]

Hi there,

I am interested in exploring the capabilities and limitations of using indexed vector stores as input for GPT and other language models (LLMs). Specifically, I am looking to understand how to retrieve detailed summaries of large documents and whether this requires a different architectural approach than retrieving specific answers to small context questions.

Based on my understanding, indexed vector stores can be used to find relevant context and insert it into the prompt alongside the question for the LLM. However, I have noticed that when attempting to retrieve detailed summaries of large documents, certain sections of the text are given disproportionate focus, while other sections seem to be ignored completely, even with specific prompting. For example, in the case of analyzing a movie script for character development, the chatbot may become fixated on a single scene rather than providing a comprehensive summary of the character arc throughout the script.

To improve my approach, I have a few specific questions that I would appreciate some guidance on:

1. How can I analyze the complete prompt that is sent to the LLM after it retrieves data from the vector database?
2. Is there a different architectural approach required for retrieving detailed summaries of large documents compared to answering specific questions with small context?
3. Are there prompt engineering approaches that I may be overlooking in my attempts to retrieve detailed summaries?
4. Why would certain context within a large text, such as a book or movie script, be almost completely ignored even if it seems relevant to me as the reader? Could this be related to the way the text is chunked?

Thank you for your help.