refactored colbert codebase

[epinzur]

• dropped multiprocess stuff after seeing in tests that it runs slower than the single process code.
• split the BaseVectorStore abstract class into BaseVectorStore and BaseDatabase abstract classes
  • BaseDatabase contains the interface for interacting with the database to manage CRUD and do the proper queries for ColBERT retrieval.
  • BaseVectorStore contains the interface for creating a LLamaIndex or LangChain VectorStore
• Created a CassandraDatabase implementation for the BaseDatabase abstract class
• Created a ColbertVectorStore implementation for the BaseVectorStore abstract class
• Created a ragstack-langchain.colbert ColbertVectorStore class that follows the standard langchain vector-store patterns
• Created a ragstack-llamaindex.colbert ColbertVectorStore class that follows the standard llamaindex vector-store patterns
• Renamed ragstack-langchain.colbert ColbertLCRetriever to ColbertRetriever and updated methods to match the standard langchain retrieval patterns
• Renamed ragstack-llamaindex.colbert ColbertLIRetriever to ColbertRetriever and updated methods to match the standard llamaindex retrieval patterns
• Added async methods for a few of the classes. More to come in a future PR.
• Updated the llamaindex and langchain integration tests to more closely follow standard ways of doing RAG ingest and retrieval for those packages.
• Dropped internal use of nest_asyncio. If running in a Jupyter environment, the user should run nest_asyncio.apply() there. It should NOT be in our package.be
• Dropped all my different helper objects (BaseChunk, DataChunk, RetrievedChunk, EmbeddedChunk, etc...) in favor of a single Chunk object.
• Created a helper class for sharing test data

Probably more stuff that I'm forgetting...

There is still some more stuff to implement on the llamaindex side, but I'd love to get a release out that is working for langchain users.

Also I'll add more robust testing later.

[mlr]

Antora site build successful! ✅
Deploying draft.
Deployment successful! View draft

[zzzming]

LGTM