Flinch

Flinch is an in-memory, real-time document database designed for fast, efficient full-text search and querying. It provides a high-performance search solution that can be seamlessly integrated into various applications. Flinch is lightweight, easy to use, and open-source, allowing users to contribute to its development and customize it to suit their needs.

Features

• In-memory database: Flinch stores documents in memory, enabling ultra-fast search performance.
• Real-time updates: Users can add, update, and delete documents in real-time, ensuring the database reflects the latest changes.
• Full-text search: Flinch includes a built-in full-text search engine that offers powerful search capabilities, including "search-as-you-type" and wildcard search.
• Lightweight and easy to use: Flinch is designed to be lightweight and has a simple API, making it easy for developers to integrate into their applications.
• Document-oriented: Flinch is a document-oriented database, allowing users to store and retrieve documents as JSON objects.
• Highly scalable: Flinch is built to handle large volumes of documents and queries efficiently, ensuring scalability as your application grows.
• Query: Flinch offers high-speed document querying capabilities, delivering query performance faster than lightning (⚡️).
• Open source: Flinch is an open-source project, enabling users to contribute to its development and customize it according to their requirements.

Note: Insertion performance may be slow due to the time taken for indexing, which is a normal occurrence.

How to Use

Run 👇it:

cargo add flinch

or add this in your Cargo.toml

flinch = "<YOUR_VERSION>"

These examples demonstrate the capabilities of Flinch as both a library and a query language, showcasing its features for document storage, retrieval, manipulation, and searching.

Example 1: Using Flinch as a Library

The first example demonstrates how to use Flinch as a library in your Rust application. It showcases various operations such as adding documents, updating values, retrieving data, performing searches, and more.

Here are the key operations performed in the code:

1. Creating a Collection: A collection is created with specific options, including index options, search options, view options, range options, and clip options.

2. Adding Documents: Documents are added to the collection using the put operation. Each document is represented as a JSON object, and it is assigned a unique key.

3. Real-time Updates: The code demonstrates real-time updates by subscribing to a channel and receiving events for document insertions and removals.

4. Retrieving Documents: The code shows how to retrieve documents using the get operation. It includes examples of getting a single document, getting multiple documents, and fetching documents based on an index.

5. Searching: The code demonstrates the search capabilities of Flinch. It shows how to perform a search query and retrieve documents that match the search query. It also showcases a "like" search using wildcard characters.

6. Views: Flinch supports views, which are predefined filters that can be applied to a collection. The code shows how to fetch documents based on a view configuration.

7. Dropping a Collection: The code demonstrates how to drop a collection, removing all its documents.

8. Subscribing to Pub/Sub Events: The code sets up a subscription to receive Pub/Sub events for document insertions and removals. It shows an example of listening to a limited number of events.

async fn library() {
    // Initialize Flinch with collection options
    let col_opts = CollectionOptions {
        name: Some(COLLECTION.to_string()),
        index_opts: vec![format!("name")],
        search_opts: vec![format!("name")],
        view_opts: vec![ViewConfig{
            prop: "age".to_string(),
            expected: "18".to_string(),
            view_name: "ADULT".to_string(),
        }],
        range_opts: vec![format!("age")],
        clips_opts: vec![format!("name")],
    };
    let database = Database::init();
    database.add(col_opts).expect("created new collection");

    // List available collections
    println!("ls Collections {:?}", database.ls());

    // Access the Flinch collection instance
    let instance = database.using(COLLECTION);
    assert!(instance.is_ok());
    let instance = instance.unwrap();
    let collection = instance.value();

    // Subscribe to real-time updates
    let (sx, mut rx) = tokio::sync::mpsc::channel(30000);
    collection.sub(sx).await.expect("subscribe to channel");

    // Insert documents into the collection
    let insert = Instant::now();
    let record_size = 10_000;
    for i in 0..record_size {
        collection.put(format!("P_{}",&i), QueryBased::from_str(
            serde_json::to_string(
                &User {
                    name: format!("julfikar{}",&i),
                    age: i,
                }
            ).unwrap().as_str()
        ).unwrap()).await.unwrap();
    }
    assert_eq!(collection.len(), record_size as usize);
    println!("insert:: {:?}", insert.elapsed());

    // Replace a document in the collection
    let x = collection.put(format!("P_{}",0), QueryBased::from_str(
        serde_json::to_string(
            &User {
                name: format!("julfikar-replace"),
                age: 10000,
            }
        ).unwrap().as_str()
   

 ).unwrap()).await;
    assert!(x.is_ok());
    println!("replaced value in {}", x.unwrap());

    // Get a single document from the collection
    let single = collection.get(&format!("P_0"));
    assert!(single.data.is_some());
    println!("single:: {:?}", single.time_taken);

    // Get multiple documents from the collection
    let multi = collection.multi_get(vec![&format!("P_1"), &format!("P_0")]);
    assert_ne!(multi.data.len(), 0);
    println!("multi:: {:?}", multi.time_taken);

    // Get an index from the collection
    let gidx = collection.get_index("julfikar100");
    assert!(gidx.data.is_some());
    println!("index:: {:?} {:?}", gidx.time_taken, gidx.data.unwrap().1.data);

    // Perform a search query in the collection
    let search = collection.search("Julfikar0");
    assert_ne!(search.data.len(), 0);
    println!("search index:: {} res {}", search.time_taken, search.data.len());

    // Perform a like search query in the collection
    let like_search = collection.like_search("Julfikar 101");
    assert_ne!(like_search.data.len(), 0);
    println!("search:: {} res {}", like_search.time_taken, like_search.data.len());

    // Fetch a view from the collection
    let view = collection.fetch_view("ADULT");
    assert_ne!(view.data.len(), 0);
    println!("view:: {} res {}", view.time_taken, view.data.len());

    // Drop the collection
    collection.drop().await;
    assert_eq!(collection.len(), 0, "after::drop");

    // Listen to pub/sub events for demonstration (limited to 10 messages)
    let mut i = 0;
    loop {
        let event = rx.recv().await.unwrap();
        match event {
            PubSubEvent::Data(d) => {
                match d {
                    ActionType::Insert(k, _v) => {
                        println!("inserted :pub/sub: {}", k);
                    }
                    ActionType::Remove(k) => {
                        println!("removed :: {}", k);
                    }
                };
            }
            PubSubEvent::Subscribed(_s) => {

            }
        };
        i += 1;
        if i == 10 {
            break;
        }
    }
}

Example 2: Querying with FLQL

The second example introduces FLQL (Flinch Query Language), which provides a powerful and concise syntax for interacting with Flinch collections. It demonstrates various FLQL queries and their functionalities.

Here's a breakdown of the FLQL queries showcased in the code:

1. Creating a Collection: This query creates a new collection with empty options.

2. Dropping a Collection: This query drops a collection from the database.

3. Checking Pointer Existence: The query checks if a specific pointer exists within a collection.

4. Collection Length: This query returns the length (number of documents) in a collection.

5. Updating or Inserting Documents: The query updates or inserts a document into a collection.

6. Conditional Update or Insert: This query performs a conditional update or insert based on a specified condition.

7. Updating or Inserting to a Pointer: The query updates or inserts a document into a pointer within a collection.

8. Getting Documents: This query retrieves documents from a collection.

9. Conditional Get: This query performs a conditional get operation based on a specified condition.

10. Getting Pointer: The query retrieves the value of a pointer from a collection.

11. Getting View: This query fetches documents based on a view configuration.

12. Getting Clip: This query fetches documents based on a clip configuration.

13. Getting Index: The query retrieves documents based on an index.

14. Getting Range: This query retrieves a range of documents based on specified start and end values.

15. Searching: This query performs a search query in the collection.

16. Conditional Search: This query performs a conditional search based on a specified condition.

17. Deleting Documents: This query deletes documents from a collection.

18. Conditional Delete: This query performs a conditional delete based on a specified condition.

19. Deleting Pointer: The query deletes a specific pointer from a collection.

20. Deleting View: This query deletes a view from a collection.

21. Deleting Clip: This query deletes a clip from a collection.

async fn query_example() {
    // Initialize Flinch with collection options
    let col_opts = CollectionOptions {
        name: Some(COLLECTION.to_string()),
        index_opts: vec![format!("name")],
        search_opts: vec![format!("name")],
        view_opts: vec![ViewConfig{
            prop: "age".to_string(),
            expected: "18".to_string(),
            view_name: "ADULT".to_string(),
        }],
        range_opts: vec![format!("age")],
        clips_opts: vec![format!("name")],
    };
    let options = serde_json::to_string(&col_opts).unwrap();
    let planner = Query::new();

    // Create a new collection
    let res = planner.exec(format!("new({});",options.as_str()).as_str()).await;
    println!("new::collection::error {:?}",res.error);

    // Insert documents into the collection
    let record_size = 2;
    for i in 0..record_size {
        let v = serde_json::to_string(
            &User {
                name: format!("julfikar{}",&i),
                age: i,
            }
        ).unwrap();
        let query = format!("put({}).into('{}');", v, &COLLECTION);
        let x = planner.exec(query.as_str()).await;
        assert_eq!(x.error, FlinchError::None);
    }

    // Get documents from the collection
    let res = planner.exec(format!("get.when('map(\"name\") == \"julfikar1\"').from('{}');",&COLLECTION).as_str()).await;
    println!("{:?}",res);

    // Get an index from the collection
    let res = planner.exec(format!("get.index('julfikar1').from('{}');",&COLLECTION).as_str()).await;
    println!("{:?}",res);

    // Perform a search query in the collection
    let res = planner.exec(format!("search.query('julfikar 1').from('{}');",&COLLECTION).as_str()).await;
    println!("{:?}",res);
}

More Query examples can be found in other repository https://github.com/mjm918/flinch-flql

new({});
drop('');
exists('').into('');
length('');
put({}).into('');
put({}).when('prop.name == \"acv\" OR prop.name STARTS_WITH \"ac\"').into('');
put({}).pointer('').into('');
get.from('');
get.when('prop.name == \"acv\" OR prop.name STARTS_WITH \"ac\"').from('');
get.pointer('').from('');
get.view('').from('');
get.clip('').from('');
delete.from('');
delete.when('prop.name == \"acv\" OR prop.name STARTS_WITH \"ac\"').from('');
delete.pointer('').from('');
delete.clip('').from('');

These FLQL queries can be executed using the Flinch Query Planner and provide a flexible and efficient way to interact with Flinch collections.