Move internal communication with DB to dedicated interface, introduce kind a DB provider

[antonGritsenko]

Right now we stick with SQlite, but it has huge disadvantage: it's using a file system. In many cases using of a file system is really complicated or even not possible: serverless scenario, docker etc.
I know, mainly TDlib used on client's end devices, but often it used for automation as dedicated service and running in environment where access to file system can be a quite complex task, even more complex than writing a custom DB provider.

Example: tclient running as a background service on a server (for automation purposes, Bot API is totally not enough for many cases).

Will be great to have kind an interface that implements access to the local DB, so we can implement it for the custom DB provider.

[AYMENJD]

An MTProto client isn't an option for you?

[azkadev]

An MTProto client isn't an option for you?

maybe he can't make mtproto client himself

[azkadev]

Right now we stick with SQlite, but it has huge disadvantage: it's using a file system. In many cases using of a file system is really complicated or even not possible: serverless scenario, docker etc. I know, mainly TDlib used on client's end devices, but often it used for automation as dedicated service and running in environment where access to file system can be a quite complex task, even more complex than writing a custom DB provider.

Example: tclient running as a background service on a server (for automation purposes, Bot API is totally not enough for many cases).

Will be great to have kind an interface that implements access to the local DB, so we can implement it for the custom DB provider.

tdlib is designed to be compact and fast for all platforms, so they chose sqlite. If you want to customize your own program/modify the tdlib library as you wish

[antonGritsenko]

An MTProto client isn't an option for you?

Well, if we can use MTProto then why the TDLib exists at all? ;) Sure, we can even use Assembler everywhere, but if we can make it simplified why not make it simplified? If no such plans, then for sure, its time to move to MTProto direct implementation.

I did check the code, and IMO it not so hard to move the DB communication to a dedicated interface, so provide kind DB Communication object instead the path to the DB. I'm not a C++ dev, so couldn't provide valuable PR for this. Maybe someone else will find this idea implementable.

[AYMENJD]

Well, if we can use MTProto then why the TDLib exists at all?

The main focus of TDLib is Messaging clients, with an additional support for bots as in Bot API.

I did check the code, and IMO it not so hard to move the DB communication to a dedicated interface, so provide kind DB Communication object instead the path to the DB

TDLib database is encrypted and optional which can be disabled and it's not meant to be accessed/managed by other than TDLib itself. Otherwise it may lead to unpredictable behaviors.

In short, TDLib is not suitable for storage-less environments with no file system access, as it relies on maintaining its state, which involves continuous reading and writing to a writable directory. For your use case, it would be better to use a custom MTProto implementation, where you have full control over everything.

[antonGritsenko]

TDLib database is encrypted and optional which can be disabled and it's not meant to be accessed/managed by other than TDLib itself. Otherwise it may lead to unpredictable behaviors.

Yes, sure, but the goal is not to have access to the DB directly out of TDLib, the goal is to have an external DB for TDLib.

[AYMENJD]

Having the external brings the unpredictable. The current TDLib design is to work with it's internal things, changing that requires significant effort and deep understanding.

[levlam]

Well, if we can use MTProto then why the TDLib exists at all?

TDLib exists to write alternative Telegram clients, because all other usages contradict Telegram API ToS. Besides that I have nothing to add to #3145 (comment).