You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
JsonCpp library used to parse scroll results is extremely slow. Scrolling through larger indices show the delay much more. In numbers: what the current implementation iterates in 85 seconds, scroll implemented using rapidjson did in 25 seconds.
Unfortunately JsonCpp is used in Scroll header interface, so the change would break API/ABI. But the speed is worth it.
Possible implementation with rapidjson (as it is header only library):
hide it completely inside elasticlient, without publishing it in public interface
make the rapidjson parsing public (header only), so each client could compile it again
First option means that parsing has to be done twice. Internally and then for data parsing itself. Otherwise there is a risk of using different versions (binary incompatible) of rapidjson.
The second option allows shared parsing (internall and client's data) without possibility of using different versions of rapidjson (one compiled into elasticlient, another version in client code).
The text was updated successfully, but these errors were encountered:
JsonCpp library used to parse scroll results is extremely slow. Scrolling through larger indices show the delay much more. In numbers: what the current implementation iterates in 85 seconds, scroll implemented using rapidjson did in 25 seconds.
Unfortunately JsonCpp is used in Scroll header interface, so the change would break API/ABI. But the speed is worth it.
Possible implementation with rapidjson (as it is header only library):
First option means that parsing has to be done twice. Internally and then for data parsing itself. Otherwise there is a risk of using different versions (binary incompatible) of rapidjson.
The second option allows shared parsing (internall and client's data) without possibility of using different versions of rapidjson (one compiled into elasticlient, another version in client code).
The text was updated successfully, but these errors were encountered: