Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go Client v7.3.0 #433

Merged
merged 31 commits into from
May 8, 2024
Merged

Go Client v7.3.0 #433

merged 31 commits into from
May 8, 2024

Conversation

khaf
Copy link
Collaborator

@khaf khaf commented May 8, 2024

This is a major feature release of the Go client and touches some of the fundamental aspects of the inner workings of it.
We suggest complete testing of your application before using it in production.

  • New Features

    • CLIENT-2238 Convert batch calls with just one key per node in sub-batches to Get requests.
      If the number keys for a sub-batch to a node is equal or less then the value set in BatchPolicy.DirectGetThreshold, the client use direct get instead of batch commands to reduce the load on the server.

    • CLIENT-2274 Use constant sized connection buffers and resize the connection buffers over time.

      The client would use a single buffer on the connection and would grow it
      per demand in case it needed a bigger buffer, but would not shrink it.
      This helped with avoiding using buffer pools and the associated
      synchronization, but resulted in excessive memory use in case there were a
      few large records in the results, even if they were infrequent.
      This changeset does two things:
      1. Will use a memory pool for large records only. Large records
      are defined as records bigger than aerospike.PoolCutOffBufferSize.
      This is a tiered pool with different buffer sizes. The pool
      uses sync.Pool under the cover, releasing unused buffers back to the
      runtime.
      2. By using bigger aerospike.DefaultBufferSize values, the user can
      imitate the old behavior, so no memory pool is used most of the time.
      3. Setting aerospike.MinBufferSize will prevent the pool using buffer sizes too small,
      having to grow them frequently.
      4. Buffers are resized every 5 seconds to the median size of buffers used over the previous period,
      within the above limits.

      This change should result in much lower memory use by the client.

    • CLIENT-2702 Support Client Transaction Metrics. The native client can now track transaction latencies using histograms. Enable using the Client.EnableMetrics API.

  • Improvements

  • Fixes

    • [CLIENT-2905] Fix inconsistency of handling in-doubt flag in errors.
    • [CLIENT-2890] Support []MapPair return in reflection.
      This fix supports unmarshalling ordered maps into map[K]V and []MapPair in the structs.

khaf and others added 30 commits April 16, 2024 14:59
This fix supports unmarshalling ordered maps into hash maps map[K]V and []MapPair in the structs.
The client would use a single buffer on the connection and would grow it
per demand in case it needed a bigger buffer, but would not shrink it.
This helped with avoiding using buffer pools and the associated
synchrinization, but resulted in hugew memory use in case there were a
few large records in the results, even if they were infrequent.
This changeset does two things:
	1. Will use a memory pool for large records only. Large records
	   are defined as records bigger than aerospike.DefaultBufferSize.
	   This is a tiered pool with different buffer sizes. The pool
	   uses sync.Pool under the cover, releasing unused buffers back to the
	   runtime.

	2. By using bigger aerospike.DefaultBufferSize values, the user can
	   immitate the old behavior, so no memory pool is used.

This change should result in much lower memory use by the client.
Signed-off-by: Swarit Pandey <[email protected]>
Add a histogram and use the median value in intervals to readjust the connection buffer down or upwards to optimize memory use.
"math/rand" is fast enough now
…b-batches to Get requests

If the number keys for a sub-batch to a node is equal or less then the value set in BatchPolicy.DirectGetThreshold, the client use direct get instead of batch commands to reduce the load on the server.
…b-batches to Get requests

If the number keys for a sub-batch to a node is equal or less then the value set in BatchPolicy.DirectGetThreshold, the client use direct get instead of batch commands to reduce the load on the server.
…le key batch commands to signle operate commands
@khaf khaf requested a review from justinlee-aerospike May 8, 2024 18:52
Copy link

@justinlee-aerospike justinlee-aerospike left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes look good

@khaf khaf merged commit 043b81d into v7 May 8, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reading a single value from a map cdt via Operate()
3 participants