establish more connections in cassandra-stress

[glommer]

In many occasions we are forced to run multiple clients not because there is a bottleneck in the client machine, but because the low number of connections.

Ideally, we would make the number of connections configurable, and then tune that. A change like this should find its way to the upstream cassandra-stress project.

Bonus point if we can have a wrapper script that somehow connects to the Scylla REST or other jmx metrics, finds out what is the number of shards, and opens as many connections as there are shards.

[avikivity]

REST isn't a good idea, because you need to set up another connection with its own configurable port number, authentication, and encryption.

Better read a new column from system.local or a new virtual table. This could then be done in the driver and apply to all applications, not just cassandra-stress.

[glommer]

I am totally fine with that.

[glommer]

As of a0076e7, cassandra-stress upstream supports setting the number of connections.

Since we also want a newer version to fix coordinated omission, I strongly suggest we upgrade.

[eyalgutkind]

Did we resolve this one?

[avikivity]

This should be resolved by bundling the shard-aware driver instead of the datastax driver.

@haaawk what's the best way to do that? IMO we should do a formal release of the shard-aware driver (i.e. with a tag, version number, and upload to the binary repo) and bundle that.

[haaawk]

@avikivity I agree that publishing our driver as maven artifact is the best way to do it. In the mean time we can just put our driver into tools instalation.

[avikivity]

What does "put our driver into tools installation" mean? Copy it to libs/? I don't want to use a non-released version (although, it's not really that terrible if it's a hidden component).

btw we're also facing a similar problem with the Python driver that is bundled in scylla-tools-java.git.

[haaawk]

I would have to double check but putting the jar in libs should work. We might need to change some build scripts too but I'm not 100% sure without giving it a try.