Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle ssl only scylla cluster setup #4114

Merged
merged 14 commits into from
Nov 22, 2024

Conversation

VAveryanov8
Copy link
Collaborator

This fixes how SM decides which port to use when connecting to Scylla
nodes.
Also adds SSL_ENABLED flag to Makefile, so that when you run
SSL_ENABLED=true make start-dev-env the scylla cluster will be created
with ssl_only config.

Fixes #4079


Please make sure that:

  • Code is split to commits that address a single change
  • Commit messages are informative
  • Commit titles have module prefix
  • Commit titles have issue nr. suffix

@VAveryanov8 VAveryanov8 force-pushed the va/fix-only-ssl-cluster-setup branch from 72dc0d5 to 378d7b8 Compare November 15, 2024 11:38
@VAveryanov8 VAveryanov8 marked this pull request as ready for review November 15, 2024 12:36
This adds SSL_ENABLED flag to Makefile, so that when you run
SSL_ENABLED=true make start-dev-env the scylla cluster will be created
with ssl_only config.
This fixes how SM decides which port to use when connecting to Scylla
nodes.
@VAveryanov8 VAveryanov8 force-pushed the va/fix-only-ssl-cluster-setup branch from 378d7b8 to 7b7e591 Compare November 18, 2024 08:13
Copy link
Collaborator

@Michal-Leszczynski Michal-Leszczynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

testing/scylla/config/scylla-ssl.yaml Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
pkg/service/cluster/service.go Outdated Show resolved Hide resolved
pkg/service/cluster/service.go Outdated Show resolved Hide resolved
pkg/service/cluster/service.go Outdated Show resolved Hide resolved
This replaces CQLAddr and CQLSSLAddr with one function which returns
correct cql addr depending on cluster configuration.
Also backup worker is modified a little bit to get cluster configuration
with tls related info.
This uses yq to delete non ssl port from scylla.yaml config and
also merges it with scylla-ssl.yaml which contains requried parameters
to enable ssl in scylla cluster.
This enables ssl only scylla cluster for the most of our integration tests
in ci.
This also fixes cqlping test so it supports a scylla cluster
with ssl.
This changes the signature of SessionConfigOption so that SingleHostSession func
can be simplified when Scylla cluster uses SSL.
This adds ssl related configuration options to cqlping integration tests config when ssl
is enabled.
This adds ssl support to repair integartion test case that uses cqlping
This refactor some parts of the tests that are using SSL_ENABLED env var.
This fixes how restore integration tests handle old Scylla versions: old versions require a restart after schema restoration.
To ensure Scylla is up and running, the tests perform a CQL ping, which should be initialized correctly when SSL is enabled.
Copy link
Collaborator

@Michal-Leszczynski Michal-Leszczynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SSL setup (both locally and on gh actions) will be really useful, thanks!

@VAveryanov8 VAveryanov8 merged commit 75fb75c into master Nov 22, 2024
51 checks passed
@VAveryanov8 VAveryanov8 deleted the va/fix-only-ssl-cluster-setup branch November 22, 2024 08:35
Michal-Leszczynski pushed a commit that referenced this pull request Dec 10, 2024
* fix: adds SSL_ENABLED flag to start scylla cluster in ssl only mode

This adds SSL_ENABLED flag to Makefile, so that when you run
SSL_ENABLED=true make start-dev-env the scylla cluster will be created
with ssl_only config.

* fix: handle ssl only scylla clusters

This fixes how SM decides which port to use when connecting to Scylla
nodes.

* fix: CQLAddr provides ssl or non-ssl addr depending on cluster conf.

This replaces CQLAddr and CQLSSLAddr with one function which returns
correct cql addr depending on cluster configuration.
Also backup worker is modified a little bit to get cluster configuration
with tls related info.

* fix(Makefile): use yq to produce scylla config with ssl enabled

This uses yq to delete non ssl port from scylla.yaml config and
also merges it with scylla-ssl.yaml which contains requried parameters
to enable ssl in scylla cluster.

* fix: typo in testing/scylla/config/scylla-ssl.yaml

Co-authored-by: karol-kokoszka <[email protected]>

* fix(test): use scylla cluster with SSL for integration tests

This enables ssl only scylla cluster for the most of our integration tests
in ci.
This also fixes cqlping test so it supports a scylla cluster
with ssl.

* fix(cluster): simplifies SingleHostSessionOption when dealing with SSL

This changes the signature of SessionConfigOption so that SingleHostSession func
can be simplified when Scylla cluster uses SSL.

* fix(test): adds ssl support to cqlping integration tests

This adds ssl related configuration options to cqlping integration tests config when ssl
is enabled.

* fix(test): adds ssl support to repair integration test

This adds ssl support to repair integartion test case that uses cqlping

* fix(test): adds ssl support to healthcheck integration tests

* fix(test): unifies how SSL_ENABLED is used in testconfig

* fix(ci): adds missing ssl-enabled option for a one entry in ci config

* refactor: moves parsing of SSL_ENABLED env var to the testconfig pkg

This refactor some parts of the tests that are using SSL_ENABLED env var.

* fix(test): use cqlping with ssl for the restore test of old scylla ver

This fixes how restore integration tests handle old Scylla versions: old versions require a restart after schema restoration.
To ensure Scylla is up and running, the tests perform a CQL ping, which should be initialized correctly when SSL is enabled.

---------

Co-authored-by: karol-kokoszka <[email protected]>
(cherry picked from commit 75fb75c)
Michal-Leszczynski pushed a commit that referenced this pull request Dec 10, 2024
* fix: adds SSL_ENABLED flag to start scylla cluster in ssl only mode

This adds SSL_ENABLED flag to Makefile, so that when you run
SSL_ENABLED=true make start-dev-env the scylla cluster will be created
with ssl_only config.

* fix: handle ssl only scylla clusters

This fixes how SM decides which port to use when connecting to Scylla
nodes.

* fix: CQLAddr provides ssl or non-ssl addr depending on cluster conf.

This replaces CQLAddr and CQLSSLAddr with one function which returns
correct cql addr depending on cluster configuration.
Also backup worker is modified a little bit to get cluster configuration
with tls related info.

* fix(Makefile): use yq to produce scylla config with ssl enabled

This uses yq to delete non ssl port from scylla.yaml config and
also merges it with scylla-ssl.yaml which contains requried parameters
to enable ssl in scylla cluster.

* fix: typo in testing/scylla/config/scylla-ssl.yaml

Co-authored-by: karol-kokoszka <[email protected]>

* fix(test): use scylla cluster with SSL for integration tests

This enables ssl only scylla cluster for the most of our integration tests
in ci.
This also fixes cqlping test so it supports a scylla cluster
with ssl.

* fix(cluster): simplifies SingleHostSessionOption when dealing with SSL

This changes the signature of SessionConfigOption so that SingleHostSession func
can be simplified when Scylla cluster uses SSL.

* fix(test): adds ssl support to cqlping integration tests

This adds ssl related configuration options to cqlping integration tests config when ssl
is enabled.

* fix(test): adds ssl support to repair integration test

This adds ssl support to repair integartion test case that uses cqlping

* fix(test): adds ssl support to healthcheck integration tests

* fix(test): unifies how SSL_ENABLED is used in testconfig

* fix(ci): adds missing ssl-enabled option for a one entry in ci config

* refactor: moves parsing of SSL_ENABLED env var to the testconfig pkg

This refactor some parts of the tests that are using SSL_ENABLED env var.

* fix(test): use cqlping with ssl for the restore test of old scylla ver

This fixes how restore integration tests handle old Scylla versions: old versions require a restart after schema restoration.
To ensure Scylla is up and running, the tests perform a CQL ping, which should be initialized correctly when SSL is enabled.

---------

Co-authored-by: karol-kokoszka <[email protected]>
(cherry picked from commit 75fb75c)
Michal-Leszczynski pushed a commit that referenced this pull request Dec 11, 2024
* fix: adds SSL_ENABLED flag to start scylla cluster in ssl only mode

This adds SSL_ENABLED flag to Makefile, so that when you run
SSL_ENABLED=true make start-dev-env the scylla cluster will be created
with ssl_only config.

* fix: handle ssl only scylla clusters

This fixes how SM decides which port to use when connecting to Scylla
nodes.

* fix: CQLAddr provides ssl or non-ssl addr depending on cluster conf.

This replaces CQLAddr and CQLSSLAddr with one function which returns
correct cql addr depending on cluster configuration.
Also backup worker is modified a little bit to get cluster configuration
with tls related info.

* fix(Makefile): use yq to produce scylla config with ssl enabled

This uses yq to delete non ssl port from scylla.yaml config and
also merges it with scylla-ssl.yaml which contains requried parameters
to enable ssl in scylla cluster.

* fix: typo in testing/scylla/config/scylla-ssl.yaml

Co-authored-by: karol-kokoszka <[email protected]>

* fix(test): use scylla cluster with SSL for integration tests

This enables ssl only scylla cluster for the most of our integration tests
in ci.
This also fixes cqlping test so it supports a scylla cluster
with ssl.

* fix(cluster): simplifies SingleHostSessionOption when dealing with SSL

This changes the signature of SessionConfigOption so that SingleHostSession func
can be simplified when Scylla cluster uses SSL.

* fix(test): adds ssl support to cqlping integration tests

This adds ssl related configuration options to cqlping integration tests config when ssl
is enabled.

* fix(test): adds ssl support to repair integration test

This adds ssl support to repair integartion test case that uses cqlping

* fix(test): adds ssl support to healthcheck integration tests

* fix(test): unifies how SSL_ENABLED is used in testconfig

* fix(ci): adds missing ssl-enabled option for a one entry in ci config

* refactor: moves parsing of SSL_ENABLED env var to the testconfig pkg

This refactor some parts of the tests that are using SSL_ENABLED env var.

* fix(test): use cqlping with ssl for the restore test of old scylla ver

This fixes how restore integration tests handle old Scylla versions: old versions require a restart after schema restoration.
To ensure Scylla is up and running, the tests perform a CQL ping, which should be initialized correctly when SSL is enabled.

---------

Co-authored-by: karol-kokoszka <[email protected]>
(cherry picked from commit 75fb75c)
Michal-Leszczynski pushed a commit that referenced this pull request Dec 11, 2024
* fix: adds SSL_ENABLED flag to start scylla cluster in ssl only mode

This adds SSL_ENABLED flag to Makefile, so that when you run
SSL_ENABLED=true make start-dev-env the scylla cluster will be created
with ssl_only config.

* fix: handle ssl only scylla clusters

This fixes how SM decides which port to use when connecting to Scylla
nodes.

* fix: CQLAddr provides ssl or non-ssl addr depending on cluster conf.

This replaces CQLAddr and CQLSSLAddr with one function which returns
correct cql addr depending on cluster configuration.
Also backup worker is modified a little bit to get cluster configuration
with tls related info.

* fix(Makefile): use yq to produce scylla config with ssl enabled

This uses yq to delete non ssl port from scylla.yaml config and
also merges it with scylla-ssl.yaml which contains requried parameters
to enable ssl in scylla cluster.

* fix: typo in testing/scylla/config/scylla-ssl.yaml

Co-authored-by: karol-kokoszka <[email protected]>

* fix(test): use scylla cluster with SSL for integration tests

This enables ssl only scylla cluster for the most of our integration tests
in ci.
This also fixes cqlping test so it supports a scylla cluster
with ssl.

* fix(cluster): simplifies SingleHostSessionOption when dealing with SSL

This changes the signature of SessionConfigOption so that SingleHostSession func
can be simplified when Scylla cluster uses SSL.

* fix(test): adds ssl support to cqlping integration tests

This adds ssl related configuration options to cqlping integration tests config when ssl
is enabled.

* fix(test): adds ssl support to repair integration test

This adds ssl support to repair integartion test case that uses cqlping

* fix(test): adds ssl support to healthcheck integration tests

* fix(test): unifies how SSL_ENABLED is used in testconfig

* fix(ci): adds missing ssl-enabled option for a one entry in ci config

* refactor: moves parsing of SSL_ENABLED env var to the testconfig pkg

This refactor some parts of the tests that are using SSL_ENABLED env var.

* fix(test): use cqlping with ssl for the restore test of old scylla ver

This fixes how restore integration tests handle old Scylla versions: old versions require a restart after schema restoration.
To ensure Scylla is up and running, the tests perform a CQL ping, which should be initialized correctly when SSL is enabled.

---------

Co-authored-by: karol-kokoszka <[email protected]>
(cherry picked from commit 75fb75c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Scylla Manager, under certain condition, is unable to use only SSL port (9142) to restore data
3 participants