Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support providing configuration from YAML files #123

Merged
merged 9 commits into from
Jul 2, 2024
Merged

Support providing configuration from YAML files #123

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ccdef32
Support providing configuration from YAML files
lukasz-antoniak Jul 1, 2024
7f47c41
Do not allow to override configuration values
lukasz-antoniak Jul 1, 2024
186f920
Support single configuration file
lukasz-antoniak Jul 1, 2024
49b874c
Apply review comments
lukasz-antoniak Jul 2, 2024
c359db6
Apply review comments
lukasz-antoniak Jul 2, 2024
b88d2fc
Reference configuration file
lukasz-antoniak Jul 2, 2024
13ba538
Reference configuration file
lukasz-antoniak Jul 2, 2024
439763e
Reference configuration file
lukasz-antoniak Jul 2, 2024
b1b08c0
Reference configuration file
lukasz-antoniak Jul 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ An overview of the proxy architecture and logical flow can be viewed [here](http

## Quick Start

In order to run the proxy, you'll need to set some environment variables to configure it properly.
In order to run the proxy, you'll need to set some environment variables or pass reference to YAML configuration file.
Below you'll find a list with the most important variables along with their default values.
The required ones are marked with a comment.
The required ones are marked with a comment. Variable names for YAML configuration file do not have `ZDM_` prefix and
are lower-cased.

```shell
ZDM_ORIGIN_CONTACT_POINTS=10.0.0.1 #required
Expand All @@ -36,7 +37,7 @@ ZDM_READ_MODE=PRIMARY_ONLY
ZDM_LOG_LEVEL=INFO
```

The environment variables must be set and exported for the proxy to work.
The environment variables (or YAM configuration file) must be set for the proxy to work.

In order to get started quickly, in your local environment, grab a copy of the binary distribution in the
[Releases](https://github.com/datastax/zdm-proxy/releases) page. For the recommended installation in a production
Expand All @@ -55,6 +56,19 @@ export ZDM_TARGET_PASSWORD=cassandra \
./zdm-proxy-v2.0.0 # run the ZDM proxy executable
```

If you prefer to use YAML configuration file, an equivalent setup would look like:

```shell
$ cat zdm-config.yml
origin_contact_points: 10.0.0.1
target_contact_points: 10.0.0.2
origin_username: cassandra
origin_password: cassandra
target_username: cassandra
target_password: cassandra
$ ./zdm-proxy-v2.0.0 --config=./zdm-config.yml # run the ZDM proxy executable
```

At this point, you should be able to connect some client such as [CQLSH](https://downloads.datastax.com/#cqlsh) to the proxy
and write data to it and the proxy will take care of forwarding the requests to both clusters concurrently.

Expand Down
2 changes: 1 addition & 1 deletion RELEASE_PROCESS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ All published container images can be found at [https://hub.docker.com/r/datasta

Before triggering the build and publish process for an official/stable release, three files need to be updated, the `RELEASE_NOTES`, `CHANGELOG` and `main.go`.

Please update the ZDM version displayed during component startup in `main.go`:
Please update the ZDM version displayed during component startup in `launch.go`:
```go
const ZdmVersionString = "2.0.0"
```
Expand Down
3 changes: 2 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ require (
github.com/google/uuid v1.1.1
github.com/jpillora/backoff v1.0.0
github.com/kelseyhightower/envconfig v1.4.0
github.com/mcuadros/go-defaults v1.2.0
github.com/prometheus/client_golang v1.3.0
github.com/prometheus/client_model v0.1.0
github.com/rs/zerolog v1.20.0
github.com/sirupsen/logrus v1.6.0
github.com/stretchr/testify v1.8.0
gopkg.in/yaml.v3 v3.0.1
)

require (
Expand All @@ -33,5 +35,4 @@ require (
github.com/prometheus/procfs v0.0.8 // indirect
golang.org/x/sys v0.3.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
7 changes: 6 additions & 1 deletion go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,15 @@ github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mcuadros/go-defaults v1.2.0 h1:FODb8WSf0uGaY8elWJAkoLL0Ri6AlZ1bFlenk56oZtc=
github.com/mcuadros/go-defaults v1.2.0/go.mod h1:WEZtHEVIGYVDqkKSWBdWKUVdRyKlMfulPaGDWIVeCWY=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/pierrec/lz4/v4 v4.0.3 h1:vNQKSVZNYUEAvRY9FaUXAF1XPbSOHJtDTiP41kzDz2E=
github.com/pierrec/lz4/v4 v4.0.3/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
Expand Down Expand Up @@ -127,8 +131,9 @@ golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8T
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
Expand Down
19 changes: 18 additions & 1 deletion proxy/launch.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ package main

import (
"context"
"flag"
"fmt"
"github.com/datastax/zdm-proxy/proxy/pkg/config"
"github.com/datastax/zdm-proxy/proxy/pkg/runner"
log "github.com/sirupsen/logrus"
Expand All @@ -10,6 +12,12 @@ import (
"syscall"
)

// TODO: to be managed externally
const ZdmVersionString = "2.2.0"

var displayVersion = flag.Bool("version", false, "display the ZDM proxy version and exit")
var configFile = flag.String("config", "", "specify path to ZDM configuration file")

func runSignalListener(cancelFunc context.CancelFunc) {
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
Expand All @@ -24,7 +32,16 @@ func runSignalListener(cancelFunc context.CancelFunc) {
}

func launchProxy(profilingSupported bool) {
conf, err := config.New().ParseEnvVars()
if *displayVersion {
fmt.Printf("ZDM proxy version %v\n", ZdmVersionString)
return
}

// Always record version information (very) early in the log
log.Infof("Starting ZDM proxy version %v", ZdmVersionString)

conf, err := config.New().LoadConfig(*configFile)

if err != nil {
log.Errorf("Error loading configuration: %v. Aborting startup.", err)
os.Exit(-1)
Expand Down
17 changes: 0 additions & 17 deletions proxy/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,27 +7,10 @@ package main

import (
"flag"
"fmt"
"os"

log "github.com/sirupsen/logrus"
)

// TODO: to be managed externally
const ZdmVersionString = "2.2.0"

var displayVersion = flag.Bool("version", false, "Display the ZDM proxy version and exit")

func main() {

flag.Parse()
if *displayVersion {
fmt.Printf("ZDM proxy version %v\n", ZdmVersionString)
os.Exit(0)
}

// Always record version information (very) early in the log
log.Infof("Starting ZDM proxy version %v", ZdmVersionString)

launchProxy(false)
}
1 change: 0 additions & 1 deletion proxy/main_profiling.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ var cpuProfile = flag.String("cpu_profile", "", "write cpu profile to the specif
var memProfile = flag.String("mem_profile", "", "write memory profile to the specified file")

func main() {

flag.Parse()

// the cpu profiling is enabled at startup and is periodically collected while the proxy is running
Expand Down
Loading
Loading