Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: zipkin-gcp #45

Closed
8 tasks
saturnism opened this issue Sep 9, 2017 · 20 comments
Closed
8 tasks

Proposal: zipkin-gcp #45

saturnism opened this issue Sep 9, 2017 · 20 comments

Comments

@saturnism
Copy link
Collaborator

saturnism commented Sep 9, 2017

There are a couple of zipkin subprojects targeting different cloud environments. We should look into how to evolve this into the same fold, e.g., zipkin-gcp. zipkin-gcp should can have the following:

  • Senders
    • Pub/Sub (sender-pubsub)
    • Stackdriver Trace (sender-stackdriver)
  • Storage
    • Stackdriver Storage (storage-stackdriver)
  • Collectors
    • Pub/Sub (collector-pubsub)
  • Autoconfigurers

We can also use spring-cloud-gcp-core as the basis for getting Project ID and Credentials.

Current stackdriver-zipkin behavior can ultimately be composed by:

  • storage-stackdriver
  • spring-cloud-gcp-core
  • autoconfigurers
@saturnism
Copy link
Collaborator Author

saturnism commented Sep 9, 2017

does it still make sense to have Stackdriver storage?

@codefromthecrypt
Copy link
Member

codefromthecrypt commented Sep 10, 2017 via email

@saturnism
Copy link
Collaborator Author

Sounds like we should def keep it!

@codefromthecrypt
Copy link
Member

codefromthecrypt commented Sep 11, 2017 via email

@mtwo
Copy link
Contributor

mtwo commented Sep 11, 2017

@saturnism when you mention that there are multiple Zipkin subprojects targeting different cloud environments, do you mean Zipkin collectors like this one that forward data to a different backend or something else?

Is the proposal that we transform this into a Zipkin collector that can forward traces to many different backends?

@saturnism
Copy link
Collaborator Author

@mtwo different zipkin collectors are definitely part of it. but also different senders as well. E.g., a zipkin sender that can send directly to Stackdriver or Pub/Sub. As well as collectors that can consume Zipkin from Pub/Sub too. We would still have the current storage module, etc.

For example, there is currently a stackdriver-zipkin collector module - but i feel it's not how zipkin defines collectors AFAIK. Collectors listens to some input sources, such as pub/sub.

@saturnism
Copy link
Collaborator Author

@mtwo also, with these modules, users have a lot of choices in terms of how they can mix & match the components to their needs. For example:

  • If you want to forward Zipkin traces via HTTP then to Stackdriver, use the start zipkin w/ Stackdriver Storage.
  • If you want to store traces in your own store, but using Pub/Sub for fire & forget messaging, then use Pub/Sub Sender, Pub/Sub Collector, and whatever you need for storage.
  • If you want to send traces from Brave to Stackdriver, use Stackdriver Sender

@mtwo
Copy link
Contributor

mtwo commented Sep 14, 2017

Got it, thanks. So with this design, a single sender would allow traces to be exported from Brave or a Zipkin server? Seems neat.

@adriancole no issues from me with moving the project to another org, let me double check with legal and let's get @bogdandrutu to comment as well.

@saturnism
Copy link
Collaborator Author

@mtwo that's the idea :) and brave is pretty awesome already.

@mtwo
Copy link
Contributor

mtwo commented Sep 19, 2017

FYI for all, @saturnism and I are on a thread to get the repo moved. We'll post back once we get confirmation.

@codefromthecrypt
Copy link
Member

codefromthecrypt commented Oct 6, 2017 via email

@mtwo
Copy link
Contributor

mtwo commented Oct 19, 2017

I asked around internally about issues with porting this and nobody raised an exception, so I think we're good to go. Did you want to just fork the code or have the repo moved (assuming that GitHub supports such an operation)?

Forking might be easier so that we can keep the collector operating as-is until the replacement is ready.

@codefromthecrypt
Copy link
Member

codefromthecrypt commented Oct 20, 2017 via email

@mtwo
Copy link
Contributor

mtwo commented Oct 20, 2017

@bogdandrutu @tags07

I don't have perms to change the name, but Bogdan or Gopi likely do. However given that this is in the Google org, it may end up being easier to fork it to the Zipkin org and then delete it here once the work is done (?).

@codefromthecrypt
Copy link
Member

codefromthecrypt commented Oct 21, 2017 via email

@mtwo
Copy link
Contributor

mtwo commented Oct 23, 2017 via email

@codefromthecrypt
Copy link
Member

FYI I have started on this, and probably need a day or two before raising a PR. When I raise the PR, it will look just like the other zipkin extensions (including build setup so we can publish it). When ready to merge, we can do a last release here, then move the repo and do a first release there. cc @anuraaga

codefromthecrypt pushed a commit to openzipkin/zipkin that referenced this issue Nov 11, 2017
gRPC uses Netty 4.1 and `netty-tcnative-boringssl-static`, which is
sensitive to Netty versions. DataStax Cassandra driver 4.0 uses Netty
4.1, but the driver isn't out, yet. By using the published shaded jar,
we can dodge the conflict for now.

I've verified that we can connect to google cloud services after this
change. I've also verified that there's no significant jar size change.

See https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
See openzipkin/zipkin-gcp#45
@codefromthecrypt
Copy link
Member

I had to make a server change to dodge netty conflicts. This is blocking work, but not for long openzipkin/zipkin#1788

codefromthecrypt pushed a commit to openzipkin/zipkin that referenced this issue Nov 12, 2017
gRPC uses Netty 4.1 and `netty-tcnative-boringssl-static`, which is
sensitive to Netty versions. DataStax Cassandra driver 4.0 uses Netty
4.1, but the driver isn't out, yet. By using the published shaded jar,
we can dodge the conflict for now.

I've verified that we can connect to google cloud services after this
change. I've also verified that there's no significant jar size change.

See https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
See openzipkin/zipkin-gcp#45
@codefromthecrypt
Copy link
Member

#48 sorry took me so long to get started on this

@codefromthecrypt
Copy link
Member

#109 is for pub/sub

abesto pushed a commit to abesto/zipkin that referenced this issue Sep 10, 2019
gRPC uses Netty 4.1 and `netty-tcnative-boringssl-static`, which is
sensitive to Netty versions. DataStax Cassandra driver 4.0 uses Netty
4.1, but the driver isn't out, yet. By using the published shaded jar,
we can dodge the conflict for now.

I've verified that we can connect to google cloud services after this
change. I've also verified that there's no significant jar size change.

See https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
See openzipkin/zipkin-gcp#45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants