-
Notifications
You must be signed in to change notification settings - Fork 0
/
dcom.slide
163 lines (91 loc) · 5.2 KB
/
dcom.slide
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
Distributed Communication
Brian Ketelsen
@bketelsen
* Distributed Communication
- Development Servers
- Introductions
- Methods of communicating between applications
- Transport and encoding options
* Methods of Communication Between Applications
There are infinite combinations of ways to communicate between running applications, but they boil down to two patterns:
- Message Queue
- Network Request/Response
* Message Queue
In the message queue pattern, a service on the network sends a message on a queue. Another service listens to that queue and processes the message -- optionally sending a response on a special response queue.
- RabbitMQ
- NATS
- NSQ
- countless others
* Message Queue Pros/Cons
Pros:
- Simple to reason about communication - send a message on a topic.
- No concern about who/what is receiving and processing the message. Implicit trust in the queue.
- Very scalable, 1 -> N subscribers processing the queue.
- Many can be configured for message durability - guarantee message is saved.
Cons:
- One more thing to manage in your stack.
- Requires special knowledge to maintain the queue service.
- Without good monitoring, it's easy to lose messages.
- Durability guarantees require more work for sysadmin & developers.
* Exercise
Message Queue
- $GOPATH/src/github.com/gophertrain/material/dcom/exercises/onenode/readme.txt
- $GOPATH/src/github.com/gophertrain/material/dcom/exercises/logger/readme.txt
* Communications Notes
There is no hard and fast rule about communicating between your services. You may even have to use multiple options to fit into your existing environment. Write your code with this in mind:
.play dcom/includes/inventory/transport/transporter.go
This example service has an interface for serving the content that accepts a `net.Listener`. Any network transport can satisfy this interface.
* RPC
There are many flavors of RPC, and the choice you make depends on the consumers of your service and what they can support.
- All Go based - Use net/rpc with `gob` encoding
- Varied consumers - use a JSON RPC protocol like net/rpc/json
* RPC
RPC stands for Remote Procedure Call. Many people have been jaded by previous implementations of RPC (Who remembers CORBA??), but it's very efficient now, and extremely useful in a distributed application architecture.
* RPC Demo
$GOPATH/src/github.com/gophertrain/material/dcom/demos/{client/server}
* Encoding
You have an abundance of options for encoding your messages over the wire. Once again, it's probably best to expect to be providing more than one encoding option for compatibility.
- JSON
- gob
- protocol buffers
- Thrift
- Avro
- more and more and more
* Encoding
Each encoding protocol has pros and cons, and potentially limits your consumers.
Recommendation:
Support JSON and protocol buffers
JSON is the standard for interop *today* and protocol buffers are the *up-and-coming* star.
* Protocol Buffers
"Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages."
Protocol buffers are defined in .proto files, and then client/server code is *generated* by the protoc compiler, using plugins for different language implementations.
* Protocol Buffers
Protocol Buffers use numbered fields in their message definitions, which guarantees backwards compatibility.
- You never modify existing fields
- But you may add more fields to a message
Old clients will continue to work, and new clients can use newer fields. Server-side code is epected to handle both gracefully.
* Protocol Buffers
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
}
Messages are strongly typed (unlike JSON), and there is a rich definition language for custom message types, optional fields, enumerations and more.
* Protocol Buffers
The only downside is that there is more tooling required to use protocol buffers as a developer.
.link https://github.com/google/protobuf Protobuf Installation
* Protocol Buffers
By themselves, protocol buffers are a great choice for message encoding. There are PB implementations for every major language and many of the minor ones.
But as Billy Mays says... "BUT WAIT!! There's MORE!"
* GRPC
You can have the best of both worlds by using GRPC.
.link http://grpc.io
GRPC is a "high performance open source universal RPC framework".
It uses protocol buffers to represent your messages *and* service definitions.
GRPC also has a variety of plugins, including one that exposes your GRPC services as JSON/REST style services. So you can have one source of truth for your service definitions and support fast RPC using GRPC or legacy systems with JSON/REST.
* GRPC Exercise
Together:
Open $GOPATH/src/google.golang.org/grpc/examples/helloworld in two different terminals.
Inspect the source code for the client and the server.
Individually: Run the server, then from another terminal run the client. (Use the same machine for both.)