forked from OpenTSDB/opentsdb
-
Notifications
You must be signed in to change notification settings - Fork 0
/
NEWS
319 lines (268 loc) · 15 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
OpenTSDB - User visible changes.
* Version 2.2.1 (2015-?-?)
Noteworthy Changes
- Generate an incrementing TSMeta request only if both enable_tsuid_incrementing and
tsd.core.meta.enable_realtime_ts are enabled. Previously, increments would run
regardless of whether or not the real time ts setting was enabled. If tsuid
incrementing is disabled then a get and optional put is executed each time without
modifying the meta counter field.
* Version 2.2.0 (2016-02-14)
Noteworthy Changes
- Rework the QueryStats output to be a bit more useful and add timings from the
various scanners and query components.
- Modify the UI to allow for group by or aggregate per tag (use the new query feature)
- Rework the UI skin with the new TSDB logo and color scheme.
- Add the QueryLog config to logback.xml so users can optionally enable logging of
all queries along with their stats.
Buf Fixes:
- Properly handle append data points in the FSCK utility.
- Fix FSCK to handle salting properly.
- Fix the IncomingDataPoints class for the CLI import tool to properly account for
salting.
- Fix the QueryStats maps by making sure the hash accounts for an unmodified list of
filters (return copies to callers so sorting won't break the hash code).
- Fix the case-insensitive wildcard filter to properly ignore the case.
- Fix the CLI dumper/scan utility to properly handle salted data.
- Fix a case where the compaction queue could grow unbounded when salting was enabled.
- Allow duplicate queries by default (as we did in the past) and users must now block
them explicitly.
- Fix the /api/stats endpoint to allow for returning a value if the max UID width is
set to 8 for any type. Previously it would throw an exception.
- Add a try catch to FSCK to debug issues where printing a problematic row would cause
a hangup or no output.
* Version 2.2.0 RC3 (2015-11-11)
Bug Fixes:
- Fix build issues where the static files were not copied into the proper location.
* Version 2.2.0 RC2 (2015-11-09)
Noteworthy Changes:
- Allow overriding the metric and tag UID widths via config file instead of
having to modify the source code.
Bug Fixes:
- OOM handling script now handles multiple TSDs installed on the same host.
- Fix a bug where queries never return if an exception is thrown from the
storage layer.
- Fix random metric UID assignment in the CLI tool.
- Fix for meta data sync when salting is enabled.
-
* Version 2.2.0 RC1 (2015-09-12)
Noteworthy Changes:
- Add the option to randomly assign UIDs to metrics to improve distribution across
HBase region servers.
- Introduce salting of data to improve distribution of high cardinality regions
across region servers.
- Introduce query stats for tracking various timings related to TSD queries.
- Add more stats endpoints including /threads, /jvm and /region_clients
- Allow for deleting UID mappings via CLI or the API
- Name the various threads for easier debugging, particularly for distinguishing
between TSD and AsyncHBase threads.
- Allow for pre-fetching all of the meta information for the tables to improve
performance.
- Update to the latest AsyncHBase with support for secure HBase clusters and RPC
timeouts.
- Allow for overriding metric and tag widths via the config file. (Be careful!)
- URLs from the API are now relative instead of absolute, allowing for easier reverse
proxy use.
- Allow for percent deviation in the Nagios check
- Let queries skip over unknown tag values that may not exist yet (via config)
- Add various query filters such as case (in)sensitive pipes, wildcards and pipes
over tag values. Filters do not work over metrics at this time.
- Add support for writing data points using Appends in HBase as a way of writing
compacted data without having to read and re-write at the top of each hour.
- Introduce an option to emit NaNs or Nulls in the JSON output when downsampling and
a bucket is missing values.
- Introduce query time flags to show the original query along with some timing stats
in the response.
- Introduce a storage exception handler plugin that will allow users to spool or
requeue data points that fail writes to HBase due to various issues.
- Rework the HTTP pipeline to support plugins with RPC implementations.
- Allow for some style options in the Gnuplot graphs.
- Allow for timing out long running HTTP queries.
- Text importer will now log and continue bad rows instead of failing.
- New percentile and count aggregators.
- Add the /api/annotations endpoint to fetch multiple annotations in one call.
- Add a class to support improved bulk imports by batching requests in memory for a
full hour before writing.
Bug Fixes:
- Modify the .rpm build to allow dashes in the name.
- Allow the Nagios check script to handle 0 values properly in checks.
- Fix FSCK where floating point values were not processed correctly (#430)
- Fix missing information from the /appi/uid/tsmeta calls (#498)
- Fix more issues with the FSCK around deleting columns that were in the list (#436)
- Avoid OOM issues over Telnet when the sending client isn't reading errors off it's
socket fast enough by blocking writes.
* Version 2.1.4 (2016-02-14)
Bug Fixes:
- Fix the meta table where the UID/TSMeta APIs were not sorting tags properly
prior to creating the row key, thus allowing for duplicates if the caller changed
the order of tags.
- Fix a situation where meta sync could hang forever if a routine threw an exception.
- Fix an NPE thrown when accessing the /logs endpoint if the Cyclic appender is not
enabled in the logback config.
- Remove an overly chatty log line in TSMeta on new time series creation.
* Version 2.1.3 (2015-11-11)
Bug Fixes:
- Fix build issues where the static files were not copied into the proper location.
* Version 2.1.2 (2015-11-09)
Bug Fixes:
- Fix the built-in UI to handle query parameter parsing properly (found when Firefox
changed their URI behavior)
- Fix comments about the Zookeeper quorum setting in various config files.
- Fix quoting in the Makefile when installing.
- Make sure builds write files in the proper location on FreeBSD.
* Version 2.1.1 (2015-09-12)
Bug Fixes:
- Relax the pgrep regex to correctly find and kill the java process in the RPM init.d
script.
- Improve query performance slightly when aggregating multiple series.
- Fix the /api/search/lookup API call to properly handle the limit parameter.
- Fix the /api/query/last endpoint to properly handle missing tsdb-meta tables.
* Version 2.1.0 (2015-05-06)
Bug Fixes:
- FSCK was not handling compacted and floating point duplicates properly. Now they
are merged correctly.
- TSMeta data updates were not loading the latest data from storage on response
- The config class will now trim spaces from booleans and integers
- On shutdown, the idle state handler could prevent the TSD from shutting down
gracefully. A new thread factory sets that thread as a daemon thread.
- TSMeta objects were not generated if multiple writes for the same data point arrived
in succession due to buffering atomic increments. Increments are no longer buffered.
- Updated paths to the deprecated Google Code repo for dependencies.
* Version 2.1.0 RC2 (2015-04-04)
Noteworthy Changes:
- Handle idle connections in Netty by closing them after some period of inactivity
- Support compressed HTTP responses
Bug Fixes:
- Various RPM script and package fixes
- Properly handle deletions of the cache directory while running
- Queries for non-extant metrics now return a 400 error
- Fix an issue with the meta sync utility where entries were not created if the counter existed
- Report stats properly when the UID is greater than 3 bytes wide
- Fix UI hangups when incorrect tags are supplied
- Log illegal put exceptions at the debug level
- Fix global annotation retrieval where some entries were missing
- Fix unit tests that were not properly mocking out clients or threads and causing JVM OOMs
- Fix accounting bugs in FSCK
- Sort aggregators in the UI
- Properly throw an exception if the user supplies an empty tag on storage or retreival
- Handle missing directories in the Config class
* Version 2.1.0 RC1 (2014-11-09)
Noteworthy Changes:
- Add a server side timeout for sockets that haven't written data in some time
- Major FSCK utility update to handle new objects, delete bad data and deal with duplicate data points.
- Optionally preload portions of the name to UID maps at startup
- Add read and write modes to the TSD to disable writing data points via telnet or HTTP
- Optionally disable the diediedie commands to prevent users from shutting down a tsd
- Optionally block the auto creation of tag keys and values
- Downsampling is now aligned on modulus bondaries so that we avoid interpolation as much as possible. Data returned is now more along the lines of what users expect, e.g. 24 data points for day when downsampled on hourly intervals instead of random points based on the span's timestamps.
- Add the /api/search/lookup endpoint and CLI endpoint for looking up time series based on the meta or data tables
- Rework of the TSD compaction code to process compactions faster
- Optionally handle duplicate data points gracefully during compaction or query time without throwing exceptions
- Add Allow-Headers CORs support
* Version 2.0.1 (2014-11-09)
Bug Fixes:
- Fix tree REST endpoint to allow modification of the strictMatch and storeFailure fields
- Fix compacted millisecond columns throwing an error during fsck
- Fix tsd startup directory permission checks
- Fix thread safety of pending UID assignment when they are complete
- Fix compaction queue flushing on tsd shutdown
- Fix RTPublisher plugin floating point value detection where it was previously posting to the wrong method
- Fix TsdbQuery.toString() to avoid looing up UIDs if an error was thrown as this can cause cause a deadlock
Noteworthy changes:
- Add Travis CI build support
- Update to Asynchbase 1.6.0 and update additional dependencies
- Modify scripts to allow easier compilation under FreeBSD
* Version 2.0.0 (2014-05-5)
API incompatible changes:
- The `TSDB' class now takes a `Config' object in argument instead of an
HBaseClient and two strings.
- The downsampling interval for the method `Query.downsample' went from
being an `int' to a `long'.
Noteworthy changes:
- Configuration can be provided in a properties file
- New Jackson JSON helper class
- GnuPlot batch file for Windows compatability
- Add relative time option "n" for 30 days
- Relative, unix epoch style timestamps work in CliQuery
- New "max" parameter for /suggest that can fetch more than the default 25
results. If not supplied, default is used
- New formalized HTTP API, deprecates many of the old HTTP API calls but it
is still backwards compatible
- New store data points over HTTP via JSON
- New annotations for recording meta data along with data points in a time
series
- New global annotations to record meta data at a specific time but not
associated with a specific series
- New meta data for metrics, tag names, tag values and time series
- New optional chunked encoding support for HTTP requests, configurable
- Millisecond resolution support for writing data points and annotations
- Variable length encoding for integer data points, any where from 1 to 8
bytes instead of using 8 bytes for every point
- CORs support for the HTTP API
- New data injest plugin framework allowing support for different data
formats
- Search plugin framework to push meta data to a search engine for access
- Real-Time publisher framework to publish data points to a secondary system
as soon as they are received at the TSD
- New aggregation functions with alternatives to interpolation including:
- zero if missing sum: returns a 0 if a data point doesn't exist
- max if missing min: returns the maximum value if a data point is missing
- min if missing max: returns the minimum value if a data point is missing
- New TSUID tracking that writes a counter to a new table to track the
different time series stored and how many data points were written
- New meta data trees that allow for flattening time series into a
heirarchical representation similar to a file system
- New meta and tree synchronization CLI tools
- New statistics showing the number of UIDs used and available for each type
- New statistics for the number of current connections to the TSD
- New options for working with rate calculations on counters to rollover
or reset on anomallys
- New Debian package compilable from the source
- New RPM package compilable from the source
* Version 1.1.1 (2013-??-??) [???????]
Noteworthy changes:
- UIDs are now assigned in a lock-less fashion.
* Version 1.1.0 (2013-03-08) [12879d7]
Noteworthy changes:
- Licensing adjustment: allow LGPLv2.1+ in addition to LGPLv3+.
- Various fixes used when customizing size of UniqueId. The default size
is 3 bytes and is a compile-time constant rarely changed in practice.
- New a new standard deviation aggregator, `dev'.
- New `fgcolor', `bgcolor' and `smooth' query parameters to /q.
- New `tz' query string parameter to allow specifying a custom time zone.
- Stop accepting connections when shutting down.
- A new `dropcaches' administrative command allows discarding in-memory
caches. Right now these are UID mappings.
- Browser history support in the web UI.
- Allow "1d-ago" style input in date boxes.
- Fix the 30d integer overflow in the web UI.
- Add the ability to use mouse for drag-to-zoom on graphs.
- Integration with Maven.
- Work around a Netty performance bug, increasing write throughput by 10x.
- Properly parse floating point values in scientific notations.
- Allow tuning the number of worker threads or using OIO.
- Fix auto-completion bug causing partial results to show in the web UI.
- Various internal bug fixes.
- Update all dependencies.
- TSDB data compaction is now enabled by default.
* Version 1.0.0 (2011-12-23) [66a6b42]
Initial release:
- Upload data points through a telnet-style protocol.
- HTTP interface to query data in ASCII and PNG.
- Efficient, fully asynchronous write path.
- Synchronous / blocking read path (to be rewritten).
- TSDB data compaction (disabled by default).
-----
Copyright (C) 2011-2012 The OpenTSDB Authors.
This library is free software: you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation, either version 2.1 of the License, or
(at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with this library. If not, see <http://www.gnu.org/licenses/>.
Local Variables:
mode: outline
End: