Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: add new kafka backend and refactor metrics sender engine #522

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

feat!: add new kafka backend and refactor metrics sender engine #522

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
# v0.14.0 ( unreleased )

### New Features

* add Kafka as available Backend - on backend and UI
* add internal buffer to store metrics and improved management/control/writes to the backend, based on time (see breaking change)
* add new dropped series metric to selfmon stats

### Fixes

* fix #343

### Breaking Changes

* refactor metrics sender engine with new output/backend refactor. The metrics are now sent in batches based on an interval and it
* UI: new Output component and InfluxDB is moved as an Output backend
* SNMPCollector OutDB refers now to an Output instead of an InfluxDB Server, a migration script is done to migrate current InfluxDBServers to Outputs

# v0.13.1 ( 2022-11-15 )

### New Features
Expand Down
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# SnmpCollector [![Go Report Card](https://goreportcard.com/badge/github.com/toni-moreno/snmpcollector)](https://goreportcard.com/report/github.com/toni-moreno/snmpcollector)

SnmpCollector is a full featured Generic SNMP data collector with Web Administration Interface Open Source tool which has as main goal simplify the configuration for getting data from any device which snmp protocol support and send resulting data to an influxdb backend.
SnmpCollector is a full featured Generic SNMP data collector with Web Administration Interface Open Source tool which has as main goal simplify the configuration for getting data from any device which snmp protocol support and send resulting data to an InfluxDB and Kafka backends.

For complete information on installation from binary package and configuration you could read the [snmpcollector wiki](https://github.com/toni-moreno/snmpcollector/wiki).

Expand All @@ -13,6 +13,7 @@ If you want to build a package yourself, or contribute. Here is a guide for how

- Go 1.5 for snmpcollector < 0.8
- Go 1.11 for snmpcollector >= 0.8
- Go 1.17 for snmpcollector >= 0.13
- NodeJS >=6.2.1

### Get Code and setup example config
Expand Down
69 changes: 53 additions & 16 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,47 +4,84 @@ go 1.17

require (
github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible
github.com/Shopify/sarama v1.36.0
github.com/go-macaron/binding v1.1.1
github.com/go-macaron/session v1.0.2
github.com/go-macaron/toolbox v0.0.0-20200329073429-4401f4ce0f55
github.com/go-sql-driver/mysql v1.6.0
github.com/google/go-cmp v0.5.3
github.com/gosnmp/gosnmp v1.32.0
github.com/gofrs/uuid v4.2.0+incompatible
github.com/google/go-cmp v0.5.9
github.com/gosnmp/gosnmp v1.34.0
github.com/influxdata/influxdb1-client v0.0.0-20200827194710-b269163b24ab
github.com/influxdata/telegraf v1.24.2
github.com/kelseyhightower/envconfig v1.4.0
github.com/lib/pq v1.10.2
github.com/mattn/go-sqlite3 v1.14.8
github.com/pkg/errors v0.9.1
github.com/sirupsen/logrus v1.4.2
github.com/sirupsen/logrus v1.9.0
github.com/spf13/viper v1.2.1
gopkg.in/macaron.v1 v1.4.0
xorm.io/xorm v1.2.5
)

require (
github.com/blues/jsonata-go v1.5.4 // indirect
github.com/caio/go-tdigest v3.1.0+incompatible // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/eapache/go-resiliency v1.3.0 // indirect
github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21 // indirect
github.com/eapache/queue v1.1.0 // indirect
github.com/fatih/color v1.13.0 // indirect
github.com/fsnotify/fsnotify v1.4.9 // indirect
github.com/go-macaron/inject v0.0.0-20200308113650-138e5925c53b // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/goccy/go-json v0.7.4 // indirect
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/json-iterator/go v1.1.11 // indirect
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
github.com/jcmturner/gofork v1.7.6 // indirect
github.com/jcmturner/gokrb5/v8 v8.4.3 // indirect
github.com/jcmturner/rpc/v2 v2.0.3 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.15.10 // indirect
github.com/konsorten/go-windows-terminal-sequences v1.0.3 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/magiconair/properties v1.8.0 // indirect
github.com/mitchellh/mapstructure v1.1.2 // indirect
github.com/magiconair/properties v1.8.6 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.1 // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml v1.8.1 // indirect
github.com/spf13/afero v1.1.2 // indirect
github.com/philhofer/fwd v1.1.1 // indirect
github.com/pierrec/lz4/v4 v4.1.15 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.37.0 // indirect
github.com/prometheus/prometheus v1.8.2-0.20210430082741-2a4b8e12bbf2 // indirect
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
github.com/spf13/afero v1.2.2 // indirect
github.com/spf13/cast v1.3.0 // indirect
github.com/spf13/jwalterweatherman v1.0.0 // indirect
github.com/spf13/pflag v1.0.3 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/syndtr/goleveldb v1.0.0 // indirect
github.com/tinylib/msgp v1.1.6 // indirect
github.com/unknwon/com v1.0.1 // indirect
golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e // indirect
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1 // indirect
golang.org/x/text v0.3.6 // indirect
gopkg.in/ini.v1 v1.62.0 // indirect
gopkg.in/yaml.v2 v2.3.0 // indirect
github.com/wavefronthq/wavefront-sdk-go v0.10.4 // indirect
github.com/xdg/scram v1.0.5 // indirect
github.com/xdg/stringprep v1.0.3 // indirect
golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 // indirect
golang.org/x/net v0.0.0-20220909164309-bea034e7d591 // indirect
golang.org/x/sys v0.0.0-20220919091848-fb04ddd9f9c8 // indirect
golang.org/x/text v0.3.7 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
xorm.io/builder v0.3.9 // indirect
)
Loading