Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[change] Find faster way to write to influxdb (eg: UDP) #458

Closed
nemesifier opened this issue Nov 2, 2022 · 1 comment · Fixed by #472
Closed

[change] Find faster way to write to influxdb (eg: UDP) #458

nemesifier opened this issue Nov 2, 2022 · 1 comment · Fixed by #472
Labels
enhancement New feature or request

Comments

@nemesifier
Copy link
Member

nemesifier commented Nov 2, 2022 •
edited
Loading

We should find a faster method for writing timeseries data, maybe UDP instead of HTTP.
I think this may change from influxdb 1.8 to 2.0 so we'd have to check this on Influxdb 2.0 #274.
Right now writing timeseries on big systems is quite a bottleneck!

See also #466.
@nemesifier nemesifier added the enhancement New feature or request label Nov 2, 2022
@nemesifier nemesifier mentioned this issue Dec 2, 2022
Closed
@nemesifier nemesifier changed the title [change] Find faster way to write to influxdb [change] Find faster way to write to influxdb (eg: UDP) Dec 2, 2022
@pandafy
Copy link
Member

pandafy commented Dec 9, 2022

The influxdb-python library supports connecting to InfluxDB using UDP.

We need to verify that the authentication works with UDP.

pandafy added a commit that referenced this issue Dec 23, 2022
@pandafy
[feature] Allowed specifying additional options for InfluxDB backend #…
f427207 
…458

- Additional options can be specified to the TIMESERIES_DATABASE setting
  which are passed to the underlying backend library. This allows using
  UDP for writing to InfluxDB.
- "openwisp_monitoring.db.backends.influxdb.client.DatabaseClient.write"
  will use "InfluxDBClient.write_points" instead of "InfluxDBClient.write"
  because the former allows writing data using the UDP protocol.

Closes #458
@pandafy pandafy mentioned this issue Dec 23, 2022
Merged
3 tasks
pandafy added a commit that referenced this issue Jan 24, 2023
@pandafy
[feature] Allowed specifying additional options for InfluxDB backend #…
4a33176 
…458

- Additional options can be specified to the TIMESERIES_DATABASE setting
  which are passed to the underlying backend library. This allows using
  UDP for writing to InfluxDB.
- "openwisp_monitoring.db.backends.influxdb.client.DatabaseClient.write"
  will use "InfluxDBClient.write_points" instead of "InfluxDBClient.write"
  because the former allows writing data using the UDP protocol.

Closes #458
pandafy added a commit that referenced this issue Feb 7, 2023
@pandafy
[feature] Allowed specifying additional options for InfluxDB backend #…
24a40b1 
…458

- Additional options can be specified to the TIMESERIES_DATABASE setting
  which are passed to the underlying backend library. This allows using
  UDP for writing to InfluxDB.
- "openwisp_monitoring.db.backends.influxdb.client.DatabaseClient.write"
  will use "InfluxDBClient.write_points" instead of "InfluxDBClient.write"
  because the former allows writing data using the UDP protocol.

Closes #458
pandafy added a commit that referenced this issue Feb 9, 2023
@pandafy
[feature] Allowed specifying additional options for InfluxDB backend #…
8fc60e9 
…458

- Additional options can be specified to the TIMESERIES_DATABASE setting
  which are passed to the underlying backend library. This allows using
  UDP for writing to InfluxDB.
- "openwisp_monitoring.db.backends.influxdb.client.DatabaseClient.write"
  will use "InfluxDBClient.write_points" instead of "InfluxDBClient.write"
  because the former allows writing data using the UDP protocol.

Closes #458
nemesifier pushed a commit that referenced this issue Feb 9, 2023
@pandafy
[feature] Added support for InfluxDB UDP mode #458
476e7a3 
- Additional options can be specified to the TIMESERIES_DATABASE setting
  which are passed to the underlying backend library. This allows using
  UDP for writing to InfluxDB.
- "openwisp_monitoring.db.backends.influxdb.client.DatabaseClient.write"
  will use "InfluxDBClient.write_points" instead of "InfluxDBClient.write"
  because the former allows writing data using the UDP protocol.
- If device data is too big to be written with UDP, HTTP is used instead.

Closes #458
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
OpenWISP Contributor's Board
Archived in project
OpenWISP Priorities for next releases
Archived in project
Milestone
No milestone
2 participants
@nemesifier @pandafy