Most topics are ignored, others are randomly parsed

[zdzichu]

Hi,

I'm aware there are multitude "does not work"-kind issues already reported, but I believe I'm providing enough data to improve the datasource.

I'm using version v1.0.0-beta.4 with grafana-10.2.6.

I see two issues:

• topic names starting / seem to be ignored. This may be related with how the key for topic is created, by path-concatenating duration and the actual topic. I also noted, that in topic_test.go there are not tests for a topic starting with a slash
• non-JSON payloads seem to be ignored completely. At the same time JSON payloads are sometimes parsed.

I was able to capture some traffic in Grafana Explorer by subscribing to __HASH__ . At the same time I was running mosquitto_sub on this topic. Here are the results:

This what was being sent:

/sonoff03/dht22/humidity 38.00
/sonoff03/gpio/12 0
/sonoff03/pwm/13 1023
/d1mini04/BH1750/lux 0.00
rtl_433//FT-004B/time 2024-05-17 13:30:55
rtl_433//FT-004B/protocol 92
rtl_433//FT-004B/temperature_C 23.2
/sonoff08/sysinfo/wifi -70.00
/shelly02/events/rpc {"src":"shelly1pmmini-6055f999fdac","dst":"/shelly02/events","method":"NotifyStatus","params":{"ts":1715945460.30,"switch:0":{"id":0,"aenergy":{"by_minute":[0.000,0.000,0.000],"minute_ts":1715945459,"total":1242.654}}}}
/shelly02/status/switch:0 {"id":0, "source":"mqtt", "output":false, "apower":0.0, "voltage":242.9, "freq":50.1, "current":0.000, "aenergy":{"total":1242.654,"by_minute":[0.000,0.000,0.000],"minute_ts":1715945459},"temperature":{"tC":34.8, "tF":94.7}}
/d1mini03/wifi/ -53.00
/sonoff01/wifi/ -34.00

And this is what MQTT datasource was able to understand:

Observation:

• most messages ended as null/undefined
• for the topic starting with rtl_433 (note: no leading slash) no value was correctly parsed
• for JSON sent to /shelly02/events/rpc some fields were interpreted, but also got duplicated (?)
• for JSON sent to /shelly02/status/switch:0 some field were interpreted, too

[NiklasCi]

The Problem with the current streaming plugins is that you can't send back series with refid's.
We just send each received message on the same query so /all/# results into a big map of values. Because the values do not arrive at the same time you see undefined and null values.

Example:
query: /all/#

--> first message on the subscription from /all/device1 with data {"value1": 123} will be stored into the messages array.
messages: [{"value1": 123}]

--> second message on the subscription from /all/device2 with data {"value2": 32.1} will be stored into the messages array.
messages: [{"value1": 123}, {"value2": 32.1}]

Grafana will create a row for each array index resulting in:

Value1	Value2
	32.1
123

where all empty fields will be replaced with undefined/null

[zdzichu]

Sorry for not being clear enough. If I specify topic as /all/device1 or /all/device2 then nothing will be received by Grafana. At least it looks like so, but:
Specifying # was to illustrate that data is in fact received.
Oh, and those topic receive integer or float values, not a JSON.

[NiklasCi]

Ok now i got it.
So it seems none json value support is needed and a fix that topics can start with a /