Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sonoff Zigbee3.0 USB Dongle pairing issues #9117

Closed
notenoughtech opened this issue Oct 11, 2021 · 127 comments
Closed

Sonoff Zigbee3.0 USB Dongle pairing issues #9117

notenoughtech opened this issue Oct 11, 2021 · 127 comments
Labels
problem Something isn't working stale Stale issues

Comments

@notenoughtech
Copy link

I got serious pairing issues with various Zigbee 3.0 end devices. Sonoff Zigbee3.0 USB Dongle was using
{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20210120,"transportrev":2},"type":"zStack3x0"}
and the latest firmware after a successful flash.
With the latest firmware, I was not able to get anything to pair - no MQTT messages about the interview process. So I reverted back to the 20210120 revision and I was able to pair Aqara TVOC and IKEA button, but Sonoff devices or Xiaomi buttons were not even entering the interview stage. The same devices were tested with ZZH and CC2531 and I was able to pair and use the devices.

There isn't much information I can share, as the log wasn't being populated.
Happy to try things out
@notenoughtech notenoughtech added the problem Something isn't working label Oct 11, 2021
@Koenkk
Copy link
Owner

Koenkk commented Oct 11, 2021

Could you flash this FW: https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20210708.zip and use a CC2531 to sniff the traffic when pairing fails? https://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html

@notenoughtech
Copy link
Author

I will give it a go this eve.

@notenoughtech
Copy link
Author

On this firmware I see pairing messages -- so there is progress

Some observations:
AqaraE1: Paired without even triggering the pairing procedure (was it waiting to pair for 2 days?)
Sonoff button: - no dice not responsive:
Oct 11 19:16:13 testpi npm[2173]: Zigbee2MQTT:error 2021-10-11 19:16:13: Failed to configure '0x00124b001cd61409', attempt 3 (Error: Bind 0x00124b001cd61409/1 genOnOff from '0x00124b0022a54d11/1' failed (AREQ - ZDO - bindRsp after 10000ms)
and also triggered
Oct 11 19:11:02 testpi npm[2173]: (node:2188) UnhandledPromiseRejectionWarning: AssertionError [ERR_ASSERTION]: No image available for imageType '14976' Oct 11 19:11:02 testpi npm[2173]: at getImageMeta (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/lib/ota/zigbeeOTA.js:24:5) Oct 11 19:11:02 testpi npm[2173]: at processTicksAndRejections (internal/process/task_queues.js:95:5) Oct 11 19:11:02 testpi npm[2173]: at isNewImageAvailable (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/lib/ota/common.js:265:18) Oct 11 19:11:02 testpi npm[2173]: at Object.isUpdateAvailable (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/lib/ota/common.js:256:23) Oct 11 19:11:02 testpi npm[2173]: at OTAUpdate.onZigbeeEvent (/opt/zigbee2mqtt/lib/extension/otaUpdate.ts:61:31)

Sonoff Motion sensor - at first I thought it would fail but took 2 min to pair and works ok now 🤷‍♂️

Lidl stuff, before not able to pair but worked just like that.

Sonoff temp sensor: gets stuck on
Oct 11 19:36:43 testpi npm[2173]: Zigbee2MQTT:info 2021-10-11 19:36:43: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"interview_successful","meta":{"description":"Temperature and humidity sensor","friendly_name":"0x00124b002269c2c8","model":"SNZB-02","supported":true,"vendor":"SONOFF"},"type":"pairing"}'

gave up after 5 min

Some improvement there over what it was before - I wasn't on the launchpad version. I need to setup sniffing to give you more sets, got lots on my plate this week so I will try to post as soon as I can

@Koenkk
Copy link
Owner

Koenkk commented Oct 12, 2021

@notenoughtech
Copy link
Author

notenoughtech commented Oct 12, 2021 via email

@Koenkk
Copy link
Owner

Koenkk commented Oct 12, 2021

If you provide a debug log + sniff starting from the moment you put the device into pairing mode I can check that.

@io53
Copy link

io53 commented Oct 12, 2021

@notenoughtech can you make sure you are not running into this linux kernel driver bug? #8663
Probably not if zzh works as it should, but worth a look.

@notenoughtech
Copy link
Author

notenoughtech commented Oct 12, 2021 via email

@Hedda
Copy link
Contributor

Hedda commented Oct 27, 2021 •
edited
Loading

@notenoughtech can you make sure you are not running into this linux kernel driver bug? #8663

He should not run into the bug mentioned in #8663 if his issue is with the new "SONOFF Zigbee 3.0 USB Dongle Plus (Model: "ZBDongle-P")" as it uses Silicon Labs CP2102N (from Silabs CP210x Family) USB-to-UART bridge chip instead of WCH CH341.

@notenoughtech you should maybe clarify in your original post that your issue here is specifically with "SONOFF Zigbee 3.0 USB Dongle Plus (Model: "ZBDongle-P" based on Texas Instruments CC2652P)" and not with their previous "Zigbee 3.0 USB Dongle (SKU: "9888010100045" based on Silicon Labs EFR32MG21)", as both are from ITead but only the "Zigbee 3.0 USB Dongle (SKU: "9888010100045") uses a CH341 USB-to-UART bridge chip.

At least that is my assumption considering the timing of this and your recent review article:

https://notenoughtech.com/home-automation/sonoff-zigbee-3-0-usb-dongle-plus/

https://www.reddit.com/r/Not_Enough_Tech/comments/q2vfq4/getting_started_with_sonoff_zigbee_30_usb_dongle/

@notenoughtech
Copy link
Author

notenoughtech commented Oct 27, 2021 via email

@Hedda
Copy link
Contributor

Hedda commented Oct 29, 2021 •
edited
Loading

FYI, perhaps not quite general knowledge but other than the obvious in avoiding having your Zigbee coordinator close to your Wi-Fi router/access-points, know that you can often get Zigbee pairing issues when got electromagnetic interference from USB 3.0 devices and cables connected to those devices are known to often interfere with Zigbee coordinator signal reception:

https://www.intel.com/content/www/us/en/products/docs/io/universal-serial-bus/usb3-frequency-interference-paper.html

Simplest solution to avoid EMF from devices is to use a long (1-2 meter) USB-extension-cable to get your ZB dongle a bit away.

Maybe little of topic, but FYI, I have collected these general best practices for avoiding pairing difficulties with Zigbee devices:

Best practices for avoiding pairing difficulties

  • If possible try to pair your Zigbee devices in their intended final location, (and not pair it next to the Zigbee coordinator and then need to move it after).
    • Pairing a Zigbee device next to the Zigbee coordinator and then moving it later can result in dropped/lost connections or other issues.
      • If the device you want to add is not brand new and as such never paired before then you always have to make sure to first manually reset the device to its factory default settings before you will be able to add/pair it.
  • Some battery-operated Zigbee devices are known to have problems with pairing if they have Low battery voltage.
    • Some people have reported replacing the battery on their newly received Xiaomi/Aqara devices solved pairing issues.
  • Check that you have enough Zigbee router devices (also known as Zigbee signal repeaters or range extenders) and if you do not have any, invest and add some mains-powered devices that will work as Zigbee routers.
    • Aim to start out with mains-powered devices before adding battery-operated devices as a "weak" Zigbee network mesh (e.g., the device is too far from the Zigbee coordinator or a Zigbee router) may prevent some devices from being paired. Zigbee router devices are also needed to increase the maximum of devices that can be connected to your Zigbee mesh network.
    • Note that some Zigbee devices are not fully compatible with all brands of Zigbee router devices. Xiaomi/Aqara devices are for example known not to work with Zigbee router devices from Centralite, General Electrics, Iris, Ledvance/OSRAM/ LIGHTIFY/Sylvania, Orvibo, PEQ, Securifi, and SmartThings/Samsung. Better results can usually be achieved by using mains-powered devices IKEA and Nue/3A Home or dedicated DIY routing devices based on Texas Instruments CC253x/CC26x2 and XBee Series 2/3 Zigbee radios.
  • Be patient as the pairing of some Zigbee devices may require multiple attempts and you may sometimes need to try again and again.
    • Some devices, like example those from Xiaomi/Aqara, are known to not be 100% compliant with the standard Zigbee specifications and may therefore require many paring attempts over 10-20 minutes or longer.

Also FYI, while on the subject of your issues I have also connected these general tips on improving Zigbee network range:

Tips on improving Zigbee network range

Both poor signal quality or signal interference can lead to transmission errors and related issues. This section contains fundamental troubleshooting tips on how to improve signal quality plus optimize range and coverage. Improving signal quality and removing sources of signal interference can in most cases maximize range and resolves problems related to transmission errors. Please try to follow at least some of these recommendations before posting on the community forums or reporting issues to developers.

  1. Adding more Zigbee routers between devices far away and the next closest router or your Zigbee coordinator. Zigbee network topology uses a "mesh network" design which means that each device that acts as a Zigbee router will extend the total range and coverage of your Zigbee network as a whole. The solution is to distribute more Zigbee routers in areas with poor reception. Note that while there are exceptions, understand that almost all permanently powered devices will serve as a Zigbee router. Thus adding more permanently powered Zigbee devices will allow greater range better coverage. There are also dedicated Zigbee routers which you can find by doing a community search for "Zigbee signal repeater" or "Zigbee range extender") (an example being IKEA Tradfri Signal Repeater). Devices that can not act as routers are typically battery-operated and known as "end devices". There are some exceptions as some end devices (e.g. Xiaomi/Aqara devices) have issues connecting through routers that are not from the same manufacturer.
  2. Use a long USB extension cable with your Zigbee coordinator adapter. This not only enables you to position the Zigbee coordinator adapter for better signal quality, but it also allows you to locate the Zigbee coordinator further away from Wi-Fi access-points/routers or other sources of 2.4GHz signals to avoid signal interference. Optimally in the best area in your home depending on your building's floorplan. Ideally, you want to place the Zigbee coordinator somewhere in the middle of your house/apartment. Building materials influence signal quality, for example, dense/thick concrete, bricks, masonry, etc. dampen all wireless signals. Place the Zigbee coordinator with some distance from walls, ceilings and floors. Also, try different orientations of your Zigbee antenna (or your whole Zigbee coordinator adapter if it has an internal antenna). Some Antennas have a stronger signal in a certain direction. Simply changing orientation can improve signal quality already. Note: A bad USB extension cable may lead to connection issues between the system and the Zigbee coordinator adapter, symptoms of this are disconnection messages.
  3. Zigbee and Wi-Fi can operate on various channels in the 2.4GHz spectrum. A busy Wi-Fi access-points/routers that are operating in the same frequency range (overlapping channels) as your Zigbee coordinator will drown out the Zigbee traffic, especially if they are located close to each other. To avoid interference between Zigbee and Wi-Fi try to choose channels without overlap. Check the channel your Wi-Fi access-points/routers are using (either by checking on the router's web interface or using a Wi-Fi analyzer app). Changing the channel of the Zigbee network usually requires reforming it and re-join/re-pair all of your Zigbee devices. It is therefor typically it's a lot easier to just change the channel used by your Wi-Fi router and/or access-points. See this article for Wi-Fi and Zigbee channels coexistance to avoid using overlapping frequency ranges.
  4. Check if updating the firmware on your Zigbee coordinator adapter and your Zigbee end devices is possible. Note that depending on your hardware the latest Zigbee coordinator firmware might not always be the one recommended by the community so it is advised to ask before upgrading.
  5. If your Zigbee coordinator adapter has a removable antenna (e.g., with an SMA-connector) then you have the option of replacing it with a high-gain antenna. Note that antennas with higher gain usually have directionality: You might have better reception on the same floor, but reception across floors might suffer. In addition, you also have the option to use an antenna extension cable if needed (usually using just a USB extension cable for your Zigbee coordinator adapter is the better alternative). This should really only be needed if you are trying to cover a long distance, like to another building or very dense/thick walls, ceilings and floors.
  6. Buy more powerful Zigbee radio hardware with a better radio range, preferably with an external antenna and based on a newer chip that offers up-to-date firmware. If you are not only experimenting with Zigbee but want a permanently stable and healthy Zigbee mesh network with potentially many devices then you should consider upgrading to a more powerful and newer Zigbee coordinator USB adapter or Ethernet to serial gateway/bridge. Generally, Zigbee adapters with an external antenna will have a better range and offer you more flexibility, therefore you will also want to avoid buying an internal Zigbee adapter unless it has a port for an external antenna.

@Hedda
Copy link
Contributor

Hedda commented Oct 29, 2021

@notenoughtech btw, regarding your range tests; Koenkk now stated current firmware doesn't yet support +20 dbm output:

#8840 (reply in thread)

That means that this CC2652P based Sonoff Zigbee 3.0 USB Dongle Plus adapter with its current firmware version only operate at +5 dBm output, so no wonder it does not perform better when compared to Electrorama’s zzh dongle which is based on CC2652R that physically only capable of +20 dBm output. Thus only once we get a new community firmware with RF switch configured will enabling +20 dBm output in application settings actually work.chiakikato also manually tested Sonoff Plus dongle hardware and confirms it support +20 dBm output:

#8840 (reply in thread)

Guess this means you have to redo the tests and update review when get new firmware.

@Koenkk
Copy link
Owner

Koenkk commented Oct 29, 2021 •
edited
Loading

@Hedda before people get confused,

CC2652R that physically only capable of +20 dBm output

This should be 5dbm, not 20dbm, so theoretically the ZZH should gave the same performance as the sonoff adapter currently.

@notenoughtech
Copy link
Author

notenoughtech commented Oct 29, 2021 via email

@Koenkk
Copy link
Owner

Koenkk commented Oct 29, 2021

@notenoughtech do you still have the pairing issues? If so could you provide me the required debug information so I can investigate this further? #9117 (comment)

@notenoughtech
Copy link
Author

notenoughtech commented Oct 29, 2021 via email

@hoggerz
Copy link

hoggerz commented Nov 16, 2021

Hi Koen,

I’ve recently purchased a couple of these, straight away with the latest 3.X.0 branch firmware, and also with the newer one you posted specifically referencing the sonoff dongle, I found issues with pairing Ikea bulbs. Oddly some that had older firmware versions paired first time with no issues, but newer firmware bulbs always had issues either pairing or setting up reporting for the bulbs. I had similar issues with some very old HUE b22 bulbs problems I will sniff the traffic of various items when I’m back home at the weekend.

hoggerz

@u8915055
Copy link

u8915055 commented Dec 1, 2021

Id like to leave a comment here as well. I purchased a couple of the sonoff zigbee 3.0 plus dongles and have not been able to get them to work. Now im new to zigbee2mqtt.. ive been a long time ZHA user through HA but want to move to z2m to get better device support.

I am running this on a debian vm, fresh install. I followed all the instructions to install z2m. I took my dongle and flashed it with 20210708. I was able to get z2m started and the web interface came up etc. It says its listening for devices to join but i cant get a single device to join. No logs no nothing (even with debug logging enabled) It just sits there. I have flashed this dongle on both a windows vm and also using the CC2538-bsl script.. both seemed to flash and verify successfully but still wont enable any devices to join.

I have about 25 zigbee devices that are all running fine with ZHA. I am running on channel 20.

I have found though that ive run into that problem with starting z2m raises that duplicate pan_id issue.. I was able to get around that by re-flashing and setting the pan_id to GENERATE in teh config file. That sometimes works. What always seems to let me get things started is to remove the external antenna. But that fact sort of suggests to me that something is happening at the stack level because with the antenna installed i hit the pan_id issue and without z2m will start.

Anyway i know im not giving a lot of cold hard facts here but that's where things stand for me. Unable to get a single device to join. Im thinking of flashing a CC2531 device and see if i can get that to accept devices on the same setup.

Happy to test or try anything. Thanks in advance.

@u8915055
Copy link

u8915055 commented Dec 1, 2021

Wanted to add some more info here.. i took my other sonoff 3.0 dongle from its package, disconnected the antenna.. replaced the flashed one, and restarted zigbee2mqtt, and then reattached the antenna. I am now able to get a few devices to connect.. it says in the web interface that its running firmware 20210708 as well.. strange why this one works and the other one doesnt. One may say i flashed it wrong but i did it a few times and did full verification on the written image and it checked out. Im going to take the original one and use it with ZHA and see if it works fine there.

Now im having other issues iwth it reading battery levels, not sure whats happening there, but at least its reading some devices.

@Hedda
Copy link
Contributor

Hedda commented Dec 1, 2021

Wanted to add some more info here.. i took my other sonoff 3.0 dongle from its package, disconnected the antenna.. replaced the flashed one, and restarted zigbee2mqtt, and then reattached the antenna. I am now able to get a few devices to connect.. it says in the web interface that its running firmware 20210708 as well.. strange why this one works and the other one doesnt. One may say i flashed it wrong but i did it a few times and did full verification on the written image and it checked out. Im going to take the original one and use it with ZHA and see if it works fine there.

Sounds like a hardware fault with the other one of your adapters if one adapter is working and the other adapter is not when both are flashed with same firmware image. Maybe try switching antenna between them to rule out one of the antennas being at fault.

FYI, once fixed know there is more general and broader discussion about ITead's Sonoff Zigbee 3.0 USB Dongle Plus here -> #8840

@Nobody931
Copy link

Nobody931 commented Dec 4, 2021 •
edited
Loading

Hello, I ordered the Sonoff Zigbee3.0 USB Dongle (ITEAD) two weeks ago and it arrived yesterday. Before I used the CC2530 with serial connection the my raspberry Pi4 which worked good, however, after updating the raspberry pi I was not able to connect the CC2530, so I order the Sonoff Dongle (as I read that it does not perform that good anymore).
I was able to connect the Dongle and pair with a xiaomi motion detector (with battery status). However, I am not able to pair my other devices which compromises of IKEA light bulbs, e1743 buttons and other supported devices. I found this issue and before I try reflashing the USB Dongle (which already shipped with the latest firmware [20210708]) I thought maybe there is another problem and used the cc2531 sniffer for capturing the communication. This is the capture of the e1743 button with a fresh battery.
logAndCapture.zip
I hope the log and capture is good enough, if necessary I can provide more log and capture other devices as well if time allows it :) Would love the get my old setup working, thank you!
tldr: Changed from CC2530 to pre-flashed Sonoff Dongle, successfully paired motion sensor, but not able to pair other (supported and functioning) devices.

@u8915055
Copy link

u8915055 commented Dec 4, 2021 via email

@flatsiedatsie
Copy link
Contributor

flatsiedatsie commented Dec 20, 2021 •
edited
Loading

My Sonoff 3.0 Plus USB stick arrived. I've been trying it on a fresh Linux device (Webthings Gateway), with a brand new random PAN_ID and Network_Key.

I tried to switch an IKEA light over from my main system (with Conbee2), to this new test system.

So far I have not been able to get it to pair.

I have tried:

  • powering off the main controller with the Conbee2 stick, just to make sure it wasn't interfering.
  • moving the Sonoff dongle and lightbulb closer together
  • many reboots and full power-off situations.

At some point I noticed an SRSP - ZDO - mgmtPermitJoinReq after 6000mserror:

2021-12-20 22:19:19.703 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/request/permit_join' with data '{"value": true}'
2021-12-20 22:19:19.705 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.707 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:19: Zigbee: allowing new devices to join.
2021-12-20 22:19:19.709 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.712 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/config/devices/get' with data ''
2021-12-20 22:19:19.713 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.714 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:error 2021-12-20 22:19:25: Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SRSP - ZDO - mgmtPermitJoinReq after 6000ms'
2021-12-20 22:19:25.717 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.772 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:25: Error: SRSP - ZDO - mgmtPermitJoinReq after 6000ms
2021-12-20 22:19:25.774 INFO   : zigbee2mqtt-adapter:     at Timeout.waiter.timer.setTimeout [as _onTimeout] (/home/pi/.webthings/data/zigbee2mqtt-adapter/zigbee2mqtt/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35)
2021-12-20 22:19:25.776 INFO   : zigbee2mqtt-adapter:     at ontimeout (timers.js:436:11)
2021-12-20 22:19:25.778 INFO   : zigbee2mqtt-adapter:     at tryOnTimeout (timers.js:300:5)
2021-12-20 22:19:25.779 INFO   : zigbee2mqtt-adapter:     at listOnTimeout (timers.js:263:5)
2021-12-20 22:19:25.781 INFO   : zigbee2mqtt-adapter:     at Timer.processTimers (timers.js:223:10)
2021-12-20 22:19:25.782 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.784 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:25: MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{},"error":"SRSP - ZDO - mgmtPermitJoinReq after 6000ms","status":"error"}'

But most of the time there's simply... nothing.

I still have to try:

  • extension cable
  • pairing different devices

I don't want to flash a different firmware, as for my purposes I'm looking for a cheap USB stick that I can recommend others to buy. It needs to be globally available, and work out of the box.

@Koenkk
Copy link
Owner

Koenkk commented Dec 21, 2021

@flatsiedatsie make sure to check https://www.zigbee2mqtt.io/guide/faq/#zigbee2mqtt-crashes-after-some-time

@dreesi
Copy link

dreesi commented Dec 21, 2021

My Sonoff 3.0 Plus USB stick arrived. I've been trying it on a fresh Linux device (Webthings Gateway), with a brand new random PAN_ID and Network_Key.

I tried to switch an IKEA light over from my main system (with Conbee2), to this new test system.

So far I have not been able to get it to pair.

I have tried:

  • powering off the main controller with the Conbee2 stick, just to make sure it wasn't interfering.
  • moving the Sonoff dongle and lightbulb closer together
  • many reboots and full power-off situations.

At some point I noticed an SRSP - ZDO - mgmtPermitJoinReq after 6000mserror:

2021-12-20 22:19:19.703 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/request/permit_join' with data '{"value": true}'
2021-12-20 22:19:19.705 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.707 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:19: Zigbee: allowing new devices to join.
2021-12-20 22:19:19.709 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.712 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/config/devices/get' with data ''
2021-12-20 22:19:19.713 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.714 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:error 2021-12-20 22:19:25: Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SRSP - ZDO - mgmtPermitJoinReq after 6000ms'
2021-12-20 22:19:25.717 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.772 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:25: Error: SRSP - ZDO - mgmtPermitJoinReq after 6000ms
2021-12-20 22:19:25.774 INFO   : zigbee2mqtt-adapter:     at Timeout.waiter.timer.setTimeout [as _onTimeout] (/home/pi/.webthings/data/zigbee2mqtt-adapter/zigbee2mqtt/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35)
2021-12-20 22:19:25.776 INFO   : zigbee2mqtt-adapter:     at ontimeout (timers.js:436:11)
2021-12-20 22:19:25.778 INFO   : zigbee2mqtt-adapter:     at tryOnTimeout (timers.js:300:5)
2021-12-20 22:19:25.779 INFO   : zigbee2mqtt-adapter:     at listOnTimeout (timers.js:263:5)
2021-12-20 22:19:25.781 INFO   : zigbee2mqtt-adapter:     at Timer.processTimers (timers.js:223:10)
2021-12-20 22:19:25.782 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.784 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:25: MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{},"error":"SRSP - ZDO - mgmtPermitJoinReq after 6000ms","status":"error"}'

But most of the time there's simply... nothing.

I still have to try:

  • extension cable
  • pairing different devices

I don't want to flash a different firmware, as for my purposes I'm looking for a cheap USB stick that I can recommend others to buy. It needs to be globally available, and work out of the box.

same for me, ordered the sonoff stick but IKEA bulbs aren't pairing. I didn't start debugging yet and will try to follow the points Koenkk advised.

@Hedda
Copy link
Contributor

Hedda commented Dec 21, 2021 •
edited
Loading

I don't want to flash a different firmware, as for my purposes I'm looking for a cheap USB stick that I can recommend others to buy. It needs to be globally available, and work out of the box.

I highly recommend that you at least change the requirement to have a relatively fast/easy way for users to upgrade the Zigbee coordinator firmware on whatever adapter you choose. Suggest that you check out this "feature/ITead_Sonoff_Zigbee-delay" branch from JelmerT patch for cc2538-bsl which makes upgrading firmware much easier than following ITead's official guide (PR JelmerT/cc2538-bsl#114 with patch for this sonoff plus dongle) -> https://github.com/JelmerT/cc2538-bsl/tree/feature/ITead_Sonoff_Zigbee-delay

Would therefore strongly suggest that you slightly change your particular "works out-of-the-box" condition requirement for fast/easy when looking at any Zigbee Coordinator adapter as upgrading the firmware at some point will always be a minimum requirement if you want a stable Zigbee network. If not direct when you bought it (because you got lucky and received stock when the manufacturer just happen to have updated to a new revision) then later on once you discover bugs on there is a security exploit.

Remember that all hardware today will need updates for firmware/drivers/software, so understand that by today's standard suggesting that users should never ever have to update their Zigbee coordinator firmware is almost like suggesting that users should be able to buy a smartphone and never update it either, (then thinking they can ask for support and not expecting being told to upgrade before getting any help).

Regardless of the standard dilemma for Zigbee DIY solutions is that you can today not really find any Zigbee coordinator adapter hardware and Zigbee software solution that is fast/easy, cheap, and good: You can only pick two of those.

image

@flatsiedatsie
Copy link
Contributor

ON TOPIC:

I just tried flashing the latest development firmware (CC1352P2_CC2652P_launchpad_coordinator_20211217) onto the Sonoff stick, using the excellent Python script recommended by @Hedda

I still cannot pair my IKEA lightbulb.

  • The Conbee stick is still unplugged from my other Raspberry Pi, so it's no interfering.
  • I'm using an extension cable now.
  • I tried keeping the Sonoff stick very close to the lightbulb as it was flashing.

I still don't see any reaction from Zigbee2MQTT.

OFF TOPIC:

highly recommend that you at least change the requirement to have a relatively fast/easy way to upgrade the Zigbee coordinator firmware on whatever adapter you choose.

Yes that's definitely something I'd like too. I'd like to build firmware updating of the USB stick into the Webthings Addon I work on. I've just updated that addon to add security support. I could add update flashing next.

feature/ITead_Sonoff_Zigbee-delay

Very nice. If I understand correctly, USB sticks like Zig-Ah-Zig-Ah also have this feature.

This is my list of desired features for a USB Zigbee stick:

  • Works with a wide range of devices, including IKEA buttons and sensors.
  • Pairing is usually succesfull on the first try.
  • Can handle enough devices for an average home and has enough range by itself.
  • Can easily be purchased all over the world and at any time through 'big' retail channels.
  • Likely comes with a working firmware at purchase time.
  • Can be upgraded through Linux command.

Nice to have:

  • Works with Zigbee2MQTT, but useful if the stick also has value for other smart home systems that might not build on Zigbee2MQTT
  • Cheap

So far I've been recommending the Conbee2 stick. I've never once upgraded it since purchasing it years ago, but it seems to work fine so far.Theoretically it can be updated from the linux command line (which I'm attempting now, but I'm not having much luck so far)

The mayor downsides of the Conbee2 are:

  • The Zigbee2MQTT project doesn't seem to reallty recommend it?
  • It's relatively expensive.

TLDR: if the Sonoff stick could be made to work well, I think it's a winner.

@Hedda
Copy link
Contributor

Hedda commented Dec 21, 2021 •
edited
Loading

feature/ITead_Sonoff_Zigbee-delay

Very nice. If I understand correctly, USB sticks like Zig-Ah-Zig-Ah also have this feature.

@flatsiedatsie Yes such "Auto BSL" (formerly known as BootStrap Loader) feature allows you to trigger entering into bootloader mode via software and is built into many new microcontrollers, including CC2538, CC2652 and CC1352, but not supported by all:

https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/README.md

Auto BSL is however not a feature of CC2530 and CC2531, however for those you can manually force it to enter bootloader and then flash them with zigpy-znp tools -> https://github.com/zigpy/zigpy-znp/blob/dev/TOOLS.md#flash-operations but on the other hand CC2530 and CC2531 are no longer recommended for Zigbee2MQTT at all, especially not s a new purchase for new installations by beginners and users new to Zigbee.

I'd like to build firmware updating of the USB stick into the Webthings Addon I work on. I've just updated that addon to add security support. I could add update flashing next.

Off-topic; Then suggest you check out and take inspiration from ESP Web Tools https://esphome.github.io/esp-web-tools/ (which support OTW flashing of firmware via a web browser using Web Serial API, though currently only for ESP based MCU chips), as well a https://github.com/electrolama/llama-bsl (fork of cc2538-bs with some added auto-detection feature and more similar features planned to make OTW flashing of firmware more user-friendly)

This is my list of desired features for a USB Zigbee stick:

  • Works with a wide range of devices, including IKEA buttons and sensors.
  • Pairing is usually succesfull on the first try.
  • Can handle enough devices for an average home and has enough range by itself.
  • Can easily be purchased all over the world and at any time through 'big' retail channels.
  • Likely comes with a working firmware at purchase time.
  • Can be upgraded through Linux command.

You need to remember that most of the things on your list do not depend on the Zigbee hardware adapter or even the Zigbee stack firmware, instead, those things will rather depend on the Zigbee software solution and the Zigbee devices that you connect. Understand that Zigbee is the wild wild west as not all manufacturers do not strictly follow the rules, and all of the rules are not even strict rules but more like guidelines. So implementing support for different Zigbee devices from hundreds or thousands of manufacturers is hard and complex. Jeedom developers recently wrote a great blog post article which makes for a good read summing all that up better than I can:

https://blog.jeedom.com/6107-parlons-zigbee/

A couple of quick tips for any new Zigbee user are; upgrade Zigbee coordinator firmware before beginning. Stay away from Aqara, Xiaomi, as well as Tuya devices. When starting to add devices begin by adding at least a few main-powered devices that will work as Zigbee router (a.k.a. repeater) devices, preferably flash CC2652 and/or CC1352 adapters with Zigbee router firmware and use those as known working Zigbee routers to create a stable network mesh at its core.

The mayor downsides of the Conbee2 are:

  • The Zigbee2MQTT project doesn't seem to reallty recommend it?
  • It's relatively expensive.

For now, recommend using Texas Instruments based adapters with Zigbee2MQTT. ConBee/RaspBee support is only experimental is Zigbee2MQTT. So is Silicon Labs (Silabs EZSP) based adapters.

https://www.zigbee2mqtt.io/information/supported_adapters.html

Texas Instruments Z-Stack 3 based asapters also work with Home Assistant's ZHA and Jeedom's Zigbee plugin

ConBee/RaspBee is currently really only good if used with dresden elektronik's own deCONZ software and not third-party.

@flatsiedatsie
Copy link
Contributor

Update: success! I was just able to pair an IKEA bulb with the Sonoff 3.0 Plus stick running the latest development coordinator firmware. It even paired on the first attempt this time.

Turns out there was an error in my code, and because of that the security wasn't being implemented. This meant that the new Raspberry Pi was still using the default PAN_ID and Network_key.

I suspect that the IKEA bulb didn't see the need to pair with the same controller it was technically already paired too.

@oferwald
Copy link

The dongle should pick up the EPID from the backup file after you reset its nvram.

I ended up resetting most of my routers on the network anyhow, which was a huge pita, but things seem to work better now.

I believe that @castorw is probably working on a better fix for this.

@SkarfyFPV
Copy link

Hi there, I am having the same problem and trying to apply this fix, but I am new to all of this and I cannot find the way (or the shell) to run this command: python zigpy_znp/tools/nvram_reset.py /dev/ttyUSB1.
I did the Home Assistant installation following the official doc: https://www.home-assistant.io/installation/raspberrypi
but I didn't install Home Assistant Container...
Please any help will be appreciated.

@SkarfyFPV
Copy link

How can I delete the NVRAM without access to the shell? Or how can I access the shell? Thanks

@oferwald
Copy link

oferwald commented Feb 2, 2022

@SkarfyFPV My suggestion is that you just take the stick out to any computer that you can install zigpy on and have shell access to.

@SkarfyFPV
Copy link

Thank you @oferwald, this is a good idea. Would you recommend using docker containers instead next time a start from scratch?

@oferwald
Copy link

oferwald commented Feb 2, 2022

@SkarfyFPV I am not sure I am in a position to recommend anything, I am a simple user of this great project.
I have z2m and ha both running on docker, it was probably the simplest way for me personally to get going. It has its advantages and disadvantages (mainly for HA, as I am missing the controller). But I am ok with my current setup. It mostly works :). I don't think that there is really a one solution that fits everyone, so my suggestion is to just try and find out what is best for you.

@SkarfyFPV
Copy link

Thank you @oferwald I really appreciate your help. I have been some time thinking about trying with docker and I think I will just get another SD and try it. Also, I see a great advantage with docker, as you can have a few containers and just run the ones you need, which seems quite good to me for quick testing. Thanks again!

@danneh82
Copy link

danneh82 commented Feb 8, 2022

Finally managed to reset the nvram and changed the EPID in the coordinator backup file. Happy to report that i have been able to get devices to pair again.

Almost completely back up and running bar one stubborn light which is refusing to go in pairing mode!

@drogfild
Copy link

drogfild commented Feb 9, 2022

@danneh82 Just to make sure (before I'm doing the same), you edited EPID in the backup file to be "dddddddddddddddd"? Previous it was same as your coordinator IEEE?

@danneh82
Copy link

danneh82 commented Feb 9, 2022

Yep exactly :)

@SkarfyFPV
Copy link

SkarfyFPV commented Feb 20, 2022 •
edited
Loading

Hi! apologies, it took me more than expected to be able to test this. The result for me was unsuccessful, executed a few times :(

As I have directly installed HASS on my Pi I don't have access to the shell to run this commands, for this reason, I had to unplug my CC2652P and plug it into another Pi where I am running Debian to be able to clear the NVRAM. Maybe this is the reason for my failure.

These are the steps I followed:

  1. Shutdown z2mqtt
  2. Unplug CC2652P from HASS and plug it into another Pi with Debian
  3. Clear the NVRAM of the coordinator with znp nvram_reset, for me it was:
    python zigpy_znp/tools/nvram_reset.py /dev/ttyUSB0
  4. Edit my coordinator_backup.json
  5. Modify the line with tag "extended_pan_id" to show as this: "extended_pan_id": "dddddddddddddddd",
  6. Plug CC2652P into HASS Pi.
  7. Restart your z2mqtt

After these steps, my network was working with the already set devices but I could NOT pair anything else. I tried a couple of different bulbs and switches without any luck.

Any other idea that comes to your mind which may help me to sort the problem?

Thank you all!

@l0rda
Copy link

l0rda commented Feb 20, 2022

@SkarfyFPV i changed channel to 25 and it works as expected

@SkarfyFPV
Copy link

@l0rda if I change the channel then nothing works. None of the previously set devices are recognised.

@rvt
Copy link

rvt commented Feb 22, 2022

Just my two cents if somebody finds this..,
clearing the nvram only did not work for me but I did the below steps, but I am not sure what step really did the fix:

  • stop zigbee2mqtt
  • Do the nvram clear
  • unplug the stick, and re-add (did you turn it off and on??)
  • Change extended_pan_id to dddddddddddddddd
  • in database.db, I removed all coordinators, for some reason I had multiple
  • start zigbee2mqtt

After this all pairing did work directly, not even I had to keep items close to the coordinator.

This was after I upgraded from the good old CC2531 chip to the sonoff zigbee3 CC2652P stick.

@lordlala
Copy link

I am having the exact same issue as @SkarfyFPV .

All of a sudden my zigbee devices stop working and become unresponsive. If I try and remove and re-add, they never seem to repair.

What a trainwreck zigbee is :(

@Hedda
Copy link
Contributor

Hedda commented Mar 17, 2022 •
edited
Loading

There are still several issues/problems when upgrading from any CC2530/CC2531 adapter to any CC2652/CC1352 adapter.

If you upgraded from CC2530/CC2531 to CC2652/CC1352 in a "production environment" and therefore do not want to spend a lot of time troubleshooting to try to find the root causes then the general recommendation today is to just remove and re-repair all of your mains-powered Zigbee Router devices, starting with the Zigbee Router devices closest to the Zigbee Coordinator adapter and then continue moving your aways outwards. That is not quite like starting from scratch but will still take a while if you have a lot of Zigbee Router devices.

@lordlala
Copy link

So, to be clear in my case I've been running this for months with absolutely no issue then all of a sudden devices stop communicating and if I remove them from zigbee2mqtt, I can never seem to re-add them, which never used to be an issue. I have no idea what changed.

@rvt
Copy link

rvt commented Mar 17, 2022

I wonder if there was somebody else that did set up a coordinators, perhaps somebody in the neighborhood... wel, at least in my DB I found coordinators that where still old, perhaps U used them once... after I removed them all and restarted zigbee2mqtt things started to work again

@hunterdrayman
Copy link

Just adding my experience of upgrading from CC2531 to CC2562P. I have all the woes mentioned here with devices not re-pairing. I removed all the routers and I do see a successful interview of a router. It appears in configuration.yaml and in the zigbee2mqtt device map but not in the front end device list. All of the sensors etc that were registered before the swap continue to talk to the new coordinator but no new ones can be added. This is after resetting the coordinator NVRAM etc.

Fortunately I took a VM snapshot of the host before replacing my trusty old CC2531 coordinator so it's now time to do a device swap and restore. Amusingly, I sent back a recently purchased switch as being DOA - I wonder if I should admit my mistake to the vendor. Ooops!

@github-actions
Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Apr 26, 2022
@SkarfyFPV
Copy link

Thank you guys for chipping in. I honestly didn't find a solution for my problem and I am just waiting for a moment where I have time enough to build the HASS again from scratch... I did try all your suggestions but sadly none worked for me.

@github-actions github-actions bot removed the stale Stale issues label May 4, 2022
@Hedda
Copy link
Contributor

Hedda commented May 4, 2022 •
edited
Loading

Tip before troubleshooting further is to be sure to follow all these general advice and best practices to get a more optimal setup.

Improving Zigbee network range

Poor signal quality or interference can lead to transmission problems and show symptoms such as or errors when sending and/or receiving Zigbee messages that will cause significant degradation in Zigbee network performance. Below is some fundamental background information and tips for a Zigbee setup starting point to achieve better signal quality, coverage and range.

Following these essential optimization tips will typically resolve many known issues caused by a bad setup of Zigbee Coordinator adapter and Zigbee network mesh or Zigbee devices. At the very least taking these actions should improve most message transmitting and receiving issues caused by not knowing the basics needed to workaround related Zigbee limitations.

Understanding common causes of problems in Zigbee setups

  • Zigbee Coordinator adapter hardware:
    • Poor placement of the Zigbee Coordinator and/or wrong orientation of Zigbee Coordinator antenna.
    • Poor / outdated / incompatible Zigbee Coordinator firmware on the Zigbee Coordinator adapter.
    • Old, outdated or underpowered Zigbee Coordinator hardware or poor Zigbee Coordinator antenna.
  • Zigbee Coordinator and Zigbee devices sensitive to all types of RFI/EMI/EMF interference:
    • Electromagnetic interference (EMI) from electronics caused by Electromagnetic Fields (EMF).
    • USB 3.0 ports and computer peripherals are known culprits of EMI/EMF disrupting Zigbee.
    • 2.4 GHz Radio Frequency Interference (RFI) from Wi-Fi Routers and Wi-Fi Access Points, or others.
    • Radio proximity as well conflicting radio frequencies (Wi-Fi) overlapping used Zigbee frequency.
  • Too few or far distance between Zigbee devices in Zigbee network mesh:
    • Too many obstacles between the Zigbee Coordinator or Zigbee Router and Zigbee End Device.
      • Zigbee network mesh depends on Zigbee Router devices to extend range and coverage.
      • All individual Zigbee devices do have poor signal penetration and not long range.
    • Too few Zigbee Router devices Zigbee Coordinator or Zigbee Router and Zigbee End Device.
      • Only mains-powered devices can be repeaters that extend network range and coverage.
      • Any battery-operated products are a Zigbee End Device and will never be a Zigbee Router.

Simple actions that should improve most Zigbee setups

  • Zigbee Coordinator adapter hardware:
    • Try different physical placement and orientations of Zigbee Coordinator or its antenna.
      • Optimal placement of Zigbee Coordinator is as much in middle of the house as possible.
      • Try to place the Zigbee coordinator at some distance way from walls, ceilings and floors.
      • Try different orientations of your Zigbee antenna (or whole Zigbee coordinator adapter).
    • Update to a later released Zigbee Coordinator firmware on the Zigbee Coordinator adapter.
      • Updating FW is easy and community often provide newer firmware than manufacturer.
    • Buy a more powerful Zigbee Coordinator adapter based on a newer/modern Zigbee chip hardware.
      • Recommended as older adapter might work but have obsolete hardware and/or firmware.
  • Zigbee Coordinator and Zigbee devices RFI sensitive as susceptible to all types of EMI/EMF interference:
    • Use a long USB extension cable to place Zigbee Coordinator away from interference and obstacles.
      • Try to make sure the USB extension cable has proper shielded (thick cables usually do).
      • Longer USB extension cable also make it easier to orient Zigbee Coordinator and antenna.
    • Only connect Zigbee Coordinator USB adapter to a USB 2.0 ports (or a powered USB 2.0 hub).
      • USB 3.0 ports and peripherals are infamously known for causing EMI/EMF interference.
    • Shield close unshielded computers and unshielded peripherals by using metal enclosures/chassis.
      • Single-board-computers and USB hard drives are especially known as source of EMI/EMF.
    • Change Wi-Fi Router and Access Points channels to not interfere with the default Zigbee channel.
      • Changing Wi-Fi channel on Wi-Fi devices is recommended over changing Zigbee channel.
    • Change Zigbee channel on Zigbee network if have too many strong sources of overlapping Wi-Fi.
      • Changing Zigbee channel is considered a last resort so make sure to do research before.
        • Normally require re-pairing all, and not all devices support all Zigbee channels.
  • Too few or far distance between Zigbee devices in Zigbee network mesh:
    • Zigbee network mesh depends on having Zigbee Router devices to extend range and coverage:
      • Add additional mains-powered Zigbee devices known to be good as Zigbee Router device.
        • Add more Zigbee Router devices to extend network mesh range and coverage.
        • Note that not all mains-powered Zigbee devices act as a Zigbee Router device.
        • Some Zigbee Router devices are know to only work well with same brand devices.
      • Buy a few known good dedicated Zigbee Router product (ex. “IKEA Tradfri Signal Repeater”).
        • Search community forums for "Zigbee signal repeater" or "Zigbee range extender".
      • Buy a few extra new Zigbee USB adapters to use after flashing with Zigbee Router firmware.
      • Reflash/reuse Zigbee USB adapters to Zigbee Router when upgrading Zigbee Coordinator.

Best practices to avoid pairing/connection difficulties

Verify that you try to follow recommended best practices to avoid pairing and/or connection issues:

  • If possible try to pair your Zigbee devices in their intended final location, (and not pair it next to the Zigbee coordinator and then need to move it after).
    • Pairing a Zigbee device next to the Zigbee coordinator and then moving it later can result in dropped/lost connections or other issues.
      • If the device you want to add is not brand new and as such never paired before then you always have to make sure to first manually reset the device to its factory default settings before you will be able to add/pair it.
  • Some battery-operated Zigbee devices are known to have problems with pairing if they have Low battery voltage.
    • Some people have reported replacing the battery on their newly received Xiaomi/Aqara devices solved pairing issues.
  • Check that you have enough Zigbee router devices (also known as Zigbee signal repeaters or range extenders) and if you do not have any, invest and add some mains-powered devices that will work as Zigbee routers.
    • Aim to start out with mains-powered devices before adding battery-operated devices as a "weak" Zigbee network mesh (e.g., the device is too far from the Zigbee coordinator or a Zigbee router) may prevent some devices from being paired. Zigbee router devices are also needed to increase the maximum of devices that can be connected to your Zigbee mesh network.
    • Note that some Zigbee devices are not fully compatible with all brands of Zigbee router devices. Xiaomi/Aqara devices are for example known not to work with Zigbee router devices from Centralite, General Electrics, Iris, Ledvance/OSRAM, LIGHTIFY/Sylvania, Orvibo, PEQ, Securifi, and SmartThings/Samsung. Better results can usually be achieved by using mains-powered devices IKEA and Nue/3A Home or dedicated DIY routing devices based on Texas Instruments CC253x/CC26x2 and XBee Series 2/3 Zigbee radios.
  • Be patient as the pairing of some Zigbee devices may require multiple attempts and you may sometimes need to try again and again.
    • Some devices, like example those from Xiaomi/Aqara, are known to not be 100% compliant with the standard Zigbee specifications and may therefore require many paring attempts over 10-20 minutes or longer.

External resources for initial Zigbee setup optimizations:

@mjanssens mjanssens mentioned this issue Jun 2, 2022
Closed
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2022

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Jun 4, 2022
@Koenkk Koenkk mentioned this issue Aug 25, 2022
Closed
@JanvdW304 JanvdW304 mentioned this issue Jan 3, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
problem Something isn't working stale Stale issues
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests