Do not crash Syncsketch addon if it cannot connect to the Syncsketch server due to wrong server or credentials

[BigRoy]

Changelog Description

Do not crash Syncsketch addon if it cannot connect to the Syncsketch server due to wrong server or credentials

Additional info

Previously this would error with:

2024-10-25 13:52:02 2024-10-25 11:52:02 ERROR      server          Error during syncsketch 0.2.0+dev setup
2024-10-25 13:52:02 
2024-10-25 13:52:02     Traceback (most recent call last):
2024-10-25 13:52:02       File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 974, in json
2024-10-25 13:52:02         return complexjson.loads(self.text, **kwargs)
2024-10-25 13:52:02                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-25 13:52:02       File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
2024-10-25 13:52:02         return _default_decoder.decode(s)
2024-10-25 13:52:02                ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-25 13:52:02       File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
2024-10-25 13:52:02         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2024-10-25 13:52:02                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-25 13:52:02       File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
2024-10-25 13:52:02         raise JSONDecodeError("Expecting value", s, err.value) from None
2024-10-25 13:52:02     json.decoder.JSONDecodeError: Expecting value: line 3 column 1 (char 2)
2024-10-25 13:52:02     
2024-10-25 13:52:02     During handling of the above exception, another exception occurred:
2024-10-25 13:52:02     
2024-10-25 13:52:02     Traceback (most recent call last):
2024-10-25 13:52:02       File "/backend/ayon_server/api/server.py", line 448, in startup_event
2024-10-25 13:52:02         await version.setup()
2024-10-25 13:52:02       File "/addons/syncsketch/0.2.0+dev/server/__init__.py", line 47, in setup
2024-10-25 13:52:02         await self.create_syncsketch_webhook()
2024-10-25 13:52:02       File "/addons/syncsketch/0.2.0+dev/server/__init__.py", line 121, in create_syncsketch_webhook
2024-10-25 13:52:02         if existing_webhooks.json():
2024-10-25 13:52:02            ^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-25 13:52:02       File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 978, in json
2024-10-25 13:52:02         raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
2024-10-25 13:52:02     requests.exceptions.JSONDecodeError: Expecting value: line 3 column 1 (char 2)

Testing notes:

1. Set up all required credentials and server - but to the wrong address that server can't connect to
2. Restart server
3. Addon should still work and not be broken.

[64qam]

lgtm

[MilaKudr]

2024-10-25 12:50:45 ERROR server Error during syncsketch 0.2.0+dev setup

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 199, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request
    raise new_e
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 693, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 214, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7ff441ada0d0>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.prdel.com', port=443): Max retries exceeded with url: /api/v2/notifications/78fd6d047b0f40dcbd96d105daf72352/webhooks/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff441ada0d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/backend/ayon_server/api/server.py", line 448, in startup_event
    await version.setup()
  File "/addons/syncsketch/0.2.0+dev/server/__init__.py", line 47, in setup
    await self.create_syncsketch_webhook()
  File "/addons/syncsketch/0.2.0+dev/server/__init__.py", line 111, in create_syncsketch_webhook
    existing_webhooks = requests.request(
                        ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.prdel.com', port=443): Max retries exceeded with url: /api/v2/notifications/78fd6d047b0f40dcbd96d105daf72352/webhooks/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff441ada0d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

2024-10-25 12:50:45 ERROR server Addon syncsketch 0.2.0+dev failed to start. Unloading.
2024-10-25 12:50:45 INFO server Unloading addon syncsketch 0.2.0+dev
2024-10-25 12:50:46 GOOD NEWS server Server is now ready to connect

[BigRoy]

Thanks @MilaKudr could you try again?

[BigRoy]

@martastain you're way more knowledgable here - should there be better ways to catch or log these requests errors and identify them, please let me know :)

[martastain]

It would be nice to handle TimeoutError at least.

I am not sure what status codes we should expect - but raising ValueError during setup here:

        if webhook_created.status_code != 200:
            logging.info(
                "Something went wrong when trying to create the Webhook.")
            raise ValueError(webhook_created.json())

Will disable the addon, while server unavailable will keep it alive, but probably defunct as it just returns.

BTW: it does not hurt in this case because it happens during server initialization, but it is not a good idea to use requests (blocking) in async functions. Use httpx instead.

[iLLiCiTiT]

@martastain could you please convert it to use httpx? We need this fix ASAP and you'll make it fastest.

[martastain]

I've replaced requests.request calls with httpx, could you pls test the changes?

[jakubjezek001]

So tested and works like a charm! Thank you @BigRoy.

PS: it was not easy to test since docker is quite bithing about the new #.#.#+dev tagging in version. The previouse #.#.#-dev.1 was more afficient for sure @dee-ynput.