Feeds are not updating #10
Comments
herndlm
changed the title
|
Fyi I started debugging this, might be related or not but there is definitely a problem in case the async job throws an exception, e.g.
I'm not a python dev, but am happy to help out here and learn a thing or the other potentially. Feel free to improve/fix this if you know what's going on, if not or you're busy, I'll try too :) UPDATE: I might have not been waiting long enough, my use case actually throws an |
|
Damn, I hoped you had a lead on this. 😅 |
|
Well I started running/using it on my raspberry pi now, so I'll take a close look there too |
|
Oh this is interesting. I can reproduce it at least. It just randomly stops as in I don't know enough about Python, the libs and the app yet but I wonder if it is dead-locking the DB or something similar. After a restart it immediately imported events. Will continue later.. Actually every or every other new unwetter update seems to lock it again 🤔 |
This seems to be fixing jplitza#10
|
I have found at least the problem that disguises the actual problem. Apparently if there is a reference to a task and the task is not finished yet (because of the SIGINT loop) it will not be garbage collected and this seems to be not triggering its error handling => if any exception is raised inside the task we will never see it unless we kill the app / task. https://bugs.python.org/issue28274 is related So I assume some HTTP or parse error is happening but we just don't see it because of the SIGINT loop. I suggest to improve this first, by failing early if the task fails, e.g. by either using https://docs.python.org/3/library/asyncio-task.html#asyncio.Task.add_done_callback or refactoring the SIGINT loop somehow. I can at least try to improve it :) After that the actual update feeds task should be hardened by e.g. logging an error and skipping a feed instead of raising an exception that ends the task. |
- fully switches to the recommended high-level API by not interfering with the loop in any way - removes the need to manually cleanup tasks by having to wait for a signal - fixes the problem with the update feeds task where exceptions where only shown on shutdown This is the first part of the fix for jplitza#10
- fully switches to the recommended high-level API by not interfering with the loop in any way - removes the need to manually cleanup tasks by having to wait for a signal - fixes the problem with the update feeds task where exceptions where only shown on shutdown This is the first part of the fix for jplitza#10
This logs all request errors via httpx that can happen from time to time (connect error, timeout, ..) and continues updating other feeds instead of terminating the task. See also https://www.python-httpx.org/exceptions/ This should greatly improve stability of the update feeds task which is related to jplitza#10
This logs all request errors via httpx that can happen from time to time (connect error, timeout, ..) and continues updating other feeds instead of terminating the task. See also https://www.python-httpx.org/exceptions/ This should greatly improve stability of the update feeds task which is related to #10
- fully switches to the recommended high-level API by not interfering with the loop in any way - removes the need to manually cleanup tasks by having to wait for a signal - fixes the problem with the update feeds task where exceptions where only shown on shutdown This is the first part of the fix for #10
Hi,
I think there might be a problem with your hosted instance as it seems to be not updating the feeds.
Output of
feedsI guess you're basically not supporting that instance but I wanted to report this, as it seems to be more global, before trying to host my own.
Btw. I think this project is a great idea and appreciate it very much!
The text was updated successfully, but these errors were encountered: