[R] Download SurvStat data via web service

[stephaneghozzi]

At the moment there are two data sources: RKI's corona dashboard and RKI's SurvStat. The first is already queried automatically, the second however has to be downloaded manually, although there is a web service https://tools.rki.de/SurvStat/SurvStatWebService.svc, which however is a bit tricky to use. See https://github.com/rgieseke/opencoviddata and in particular https://github.com/rgieseke/opencoviddata/blob/main/scripts/fetch-state.py for a Python implementation.

[JonasCir]

Doing this in python looks quite simple and effective. Both of the tools use the same descriptor file to interact with the service. Before including the code, however, we need to reach out to the owner of the repo as no License is specified for the code.

[stephaneghozzi]

Good point... A bit a question of what is done in R and what in Python... But that doesn't really matter if the code is modular :)

[JonasCir]

I think we could run the python script in the very beginning to store the data as CSV and then spin up R. Also I found a way to super conviniently call R from python rpy2, it is really an amazing piece of software. I already used it for SORMAS-Stats, just in case we need it.

[rgieseke]

Good point on a missing license ... i have also (uncommitted) scripts for Landkreise (#23) ...

[rgieseke]

Just added a BSD license (rgieseke/opencoviddata@2f28246)

[stephaneghozzi]

😮 and I was there writing a long email... Thanks! 🙏

[rgieseke]

Cheers! Pushed the county script as well so you have that too. I think i remember there was someone on Twitter who said they have an (unpublished) R-wrapper for SurvStat but these Python scripts to CSV are probably self-contained enough that you can get them to run if you want (or re-do the API calls in your favourite environment).
The state version has run pretty well in the daily GitHub CI action.

[JonasCir]

@rgieseke That's awesome! Thank you so much :)

[stephaneghozzi]

yes... I think I saw the tweet and but couldn't find it afterwards... (Thing is, former colleagues at RKI did develop such solutions, but we could never determine whether it was ok to make them publicly available... a real shame...)

[rgieseke]

Yeah ... i mean technically it is sort of self-documenting (https://tools.rki.de/SurvStat/SurvStatWebService.svc?wsdl) but a lot i had to figure out from looking at the field names in the website ...

[stephaneghozzi]

we'll try and extend it to dimension "Falldefinitionskategorie" and possibly age group and sex, but that shouldn't be a problem given all you've done already

[JonasCir]

If anything is missing, I'm happy to contribute back :D

[JonasCir]

I drop you both a mail :)

[JonasCir]

Thank yo us much :)

[JonasCir]

@rgieseke I sent an email to the email address linked on your GitHub profile

[JonasCir]

Continued in #27