gather-external-docs is broken (again...)

[Confectrician]

And again with have to deal with a refactoring issue.

The gather external docs job (and due to this the complete addon docs build) is completely broken because of
https://github.com/openhab/openhab2-addons/issues/5222 and
https://github.com/openhab/openhab2-addons/pull/5555

Our build process is build up on feature.xml files in many parts
and especially the removed one was used for a big part.
Seems finally the time to rework the whole script has come now.

A ping to @openhab/2-x-add-ons-maintainers for info.
I know docs are a necessary part but not the most beloved one.
I would like to kindly ask you to think of the docs too, when reviewing some bigger changes that are not only related to one binding and will change folder structures.
This way we could have had prepared something maybe.

I am already following the activity in openhab2-addons for a while,
but its not that easy to be up to date all the time, if one is just reading most of the time.

[davidgraeff]

The feature file should not be relevant to docs. You can parse the pom.xml and binding.xml files for addon information, can't you?

[Confectrician]

should not be relevant

Should and shouldn't don't matter here.
Someone used those files in the past for the scripts and now we got a problem.
The scripts were already here as i joined in and i didn't question them so far.

Yes, the pom.xml could be a valid base too.
Anyway we need a bigger rework now and it would have been nice to be able to prepare this.

[J-N-K]

Sorry for that. I‘ll have a look if I can help here.

[Hilbrand]

Where are those scripts located? I did a quick search here but couldn't find them.

[Confectrician]

I will have look tomorrow too.
Maybe this is also a good chance to simplify and refactor the existing buildchain a bit and make the best out of it.

Where are those scripts located? I did a quick search here but couldn't find them.

You can find them in the final branch of this repository.

[davidgraeff]

Wow. Different branches. A rework is definitely necessary IMO.
Why is openhab-docs actually responsible for addon gathering?
That should be part of the website repo, I think.

[Confectrician]

The different branches approach was done on purpose and i definetely want to keep this.

Why is openhab-docs actually responsible for addon gathering?

This is a historically grown structure...
Docs were build by github pages/jenkins before we moved to netlify/vuepress.
Based on that every source had to be placed in this repository.

The final branch was introduced, to make contributing easier.
We have lost many interested people, because they tried to edit addon readmes here, which would then get overriden by the next build.
This was very frustrating especially for first time contributors and only a few invested the work and transferred their work to the correct place of the readme file.

We then separated contents (with those branches) and added the "edit this page links" aside.
The master branch of this repository contains onle the docs contents, that are safely editable from here.

This way, we tried to lower the entry hurdle and bring more people to the docs.
And this is still difficult enough for non programmers who would like to contribute something back to openHAB.

[davidgraeff]

There should be no intermediate, assembly only repository or branch, that's what I mean. Only content.

IMO no build should happen here. The website repo build would gather and assemble everything. And for generating previews we would checkout the website repo in our Travis script and use the very same build procedure (excluding the expensive gather step maybe).

[J-N-K]

@Confectrician I found the groovy-script that processes addons feature-files. It is not so easy to change that, otherwise we would have to keep the list of excluded and included features and the names in different places in sync.

But I have another idea: How often is that script executed? Every time a new snapshot is build? Each build aggregates the features.xml so that it is the same as it was before, it's just nmot part of the repo anymore but still deployed for the distribution. Do you think we can pick it up from there and parse that instead of the source file?

[davidgraeff]

I have also created this openhab/website#186 for an idea

[Confectrician]

Do you think we can pick it up from there and parse that instead of the source file?

Where does it get deployed to?
We could of course add a wget or a clone of the distro repo to the bash script.
for the rest we would just need to adaopt the pom.xml and copy it in the right place.

For Mid/Long term i want to simplify that anyways.
I have simply no groovy knowledge and have to invest a huge amount of time, everytime something breaks.
Anyways this would be a good short term solution to keep the docs up to date for now.

[J-N-K]

https://openhab.jfrog.io/openhab/libs-snapshot/org/openhab/distro/openhab-addons/2.5.0-SNAPSHOT/

The file name changes but can be extracted from maven-metadata.xml

[Confectrician]

Hey @J-N-K sorry for the delay.

This looks good. We can even catche the correct snapshot folder from https://openhab.jfrog.io/openhab/libs-snapshot/org/openhab/distro/openhab-addons/ with the metadata.xml over there.

I will try to get a fix now with a fixed snapshot folder.
I think 2.6.0-Snapshot is far away enough to work on a reliable solution, when the website is already building again.

[Confectrician]

Working again. 🙌

Anyway we should aim for a simplification on mid and long term.

[Confectrician]

Thanks to @J-N-K who gave me some support and his eyes for the jfrog part.