New Documentation for 22.04, repository creation etc.

[flyersa]

Hi all,

before all, YES i am aware we are not in the SCS meetings atm (we just cant in the currently with all the meeting overlaps) and we lack a bit behind on all the updates.

However, we noticed now for a new installation the osism docs refer to Ubuntu 22.04 for the install, refering cookiecutter only as "previously used". So how is someone supposed to create those repos now?

Also we figured out on our last installation where we tried 22.04 (and stil run it) that it doesnt work very well with OVS as example at all. (Dont say use OVN, this brings in alot of other issues too)

Default Ubuntu 22.04 (MaaS) installations have no working security groups and networking functionality without disabling bridge arp and so on and the only workaround is to use sgs in ovs and not in iptables (which isnt bad at all). But without tuning the sysctl settings we could not pass any traffic on any devices on 2 different 22.04 installations.

So im curious if 22.04 is now declared stable and was really tested? our last 22.04 installation is just 3 weeks behind and i doubt something changed alot.

We also emailed the issue during the installation phase for this customer but got not really a reply. Im really curious if really someone else use this for alot of production deployments? we certainly do and we get everytime on a new installation overwhelmed with the new documentation and workflow changes. How are we supposed to upgrade all of this to the new workflow and so on? can we throttle a bit and make proper announcements? We cant force anyone to read hundreds of gitlab commits.

And again, yes WE WILL participate in the calls and i dont blame anyone, but email is also a normal communication way that doesnt end in endless meetings. Or a Mailinglist perhaps makes sense (or multiple? osism-announce? osism-discussion etc. ?)

Dont get me wrong, i really like the progress and it is moving in the right direction fast paced. But its a bit too fast paced imho. It is a great product and you do a really good job on all of this. I also blame us not taking part in the meetings, i guess i need to force us todo that somehow.

But production wise in the current state with all the rapid changes in workflow, documentation and so on its rather something i would say NO sadly to clients.

[berendt]

However, we noticed now for a new installation the osism docs refer to Ubuntu 22.04 for the install, refering cookiecutter only as "previously used". So how is someone supposed to create those repos now?

Ubuntu 22.04 is the new default for the upcoming 5.0.0 release.

Default Ubuntu 22.04 (MaaS) installations have no working security groups and networking functionality without disabling bridge arp and so on and the only workaround is to use sgs in ovs and not in iptables (which isnt bad at all). But without tuning the sysctl settings we could not pass any traffic on any devices on 2 different 22.04 installations.

We do not support MAAS. Currently there is osism/node-image for provisioning new nodes.

We also emailed the issue during the installation phase for this customer but got not really a reply.

Emails to personalized accounts in the run-up to Christmas is not a good way to get support. Especially if you don't even ask there if you don't get an answer. We have [email protected] as well as this repository for that.

can we throttle a bit and make proper announcements?

Annoucements take place weekly in the SCS Weekly and in the team meetings. There is also release.osism.tech where we summarise things. If you do not have time to attend the SCS meetings, there are notes for all meetings. And there are also etherpads for all meetings so that you can also participate and take part directly. Without joining a meeting in person.

We cant force anyone to read hundreds of gitlab commits.

If you use latest you have to read it. Otherwise use the stable releases. Not much changes there between the releases. 4.2.0 is coming in the next few days.

But production wise in the current state with all the rapid changes in workflow, documentation and so on it's rather something i would say NO sadly to clients.

There is the stable release for productive environments. The many changes are on the main branches. Not in the stable releases. Main is not intended to be used with customers. I agree with that.

And again, yes WE WILL participate in the calls and i dont blame anyone, but email is also a normal communication way that doesnt end in endless meetings. Or a Mailinglist perhaps makes sense (or multiple? osism-announce? osism-discussion etc. ?

We use our channel on Matrix as well as release.osism.tech. in SCS it is currently a meeting / week (Wednesday, 10:00) that takes 1h. We can gladly provide another mailing list. So far there has not been the request for it.

If there are problems, please start opening issues and share them. What is not working with Ubuntu 22.04 in combination with OVS? What does not work with OVN? If the documentation is not sufficient, then expand and/or open Issue to the places.

[garloff]

@flyersa: Care to work out what is broken with security groups and iptables in 22.04 with OVS?
So we can double-check that this is handled?
Opening a separate issue is probably best to separate it from the more generic discussion how SCS/OSISM may or may not be too-fast paced and how the communication may or may not work ...

There is a lot happening in SCS and it may be hard to follow everything.
But as Christian says: We have stable releases for a reason. And we do the Thu afternoon community call, where you get an overview over all ongoing activities in less than an hour per week. You then know where to do deep dives. There's minutes for all team calls and almost all SIGs and we have good discipline to link the related docs (typically github issues), so you can dig in to whatever topics you find there that are relevant to you.
To be honest, we have invested a lot of thinking into making it easier to follow progress in our community especially for people that can not invest two-digit amounts of hours per week. But it may be very hard if you have less than an hour per week and I am not sure we can fix that.

[flyersa]

I will open a case with our u 22.04 findings. And yes i totaly understand the part with the meetings,as i said i blame us for this and we try to fix this. But i stil think its not good todo this kind of heavy changes every release to end users. We stil follow up to some extend and are kind of involved, other user may not and ma find this not productive. But guess we can discuss this outside of here. Kurt Garloff ***@***.***> schrieb am Mo., 23. Jan. 2023, 18:31:

…

@flyersa <https://github.com/flyersa>: Care to work out what is broken with security groups and iptables in 22.04 with OVS? So we can double-check that this is handled? Opening a separate issue is probably best to separate it from the more generic discussion how SCS/OSISM may or may not be too-fast paced and how the communication may or may not work ... There is a lot happening in SCS and it may be hard to follow everything. But as Christian says: We have stable releases for a reason. And we do the Thu afternoon community call, where you get an overview over all ongoing activities in less than an hour per week. You then know where to do deep dives. There's minutes for all team calls and almost all SIGs and we have good discipline to link the related docs (typically github issues), so you can dig in to whatever topics you find there that are relevant to you. To be honest, we have invested a lot of thinking into making it easier to follow progress in our community especially for people that can not invest two-digit amounts of hours per week. But it may be very hard if you have less than an hour per week and I am not sure we can fix that. — Reply to this email directly, view it on GitHub <#415 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABBJ7774GOU6NLRQ3EAUZ43WT26ABANCNFSM6AAAAAAUD74J3E> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[berendt]

It would be nice if this issue would still come so that we can fix the problems in Ubuntu 22.04.

[berendt]

I have migrated this into a discussion. I think this should be discussed here and not somewhere else.

[flyersa]

to pick this up again

"We do not support MAAS. Currently there is osism/node-image for provisioning new nodes."

You SHOULD support MAAS, your major installations use MAAS (Plusserver, Wavestack, all of our clients which are like 15 installations) and your node-image is far away from being productive usable. It is no magic, MAAS will not deploy something magic, but a base ubuntu image. If you say you do not support MAAS then almost all osism installations are basically not supported.

Also there is no need to reinvent the wheel for deployment tools, MAAS, Cobbler etc. are there for a reason and matured enough and do not need replacement, and imho you should adopt to that instead of adding something new again.

[berendt]

We use Ironic for bare-metal provisioning and that is the primary solution we have for this. Since years.

osism/node-image is only used for the provision of the manager node and, if necessary, the control nodes. The osism/node-image simply makes a minimal Ubuntu installation, nothing more.

However, there is nothing to be said against deploying the initial systems with MAAS. If this performs a minimal Ubuntu installation, that's fine.

[flyersa]

and still people tend to use MAAS instead of your ironic solution. Also your node images have partially some weird LVM Settings as example that not everyone likes. I remember myself a session where you tried to show us how that provisioning works and nothing did work at all. I think operators should be free to to pick whatever they want to base provisioning there own equipment. You need Ubuntu XY, thats all. It may not be 99%, but the core heavy users are using MAAS and im sure it will stay that way for a while.

[berendt]

The osism/node-image is one option to bootstrap the nodes. You can also do the installation by hand or with MAAS or with Cobbler. It's your choice. There is no default and no lock on a particular solution.

[frosty-geek]

I can confirm that plusserver has no short term plans to switch hardware deployments away from MAAS.

My guess is that @berendt meant to say it's out-of-scope for OSISM, but I do agree if 99% of the OSISM/SCS Installations use MAAS why not get it in-scope?

[berendt]

Because we have Ironic for it. Someone can document how to provide the initial nodes with MAAS. But for fleet management, Ironic is the component of choice.

[berendt]

Opened #427 for discussion of MAAS in the context of the SCS project. The 99% is not quite correct ;)

[fkr]

@flyersa In this weeks IaaS call in SCS we've discussed MAAS. @berendt documented the outcome in #427 - furthermore it is visible in our minutes. I very much would like to invite you again to join our calls. I understood from previous comments that available time is hard constraint on your side - we accommodate that in the SCS project with the community call on thursdays at 15:05 CET which allows to get a good overview in just 45 minutes.
Feel invited to get back to me, I reached out numerous time in order to better understand your (and your customers) requirements .

[flyersa]

Hi Felix,

i did read the minutes. Thanks alot. I will attend those meetings starting in march. Where did you try to reach out? I am happy to schedule a call. Can you drop me a mail?

[fkr]

I will attend those meetings starting in march.

cool! Very much loooking forward.

Where did you try to reach out? I am happy to schedule a call. Can you drop me a mail?

I called your cell phone numerous times and over the course of the last few months I wrote several mails inviting you to our Lean SCS Operator Coffee.
I forwarded you the past mails again.

[flyersa]

hmm strange, i usually answer my calls. Found the mail now. gmail sorted it into spam for some reason