More focused software engineering for scientific communities

[selsayed]

Software in the scientific communities is seldomly written by software engineers nor by veteran coders. Large chunks are created by individuals with little experience of the processes and recommendations of creating healthy and maintainable software. While tools such as CI are a good method to invoke some code standards, the background and goals of most scientists writing code results in them attempting to circumvent what is considered by them to be an annoyance.

One considerably more reasonable approach is to educate. Most important here is to focus on solidifying the reasons for specific approaches to coding that only show results much later in the coding process. As it is not possible to educate every scientific coder, the main difficulty is to wisely allocate resources. A bigger challenge yet for those embarking on this educational task, is the divergent goals. While software engineers tend to focus on Readability and Maintainability of code, scientists are much more focused on shorter write times and faster results. This can only be resolved by the software engineers better understanding, not of the scientific field specifically, but the goals and the approaches to science that the scientific communities adopt.

In this breakout session, we would like to discuss with a wide audience the methods by which software engineers can better approach the scientific community and further their understanding of how the scientific process within operates. Furthermore, we would like to discuss, how we can better build centers supporting the sciences and how young scientists can be made aware of good and healthy coding skills.

Our background is in the support of scientific communities using HPC systems. We have experienced first hand how code quality stifles porting to newer technologies and optimizing for better performance. Additionally, we have experienced the gap between scientific communities' strive for larger simulations and better results, while being uninterested in investing time and effort in improving code quality. This has both effected the scientific communities as well as our own research into HPC (such as unused prototypes due to requiring high porting efforts). We have been cooping by blaming scientific code quality, creating simplified examples to run on prototypes as proof of concept and in limited cases porting codes ourselves. However, we identified the divergent goals, the misunderstanding of intent and the funding structures as primary causes. We wish to discuss how we can address these issues at their root.

[pancetta]

Hi @selsayed, thanks for the proposal! So, this BOS will be organized as a discussion, yes? Do you have collaborators in mind? How long do you think the BOS should be? We plan with slots of 90 minutes with the default length of one slot.

[selsayed]

Hi,
Yes, my thought was to make it into a discussion. We could have a few slides to prime the topic, giving examples or historical issues. But I would limit those to not more than 10-15min.
My current collaborator on the subject is @jayeshbadwaik. He helped as well with formulating the above issue. But I think we can find a few more colleagues that would be interested to discuss this topic.
90min sounds like a good amount of time. I suspect, depending on the participants and their experience we might have material for a much longer discussions. But as a priming session, I think the 90 min should be a good start to get to know more with the same or similar issues.

[HeidiSeibold]

I added a label to this break-out. Can you check if you feel it is appropriate and change it if not? Let me know if you have any questions.

[selsayed]

Thanks. Yes, I believe it is appropriate.

[HeidiSeibold]

So what I need from you to be able to make a decision on this breakout are the following infos:

Who could be people interested in collaborating on this?

(feel free to tag them with their GitHub username if they have one)

Sounds like you (@mmesiti), @chillenzer and @chryswoods are obvious choices so far. Anybody else?

How much time do you need for this?

(90 minutes or multiples thereof)

Abstract

(Can be short)

[HeidiSeibold]

Hi @selsayed can you please respond by Tuesday morning, thanks 😃

[selsayed]

Hi @HeidiSeibold,
for now I've collaborated this breakout with @jayeshbadwaik and gotten some corrections from Carolin Penke. Most definitely @mmesiti, @chillenzer and @chryswoods are welcome to join and even further refine the topic.

Abstract:

The major challenge for introducing and expanding the software engineering practices in the scientific communities is what can be perceived as divergent goals. While software engineering tends to focus on for example Readability and Maintainability of code, scientists are much more focused on shorter write times and faster results. The very processes that are placed to improve on the software engineering practices are thus occasionally circumvented or at the least considered a nuisance. Scientists are also regularly asked to place higher priority on code rewrite for adopting better coding methods, newer packages or making room for new platform support, which from their perspective would not lead to scientific results in the short term.
The reasoning behind adoption of RSE is obvious to those with long experience in the field. We need to clarify what scientists can gain and how we can adapt software engineering methodology to accommodate their goals.
To that end some suggested talking points for this session could include:

1. How to clarify the cost benefit of applying software engineering methods to scientific communities? As well as funding agencies allowing for adding expert SE to a project.
2. How to quickly introduce young scientists to software engineering without slowing down their master or doctoral progress?
3. How to address the missing Dev stage of code development, where much of the written code goes from research directly into operations? DevOps is a luxury, scientific communities do ResOps.
4. How can Co-Design of middleware and next gen prototypes become more beneficial to scientific communities in the short run?

In short, part of RSE focuses on getting researchers to adopt software engineering. The intention from the suggested topic is to reverse it by asking how software engineering can further understand and adapt to researchers needs beyond simple tooling.

[pancetta]

Hi! I just added the "Accepted" label to this BOS. Welcome on board!
https://un-derse23.sciencesconf.org/program

[led02]

Will be done by a colleague of him.

[selsayed]

Will be done by a colleague of him.

@jayeshbadwaik will take over.
Please feel free to forward me and him any additional notes or talking points that we can include in our preparations.

[pancetta]

Hi all, the unconference is only 3 weeks away now! On day 1 there will be a breakout blitz where all session organizers should advertise their sessions. 1 minute, 1 slide, let people know what you intend to do. Please prepare this slide in advance and add it right here (PDF please), by September 20.

[pancetta]

ping

[selsayed]

ping

Hi, we are in the process of preparing the session. Anything we should be aware of?

[pancetta]

I just need the blitz for now, see above.

[selsayed]

🤦‍♂️ Somehow this missed my todos. Apologize, we will add it here by end of day.

[jbadwaik]

Adapting Software Engineering for Scientists

[pancetta]

Here is the main hub for taking notes: https://pad.gwdg.de/FkFJTslFQhq-UF3Es6q4rw#

[pancetta]

Have fun with the session(s)! Please add the pad you're using also here for people to see what you did.

If possible, please prepare a 1 minute wrap up of your session for the farewell session on Thursday afternoon! What did you do in the session, how would you like to continue, how can people contribute after the unconference etc. We'll go through the blitz slides again one by one as in the blitz session.