Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2024 meetings #188

Open
oliver-sanders opened this issue Feb 19, 2024 · 5 comments
Open

2024 meetings #188

oliver-sanders opened this issue Feb 19, 2024 · 5 comments

Comments

@oliver-sanders
Copy link
Member

Issue meeting notes, follows on from #163

@oliver-sanders
Copy link
Member Author

oliver-sanders commented Feb 19, 2024

2024-02-19T20:00Z

Agenda:


quick summary notes

Cylc-8.3.0

  • aim for March, this is pretty urgent
  • triage: drop all non-essential issues back to 8.4 or 8.3.x
  • cylc set is essential, but we can push cylc remove and skip-mode back
  • the remaining optional outputs work is less critical, since we took expire out of the mix

Cylc-8.2.5

  • UK team still think this is worth releasing, before 8.3.0 gets out
  • (NIWA side: I don't mind either way, no pressing 8.2.4 bugs are affecting us that I'm aware of)
  • OS to review 8.2.x and 8.2.5 milestones tomorrow

Migration:

  • high support workload at the moment, making it hard to get development time

Open questions (link above)

  • we all need to put a bit of time into considering those, they're piling up

Multi-user gscan

  • this is viewed as a requirement at BOM, due to a significant number of production role accounts
  • in principle the UI can manage multiple subscriptions to multiple (user-specific) UI Servers, and users would appear at the top level of the gscan hierarchy, but to date we've had to concentrate on efficiency and stability of the single-user subscription
  • to do multi-user properly there's quite a lot to consider, e.g. what determines which other users you can see, load on the UI for many users, etc.
    -- it may be an option for BOM to do this via a local UI extension (even a separate "gscan view" perhaps); but note that development and code review resources are currently very limited, so it might have to be maintained separately for a good while with no guarantee to be merged as-is

AWS Batch

  • some MO work going is going on now with Cylc on AWS
  • NIWA has a dev 1-day per week for a while, considering cloud platform support in Cylc (just starting)

@oliver-sanders
Copy link
Member Author

oliver-sanders commented Apr 11, 2024

Next Meeting

Agenda:

@hjoliver
Copy link
Member

hjoliver commented Jun 5, 2024

5 June Meeting

Agenda:

Primarily: 8.3.0 release. The previous agenda still applies!

  • excuses: HO workflow-state PR (and N*w*-state fubar)
  • (probably get OS command validation PR in too?)

Other: ...

@oliver-sanders
Copy link
Member Author

oliver-sanders commented Jul 12, 2024

2024-11-04

Agenda:

  • 8.3.6 progress
  • Review roadmap.
    • Anything to add or remove?
  • (quick) Wrapper script sets PATH when running cylc hub which breaks version selection for cylc play in UI
    • We need configurable-http-proxy to be in the $PATH for cylc hub.
    • We are currently putting the whole UIS env bin directory in the $PATH which bypasses the wrapper causing problems.
    • The best solution I can think of (under the current approach) is to require a wrapper script for configurable-http-proxy to match the ones we already have for cylc, rose and isodatetime.
    • These wrapper scripts are an installation pain, but without changing approach (topic for another day?), this feels like the way forward for now.
    • Question: Agreed? Or any other ideas?
    • Action: New suggestion on the issue, can resolve offline.
  • (quick) skip mode: consider using special "skip" task messages
    • If an output was completed in skip mode, save the message as satisfied by skip mode rather than, e.g. succeeded.
    • Question: Any objections?
    • Answer: No.
  • (quick) data-store: incorrect information for n>0 xtriggers (agreement on issue)
    • Xtriggers are not evaluated until at least one pre-requisite has been satisfied.
    • But xtriggers will show up e.g. in the GUI whether they have been evaluated or not.
    • This can make it look like an xtrigger (e.g. wall_clock) is unsatisfied in the GUI when it is actually unevaluated which can confuse users into thinking xtriggers aren't working.
    • Question: What should we do? (see issue for options)
  • Support EmPy v4?
    • Cylc's EmPy support is currently broken, it's probably an easy fix (PR up), but do we want to continue support?
    • EmPy:
      • Has a single maintainer.
      • Is a closed-repository project (i.e. effectively closed source, you can acquire their releases, but you can't access the source VCS, raise a PR or anything like that).
      • Has an infrequent release cycle and a history of year-long gaps between releases.
      • Does not maintain their conda-forge feedstock.
    • Question: Given we currently do not have any EmPy users, do we want to continue support for this system?
    • Answer: No.
  • Housekeep broadcasts on task completion
    • How do we want broadcast housekeeping to work at Cylc8?
    • We used to clear broadcasts when the cycle they applied to "closed".
    • This means you will get a different result if you re-trigger a historical task which doesn't really make sense.
    • We could consider a different housekeep strategy (e.g. see discussion on issue) or opt not to housekeep broadcasts at all.
    • Ensure we consider the implications of whatever we decide for reflow use cases.
    • Question: [How] should we housekeep broadcasts?
    • Answer: Don't auto-expire broadcasts, maintain the data-store window worth of broadcasts in memory.
  • CYLC_SHARE_CYCLE_DIR
    • Some concern on the FS impact of per-cycle directories.
    • Is this legitimate? We are already doing this for the log directories.
      • (HO: not very legit; we can provide them, but they don't have to be used, and can be hand-rolled if not provided)
    • Should we develop the housekeeping tools for this before implementing?
    • Question: Should we press ahead with this, or stick it on ice?
  • How to prevent a future sub-graph from running
    • This is a new use case made possible by SoD allowing downstream tasks to be removed implicitly by removing the required upstream(s).
    • At Cylc 7, only explicit task removal was possible (although users usually reset tasks to succeeded to skip them).
    • (HO: the above two points are only partly true. In Cylc 7 you have to explicity remove the first pre-spawned task of an unused branch but anything downstream of that would in fact be "implicitly removed")
    • What are the use cases for implicit task removal (as opposed to removing/skipping by task/family/cycle)?
      • (HO: primarily the classic non-clock task expire use case: external circumstances say I don't need to run task-x and the sub-graph downstream of it)
    • How should downstream impacts be handled (to avoid stall)?
    • (HO: superseded by Handling future final-incomplete tasks cylc-flow#6383)
  • BoM corrupted DB issue
    • How are these databases getting corrupted? Need information from BoM.
    • Action: Close this issue.

@hjoliver
Copy link
Member

hjoliver commented Dec 1, 2024

5/6 Dec 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants