Proactive use of hashing

[jwhite242]

Just continuing the conversation on the possibilities for being more proactive on hashing workspace paths and script file names. Would be great for the users if the option could be flipped on automatically to avoid finding the issues after running a large study. How to notify the users of it is an interesting problem given the background construction of everything; could the front end could do a pass on finding max param combo lengths before handing off everything to conductor?

[FrankD412]

I meant to get to this sooner -- I like the idea of being proactive about hashing, but I think it's a bad idea to force it on the user. It's Maestro's front end that constructs the study and sets the hashing mode, with the Study class carrying that setting over to the

The following flow is what I could see working assuming there's a way to check the right properties in the Maestro front end:

1. Check the number of parameters, maybe the length of the names, and store that.
2. If available, get the OS path limit (I wonder if os.path has something for this).
3. If you're within some threshold, there's a likelihood you'll go over -- have Maestro print a message that they're likely to run over the OS specified path length limit and ask them if they'd like to hash parameterized workspaces. If no, either abort the study or let it continue but warn the user that it has a high probability to run into failures.
4. If yes, go ahead and set hashing in the instance of the Study class -- everything proceeds as normal after this.

How does that sound, @jwhite242?

Folks who might want to offer some input: @jsemler @crkrenn @daub1

[crkrenn]

I think a warning if you are going over is a good idea. https://www.tutorialspoint.com/How-to-get-the-maximum-file-name-length-limit-using-Python https://docs.python.org/2/library/statvfs.html And, is there a file name limit or path name limit at llnl? A solution may be to just use 2 or three directories instead of 1. E.g. var-Val/var-Val/var-Val vs var-Val_var-Val_var-Val

…

--- Sent from Workspace ONE Boxer<https://whatisworkspaceone.com/boxer> On September 20, 2020 at 11:01:52 AM PDT, Francesco Di Natale <[email protected]> wrote: I meant to get to this sooner -- I like the idea of being proactive about hashing, but I think it's a bad idea to force it on the user. It's Maestro's front end that constructs the study and sets the hashing mode, with the Study class carrying that setting over to the The following flow is what I could see working assuming there's a way to check the right properties in the Maestro front end: 1. Check the number of parameters, maybe the length of the names, and store that. 2. If available, get the OS path limit (I wonder if os.path has something for this). 3. If you're within some threshold, there's a likelihood you'll go over -- have Maestro print a message that they're likely to run over the OS specified path length limit and ask them if they'd like to hash parameterized workspaces. If no, either abort the study or let it continue but warn the user that it has a high probability to run into failures. 4. If yes, go ahead and set hashing in the instance of the Study class -- everything proceeds as normal after this. How does that sound, @jwhite242<https://github.com/jwhite242>? Folks who might want to offer some input: @jsemler<https://github.com/jsemler> @crkrenn<https://github.com/crkrenn> @daub1<https://github.com/daub1> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#320 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABOJ5V6TXFVTZ2QD6BWQQJDSGY7QTANCNFSM4RLYSSPA>.