[#432] Implement CLI iox2-config #468

brosier01 · 2024-10-14T21:53:07Z

Notes for Reviewer

Create a new CLI for iceoryx2 : iox2-config
iox2-config can show the configuration currently in use and generate a new configuration file at the default location iceoryx2 is looking for.
Remove the print_system_configuration() function in iceoryx2-bb/posix/src/system_configuration.rs file and move it into the CLI iox2-config

Pre-Review Checklist for the PR Author

Checklist for the PR Reviewer

Commits are properly organized and messages are according to the guideline
~~Unit tests have been written for new behavior~~
Public API is documented
PR title describes the changes

Post-review Checklist for the PR Author

All open points are addressed and tracked via issues

References

Relates to #432

elfenpiff · 2024-10-14T22:35:17Z

@brosier01 don't close it - it looked good!

orecham

In general looks pretty good, just one real blocker re: config location and also you'll need to address the static analysis CI findings.

Thanks for getting involved and welcome!

doc/release-notes/iceoryx2-unreleased.md

orecham · 2024-10-14T22:47:15Z

doc/release-notes/iceoryx2-unreleased.md

-   // old
-   let fuu = hello().is_it_me_you_re_looking_for()
+```bash
+   cargo run --bin iox2-config show


Suggested change

cargo run --bin iox2-config show

iox2 config show

doc/release-notes/iceoryx2-unreleased.md

iceoryx2-cli/iox2-config/src/commands.rs

orecham · 2024-10-14T23:06:31Z

iceoryx2-cli/iox2-config/src/commands.rs

+}
+
+pub fn show() -> Result<()> {
+    print_system_configuration();


Does this print the default in-built config if there is no config file being loaded from file?

print_system_configuration() displays the whole system configuration with all limits, features and details to the console. It does not print the built-in config but the system configuration. print_system_configuration() function is independent of the config file.

@elfenpiff Should the defaults be added to the output of iox2 config show, under a different sub-heading ? I would think yes, but what did you have in mind?

@elfenpiff Will need your feedback on this. What was your intention for this command?

@elfenpiff I implemented this to provide me with some debug output of the configuration of the POSIX system, meaning the POSIX configuration that states like how many semaphores can be created or what the maximum value of a semaphore could be.

Windows, Mac Os, FreeBSD, and Linux have different values here, and in the early days of iceoryx2, it was very helpful when analyzing bugs. Now, everything is abstracted away with the iceoryx2_pal and iceoryx2_cal layer but this still makes sense for mission-critical environments where we define the minimum requirements we have for the system (somewhere) and then this would print out the systems requirement and would also highlight the things which would violate our requirements.

A practical example is the iceoryx2_bb_threadsafe::trigger_queue. It uses two semaphores to trigger the producer/consumer when changes happen. But when the semaphores max value is 1024 it means the trigger queue can have a capacity of at most 1024 elements, otherwise the underlying semaphore would overflow and break the queue.

Another example is a gateway. When this subscribes to thousands of services it must be able to memory map thousands of shared memories and also hold thousands of fd's. By default the max number of fd's is 1024 so here it may makes sense to print a warning on gateway startup when the config does not fit.

@elfenpiff Apologies, what is your advice here ?

@orecham My apologies!

Should the defaults be added to the output of iox2 config show, under a different sub-heading ?

Yes, this would make sense.

@elfenpiff My apologies, did you say I had to set up a new subtitle? Like “iox2 config show default” which shows the default configuration? Am I right ?

@brosier01 Yeah, if you could additionally print the contents of the config file under another subtitle, this would be perfect.

Let us know if you hit problems or it turns out more difficult than it seems.

orecham

I tested the CLI on the different platforms. A few more things to resolve before it can be merged.

orecham · 2024-10-16T13:38:06Z

iceoryx2-cli/iox2-config/src/commands.rs

+}
+
+pub fn generate() -> Result<()> {
+    let config_dir = dirs::config_dir().unwrap().join("iceoryx2/");


This results in a directory C:\Users\orecham\AppData\Roaming\iceoryx2/config.toml on Windows (the last slash should be a backslash.) I think you can just remove the / here and rely on .join() adding the proper separators.

On a side note. Windows also supports the forward slash as path separator and if you need to pass a path as string I would recommend to use C:/Windows/foo or you might end up with something like C:\\\\Windows\\\\foo, depending on where the string is defined and how many layers it needs to pass until it reaches the actual Windows call.

orecham · 2024-10-16T13:42:38Z

iceoryx2-cli/iox2-config/src/commands.rs

+
+/// Prints the whole system configuration with all limits, features and details to the console.
+pub fn print_system_configuration() {
+    const HEADER_COLOR: &str = "\x1b[4;92m";


In PowerShell on Windows, these codes don't seem to be parsed correctly:

PS C:\Users\orecham> iox2 config show ←[4;92mposix system configuration←[0m ←[4;92msystem info←[0m ←[0;37mPosixVersion ←[0m ←[0;94m200809←[0m ←[0;37mPageSize ←[0m ←[0;94m4096←[0m ←[0;37mNumberOfClockTicksPerSecond ←[0m ←[0;94m0←[0m ←[0;37mNumberOfCpuCores ←[0m ←[0;94m16←[0m ←[4;92mlimits←[0m ←[0;37mMaxLengthOfHostname ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxLengthOfLoginName ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSupplementaryGroupIds ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfOpenFiles ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSimultaneousProcessesPerUser ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxLengthOfArguments ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfOpenStreams ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSymbolicLinks ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxLengthOfTerminalDeviceName ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfBytesInATimezone ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSemaphores ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxSemaphoreValue ←[0m ←[0;94m2147483646←[0m ←[0;37mMaxNumberOfOpenMessageQueues ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxMessageQueuePriority ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfThreads ←[0m ←[0;94m1024←[0m ←[0;37mMaxSizeOfPasswordBuffer ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxSizeOfGroupBuffer ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxPathLength ←[0m ←[0;94m260←[0m ←[0;37mMaxFileNameLength ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxUnixDomainSocketNameLength ←[0m ←[0;94m108←[0m ←[0;37mMinStackSizeOfThread ←[0m ←[0;94m1048576←[0m ←[4;92moptions←[0m ←[0;90mAdvisoryInfo ←[0m ←[0;90mfalse←[0m ←[0;90mCpuTime ←[0m ←[0;90mfalse←[0m ←[0;90mFsync ←[0m ←[0;90mfalse←[0m ←[0;90mIpv6 ←[0m ←[0;90mfalse←[0m ←[0;90mMemLock ←[0m ←[0;90mfalse←[0m ←[0;90mMemLockRange ←[0m ←[0;90mfalse←[0m ←[0;90mMessagePassing ←[0m ←[0;90mfalse←[0m ←[0;90mPrioritizedIo ←[0m ←[0;90mfalse←[0m ←[0;90mPriorityScheduling ←[0m ←[0;90mfalse←[0m ←[0;90mRegularExpressions ←[0m ←[0;90mfalse←[0m ←[0;90mSpawn ←[0m ←[0;90mfalse←[0m ←[0;90mProcessSporadicServer ←[0m ←[0;90mfalse←[0m ←[0;90mSynchronizedIo ←[0m ←[0;90mfalse←[0m ←[0;90mThreadStackAddress ←[0m ←[0;90mfalse←[0m ←[0;90mThreadStackSize ←[0m ←[0;90mfalse←[0m ←[0;90mThreadCpuTimeClock ←[0m ←[0;90mfalse←[0m ←[0;90mThreadExecutionScheduling ←[0m ←[0;90mfalse←[0m ←[0;90mThreadProcessSharedSynchronization ←[0m ←[0;90mfalse←[0m ←[0;90mMutexPriorityInheritance ←[0m ←[0;90mfalse←[0m ←[0;90mMutexPriorityProtection ←[0m ←[0;90mfalse←[0m ←[0;90mRobustMutexPriorityInhertiance ←[0m ←[0;90mfalse←[0m ←[0;90mRobustMutexPriorityProtection ←[0m ←[0;90mfalse←[0m ←[0;90mThreadSporadicServer ←[0m ←[0;90mfalse←[0m ←[0;90mTrace ←[0m ←[0;90mfalse←[0m ←[0;90mTraceEventFilter ←[0m ←[0;90mfalse←[0m ←[0;90mTraceInherit ←[0m ←[0;90mfalse←[0m ←[0;90mTraceLog ←[0m ←[0;90mfalse←[0m ←[0;90mTypedMemoryObjects ←[0m ←[0;90mfalse←[0m ←[4;92mfeatures←[0m ←[0;37mBarriers ←[0m ←[0;94mtrue←[0m ←[0;90mAsynchronousIo ←[0m ←[0;90mfalse←[0m ←[0;90mClockSelection ←[0m ←[0;90mfalse←[0m ←[0;90mJobControl ←[0m ←[0;90mfalse←[0m ←[0;37mMappedFiles ←[0m ←[0;94mtrue←[0m ←[0;90mMemoryProtection ←[0m ←[0;90mfalse←[0m ←[0;90mMonotonicClock ←[0m ←[0;90mfalse←[0m ←[0;90mRawSockets ←[0m ←[0;90mfalse←[0m ←[0;37mReaderWriterLocks ←[0m ←[0;94mtrue←[0m ←[0;90mRealtimeSignals ←[0m ←[0;90mfalse←[0m ←[0;90mSavedUserAndGroupIds ←[0m ←[0;90mfalse←[0m ←[0;37mSemaphores ←[0m ←[0;94mtrue←[0m ←[0;37mSharedMemoryObjects ←[0m ←[0;94mtrue←[0m ←[0;90mShell ←[0m ←[0;90mfalse←[0m ←[0;37mSpinLocks ←[0m ←[0;94mtrue←[0m ←[0;37mThreadSafeFunctions ←[0m ←[0;94mtrue←[0m ←[0;90mThreads ←[0m ←[0;90mfalse←[0m ←[0;37mTimeouts ←[0m ←[0;94mtrue←[0m ←[0;37mTimers ←[0m ←[0;94mtrue←[0m ←[0;90mOpenRealtimeOptionGroup ←[0m ←[0;90mfalse←[0m ←[0;90mOpenRealtimeThreadsOptionGroup ←[0m ←[0;90mfalse←[0m ←[4;92mprocess resource limits←[0m ←[0;37mMaxCoreFileSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxConsumableCpuTime ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxDataSegmentSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxFileSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxNumberOfOpenFileDescriptors ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxStackSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxSizeOfTotalMemory ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m

A crate like colored, like used in the rest of the CLI, should be used for cross-platform coloring.

I know this is not your code, but this would need to be updated for UX purposes on Windows.

iceoryx2-cli/iox2-config/src/commands.rs

orecham · 2024-10-16T13:57:05Z

iceoryx2-cli/iox2-config/src/commands.rs

+use std::io::Write;
+
+/// Prints the whole system configuration with all limits, features and details to the console.
+pub fn print_system_configuration() {


This function has an error on macOS:

0 [F] "ProcessResourceLimit::soft_limit" | This should never happen! Unable to acquire soft limit for MaxSizeOfTotalMemory due to an unknown error (errno { name = "EINVAL", value = 22, details = "Invalid argument" }).

We should make it possible for the CI to catch stuff like this. Could you add a test that executes this function?

As for the reason, it's not immediately obvious to me. For this PR, one workaround could be to update the code to catch errors from the calls it uses to get the information it prints. The function could then continue even if a call has an error, and print a message to the user explaining which information was not retrievable.

cc @elfenpiff

FYI the bug is probably due to: #473

Not for this PR though - handling the error as described above would be sufficient.

iceoryx2-cli/iox2-config/src/commands.rs

orecham

The generated configs are now being used and there is a confirmation message for overwriting a pre-existing generated config - nice.

Some more things left to address, see unresolved threads.

orecham · 2024-10-24T16:04:32Z

iceoryx2/src/config.rs

-                &FilePath::new_unchecked(DEFAULT_CONFIG_FILE)
+                &FilePath::new_unchecked(
+                    dirs::config_dir()
+                        .unwrap()


Won't this panic if the directory doesn't exist. The None case should be handled explicitly.

iceoryx2/src/config.rs

orecham · 2024-10-24T16:15:07Z

iceoryx2/src/config.rs

@@ -69,22 +69,20 @@
 //! # }
 //! ```

+use dirs;


Rather than using this directly, an abstraction should be implemented.

This allows for swapping out the implementation and will make it easier to isolate for safety certification purposes. I haven't looked at the implementation of dirs closely, but it might not be suitable for certification e.g. if it uses heap.

The cleanest way would probably be to create a "concept" in the concept abstraction layer (iceoryx2-cal) for configuration storage, then add one implementation that uses dirs.

This will allow for:

Swapping out an implementation to a safety-certifiable version

Implementation of the concept for arbitrary environments that don't have a file system

cc. @elfenpiff @elBoberido

orecham · 2024-10-24T16:17:23Z

iceoryx2-cli/iox2-config/src/commands.rs

+}
+
+pub fn show() -> Result<()> {
+    print_system_configuration();


@elfenpiff Will need your feedback on this. What was your intention for this command?

iceoryx2-cli/iox2-config/src/commands.rs

orecham · 2024-10-24T16:30:33Z

iceoryx2/src/config.rs

 use iceoryx2_bb_container::semantic_string::SemanticString;
 use iceoryx2_bb_elementary::lazy_singleton::*;
 use iceoryx2_bb_posix::{file::FileBuilder, shared_memory::AccessMode};
 use iceoryx2_bb_system_types::file_name::FileName;
 use iceoryx2_bb_system_types::file_path::FilePath;
 use iceoryx2_bb_system_types::path::Path;
 use serde::{Deserialize, Serialize};
-use std::time::Duration;
+use std::{os::unix::ffi::OsStrExt, time::Duration};


This will not work on Windows (unix-only).

Hi @orecham I fixed my code by adding a new crate abastraction layer to get the default directory of the configuration file. And I've added a new sub-command: now I can display the system configuration and show the current iceoryx2 configuration.

brosier01 · 2024-11-19T21:13:34Z

Hi @orecham @elfenpiff, is there any way to run CI locally?

elfenpiff · 2024-11-20T06:27:27Z

@brosier01 Yes there is. You can use the ./internal/scripts/iceoryx2_env.sh to enter the right docker container, depending on what target fails. The syntax is ./internal/scripts/iceoryx2_env.sh enter ubuntu:22.04 to start the ubuntu docker container for instance.

When you entered it you can run the usual cargo commands. Under .github/workflows/build-test.yml you find the test definitions. It is a bit hands on but we did not yet have time to add individual build & test scripts to ./internal/scripts/.

Co-authored-by: orecham <[email protected]>

…file

…rm coloring + implementation of user confirmation required before deleting current configuration file

…file with the standard configuration path

…the configuration file + add a subcommand to display the contents of the configuration file

brosier01 · 2024-11-26T18:57:36Z

Hi @orecham, I think I've implemented all the elements described in the original problem. Do you see any other features I need to work on ?

orecham · 2024-11-27T12:36:22Z

Hi @orecham, I think I've implemented all the elements described in the original problem. Do you see any other features I need to work on ?

@brosier01 I will be able to review sometime today or tomorrow, apologies for the delay.

brosier01 closed this Oct 14, 2024

brosier01 deleted the iox2-432-CLI-displaying-full-system-configuration branch October 14, 2024 22:27

brosier01 restored the iox2-432-CLI-displaying-full-system-configuration branch October 14, 2024 22:28

brosier01 reopened this Oct 14, 2024

orecham reviewed Oct 14, 2024

View reviewed changes

orecham assigned brosier01 Oct 14, 2024

orecham reviewed Oct 14, 2024

View reviewed changes

orecham changed the title ~~[eclipse-iceoryx#432] Implement CLI iox2-config~~ [#432] Implement CLI iox2-config Oct 15, 2024

brosier01 force-pushed the iox2-432-CLI-displaying-full-system-configuration branch from ff8654f to 973c574 Compare October 15, 2024 22:28

orecham reviewed Oct 16, 2024

View reviewed changes

brosier01 requested a review from orecham October 23, 2024 17:26

orecham reviewed Oct 24, 2024

View reviewed changes

brosier01 requested review from orecham and elfenpiff November 11, 2024 19:00

brosier01 requested a review from elBoberido November 19, 2024 20:40

brosier01 force-pushed the iox2-432-CLI-displaying-full-system-configuration branch from e7798a3 to 037781e Compare November 19, 2024 21:07

brosier01 force-pushed the iox2-432-CLI-displaying-full-system-configuration branch from 037781e to 82f5387 Compare November 19, 2024 21:27

brosier01 force-pushed the iox2-432-CLI-displaying-full-system-configuration branch from 82f5387 to 08a4135 Compare November 20, 2024 19:59

brosier01 and others added 8 commits November 26, 2024 19:45

[eclipse-iceoryx#432] Implement CLI iox2-config

94c6204

[eclipse-iceoryx#432] Implement CLI iox2-config

2c1797c

Update doc/release-notes/iceoryx2-unreleased.md

ecd086f

Co-authored-by: orecham <[email protected]>

Update doc/release-notes/iceoryx2-unreleased.md

624ee0c

Co-authored-by: orecham <[email protected]>

[eclipse-iceoryx#432] Set default location for default configuration …

74ad90f

…file

[eclipse-iceoryx#432] Fix print_system_configuration for cross-platfo…

b96b5f7

…rm coloring + implementation of user confirmation required before deleting current configuration file

[eclipse-iceoryx#432] Replace the relative path of the configuration …

f6558d0

…file with the standard configuration path

[eclipse-iceoryx#432] Add an abstraction layer to obtain the path to …

41d20b8

…the configuration file + add a subcommand to display the contents of the configuration file

brosier01 force-pushed the iox2-432-CLI-displaying-full-system-configuration branch from 08a4135 to 41d20b8 Compare November 26, 2024 18:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#432] Implement CLI iox2-config #468

[#432] Implement CLI iox2-config #468

brosier01 commented Oct 14, 2024 •

edited

Loading

elfenpiff commented Oct 14, 2024

orecham left a comment •

edited

Loading

orecham Oct 14, 2024

orecham Oct 14, 2024

brosier01 Oct 15, 2024

orecham Oct 16, 2024

orecham Oct 24, 2024

elfenpiff Oct 30, 2024 •

edited

Loading

orecham Oct 30, 2024

elfenpiff Oct 31, 2024

brosier01 Nov 2, 2024

orecham Nov 2, 2024

orecham left a comment

orecham Oct 16, 2024 •

edited

Loading

elBoberido Oct 16, 2024

orecham Oct 16, 2024

orecham Oct 16, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading

orecham left a comment

orecham Oct 24, 2024

orecham Oct 24, 2024

orecham Oct 24, 2024

orecham Oct 24, 2024

brosier01 Nov 20, 2024

brosier01 commented Nov 19, 2024

elfenpiff commented Nov 20, 2024

brosier01 commented Nov 26, 2024

orecham commented Nov 27, 2024

[#432] Implement CLI iox2-config #468

Are you sure you want to change the base?

[#432] Implement CLI iox2-config #468

Conversation

brosier01 commented Oct 14, 2024 • edited Loading

Notes for Reviewer

Pre-Review Checklist for the PR Author

Checklist for the PR Reviewer

Post-review Checklist for the PR Author

References

elfenpiff commented Oct 14, 2024

orecham left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elfenpiff Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

orecham left a comment

Choose a reason for hiding this comment

orecham Oct 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

orecham Oct 16, 2024 • edited Loading

Choose a reason for hiding this comment

orecham Oct 16, 2024 • edited Loading

Choose a reason for hiding this comment

orecham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brosier01 commented Nov 19, 2024

elfenpiff commented Nov 20, 2024

brosier01 commented Nov 26, 2024

orecham commented Nov 27, 2024

brosier01 commented Oct 14, 2024 •

edited

Loading

orecham left a comment •

edited

Loading

elfenpiff Oct 30, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading