UCS/CONFIG/PARSER: Handle NULL entries in sparse arrays #10305

michal-shalev · 2024-11-15T19:42:48Z

What?

Following #10172
Introduced UCS_CONFIG_TYPE_FLAGS to support sparse parsing of configuration flags.
Removed unnecessary flags from UCX_EXTRA_OP_ATTR_FLAGS.

Why?

UCS_CONFIG_TYPE_BITMAP required contiguous indices, leading to core dumps when NULL values were accessed due to gaps in the array.
UCS_CONFIG_TYPE_FLAGS enables sparse arrays for flags, eliminating the need for unnecessary placeholders and simplifying the flag configuration.

How?

Implemented UCS_CONFIG_TYPE_FLAGS to loop until the specified number of arguments instead of relying on NULL-terminated arrays.

…r_flags

iyastreb · 2024-11-18T07:56:21Z

src/ucs/sys/string.c

+{
+    size_t i;
+
+    /* Ingnores NULL entries */


iyastreb · 2024-11-18T08:00:03Z

src/ucs/sys/string.h

+ *
+ * @param str          String to search for.
+ * @param string_list  Array of strings.
+ * @param num_of_args  Number of strings in the array.


I would propose to name this arg similar to array, eg str_array and str_array_size

src/ucp/core/ucp_context.c

iyastreb · 2024-11-18T08:03:59Z

src/ucp/core/ucp_context.c

@@ -544,7 +546,7 @@ static ucs_config_field_t ucp_context_config_table[] = {
   "in addition to what is set explicitly by the user. \n"
   "Possible values are: no_imm_cmpl, fast_cmpl, force_imm_cmpl, multi_send.",
   ucs_offsetof(ucp_context_config_t, extra_op_attr_flags),
-   UCS_CONFIG_TYPE_BITMAP(ucp_extra_op_attr_flags_names)},
+   UCS_CONFIG_TYPE_FLAGS((const void *)&ucp_extra_op_attr_flags_args)},


Could you please add few unit tests for this new option (test_config.cc)

iyastreb · 2024-11-18T08:05:38Z

src/ucs/config/parser.c

+            break;
+        }
+
+        ucs_assertv(i < (sizeof(uint64_t) * 8), "bit %d overflows for '%s'", i,


maybe use CHAR_BIT const instead of 8

iyastreb · 2024-11-18T08:06:55Z

src/ucs/config/parser.c

+    }
+
+    ret = 1;
+    *((uint64_t*)dest) = 0;


I would propose to create a variable to avoid repetitive casts:
uint64_t *flags = dest;

iyastreb · 2024-11-18T08:08:03Z

src/ucs/config/parser.c

+
+    ret = 1;
+    *((uint64_t*)dest) = 0;
+    p = strtok_r(str, ",", &saveptr);


consider using ucs_string_split

tvegas1

is it possible to fix UCS_CONFIG_TYPE_BITMAP for sparse, instead of introducing new UCS_CONFIG_TYPE_FLAGS?

tvegas1 · 2024-11-18T08:21:13Z

src/ucs/config/parser.c

+    ucs_config_flags_args_t *args = (ucs_config_flags_args_t*)arg;
+    int written                   = snprintf(buf, max, "comma-separated list of: ");
+
+    if (written < 0 || written >= max) {


tvegas1 · 2024-11-18T08:21:45Z

src/ucs/config/parser.c

+    }
+
+    /* Ingores NULL entries */
+    for (i = 0; i < num_of_args && ptr < end; ++i) {


() and (), everywhere

src/ucp/core/ucp_context.c

brminich · 2024-11-18T08:27:32Z

src/ucs/config/parser.h

@@ -384,6 +393,11 @@ void ucs_config_help_generic(char *buf, size_t max, const void *arg);
                                    ucs_config_help_bitmap,      ucs_config_doc_nop, \
                                    t}

+#define UCS_CONFIG_TYPE_FLAGS(t)   {ucs_config_sscanf_flags, ucs_config_sprintf_bitmap, \


Suggested change

#define UCS_CONFIG_TYPE_FLAGS(t) {ucs_config_sscanf_flags, ucs_config_sprintf_bitmap, \

#define UCS_CONFIG_TYPE_FLAGS(t) {ucs_config_sscanf_flags, ucs_config_sprintf_flags, \

?

great catch, fixed

brminich · 2024-11-18T08:30:43Z

src/ucs/config/parser.c

+            continue;
+        }
+
+        if (ptr > buf + 1 && ptr < end) {


Suggested change

if (ptr > buf + 1 && ptr < end) {

if ((ptr > (buf + 1)) && (ptr < end)) {

thanks, done

brminich · 2024-11-18T08:39:37Z

src/ucs/config/parser.c

+    ucs_config_flags_args_t *args = (ucs_config_flags_args_t*)arg;
+    int written                   = snprintf(buf, max, "comma-separated list of: ");
+
+    if (written < 0 || written >= max) {


Suggested change

if (written < 0 || written >= max) {

if ((written < 0) || (written >= max)) {

michal-shalev · 2024-11-18T16:47:11Z

is it possible to fix UCS_CONFIG_TYPE_BITMAP for sparse, instead of introducing new UCS_CONFIG_TYPE_FLAGS?

@tvegas1
The alternative is to change UCS_CONFIG_TYPE_BITMAP instances unrelated to the flags, but it is not critical for me.

yosefe · 2024-11-18T16:52:33Z

is it possible to fix UCS_CONFIG_TYPE_BITMAP for sparse, instead of introducing new UCS_CONFIG_TYPE_FLAGS?

We could do it nicely by adding another "unsigned long" field to ucs_config_parser_t to hold the count, something like

typedef struct ucs_config_parser {
    ...
    void                     (*help)(char *buf, size_t max, const void *arg);
    void                     (*doc)(ucs_string_buffer_t *strb, const void *arg);
    const void               *arg_ptr;
    unsigned long            arg_ulong;
};

And setting arg_ulong by ucs_static_array_size

michal-shalev · 2024-11-18T17:05:20Z

is it possible to fix UCS_CONFIG_TYPE_BITMAP for sparse, instead of introducing new UCS_CONFIG_TYPE_FLAGS?

We could do it nicely by adding another "unsigned long" field to ucs_config_parser_t to hold the count, something like
typedef struct ucs_config_parser {
    ...
    void                     (*help)(char *buf, size_t max, const void *arg);
    void                     (*doc)(ucs_string_buffer_t *strb, const void *arg);
    const void               *arg_ptr;
    unsigned long            arg_ulong;
};
And setting arg_ulong by ucs_static_array_size

We could but it will require changes in all the parsers, which is more than the scope of the flags, LMK WDYT @yosefe

brminich · 2024-11-18T17:26:05Z

src/ucs/sys/string.h

+ * Find a string in an array of strings with NULL entries.
+ *
+ * @param str          String to search for.
+ * @param string_list  Array of strings.


i'd call it str_array to be consistent with str_array_size

yosefe · 2024-11-18T17:39:34Z

is it possible to fix UCS_CONFIG_TYPE_BITMAP for sparse, instead of introducing new UCS_CONFIG_TYPE_FLAGS?

We could do it nicely by adding another "unsigned long" field to ucs_config_parser_t to hold the count, something like
typedef struct ucs_config_parser {
    ...
    void                     (*help)(char *buf, size_t max, const void *arg);
    void                     (*doc)(ucs_string_buffer_t *strb, const void *arg);
    const void               *arg_ptr;
    unsigned long            arg_ulong;
};
And setting arg_ulong by ucs_static_array_size
We could but it will require changes in all the parsers, which is more than the scope of the flags, LMK WDYT @yosefe

I think it's worth it , but would appreciate @gleon99 @brminich @tvegas1 opinions

tvegas1 · 2024-11-20T06:43:10Z

We could but it will require changes in all the parsers, which is more than the scope of the flags, LMK WDYT @yosefe

I think it's worth it , but would appreciate @gleon99 @brminich @tvegas1 opinions

I would try to have only one UCS_CONFIG_TYPE_BITMAP that supports holes because _TYPE_FLAGS seems to be more or less the same, if that's possible.

michal-shalev added 2 commits November 15, 2024 19:28

UCS/CONFIG/PARSER: Handle NULL entries in sparse arrays

a2eba22

UCP/CONTEXT/CONFIG: Remove no_imm_cmpl and force_imm_cmpl from op_att…

0b9171d

…r_flags

michal-shalev requested review from brminich, yosefe, Artemy-Mellanox and gleon99 November 15, 2024 19:42

UCS/CONFIG/PARSER: Fix code style

2247974

michal-shalev self-assigned this Nov 16, 2024

michal-shalev added the Ready for Review label Nov 16, 2024

iyastreb reviewed Nov 18, 2024

View reviewed changes

tvegas1 reviewed Nov 18, 2024

View reviewed changes

brminich reviewed Nov 18, 2024

View reviewed changes

michal-shalev added 3 commits November 18, 2024 17:04

UCS/CONFIG/PARSER: fix codespell

cf20399

UCS/CONFIG/PARSER: fix codespell 2.0

0cb3f67

UCS/CONFIG/PARSER: PR fixes

2a29c90

brminich reviewed Nov 18, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UCS/CONFIG/PARSER: Handle NULL entries in sparse arrays #10305

UCS/CONFIG/PARSER: Handle NULL entries in sparse arrays #10305

michal-shalev commented Nov 15, 2024 •

edited

Loading

iyastreb Nov 18, 2024

michal-shalev Nov 18, 2024

iyastreb Nov 18, 2024

michal-shalev Nov 18, 2024

iyastreb Nov 18, 2024

iyastreb Nov 18, 2024

iyastreb Nov 18, 2024

iyastreb Nov 18, 2024

tvegas1 left a comment

tvegas1 Nov 18, 2024

michal-shalev Nov 18, 2024

tvegas1 Nov 18, 2024

michal-shalev Nov 18, 2024

brminich Nov 18, 2024

michal-shalev Nov 18, 2024

brminich Nov 18, 2024

michal-shalev Nov 18, 2024

brminich Nov 18, 2024

michal-shalev Nov 18, 2024

michal-shalev commented Nov 18, 2024

yosefe commented Nov 18, 2024 •

edited

Loading

michal-shalev commented Nov 18, 2024 •

edited

Loading

brminich Nov 18, 2024

yosefe commented Nov 18, 2024

tvegas1 commented Nov 20, 2024

	#define UCS_CONFIG_TYPE_FLAGS(t) {ucs_config_sscanf_flags, ucs_config_sprintf_bitmap, \
	#define UCS_CONFIG_TYPE_FLAGS(t) {ucs_config_sscanf_flags, ucs_config_sprintf_flags, \

	if (ptr > buf + 1 && ptr < end) {
	if ((ptr > (buf + 1)) && (ptr < end)) {

	if (written < 0 \|\| written >= max) {
	if ((written < 0) \|\| (written >= max)) {

UCS/CONFIG/PARSER: Handle NULL entries in sparse arrays #10305

Are you sure you want to change the base?

UCS/CONFIG/PARSER: Handle NULL entries in sparse arrays #10305

Conversation

michal-shalev commented Nov 15, 2024 • edited Loading

What?

Why?

How?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tvegas1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michal-shalev commented Nov 18, 2024

yosefe commented Nov 18, 2024 • edited Loading

michal-shalev commented Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

yosefe commented Nov 18, 2024

tvegas1 commented Nov 20, 2024

michal-shalev commented Nov 15, 2024 •

edited

Loading

yosefe commented Nov 18, 2024 •

edited

Loading

michal-shalev commented Nov 18, 2024 •

edited

Loading