mdtest 3.2.0 segfaults if run naively

[glennklockwood]

mdtest used to have sensible defaults if it was run without any command line options (e.g., mpirun -np 1 mdtest -n 1). Now it just segfaults:

$ mpirun -np 1 /home/glock/ior/test/_inst/bin/mdtest -n 1
-- started at 01/18/2019 15:37:02 --

mdtest-1.9.3 was launched with 1 total task(s) on 1 node(s)
Command line used: /home/glock/ior/test/_inst/bin/mdtest "-n" "1"

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   EXIT CODE: 139
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)

Backtrace is

(gdb) bt
#0  0x00002adfc2575e29 in strtok () from /lib64/libc.so.6
#1  0x00000000004074a1 in parse_dirpath (dirpath_arg=0x412a9b "./out") at mdtest.c:214
#2  0x000000000040d871 in mdtest_run (argc=<optimized out>, argv=<optimized out>, 
    world_com=world_com@entry=1140850688, world_out=<optimized out>) at mdtest.c:2263
#3  0x0000000000402bfc in main (argc=3, argv=0x7fff6d7b2248) at mdtest-main.c:8

It looks like there is no default for -d and the result is a segfault. Not a great out-of-box experience ☹️ . Running with -d . works around this, but we should probably made the default for -d into $PWD (or an error message).

[glennklockwood]

This might be something weird with my environment. strtok is where something breaks, and even a simple reproducer like this segfaults:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
    char *dirpath_arg = "./out";
    char *token;
    char delimiter_string[3] = { '@', '\n', '\0' };
    int i;

    token = strtok(dirpath_arg, delimiter_string);
    while (token != NULL) {
        token = strtok(NULL, delimiter_string);
        i++;
    }
    return 0;
}

This is on CentOS Linux release 7.2.1511 with glibc-2.17.106.el7_2.8. It does not happen on SLES 12sp3 or macOS. In the event that is some weird environmental thing that might affect someone else, I'll leave this open until I track down the specific cause. Until then though, this is not a high priority.

[JulianKunkel]

Weird, since the default is ./out as seen from your stacktrace, too: #0 0x00002adfc2575e29 in strtok () from /lib64/libc.so.6 #1 0x00000000004074a1 in parse_dirpath (dirpath_arg=0x412a9b "./out") at mdtest.c:214 I believe that may be some memory protection issue as strtok is manipulating the memory. Have a look if the fix here solves it: https://github.com/hpc/ior/tree/fix-127 Am Fr., 18. Jan. 2019 um 23:40 Uhr schrieb Glenn K. Lockwood < [email protected]>:

…

mdtest used to have sensible defaults if it was run without any command line options (e.g., mpirun -np 1 mdtest -n 1). Now it just segfaults: $ mpirun -np 1 /home/glock/ior/test/_inst/bin/mdtest -n 1 -- started at 01/18/2019 15:37:02 -- mdtest-1.9.3 was launched with 1 total task(s) on 1 node(s) Command line used: /home/glock/ior/test/_inst/bin/mdtest "-n" "1" =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 139 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) Backtrace is (gdb) bt #0 0x00002adfc2575e29 in strtok () from /lib64/libc.so.6 #1 0x00000000004074a1 in parse_dirpath (dirpath_arg=0x412a9b "./out") at mdtest.c:214 #2 0x000000000040d871 in mdtest_run (argc=<optimized out>, argv=<optimized out>, ***@***.***=1140850688, world_out=<optimized out>) at mdtest.c:2263 #3 0x0000000000402bfc in main (argc=3, argv=0x7fff6d7b2248) at mdtest-main.c:8 It looks like there is no default for -d and the result is a segfault. Not a great out-of-box experience ☹️ . Running with -d . works around this, but we should probably made the default for -d into $PWD (or an error message). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#127>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE1uylk-3dROPr934Fhuf6SNOuFzxvobks5vEltjgaJpZM4aI0Ls> .

-- Dr. Julian Kunkel Lecturer, Department of Computer Science +44 (0) 118 378 8218 http://www.cs.reading.ac.uk/ https://hps.vi4io.org/ PGP Fingerprint: 1468 1A86 A908 D77E B40F 45D6 2B15 73A5 9D39 A28E

[glennklockwood]

Thanks Julian! That indeed fixed it. I added an OOM check but otherwise this looks great and you can merge into master if my change looks fine.

Do you think this this is worth backporting to the 3.2 release?

[JulianKunkel]

Cool. The patch should be able to be integrated into the 3.2. release without a problem. I wouldn't release a new bugfix release just for it (until people scream), let's see what else to fix! Am Do., 24. Jan. 2019 um 00:04 Uhr schrieb Glenn K. Lockwood < [email protected]>:

…

Thanks Julian! That indeed fixed it. I added an OOM check but otherwise this looks great and you can merge into master if my change looks fine. Do you think this this is worth backporting to the 3.2 release? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#127 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE1uymDvxj9NOBQrDWnuhUGtqNsIxLqOks5vGPiqgaJpZM4aI0Ls> .

-- Dr. Julian Kunkel Lecturer, Department of Computer Science +44 (0) 118 378 8218 http://www.cs.reading.ac.uk/ https://hps.vi4io.org/ PGP Fingerprint: 1468 1A86 A908 D77E B40F 45D6 2B15 73A5 9D39 A28E

[glennklockwood]

I merged the two commits on fix-127 into the following branches:

1. master
2. rc
3. 3.2

The 3.2 branch was branched off of the 3.2.0 tag (of the rc branch), and that is where 3.2.1 and any subsequent maintenance releases for 3.2 will go.

I think fix-127 can be deleted and this can be closed now.