perl 5.36.1 builds with gcc-13.1

[shawnlaffan]

Given the discussion in #93 regarding PDL builds, I've had a go at using gcc-13.1.

I'll push commits to the wip_536 branch soon but below are some notes.

1. I updated the GNUmakefile to check for libmcfgthread-1.dll, as per [Win32] Fix win32/GNUmakefile to accommodate gcc-13 Perl/perl5#21038
2. The GNUMakefile uses -Os (which we have already been using for 5.36.0, FWIW).
3. libmcfgthread-1.dll needs to be copied to the perl/bin dir, the same as for libwinpthread-1.dll

The build ran to completion but then we hit some test fails. Several of these are already known.

1. op/magic.t is under Failed test 186 - ENV store downgrades utf8 in setenv at op/magic.t line 75 (also test 187) #60
2. porting/customized.t is because we are patching a module and have not updated the tests.
3. cpan/ExtUtils-MakeMaker/t/basic.t seems to be a docker issue and is also in 5.37.11: 5.37.11 build  #97 (comment)
4. cpan/ExtUtils-PL2Bat/t/make_executable.t has an open issue at Test Failures on Windows Perl-Toolchain-Gang/extutils-pl2bat#7 and is probably to do with the docker setup as it builds locally without error. It did not affect the gcc-10 builds.
5. The TODO passes have been occurring for the previous builds with gcc-10.

Test Summary Report
-------------------
op/magic.t                                                         (Wstat: 0 Tests: 208 Failed: 2)
  Failed tests:  186-187
op/taint.t                                                         (Wstat: 0 Tests: 1058 Failed: 2)
  Failed tests:  1, 463
porting/customized.t                                               (Wstat: 0 Tests: 44 Failed: 1)
  Failed test:  22
win32/stat.t                                                       (Wstat: 0 Tests: 50 Failed: 0)
  TODO passed:   42
../cpan/ExtUtils-MakeMaker/t/basic.t                               (Wstat: 65280 (exited 255) Tests: 188 Failed: 0)
  Non-zero exit status: 255
../cpan/ExtUtils-PL2Bat/t/make_executable.t                        (Wstat: 11776 (exited 46) Tests: 112 Failed: 46)
  Failed tests:  5-7, 10-11, 15-16, 20-21, 25-26, 30-31
                35-36, 40-42, 45-46, 50-51, 55-56, 60-61
                65-66, 70-71, 75-76, 80-81, 85-86, 90-91
                95-96, 100-101, 105-106, 110-111
  Non-zero exit status: 46
../dist/Net-Ping/t/450_service.t                                   (Wstat: 0 Tests: 26 Failed: 0)
  TODO passed:   9
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2766, Tests=1151652, 1713 wallclock secs (122.56 usr + 16.17 sys = 138.74 CPU)
Result: FAIL
gmake: *** [GNUmakefile:1815: test] Error 51

[shawnlaffan]

Extlibs files are at https://github.com/StrawberryPerl/build-extlibs/releases/tag/dev_gcc13.1_20230502

The docker instances are the same as for https://github.com/StrawberryPerl/spbuild/tree/main/5.34 except the winlibs URL was updated to the gcc-13.1 version.

[shawnlaffan]

Commits:
704d296
2ff231a
25410d3

[sisyphus]

cpan/ExtUtils-PL2Bat/t/make_executable.t has an open issue at ....

I struck those ../cpan/ExtUtils-MakeMaker/t/* issues when I first built perl using gcc-13.
But they, along with t/op/taint.t failures that I was also getting, went away when I fixed the GNUmakefile to copy libmcfgthread-1.dll to the t directory - and I haven't seen them since.

Not sure if that piece of information is in any way helpful .... ;-(

Cheers,
Rob

[shawnlaffan]

Yes, the errors go away when libmcfgthread-1.dll is copied across.

[shawnlaffan]

PDL still fails with the same errors, though.

[sisyphus]

PDL` still fails with the same errors, though.

That pesky niceslice issue again ?
I didn't experience any issue with PDL-2.082 and perl-5.36.1 built using gcc-13.1.0 ... but that was LH-Mouse's gcc-13.1.0.
I'll do a build of perl-5.36.1 using winlibs' gcc-13.1.0 and report back on how that handles PDL-2.082 for me.

Cheers,
Rob

[shawnlaffan]

Thanks Rob. Will be interesting to see if it can be replicated using a different machine.

[mohawk2]

FYI there is now a PDL 2.083 and it would be helpful to see if that makes a difference?

[sisyphus]

Hmmm ... if perl-5.36.1 is built using LH_Mouse's 1310 compiler then PDL-2.082 and 2.083 build, test and install fine.
But if perl-5.36.1 is built using winlibs' 1310 compiler then both 2.082 and 2.083 suffer the "niceslice" failure.
I'll try and digest the implications of this tomorrow. (Feel free to unravel it all for me ;-)

TBC, as is to be expected, in all cases I build PDL using the actual compiler that built perl-5.36.1.

With LH_Mouse's compiler, PDL-2.082 and 2.083 require the Math/Basic/Makefile.PL and IO/Browser/Makefile.pl patches that I've already mentioned somewhere in some previous post.
LH_Mouse explained to me that his decision to change the default -masm setting was a personal one. No-one else is bound to adopt it, and Brecht Sanders clearly hasn't.

IIRC, PDL-2.082 is fine with winlibs-13.1.0, perl-5.37.11.
Maybe just forget about the Strawberry PDL build until 5.38 ?? (5.38.0 is not very far away.)

Cheers,
Rob

[mohawk2]

It would be good to have Strawberry PDL builds for all versions of PDL, obviously. But I don't have a lot of time to invest in helping with that currently, so those who do get to make the calls :-)

[shawnlaffan]

@sisyphus - just to be absolutely sure, did you also compile PDL with OPTIMIZE=-Os?

PDL compiled with gcc-10 when using OPTIMIZE=-Og (see #93 (comment)).

I'll try to have a go later today.

[shawnlaffan]

But on the good news front, since this issue is for the full build with gcc-13.1, the CPAN step worked to completion and the system generated all the requisite outputs. This was for a config without the PDL component.

I'll upload the artefacts later today for people to test.

[sisyphus]

@sisyphus - just to be absolutely sure, did you also compile PDL with OPTIMIZE=-Os?

I believe so.
perl -V:optimize reports -Osand AFAIK I did nothing to override it when building PDL.
So ... unless the PDL build is doing something to change it, optimization level will be -Os.

Cheers,
Rob

[shawnlaffan]

I also built with perl Makefile.PL OPTIMIZE=-Os and got the usual errors.

[sisyphus]

I also built with perl Makefile.PL OPTIMIZE=-Os and got the usual errors.

I tried perl Makefile.PL CC="gcc -masm=intel" (thereby adopting LH_Mouse's preferred AS syntax) just in case that was the key to PDL success with winlibs-13.1.0 on perl-5.36.1.
But the niceslice tests still blew up in the same way.

So I went one step further and built perl-5.36.1 with winlibs-13.1.0 with CC set to gcc -masm=intel. After providing a bit of manual assistance along the way, that perl build worked out well and passed all tests.
But the PDL builds (2.082 && 2.083) still threw up the usual niceslice failures.

My main interest in LH_Mouse's compiler is that his 32-bit 13.1.0 is the only compiler I've ever struck that produces MSWin32-x86-multi-thread-quadmath && MSWin32-x86-mutli-thread-64int-quadmath builds that pass all tests.
With all of the winlibs 32-bit compilers, perls of those 2 configurations consistently fail a number of threads-related tests.

Admittedly, it's only 32-bit quadmath builds - which are hardly high priority.
But it was a nice (ie uplifting) experience to see all test pass for those 2 configurations.

Cheers,
Rob

[shawnlaffan]

And just as another data point I tried with perl Makefile.PL OPTIMIZE=-O1 and the niceslice failures are still present.

I'm building one with perl Makefile.PL OPTIMIZE="-g -O2 to see if we can get a gdb backtrace from it (assuming that's a reasonable thing to do, and because running with just -g produces a working PDL with gcc-10).

[shawnlaffan]

There's also the brute force option of repeatedly building PDL with the -O1 optimisation set but with one turned off at a time. That assumes there are no interactions between optimisations, of course, but it's otherwise a simple loop.

The list can be obtained by running gcc -O1 -Q --help=optimizers 2>&1 | findstr enabled > O1.txt

[shawnlaffan]

GDB backtrace below after building with OPTIMIZE="-g -O2".

@mohawk2 - does this help with diagnosis? Or do we also need a perl built with debugging?

(The gdb warning is probably due to the docker env, or because I removed 60+MB of python stuff from the winlibs directory tree).

Starting program: C:\strawberry\perl\bin\perl.exe -Mblib -MPDL -E "my $p=pdl[6,6]; say $p; my $y = (all $p==6) ? 'yes' :
 'no'; say $y"
[New Thread 3064.0x484]
[New Thread 3064.0x6c0]
[6 6]

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffc756b3187 in boot_PDL__Core ()
   from C:\spbuild\msys64\.cpanm\work\1683075989.2688\PDL-2.083\blib\arch\auto\PDL\Core\Core.xs.dll
(gdb) bt
warning: could not convert 'main' from the host encoding (CP65001) to UTF-32.
This normally should not happen, please file a bug report.
#0  0x00007ffc756b3187 in boot_PDL__Core ()
   from C:\spbuild\msys64\.cpanm\work\1683075989.2688\PDL-2.083\blib\arch\auto\PDL\Core\Core.xs.dll
#1  0x00007ffc756a58f3 in boot_PDL__Core ()
   from C:\spbuild\msys64\.cpanm\work\1683075989.2688\PDL-2.083\blib\arch\auto\PDL\Core\Core.xs.dll
#2  0x00007ffc75514ef2 in ?? ()
   from C:\spbuild\msys64\.cpanm\work\1683075989.2688\PDL-2.083\blib\arch\auto\PDL\Ops\Ops.xs.dll
#3  0x00007ffc77746875 in perl536!Perl_caller_cx () from C:\strawberry\perl\bin\perl536.dll
#4  0x00007ffc777aea22 in perl536!Perl_runops_standard () from C:\strawberry\perl\bin\perl536.dll
#5  0x00007ffc77769e9e in perl536!Perl_amagic_call () from C:\strawberry\perl\bin\perl536.dll
#6  0x00007ffc7776a45b in perl536!Perl_try_amagic_bin () from C:\strawberry\perl\bin\perl536.dll
#7  0x00007ffc7775c17a in perl536!Perl_packlist () from C:\strawberry\perl\bin\perl536.dll
#8  0x00007ffc777aea22 in perl536!Perl_runops_standard () from C:\strawberry\perl\bin\perl536.dll
#9  0x00007ffc777722a4 in perl_run () from C:\strawberry\perl\bin\perl536.dll
#10 0x00007ffc777bd95f in perl536!RunPerl () from C:\strawberry\perl\bin\perl536.dll
#11 0x00007ff61f801340 in ?? ()
#12 0x00007ff61f801146 in ?? ()
#13 0x00007ffc82577ab4 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#14 0x00007ffc8521a351 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#15 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)

[shawnlaffan]

The main topics of this issue have been handled now that #93 has been resolved. The remaining topics have their own issues.

I'll close this issue but please add a note if there is a reason to reopen it.

[shawnlaffan]

I should have noted this before closing.

A new dev release with a working PDL is available from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5361_20230524_gcc13