audit winlibs tools in distribution

[shawnlaffan]

It seems several of the utilities distributed with SP 5.38 and 5.36 interfere with other systems. Some of these utilities were added for convenience so are not strictly needed.

Two examples are make.exe (#148) and cmake.exe (actions/runner-images#8598).

GitHub actions have pinned SP to 5.32 for now. actions/runner-images#8622

We need to run an audit to determine what we really need for SP to run. This might involve building our own binutils, which is not very hard.

This is partly a redux of #57

[sisyphus]

Some of these utilities were added for convenience so are not strictly needed.

An alternative to removing them from the distro is to relocate them to a separate folder that is not in the path.
Users can then still conveniently access those utilities (at their own peril) by adding that "separate location" to the path.

I haven't quite got my head around the problem associated with the name of the make utility.
But if you want to build a strawberry perl whose $Config{make} is "make" instead of "gmake", just set PLMAKE to "make" - either by editing GNUmakefile or by specifying PLMAKE=make on the command line.
That worked fine for me, and I conducted the additional test of using this perl to successfully build Math::GMPz.
I guess it's possible that there's a buggy module somewhere on CPAN that won't allow for a Windows make utility named "make",
(I was actually a bit surprised, and a little disappointed, that SP chose the name "gmake" over "make" in the first place.)

No such luck with setting PLMAKE to "mingw32-make", During the perl build, dist/Time-HiRes fails to build.
It's not immediately apparent to me what the problem is, but I'm guessing it's because EU::MM doesn't cater for a make utility named "mingw32-make". I haven't investigated.

[shawnlaffan]

An alternative to removing them from the distro is to relocate them to a separate folder that is not in the path. Users can then still conveniently access those utilities (at their own peril) by adding that "separate location" to the path.

We could use .../c/opt/bin or .../opt. (That assumes we still want to distribute the winlibs generated libs and tools).

WRT make, I think we are better off not providing it since it clashes with other systems. gmake is less common so has fewer clashes. The question remains whether we build our own or keep the winlibs copy, with its known recursion limits due to the name differing from the original.

[sisyphus]

The question remains whether we build our own or keep the winlibs copy

I think that if building your own make utility named "gmake"will avoid the recursion issue, then that's a good reason to do precisely that.
I was, however, thinking that the only way to avoid the recursion issue might be to name the make utility "make" (as that's what it really is, in the final analysis) - though I guess you could probably patch the source to match the name of the executable, if that is needed.

Anyway, as I said, I don't really have a handle on what's causing this issue, or on how to work around it.

[shawnlaffan]

It was with the utility renamed to make that the issue first manifested (#148).

I should hopefully have some time next week to look at this.

[sisyphus]

I'm now seeing something pretty evil in the "make.exe" and "gmake.exe" that ship with Strawberry Perl - and I suspect (unproven) that the problem might not be that they are duplicates of "mingw32-make.exe" but that they are GNU make version 4.4.1.

I've lately tried building perl-5.39.5 Using the toolchain provided by Strawberry Perl 5,38,0,
In the win32 directory of a freshly unpacked perl-5.39.5 source I've tried building with both:

make CCTYPE=GCC CCHOME=C:\sp\_64\sp-5.38.0\c -j8 test

and

gmake CCTYPE=GCC CCHOME=C:\sp\_64\sp-5.38.0\c -j8 test

In both case, the process concludes with

...
Test Summary Report
-------------------
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
../lib/locale.t                                                    (Wstat: 0 Tests: 682 Failed: 69)
  Failed tests:  381-449
Files=2887, Tests=1129542, 1121 wallclock secs (11.47 usr +  0.78 sys = 12.25 CPU)
Result: FAIL
Finished test run at Wed Nov 29 21:45:29 2023.
gmake: *** [GNUmakefile:1829: test] Error 69
rm ..\perly.c

Now, the lib/locale.t failure happens because Strawberry's gcc uses the MSVC runtime by default. (That's easily fixed by using a gcc that instead uses the UCRT.)
But that insidious "rm ..\perly.c" right at the very end can only be described as "evil". (Feel free to insert expletives.)
Any subsequent attempt to "make install" or "make distclean" is doomed if perly.c is not reinstated.

I also have versions 4.2.1 of "gmake.exe" and "make.exe". I'm not sure of their origin, but their sha256 checksums confirm that they are exactly the same.
If I replace the Strawberry Perl gmake/make executables with these 2 versions, then perly.c does not get clobbered at the end

I don't rule out the possibility that building your own gmake-4.4.1 and make-4.4.1 will fix the issue .... but bear in mind the possibility that it may not.
(Even with my own builds of perl-5.39.5 using a gcc-13.2.0 that defaults to UCRT, I strike this problem if I use the 4.4.1 versions of make/gmake that Strawberry provides.)

[shawnlaffan]

What happens if you use mingw32-make? The make and gmake are just copies of mingw32-make and that appears to have caused the issues under #148. If mingw32-make does not work with 4.4.1 we might need to build an earlier version.

[sisyphus]

What happens if you use mingw32-make?

Same problem - perly.c gets clobbered

Firstly, you don't need to run make test to see the problem. If you just run the make step (which takes far less time) you'll still see the rm ..\perly.c at the end.
It's probably worthwhile someone checking this, in case it's just some weird problem specific to my Windows 11 system.

Secondly, note that PLMAKE is set to gmake in the GNUmakefile, so irrespective of the name of the make executable you execute, "gmake.exe" is going to be called internally unless you overwrite the PLMAKE setting to something else.

Thirdly, I wonder if there is some way to fix this problem from within the perl source. Of course, I have no idea what that "some way" would be.
At this stage, using the 4.4.1 versions, I've tried:
building perl using make.exe, specifying PLMAKE=gmake
building perl using make.exe, specifying PLMAKE=make
building perl using gmake.exe, specifying PLMAKE=gmake
building perl using gmake.exe, specifying PLMAKE=make
building perl using mingw32-make.exe, specifying PLMAKE=gmake
building perl using mingw32-make.exe, specifying PLMAKE=mingw32-make

In all cases,perly.c is removed as the last action taken by the process.
I've just now left out the "-j8" argument in case it was somehow responsible, by running gmake CCTYPE=GCC CCHOME=C:\sp_64\sp-5.38.0\c in the win32 directory of a freshly unpacked perl-5.39.5 source - and perly.c still gets clobbered at the end.
It'd be good if someone else could find the few minutes needed to check on that. You just need to alter the CCHOME argument to reflect the location in which Strawberry's "C" folder is located.

YMMV if you try running the the "make test" step, having specified PLMAKE to be other than "gmake".

[shawnlaffan]

These links seem to be relevant:
https://unix.stackexchange.com/questions/517190/what-causes-make-to-delete-intermediate-files
https://stackoverflow.com/questions/47447369/gnu-make-removing-intermediate-files

Given their contents, maybe something is messing with perly.c and changing its timestamp. The make process then assumes it is an intermediate file and cleans it up. But if this is the case then I don't know why it happens with make 4.4.1 and not 4.2.

What happens if you add this line somewhere in GNUMakefile?

.PRECIOUS: ../perly.c

[sisyphus]

The suggested ".PRECIOUS" insertion made no difference - perly.c still gets removed,
I'll try to track down when this behaviour kicks in .

UPDATE 1: It's new in 5.39.5 - likely either a change to GNUmakefile or to the handling of perly.c. There's no clobbering of
perly.c when building 5.39.4.
UPDATE 2: Seems to NOT be the GNUmakefile. The 5.39.4 GNUmakefile works fine with perl-5.39.5, but perly.c still gets clobbered. Suspicion falls on handling of perly.c.

[shawnlaffan]

Thanks @sisyphus

If I get a chance I'll have a go as well.

[sisyphus]

Seems that all I have to do in order to build perl-5.39.5 using gmake-4.4.1 is to touch perly.c
See Perl/perl5#21651 - which I've just re-opened, now that I've finally got a handle on what's happening.

Thanks @shawnlaffan.

UPDATE: The issue surrounding perly.c (Perl/perl5#21651) has since been fixed and closed.

[MasterInQuestion]

    "We need to run an audit to determine what we really need for SP to run."
<^>    For reference: https://github.com/exiftool/exiftool/issues/260