Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strawberry Perl POC and Status of 5.34.1 #39

Closed
rafulbrig opened this issue May 4, 2022 · 82 comments
Closed

Strawberry Perl POC and Status of 5.34.1 #39

rafulbrig opened this issue May 4, 2022 · 82 comments

Comments

@rafulbrig
Copy link

I am preparing to upgrade to the latest release of Strawberry Perl (which shows as 5.32.1.1). I then noticed that Perl itself is at release 5.34.1. I tried to find the name of the POC for Strawberry Perl, but was unsuccessful. What is the contact info for the current manager of Strawberry Perl releases?

I am trying to find out if the upgrade to Strawberry Perl to release 5.34.1 is imminent. If it is, I am happy to wait before upgrading, so I do not have to upgrade twice. Any info you can provide will be most appreciated. Thanks.

@genio
Copy link
Member

genio commented May 4, 2022

We are trying to get things working. Life has hindered some of us from being able to work on it some. It's in the works, but I can't give an honest estimate of the time it will take at the moment. Apologies for that.

@rafulbrig
Copy link
Author

Thank you Genio. I appreciate all the effort involved in a new release.

@aero
Copy link

aero commented Jun 28, 2022

Perl 5.36.0 released.
I would like to see the new version of strawberry perl.

@ludovicroland
Copy link

ludovicroland commented Jul 11, 2022

Hello :)

Same here, I'd love to test Perl 5.34 in order to use, at least, the native try catch blocks :)

Hope to see an update soon 🤞

@sisyphus
Copy link

I would recommend building perl yourself.
It's really easy - and the more people that are building perl on windows, then the better our chances that it will stay that way .... and the better our chances of getting the "messy bits" (see below) fixed.

Start by grabbing the perl source:
https://cpan.metacpan.org/authors/id/X/XS/XSAWYERX/perl-5.34.0.tar.gz
https://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl-5.36.0.tar.gz

(It wasn't until I finished writing this that I realized you had specified 5.34.0, not 5.36.0.)

Untar that perl source into some directory - say C:\straw-comp.
Then cd to the C:\straw-comp\perl-5.36.0\win32 folder.

Given that I want to install a 64 bit build of perl-5.36.0 into C:\straw\perl-5.36.0, I'll need to include the following command line argument:

INST_TOP=C:\straw\perl-5.36.0

(Modify accordingly if you want to install it elsewhere.)

I'll also want to specify CCHOME, which is the directory that contains
bin/gcc.exe. For me, that directory is C:_64\strawberry-5.32.0\c.
Therefore, I will need to include the command line argument:

CCHOME=C:\_64\strawberry-5.32.0\c 

(Modify accordingly if your CCHOME is somewhere else.)

##############
The messy bits
##############

Annoyingly, there's some cpan/IO-Compress/t/ tests that hang for me.
To avoid this issue during the running of the perl test suite, I first set the following environment variable:

set IO_COMPRESS_SKIP_STDIN_TESTS=1

Then, after I've built perl with 'gmake .... test', I clear that environment variable, cd to 'cpan/IO-Compress', run 'gmake test' and sit back while all test pass.

There might also be a hang with IPC-Cmd/t/01_IPC-Cmd.t. (Let me know if you see that one.)
I actually patch that test file to skip all tests if $ENV{IPC_CMD_SKIP_TESTS} is set to a true value.
It's a simple patch - just insert the following at the beginning of the file, before building perl:

if($ENV{IPC_CMD_SKIP_TESTS}) {
  ok('1' eq '1', 'test 1');
  exit 0;
}

And, of course, I also:

set IPC_CMD_SKIP_TESTS=1

Then, after I've built perl with 'gmake ... test', and still in the 'win32' folder, I clear that environment variable and run:

..\perl -I..\lib ../cpan/IPC-Cmd/t/01_IPC-Cmd.t >out.txt

I can then examine out.txt, whereupon I clearly see that all tests have passed.

If a test hangs, you can just kill it using Process Explorer (if you have that).
If you kill it with Ctrl-C, then the running of the test suite aborts immediately.

#####
Build it
#####

Run 'gmake test':
C:\straw-comp\perl-5.36.0\win32> gmake INST_TOP=C:\straw\perl-5.36.0 CCHOME=C:_64\strawberry-5.32.0\c test

Then run 'gmake install', repeating the same options as were used for 'gmake test'.
C:\straw-comp\perl-5.36.0\win32> gmake INST_TOP=C:\straw\perl-5.36.0 CCHOME=C:_64\strawberry-5.32.0\c install

That's it. Then just run:
set PATH=C:\straw\perl-5.36.0\bin;%PATH% perl -V
and all should be good.
I've actually just gone through the above procedure, using my 64-bit Strawberry Perl-5.32.0 installation.
Apart from the afore-mentioned hangs, it was perfect.

#############
SOME OTHER OPTIONS:
#############

Not actually necessary, but I recommend adding

USE_MINGW_ANSI_STDIO=define BUILDOPTEXTRA=-fdiagnostics-color=never

to the above command line args.

If your Strawberry Perl (and therefore gcc also) is 32-bit then you'll need to add

WIN64=undef

to the above command line args.

If your Strawberry Perl (and therefore gcc also) is 32-bit and you want 64-bit integers then you'll need to add

USE_64_BIT_INT=define

to the above command line args.

Alternatively, if you want your nvtype to be 'long double', then you'll need to add:

USE_LONG_DOUBLE=define

to the above command line args.

Alternatively, if you want your nvtype to be '__float128', then you'll need to add:

USE_QUADMATH=define I_QUADMATH=define

to the above command line args.

If you want to re-use the same source to build a perl with a different configuration, or using a different compiler, just run 'make distclean' first.

Cheers,
Rob

@aero
Copy link

aero commented Jul 13, 2022

Besides it could be built own windows perl.

The new language version announcement with binaries for major OSes is considered common and basic.

See the new kids on the block & hipster language zig's download page( https://ziglang.org/download/ ).
Sister languge Raku is also doing so. https://rakudo.org/downloads

Why not for perl itself?

I think this is a matter that TPF(the perl foundation) should be concerned about.

@genio
Copy link
Member

genio commented Jul 13, 2022

@aero It's an extremely complex process being completed by volunteers. The TPF doesn't have anything to do with the Strawberry build of Windows to my knowledge. We're making progress on it every day, but we're trying to do so in a way that will make future builds less opaque and allow more people to work on it. Please don't come in here being rude and saying because others do it, we should be able to with no problem.

Understand the entirety of the problem set starting with some of the information here: https://github.com/StrawberryPerl/spbuild and then start working through the steps. None of it's simple on Windows.

@aero
Copy link

aero commented Jul 13, 2022

It's an extremely complex process being completed by volunteers. The TPF doesn't have anything to do with the Strawberry build of Windows to my knowledge. We're making progress on it every day, but we're trying to do so in a way that will make future builds less opaque and allow more people to work on it. Please don't come in here being rude and saying because others do it, we should be able to with no problem.

I'm sorry if it felt rude.

I hope perl has multiple options like python.
Official release:

I mentioned TPF because I think TPF needs to support or be responsible for the official windows perl package like other major languages by donation or grant apart from Strawberry perl.

Understand the entirety of the problem set starting with some of the information here: https://github.com/StrawberryPerl/spbuild and then start working through the steps. None of it's simple on Windows.

Thank you for your unseen efforts.

@sisyphus
Copy link

@genio, I don't think @aero was being rude.
My take is that he was simply wishing that the perl developers would release perl binaries for windows (and other systems) - just as the raku developers do with raku, and the python developers do with python.
He took it a bit further by also expressing his view that it would be in the best interests of perl itself if the perl developers were to do precisely that.

Of course, they never have, and they probably never will - but @aero's POV on this makes reasonable sense to me.

I don't think he was suggesting that it was time for the perl developers to take responsibility for the release of Strawberry Perl.
If they can't be bothered releasing CORE perl binaries, then they're certainly not going to get involved in providing a gcc toolchain, 3rd party C libraries and associated perl extensions as well.

Cheers,
Rob

@blues1875
Copy link

Thanks for that great HOWTO, Rob (sisyphus). I decided to have a go myself and managed to build 5.36 starting with my SP 5.32 installation, as you did. Interestingly, I had no problem with IO::Compress, but my build failed on HTTP::Tiny. My solution was to move that folder out of the cpan folder and running gmake again. I'll do some more experimenting when I get time (you asked us to let you know about test failures, but I'm not sure where; I don't think this is the correct forum).

As others have pointed out, Strawberry Perl is more than just the latest perl, so I kept the rest of the distribution, swapping the original strawberry/perl folder with the new one. I bootstrapped cpanm by copying the fat-packed script over as cpanm.pl and running

perl cpanm.pl App::cpanminus

After that, I could build and install other modules in the usual way, e.g.

cpanm Moo

I did hit a snag with namespace::clean where it complained about the missing libstdc++-6.dll. I guess this is where SP is more than just perl... Any advice on that? I just copied that and the other additional DLLs over from SP 5.32 and re-ran cpanm. That appeared to work and some simple scripts are running fine, but I just worry that I've left a ticking time bomb in there!

@sisyphus
Copy link

sisyphus commented Jul 17, 2022 via email

@blues1875
Copy link

I just hit another problem with my new 5.36 build: Perl-Critic wouldn't build because of a problem with Module-Pluggable, provoked by an update to File-Find. See simonwistow/Module-Pluggable#24

@sisyphus
Copy link

Interesting. There's a number of Linux smokers that would have detected this problem when File-Find-1.39 was first introduced to bleadperl ...., if it was something that affected Linux.
But, of course, there are no such Windows smokers, so it goes undetected until a user actually hits the issue some time later.

I gather that the problem is not that File::Find is ding something it shouldn't ... instead it's a case of Module::Pluggable making an assumption that it shouldn't ?

It took me a bit longer than it should have to work out what the patch at simonwistow/Module-Pluggable#24 actually was.
For others who are also a bit slow, it's:

--- Object.pm_orig	2022-07-25 11:42:24 +1000
+++ Object.pm	2022-07-25 12:12:02 +1000
@@ -66,6 +66,7 @@
     $self->{'on_instantiate_error'} ||= sub { my ($plugin, $err) = @_; carp "Couldn't instantiate $plugin: $err"; return 0 };
 
     # default whether to follow symlinks
+    $self->{'follow_symlinks'} = 0 if ($^O eq 'MSWin32' && !exists $self->{'follow_symlinks'});
     $self->{'follow_symlinks'} = 1 unless exists $self->{'follow_symlinks'};
 
     # check to see if we're running under test

The author of Module::Pluggable hasn't released anything on CPAN for nearly 7 years, so I'm not sure how long we might be waiting for the patch to be implemented and a fixed version of Module::Pluggable released.

Cheers,
Rob

@blues1875
Copy link

This has been quite an interesting exercise! I think you are right to encourage others to build their own perl... the devil is in the details, as always! I had problems with Module::Find, too. That module also uses File::Find and is related to symlinks, but that one is a bit harder to figure out. Once SP 5.36 (or even 5.34) is released, I'm sure more people will find more of these problems, but I understand that there is a lot of work involved with putting together a new release of SP--many thanks to you guys. I hope my experience helps, if just a bit. Building our own perl and finding these glitches is a good way for the rest of us to contribute.

Glenn

@sisyphus
Copy link

I suspect that the "real spanner in the works" (and the the thing that needs fixing) is File::Find.
However, I don't really know.
The telling part of the diff between File-Find-1.37 and File-Find-1.39 is:

....
-our $VERSION = '1.37';
+our $VERSION = '1.39';
 require Exporter;
 require Cwd;
 
@@ -161,9 +161,8 @@
     $pre_process       = $wanted->{preprocess};
     $post_process      = $wanted->{postprocess};
     $no_chdir          = $wanted->{no_chdir};
-    $full_check        = $Is_Win32 ? 0 : $wanted->{follow};
-    $follow            = $Is_Win32 ? 0 :
-                             $full_check || $wanted->{follow_fast};
+    $full_check        = $wanted->{follow};
+    $follow            = $full_check || $wanted->{follow_fast};
     $follow_skip       = $wanted->{follow_skip};
     $untaint           = $wanted->{untaint};
     $untaint_pat       = $wanted->{untaint_pattern};
@@ -840,6 +839,9 @@
....

It doesn't feel right to me that such changes have been made, given that they're breaking both Module::Pluggable and Module::Find. (And I think it's quite possible that the developers who made those changes are not even aware of those breakages.)

Should we question the validity of those changes to Find.pm by raising an Issue at https://github.com/Perl/perl5/issues ??

Cheers,
Rob

@genio
Copy link
Member

genio commented Jul 25, 2022

@sisyphus You might want to float this in #p5p on magnet and get some conversation going there before putting in a bug report. They'll help make sure it has all of the info int it that it needs

@bekuno
Copy link

bekuno commented Jul 25, 2022

The change was intentional to support Win32 symlinks, see
Perl/perl5@0d00729

@jkeenan
Copy link

jkeenan commented Jul 25, 2022

Should we question the validity of those changes to Find.pm by raising an Issue at https://github.com/Perl/perl5/issues ??

Well, you're getting pretty far away from the original complaint raised in this issue, so you should probably start a fresh ticket somewhere. If you believe the problem is in the Perl core distribution -- as distinct from Strawberry Perl or some (non-core) CPAN module, then you should probably check your results both before and at this commit:

$ gitshowf 0d00729c03a
commit 0d00729c03a1f68e1b51e986d1ce9000b0e3d301
Author:     Tony Cook <[email protected]>
AuthorDate: Wed Oct 7 12:07:31 2020 +1100
Commit:     Tony Cook <[email protected]>
CommitDate: Tue Dec 1 15:29:33 2020 +1100

    File::Find: support Win32 symlinks
    
    find.t, taint.t: check that symlink() works under the current
    permissions/filesystem rather than assuming it will work
    
    find.t: since symlinks are now available, an earlier test block
    set $FileFileTests_OK, and the tests in this Win32 block don't use
    either of the follow options, which is required for fast file tests.
    
    taint.t: ensure we get "/" separated names to match File::Find's output

@blues1875
Copy link

Here is another issue with perl 5.36, @sisyphus. Path-Class fails build tests. See kenahoo/Path-Class#55.

Path-Class is a dependency for DBIx-Class-Schema-Loader.

@sisyphus
Copy link

sisyphus commented Aug 1, 2022

Path-Class fails build tests.

Yeah ... there's an issue with t/01-basic.t on Windows whereby tests like:
ok $file3, 'dir/foo/bar.txt';
are failing because $file3 is actually 'dir\foo\bar.txt'.

To overcome this, t/01-basic.t begins with:

BEGIN {
  $^O = 'Unix'; # Test in Unix mode
}

and that takes care of that particular issue nicely. (There's at least one other Windows-vs-Unix incompatibility that it also attends to.)
Unfortunately, on perl-5.34.0 and later, it also leads to the fatal error "Can't call method "relative" on an undefined value ...".

I think the first thing that needs to be established is whether this hack of overriding $^O is deemed a valid technique.
(I think this means raising another Perl5 Issue.)
If it's not valid, then an alternative method of correctly performing the tests is required.
Otherwise, the functionality that allowed the $^O override to work in perl-5.32 and earlier needs to be restored.

My reading of the "$^O" documentation in perlvar leads me to think that this technique is invalid - but there's no specific forbidding of overriding it.

Have a play around with removing the $^O override in t/01-basic.t. And check it on both perl-5.32 and perl-5.36.
What do you think ?

Cheers,
Rob

@blues1875
Copy link

I removed the $^O override. It doesn't bale out now on the fatal error, but it does fail 30 if the 78 tests with lots of complaints about backslashes like this:

# Test 6 got: "dir\\bar.txt" (t/01-basic.t at line 20)
#   Expected: "dir/bar.txt"

I guess I should report this over on Path-Class issues.

@sisyphus
Copy link

sisyphus commented Aug 2, 2022 via email

@kenneth-olwing
Copy link

Hi all,

I also have an interest in seeing new Strawberry dists...OR, in fact I may consider building my own Perl dist given the insights gained here. I already have a complete build process to do that on Linux, while it currently piggybacks on Strawberry, and adds a huge number of modules for our inhouse deployable Perl in order for me to ensure I have 'the same' stuff on our platforms.

Regardless of that, I'm interested in helping in various areas to the extent I'm able, with the goal of getting new Strawberry dists out. If that helps me in 'building 'my own dist', so much the better ;-). Since there's such a multitude of things to be aware of, the more of us the merrier I think when ferreting all the itty-bitty issues out in the open...

So in regard to that I seem to have a few other issues than already discussed above which I would like to compare notes on with others.

======

I currently see two issues with building perl-5.36.0 ('raw', figure this is best before trying to tackle the Perl::Dist::Strawberry build system):

For reference, I use the 'CCHOME' in a portable Strawberry 5.32.1 to build, a freshly unpacked perl-5.36.0.tar.gz and

gmake CCHOME=c:\ken1\slask\straw\SP-5.32.1.1\c INST_TOP=c:\ken1\slask\straw\ken1-perl-5.36.0 USE_MINGW_ANSI_STDIO=define

(note that using 5.34, I was forced to add "USE_64_BIT_INT=define" despite it shouldn't be needed...)

  1. Build error in ext/Errno

NOTE: this problem will be reported on the https://github.com/Perl/perl5, but if you want to build 5.36.0, it can be patched to work around it. I haven't been able to test yet, but it seems that blead as of now should have the same problem.

During build of ext/Errno, the Errno_PL.PM is invoked, which basically as I understand it, collects all the possible include files in
the CCHOME, creates a tmp .c file which #include's them all, runs gcc to have it spit out all #defines, and then interprets that to find all possible error codes to create Errno.pm with. I think.

The problem (which apparently happens only with 5.36.0 [?]) is that the gcc run commonly errors out with something like this:

gmake[1]: Entering directory 'C:/ken1/slask/straw/perl-5.36.0/ext/Errno'
"C:\ken1\slask\straw\perl-5.36.0\miniperl.exe" "-I..\..\lib" Errno_pm.PL Errno.pm 
In file included from includes.c:2:
c:/Users/ken1/slask/straw/SP-5.32.1.1/c/lib/gcc/x86_64-w64-mingw32/8.3.0/include/avx5124fmapsintrin.h:25:3: error: #error "Never use <avx5124fmapsintrin.h> directly; include <x86intrin.h> instead."
 # error "Never use <avx5124fmapsintrin.h> directly; include <x86intrin.h> instead."
   ^~~~~

Unfortunately, this isn't trapped by the code (nor will the make file trap it even if the code would exit with an error exitcode).
It should also be noted that sometimes it doesn't happen, or you may get other variations of this
error: there's a number of include files that is coded in this manner, and since they are detected initially using a hash, the order in which they get listed as includes is not defined. It should be noted that there is an attempt at fixing this in the Errno_PL.PM code, it's just that it's only good for one or two particular include files (see Perl/perl5#18026 for this).
I'm unsure if this error actually makes a big difference, but still, it shouldn't happen.

If someone is working on a perl-5.36.0.pp file in P::D::S, this problem can be patched by making the file list be treated in
sorted order (no, I'm aware it's not a proper fix, it just happens to make the build happy as it makes the include files
come in acceptable and dependable order...):

diff --git a/ext/Errno/Errno_pm.PL b/ext/Errno/Errno_pm.PL
index ee2f4a3..2ad6cce 100644
--- a/ext/Errno/Errno_pm.PL
+++ b/ext/Errno/Errno_pm.PL
@@ -21,7 +21,7 @@ if ($Config{gccversion} ne '' && $^O eq 'MSWin32') {
     my %seen;
     open INCS, '>', 'includes.c' or
        die "Cannot open includes.c";
-    foreach $file (@files) {
+    foreach $file (sort @files) {
        next if $file eq 'errno.c';
        next unless -f $file;
        if ( $file eq 'avx512vpopcntdqvlintrin.h' || $file eq 'avx512bwintrin.h' ) {

  1. Test errors
Test Summary Report
-------------------
re/reg_mesg.t                                                      (Wstat: 65280 (exited 255) Tests: 339 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
re/regexp_unicode_prop_thr.t                                       (Wstat: 65280 (exited 255) Tests: 4 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 1110 tests but ran 4.
porting/authors.t                                                  (Wstat: 0 Tests: 1 Failed: 1)
  Failed test:  1
porting/pending-author.t                                           (Wstat: 0 Tests: 1 Failed: 1)
  Failed test:  1
porting/regen.t                                                    (Wstat: 512 (exited 2) Tests: 15 Failed: 1)
  Failed test:  15
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 42 tests but ran 15.
win32/popen.t                                                      (Wstat: 2304 (exited 9) Tests: 0 Failed: 0)
  Non-zero exit status: 9
  Parse errors: No plan found in TAP output
../dist/Storable/t/utf8.t                                          (Wstat: 65280 (exited 255) Tests: 5 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 6 tests but ran 5.
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2766, Tests=1147620, 4469 wallclock secs (80.92 usr + 155.01 sys = 235.94 CPU)
Result: FAIL
gmake: *** [GNUmakefile:1810: test] Error 3

Some are inconsequential, but some individual errors, as well as some test runs exiting with 255 indicating some kind of core (seems to be unicode/utf8 related stuff though...some C libs that are broken on win32?).

Haven't tried to follow up on them, but anyone seeing the same and/or have some ideas on why?

===========

Lastly...this issue seems to be slightly overcrowded in discussing issues a bit beyond tracking the original subject, clouding it a bit. And I do the same...:-)

Is there a more convenient forum to discuss selfbuilding perl on win32 considering this? In particular to jointly help with the wish to see new Strawberry dists.

ken1

@sisyphus
Copy link

sisyphus commented Aug 3, 2022

(note that using 5.34, I was forced to add "USE_64_BIT_INT=define" despite it shouldn't be needed...)

Oooo ... alarm bells ... this should only be the case if the gcc compiler is 32 bit.
And if you are using a 32-bit gcc compiler on a 64-bit Windows system then you'll need to add 'WIN64=undef'.

I have struck nothing remotely like the errors you are reporting.

It's probably a good idea to create a perl5 issue about this - rather than further pollute this thread. (I will follow that issue with interest.)
When you do create that Issue, please provide the output of running gcc --version, as I feel that's something we'll need to see.
If the one line of output produced by that command matches x86_64 then it's a 64 bit compiler ... else it's a 32-bit compiler,
Also, make sure you identify which version of Windows you're running - as that could also be relevant.

Cheers,
Rob

@kenneth-olwing
Copy link

kenneth-olwing commented Aug 3, 2022

Rob, thanks for your comment.

Re: polluting this thread (in general) - I agree; any suggestions where these type of postings could/should be made 'best'? Having said that I'll give some short replies here:

Re: my first problem, I've just filed Perl/perl5#20033; for me it's quite real and very reproducible and I'm guessing you actually can recreate it too; just run the Errno_pm.PL script a number of times per my example if you're curious. Then again, no biggie.

Re: 5.34 and USE_64_BIT_INT - I'm 64 bit, Strawberry 5.32.1, Win11 as build toolchain and certainly see that gcc, just as with 5.36 so it's confusing - as you say it shouldn't be necessary. I'll retest and ensure reproducibility and file an issue,,,

Finally: apart from the tests you mention giving problems (which they don't for me!), are you otherwise getting a clean test run? Only major difference (?) I can think of is that I'm using Win11...I'll soon be firing up my work computer with Win10 and try things there but...I'm certainly curious what I might be doing different/wrong...:-/

Thanks again,
ken1

@genio
Copy link
Member

genio commented Aug 3, 2022

While I don't mind having this issue open for you to keep the information somewhere, it might be helpful for you to join the #p5p IRC channel as they'd have the most expertise in building Perl in general.

@kenneth-olwing
Copy link

Thank you!

@sisyphus
Copy link

sisyphus commented Aug 3, 2022

I can reproduce the behaviour with perl Errno_pm.pl, but in building a couple of hundred different configurations of current perls on this windows box over the last couple of years it's not something that has ever cropped up.
(Once perl has been built, I'm not in the habit of ever running perl Errno_pm.pl :-)

I'll keep an eye on Perl/perl5#20033 and chime in if I've got anything to add - though I might not get a chance until early next week.

I've double-checked on the perl-5.36.0 that I built with Strawberry's gcc-8.3.0 toolchain. I definitely didn't explicitly define USE_64_BIT_INT, though $Config{use64bitint} reports 'define', $Config{ivtype} is 'long long' and $Config{ivsize} is 8.
The GNUmakefile contains:

ifeq ($(WIN64),define)
USE_64_BIT_INT	= define
endif

Unless that condition is evaluated as false, I don't see how USE_64_BIT_INT could be anything other than 'define'.

Cheers,
Rob

@shawnlaffan
Copy link
Contributor

New dev release for 5.36.1, compiled using gcc-13.1, is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5361_20230506_gcc13

@shawnlaffan
Copy link
Contributor

A second dev 5.36.1 dev release is now available from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5361_20230524_gcc13

The main change is that PDL now builds.

The list of open issues is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/labels/perl5.36

The ones I think we need to resolve for a proper release are BerkeleyDB and DBD::Oracle.

@PhilterPaper
Copy link

Regarding BerkeleyDB, I don't use it (it came preinstalled with Strawberry), but have successfully upgraded it in 5.24, 5.26, and 5.32.

Regarding DBD::Oracle, I don't use it (it came preinstalled with Strawberry), and have never been able to upgrade it. It appears to be missing some sort of Oracle client. I think consideration should be given to omitting it entirely, as most systems are not going to have Oracle installed by default. Ditto for DBD::Pg, as most systems are not going to have PostgreSQL installed by default. At worst, perhaps you could have dummy stubs for DBD::Oracle and DBD::Pg just to tell the user that Perl support is available, but not attempt an upgrade (e.g., an impossibly high version number, and instructions on how to install the real thing)?

It would be interesting to see if IPC::Run will install and upgrade, as I have not been able to upgrade it since 0.96. Math::MPFR chronically fails to upgrade earlier than 5.32, but as of 5.32 it appears to be OK. Finally 'B' only upgrades on the current (5.36?) Perl -- I have asked CPAN to consider putting in some guards so cpan doesn't try to upgrade it on older Perls. I don't know if you can do anything on your end. I don't use any of these packages, so it's merely a nuisance that the cpan tool tries (and fails) to upgrade them (I get around that by having a wrapper to exclude problem packages).

@shawnlaffan
Copy link
Contributor

Thanks @PhilterPaper

The issue with BerkeleyDB is that it does not build against libdb built with gcc-13. Currently the SP5.36 dev releases are re-using the libdb from SP5.32. This works but some builds show warnings about not being able to find gcc-8 files, warnings I am assuming come from libdb files.

WRT DBD::Oracle I'm starting to think the same way as you. If I understand correctly, the license means we cannot redistribute its binaries. They are not provided with SP5.32, and probably earlier. If DBD::Oracle builds cleanly when the binaries are available (and the user sets the right env vars) then we perhaps don't need to provide it.

WRT DBD::Pg, Strawberry Perl provides the Postgres binaries. The possible issue is when the user tries to use its PL/Perl interface. FWIW, it seems not to be exercised in the DBD::Pg tests.

SP5.36 come with the latest IPC::Run.

There is also the spectre of build failures due to changes in gcc-13 (probably anything since gcc-8.3). We already patch in a number of perl 5.37 changes to get things to build (e.g. PDL), and it is possible some other modules will also show similar failures (maybe those for Tk are a manifestation of this). This can perhaps be considered just a normal part of the build cycle and thus not a blocker to any SP releases as those modules will need to update eventually anyway.

@sisyphus
Copy link

Math::MPFR chronically fails to upgrade earlier than 5.32,

Now fixed in the github repo (https://github.com/sisyphus/math-mpfr), and therefore will be fixed in Math-MPFR-4.27 when it is released.
At least, that (github) version of Math::MPFR now builds and installs fine for me on Strawberry Perl-5.24.4 - but if there are yet other problems to be dealt with, please open an Issue at https://github.com/sisyphus/math-mpfr/issues.

Cheers,
Rob

@PhilterPaper
Copy link

There is also the spectre of build failures due to changes in gcc-13 (probably anything since gcc-8.3). We already patch in a number of perl 5.37 changes to get things to build (e.g. PDL), and it is possible some other modules will also show similar failures (maybe those for Tk are a manifestation of this). This can perhaps be considered just a normal part of the build cycle and thus not a blocker to any SP releases as those modules will need to update eventually anyway.

That worries me. Ideally, you should not custom patch any package to get it to build on Windows/Strawberry, as opposed to a patch (to that package's basic source) that is universally distributed and works on all platforms, or at least, can conditionally build on Strawberry. There's no point in shipping Strawberry with a custom build of some package, when the package's next update on CPAN will then fail. At least, that's how I'm reading (misreading?) the quoted post.

Patches to Perl itself, to get it to play nice on Windows, are another matter. Presumably no one with Strawberry Perl installed is going to upgrade their Perl with a non-Strawberry replacement, so as long as Strawberry Perl language and behavior meets specs for Perl programs, that should be OK. There may still be the issue of a program or package developed on Strawberry, failing to run properly on non-Strawberry platforms, but hopefully that can be avoided by either tweaking Strawberry Perl, or doing something with the OS detected in the program.

TL;DR: if a package on CPAN can be built (without customization) for shipping with Strawberry, it should be possible to install/update directly from CPAN, and work.

@shawnlaffan
Copy link
Contributor

TL;DR: if a package on CPAN can be built (without customization) for shipping with Strawberry, it should be possible to install/update directly from CPAN, and work.

@PhilterPaper - We're in agreement.

Exceptions for patching apply when a module has a reasonable number of dependencies (particularly in the SP builds), the issues have been reported upstream, and where a new release seems unlikely or not imminent. The four I am building patched versions of are File::Find::Rule, Socket6, Pod::Parser and String::Escape.

There are four others that have been patched for previous builds and which I have not looked into in any detail, except to note most do not have recent releases on CPAN. To see them, search for perl-modules-patched in the build config.

One point to note from the build config is that tests are skipped, in whole or in part, for many of the modules. Some of these are failures only on docker. Others I have not looked into but I do think it is reasonable to trust the decisions of the previous maintainers.

@PhilterPaper
Copy link

Good point about the dependencies chain -- I hadn't really considered that. If package X won't build, knocking dependent packages out of the build, and so on and so on, any idea how much of the package list would have to be removed in all?

I presume that you're keeping an eye out for problems in the tools themselves, rather than a fault in the end package itself? Compiler and linker bugs, library bugs, etc. Of course, getting such a bug fixed may set off a chain of failures in packages that were "fixed" to get around a tool bug in the first place!

@shawnlaffan
Copy link
Contributor

shawnlaffan commented Jun 5, 2023

I presume that you're keeping an eye out for problems in the tools themselves, rather than a fault in the end package itself?

I'm watching the winlibs repo if that's what you mean. I also have the occasional look at the gcc development but am happy leaving that to the experts. Edit - and winlibs has a new gcc-13.1 release which we need to try StrawberryPerl/spbuild#17

WRT the perl dependency chains, we build more than 230 packages from CPAN, plus their dependencies. The Task::Kensho checks in #103 have also been useful. This gives a pretty good sample.

The main breakages for SP 5.36.1 are now related to symlinks and directory separators. The latter is easily worked around by path normalisation. The latter might need more investigation.

There are other issues that might be related to memory alignment but which seem to be Strawberry specific. #102, #98, #26 (comment) (Edit - I meant don't seem to be Strawberry specific).

@shawnlaffan
Copy link
Contributor

I've just uploaded a new dev release. https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5361_20230609_gcc13

The main change is that we now use release 5 of the winlibs gcc-13 stack. Compilation using its gcc is substantially faster than previous versions.

Unless some substantive new issues are identified I am thinking the next will be a proper release.

@shawnlaffan
Copy link
Contributor

Perl 5.38.0 RC1 builds with the same test failures as 5.36.1 (#60), which is good news.

A portable version is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5380RC1_20230618_gcc13 if people want to test their favourite modules.

PDL and Inline::C work. I have not tested the Task::Kensho list yet. #103 (comment)

@shawnlaffan
Copy link
Contributor

@shawnlaffan
Copy link
Contributor

A version of Strawberry Perl 5.38.0 is at https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5380_20230704_gcc13

@genio - If no issues are reported then we could likely promote those to final for 5.38.0. A 5.36.1 can follow, and perhaps a 5.34 if there is an actual need and the patches we are applying to 5.36 apply cleanly.

@shawnlaffan
Copy link
Contributor

Discussion on this topic has generalised to any release of Strawberry perl, and now that 5.36.1 and 5.38.0 are out the original intent (release a new version) would seem to be satisfied.

I've opened #124 for a 5.34 specific release.

I will close this issue in a few days unless there is a need to keep it open.

@PhilterPaper
Copy link

That's wonderful news! I look forward to both versions showing up on StrawberryPerl.com for download and trying out one or both (at least, 5.38). I might wait a few days to see if 5.38.1 comes out; usually .0 releases of any software need to be updated fairly quickly. Thanks for all the effort you've put in!

@shawnlaffan
Copy link
Contributor

5.38.1 is likely quite some time off. For example, 5.36.1 was about 11 months after 5.36.0.

If you want to avoid .0 releases then Strawberry Perl 5.36.1.1 is available.

The website is a work in progress as we don't yet have access. In the meantime you can download from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/SP_5380_5361

@gregoa
Copy link

gregoa commented Jul 27, 2023 via email

@genio
Copy link
Member

genio commented Jul 28, 2023

We haven't made any plans to build and test a 32-bit version at this point in time. That being said, are there real world scenarios out there of 32-bit Windows in the wild? Note that this isn't intended to come off as a criticism or plain snide in any way, but a genuine question.

I mean, I know there are some "can't ever ever ever upgrade that windows 7 machine that's running my CNC machine" and stuff like that, but are those edge cases actually running/using Perl? Even if so, are those users brave enough to update anything on those machines?

I'm asking because it's quite a bit of work. We'd need to rebuild everything with a 32-bit compiler and go through all the testing and whatnot again. That's a lot of work if it's not something used but just kind of, sort of available on someone's testing rig.

@shawnlaffan
Copy link
Contributor

A cut down 32-bit version (fewer external libs and perl modules) would likely be sufficient for most needs but it does still involve time on our part.

Discussion for a possible 32 bit release is under #96

@kenneth-olwing
Copy link

Let me first say:

===

Thanks to everyone involved for the incredible amount of work I suspect has been necessary in getting this on track again!

===

It is hard to overstate the importance of having a Strawberry dist for me to build on in terms of serving several hundred users in my organization, on both Windows and Linux where I try to make available Perl installations that are as "identical" as possible, from Perl base version to collection of modules. This in order to be as sure as possible that the multiplatform apps and tools we have can be expected to work "everywhere".

I have for the past month been busy getting my build system on track for creating the above with 5.38 as the base (building from source on Linux, but "cheating" on Windows by leveraging Strawberry, especially for getting the toolchain and many other libs sorted out). The last I did this with was 5.30 a few years back.

Mostly this has been working fine, but I note some modules that I know installed on 5.30 but not now. I'm not organized yet to track things down and properly compare whys and hows, but I expect to report a few later here for a first diagnosis.

Again, kudos!

ken1

@shawnlaffan
Copy link
Contributor

You're welcome.

In terms of the modules, we do have several open issues for those we know to have errors. There is also the Task::Kensho list which is part of a large issue thread but the status of modules is in this comment: #103 (comment)

@gregoa
Copy link

gregoa commented Jul 29, 2023 via email

@sisyphus
Copy link

sisyphus commented Jul 29, 2023

And this also means that I wouldn't need a full-blown all-batteries-included Strawberry Perl but just a perl core Windows
32-bit build

The Strawberry Perl developers (of which I am not one) might be too polite to point out that it's really quite easy and straightforward to build such a perl yourself.

To put it bluntly and simply, all you need is the gmake utility, the perl-5.38.0 source, and a 32-bit mingw-w64 port of gcc such as
CORRECTION: At this point I originally provided a link to a winlibs UCRT compiler - but for 5.38.0 we need an MSVCRT compiler.
Here is the corrected link
https://github.com/brechtsanders/winlibs_mingw/releases/download/13.1.0-16.0.5-11.0.0-msvcrt-r5/winlibs-i686-posix-dwarf-gcc-13.1.0-mingw-w64msvcrt-11.0.0-r5.zip (That's what SP will probably be using for any 32-bit builds they do release ... though, of course, that's not my call.)

UPDATE: One other thing I initially failed to mention in this post is that the changes made by Perl/perl5#21142 need to be made, (The diff is at Perl/perl5@89d9037.)
Having unzipped the source and applied PR21142, it's then just a matter of cd'ing to the win32 directory (in the cmd.exe shell) and running gmake args test followed by gmake args install

If you need assistance with specifying "args" correctly, just post back here (or even contact me directly) and help will be forthcoming.
We'd need to know:

  1. do you want 32-bit or 64-bit integer (IV);
  2. what nvtype do you want (double/long double/__float128);
  3. into what location is perl to be installed;
  4. what is the full path to your mingw-w64 "bin" folder.

Mind you, I'm hoping that SP does release a 32-bit perl - because perl should still support such architectures, and having 32-bit SP builds out and about will help keep 'em honest and on their toes.
AFAICT, the only point in having a 64-bit perl is if you need to have 64-bit pointers. (AFAIK, all scripts that I've ever written work just as well if pointers are 32 bits.)

UPDATE: The PR at Perl/perl5#21324 resolves the following issue with winlibs compilers. With those patches in place, winlibs compilers produce a 32-bit quadmath build of perl, with all tests passing - and the next paragraph can be ignored.

In the unlikely event that you want to build a 32-bit perl with nvtype of __float128, don't use the winlibs compilers (as they fail numerous threads-related tests).
For 32-bit quadmath builds, I've found that https://gcc-mcf.lhmouse.com/mingw-w64-gcc-mcf_20230419_13.0.1_x86-msvcrt_92844459e964d9e5547d3e3662188bda9b664e4e.7z builds a 32-bit quadmath perl-5.38.0 that passes all tests.

Disclaimer: I don't really want to make any claims about Windows Vista's capability to build perl. I think it should be ok, but I was unfortunate enough to once have a Vista system, and I well remember that it was a shitheap.

Cheers,
Rob

'

@shawnlaffan
Copy link
Contributor

I'll close this issue now, as promised in #39 (comment)

If there are additional problems then please open a new issue. If there is a general question then the discussion forum can be used. https://github.com/StrawberryPerl/Perl-Dist-Strawberry/discussions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests