add python 3.12 to the build and drop python 3.8 #4716

ahnitz · 2024-04-25T02:35:42Z

Motivation

This moves pycbc to 3.09 as the minimum python version. This is consistent with upstream numpy release schedule which we in principle try to sync to. This will reduce the number of tests and allow people to being using some more modern python structures. Python 3.12 is also added to the test infrastructure and build.

Links to any issues or associated PRs

Testing performed

The CI will hopefully provide the needed tests. Iteration may be needed to ensure the various env builds all work now.

The author of this pull request confirms they will adhere to the code of conduct

ahnitz · 2024-04-25T03:17:24Z

Ok, I'm tracking through various incompatibilities. One issue is the installation of sbank. We need newer wheels. I've started a PR to see if this is easy on the sbank end.
gwastro/sbank#57

ahnitz · 2024-04-25T13:34:14Z

Ok, there a couple other problems here. (1) there are some places we are not actually 3.12 compliant, I am going through to fix some of these. (2) the docker / venv builds need to be moved to a later python version to match (currently broken). (3) I am seeing some build issues with some upstream libraries. These might be workable, but I'm having trouble actually getting some libraries like ligo.lw to actually build locally for testing (somehow seems to work on github??).

titodalcanton · 2024-04-25T14:15:47Z

@maxtrevor note that we are using 3.9 for PyCBC Live at the moment; if we decide to move to a PyCBC version that includes this change, we will probably need to rebuild the testing and production envs from scratch.

ahnitz · 2024-04-25T16:13:09Z

Ok, I'll revert dropping python 3.9.

spxiwh · 2024-05-07T15:15:19Z

SBank python 3.12 wheels are available now.

ahnitz · 2024-06-19T15:46:14Z

Ok, I think for this to make the most sense we need to
(1) drop python 3.8 / 3.9 support
(2) build pycbc with numpy 2.0 as recommended https://numpy.org/devdocs/dev/depending_on_numpy.html#adding-a-dependency-on-numpy
(3) use < 2.0 for numpy for all testing until upstream packages are stabilized with numpy 2.0 support. Many do not seem to be working yet.
(4) fix up the venv / docker builds (can someone help here who understands these??) @spxiwh

ahnitz · 2024-06-24T20:16:12Z

Note, at the moment, docker / venv are kept at python 3.9, but this seems to work and notably numpy is making releases for python3.9 with numpy 2.0. I don't expect this to last, so it would be good to advance these versions at some point, but I think we can postpone dropping 3.9 if needed.

ahnitz · 2024-07-22T21:09:11Z

@spxiwh Can you please review? There is one blocking issue now, which is that the tutorials need to be updated to work and that is a blocking issue for merging, however, I don't think that should involve changes to pycbc unless we want to keep monkey patching emcee 2 to work. I think it's the right time to drop though for future python versions.

@cdcapano and I will need to address
gwastro/PyCBC-Tutorials#34

But it would be good to get any feedback on the remainder of the PR or if we are otherwise good to go once the tutorial tests pass (which again should just be a rerun of the CI once we address the tutorials themselves).

titodalcanton · 2024-07-23T12:41:12Z

#4821 seems related to this.

spxiwh

This all looks reasonably clear, and sensible. I left some minor comments, but am happy to approve once tests work.

spxiwh · 2024-07-27T12:56:40Z

.github/workflows/basic-tests.yml

@@ -13,7 +13,7 @@ jobs:
      max-parallel: 60
      matrix:
        os: [ubuntu-20.04]
-        python-version: [3.8, 3.9, '3.10', '3.11']


Why are we dropping 3.9 from the basic tests as well?

I'll add this back. 3.9 is past EOL, but we can remove later after the docker / venv images are also updated and pycbc live is moved to a newer python version. I believe those are the hangups for 3.9.

spxiwh · 2024-07-27T12:59:34Z

bin/all_sky_search/pycbc_cut_merge_triggers_to_tmpltbank

+import importlib.util
+import importlib.machinery
+
+def load_source(modname, filename):


Can we move this function to some appropriate module, rather than defining in executables?

This is duplicated in a few places, and this avoids that ...

spxiwh · 2024-07-27T13:00:41Z

pyproject.toml

-            "numpy==1.23.4; python_version=='3.11'",
-            "numpy; python_version >= '3.12'",
+            "cython>=0.29.21",
+            "numpy>=2.0.0",


Should we change PR title to indicate that this now makes pycbc require numpy 2 as well as dropping python versions?

Is it actually necessary to require Numpy 2? As far as I understood it was supposed to maintain some level of backward compatibility.

We don't require numpy 2.0. However, to work with 2.0 it must be built with 2.0. They have added in 2.0 the ability for ABI to be backwards compatible, so when it is built with 2.0 it will still work with 1.XX.

See

https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-0-specific-advice

That's why i did not update the setup.py or requirements file at hte same time, only the pyproject.toml which dictates how the build itself is done.

spxiwh · 2024-07-27T13:01:27Z

requirements-igwn.txt

@@ -1,4 +1,5 @@
 # For LDG service access
+ligo-proxy-utils


What is this needed for?

@spxiwh That's a question for you all actually. If it's not needed we can remove it. However, it was hard coded in the workflow whereas if it is needed it should be in the requirements rather than installed by hand.

* drop python3.7, 3.8, 3.9 * add missing quotes * banish distutils * banish p39 in more places * tox needs some things early * move here * maybe? * try el9 * must use 311 for el9 * this might be the right name * updates * try this * more stuff * update * do not use imp module * more places where imp was used * bring back 39 * more reverts * update * update * fixes * this should be here * cc * fix web url * add back 3.9 to basic tests * centralize python12 backwards compatibility function * cc * simplify setup.py, ensure math is linked * cc

ahnitz requested a review from spxiwh April 25, 2024 02:37

ahnitz added the work in progress label Apr 25, 2024

ahnitz force-pushed the p12 branch from ad7a7e2 to 8080c3b Compare June 19, 2024 15:21

ahnitz force-pushed the p12 branch from ea3da6b to 73eb94b Compare June 26, 2024 15:43

ahnitz changed the title ~~add python 3.12 to the build and drop python 3.8, 3.9~~ add python 3.12 to the build and drop python 3.8 Jul 22, 2024

ahnitz force-pushed the p12 branch from b7b1a65 to 4df285a Compare July 22, 2024 20:35

ahnitz mentioned this pull request Jul 22, 2024

All tutorials should drop use of emcee_pt gwastro/PyCBC-Tutorials#34

Open

ahnitz removed the work in progress label Jul 22, 2024

spxiwh approved these changes Jul 27, 2024

View reviewed changes

ahnitz added 12 commits July 29, 2024 15:13

drop python3.7, 3.8, 3.9

4ccb200

add missing quotes

2885b47

banish distutils

8530850

banish p39 in more places

7695c5d

tox needs some things early

00b26ab

move here

5f74135

maybe?

8daba3d

try el9

15e7321

must use 311 for el9

283472b

this might be the right name

a887f5a

updates

e90f2a6

try this

583e124

ahnitz added 14 commits July 29, 2024 15:13

more stuff

8ff2e03

update

e2e5992

do not use imp module

2206e4b

more places where imp was used

3848bf0

bring back 39

b52231b

more reverts

46988c7

update

180f085

update

8a66dca

fixes

549ae44

this should be here

6a3ace0

cc

d94bd19

fix web url

2b9b52b

add back 3.9 to basic tests

2b25d21

centralize python12 backwards compatibility function

b9d35b2

ahnitz force-pushed the p12 branch from 4df285a to b9d35b2 Compare July 29, 2024 19:17

ahnitz added 2 commits July 29, 2024 15:35

cc

92bb4db

simplify setup.py, ensure math is linked

8cf57e4

ahnitz force-pushed the p12 branch from 28814d4 to 8cf57e4 Compare July 30, 2024 00:10

cc

319ec0b

ahnitz merged commit 013bdd1 into gwastro:master Jul 30, 2024
30 checks passed

This was referenced Jul 30, 2024

allow numpy 2.0 to run #4829

Merged

don't forget to actually deploy the documentation #4830

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add python 3.12 to the build and drop python 3.8 #4716

add python 3.12 to the build and drop python 3.8 #4716

ahnitz commented Apr 25, 2024 •

edited

Loading

ahnitz commented Apr 25, 2024

ahnitz commented Apr 25, 2024

titodalcanton commented Apr 25, 2024

ahnitz commented Apr 25, 2024

spxiwh commented May 7, 2024

ahnitz commented Jun 19, 2024

ahnitz commented Jun 24, 2024

ahnitz commented Jul 22, 2024

titodalcanton commented Jul 23, 2024 •

edited

Loading

spxiwh left a comment

spxiwh Jul 27, 2024

ahnitz Jul 29, 2024

spxiwh Jul 27, 2024

spxiwh Jul 27, 2024

spxiwh Jul 27, 2024

titodalcanton Jul 29, 2024 •

edited

Loading

ahnitz Jul 29, 2024

spxiwh Jul 27, 2024

ahnitz Jul 29, 2024

add python 3.12 to the build and drop python 3.8 #4716

add python 3.12 to the build and drop python 3.8 #4716

Conversation

ahnitz commented Apr 25, 2024 • edited Loading

Motivation

Contents

Links to any issues or associated PRs

Testing performed

ahnitz commented Apr 25, 2024

ahnitz commented Apr 25, 2024

titodalcanton commented Apr 25, 2024

ahnitz commented Apr 25, 2024

spxiwh commented May 7, 2024

ahnitz commented Jun 19, 2024

ahnitz commented Jun 24, 2024

ahnitz commented Jul 22, 2024

titodalcanton commented Jul 23, 2024 • edited Loading

spxiwh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

titodalcanton Jul 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ahnitz commented Apr 25, 2024 •

edited

Loading

titodalcanton commented Jul 23, 2024 •

edited

Loading

titodalcanton Jul 29, 2024 •

edited

Loading