From 2407d6ae12e5d13c73ea26ee881d8f8f5e80d0c6 Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Tue, 19 Jan 2021 22:22:51 +0100 Subject: [PATCH 01/17] Update ReleaseGuide.md Apply naming scheme repo-vX.X.X.X for release branch --- docs/ReleaseGuide.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/ReleaseGuide.md b/docs/ReleaseGuide.md index 55b997ce..8d86194b 100644 --- a/docs/ReleaseGuide.md +++ b/docs/ReleaseGuide.md @@ -1,11 +1,11 @@ ## Guide to release new version of python-bindings The developer who is releasing a new version of the python-bindings is expected to follow this work flow: -The release of the `python-bindings` repository is made directly from a release branch called `release_v2.1.1.1`. This branch is mainly needed to help other developers with testing. +The release of the `python-bindings` repository is made directly from a release branch called `python-bindings-v2.1.1.1`. This branch is mainly needed to help other developers with testing. -1. Create a branch called `release_v2.1.1.1` from the latest commit of the `develop` branch. +1. Create a branch called `python-bindings-v2.1.1.1` from the latest commit of the `develop` branch. -2. [Open a Pull Request `master` <-- `release_v2.1.1.1`](https://github.com/precice/python-bindings/compare/master...master) named after the version (i.e. `Release v2.1.1.1`) and briefly describe the new features of the release in the PR description. +2. [Open a Pull Request `master` <-- `python-bindings-v2.1.1.1`](https://github.com/precice/python-bindings/compare/master...master) named after the version (i.e. `Release v2.1.1.1`) and briefly describe the new features of the release in the PR description. 3. Bump the version in the following places: @@ -15,11 +15,11 @@ The release of the `python-bindings` repository is made directly from a release 4. [Draft a New Release](https://github.com/precice/python-bindings/releases/new) in the `Releases` section of the repository page in a web browser. The release tag needs to be the exact version number (i.e.`v2.1.1.1` or `v2.1.1.1rc1`, compare to [existing tags](https://github.com/precice/python-bindings/tags)). Use `@target:master`. Release title is also the version number (i.e. `v2.1.1.1` or `v2.1.1.1rc1`, compare to [existing releases](https://github.com/precice/python-bindings/tags)). * *Note:* We use the [python-versioneer](https://github.com/python-versioneer/python-versioneer/) for maintaining the version. Therefore the tag directly defines the version in all relevant places. - * *Note:* If it is a pre-release then the option *This is a pre-release* needs to be selected at the bottom of the page. Use `@target:release_v2.1.1.1` for a pre-release, since we will never merge a pre-release into master. + * *Note:* If it is a pre-release then the option *This is a pre-release* needs to be selected at the bottom of the page. Use `@target:python-bindings-v2.1.1.1` for a pre-release, since we will never merge a pre-release into master. a) If a pre-release is made: Directly hit the "Publish release" button in your Release Draft. Now you can check the artifacts (e.g. release on [PyPI](https://pypi.org/project/pyprecice/#history)) of the release. *Note:* As soon as a new tag is created github actions will take care of deploying the new version on PyPI using [this workflow](https://github.com/precice/python-bindings/actions?query=workflow%3A%22Upload+Python+Package%22). - b) If this is a "real" release: As soon as one approving review is made, merge the release PR (from `release_v2.1.1.1`) into `master`. + b) If this is a "real" release: As soon as one approving review is made, merge the release PR (from `python-bindings-v2.1.1.1`) into `master`. 6. Merge `master` into `develop` for synchronization of `develop`. From 912790929a991928c9e0bfeebae0e60e9c0324d8 Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Tue, 19 Jan 2021 22:26:16 +0100 Subject: [PATCH 02/17] Update ReleaseGuide.md Addition to 2407d6a --- docs/ReleaseGuide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ReleaseGuide.md b/docs/ReleaseGuide.md index 8d86194b..18dcbe35 100644 --- a/docs/ReleaseGuide.md +++ b/docs/ReleaseGuide.md @@ -9,7 +9,7 @@ The release of the `python-bindings` repository is made directly from a release 3. Bump the version in the following places: - * `CHANGELOG.md` on `release v2.1.1.1`. + * `CHANGELOG.md` on `python-bindings-v2.1.1.1`. * There is no need to bump the version anywhere else, since we use the [python-versioneer](https://github.com/python-versioneer/python-versioneer/) for maintaining the version everywhere else. 4. [Draft a New Release](https://github.com/precice/python-bindings/releases/new) in the `Releases` section of the repository page in a web browser. The release tag needs to be the exact version number (i.e.`v2.1.1.1` or `v2.1.1.1rc1`, compare to [existing tags](https://github.com/precice/python-bindings/tags)). Use `@target:master`. Release title is also the version number (i.e. `v2.1.1.1` or `v2.1.1.1rc1`, compare to [existing releases](https://github.com/precice/python-bindings/tags)). From f743837835155a33912467aef1a05f0c52eff404 Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Wed, 20 Jan 2021 14:31:01 +0100 Subject: [PATCH 03/17] Clearify intend of README.md --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index c48f23e5..a573f674 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,10 @@ Python language bindings for the C++ library preCICE This package provides python language bindings for the C++ library [preCICE](https://github.com/precice/precice). Note that the first three digits of the version number of the bindings indicate the preCICE version that the bindings support. The last digit represents the version of the bindings. Example: `v2.0.0.1` and `v2.0.0.2` of the bindings represent versions `1` and `2` of the bindings that are compatible with preCICE `v2.0.0`. +# User documentation + +Please refer to [precice.org](https://www.precice.org/installation-bindings-python.html) for information on how to install and use the python bindings. Information below is intended for advanced users and developers. + # Required dependencies **preCICE**: Refer to [the preCICE wiki](https://github.com/precice/precice/wiki#1-get-precice) for information on building and installation. From d029401cc1aec448439f0108eeb353b028388a53 Mon Sep 17 00:00:00 2001 From: BenjaminRodenberg Date: Wed, 20 Jan 2021 18:10:19 +0100 Subject: [PATCH 04/17] Fix command. --- spack/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spack/README.md b/spack/README.md index 5135e02b..0c721e06 100644 --- a/spack/README.md +++ b/spack/README.md @@ -4,7 +4,7 @@ The Spack package `py-pyprecice` provides the python bindings via Spack and was Note that the file `var/spack/repos/builtin/packages/py-pyprecice/package.py` is a template that can be specified using `jinja2`. Run `python3 jinja-instrantiate.py` to do so. You may use the `--branch` argument to specify a branch. Example: ``` -python3 jinja-instantiate --branch feature > my_package_script.py +python3 jinja-instantiate.py --branch feature > my_package_script.py ``` ## Docker image ci-spack-pyprecice-deps-1804 From 4030e55585261207da3332cc79fb098f4bad9d53 Mon Sep 17 00:00:00 2001 From: BenjaminRodenberg Date: Wed, 20 Jan 2021 18:15:57 +0100 Subject: [PATCH 05/17] Adds checksum for version 2.2.0.1. --- spack/var/spack/repos/builtin/packages/py-pyprecice/package.py | 1 + 1 file changed, 1 insertion(+) diff --git a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py index 07e1ea48..92665519 100644 --- a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py +++ b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py @@ -19,6 +19,7 @@ class PyPyprecice(PythonPackage): # Always prefer final version of release candidate version("develop", branch="{{ branch }}") + version('2.2.0.1', sha256='032fa58193cfa69e3be37557977056e8f507d89b40c490a351d17271269b25ad') version('2.1.1.2', sha256='363eb3eeccf964fd5ee87012c1032353dd1518662868f2b51f04a6d8a7154045') version("2.1.1.1", sha256="972f574549344b6155a8dd415b6d82512e00fa154ca25ae7e36b68d4d2ed2cf4") version("2.1.0.1", sha256="ac5cb7412c6b96b08a04fa86ea38e52d91ea739a3bd1c209baa93a8275e4e01a") From 814db227d590b62c77f0b6dd2e1144bbaad592d8 Mon Sep 17 00:00:00 2001 From: BenjaminRodenberg Date: Wed, 20 Jan 2021 18:25:40 +0100 Subject: [PATCH 06/17] Fix precice dependency. --- spack/var/spack/repos/builtin/packages/py-pyprecice/package.py | 1 + 1 file changed, 1 insertion(+) diff --git a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py index 92665519..096203a1 100644 --- a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py +++ b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py @@ -38,6 +38,7 @@ class PyPyprecice(PythonPackage): depends_on("mpi", when="+mpi") depends_on("openssh") depends_on("precice", when="@develop") + depends_on("precice@2.2.0", when="@2.2.0.1:2.2.0.99") depends_on("precice@2.1.1", when="@2.1.1.1:2.1.1.99") depends_on("precice@2.1.0", when="@2.1.0.1:2.1.0.99") depends_on("precice@2.0.2", when="@2.0.2.1:2.0.2.99") From 1820868f926b732343746caecb3edfb50183e9d2 Mon Sep 17 00:00:00 2001 From: Oguz Ziya Koseomur <34319260+oguzziya@users.noreply.github.com> Date: Wed, 3 Mar 2021 21:47:21 +0100 Subject: [PATCH 07/17] ValueError FAQ (#85) Co-authored-by: Benjamin Rodenberg --- README.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a573f674..193a3c54 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ Please refer to [precice.org](https://www.precice.org/installation-bindings-pyth We recommend using pip3 (version 19.0.0 or newer required) for the sake of simplicity. You can check your pip3 version via `pip3 --version`. To update pip3, use the following line: ``` -$ pip3 install --upgrade pip +$ pip3 install --user --upgrade pip ``` ## Using pip3 @@ -268,6 +268,20 @@ python setup.py install --user ``` +### ValueError while importing preCICE +If you face the error: + +```bash +ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject +``` + +make sure that you are using an up-to-date version of NumPy. You can update NumPy with + +```bash +pip3 install numpy --upgrade +``` + + # Contributors * [Benjamin Rüth](https://github.com/BenjaminRueth) From 1849a41398f88b76e0c7f62ab6dc0cc4af3b6255 Mon Sep 17 00:00:00 2001 From: ajaust Date: Thu, 4 Mar 2021 11:30:42 +0100 Subject: [PATCH 08/17] Update requirements.txt Allow for `pyprecice` that are newer than 2.0. --- solverdummy/requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solverdummy/requirements.txt b/solverdummy/requirements.txt index de12c412..bd681547 100644 --- a/solverdummy/requirements.txt +++ b/solverdummy/requirements.txt @@ -1,3 +1,3 @@ -pyprecice==2.0 +pyprecice>=2.0 argparse>=1.4 numpy>=1.16 From d2216f6ff9e74635c93d09a04313e58f8cf8d06a Mon Sep 17 00:00:00 2001 From: BenjaminRodenberg Date: Thu, 4 Mar 2021 15:37:56 +0100 Subject: [PATCH 09/17] Get upstream version from https://github.com/spack/spack/pull/21881 --- .../builtin/packages/py-pyprecice/package.py | 23 ++++++++----------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py index 096203a1..64e5a755 100644 --- a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py +++ b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py @@ -1,4 +1,4 @@ -# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other +# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) @@ -15,12 +15,12 @@ class PyPyprecice(PythonPackage): homepage = "https://www.precice.org" git = "https://github.com/precice/python-bindings.git" url = "https://github.com/precice/python-bindings/archive/v2.0.0.1.tar.gz" - maintainers = ["ajaust", "BenjaminRueth"] + maintainers = ["ajaust", "BenjaminRodenberg"] # Always prefer final version of release candidate version("develop", branch="{{ branch }}") version('2.2.0.1', sha256='032fa58193cfa69e3be37557977056e8f507d89b40c490a351d17271269b25ad') - version('2.1.1.2', sha256='363eb3eeccf964fd5ee87012c1032353dd1518662868f2b51f04a6d8a7154045') + version('2.1.1.2', sha256='363eb3eeccf964fd5ee87012c1032353dd1518662868f2b51f04a6d8a7154045') version("2.1.1.1", sha256="972f574549344b6155a8dd415b6d82512e00fa154ca25ae7e36b68d4d2ed2cf4") version("2.1.0.1", sha256="ac5cb7412c6b96b08a04fa86ea38e52d91ea739a3bd1c209baa93a8275e4e01a") version("2.0.2.1", sha256="c6fca26332316de041f559aecbf23122a85d6348baa5d3252be4ddcd5e94c09a") @@ -28,16 +28,12 @@ class PyPyprecice(PythonPackage): version("2.0.0.2", sha256="5f055d809d65ec2e81f4d001812a250f50418de59990b47d6bcb12b88da5f5d7") version("2.0.0.1", sha256="96eafdf421ec61ad6fcf0ab1d3cf210831a815272984c470b2aea57d4d0c9e0e") - # Import module as a test - import_modules = ["precice"] - + # Older versions of the bindings checked versions via pip. This patch + # removes the pip dependency. + # See also https://github.com/spack/spack/pull/19558 patch("deactivate-version-check-via-pip.patch", when="@:2.1.1.1") - variant("mpi", default=True, description="Enables MPI support") - - depends_on("mpi", when="+mpi") - depends_on("openssh") - depends_on("precice", when="@develop") + depends_on("precice@develop", when="@develop") depends_on("precice@2.2.0", when="@2.2.0.1:2.2.0.99") depends_on("precice@2.1.1", when="@2.1.1.1:2.1.1.99") depends_on("precice@2.1.0", when="@2.1.0.1:2.1.0.99") @@ -47,9 +43,8 @@ class PyPyprecice(PythonPackage): depends_on("python@3:", type=("build", "run")) depends_on("py-setuptools", type="build") - depends_on("py-wheel", type="build") depends_on("py-numpy", type=("build", "run")) - depends_on("py-mpi4py", type=("build", "run"), when="+mpi") + depends_on("py-mpi4py", type=("build", "run")) depends_on("py-cython@0.29:", type=("build")) phases = ['install_lib', 'build_ext', 'install'] @@ -61,5 +56,7 @@ def build_ext_args(self, spec, prefix): ] def install(self, spec, prefix): + # Older versions of the bindings had a non-standard installation routine + # See also https://github.com/spack/spack/pull/19558#discussion_r513123239 if self.version <= Version("2.1.1.1"): self.setup_py("install", "--prefix={0}".format(prefix)) From 5d725fdb09108ff8acb153229a647a033a9449d0 Mon Sep 17 00:00:00 2001 From: BenjaminRodenberg Date: Thu, 4 Mar 2021 21:32:44 +0100 Subject: [PATCH 10/17] Adds openssh dependency again. --- spack/var/spack/repos/builtin/packages/py-pyprecice/package.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py index 64e5a755..23f16bc9 100644 --- a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py +++ b/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py @@ -41,6 +41,8 @@ class PyPyprecice(PythonPackage): depends_on("precice@2.0.1", when="@2.0.1.1:2.0.1.99") depends_on("precice@2.0.0", when="@2.0.0.1:2.0.0.99") + depends_on("openssh") # needed by OpenMPI. See https://github.com/precice/python-bindings/pull/76 + depends_on("python@3:", type=("build", "run")) depends_on("py-setuptools", type="build") depends_on("py-numpy", type=("build", "run")) From 85245af97288c54b53d8ce26067408218aa99a0d Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Thu, 4 Mar 2021 21:47:22 +0100 Subject: [PATCH 11/17] Increase timeout --- .github/workflows/build-spack.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build-spack.yml b/.github/workflows/build-spack.yml index 4ebdcd6a..39fcf4fe 100644 --- a/.github/workflows/build-spack.yml +++ b/.github/workflows/build-spack.yml @@ -8,7 +8,7 @@ jobs: build_spack: name: build_spack runs-on: ubuntu-latest - timeout-minutes: 5 + timeout-minutes: 15 container: benjaminrueth/ci-spack-pyprecice-deps-1804 defaults: run: From 86f249d967f234922a78425178c3eae18ce37cae Mon Sep 17 00:00:00 2001 From: BenjaminRodenberg Date: Tue, 9 Mar 2021 12:51:11 +0100 Subject: [PATCH 12/17] Throw warning, if user calls wrong interface function. See #92. --- precice/__init__.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/precice/__init__.py b/precice/__init__.py index 40ff7a75..8c28e592 100644 --- a/precice/__init__.py +++ b/precice/__init__.py @@ -1,7 +1,16 @@ __version__ = "unknown" +import warnings from cyprecice import Interface, action_read_iteration_checkpoint, action_write_iteration_checkpoint, action_write_initial_data, get_version_information + +def SolverInterface(*args): + """ + This is just a dummy function to avoid wrong usage of the interface. Please use precice.Interface, if you want to establish a connection to preCICE. See https://github.com/precice/python-bindings/issues/92 for more information. + """ + warnings.warn("please use precice.Interface to create the interface to C++ preCICE. Note that this function (precice.SolverInterface) does not do anything but throwing this warning. See https://github.com/precice/python-bindings/issues/92 for more information.") + + from ._version import get_versions __version__ = get_versions()['version'] del get_versions From 5fed8bdb7b43e5e915ffead9f164d9c82b74266f Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Tue, 23 Mar 2021 18:43:58 +0100 Subject: [PATCH 13/17] Overhaul spack CI (#89) * Experiment with architecture proposed in https://github.com/haampie-spack/ci-example. * Add workflow for building dependencies from https://github.com/haampie-spack/ci-example. * Use spack dev-build, don't use jinja. * Build env on a weekly basis via cron job. * Do build for generic architecture x86_64. #81 --- .github/workflows/build-env.yml | 28 +++++++++++++++++++ .github/workflows/build-spack.yml | 27 ++++-------------- spack/README.md | 19 ++++--------- spack/ci-spack-pyprecice-deps-1804.dockerfile | 17 ++++++----- spack/jinja-instantiate.py | 15 ---------- .../deactivate-version-check-via-pip.patch | 0 .../packages/py-pyprecice/package.py | 2 +- spack/repo/repo.yaml | 2 ++ 8 files changed, 53 insertions(+), 57 deletions(-) create mode 100644 .github/workflows/build-env.yml delete mode 100644 spack/jinja-instantiate.py rename spack/{var/spack/repos/builtin => repo}/packages/py-pyprecice/deactivate-version-check-via-pip.patch (100%) rename spack/{var/spack/repos/builtin => repo}/packages/py-pyprecice/package.py (98%) create mode 100644 spack/repo/repo.yaml diff --git a/.github/workflows/build-env.yml b/.github/workflows/build-env.yml new file mode 100644 index 00000000..273d90f0 --- /dev/null +++ b/.github/workflows/build-env.yml @@ -0,0 +1,28 @@ +name: Update build environment + +on: + workflow_dispatch: # Trigger by hand from the UI + schedule: + - cron: '0 0 * * 0' # Schedule it every Sunday + +jobs: + build-spack-pyprecice-deps: + name: Builds the baseimage for spack providing dependencies + runs-on: ubuntu-latest + steps: + - name: Checkout Repository + uses: actions/checkout@v2 + - name: Set up Docker + uses: docker/setup-buildx-action@v1 + - name: Login to registry + uses: docker/login-action@v1 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + - name: Build and push + uses: docker/build-push-action@v2 + with: + push: true + context: . + file: spack/ci-spack-pyprecice-deps-1804.dockerfile + tags: precice/ci-spack-pyprecice-deps-1804 diff --git a/.github/workflows/build-spack.yml b/.github/workflows/build-spack.yml index 39fcf4fe..1d734b7c 100644 --- a/.github/workflows/build-spack.yml +++ b/.github/workflows/build-spack.yml @@ -9,32 +9,17 @@ jobs: name: build_spack runs-on: ubuntu-latest timeout-minutes: 15 - container: benjaminrueth/ci-spack-pyprecice-deps-1804 + container: precice/ci-spack-pyprecice-deps-1804 defaults: run: shell: "bash --login -eo pipefail {0}" steps: - name: Checkout Repository uses: actions/checkout@v2 - - name: Install pip3 + - name: Move Package Script run: | - curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py - python3 get-pip.py - - name: Install jinja2 - run: pip3 install --user jinja2 - - name: Extract branch name # from https://stackoverflow.com/a/58035262/5158031 - shell: bash - run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})" - id: extract_branch - - name: Specify Package Script + cp spack/repo/packages/py-pyprecice/package.py /py-pyprecice-repo/packages/py-pyprecice/ + cp spack/repo/packages/py-pyprecice/*.patch /py-pyprecice-repo/packages/py-pyprecice/ + - name: Try to build py-pyprecice with spack and test it run: | - python3 spack/jinja-instantiate.py --branch ${{ steps.extract_branch.outputs.branch }} > /opt/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py - ls spack/var/spack/repos/builtin/packages/py-pyprecice - cp spack/var/spack/repos/builtin/packages/py-pyprecice/*.patch /opt/spack/var/spack/repos/builtin/packages/py-pyprecice/ - ls spack/var/spack/repos/builtin/packages/py-pyprecice - - name: Try to build py-pyprecice with spack - run: | - . /opt/spack/share/spack/setup-env.sh && spack env activate precice && spack add py-pyprecice@develop ^precice@develop && spack concretize -f && spack install -v - - name: Import precice and print version - run: | - . /opt/spack/share/spack/setup-env.sh && spack env activate precice && python3 -c "import precice; print(precice.__version__)" + . /opt/spack/share/spack/setup-env.sh && spack env activate ci && spack arch && spack find && spack dev-build py-pyprecice@develop target=x86_64 && spack load precice py-numpy py-mpi4py py-cython openssh openmpi && mkdir runner && cd runner && python3 -c "import precice; print(precice.__version__)" diff --git a/spack/README.md b/spack/README.md index 0c721e06..71c8bc42 100644 --- a/spack/README.md +++ b/spack/README.md @@ -2,20 +2,13 @@ The Spack package `py-pyprecice` provides the python bindings via Spack and was submitted via https://github.com/spack/spack/pull/19558. This folder contains the Spack package script that can be used for testing whether the Spack installation works. -Note that the file `var/spack/repos/builtin/packages/py-pyprecice/package.py` is a template that can be specified using `jinja2`. Run `python3 jinja-instrantiate.py` to do so. You may use the `--branch` argument to specify a branch. Example: -``` -python3 jinja-instantiate.py --branch feature > my_package_script.py -``` +## Docker image `precice/ci-spack-pyprecice-deps-1804` -## Docker image ci-spack-pyprecice-deps-1804 +The workflow `build_env` in `.github/workflows/build-spack.yml` creates the image `precice/ci-spack-pyprecice-deps-1804`. This image contains all dependencies of `py-pyprecice@develop`. + +The workflow `build_spack` in `.github/workflows/build-spack.yml` uses the image `precice/ci-spack-pyprecice-deps-1804` to reduce build time. The workflow uses the `package.py` from this repository to build the latest version of the bindings and run a small test on this version. -The test `build_spack` in `.github/workflows/build-spack.yml` uses the image `benjaminrueth/ci-spack-pyprecice-deps-1804` to reduce build time. Use the file `ci-spack-pyprecice-deps-1804.dockerfile` from this folder for building this image: -``` -docker build -f ci-spack-pyprecice-deps-1804.dockerfile -t USERNAME/ci-spack-pyprecice-deps-1804.dockerfile . -docker push USERNAME/ci-spack-pyprecice-deps-1804.dockerfile -``` ## When a new spack release is necessary -* Add checksum of newest version(s) to https://github.com/precice/python-bindings/blob/develop/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py. You can get checksum for any released version by running `spack checksum py-pyprecice`. -* Run `python3 jinja-instantiate > package.py` to generate the latest version of `package.py`. -* Use `package.py` together with the patches provided in `python-bindings/spack/var/spack/repos/builtin/packages/py-pyprecice` to [create a pull request for Spack](https://github.com/spack/spack/compare) and submit the new release. +* Add checksum of newest version(s) to https://github.com/precice/python-bindings/blob/develop/spack/repo/packages/py-pyprecice/package.py. You can get checksum for any released version by running `spack checksum py-pyprecice`. +* Use `package.py` together with the patches provided in `python-bindings/spack/repo/packages/py-pyprecice` to [create a pull request for Spack](https://github.com/spack/spack/compare) and submit the new release. diff --git a/spack/ci-spack-pyprecice-deps-1804.dockerfile b/spack/ci-spack-pyprecice-deps-1804.dockerfile index ada55360..fb7c3cfa 100644 --- a/spack/ci-spack-pyprecice-deps-1804.dockerfile +++ b/spack/ci-spack-pyprecice-deps-1804.dockerfile @@ -1,8 +1,11 @@ # Build stage with Spack pre-installed and ready to be used -FROM spack/ubuntu-bionic:latest -RUN spack env create precice -# Install dependencies for precice and py-pyprecice -RUN spack env activate precice \ - && spack add boost@1.74.0 cmake@3.18.4 eigen@3.3.8 openssh petsc@3.14.1 pkgconfig precice@develop py-setuptools py-wheel py-cython py-numpy py-mpi4py \ - && spack concretize -f \ - && spack install +FROM spack/ubuntu-bionic:latest + +# Mount the current sources into the build container +# and build the default environment +ADD ./spack/repo /py-pyprecice-repo +RUN spack --color=always env create --without-view ci && \ + spack --color=always -e ci add py-pyprecice@develop target=x86_64 && \ + spack --color=always -e ci repo add /py-pyprecice-repo && \ + spack --color=always -e ci install --fail-fast --only=dependencies && \ + spack --color=always clean -a diff --git a/spack/jinja-instantiate.py b/spack/jinja-instantiate.py deleted file mode 100644 index e26877b0..00000000 --- a/spack/jinja-instantiate.py +++ /dev/null @@ -1,15 +0,0 @@ -import jinja2 -import argparse -from os import path - -parser = argparse.ArgumentParser() -parser.add_argument('--branch', type=str, help="Branch to be checked out", - default="develop") -args = parser.parse_args() - -f = open(path.join(path.dirname(__file__), - "var/spack/repos/builtin/packages/py-pyprecice/package.py"), - "r") - -template = jinja2.Template(f.read()) -print(template.render(branch=args.branch)) diff --git a/spack/var/spack/repos/builtin/packages/py-pyprecice/deactivate-version-check-via-pip.patch b/spack/repo/packages/py-pyprecice/deactivate-version-check-via-pip.patch similarity index 100% rename from spack/var/spack/repos/builtin/packages/py-pyprecice/deactivate-version-check-via-pip.patch rename to spack/repo/packages/py-pyprecice/deactivate-version-check-via-pip.patch diff --git a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py b/spack/repo/packages/py-pyprecice/package.py similarity index 98% rename from spack/var/spack/repos/builtin/packages/py-pyprecice/package.py rename to spack/repo/packages/py-pyprecice/package.py index 23f16bc9..dd65c005 100644 --- a/spack/var/spack/repos/builtin/packages/py-pyprecice/package.py +++ b/spack/repo/packages/py-pyprecice/package.py @@ -18,7 +18,7 @@ class PyPyprecice(PythonPackage): maintainers = ["ajaust", "BenjaminRodenberg"] # Always prefer final version of release candidate - version("develop", branch="{{ branch }}") + version("develop", branch="develop") version('2.2.0.1', sha256='032fa58193cfa69e3be37557977056e8f507d89b40c490a351d17271269b25ad') version('2.1.1.2', sha256='363eb3eeccf964fd5ee87012c1032353dd1518662868f2b51f04a6d8a7154045') version("2.1.1.1", sha256="972f574549344b6155a8dd415b6d82512e00fa154ca25ae7e36b68d4d2ed2cf4") diff --git a/spack/repo/repo.yaml b/spack/repo/repo.yaml new file mode 100644 index 00000000..61517481 --- /dev/null +++ b/spack/repo/repo.yaml @@ -0,0 +1,2 @@ +repo: + namespace: 'spack' From 8c66cc3e03e311fea8594a607b119cedc4cec653 Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Tue, 23 Mar 2021 18:54:58 +0100 Subject: [PATCH 14/17] Don't install openssh (#95) --- spack/repo/packages/py-pyprecice/package.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/spack/repo/packages/py-pyprecice/package.py b/spack/repo/packages/py-pyprecice/package.py index dd65c005..65817306 100644 --- a/spack/repo/packages/py-pyprecice/package.py +++ b/spack/repo/packages/py-pyprecice/package.py @@ -41,8 +41,6 @@ class PyPyprecice(PythonPackage): depends_on("precice@2.0.1", when="@2.0.1.1:2.0.1.99") depends_on("precice@2.0.0", when="@2.0.0.1:2.0.0.99") - depends_on("openssh") # needed by OpenMPI. See https://github.com/precice/python-bindings/pull/76 - depends_on("python@3:", type=("build", "run")) depends_on("py-setuptools", type="build") depends_on("py-numpy", type=("build", "run")) From 2c6863045a984cad86210d9acb9661436e782e3a Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Tue, 6 Apr 2021 20:54:26 +0200 Subject: [PATCH 15/17] Improved error messages for bindings (#9) * Adding error messages to check length of values and vertex id arrays provided for write commands * Adding provided and expected values in assertion statements Co-authored-by: ishaandesai --- cyprecice/cyprecice.pyx | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/cyprecice/cyprecice.pyx b/cyprecice/cyprecice.pyx index ad0db10f..0c48fd88 100644 --- a/cyprecice/cyprecice.pyx +++ b/cyprecice/cyprecice.pyx @@ -453,7 +453,7 @@ cdef class Interface: if len(position) > 0: dimensions = len(position) - assert(dimensions == self.get_dimensions()) + assert dimensions == self.get_dimensions(), "Dimensions of vertex coordinate in set_mesh_vertex does not match with dimensions in problem definition. Provided dimensions: {}, expected dimensions: {}".format(dimensions, self.get_dimensions()) elif len(position) == 0: dimensions = self.get_dimensions() @@ -530,7 +530,7 @@ cdef class Interface: if len(positions) > 0: size, dimensions = positions.shape - assert(dimensions == self.get_dimensions()) + assert dimensions == self.get_dimensions(), "Dimensions of vertex coordinates in set_mesh_vertices does not match with dimensions in problem definition. Provided dimensions: {}, expected dimensions: {}".format(dimensions, self.get_dimensions()) elif len(positions) == 0: size = positions.shape[0] dimensions = self.get_dimensions() @@ -640,7 +640,7 @@ cdef class Interface: if len(positions) > 0: size, dimensions = positions.shape - assert(dimensions == self.get_dimensions()) + assert dimensions == self.get_dimensions(), "Dimensions of position coordinates in get_mesh_vertex_ids_from_positions does not match with dimensions in problem definition. Provided dimensions: {}, expected dimensions: {}".format(dimensions, self.get_dimensions()) elif len(positions) == 0: size = positions.shape[0] dimensions = self.get_dimensions() @@ -892,15 +892,15 @@ cdef class Interface: if len(values) > 0: size, dimensions = values.shape - assert(dimensions == self.get_dimensions()) + assert dimensions == self.get_dimensions(), "Dimensions of vector data in write_block_vector_data does not match with dimensions in problem definition. Provided dimensions: {}, expected dimensions: {}".format(dimensions, self.get_dimensions()) if len(values) == 0: size = 0 cdef np.ndarray[int, ndim=1] _vertex_ids = np.ascontiguousarray(vertex_ids, dtype=np.int32) cdef np.ndarray[double, ndim=1] _values = np.ascontiguousarray(values.flatten(), dtype=np.double) - assert(_values.size == size * self.get_dimensions()) - assert(_vertex_ids.size == size) + assert _values.size == size * self.get_dimensions(), "Vector data is not provided for all vertices in write_block_vector_data. Check length of input data provided. Provided size: {}, expected size: {}".format(_values.size, size * self.get_dimensions()) + assert _vertex_ids.size == size, "Vertex IDs are of incorrect length in write_block_vector_data. Check length of vertex ids input. Provided size: {}, expected size: {}".format(_vertex_ids.size, size) self.thisptr.writeBlockVectorData (data_id, size, _vertex_ids.data, _values.data) @@ -941,11 +941,14 @@ cdef class Interface: >>> interface.write_vector_data(data_id, vertex_id, value) """ check_array_like(value, "value", "write_vector_data") + assert len(value) > 0, "Input vector data is empty in write_vector_data" - assert(len(value) > 0) dimensions = len(value) - assert(dimensions == self.get_dimensions()) + + assert dimensions == self.get_dimensions(), "Dimensions of vector data in write_vector_data does not match with dimensions in problem definition. Provided dimensions: {}, expected dimensions: {}".format(dimensions, self.get_dimensions()) + cdef np.ndarray[np.double_t, ndim=1] _value = np.ascontiguousarray(value, dtype=np.double) + self.thisptr.writeVectorData (data_id, vertex_id, _value.data) def write_block_scalar_data (self, data_id, vertex_ids, values): @@ -985,13 +988,11 @@ cdef class Interface: if len(values) == 0: size = 0 - cdef np.ndarray[int, ndim=1] _vertex_ids = np.ascontiguousarray(vertex_ids, dtype=np.int32) cdef np.ndarray[double, ndim=1] _values = np.ascontiguousarray(values, dtype=np.double) - assert(_values.size == size) - assert(_vertex_ids.size == size) - + assert _values.size == size, "Scalar data is not provided for all vertices in write_block_scalar_data. Check size of input data provided. Provided size: {}, expected size: {}".format(_values.size, size) + assert _vertex_ids.size == size, "Vertex IDs are of incorrect length in write_block_scalar_data. Check size of vertex ids input. Provided size: {}, expected size: {}".format(_vertex_ids.size, size) self.thisptr.writeBlockScalarData (data_id, size, _vertex_ids.data, _values.data) def write_scalar_data (self, data_id, vertex_id, double value): From f1912bd97423ae7be8633cecc69cd9417955b5db Mon Sep 17 00:00:00 2001 From: Ishaan Desai Date: Wed, 7 Apr 2021 12:51:01 +0200 Subject: [PATCH 16/17] Bump version in CHANGELOG --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1800a966..d0113f44 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ All notable changes to this project will be documented in this file. +## 2.2.0.2 +* Improved error messgaes for all assertions + ## 2.2.0.1 * Format complete codebase according to PEP8 and test formatting. https://github.com/precice/python-bindings/pull/82 From 73998fa65a433708ea11d481409ce3ec397ae50c Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Wed, 7 Apr 2021 13:48:53 +0200 Subject: [PATCH 17/17] Update CHANGELOG.md --- CHANGELOG.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d0113f44..b6e35e9c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,9 @@ All notable changes to this project will be documented in this file. ## 2.2.0.2 -* Improved error messgaes for all assertions + +* Improved error messgaes for all assertions. https://github.com/precice/python-bindings/pull/9 +* Improve CI w.r.t spack package. https://github.com/precice/python-bindings/pull/89 ## 2.2.0.1