From 4d5121d72762848a0979bbef011b7a3b415cd4f1 Mon Sep 17 00:00:00 2001 From: Maximilian Linhoff Date: Mon, 18 Nov 2024 17:59:19 +0100 Subject: [PATCH 1/4] Add changelog for 0.23 --- CHANGES.rst | 86 +++++++++++++++++++++++++++++++++++ docs/changes/2411.feature.rst | 3 -- docs/changes/2609.feature.rst | 1 - docs/changes/2625.api.rst | 9 ---- docs/changes/2625.feature.rst | 8 ---- docs/changes/2627.bugfix.rst | 6 --- docs/changes/2633.api.rst | 7 --- docs/changes/2636.feature.rst | 4 -- docs/changes/2641.api.rst | 16 ------- docs/changes/2648.api.rst | 10 ---- docs/changes/2650.feature.rst | 3 -- 11 files changed, 86 insertions(+), 67 deletions(-) delete mode 100644 docs/changes/2411.feature.rst delete mode 100644 docs/changes/2609.feature.rst delete mode 100644 docs/changes/2625.api.rst delete mode 100644 docs/changes/2625.feature.rst delete mode 100644 docs/changes/2627.bugfix.rst delete mode 100644 docs/changes/2633.api.rst delete mode 100644 docs/changes/2636.feature.rst delete mode 100644 docs/changes/2641.api.rst delete mode 100644 docs/changes/2648.api.rst delete mode 100644 docs/changes/2650.feature.rst diff --git a/CHANGES.rst b/CHANGES.rst index 82d906de031..bd4714b8a7d 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -1,3 +1,89 @@ +ctapipe v0.23.0 (2024-11-18) +============================ + + +API Changes +----------- + +- Add possibility to use ``HIPPARCOS`` catalog to get star positions + + - add catalogs enumerator to ``ctapipe.utils.astro`` + - update ``get_bright_stars`` in ``ctapipe.utils.astro`` to allow catalog selection + - ensure application of proper motion + - add possibility to select stars on proximity to a given position and on apparent magnitude + - Bundle star catalogs in the python package + - API change: ``ctapipe.utils.astro.get_bright_stars`` now requires a timestamp to apply proper motion. + In the prevision revision, the time argument was missing and the proper motion was not applied. [`#2625 `__] + +- Move the simulated shower distribution from something + that was specific to ``SimTelEventSource`` to a general interface + of ``EventSource``. Implement the new interface in both ``SimTelEventSource`` + and ``HDF5EventSource`` and adapt writing of this information in ``DataWriter``. + + This makes sure that the ``SimulatedShowerDistribution`` information is always + included, also when running ``ctapipe-process`` consecutively. [`#2633 `__] + +- The following dependencies are now optional: + + * eventio, used for ``ctapipe.io.SimTelEventSource``. + * matplotlib, used ``ctapipe.visualiation.CameraDisplay``, ``ctapipe.visualiation.ArrayDisplay``, + and most default visualiation tasks, e.g. ``.peek()`` methods. + * iminuit, used for the ``ctapipe.image.muon`` and ``ctapipe.reco.impact`` fitting routines. + * bokeh, for ``ctapipe.visualiation.bokeh`` + + Code that needs these dependencies will now raise ``ctapipe.exceptions.OptionalDependencyMissing`` + in case such functionality is used and the dependency in question is not installed. + + These packages will now longer be installed by default when using e.g. ``pip install ctapipe``. + + If you want to install ctapipe with all optional dependencies included, do ``pip install "ctapipe[all]"``. + + For ``conda``, we will publish to packages: ``ctapipe`` will include all optional dependencies + and a new ``ctapipe-base`` package will only include the required dependencies. + + [`#2641 `__] + +- * Add possibility to directly pass the reference metadata to + ``Provenance.add_input_file``. + * Remove the call to ``Provenace.add_input_file`` from the + ``EventSource`` base class. + * Add the proper calls to ``Provenance.add_input_file`` in + ``HDF5EventSource`` (providing the metadata) and + ``SimTelEventSource`` (not providing metadata yet, but avoiding a warning) + * Plugin implementations of ``EventSource`` should make sure they + register their input files using ``Provenance.add_input_file``, preferably + providing also the reference metadata. [`#2648 `__] + + +Bug Fixes +--------- + +- Fix that hdf5 files created with older versions of ctapipe, e.g. + the public dataset created with 0.17 can be read by ctapipe-process. + These files are missing the subarray reference location, which was + introduced in later versions of ctapipe. A dummy location (lon=0, lat=0) + is used for these now, the same value is already used for simtel files + lacking this information. [`#2627 `__] + +New Features +------------ + +- Add option ``override_obs_id`` to ``SimTelEventSource`` which allows + assigning new, unique ``obs_ids`` in case productions reuse CORSIKA run + numbers. [`#2411 `__] + +- Add calibration calculators which aggregates statistics, detects outliers, handles faulty data chunks. [`#2609 `__] + +- Update ``CameraCalibrator`` in ``ctapipe.calib.camera.calibrator`` allowing it to correctly calibrate variance images generated with the ``VarianceExtractor``. + - If the ``VarianceExtractor`` is used for the ``CameraCalibrator`` the element-wise square of the relative and absolute gain calibration factors are applied to the image; + - For other image extractors the plain factors are still applied. + - The ``VarianceExtractor`` provides no peak time and the calibrator will skip shifting the peak time for extractors like the ``VarianceExtractor`` that similarly do not provide a peak time [`#2636 `__] + +- Add ``__repr__`` methods to all objects that were missing + them in ``ctapipe.io.metadata``, update the existing ones + for consistency. [`#2650 `__] + + ctapipe v0.22.0 (2024-09-12) ============================ diff --git a/docs/changes/2411.feature.rst b/docs/changes/2411.feature.rst deleted file mode 100644 index 49cb45370dc..00000000000 --- a/docs/changes/2411.feature.rst +++ /dev/null @@ -1,3 +0,0 @@ -Add option ``override_obs_id`` to ``SimTelEventSource`` which allows -assigning new, unique ``obs_ids`` in case productions reuse CORSIKA run -numbers. diff --git a/docs/changes/2609.feature.rst b/docs/changes/2609.feature.rst deleted file mode 100644 index fac55b285f6..00000000000 --- a/docs/changes/2609.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add calibration calculators which aggregates statistics, detects outliers, handles faulty data chunks. diff --git a/docs/changes/2625.api.rst b/docs/changes/2625.api.rst deleted file mode 100644 index 2e10e749fbd..00000000000 --- a/docs/changes/2625.api.rst +++ /dev/null @@ -1,9 +0,0 @@ -Add possibility to use ``HIPPARCOS`` catalog to get star positions - -- add catalogs enumerator to ``ctapipe.utils.astro`` -- update ``get_bright_stars`` in ``ctapipe.utils.astro`` to allow catalog selection -- ensure application of proper motion -- add possibility to select stars on proximit to a given position and on apparent magnitude -- move cached star catalogs to ``ctapipe/resources/`` -- API change: ``ctapipe.utils.astro.get_bright_stars`` now requires a timestamp to apply proper motion. - In the prevision revision, the time argument was missing and the proper motion was not applied. diff --git a/docs/changes/2625.feature.rst b/docs/changes/2625.feature.rst deleted file mode 100644 index cb611de78e4..00000000000 --- a/docs/changes/2625.feature.rst +++ /dev/null @@ -1,8 +0,0 @@ -Add possibility to use ``HIPPARCOS`` catalog to get star positions - -- add catalogs enumerator to ``ctapipe.utils.astro`` -- update ``get_bright_stars`` in ``ctapipe.utils.astro`` to allow catalog selection -- ensure application of proper motion -- add possibility to select stars on proximit to a given position and on apparent magnitude -- move cached star catalogs to ``ctapipe/resources/`` -- API change: ``ctapipe.utils.astro.get_star_catalog`` now requires to give a time to apply proper motion, before there was no time argument diff --git a/docs/changes/2627.bugfix.rst b/docs/changes/2627.bugfix.rst deleted file mode 100644 index 4ed358e1ad8..00000000000 --- a/docs/changes/2627.bugfix.rst +++ /dev/null @@ -1,6 +0,0 @@ -Fix that hdf5 files created with older versions of ctapipe, e.g. -the public dataset created with 0.17 can be read by ctapipe-process. -These files are missing the subarray reference location, which was -introduced in later versions of ctapipe. A dummy location (lon=0, lat=0) -is used for these now, the same value is already used for simtel files -lacking this information. diff --git a/docs/changes/2633.api.rst b/docs/changes/2633.api.rst deleted file mode 100644 index 2f76d166ad2..00000000000 --- a/docs/changes/2633.api.rst +++ /dev/null @@ -1,7 +0,0 @@ -Move the simulated shower distribution from something -that was specific to ``SimTelEventSource`` to a general interface -of ``EventSource``. Implement the new interface in both ``SimTelEventSource`` -and ``HDF5EventSource`` and adapt writing of this information in ``DataWriter``. - -This makes sure that the ``SimulatedShowerDistribution`` information is always -included, also when running ``ctapipe-process`` consecutively. diff --git a/docs/changes/2636.feature.rst b/docs/changes/2636.feature.rst deleted file mode 100644 index d8bdc488322..00000000000 --- a/docs/changes/2636.feature.rst +++ /dev/null @@ -1,4 +0,0 @@ -Update ``CameraCalibrator`` in ``ctapipe.calib.camera.calibrator`` allowing it to correctly calibrate variance images generated with the ``VarianceExtractor``. - - If the ``VarianceExtractor`` is used for the ``CameraCalibrator`` the element-wise square of the relative and absolute gain calibration factors are applied to the image; - - For other image extractors the plain factors are still applied. - - The ``VarianceExtractor`` provides no peak time and the calibrator will skip shifting the peak time for extractors like the ``VarianceExtractor`` that similarly do not provide a peak time diff --git a/docs/changes/2641.api.rst b/docs/changes/2641.api.rst deleted file mode 100644 index f48a8c2b59d..00000000000 --- a/docs/changes/2641.api.rst +++ /dev/null @@ -1,16 +0,0 @@ -The following dependencies are now optional: - -* eventio, used for ``ctapipe.io.SimTelEventSource``. -* matplotlib, used ``ctapipe.visualiation.CameraDisplay``, ``ctapipe.visualiation.ArrayDisplay``, - and most default visualiation tasks, e.g. ``.peek()`` methods. -* iminuit, used for the ``ctapipe.image.muon`` and ``ctapipe.reco.impact`` fitting routines. -* bokeh, for ``ctapipe.visualiation.bokeh`` - -Code that needs these dependencies will now raise ``ctapipe.exceptions.OptionalDependencyMissing`` -in case such functionality is used and the dependency in question is not installed. - -These packages will now longer be installed by default when using e.g. ``pip install ctapipe``. - -If you want to install ctapipe with all optional dependencies included, do ``pip install "ctapipe[all]"``. - -The ``conda`` package will also no longer directly depend on these packages. diff --git a/docs/changes/2648.api.rst b/docs/changes/2648.api.rst deleted file mode 100644 index 01dbd2bc725..00000000000 --- a/docs/changes/2648.api.rst +++ /dev/null @@ -1,10 +0,0 @@ -* Add possibility to directly pass the reference metadata to - ``Provenance.add_input_file``. -* Remove the call to ``Provenace.add_input_file`` from the - ``EventSource`` base class. -* Add the proper calls to ``Provenance.add_input_file`` in - ``HDF5EventSource`` (providing the metadata) and - ``SimTelEventSource`` (not providing metadata yet, but avoiding a warning) -* Plugin implementations of ``EventSource`` should make sure they - register their input files using ``Provenance.add_input_file``, preferably - providing also the reference metadata. diff --git a/docs/changes/2650.feature.rst b/docs/changes/2650.feature.rst deleted file mode 100644 index 9035322f858..00000000000 --- a/docs/changes/2650.feature.rst +++ /dev/null @@ -1,3 +0,0 @@ -Add ``__repr__`` methods to all objects that were missing -them in ``ctapipe.io.metadata``, update the existing ones -for consistency. From 042acc59fe52ae341d568805ff40f3c62c490081 Mon Sep 17 00:00:00 2001 From: Maximilian Linhoff Date: Mon, 18 Nov 2024 18:01:46 +0100 Subject: [PATCH 2/4] Add note about ctapipe-base to getting started --- docs/user-guide/getting-started.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/user-guide/getting-started.rst b/docs/user-guide/getting-started.rst index 08318f617ec..786388a2229 100644 --- a/docs/user-guide/getting-started.rst +++ b/docs/user-guide/getting-started.rst @@ -56,6 +56,14 @@ You can install them individually, or if you just want to get ``ctapipe`` with a $ pip install 'ctapipe[all]' +The ``conda`` package ``ctapipe`` includes all optional dependencies, if you want to install +a minimal version of ``ctapipe`` only including required dependencies, you can use the +``ctapipe-base`` package: + +.. code-block:: console + + $ mamba install -c conda-forge ctapipe-base + How To Get a Specific Version ----------------------------- From dd1993293ee4dbb1466dbe67b717fc2755c897f5 Mon Sep 17 00:00:00 2001 From: Maximilian Linhoff Date: Tue, 19 Nov 2024 10:37:24 +0100 Subject: [PATCH 3/4] Fix typos --- CHANGES.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGES.rst b/CHANGES.rst index bd4714b8a7d..5ea3581bde9 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -26,8 +26,8 @@ API Changes - The following dependencies are now optional: * eventio, used for ``ctapipe.io.SimTelEventSource``. - * matplotlib, used ``ctapipe.visualiation.CameraDisplay``, ``ctapipe.visualiation.ArrayDisplay``, - and most default visualiation tasks, e.g. ``.peek()`` methods. + * matplotlib, used ``ctapipe.visualization.CameraDisplay``, ``ctapipe.visualization.ArrayDisplay``, + and most default visualization tasks, e.g. ``.peek()`` methods. * iminuit, used for the ``ctapipe.image.muon`` and ``ctapipe.reco.impact`` fitting routines. * bokeh, for ``ctapipe.visualiation.bokeh`` @@ -58,7 +58,7 @@ API Changes Bug Fixes --------- -- Fix that hdf5 files created with older versions of ctapipe, e.g. +- Fix ensures that hdf5 files created with older versions of ctapipe, e.g. the public dataset created with 0.17 can be read by ctapipe-process. These files are missing the subarray reference location, which was introduced in later versions of ctapipe. A dummy location (lon=0, lat=0) From 4a43863b70eb53e76c06e084d4ae75ef1bf00ba1 Mon Sep 17 00:00:00 2001 From: Maximilian Linhoff Date: Tue, 19 Nov 2024 14:45:44 +0100 Subject: [PATCH 4/4] Update changelog --- CHANGES.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/CHANGES.rst b/CHANGES.rst index 5ea3581bde9..e2fa7538ad0 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -58,10 +58,11 @@ API Changes Bug Fixes --------- -- Fix ensures that hdf5 files created with older versions of ctapipe, e.g. +- Fix ensuring that hdf5 files created with older versions of ctapipe, e.g. the public dataset created with 0.17 can be read by ctapipe-process. - These files are missing the subarray reference location, which was - introduced in later versions of ctapipe. A dummy location (lon=0, lat=0) + These files contain pointing information at a different location and + are missing the subarray reference location, which was introduced + in later versions of ctapipe. A dummy location (lon=0, lat=0) is used for these now, the same value is already used for simtel files lacking this information. [`#2627 `__]