PySEP v0.5.0 (#130)

* FEATURE: improves FORCESOLUTION read function to deal with SPECFEM3D_GLOBE format, previously could only handle 3D_CARTESIAN. adds more detailed docstring for the function

* utils.io.read_forcesolution changed input variable 'default_time' -> 'origin_time' to match other read functions and be slightly more intuitive

* REFORMAT read_forcesolution to return an ObsPy Event object rather than a PySEP SOURCE object, so that it can be parsed and used by Pyflex which is expecting ObsPy Event or Catalog objects to define sources

* added function 'read_events_plus' (originally defined in Pyatoa) which provides additional file format support to ObsPy's read_events function

* removed Source class from io.mt which was an old holdover for reading in FORCESOLUTION and SOURCE files from SPECFEM2D/3D. the revised method simply returns barebones ObsPy Event objects which is much cleaner and avoids requiring having to define an entirely new class. Does not affect PySEP routines, these were mainly used by external libraries calling PySEP

* General Improvements: improve recsec log, single-source version, bugfix zero trace length (#121)

* bumping version numbers to 0.5.0-dev to signify backwards-incompatible changes and that this is the 'dev' version of the code

* Issue #120 single-sourcing version number from pyproject.toml and sourcing version number in docs and __init__ directly from this value

* code fix conf finding version number,
update 'how_to_version_release' to only bump pyproject.toml version number

* #117 added a new pre-preprocessing curtail function that removes Traces whose trace lenght is 0

* #116 RecSec now logs amplitude and amplitude ratios if both obs and syn are plotted together

* amp ratio log messages now for abs max

* update CHANGELOG

* updates RTD Yaml file to comply with build.image deprecation

* attempt fix failing RTD build by swapping Python for Mamba

* removed 'read_sem' logger warning about not providing an origin time by setting default origin time to 1970-01-01T00:00:00 directly in the input variables

* added read functions to main package imports (read_sem, read_stations, read_events_plus)

* Bugfix: incorrect RecSec xlim_s indexing (#123)

* bugfix: xlim_s was using time and not sample to index the waveform when selecting max amplitudes for normalization, causing incorrect normalization to occur

* tests caught edge case where indicies were None and could not be converted to integer samples. fixing

* Update RecordSection data reading (#124)

* BUGFIX: in read_events_plus, read_specfem3d_cmtsolution_cartesian was returning an Event and not a catalog object. This was probably fine but not consistent with the other returns

* new recsec.read_data function that replaces the old generate_synthetic_data function, contains logic for taking paths and return streams and can handle both obs and syn data as well as SAC formatted syn data (#122)
bugfix: added a catch in recsec to kill the object if no stream data are found

* API CHANGE: RecSec parameter 'cmtsolution' has been changed to 'source' for generality because the file does not need to be a cmtsolution, can be a forcesolution or source as well

* small bugfixes to get synthetic stream reading working

* adding Cartesian generated synthetics to test data and added new test data to make sure read_sem can handle these appropriately

* removing some of the test data, only need a few traces

* update API recsec cmtsolution -> source

* update changelog

* BUGFIX: RecSec was unable to read different source file formats. New parameter added '--srcfmt' which allows the User to set the file format manually. If srcfmt is not provided, RecSec will attempt to guess the file format based on the file name, checking against a list of common SPECFEM source file format types

* BUGFIX: function 'read_specfem3d_cmtsolution_cartesian' can is now less strict about how it expects the incoming file to be formatted

* removing debug statement left in read forcesolution function

* #126 updating example config files to fix inconsistencies

* BUGFIX: RecordSection was incorrectly using indices in units of time, and not samples, to index the visible portion of the waveform for use in determining the local max amplitude within the time window. This would have led to incorrectly scaled waveforms when using the option 'xlim_s'. This bugfix sets the correct index unit, and ensures that all indices coming out of getter function 'get_xlim' are integers so that they directly be used to index waveform data

* Update recsec.rst

Accommodating parameter name change from 'cmtsolution' to 'source' in the docs to match recent update in #124

* pysep internal variable _legacy_naming -> legacy_naming so that it can be used in parameter file

* recsec #127 added parameters 'obs_wildcard' and 'syn_wildcard' to allow user to determine which files are read in

* change feature so that 'syn_wildcard' defaults to 'wildcard' unless explicitely provided

* removed parameters 'log_level' and 'legacy_naming' from the default config writing function, remade the template config, added new test to check the example config files are still in order and removed some unncessary parameters from example configs

* update changelog

.readthedocs.yaml

@@ -1,8 +1,13 @@
 version: 2
 
+build:
+    os: "ubuntu-22.04"
+    tools:
+        python: "mambaforge-22.9"
+
 sphinx:
     builder: html
-    configuration: ./docs/conf.py
+    configuration: docs/conf.py
 
 conda:
-    environment: ./docs/environment.yml
+    environment: docs/environment.yml

CHANGELOG.md

@@ -64,7 +64,47 @@
   - PySEP warns when config parameters are not used by the program
 
 
-## Version 0.4.1 (Master/Devel)
+## Version 0.4.1 
 - Adds Tutorial documentation following GEOS626 lab (thanks, Aakash!)
 - Adds version release documentation
 - Slightly modifies pysep-docs conda environment to accomodate converted nbooks
+
+## Version 0.5.0 (Master)
+- Improves functions 'read_forcesolution' and 'read_source', which now return
+  `obspy.core.event.Event` objects, rather than the makeshift Source objects 
+- 'read_forcesolution' can now handle FORCESOLUTION files from both SPECFEM3D
+  and SPECFEM3D_GLOBE
+- Added function `read_events_plus` that provides additional support to 
+  Obspys `read_events` function by allowing for support of FORCESOLUTION and
+  SOURCE files from SPECFEM2D/3D/3D_GLOBE
+- Remove the `Source` class from `Pysep.utils.io.mt`. This was a remnant of the
+  old Pyatoa approach to building an Obspy Event-like object to mimic certain
+  behaviors. This has been replaced by read functions which simply return Events
+- #117: New quality control function that removes Traces with array length <= 1,
+  which would cause preprocessing to fail
+- #116: RecSec now logs absmax amplitudes and absmax amplitude ratios IFF both
+  `st` and `st_syn` are provided
+- #120: Version number is now only sourced from `pyproject.toml`, other 
+  locations now reference this file to determine version number
+- #124:
+  - API Change: RecordSection parameter `cmtsolution` has been **renamed** to 
+	`source`. 
+  - RecordSection now only expects readable files in  --pysep_path or 
+	--syn_path.
+	- New `RecordSection.read_data()`function which handles data reading logic 
+	and can read both obs data (.SAC from PySEP) and syn data 
+	(SPECFEM ASCII files or SAC files)
+	- Bugfix: Added an exit catch in RecordSection to stop the workflow if no 
+	data is available
+- Bugfix: RecSec unable to read different `source` file formats. New parameter
+  'srcfmt' allows User to set this manually. If not given, RecSec will attempt
+  to guess the file format based on the name of the file.
+- Bugfix: `read_specfem3d_cmtsolution_cartesian` was unable to handle 
+	Flinn-Engdahl regions that had spaces in them. Also it was unable to 
+	handle extra lines in the file. 
+- Bugfix #126: Fixed some incorrect parameters in example config files
+- PySEP variable namechange: `_legacy_naming` -> 'legacy_naming`
+- New RecSec Parameters `wildcard` and `syn_wildcard` to specify how to 
+	search for data to plot with RecSec
+- Removed unnused parameters 'legacy_naming' and 'log_level' from 
+	`Pysep.write_config`

docs/conf.py

@@ -18,7 +18,12 @@ def setup(app):
 copyright = '2023, adjTomo Dev Team'
 author = 'adjTomo Dev Team'
 release = ''
-version = '0.4.1'
+# Grab version number from 'pyproject.toml'
+with open("../pyproject.toml", "r") as f:
+    _lines = f.readlines()
+for _line in _lines:
+    if _line.startswith("version"):
+        version = _line.split('"')[1].strip()
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

docs/how_to_version_release.md

@@ -8,8 +8,6 @@ patch), or provide reason in PR for why certain points are not checked.
 ## Prior to PR merge:
 - [ ] Merge `devel` -> `master`
 - [ ] Bump version number `pyproject.toml`
-- [ ] Bump version number `docs/conf.py`
-- [ ] Bump version number `pysep/__init__.py`
 - [ ] Ensure all tests still pass, fix broken tests
 - [ ] Update `CHANGELOG` to include all major changes since last version
 

docs/recsec.rst

@@ -158,13 +158,13 @@ structure:
 
 .. code:: bash
 
-    recsec --syn_path OUTPUT_FILES/ --cmtsolution DATA/CMTSOLUTION --stations DATA/STATIONS
+    recsec --syn_path OUTPUT_FILES/ --source DATA/CMTSOLUTION --stations DATA/STATIONS
 
 Or when scripting,
 
 .. code:: python
 
-    plotw_rs(syn_path="OUTPUT_FILES", cmtsolution="DATA/CMTSOLUTION",
+    plotw_rs(syn_path="OUTPUT_FILES", source="DATA/CMTSOLUTION",
              stations="DATA/STATIONS")
 
 You can also directly feed in an ObsPy stream containing your synthetic data
@@ -188,7 +188,7 @@ and tell RecSec to preprocess both data streams identically
 
 .. code:: bash
 
-    recsec --pysep_path ./SAC --syn_path OUTPUT_FILES/ --cmtsolution DATA/CMTSOLUTION --stations DATA/STATIONS --preprocess both
+    recsec --pysep_path ./SAC --syn_path OUTPUT_FILES/ --source DATA/CMTSOLUTION --stations DATA/STATIONS --preprocess both
 
 Preprocessing flags can be applied to the observed data only (`st`), synthetic
 data only (`st_syn`) or both (`both`). See the `preprocess` parameter in the
@@ -220,15 +220,15 @@ For SPECFEM3D_Cartesian this would look like
 
 .. code:: bash
 
-    recsec --syn_path OUTPUT_FILES --cmtsolution DATA/CMTSOLUTION --stations DATA/STATIONS --cartesian
+    recsec --syn_path OUTPUT_FILES --source DATA/CMTSOLUTION --stations DATA/STATIONS --cartesian
 
 
 For SPECFEM2D, the source file may not be a CMTSOLUTION. Additionally, the 
 default seismogram components may not be defined in ZNE
 
 .. code:: bash
 
-    recsec --syn_path OUTPUT_FILES --cmtsolution DATA/SOURCE --stations DATA/STATIONS --components Y --cartesian
+    recsec --syn_path OUTPUT_FILES --source DATA/SOURCE --stations DATA/STATIONS --components Y --cartesian
 
 
 While scripting, the input parameter `cartesian` can be used:
@@ -252,14 +252,14 @@ and provide paths to both `--pysep_path` and `--syn_path`.
 .. note::
 
     RecSec makes the assumption that both sets of synthetics share the
-    same metadata provided in the `--cmtsolution` and `--stations` flags.
+    same metadata provided in the `--source` and `--stations` flags.
 
 Let's say you've stored your 'data' in a directory called 'observed/' and your
 synthetics in a directory called 'synthetic/'
 
 .. code:: bash
 
-    recsec --pysep_path observed/ --syn_path synthetic/ --cmtsolution DATA/CMTSOLUTION --stations DATA/STATIONS --synsyn
+    recsec --pysep_path observed/ --syn_path synthetic/ --source DATA/CMTSOLUTION --stations DATA/STATIONS --synsyn
 
 
 

pyproject.toml

@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pysep-adjtomo"
-version = "0.4.1"
+version = "0.5.0"
 description = "Python Seismogram Extraction and Processing"
 readme = "README.md"
-requires-python = ">=3.7"
+requires-python = ">=3.8"
 license = {file = "LICENSE.txt"}
 authors = [
     {name = "adjTomo Dev Team"},

pysep/__init__.py

@@ -1,6 +1,8 @@
 import logging
+from importlib.metadata import version
 
-__version__ = "0.4.1"
+# Defines version number from 'pyproject.toml'
+__version__ = version("pysep-adjtomo")
 
 logger = logging.getLogger("pysep")
 logger.setLevel("INFO")
@@ -15,3 +17,4 @@
 from pysep.pysep import Pysep, get_data  # NOQA
 from pysep.recsec import RecordSection, plotw_rs  # NOQA
 from pysep.declust import Declust  # NOQA
+from pysep.utils.io import read_sem, read_stations, read_events_plus  # NOQA

pysep/configs/alvizuri_tape_2018/2006-10-09T013528_NORTH_KOREA.yaml

@@ -47,4 +47,3 @@ remove_insufficient_length: true
 remove_masked_data: true
 fill_data_gaps: false
 gap_fraction: 1.0
-log_level: DEBUG

pysep/configs/misc_events/1988-02-15T181000_KERNVILLE_EXPLOSION.yaml

@@ -46,4 +46,3 @@ remove_insufficient_length: true
 remove_masked_data: true
 fill_data_gaps: false
 gap_fraction: 1.0
-log_level: DEBUG

pysep/configs/misc_events/2015-10-18T051831_TAAN_FJORD_LANDSLIDE.yaml

@@ -50,4 +50,3 @@ remove_insufficient_length: true
 remove_masked_data: true
 fill_data_gaps: false
 gap_fraction: 1.0
-log_level: DEBUG

pysep/configs/misc_events/2016-01-24T103037_INISKIN.yaml

@@ -47,4 +47,3 @@ remove_insufficient_length: true
 remove_masked_data: true
 fill_data_gaps: false
 gap_fraction: 1.0
-log_level: DEBUG

pysep/configs/mtuq_workshop_2022/2014-08-25T161903_ICELAND.yaml

@@ -11,7 +11,7 @@ seconds_before_event: 20
 seconds_after_event: 20
 event_latitude: 64.612
 event_longitude: -17.472
-event_depth_km: 50.0
+event_depth_km: 5.0
 event_magnitude: 4.6
 networks: Z7
 stations: RODG,DYSA,LIND,LOKT,LAUF,KALF,HELI,FAG,SVIN,K250

pysep/configs/mtuq_workshop_2022/2017-09-03T033001_NORTH_KOREA.yaml

@@ -19,7 +19,7 @@ channels: BH?
 locations: '00'
 reference_time: '2017-09-03T03:30:01.760000Z'
 seconds_before_ref: 100
-seconds_after_ref: 300
+seconds_after_ref: 1000
 phase_list:
 - ttall
 mindistance_km: 0

pysep/configs/template_config.yaml

@@ -1,48 +1,48 @@
-# PYSEP Config file - Template v0.1.0
+# PySEP Template config v0.5.0
+event_tag: null
 config_file: null
 client: IRIS
 client_debug: false
 timeout: 600
+taup_model: ak135
+use_mass_download: false
 event_selection: default
-origin_time: 2009-04-07T20:12:55.351
+origin_time: null
 seconds_before_event: 20
 seconds_after_event: 20
-event_latitude: 61.45420
-event_longitude: -149.7428
-event_depth_km: 33.033
-event_magnitude: 4.6
-networks: 'AK,YV,AV,AT,XZ,PN'
+event_latitude: null
+event_longitude: null
+event_depth_km: null
+event_magnitude: null
+networks: '*'
 stations: '*'
-channels: 'BH?'
+channels: '*'
 locations: '*'
 reference_time: null
 seconds_before_ref: 100
 seconds_after_ref: 300
-phase_list: null
-llnl_db_path: /store/raw/LLNL/UCRL-MI-222502/westernus.wfdisc
-min_dist: 0
-max_dist: 300.0
-min_az: 0
-max_az: 360
-min_lat: null
-max_lat: null
-min_lon: null
-max_lon: null
+phase_list:
+- ttall
+mindistance_km: 0
+maxdistance_km: 20000.0
+minazimuth: 0
+maxazimuth: 360
+minlatitude: null
+maxlatitude: null
+minlongitude: null
+maxlongitude: null
 demean: true
 detrend: true
-taper_percentage: 0
+taper_percentage: 0.0
 rotate: null
 remove_response: true
 output_unit: VEL
 water_level: 60
 pre_filt: default
-scale_factor: 100.0
-resample_freq: 50
-remove_clipped: false
-output_dir: ./
-write_files: !!set
-  all: null
-plot_files: !!set
-  all: null
-log_level: DEBUG
-event_tag: null
+scale_factor: 1
+resample_freq: null
+remove_clipped: true
+remove_insufficient_length: true
+remove_masked_data: true
+fill_data_gaps: false
+gap_fraction: 1.0

pysep/pysep.py

@@ -532,7 +532,7 @@ def __init__(self, config_file=None, event_selection="default",
         self.write_files = write_files
         self.plot_files = plot_files
         self.log_level = log_level
-        self._legacy_naming = legacy_naming
+        self.legacy_naming = legacy_naming
         self._overwrite = overwrite
         self._overwrite_event_tag = overwrite_event_tag
 
@@ -1580,7 +1580,7 @@ def write(self, write_files=None, _return_filenames=False, _subset=None,
                 self.st.write(fid, format="MSEED")
 
         # Used for determining where to save SAC files
-        if self._legacy_naming:
+        if self.legacy_naming:
             _output_dir = self.output_dir
         else:
             _output_dir = os.path.join(self.output_dir, sac_subdir)
@@ -1632,7 +1632,7 @@ def _write_sac(self, st, output_dir=os.getcwd(), components=None):
         for tr in st:
             if components and tr.stats.component not in components:
                 continue
-            if self._legacy_naming:
+            if self.legacy_naming:
                 # Legacy: e.g., 20000101000000.NN.SSS.LL.CC.c
                 _trace_id = f"{tr.get_id()[:-1]}.{tr.get_id()[-1].lower()}"
                 if self.event_tag:
@@ -1680,10 +1680,9 @@ def write_config(self, fid=None, overwrite=False):
                     if not key.startswith("_")}
         # Internal attributes that don't need to go into the written config
         attr_remove_list = ["st", "st_raw", "event", "inv", "c", "write_files",
-                            "plot_files", "output_dir", "station_ids", "kwargs"]
+                            "plot_files", "output_dir", "station_ids", "kwargs",
+                            "llnl_db_path", "log_level", "legacy_naming"]
 
-        if self.client.upper() != "LLNL":
-            attr_remove_list.append("llnl_db_path")  # not important unless LLNL
         for key in attr_remove_list:
             del(dict_out[key])
         # Write times in as strings not UTCDateTime
@@ -1736,7 +1735,7 @@ def _event_tag_and_output_dir(self):
         # Default behavior, auto-generate event tag
         if self._overwrite_event_tag is None:
             # Options for choosing how to name things. Legacy or new-style
-            if self._legacy_naming:
+            if self.legacy_naming:
                 logger.debug("reverting to legacy style file naming")
                 event_tag = format_event_tag_legacy(self.event)
             else: