Skip to content

Commit

Permalink
Merge pull request #88 from e-koch/parallel_checks
Browse files Browse the repository at this point in the history 
Parallel checks and debugging for empty filament cases
  • Loading branch information
@e-koch
e-koch authored Sep 30, 2024
2 parents 8bfaa51 + cfdb88f commit a5d2598
Show file tree
Hide file tree
Showing 3 changed files with 128 additions and 62 deletions.
114 changes: 79 additions & 35 deletions examples/tutorial.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
},
{
"cell_type": "code",
"execution_count": 120,
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -53,7 +53,7 @@
},
{
"cell_type": "code",
"execution_count": 121,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -67,14 +67,14 @@
},
{
"cell_type": "code",
"execution_count": 122,
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/eric/Dropbox/code_development/filaments/build/lib.linux-x86_64-3.6/fil_finder/filfinder2D.py:138: UserWarning: No beam width given. Using 0 pixels.\n",
"/Users/ekoch/Library/CloudStorage/Dropbox/code_development/FilFinder/fil_finder/filfinder2D.py:150: UserWarning: No beam width given. Using 0 pixels.\n",
" warnings.warn(\"No beam width given. Using 0 pixels.\")\n"
]
}
Expand All @@ -96,7 +96,7 @@
},
{
"cell_type": "code",
"execution_count": 123,
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -106,25 +106,20 @@
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
"collapsed": true,
"jupyter": {
"outputs_hidden": true
}
},
"source": [
"If [spectral-cube](https://spectral-cube.readthedocs.io/en/latest/) is installed, the `Projection` or `Slice` classes can also be passed to `FilFinder2D`:"
]
},
{
"cell_type": "code",
"execution_count": 124,
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: A 'NAXIS1' keyword already exists in this header. Inserting duplicate keyword. [astropy.io.fits.header]\n"
]
}
],
"outputs": [],
"source": [
"from spectral_cube import Projection\n",
"\n",
Expand All @@ -142,12 +137,14 @@
"\n",
"Note that numerical inputs must be given as `~astropy.units.Quantity` object with the appropriate unit.\n",
"\n",
"**Distance** -- To facilitate conversions to physical units, a distance can be given to `FilFinder2D`:"
"### Distance \n",
"\n",
"To facilitate conversions to physical units, a distance can be given to `FilFinder2D`:"
]
},
{
"cell_type": "code",
"execution_count": 125,
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -158,25 +155,27 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"**Angular Scale** -- If no header information is given, the pixel-to-angular conversion can be given:"
"### Angular Scale\n",
"\n",
"If no header information is given, the pixel-to-angular conversion can be given:"
]
},
{
"cell_type": "code",
"execution_count": 126,
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/eric/Dropbox/code_development/filaments/build/lib.linux-x86_64-3.6/fil_finder/base_conversions.py:55: UserWarning: Cannot find 'BMAJ' in the header. Try installing the `radio_beam` package for loading header information.\n",
"/Users/ekoch/Library/CloudStorage/Dropbox/code_development/FilFinder/fil_finder/base_conversions.py:55: UserWarning: Cannot find 'BMAJ' in the header. Try installing the `radio_beam` package for loading header information.\n",
" warn(\"Cannot find 'BMAJ' in the header. Try installing\"\n",
"/home/eric/Dropbox/code_development/filaments/build/lib.linux-x86_64-3.6/fil_finder/base_conversions.py:63: UserWarning: Cannot find 'BMIN' in the header. Assuming circular beam.\n",
"/Users/ekoch/Library/CloudStorage/Dropbox/code_development/FilFinder/fil_finder/base_conversions.py:63: UserWarning: Cannot find 'BMIN' in the header. Assuming circular beam.\n",
" warn(\"Cannot find 'BMIN' in the header. Assuming circular beam.\")\n",
"/home/eric/Dropbox/code_development/filaments/build/lib.linux-x86_64-3.6/fil_finder/base_conversions.py:69: UserWarning: Cannot find 'BPA' in the header. Assuming PA of 0.\n",
"/Users/ekoch/Library/CloudStorage/Dropbox/code_development/FilFinder/fil_finder/base_conversions.py:69: UserWarning: Cannot find 'BPA' in the header. Assuming PA of 0.\n",
" warn(\"Cannot find 'BPA' in the header. Assuming PA of 0.\")\n",
"/home/eric/Dropbox/code_development/filaments/build/lib.linux-x86_64-3.6/fil_finder/filfinder2D.py:138: UserWarning: No beam width given. Using 0 pixels.\n",
"/Users/ekoch/Library/CloudStorage/Dropbox/code_development/FilFinder/fil_finder/filfinder2D.py:150: UserWarning: No beam width given. Using 0 pixels.\n",
" warnings.warn(\"No beam width given. Using 0 pixels.\")\n"
]
}
Expand All @@ -189,12 +188,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"**Beamwidth** -- If the major axis of the beam is contained in the header, it will be automatically read in. If that information is not in the header, the beam size can be passed separately:"
"### Beam width \n",
"\n",
"If the major axis of the beam is contained in the header, it will be automatically read in. If that information is not in the header, the beam size can be passed separately:"
]
},
{
"cell_type": "code",
"execution_count": 127,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -205,12 +206,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"**Custom Filament Masks** -- If you have a pre-computed filament mask, the mask array can be passed:"
"### Custom Filament Masks\n",
"\n",
"If you have a pre-computed filament mask, the mask array can be passed:"
]
},
{
"cell_type": "code",
"execution_count": 128,
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -226,12 +229,14 @@
"source": [
"The custom mask must have the same shape as the inputed image.\n",
"\n",
"**Save Name** -- A prefix for saved plots and table can be given:"
"### Save Name\n",
"\n",
"A prefix for saved plots and table can be given:"
]
},
{
"cell_type": "code",
"execution_count": 129,
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -247,7 +252,7 @@
},
{
"cell_type": "code",
"execution_count": 130,
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -261,6 +266,42 @@
"The beamwidth is $24''$ and is defined in the header."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Parallel Processing options\n",
"\n",
"`FilFinder2D` now allows operations over invidividual filaments to be parallelized with the `pool` and `nthreads` kwargs.\n",
"\n",
"The default is `None` which will trigger creating a `oncurrent.futures.ProcessPoolExecutor` with `nthread` processes (default is 1).\n",
"\n",
"Alternatively, you can use [`mpi4py.futures.MPIPoolExecutor`](https://mpi4py.readthedocs.io/en/stable/mpi4py.futures.html#mpi4py.futures.MPIPoolExecutor) as the `pool` when working in an MPI environment. By default the `max_workers` will be set by the [`MPI4PY_FUTURES_MAX_WORKERS`](https://mpi4py.readthedocs.io/en/stable/mpi4py.futures.html#envvar-MPI4PY_FUTURES_MAX_WORKERS) environment variable.\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"from concurrent.futures import ProcessPoolExecutor\n",
"\n",
"mypool = ProcessPoolExecutor(max_workers=2)\n",
"fil = FilFinder2D(hdu, save_name=\"FilFinder_Output\", pool=mypool)\n",
"\n",
"# is equivalent to\n",
"\n",
"fil = FilFinder2D(hdu, save_name=\"FilFinder_Output\", pool=None, nthreads=2)\n",
"\n",
"# Using mpi4py:\n",
"# from mpi4py.futures import MPIPoolExecutor\n",
"\n",
"# mypool = MPIPoolExecutor(max_workers=None)\n",
"# fil = FilFinder2D(hdu, save_name=\"FilFinder_Output\", pool=mypool)\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -272,7 +313,10 @@
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
"collapsed": true,
"jupyter": {
"outputs_hidden": true
}
},
"source": [
"Prior to creating the mask, it can be helpful to first *flatten* the image of bright compact sources. `FilFinder2D` uses an arctan transform, where the data are first normalized by some percentile value of the data:"
Expand Down Expand Up @@ -2169,7 +2213,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -2183,9 +2227,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 4
}
Loading

0 comments on commit a5d2598

Please sign in to comment.