- Fixed the config_server on osx.
- Added tox to testing install option to fix auto PyPi builds.
- Config server not properly saving entries to local file.
- Config server doesn't initially parse config file so it doesn't need to deserialize on response.
- The
panoptes-utils image watch <PATH>
command with default processing that will convertCR2
files toJPG
andFITS
and then plate-solve theFITS
files. - GHA downloads a
CR2
file for testing. - Plot directives for documentation.
- Testing now includes
--test-solve
for plate-solving in GHA. - Local tests only uses
memory
database. - Rearranged some functions in the
panoptes.utils.images
namespace.
- Unused stamp plotting functions.
- Testing of config servers on GHA.
- CountdownTimer.is_non_blocking predicate that wasn't being used.
- Extra serial protocol handlers.
panoptes-utils
cli with sub-commands for converting CR2 to FITS and for plate-solving FITS.
- Updating
pyscaffold
to4.2.3
, which allows forpyproject.toml
andtox
usage. - Updating example SerialDevice notebook and fixing deps.
- Config server can now run on arbitrary ports better.
- Removing unused functions: DelaySigTerm, strings_to_params.
- Fixes to support 420 namespaces and pyscaffold.
- Don't deserialize all strings as dates unless they actually look like a date.
- This release removes the Docker components from the repository, pushing some of the dependencies to
POCS
. This is an attempt to make this module slimmer and more useful. - The
astrometry.net
plate solving tests have been removed since we no longer build and test against a Docker image. The plate solving will continue to transition to the self-containedplate-solver
repository.
- New
panoptes.utils.serial.device.SerialDevice
that will replace thers232.SerialData
. New class uses the threaded reader with optional callback for better asynchronous reading from a serial device. Example notebook included. #274 ObservationPathInfo
allows for easy parsing of the standard PANOPTES file name scheme, as provided byPATH_MATCHER
. #282extract_metadata
pulls FITS headers into a common dictionary structure. #282- Docker updates: Add
PYTHONUNBUFFERED
to docker files; Removeversion
anddeploy
keys from docker compose files. #284
- The serial protocol handlers were moved to the
panoptes.utils.serial.handlers
namespace. #274 - Testing Dockerfile has
privileged
permission to get deviceloop
. #275 - Dockerfile: update
conda
in Dockerfile before installing environment; installpanoptes-utils
module in user-editable mode. #277 - Dockerfile: use
condaforge/miniforge3
as the base, which reduces image size. Push multi-stage builds for better caching. #278, #279 - Consistent multi-stage names in Dockerfile; added
jupyter_console
. #280 - Change behavior of
parse_config_directories
so thebase
entry must always exist or be missing (in which case the cwd is used). Remove themust_exist
param. #283
- Don't nest storage objects for the
PanFileDB
. #280
panoptes.utils.rs232.find_serial_port()
can be used to look up a serial port from the vendor and product hex ids. #269
panoptes.utils.config.config.client.get_config()
changed sodefault
is the second parameter and function made less noisy overall. #272- Updated to
Pillow>=8.1.1
for dependabot alert. #273.
panoptes.utils.logging
which is just replaced byfrom loguru import logger
. # 272
- Added an
RGB
IntEnum for easy iterating and consistent array access of rgb data. #265 - Added
save_rgb_bg_fits
that will save a FITS files with seven extensions: combined rgb map, and then the background and rms maps for each color. #265
get_rgb_background
only accepts adata
argument and a filename can no longer be passed. #265- Updated defaults for
get_rgb_background
. #265 get_stamp_slice
hasas_slices
param added with defaultTrue
for legacy behavior. IfFalse
then just the four points are returned. #265- Test coverage will skip noqa markers and pytest will run all tests. #265
- Added ruamel.yaml to base dependencies instead of just
config
extras. #265
- The timeout parameter is now passed from get_solve_field to solve_field. #266
- Docker musical chairs:
- Add Dockerfile to be
panoptes-utils
but don't use for testing. (#264) - Generic
panoptes-utils
withpanoptes-config-server
as default command example. (#264) - Move conda
environment
file intodocker
folder. (#264)
- Add Dockerfile to be
parse_config_directories
no longer modifies dictionary in place. (#264)
- Clean out the Contributing guide to point to POCS. (#264)
- Removing
pillow<7
requirement. (#264)
- Removing unused new cli experiment. (#264)
- Removed all
PANDIR
andPANLOG
references. Closes #263. (#264) - Removed
astroplan
from dependencies. (#264)
- Conda environment file (@wtgee #260)
- A cli script for
panoptes-utils
. This will eventually take over thepanoptes-config-server
and any other simple commands needed. For now can usepanoptes-utils tests run
for building and running tests.
- Updated testing to use specific docker image. Updated run script for more specific options on starting testing config server. (@wtgee #260)
- Different
extras
install options:config
,docs
,images
,testing
, andsocial
. (@wtgee #260) - Use
loguru
for logging directly in all files, rather than needless import. (@wtgee #260) - Make the
CountdownTimer.sleep()
less noisy. (#259)
- Plate solving extras option parsing correctly. (@wtgee #260)
- Explicit imports for some of the utils. (@wtgee #260)
- Remove Docker services except for testing. (@wtgee #260)
- Removed unused files: (@wtgee #260)
- Example data fetching notebook removed.
- Hotspot script removed.
- Data (astrometry index and IERS) download removed.
wait-for-it.sh
script removed.- TheSkyX utils moving to POCS directly.
- Removed
moving_average
. - Removed pipeline functions for getting
image_id
andsequence_id
.
- Added
oh-my-zsh
install file directly to ease some issues with GCP builds. (@wtgee #257) - Added
source-extractor
to dependencies but with no custom config files. (@wtgee #257) - Config Server:
- Option to start a heartbeat or not. (@wtgee #248)
- Reverting back to
python=3.7
for compatibility w/ GCP notebooks. (@wtgee #255) - Freezing
astropy<=4.0.1
while we wait forastroplan
to get pushed. (@wtgee #255) - Changed the horizon module to use numpy interpolation so we don't need to explicitly install scipy. (@wtgee #248)
altaz_to_radec
accepts astropy quantities. (@wtgee #250)- Downloaded helper script doesn't have
python3
hardcoded. (@wtgee #250) - Docker Tools (@wtgee #248):
- Conda environment built from
resources/environment.yaml
. (@wtgee #252) - Adds a "developer" dockerfile and compose file to install things for developers. (@wtgee #248)
- Docker CMD will run ipython. (@wtgee #248)
- docker-compose file will start a jupyter-lab instance. (@wtgee #248)
- Conda environment built from
- Fixed the
oh-my-zsh
path for Docker install. (@wtgee #256) - Return testing output from docker container, passint exit status. (@wtgee #256)
- The
stars
module, which has been moved topanoptes-pipeline
. (@wtgee #251) - The
metadata
module, which has been moved topanoptes-pipeline
. (@wtgee #252) - Docker Tools (@wtgee #248):
- Remove
source-extractor
frompanoptes-utils
and move topanoptes-pipeline
. (@wtgee #252) - Remove
imagemagick
frompanoptes-utils
. This is used for adding titles to JPGs. (@wtgee #252) - Don't install a separate conda environment, just use the base to help reduce image size, complexity. (@wtgee #252)
- Cleanup unused dependencies. (@wtgee @252)
- Remove
- Testing:
- Adios travis! (@wtgee #252)
- Add bit_depth argument to mask_saturated, no longer convert to float64 by default (@AnthonyHorton #244)
- Single cloudbuild file for both
panoptes-base
andpanoptes-utils
. (#242) - Add
astropy
channel. - Remove the miniforge installer from the docker image and clean up build args. (@wtgee #245)
- Changed relative to absolute imports. (@wtgee #246)
- Config server startup controlled via envvars, incorporating
python-dotenv
(@wtgee #241):PANOPTES_CONFIG_HOST
andPANOPTES_CONFIG_PORT
PANOPTES_CONFIG_FILE
- Config server updates (@wtgee #241):
- Config server with project dir mounted can be started via
docker/docker-testing.yaml
. - Config server flask instances are run on
gevent
wsgi server instead of development server. host
andport
are specified at the top-level command, e.g.panoptes-config-server --host foobar --port 9999 get
.host
andport
respect the above envvars above.- Options changed from
auto-save
andignore-local
tosave-local
andload-local
. run
command adds the``config_server.running=True`` entry to the server.stop
command added that setsconfig_server.running=False
to break loop.
- Config server with project dir mounted can be started via
- Testing (@wtgee #241):
- All testing is started from
scripts/test-software.sh
. - The
panoptes-config-server
is started as an external service, not in the pytest conf. - Added a
tests/env
file that is used by the docker compose file for setting vars inside the running containers. - Config server uses
tests/testing.yaml
for all testing. - Testing logs are stored in
./logs
, relative to the project root outside the container. - Coverage file is stored in
/var/panoptes/logs
. - A
scripts/wait-for-it.sh
script added to check that config server is running properly before starting tests.
- All testing is started from
scripts/setup-local-environment.sh
properly uses new base image if requested. (@wtgee #241)- Docker images: fixed the created
ssh
directory for$PANUSER
. (@wtgee #240)
This release is mostly cleanup and testing of our autobuild features.
- Splitting the
panoptes-base
files into separate folder. (#238) - Consolidate the GitHub Actions for building and publishing a release package. (#239)
- Fix Github Actions for building releases. (#238)
- Google Cloud Build of Docker images.
panoptes-base
andpanoptes-utils
are built for each PR as well as on merges todevelop
andmaster
. (#237)
- GitHub Actions
- If a semantically tagged branch is pushed to GH, a release will automatically be generated and a package will be built and sent to PyPi. (#237)
- Changelog fixes. (#237)
- Simplified docker docker images. (#227)
- Consolidation of Dockerfile to support images:
panoptes-base
serves as a base image for all docker services.panoptes-utils:latest
installs editablepanoptes-utils
module from githubdevelop
branch.panoptes-utils:develop
is used for testing and can be built locally with thedocker/setup-local-environment.sh
script.- Tests on GH and Travis use the
docker/setup-local-environment.sh
script for building test images. - miniforge used to install a
conda
environment withconda-forge
as default channels. Forces 64bit support. - Multi-arch builds are supported on
gcr.io
via thecloudbuild.yaml
file. Built withbuildx
plugin to docker. Currentlylinux/amd64
andlinux/arm64
. - Extra zsh plugins in the docker image.
- Properly disable auto-update of zsh.
- Cleanup of entrypoint for better loading of environment.
Pillow
fights.
- Dependencies:
pyarrow
too hard to build on arm.hvplot
andholoviews
not needed in default install.
- Changed
dir
todirectory
in disk space check. (#226) - Pass the git folder to the build context when making local docker images. (#226)
- Added
arm64
build for Docker based offubuntu
image. (#223)
- Docker
- Changed base image to
ubuntu
. (#223) amd64
andarm64
images built by default. (#223)- Ubuntu has changed
sextractor
tosource-extractor
(yay). (#223)
- Changed base image to
- Config Server
- Better parsing of directories entry in config server. (#222)
- Make config server less noisy. (#222)
- Bump PyYaml to latest for security warning. (#222)
- Remove pendulum because too hard to build on arm processors. (#223)
Moving to python 3.8.
- Breaking Python minimum version changed to
3.8
. (#217) - Running pytest locally will generate coverage report in terminal. (#218)
- Lots of documentation. (#218)
- Removing the environment section from the readme. (#218)
- Config Server (#217)
- Better logging.
- Cleaning up doctests.
- Removing all dynamic server items from this repo as they are not needed.
- Wait for config_server to start.
- Fixing starting within fixture.
- Config items no longer assume any defaults for either directories or files. A config file name is always required and it should always be an absolute path. (#218)
- Adding test file for config items. (#218)
panoptes-config-server
re-worked and now includesrun
,get
, andset
subcomamnds. (#221)
- Testing (#218)
- Log files are rotated for each testing run.
- Fix env vars (mostly need to make sure the
export
option exists in theenv
file. - Pytest commands moved to
setup.cfg
instead ofrun-tests.sh
- Remove old markers
- Setting
--strict-markers
options. - Add
astrometry
marker for tests requiring solve andtheskyx
marker for running alongside TheSkyX. - Coverage reports generated in xml and output in terminal.
- Serializers update. (#217)
- Make the parsing and serializing functions public.
- Use pendulum for parsing times instead of astropy Time.
- Better naming of public functions. (#218)
Straight past 0.2.19
.
- Removed
bin/panoptes-config-server
and created an entry_point insetup.cfg
. (#212) - Removed old developer items in favor of those in
panoptes-pocs
. (#212) - Consolidate docker files, consistent naming with other repos. (#210, #212)
0.2.16
was released with an error and this is a hotfix.
- Added CR2 file testing to GitHub Actions. (#125, #205)
- A
wait_for_events
generic utility, mostly pulled from POCS. (#92, #206) * Supports singlecallback
that can be used for interrupting, custom logging, etc. (#208)
- Remove the
validate_collection
requirement from the database types, making any collection is now valid. (#204) - Rearrange some of the
panoptes.utils.database
modules. (#204)
- Remove
error.InvalidCollection
. (#204) - Unused items in
conftest.py
. (#204)
- Convert to pyscaffold. (#198)
- Proper namespace package
panoptes
. - Move items to
src
folder. - Fix version number.
- Fix build.
- Fix documentation #27.
- Move all project config to
setup.cfg
. - Base Docker image is run by root only.
- Added a
testing
Dockerfile and cleaned uplatest
anddevelop
.
- Proper namespace package
- Breaking Removing all zmq based messaging services. (#200)
Add snappy decompression for parquet;
pyarrow
instead offastparquet
(#193)Password-less sudo for panoptes user on dev docker image (#193)
get_metadata
has an optional progress bar. (#194)Add
bayer.get_stamp_slice
for getting a stamp slice while respecting the superpixel. This was removed awhile ago and has been re-added and improved. (#196) * Adjusting the offsets so the center pixel is always:G2 B R G1
- Fix time-based search (#193)
- Fix the build (#197)
* Removed
versioneer
in favor ofsetuptools-scm
for workin version numbers. * Removed the MANIFEST.in * Added a simplepyproject.toml
.
- Breaking Only support getting stars directly from the WCS, not the footprint. (#194)
*
get_stars_from_footprint
->get_stars_from_wcs
* Better logging * Consistent column names for dtypes * Vmag bin comes from sql. * Allow for different RA/Dec column names. * Better catalog match function. sextractor
param changes. (#194)- Breaking
panoptes.utils.logger
->panoptes.utils.logger
so we canfrom panoptes.utils import logger
(#197) - Breaking The
panoptes.utils.data.assets
module was removed and the Downloader
class is placed directly within thescripts/download-data.py
file. (#197)
- Breaking The
- The
panopes-utils
module is not installed in editable mode in thelatest
docker image. (#197) * Slight clean up of latest.Dockerfile
- Fix some passing of options between
get_solve_field
andsolve_field
that was leading to double parameter issues. (#189)
- The
panoptes.utils.data
functions use static versions of the file rather than firestore. (#192) - Updated development environment (#191)
get_metadata
filter the fields at the parquet level. (#194)
Quick release to get the panoptes.utils.sources
into the package.
panoptes.utils.sources
not included in package. (#187, #188)
- Ability to pass credentials to underlying google client functions. (#187)
- Data
- Added basic data access components for getting observation and image metadata. (#178, #181)
- Added a
search_observations
function for searching by various criteria. (#181) - Uses anonymous credentials to connect to firestore.
- Added a basic notebook demonstrating features.
- Added a
- Adding
holoviews
andhvplot
as required dependencies.
- FITS Utils fixes:
- Fix docstring return types for some functions. (#173)
fpack
/funpack
andget_solve_field
were not properly overwriting FITS files- under certain conditions when an uncompressed file of the same name was present alongside the compressed version. (#175)
- Properly pass
args
andkwargs
toastropy.io.fits.getdata
. (#180)
- Docker
- Changed developer tag from
dev
todevelop
. (#174)
- Changed developer tag from
- FITS Utils changes (#173):
- Uncompressed file is always used for solve because we were occasionally seeing odd errors as described in dstndstn/astrometry.net#182. (#173)
warning: get_solve_field
willoverwrite
by default.- Better log output for solving.
- Better checking for solved file at end (via
is_celestial
). - Cleanup the cleanup of solve files, removing
remove_extras
option. - Pass
kwargs
to underlyingwriteto
method forwrite_fits
. Needed for, e.g.overwrite
. - Allow additional options to be passed to solve field functions without having to override all options. (#180)
- Changed default options in
get_solve_field
to usescale-low
andscale-high
instead ofradius
(which - requires an
ra
anddec
). (#180)
- Changed default options in
- Changed
bin/panoptes-dev
->bin/panoptes-develop
for naming consistency. (#175) - Data
- BREAKING The
panoptes.utils.data.py
has moved into thepanoptes.utils.data
namespace with the relevant existingDownloader
class placed in theassets.py
module. (#181) - Changed the
get_data
(and images and observations equivalent) toget_metadata
. (#181)
- BREAKING The
- FITS Utils removals (#173):
- Removing unused and confusing
improve_wcs
. - PanLogger class moved to POCS. (#186)
- Removing unused and confusing
get_stars_from_footpr int
can accept aWCS
directly instead of just the output fromcalc_footprint()
. (#164)- Ability to create different tags for the docker image. The
develop
directory is now used to create adevelop
image and is provided along withlatest
. (#165) get_rgb_backgrounds(return_separate-True)
will now return theBackground2D
objects. (#166)- Added BigQuery pandas dependencies. (#168)
- Added a developer image at
panoptes-utils:dev
, which is also auto-built along with thelatest
in the cloudbuild. Offers ajupyter-lab
instance along with a number of plotting modules. Can be easily started viapanoptes-dev
. (#170, #171)
image_id_from_path
andsequence_id_from_path
can recognize a zero in thecamera_id
andNone
when no match. (#163)- Fixed the bigquery client param for star lookup. (#164)
- Unquote paths before id matching. (#169)
- Do WCS match for all unmatched sources, not just matched sources. (#172)
- Docker entrypoint no longer tries to activate service account if
$GOOGLE_APPLICATION_CREDENTIALS
is found. The python client libraries will recognize the env var so this means we can avoid installinggcloud
utilities just to activate. (#165) - The
sources
module does not require a BigQuery client to be passed but can start it's own. A warning is given if$GOOGLE_APPLICATION_CREDENTIALS
is not found. (#167) lookup_point_sources
updates: default vmag range expanded so less false positive matches [4,18). (#168)- Removed TOC from changelog. (#170)
- Sextractor param changes: (#171)
* Threshold for detection changed from 3 pixels to 10 pixels.
* Seeing changed from 0.7 arcsec to 15.3 arcsec. (Isn't used.)
* Removed
class_star
from sextractor results.
Pointless version bump because of issue with PyPi.
Thanks first-time contributer @preethi524! 🎉
- Ability to return separate RGB backgrounds. (#162)
- Increase coverage. (#161)
- Basic serialization of
Exception
. (#160)
- Add
args
andkwargs
toget_rgb_background
. (#160)
get_rgb_background
added to thebayer
module. (#158)getwcs
thin-wrapper added tofits
module. (#158)- Added
sources
utils. (#158)
- Changed scope of test data files to
function
. (#158)
- Docker
- Change to
python:3.8-slim-buster
for base image. Onlyamd64
support for now. (#155) - Simplified docker files. (#155)
- Switching from Travis to GHA: (#155)
- Travis builds docker image before testing.
- Travis doesn't upload coverage.
- Don't update module inside container during entrypoint.
- Fixed user permissions for $HOME and $PANDIR. (#155)
- The docker container only really likes it when user id
1000
is running the system. - Remove GCP Cloud SQL proxy support.
- Installed
sextractor
. (#158) - Added
pandas
. (#158) - Default
panoptes
user has passwordpanoptes
. (#158)
- Change to
- Docker (#155)
- Remove anaconda
- Polar alignment utils (#156)
- Github Actions testing and coverage upload. (#145) * Log files for testing are created as an artifact.
PanLogger
helper class added. Mostly handles formatting but can also track handlers. (#145)
- Fixed top-level namespace so we can have other
panoptes
repos. (#150, fixes #137)
- Data files for testing are copied before tests. Allows for reuse of unsolved fits file. (#144)
- Fix astrometry data file directories in Docker images. (#144)
- The docker image no longer updates
panoptes-utils
when usingrun-tests.sh
. (#145)
- Disallow zipped packages, which also interfere with namespace (#142)
photutils
dependency for rectangular apertures in theshow_stamps
method.
Small point release to correct namespace and remove some bloat.
- Fixed top-level namespace so we can have other
panoptes
repos. (#137)
- Dependencies that will be deprecated soon and are causing bloat:
photutils
,scikit-image
. (#138)
- Fixed top-level namespace so we can have other
panoptes
repos (#137, #150).
Mostly some cleanup from the v0.2.0
release based on integrating all the changes into POCS.
- Misc bugs introduced as part of last release, including to
download-data.py
script. - Custom exceptions now properly pass
kwargs
through to parent (#135).
- New script for downloading data,
scripts/download-data.py
. This helped resolve some issues with the relative imports introduced inv0.2.0
and is cleaner. (#129) - All dependencies are smashed into one "feature" in
setup.py
to makepip-tools
work well. This will fix the docker image problems introduced inv0.2.1
. (#136)
- The
get_root_logger
and associated tests (#134).
First big overhaul of the repository. Pulls in features that were duplicated or scattered across POCS and PIAA. Removes a lot of code that wasn't being used or was otherwise clutter. Overhauls the logging system to use [loguru
](https://github.com/Delgan/loguru) so things are simplified. Updates to documentation.
- Config Server
- See the description in the [README](README.md)
- Versioneer for version strings (#123).
- Read the docs config (#123).
- IERS Mirror (#65, #67)
- Docker updates
- See #68 and #75 for list.
- Logging:
- Switch to loguru. This simplifies our logging system. Also gives us access to the
trace
(lower thandebug
, good for hardware and other debug we don't need to see during operation) andsuccess
(higher thaninfo
) levels, which would be nice to start implementing. (#123) - Consistent use of relative imports. (#123)
- Documentation updates. (#97, #119, #120, #123)
- Repo cleanup. (#97, #123)
- Using GitHub Actions for testing. (#100, #101)
- Using pip-tools for dependency management.
Changes and cleanup on the way to a (more) stable release. See 0.2.0
for list of changes.
Bringing things in line with updates to POCS
for docker and panoptes-utils
use.
- Serial handlers move to panoptes-utils from POCS.
- Tests and coverage.
improve_wcs
(moved from PIAA).~utils.fits.getdata
to match other fits convenience functions, allowing for fpack files.
- Serialization fixes.
- Use our serialization everywhere (e.g. messaging)
- Closes #panoptes/POCS/issues/818
- Closes #panoptes/POCS/issues/103
- Setup/Install:
- Scripts are renamed to have
panoptes
prefix. - Scripts are installed as part of setup.
- Script improvements to make more robust and portable.
- Scripts are renamed to have
- Docker Updates:
- Don't use anaconda.
- Testing:
- Overhaul of config_server use in testing.
- Testing config file is separated from any regular config files.
- Logging:
- Silence some 3rd party logs.
- Added bayer utilities. 📷
- Added Cloud SQL utilities. ☁️
- Breaking Changed namespace so no underscores, i.e.
from panoptes.utils import time
. - Docker updates:
- Use slim python images and not anaconda on amd64.
- Adding zsh as default shell along with some customizations.
- Entrypoint script properly authenticates to google cloud if possible.
- Added amd64 only build scripts.
- Docker containers created:
panoptes-base
for base OS and system packages, including astrometry.net and friends.panoptes-utils
for container containing base utilities.- Script for building containers in GCR.
- Consistent JSON and YAML serializers.
- Configuration Server (Flask/JSON microservice).
- Minimum Python version is 3.6
- Default PanDB type is changed to
memory
. - Documentation updates.
- Bux fixes and code improvements.
- Added a change log. Yay.
- Drop
orjson
and revert tojson
for the JSON serializers.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).