v0.6.0 Release #92
Merged
v0.6.0 Release #92
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Dev get records
* update version in prep for pipestat table * add table function * add initial stats function from looper, remove counter * fixing _create_stats_summary and get_file_for_project * fix sample_level stats reporting * add object reporting * remove redundancies * lint * add assertion for table generation to pre-existing test * use pipeline_type when retrieving samples * Fix stats table generation output to look better * fix return types * fix list vs List * fix typo for stats table * fix doc strings * update func names for disambiguation * fix key error issue in html_reports * clean up * adjust docstrings * update docstrings Co-authored-by: Nathan Sheffield <[email protected]> * adjust docstrings, rename html_reports_pipestat.py to reports.py * adjust LOGGER.info string * simplify objs and stats generation * remove old todos * remove sample_name, add pipeline_name to project field definitions * work towards using project_name instead of sample_name from project-level pipelines * fix check record bug, add better error message for pipeline type. * add checks for pipeline types, lint * fix get_samples with pipeline type * fix fetch_pipeline_results * add basic tests for project_level, fix associated bugs * allow table creation to use project_name if applicable * add passing project name to file_backend * refactor to use r_id as input * update doc strings * more refactoring of records vs samples to be more general * move table generation to reports.py * allow status file dir path to be associated with config path OR the file path to ensure looper compatibility. * add output_dir as a parameter for placing reports and stat files. Otherwise defaults to results_file dir or config_file dir * add obtaining schema_path from pipestat config for Looper compatibility * added select function to main pipestat class * added select_txt and select_distinct function to main pipestat class * added get_one_record function to main pipestat class #70 * Removing CLI req that one must supply config and schema because the user can supply schema within the config. For Looper compatibility. * Revert "added get_one_record function to main pipestat class #70" This reverts commit b477060. * Revert "added select_txt and select_distinct function to main pipestat class" This reverts commit 4588f4f. * Revert "added select function to main pipestat class" This reverts commit ee98145. * Add basic glossary page in the form of a table. Addresses: pepkit/looper#290 * remove redundant tag * add RecordNotFoundError across both backends #74 * Add PipelineTypeNotSuppliedError to hide dbbackend information #74 * Initial work towards a fastapi implementation of pipestat reader #22 * Initial work to separate classes #78 * working implementation of child classes for reporting using DBBACKEND. Manual testing confirmed. pytest Tests broken. #78 * implement report for filebackend * fix inheritance issue with pipeline_type * typo * major refactoring, many tests still broken #78 * Fix signatures for report and explicitly state input arguments. * remove pipeline type check * add projectpipestamanager and refactor project_name to record_identifier * fix get_status * fix tests * fix all tests #78 * add todo * code cleanup and remove getting_table_name * clean up * partial removal of pipeline_type logic from DB backend * remove pipeline_type input arguments from File Backend * clean up * add simple class to wrap SamplePipestatManager and ProjectPipestatManager * change pipestatmanager to mutable mapping #78 * lowercase attributes #78 * polish PipestatBoss #78 * env_vars for pipeline_type * update docs * working proof of concept to retrieve results #22 * update readme and add getting output schema * add else catch for outptu schema and fix typo * add ability to run with uvicorn if calling reader.py directly. * add __init__.py and main func * add more endpoints #22 * update endpoints, add Query * allow for more complex filtering using post and pydantic models #22 * Add image and file endpoints #22 * lint * Add basic arg parser to pass absolute path to config file * Add endpoint for get_records() * clean up * modify get_records to return new structure #75 * fix html report generation using get_records * attempt global creation of psm, does not work * fix creation of global psm * change fetching by filetype * add cli option for pipestat serve * simplify and fix pipestat serve #22 * add host and port arguments #22 * add config error * update readme --------- Co-authored-by: nsheff <[email protected]>
* first pass to allow for output_schema as a true JSON schema #85 * remove unused import * simplify logic, extend _safe_pop_one_mapping to handles multiple keys * disambiguation key vs keys in parsed schema * make samples and project objects instead of arrays, add more test assertions, clean up docstrings. * add status data to string representation of ParsedSchema
* begin work on file inking via pipestat * continue implementing link for an output_dir OR the results_file.yaml #89 * implemented symlinks for filebackend given output_dir #89 * implemented symlinks for filebackend using results.yaml, polish tests #89 * fix typo, clean up * more lcean up * begin rework to be backend agnostic * refactor to place in folders specific to a result_identifier * add more complex types, add temp directory for better testing * fix recursive finding of paths * clean up, move to abstract class, confirm works for both backends * remove unused test files * add jinja2 to requirements-all.txt #91 * complex example with collision * add warning * add cli option * remove unused functions
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #92 +/- ##
===========================================
- Coverage 88.32% 49.64% -38.68%
===========================================
Files 17 21 +4
Lines 2252 3501 +1249
===========================================
- Hits 1989 1738 -251
- Misses 263 1763 +1500 ☔ View full report in Codecov by Sentry. |
…are not parsing correctly, tests broken #93
Fix reporting in pipestat summarize
…e", add try except
|
Closes milestone: https://github.com/pepkit/pipestat/milestone/3 |
This was referenced Dec 22, 2023
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[0.6.0] - 2023-10-XX
Added
select_records, which allows for a single API for selecting attributes (result_identifiers) given filter_conditions and/or columnsretrieve_one, andretrieve_manywhich allows for selecting one or multiple records given record_identifierpipestat serve --config "config.yaml"SamplePipestatManagerandProjectSamplePipestatManagerwhich are sample/project specific PipestatManager objects.SamplePipestatManagerandProjectSamplePipestatManagerclasses.to_dictmethods to parsed schema object.select_distinctfunction which retrieves unique results for a list of attributes.pipestat linkwhich creates a directory of symlinks for reported resultslist_recent_resultswhich allows for retrieving records filtered via a start and end timepsm["sample1", "name_of_something"] = "name_of_something_string"orresult = psm["sample1"]Fixed
pipeline_namecolumn not populating in postgres db backend.Changed
retrieve,get_one_record,get_records functionget_ormand replace withget_modelget_table_namefunctionsample_name->record_identifierpipeline_typehas been removed from most functions