-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add configuration example in docs and improve how-to section
- Loading branch information
1 parent
1f65be6
commit b211e25
Showing
3 changed files
with
86 additions
and
32 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,41 +1,60 @@ | ||
## Command line | ||
--- | ||
hide: | ||
- navigation | ||
--- | ||
### Conversion | ||
|
||
You can use the odpy command within a terminal to access the different functions of the `ocean-data-parser`. | ||
??? question "Convert files to netcdf" | ||
|
||
For more info use the command: | ||
``` | ||
odpy --help | ||
``` | ||
You can use the odpy command within a terminal to access the different functions of the `ocean-data-parser`. | ||
|
||
!!! Info | ||
For more details on how to use the command line interface see [commmand line section](cli.md). | ||
Convert all cnv files in subdiretories by using the seabird.cnv parser and save to output directory: | ||
```console | ||
odpy convert --input-path input/**/*.cnv --parser=seabird.cnv --output_path=output | ||
``` | ||
|
||
See [commmand line section](cli.md) for more detail or use the command: `odpy convert --help` | ||
|
||
## Import a specific parser | ||
??? question "Avoid reconverting over again the same files" | ||
|
||
As an example, to load a compatible file you can use the automated parser detection method: | ||
The `ocean-data-parser` provides a file retristry which can be used to: | ||
- track which files were converted and outputted where | ||
- error associated with each files | ||
- file modified time | ||
- file hash | ||
|
||
```python | ||
from ocean_data_parser import read | ||
If activated, a registry file (*.csv/*.parquet) will be saved and any time a conversion is rerun. odpy will first compare the available files to the already parsed files available within the registry and will only convert the ones which have changes. Those changes are primarily based on the file modified time, each modified file is then rehashed and if that hash is different the file will be reconverted and the output overwritten (default) | ||
|
||
# Load a file to an xarray object | ||
ds = read.file('Path to file') | ||
### Parser handling | ||
|
||
# Save to netcdf | ||
ds.to_netcdf('save-path.nc') | ||
``` | ||
??? question "Load any compatible files in my own project" | ||
|
||
!!! warning | ||
The parser detection method relies on the file extension and the first few lines present within the given file. | ||
|
||
Or specify the specific parser to use for this file format: | ||
``` python | ||
from ocean_data_parser.parsers import seabird | ||
To load a compatible file you can use the automated parser detection method: | ||
|
||
# Load a seabird cnv file as an xarray dataset | ||
ds = seabird.cnv('Path to seabird cnv file') | ||
```python | ||
from ocean_data_parser import read | ||
|
||
# Save to netcdf | ||
ds.to_netcdf('save-path.nc') | ||
``` | ||
The `ocean-data-parser` can then be used within either a python package, script or jupyter notebook. See [documentation Notebook section](https://cioos-siooc.github.io/ocean-data-parser) for examples on how to use the package within a jupyter notebook. | ||
# Load a file to an xarray object | ||
ds = read.file('Path to file') | ||
|
||
# Save to netcdf | ||
ds.to_netcdf('save-path.nc') | ||
``` | ||
|
||
:warning: The parser detection method relies on the file extension and the first few lines present within the given file. It is preferable to define a specific parser when a tool is used in production. | ||
|
||
|
||
|
||
??? question "Load a file with a specific parser in my own project" | ||
You can import a specific parser via the `ocean_data_parser.parser` | ||
``` python | ||
from ocean_data_parser.parsers import seabird | ||
|
||
# Load a seabird cnv file as an xarray dataset | ||
ds = seabird.cnv('Path to seabird cnv file') | ||
|
||
# Save to netcdf | ||
ds.to_netcdf('save-path.nc') | ||
``` | ||
The `ocean-data-parser` can then be used within either a python package, script or jupyter notebook. See [documentation Notebook section](https://cioos-siooc.github.io/ocean-data-parser) for examples on how to use the package within a jupyter notebook. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters