Skip to content

Commit

Permalink
Include readme in docs (xpublish-community#195)
Browse files Browse the repository at this point in the history
Updates the readme some, including fixing badges, and includes elements of the readme elsewhere in the docs to reduce duplication.
  • Loading branch information
abkfenris authored May 12, 2023
1 parent 2056222 commit 81ecbbc
Show file tree
Hide file tree
Showing 5 changed files with 104 additions and 98 deletions.
78 changes: 78 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Xpublish

Publish Xarray Datasets to the web

<!-- badges-start -->

[![PyPI](https://img.shields.io/pypi/v/xpublish)](https://pypi.org/project/xpublish/)
[![Conda](https://img.shields.io/conda/v/conda-forge/xpublish)](https://anaconda.org/conda-forge/xpublish)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/xpublish)
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/xpublish-community/xpublish/main)

[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/xpublish-community/xpublish/main.yaml?logo=github)](https://github.com/xpublish-community/xpublish/actions/workflows/main.yaml)
[![Documentation Status](https://readthedocs.org/projects/xpublish/badge/?version=latest)](https://xpublish.readthedocs.io/en/latest/?badge=latest)
[![](https://codecov.io/gh/xpublish-community/xpublish/branch/main/graph/badge.svg)](https://codecov.io/gh/xpublish-community/xpublish)
[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/xpublish-community/xpublish/main.svg)](https://results.pre-commit.ci/latest/github/xpublish-community/xpublish/main)

<!-- badges-end -->

## A quick example

**Serverside: Publish a Xarray Dataset through a rest API**

<!-- server-example-start -->

```python
ds.rest.serve(host="0.0.0.0", port=9000)
```

<!-- server-example-end -->

**Client-side: Connect to a published dataset**

<!-- client-example-start -->

The published datasets can be accessed from various kinds of client applications, e.g., from within Python using Zarr and fsspec.

```python
import xarray as xr
import zarr
from fsspec.implementations.http import HTTPFileSystem

fs = HTTPFileSystem()
http_map = fs.get_mapper("http://0.0.0.0:9000/zarr/")

# open as a zarr group
zg = zarr.open_consolidated(http_map, mode="r")

# or open as another Xarray Dataset
ds = xr.open_zarr(http_map, consolidated=True)
```

Or to explore other access methods, open [http://0.0.0.0:9000/docs](http://0.0.0.0:9000/docs) in a browser.

<!-- client-example-end -->

## Why?

Xpublish lets you serve/share/publish Xarray Datasets via a web application.

The data and/or metadata in the Xarray Datasets can be exposed in various forms through [pluggable REST API endpoints](https://xpublish.readthedocs.io/en/latest/user-guide/plugins.html).
Efficient, on-demand delivery of large datasets may be enabled with Dask on the server-side.

Xpublish's [plugin ecosystem](https://xpublish.readthedocs.io/en/latest/ecosystem/index.html#plugins) has capabilities including:

- publish on-demand or derived data products
- turning xarray objects into streaming services (e.g. OPeNDAP)

## How?

Under the hood, Xpublish is using a web app (FastAPI) that is exposing a
REST-like API with builtin and/or user-defined endpoints.

For example, Xpublish provides by default a minimal Zarr compatible REST-like API with the following endpoints:

- `zarr/.zmetadata`: returns Zarr-formatted metadata keys as json strings.
- `zarr/var/0.0.0`: returns a variable data chunk as a binary string.

Futher endpoints can be added by installing or writing plugins.
77 changes: 0 additions & 77 deletions README.rst

This file was deleted.

4 changes: 4 additions & 0 deletions docs/source/getting-started/tutorial/using-plugins.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Using Plugins

```{versionadded} 0.3.0
Plugins were released in [0.3.0](https://github.com/xpublish-community/xpublish/releases/tag/0.3.0).
```

Much of the power of Xpublish comes from it's [ecosystem](../../ecosystem/index) of plugins, which can quickly extend Xpublish with new capabilities.

```{note}
Expand Down
39 changes: 18 additions & 21 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

**Useful links:** [Installation](getting-started/installation) | [Source Repository](https://github.com/xpublish-community/xpublish/) | [Issue Tracker](https://github.com/xpublish-community/xpublish/issues) | [Q&A Support](https://github.com/xpublish-community/xpublish/discussions/categories/q-a?discussions_q=category%3AQ%26A+) | [Slack Channel](./ecosystem/index.md#slack)

```{include} ../../README.md
---
start-after: <!-- badges-start -->
end-before: <!-- badges-end -->
---
```

## Xpublish is

````{grid} 3
Expand Down Expand Up @@ -73,37 +80,27 @@ The contributing guidelines will guide you through the process of improving Xpub
````

### Xpublish lets you easily publish Xarray Datasets via a REST API.
## A quick intro

_You can run a short example application in a live session here:_ [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/xpublish-community/xpublish/master)

On the server-side, one or more datasets can be published using the
{class}`xpublish.Rest` class or the {attr}`xarray.Dataset.rest` accessor, e.g.,

```python
ds.rest.serve(host="0.0.0.0", port=9000)
```{include} ../../README.md
---
start-after: <!-- server-example-start -->
end-before: <!-- server-example-end -->
---
```

Those datasets can be accessed from various kinds of client applications, e.g.,
from within Python using Zarr and fsspec.

```python
import xarray as xr
import zarr
from fsspec.implementations.http import HTTPFileSystem

fs = HTTPFileSystem()
http_map = fs.get_mapper("http://0.0.0.0:9000/zarr/")

# open as a zarr group
zg = zarr.open_consolidated(http_map, mode="r")

# or open as another Xarray Dataset
ds = xr.open_zarr(http_map, consolidated=True)
```{include} ../../README.md
---
start-after: <!-- client-example-start -->
end-before: <!-- client-example-end -->
---
```

Or to explore other access methods, open [http://0.0.0.0:9000/docs](http://0.0.0.0:9000/docs) in a browser.

```{toctree}
---
caption: Documentation Contents
Expand Down
4 changes: 4 additions & 0 deletions docs/source/user-guide/plugins.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Plugins

```{versionadded} 0.3.0
Plugins were released in [0.3.0](https://github.com/xpublish-community/xpublish/releases/tag/0.3.0).
```

While {py:class}`fastapi.APIRouter` can get you started building new endpoints
for datasets quickly, the real extendability of Xpublish comes from it's plugin system.

Expand Down

0 comments on commit 81ecbbc

Please sign in to comment.