Skip to content

Commit

Permalink
deploy: 6c8ae21
Browse files Browse the repository at this point in the history
  • Loading branch information
tinaok committed Oct 31, 2023
1 parent eaeef84 commit 4f0e137
Show file tree
Hide file tree
Showing 8 changed files with 2,650 additions and 2,650 deletions.
6 changes: 3 additions & 3 deletions _sources/part3/chunking_introduction.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -187,15 +187,15 @@
"__Chunking__ is splitting a dataset into small pieces. \n",
"\n",
"Original dataset, in one piece, \n",
"<img src=\"../figures/notchunked.png\" width=\"200\" height=\"100\">\n",
"<img src=\"https://github.com/pangeo-data/pangeo-openeo-BiDS-2023/tree/main/tutorial/figures/notchunked.png\" width=\"200\" height=\"100\">\n",
"\n",
"and we split it into several smaller pieces. \n",
"<img src=\"../figures/chunked.png\" width=\"200\" height=\"100\">\n",
"<img src=\"https://github.com/pangeo-data/pangeo-openeo-BiDS-2023/tree/main/tutorial/figures/chunked.png\" width=\"200\" height=\"100\">\n",
"\n",
"We split it into pieces so that we can process our data block by block or __chunk__ by __chunk__.\n",
"\n",
"In our case, for the moment, we used stackstac without specifying 'chunk' explicitly. The dataset is composed of 8MiB each, each contains, 1 time step, 1 band, 1024 x 1024 on x and y direction. \n",
"<img src=\"../figures/chunk_original.png\" width=\"200\" height=\"100\">\n",
"<img src=\"https://github.com/pangeo-data/pangeo-openeo-BiDS-2023/tree/main/tutorial/figures/chunk_original.png\" width=\"200\" height=\"100\">\n",
"\n",
"\n",
"If we have too small chunk size, we will divide our work flow in too small pieces, which can create too many communications, too many 'distirbution' overheads. \n",
Expand Down
8 changes: 4 additions & 4 deletions _sources/part3/scaling_dask.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -272,11 +272,11 @@
"you will just need too look at the html link you have for your jupyterlab, and Dask dashboard port number, as highlighted in the figure below.\n",
"\n",
"\n",
"<img src=\"../figures/dashboardlink.png\" width=\"50%\">\n",
"<img src=\"../figures/dasklab.png\" width=\"80%\">\n",
"<img src=\"https://github.com/pangeo-data/pangeo-openeo-BiDS-2023/tree/main/tutorial/figures/dashboardlink.png\" width=\"50%\">\n",
"<img src=\"https://github.com/pangeo-data/pangeo-openeo-BiDS-2023/tree/main/tutorial/figures/dasklab.png\" width=\"80%\">\n",
"\n",
"Then click the orange icon indicated in the above figure, and type 'your' dashboard link (normally, you just need to replace 'todaka' to 'your username'). \n",
"<img src=\"../figures/daskclick.png\" width=\"30%\">\n",
"<img src=\"https://github.com/pangeo-data/pangeo-openeo-BiDS-2023/tree/main/tutorial/figures/daskclick.png\" width=\"30%\">\n",
"\n",
"\n",
"\n",
Expand All @@ -285,7 +285,7 @@
"You can click several buttons indicated with blue arrows in above figures, then drag and drop to place them as your convenience. \n",
"\n",
"\n",
"<img src=\"../figures/exampledasklab.png\" width=\"50%\">\n",
"<img src=\"https://github.com/pangeo-data/pangeo-openeo-BiDS-2023/tree/main/tutorial/figuresexampledasklab.png\" width=\"50%\">\n",
"\n",
"\n",
"It's really helpfull to understand your computation and how it is distributed."
Expand Down
10 changes: 5 additions & 5 deletions part1/visualization.html

Large diffs are not rendered by default.

170 changes: 85 additions & 85 deletions part1/xarray_pitfalls.html

Large diffs are not rendered by default.

2,854 changes: 1,427 additions & 1,427 deletions part3/chunking_introduction.html

Large diffs are not rendered by default.

2,108 changes: 1,054 additions & 1,054 deletions part3/data_exploitability_pangeo.html

Large diffs are not rendered by default.

142 changes: 71 additions & 71 deletions part3/scaling_dask.html

Large diffs are not rendered by default.

Loading

0 comments on commit 4f0e137

Please sign in to comment.