Skip to content

Commit

Permalink
Add model from huggingface in container
Browse files Browse the repository at this point in the history
Updates Dockerfile to download the model file from huggingface
instead of datacommons.

Some fixes were needed to install requirements due to changes
to the upstream container and various packages:
- setuptools is manually installed with support for use_2to3
  - pipenv needs use_2to3 to install pycallgraph
- pillow is now pinned
   - PIL.Image.LINEAR is missing in later versions
- scikit-image is now pinned
  - measure.perimeter() neighbourhood param has been renamed
  • Loading branch information
johnbradley committed Jul 24, 2023
1 parent 8abd71d commit e7656d5
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 8 deletions.
15 changes: 9 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
FROM ghcr.io/imageomics/dataverse-access:1 as model_fetcher
ARG DATAVERSE_API_TOKEN
ENV DATAVERSE_URL=https://datacommons.tdai.osu.edu/
ENV MODEL_DV_DOI=doi:10.5072/FK2/MMX6FY
FROM alpine/git:2.40.1 as model_fetcher
ENV MODEL_REPO_URL=https://huggingface.co/imageomics/Drexel-metadata-generator

# Download model_final.pth
RUN mkdir -p /model \
&& dva download $MODEL_DV_DOI /model
&& cd /model \
&& git clone --depth=1 ${MODEL_REPO_URL}
RUN ls /model

FROM python:3.8.10-slim-buster
LABEL "org.opencontainers.image.authors"="John Bradley <[email protected]>"
Expand All @@ -27,11 +27,14 @@ ENV PATH="/pipeline:${PATH}"

COPY Pipfile /pipeline/.

# Fix issue installing pycallgraph related to use_2to3 being removed from setuptools
RUN pip install setuptools==57.0.0

# Install requirements
RUN pipenv install --skip-lock --system && pipenv --clear

COPY config /pipeline/config
COPY --from=model_fetcher /model/model_final.pth \
COPY --from=model_fetcher /model/Drexel-metadata-generator/model_final.pth \
/pipeline/output/enhanced/model_final.pth

COPY gen_metadata.py /pipeline
Expand Down
4 changes: 2 additions & 2 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ verify_ssl = false
[packages]
numpy = "*"
pynrrd = "*"
pillow = "*"
pillow = "==9.4.0"
opencv-python = "*"
ipython = "*"
scikit-image = "*"
scikit-image = "==0.19.3"
pandas = "*"
iopath = "*"
imutils = "*"
Expand Down

0 comments on commit e7656d5

Please sign in to comment.