Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Containerization #38

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Containerization #38

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
194653b
versions in requirements
leonardomra Oct 26, 2020
08e2689
Dockerfile for services added
leonardomra Oct 26, 2020
334b5fa
Docker info in Readme
leonardomra Oct 26, 2020
92f709a
Update: from title to subtitle for docker section
leonardomra Oct 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
FROM python:3.7.3-stretch

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

COPY requirements.txt /usr/src/app/
RUN pip install --upgrade pip
RUN pip3 install --no-cache-dir -r requirements.txt
RUN apt-get update && apt-get install -y wget && apt-get install coreutils && apt-get install unzip && apt-get install -y default-jre

# Install GNormPlus
RUN wget https://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/tmTools/download/GNormPlus/GNormPlusJava.zip \
&& unzip GNormPlusJava.zip && rm GNormPlusJava.zip
RUN wget -O /usr/src/app/GNormPlusJava/crfpp-0.58.tar.gz https://drive.google.com/uc?id=0B4y35FiV1wh7QVR6VXJ5dWExSTQ
RUN tar xvfz /usr/src/app/GNormPlusJava/crfpp-0.58.tar.gz
RUN cp -rf /usr/src/app/CRF++-0.58/* /usr/src/app/GNormPlusJava/CRF
RUN cd /usr/src/app/GNormPlusJava/CRF/ && sh ./configure
RUN cd /usr/src/app/GNormPlusJava/CRF/ && make
RUN cd /usr/src/app/GNormPlusJava/CRF/ && make install
RUN chmod 764 /usr/src/app/GNormPlusJava/Ab3P
RUN cd /usr/src/app/GNormPlusJava/ && sed -i 's/= All/= 9606/g' setup.txt; echo "FocusSpecies: from All to 9606 (Human)"
RUN cd /usr/src/app/GNormPlusJava/ && sh Installation.sh
RUN cd /usr/src/app/GNormPlusJava/ && wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1g-JlhqeDIlZX5YFk8Y27_M8BXUXcQRSX' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1g-JlhqeDIlZX5YFk8Y27_M8BXUXcQRSX" -O GNormPlusServer.jar && rm -rf /tmp/cookies.txt

# Install tmVar2
RUN cd /usr/src/app/ && wget ftp://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/tmVar2/tmVarJava.zip \
&& unzip tmVarJava.zip && rm tmVarJava.zip
RUN cp -rf /usr/src/app/CRF++-0.58/* /usr/src/app/tmVarJava/CRF && rm -R /usr/src/app/CRF++-0.58 && rm /usr/src/app/GNormPlusJava/crfpp-0.58.tar.gz
RUN cd /usr/src/app/tmVarJava/CRF/ && sh ./configure
RUN cd /usr/src/app/tmVarJava/CRF/ && make
RUN cd /usr/src/app/tmVarJava/CRF/ && make install
RUN chmod 764 /usr/src/app/tmVarJava/CRF/crf_test
RUN cd /usr/src/app/tmVarJava/ && sh Installation.sh
RUN cd /usr/src/app/tmVarJava/ && wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1kQYzLHLFLsU9qKpRRGjXkIYmaYK6bPJm' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1kQYzLHLFLsU9qKpRRGjXkIYmaYK6bPJm" -O tmVar2Server.jar && rm -rf /tmp/cookies.txt
RUN cd /usr/src/app/tmVarJava/ && wget https://repo1.maven.org/maven2/org/xerial/sqlite-jdbc/3.20.0/sqlite-jdbc-3.20.0.jar
RUN cd /usr/src/app/tmVarJava/ && wget https://repo1.maven.org/maven2/edu/stanford/nlp/stanford-corenlp/3.5.2/stanford-corenlp-3.5.2.jar

# Copy Project into App
COPY . /usr/src/app
RUN mkdir /usr/src/app/logs
RUN touch /usr/src/app/logs/nohup_BERN.out \
&& touch /usr/src/app/logs/nohup_disease_normalize.out \
&& touch /usr/src/app/logs/nohup_drug_normalize.out \
&& touch /usr/src/app/logs/nohup_gene_normalize.out \
&& touch /usr/src/app/logs/nohup_gnormplus.out \
&& touch /usr/src/app/logs/nohup_mutation_normalize.out \
&& touch /usr/src/app/logs/nohup_species_normalize.out \
&& touch /usr/src/app/logs/nohup_tmvar.out

# Download normalization resources and pre-trained BioBERT NER models
RUN cd /usr/src/app/scripts && sh download_norm.sh \
&& cd /usr/src/app/scripts && sh download_biobert_ner_models.sh

EXPOSE 8888

# Execute
CMD (cd /usr/src/app/GNormPlusJava/ && nohup java -Xmx16G -Xms16G -jar /usr/src/app/GNormPlusJava/GNormPlusServer.jar 18895 >> /usr/src/app/logs/nohup_gnormplus.out 2>&1 &) \
&& (cd /usr/src/app/tmVarJava/ && nohup java -Xmx8G -Xms8G -jar /usr/src/app/tmVarJava/tmVar2Server.jar 18896 >> /usr/src/app/logs/nohup_tmvar.out 2>&1 &) \
&& (cd /usr/src/app/ && sh load_dicts.sh) \
&& echo $CUDA_VISIBLE_DEVICES && export CUDA_VISIBLE_DEVICES=0 \
&& (nohup python3 -u /usr/src/app/server.py --port 8888 --gnormplus_home /usr/src/app/GNormPlusJava --gnormplus_port 18895 --tmvar2_home /usr/src/app/tmVarJava --tmvar2_port 18896 >> /usr/src/app/logs/nohup_BERN.out 2>&1 &) \
&& tail -f logs/nohup_BERN.out
7 changes: 6 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,12 @@ tail -F logs/nohup_BERN.out
print(result_dict)
```

## BERN in Docker

Build and run BERN in a single container::

docker build --tag bern_service_component:1.0 . && docker run --publish 8888:8888 --detach --name bern bern_service_component:1.0

## Result
<details>
<summary>See a result example in JSON (PMID:29446767) </summary>
Expand Down Expand Up @@ -347,7 +353,6 @@ nohup python3 -u server.py --port 8888 --gnormplus_home ~/bern/GNormPlusJava --g
tail -F logs/nohup_BERN.out
```


## Troubleshooting
* Trouble: It takes a long time to get results.
* Solution: Make sure TensorFlow is using a GPU.
Expand Down
10 changes: 5 additions & 5 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
numpy
pandas
requests
tensorflow-gpu
xmltodict
numpy==1.19.2
pandas==1.1.3
requests==2.24.0
tensorflow-gpu==1.15
xmltodict==0.12.0