Skip to content

Commit

Permalink
Update Node.js to v16 and update frontend dependencies (#519)
Browse files Browse the repository at this point in the history 
* Use npm audit to update npm packages

* Use Node.js version 16 in Docker development

also changed shell to bash
We use nvm to install Node.js as nodesource distributions somehow
always install Node.js 18 instead
(see nodesource/distributions#1583)

* Use node after installation

* Split installation of Node.js to multiple RUN commands

Also alias the default version
If you want to see the full output during Docker build,
you can use the "--progress=plain" option.

* Uniy sass packages and update yarn lock

* Reflect changes in production and test Dockerfiles

* Add comments to Dockerfile and tighten run commands

* Add sass-loader back as dependency

* Switch back to old version of sass-loader

This is because we still use webpacker v4 currently.
We will upgrade this depenedency when we switch from @rails/webpacker
to webpack in the future.

* Use "-no-install-recommends" for yarn install in prod

This is so that Node.js is not installed again, which would result in v18,
which we don't want at the moment.

* Use consistent ${} syntax in Dockerfile & fix Node path

* Copy Node.js over to /usr/local

This ensures that the app user can access node that was previously
installed with nvm in /root/.nvm

* Apply quote arguments suggestion (in code review)

Co-authored-by: Christian Heusel <[email protected]>

* Quote arguments in other Dockerfiles as well

* Replace deprecated apt-key and improve Dockerfiles

- Replaced Yarn installation with "corepack", a new binary shipped with
Node.js starting with v16. This also means that Yarn is upgraded from v1 to v2
with a different CLI, e.g. we now use "yarn workspaces focus"
instead of "yarn install" to install the dependencies for the workspace
in the current working directory.
- Update GPG key handling during PostgreSQL installation
- Added some more comments to the Dockerfile
- Moved copying of Node.js up in the Dockerfile

More yarn-related changed in subsequent commits.

* Upgrade to Yarn 2 and add packages for non-error webpack build

* Copy Yarn to usr folder alongside other Node.js tooling

* Activate Yarn for app user

* Only copy node over to usr folder

* Go back to Yarn1 and explicitly set Yarn version

- Added a fix so that corepack really installs the version we want
and not some other old version
- with Yarn2+, building curerntly fails, so we switch back to Yarn1 and
deal with an upgrade of Yarn in the future

* Remove ".yarn" from .gitignore

Yarn1 does not generate this folder

* Remove unnecessary dependencies from package.json

"unnecessary" means dependencies that were added when trying out Yarn2+

* Add back missing "--production=false" to "yarn install"

* Use "ruby:3.1.4-bullseye" to have PostgreSQL available

* Add missing `apt update` in dev and test Dockerfiles

Co-authored-by: Christian Heusel <[email protected]>

* Group `update` and `install` in one RUN statement

(in prod Dockerfile)

* Remove superfluous newline

* Make Node.js version logging one RUN statement

* Explicitly set `--production=true` during `yarn install`

---------

Co-authored-by: Christian Heusel <[email protected]>
  • Loading branch information
@Splines @christian-heusel
Splines and christian-heusel committed Jun 27, 2023
1 parent 3bccb52 commit de82635
Show file tree
Hide file tree
Showing 5 changed files with 1,983 additions and 2,699 deletions.
56 changes: 46 additions & 10 deletions docker/development/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,27 +7,63 @@ RUN GOOS=js GOARCH=wasm go build -o pdfcomprezzor.wasm
RUN cp "$(go env GOROOT)/misc/wasm/wasm_exec.js" .

# Now build the actual mampf application
FROM ruby:3.1.4
# https://hub.docker.com/_/ruby/
FROM ruby:3.1.4-bullseye
ENV RAILS_ENV=production

EXPOSE 3000
EXPOSE 9394

# https://github.com/nodesource/distributions#installation-instructions
RUN curl -sL https://deb.nodesource.com/setup_14.x | bash - && \
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
curl -sS https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - && echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" | tee /etc/apt/sources.list.d/pgdg.list && \
apt-get update && \
apt-get install --no-install-recommends -y ffmpeg ghostscript imagemagick \
libarchive-tools nodejs pdftk postgresql-client-13 sqlite3 wget \
wait-for-it yarn shared-mime-info && \
rm -rf /var/lib/apt/lists/* && apt-get clean
# use "--login" option, so that .bashrc is sourced in new shells
SHELL ["/bin/bash", "--login", "-c"]

# Install Node.js
# https://github.com/nodesource/distributions/issues/1583#issuecomment-1597489401
# https://stackoverflow.com/a/57546198/
# Unfortunately, we have to explicitly specify the node version here
# and cannot use 16.x as we need to put the node binary into the PATH
# and therefore require the exact version to find the folder
ENV NODE_VERSION=16.20.1
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
ENV NVM_DIR=/root/.nvm
RUN source "${NVM_DIR}/nvm.sh" && nvm install "${NODE_VERSION}" && \
nvm use "${NODE_VERSION}" && nvm alias default "${NODE_VERSION}"

ENV NODE_PATH="${NVM_DIR}/versions/node/v${NODE_VERSION}/bin/"
ENV PATH="${NODE_PATH}:${PATH}"

RUN nvm current
RUN node --version
RUN npm --version

# Install Yarn (see https://yarnpkg.com/getting-started/install)
# https://github.com/nodejs/corepack#corepack-prepare--nameversion
ENV YARN_VERSION=1.22.19
RUN corepack enable
RUN corepack prepare "yarn@${YARN_VERSION}" --activate
RUN which yarn; yarn --version
# even though this is not specified in the corepack documentation, we need to
# run "set version", otherwise the app user will have an old version of
# yarn avaialble instead of our specified version
RUN yarn set version "${YARN_VERSION}"

# Install other dependencies
# Note that postgresql-client-13 is available through debian bullseye
# that the ruby image is based on
RUN apt update && \
apt-get install -y --no-install-recommends \
ffmpeg imagemagick pdftk ghostscript shared-mime-info \
libarchive-tools postgresql-client-13 sqlite3 wget wait-for-it

# Setup ImageMagick
RUN sed -i '/disable ghostscript format types/,+6d' /etc/ImageMagick-6/policy.xml

WORKDIR /usr/src/app
ENTRYPOINT ["./entrypoint.sh"]

COPY ./Gemfile ./Gemfile.lock ./yarn.lock ./package.json /usr/src/app/
RUN bundle install
RUN yarn install --production=false

COPY --from=build-pdfcomprezzor /go/src/pdfcomprezzor.wasm /go/src/wasm_exec.js /usr/src/app/public/pdfcomprezzor/
COPY --from=build-pdfcomprezzor /go/src/pdfcomprezzor.wasm /go/src/wasm_exec.js /
61 changes: 51 additions & 10 deletions docker/production/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,29 +16,70 @@ EXPOSE 9394

ENTRYPOINT ["/usr/src/app/docker/entrypoint-worker.sh"]

RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN curl -sL https://deb.nodesource.com/setup_14.x | bash
# update the sources with the repos set up
RUN apt-get update
# install all the dependencies
RUN apt-get install -y --no-install-recommends \
# use "--login" option, so that .bashrc is sourced in new shells
SHELL ["/bin/bash", "--login", "-c"]

# Install Node.js
# https://github.com/nodesource/distributions/issues/1583#issuecomment-1597489401
# https://stackoverflow.com/a/57546198/
# Unfortunately, we have to explicitly specify the node version here
# and cannot use 16.x as we need to put the node binary into the PATH
# and therefore require the exact version to find the folder
ENV NODE_VERSION=16.20.1
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
ENV NVM_DIR=/root/.nvm
RUN source "${NVM_DIR}/nvm.sh" && nvm install "${NODE_VERSION}" && \
nvm use "${NODE_VERSION}" && nvm alias default "${NODE_VERSION}"

ENV NODE_PATH="${NVM_DIR}/versions/node/v${NODE_VERSION}/bin/"
ENV PATH="${NODE_PATH}:${PATH}"

RUN nvm current; node --version
RUN npm --version

# Install Yarn (see https://yarnpkg.com/getting-started/install)
# https://github.com/nodejs/corepack#corepack-prepare--nameversion
ENV YARN_VERSION=1.22.19
RUN corepack enable
RUN corepack prepare "yarn@${YARN_VERSION}" --activate
RUN which yarn; yarn --version
# even though this is not specified in the corepack documentation, we need to
# run "set version", otherwise the app user will have an old version of
# yarn avaialble instead of our specified version
RUN yarn set version "${YARN_VERSION}"

# Make Node.js tooling available for other users
# see https://www.digitalocean.com/community/tutorials/how-to-install-node-js-with-nvm-node-version-manager-on-a-vps
# this command copies the currently active nvm node tooling to /usr/local/
# so that it can be used by the app user later
RUN n=$(which node); n=${n%/bin/node}; chmod 755 $n/bin/node; chmod 755 $n/bin/yarn; cp -r $n/{bin,lib,share} /usr/local

# Install other dependencies
RUN apt-get update && \
apt-get install -y --no-install-recommends \
ffmpeg imagemagick pdftk ghostscript rsync shared-mime-info
RUN apt-get install -y nodejs yarn

# Setup ImageMagick
RUN sed -i '/disable ghostscript format types/,+6d' /etc/ImageMagick-6/policy.xml

# Manage users
RUN groupadd -g 501 app && useradd -g 501 -u 501 -m -d /usr/src/app app && \
mkdir /private /caches && chown app:app /private /caches


# ============= Now switch to the app user
WORKDIR /usr/src/app
USER app

# Log Node.js tooling versions that the `app` user uses
RUN which node; node --version; which yarn; yarn --version

COPY --from=build-pdfcomprezzor /go/src/pdfcomprezzor.wasm /go/src/wasm_exec.js /usr/src/app/public/pdfcomprezzor/

COPY --chown=app:app ./Gemfile ./Gemfile.lock ./yarn.lock ./package.json /usr/src/app/
RUN bundle install
RUN yarn install --production=true

RUN bundle install && \
yarn install --production=false
COPY --chown=app:app . /usr/src/app
RUN cp -r $(bundle info --path sidekiq)/web/assets /usr/src/app/public/sidekiq && \
SECRET_KEY_BASE="$(bundle exec rails secret)" DB_ADAPTER=nulldb bundle exec rails assets:precompile
59 changes: 47 additions & 12 deletions docker/run_cypress_tests/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,61 @@ RUN GOOS=js GOARCH=wasm go build -o pdfcomprezzor.wasm
RUN cp "$(go env GOROOT)/misc/wasm/wasm_exec.js" .

# Now build the actual mampf application
FROM ruby:3.1.4
# https://hub.docker.com/_/ruby/
FROM ruby:3.1.4-bullseye
ENV RAILS_ENV=production

EXPOSE 3000

# https://github.com/nodesource/distributions#installation-instructions
RUN curl -sL https://deb.nodesource.com/setup_14.x | bash - && \
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
curl -sS https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - && echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" | tee /etc/apt/sources.list.d/pgdg.list && \
apt-get update && \
apt-get install --no-install-recommends -y ffmpeg ghostscript imagemagick \
libarchive-tools nodejs pdftk postgresql-client-13 sqlite3 wget \
wait-for-it yarn shared-mime-info && \
rm -rf /var/lib/apt/lists/* && apt-get clean
# use "--login" option, so that .bashrc is sourced in new shells
SHELL ["/bin/bash", "--login", "-c"]

# Install Node.js
# https://github.com/nodesource/distributions/issues/1583#issuecomment-1597489401
# https://stackoverflow.com/a/57546198/
# Unfortunately, we have to explicitly specify the node version here
# and cannot use 16.x as we need to put the node binary into the PATH
# and therefore require the exact version to find the folder
ENV NODE_VERSION=16.20.1
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
ENV NVM_DIR=/root/.nvm
RUN source "${NVM_DIR}/nvm.sh" && nvm install "${NODE_VERSION}" && \
nvm use "${NODE_VERSION}" && nvm alias default "${NODE_VERSION}"

ENV NODE_PATH="${NVM_DIR}/versions/node/v${NODE_VERSION}/bin/"
ENV PATH="${NODE_PATH}:${PATH}"

RUN nvm current
RUN node --version
RUN npm --version

# Install Yarn (see https://yarnpkg.com/getting-started/install)
# https://github.com/nodejs/corepack#corepack-prepare--nameversion
ENV YARN_VERSION=1.22.19
RUN corepack enable
RUN corepack prepare "yarn@${YARN_VERSION}" --activate
RUN which yarn; yarn --version
# even though this is not specified in the corepack documentation, we need to
# run "set version", otherwise the app user will have an old version of
# yarn avaialble instead of our specified version
RUN yarn set version "${YARN_VERSION}"

# Install other dependencies
# Note that postgresql-client-13 is available through debian bullseye
# that the ruby image is based on
RUN apt update && \
apt-get install -y --no-install-recommends \
ffmpeg imagemagick pdftk ghostscript shared-mime-info \
libarchive-tools postgresql-client-13 sqlite3 wget wait-for-it

# Setup ImageMagick
RUN sed -i '/disable ghostscript format types/,+6d' /etc/ImageMagick-6/policy.xml

WORKDIR /usr/src/app

COPY ./Gemfile ./Gemfile.lock ./yarn.lock ./package.json /usr/src/app/
RUN bundle install && \
yarn install --production=false
RUN bundle install
RUN yarn install --production=false

COPY --from=build-pdfcomprezzor /go/src/pdfcomprezzor.wasm /go/src/wasm_exec.js /usr/src/app/public/pdfcomprezzor/
COPY --from=build-pdfcomprezzor /go/src/pdfcomprezzor.wasm /go/src/wasm_exec.js /
Expand Down
21 changes: 11 additions & 10 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,24 @@
"name": "mampf",
"private": true,
"dependencies": {
"@rails/webpacker": "5.4.3",
"@webpack-cli/serve": "^1.6.1",
"@rails/webpacker": "5.4.4",
"@webpack-cli/serve": "^1.7.0",
"coffee-loader": "^1.0.1",
"coffeescript": "2.6.1",
"core-js": "^3.21.1",
"core-js": "^3.31.0",
"css-loader": "^5.2.7",
"friendly-challenge": "^0.9.1",
"friendly-challenge": "^0.9.12",
"imports-loader": "^1.2.0",
"jquery-datetimepicker": "^2.5.21",
"moment": "^2.29.1",
"node-sass": "^7.0.1",
"regenerator-runtime": "^0.13.7",
"moment": "^2.29.4",
"regenerator-runtime": "^0.13.11",
"sass": "^1.63.4",
"sass-loader": "^10.1.1",
"webpack-cli": "^4.9.2",
"webpack-dev-server": "^4.7.4"
"webpack": "^4.46.0",
"webpack-cli": "^4.10.0",
"webpack-dev-server": "^4.15.1"
},
"scripts": {
"lint": "eslint ."
}
}
}
Loading

0 comments on commit de82635

Please sign in to comment.