Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

必要なライブラリを含めたリリースをできるようにした #246

Closed
104 changes: 104 additions & 0 deletions .github/actions/download-cuda-libraries/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
name: download cuda libraries
description: cuda libraryをダウンロードする
inputs:
cudnn_download_url:
description: cudaのダウンロードURL
required: true
artifact_name:
description: artifactの名前
required: true
cuda_version:
description: cudaのバージョン
required: true
cuda_lib_dir:
description: libraryは配置されているディレクトリ
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: libraryは配置されているディレクトリ
description: libraryが配置されているディレクトリ

required: true
extract_dir:
description: 出力先のディレクトリ
required: true
runs:
using: "composite"
steps:
- name: Prepare CUDA cache
uses: actions/cache@v2
id: cuda-library-cache
with:
key: v1-cuda-library-cache-${{ inputs.artifact_name }}-${{ inputs.cuda_version }}
path: download/cuda
- name: Prepare cuDNN cache
uses: actions/cache@v2
id: cudnn-library-cache
with:
key: v1-cudnn-library-cache-${{ inputs.artifact_name }}-${{ inputs.cudnn_download_url }}
path: download/cudnn
- uses: Jimver/[email protected]
id: cuda-toolkit
with:
method: network
cuda: ${{ inputs.cuda_version }}
- name: Extract CUDA Library
if: steps.cuda-library-cache.outputs.cache-hit != 'true'
shell: bash
run: |
set -eux
CUDA_ROOT=$( echo "${{ steps.cuda-toolkit.outputs.CUDA_PATH }}" | tr '\\' '/' )
mkdir -p download/cuda/bin
cp -v "$CUDA_ROOT/${{ inputs.cuda_lib_dir }}/"*.{so*,dll} download/cuda/bin/ || true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CUDNNはOSごとに処理が別れているので、こちらもそうして引数を少なくすると見通しが良くなりそう?

- name: Extract CUDNN Library
if: steps.cudnn-library-cache.outputs.cache-hit != 'true'
shell: bash
run: |
set -eux
mkdir -p download/cudnn_tmp
if [[ "${{ inputs.artifact_name }}" =~ "linux" ]]; then
curl -L "${{ matrix.cudnn_download_url }}" > download/cudnn.tar.xz
tar -Jxvf download/cudnn.tar.xz -C download/cudnn_tmp
mkdir -p download/cudnn_tmp/bin
mv -v download/cudnn_tmp/**/lib/* download/cudnn_tmp/bin
elif [[ "${{ inputs.artifact_name }}" =~ "windows" ]];then
curl -L "${{ matrix.cudnn_download_url }}" > download/cudnn.zip
unzip download/cudnn.zip cudnn*/bin/*.dll -d download/cudnn_tmp
mkdir -p ./download/cudnn_tmp/bin
mv -v ./download/cudnn_tmp/cudnn*/bin/* ./download/cudnn_tmp/bin
fi
mkdir -p download/cudnn/bin
mv download/cudnn_tmp/bin/*.{so*,dll} download/cudnn/bin/ || true
rm -rf download/cudnn_tmp
rm -f download/cudnn.*
mkdir -p ${{ inputs.extract_dir }}
cp -v docs/licenses/cuda/* "${{ inputs.extract_dir }}"
- name: Extract Libraries for Windows
shell: bash
if: contains(inputs.artifact_name,'windows')
run: |
# CUDA
ln -sf "$(pwd)/download/cuda/bin"/cublas64_*.dll "${{ inputs.extract_dir }}"
ln -sf "$(pwd)/download/cuda/bin"/cublasLt64_*.dll "${{ inputs.extract_dir }}"
ln -sf "$(pwd)/download/cuda/bin"/cudart64_*.dll "${{ inputs.extract_dir }}"
ln -sf "$(pwd)/download/cuda/bin"/cufft64_*.dll "${{ inputs.extract_dir }}"
ln -sf "$(pwd)/download/cuda/bin"/curand64_*.dll "${{ inputs.extract_dir }}"

# cuDNN
ln -sf "$(pwd)/download/cudnn/bin"/cudnn64_*.dll "${{ inputs.extract_dir }}"
ln -sf "$(pwd)/download/cudnn/bin"/cudnn_*_infer64*.dll "${{ inputs.extract_dir }}"
- name: Extract Libraries for Linux
shell: bash
if: contains(inputs.artifact_name,'linux')
run: |
# CUDA
ln -sf "$(pwd)/download/cuda/bin"/libcublas.so.* "${{ inputs.extract_dir }}"
rm -f "${{ inputs.extract_dir }}"/libcublas.so.*.*
ln -sf "$(pwd)/download/cuda/bin"/libcublasLt.so.* "${{ inputs.extract_dir }}"
rm -f "${{ inputs.extract_dir }}"/libcublasLt.so.*.*
ln -sf "$(pwd)/download/cuda/bin"/libcudart.so.* "${{ inputs.extract_dir }}"
rm -f "${{ inputs.extract_dir }}"/libcudart.so.*.*
ln -sf "$(pwd)/download/cuda/bin"/libcufft.so.* "${{ inputs.extract_dir }}"
rm -f "${{ inputs.extract_dir }}"/libcufft.so.*.*
ln -sf "$(pwd)/download/cuda/bin"/libcurand.so.* "${{ inputs.extract_dir }}"
rm -f "${{ inputs.extract_dir }}"/libcurand.so.*.*

# cuDNN
ln -sf "$(pwd)/download/cudnn/bin"/libcudnn.so.* "${{ inputs.extract_dir }}"
rm -f "${{ inputs.extract_dir }}"/libcudnn.so.*.*
ln -sf "$(pwd)/download/cudnn/bin"/libcudnn_*_infer.so.* "${{ inputs.extract_dir }}"
rm -f "${{ inputs.extract_dir }}"/libcudnn_*_infer.so.*.*
36 changes: 36 additions & 0 deletions .github/actions/upload-artifact/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: upload artifact
description: artifactをzipでアップロードする
inputs:
artifact_dir:
description: 作業ディレクトリ
required: true
assert_name:
description: artifactの名前
required: true
repo_token:
description: github token
required: true
version:
description: version
required: true
runs:
using: "composite"
steps:
- name: Archive artifact
if: (!contains(inputs.assert_name,'windows'))
shell: bash
run: |
cd "${{ inputs.artifact_dir }}"
zip -r "../${{ inputs.assert_name }}.zip" "${{ inputs.assert_name }}"
- name: Archive artifact (Windows)
if: contains(inputs.assert_name, 'windows')
shell: bash
run: |
powershell Compress-Archive -Path "${{ inputs.artifact_dir }}/${{ inputs.assert_name }}" -DestinationPath "${{ inputs.assert_name }}.zip"
- name: Upload to Release
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ inputs.repo_token }}
prerelease: true
tag: ${{ inputs.version }}
file: ${{ inputs.assert_name }}.zip
65 changes: 49 additions & 16 deletions .github/workflows/build_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,21 @@ jobs:
target: x86_64-pc-windows-msvc
artifact_name: windows-x64-directml
use_cuda: false
directml_full: true
directml_platform: x64-win
- os: windows-latest
additional-features: directml
target: x86_64-pc-windows-msvc
artifact_name: windows-x64-directml-min
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

他と命名規則合わせるために、こっちを無印にして、同梱版を-fullにするのはどうでしょう

use_cuda: false
- os: windows-latest
additional-features: ""
target: x86_64-pc-windows-msvc
artifact_name: windows-x64-cuda
use_cuda: true
cudnn_download_url: https://developer.download.nvidia.com/compute/redist/cudnn/v8.4.1/local_installers/11.6/cudnn-windows-x86_64-8.4.1.50_cuda11.6-archive.zip
cuda_version: 11.7.0
Comment on lines +54 to +55
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CUDNNが想定するCUDAのバージョン(11.6?)とCUDAのバージョン(11.7.0)が合ってないかも

cuda_lib_dir: bin
- os: windows-latest
additional-features: ""
target: i686-pc-windows-msvc
Expand All @@ -59,6 +69,9 @@ jobs:
target: x86_64-unknown-linux-gnu
artifact_name: linux-x64-gpu
use_cuda: true
cudnn_download_url: https://developer.download.nvidia.com/compute/redist/cudnn/v8.4.1/local_installers/11.6/cudnn-linux-x86_64-8.4.1.50_cuda11.6-archive.tar.xz
cuda_version: 11.7.0
cuda_lib_dir: lib64
- os: macos-latest
additional-features: ""
target: aarch64-apple-darwin
Expand All @@ -83,6 +96,10 @@ jobs:
- name: Set ASSET_NAME env var
shell: bash
run: echo "ASSET_NAME=voicevox_core-${{ matrix.artifact_name }}-${{ env.VERSION }}" >> $GITHUB_ENV
- name: Set CUDA_LIB_ASSET_NAME env var
shell: bash
if: ${{ matrix.use_cuda }}
run: echo "CUDA_LIB_ASSET_NAME=voicevox_core-${{ matrix.artifact_name }}-cuda-lib-${{ env.VERSION }}" >> $GITHUB_ENV
- name: Organize artifact
shell: bash
run: |
Expand All @@ -95,6 +112,24 @@ jobs:
rm -f artifact/${{ env.ASSET_NAME }}/libonnxruntime.{so,dylib}
cp -v README.md "artifact/${{ env.ASSET_NAME }}/README.txt"
echo "${{ env.VERSION }}" > "artifact/${{ env.ASSET_NAME }}/VERSION"
- name: Organize directml artifact
shell: bash
if: ${{ matrix.directml_full }}
run: |
directml_link=$(curl -s https://api.nuget.org/v3/registration5-semver1/microsoft.ai.directml/index.json | jq ".items[0].items | max_by(.catalogEntry.version) | .catalogEntry.packageContent" | sed "s/\"//g")
curl -sO $directml_link
unzip -d ./ext_directml ./microsoft.ai.directml.*.nupkg
cp -v "ext_directml/bin/${{ matrix.directml_platform }}/DirectML.dll" "artifact/${{ env.ASSET_NAME }}"
cp -v "ext_directml/bin/${{ matrix.directml_platform }}/LICENSE.txt" "artifact/${{ env.ASSET_NAME }}/DirectML_LICENSE.txt"
- name: Organize cuda lib artifact
if: ${{ matrix.use_cuda }}
uses: ./.github/actions/download-cuda-libraries
with:
cudnn_download_url: ${{ matrix.cudnn_download_url }}
artifact_name: ${{ matrix.artifact_name }}
cuda_version: ${{ matrix.cuda_version }}
cuda_lib_dir: ${{ matrix.cuda_lib_dir }}
extract_dir: artifact/${{ env.CUDA_LIB_ASSET_NAME }}
- name: Code signing (Windows)
if: startsWith(matrix.os, 'windows') && github.event.inputs.code_signing == 'true'
shell: bash
Expand All @@ -103,21 +138,19 @@ jobs:
env:
CERT_BASE64: ${{ secrets.CERT_BASE64 }}
CERT_PASSWORD: ${{ secrets.CERT_PASSWORD }}
- name: Archive artifact
shell: bash
if: (!contains(matrix.os , 'windows'))
run: |
cd artifact
zip -r "../${{ env.ASSET_NAME }}.zip" "${{ env.ASSET_NAME }}"
- name: Archive artifact (Windows)
if: contains(matrix.os, 'windows')
run: |
powershell Compress-Archive -Path "artifact/${{ env.ASSET_NAME }}" -DestinationPath "${{ env.ASSET_NAME }}.zip"
- name: Upload to Release
if: env.VERSION != 'DEBUG' && env.SKIP_UPLOADING_RELEASE_ASSET == '0'
uses: svenstaro/upload-release-action@v2
- name: Upload artifact
uses: ./.github/actions/upload-artifact
if: ${{ env.VERSION != 'DEBUG' && env.SKIP_UPLOADING_RELEASE_ASSET == '0' }}
with:
artifact_dir: artifact
assert_name: ${{ env.ASSET_NAME }}
repo_token: ${{ secrets.GITHUB_TOKEN }}
version: ${{ env.VERSION }}
- name: Upload cuda artifact
if: ${{ env.VERSION != 'DEBUG' && env.SKIP_UPLOADING_RELEASE_ASSET == '0' && (matrix.use_cuda) }}
uses: ./.github/actions/upload-artifact
with:
artifact_dir: artifact
assert_name: ${{ env.CUDA_LIB_ASSET_NAME }}
repo_token: ${{ secrets.GITHUB_TOKEN }}
prerelease: true
tag: ${{ env.VERSION }}
file: ${{ env.ASSET_NAME }}.zip
version: ${{ env.VERSION }}
Loading