Skip to content

Latest commit

 

History

History
39 lines (27 loc) · 1.94 KB

README.md

File metadata and controls

39 lines (27 loc) · 1.94 KB
Raw

QSync

Official resporitory for "IPDPS' 24 QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices".

Description

QSync aims to explore the potential of removing unnecessary quantized operations to improve training accuracy. It achieves this through the following components:

  • Quantization perturbation indicator/Replayer for analyzing the global data flow graph's memory and latency under mixed-precision (Predictor)
  • Allocator for selecting the optimal quantized operations for training (Allocator / Syncer)
  • Support for low-precision backends (CUTLASS, CUDNN) (LP-PyTorch)

In particular, QSync addresses a specific practical scenario: hybrid-cluster training, which involves inference GPUs with power capabilities (memory, compute) and training GPUs with higher capabilities.

The provided scripts support both convolution-based and transformer-based models.

NOTE: The project is a bit old. The performance of kernel implementation may not catch up with latest PyTorch.

Set Environment

Clone the repo git clone --recursive https://github.com/bytedance/QSync.git

Docker

  • run build.sh under dockerfile
  • run run.sh, specifiying the necessary path mounting inside.
  • run pip install -e . right in the root folder of QSync, compilation of kernels will start.

Manual Installation

  • Some libs may hard to install without proxy. Change <abspath_to_root> in m_install.sh to the absolute path to the root folder. Then
  1. bash m_install.sh
  2. make

Usage

QSync is implemented under the qsync folder, composed of syncer, predictor and LpTorch.

  • to use LpTorch and convert your model to mixed-biwdith model, use model = QModule(model)
  • See detail for usage of predictor and syncer in the corresponding page.
  • See sample under benchmark_convs / benchmark_transformers

notice the cross-node cost modeling is not as accurate as single-node is. Extra efforts required to align the communication start.