Code

Fix mqa parallelization#51

Code

Open

thomasw21 wants to merge 3 commits intomulti-query-attentionbigcode-project/Megatron-LM:multi-query-attentionfrom thomas/fix_mqa_parallelizationbigcode-project/Megatron-LM:thomas/fix_mqa_parallelizationCopy head branch name to clipboard

+27-24

Commits

Commits on May 11, 2023

• Don't need to run communication for kv

  thomasw21

  committed
  33d14e6

• Woops

  thomasw21

  committed
  67f733c

Commits on May 12, 2023

• I think this fixes sequence parallel

  thomasw21

  committed
  080d1c0