Code

Mtf#47

Code

Open

Muennighoff wants to merge 24 commits intomulti-query-attentionbigcode-project/Megatron-LM:multi-query-attentionfrom mtfbigcode-project/Megatron-LM:mtfCopy head branch name to clipboard

+1,031-35

Commits

Commits on Mar 6, 2023

• add santacoder example script

  RaymondLi0

  committed
  ccf78e0

Commits on Apr 14, 2023

• Add MTF (alike bloomz/mt0)

  Muennighoff

  committed
  d661d04

• Merge

  Muennighoff

  committed
  2aa3e5a

Commits on Apr 15, 2023

• Fix logging

  Muennighoff

  committed
  181593c

• Add

  Muennighoff

  committed
  005dbb0

• Fix warn

  Muennighoff

  committed
  b64ade1

• Add PAD

  Muennighoff

  committed
  9c66466

• eos > eod

  Muennighoff

  committed
  65c81ae

• Add size

  Muennighoff

  committed
  fb234a3

• Fix args

  Muennighoff

  committed
  9dde9db

• Make causal

  Muennighoff

  committed
  cbd0beb

• Mov pos to dev

  Muennighoff

  committed
  2d8e414

• Packed MTF mask

  Muennighoff

  committed
  bd22057

• Merge pull request #48 from bigcode-project/packedmtfmask

  Show description for c790969
  Muennighoff

  authored
  c790969

• Add reset progress

  Muennighoff

  committed
  168e314

Commits on Apr 16, 2023

• Copy tensors after loading ckpt

  Muennighoff

  committed
  3991520

Commits on Apr 17, 2023

• Fix spec toks; Opt ckpt loading; newline

  Muennighoff

  committed
  ad603d6

Commits on Apr 19, 2023

• Fix tok

  Muennighoff

  committed
  14bc3e5

Commits on Apr 20, 2023

• Fix var

  Muennighoff

  committed
  5dc080f

• Clean init of optimizer when not laoaded

  Muennighoff

  committed
  b4609a7

• Rmv debug

  Muennighoff

  committed
  a828682

• Debug

  Muennighoff

  committed
  57184fc

• Clean comments

  Muennighoff

  committed
  6c80f79

• Attn

  Muennighoff

  committed
  9192641