Skip to content

Commit

Permalink
[PyTorch] Move block_table argument to FA varlen function (#1222)
Browse files Browse the repository at this point in the history
move block_table arg to varlen_func section

Signed-off-by: Charlene Yang <[email protected]>
  • Loading branch information
cyanguwa authored Oct 3, 2024
1 parent fb74961 commit 10cceae
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions transformer_engine/pytorch/attention.py
Original file line number Diff line number Diff line change
Expand Up @@ -5012,12 +5012,12 @@ def forward(
fa_optional_forward_kwargs["alibi_slopes"] = alibi_slopes
if _flash_attn_2_4_1_plus:
fa_optional_forward_kwargs["deterministic"] = self.deterministic
if _flash_attn_2_5_7_plus:
fa_optional_forward_kwargs["block_table"] = None
fa_optional_forward_args_thd = []
if qkv_format in ["bshd", "sbhd"] and "padding" not in attn_mask_type:
func = flash_attn_func if not _use_flash_attn_3 else flash_attn_func_v3
else:
if _flash_attn_2_5_7_plus:
fa_optional_forward_kwargs["block_table"] = None
func = (
flash_attn_varlen_func
if not _use_flash_attn_3
Expand Down

0 comments on commit 10cceae

Please sign in to comment.