Attention of instruction-tuned GPTs

[Bachstelze]

How does the attention look in instruction-tuned GPTs?
reference: jessevig/bertviz#128

[jwergieluk]

Good question: The library builds upon nanoGPT, so instruction-tuned GPT2 models shouldn't be a problem (probably a matter of updating the list of available models). Which model are interested in specifically?

[iBibek]

LLAMA-2 series models

[jwergieluk]

llama2 are not gpt2 models, AFAIK. So not supported at the moment.