Pinned Loading

1. fft-conv-pytorch fft-conv-pytorch Public

   Implementation of 1D, 2D, and 3D FFT convolutions in PyTorch. Much faster than direct convolutions for large kernel sizes.

   Python 477 58

2. yet-another-retnet yet-another-retnet Public

   A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (https://arxiv.org/pdf/2307.08621.pdf)

   Python 100 15

3. grouped-query-attention-pytorch grouped-query-attention-pytorch Public

   (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints" (https://arxiv.org/pdf/2305.13245.pdf)

   Python 135 8

4. transformer-from-scratch transformer-from-scratch Public

   Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch

   Python 90 19

5. clip-text-decoder clip-text-decoder Public

   Generate text captions for images from their embeddings.

   Python 104 7

6. soft-mixture-of-experts soft-mixture-of-experts Public

   PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)

   Python 65 5