Skip to content

Latest commit

 

History

History
125 lines (94 loc) · 8.72 KB

README.md

File metadata and controls

125 lines (94 loc) · 8.72 KB
Raw

Reading group for Fall 23

This reading group will go on from mid August till first week of december. We have approximately 14 weeks which means we can hold 28 presentations in total.

Relevant Links

  1. (webpage link)
  2. Sign up sheet (spreadsheet)
  3. Unanswered questions doc (doc)

Contributors

Name Role
Aditya Desai TBD
Zhenghui Guo R2
Gaurav Gupta R3
Masa Maksimovic R2
Atishay Jain R1
Sanya Garg R2
Benjamin Meisburger R2
Apoorv Walia R1
Jonah Yi R1
Name Title Suggested to be read by date Unanswered questions updated? slides
Aditya Desai Moments, deviations, chernoff/Hoeffding All 08/28/2023 Yes N.A
Tony Zhang Near neighbor graph, inner product transformation, inner product search All 09/06/2023 Yes Slides
Jonah Yi Learning to Route All 09/06/2023 Yes Slides
Ben Meisburger Deep Gradient Compression All 09/11/2023 Yes Slides
Masa Maksimovic Convergence in ML All 09/11/2023 Yes ❗
Zhenghui Guo(Kevin) KV cache inference All 09/18/2023 Yes Slides
Atishay Jain Knowledge neurons All 09/18/2023 ❗ ❗
Gaurav Vector Databases All 09/25/2023 ❗ ❗
Apoorv Randomized projection based compression All 10/16/2023 ❗ ❗
Aditya Desai Recent results on projection based compression All 10/16/2023 ❗ ❗
Zhenghui Guo(Kevin) You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling All 10/23/23 ❗ [Slides]
Jonah Yi Accelerating Large-Scale Inference with Anisotropic Vector Quantization All 10/30/23 ❗ Slides
Benjamin Meisburger ROSE: Robust Caching for Amazon Search 11/13/23 Slides
Jonah Yi ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms All 11/20/23 ❗ Slides

Announcements (Format - Title : Date of announcement) . Please put latest first.

Presentation by Jonah Yi 20th Nov.

  1. ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms: https://arxiv.org/pdf/1807.05614.pdf
  2. A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search: https://arxiv.org/pdf/2101.12631.pdf

Presentation by Benjamin Meisburger 13th Nov.

  1. Robust Caching for Amazon Search: https://assets.amazon.science/dc/5e/b919974a4abdba7a3cd82c1bc86f/rose-robust-caches-for-amazon-product-search.pdf

Presentation by Jonah Yi 30th Oct.

  1. Accelerating Large-Scale Inference with Anisotropic Vector Quantization: https://arxiv.org/pdf/1908.10396.pdf

Presentation by Zhenghui Guo(Kevin) 23rd Oct.

  1. You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling: https://proceedings.mlr.press/v139/zeng21a/zeng21a.pdf

Presentation by Sanya Garg: 235d Oct.

  1. Learning without Forgetting for Vision-Language Models: https://arxiv.org/pdf/2305.19270.pdf

Presentation by Apoorv Walia: 16th Oct.

  1. Introduction to Parameter Sharing Methods (Papers - Roast: Efficient Model Compression with Random Operation Access Specific Tile Hashing - https://arxiv.org/pdf/2207.10702.pdf)
  2. Robe: Random Offset Block Embedding Array - https://arxiv.org/pdf/2108.02191.pdf
  3. Compressing Neural Networks with the Hashing Trick - https://arxiv.org/pdf/1504.04788.pdf

Presentation by Aditya Desai: 16th Oct.

  1. Results from "In defense of parameter sharing for model compression"

Presentation by Gaurav Gupta: 25th Sept.

  1. Introduction to Vector databases (paper- Milvus: A purpose-built vector data management system, https://dl.acm.org/doi/abs/10.1145/3448016.3457550)

Presentation by Atishay Jain: 18th Sept.

  1. Knowledge Neurons in Pretrained Transformers: https://arxiv.org/abs/2104.08696
  2. Locating and Editing Factual Associations in GPT: https://arxiv.org/abs/2202.05262

Presentation by Zhenghui Guo(kevin): 18th Sept.

  1. Efficient Memory Management for Large Language Model Serving with PagedAttention: https://arxiv.org/pdf/2309.06180.pdf

Presentation by Ben Meisburger: 11th Sept.

  1. Deep Gradient Compression: https://openreview.net/pdf?id=SkhQHMW0W

Presentation by Masa Maksimovic: 11th Sept.

  1. Convergence proofs for some simple settings: https://akyrillidis.github.io/comp414-514/schedule/?fbclid=IwAR1_ImKSLRQFgEGENLVBJD5stwFKf7fogHwse-w-NBrHxFVdMHl7iLRMUVA
  2. A convergence theory for deep learning: https://arxiv.org/abs/1811.03962v1

Presentation by Tony Zhang : 1st Sept.

  1. Hierarchical Navigable Small World (HNSW) data structure for approximate near neighbor search: https://arxiv.org/abs/1603.09320
  2. Vector transformation for making them more amenable to inner product search: https://arxiv.org/abs/1405.5869
  3. IP-NSW for maximum inner product search: https://proceedings.neurips.cc/paper/2018/hash/229754d7799160502a143a72f6789927-Abstract.html

Presentation by Jonah Yi : 1st Sept.

Learning to Route (LTR) in Similarity Graphs: https://arxiv.org/abs/1905.10987

Location Confirmed! DH1049 : 30th August

We have confirmed booking for DH1049 every monday (except Oct 16 where we will book a library room)

Presentation by Aditya Desai : 26th August

I will aim to cover parts of chapter 3 and chapter 4 from http://lib.ysu.am/open_books/413311.pdf

Roles in Fall 23 : 9th August 2023

We have the following roles in this edition.

R1:

This is focused research presentations. You will have to declare a particular problem that you are looking at and make three presentations:

  1. The first presentation will be motivating the problem (signifiance, impact, urgency, etc) and some related work that people have done. (Example : Embedding table compression is important, why, how have people solved this in the past, what problems remain, etc)
  2. Fundamentals of the approach. This is a "teaching presentation" where you explain the fundamental toolkit that you are using. (Example: I am going to use random projection based approach. what are random projections, how to analyse their usefulness, what are hash functions, why they are important, analysing hash functions)
  3. Your proposed method and results . (Example, the recipe to apply random projection based compression in embedding tables, results - quality, efficiency, etc)

R2:

This is more general research presentations. You will pick an area of research (for example, training efficiency, model compression, ml on edge, etc). You ll be making 2 presentations. This is similar to how things ran in summer edition of reading group. You ll keep reading papers on a single topic and summarize your learnings in two presentations.

R3:

This is a role of "moderating presentations". You ll not be presenting yourself. But you ll shadow readings of others if possible and actively discuss while they are presenting. The goal of this role is to keep the presentations engaging and to get the best out of it.

R4:

This is a role of "general audience and shadow". You ll read papers from one of the broad topics shadowing other presenters, listen in to presentations and participate in discussions.

❗ Add your name to the table above about roles.

How to participate : 9th August 2023 ❗❗

  1. If you are participating in the reading group, send me your github id at [email protected] . I will add you to the collaborators so that you can change the readme and thus update the webpage.
  2. It will a collaborative effort to keep this webpage updated. So make sure to get yourself added to collaborators and update portions that are relevant to you over the period of entire edition.

Previous Editions

Summer 2023