From 7e6f71e8534ded3460d930de0e6c99310cbb61a4 Mon Sep 17 00:00:00 2001 From: Jennifer Gerhold Date: Thu, 25 Jul 2024 11:32:22 -0700 Subject: [PATCH] Updated Latest News and added Blog section Signed-off-by: Jennifer Gerhold --- README.md | 116 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 76 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index cb2a357fd7ed..437e9c5337bc 100644 --- a/README.md +++ b/README.md @@ -10,10 +10,38 @@ # **NVIDIA NeMo Framework** ## Latest News +
- Large Language Models and Multimodal + Large Language Models and Multimodal Models +
+ + + New Llama 3.1 Support + (2024-07-23) + + The NeMo Framework now supports training and customizing the Llama 3.1 collection of LLMs from Meta. +

+
+ + + Accelerate your Generative AI Distributed Training Workloads with the NVIDIA NeMo Framework on Amazon EKS + (2024-07-16) + + NVIDIA NeMo Framework now runs distributed training workloads on an Amazon Elastic Kubernetes Service (Amazon EKS) cluster. For step-by-step instructions on creating an EKS cluster and running distributed training workloads with NeMo, see our [GitHub repository](https://github.com/aws-samples/awsome-distributed-training/tree/main/3.test_cases/2.nemo-launcher/EKS). +

+
+
+ + + NVIDIA NeMo Accelerates LLM Innovation with Hybrid State Space Model Support + (2024/06/17) + + NVIDIA NeMo and Megatron Core now support pre-training and fine-tuning of state space models (SSMs). NeMo also supports training models based on the Griffin architecture as described by Google DeepMind. +

+
+
NVIDIA releases 340B base, instruct, and reward models pretrained on a total of 9T tokens. @@ -46,45 +74,6 @@ The walkthrough includes detailed instructions on how to set up a Google Cloud Project and pre-train a GPT model using the NeMo Framework.

-
- - - Bria Builds Responsible Generative AI for Enterprises Using NVIDIA NeMo, Picasso - (2024/03/06) - - Bria, a Tel Aviv startup at the forefront of visual generative AI for enterprises now leverages the NVIDIA NeMo Framework. - The Bria.ai platform uses reference implementations from the NeMo Multimodal collection, trained on NVIDIA Tensor Core GPUs, to enable high-throughput and low-latency image generation. - Bria has also adopted NVIDIA Picasso, a foundry for visual generative AI models, to run inference. -

-
-
- - - New NVIDIA NeMo Framework Features and NVIDIA H200 - (2023/12/06) - - NVIDIA NeMo Framework now includes several optimizations and enhancements, - including: - 1) Fully Sharded Data Parallelism (FSDP) to improve the efficiency of training large-scale AI models, - 2) Mix of Experts (MoE)-based LLM architectures with expert parallelism for efficient LLM training at scale, - 3) Reinforcement Learning from Human Feedback (RLHF) with TensorRT-LLM for inference stage acceleration, and - 4) up to 4.2x speedups for Llama 2 pre-training on NVIDIA H200 Tensor Core GPUs. -

- - H200-NeMo-performance -

-
-
- - - NVIDIA now powers training for Amazon Titan Foundation models - (2023/11/28) - - NVIDIA NeMo Framework now empowers the Amazon Titan foundation models (FM) with efficient training of large language models (LLMs). - The Titan FMs form the basis of Amazon’s generative AI service, Amazon Bedrock. - The NeMo Framework provides a versatile framework for building, customizing, and running LLMs. -

-
@@ -604,6 +593,53 @@ to the `gh-pages-src` branch of this repository. For detailed information, please consult the README located at the [gh-pages-src branch](https://github.com/NVIDIA/NeMo/tree/gh-pages-src#readme). +## Blogs + + +
+ Large Language Models and Multimodal Models +
+ + + Bria Builds Responsible Generative AI for Enterprises Using NVIDIA NeMo, Picasso + (2024/03/06) + + Bria, a Tel Aviv startup at the forefront of visual generative AI for enterprises now leverages the NVIDIA NeMo Framework. + The Bria.ai platform uses reference implementations from the NeMo Multimodal collection, trained on NVIDIA Tensor Core GPUs, to enable high-throughput and low-latency image generation. + Bria has also adopted NVIDIA Picasso, a foundry for visual generative AI models, to run inference. +

+
+
+ + + New NVIDIA NeMo Framework Features and NVIDIA H200 + (2023/12/06) + + NVIDIA NeMo Framework now includes several optimizations and enhancements, + including: + 1) Fully Sharded Data Parallelism (FSDP) to improve the efficiency of training large-scale AI models, + 2) Mix of Experts (MoE)-based LLM architectures with expert parallelism for efficient LLM training at scale, + 3) Reinforcement Learning from Human Feedback (RLHF) with TensorRT-LLM for inference stage acceleration, and + 4) up to 4.2x speedups for Llama 2 pre-training on NVIDIA H200 Tensor Core GPUs. +

+ + H200-NeMo-performance +

+
+
+ + + NVIDIA now powers training for Amazon Titan Foundation models + (2023/11/28) + + NVIDIA NeMo Framework now empowers the Amazon Titan foundation models (FM) with efficient training of large language models (LLMs). + The Titan FMs form the basis of Amazon’s generative AI service, Amazon Bedrock. + The NeMo Framework provides a versatile framework for building, customizing, and running LLMs. +

+
+
+ + ## Licenses - [NeMo GitHub Apache 2.0