From aebd23e93c6fdadb18fc7136e57d9cb0a28acb94 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Sat, 26 Oct 2024 10:27:58 -0400 Subject: [PATCH 1/6] dl primer updates --- contents/dl_primer/dl_primer.qmd | 34 +++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/contents/dl_primer/dl_primer.qmd b/contents/dl_primer/dl_primer.qmd index 08718e5b..8b7c748f 100644 --- a/contents/dl_primer/dl_primer.qmd +++ b/contents/dl_primer/dl_primer.qmd @@ -10,7 +10,9 @@ Resources: [Slides](#sec-deep-learning-primer-resource), [Videos](#sec-deep-lear ![_DALL·E 3 Prompt: Photo of a classic classroom with a large blackboard dominating one wall. Chalk drawings showcase a detailed deep neural network with several hidden layers, and each node and connection is precisely labeled with white chalk. The rustic wooden floor and brick walls provide a contrast to the modern concepts. Surrounding the room, posters mounted on frames emphasize deep learning themes: convolutional networks, transformers, neurons, activation functions, and more._](images/png/cover_dl_primer.png) -This section briefly introduces deep learning, starting with an overview of its history, applications, and relevance to embedded AI systems. It examines the core concepts like neural networks, highlighting key components like perceptrons, multilayer perceptrons, activation functions, and computational graphs. The primer also briefly explores major deep learning architecture, contrasting their applications and uses. Additionally, it compares deep learning to traditional machine learning to equip readers with the general conceptual building blocks to make informed choices between deep learning and traditional ML techniques based on problem constraints, setting the stage for more advanced techniques and applications that will follow in subsequent chapters. +This section serves as a primer for deep learning, providing systems practitioners with essential context and foundational knowledge needed to implement deep learning solutions effectively. Rather than delving into theoretical depths, we focus on key concepts, architectures, and practical considerations relevant to systems implementation. We begin with an overview of deep learning's evolution and its particular significance in embedded AI systems. Core concepts like neural networks are introduced with an emphasis on implementation considerations rather than mathematical foundations. + +The primer explores major deep learning architectures from a systems perspective, examining their practical implications and resource requirements. We also compare deep learning to traditional machine learning approaches, helping readers make informed architectural choices based on real-world system constraints. This high-level overview sets the context for the more detailed systems-focused techniques and optimizations covered in subsequent chapters. ::: {.callout-tip} @@ -30,7 +32,11 @@ This section briefly introduces deep learning, starting with an overview of its ### Definition and Importance -Deep learning, a specialized area within machine learning and artificial intelligence (AI), utilizes algorithms modeled after the structure and function of the human brain, known as artificial neural networks. This field is a foundational element in AI, driving progress in diverse sectors such as computer vision, natural language processing, and self-driving vehicles. Its significance in embedded AI systems is highlighted by its capability to handle intricate calculations and predictions, optimizing the limited resources in embedded settings. @fig-ai-ml-dl illustrates the chronological development and relative segmentation of the three fields. +Deep learning, a specialized area within machine learning and artificial intelligence (AI), utilizes algorithms modeled after the structure and function of the human brain, known as artificial neural networks. This field is a foundational element in AI, driving progress in diverse sectors such as computer vision, natural language processing, and self-driving vehicles. Its significance in embedded AI systems is highlighted by its capability to handle intricate calculations and predictions, optimizing the limited resources in embedded settings. + +@fig-ai-ml-dl provides a visual representation of how deep learning fits within the broader context of AI and machine learning. The diagram illustrates the chronological development and relative segmentation of these three interconnected fields, showcasing deep learning as a specialized subset of machine learning, which in turn is a subset of AI. + +As depicted in the figure, AI represents the overarching field, encompassing all computational methods that mimic human cognitive functions. Machine learning, shown as a subset of AI, includes algorithms capable of learning from data. Deep learning, the smallest subset in the diagram, specifically involves neural networks that are able to learn more complex patterns from large volumes of data. ![The diagram illustrates artificial intelligence as the overarching field encompassing all computational methods that mimic human cognitive functions. Machine learning is a subset of AI that includes algorithms capable of learning from data. Deep learning, a further subset of ML, specifically involves neural networks that are able to learn more complex patterns in large volumes of data. Source: NVIDIA.](images/png/ai_dl_progress_nvidia.png){#fig-ai-ml-dl} @@ -40,7 +46,7 @@ The idea of deep learning has origins in early artificial neural networks. It ha The term "deep learning" became prominent in the 2000s, characterized by advances in computational power and data accessibility. Important milestones include the successful training of deep networks like AlexNet [@krizhevsky2012imagenet] by [Geoffrey Hinton](https://amturing.acm.org/award_winners/hinton_4791679.cfm), a leading figure in AI, and the renewed focus on neural networks as effective tools for data analysis and modeling. -Deep learning has recently seen exponential growth, transforming various industries. Computational growth followed an 18-month doubling pattern from 1952 to 2010, which then accelerated to a 6-month cycle from 2010 to 2022, as shown in @fig-trends. Concurrently, we saw the emergence of large-scale models between 2015 and 2022, appearing 2 to 3 orders of magnitude faster and following a 10-month doubling cycle. +Deep learning has recently seen exponential growth, transforming various industries. @fig-trends illustrates this remarkable progression, highlighting two key trends in the field. First, the graph shows that computational growth followed an 18-month doubling pattern from 1952 to 2010. This trend then dramatically accelerated to a 6-month doubling cycle from 2010 to 2022, indicating a significant leap in computational capabilities. Second, the figure depicts the emergence of large-scale models between 2015 and 2022. These models appeared 2 to 3 orders of magnitude faster than the general trend, following an even more aggressive 10-month doubling cycle. This rapid scaling of model sizes represents a paradigm shift in deep learning capabilities. ![Growth of deep learning models.](https://epochai.org/assets/images/posts/2022/compute-trends.png){#fig-trends} @@ -54,11 +60,11 @@ Organizations worldwide recognize deep learning's transformative potential and i ### Applications of Deep Learning -Deep learning is extensively used across numerous industries today, and its transformative impact on society is evident. In finance, it powers stock market prediction, risk assessment, and fraud detection. For instance, deep learning algorithms can predict stock market trends, guide investment strategies, and improve financial decisions. In marketing, it drives customer segmentation, personalization, and content optimization. Deep learning analyzes consumer behavior and preferences to enable highly targeted advertising and personalized content delivery. In manufacturing, deep learning streamlines production processes and enhances quality control by continuously analyzing large volumes of data. This allows companies to boost productivity and minimize waste, leading to the production of higher quality goods at lower costs. In healthcare, machine learning aids in diagnosis, treatment planning, and patient monitoring. Similarly, deep learning can make medical predictions that improve patient diagnosis and save lives. The benefits are clear: machine learning predicts with greater accuracy than humans and does so much more quickly. @fig-deeplearning further illustrates some applications of deep learning. +Deep learning is extensively used across numerous industries today, with its transformative impact evident in various sectors, as illustrated in @fig-deeplearning. In finance, it powers stock market prediction, risk assessment, and fraud detection, guiding investment strategies and improving financial decisions. Marketing leverages deep learning for customer segmentation and personalization, enabling highly targeted advertising and content optimization based on consumer behavior analysis. In manufacturing, it streamlines production processes and enhances quality control, allowing companies to boost productivity and minimize waste. Healthcare benefits from deep learning in diagnosis, treatment planning, and patient monitoring, potentially saving lives through improved medical predictions. -Deep learning enhances everyday products, such as strengthening Netflix's recommender systems to provide users with more [personalized recommendations](https://dl.acm.org/doi/abs/10.1145/3543873.3587675). At Google, deep learning models have driven significant improvements in [Google Translate](https://research.google/blog/recent-advances-in-google-translate/), enabling it to handle over [100 languages](https://cloud.google.com/translate/docs/languages). Autonomous vehicles from companies like Waymo, Cruise, and Motional have become a reality through the use of deep learning in their [perception system](https://motional.com/news/technically-speaking-improving-av-perception-through-transformative-machine-learning). Additionally, Amazon employs deep learning at the edge in their Alexa devices to perform [keyword spotting](https://towardsdatascience.com/how-amazon-alexa-works-your-guide-to-natural-language-processing-ai-7506004709d3). +![Deep learning applications, benefits, and implementations across various industries including finance, marketing, manufacturing, and healthcare. Source: [Leeway Hertz](https://www.leewayhertz.com/what-is-deep-learning/)](images/png/deeplearning.png){#fig-deeplearning} -![Deep learning applications, benefits and implementations. Source: [Leeway Hertz](https://www.leewayhertz.com/what-is-deep-learning/)](images/png/deeplearning.png){#fig-deeplearning} +Beyond these core industries, deep learning enhances everyday products and services. Netflix uses it to strengthen its recommender systems, providing users with more [personalized recommendations](https://dl.acm.org/doi/abs/10.1145/3543873.3587675). Google has significantly improved its Translate service, now handling over [100 languages](https://cloud.google.com/translate/docs/languages) with increased accuracy, as highlighted in their [recent advances](https://research.google/blog/recent-advances-in-google-translate/). Autonomous vehicles from companies like Waymo, Cruise, and Motional have become a reality through deep learning in their [perception system](https://motional.com/news/technically-speaking-improving-av-perception-through-transformative-machine-learning). Additionally, Amazon employs deep learning at the edge in Alexa devices for tasks such as [keyword spotting](https://towardsdatascience.com/how-amazon-alexa-works-your-guide-to-natural-language-processing-ai-7506004709d3). These applications demonstrate how machine learning often predicts and processes information with greater accuracy and speed than humans, revolutionizing various aspects of our daily lives. ### Relevance to Embedded AI @@ -106,12 +112,12 @@ A perceptron can be configured to perform either regression or classification ta ### Multilayer Perceptrons -Multilayer perceptrons (MLPs) are an evolution of the single-layer perceptron model, featuring multiple layers of nodes connected in a feedforward manner. In a feedforward network, information moves in only one direction - from the input layer, through the hidden layers, to the output layer, without any cycles or loops. This structure is illustrated in @fig-mlp. The network layers include an input layer for data reception, several hidden layers for data processing, and an output layer for final result generation. - -While a single perceptron is limited in its capacity to model complex patterns, the real strength of neural networks emerges from the assembly of multiple layers. Each layer consists of numerous perceptrons working together, allowing the network to capture intricate and non-linear relationships within the data. With sufficient depth and breadth, these networks can approximate virtually any function, no matter how complex. +Multilayer perceptrons (MLPs) are an evolution of the single-layer perceptron model, featuring multiple layers of nodes connected in a feedforward manner. @fig-mlp provides a visual representation of this structure. As illustrated in the figure, information in a feedforward network moves in only one direction - from the input layer on the left, through the hidden layers in the middle, to the output layer on the right, without any cycles or loops. ![Multilayer Perceptron. Source: Wikimedia - Charlie.](https://www.nomidl.com/wp-content/uploads/2022/04/image-7.png){width=70% #fig-mlp} +While a single perceptron is limited in its capacity to model complex patterns, the real strength of neural networks emerges from the assembly of multiple layers. Each layer consists of numerous perceptrons working together, allowing the network to capture intricate and non-linear relationships within the data. With sufficient depth and breadth, these networks can approximate virtually any function, no matter how complex. + ### Training Process A neural network receives an input, performs a calculation, and produces a prediction. The prediction is determined by the calculations performed within the sets of perceptrons found between the input and output layers. These calculations depend primarily on the input and the weights. Since you do not have control over the input, the objective during training is to adjust the weights in such a way that the output of the network provides the most accurate prediction. @@ -233,7 +239,13 @@ These architectures serve specific purposes and excel in different domains, offe ### Traditional ML vs Deep Learning -Deep learning extends traditional machine learning by utilizing neural networks to discern patterns in data. In contrast, traditional machine learning relies on a set of established algorithms such as decision trees, k-nearest neighbors, and support vector machines, but does not involve neural networks. To briefly highlight the differences, @tbl-mlvsdl illustrates the contrasting characteristics between traditional ML and deep learning. @fig-ml-dl further explains the differences between Machine Learning and Deep Learning. +Deep learning extends traditional machine learning by utilizing neural networks to discern patterns in data. In contrast, traditional machine learning relies on a set of established algorithms such as decision trees, k-nearest neighbors, and support vector machines, but does not involve neural networks. @fig-ml-dl provides a visual comparison of Machine Learning and Deep Learning, highlighting their key differences in approach and capabilities. + +![Comparing Machine Learning and Deep Learning. Source: [Medium](https://aoyilmaz.medium.com/understanding-the-differences-between-deep-learning-and-machine-learning-eb41d64f1732)](images/png/mlvsdl.png){#fig-ml-dl} + +As shown in the figure, deep learning models can process raw data directly and automatically extract relevant features, while traditional machine learning often requires manual feature engineering. The figure also illustrates how deep learning models can handle more complex tasks and larger datasets compared to traditional machine learning approaches. + +To further highlight the differences, @tbl-mlvsdl provides a more detailed comparison of the contrasting characteristics between traditional ML and deep learning. This table complements the visual representation in @fig-ml-dl by offering specific points of comparison across various aspects of these two approaches. +-------------------------------+-----------------------------------------------------------+--------------------------------------------------------------+ | Aspect | Traditional ML | Deep Learning | @@ -253,8 +265,6 @@ Deep learning extends traditional machine learning by utilizing neural networks : Comparison of traditional machine learning and deep learning. {#tbl-mlvsdl .striped .hover} -![Comparing Machine Learning and Deep Learning. Source: [Medium](https://aoyilmaz.medium.com/understanding-the-differences-between-deep-learning-and-machine-learning-eb41d64f1732)](images/png/mlvsdl.png){#fig-ml-dl} - ### Choosing Traditional ML vs. DL #### Data Availability and Volume From cbf3785f40a442828a51c8df9fa6f42836d25503 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Sat, 26 Oct 2024 10:28:08 -0400 Subject: [PATCH 2/6] ml systems update --- contents/ml_systems/ml_systems.qmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contents/ml_systems/ml_systems.qmd b/contents/ml_systems/ml_systems.qmd index 9a0ac851..6a80338b 100644 --- a/contents/ml_systems/ml_systems.qmd +++ b/contents/ml_systems/ml_systems.qmd @@ -12,7 +12,7 @@ Resources: [Slides](#sec-ml-systems-resource), [Videos](#sec-ml-systems-resource Machine learning (ML) systems, built on the foundation of computing systems, hold the potential to transform our world. These systems, with their specialized roles and real-time computational capabilities, represent a critical junction where data and computation meet on a micro-scale. They are specifically tailored to optimize performance, energy usage, and spatial efficiency—key factors essential for the successful implementation of ML systems. -As this chapter progresses, we will explore ML systems' complex and fascinating world. We'll gain insights into their structural design and operational features and understand their key role in powering ML applications. Starting with the basics of microcontroller units, we will examine the interfaces and peripherals that improve their functionalities. This chapter is designed to be a comprehensive guide elucidating the nuanced aspects of ML systems. +As this chapter progresses, we will explore ML systems' complex and fascinating world. We'll gain insights into their structural design and operational features and understand their key role in powering ML applications. Starting with the basics of microcontroller units, we will examine the interfaces and peripherals that improve their functionalities. This chapter is designed to be a comprehensive guide that explains the nuanced aspects of different ML systems. ::: {.callout-tip} From b961f753c8409ab6b49e38e78bd13792fc183eb8 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Mon, 28 Oct 2024 10:22:25 -0400 Subject: [PATCH 3/6] Bravo's fixes --- contents/labs/raspi/object_detection/object_detection.qmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/contents/labs/raspi/object_detection/object_detection.qmd b/contents/labs/raspi/object_detection/object_detection.qmd index 1bd4dac1..22b81904 100644 --- a/contents/labs/raspi/object_detection/object_detection.qmd +++ b/contents/labs/raspi/object_detection/object_detection.qmd @@ -100,11 +100,11 @@ As we saw in the introduction, given an image or a video stream, an object detec > You can test some common models online by visiting [Object Detection - MediaPipe Studio](https://mediapipe-studio.webapps.google.com/studio/demo/object_detector) -On [Kaggle](https://www.kaggle.com/models?id=298,130,299), we can find the most common pre-trained tflite models to use with the Raspi, [ssd_mobilenet_v1,](https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1/tfLite) and [efficiendet](https://www.kaggle.com/models/tensorflow/efficientdet/tfLite). Those models were trained on the COCO (Common Objects in Context) dataset, with over 200,000 labeled images in 91 categories. Go, download the models, and upload them to the `./models` folder in the Raspi. +On [Kaggle](https://www.kaggle.com/models?id=298,130,299), we can find the most common pre-trained tflite models to use with the Raspi, [ssd_mobilenet_v1,](https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1/tfLite) and [EfficientDet](https://www.kaggle.com/models/tensorflow/efficientdet/tfLite). Those models were trained on the COCO (Common Objects in Context) dataset, with over 200,000 labeled images in 91 categories. Go, download the models, and upload them to the `./models` folder in the Raspi. > Alternatively[,](https://github.com/Mjrovai/EdgeML-with-Raspberry-Pi/tree/main/OBJ_DETEC/models) you can find the models and the COCO labels on [GitHub](https://github.com/Mjrovai/EdgeML-with-Raspberry-Pi/tree/main/OBJ_DETEC/models). -For the first part of this lab, we will focus on a pre-trained 300x300 SSD-Mobilenet V1 model and compare it with the 320x320 EfficientDet-lite0, also trained using the COCO 2017 dataset. Both models were converted to a TensorFlow Lite format (4.2MB for the SSD Mobilenet and 4.6MB for the EfficienDet). +For the first part of this lab, we will focus on a pre-trained 300x300 SSD-Mobilenet V1 model and compare it with the 320x320 EfficientDet-lite0, also trained using the COCO 2017 dataset. Both models were converted to a TensorFlow Lite format (4.2MB for the SSD Mobilenet and 4.6MB for the EfficientDet). > SSD-Mobilenet V2 or V3 is recommended for transfer learning projects, but once the V1 TFLite model is publicly available, we will use it for this overview. From 355ab009437d3c79efd8005d019e480785ed30bd Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Mon, 28 Oct 2024 10:27:10 -0400 Subject: [PATCH 4/6] Bravo's suggested fixes --- .../arduino/nicla_vision/nicla_vision.qmd | 6 +- contents/labs/labs.qmd | 10 +- .../image_classification.qmd | 36 ++--- contents/labs/raspi/llm/llm.qmd | 126 +++++++++--------- contents/labs/raspi/raspi.qmd | 8 +- contents/labs/raspi/setup/setup.qmd | 52 ++++---- contents/labs/seeed/xiao_esp32s3/kws/kws.qmd | 6 +- .../labs/seeed/xiao_esp32s3/setup/setup.qmd | 12 +- .../labs/seeed/xiao_esp32s3/xiao_esp32s3.qmd | 8 +- contents/optimizations/optimizations.qmd | 8 +- .../privacy_security/privacy_security.qmd | 2 +- 11 files changed, 137 insertions(+), 137 deletions(-) diff --git a/contents/labs/arduino/nicla_vision/nicla_vision.qmd b/contents/labs/arduino/nicla_vision/nicla_vision.qmd index 50990d7c..c19e7b11 100644 --- a/contents/labs/arduino/nicla_vision/nicla_vision.qmd +++ b/contents/labs/arduino/nicla_vision/nicla_vision.qmd @@ -6,9 +6,9 @@ These labs provide a unique opportunity to gain practical experience with machin ## Pre-requisites -- **Nicla Vision Board** : Ensure you have the Nicla Vision board. -- **USB Cable** : For connecting the board to your computer. -- **Network** : With internet access for downloading necessary software. +- **Nicla Vision Board**: Ensure you have the Nicla Vision board. +- **USB Cable**: For connecting the board to your computer. +- **Network**: With internet access for downloading necessary software. ## Setup diff --git a/contents/labs/labs.qmd b/contents/labs/labs.qmd index a61d02d5..10fc8474 100644 --- a/contents/labs/labs.qmd +++ b/contents/labs/labs.qmd @@ -54,15 +54,15 @@ These labs are designed for: Each lab follows a structured approach: -1. **Introduction** : Explore the application and its significance in real-world scenarios. +1. **Introduction**: Explore the application and its significance in real-world scenarios. -2. **Setup** : Step-by-step instructions to configure the hardware and software environment. +2. **Setup**: Step-by-step instructions to configure the hardware and software environment. -3. **Deployment** : Guidance on training and deploying the pre-trained ML models on supported devices. +3. **Deployment**: Guidance on training and deploying the pre-trained ML models on supported devices. -4. **Exercises** : Hands-on tasks to modify and experiment with model parameters. +4. **Exercises**: Hands-on tasks to modify and experiment with model parameters. -5. **Discussion** : Analysis of results, potential improvements, and practical insights. +5. **Discussion**: Analysis of results, potential improvements, and practical insights. ## Troubleshooting and Support diff --git a/contents/labs/raspi/image_classification/image_classification.qmd b/contents/labs/raspi/image_classification/image_classification.qmd index b8631dc6..472934db 100644 --- a/contents/labs/raspi/image_classification/image_classification.qmd +++ b/contents/labs/raspi/image_classification/image_classification.qmd @@ -777,19 +777,19 @@ This Python script creates a web-based interface for capturing and organizing im #### Key Features: -1. **Web Interface** : Accessible from any device on the same network as the Raspberry Pi. -2. **Live Camera Preview** : This shows a real-time feed from the camera. -3. **Labeling System** : Allows users to input labels for different categories of images. -4. **Organized Storage** : Automatically saves images in label-specific subdirectories. -5. **Per-Label Counters** : Keeps track of how many images are captured for each label. -6. **Summary Statistics** : Provides a summary of captured images when stopping the capture process. +1. **Web Interface**: Accessible from any device on the same network as the Raspberry Pi. +2. **Live Camera Preview**: This shows a real-time feed from the camera. +3. **Labeling System**: Allows users to input labels for different categories of images. +4. **Organized Storage**: Automatically saves images in label-specific subdirectories. +5. **Per-Label Counters**: Keeps track of how many images are captured for each label. +6. **Summary Statistics**: Provides a summary of captured images when stopping the capture process. #### Main Components: -1. **Flask Web Application** : Handles routing and serves the web interface. -2. **Picamera2 Integration** : Controls the Raspberry Pi camera. -3. **Threaded Frame Capture** : Ensures smooth live preview. -4. **File Management** : Organizes captured images into labeled directories. +1. **Flask Web Application**: Handles routing and serves the web interface. +2. **Picamera2 Integration**: Controls the Raspberry Pi camera. +3. **Threaded Frame Capture**: Ensures smooth live preview. +4. **File Management**: Organizes captured images into labeled directories. #### Key Functions: @@ -1435,10 +1435,10 @@ The code creates a web application for real-time image classification using a Ra #### Key Components: -1. **Flask Web Application** : Serves the user interface and handles requests. -2. **PiCamera2** : Captures images from the Raspberry Pi camera module. -3. **TensorFlow Lite** : Runs the image classification model. -4. **Threading** : Manages concurrent operations for smooth performance. +1. **Flask Web Application**: Serves the user interface and handles requests. +2. **PiCamera2**: Captures images from the Raspberry Pi camera module. +3. **TensorFlow Lite**: Runs the image classification model. +4. **Threading**: Manages concurrent operations for smooth performance. #### Main Features: @@ -1491,10 +1491,10 @@ The code creates a web application for real-time image classification using a Ra #### Key Concepts: -1. **Concurrent Operations** : Using threads to handle camera capture and classification separately from the web server. -2. **Real-time Updates** : Frequent updates to the classification results without page reloads. -3. **Model Reuse** : Loading the TFLite model once and reusing it for efficiency. -4. **Flexible Configuration** : Allowing users to adjust the confidence threshold on the fly. +1. **Concurrent Operations**: Using threads to handle camera capture and classification separately from the web server. +2. **Real-time Updates**: Frequent updates to the classification results without page reloads. +3. **Model Reuse**: Loading the TFLite model once and reusing it for efficiency. +4. **Flexible Configuration**: Allowing users to adjust the confidence threshold on the fly. #### Usage: diff --git a/contents/labs/raspi/llm/llm.qmd b/contents/labs/raspi/llm/llm.qmd index d4716090..90098990 100644 --- a/contents/labs/raspi/llm/llm.qmd +++ b/contents/labs/raspi/llm/llm.qmd @@ -46,13 +46,13 @@ GenAI provides the conceptual framework for AI-driven content creation, with LLM Large Language Models (LLMs) are advanced artificial intelligence systems that understand, process, and generate human-like text. These models are characterized by their massive scale in terms of the amount of data they are trained on and the number of parameters they contain. Critical aspects of LLMs include: -1. **Size** : LLMs typically contain billions of parameters. For example, GPT-3 has 175 billion parameters, while some newer models exceed a trillion parameters. +1. **Size**: LLMs typically contain billions of parameters. For example, GPT-3 has 175 billion parameters, while some newer models exceed a trillion parameters. -2. **Training Data** : They are trained on vast amounts of text data, often including books, websites, and other diverse sources, amounting to hundreds of gigabytes or even terabytes of text. +2. **Training Data**: They are trained on vast amounts of text data, often including books, websites, and other diverse sources, amounting to hundreds of gigabytes or even terabytes of text. -3. **Architecture** : Most LLMs use [transformer-based architectures](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)), which allow them to process and generate text by paying attention to different parts of the input simultaneously. +3. **Architecture**: Most LLMs use [transformer-based architectures](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)), which allow them to process and generate text by paying attention to different parts of the input simultaneously. -4. **Capabilities** : LLMs can perform a wide range of language tasks without specific fine-tuning, including: +4. **Capabilities**: LLMs can perform a wide range of language tasks without specific fine-tuning, including: - Text generation - Translation - Summarization @@ -60,17 +60,17 @@ Large Language Models (LLMs) are advanced artificial intelligence systems that u - Code generation - Logical reasoning -5. **Few-shot Learning** : They can often understand and perform new tasks with minimal examples or instructions. +5. **Few-shot Learning**: They can often understand and perform new tasks with minimal examples or instructions. -6. **Resource-Intensive** : Due to their size, LLMs typically require significant computational resources to run, often needing powerful GPUs or TPUs. +6. **Resource-Intensive**: Due to their size, LLMs typically require significant computational resources to run, often needing powerful GPUs or TPUs. -7. **Continual Development** : The field of LLMs is rapidly evolving, with new models and techniques constantly emerging. +7. **Continual Development**: The field of LLMs is rapidly evolving, with new models and techniques constantly emerging. -8. **Ethical Considerations** : The use of LLMs raises important questions about bias, misinformation, and the environmental impact of training such large models. +8. **Ethical Considerations**: The use of LLMs raises important questions about bias, misinformation, and the environmental impact of training such large models. -9. **Applications** : LLMs are used in various fields, including content creation, customer service, research assistance, and software development. +9. **Applications**: LLMs are used in various fields, including content creation, customer service, research assistance, and software development. -10. **Limitations** : Despite their power, LLMs can produce incorrect or biased information and lack true understanding or reasoning capabilities. +10. **Limitations**: Despite their power, LLMs can produce incorrect or biased information and lack true understanding or reasoning capabilities. We must note that we use large models beyond text, calling them *multi-modal models*. These models integrate and process information from multiple types of input simultaneously. They are designed to understand and generate content across various forms of data, such as text, images, audio, and video. @@ -94,17 +94,17 @@ SLMs are compact versions of LLMs designed to run efficiently on resource-constr Key characteristics of SLMs include: -1. **Reduced parameter count** : Typically ranging from a few hundred million to a few billion parameters, compared to two-digit billions in larger models. +1. **Reduced parameter count**: Typically ranging from a few hundred million to a few billion parameters, compared to two-digit billions in larger models. -2. **Lower memory footprint** : Requiring, at most, a few gigabytes of memory rather than tens or hundreds of gigabytes. +2. **Lower memory footprint**: Requiring, at most, a few gigabytes of memory rather than tens or hundreds of gigabytes. -3. **Faster inference time** : Can generate responses in milliseconds to seconds on edge devices. +3. **Faster inference time**: Can generate responses in milliseconds to seconds on edge devices. -4. **Energy efficiency** : Consuming less power, making them suitable for battery-powered devices. +4. **Energy efficiency**: Consuming less power, making them suitable for battery-powered devices. -5. **Privacy-preserving** : Enabling on-device processing without sending data to cloud servers. +5. **Privacy-preserving**: Enabling on-device processing without sending data to cloud servers. -6. **Offline functionality** : Operating without an internet connection. +6. **Offline functionality**: Operating without an internet connection. SLMs achieve their compact size through various techniques such as knowledge distillation, model pruning, and quantization. While they may not match the broad capabilities of larger models, SLMs excel in specific tasks and domains, making them ideal for targeted applications on edge devices. @@ -120,25 +120,25 @@ For more information on SLMs, the paper, [LLM Pruning and Distillation in Practi [Ollama](https://ollama.com/) is an open-source framework that allows us to run language models (LMs), large or small, locally on our machines. Here are some critical points about Ollama: -1. **Local Model Execution** : Ollama enables running LMs on personal computers or edge devices such as the Raspi-5, eliminating the need for cloud-based API calls. +1. **Local Model Execution**: Ollama enables running LMs on personal computers or edge devices such as the Raspi-5, eliminating the need for cloud-based API calls. -2. **Ease of Use** : It provides a simple command-line interface for downloading, running, and managing different language models. +2. **Ease of Use**: It provides a simple command-line interface for downloading, running, and managing different language models. -3. **Model Variety** : Ollama supports various LLMs, including Phi, Gemma, Llama, Mistral, and other open-source models. +3. **Model Variety**: Ollama supports various LLMs, including Phi, Gemma, Llama, Mistral, and other open-source models. -4. **Customization** : Users can create and share custom models tailored to specific needs or domains. +4. **Customization**: Users can create and share custom models tailored to specific needs or domains. -5. **Lightweight** : Designed to be efficient and run on consumer-grade hardware. +5. **Lightweight**: Designed to be efficient and run on consumer-grade hardware. -6. **API Integration** : Offers an API that allows integration with other applications and services. +6. **API Integration**: Offers an API that allows integration with other applications and services. -7. **Privacy-Focused** : By running models locally, it addresses privacy concerns associated with sending data to external servers. +7. **Privacy-Focused**: By running models locally, it addresses privacy concerns associated with sending data to external servers. -8. **Cross-Platform** : Available for macOS, Windows, and Linux systems (our case, here). +8. **Cross-Platform**: Available for macOS, Windows, and Linux systems (our case, here). -9. **Active Development** : Regularly updated with new features and model support. +9. **Active Development**: Regularly updated with new features and model support. -10. **Community-Driven** : Benefits from community contributions and model sharing. +10. **Community-Driven**: Benefits from community contributions and model sharing. To learn more about what Ollama is and how it works under the hood, you should see this short video from [Matt Williams](https://www.youtube.com/@technovangelist), one of the founders of Ollama: @@ -205,14 +205,14 @@ Using the option `--verbose` when calling the model will generate several statis Each metric gives insights into how the model processes inputs and generates outputs. Here’s a breakdown of what each metric means: -- **Total Duration (2.620170326s)** : This is the complete time taken from the start of the command to the completion of the response. It encompasses loading the model, processing the input prompt, and generating the response. -- **Load Duration (39.947908ms)** : This duration indicates the time to load the model or necessary components into memory. If this value is minimal, it can suggest that the model was preloaded or that only a minimal setup was required. -- **Prompt Eval Count (32 tokens)** : The number of tokens in the input prompt. In NLP, tokens are typically words or subwords, so this count includes all the tokens that the model evaluated to understand and respond to the query. -- **Prompt Eval Duration (1.644773s)** : This measures the model's time to evaluate or process the input prompt. It accounts for the bulk of the total duration, implying that understanding the query and preparing a response is the most time-consuming part of the process. -- **Prompt Eval Rate (19.46 tokens/s)** : This rate indicates how quickly the model processes tokens from the input prompt. It reflects the model’s speed in terms of natural language comprehension. -- **Eval Count (8 token(s))** : This is the number of tokens in the model’s response, which in this case was, “The capital of France is Paris.” -- **Eval Duration (889.941ms)** : This is the time taken to generate the output based on the evaluated input. It’s much shorter than the prompt evaluation, suggesting that generating the response is less complex or computationally intensive than understanding the prompt. -- **Eval Rate (8.99 tokens/s)** : Similar to the prompt eval rate, this indicates the speed at which the model generates output tokens. It's a crucial metric for understanding the model's efficiency in output generation. +- **Total Duration (2.620170326s)**: This is the complete time taken from the start of the command to the completion of the response. It encompasses loading the model, processing the input prompt, and generating the response. +- **Load Duration (39.947908ms)**: This duration indicates the time to load the model or necessary components into memory. If this value is minimal, it can suggest that the model was preloaded or that only a minimal setup was required. +- **Prompt Eval Count (32 tokens)**: The number of tokens in the input prompt. In NLP, tokens are typically words or subwords, so this count includes all the tokens that the model evaluated to understand and respond to the query. +- **Prompt Eval Duration (1.644773s)**: This measures the model's time to evaluate or process the input prompt. It accounts for the bulk of the total duration, implying that understanding the query and preparing a response is the most time-consuming part of the process. +- **Prompt Eval Rate (19.46 tokens/s)**: This rate indicates how quickly the model processes tokens from the input prompt. It reflects the model’s speed in terms of natural language comprehension. +- **Eval Count (8 token(s))**: This is the number of tokens in the model’s response, which in this case was, “The capital of France is Paris.” +- **Eval Duration (889.941ms)**: This is the time taken to generate the output based on the evaluated input. It’s much shorter than the prompt evaluation, suggesting that generating the response is less complex or computationally intensive than understanding the prompt. +- **Eval Rate (8.99 tokens/s)**: Similar to the prompt eval rate, this indicates the speed at which the model generates output tokens. It's a crucial metric for understanding the model's efficiency in output generation. This detailed breakdown can help understand the computational demands and performance characteristics of running SLMs like Llama on edge devices like the Raspberry Pi 5. It shows that while prompt evaluation is more time-consuming, the actual generation of responses is relatively quicker. This analysis is crucial for optimizing performance and diagnosing potential bottlenecks in real-time applications. @@ -526,10 +526,10 @@ As a result, we will have the model response in a JSON format: As we can see, several pieces of information are generated, such as: -- **response** : the main output text generated by the model in response to our prompt. +- **response**: the main output text generated by the model in response to our prompt. - `The capital of France is **Paris**. 🇫🇷` -- **context** : the token IDs representing the input and context used by the model. Tokens are numerical representations of text used for processing by the language model. +- **context**: the token IDs representing the input and context used by the model. Tokens are numerical representations of text used for processing by the language model. - `[106, 1645, 108, 1841, 603, 573, 6037, 576, 6081, 235336, 107, 108,` ` 106, 2516, 108, 651, 6037, 576, 6081, 603, 5231, 29437, 168428, ` ` 235248, 244304, 241035, 235248, 108]` @@ -537,11 +537,11 @@ As we can see, several pieces of information are generated, such as: The Performance Metrics: -- **total_duration** : The total time taken for the operation in nanoseconds. In this case, approximately 24.26 seconds. -- **load_duration** : The time taken to load the model or components in nanoseconds. About 19.83 seconds. -- **prompt_eval_duration** : The time taken to evaluate the prompt in nanoseconds. Around 16 nanoseconds. -- **eval_count** : The number of tokens evaluated during the generation. Here, 14 tokens. -- **eval_duration** : The time taken for the model to generate the response in nanoseconds. Approximately 2.5 seconds. +- **total_duration**: The total time taken for the operation in nanoseconds. In this case, approximately 24.26 seconds. +- **load_duration**: The time taken to load the model or components in nanoseconds. About 19.83 seconds. +- **prompt_eval_duration**: The time taken to evaluate the prompt in nanoseconds. Around 16 nanoseconds. +- **eval_count**: The number of tokens evaluated during the generation. Here, 14 tokens. +- **eval_duration**: The time taken for the model to generate the response in nanoseconds. Approximately 2.5 seconds. But, what we want is the plain 'response' and, perhaps for analysis, the total duration of the inference, so let's change the code to extract it from the dictionary: @@ -708,11 +708,11 @@ from pydantic import BaseModel, Field import instructor ``` -- **sys** : Provides access to system-specific parameters and functions. It's used to get command-line arguments. -- **haversine** : A function from the haversine library that calculates the distance between two geographic points using the Haversine formula. -- **openAI** : A module for interacting with the OpenAI API (although it's used in conjunction with a local setup, Ollama). Everything is off-line here. -- **pydantic** : Provides data validation and settings management using Python-type annotations. It's used to define the structure of expected response data. -- **instructor** : A module is used to patch the OpenAI client to work in a specific mode (likely related to structured data handling). +- **sys**: Provides access to system-specific parameters and functions. It's used to get command-line arguments. +- **haversine**: A function from the haversine library that calculates the distance between two geographic points using the Haversine formula. +- **openAI**: A module for interacting with the OpenAI API (although it's used in conjunction with a local setup, Ollama). Everything is off-line here. +- **pydantic**: Provides data validation and settings management using Python-type annotations. It's used to define the structure of expected response data. +- **instructor**: A module is used to patch the OpenAI client to work in a specific mode (likely related to structured data handling). ### 2. Defining Input and Model @@ -723,11 +723,11 @@ mylat = -33.33 # Latitude of Santiago de Chile mylon = -70.51 # Longitude of Santiago de Chile ``` -- **country** : On a Python script, getting the country name from command-line arguments is possible. On a Jupyter notebook, we can enter its name, for example, +- **country**: On a Python script, getting the country name from command-line arguments is possible. On a Jupyter notebook, we can enter its name, for example, - `country = "France"` -- **MODEL** : Specifies the model being used, which is, in this example, the phi3.5. -- **mylat** **and** **mylon** : Coordinates of Santiago de Chile, used as the starting point for the distance calculation. +- **MODEL**: Specifies the model being used, which is, in this example, the phi3.5. +- **mylat** **and** **mylon**: Coordinates of Santiago de Chile, used as the starting point for the distance calculation. ### 3. Defining the Response Data Structure @@ -738,7 +738,7 @@ class CityCoord(BaseModel): lon: float = Field(..., description="Decimal Longitude of the city") ``` -- **CityCoord** : A Pydantic model that defines the expected structure of the response from the LLM. It expects three fields: city (name of the city), lat (latitude), and lon (longitude). +- **CityCoord**: A Pydantic model that defines the expected structure of the response from the LLM. It expects three fields: city (name of the city), lat (latitude), and lon (longitude). ### 4. Setting Up the OpenAI Client @@ -752,8 +752,8 @@ client = instructor.patch( ) ``` -- **OpenAI** : This setup initializes an OpenAI client with a local base URL and an API key (ollama). It uses a local server. -- **instructor.patch** : Patches the OpenAI client to work in JSON mode, enabling structured output that matches the Pydantic model. +- **OpenAI**: This setup initializes an OpenAI client with a local base URL and an API key (ollama). It uses a local server. +- **instructor.patch**: Patches the OpenAI client to work in JSON mode, enabling structured output that matches the Pydantic model. ### 5. Generating the Response @@ -772,11 +772,11 @@ resp = client.chat.completions.create( ) ``` -- **client.chat.completions.create** : Calls the LLM to generate a response. -- **model** : Specifies the model to use (llava-phi3). -- **messages** : Contains the prompt for the LLM, asking for the latitude and longitude of the capital city of the specified country. -- **response_model** : Indicates that the response should conform to the CityCoord model. -- **max_retries** : The maximum number of retry attempts if the request fails. +- **client.chat.completions.create**: Calls the LLM to generate a response. +- **model**: Specifies the model to use (llava-phi3). +- **messages**: Contains the prompt for the LLM, asking for the latitude and longitude of the capital city of the specified country. +- **response_model**: Indicates that the response should conform to the CityCoord model. +- **max_retries**: The maximum number of retry attempts if the request fails. ### 6. Calculating the Distance @@ -786,12 +786,12 @@ print(f"Santiago de Chile is about {int(round(distance, -1)):,} \ kilometers away from {resp.city}.") ``` -- **haversine** : Calculates the distance between Santiago de Chile and the capital city returned by the LLM using their respective coordinates. -- **(mylat, mylon)** : Coordinates of Santiago de Chile. -- **resp.city** : Name of the country's capital -- **(resp.lat, resp.lon)** : Coordinates of the capital city are provided by the LLM response. -- **unit='km'** : Specifies that the distance should be calculated in kilometers. -- **print** : Outputs the distance, rounded to the nearest 10 kilometers, with thousands of separators for readability. +- **haversine**: Calculates the distance between Santiago de Chile and the capital city returned by the LLM using their respective coordinates. +- **(mylat, mylon)**: Coordinates of Santiago de Chile. +- **resp.city**: Name of the country's capital +- **(resp.lat, resp.lon)**: Coordinates of the capital city are provided by the LLM response. +- **unit='km'**: Specifies that the distance should be calculated in kilometers. +- **print**: Outputs the distance, rounded to the nearest 10 kilometers, with thousands of separators for readability. **Running the code** diff --git a/contents/labs/raspi/raspi.qmd b/contents/labs/raspi/raspi.qmd index 02241339..2e6dfe95 100644 --- a/contents/labs/raspi/raspi.qmd +++ b/contents/labs/raspi/raspi.qmd @@ -6,13 +6,13 @@ These labs offer invaluable hands-on experience with machine learning systems, l ## Pre-requisites -- **Raspberry Pi** : Ensure you have at least one of the boards: the Raspberry Pi Zero 2W, Raspberry Pi 4 or 5 for the Vision Labs, and the Raspberry 5 for the GenAi lab. -- **Power Adapter** : To Power on the boards. +- **Raspberry Pi**: Ensure you have at least one of the boards: the Raspberry Pi Zero 2W, Raspberry Pi 4 or 5 for the Vision Labs, and the Raspberry 5 for the GenAi lab. +- **Power Adapter**: To Power on the boards. - Raspberry Pi Zero 2-W: 2.5W with a Micro-USB adapter - Raspberry Pi 4 or 5: 3.5W with a USB-C adapter -- **Network** : With internet access for downloading the necessary software and controlling the boards remotely. -- **SD Card (32GB minimum) and an SD card Adapter** : For the Raspberry Pi OS. +- **Network**: With internet access for downloading the necessary software and controlling the boards remotely. +- **SD Card (32GB minimum) and an SD card Adapter**: For the Raspberry Pi OS. ## Setup diff --git a/contents/labs/raspi/setup/setup.qmd b/contents/labs/raspi/setup/setup.qmd index 806455d5..4cf0a292 100644 --- a/contents/labs/raspi/setup/setup.qmd +++ b/contents/labs/raspi/setup/setup.qmd @@ -12,17 +12,17 @@ The Raspberry Pi is a powerful and versatile single-board computer that has beco ### Key Features -1. **Computational Power** : Despite their small size, Raspberry Pis offers significant processing capabilities, with the latest models featuring multi-core ARM processors and up to 8GB of RAM. +1. **Computational Power**: Despite their small size, Raspberry Pis offers significant processing capabilities, with the latest models featuring multi-core ARM processors and up to 8GB of RAM. -2. **GPIO Interface** : The 40-pin GPIO header allows direct interaction with sensors, actuators, and other electronic components, facilitating hardware-software integration projects. +2. **GPIO Interface**: The 40-pin GPIO header allows direct interaction with sensors, actuators, and other electronic components, facilitating hardware-software integration projects. -3. **Extensive Connectivity** : Built-in Wi-Fi, Bluetooth, Ethernet, and multiple USB ports enable diverse communication and networking projects. +3. **Extensive Connectivity**: Built-in Wi-Fi, Bluetooth, Ethernet, and multiple USB ports enable diverse communication and networking projects. -4. **Low-Level Hardware Access** : Raspberry Pis provides access to interfaces like I2C, SPI, and UART, allowing for detailed control and communication with external devices. +4. **Low-Level Hardware Access**: Raspberry Pis provides access to interfaces like I2C, SPI, and UART, allowing for detailed control and communication with external devices. -5. **Real-Time Capabilities** : With proper configuration, Raspberry Pis can be used for soft real-time applications, making them suitable for control systems and signal processing tasks. +5. **Real-Time Capabilities**: With proper configuration, Raspberry Pis can be used for soft real-time applications, making them suitable for control systems and signal processing tasks. -6. **Power Efficiency** : Low power consumption enables battery-powered and energy-efficient designs, especially in models like the Pi Zero. +6. **Power Efficiency**: Low power consumption enables battery-powered and energy-efficient designs, especially in models like the Pi Zero. ### Raspberry Pi Models (covered in this book) @@ -36,21 +36,21 @@ The Raspberry Pi is a powerful and versatile single-board computer that has beco ### Engineering Applications -1. **Embedded Systems Design** : Develop and prototype embedded systems for real-world applications. +1. **Embedded Systems Design**: Develop and prototype embedded systems for real-world applications. -2. **IoT and Networked Devices** : Create interconnected devices and explore protocols like MQTT, CoAP, and HTTP/HTTPS. +2. **IoT and Networked Devices**: Create interconnected devices and explore protocols like MQTT, CoAP, and HTTP/HTTPS. -3. **Control Systems** : Implement feedback control loops, PID controllers, and interface with actuators. +3. **Control Systems**: Implement feedback control loops, PID controllers, and interface with actuators. -4. **Computer Vision and AI** : Utilize libraries like OpenCV and TensorFlow Lite for image processing and machine learning at the edge. +4. **Computer Vision and AI**: Utilize libraries like OpenCV and TensorFlow Lite for image processing and machine learning at the edge. -5. **Data Acquisition and Analysis** : Collect sensor data, perform real-time analysis, and create data logging systems. +5. **Data Acquisition and Analysis**: Collect sensor data, perform real-time analysis, and create data logging systems. -6. **Robotics** : Build robot controllers, implement motion planning algorithms, and interface with motor drivers. +6. **Robotics**: Build robot controllers, implement motion planning algorithms, and interface with motor drivers. -7. **Signal Processing** : Perform real-time signal analysis, filtering, and DSP applications. +7. **Signal Processing**: Perform real-time signal analysis, filtering, and DSP applications. -8. **Network Security** : Set up VPNs, firewalls, and explore network penetration testing. +8. **Network Security**: Set up VPNs, firewalls, and explore network penetration testing. This tutorial will guide you through setting up the most common Raspberry Pi models, enabling you to start on your machine learning project quickly. We'll cover hardware setup, operating system installation, and initial configuration, focusing on preparing your Pi for Machine Learning applications. @@ -60,23 +60,23 @@ This tutorial will guide you through setting up the most common Raspberry Pi mod ![](images/jpeg/zero-hardware.jpg) -- **Processor** : 1GHz quad-core 64-bit Arm Cortex-A53 CPU -- **RAM** : 512MB SDRAM -- **Wireless** : 2.4GHz 802.11 b/g/n wireless LAN, Bluetooth 4.2, BLE -- **Ports** : Mini HDMI, micro USB OTG, CSI-2 camera connector -- **Power** : 5V via micro USB port +- **Processor**: 1GHz quad-core 64-bit Arm Cortex-A53 CPU +- **RAM**: 512MB SDRAM +- **Wireless**: 2.4GHz 802.11 b/g/n wireless LAN, Bluetooth 4.2, BLE +- **Ports**: Mini HDMI, micro USB OTG, CSI-2 camera connector +- **Power**: 5V via micro USB port ### Raspberry Pi 5 ![](images/jpeg/r5-hardware.jpg) -- **Processor** : +- **Processor**: - Pi 5: Quad-core 64-bit Arm Cortex-A76 CPU @ 2.4GHz - Pi 4: Quad-core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz -- **RAM** : 2GB, 4GB, or 8GB options (8GB recommended for AI tasks) -- **Wireless** : Dual-band 802.11ac wireless, Bluetooth 5.0 -- **Ports** : 2 × micro HDMI ports, 2 × USB 3.0 ports, 2 × USB 2.0 ports, CSI camera port, DSI display port -- **Power** : 5V DC via USB-C connector (3A) +- **RAM**: 2GB, 4GB, or 8GB options (8GB recommended for AI tasks) +- **Wireless**: Dual-band 802.11ac wireless, Bluetooth 5.0 +- **Ports**: 2 × micro HDMI ports, 2 × USB 3.0 ports, 2 × USB 2.0 ports, CSI camera port, DSI display port +- **Power**: 5V DC via USB-C connector (3A) > In the labs, we will use different names to address the Raspberry: `Raspi`, `Raspi-5`, `Raspi-Zero`, etc. Usually, `Raspi` is used when the instructions or comments apply to every model. @@ -124,14 +124,14 @@ Follow the steps to install the OS in your Raspi. 2. Insert a microSD card into your computer (a 32GB SD card is recommended) . 3. Open Raspberry Pi Imager and select your Raspberry Pi model. 4. Choose the appropriate operating system: - - **For Raspi-Zero** : For example, you can select: + - **For Raspi-Zero**: For example, you can select: `Raspberry Pi OS Lite (64-bit)`. ![img](images/png/zero-burn.png) > Due to its reduced SDRAM (512MB), the recommended OS for the Raspi-Zero is the 32-bit version. However, to run some machine learning models, such as the YOLOv8 from Ultralitics, we should use the 64-bit version. Although Raspi-Zero can run a *desktop*, we will choose the LITE version (no Desktop) to reduce the RAM needed for regular operation. - - For **Raspi-5** : We can select the full 64-bit version, which includes a desktop: + - For **Raspi-5**: We can select the full 64-bit version, which includes a desktop: `Raspberry Pi OS (64-bit)` ![](images/png/r5-burn.png) diff --git a/contents/labs/seeed/xiao_esp32s3/kws/kws.qmd b/contents/labs/seeed/xiao_esp32s3/kws/kws.qmd index 2c9efecf..e1e6f81f 100644 --- a/contents/labs/seeed/xiao_esp32s3/kws/kws.qmd +++ b/contents/labs/seeed/xiao_esp32s3/kws/kws.qmd @@ -99,11 +99,11 @@ The I2S protocol consists of at least three lines: ![](https://hackster.imgix.net/uploads/attachments/1594628/image_8CRJmXD9Fr.png?auto=compress%2Cformat&w=740&h=555&fit=max) -**1. Bit (or Serial) clock line (BCLK or CLK)** : This line toggles to indicate the start of a new bit of data (pin IO42). +**1. Bit (or Serial) clock line (BCLK or CLK)**: This line toggles to indicate the start of a new bit of data (pin IO42). -**2. Word select line (WS)** : This line toggles to indicate the start of a new word (left channel or right channel). The Word select clock (WS) frequency defines the sample rate. In our case, L/R on the microphone is set to ground, meaning that we will use only the left channel (mono). +**2. Word select line (WS)**: This line toggles to indicate the start of a new word (left channel or right channel). The Word select clock (WS) frequency defines the sample rate. In our case, L/R on the microphone is set to ground, meaning that we will use only the left channel (mono). -**3. Data line (SD)** : This line carries the audio data (pin IO41) +**3. Data line (SD)**: This line carries the audio data (pin IO41) In an I2S data stream, the data is sent as a sequence of frames, each containing a left-channel word and a right-channel word. This makes I2S particularly suited for transmitting stereo audio data. However, it can also be used for mono or multichannel audio with additional data lines. diff --git a/contents/labs/seeed/xiao_esp32s3/setup/setup.qmd b/contents/labs/seeed/xiao_esp32s3/setup/setup.qmd index 2b836e52..8caa62e0 100644 --- a/contents/labs/seeed/xiao_esp32s3/setup/setup.qmd +++ b/contents/labs/seeed/xiao_esp32s3/setup/setup.qmd @@ -10,12 +10,12 @@ The [XIAO ESP32S3 Sense](https://www.seeedstudio.com/XIAO-ESP32S3-Sense-p-5639.h **XIAO ESP32S3 Sense Main Features** -- **Powerful MCU Board** : Incorporate the ESP32S3 32-bit, dual-core, Xtensa processor chip operating up to 240 MHz, mounted multiple development ports, Arduino / MicroPython supported -- **Advanced Functionality** : Detachable OV2640 camera sensor for 1600 * 1200 resolution, compatible with OV5640 camera sensor, integrating an additional digital microphone -- **Elaborate Power Design** : Lithium battery charge management capability offers four power consumption models, which allows for deep sleep mode with power consumption as low as 14μA -- **Great Memory for more Possibilities** : Offer 8MB PSRAM and 8MB FLASH, supporting SD card slot for external 32GB FAT memory -- **Outstanding RF performance** : Support 2.4GHz Wi-Fi and BLE dual wireless communication, support 100m+ remote communication when connected with U.FL antenna -- **Thumb-sized Compact Design** : 21 x 17.5mm, adopting the classic form factor of XIAO, suitable for space-limited projects like wearable devices +- **Powerful MCU Board**: Incorporate the ESP32S3 32-bit, dual-core, Xtensa processor chip operating up to 240 MHz, mounted multiple development ports, Arduino / MicroPython supported +- **Advanced Functionality**: Detachable OV2640 camera sensor for 1600 * 1200 resolution, compatible with OV5640 camera sensor, integrating an additional digital microphone +- **Elaborate Power Design**: Lithium battery charge management capability offers four power consumption models, which allows for deep sleep mode with power consumption as low as 14μA +- **Great Memory for more Possibilities**: Offer 8MB PSRAM and 8MB FLASH, supporting SD card slot for external 32GB FAT memory +- **Outstanding RF performance**: Support 2.4GHz Wi-Fi and BLE dual wireless communication, support 100m+ remote communication when connected with U.FL antenna +- **Thumb-sized Compact Design**: 21 x 17.5mm, adopting the classic form factor of XIAO, suitable for space-limited projects like wearable devices ![](./images/png/xiao_pins.png) diff --git a/contents/labs/seeed/xiao_esp32s3/xiao_esp32s3.qmd b/contents/labs/seeed/xiao_esp32s3/xiao_esp32s3.qmd index 0d77dd03..90064ee8 100644 --- a/contents/labs/seeed/xiao_esp32s3/xiao_esp32s3.qmd +++ b/contents/labs/seeed/xiao_esp32s3/xiao_esp32s3.qmd @@ -6,10 +6,10 @@ These labs provide a unique opportunity to gain practical experience with machin ## Pre-requisites -- **XIAO ESP32S3 Sense Board** : Ensure you have the XIAO ESP32S3 Sense Board. -- **USB-C Cable** : This is for connecting the board to your computer. -- **Network** : With internet access for downloading necessary software. -- **SD Card and an SD card Adapter** : This saves audio and images (optional). +- **XIAO ESP32S3 Sense Board**: Ensure you have the XIAO ESP32S3 Sense Board. +- **USB-C Cable**: This is for connecting the board to your computer. +- **Network**: With internet access for downloading necessary software. +- **SD Card and an SD card Adapter**: This saves audio and images (optional). ## Setup diff --git a/contents/optimizations/optimizations.qmd b/contents/optimizations/optimizations.qmd index 826dd868..285ab5c5 100644 --- a/contents/optimizations/optimizations.qmd +++ b/contents/optimizations/optimizations.qmd @@ -91,10 +91,10 @@ A widely adopted and effective strategy for systematically pruning structures re There are several techniques for assigning these importance scores: -* **Weight Magnitude-Based Pruning** : This approach assigns importance scores to a structure by evaluating the aggregate magnitude of their associated weights. Structures with smaller overall weight magnitudes are considered less critical to the network's performance. -* **Gradient-Based Pruning** : This technique utilizes the gradients of the loss function with respect to the weights associated with a structure. Structures with low cumulative gradient magnitudes, indicating minimal impact on the loss when altered, are prime candidates for pruning. -* **Activation-Based Pruning** : This method tracks how often a neuron or filter is activated by storing this information in a parameter called the activation counter. Each time the structure is activated, the counter is incremented. A low activation count suggests that the structure is less relevant. -* **Taylor Expansion-Based Pruning** : This approach approximates the change in the loss function from removing a given weight. By assessing the cumulative loss disturbance from removing all the weights associated with a structure, you can identify structures with negligible impact on the loss, making them suitable candidates for pruning. +* **Weight Magnitude-Based Pruning**: This approach assigns importance scores to a structure by evaluating the aggregate magnitude of their associated weights. Structures with smaller overall weight magnitudes are considered less critical to the network's performance. +* **Gradient-Based Pruning**: This technique utilizes the gradients of the loss function with respect to the weights associated with a structure. Structures with low cumulative gradient magnitudes, indicating minimal impact on the loss when altered, are prime candidates for pruning. +* **Activation-Based Pruning**: This method tracks how often a neuron or filter is activated by storing this information in a parameter called the activation counter. Each time the structure is activated, the counter is incremented. A low activation count suggests that the structure is less relevant. +* **Taylor Expansion-Based Pruning**: This approach approximates the change in the loss function from removing a given weight. By assessing the cumulative loss disturbance from removing all the weights associated with a structure, you can identify structures with negligible impact on the loss, making them suitable candidates for pruning. The idea is to measure, either directly or indirectly, the contribution of each component to the model's output. Structures with minimal influence according to the defined criteria are pruned first. This enables selective, optimized pruning that maximally compresses models while preserving predictive capacity. In general, it is important to evaluate the impact of removing particular structures on the model's output, with recent works such as [@rachwan2022winning] and [@lubana2020gradient] investigating combinations of techniques like magnitude-based pruning and gradient-based pruning. diff --git a/contents/privacy_security/privacy_security.qmd b/contents/privacy_security/privacy_security.qmd index 4eecea48..5e080a9f 100644 --- a/contents/privacy_security/privacy_security.qmd +++ b/contents/privacy_security/privacy_security.qmd @@ -510,7 +510,7 @@ The fundamentals of TEEs contain four main parts: Here are some examples of TEEs that provide hardware-based security for sensitive applications: -* **[ARMTrustZone](https://www.arm.com/technologies/trustzone-for-cortex-m):**This technology creates secure and normal world execution environments isolated using hardware controls and implemented in many mobile chipsets. +* **[ARMTrustZone](https://www.arm.com/technologies/trustzone-for-cortex-m):** This technology creates secure and normal world execution environments isolated using hardware controls and implemented in many mobile chipsets. * **[IntelSGX](https://www.intel.com/content/www/us/en/architecture-and-technology/software-guard-extensions.html):** Intel's Software Guard Extensions provide an enclave for code execution that protects against various software-based threats, specifically targeting O.S. layer vulnerabilities. They are used to safeguard workloads in the cloud. From 020101b35d55d6eaf164ef07576b1b68da4e621c Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 28 Oct 2024 21:08:56 +0000 Subject: [PATCH 5/6] Update readme and contributors.qmd with contributors --- .all-contributorsrc | 256 +++++++++++++++++++------------------- README.md | 64 +++++----- contents/contributors.qmd | 64 +++++----- 3 files changed, 192 insertions(+), 192 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index d7c41af0..6f6ea966 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -69,13 +69,6 @@ "profile": "https://github.com/kai4avaya", "contributions": [] }, - { - "login": "JaredP94", - "name": "Jared Ping", - "avatar_url": "https://avatars.githubusercontent.com/JaredP94", - "profile": "https://github.com/JaredP94", - "contributions": [] - }, { "login": "Sara-Khosravi", "name": "Sara Khosravi", @@ -84,10 +77,10 @@ "contributions": [] }, { - "login": "mpstewart1", - "name": "Matthew Stewart", - "avatar_url": "https://avatars.githubusercontent.com/mpstewart1", - "profile": "https://github.com/mpstewart1", + "login": "JaredP94", + "name": "Jared Ping", + "avatar_url": "https://avatars.githubusercontent.com/JaredP94", + "profile": "https://github.com/JaredP94", "contributions": [] }, { @@ -97,6 +90,13 @@ "profile": "https://github.com/ishapira1", "contributions": [] }, + { + "login": "mpstewart1", + "name": "Matthew Stewart", + "avatar_url": "https://avatars.githubusercontent.com/mpstewart1", + "profile": "https://github.com/mpstewart1", + "contributions": [] + }, { "login": "Maximilian Lam", "name": "Maximilian Lam", @@ -146,13 +146,6 @@ "profile": "https://github.com/korneelf1", "contributions": [] }, - { - "login": "zishenwan", - "name": "Zishen Wan", - "avatar_url": "https://avatars.githubusercontent.com/zishenwan", - "profile": "https://github.com/zishenwan", - "contributions": [] - }, { "login": "colbybanbury", "name": "Colby Banbury", @@ -160,6 +153,13 @@ "profile": "https://github.com/colbybanbury", "contributions": [] }, + { + "login": "zishenwan", + "name": "Zishen Wan", + "avatar_url": "https://avatars.githubusercontent.com/zishenwan", + "profile": "https://github.com/zishenwan", + "contributions": [] + }, { "login": "DivyaAmirtharaj", "name": "Divya Amirtharaj", @@ -167,6 +167,13 @@ "profile": "https://github.com/DivyaAmirtharaj", "contributions": [] }, + { + "login": "srivatsankrishnan", + "name": "Srivatsan Krishnan", + "avatar_url": "https://avatars.githubusercontent.com/srivatsankrishnan", + "profile": "https://github.com/srivatsankrishnan", + "contributions": [] + }, { "login": "ma3mool", "name": "Abdulrahman Mahmoud", @@ -175,10 +182,10 @@ "contributions": [] }, { - "login": "srivatsankrishnan", - "name": "Srivatsan Krishnan", - "avatar_url": "https://avatars.githubusercontent.com/srivatsankrishnan", - "profile": "https://github.com/srivatsankrishnan", + "login": "James-QiuHaoran", + "name": "Haoran Qiu", + "avatar_url": "https://avatars.githubusercontent.com/James-QiuHaoran", + "profile": "https://github.com/James-QiuHaoran", "contributions": [] }, { @@ -188,13 +195,6 @@ "profile": "https://github.com/arnaumarin", "contributions": [] }, - { - "login": "eezike", - "name": "Emeka Ezike", - "avatar_url": "https://avatars.githubusercontent.com/eezike", - "profile": "https://github.com/eezike", - "contributions": [] - }, { "login": "aptl26", "name": "Aghyad Deeb", @@ -202,20 +202,6 @@ "profile": "https://github.com/aptl26", "contributions": [] }, - { - "login": "James-QiuHaoran", - "name": "Haoran Qiu", - "avatar_url": "https://avatars.githubusercontent.com/James-QiuHaoran", - "profile": "https://github.com/James-QiuHaoran", - "contributions": [] - }, - { - "login": "ELSuitorHarvard", - "name": "ELSuitorHarvard", - "avatar_url": "https://avatars.githubusercontent.com/ELSuitorHarvard", - "profile": "https://github.com/ELSuitorHarvard", - "contributions": [] - }, { "login": "AditiR-42", "name": "Aditi Raju", @@ -244,6 +230,13 @@ "profile": "https://github.com/jared-ni", "contributions": [] }, + { + "login": "ELSuitorHarvard", + "name": "ELSuitorHarvard", + "avatar_url": "https://avatars.githubusercontent.com/ELSuitorHarvard", + "profile": "https://github.com/ELSuitorHarvard", + "contributions": [] + }, { "login": "Ekhao", "name": "Emil Njor", @@ -251,13 +244,6 @@ "profile": "https://github.com/Ekhao", "contributions": [] }, - { - "login": "BaeHenryS", - "name": "Henry Bae", - "avatar_url": "https://avatars.githubusercontent.com/BaeHenryS", - "profile": "https://github.com/BaeHenryS", - "contributions": [] - }, { "login": "leo47007", "name": "Yu-Shun Hsiao", @@ -266,10 +252,10 @@ "contributions": [] }, { - "login": "mmaz", - "name": "Mark Mazumder", - "avatar_url": "https://avatars.githubusercontent.com/mmaz", - "profile": "https://github.com/mmaz", + "login": "BaeHenryS", + "name": "Henry Bae", + "avatar_url": "https://avatars.githubusercontent.com/BaeHenryS", + "profile": "https://github.com/BaeHenryS", "contributions": [] }, { @@ -280,17 +266,10 @@ "contributions": [] }, { - "login": "marcozennaro", - "name": "Marco Zennaro", - "avatar_url": "https://avatars.githubusercontent.com/marcozennaro", - "profile": "https://github.com/marcozennaro", - "contributions": [] - }, - { - "login": "euranofshin", - "name": "Eura Nofshin", - "avatar_url": "https://avatars.githubusercontent.com/euranofshin", - "profile": "https://github.com/euranofshin", + "login": "mmaz", + "name": "Mark Mazumder", + "avatar_url": "https://avatars.githubusercontent.com/mmaz", + "profile": "https://github.com/mmaz", "contributions": [] }, { @@ -307,6 +286,20 @@ "profile": "https://github.com/ShvetankPrakash", "contributions": [] }, + { + "login": "marcozennaro", + "name": "Marco Zennaro", + "avatar_url": "https://avatars.githubusercontent.com/marcozennaro", + "profile": "https://github.com/marcozennaro", + "contributions": [] + }, + { + "login": "euranofshin", + "name": "Eura Nofshin", + "avatar_url": "https://avatars.githubusercontent.com/euranofshin", + "profile": "https://github.com/euranofshin", + "contributions": [] + }, { "login": "arbass22", "name": "Andrew Bass", @@ -322,10 +315,10 @@ "contributions": [] }, { - "login": "gnodipac886", - "name": "gnodipac886", - "avatar_url": "https://avatars.githubusercontent.com/gnodipac886", - "profile": "https://github.com/gnodipac886", + "login": "Emeka Ezike", + "name": "Emeka Ezike", + "avatar_url": "https://www.gravatar.com/avatar/af39c27c6090c50a1921a9b6366e81cc?d=identicon&s=100", + "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", "contributions": [] }, { @@ -336,17 +329,10 @@ "contributions": [] }, { - "login": "Gjain234", - "name": "Gauri Jain", - "avatar_url": "https://avatars.githubusercontent.com/Gjain234", - "profile": "https://github.com/Gjain234", - "contributions": [] - }, - { - "login": "Allen-Kuang", - "name": "Allen-Kuang", - "avatar_url": "https://avatars.githubusercontent.com/Allen-Kuang", - "profile": "https://github.com/Allen-Kuang", + "login": "BrunoScaglione", + "name": "Bruno Scaglione", + "avatar_url": "https://avatars.githubusercontent.com/BrunoScaglione", + "profile": "https://github.com/BrunoScaglione", "contributions": [] }, { @@ -357,10 +343,17 @@ "contributions": [] }, { - "login": "BrunoScaglione", - "name": "Bruno Scaglione", - "avatar_url": "https://avatars.githubusercontent.com/BrunoScaglione", - "profile": "https://github.com/BrunoScaglione", + "login": "Allen-Kuang", + "name": "Allen-Kuang", + "avatar_url": "https://avatars.githubusercontent.com/Allen-Kuang", + "profile": "https://github.com/Allen-Kuang", + "contributions": [] + }, + { + "login": "Gjain234", + "name": "Gauri Jain", + "avatar_url": "https://avatars.githubusercontent.com/Gjain234", + "profile": "https://github.com/Gjain234", "contributions": [] }, { @@ -370,6 +363,13 @@ "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", "contributions": [] }, + { + "login": "gnodipac886", + "name": "gnodipac886", + "avatar_url": "https://avatars.githubusercontent.com/gnodipac886", + "profile": "https://github.com/gnodipac886", + "contributions": [] + }, { "login": "serco425", "name": "Sercan Ayg\u00fcn", @@ -384,13 +384,6 @@ "profile": "https://github.com/AbenezerKb", "contributions": [] }, - { - "login": "BravoBaldo", - "name": "Baldassarre Cesarano", - "avatar_url": "https://avatars.githubusercontent.com/BravoBaldo", - "profile": "https://github.com/BravoBaldo", - "contributions": [] - }, { "login": "emmanuel2406", "name": "Emmanuel Rassou", @@ -399,10 +392,10 @@ "contributions": [] }, { - "login": "bilgeacun", - "name": "Bilge Acun", - "avatar_url": "https://avatars.githubusercontent.com/bilgeacun", - "profile": "https://github.com/bilgeacun", + "login": "jasonlyik", + "name": "Jason Yik", + "avatar_url": "https://avatars.githubusercontent.com/jasonlyik", + "profile": "https://github.com/jasonlyik", "contributions": [] }, { @@ -412,13 +405,6 @@ "profile": "https://github.com/abigailswallow", "contributions": [] }, - { - "login": "YLab-UChicago", - "name": "yanjingl", - "avatar_url": "https://avatars.githubusercontent.com/YLab-UChicago", - "profile": "https://github.com/YLab-UChicago", - "contributions": [] - }, { "login": "YangZhou1997", "name": "Yang Zhou", @@ -427,10 +413,10 @@ "contributions": [] }, { - "login": "jasonlyik", - "name": "Jason Yik", - "avatar_url": "https://avatars.githubusercontent.com/jasonlyik", - "profile": "https://github.com/jasonlyik", + "login": "bilgeacun", + "name": "Bilge Acun", + "avatar_url": "https://avatars.githubusercontent.com/bilgeacun", + "profile": "https://github.com/bilgeacun", "contributions": [] }, { @@ -440,13 +426,6 @@ "profile": "https://github.com/happyappledog", "contributions": [] }, - { - "login": "ciyer64", - "name": "Curren Iyer", - "avatar_url": "https://avatars.githubusercontent.com/ciyer64", - "profile": "https://github.com/ciyer64", - "contributions": [] - }, { "login": "jessicaquaye", "name": "Jessica Quaye", @@ -455,10 +434,10 @@ "contributions": [] }, { - "login": "skmur", - "name": "Sonia Murthy", - "avatar_url": "https://avatars.githubusercontent.com/skmur", - "profile": "https://github.com/skmur", + "login": "vitasam", + "name": "The Random DIY", + "avatar_url": "https://avatars.githubusercontent.com/vitasam", + "profile": "https://github.com/vitasam", "contributions": [] }, { @@ -469,17 +448,10 @@ "contributions": [] }, { - "login": "vijay-edu", - "name": "Vijay Edupuganti", - "avatar_url": "https://avatars.githubusercontent.com/vijay-edu", - "profile": "https://github.com/vijay-edu", - "contributions": [] - }, - { - "login": "vitasam", - "name": "The Random DIY", - "avatar_url": "https://avatars.githubusercontent.com/vitasam", - "profile": "https://github.com/vitasam", + "login": "skmur", + "name": "Sonia Murthy", + "avatar_url": "https://avatars.githubusercontent.com/skmur", + "profile": "https://github.com/skmur", "contributions": [] }, { @@ -489,6 +461,13 @@ "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", "contributions": [] }, + { + "login": "Baldassarre Cesarano", + "name": "Baldassarre Cesarano", + "avatar_url": "https://www.gravatar.com/avatar/13b816dd84837bb4700a55f47a70763e?d=identicon&s=100", + "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", + "contributions": [] + }, { "login": "Annie Laurie Cook", "name": "Annie Laurie Cook", @@ -496,6 +475,13 @@ "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", "contributions": [] }, + { + "login": "Vijay Edupuganti", + "name": "Vijay Edupuganti", + "avatar_url": "https://www.gravatar.com/avatar/b15b6e0e9adf58099905c1a0fd474cb9?d=identicon&s=100", + "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", + "contributions": [] + }, { "login": "Jothi Ramaswamy", "name": "Jothi Ramaswamy", @@ -510,6 +496,13 @@ "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", "contributions": [] }, + { + "login": "Curren Iyer", + "name": "Curren Iyer", + "avatar_url": "https://www.gravatar.com/avatar/bd53d146aa888548c8db4da02bf81e7a?d=identicon&s=100", + "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", + "contributions": [] + }, { "login": "Fatima Shah", "name": "Fatima Shah", @@ -517,6 +510,13 @@ "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", "contributions": [] }, + { + "login": "yanjingl", + "name": "yanjingl", + "avatar_url": "https://www.gravatar.com/avatar/f5d58ba6aa9b00189d4c018d370e8f43?d=identicon&s=100", + "profile": "https://github.com/harvard-edge/cs249r_book/graphs/contributors", + "contributions": [] + }, { "login": "a-saraf", "name": "a-saraf", diff --git a/README.md b/README.md index 5a8d3c50..23e1a591 100644 --- a/README.md +++ b/README.md @@ -96,12 +96,12 @@ This project follows the [all-contributors](https://allcontributors.org) specifi shanzehbatool
shanzehbatool
Elias Nuwara
Elias Nuwara
kai4avaya
kai4avaya
- Jared Ping
Jared Ping
+ Sara Khosravi
Sara Khosravi
- Sara Khosravi
Sara Khosravi
- Matthew Stewart
Matthew Stewart
+ Jared Ping
Jared Ping
Itai Shapira
Itai Shapira
+ Matthew Stewart
Matthew Stewart
Maximilian Lam
Maximilian Lam
Jayson Lin
Jayson Lin
@@ -113,79 +113,79 @@ This project follows the [all-contributors](https://allcontributors.org) specifi Korneel Van den Berghe
Korneel Van den Berghe
- Zishen Wan
Zishen Wan
Colby Banbury
Colby Banbury
+ Zishen Wan
Zishen Wan
Divya Amirtharaj
Divya Amirtharaj
- Abdulrahman Mahmoud
Abdulrahman Mahmoud
Srivatsan Krishnan
Srivatsan Krishnan
+ Abdulrahman Mahmoud
Abdulrahman Mahmoud
+ Haoran Qiu
Haoran Qiu
marin-llobet
marin-llobet
- Emeka Ezike
Emeka Ezike
Aghyad Deeb
Aghyad Deeb
- Haoran Qiu
Haoran Qiu
- ELSuitorHarvard
ELSuitorHarvard
- - Aditi Raju
Aditi Raju
Michael Schnebly
Michael Schnebly
+ + oishib
oishib
Jared Ni
Jared Ni
+ ELSuitorHarvard
ELSuitorHarvard
Emil Njor
Emil Njor
+ Yu-Shun Hsiao
Yu-Shun Hsiao
Henry Bae
Henry Bae
- Yu-Shun Hsiao
Yu-Shun Hsiao
- Mark Mazumder
Mark Mazumder
Jae-Won Chung
Jae-Won Chung
- Marco Zennaro
Marco Zennaro
+ Mark Mazumder
Mark Mazumder
+ Pong Trairatvorakul
Pong Trairatvorakul
+ Shvetank Prakash
Shvetank Prakash
+ Marco Zennaro
Marco Zennaro
Eura Nofshin
Eura Nofshin
- Pong Trairatvorakul
Pong Trairatvorakul
- Shvetank Prakash
Shvetank Prakash
Andrew Bass
Andrew Bass
Jennifer Zhou
Jennifer Zhou
+ Emeka Ezike
Emeka Ezike
- gnodipac886
gnodipac886
Alex Oesterling
Alex Oesterling
- Gauri Jain
Gauri Jain
- Allen-Kuang
Allen-Kuang
+ Bruno Scaglione
Bruno Scaglione
Fin Amin
Fin Amin
+ Allen-Kuang
Allen-Kuang
+ Gauri Jain
Gauri Jain
- Bruno Scaglione
Bruno Scaglione
Fatima Shah
Fatima Shah
+ gnodipac886
gnodipac886
Sercan Aygün
Sercan Aygün
Abenezer
Abenezer
- Baldassarre Cesarano
Baldassarre Cesarano
- - Emmanuel Rassou
Emmanuel Rassou
- Bilge Acun
Bilge Acun
- abigailswallow
abigailswallow
- yanjingl
yanjingl
- Yang Zhou
Yang Zhou
Jason Yik
Jason Yik
+ abigailswallow
abigailswallow
+ Yang Zhou
Yang Zhou
+ Bilge Acun
Bilge Acun
happyappledog
happyappledog
- Curren Iyer
Curren Iyer
- Jessica Quaye
Jessica Quaye
- Sonia Murthy
Sonia Murthy
- Shreya Johri
Shreya Johri
- Vijay Edupuganti
Vijay Edupuganti
+ Jessica Quaye
Jessica Quaye
The Random DIY
The Random DIY
+ Shreya Johri
Shreya Johri
+ Sonia Murthy
Sonia Murthy
Costin-Andrei Oncescu
Costin-Andrei Oncescu
- Annie Laurie Cook
Annie Laurie Cook
+ Baldassarre Cesarano
Baldassarre Cesarano
+ Annie Laurie Cook
Annie Laurie Cook
+ Vijay Edupuganti
Vijay Edupuganti
Jothi Ramaswamy
Jothi Ramaswamy
Batur Arslan
Batur Arslan
+ + + Curren Iyer
Curren Iyer
Fatima Shah
Fatima Shah
+ yanjingl
yanjingl
a-saraf
a-saraf
songhan
songhan
diff --git a/contents/contributors.qmd b/contents/contributors.qmd index ddb925eb..6115155b 100644 --- a/contents/contributors.qmd +++ b/contents/contributors.qmd @@ -83,12 +83,12 @@ We extend our sincere thanks to the diverse group of individuals who have genero shanzehbatool
shanzehbatool
Elias Nuwara
Elias Nuwara
kai4avaya
kai4avaya
- Jared Ping
Jared Ping
+ Sara Khosravi
Sara Khosravi
- Sara Khosravi
Sara Khosravi
- Matthew Stewart
Matthew Stewart
+ Jared Ping
Jared Ping
Itai Shapira
Itai Shapira
+ Matthew Stewart
Matthew Stewart
Maximilian Lam
Maximilian Lam
Jayson Lin
Jayson Lin
@@ -100,79 +100,79 @@ We extend our sincere thanks to the diverse group of individuals who have genero Korneel Van den Berghe
Korneel Van den Berghe
- Zishen Wan
Zishen Wan
Colby Banbury
Colby Banbury
+ Zishen Wan
Zishen Wan
Divya Amirtharaj
Divya Amirtharaj
- Abdulrahman Mahmoud
Abdulrahman Mahmoud
Srivatsan Krishnan
Srivatsan Krishnan
+ Abdulrahman Mahmoud
Abdulrahman Mahmoud
+ Haoran Qiu
Haoran Qiu
marin-llobet
marin-llobet
- Emeka Ezike
Emeka Ezike
Aghyad Deeb
Aghyad Deeb
- Haoran Qiu
Haoran Qiu
- ELSuitorHarvard
ELSuitorHarvard
- - Aditi Raju
Aditi Raju
Michael Schnebly
Michael Schnebly
+ + oishib
oishib
Jared Ni
Jared Ni
+ ELSuitorHarvard
ELSuitorHarvard
Emil Njor
Emil Njor
+ Yu-Shun Hsiao
Yu-Shun Hsiao
Henry Bae
Henry Bae
- Yu-Shun Hsiao
Yu-Shun Hsiao
- Mark Mazumder
Mark Mazumder
Jae-Won Chung
Jae-Won Chung
- Marco Zennaro
Marco Zennaro
+ Mark Mazumder
Mark Mazumder
+ Pong Trairatvorakul
Pong Trairatvorakul
+ Shvetank Prakash
Shvetank Prakash
+ Marco Zennaro
Marco Zennaro
Eura Nofshin
Eura Nofshin
- Pong Trairatvorakul
Pong Trairatvorakul
- Shvetank Prakash
Shvetank Prakash
Andrew Bass
Andrew Bass
Jennifer Zhou
Jennifer Zhou
+ Emeka Ezike
Emeka Ezike
- gnodipac886
gnodipac886
Alex Oesterling
Alex Oesterling
- Gauri Jain
Gauri Jain
- Allen-Kuang
Allen-Kuang
+ Bruno Scaglione
Bruno Scaglione
Fin Amin
Fin Amin
+ Allen-Kuang
Allen-Kuang
+ Gauri Jain
Gauri Jain
- Bruno Scaglione
Bruno Scaglione
Fatima Shah
Fatima Shah
+ gnodipac886
gnodipac886
Sercan Aygün
Sercan Aygün
Abenezer
Abenezer
- Baldassarre Cesarano
Baldassarre Cesarano
- - Emmanuel Rassou
Emmanuel Rassou
- Bilge Acun
Bilge Acun
- abigailswallow
abigailswallow
- yanjingl
yanjingl
- Yang Zhou
Yang Zhou
Jason Yik
Jason Yik
+ abigailswallow
abigailswallow
+ Yang Zhou
Yang Zhou
+ Bilge Acun
Bilge Acun
happyappledog
happyappledog
- Curren Iyer
Curren Iyer
- Jessica Quaye
Jessica Quaye
- Sonia Murthy
Sonia Murthy
- Shreya Johri
Shreya Johri
- Vijay Edupuganti
Vijay Edupuganti
+ Jessica Quaye
Jessica Quaye
The Random DIY
The Random DIY
+ Shreya Johri
Shreya Johri
+ Sonia Murthy
Sonia Murthy
Costin-Andrei Oncescu
Costin-Andrei Oncescu
- Annie Laurie Cook
Annie Laurie Cook
+ Baldassarre Cesarano
Baldassarre Cesarano
+ Annie Laurie Cook
Annie Laurie Cook
+ Vijay Edupuganti
Vijay Edupuganti
Jothi Ramaswamy
Jothi Ramaswamy
Batur Arslan
Batur Arslan
+ + + Curren Iyer
Curren Iyer
Fatima Shah
Fatima Shah
+ yanjingl
yanjingl
a-saraf
a-saraf
songhan
songhan
From fee43d5a204678ea307f415602331c9b6ba91ee9 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Mon, 28 Oct 2024 17:12:56 -0400 Subject: [PATCH 6/6] Marcelo committed the labs Co-authored-by: Marcelo Rovai --- contents/labs/labs.qmd | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/contents/labs/labs.qmd b/contents/labs/labs.qmd index 10fc8474..cd971e6f 100644 --- a/contents/labs/labs.qmd +++ b/contents/labs/labs.qmd @@ -41,13 +41,13 @@ These labs are designed for: +----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+ | Keyword Spotting (KWS) | | | | +----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+ -| Image Classification | | | Coming soon. | +| Image Classification | | | | +----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+ -| Object Detection | | | Coming soon. | +| Object Detection | | | | +----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+ | Motion Detection | | | | +----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+ -| Small Language Models (SLM)| | | Coming soon. | +| Small Language Models (SLM)| | | | +----------------------------+----------------------------------------------------------------+----------------------------------------------------------------------------+----------------------------------------------+ ## Lab Structure