From 4da9bf3dbba1b7b84ab5c2db90f27826a4955645 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Tue, 11 Jun 2024 11:25:57 -0600 Subject: [PATCH 1/3] I broke the .callout in a prev. push --- contents/ai_for_good/ai_for_good.qmd | 6 +++--- contents/benchmarking/benchmarking.qmd | 6 +++--- contents/data_engineering/data_engineering.qmd | 10 +++++----- contents/dl_primer/dl_primer.qmd | 6 +++--- contents/efficient_ai/efficient_ai.qmd | 2 +- contents/frameworks/frameworks.qmd | 8 ++++---- contents/hw_acceleration/hw_acceleration.qmd | 4 ++-- contents/ml_systems/ml_systems.qmd | 4 ++-- contents/ondevice_learning/ondevice_learning.qmd | 8 ++++---- contents/ops/ops.qmd | 4 ++-- contents/optimizations/optimizations.qmd | 8 ++++---- contents/privacy_security/privacy_security.qmd | 6 +++--- contents/responsible_ai/responsible_ai.qmd | 2 +- contents/robust_ai/robust_ai.qmd | 10 +++++----- contents/sustainable_ai/sustainable_ai.qmd | 6 +++--- contents/training/training.qmd | 12 ++++++------ contents/workflow/workflow.qmd | 2 +- 17 files changed, 52 insertions(+), 52 deletions(-) diff --git a/contents/ai_for_good/ai_for_good.qmd b/contents/ai_for_good/ai_for_good.qmd index 08db6d98..26a3a8c7 100644 --- a/contents/ai_for_good/ai_for_good.qmd +++ b/contents/ai_for_good/ai_for_good.qmd @@ -78,7 +78,7 @@ Widespread TinyML applications can help digitize smallholder farms to increase p With greater investment and integration into rural advisory services, TinyML could transform small-scale agriculture and improve farmers' livelihoods worldwide. The technology effectively brings the benefits of precision agriculture to disconnected regions most in need. -:::{#exr-agri.callout-caution collapse="true"} +:::{#exr-agri .callout-caution collapse="true"} ### Crop Yield Modeling @@ -125,7 +125,7 @@ An on-device algorithm for early and timely life-threatening VA detection will i The champion, GaTech EIC Lab, obtained 0.972 in $F_\beta$ (F1 score with a higher weight to recall), 1.747 ms in latency, and 26.39 kB in memory footprint with a deep neural network. An ICD with an on-device VA detection algorithm was [implanted in a clinical trial](https://youtu.be/vx2gWzAr85A?t=2359). -:::{#exr-hc.callout-caution collapse="true"} +:::{#exr-hc .callout-caution collapse="true"} ### Clinical Data: Unlocking Insights with Named Entity Recognition @@ -253,7 +253,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises - @exr-agri diff --git a/contents/benchmarking/benchmarking.qmd b/contents/benchmarking/benchmarking.qmd index 9f967893..04d67edf 100644 --- a/contents/benchmarking/benchmarking.qmd +++ b/contents/benchmarking/benchmarking.qmd @@ -135,7 +135,7 @@ These types of microbenchmarks include zooming into very specific operations or Example: [DeepBench](https://github.com/baidu-research/DeepBench), introduced by Baidu, is a good example of something that assesses the above. DeepBench assesses the performance of basic operations in deep learning models, providing insights into how different hardware platforms handle neural network training and inference. -:::{#exr-cuda.callout-caution collapse="true"} +:::{#exr-cuda .callout-caution collapse="true"} ### System Benchmarking - Tensor Operations @@ -449,7 +449,7 @@ Metrics: We will measure the following metrics: By measuring these metrics, we can assess the performance of the object detection model on the edge device and identify any potential bottlenecks or areas for optimization to enhance real-time processing capabilities. -:::{#exr-perf.callout-caution collapse="true"} +:::{#exr-perf .callout-caution collapse="true"} ### Inference Benchmarks - MLPerf @@ -821,7 +821,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/data_engineering/data_engineering.qmd b/contents/data_engineering/data_engineering.qmd index 7c771b4b..be79ce62 100644 --- a/contents/data_engineering/data_engineering.qmd +++ b/contents/data_engineering/data_engineering.qmd @@ -123,7 +123,7 @@ In this context, using KWS as an example, we can break each of the steps out as 7. **Iterative Feedback and Refinement:** Once a prototype KWS system is developed, it's crucial to test it in real-world scenarios, gather feedback, and iteratively refine the model. This ensures that the system remains aligned with the defined problem and objectives. This is important because the deployment scenarios change over time as things evolve. -:::{#exr-kws.callout-caution collapse="true"} +:::{#exr-kws .callout-caution collapse="true"} ### Keyword Spotting with TensorFlow Lite Micro @@ -174,7 +174,7 @@ Web scraping can yield inconsistent or inaccurate data. For example, the photo i ![A picture of old traffic lights (1914). Credit: [Vox.](https://www.vox.com/2015/8/5/9097713/when-was-the-first-traffic-light-installed)](images/jpg/1914_traffic.jpeg){#fig-traffic-light} -:::{#exr-ws.callout-caution collapse="true"} +:::{#exr-ws .callout-caution collapse="true"} ### Web Scraping @@ -219,7 +219,7 @@ While synthetic data offers numerous advantages, it is essential to use it judic ![Increasing training data size with synthetic data generation. Credit: [AnyLogic](https://www.anylogic.com/features/artificial-intelligence/synthetic-data/).](images/jpg/synthetic_data.jpg){#fig-synthetic-data} -:::{#exr-sd.callout-caution collapse="true"} +:::{#exr-sd .callout-caution collapse="true"} ### Synthetic Data Let us learn about synthetic data generation using Generative Adversarial Networks (GANs) on tabular data. We'll take a hands-on approach, diving into the workings of the CTGAN model and applying it to the Synthea dataset from the healthcare domain. From data preprocessing to model training and evaluation, we'll go step-by-step, learning how to create synthetic data, assess its quality, and unlock the potential of GANs for data augmentation and real-world applications. @@ -304,7 +304,7 @@ Maintaining the integrity of the data infrastructure is a continuous endeavor. T There is a boom in data processing pipelines, commonly found in ML operations toolchains, which we will discuss in the MLOps chapter. Briefly, these include frameworks like MLOps by Google Cloud. It provides methods for automation and monitoring at all steps of ML system construction, including integration, testing, releasing, deployment, and infrastructure management. Several mechanisms focus on data processing, an integral part of these systems. -:::{#exr-dp.callout-caution collapse="true"} +:::{#exr-dp .callout-caution collapse="true"} ### Data Processing @@ -493,7 +493,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/dl_primer/dl_primer.qmd b/contents/dl_primer/dl_primer.qmd index 4dd685ad..0c4d51c3 100644 --- a/contents/dl_primer/dl_primer.qmd +++ b/contents/dl_primer/dl_primer.qmd @@ -117,7 +117,7 @@ MLPs are basic deep learning architectures comprising three layers: an input lay In embedded AI systems, MLPs can function as compact models for simpler tasks like sensor data analysis or basic pattern recognition, where computational resources are limited. Their ability to learn non-linear relationships with relatively less complexity makes them a suitable choice for embedded systems. -:::{.callout-caution #exr-mlp collapse="false"} +:::{ .callout-caution #exr-mlp collapse="false"} ##### Multilayer Perceptrons (MLPs) @@ -140,7 +140,7 @@ CNNs are mainly used in image and video recognition tasks. This architecture emp In embedded AI, CNNs are crucial for image and video recognition tasks, where real-time processing is often needed. They can be optimized for embedded systems using techniques like quantization and pruning to minimize memory usage and computational demands, enabling efficient object detection and facial recognition functionalities in devices with limited computational resources. -:::{.callout-caution #exr-cnn collapse="false"} +:::{ .callout-caution #exr-cnn collapse="false"} ### Convolutional Neural Networks (CNNs) @@ -282,7 +282,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/efficient_ai/efficient_ai.qmd b/contents/efficient_ai/efficient_ai.qmd index c1a73640..3aa6de2c 100644 --- a/contents/efficient_ai/efficient_ai.qmd +++ b/contents/efficient_ai/efficient_ai.qmd @@ -204,7 +204,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu - [Continuous Evaluation Challenges for TinyML.](https://docs.google.com/presentation/d/1OuhwH5feIwPivEU6pTDyR3QMs7AFstHLiF_LB8T5qYQ/edit?usp=drive_link&resourcekey=0-DZxIuVBUbJawuFh0AO-Pvw) ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises diff --git a/contents/frameworks/frameworks.qmd b/contents/frameworks/frameworks.qmd index 7aa56dab..406cc222 100644 --- a/contents/frameworks/frameworks.qmd +++ b/contents/frameworks/frameworks.qmd @@ -120,7 +120,7 @@ The Parameter Server (PS) architecture is a popular design for distributing the **Computation:** The worker processes, which could be run in parallel, were stateless and purely computational. They processed data and computed gradients without maintaining any state or long-term memory [@li2014communication]. -:::{#exr-tfc.callout-caution collapse="true"} +:::{#exr-tfc .callout-caution collapse="true"} ### TensorFlow Core @@ -129,7 +129,7 @@ Let's comprehensively understand core machine learning algorithms using TensorFl [![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/drive/15Cyy2H7nT40sGR7TBN5wBvgTd57mVKay#scrollTo=IEeIRxlbx0wY) ::: -:::{#exr-tfl.callout-caution collapse="true"} +:::{#exr-tfl .callout-caution collapse="true"} ### TensorFlow Lite @@ -166,7 +166,7 @@ TensorFlow includes libraries to help users develop and deploy more use-case-spe One of TensorFlow's biggest advantages is its integration with Keras, though, as we will cover in the next section, Pytorch recently added a Keras integration. Keras is another ML framework built to be extremely user-friendly and, as a result, has a high level of abstraction. We will cover Keras in more depth later in this chapter. However, when discussing its integration with TensorFlow, it was important to note that it was originally built to be backend-agnostic. This means users could abstract away these complexities, offering a cleaner, more intuitive way to define and train models without worrying about compatibility issues with different backends. TensorFlow users had some complaints about the usability and readability of TensorFlow's API, so as TF gained prominence, it integrated Keras as its high-level API. This integration offered major benefits to TensorFlow users since it introduced more intuitive readability and portability of models while still taking advantage of powerful backend features, Google support, and infrastructure to deploy models on various platforms. -:::{#exr-k.callout-caution collapse="true"} +:::{#exr-k .callout-caution collapse="true"} ### Exploring Keras: Building, Training, and Evaluating Neural Networks @@ -759,7 +759,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu * [TFLite Micro NN Operations.](https://docs.google.com/presentation/d/1ZwLOLvYbKodNmyuKKGb_gD83NskrvNmnFC0rvGugJlY/edit?usp=drive_link) ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/hw_acceleration/hw_acceleration.qmd b/contents/hw_acceleration/hw_acceleration.qmd index 7a0f71fc..091003fb 100644 --- a/contents/hw_acceleration/hw_acceleration.qmd +++ b/contents/hw_acceleration/hw_acceleration.qmd @@ -594,7 +594,7 @@ Programming models provide abstractions to map computations and data onto hetero Key challenges include expressing parallelism, managing memory across devices, and matching algorithms to hardware capabilities. Abstractions must balance portability with allowing hardware customization. Programming models enable developers to harness accelerators without hardware expertise. These details are discussed in the [AI frameworks](../frameworks/frameworks.qmd) section. -:::{#exr-tvm.callout-caution collapse="true"} +:::{#exr-tvm .callout-caution collapse="true"} ### Software for AI Hardware - TVM @@ -1021,7 +1021,7 @@ Here is a curated list of resources to support students and instructors in their * *Coming soon.* ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises * @exr-tvm diff --git a/contents/ml_systems/ml_systems.qmd b/contents/ml_systems/ml_systems.qmd index 478371eb..176dff3d 100644 --- a/contents/ml_systems/ml_systems.qmd +++ b/contents/ml_systems/ml_systems.qmd @@ -162,7 +162,7 @@ TinyML excels in low-power and resource-constrained settings. These environments ![Examples of TinyML device kits. Credit: [Widening Access to Applied Machine Learning with TinyML.](https://arxiv.org/pdf/2106.04008.pdf)](images/jpg/tiny_ml.jpg){#fig-tinyml-example} -:::{#exr-tinyml.callout-caution collapse="true"} +:::{#exr-tinyml .callout-caution collapse="true"} ### TinyML with Arduino @@ -278,7 +278,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/ondevice_learning/ondevice_learning.qmd b/contents/ondevice_learning/ondevice_learning.qmd index e7de5e87..67ae20b9 100644 --- a/contents/ondevice_learning/ondevice_learning.qmd +++ b/contents/ondevice_learning/ondevice_learning.qmd @@ -296,7 +296,7 @@ There are advantages to reusing the features: 1. **Hierarchical Feature Learning:** Deep learning models, particularly Convolutional Neural Networks (CNNs), can learn hierarchical features. Lower layers typically learn generic features like edges and shapes, while higher layers learn more complex and task-specific features. Transfer learning allows us to reuse the generic features learned by a model and finetune the higher layers for our specific task. 2. **Boosting Performance:** Transfer learning has been proven to boost the performance of models on tasks with limited data. The knowledge gained from the source task can provide a valuable starting point and lead to faster convergence and improved accuracy on the target task. -:::{#exr-tlb.callout-caution collapse="true"} +:::{#exr-tlb .callout-caution collapse="true"} ### Transfer Learning @@ -478,7 +478,7 @@ To accomplish this goal, Google employed its algorithm DP-FTRL, which provides a ![Differential Privacy in G Board. Credit: Zheng et al., ([2023](https://arxiv.org/abs/2305.18465)).](images/png/ondevice_gboard_approach.png){#fig-differential-privacy} -:::{#exr-flg.callout-caution collapse="true"} +:::{#exr-flg .callout-caution collapse="true"} ### Federated Learning - Text Generation @@ -488,7 +488,7 @@ Have you ever used those smart keyboards to suggest the next word? With federate ::: -:::{#exr-fli.callout-caution collapse="true"} +:::{#exr-fli .callout-caution collapse="true"} ### Federated Learning - Image Classification @@ -734,7 +734,7 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises diff --git a/contents/ops/ops.qmd b/contents/ops/ops.qmd index 587de587..cd8cc2f2 100644 --- a/contents/ops/ops.qmd +++ b/contents/ops/ops.qmd @@ -728,7 +728,7 @@ Deploying low-powered devices also presents inherent challenges. Optimized model While impressive results are achievable, users shouldn't view Edge Impulse as a "Push Button ML" solution. Careful project scoping, data collection, model evaluation, and testing are still essential. As with any development tool, reasonable expectations and diligence in application are advised. However, Edge Impulse can accelerate embedded ML prototyping and deployment for developers willing to invest the requisite data science and engineering effort. -:::{#exr-ei.callout-caution collapse="true"} +:::{#exr-ei .callout-caution collapse="true"} ### Edge Impulse @@ -921,7 +921,7 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/optimizations/optimizations.qmd b/contents/optimizations/optimizations.qmd index 1575b4f2..6b0868e6 100644 --- a/contents/optimizations/optimizations.qmd +++ b/contents/optimizations/optimizations.qmd @@ -227,7 +227,7 @@ Last but not least, adherence to legal and ethical guidelines is paramount, espe ![Sparse weight matrix.](images/jpg/modeloptimization_sprase_matrix.jpeg){#fig-sparse-matrix} -:::{#exr-p.callout-caution collapse="true"} +:::{#exr-p .callout-caution collapse="true"} ### Pruning @@ -299,7 +299,7 @@ The work of Tamara G. Kolda and Brett W. Bader, ["Tensor Decompositions and Appl ![Tensor decomposition. Credit: @xinyu.](images/png/modeloptimization_tensor_decomposition.png){#fig-tensor-decomposition} -:::{#exr-mc.callout-caution collapse="true"} +:::{#exr-mc .callout-caution collapse="true"} ### Scalable Model Compression with TensorFlow @@ -343,7 +343,7 @@ Similarly, MorphNet is a neural network optimization framework designed to autom TinyNAS and MorphNet represent a few of the many significant advancements in the field of systematic neural network optimization, allowing architectures to be systematically chosen and generated to fit perfectly within problem constraints. -:::{#exr-md.callout-caution collapse="true"} +:::{#exr-md .callout-caution collapse="true"} ### Edge-Aware Model Design @@ -977,7 +977,7 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/privacy_security/privacy_security.qmd b/contents/privacy_security/privacy_security.qmd index 365f372e..a4025ea9 100644 --- a/contents/privacy_security/privacy_security.qmd +++ b/contents/privacy_security/privacy_security.qmd @@ -831,7 +831,7 @@ The system uses hardware-based secure random number generation to sample from th Multiple third-party audits have verified that Apple's system provides rigorous differential privacy protections in line with their stated policies. Of course, assumptions around composition over time and potential re-identification risks still apply. Apple's deployment shows how differential Privacy can be realized in large real-world products when backed by sufficient engineering resources. -:::{#exr-dptf.callout-caution collapse="true"} +:::{#exr-dptf .callout-caution collapse="true"} ### Differential Privacy - TensorFlow Privacy @@ -965,7 +965,7 @@ For many real-time and embedded applications, fully homomorphic encryption remai **Hybrid Designs:** Rather than encrypting entire workflows, selective application of homomorphic encryption to critical subcomponents can achieve protection while minimizing overheads. -:::{#exr-he.callout-caution collapse="true"} +:::{#exr-he .callout-caution collapse="true"} ### Homomorphic Encryption @@ -1110,7 +1110,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/responsible_ai/responsible_ai.qmd b/contents/responsible_ai/responsible_ai.qmd index 69e74334..5ae8cdf2 100644 --- a/contents/responsible_ai/responsible_ai.qmd +++ b/contents/responsible_ai/responsible_ai.qmd @@ -472,7 +472,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/robust_ai/robust_ai.qmd b/contents/robust_ai/robust_ai.qmd index edd1869d..68325495 100644 --- a/contents/robust_ai/robust_ai.qmd +++ b/contents/robust_ai/robust_ai.qmd @@ -321,7 +321,7 @@ Anomaly detection algorithms can be applied to the model's predictions or interm ![n-bits representation of the Reed-Solomon codes (Credit: [GeeksforGeeks](https://www.geeksforgeeks.org/what-is-reed-solomon-code/))](./images/png/Reed-Solomon.png){#fig-Reed-Solomon} -:::{#exr-ad.callout-caution collapse="true"} +:::{#exr-ad .callout-caution collapse="true"} ### Anomaly Detection @@ -445,7 +445,7 @@ The presence of adversarial vulnerabilities also complicates the deployment and The impact of adversarial attacks on ML systems is significant and multifaceted. These attacks expose ML models' vulnerabilities, from degrading model performance and raising security and safety concerns to challenging model trustworthiness and interpretability. Developers and researchers must prioritize the development of robust defenses and countermeasures to mitigate the risks posed by adversarial attacks. By addressing these challenges, we can build more secure, reliable, and trustworthy ML systems that can withstand the ever-evolving landscape of adversarial threats. -:::{#exr-aa.callout-caution collapse="true"} +:::{#exr-aa .callout-caution collapse="true"} ### Adversarial Attacks @@ -573,7 +573,7 @@ On the other hand, this tool can be used maliciously and can affect legitimate a ![Data poisoning (Credit: @shan2023prompt)](images/png/image14.png){#fig-poisoning} -:::{#exr-pa.callout-caution collapse="true"} +:::{#exr-pa .callout-caution collapse="true"} ### Poisoning Attacks @@ -846,7 +846,7 @@ Detecting and mitigating software faults in machine learning frameworks is essen Adopting a proactive and systematic approach to fault detection and mitigation can significantly improve ML systems' robustness, reliability, and maintainability. By investing in comprehensive testing, monitoring, and fault-tolerant design practices, organizations can minimize the impact of software faults and ensure their ML systems' smooth operation in production environments. -:::{#exr-ft.callout-caution collapse="true"} +:::{#exr-ft .callout-caution collapse="true"} ### Fault Tolerance @@ -1037,7 +1037,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu * *Coming soon.* ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/sustainable_ai/sustainable_ai.qmd b/contents/sustainable_ai/sustainable_ai.qmd index 9ff412c3..2142c34b 100644 --- a/contents/sustainable_ai/sustainable_ai.qmd +++ b/contents/sustainable_ai/sustainable_ai.qmd @@ -202,7 +202,7 @@ Additionally, the manufacturing and disposal of AI hardware add to the carbon fo As is clear from the above, a proper Life Cycle Analysis is necessary to portray all relevant aspects of the emissions caused by AI. Another method is carbon accounting, which quantifies the amount of carbon dioxide emissions directly and indirectly associated with AI operations. This measurement typically uses $\textrm{CO}_2$ equivalents, allowing for a standardized way of reporting and assessing emissions. -:::{#exr-cf.callout-caution collapse="true"} +:::{#exr-cf .callout-caution collapse="true"} ### AI's Carbon Footprint @@ -316,7 +316,7 @@ AI computes the majority of emissions in the lifecycle due to continuous high-po The disposal stage impacts include air and water pollution from toxic materials in devices, challenges associated with complex electronics recycling, and contamination when improperly handled. Harmful compounds from burned e-waste are released into the atmosphere. At the same time, landfill leakage of lead, mercury, and other materials poses risks of soil and groundwater contamination if not properly controlled. Implementing effective electronics recycling is crucial. -:::{#exr-mle.callout-caution collapse="true"} +:::{#exr-mle .callout-caution collapse="true"} ### Tracking ML Emissions @@ -682,7 +682,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/training/training.qmd b/contents/training/training.qmd index 82917754..cc4b75d8 100644 --- a/contents/training/training.qmd +++ b/contents/training/training.qmd @@ -265,7 +265,7 @@ The entire backpropagation process can be complex, especially for very deep netw As seen above, intermediate activations $A_i$ are reused in backpropagation. To improve performance, these activations are cached from the forward pass to avoid being recomputed. However, activations must be kept in memory between the forward and backward passes, leading to higher memory usage. If the network and batch size are large, this may lead to memory issues. Similarly, the derivatives with respect to each layer's outputs are cached to avoid recomputation. ::: -:::{#exr-nn.callout-caution collapse="true"} +:::{#exr-nn .callout-caution collapse="true"} ### Neural Networks with Backpropagation and Gradient Descent @@ -590,7 +590,7 @@ With the defined search space and optimization goals, the EON Tuner leverages Ba The EON Tuner frees machine learning engineers from the demandingly iterative process of hand-tuning models by automatically tuning models for embedded deployment. The tool integrates seamlessly into the Edge Impulse workflow, taking models from concept to efficiently optimized implementations on microcontrollers. The expertise encapsulated in EON Tuner regarding ML model optimization for microcontrollers ensures beginner and experienced developers alike can rapidly iterate to models fitting their project needs. -:::{#exr-hpt.callout-caution collapse="true"} +:::{#exr-hpt .callout-caution collapse="true"} ### Hyperparameter Tuning @@ -711,7 +711,7 @@ Therefore, formal early stopping methods require monitoring a metric like valida Simple, early-stopping rules stop immediately at the first post-peak degradation. More robust methods introduce a patience parameter—the number of degrading epochs permitted before stopping. This avoids prematurely halting training due to transient fluctuations. Typical patience windows range from 50 to 200 validation batches. Wider windows incur the risk of overfitting. Formal tuning strategies can determine optimal patience. -:::{#exr-r.callout-caution collapse="true"} +:::{#exr-r .callout-caution collapse="true"} ### Regularization @@ -761,7 +761,7 @@ He overcomes this by sampling weights from a distribution with a variance set ba This specialized Initialization allows ReLU networks to converge efficiently right from the start. The choice between Xavier and He must match the intended network activation function. -:::{#exr-wi.callout-caution collapse="true"} +:::{#exr-wi .callout-caution collapse="true"} ### Weight Initialization @@ -846,7 +846,7 @@ Here are the summarizing pros and cons of these various standard activation func | ReLU | Computationally efficient
Introduces sparsity
Avoids vanishing gradients | "Dying ReLU" units
Not bounded | | Softmax | Used for the last layer to normalize vector outputs to be a probability distribution; typically used for classification tasks | - | -:::{#exr-af.callout-caution collapse="true"} +:::{#exr-af .callout-caution collapse="true"} ### Activation Functions @@ -1070,7 +1070,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/workflow/workflow.qmd b/contents/workflow/workflow.qmd index 466477f3..24d89f1b 100644 --- a/contents/workflow/workflow.qmd +++ b/contents/workflow/workflow.qmd @@ -121,7 +121,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{.callout-caution collapse="false"} +:::{ .callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. From 2b942de9ba3e1c5b77c6f2784c988e8fbed41f45 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Tue, 11 Jun 2024 11:33:48 -0600 Subject: [PATCH 2/3] Fix spacing issues Fix :::{ .callout- to :::{.callout- --- contents/ai_for_good/ai_for_good.qmd | 2 +- contents/benchmarking/benchmarking.qmd | 2 +- contents/data_engineering/data_engineering.qmd | 2 +- contents/dl_primer/dl_primer.qmd | 6 +++--- contents/efficient_ai/efficient_ai.qmd | 2 +- contents/frameworks/frameworks.qmd | 2 +- contents/hw_acceleration/hw_acceleration.qmd | 2 +- contents/ml_systems/ml_systems.qmd | 2 +- contents/ondevice_learning/ondevice_learning.qmd | 2 +- contents/ops/ops.qmd | 2 +- contents/optimizations/optimizations.qmd | 2 +- contents/privacy_security/privacy_security.qmd | 2 +- contents/responsible_ai/responsible_ai.qmd | 2 +- contents/robust_ai/robust_ai.qmd | 2 +- contents/sustainable_ai/sustainable_ai.qmd | 2 +- contents/training/training.qmd | 2 +- contents/workflow/workflow.qmd | 2 +- 17 files changed, 19 insertions(+), 19 deletions(-) diff --git a/contents/ai_for_good/ai_for_good.qmd b/contents/ai_for_good/ai_for_good.qmd index 26a3a8c7..a5cdb385 100644 --- a/contents/ai_for_good/ai_for_good.qmd +++ b/contents/ai_for_good/ai_for_good.qmd @@ -253,7 +253,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises - @exr-agri diff --git a/contents/benchmarking/benchmarking.qmd b/contents/benchmarking/benchmarking.qmd index 04d67edf..8eb1b99d 100644 --- a/contents/benchmarking/benchmarking.qmd +++ b/contents/benchmarking/benchmarking.qmd @@ -821,7 +821,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/data_engineering/data_engineering.qmd b/contents/data_engineering/data_engineering.qmd index be79ce62..e70a90ab 100644 --- a/contents/data_engineering/data_engineering.qmd +++ b/contents/data_engineering/data_engineering.qmd @@ -493,7 +493,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/dl_primer/dl_primer.qmd b/contents/dl_primer/dl_primer.qmd index 0c4d51c3..4dd685ad 100644 --- a/contents/dl_primer/dl_primer.qmd +++ b/contents/dl_primer/dl_primer.qmd @@ -117,7 +117,7 @@ MLPs are basic deep learning architectures comprising three layers: an input lay In embedded AI systems, MLPs can function as compact models for simpler tasks like sensor data analysis or basic pattern recognition, where computational resources are limited. Their ability to learn non-linear relationships with relatively less complexity makes them a suitable choice for embedded systems. -:::{ .callout-caution #exr-mlp collapse="false"} +:::{.callout-caution #exr-mlp collapse="false"} ##### Multilayer Perceptrons (MLPs) @@ -140,7 +140,7 @@ CNNs are mainly used in image and video recognition tasks. This architecture emp In embedded AI, CNNs are crucial for image and video recognition tasks, where real-time processing is often needed. They can be optimized for embedded systems using techniques like quantization and pruning to minimize memory usage and computational demands, enabling efficient object detection and facial recognition functionalities in devices with limited computational resources. -:::{ .callout-caution #exr-cnn collapse="false"} +:::{.callout-caution #exr-cnn collapse="false"} ### Convolutional Neural Networks (CNNs) @@ -282,7 +282,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/efficient_ai/efficient_ai.qmd b/contents/efficient_ai/efficient_ai.qmd index 3aa6de2c..c1a73640 100644 --- a/contents/efficient_ai/efficient_ai.qmd +++ b/contents/efficient_ai/efficient_ai.qmd @@ -204,7 +204,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu - [Continuous Evaluation Challenges for TinyML.](https://docs.google.com/presentation/d/1OuhwH5feIwPivEU6pTDyR3QMs7AFstHLiF_LB8T5qYQ/edit?usp=drive_link&resourcekey=0-DZxIuVBUbJawuFh0AO-Pvw) ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises diff --git a/contents/frameworks/frameworks.qmd b/contents/frameworks/frameworks.qmd index 406cc222..6ed6d934 100644 --- a/contents/frameworks/frameworks.qmd +++ b/contents/frameworks/frameworks.qmd @@ -759,7 +759,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu * [TFLite Micro NN Operations.](https://docs.google.com/presentation/d/1ZwLOLvYbKodNmyuKKGb_gD83NskrvNmnFC0rvGugJlY/edit?usp=drive_link) ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/hw_acceleration/hw_acceleration.qmd b/contents/hw_acceleration/hw_acceleration.qmd index 091003fb..11344a28 100644 --- a/contents/hw_acceleration/hw_acceleration.qmd +++ b/contents/hw_acceleration/hw_acceleration.qmd @@ -1021,7 +1021,7 @@ Here is a curated list of resources to support students and instructors in their * *Coming soon.* ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises * @exr-tvm diff --git a/contents/ml_systems/ml_systems.qmd b/contents/ml_systems/ml_systems.qmd index 176dff3d..cf5ad5cf 100644 --- a/contents/ml_systems/ml_systems.qmd +++ b/contents/ml_systems/ml_systems.qmd @@ -278,7 +278,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/ondevice_learning/ondevice_learning.qmd b/contents/ondevice_learning/ondevice_learning.qmd index 67ae20b9..64bcd631 100644 --- a/contents/ondevice_learning/ondevice_learning.qmd +++ b/contents/ondevice_learning/ondevice_learning.qmd @@ -734,7 +734,7 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises diff --git a/contents/ops/ops.qmd b/contents/ops/ops.qmd index cd8cc2f2..623f43fe 100644 --- a/contents/ops/ops.qmd +++ b/contents/ops/ops.qmd @@ -921,7 +921,7 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/optimizations/optimizations.qmd b/contents/optimizations/optimizations.qmd index 6b0868e6..6320400e 100644 --- a/contents/optimizations/optimizations.qmd +++ b/contents/optimizations/optimizations.qmd @@ -977,7 +977,7 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/privacy_security/privacy_security.qmd b/contents/privacy_security/privacy_security.qmd index a4025ea9..eeea1532 100644 --- a/contents/privacy_security/privacy_security.qmd +++ b/contents/privacy_security/privacy_security.qmd @@ -1110,7 +1110,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/responsible_ai/responsible_ai.qmd b/contents/responsible_ai/responsible_ai.qmd index 5ae8cdf2..69e74334 100644 --- a/contents/responsible_ai/responsible_ai.qmd +++ b/contents/responsible_ai/responsible_ai.qmd @@ -472,7 +472,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/robust_ai/robust_ai.qmd b/contents/robust_ai/robust_ai.qmd index 68325495..2eb5d771 100644 --- a/contents/robust_ai/robust_ai.qmd +++ b/contents/robust_ai/robust_ai.qmd @@ -1037,7 +1037,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu * *Coming soon.* ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/sustainable_ai/sustainable_ai.qmd b/contents/sustainable_ai/sustainable_ai.qmd index 2142c34b..4674376d 100644 --- a/contents/sustainable_ai/sustainable_ai.qmd +++ b/contents/sustainable_ai/sustainable_ai.qmd @@ -682,7 +682,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/training/training.qmd b/contents/training/training.qmd index cc4b75d8..05b5b558 100644 --- a/contents/training/training.qmd +++ b/contents/training/training.qmd @@ -1070,7 +1070,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/workflow/workflow.qmd b/contents/workflow/workflow.qmd index 24d89f1b..466477f3 100644 --- a/contents/workflow/workflow.qmd +++ b/contents/workflow/workflow.qmd @@ -121,7 +121,7 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. From 0b28e14f2f306cb8446c6f4c2caf860039fdf163 Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Tue, 11 Jun 2024 14:55:02 -0600 Subject: [PATCH 3/3] Adding video callouts + end of section resources --- _quarto.yml | 5 ++ contents/ai_for_good/ai_for_good.qmd | 36 ++++++-- contents/benchmarking/benchmarking.qmd | 10 ++- .../data_engineering/data_engineering.qmd | 10 ++- contents/dl_primer/dl_primer.qmd | 41 +++++++-- contents/efficient_ai/efficient_ai.qmd | 8 +- contents/frameworks/frameworks.qmd | 10 ++- contents/hw_acceleration/hw_acceleration.qmd | 39 ++++++-- contents/ml_systems/ml_systems.qmd | 11 ++- .../ondevice_learning/ondevice_learning.qmd | 29 +++++- contents/ops/ops.qmd | 47 ++++++++-- contents/optimizations/optimizations.qmd | 19 +++- .../privacy_security/privacy_security.qmd | 43 ++++++--- contents/responsible_ai/responsible_ai.qmd | 19 +++- contents/robust_ai/robust_ai.qmd | 10 ++- contents/sustainable_ai/sustainable_ai.qmd | 10 ++- contents/training/training.qmd | 89 +++++++++++++++++-- contents/workflow/workflow.qmd | 10 ++- 18 files changed, 377 insertions(+), 69 deletions(-) diff --git a/_quarto.yml b/_quarto.yml index 3bab6130..94b43469 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -320,5 +320,10 @@ crossref: key: exr latex-env: exr + - kind: float + reference-prefix: Video + key: vid + latex-env: vid + editor: render-on-save: true diff --git a/contents/ai_for_good/ai_for_good.qmd b/contents/ai_for_good/ai_for_good.qmd index 26a3a8c7..a1f1eee4 100644 --- a/contents/ai_for_good/ai_for_good.qmd +++ b/contents/ai_for_good/ai_for_good.qmd @@ -5,7 +5,7 @@ bibliography: ai_for_good.bib # AI for Good {#sec-ai_for_good} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-ai-for-good-resource), [Exercises](#sec-ai-for-good-resource), [Labs](#sec-ai-for-good-resource) +Resources: [Slides](#sec-ai-for-good-resource), [Videos](#sec-ai-for-good-resource), [Exercises](#sec-ai-for-good-resource), [Labs](#sec-ai-for-good-resource) ::: ![_DALL·E 3 Prompt: Illustration of planet Earth wrapped in shimmering neural networks, with diverse humans and AI robots working together on various projects like planting trees, cleaning the oceans, and developing sustainable energy solutions. The positive and hopeful atmosphere represents a united effort to create a better future._](images/png/cover_ai_good.png) @@ -162,18 +162,24 @@ Researchers from Moulay Ismail University of Meknes in Morocco [@bamoumen2022tin In disaster response, speed and safety are paramount. But rubble and wreckage create hazardous, confined environments that impede human search efforts. TinyML enables nimble drones to assist rescue teams in these dangerous scenarios. -When buildings collapse after earthquakes, small drones can prove invaluable. Equipped with TinyML navigation algorithms, micro-sized drones like the [CrazyFlie](https://www.bitcraze.io/) can traverse cramped voids and map pathways beyond human reach [@duisterhof2019learning]. Obstacle avoidance allows the drones to weave through unstable debris. This autonomous mobility lets them rapidly sweep areas humans cannot access. +When buildings collapse after earthquakes, small drones can prove invaluable. Equipped with TinyML navigation algorithms, micro-sized drones like the [CrazyFlie](https://www.bitcraze.io/) can traverse cramped voids and map pathways beyond human reach [@duisterhof2019learning]. Obstacle avoidance allows the drones to weave through unstable debris. This autonomous mobility lets them rapidly sweep areas humans cannot access. @vid-l2seek presents the [@duisterhof2019learning] paper on deep reinforcement learning using drones for source-seeking. -The video below presents the [@duisterhof2019learning] paper on deep reinforcement learning using drones for source-seeking. +:::{#vid-l2seek .callout-important} + +# Learning to Seek {{< video https://www.youtube.com/watch?v=wmVKbX7MOnU >}} -Crucially, onboard sensors and TinyML processors analyze real-time data to identify signs of survivors. Thermal cameras detect body heat, microphones pick up calls for help, and gas sensors warn of leaks [@duisterhof2021sniffy]. Processing data locally using TinyML allows for quick interpretation to guide rescue efforts. As conditions evolve, the drones can adapt by adjusting their search patterns and priorities. +::: + +Crucially, onboard sensors and TinyML processors analyze real-time data to identify signs of survivors. Thermal cameras detect body heat, microphones pick up calls for help, and gas sensors warn of leaks [@duisterhof2021sniffy]. Processing data locally using TinyML allows for quick interpretation to guide rescue efforts. As conditions evolve, the drones can adapt by adjusting their search patterns and priorities. @vid-sniffybug is an overview of autonomous drones for gas leak detection. -The following video is an overview of autonomous drones for gas leak detection. +:::{#vid-sniffybug .callout-important} {{< video https://www.youtube.com/watch?v=hj_SBSpK5qg >}} +::: + Additionally, coordinated swarms of drones unlock new capabilities. By collaborating and sharing insights, drone teams comprehensively view the situation. Blanketing disaster sites allows TinyML algorithms to fuse and analyze data from multiple vantage points, amplifying situational awareness beyond individual drones [@duisterhof2021sniffy]. Most importantly, initial drone reconnaissance enhances safety for human responders. Keeping rescue teams at a safe distance until drone surveys assess hazards saves lives. Once secured, drones can guide precise personnel placement. @@ -198,12 +204,14 @@ Technology has immense potential to break down barriers faced by people with dis With machine learning algorithms running locally on microcontrollers, compact accessibility tools can operate in real time without reliance on connectivity. The [National Institute on Deafness and Other Communication Disorders (NIDCD)](https://www.nidcd.nih.gov/health/statistics/quick-statistics-hearing) states that 20% of the world's population has some form of hearing loss. Hearing aids leveraging TinyML could recognize multiple speakers and amplify the voice of a chosen target in crowded rooms. This allows people with hearing impairments to focus on specific conversations. -Similarly, mobility devices could use on-device vision processing to identify obstacles and terrain characteristics. This enables enhanced navigation and safety for the visually impaired. Companies like [Envision](https://www.letsenvision.com/) are developing smart glasses, converting visual information into speech, with embedded TinyML to guide blind people by detecting objects, text, and traffic signals. +Similarly, mobility devices could use on-device vision processing to identify obstacles and terrain characteristics. This enables enhanced navigation and safety for the visually impaired. Companies like [Envision](https://www.letsenvision.com/) are developing smart glasses, converting visual information into speech, with embedded TinyML to guide blind people by detecting objects, text, and traffic signals. @vid-envision below shows the different real-life use cases of the Envision visual aid glasses. -The video below shows the different real-life use cases of the Envision visual aid glasses. +:::{#vid-envision .callout-important} {{< video https://www.youtube.com/watch?v=oGWinIKDOdc >}} +::: + TinyML could even power responsive prosthetic limbs. By analyzing nerve signals and sensory data like muscle tension, prosthetics and exoskeletons with embedded ML can move and adjust grip dynamically, making control more natural and intuitive. Companies are creating affordable, everyday bionic hands using TinyML. For those with speech difficulties, voice-enabled devices with TinyML can generate personalized vocal outputs from non-verbal inputs. Pairs by Anthropic translates gestures into natural speech tailored for individual users. By enabling more customizable assistive tech, TinyML makes services more accessible and tailored to individual needs. And through translation and interpretation applications, TinyML can break down communication barriers. Apps like Microsoft Translator offer real-time translation powered by TinyML algorithms. @@ -253,12 +261,24 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-l2seek + +- @vid-sniffybug + +- @vid-envision + +::: + +:::{.callout-caution collapse="false"} #### Exercises - @exr-agri - @exr-hc + ::: :::{.callout-warning collapse="false"} diff --git a/contents/benchmarking/benchmarking.qmd b/contents/benchmarking/benchmarking.qmd index 04d67edf..0665880f 100644 --- a/contents/benchmarking/benchmarking.qmd +++ b/contents/benchmarking/benchmarking.qmd @@ -5,7 +5,7 @@ bibliography: benchmarking.bib # Benchmarking AI {#sec-benchmarking_ai} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-benchmarking-ai-resource), [Exercises](#sec-benchmarking-ai-resource), [Labs](#sec-benchmarking-ai-resource) +Resources: [Slides](#sec-benchmarking-ai-resource), [Videos](#sec-benchmarking-ai-resource), [Exercises](#sec-benchmarking-ai-resource), [Labs](#sec-benchmarking-ai-resource) ::: ![_DALL·E 3 Prompt: Photo of a podium set against a tech-themed backdrop. On each tier of the podium, there are AI chips with intricate designs. The top chip has a gold medal hanging from it, the second one has a silver medal, and the third has a bronze medal. Banners with 'AI Olympics' are displayed prominently in the background._](images/png/cover_ai_benchmarking.png) @@ -821,7 +821,13 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/data_engineering/data_engineering.qmd b/contents/data_engineering/data_engineering.qmd index be79ce62..b57d07b1 100644 --- a/contents/data_engineering/data_engineering.qmd +++ b/contents/data_engineering/data_engineering.qmd @@ -5,7 +5,7 @@ bibliography: data_engineering.bib # Data Engineering {#sec-data_engineering} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-data-engineering-resource), [Exercises](#sec-data-engineering-resource), [Labs](#sec-data-engineering-resource) +Resources: [Slides](#sec-data-engineering-resource), [Videos](#sec-data-engineering-resource), [Exercises](#sec-data-engineering-resource), [Labs](#sec-data-engineering-resource) ::: ![_DALL·E 3 Prompt: Create a rectangular illustration visualizing the concept of data engineering. Include elements such as raw data sources, data processing pipelines, storage systems, and refined datasets. Show how raw data is transformed through cleaning, processing, and storage to become valuable information that can be analyzed and used for decision-making._](images/png/cover_data_engineering.png) @@ -493,7 +493,13 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/dl_primer/dl_primer.qmd b/contents/dl_primer/dl_primer.qmd index 0c4d51c3..004933f9 100644 --- a/contents/dl_primer/dl_primer.qmd +++ b/contents/dl_primer/dl_primer.qmd @@ -5,7 +5,7 @@ bibliography: dl_primer.bib # DL Primer {#sec-dl_primer} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-deep-learning-primer-resource), [Exercises](#sec-deep-learning-primer-resource), [Labs](#sec-deep-learning-primer-resource) +Resources: [Slides](#sec-deep-learning-primer-resource), [Videos](#sec-deep-learning-primer-resource), [Exercises](#sec-deep-learning-primer-resource), [Labs](#sec-deep-learning-primer-resource) ::: ![_DALL·E 3 Prompt: Photo of a classic classroom with a large blackboard dominating one wall. Chalk drawings showcase a detailed deep neural network with several hidden layers, and each node and connection is precisely labeled with white chalk. The rustic wooden floor and brick walls provide a contrast to the modern concepts. Surrounding the room, posters mounted on frames emphasize deep learning themes: convolutional networks, transformers, neurons, activation functions, and more._](images/png/cover_dl_primer.png) @@ -91,22 +91,40 @@ Multilayer perceptrons (MLPs) are an evolution of the single-layer perceptron mo The forward pass is the initial phase where data moves through the network from the input to the output layer. During this phase, each layer performs specific computations on the input data, using weights and biases before passing the resulting values to subsequent layers. The final output of this phase is used to compute the loss, indicating the difference between the predicted output and actual target values. -The video below explains how neural networks work using handwritten digit recognition as an example application. It also touches on the math underlying neural nets. +@vid-nn below explains how neural networks work using handwritten digit recognition as an example application. It also touches on the math underlying neural nets. + +:::{#vid-nn .callout-important} + +# Neural Networks {{< video https://www.youtube.com/embed/aircAruvnKk?si=qfkBf8MJjC2WSyw3 >}} +::: + #### Backward Pass (Backpropagation) Backpropagation is a key algorithm in training deep neural networks. This phase involves calculating the gradient of the loss function concerning each weight using the chain rule, effectively moving backward through the network. The gradients calculated in this step guide the adjustment of weights to minimize the loss function, thereby enhancing the network's performance with each iteration of training. Grasping these foundational concepts paves the way to understanding more intricate deep learning architectures and techniques, fostering the development of more sophisticated and productive applications, especially within embedded AI systems. -The following two videos build upon the previous one. They cover gradient descent and backpropagation in neural networks. +@vid-gd and @vid-bp build upon @vid-nn. They cover gradient descent and backpropagation in neural networks. + +:::{#vid-gd .callout-important} + +# Gradient descent {{< video https://www.youtube.com/watch?v=IHZwWFHWa-w&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&index=2 >}} +::: + +:::{#vid-bp .callout-important} + +# Backpropagation + {{< video https://www.youtube.com/watch?v=Ilg3gGewQ5U&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&index=3 >}} +::: + ### Model Architectures Deep learning architectures refer to the various structured approaches that dictate how neurons and layers are organized and interact in neural networks. These architectures have evolved to tackle different problems and data types effectively. This section overviews some well-known deep learning architectures and their characteristics. @@ -117,7 +135,7 @@ MLPs are basic deep learning architectures comprising three layers: an input lay In embedded AI systems, MLPs can function as compact models for simpler tasks like sensor data analysis or basic pattern recognition, where computational resources are limited. Their ability to learn non-linear relationships with relatively less complexity makes them a suitable choice for embedded systems. -:::{ .callout-caution #exr-mlp collapse="false"} +:::{.callout-caution #exr-mlp collapse="false"} ##### Multilayer Perceptrons (MLPs) @@ -140,7 +158,7 @@ CNNs are mainly used in image and video recognition tasks. This architecture emp In embedded AI, CNNs are crucial for image and video recognition tasks, where real-time processing is often needed. They can be optimized for embedded systems using techniques like quantization and pruning to minimize memory usage and computational demands, enabling efficient object detection and facial recognition functionalities in devices with limited computational resources. -:::{ .callout-caution #exr-cnn collapse="false"} +:::{.callout-caution #exr-cnn collapse="false"} ### Convolutional Neural Networks (CNNs) @@ -282,7 +300,18 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-nn + +- @vid-gd + +- @vid-bp + +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/efficient_ai/efficient_ai.qmd b/contents/efficient_ai/efficient_ai.qmd index 3aa6de2c..d9b8284a 100644 --- a/contents/efficient_ai/efficient_ai.qmd +++ b/contents/efficient_ai/efficient_ai.qmd @@ -204,7 +204,13 @@ These slides are a valuable tool for instructors to deliver lectures and for stu - [Continuous Evaluation Challenges for TinyML.](https://docs.google.com/presentation/d/1OuhwH5feIwPivEU6pTDyR3QMs7AFstHLiF_LB8T5qYQ/edit?usp=drive_link&resourcekey=0-DZxIuVBUbJawuFh0AO-Pvw) ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + +:::{.callout-caution collapse="false"} #### Exercises diff --git a/contents/frameworks/frameworks.qmd b/contents/frameworks/frameworks.qmd index 406cc222..b29358d9 100644 --- a/contents/frameworks/frameworks.qmd +++ b/contents/frameworks/frameworks.qmd @@ -5,7 +5,7 @@ bibliography: frameworks.bib # AI Frameworks {#sec-ai_frameworks} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-ai-frameworks-resource), [Exercises](#sec-ai-frameworks-resource), [Labs](#sec-ai-frameworks-resource) +Resources: [Slides](#sec-ai-frameworks-resource), [Videos](#sec-ai-frameworks-resource), [Exercises](#sec-ai-frameworks-resource), [Labs](#sec-ai-frameworks-resource) ::: ![_DALL·E 3 Prompt: Illustration in a rectangular format, designed for a professional textbook, where the content spans the entire width. The vibrant chart represents training and inference frameworks for ML. Icons for TensorFlow, Keras, PyTorch, ONNX, and TensorRT are spread out, filling the entire horizontal space, and aligned vertically. Each icon is accompanied by brief annotations detailing their features. The lively colors like blues, greens, and oranges highlight the icons and sections against a soft gradient background. The distinction between training and inference frameworks is accentuated through color-coded sections, with clean lines and modern typography maintaining clarity and focus._](images/png/cover_ml_frameworks.png) @@ -759,7 +759,13 @@ These slides are a valuable tool for instructors to deliver lectures and for stu * [TFLite Micro NN Operations.](https://docs.google.com/presentation/d/1ZwLOLvYbKodNmyuKKGb_gD83NskrvNmnFC0rvGugJlY/edit?usp=drive_link) ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/hw_acceleration/hw_acceleration.qmd b/contents/hw_acceleration/hw_acceleration.qmd index 091003fb..e85db4a5 100644 --- a/contents/hw_acceleration/hw_acceleration.qmd +++ b/contents/hw_acceleration/hw_acceleration.qmd @@ -5,7 +5,7 @@ bibliography: hw_acceleration.bib # AI Acceleration {#sec-ai_acceleration} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-ai-acceleration-resource), [Exercises](#sec-ai-acceleration-resource), [Labs](#sec-ai-acceleration-resource) +Resources: [Slides](#sec-ai-acceleration-resource), [Videos](#sec-ai-acceleration-resource), [Exercises](#sec-ai-acceleration-resource), [Labs](#sec-ai-acceleration-resource) ::: ![_DALL·E 3 Prompt: Create an intricate and colorful representation of a System on Chip (SoC) design in a rectangular format. Showcase a variety of specialized machine learning accelerators and chiplets, all integrated into the processor. Provide a detailed view inside the chip, highlighting the rapid movement of electrons. Each accelerator and chiplet should be designed to interact with neural network neurons, layers, and activations, emphasizing their processing speed. Depict the neural networks as a network of interconnected nodes, with vibrant data streams flowing between the accelerator pieces, showcasing the enhanced computation speed._](images/png/cover_ai_hardware.png) @@ -766,10 +766,16 @@ By designing the wafer as one integrated logic unit, data transfer between eleme However, the ultra-large-scale also poses difficulties for manufacturability and yield with wafer-scale designs. Defects in any region of the wafer can make (certain parts of) the chip unusable. Specialized lithography techniques are required to produce such large dies. So, wafer-scale integration pursues the maximum performance gains from integration but requires overcoming substantial fabrication challenges. -The following video will provide additional context. +@vid-wfscale provides additional context about wafer-scale AI chips. + +:::{#vid-wfscale .callout-important} + +# Wafer-scale AI Chips {{< video https://www.youtube.com/watch?v=Fcob512SJz0 >}} +::: + #### Chiplets for AI Chiplet design refers to a semiconductor architecture in which a single integrated circuit (IC) is constructed from multiple smaller, individual components known as chiplets. Each chiplet is a self-contained functional block, typically specialized for a specific task or functionality. These chiplets are then interconnected on a larger substrate or package to create a cohesive system. @fig-chiplet illustrates this concept. For AI hardware, chiplets enable the mixing of different types of chips optimized for tasks like matrix multiplication, data movement, analog I/O, and specialized memories. This heterogeneous integration differs greatly from wafer-scale integration, where all logic is manufactured as one monolithic chip. Companies like Intel and AMD have adopted chiplet designs for their CPUs. @@ -803,12 +809,18 @@ Intel and IBM are leading commercial efforts in neuromorphic hardware. Intel's L Spiking neural networks (SNNs) [@maass1997networks] are computational models for neuromorphic hardware. Unlike deep neural networks communicating via continuous values, SNNs use discrete spikes that are more akin to biological neurons. This allows efficient event-based computation rather than constant processing. Additionally, SNNs consider the temporal and spatial characteristics of input data. This better mimics biological neural networks, where the timing of neuronal spikes plays an important role. However, training SNNs remains challenging due to the added temporal complexity. @fig-spiking provides an overview of the spiking methodology: (a) Diagram of a neuron; (b) Measuring an action potential propagated along the axon of a neuron. Only the action potential is detectable along the axon; (c) The neuron's spike is approximated with a binary representation; (d) Event-Driven Processing; (e) Active Pixel Sensor and Dynamic Vision Sensor. -You can also watch the video linked below for a more detailed explanation. - ![Neuromoprhic spiking. Credit: @eshraghian2023training.](images/png/aimage4.png){#fig-spiking} +You can also watch @vid-snn linked below for a more detailed explanation. + +:::{#vid-snn .callout-important} + +# Neuromorphic Computing + {{< video https://www.youtube.com/watch?v=yihk_8XnCzg >}} +::: + Specialized nanoelectronic devices called memristors [@chua1971memristor] are synaptic components in neuromorphic systems. Memristors act as nonvolatile memory with adjustable conductance, emulating the plasticity of real synapses. Memristors enable in-situ learning without separate data transfers by combining memory and processing functions. However, memristor technology has yet to reach maturity and scalability for commercial hardware. The integration of photonics with neuromorphic computing [@shastri2021photonics] has recently emerged as an active research area. Using light for computation and communication allows high speeds and reduced energy consumption. However, fully realizing photonic neuromorphic systems requires overcoming design and integration challenges. @@ -995,7 +1007,11 @@ While significant research remains, foundation models represent the most transfo If you are interested in ML-aided computer architecture design [@krishnan2023archgym], we encourage you to read [Architecture 2.0](https://www.sigarch.org/architecture-2-0-why-computer-architects-need-a-data-centric-ai-gymnasium/). -Alternatively, you can watch the below video. +Alternatively, you can watch @vid-arch for more details. + +:::{#vid-arch .callout-important} + +# Architecture 2.0 {{< video https://www.youtube.com/watch?v=F5Eieaz7u1I&ab_channel=OpenComputeProject >}} @@ -1021,7 +1037,18 @@ Here is a curated list of resources to support students and instructors in their * *Coming soon.* ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-wfscale + +- @vid-snn + +- @vid-arch + +::: + +:::{.callout-caution collapse="false"} #### Exercises * @exr-tvm diff --git a/contents/ml_systems/ml_systems.qmd b/contents/ml_systems/ml_systems.qmd index 176dff3d..b7effe45 100644 --- a/contents/ml_systems/ml_systems.qmd +++ b/contents/ml_systems/ml_systems.qmd @@ -5,7 +5,7 @@ bibliography: ml_systems.bib # ML Systems {#sec-ml_systems} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-ml-systems-resource), [Exercises](#sec-ml-systems-resource), [Labs](#sec-ml-systems-resource) +Resources: [Slides](#sec-ml-systems-resource), [Videos](#sec-ml-systems-resource), [Exercises](#sec-ml-systems-resource), [Labs](#sec-ml-systems-resource) ::: ![_DALL·E 3 Prompt: Illustration in a rectangular format depicting the merger of embedded systems with Embedded AI. The left half of the image portrays traditional embedded systems, including microcontrollers and processors, detailed and precise. The right half showcases the world of artificial intelligence, with abstract representations of machine learning models, neurons, and data flow. The two halves are distinctly separated, emphasizing the individual significance of embedded tech and AI, but they come together in harmony at the center._](images/png/cover_ml_systems.png) @@ -278,7 +278,14 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/ondevice_learning/ondevice_learning.qmd b/contents/ondevice_learning/ondevice_learning.qmd index 67ae20b9..b5c9b0b2 100644 --- a/contents/ondevice_learning/ondevice_learning.qmd +++ b/contents/ondevice_learning/ondevice_learning.qmd @@ -5,7 +5,7 @@ bibliography: ondevice_learning.bib # On-Device Learning {#sec-ondevice_learning} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-on-device-learning-resource), [Exercises](#sec-on-device-learning-resource), [Labs](#sec-on-device-learning-resource) +Resources: [Slides](#sec-on-device-learning-resource), [Videos](#sec-on-device-learning-resource), [Exercises](#sec-on-device-learning-resource), [Labs](#sec-on-device-learning-resource) ::: ![_DALL·E 3 Prompt: Drawing of a smartphone with its internal components exposed, revealing diverse miniature engineers of different genders and skin tones actively working on the ML model. The engineers, including men, women, and non-binary individuals, are tuning parameters, repairing connections, and enhancing the network on the fly. Data flows into the ML model, being processed in real-time, and generating output inferences._](images/png/cover_ondevice_learning.png) @@ -396,10 +396,16 @@ The complexity of the source model can also impact the success of transfer learn By considering these factors, ML practitioners can make informed decisions about when and how to utilize transfer learning, ultimately leading to more successful model performance on the target task. The success of transfer learning hinges on the degree of similarity between the source and target domains. Overfitting is risky, especially when finetuning occurs on a limited dataset. On the computational front, certain pre-trained models, owing to their size, might not comfortably fit into the memory constraints of some devices or may run prohibitively slowly. Over time, as data evolves, there is potential for model drift, indicating the need for periodic re-training or ongoing adaptation. -Learn more about transfer learning in the video below. +Learn more about transfer learning in @vid-tl below. + +:::{#vid-tl .callout-important} + +# Transfer Learning {{< video https://www.youtube.com/watch?v=FQM13HkEfBk >}} +::: + ## Federated Machine Learning {#sec-fl} Federated Learning Overview @@ -418,10 +424,16 @@ This summarizes the basic algorithm for federated learning on the right. For eac With this proposed structure, there are a few key vectors for further optimizing federated learning. We will outline each in the following subsections. -The following video is an overview of federated learning. +@vid-fl gives an overview of federated learning. + +:::{#vid-fl .callout-important} + +# Transfer Learning {{< video https://www.youtube.com/watch?v=zqv1eELa7fs >}} +::: + ### Communication Efficiency One of the key bottlenecks in federated learning is communication. Every time a client trains the model, they must communicate their updates back to the server. Similarly, once the server has averaged all the updates, it must send them back to the client. This incurs huge bandwidth and resource costs on large networks of millions of devices. As the field of federated learning advances, a few optimizations have been developed to minimize this communication. To address the footprint of the model, researchers have developed model compression techniques. In the client-server protocol, federated learning can also minimize communication through the selective sharing of updates on clients. Finally, efficient aggregation techniques can also streamline the communication process. @@ -734,7 +746,16 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-tl + +- @vid-fl + +::: + +:::{.callout-caution collapse="false"} #### Exercises diff --git a/contents/ops/ops.qmd b/contents/ops/ops.qmd index cd8cc2f2..555df8db 100644 --- a/contents/ops/ops.qmd +++ b/contents/ops/ops.qmd @@ -5,7 +5,7 @@ bibliography: ops.bib # ML Operations {#sec-mlops} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-embedded-aiops-resource), [Exercises](#sec-embedded-aiops-resource), [Labs](#sec-embedded-aiops-resource) +Resources: [Slides](#sec-embedded-aiops-resource), [Videos](#sec-embedded-aiops-resource), [Exercises](#sec-embedded-aiops-resource), [Labs](#sec-embedded-aiops-resource) ::: ![_DALL·E 3 Prompt: Create a detailed, wide rectangular illustration of an AI workflow. The image should showcase the process across six stages, with a flow from left to right: 1. Data collection, with diverse individuals of different genders and descents using a variety of devices like laptops, smartphones, and sensors to gather data. 2. Data processing, displaying a data center with active servers and databases with glowing lights. 3. Model training, represented by a computer screen with code, neural network diagrams, and progress indicators. 4. Model evaluation, featuring people examining data analytics on large monitors. 5. Deployment, where the AI is integrated into robotics, mobile apps, and industrial equipment. 6. Monitoring, showing professionals tracking AI performance metrics on dashboards to check for accuracy and concept drift over time. Each stage should be distinctly marked and the style should be clean, sleek, and modern with a dynamic and informative color scheme._](images/png/cover_ml_ops.png) @@ -90,10 +90,16 @@ While DevOps and MLOps share similarities in their goals and principles, they di : Comparison of DevOps and MLOps. {#tbl-mlops} -Learn more about ML Lifecycles through a case study featuring speech recognition. +Learn more about ML Lifecycles through a case study featuring speech recognition in @vid-mlops. + +:::{#vid-mlops .callout-important} + +# MLOps {{< video https://www.youtube.com/watch?v=YJsRD_hU4tc&list=PLkDaE6sCZn6GMoA0wbpJLi3t34Gd8l0aK&index=3 >}} +::: + ## Key Components of MLOps In this chapter, we will provide an overview of the core components of MLOps, an emerging set of practices that enables robust delivery and lifecycle management of ML models in production. While some MLOps elements like automation and monitoring were covered in previous chapters, we will integrate them into an integrated framework and expand on additional capabilities like governance. Additionally, we will describe and link to popular tools used within each component, such as [LabelStudio](https://labelstud.io/) for data labeling. By the end, we hope that you will understand the end-to-end MLOps methodology that takes models from ideation to sustainable value creation within organizations. @@ -114,10 +120,16 @@ For instance, a pipeline may ingest data from [PostgreSQL](https://www.postgresq In an industrial predictive maintenance use case, sensor data is ingested from devices into S3. A perfect pipeline processes the sensor data, joining it with maintenance records. The enriched dataset is stored in Feast so models can easily retrieve the latest data for training and predictions. -The video below is a short overview of data pipelines. +@vid-datapipe below is a short overview of data pipelines. + +:::{#vid-datapipe .callout-important} + +# Data Pipelines {{< video https://www.youtube.com/watch?v=gz-44N3MMOA&list=PLkDaE6sCZn6GMoA0wbpJLi3t34Gd8l0aK&index=33 >}} +::: + ### CI/CD Pipelines Continuous integration and continuous delivery (CI/CD) pipelines actively automate the progression of ML models from initial development into production deployment. Adapted for ML systems, CI/CD principles empower teams to rapidly and robustly deliver new models with minimized manual errors. @@ -218,8 +230,14 @@ After deployment, comprehensive monitoring enables teams to maintain confidence Watch the video below to learn more about monitoring. +:::{#vid-monitoring .callout-important} + +# Model Monitoring + {{< video https://www.youtube.com/watch?v=hq_XyP9y0xg&list=PLkDaE6sCZn6GMoA0wbpJLi3t34Gd8l0aK&index=7 >}} +::: + ### Governance MLOps teams actively establish proper governance practices as a critical component. Governance provides oversight into ML models to ensure they are trustworthy, ethical, and compliant. Without governance, significant risks exist of models behaving in dangerous or prohibited ways when deployed in applications and business processes. @@ -244,10 +262,16 @@ For example, a data scientist may use Weights & Biases to analyze an anomaly det Enabling transparency, traceability, and communication via MLOps empowers teams to remove bottlenecks and accelerate the delivery of impactful ML systems. -The following video covers key challenges in model deployment, including concept drift, model drift, and software engineering issues. +@vid-deploy covers key challenges in model deployment, including concept drift, model drift, and software engineering issues. + +:::{#vid-deploy .callout-important} + +# Deployment Challenges {{< video https://www.youtube.com/watch?v=UyEtTyeahus&list=PLkDaE6sCZn6GMoA0wbpJLi3t34Gd8l0aK&index=5 >}} +::: + ## Hidden Technical Debt in ML Systems Technical debt is increasingly pressing for ML systems (see Figure 14.2). This metaphor, originally proposed in the 1990s, likens the long-term costs of quick software development to financial debt. Just as some financial debt powers beneficial growth, carefully managed technical debt enables rapid iteration. However, left unchecked, accumulating technical debt can outweigh any gains. @@ -921,7 +945,20 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-mlops + +- @vid-datapipe + +- @vid-monitoring + +- @vid-deploy + +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/optimizations/optimizations.qmd b/contents/optimizations/optimizations.qmd index 6b0868e6..3da94ec9 100644 --- a/contents/optimizations/optimizations.qmd +++ b/contents/optimizations/optimizations.qmd @@ -5,7 +5,7 @@ bibliography: optimizations.bib # Model Optimizations {#sec-model_optimizations} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-model-optimizations-resource), [Exercises](#sec-model-optimizations-resource), [Labs](#sec-model-optimizations-resource) +Resources: [Slides](#sec-model-optimizations-resource), [Videos](#sec-model-optimizations-resource), [Exercises](#sec-model-optimizations-resource), [Labs](#sec-model-optimizations-resource) ::: ![_DALL·E 3 Prompt: Illustration of a neural network model represented as a busy construction site, with a diverse group of construction workers, both male and female, of various ethnicities, labeled as 'pruning', 'quantization', and 'sparsity'. They are working together to make the neural network more efficient and smaller, while maintaining high accuracy. The 'pruning' worker, a Hispanic female, is cutting unnecessary connections from the middle of the network. The 'quantization' worker, a Caucasian male, is adjusting or tweaking the weights all over the place. The 'sparsity' worker, an African female, is removing unnecessary nodes to shrink the model. Construction trucks and cranes are in the background, assisting the workers in their tasks. The neural network is visually transforming from a complex and large structure to a more streamlined and smaller one._](images/png/cover_model_optimizations.png) @@ -717,10 +717,16 @@ In addition to being an indispensable technique for many edge processors, quanti Thus, quantization combined with efficient low-precision logic and dedicated deep learning accelerators, has been one crucial driving force for the evolution of such edge processors. -The video below is a lecture on quantization and the different quantization methods. +@vid-quant is a lecture on quantization and the different quantization methods. + +:::{#vid-quant .callout-important} + +# Quantization {{< video https://www.youtube.com/watch?v=AlASZb93rrc >}} +::: + ## Efficient Hardware Implementation {#sec-model_ops_hw} Efficient hardware implementation transcends the selection of suitable components; it requires a holistic understanding of how software will interact with underlying architectures. The essence of achieving peak performance in TinyML applications lies not only in refining algorithms to hardware but also in ensuring that the hardware is strategically tailored to support these algorithms. This synergy between hardware and software is crucial. As we delve deeper into the intricacies of efficient hardware implementation, the significance of a co-design approach, where hardware and software are developed in tandem, becomes increasingly evident. This section provides an overview of the techniques of how hardware and the interactions between hardware and software can be optimized to improve models performance. @@ -977,7 +983,14 @@ These slides serve as a valuable tool for instructors to deliver lectures and fo ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-quant + +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/privacy_security/privacy_security.qmd b/contents/privacy_security/privacy_security.qmd index a4025ea9..adb12f6f 100644 --- a/contents/privacy_security/privacy_security.qmd +++ b/contents/privacy_security/privacy_security.qmd @@ -5,7 +5,7 @@ bibliography: privacy_security.bib # Security & Privacy {#sec-security_privacy} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-security-and-privacy-resource), [Exercises](#sec-security-and-privacy-resource), [Labs](#sec-security-and-privacy-resource) +Resources: [Slides](#sec-security-and-privacy-resource), [Videos](#sec-security-and-privacy-resource), [Exercises](#sec-security-and-privacy-resource), [Labs](#sec-security-and-privacy-resource) ::: ![_DALL·E 3 Prompt: An illustration on privacy and security in machine learning systems. The image shows a digital landscape with a network of interconnected nodes and data streams, symbolizing machine learning algorithms. In the foreground, there's a large lock superimposed over the network, representing privacy and security. The lock is semi-transparent, allowing the underlying network to be partially visible. The background features binary code and digital encryption symbols, emphasizing the theme of cybersecurity. The color scheme is a mix of blues, greens, and grays, suggesting a high-tech, digital environment._](images/png/cover_security_privacy.png) @@ -84,22 +84,30 @@ This breach was significant due to its sophistication; Stuxnet specifically targ ### Jeep Cherokee Hack -The Jeep Cherokee hack was a groundbreaking event demonstrating the risks inherent in increasingly connected automobiles [@miller2019lessons]. In a controlled demonstration, security researchers remotely exploited a vulnerability in the Uconnect entertainment system, which had a cellular connection to the internet. They were able to control the vehicle's engine, transmission, and brakes, alarming the automotive industry into recognizing the severe safety implications of cyber vulnerabilities in vehicles. +The Jeep Cherokee hack was a groundbreaking event demonstrating the risks inherent in increasingly connected automobiles [@miller2019lessons]. In a controlled demonstration, security researchers remotely exploited a vulnerability in the Uconnect entertainment system, which had a cellular connection to the internet. They were able to control the vehicle's engine, transmission, and brakes, alarming the automotive industry into recognizing the severe safety implications of cyber vulnerabilities in vehicles. @vid-jeephack below is a short documentary of the attack. -The video below is a short documentary of the attack. +:::{#vid-jeephack .callout-important} -{{< video https://www.youtube.com/watch?v=MK0SrxBC1xs&ab_channel=WIRED title="Hackers Remotely Kill a Jeep on a Highway" >}} +# Jeep Cherokee Hack + +{{< video https://www.youtube.com/watch?v=MK0SrxBC1xs&ab_channel=WIRED>}} + +::: While this wasn't an attack on an ML system per se, the reliance of modern vehicles on embedded systems for safety-critical functions has significant parallels to the deployment of ML in embedded systems, underscoring the need for robust security at the hardware level. ### Mirai Botnet -The Mirai botnet involved the infection of networked devices such as digital cameras and DVR players [@antonakakis2017understanding]. In October 2016, the botnet was used to conduct one of the largest [DDoS](https://www.cloudflare.com/learning/ddos/what-is-a-ddos-attack/) attacks, disrupting internet access across the United States. The attack was possible because many devices used default usernames and passwords, which were easily exploited by the Mirai malware to control the devices. +The Mirai botnet involved the infection of networked devices such as digital cameras and DVR players [@antonakakis2017understanding]. In October 2016, the botnet was used to conduct one of the largest [DDoS](https://www.cloudflare.com/learning/ddos/what-is-a-ddos-attack/) attacks, disrupting internet access across the United States. The attack was possible because many devices used default usernames and passwords, which were easily exploited by the Mirai malware to control the devices. @vid-mirai explains how the Mirai Botnet works. + +:::{#vid-mirai .callout-important} -The following video presentation explains how the Mirai Botnet works. +# Jeep Cherokee Hack {{< video https://www.youtube.com/watch?v=1pywzRTJDaY >}} +::: + Although the devices were not ML-based, the incident is a stark reminder of what can happen when numerous embedded devices with poor security controls are networked, which is becoming more common with the growth of ML-based IoT devices. ### Implications @@ -363,11 +371,15 @@ Let's look at the power analysis chart when we enter an incorrect password in @f ![Power analysis of an encryption device with a wrong password. Credit: [Colin O'Flynn.](https://www.youtube.com/watch?v=2iDLfuEBcs8)](images/png/image15.png){#fig-encryption3} -The example above shows how we can infer information about the encryption process and the secret key by analyzing different inputs and trying to 'eavesdrop' on the device's operations on each input byte. +The example above shows how we can infer information about the encryption process and the secret key by analyzing different inputs and trying to 'eavesdrop' on the device's operations on each input byte. For a more detailed explanation, watch @vid-powerattack below. -For a more detailed explanation, watch the video below. +:::{#vid-powerattack .callout-important} -{{< video title="ECED4406 - 0x501 Power Analysis Attacks" >}} +# Power Attack + +{{< video >}} + +::: Another example is an ML system for speech recognition, which processes voice commands to perform actions. By measuring the time it takes for the system to respond to commands or the power used during processing, an attacker could infer what commands are being processed and thus learn about the system's operational patterns. Even more subtle, the sound emitted by a computer's fan or hard drive could change in response to the workload, which a sensitive microphone could pick up and analyze to determine what kind of operations are being performed. @@ -1110,7 +1122,18 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-jeephack + +- @vid-mirai + +- @vid-powerattack + +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/responsible_ai/responsible_ai.qmd b/contents/responsible_ai/responsible_ai.qmd index 5ae8cdf2..0d7834f4 100644 --- a/contents/responsible_ai/responsible_ai.qmd +++ b/contents/responsible_ai/responsible_ai.qmd @@ -5,7 +5,7 @@ bibliography: responsible_ai.bib # Responsible AI {#sec-responsible_ai} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-responsible-ai-resource), [Exercises](#sec-responsible-ai-resource), [Labs](#sec-responsible-ai-resource) +Resources: [Slides](#sec-responsible-ai-resource), [Videos](#sec-responsible-ai-resource), [Exercises](#sec-responsible-ai-resource), [Labs](#sec-responsible-ai-resource) ::: ![_DALL·E 3 Prompt: Illustration of responsible AI in a futuristic setting with the universe in the backdrop: A human hand or hands nurturing a seedling that grows into an AI tree, symbolizing a neural network. The tree has digital branches and leaves, resembling a neural network, to represent the interconnected nature of AI. The background depicts a future universe where humans and animals with general intelligence collaborate harmoniously. The scene captures the initial nurturing of the AI as a seedling, emphasizing the ethical development of AI technology in harmony with humanity and the universe._](images/png/cover_responsible_ai.png) @@ -86,10 +86,16 @@ Putting AI systems into real-world operation requires ensuring they are safe, re Adversarial attacks that subtly alter input data can also fool ML models and cause dangerous failures if systems are not resistant. Deepfakes represent another emerging threat area. -Below is a deepfake video of Barack Obama that went viral a few years ago. +@vid-fakeobama is a deepfake video of Barack Obama that went viral a few years ago. + +:::{#vid-fakeobama .callout-important} + +# Fake Obama {{< video https://www.youtube.com/watch?v=AmUC4m6w1wo&ab_channel=BBCNews >}} +::: + Promoting safety requires extensive testing, risk analysis, human oversight, and designing systems that combine multiple weak models to avoid single points of failure. Rigorous safety mechanisms are essential for the responsible deployment of capable AI. ### Accountability and Governance @@ -472,7 +478,14 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-fakeobama + +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/robust_ai/robust_ai.qmd b/contents/robust_ai/robust_ai.qmd index 68325495..8e670e20 100644 --- a/contents/robust_ai/robust_ai.qmd +++ b/contents/robust_ai/robust_ai.qmd @@ -5,7 +5,7 @@ bibliography: robust_ai.bib # Robust AI {#sec-robust_ai} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-robust-ai-resource), [Exercises](#sec-robust-ai-resource), [Labs](#sec-robust-ai-resource) +Resources: [Slides](#sec-robust-ai-resource), [Videos](#sec-robust-ai-resource), [Exercises](#sec-robust-ai-resource), [Labs](#sec-robust-ai-resource) ::: ![_DALL·E 3 Prompt: Create an image featuring an advanced AI system symbolized by an intricate, glowing neural network, deeply nested within a series of progressively larger and more fortified shields. Each shield layer represents a layer of defense, showcasing the system's robustness against external threats and internal errors. The neural network, at the heart of this fortress of shields, radiates with connections that signify the AI's capacity for learning and adaptation. This visual metaphor emphasizes not only the technological sophistication of the AI but also its resilience and security, set against the backdrop of a state-of-the-art, secure server room filled with the latest in technological advancements. The image aims to convey the concept of ultimate protection and resilience in the field of artificial intelligence._](./images/png/cover_robust_ai.png) @@ -1037,7 +1037,13 @@ These slides are a valuable tool for instructors to deliver lectures and for stu * *Coming soon.* ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/sustainable_ai/sustainable_ai.qmd b/contents/sustainable_ai/sustainable_ai.qmd index 2142c34b..d99fbd97 100644 --- a/contents/sustainable_ai/sustainable_ai.qmd +++ b/contents/sustainable_ai/sustainable_ai.qmd @@ -5,7 +5,7 @@ bibliography: sustainable_ai.bib # Sustainable AI {#sec-sustainable_ai} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-sustainable-ai-resource), [Exercises](#sec-sustainable-ai-resource), [Labs](#sec-sustainable-ai-resource) +Resources: [Slides](#sec-sustainable-ai-resource), [Videos](#sec-sustainable-ai-resource), [Exercises](#sec-sustainable-ai-resource), [Labs](#sec-sustainable-ai-resource) ::: ![_DALL·E 3 Prompt: 3D illustration on a light background of a sustainable AI network interconnected with a myriad of eco-friendly energy sources. The AI actively manages and optimizes its energy from sources like solar arrays, wind turbines, and hydro dams, emphasizing power efficiency and performance. Deep neural networks spread throughout, receiving energy from these sustainable resources._](images/png/cover_sustainable_ai.png) @@ -682,7 +682,13 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/training/training.qmd b/contents/training/training.qmd index cc4b75d8..d7d2a2d1 100644 --- a/contents/training/training.qmd +++ b/contents/training/training.qmd @@ -5,7 +5,7 @@ bibliography: training.bib # AI Training {#sec-ai_training} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-ai-training-resource), [Exercises](#sec-ai-training-resource), [Labs](#sec-ai-training-resource) +Resources: [Slides](#sec-ai-training-resource), [Videos](#sec-ai-frameworks-resource), [Exercises](#sec-ai-training-resource), [Labs](#sec-ai-training-resource) ::: ![_DALL·E 3 Prompt: An illustration for AI training, depicting a neural network with neurons that are being repaired and firing. The scene includes a vast network of neurons, each glowing and firing to represent activity and learning. Among these neurons, small figures resembling engineers and scientists are actively working, repairing and tweaking the neurons. These miniature workers symbolize the process of training the network, adjusting weights and biases to achieve convergence. The entire scene is a visual metaphor for the intricate and collaborative effort involved in AI training, with the workers representing the continuous optimization and learning within a neural network. The background is a complex array of interconnected neurons, creating a sense of depth and complexity._](images/png/ai_training.png) @@ -315,10 +315,16 @@ The test set acts as a completely unseen dataset that the model did not see duri The relative proportions of the training, validation, and test sets can vary based on data size and application. However, following the general guidelines for a 60/20/20 split is a good starting point. Careful data splitting ensures models are properly trained, tuned, and evaluated to achieve the best performance. -The video below explains how to properly split the dataset into training, validation, and testing sets, ensuring an optimal training process. +@vid-train-dev-test explains how to properly split the dataset into training, validation, and testing sets, ensuring an optimal training process. + +:::{#vid-train-dev-test .callout-important} + +# Train/Dev/Test Sets {{< video https://www.youtube.com/watch?v=1waHlpKiNyY >}} +::: + ### Common Pitfalls and Mistakes #### Insufficient Training Data @@ -341,10 +347,16 @@ Consider @fig-over-under-fitting where we try to classify/split datapoints into ![Fitting the data overtime. Credit: IBM.](images/png/aitrainingfit.png){#fig-fitting-time} -The video below provides an overview of bias and variance and the relationship between the two concepts and model accuracy. +@vid-bias provides an overview of bias and variance and the relationship between the two concepts and model accuracy. + +:::{#vid-bias .callout-important} + +# Bias/Variance {{< video https://www.youtube.com/watch?v=SjQyLhQIXSM >}} +::: + #### Data Leakage Between Sets Data leakage refers to the unintentional transfer of information between the training, validation, and test sets. This violates the fundamental assumption that the splits are completely separated. Data leakage leads to seriously compromised evaluation results and inflated performance metrics. @@ -600,10 +612,16 @@ Get ready to unlock the secrets of hyperparameter tuning and take your PyTorch m ::: -The video below explains the systematic organization of the hyperparameter tuning process. +@vid-hyperparameter explains the systematic organization of the hyperparameter tuning process. + +:::{#vid-hyperparameter .callout-important} + +# Hyperparameter {{< video https://www.youtube.com/watch?v=AXDByU3D1hA&list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&index=24 >}} +::: + ## Regularization Regularization is a critical technique for improving the performance and generalizability of machine learning models in applied settings. It refers to mathematically constraining or penalizing model complexity to avoid overfitting the training data. Without regularization, complex ML models are prone to overfitting the dataset and memorizing peculiarities and noise in the training set rather than learning meaningful patterns. They may achieve high training accuracy but perform poorly when evaluating new unseen inputs. @@ -664,12 +682,26 @@ Where: The choice between L1 and L2 depends on the expected model complexity and whether intrinsic feature selection is needed. Both require iterative tuning across a validation set to select the optimal α hyperparameter. -The two videos below explain how regularization works and can help reduce model overfitting to improve performance. +@vid-regularization and @vid-whyreg explains how regularization works. + +:::{#vid-regularization .callout-important} + +# Regularization {{< video https://www.youtube.com/watch?v=6g0t3Phly2M&list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&index=4 >}} +::: + +@vid-whyreg explains how regularization can help reduce model overfitting to improve performance. + +:::{#vid-whyreg .callout-important} + +# Why Regularization Reduces Overfitting + {{< video https://www.youtube.com/watch?v=NyG-7nRpsW8&list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&index=5 >}} +::: + ### Dropout Another widely adopted regularization method is dropout [@srivastava2014dropout]. During training, dropout randomly sets a fraction $p$ of node outputs or hidden activations to zero. This encourages greater information distribution across more nodes rather than reliance on a small number of nodes. Come prediction time; the full neural network is used, with intermediate activations scaled by $p$ to maintain output magnitudes. GPU optimizations make implementing dropout efficiently straightforward via frameworks like PyTorch and TensorFlow. @@ -698,10 +730,16 @@ Where: The key hyperparameter is $p$, the fraction of nodes dropped, often set between 0.2 and 0.5. Larger networks tend to benefit from more dropout, while small networks risk underfitting if too many nodes are cut out. Trial and error combined with monitoring validation performance helps tune the dropout level. -The following video discusses the intuition behind the dropout regularization technique and how it works. +@vid-dropout discusses the intuition behind the dropout regularization technique and how it works. + +:::{#vid-dropout .callout-important} + +# Dropout {{< video https://www.youtube.com/watch?v=ARq74QuavAo&list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&index=7 >}} +::: + ### Early Stopping The intuition behind early stopping involves tracking model performance on a held-out validation set across training epochs. At first, increases in training set fitness accompany gains in validation accuracy as the model picks up generalizable patterns. After some point, however, the model starts overfitting - latching onto peculiarities and noise in the training data that don't apply more broadly. The validation performance peaks and then degrades if training continues. Early stopping rules halt training at this peak to prevent overfitting. This technique demonstrates how ML pipelines must monitor system feedback, not just unquestioningly maximize performance on a static training set. The system's state evolves, and the optimal endpoints change. @@ -721,10 +759,16 @@ Battling Overfitting: Unlock the Secrets of Regularization! Overfitting is like ::: -The following video covers a few other regularization methods that can reduce model overfitting. +@vid-otherregs covers a few other regularization methods that can reduce model overfitting. + +:::{#vid-otherregs .callout-important} + +# Other Regularization Methods {{< video https://www.youtube.com/watch?v=BOCLq2gpcGU&list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&index=8 >}} +::: + ## Weight Initialization Proper Initializing the weights in a neural network before training is a vital step directly impacting model performance. Randomly initializing weights to very large or small values can lead to problems like vanishing/exploding gradients, slow convergence of training, or getting trapped in poor local minima. Proper weight initialization accelerates model convergence during training and carries implications for system performance at inference time in production environments. Some key aspects include: @@ -771,10 +815,16 @@ Get your neural network off to a strong start with weight initialization! How yo ::: -The video below emphasizes the importance of deliberately selecting initial weight values over random choices. +@vid-weightinit emphasizes the importance of deliberately selecting initial weight values over random choices. + +:::{#vid-weightinit .callout-important} + +# Weight Initialization {{< video https://www.youtube.com/watch?v=s2coXdufOzE&list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&index=11 >}} +::: + ## Activation Functions Activation functions play a crucial role in neural networks. They introduce nonlinear behaviors that allow neural nets to model complex patterns. Element-wise activation functions are applied to the weighted sums coming into each neuron in the network. Without activation functions, neural nets would be reduced to linear regression models. @@ -1070,7 +1120,28 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +- @vid-train-dev-test + +- @vid-bias + +- @vid-hyperparameter + +- @vid-regularization + +- @vid-whyreg + +- @vid-dropout + +- @vid-otherregs + +- @vid-weightinit + +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding. diff --git a/contents/workflow/workflow.qmd b/contents/workflow/workflow.qmd index 24d89f1b..179e670b 100644 --- a/contents/workflow/workflow.qmd +++ b/contents/workflow/workflow.qmd @@ -5,7 +5,7 @@ bibliography: workflow.bib # AI Workflow {#sec-ai_workflow} ::: {.content-visible when-format="html"} -Resources: [Slides](#sec-ai-workflow-resource), [Exercises](#sec-ai-workflow-resource), [Labs](#sec-ai-workflow-resource) +Resources: [Slides](#sec-ai-workflow-resource), [Videos](#sec-ai-workflow-resource), [Exercises](#sec-ai-workflow-resource), [Labs](#sec-ai-workflow-resource) ::: ![_DALL·E 3 Prompt: Create a rectangular illustration of a stylized flowchart representing the AI workflow/pipeline. From left to right, depict the stages as follows: 'Data Collection' with a database icon, 'Data Preprocessing' with a filter icon, 'Model Design' with a brain icon, 'Training' with a weight icon, 'Evaluation' with a checkmark, and 'Deployment' with a rocket. Connect each stage with arrows to guide the viewer horizontally through the AI processes, emphasizing these steps' sequential and interconnected nature._](images/png/cover_ai_workflow.png) @@ -121,7 +121,13 @@ These slides are a valuable tool for instructors to deliver lectures and for stu ::: -:::{ .callout-caution collapse="false"} +:::{.callout-important collapse="false"} +#### Videos + +* *Coming soon.* +::: + +:::{.callout-caution collapse="false"} #### Exercises To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding.