Skip to content

Commit

Permalink
Removed duplicate copy of a section
Browse files Browse the repository at this point in the history
  • Loading branch information
@profvjreddi
profvjreddi committed Dec 17, 2024
1 parent 1e13ef3 commit 3a7d2e1
Showing 1 changed file with 0 additions and 58 deletions.
58 changes: 0 additions & 58 deletions contents/core/ml_systems/ml_systems.qmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -402,64 +402,6 @@ In environmental monitoring, Tiny ML enables real-time data analysis from variou

In summary, Tiny ML serves as a trailblazer in the evolution of machine learning, fostering innovation across various fields by bringing intelligence directly to the edge. Its potential to transform our interaction with technology and the world is immense, promising a future where devices are connected, intelligent, and capable of making real-time decisions and responses.

## ML System Comparison

We can now synthesize our understanding by examining how the various ML system approaches compare across different dimensions. This synthesis is particularly important as system designers often face tradeoffs between different deployment options when implementing ML solutions.

The relationship between computational resources and deployment location forms one of the most fundamental comparisons across ML systems. As we move from cloud deployments to tiny devices, we observe a dramatic reduction in available computing power, storage, and energy consumption. Cloud ML systems, with their data center infrastructure, can leverage virtually unlimited resources, processing data at the scale of petabytes and training models with billions of parameters. Edge ML systems, while more constrained, still offer significant computational capability through specialized hardware like edge GPUs and neural processing units. Mobile ML represents a middle ground, balancing computational power with energy efficiency on devices like smartphones and tablets. At the far end of the spectrum, TinyML operates under severe resource constraints, often limited to kilobytes of memory and milliwatts of power consumption.

The operational characteristics of these systems reveal another important dimension of comparison. @tbl-big_vs_tiny provides a comprehensive view of how these systems differ across various operational aspects. Latency, for instance, shows a clear pattern: cloud systems typically incur delays of 100-1000ms due to network communication, while edge systems reduce this to 10-100ms by processing data locally. Mobile ML achieves even lower latencies of 5-50ms for many tasks, and TinyML systems can respond in 1-10ms for simple inferences. This latency gradient illustrates how moving computation closer to the data source can improve real-time processing capabilities.

+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Aspect | Cloud ML | Edge ML | Mobile ML | Tiny ML |
+:=========================+:=========================================================+:=========================================================+:==========================================================+:=========================================================+
| Processing Location | Centralized cloud servers (Data Centers) | Local edge devices (gateways, servers) | Smartphones and tablets | Ultra-low-power microcontrollers and embedded systems |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Latency | High (100ms-1000ms+) | Moderate (10-100ms) | Low-Moderate (5-50ms) | Very Low (1-10ms) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Data Privacy | Basic-Moderate (Data leaves device) | High (Data stays in local network) | High (Data stays on phone) | Very High (Data never leaves sensor) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Computational Power | Very High (Multiple GPUs/TPUs) | High (Edge GPUs) | Moderate (Mobile NPUs/GPUs) | Very Low (MCU/tiny processors) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Energy Consumption | Very High (kW-MW range) | High (100s W) | Moderate (1-10W) | Very Low (mW range) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Scalability | Excellent (virtually unlimited) | Good (limited by edge hardware) | Moderate (per-device scaling) | Limited (fixed hardware) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Cost | High ($1000s+/month) | Moderate ($100s-1000s) | Low ($0-10s) | Very Low ($1-10s) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Connectivity Required | Constant high-bandwidth | Intermittent | Optional | None |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Real-time Processing | Dependent on network | Good | Very Good | Excellent |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Storage Capacity | Unlimited (petabytes+) | Large (terabytes) | Moderate (gigabytes) | Very Limited (kilobytes-megabytes) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Primary Use Cases | Big Data Analytics, Training, Complex AI Models | Smart Manufacturing, Video Analytics, IoT Hubs | AR/VR Apps, Mobile Gaming, Photo/Video Processing | Sensor Processing, Gesture Detection, Keyword Spotting |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Development Complexity | High (cloud expertise needed) | Moderate-High (edge+networking) | Moderate (mobile SDKs) | High (embedded expertise) |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Deployment Speed | Fast | Moderate | Fast | Slow |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Hardware Requirements | Cloud infrastructure | Edge servers/gateways | Modern smartphones | MCUs/embedded systems |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Framework Support | All ML frameworks | Most frameworks | Mobile-optimized (TFLite, CoreML) | Tiny ML frameworks |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Model Size Limits | None | Several GB | 10s-100s MB | Bytes-KB range |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Battery Impact | N/A | N/A | Moderate | Minimal |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+
| Offline Capability | None | Good | Excellent | Complete |
+--------------------------+----------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------------------+----------------------------------------------------------+

: Comparison of feature aspects across Cloud ML, Edge ML, and Tiny ML. {#tbl-big_vs_tiny .hover .striped}

Privacy and data handling represent another crucial axis of comparison. Cloud ML requires data to leave the device, potentially raising privacy concerns despite robust security measures. Edge ML improves privacy by keeping data within local networks, while Mobile ML further enhances this by processing sensitive information directly on personal devices. TinyML offers the strongest privacy guarantees, as data never leaves the sensor or microcontroller where it's collected.

Development complexity and deployment considerations also vary significantly across these paradigms. Cloud ML benefits from mature development tools and frameworks but requires expertise in cloud infrastructure. Edge ML demands knowledge of both ML and networking protocols, while Mobile ML developers must understand mobile-specific optimizations and platform constraints. TinyML development, though targeting simpler devices, often requires specialized knowledge of embedded systems and careful optimization to work within severe resource constraints.

Cost structures differ markedly as well. Cloud ML typically involves ongoing operational costs for computation and storage, often running into thousands of dollars monthly for large-scale deployments. Edge ML requires significant upfront investment in edge devices but may reduce ongoing costs. Mobile ML leverages existing consumer devices, minimizing additional hardware costs, while TinyML solutions can be deployed for just a few dollars per device, though development costs may be higher.

These comparisons reveal that each paradigm has distinct advantages and limitations. Cloud ML excels at complex, data-intensive tasks but requires constant connectivity. Edge ML offers a balance of computational power and local processing. Mobile ML provides personalized intelligence on ubiquitous devices. TinyML enables ML in previously inaccessible contexts but requires careful optimization. Understanding these tradeoffs is crucial for selecting the appropriate deployment strategy for specific applications and constraints.

## Hybrid ML

Systems architects rarely confine themselves to a single approach, instead combining various paradigms to create more nuanced solutions. These ``Hybrid ML'' approaches leverage the complementary strengths we've analyzed---from cloud's computational power to tiny's efficiency---while mitigating their individual limitations. Architects create new architectural patterns that balance competing demands for performance, privacy, and resource efficiency, opening up possibilities for more sophisticated ML applications that better meet complex real-world requirements.
Expand Down

0 comments on commit 3a7d2e1

Please sign in to comment.