Skip to content

Commit

Permalink
rename crate to "vm-memory"
Browse files Browse the repository at this point in the history
Rename create from "memory-model" to "vm-memory".

Signed-off-by: Liu Jiang <[email protected]>
  • Loading branch information
jiangliu committed Feb 28, 2019
1 parent cac2eb1 commit 83f6111
Show file tree
Hide file tree
Showing 6 changed files with 51 additions and 49 deletions.
8 changes: 4 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
[package]
name = "memory-model"
name = "vm-memory"
version = "0.1.0"
authors = ["Liu Jiang <[email protected]>"]

[features]
default = ["memory-backend-mmap"]
memory-backend-mmap = []
default = ["vm-memory-backend-mmap"]
vm-memory-backend-mmap = []

[dependencies]
libc = ">=0.2.39"
matches = ">=0"

[dev-dependencies]
matches = ">=0"
tempfile = ">=3.0.2"
46 changes: 24 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,43 @@
# memory-model
# vm-memory
A library to manage and access virtual machine's physical memory.

The `memory-model` crate aims to provide a set of stable traits for consumers to access virtual machine's physical memory. Based on these common traits, typical consumers like hypervisors, virtio backend drivers, vhost-user drivers could access guest's physical memory without knowing the implementation details. And thus virtual device backend drivers based on this crate may be reused by different hypervisors.
For a typical hypervisor, there are seveval components, such as boot loader, virtual device drivers, virtio backend drivers and vhost drivers etc, need to access VM's physical memory. The `vm-memory` crate aims to provide a set of stable traits to decouple VM memory consumers from VM memory providers. Based on these common traits, typical VM memory consumers could access VM's physical memory without knowing the implementation details of VM memory provider. Thus common hypervisor components, such as boot loader, virtual device drivers, virtio backend drivers and vhost drivers could be shared and reused by multiple hypervisors.

On the other hand, this crate dosen't define the way how the underline mechanism is implemented to access guest's physical memory. For light-wieght hypervisors like crosvm and firecracker, they may make some assumptions about the structure of virtual machine's physical memory and implement a light-weight backend to access guest's memory. For hypervisors like qemu, a high performance and full functionality backend may be implemented with less assumptions.
The `vm-memory` crate focuses on defining consumer side interfaces to access VM's physical memory, and it dosen't define the way how the underline VM memory provider is implemented. For light-wieght hypervisors like crosvm and firecracker, they may make some assumptions about the structure of VM's physical memory and implement a light-weight backend to access VM's physical memory. For hypervisors like qemu, a high performance and full functionality backend may be implemented with less assumptions.

This crate is derived from two upstream projects:
- [crosvm project](https://chromium.googlesource.com/chromiumos/platform/crosvm/) commit 186eb8b0db644892e8ffba8344efe3492bb2b823
- [firecracker project](https://firecracker-microvm.github.io/) commit 80128ea61b305a27df1f751d70415b04b503eae7

To be hypervisor neutral, the high level abstraction has been heavily refactored. It could be divided into four parts as:
To be hypervisor neutral, the high level abstraction has been heavily refactored. The new `vm-memory` crate could be divided into four logic parts as:

### Abstraction of Generic Address Space
Build generic abstractions to describe and access an address space as below:
- AddressValue: stores the raw value of an address. Typically u32, u64 or usize is used to store the raw value. But pointers, such as \*u8, can't be used because it doesn't implement the Add and Sub traits.
- Address: encapsulates an AddressValue object and defines methods to access it.
- AddressValue: Stores the raw value of an address. Typically u32, u64 or usize is used to store the raw value. But pointers, such as \*u8, can't be used because it doesn't implement the Add and Sub traits.
- Address: Encapsulates an AddressValue object and defines methods to access it.
- Bytes: Common trait for volatile access to memory. The `Bytes` trait can be parameterized with newtypes that represent addresses, in order to enforce that addresses are used with the right "kind" of volatile memory.
- VolatileMemory: Basic implementation of volatile access to memory, implements `Bytes<usize>`.

To make the abstraction as generic as possible, all the core traits only define methods to access the address space are defined here, and they never define methods to manage (create, delete, insert, remove etc) address spaces. By this way, the address space consumers (virtio device drivers, vhost-user drivers and boot loaders etc) may be decoupled from the address space provider (typically a hypervisor).
To make the abstraction as generic as possible, all of above core traits only define methods to access the address space, and they never define methods to manage (create, delete, insert, remove etc) address spaces. By this way, the address space consumers (virtio device drivers, vhost-user drivers and boot loaders etc) may be decoupled from the address space provider (typically a hypervisor).

### Specialization for Virtual Machine Physical Address Space
The generic address space crates are specialized to access guest's physical memory with following traits:
- GuestAddress: represents a guest physical address (GPA). On ARM64, a 32-bit hypervisor may be used to support a 64-bit guest. For simplicity, u64 is used to store the the raw value no matter the guest a 32-bit or 64-bit virtual machine.
- GuestMemoryRegion: used to represent a continuous region of guest's physical memory.
The generic address space crates are specialized to access VM's physical memory with following traits:
- GuestAddress: represents a guest physical address (GPA). On ARM64, a 32-bit hypervisor may be used to support a 64-bit VM. For simplicity, u64 is used to store the the raw value no matter it is a 32-bit or 64-bit virtual machine.
- GuestMemoryRegion: used to represent a continuous region of VM's physical memory.
- GuestMemory: used to represent a collection of GuestMemoryRegion objects. The main responsibilities of the GuestMemory trait are:
- hide the detail of accessing guest's physical address (for example complex hierarchical structures).
- hide the detail of accessing VM's physical address (for example complex hierarchical structures).
- map a request address to a GuestMemoryRegion object and relay the request to it.
- handle cases where an access request spanning two or more GuestMemoryRegion objects.

The virtual machine memory consumers, such as virtio device drivers, vhost drivers and boot loaders etc, should only rely on traits defined here to access guest's memory.
The VM memory consumers, such as virtio device drivers, vhost drivers and boot loaders etc, should only rely on traits and structs defined here to access VM's physical memory.

### A Sample and Default Backend Implementation Based on mmap()
Provide a default and sample implementation of the GuestMemory trait by mmapping guest's memory into current process. Three data structures are introduced here:
- MmapRegion: mmap a continous range of guest's physical memory into current and provide methods to access the mmapped memory.
- GuestRegionMmap: a wrapper structure to map guest physical address into (mmap\_region, offset) tuple.
- GuestMemoryMmap: manage a collection of GuestRegionMmap objects for a virtual machine.
Provide a default and sample implementation of the GuestMemory trait by mmapping VM's physical memory into current process. Three data structures are defined here:
- MmapRegion: mmap a continous range of VM's physical memory into current and provide methods to access the mmapped memory.
- GuestRegionMmap: a wrapper structure to map VM's physical address into (mmap\_region, offset) tuple.
- GuestMemoryMmap: manage a collection of GuestRegionMmap objects for a VM.

One of the main responsibilities of the GuestMemoryMmap object is to handle the use cases where an access request crosses the memory region boundary. This scenario may be triggered when memory hotplug is supported. So there's a tradeoff between functionality code and complexity:
One of the main responsibilities of the GuestMemoryMmap object is to handle the use cases where an access request crosses the memory region boundary. This scenario may be triggered when memory hotplug is supported. So there's a tradeoff between functionality and code complexity:
- use following pattern for simplicity which fails when the request crosses region boundary. It's current default behavior in the crosvm and firecracker project.
```rust
let guest_memory_mmap: GuestMemoryMmap = ...
Expand All @@ -53,13 +55,12 @@ One of the main responsibilities of the GuestMemoryMmap object is to handle the

### Utilities and Helpers
Following utility and helper traits/macros are imported from the [crosvm project](https://chromium.googlesource.com/chromiumos/platform/crosvm/) with minor changes:
- Bytes: Common trait for volatile access to memory. The `Bytes` trait can be parameterized with newtypes that represent addresses, in order to enforce that addresses are used with the right "kind" of volatile memory.
- DataInit: Types for which it is safe to initialize from raw data. A type `T` is `DataInit` if and only if it can be initialized by reading its contents from a byte array. This is generally true for all plain-old-data structs. It is notably not true for any type that includes a reference.
- {Le,Be}\_{16,32,64}: Explicit endian types useful for embedding in structs or reinterpreting data.
- VolatileMemory: Basic implementation of volatile access to memory, implements `Bytes<usize>`.

### Relationship among Traits and Structs
traits:
- Address: AddressValue
- GuestMemoryRegion: Bytes<MemoryRegionAddress, E = Error> (must be implemented)
- GuestMemory: Bytes<GuestAddress> (generic implementation)

Expand All @@ -76,12 +77,13 @@ structs:
# Usage
First, add the following to your `Cargo.toml`:
```toml
memory-model = "0.1"
vm-memory = "0.1"
```
Next, add this to your crate root:
```rust
extern crate memory-model;
extern crate vm-memory;
```

# TODO List
- Import more interfaces from guest\_memory.rs and mmap.rs
- Abstraction layer to seperate VM memory management from VM memory accessor.
- Help needed to refine documentation and usage examples.
2 changes: 1 addition & 1 deletion src/endian.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
//! # Examples
//!
//! ```
//! # use memory_model::*;
//! # use vm_memory::*;
//! let b: Be32 = From::from(3);
//! let l: Le32 = From::from(3);
//!
Expand Down
4 changes: 2 additions & 2 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ pub use endian::*;
pub mod guest_memory;
pub use guest_memory::*;

#[cfg(feature = "memory-backend-mmap")]
#[cfg(feature = "vm-memory-backend-mmap")]
pub mod mmap;
#[cfg(feature = "memory-backend-mmap")]
#[cfg(feature = "vm-memory-backend-mmap")]
pub use mmap::*;

pub mod volatile_memory;
Expand Down
8 changes: 4 additions & 4 deletions src/mmap.rs
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ impl Bytes<MemoryRegionAddress> for GuestRegionMmap {
/// * Write a slice at guest address 0x1200.
///
/// ```
/// # use memory_model::{Bytes, GuestAddress, GuestMemoryMmap};
/// # use vm_memory::{Bytes, GuestAddress, GuestMemoryMmap};
/// # let start_addr = GuestAddress(0x1000);
/// # let mut gm = GuestMemoryMmap::new(&vec![(start_addr, 0x400)]).unwrap();
/// let res = gm.write(&[1,2,3,4,5], GuestAddress(0x1200)).unwrap();
Expand All @@ -202,7 +202,7 @@ impl Bytes<MemoryRegionAddress> for GuestRegionMmap {
/// * Read a slice of length 16 at guestaddress 0x1200.
///
/// ```
/// # use memory_model::{Bytes, GuestAddress, GuestMemoryMmap};
/// # use vm_memory::{Bytes, GuestAddress, GuestMemoryMmap};
/// # let start_addr = GuestAddress(0x1000);
/// # let mut gm = GuestMemoryMmap::new(&vec![(start_addr, 0x400)]).unwrap();
/// let buf = &mut [0u8; 16];
Expand Down Expand Up @@ -235,7 +235,7 @@ impl Bytes<MemoryRegionAddress> for GuestRegionMmap {
/// * Read bytes from /dev/urandom
///
/// ```
/// # use memory_model::{Address, Bytes, GuestAddress, GuestMemoryMmap};
/// # use vm_memory::{Address, Bytes, GuestAddress, GuestMemoryMmap};
/// # use std::fs::File;
/// # use std::path::Path;
/// # let start_addr = GuestAddress(0x1000);
Expand Down Expand Up @@ -268,7 +268,7 @@ impl Bytes<MemoryRegionAddress> for GuestRegionMmap {
/// * Write 128 bytes to /dev/null
///
/// ```
/// # use memory_model::{Address, Bytes, GuestAddress, GuestMemoryMmap};
/// # use vm_memory::{Address, Bytes, GuestAddress, GuestMemoryMmap};
/// # use std::fs::OpenOptions;
/// # let start_addr = GuestAddress(0x1000);
/// # let gm = GuestMemoryMmap::new(&vec![(start_addr, 0x400)]).unwrap();
Expand Down
32 changes: 16 additions & 16 deletions src/volatile_memory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ pub type Result<T> = result::Result<T, Error>;
/// # Examples
///
/// ```
/// # use memory_model::volatile_memory::*;
/// # use vm_memory::volatile_memory::*;
/// # fn get_slice(offset: usize, count: usize) -> Result<()> {
/// let mem_end = calc_offset(offset, count)?;
/// if mem_end > 100 {
Expand Down Expand Up @@ -211,7 +211,7 @@ impl<'a> VolatileSlice<'a> {
/// ```
/// # use std::fs::File;
/// # use std::path::Path;
/// # use memory_model::VolatileMemory;
/// # use vm_memory::VolatileMemory;
/// # fn test_write_null() -> Result<(), ()> {
/// let mut mem = [0u8; 32];
/// let mem_ref = &mut mem[..];
Expand Down Expand Up @@ -243,7 +243,7 @@ impl<'a> VolatileSlice<'a> {
/// # Examples
///
/// ```
/// # use memory_model::VolatileMemory;
/// # use vm_memory::VolatileMemory;
/// # fn test_write_null() -> Result<(), ()> {
/// let mut mem = [0u8; 32];
/// let mem_ref = &mut mem[..];
Expand All @@ -268,7 +268,7 @@ impl<'a> VolatileSlice<'a> {
/// ```
/// # use std::fs::File;
/// # use std::path::Path;
/// # use memory_model::VolatileMemory;
/// # use vm_memory::VolatileMemory;
/// # fn test_write_null() -> Result<(), ()> {
/// let mut mem = [0u8; 32];
/// let mem_ref = &mut mem[..];
Expand Down Expand Up @@ -307,7 +307,7 @@ impl<'a> VolatileSlice<'a> {
/// ```
/// # use std::fs::File;
/// # use std::path::Path;
/// # use memory_model::VolatileMemory;
/// # use vm_memory::VolatileMemory;
/// # fn test_write_null() -> Result<(), ()> {
/// # let mut mem = [0u8; 32];
/// # let mem_ref = &mut mem[..];
Expand All @@ -333,7 +333,7 @@ impl<'a> VolatileSlice<'a> {
/// ```
/// # use std::fs::File;
/// # use std::path::Path;
/// # use memory_model::VolatileMemory;
/// # use vm_memory::VolatileMemory;
/// # fn test_write_null() -> Result<(), ()> {
/// # let mut mem = [0u8; 32];
/// # let mem_ref = &mut mem[..];
Expand All @@ -360,7 +360,7 @@ impl<'a> VolatileSlice<'a> {
/// ```
/// # use std::fs::File;
/// # use std::path::Path;
/// # use memory_model::VolatileMemory;
/// # use vm_memory::VolatileMemory;
/// # fn test_write_null() -> Result<(), ()> {
/// # let mut mem = [0u8; 32];
/// # let mem_ref = &mut mem[..];
Expand All @@ -386,7 +386,7 @@ impl<'a> VolatileSlice<'a> {
/// ```
/// # use std::fs::File;
/// # use std::path::Path;
/// # use memory_model::VolatileMemory;
/// # use vm_memory::VolatileMemory;
/// # fn test_write_null() -> Result<(), ()> {
/// # let mut mem = [0u8; 32];
/// # let mem_ref = &mut mem[..];
Expand Down Expand Up @@ -423,7 +423,7 @@ impl<'a> Bytes<usize> for VolatileSlice<'a> {
/// * Write a slice at offset 256.
///
/// ```
/// # use memory_model::{Bytes, VolatileMemory};
/// # use vm_memory::{Bytes, VolatileMemory};
/// # let mut mem = [0u8; 1024];
/// # let mut mem_ref = &mut mem[..];
/// # let vslice = mem_ref.as_volatile_slice();
Expand Down Expand Up @@ -452,7 +452,7 @@ impl<'a> Bytes<usize> for VolatileSlice<'a> {
/// * Read a slice of size 16 at offset 256.
///
/// ```
/// # use memory_model::{Bytes, VolatileMemory};
/// # use vm_memory::{Bytes, VolatileMemory};
/// # let mut mem = [0u8; 1024];
/// # let mut mem_ref = &mut mem[..];
/// # let vslice = mem_ref.as_volatile_slice();
Expand Down Expand Up @@ -480,7 +480,7 @@ impl<'a> Bytes<usize> for VolatileSlice<'a> {
/// * Write a slice at offset 256.
///
/// ```
/// # use memory_model::{Bytes, VolatileMemory};
/// # use vm_memory::{Bytes, VolatileMemory};
/// # let mut mem = [0u8; 1024];
/// # let mut mem_ref = &mut mem[..];
/// # let vslice = mem_ref.as_volatile_slice();
Expand All @@ -505,7 +505,7 @@ impl<'a> Bytes<usize> for VolatileSlice<'a> {
/// * Read a slice of size 16 at offset 256.
///
/// ```
/// # use memory_model::{Bytes, VolatileMemory};
/// # use vm_memory::{Bytes, VolatileMemory};
/// # let mut mem = [0u8; 1024];
/// # let mut mem_ref = &mut mem[..];
/// # let vslice = mem_ref.as_volatile_slice();
Expand All @@ -532,7 +532,7 @@ impl<'a> Bytes<usize> for VolatileSlice<'a> {
/// * Read bytes from /dev/urandom
///
/// ```
/// # use memory_model::{Bytes, VolatileMemory};
/// # use vm_memory::{Bytes, VolatileMemory};
/// # use std::fs::File;
/// # use std::path::Path;
/// # fn test_read_random() -> Result<u32, ()> {
Expand Down Expand Up @@ -567,7 +567,7 @@ impl<'a> Bytes<usize> for VolatileSlice<'a> {
/// * Write 128 bytes to /dev/null
///
/// ```
/// # use memory_model::{Bytes, VolatileMemory};
/// # use vm_memory::{Bytes, VolatileMemory};
/// # use std::fs::File;
/// # use std::path::Path;
/// # fn test_write_null() -> Result<(), ()> {
Expand Down Expand Up @@ -615,7 +615,7 @@ impl<'a> VolatileMemory for VolatileSlice<'a> {
/// # Examples
///
/// ```
/// # use memory_model::VolatileRef;
/// # use vm_memory::VolatileRef;
/// let mut v = 5u32;
/// assert_eq!(v, 5);
/// let v_ref = unsafe { VolatileRef::new(&mut v as *mut u32) };
Expand Down Expand Up @@ -656,7 +656,7 @@ impl<'a, T: DataInit> VolatileRef<'a, T> {
///
/// ```
/// # use std::mem::size_of;
/// # use memory_model::VolatileRef;
/// # use vm_memory::VolatileRef;
/// let v_ref = unsafe { VolatileRef::new(0 as *mut u32) };
/// assert_eq!(v_ref.len(), size_of::<u32>() as usize);
/// ```
Expand Down

0 comments on commit 83f6111

Please sign in to comment.