ANE support #159

Change the error type of the constructors of `Arc` and `UniqueArc` to be `AllocError` instead of `Error`. This makes the API more clear as to what can go wrong when calling `try_new` or its variants. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Gary Guo <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This function mirrors `UnsafeCell::raw_get`. It avoids creating a reference and allows solely using raw pointers. The `pin-init` API will be using this, since uninitialized memory requires raw pointers. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This API is used to facilitate safe pinned initialization of structs. It replaces cumbersome `unsafe` manual initialization with elegant safe macro invocations. Due to the size of this change it has been split into six commits: 1. This commit introducing the basic public interface: traits and functions to represent and create initializers. 2. Adds the `#[pin_data]`, `pin_init!`, `try_pin_init!`, `init!` and `try_init!` macros along with their internal types. 3. Adds the `InPlaceInit` trait that allows using an initializer to create an object inside of a `Box<T>` and other smart pointers. 4. Adds the `PinnedDrop` trait and adds macro support for it in the `#[pin_data]` macro. 5. Adds the `stack_pin_init!` macro allowing to pin-initialize a struct on the stack. 6. Adds the `Zeroable` trait and `init::zeroed` function to initialize types that have `0x00` in all bytes as a valid bit pattern. -- In this section the problem that the new pin-init API solves is outlined. This message describes the entirety of the API, not just the parts introduced in this commit. For a more granular explanation and additional information on pinning and this issue, view [1]. Pinning is Rust's way of enforcing the address stability of a value. When a value gets pinned it will be impossible for safe code to move it to another location. This is done by wrapping pointers to said object with `Pin<P>`. This wrapper prevents safe code from creating mutable references to the object, preventing mutable access, which is needed to move the value. `Pin<P>` provides `unsafe` functions to circumvent this and allow modifications regardless. It is then the programmer's responsibility to uphold the pinning guarantee. Many kernel data structures require a stable address, because there are foreign pointers to them which would get invalidated by moving the structure. Since these data structures are usually embedded in structs to use them, this pinning property propagates to the container struct. Resulting in most structs in both Rust and C code needing to be pinned. So if we want to have a `mutex` field in a Rust struct, this struct also needs to be pinned, because a `mutex` contains a `list_head`. Additionally initializing a `list_head` requires already having the final memory location available, because it is initialized by pointing it to itself. But this presents another challenge in Rust: values have to be initialized at all times. There is the `MaybeUninit<T>` wrapper type, which allows handling uninitialized memory, but this requires using the `unsafe` raw pointers and a casting the type to the initialized variant. This problem gets exacerbated when considering encapsulation and the normal safety requirements of Rust code. The fields of the Rust `Mutex<T>` should not be accessible to normal driver code. After all if anyone can modify the fields, there is no way to ensure the invariants of the `Mutex<T>` are upheld. But if the fields are inaccessible, then initialization of a `Mutex<T>` needs to be somehow achieved via a function or a macro. Because the `Mutex<T>` must be pinned in memory, the function cannot return it by value. It also cannot allocate a `Box` to put the `Mutex<T>` into, because that is an unnecessary allocation and indirection which would hurt performance. The solution in the rust tree (e.g. this commit: [2]) that is replaced by this API is to split this function into two parts: 1. A `new` function that returns a partially initialized `Mutex<T>`, 2. An `init` function that requires the `Mutex<T>` to be pinned and that fully initializes the `Mutex<T>`. Both of these functions have to be marked `unsafe`, since a call to `new` needs to be accompanied with a call to `init`, otherwise using the `Mutex<T>` could result in UB. And because calling `init` twice also is not safe. While `Mutex<T>` initialization cannot fail, other structs might also have to allocate memory, which would result in conditional successful initialization requiring even more manual accommodation work. Combine this with the problem of pin-projections -- the way of accessing fields of a pinned struct -- which also have an `unsafe` API, pinned initialization is riddled with `unsafe` resulting in very poor ergonomics. Not only that, but also having to call two functions possibly multiple lines apart makes it very easy to forget it outright or during refactoring. Here is an example of the current way of initializing a struct with two synchronization primitives (see [3] for the full example): struct SharedState { state_changed: CondVar, inner: Mutex<SharedStateInner>, } impl SharedState { fn try_new() -> Result<Arc<Self>> { let mut state = Pin::from(UniqueArc::try_new(Self { // SAFETY: `condvar_init!` is called below. state_changed: unsafe { CondVar::new() }, // SAFETY: `mutex_init!` is called below. inner: unsafe { Mutex::new(SharedStateInner { token_count: 0 }) }, })?); // SAFETY: `state_changed` is pinned when `state` is. let pinned = unsafe { state.as_mut().map_unchecked_mut(|s| &mut s.state_changed) }; kernel::condvar_init!(pinned, "SharedState::state_changed"); // SAFETY: `inner` is pinned when `state` is. let pinned = unsafe { state.as_mut().map_unchecked_mut(|s| &mut s.inner) }; kernel::mutex_init!(pinned, "SharedState::inner"); Ok(state.into()) } } The pin-init API of this patch solves this issue by providing a comprehensive solution comprised of macros and traits. Here is the example from above using the pin-init API: #[pin_data] struct SharedState { #[pin] state_changed: CondVar, #[pin] inner: Mutex<SharedStateInner>, } impl SharedState { fn new() -> impl PinInit<Self> { pin_init!(Self { state_changed <- new_condvar!("SharedState::state_changed"), inner <- new_mutex!( SharedStateInner { token_count: 0 }, "SharedState::inner", ), }) } } Notably the way the macro is used here requires no `unsafe` and thus comes with the usual Rust promise of safe code not introducing any memory violations. Additionally it is now up to the caller of `new()` to decide the memory location of the `SharedState`. They can choose at the moment `Arc<T>`, `Box<T>` or the stack. -- The API has the following architecture: 1. Initializer traits `PinInit<T, E>` and `Init<T, E>` that act like closures. 2. Macros to create these initializer traits safely. 3. Functions to allow manually writing initializers. The initializers (an `impl PinInit<T, E>`) receive a raw pointer pointing to uninitialized memory and their job is to fully initialize a `T` at that location. If initialization fails, they return an error (`E`) by value. This way of initializing cannot be safely exposed to the user, since it relies upon these properties outside of the control of the trait: - the memory location (slot) needs to be valid memory, - if initialization fails, the slot should not be read from, - the value in the slot should be pinned, so it cannot move and the memory cannot be deallocated until the value is dropped. This is why using an initializer is facilitated by another trait that ensures these requirements. These initializers can be created manually by just supplying a closure that fulfills the same safety requirements as `PinInit<T, E>`. But this is an `unsafe` operation. To allow safe initializer creation, the `pin_init!` is provided along with three other variants: `try_pin_init!`, `try_init!` and `init!`. These take a modified struct initializer as a parameter and generate a closure that initializes the fields in sequence. The macros take great care in upholding the safety requirements: - A shadowed struct type is used as the return type of the closure instead of `()`. This is to prevent early returns, as these would prevent full initialization. - To ensure every field is only initialized once, a normal struct initializer is placed in unreachable code. The type checker will emit errors if a field is missing or specified multiple times. - When initializing a field fails, the whole initializer will fail and automatically drop fields that have been initialized earlier. - Only the correct initializer type is allowed for unpinned fields. You cannot use a `impl PinInit<T, E>` to initialize a structurally not pinned field. To ensure the last point, an additional macro `#[pin_data]` is needed. This macro annotates the struct itself and the user specifies structurally pinned and not pinned fields. Because dropping a pinned struct is also not allowed to break the pinning invariants, another macro attribute `#[pinned_drop]` is needed. This macro is introduced in a following commit. These two macros also have mechanisms to ensure the overall safety of the API. Additionally, they utilize a combined proc-macro, declarative macro design: first a proc-macro enables the outer attribute syntax `#[...]` and does some important pre-parsing. Notably this prepares the generics such that the declarative macro can handle them using token trees. Then the actual parsing of the structure and the emission of code is handled by a declarative macro. For pin-projections the crates `pin-project` [4] and `pin-project-lite` [5] had been considered, but were ultimately rejected: - `pin-project` depends on `syn` [6] which is a very big dependency, around 50k lines of code. - `pin-project-lite` is a more reasonable 5k lines of code, but contains a very complex declarative macro to parse generics. On top of that it would require modification that would need to be maintained independently. Link: https://rust-for-linux.com/the-safe-pinned-initialization-problem [1] Link: https://github.com/Rust-for-Linux/linux/tree/0a04dc4ddd671efb87eef54dde0fb38e9074f4be [2] Link: https://github.com/Rust-for-Linux/linux/blob/f509ede33fc10a07eba3da14aa00302bd4b5dddd/samples/rust/rust_miscdev.rs [3] Link: https://crates.io/crates/pin-project [4] Link: https://crates.io/crates/pin-project-lite [5] Link: https://crates.io/crates/syn [6] Co-developed-by: Gary Guo <[email protected]> Signed-off-by: Gary Guo <[email protected]> Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Wedson Almeida Filho <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

Add the following initializer macros: - `#[pin_data]` to annotate structurally pinned fields of structs, needed for `pin_init!` and `try_pin_init!` to select the correct initializer of fields. - `pin_init!` create a pin-initializer for a struct with the `Infallible` error type. - `try_pin_init!` create a pin-initializer for a struct with a custom error type (`kernel::error::Error` is the default). - `init!` create an in-place-initializer for a struct with the `Infallible` error type. - `try_init!` create an in-place-initializer for a struct with a custom error type (`kernel::error::Error` is the default). Also add their needed internal helper traits and structs. Co-developed-by: Gary Guo <[email protected]> Signed-off-by: Gary Guo <[email protected]> Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] [ Fixed three typos. ] Signed-off-by: Miguel Ojeda <[email protected]>

…ters The `InPlaceInit` trait that provides two functions, for initializing using `PinInit<T, E>` and `Init<T>`. It is implemented by `Arc<T>`, `UniqueArc<T>` and `Box<T>`. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

The `PinnedDrop` trait that facilitates destruction of pinned types. It has to be implemented via the `#[pinned_drop]` macro, since the `drop` function should not be called by normal code, only by other destructors. It also only works on structs that are annotated with `#[pin_data(PinnedDrop)]`. Co-developed-by: Gary Guo <[email protected]> Signed-off-by: Gary Guo <[email protected]> Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

The `stack_pin_init!` macro allows pin-initializing a value on the stack. It accepts a `impl PinInit<T, E>` to initialize a `T`. It allows propagating any errors via `?` or handling it normally via `match`. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Reviewed-by: Gary Guo <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

Add the `Zeroable` trait which marks types that can be initialized by writing `0x00` to every byte of the type. Also add the `init::zeroed` function that creates an initializer for a `Zeroable` type that writes `0x00` to every byte. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

Add `pin-init` API macros and traits to the prelude. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This function allows to easily initialize `Opaque` with the pin-init API. `Opaque::ffi_init` takes a closure and returns a pin-initializer. This pin-initiailizer calls the given closure with a pointer to the inner `T`. Co-developed-by: Gary Guo <[email protected]> Signed-off-by: Gary Guo <[email protected]> Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Link: https://lore.kernel.org/r/[email protected] [ Fixed typo. ] Signed-off-by: Miguel Ojeda <[email protected]>

`UniqueArc::try_new_uninit` calls `Arc::try_new(MaybeUninit::uninit())`. This results in the uninitialized memory being placed on the stack, which may be arbitrarily large due to the generic `T` and thus could cause a stack overflow for large types. Change the implementation to use the pin-init API which enables in-place initialization. In particular it avoids having to first construct and then move the uninitialized memory from the stack into the final location. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

Add two functions `init_with` and `pin_init_with` to `UniqueArc<MaybeUninit<T>>` to initialize the memory of already allocated `UniqueArc`s. This is useful when you want to allocate memory check some condition inside of a context where allocation is forbidden and then conditionally initialize an object. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Reviewed-by: Andreas Hindborg <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This makes it possible to use `T` as a `impl Init<T, E>` for every error type `E` instead of just `Infallible`. Signed-off-by: Benno Lossin <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

Benno has been involved with the Rust for Linux project for the better part of a year now. He has been working on solving the safe pinned initialization problem [1], which resulted in the pin-init API patch series [2] that allows to reduce the need for `unsafe` code in the kernel. He is also working on the field projection RFC for Rust [3] to bring pin-init as a language feature. His expertise with the language will be very useful to have around in the future if Rust grows within the kernel, thus add him to the `RUST` entry as reviewer. Link: https://rust-for-linux.com/the-safe-pinned-initialization-problem [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: rust-lang/rfcs#3318 [3] Cc: Benno Lossin <[email protected]> Reviewed-by: Alex Gaynor <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

It is a wrapper around C's `lock_class_key`, which is used by the synchronisation primitives that are checked with lockdep. This is in preparation for introducing Rust abstractions for these primitives. Cc: Peter Zijlstra <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Will Deacon <[email protected]> Cc: Waiman Long <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Co-developed-by: Boqun Feng <[email protected]> Signed-off-by: Boqun Feng <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Benno Lossin <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

They are generic Rust implementations of a lock and a lock guard that contain code that is common to all locks. Different backends will be introduced in subsequent commits. Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Suggested-by: Gary Guo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Link: https://lore.kernel.org/r/[email protected] [ Fixed typo. ] Signed-off-by: Miguel Ojeda <[email protected]>

This is the `struct mutex` lock backend and allows Rust code to use the kernel mutex idiomatically. Cc: Peter Zijlstra <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Will Deacon <[email protected]> Cc: Waiman Long <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This is the `spinlock_t` lock backend and allows Rust code to use the kernel spinlock idiomatically. Cc: Peter Zijlstra <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Will Deacon <[email protected]> Cc: Waiman Long <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This is an owned reference to an object that is always ref-counted. This is meant to be used in wrappers for C types that have their own ref counting functions, for example, tasks, files, inodes, dentries, etc. Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Reviewed-by: Gary Guo <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

It is an abstraction for C's `struct task_struct`. It implements `AlwaysRefCounted`, so the refcount of the wrapped object is managed safely on the Rust side. Cc: Ingo Molnar <[email protected]> Cc: Peter Zijlstra <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This allows Rust code to get a reference to the current task without having to increment the refcount, but still guaranteeing memory safety. Cc: Ingo Molnar <[email protected]> Cc: Peter Zijlstra <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This allows us to have data protected by a lock despite not being wrapped by it. Access is granted by providing evidence that the lock is held by the caller. Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Reviewed-by: Benno Lossin <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

It releases the lock, executes some function provided by the caller, then reacquires the lock. This is preparation for the implementation of condvars, which will sleep after between unlocking and relocking. We need an explicit `relock` method for primitives like `SpinLock` that have an irqsave variant: we use the guard state to determine if the lock was originally acquired with the regular `lock` function or `lock_irqsave`. Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [ Removed the irqsave bits as discussed. ] Signed-off-by: Miguel Ojeda <[email protected]>

This is the traditional condition variable or monitor synchronisation primitive. It is implemented with C's `wait_queue_head_t`. It allows users to release a lock and go to sleep while guaranteeing that notifications won't be missed. This is achieved by enqueuing a wait entry before releasing the lock. Cc: Peter Zijlstra <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Will Deacon <[email protected]> Cc: Waiman Long <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Reviewed-by: Alice Ryhl <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

This crate mirrors the `bindings` crate, but will contain only UAPI bindings. Unlike the bindings crate, drivers may directly use this crate if they have to interface with userspace. Initially, just bind the generic ioctl stuff. In the future, we would also like to add additional checks to ensure that all types exposed by this crate satisfy UAPI-safety guarantees (that is, they are safely castable to/from a "bag of bits"). [ Miguel: added support for the `rustdoc` and `rusttest` targets, since otherwise they fail, and we want to keep them working. ] Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Signed-off-by: Asahi Lina <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Miguel Ojeda <[email protected]>

Add simple 1:1 wrappers of the C ioctl number manipulation functions. Since these are macros we cannot bindgen them directly, and since they should be usable in const context we cannot use helper wrappers, so we'll have to reimplement them in Rust. Thankfully, the C headers do declare defines for the relevant bitfield positions, so we don't need to duplicate that. Signed-off-by: Asahi Lina <[email protected]> Link: https://lore.kernel.org/r/[email protected] [ Moved the `#![allow(non_snake_case)]` to the usual place. ] Signed-off-by: Miguel Ojeda <[email protected]>

This commit provides the build flags for Rust for AArch64. The core Rust support already in the kernel does the rest. The Rust samples have been tested with this commit. [jcunliffe: Arm specific parts taken from Miguel's upstream tree] Signed-off-by: Miguel Ojeda <[email protected]> Co-developed-by: Jamie Cunliffe <[email protected]> Signed-off-by: Jamie Cunliffe <[email protected]> Signed-off-by: Vincenzo Palazzo <[email protected]> Reviewed-by: Vincenzo Palazzo <[email protected]> Reviewed-by: Gary Guo <[email protected]>

Enable the PAC ret and BTI options in the Rust build flags to match the options that are used when building C. Signed-off-by: Jamie Cunliffe <[email protected]> Reviewed-by: Vincenzo Palazzo <[email protected]>

Disable the neon and fp target features to avoid fp & simd registers. The use of fp-armv8 will cause a warning from rustc about an unknown feature that is specified. The target feature is still passed through to LLVM, this behaviour is documented as part of the warning. This will be fixed in a future version of the rustc toolchain. Signed-off-by: Jamie Cunliffe <[email protected]> Reviewed-by: Vincenzo Palazzo <[email protected]>

This module is intended to contain functions related to kernel timekeeping and time. Initially, this just wraps ktime_get() and ktime_get_boottime() and returns them as core::time::Duration instances. Signed-off-by: Asahi Lina <[email protected]>

The XArray is an abstract data type which behaves like a very large array of pointers. Add a Rust abstraction for this data type. The initial implementation uses explicit locking on get operations and returns a guard which blocks mutation, ensuring that the referenced object remains alive. To avoid excessive serialization, users are expected to use an inner type that can be efficiently cloned (such as Arc<T>), and eagerly clone and drop the guard to unblock other users after a lookup. Future variants may support using RCU instead to avoid mutex locking. This abstraction also introduces a reservation mechanism, which can be used by alloc-capable XArrays to reserve a free slot without immediately filling it, and then do so at a later time. If the reservation is dropped without being filled, the slot is freed again for other users, which eliminates the need for explicit cleanup code. Signed-off-by: Asahi Lina <[email protected]> --- Hi everyone! This abstraction is part of the set of dependencies for the drm/asahi Apple M1/M2 GPU driver. The branch at [1] contains the full series of patches rebased on upstream leading to the complete driver, for reference on how it is intended to be used. Thank you everyone who helped review this on GitHub [2]! I hope I didn't forget any CCs... Note that I dropped the convenience `Deref` impl for `Guard`, since I couldn't figure out how to do it safely. Suggestions welcome, or we can leave it for a future improvement ^^ [1] https://github.com/AsahiLinux/linux/tree/gpu/rebase-20230224 [2] Rust-for-Linux#952 Changes in v3: - Updated to the error v2/v3 series API. - Renamed `err` to `ret` for consistency with the other instance. - Link to v2: https://lore.kernel.org/r/[email protected] Changes in v2: - Added Pin requirement for all XArray operations, to close a soundness hole due to the lock in the XArray (locks are not safe to move while locked). Creation does not require pinning in place, since the lock cannot be acquired at that point. - Added safety note to Drop impl about why we don't need to do the lock unlock dance to ensure soundness in case of a dropped lock guard. - Downstream drm/asahi driver was also rebased on this version to prove it works (previously it was still on a pre-v1 version). - This still depends on the Error series (v1). v2 of that will need a trivial rename of Error::from_kernel_errno -> Error::from_errno. If this version of XArray ends up looking good, I'll send a trivial v4 of XArray with the rename, after sending the v2 of the Error series. - Link to v1: https://lore.kernel.org/r/[email protected]

Some traits exposed by the kernel crate may not be intended to be implemented by downstream modules. Add a Sealed trait to allow avoiding this using the sealed trait pattern. Signed-off-by: Asahi Lina <[email protected]>

Add a RawDevice trait which can be implemented by any type representing a device class (such as a PlatformDevice), and a Device type which represents an owned reference to a generic struct device. Lina: Rewrote commit message, dropped the Amba bits, and squashed in simple changes to the core RawDevice/Device code from latter commits in rust-for-linux/rust. Also include the rust_helper_dev_get_drvdata helper which will be needed by consumers later on. Co-developed-by: Miguel Ojeda <[email protected]> Signed-off-by: Miguel Ojeda <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Signed-off-by: Asahi Lina <[email protected]>

The io_pgtable subsystem implements page table management for various IOMMU page table formats. This abstraction allows Rust drivers for devices with an embedded MMU to use this shared code. Signed-off-by: Asahi Lina <[email protected]>

RTKit is Apple's proprietary real-time operating system framework, used across many subdevices on Apple Silicon platforms including NVMe, system management, GPU, etc. Add Rust abstractions for this subsystem, so that it can be used by upcoming Rust drivers. Note: Although ARM64 support is not yet merged, this can be built on amd64 with CONFIG_COMPILE_TEST=y. Signed-off-by: Asahi Lina <[email protected]>

Add Rust counterparts to these C macros. `container_of` is useful for C struct subtyping, to recover the original pointer to the container structure. `offset_of` is useful for struct-relative addressing. Lina: Rewrote commit message and squashed in later changes from rust-for-linux/linux, attributed below. Co-authored-by: Miguel Ojeda <[email protected]> Signed-off-by: Miguel Ojeda <[email protected]> Co-authored:by: Léo Lanteri Thauvin <[email protected]> Signed-off-by: Léo Lanteri Thauvin <[email protected]> Co-authored-by: Wedson Almeida Filho <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Signed-off-by: Asahi Lina <[email protected]>

Commit reference: 3dfc5eb

Computed based on the PAGE_SHIFT macro from C. // Co-authored-by: Adam Bratschi-Kaye <[email protected]> // Co-authored-by: Miguel Ojeda <[email protected]> // Signed-off-by: Asahi Lina <[email protected]>

Needed by the rust-for-linux/rust module_param module. Signed-off-by: Asahi Lina <[email protected]>

Commit reference: 3dfc5eb

Upgrade the Rust version from 1.62.0 to 1.66.0. The overwhelming majority of the commit is about upgrading our `alloc` fork to the new version from upstream [1]. License has not changed [2][3] (there were changes in the `COPYRIGHT` file, but unrelated to `alloc`). As in the previous version upgrades (done out of tree so far), upgrading `alloc` requires checking that our small additions (`try_*`) still match their original (non`-try_*`) versions. With this version upgrade, the following unstable Rust features were stabilized: `bench_black_box` (1.66.0), `const_ptr_offset_from` (1.65.0), `core_ffi_c` (1.64.0) and `generic_associated_types` (1.65.0). Thus remove them. This also implies that only two unstable features remain allowed for non-`rust/` code: `allocator_api` and `const_refs_to_cell`. There are some new Clippy warnings that we are triggering (i.e. introduced since 1.62.0), as well as a few others that were not triggered before, thus allow them in this commit and clean up or remove them as needed later on: - `borrow_deref_ref` (new in 1.63.0). - `explicit_auto_deref` (new in 1.64.0). - `bool_to_int_with_if` (new in 1.65.0). - `needless_borrow`. - `type_complexity`. - `unnecessary_cast` (allowed only on `CONFIG_ARM`). Furthermore, `rustdoc` lint `broken_intra_doc_links` is triggering on links pointing to `macro_export` `macro_rules` defined in the same module (i.e. appearing in the crate root). However, even if the warning appears, the link still gets generated like in previous versions, thus it is a bit confusing. An issue has been opened upstream [4], and it appears that the link still being generated was a compatibility measure, thus we will need to adapt to the new behavior (done in the next patch). In addition, there is an added `#[const_trait]` attribute in `RawDeviceId`, due to 1.66.0's PR "Require `#[const_trait]` on `Trait` for `impl const Trait`") [5]. Finally, the `-Aunused-imports` was added for compiling `core`. This was fixed upstream for 1.67.0 in PR "Fix warning when libcore is compiled with no_fp_fmt_parse" [6], and prevented for the future for that `cfg` in PR "core: ensure `no_fp_fmt_parse builds` are warning-free" [7]. Reviewed-by: Björn Roy Baron <[email protected]> Reviewed-by: Martin Rodriguez Reboredo <[email protected]> Tested-by: Martin Rodriguez Reboredo <[email protected]> Reviewed-by: Gary Guo <[email protected]> Reviewed-by: Vincenzo Palazzo <[email protected]> Reviewed-by: Alice Ferrazzi <[email protected]> Tested-by: Alice Ferrazzi <[email protected]> Reviewed-by: Neal Gompa <[email protected]> Tested-by: Neal Gompa <[email protected]> Link: Rust-for-Linux#947 Link: https://github.com/rust-lang/rust/tree/1.66.0/library/alloc/src [1] Link: https://github.com/rust-lang/rust/blob/1.66.0/library/alloc/Cargo.toml#L4 [2] Link: https://github.com/rust-lang/rust/blob/1.66.0/COPYRIGHT [3] Link: rust-lang/rust#106142 [4] Link: rust-lang/rust#100982 [5] Link: rust-lang/rust#105434 [6] Link: rust-lang/rust#105811 [7] Signed-off-by: Miguel Ojeda <[email protected]>

Co-developed-by: Wedson Almeida Filho <[email protected]> Signed-off-by: Wedson Almeida Filho <[email protected]> Signed-off-by: Asahi Lina <[email protected]>

Commit reference: 3dfc5eb

Commit reference: 3dfc5eb Signed-off-by: Asahi Lina <[email protected]>

Commit reference: 3dfc5eb

This abstraction enables Rust drivers to walk Device Tree nodes and query their properties. Signed-off-by: Asahi Lina <[email protected]>

…ation In order for modpost to work and correctly generate module aliases from device ID tables, it needs those tables to exist as global symbols with a specific name. Additionally, modpost checks the size of the symbol, so it cannot contain trailing data. To support this, split IdArrayIds out of IdArray. The former contains just the IDs. Then split out the device table definition macro from the macro that defines the device table for a given bus driver, and add another macro to declare a device table as a module device table. Drivers can now define their ID table once, and then specify that it should be used for both the driver and the module: // Generic OF Device ID table. kernel::define_of_id_table! {ASAHI_ID_TABLE, &'static hw::HwConfig, [ (of::DeviceId::Compatible(b"apple,agx-t8103"), Some(&hw::t8103::HWCONFIG)), (of::DeviceId::Compatible(b"apple,agx-t8112"), Some(&hw::t8112::HWCONFIG)), // ... ]} /// Platform Driver implementation for `AsahiDriver`. impl platform::Driver for AsahiDriver { /// Data associated with each hardware ID. type IdInfo = &'static hw::HwConfig; // Assign the above OF ID table to this driver. kernel::driver_of_id_table!(ASAHI_ID_TABLE); // ... } // Export the OF ID table as a module ID table, to make modpost/autoloading work. kernel::module_of_id_table!(MOD_TABLE, ASAHI_ID_TABLE); Signed-off-by: Asahi Lina <[email protected]>

This patch adds a logic similar to `devm_platform_ioremap_resource` function adding: - `IoResource` enumerated type that groups the `IORESOURCE_*` macros. - `get_resource()` method that is a binding of `platform_get_resource` - `ioremap_resource` that is newly written method similar to `devm_platform_ioremap_resource`. Lina: Removed `bit` dependency and rebased Co-developed-by: Asahi Lina <[email protected]> Signed-off-by: Maciej Falkowski <[email protected]>

Allows drivers to configure the DMA masks for a device. Implemented here, not in device, because it requires a mutable platform device reference this way (device::Device is a safely clonable reference). Signed-off-by: Asahi Lina <[email protected]>

Apple SoCs require non-posted mappings for MMIO, and this is automatically handled by devm_ioremap_resource() and friends via a resource flag. Implement the same logic in kernel::io_mem, so it can work the same way. Signed-off-by: Asahi Lina <[email protected]>

Commit reference: 3dfc5eb

TODO: This isn't abstracted properly yet Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Commit reference: 3dfc5eb

Apple Silicon SoCs (M1, M2, etc.) have a GPU with an ARM64 firmware coprocessor. The firmware and the GPU share page tables in the standard ARM64 format (the firmware literally sets the base as its TTBR0/1 registers). TTBR0 covers the low half of the address space and is intended to be per-GPU-VM (GPU user mappings and kernel-managed buffers), while TTBR1 covers the upper half and is global (firmware code, data, management structures shared with the AP, and a few GPU-accessible data structures). In typical Apple fashion, the permissions are interpreted differently from traditional ARM PTEs. By default, firmware mappings use Apple SPRR permission remapping. The firmware only uses that for its own code/data/MMIO mappings, and those pages are not accessible by the GPU hardware. We never need to touch/manage these mappings, so this patch does not support them. When a specific bit is set in the PTEs, permissions switch to a different scheme which supports various combinations of firmware/GPU access. This is the mode intended to be used by AP GPU drivers, and what we implement here. The prot bits are interpreted as follows: - IOMMU_READ and IOMMU_WRITE have the usual meaning. - IOMMU_PRIV creates firmware-only mappings (no GPU access) - IOMMU_NOEXEC creates GPU-only structures (no FW access) - Otherwise structures are accessible by both GPU and FW - IOMMU_MMIO creates Device mappings for firmware - IOMMU_CACHE creates Normal-NC mappings for firmware (cache-coherent from the point of view of the AP, but slower) - Otherwise creates Normal mappings for firmware (this requires manual cache management on the firmware side, as it is not coherent with the SoC fabric) GPU-only mappings (textures/etc) are expected to use IOMMU_CACHE and are seemingly coherent with the CPU (or otherwise the firmware/GPU already issue the required cache management operations when correctly configured). There is a GPU-RO/FW-RW mode, but it is not currently implemented (it doesn't seem to be very useful for the driver). There seems to be no real noexec control (i.e. for shaders) on the GPU side. All of these mappings are implicitly noexec for the firmware. Drivers are expected to fully manage per-user (TTBR0) page tables, but ownership of shared kernel (TTBR1) page tables is shared between the firmware and the AP OS. We handle this by simply using a smaller IAS to drop down one level of page tables, so the driver can install a PTE in the top-level (firmware-initialized) page table directly and just add an offset to the VAs passed into the io_pgtable code. This avoids having to have any special handling for this here. The firmware-relevant data structures are small, so we do not expect to ever require more VA space than one top-level PTE covers (IAS=36 for the next level, 64 GiB). Only 16K page mode is supported. The coprocessor MMU supports huge pages as usual for ARM64, but the GPU MMU does not, so we do not enable them. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

DRM drivers need to be able to declare which driver-specific ioctls they support. This abstraction adds the required types and a helper macro to generate the ioctl definition inside the DRM driver. Note that this macro is not usable until further bits of the abstraction are in place (but it will not fail to compile on its own, if not called). Signed-off-by: Asahi Lina <[email protected]>

Add the initial abstractions for DRM drivers and devices. These go together in one commit since they are fairly tightly coupled types. A few things have been stubbed out, to be implemented as further bits of the DRM subsystem are introduced. Signed-off-by: Asahi Lina <[email protected]>

A DRM File is the DRM counterpart to a kernel file structure, representing an open DRM file descriptor. Add a Rust abstraction to allow drivers to implement their own File types that implement the DriverFile trait. Signed-off-by: Asahi Lina <[email protected]>

The DRM GEM subsystem is the DRM memory management subsystem used by most modern drivers. Add a Rust abstraction to allow Rust DRM driver implementations to use it. Signed-off-by: Asahi Lina <[email protected]>

There doesn't seem to be a way for the Rust bindings to get a compile-time constant reference to drm_gem_shmem_vm_ops, so we need to duplicate that structure in Rust... this isn't nice... Signed-off-by: Asahi Lina <[email protected]>

The DRM shmem helper includes common code useful for drivers which allocate GEM objects as anonymous shmem. Add a Rust abstraction for this. Drivers can choose the raw GEM implementation or the shmem layer, depending on their needs. Signed-off-by: Asahi Lina <[email protected]>

drm_mm provides a simple range allocator, useful for managing virtual address ranges. Add a Rust abstraction to expose this module to Rust drivers. Signed-off-by: Asahi Lina <[email protected]>

DMA fences are the internal synchronization primitive used for DMA operations like GPU rendering, video en/decoding, etc. Add an abstraction to allow Rust drivers to interact with this subsystem. Note: This uses a raw spinlock living next to the fence, since we do not interact with it other than for initialization. TODO: Expose this to the user at some point with a safe abstraction. Signed-off-by: Asahi Lina <[email protected]>

DRM Sync Objects are a container for a DMA fence, and can be waited on signaled, exported, and imported from userspace. Add a Rust abstraction so Rust DRM drivers can support this functionality. Signed-off-by: Asahi Lina <[email protected]>

Some hardware may require more complex resource utilization accounting than the simple job count supported by drm_sched internally. Add a can_run_job callback to allow drivers to implement more logic before deciding whether to run a GPU job. Signed-off-by: Asahi Lina <[email protected]>

The GPU scheduler manages scheduling GPU jobs and dependencies between them. This Rust abstraction allows Rust DRM drivers to use this functionality. Signed-off-by: Asahi Lina <[email protected]>

Drivers may want to support driver-private objects, which cannot be shared. This allows them to share a single lock and enables other optimizations. Add an `exportable` field to drm_gem_object, which blocks PRIME export if set to false. It is initialized to true in drm_gem_private_object_init. Signed-off-by: Asahi Lina <[email protected]>

This allows drivers to control whether a given GEM object is allowed to be exported via PRIME to other drivers.

Signed-off-by: Asahi Lina <[email protected]>

They could complete in either order, and not forcing flushes on both can leave us with stuck stamps... Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Hector Martin <[email protected]>

drm_sched_fini() currently leaves any pending jobs dangling, which causes segfaults and other badness when job completion fences are signaled after the scheduler is torn down. Explicitly detach all jobs from their completion callbacks and free them. This makes it possible to write a sensible safe abstraction for drm_sched, without having to externally duplicate the tracking of in-flight jobs. This shouldn't regress any existing drivers, since calling drm_sched_fini() with any pending jobs is broken and this change should be a no-op if there are no pending jobs. Signed-off-by: Asahi Lina <[email protected]>

Instead of invalidating the GPU context when the Queue gets dropped (at which point jobs might still be running), wrap the object and put the drop logic in the wrapper, which is inside the Arc<>. Since the WorkQueues stay alive until all commands are complete (via the EventManager reference), that means the GPU context will only be invalidated when there is no work in progress or queued for it and the Arc<> loses its last reference. Signed-off-by: Asahi Lina <[email protected]>

Seen on 12.3: asahi 206400000.gpu: Allocator: Corruption after object of type asahi::fw::fragment::RunFragmentG13V12_3 at 0xffffffa00009be00:0x928 + 0x0..0x5 5 bytes sounds like exactly the two fields we had as >= 13_0B4 at the end of the structs, so let's just enable them for all versions. Plus we already had them for compute! Signed-off-by: Asahi Lina <[email protected]>

This was probably papering over the issue in the previous commit. Doing this makes both command types have the same structure at the end, which makes sense... let's see how it goes. Signed-off-by: Asahi Lina <[email protected]>

And use Rust syntax to make it clear that it is. Signed-off-by: Asahi Lina <[email protected]>

This is the same behavior as kstrtol/kstrtoul, and allows simple `echo 0 > /sys/module/foo/parameters/bar` commands to work. Signed-off-by: Asahi Lina <[email protected]>

Increase timeouts to 1s, but switch to sleeping between polls at the 10ms boundary so we don't busy-loop when the firmware is too busy to service rings (which can happen in rare cases). Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Still untested, but it might even work now? Signed-off-by: Asahi Lina <[email protected]>

This needs to go into the UAPI... Signed-off-by: Asahi Lina <[email protected]>

Looks like these fields of the commands count events (actual GPU commands), not work commands. The previous implementation was mixing these things up... Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This is useful so the driver won't try to abort everything, which leaves memory mostly untouched so it can be inspected from the hypervisor. Signed-off-by: Asahi Lina <[email protected]>

Makes sure object sizes are 16K aligned on 4K kernels. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This is just a guess, but worst case we over-invalidate which is better than under-invalidating. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

A signaled scheduler fence can outlive its scheduler, since fences are independencly reference counted. Therefore, we can't reference the scheduler in the get_timeline_name() implementation. Fixes oopses on `cat /sys/kernel/debug/dma_buf/bufinfo` when shared dma-bufs reference fences from GPU schedulers that no longer exist. Signed-off-by: Asahi Lina <[email protected]>

Treat this like a GPU firmware crash, since it's at the very least a deadlock and a bad place to be in... Signed-off-by: Asahi Lina <[email protected]>

We need this to offload freeing completed work to the global GPU manager, since doing it in the event handler context is a recipe for deadlocks if the GPU faults. Signed-off-by: Asahi Lina <[email protected]>

If we drop work in the main event completion thread, we can deadlock when we try to invalidate GPU contexts if the firmware is stuck. Instead, stash completed work in the global GPU manager and free it whenever we allocate anything. In the future this should move to a dedicated cleanup thread or something like that... Signed-off-by: Asahi Lina <[email protected]>

Since workqueues get freed in the event thread, that can drop the GpuContext there and deadlock when the firmware halts. Delegate it to the GpuManager too. Signed-off-by: Asahi Lina <[email protected]>

Buffer is already internally locked. Signed-off-by: Asahi Lina <[email protected]>

It looks like the firmware can clean up currently mapped buffers at any arbitrary time. Let's assume this is cleaned up by context inval (which is the only explicit inval macOS does). Signed-off-by: Asahi Lina <[email protected]>

This should make Mesa fail the driver load and fall back to software rendering early. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Also set the large_tib flag properly (implied from tib size for now, not sure if this should be a separate flag?) Signed-off-by: Asahi Lina <[email protected]>

These change all the time and we don't use them, so let's just allocate them as big opaque blobs and stop worrying about them. Signed-off-by: Asahi Lina <[email protected]>

We will not be supporting this version in the end. Signed-off-by: Asahi Lina <[email protected]>

Also remove V13_2 from the build list, since we will not support it. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This includes new coefficients for t8112 that are needed for the correct table calculation code. Signed-off-by: Asahi Lina <[email protected]>

This was already missing in t8112, since the tables aren't static and vary device to device. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This makes stuff like: #[ver(...)] if foo { // version and runtime condition specific code } work. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Includes preliminary T6022 and T6020 data (untested). Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

TODO: G13(X) Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This removes some diff noise. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This is a mitigation introduced in macOS 13.3. We were already OK without mapping the kernel address space to user contexts, but it seems we need to do that for a new G14X thing. Doing this keeps the buffer manager stuff safe. Signed-off-by: Asahi Lina <[email protected]>

We were avoiding this to avoid security implications, but it seems G14X needs to read command buffers directly from the hardware as well as write to a new scene object... Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This is ugly, we need a better way of expressing this. Signed-off-by: Asahi Lina <[email protected]>

We will use this for command buffers, since G14X needs that. Signed-off-by: Asahi Lina <[email protected]>

Needed by G14X Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

The first two job params structs got replaced with a register write array. Also shuffle around paddings a bit to make it make more sense. This does not yet populate the registers. Signed-off-by: Asahi Lina <[email protected]>

Needed for G14X. Signed-off-by: Asahi Lina <[email protected]>

This allows self-referential pointers. We already had this in the inner cb for back-references, so this allows direct self-references. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Useful when converting GPU pointers to raw register values. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Also be more conservative about things, using the worst-case sizes instead of the real cluster count. It's unclear what goes down with lower-cluster-count machines anyway, and the buffers (except the tilemaps) aren't large enough to be worth optimizing more. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This lets us play around with unknown stuff from userspace, without having to commit to adding it to the UAPI (or knowing whether it's safe to expose at all). Signed-off-by: Asahi Lina <[email protected]>

This is required to make PinInit<T> work as a return value from a trait function. We indirect via an associated type to avoid return_position_impl_trait_in_trait, which is an incomplete feature. Signed-off-by: Asahi Lina <[email protected]>

This allows initializing a lock using pin_init!(), instead of requiring the inner data to be passed through the stack. Signed-off-by: Asahi Lina <[email protected]>

This requires type_alias_impl_trait. Signed-off-by: Asahi Lina <[email protected]>

Replace all usage of smutex::Mutex with real mutexes using pin_init!(). This could still use some optimization (there's redundant re-boxing going on), but for now it works and runs. Signed-off-by: Asahi Lina <[email protected]>

This lets us get rid of a bunch of Pin<Box<>> and Box<> indirection, though we still need temporary boxes due to borrow checker issues during initialization. Signed-off-by: Asahi Lina <[email protected]>

When using `quote!` as part of an expression that was not the last one in a function, the `#[allow(clippy::vec_init_then_push)]` attribute would be present on an expression, which is not allowed. This patch refactors that part of the macro to use a statement instead. Signed-off-by: Benno Lossin <[email protected]>

… function Other macros might also want to parse generics. Additionally this makes the code easier to read, as the next commit will introduce more code in `#[pin_data]`. Also add more comments to explain how parsing generics work. Signed-off-by: Benno Lossin <[email protected]>

When using `#[pin_data]` on a struct that used `Self` in the field types, a type error would be emitted when trying to use `pin_init!`. Since an internal type would be referenced by `Self` instead of the defined struct. This patch fixes this issue by replacing all occurrences of `Self` in the `#[pin_data]` macro with the concrete type circumventing the issue. Since rust allows type definitions inside of blocks, which are expressions, the macro also checks for these and emits a compile error when it finds `trait`, `enum`, `union`, `struct` or `impl`. These keywords allow creating new `Self` contexts, which conflicts with the current implementation of replacing every `Self` ident. If these were allowed, some `Self` idents would be replaced incorrectly. Signed-off-by: Benno Lossin <[email protected]> Reported-by: Alice Ryhl <[email protected]>

Also improve the explaining comments. Signed-off-by: Benno Lossin <[email protected]>

Add a derive proc-macro for the `Zeroable` trait. The macro supports structs where every field implements the `Zeroable` trait. This way `unsafe` implementations can be avoided. Suggested-by: Asahi Lina <[email protected]> Signed-off-by: Benno Lossin <[email protected]>

Use hygienic identifiers for the guards instead of the field names. This makes the init macros feel more like normal struct initializers, since assigning identifiers with the name of a field does not create conflicts. Suggested-by: Asahi Lina <[email protected]> Signed-off-by: Benno Lossin <[email protected]>

In the implementation of the init macros there is a `if false` statement that type checks the initializer to ensure every field is initialized. Since the next patch has a stack variable to store the struct, the function might allocate too much memory on debug builds. Putting the struct into a closure ensures that even in debug builds no stack overflow error is caused. In release builds this was not a problem since the code was optimized away due to the `if false`. Signed-off-by: Benno Lossin <[email protected]>

… fields Add the struct update syntax to the init macros, but only for `..Zeroable::zeroed()`. Adding this at the end of the struct initializer allows one to omit fields from the initializer, these fields will be initialized with 0x00 set to every byte. Only types that implement the `Zeroable` trait can utilize this. Suggested-by: Asahi Lina <[email protected]> Signed-off-by: Benno Lossin <[email protected]>

Add two functions `pin_init_array_from_fn` and `init_array_from_fn` that take a function that generates initializers for `T` from usize, the added functions then return an initializer for `[T; N]` where every element is initialized by an element returned from the generator function. Suggested-by: Asahi Lina <[email protected]> Signed-off-by: Benno Lossin <[email protected]>

By analogy to Default::default(), this just returns the zeroed representation of the type directly. init::zeroed() is the version that returns an initializer. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This makes directly initializing structures with a type name that isn't in the current scope work properly. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This was previously breaking the init!() macros which use <- as part of their syntax (and would've also broken operators like <). Restrict the angle bracket nesting parsing to only start after ::, unless we're in a struct definition. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

All users are now converted to new_init_prealloc() Signed-off-by: Asahi Lina <[email protected]>

These functinos allow users to chain post-processing code to an initializer, which can be used to mutate the (fully initialized) object in place as part of an initializer chain. Ideally this would be a trait function (e.g. init!(...).and_then(...)), but that requires return_position_impl_trait_in_trait... Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

All usage has been replaced with init!() and friends! 🎉 Signed-off-by: Asahi Lina <[email protected]>

Switch from being a refcount wrapper itself to a transparent wrapper around `bindings::drm_device`. The refcounted type then becomes ARef<Device<T>>. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

We want to use caller_location to uniquely identify callsites, to automatically create lockdep classes without macros. The location filename in local code uses the relative path passed to the compiler, but if that code is generic and instantiated from another crate, the path becomes absolute. To make this work and keep the paths consistent, always pass an absolute path to the compiler. Then the Location path is always identical regardless of how the code is being compiled. Signed-off-by: Asahi Lina <[email protected]>

This allows Rust code to use the Hasher interface with the kernel siphash implementation. Signed-off-by: Asahi Lina <[email protected]>

Lock classes aren't used without lockdep. The C side declares the key as an empty struct in that case, but let's make it an explicit ZST in Rust, implemented in a separate module. This allows us to more easily guarantee that the lockdep code will be trivially optimized out without CONFIG_LOCKDEP, including LockClassKey arguments that are passed around. Depending on whether CONFIG_LOCKDEP is enabled or not, we then import the real lockdep implementation or the dummy one. Signed-off-by: Asahi Lina <[email protected]>

We want to be able to handle dynamic lock class creation and using pointers to things that aren't a real lock_class_key as lock classes. Doing this by casting around Rust references is difficult without accidentally invoking UB. Instead, switch LockClassKey to being a raw pointer wrapper around a lock_class_key, which means there is no UB possible on the Rust side just by creating and consuming these objects. The C code also should never actually dereference lock classes, only use their address (possibly with an offset). We still provide a dummy ZST version of this wrapper, to be used when lockdep is disabled. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Using macros to create lock classes all over the place is unergonomic, and makes it impossible to add new features that require lock classes to code such as Arc<> without changing all callers. Rust has the ability to track the caller's identity by file/line/column number, and we can use that to dynamically generate lock classes instead. Signed-off-by: Asahi Lina <[email protected]>

Use the new automagic lock class code to remove the lock class and name parameters from Lock::new() and Lock::pin_init(). The old functions are renamed to new_with_class() and pin_init_with_class() respectively. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

This allows Rust code to explicitly integrate types with lockdep. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Asahi Lina <[email protected]>

Sidharth reports that on M2, the PMU never generates any interrupt when using 'perf record', which is a annoying as you get no sample. I'm temped to say "no sample, no problem", but others may have a different opinion. Upon investigation, it appears that the counters on M2 are significantly different from the ones on M1, as they count on 64 bits instead of 48. Which of course, in the fine M1 tradition, means that we can only use 63 bits, as the top bit is used to signal the interrupt... This results in having to introduce yet another flag to indicate yet another odd counter width. Who knows what the next crazy implementation will do... With this, perf can work out the correct offset, and 'perf record' works as intended. Tested on M2 and M2-Pro CPUs. Cc: Janne Grunau <[email protected]> Cc: Hector Martin <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Will Deacon <[email protected]> Fixes: 7d0bfb7 ("drivers/perf: apple_m1: Add Apple M2 support") Reported-by: Sidharth Kshatriya <[email protected]> Signed-off-by: Marc Zyngier <[email protected]>

This is a hardcoded charge threshold feature present in firmware 13.0 or newer. Userspace settings are rounded to one of the two possible behaviors. Signed-off-by: Hector Martin <[email protected]>

Apple M2 devices have an MTP coprocessor embedded in the SoC that handles HID for the integrated touchpad/keyboard, and communicates over the DockChannel interface. This driver implements this new interface. Signed-off-by: Hector Martin <[email protected]>

This driver can be used for coprocessors that do some background task or communicate out-of-band, and do not do any mailbox I/O beyond the standard RTKit initialization. Signed-off-by: Hector Martin <[email protected]>

Probably pointless but might be helpful to debug issues. Gets rid of 'spi_hid_apple_status_ok' is unused compiler warning. Signed-off-by: Janne Grunau <[email protected]>

The trackpad has to request multi touch reports during resume. Signed-off-by: Janne Grunau <[email protected]>

Working suspend and resume. The keyboard can not yet used as wakeup source. Most likely caused by missing irq_set_wake support in the gpio/pinctrl driver. Signed-off-by: Janne Grunau <[email protected]>

This maximum is arbitrary. Recent Apple devices have some vendor-defined reports with 16384 here which fail to parse without this, so let's bump it to that. This value is used as follows: report->size += parser->global.report_size * parser->global.report_count; [...] /* Total size check: Allow for possible report index byte */ if (report->size > (max_buffer_size - 1) << 3) { hid_err(parser->device, "report is too long\n"); return -1; } All of these fields are unsigned integers, and report_count is bounded by HID_MAX_USAGES (12288). Therefore, as long as the respective maximums do not overflow an unsigned integer (let's say a signed integer just in case), we're safe. This holds for 16384. Signed-off-by: Hector Martin <[email protected]>

On at least some SPI devices (e.g. recent Apple Silicon machines), the Broadcom touch controller is prone to crashing. When this happens, the STM eventually notices and resets it. It then notifies the driver via HID report 0x60, and the driver needs to re-enable MT mode to make things work again. This poses an additional issue: the hidinput core will close the low-level transport while the device is closed, which can cause us to miss a reset notification. To fix this, override the input open/close callbacks and send the MT enable every time the HID device is opened, instead of only once on probe. This should increase general robustness, even if the reset mechanism doesn't work for some reason, so it's worth doing it for USB devices too. MTP devices are exempt since they do not require the MT enable at all. Signed-off-by: Hector Martin <[email protected]>

On a filesystem with a non-zero stripe unit and a large sequential write, delayed allocation will set a minimum allocation length of the stripe unit. If allocation fails because there are no extents long enough for an aligned minlen allocation, it is supposed to fall back to unaligned allocation which allows single block extents to be allocated. When the allocator code was rewritting in the 6.3 cycle, this fallback was broken - the old code used args->fsbno as the both the allocation target and the allocation result, the new code passes the target as a separate parameter. The conversion didn't handle the aligned->unaligned fallback path correctly - it reset args->fsbno to the target fsbno on failure which broke allocation failure detection in the high level code and so it never fell back to unaligned allocations. This resulted in a loop in writeback trying to allocate an aligned block, getting a false positive success, trying to insert the result in the BMBT. This did nothing because the extent already was in the BMBT (merge results in an unchanged extent) and so it returned the prior extent to the conversion code as the current iomap. Because the iomap returned didn't cover the offset we tried to map, xfs_convert_blocks() then retries the allocation, which fails in the same way and now we have a livelock. Reported-and-tested-by: Brian Foster <[email protected]> Fixes: 8584332 ("xfs: factor xfs_bmap_btalloc()") Signed-off-by: Dave Chinner <[email protected]> Reviewed-by: Darrick J. Wong <[email protected]>

Newer devices don't use this. We still don't know what it does, but let's keep to the same behavior macOS has here. Signed-off-by: Hector Martin <[email protected]>

If a device is already powered, leave it powered. Otherwise port setup done by u-boot breaks. Signed-off-by: Hector Martin <[email protected]>

This is the counterpart to b50255c. In MSI mode we can still get MSIs even with IRQs disabled, so add an explicit gate for it. Signed-off-by: Hector Martin <[email protected]>

Block count needs to be computed with an extra factor of the sample count. Signed-off-by: Asahi Lina <[email protected]>

Signed-off-by: Hector Martin <[email protected]>

Signed-off-by: Eileen Yoon <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ANE support #159

ANE support #159

Commits on May 30, 2023

Commits on Jun 1, 2023

Commits on Jun 4, 2023

Commits on Jun 6, 2023

Commits on Jun 7, 2023

Commits on Jun 15, 2023

Commits on Jun 17, 2023

Commits on Jun 21, 2023

ANE support #159

Are you sure you want to change the base?

ANE support #159

Commits on May 30, 2023

Commits on Jun 1, 2023

Commits on Jun 4, 2023

Commits on Jun 6, 2023

Commits on Jun 7, 2023

Commits on Jun 15, 2023

Commits on Jun 17, 2023

Commits on Jun 21, 2023