bpf, mm: Introduce __GFP_TRYLOCK #4765
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![kernel-patches-daemon-bpf-rc[bot]](https://avatars.githubusercontent.com/u/70728002?s=60&v=4){kind=small}
|
Upstream branch: 4d33dc1 |
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
bpf-next_base
branch
from
05fce66 to
e666082
Compare
added 5 commits
December 10, 2024 09:03
Tracing BPF programs execute from tracepoints and kprobes where running context is unknown, but they need to request additional memory. The prior workarounds were using pre-allocated memory and BPF specific freelists to satisfy such allocation requests. Instead, introduce __GFP_TRYLOCK flag that makes page allocator accessible from any context. It relies on percpu free list of pages that rmqueue_pcplist() should be able to pop the page from. If it fails (due to IRQ re-entrancy or list being empty) then try_alloc_pages() attempts to spin_trylock zone->lock and refill percpu freelist as normal. BPF program may execute with IRQs disabled and zone->lock is sleeping in RT, so trylock is the only option. In theory we can introduce percpu reentrance counter and increment it every time spin_lock_irqsave(&zone->lock, flags) is used, but we cannot rely on it. Even if this cpu is not in page_alloc path the spin_lock_irqsave() is not safe, since BPF prog might be called from tracepoint where preemption is disabled. So trylock only. Note, free_page and memcg are not taught about __GFP_TRYLOCK yet. The support comes in the next patches. This is a first step towards supporting BPF requirements in SLUB and getting rid of bpf_mem_alloc. That goal was discussed at LSFMM: https://lwn.net/Articles/974138/ Signed-off-by: Alexei Starovoitov <[email protected]>
Introduce free_pages_nolock() that can free a page without taking locks. It relies on trylock only and can be called from any context. Signed-off-by: Alexei Starovoitov <[email protected]>
Similar to local_lock_irqsave() introduce local_trylock_irqsave(). It uses spin_trylock in PREEMPT_RT and always succeeds when !RT. Signed-off-by: Alexei Starovoitov <[email protected]>
Teach memcg to operate under __GFP_TRYLOCK conditions when spinning locks cannot be used. The end result is __memcg_kmem_charge_page() and __memcg_kmem_uncharge_page() become lockless. Signed-off-by: Alexei Starovoitov <[email protected]>
Unconditionally use __GFP_ACCOUNT in try_alloc_pages(). The caller is responsible to setup memcg correctly. All BPF memory accounting is memcg based. Signed-off-by: Alexei Starovoitov <[email protected]>
|
Upstream branch: 6e8ba49 |
Use try_alloc_pages() and free_pages_nolock() Signed-off-by: Alexei Starovoitov <[email protected]>
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
series/916186=>bpf-next
branch
from
9dcb7b3 to
c400ce9
Compare
|
At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=916186 expired. Closing PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: bpf, mm: Introduce __GFP_TRYLOCK
version: 2
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=916186