FROMLIST: BACKPORT: mm/page_alloc: Protect PCP lists with a spinlock
Currently the PCP lists are protected by using local_lock_irqsave to
prevent migration and IRQ reentrancy but this is inconvenient. Remote
draining of the lists is impossible and a workqueue is required and
every task allocation/free must disable then enable interrupts which is
expensive.
As preparation for dealing with both of those problems, protect the
lists with a spinlock. The IRQ-unsafe version of the lock is used
because IRQs are already disabled by local_lock_irqsave. spin_trylock
is used in preparation for a time when local_lock could be used instead
of lock_lock_irqsave.
The per_cpu_pages still fits within the same number of cache lines after
this patch relative to before the series.
struct per_cpu_pages {
spinlock_t lock; /* 0 4 */
int count; /* 4 4 */
int high; /* 8 4 */
int batch; /* 12 4 */
short int free_factor; /* 16 2 */
short int expire; /* 18 2 */
/* XXX 4 bytes hole, try to pack */
struct list_head lists[13]; /* 24 208 */
/* size: 256, cachelines: 4, members: 7 */
/* sum members: 228, holes: 1, sum holes: 4 */
/* padding: 24 */
} __attribute__((__aligned__(64)));
Signed-off-by:
Mel Gorman <mgorman@techsingularity.net>
Link: https://lore.kernel.org/all/20220420095906.27349-6-mgorman@techsingularity.net/
Link: https://lore.kernel.org/all/20220429091321.GB3441@techsingularity.net/
Conflicts:
mm/page_alloc.c
1. per_cpu_pages are updated from 44042b44 at 5.13 so conflicted
Since we don't need to have high-order page pcp atm, skip the patch.
Bug: 230899966
Signed-off-by:
Minchan Kim <minchan@google.com>
Change-Id: I03ff1c22301e7f8735947e71413376ea143e855a
Loading
Please sign in to comment