BACKPORT: mm: page_alloc: close migratetype race between freeing and stealing
There are three freeing paths that read the page's migratetype optimistically before grabbing the zone lock. When this races with block stealing, those pages go on the wrong freelist. The paths in question are: - when freeing >costly orders that aren't THP - when freeing pages to the buddy upon pcp lock contention - when freeing pages that are isolated - when freeing pages initially during boot - when freeing the remainder in alloc_pages_exact() - when "accepting" unaccepted VM host memory before first use - when freeing pages during unpoisoning None of these are so hot that they would need this optimization at the cost of hampering defrag efforts. Especially when contrasted with the fact that the most common buddy freeing path - free_pcppages_bulk - is checking the migratetype under the zone->lock just fine. In addition, isolated pages need to look up the migratetype under the lock anyway, which adds branches to the locked section, and results in a double lookup when the pages are in fact isolated. Move the lookups into the lock. Link: https://lkml.kernel.org/r/20240320180429.678181-8-hannes@cmpxchg.org Signed-off-by:Johannes Weiner <hannes@cmpxchg.org> Reported-by:
Vlastimil Babka <vbabka@suse.cz> Reviewed-by:
Vlastimil Babka <vbabka@suse.cz> Tested-by:
Baolin Wang <baolin.wang@linux.alibaba.com> Cc: David Hildenbrand <david@redhat.com> Cc: "Huang, Ying" <ying.huang@intel.com> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Zi Yan <ziy@nvidia.com> Signed-off-by:
Andrew Morton <akpm@linux-foundation.org> Bug: 406708503 (cherry picked from commit 55612e80) [ 1. Calling get_pfnblock_migratetype() under the zone->lock in __free_pages_ok to preserve the vendor hook. 2. In the patch, the function free_unref_folios has been modified, and the corresponding function in android15-6.6 is free_unref_page_list. ] Change-Id: I785fdb603b226845e1b1152116244ecb97502375 Signed-off-by:
yipeng xiang <yipengxiang@honor.corp-partner.google.com>
Loading
Please sign in to comment