af_unix: Run GC on only one CPU.
If more than 16000 inflight AF_UNIX sockets exist and the garbage
collector is not running, unix_(dgram|stream)_sendmsg() call unix_gc().
Also, they wait for unix_gc() to complete.
In unix_gc(), all inflight AF_UNIX sockets are traversed at least once,
and more if they are the GC candidate. Thus, sendmsg() significantly
slows down with too many inflight AF_UNIX sockets.
There is a small window to invoke multiple unix_gc() instances, which
will then be blocked by the same spinlock except for one.
Let's convert unix_gc() to use struct work so that it will not consume
CPUs unnecessarily.
Note WRITE_ONCE(gc_in_progress, true) is moved before running GC.
If we leave the WRITE_ONCE() as is and use the following test to
call flush_work(), a process might not call it.
CPU 0 CPU 1
--- ---
start work and call __unix_gc()
if (work_pending(&unix_gc_work) || <-- false
READ_ONCE(gc_in_progress)) <-- false
flush_work(); <-- missed!
WRITE_ONCE(gc_in_progress, true)
Signed-off-by:
Kuniyuki Iwashima <kuniyu@amazon.com>
Link: https://lore.kernel.org/r/20240123170856.41348-5-kuniyu@amazon.com
Signed-off-by:
Jakub Kicinski <kuba@kernel.org>
Loading
-
mentioned in commit bc696f2a
-
mentioned in commit 7314b742
-
mentioned in commit e9c42895
-
mentioned in commit 2f24ebe1
-
mentioned in commit e97d589f
-
mentioned in commit 8e70d5f4
-
mentioned in commit e092bc41
-
mentioned in commit fc19eecf
-
mentioned in commit f6432126
-
mentioned in commit 2048b4e8
-
mentioned in commit 25e3c322
-
mentioned in commit 564030a7
-
mentioned in commit a6cabfe8
-
mentioned in commit c870e78e
-
mentioned in commit 0cfa76de
-
mentioned in commit 5161a2f2
-
mentioned in commit 43f87ad9
-
mentioned in commit 770d3267
-
mentioned in commit ea250ef9
-
mentioned in commit 37f0ad3d
-
mentioned in commit 50e0bbc5
-
mentioned in commit ab84c178
-
mentioned in commit c5e471b4
Please sign in to comment