Joel Fernandes (Google) 3cb278e73b
rcu: Make call_rcu() lazy to save power ...
Implement timer-based RCU callback batching (also known as lazy
callbacks). With this we save about 5-10% of power consumed due
to RCU requests that happen when system is lightly loaded or idle.
By default, all async callbacks (queued via call_rcu) are marked
lazy. An alternate API call_rcu_hurry() is provided for the few users,
for example synchronize_rcu(), that need the old behavior.
The batch is flushed whenever a certain amount of time has passed, or
the batch on a particular CPU grows too big. Also memory pressure will
flush it in a future patch.
To handle several corner cases automagically (such as rcu_barrier() and
hotplug), we re-use bypass lists which were originally introduced to
address lock contention, to handle lazy CBs as well. The bypass list
length has the lazy CB length included in it. A separate lazy CB length
counter is also introduced to keep track of the number of lazy CBs.
[ paulmck: Fix formatting of inline call_rcu_lazy() definition. ]
[ paulmck: Apply Zqiang feedback. ]
[ paulmck: Apply s/call_rcu_flush/call_rcu_hurry/ feedback from Tejun Heo. ]
Suggested-by: Paul McKenney <paulmck@kernel.org >
Acked-by: Frederic Weisbecker <frederic@kernel.org >
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org >
Signed-off-by: Paul E. McKenney <paulmck@kernel.org >
2022-11-29 14:02:23 -08:00
2022-10-04 19:18:08 +02:00
2022-10-16 11:08:19 -07:00
2022-10-03 11:08:33 +01:00
2022-09-28 16:09:49 +01:00
2022-10-07 12:05:29 -07:00
2022-10-06 13:31:13 +02:00
2022-10-08 08:56:37 -07:00
2022-10-10 09:27:46 -07:00
2022-10-07 16:48:26 -07:00
2022-10-11 10:53:25 -07:00
2022-10-03 21:52:33 -07:00
2022-09-29 15:20:14 +01:00
2022-10-12 11:00:22 -07:00
2022-10-08 10:06:48 -07:00
2022-10-07 16:13:55 -07:00
2022-10-04 19:36:53 -07:00
2022-10-13 09:58:42 -07:00
2022-10-07 16:48:26 -07:00
2022-10-10 13:28:06 -07:00
2022-10-05 11:05:17 -06:00
2022-09-29 16:38:05 -07:00
2022-09-26 12:19:12 -07:00
2022-10-12 11:00:22 -07:00
2022-09-26 19:17:28 -06:00
2022-10-07 09:19:14 -07:00
2022-10-10 11:26:40 -06:00
2022-09-29 09:25:47 -07:00
2022-10-10 17:53:04 -07:00
2022-09-26 19:46:29 -07:00
2022-09-26 10:13:13 -07:00
2022-09-26 10:13:13 -07:00
2022-10-14 13:03:00 -07:00
2022-10-14 13:03:00 -07:00
2022-10-14 13:44:44 -07:00
2022-09-30 14:50:11 -07:00
2022-10-10 10:21:22 -07:00
2022-10-10 17:53:04 -07:00
2022-10-10 17:53:04 -07:00
2022-10-03 14:03:19 -07:00
2022-09-26 10:13:14 -07:00
2022-09-30 14:32:35 +02:00
2022-10-16 10:45:17 -07:00
2022-10-12 18:51:49 -07:00
2022-10-10 19:45:17 -07:00
2022-09-26 19:46:07 -07:00
2022-09-27 13:26:16 +02:00
2022-10-11 18:51:11 -07:00
2022-09-29 04:40:15 +09:00
2022-10-04 13:31:45 -07:00
2022-09-26 20:40:43 -07:00
2022-10-06 05:57:36 -07:00
2022-10-10 17:53:04 -07:00
2022-10-10 19:45:17 -07:00
2022-10-03 20:33:41 -07:00
2022-09-26 12:27:55 +02:00
2022-09-27 14:48:26 -04:00
2022-10-03 14:03:30 -07:00
2022-10-03 14:03:20 -07:00
2022-09-28 16:17:40 +01:00
2022-10-03 14:02:45 -07:00
2022-10-12 11:16:58 -07:00
2022-10-12 11:00:22 -07:00
2022-10-03 14:03:21 -07:00
2022-10-12 16:30:56 -06:00
2022-10-04 08:13:20 -06:00
2022-09-26 15:52:31 +02:00
2022-10-10 13:28:06 -07:00
2022-09-26 13:31:20 +02:00
2022-10-03 14:21:44 -07:00
2022-09-28 14:21:53 +01:00
2022-10-03 14:21:43 -07:00
2022-09-28 08:56:25 +02:00
2022-10-03 14:02:59 -07:00
2022-10-03 14:03:33 -07:00
2022-10-03 14:03:19 -07:00
2022-10-03 14:03:21 -07:00
2022-10-03 14:03:25 -07:00
2022-09-26 20:30:39 -07:00
2022-10-03 14:02:43 -07:00
2022-09-26 12:37:21 -04:00
2022-10-07 10:48:49 -07:00
2022-10-06 17:31:02 -07:00
2022-10-03 17:51:52 -07:00
2022-09-26 19:46:13 -07:00
2022-10-04 12:06:58 -06:00
2022-10-10 17:53:04 -07:00
2022-10-03 14:03:29 -07:00
2022-09-26 19:46:12 -07:00
2022-10-12 18:51:49 -07:00
2022-10-12 18:51:49 -07:00
2022-10-03 14:03:35 -07:00
2022-09-26 19:46:18 -07:00
2022-10-03 14:03:19 -07:00
2022-10-03 14:03:28 -07:00
2022-10-10 17:53:04 -07:00
2022-09-26 10:13:13 -07:00
2022-10-10 12:49:34 -07:00
2022-10-03 14:03:15 -07:00
2022-10-11 17:42:55 -06:00
2022-09-27 09:22:09 +02:00
2022-09-28 14:11:28 +01:00
2022-09-30 16:23:48 -05:00
2022-10-03 17:34:32 -07:00
2022-09-26 19:46:27 -07:00
2022-09-26 19:46:29 -07:00
2022-09-26 19:46:09 -07:00
2022-09-26 19:46:09 -07:00
2022-10-03 14:03:04 -07:00
2022-10-10 17:53:04 -07:00
2022-10-04 09:46:22 -07:00
2022-10-11 11:08:18 -07:00
2022-10-03 14:21:43 -07:00
2022-10-04 13:32:08 +02:00
2022-09-26 19:46:08 -07:00
2022-10-03 17:33:57 -07:00
2022-09-23 11:55:36 +01:00
2022-10-04 20:52:38 +02:00
2022-10-04 20:52:38 +02:00
2022-10-11 17:42:58 -06:00
2022-10-04 15:55:55 +02:00
2022-09-24 15:03:25 +02:00
2022-10-11 17:42:58 -06:00
2022-11-29 14:02:23 -08:00
2022-10-04 15:59:36 +02:00
2022-09-28 10:46:21 -06:00
2022-09-23 14:25:05 +02:00
2022-09-28 20:27:43 -04:00
2022-10-10 17:53:04 -07:00
2022-10-14 12:28:43 -07:00
2022-10-09 14:05:15 -07:00
2022-09-22 16:30:14 +02:00
2022-09-25 09:28:57 +02:00
2022-10-03 11:08:32 +01:00
2022-10-03 14:03:33 -07:00
2022-10-12 16:30:34 -06:00
2022-10-15 21:42:05 +02:00
2022-10-10 17:53:04 -07:00
2022-10-03 14:03:18 -07:00
2022-10-08 08:56:37 -07:00
2022-10-10 09:10:28 -07:00
2022-10-03 14:03:36 -07:00
2022-10-03 14:03:36 -07:00
2022-09-26 19:46:05 -07:00
2022-09-26 19:46:05 -07:00
2022-09-29 15:20:29 +02:00
2022-10-03 17:44:18 -07:00
2022-10-04 11:21:06 +02:00
2022-09-28 20:28:13 -04:00
2022-09-25 12:39:52 -04:00
2022-10-03 14:03:18 -07:00
2022-09-29 10:17:37 -04:00
2022-09-26 19:46:21 -07:00
2022-10-07 09:32:40 -04:00
2022-10-07 08:10:52 -06:00
2022-10-07 09:32:40 -04:00
2022-09-26 19:46:18 -07:00
2022-09-26 19:46:18 -07:00
2022-09-28 10:00:16 +02:00
2022-10-07 15:00:25 +02:00
2022-10-03 14:03:08 -07:00
3cb278e73b