11.2 RCU and Counters for Removable I/O Devices
Keywords: perfbook_html

#
11.2 RCU and Counters for Removable I/O Devices

Section showed a fanciful pair of code fragments for dealing with counting I/O accesses to removable devices. These code fragments suffered from high overhead on the fastpath (starting an I/O) due to the need to acquire a reader-writer lock.

This section shows how RCU may be used to avoid this overhead.

The code for performing an I/O is quite similar to the original, with an RCU read-side critical section be substituted for the reader-writer lock read-side critical section in the original:

      1 rcu_read_lock();
      2 if (removing) {
      3   rcu_read_unlock();
      4   cancel_io();
      5 } else {
      6   add_count(1);
      7   rcu_read_unlock();
      8   do_io();
      9   sub_count(1);
     10 }

The RCU read-side primitives have minimal overhead, thus speeding up the fastpath, as desired.

The updated code fragment removing a device is as follows:

      1 spin_lock(&mylock);
      2 removing = 1;
      3 sub_count(mybias);
      4 spin_unlock(&mylock);
      5 synchronize_rcu();
      6 while (read_count() != 0) {
      7   poll(NULL, 0, 1);
      8 }
      9 remove_device();

Here we replace the reader-writer lock with an exclusive spinlock and add a synchronize_rcu() to wait for all of the RCU read-side critical sections to complete. Because of the synchronize_rcu(), once we reach line 6, we know that all remaining I/Os have been accounted for.

Of course, the overhead of synchronize_rcu() can be large, but given that device removal is quite rare, this is usually a good tradeoff.