circuit_unlink_all_from_channel() is a performance bottleneck?

In https://lists.torproject.org/pipermail/tor-relays/2013-September/002715.html Andy reports some profiling on his fast exit. Top on his list is

cycle%   image              symbol
------   -----              ------
 19.37%  tor                circuit_unlink_all_from_channel

It looks like that function crawls the whole circuit list every time we close a connection. With circuit lists being huge these days, it is plausible that it takes a lot of effort to crawl it.