Form a plan for killing off 0.2.2 and 0.2.3

#9476 (moved) is looking tricky; we should make a plan for what to do if it turns out maxmially tricky, and a plan for what to do when we're finally done with 0.2.3 as well.