Fix a heap overflow found by debuger, and make it harder to make that mistake again

Our public key functions assumed that they were always writing into a
large enough buffer.  In one case, they weren't.

(Incorporates fixes from sebastian)