Patch from croup: rewrite the logic of get_next_token() to do the right thing with input that ends at weird places, or aligns with block boundaries after mmap.  should fix bug 455.  Needs fuzzing.


svn:r10847