Bug 1489744 - Fix a bounds violation crash in the prefs parser. r=glandium, a=RyanVM

Currently, if a get_char() call returns EOF, the index moves beyond the
buffer's bounds and get_char() cannot be called again without triggering a
panic. As a result, everywhere that encounters an EOF and then does subsequent
parsing ungets the EOF... except there was one place that failed to do that:
the match case for CharKind::Slash in get_token(). This meant that a single '/'
at the end of the input could trigger a bounds violation (but only if it is the
start of a new token).

This EOF-unget requirement is subtle and easy to get wrong, so this patch
eliminates it. get_char() now can be called repeatedly after an EOF, and will
return EOF on each subsequent call. This means that some of the existing
unget_char() calls can be removed. Some others are still necessary to get line
numbers correct in error messages, but the outcome of mishandled cases is now
much less drastic -- an incorrect line number in an error message instead of a
panic.

The patch also clarifies a couple of related comments.

--HG--
extra : source : 5097db630d1fb51f7f3fcd5523a924fa823e77ce
extra : intermediate-source : 2676e53112584b2c08b9a1bf02e15cd9bb0d0ee5