Also, fixed one test file that didn't have its syntax error as the last line.

Always keep 'token' outside.
Always return the string to upper levels inside.

Partially addresses issue #513.

The sequence

  #define m()
  int m"

creates a token of no length (a string of 0 size).  Protect
against a string of 0 size as well as the existing protect
against a null string.

This would look ahead for a second #, for token pasting, and if not
found, backup one token.  This is fine, unless at the end of line,
which would backup the #, rather than the look ahead.