lexer: Preserve keyword, regexp flags until processing non-comment tokens

When the lexer has either the `no_keyword` or `no_regexp` flag set, it
should retain these flags until it encounters a non-comment token.
Only then should it stop preventing the interpretation of tokens beginning
with a letter or forward slash as keywords or regular expression literals,
respectively.

Previously, these flags were being reset too early when processing comments,
which could cause incorrect parsing when comments appeared between relevant
tokens. The flags should only be reset after consuming an actual non-comment
token to ensure consistent parsing behavior.

Fixes: #250
Fixes: 855854f ("lexer: emit comment and template statement block tokens")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>

0 files changed, 0 insertions, 0 deletions