lexer: properly handle string escape sequences at buffer boundary

While parsing string literals, actually consume the backslash introducing an
escape sequence to prevent it from ending up in the produced string if the
scanner is at the end of the buffer and the remaining buffer contents are
flushed after the consumer loop.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>

1 files changed, 12 insertions, 0 deletions

diff --git a/tests/03_bugs/06_lexer_escape_at_boundary b/tests/03_bugs/06_lexer_escape_at_boundary
new file mode 100644
index 0000000..e80b0a1
--- /dev/null
+++ b/tests/03_bugs/06_lexer_escape_at_boundary
@@ -0,0 +1,12 @@
+When the lexer processed a backslash introducing a string escape directly
+at the buffer boundary, the backslash was incorrectly retained.
+
+-- Testcase --
+{%
+	print("abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijkl\n");
+%}
+-- End --
+
+-- Expect stdout --
+abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijkl
+-- End --