How to avoid trailing whitespace getting consumed

[nnmm]

Take the PEG

// I want to allow space between b and a, but not after the last a
baa = !{ "b" ~ "a"* }
baa_list = ${ baa ~ "\n" ~ baa}
WHITESPACE    = _{ " " | "\t" | "\r" | "\n" }

and the input

b
ba

I would expect the input to parse, but it doesn't:

 --> 1:1
  |
1 | b
  | ^---
  |
  = expected baa_list

It's as if the ~ after "b" eats the whitespace following it, even though the "a" is not matched. Is that a bug? If no, how can I stop rules with a repetition at the end from consuming whitespace?

There is a similar problem with + instead of *.

[notpeter]

It's as if the ~ after "b" eats the whitespace following it, even though the "a" is not matched. Is that a bug? If no, how can I stop rules with a repetition at the end from consuming whitespace?

PEST parsers are greedy, so when they match they consume the input and there's going back. So with implicit whitespace
the ~ in baa ~ "\n"becomesbaa ~ WHITESPACE* ~ "\n". WHITESPACE gobbles the newline(s) and so it doesn't find "\n"` and fails.

You should take the newlines out of WHITESPACE and manually account for them. You should only put newlines in WHITESPACE if you are working in a language where newlines, " " and "\t" are all optional/trivial -- this happens in a language with explicit statement terminators (";" in C/C++/Rust) or never required (Lua). Basically if your newlines have semantic meaning (e.g. they are a statement separator) they shouldn't be included in WHITESPACE

Maybe something like this:

baa_list = ${ baa ~ NEWLINES ~ baa}
baa = !{ "b" ~ "a"* }
WHITESPACE = _{ " " | "\t" }
NEWLINES = _{ ("\n" | "\r\n" | "\r")+ }

Or if you wanted to be more pedantic try this PESTbin

---

Note: It took me a minute to realize that your example has whitespace after ba because markdown considers trailing whitespace trivial (meta funny). To render trailing white space use use <pre>.

b
ba