Tokenizer

Fields of this type

Fields

buffer:[:0]const u8
index:usize

For debugging purposes.

Functions

dump: For debugging purposes.
init
next: After this returns invalid, it will reset on the next newline, returning tokens starting from there.