Potentail unsound issue in libsql-sqlite3-parser

Hi, thanks for your time to read this issue. We cannot find a repo for this crate, so we report the issue here. If it's not the correct place, plet let us know.

Our static analyzer find a potential unsound issue in the utf-8 parsing, where it doesn't provide enough check to ensure the soundness.

https://github.com/tursodatabase/libsql/blob/81459627f117143aed29ae797c6d1355e4c4b694/vendored/sqlite3-parser/src/dialect/mod.rs#L60-L62

A potentail PoC code is like:

use libsql_sqlite3_parser::lexer::sql::{Parser, Tokenizer};
use fallible_iterator::FallibleIterator;


fn main() {
    let invalid_utf8: &[u8] = &[0xC0, 0x80]; 
    let sql = "SELECT ? ";
    let mut v = Vec::new();
    v.extend_from_slice(sql.as_bytes());
    v.extend_from_slice(invalid_utf8);
    let mut parser = Parser::new(&*v);
    let mut cmd = parser.next().unwrap();
    println!("{:?}", cmd)
}

Thanks again for your time.

Start with vendored/sqlite3-parser/src/dialect/mod.rs at lines 60-62 and inspect the UTF-8 parsing path. Reproduce the reported behavior with the provided Rust PoC using invalid UTF-8 bytes, then verify that parsing rejects or safely handles the input and add regression coverage for the case.