Bug: Special Token Mapping Incorrect

Describe the bug
The tokenizer_stream.decode api results in incorrect mapping when special tokens are included in the models tokenizer. The token IDs which are emitted are correct (and confirmed by directly checking against the torch api and computed argmax(softmax(logit)) directly, however the mapping of the resulting token id to the final string is wrong

To Reproduce
Steps to reproduce the behavior:

def infer_onnx(target_model_path, user_queries, model_name):

    print("-"*30)
    print("onnx eval")
    print("-"*30)

    model = og.Model(target_model_path)
    tokenizer_og = og.Tokenizer(model)
    tokenizer_stream = tokenizer_og.create_stream()
    
    search_options = {}
    function_store_target = []
    arg_store_target = []
    try:
        for query in tqdm(user_queries):
            prompt = format_query(query, model_name)
            tokenizer = AutoTokenizer.from_pretrained("/<path_to_model_with_special_tokens>/phi3.5-mini/safetensors") 

            messages = [ 
                {"role": "system", "content": "You are a helpful assistant."}, 
                {"role": "user", "content": query},
            ] 
            
          
            input_tokens = tokenizer.apply_chat_template(messages, tokenize=True)

            params = og.GeneratorParams(model)
            params.set_search_options(**search_options)
            generator = og.Generator(model, params)
            generator.append_tokens(input_tokens)

            response = ""
            while not generator.is_done():
                generator.generate_next_token()
                
                # check with torch
                logits = torch.from_numpy(generator.get_output("logits"))[:, -1, :]
                logits_scaled = F.softmax(logits, dim=-1)
                idx_torch = torch.argmax(logits_scaled)
                new_token_og = generator.get_next_tokens()[0]
                print(idx_torch == new_token_og)
                 # check with torch
                
                  response += tokenizer_stream.decode(new_token_og)

            function, args_ = extract_functions_and_args(response)
            function_store_target.append(function)
            arg_store_target.append(args_)

    except KeyboardInterrupt:
        print("  --control+c pressed, aborting generation--")
        print()
        del generator
    
    return function_store_target, arg_store_target

Expected behavior
The token IDs from torch and onnxgenai always match, however the outputs are completely off

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS] -> Mac M3
• Version [e.g. 22] -> OGA nightly build

Start with tokenizer_og.create_stream() and tokenizer_stream.decode(new_token_og) in the reproduction script, then compare the decoded text with the Hugging Face tokenizer configured by apply_chat_template. Reproduce the special-token case and verify that matching token IDs produce matching strings; done means the stream decodes those IDs correctly.