EsotericSoftware / EsotericSoftware/yamlbeans

trouble with the tokenizer related to the colon (:) character

Open
#175 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
568
Forks
146
PR merge metrics
No merged PRs in 30d

Description

Hi,

Here is a test method to add to TokenizerTest if useful:
```java
@Test
public void testColon() throws FileNotFoundException {
Tokenizer tokenizer = new Tokenizer("- { setname: asciio, name: perl:app-asciio }");
while (tokenizer.peekNextToken()!=null) {
tokenizer.getNextToken();
}
}
```

and the exception stack trace is:
```
com.esotericsoftware.yamlbeans.tokenizer.Tokenizer$TokenizerException: Line 0, column 49: While scanning a plain scalar, found unexpected ':'. See: http://pyyaml.org/wiki/YAMLColonInFlowContext
at com.esotericsoftware.yamlbeans.tokenizer.Tokenizer.scanPlain(Tokenizer.java:941)
at com.esotericsoftware.yamlbeans.tokenizer.Tokenizer.fetchPlain(Tokenizer.java:522)
at com.esotericsoftware.yamlbeans.tokenizer.Tokenizer.fetchMoreTokens(Tokenizer.java:311)
at com.esotericsoftware.yamlbeans.tokenizer.Tokenizer.peekNextToken(Tokenizer.java:120)
at com.esotericsoftware.yamlbeans.tokenizer.TokenizerTest.testColon(TokenizerTest.java:201)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:93)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
```

Regards,
Patrice

ps: my sample string is extracted from the following file:
https://github.com/repology/repology-rules/blob/master/800.renames-and-merges/a.yaml
(there are many more cases in this project)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.