beehive-lab / beehive-lab/GPULlama3.java
Add Junit5 and a set of initial tests
Open
good first issue
- Dominant language
- Java
- Stars
- 274
- Forks
- 37
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 2
Description
## Description:
This repo currently doesn’t have any unit tests. Let’s add JUnit 5 support and create an initial test for the Tokenizer class to ensure tokenization behaves as expected.
## Tasks:
* Add JUnit 5 as a dependency using Maven.
* Create a test class:
src/test/java/llama/TokenizerTest.java
* Add a test that checks tokenization for a simple prompt. Example:
```java
@Test
void testBasicTokenization() {
Tokenizer tokenizer = new Tokenizer("path/to/tokenizer.model");
int[] tokens = tokenizer.tokenize("Hello world");
assertNotNull(tokens);
assertTrue(tokens.length > 0);
}
```
Contributor guide
Assessment
This issue has not been assessed yet.