beehive-lab / beehive-lab/GPULlama3.java

Add Junit5 and a set of initial tests

Open
#14 3 comments 0 reactions 2 assignees Claimed by @ayush0407 View on GitHub
good first issue
Dominant language
Java
Stars
274
Forks
37
Avg merge
5d 1h
Merged PRs (30d)
2

Description

## Description:
This repo currently doesn’t have any unit tests. Let’s add JUnit 5 support and create an initial test for the Tokenizer class to ensure tokenization behaves as expected.

## Tasks:

* Add JUnit 5 as a dependency using Maven.
* Create a test class:
src/test/java/llama/TokenizerTest.java
* Add a test that checks tokenization for a simple prompt. Example:

```java
@Test
void testBasicTokenization() {
Tokenizer tokenizer = new Tokenizer("path/to/tokenizer.model");
int[] tokens = tokenizer.tokenize("Hello world");
assertNotNull(tokens);
assertTrue(tokens.length > 0);
}

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.