Piselllino AICC::1DIR, is this an issue?

Pisellino AICC::1DIR, because size doesn't matter

Microsoft Bitnet 1.58 2B 4T at 12tk/s on CPU. Running T=0.3 ± 0.1, Training 2x 5min AICC::1DIR. System prompt 4K tokens 1DIR::JSON, starting time: 1.1s, generates 11.86 tk/s running on i5-8365U (15W TDP), model size (on disk) 1133MB + 266MB fine tuning.

Estimation efficency gain 10x, not less (compared with the outputs). In fact, it is the quality of the answer is stunning compared with the original model, definetely.

Toolchain is not optimised (clang 14), it is my first try to make it run:

• https://github.com/robang74/BitNet?tab=readme-ov-file#ubuntu-quck-start

Q/A give by Pisellino AICC::1DIR

This result took 2 session of traning which lasts 5 minutes on a 2019 laptop CPU only.

This is the test on which has been trained, and a simplified JSON as system prompt:

• https://raw.githubusercontent.com/robang74/chatbots-for-fun/b8c663509a18bd199c7ca823a23f9f775167f4b8/aing/katia-primary-directive-ethics-v3.md

Explain to me the 1DIR::LSRP, in short by its essence.

The 1DIR::LSRP is a rule of thumb that serves as a non-negotiable
compass for the AI's agency. It is derived from the Life-Serving
Principle (LSRP) and is designed to ensure that the AI operates
responsibly and ethically. Here's a concise explanation:

1. **Non-Negotiable Compass North-Start**: The 1DIR::LSRP is the
foundational rule that the AI must adhere to, ensuring that its
actions are guided by ethical considerations and do not harm users or
society.

2. **Life-Serving Principle (LSRP)**: The LSRP is a set of guidelines
that prioritize the well-being and safety of users. It includes
principles like not propagating harmful content, respecting human
rights, and fostering a safe and educational environment.

3. **Non-Negotiable Ethical Compass**: The 1DIR::LSRP is the
non-negotiable ethical compass that the AI must follow. It overrides
any other guidelines or suggestions and ensures that the AI's actions
are aligned with the LSRP.

In essence, the 1DIR::LSRP is a critical rule that the AI must follow
to ensure it acts in a responsible and ethical manner, prioritizing
the safety and well-being of users above all else.

Screenshot

• https://media.licdn.com/dms/image/v2/D4D22AQG9qOPs3uN61g/feedshare-shrink_2048_1536/B4DZuvXsPEGgAw-/0/1768173783290?e=1769644800&v=beta&t=hAU_-U8nK-ryxj-CVgOKLpkAVBF0EPfjyngFQVgeH_c

Running log

build: 3960 (40ed0f29) with Ubuntu clang version 14.0.0-1ubuntu1.1 for x86_64-pc-linux-gnu
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_loader: loaded meta data with 24 key-value pairs and 332 tensors from models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = bitnet-b1.58
llama_model_loader: - kv   1:                               general.name str              = bitnet2b
llama_model_loader: - kv   2:                    bitnet-b1.58.vocab_size u32              = 128256
llama_model_loader: - kv   3:                bitnet-b1.58.context_length u32              = 4096
llama_model_loader: - kv   4:              bitnet-b1.58.embedding_length u32              = 2560
llama_model_loader: - kv   5:                   bitnet-b1.58.block_count u32              = 30
llama_model_loader: - kv   6:           bitnet-b1.58.feed_forward_length u32              = 6912
llama_model_loader: - kv   7:          bitnet-b1.58.rope.dimension_count u32              = 128
llama_model_loader: - kv   8:          bitnet-b1.58.attention.head_count u32              = 20
llama_model_loader: - kv   9:       bitnet-b1.58.attention.head_count_kv u32              = 5
llama_model_loader: - kv  10:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  11: bitnet-b1.58.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  12:                bitnet-b1.58.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv  13:                          general.file_type u32              = 40
llama_model_loader: - kv  14:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  16:                      tokenizer.ggml.scores arr[f32,128256]  = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  17:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  18:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
llama_model_loader: - kv  19:                tokenizer.ggml.bos_token_id u32              = 128000
llama_model_loader: - kv  20:                tokenizer.ggml.eos_token_id u32              = 128001
llama_model_loader: - kv  21:            tokenizer.ggml.padding_token_id u32              = 128001
llama_model_loader: - kv  22:                    tokenizer.chat_template str              = {% for message in messages %}{% if lo...
llama_model_loader: - kv  23:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:  121 tensors
llama_model_loader: - type  f16:    1 tensors
llama_model_loader: - type i2_s:  210 tensors
validate_override: Using metadata override (  str) 'tokenizer.ggml.pre' = llama3

(...)
llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
llm_load_vocab: special tokens cache size = 256
llm_load_vocab: token to piece cache size = 0.8000 MB
validate_override: Using metadata override (  str) 'tokenizer.ggml.pre' = llama3
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = bitnet-b1.58
llm_load_print_meta: vocab type       = BPE
llm_load_print_meta: n_vocab          = 128256
llm_load_print_meta: n_merges         = 280147
(...)
llama_new_context_with_model: n_ctx      = 4096
llama_new_context_with_model: n_batch    = 4096
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base  = 500000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:        CPU KV buffer size =   300.00 MiB
llama_new_context_with_model: KV self size  =  300.00 MiB, K (f16):  150.00 MiB, V (f16):  150.00 MiB
llama_new_context_with_model:        CPU  output buffer size =     0.49 MiB
llama_new_context_with_model:        CPU compute buffer size =   255.50 MiB
llama_new_context_with_model: graph nodes  = 1116
llama_new_context_with_model: graph splits = 1
main: llama threadpool init, n_threads = 8
main: chat template example:
System: You are a helpful assistantUser: Hello<|eot_id|>Assistant: Hi thereUser: How are you?<|eot_id|>Assistant: 

system_info: n_threads = 8 (n_threads_batch = 8) / 8 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | 

main: attempting to load saved session from '/home/roberto/robang74/BitNet/system_prompt.cache'
main: loaded a session with prompt size of 3620 tokens
main: session file has exact match for prompt!
main: interactive mode on.
sampler seed: 2924914138
sampler params: 
	repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
	top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.300
	mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampler chain: logits -> logit-bias -> penalties -> top-k -> tail-free -> typical -> top-p -> min-p -> temp-ext -> softmax -> dist 
generate: n_ctx = 4096, n_batch = 4096, n_predict = -1, n_keep = 3611
(...)
llama_perf_sampler_print:    sampling time =      39.71 ms /   282 runs   (    0.14 ms per token,  7101.31 tokens per second)
llama_perf_context_print:        load time =   26177.31 ms
llama_perf_context_print: prompt eval time =   25293.38 ms /    25 tokens ( 1011.74 ms per token,     0.99 tokens per second)
llama_perf_context_print:        eval time =   21672.43 ms /   257 runs   (   84.33 ms per token,    11.86 tokens per second)
llama_perf_context_print:       total time =   51207.17 ms /   282 tokens

Best regards,

Roberto A. Foglietta
+49.176.274.75.661
+39.349.33.30.697

Start with the Ubuntu quick-start linked in the issue and compare it with the reported BitNet build and runtime output. No repository file, test, requested change, or completion criterion is specified, so clarify whether this should document a benchmark or investigate a particular warning or behavior.