jina-ai / jina-ai/node-DeepResearch

[Feature] You can set your goals higher, for instance, an "AI self-evolution program"

Open
#85 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
5.2k
Forks
461
PR merge metrics
No merged PRs in 30d

Description

Your approach is very similar to CoRAG(Chain-of-Retrieval Augmented Generation technology, but it's a phase-specific function. I suggest incorporating a vector database (such as Weaviate) into your model to address the time-consuming issue of cyclic searches. The proposed solution is described as:
1. Equip your large model with a system vector database as the knowledge memory for the LLM model, to store accumulated historical knowledge data and provide knowledge for the LLM model's inference.
2. Add a small LLM model to your inference LLM model as a knowledge data filtering module, to filter out a large amount of incorrect, false, or violent information from web searches.
3. Store data retrieved from web searches in a temporary vector database in a knowledge format after filtering, allowing the model to directly retrieve information from the temporary and system vector databases during inference, reducing latency.
4. Mark erroneous knowledge discovered during inference in the temporary vector database and update it; retain correct knowledge.
5. After inference, remember to rewrite the remaining correct knowledge from the temporary knowledge base back into the system vector database when presenting results to the user.
6. Provide the knowledge base with import and export functions for external data, as the knowledge data accumulated by your LLM model is a valuable asset that can generate revenue for your company and build an AI business ecosystem.
7. Accumulated knowledge can make your LLM model's inference more accurate, improving user experience. The more it is used, the more knowledge is accumulated, and the higher the intelligence level of your LLM model.
8. Later, if you want to update your LLM model, the accumulated high-value knowledge data can be used for reinforcement learning of your LLM model, promoting the enhancement of its intelligence.
9. The ultimate rational outcome is the construction of an evolving LLM model, where accumulated knowledge can build an AI business ecosystem and generate revenue for your company.

Finally, I sincerely wish you and your team every success.
Thank you.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.