InternLM / InternLM/InternLM-XComposer

The model's Grounding capability is still unsatisfactory.(模型的Grounding能力不尽人意)

Open
#329 0 comments 0 reactions 1 assignee Claimed by @LightDXY View on GitHub
Dominant language
Python
Stars
2.9k
Forks
175
PR merge metrics
No merged PRs in 30d

Description

The grounding ability of the fine-tuned model still falls short of meeting production requirements, showing a significant gap compared to the CogAgent model.

## examples

![68e2fb4e6c95e66c829f8992aa6fb5a1](https://github.com/InternLM/InternLM-XComposer/assets/29700371/8f74c1cb-b558-45de-8858-0604620d9f9d)
{"query": " In the photograph, could you pinpoint the location of \"ACHADOS E PERDIDOS\" and tell me its bounding boxes?", "label": "The bounding box is [475, 12, 578, 28]", "response": "The bounding box is [528, 69, 643, 119]"}

![a44b4236091c5d3169ae89a3d4e815a2](https://github.com/InternLM/InternLM-XComposer/assets/29700371/ecf614db-5b56-4f23-8875-ac833f4a12fe)
{"query": " In, can you guide me to the location of \"THE BIG WAVES JOURNAL\" by providing bounding boxes?", "label": "The bounding box is [522, 0, 628, 82]", "response": "The bounding box is [593, 88, 680, 123]"}

![fc90916946817c44ca102f46343a3698](https://github.com/InternLM/InternLM-XComposer/assets/29700371/0d928719-71d2-4f75-8a40-aa3f7376ad50)

{"query": " Help me to locate \"Vinyl Fencing\" in and give me its bounding boxes, please.", "label": "The bounding box is [329, 803, 375, 819]", "response": "The bounding box is [352, 839, 423, 857]"}

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.