InternLM / InternLM/lmdeploy

[Feature] Throw exception when response error

Open
#1,610 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Motivation

The class `APIClient` used requests and didn't deal with error which may lead confusing.

```python
def chat_completions_v1(self,
....
response = requests.post(self.chat_completions_v1_url,
headers=self.headers,
json=pload,
stream=stream)
for chunk in response.iter_lines(chunk_size=8192,
decode_unicode=False,
delimiter=b'\n'):
if chunk:
if stream:
decoded = chunk.decode('utf-8')
if decoded == 'data: [DONE]':
continue
if decoded[:6] == 'data: ':
decoded = decoded[6:]
output = json_loads(decoded)
yield output
else:
decoded = chunk.decode('utf-8')
output = json_loads(decoded)
yield output
```

### Related resources

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.