vllm-project / vllm-project/aibrix

Consider to move grpc-ext-proc Server to Python Code Base

Open
#615 0 comments 0 reactions 0 assignees View on GitHub
area/gateway kind/enhancement priority/important-longterm
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 19h
Merged PRs (30d)
98

Description

### 🚀 Feature Description and Motivation

There is few cases for migrating the grpc-ext-proc server to a Python code base. This change is driven by two main factors that would significantly benefit the development and research processes related to the server.

- Python as a Fast Prototyping Language by researchers

Researchers often rely on Python for its ease of use and rapid development capabilities. This allows researchers to iterate on their prototypes faster, saving time in the development cycle. When working on the grpc-ext-proc server, the ability to quickly test and modify concepts is crucial, and Python can provide this advantage.

- Availability of Native Tools in Python

Python has native implementations of several important tools such as tokenizers and profiling tools. For example, popular natural language processing tokenizers are readily available in Python without the need for complex 3rd party bindings.

### Use Case

- Successfully migrate the grpc-ext-proc server to a Python code base.
- Ensure that all existing functionality of the server is maintained during and after the migration.
- Integrate the native Python tools (tokenizers, profiling tools) into the new Python - based server in a seamless manner.

### Proposed Solution

- Create a plan for the migration, including a breakdown of tasks, dependencies, and potential roadblocks.
- Start with grpc-ext-proc migration of a small, isolated part of the grpc-ext-proc server to assess the feasibility and potential challenges.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Begin by inventorying the existing grpc-ext-proc server and its functionality, then document migration tasks, dependencies, and roadblocks before selecting a small isolated part for a feasibility trial. Done means the server is migrated to Python, existing functionality is preserved, and the native Python tools are integrated.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, grpc, python
Domain
api, backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.