[FEATURE] Can the framework be combined with Faster Transformer?
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 585
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
Is your feature request related to a problem? Please describe.
There are other speedup methods for transformers like FasterTransformer.
Describe the solution you'd like
Can you describe how your method compares to FT method and if it can be combined and potentially show an example?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review BMInf's current inference approach and NVIDIA FasterTransformer documentation first. Compare the two methods, determine whether they can be combined, and define a concrete example demonstrating the integration if it is feasible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100