InternLM / InternLM/Tutorial

Conda环境和下载pip包(torch)在8GB A100上很慢

Open
#1,788 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2k
Forks
1.5k
PR merge metrics
No merged PRs in 30d

Description

Hello,

In 1.2. 基础关卡, 第 2 关 (the demo which gives 30% A100 computing access), it takes a very long time to set up the "demo" conda environment (approximately 5 minutes); when I install a pip package it takes 25 minutes, for 30 minutes in total.

Mainly, the issue is installing the packages; downloading works okay but when installing the collected packages it takes quite a long time.

If creating the demo conda environment takes time due to the small amount of CPU resources in the virtual machine, maybe it would be a good idea to do either or both of two things:

1. Tell learners in advance to partition some of their time specifically to only download the conda environment and LLM, or
2. Create the demo conda environment on all the 10% A100 machines (which should be possible as the download size is ~5 GB, while the total memory available to users is ~40 GB.)

Do you think this a good idea? Thank you very much!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the 1.2 Basic Level, Lesson 2 demo setup described in the issue and measure conda-environment creation and pip installation time on the 8GB and 10% A100 machines. Decide whether the work is learner guidance, pre-creating the demo environment, or both. Done means the chosen approach is implemented and the setup-time expectations or results are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
devops, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.