Allow to work OpenAI Gym
- Dominant language
- Jupyter Notebook
- Stars
- 772
- Forks
- 229
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
As a reinforcement learning researcher/developer, needs to use [OpenAI Gym](https://github.com/openai/gym) to test the algorithms because it is the common library for reinforcement learning ([4000+ citation](https://scholar.google.com/citations?view_op=view_citation&hl=ja&user=itSa94cAAAAJ&citation_for_view=itSa94cAAAAJ:HbR8gkJAVGIC) and 27000+ stars in GitHub). Studio Lab does not have enough libraries to render the OpenAI Gym environments such as `BipedalWalker-v2` that depends on BoX2D.
**Describe the solution you'd like**
Pre-install the enough libraries for OpenAI Gym or allows to install libraries by `apt`.
[From the `Dockerfile` for OpenAI Gym test environment](https://github.com/openai/gym/blob/master/py.Dockerfile#L4), we will need the following libraries.
```
apt-get -y update && apt-get install -y unzip libglu1-mesa-dev libgl1-mesa-dev libosmesa6-dev xvfb patchelf ffmpeg cmake swig
```
**Describe alternatives you've considered**
We can install some libraries from `conda`, but it did not work successfully.
For example, the following is the simple `BipedalWalker-v3` test code.
```py
import gym
import numpy as np
env = gym.make("BipedalWalker-v3")
for trial in range(1):
observation = env.reset()
done = False
limit = 10
step = 0
while not done or step < limit:
env.render()
action = env.action_space.sample()
observation, reward, done, info = env.step(action)
print(f"Step {step}: reward = {reward}")
step += 1
```
This code causes the following error despite installing `freeglut`, `ffmpeg`, `swig` from `conda`. And we will need `xvfb` to render the screen of environment.
```
Traceback (most recent call last):
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/classic_control/rendering.py", line 27, in
from pyglet.gl import *
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/__init__.py", line 95, in
from pyglet.gl.gl import *
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/gl.py", line 45, in
from pyglet.gl.lib import link_GL as _link_function
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/lib.py", line 149, in
from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/lib_glx.py", line 46, in
glu_lib = pyglet.lib.load_library('GLU')
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/lib.py", line 168, in load_library
raise ImportError('Library "%s" not found.' % names[0])
ImportError: Library "GLU" not found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/studio-lab-user/rl.py", line 13, in
env.render()
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/core.py", line 295, in render
return self.env.render(mode, **kwargs)
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/box2d/bipedal_walker.py", line 491, in render
from gym.envs.classic_control import rendering
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/classic_control/rendering.py", line 29, in
raise ImportError(
ImportError:
Error occurred while running `from pyglet.gl import *`
HINT: make sure you have OpenGL installed. On Ubuntu, you can run 'apt-get install python-opengl'.
If you're running on a server, you may need a virtual frame buffer; something like this should work:
'xvfb-run -s "-screen 0 1400x900x24" python '
```
The definition of `rl` environment to run the test code is the following.
```
name: rl
channels:
- conda-forge
dependencies:
- _libgcc_mutex=0.1=conda_forge
- _openmp_mutex=4.5=2_gnu
- aom=3.3.0=h27087fc_1
- bzip2=1.0.8=h7f98852_4
- ca-certificates=2022.5.18.1=ha878542_0
- cloudpickle=2.1.0=pyhd8ed1ab_0
- ffmpeg=5.0.1=habc3f16_3
- freeglut=3.2.2=h9c3ff4c_1
- freetype=2.10.4=h0708190_1
- gettext=0.19.8.1=h73d1719_1008
- gmp=6.2.1=h58526e2_0
- gnutls=3.7.6=hbf5b4be_4
- gym-box2d=0.21.0=py310hff52083_2
- icu=70.1=h27087fc_0
- lame=3.100=h7f98852_1001
- ld_impl_linux-64=2.36.1=hea4e1c9_2
- libblas=3.9.0=15_linux64_openblas
- libcblas=3.9.0=15_linux64_openblas
- libdrm=2.4.111=h166bdaf_0
- libffi=3.4.2=h7f98852_5
- libgcc-ng=12.1.0=h8d9b700_16
- libgfortran-ng=12.1.0=h69a702a_16
- libgfortran5=12.1.0=hdcd56e2_16
- libgomp=12.1.0=h8d9b700_16
- libiconv=1.16=h516909a_0
- libidn2=2.3.2=h7f98852_0
- liblapack=3.9.0=15_linux64_openblas
- libnsl=2.0.0=h7f98852_0
- libopenblas=0.3.20=pthreads_h78a6416_0
- libpciaccess=0.16=h516909a_0
- libpng=1.6.37=h21135ba_2
- libstdcxx-ng=12.1.0=ha89aaad_16
- libtasn1=4.18.0=h166bdaf_1
- libunistring=0.9.10=h7f98852_0
- libuuid=2.32.1=h7f98852_1000
- libva=2.14.0=h7f98852_0
- libvpx=1.11.0=h9c3ff4c_3
- libxcb=1.13=h7f98852_1004
- libxml2=2.9.14=h22db469_0
- libzlib=1.2.12=h166bdaf_0
- ncurses=6.3=h27087fc_1
- nettle=3.8=hc379101_0
- openh264=2.1.1=h780b84a_0
- openssl=3.0.3=h166bdaf_0
- p11-kit=0.23.21=hc5aa10d_4
- pip=22.1.2=pyhd8ed1ab_0
- pthread-stubs=0.4=h36c2ea0_1001
- pyopengl=3.1.6=pyhd8ed1ab_1
- python=3.10.4=h2660328_0_cpython
- python_abi=3.10=2_cp310
- readline=8.1.2=h0f457ee_0
- sqlite=3.38.5=h4ff8645_0
- svt-av1=1.1.0=h27087fc_1
- tk=8.6.12=h27826a3_0
- tzdata=2022a=h191b570_0
- wheel=0.37.1=pyhd8ed1ab_0
- x264=1!161.3030=h7f98852_1
- x265=3.5=h924138e_3
- xorg-fixesproto=5.0=h7f98852_1002
- xorg-inputproto=2.3.2=h7f98852_1002
- xorg-kbproto=1.0.7=h7f98852_1002
- xorg-libx11=1.7.2=h7f98852_0
- xorg-libxau=1.0.9=h7f98852_0
- xorg-libxdmcp=1.1.3=h7f98852_0
- xorg-libxext=1.3.4=h7f98852_1
- xorg-libxfixes=5.0.3=h7f98852_1004
- xorg-libxi=1.7.10=h7f98852_0
- xorg-xextproto=7.3.0=h7f98852_1002
- xorg-xproto=7.0.31=h7f98852_1007
- xz=5.2.5=h516909a_1
- zlib=1.2.12=h166bdaf_0
- pip:
- box2d-py==2.3.8
- future==0.18.2
- gym==0.21.0
- numpy==1.22.4
- pyglet==1.5.26
- setuptools==62.3.3
```
**Additional context**
This issue came from https://github.com/aws/studio-lab-examples/issues/118
Contributor guide
Research direction
The issue names OpenAI Gym's py.Dockerfile, the rl conda environment, and the BipedalWalker test script; start by checking how Studio Lab provisions system packages and reproducing the GLU/xvfb failure. Done means a documented or supported dependency path lets the supplied script render BipedalWalker, including the listed native libraries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- infrastructure, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100