aws / aws/studio-lab-examples

Allow to work OpenAI Gym

Open
#124 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
772
Forks
229
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**

As a reinforcement learning researcher/developer, needs to use [OpenAI Gym](https://github.com/openai/gym) to test the algorithms because it is the common library for reinforcement learning ([4000+ citation](https://scholar.google.com/citations?view_op=view_citation&hl=ja&user=itSa94cAAAAJ&citation_for_view=itSa94cAAAAJ:HbR8gkJAVGIC) and 27000+ stars in GitHub). Studio Lab does not have enough libraries to render the OpenAI Gym environments such as `BipedalWalker-v2` that depends on BoX2D.

**Describe the solution you'd like**
Pre-install the enough libraries for OpenAI Gym or allows to install libraries by `apt`.
[From the `Dockerfile` for OpenAI Gym test environment](https://github.com/openai/gym/blob/master/py.Dockerfile#L4), we will need the following libraries.

```
apt-get -y update && apt-get install -y unzip libglu1-mesa-dev libgl1-mesa-dev libosmesa6-dev xvfb patchelf ffmpeg cmake swig
```

**Describe alternatives you've considered**
We can install some libraries from `conda`, but it did not work successfully.

For example, the following is the simple `BipedalWalker-v3` test code.

```py
import gym
import numpy as np

env = gym.make("BipedalWalker-v3")

for trial in range(1):
observation = env.reset()
done = False
limit = 10
step = 0
while not done or step < limit:
env.render()
action = env.action_space.sample()
observation, reward, done, info = env.step(action)
print(f"Step {step}: reward = {reward}")
step += 1
```

This code causes the following error despite installing `freeglut`, `ffmpeg`, `swig` from `conda`. And we will need `xvfb` to render the screen of environment.

```
Traceback (most recent call last):
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/classic_control/rendering.py", line 27, in
from pyglet.gl import *
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/__init__.py", line 95, in
from pyglet.gl.gl import *
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/gl.py", line 45, in
from pyglet.gl.lib import link_GL as _link_function
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/lib.py", line 149, in
from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/gl/lib_glx.py", line 46, in
glu_lib = pyglet.lib.load_library('GLU')
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/pyglet/lib.py", line 168, in load_library
raise ImportError('Library "%s" not found.' % names[0])
ImportError: Library "GLU" not found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/studio-lab-user/rl.py", line 13, in
env.render()
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/core.py", line 295, in render
return self.env.render(mode, **kwargs)
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/box2d/bipedal_walker.py", line 491, in render
from gym.envs.classic_control import rendering
File "/home/studio-lab-user/.conda/envs/rl/lib/python3.10/site-packages/gym/envs/classic_control/rendering.py", line 29, in
raise ImportError(
ImportError:
Error occurred while running `from pyglet.gl import *`
HINT: make sure you have OpenGL installed. On Ubuntu, you can run 'apt-get install python-opengl'.
If you're running on a server, you may need a virtual frame buffer; something like this should work:
'xvfb-run -s "-screen 0 1400x900x24" python '
```

The definition of `rl` environment to run the test code is the following.

```
name: rl
channels:
- conda-forge
dependencies:
- _libgcc_mutex=0.1=conda_forge
- _openmp_mutex=4.5=2_gnu
- aom=3.3.0=h27087fc_1
- bzip2=1.0.8=h7f98852_4
- ca-certificates=2022.5.18.1=ha878542_0
- cloudpickle=2.1.0=pyhd8ed1ab_0
- ffmpeg=5.0.1=habc3f16_3
- freeglut=3.2.2=h9c3ff4c_1
- freetype=2.10.4=h0708190_1
- gettext=0.19.8.1=h73d1719_1008
- gmp=6.2.1=h58526e2_0
- gnutls=3.7.6=hbf5b4be_4
- gym-box2d=0.21.0=py310hff52083_2
- icu=70.1=h27087fc_0
- lame=3.100=h7f98852_1001
- ld_impl_linux-64=2.36.1=hea4e1c9_2
- libblas=3.9.0=15_linux64_openblas
- libcblas=3.9.0=15_linux64_openblas
- libdrm=2.4.111=h166bdaf_0
- libffi=3.4.2=h7f98852_5
- libgcc-ng=12.1.0=h8d9b700_16
- libgfortran-ng=12.1.0=h69a702a_16
- libgfortran5=12.1.0=hdcd56e2_16
- libgomp=12.1.0=h8d9b700_16
- libiconv=1.16=h516909a_0
- libidn2=2.3.2=h7f98852_0
- liblapack=3.9.0=15_linux64_openblas
- libnsl=2.0.0=h7f98852_0
- libopenblas=0.3.20=pthreads_h78a6416_0
- libpciaccess=0.16=h516909a_0
- libpng=1.6.37=h21135ba_2
- libstdcxx-ng=12.1.0=ha89aaad_16
- libtasn1=4.18.0=h166bdaf_1
- libunistring=0.9.10=h7f98852_0
- libuuid=2.32.1=h7f98852_1000
- libva=2.14.0=h7f98852_0
- libvpx=1.11.0=h9c3ff4c_3
- libxcb=1.13=h7f98852_1004
- libxml2=2.9.14=h22db469_0
- libzlib=1.2.12=h166bdaf_0
- ncurses=6.3=h27087fc_1
- nettle=3.8=hc379101_0
- openh264=2.1.1=h780b84a_0
- openssl=3.0.3=h166bdaf_0
- p11-kit=0.23.21=hc5aa10d_4
- pip=22.1.2=pyhd8ed1ab_0
- pthread-stubs=0.4=h36c2ea0_1001
- pyopengl=3.1.6=pyhd8ed1ab_1
- python=3.10.4=h2660328_0_cpython
- python_abi=3.10=2_cp310
- readline=8.1.2=h0f457ee_0
- sqlite=3.38.5=h4ff8645_0
- svt-av1=1.1.0=h27087fc_1
- tk=8.6.12=h27826a3_0
- tzdata=2022a=h191b570_0
- wheel=0.37.1=pyhd8ed1ab_0
- x264=1!161.3030=h7f98852_1
- x265=3.5=h924138e_3
- xorg-fixesproto=5.0=h7f98852_1002
- xorg-inputproto=2.3.2=h7f98852_1002
- xorg-kbproto=1.0.7=h7f98852_1002
- xorg-libx11=1.7.2=h7f98852_0
- xorg-libxau=1.0.9=h7f98852_0
- xorg-libxdmcp=1.1.3=h7f98852_0
- xorg-libxext=1.3.4=h7f98852_1
- xorg-libxfixes=5.0.3=h7f98852_1004
- xorg-libxi=1.7.10=h7f98852_0
- xorg-xextproto=7.3.0=h7f98852_1002
- xorg-xproto=7.0.31=h7f98852_1007
- xz=5.2.5=h516909a_1
- zlib=1.2.12=h166bdaf_0
- pip:
- box2d-py==2.3.8
- future==0.18.2
- gym==0.21.0
- numpy==1.22.4
- pyglet==1.5.26
- setuptools==62.3.3
```

**Additional context**

This issue came from https://github.com/aws/studio-lab-examples/issues/118

Contributor guide

Open the contributing guide

Research direction

The issue names OpenAI Gym's py.Dockerfile, the rl conda environment, and the BipedalWalker test script; start by checking how Studio Lab provisions system packages and reproducing the GLU/xvfb failure. Done means a documented or supported dependency path lets the supplied script render BipedalWalker, including the listed native libraries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
infrastructure, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.