You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/vector-environment.md
+29-4Lines changed: 29 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,28 @@
4
4
5
5
The Arcade Learning Environment (ALE) Vector Environment provides a high-performance implementation for running multiple Atari environments in parallel. This implementation utilizes native C++ code with multi-threading to achieve significant performance improvements, especially when running many environments simultaneously.
6
6
7
-
The vector environment is equivalent to `FrameStackObservation(AtariPreprocessing(gym.make("ALE/{AtariGame}-v5")), stack_size=4)`.
7
+
The vector environment is equivalent to FrameStackObservation + AtariPreprocessing from Gymnasium as
8
+
```
9
+
gym_envs = gym.vector.SyncVectorEnv(
10
+
[
11
+
lambda: gym.wrappers.FrameStackObservation(
12
+
gym.wrappers.AtariPreprocessing(
13
+
gym.make(env_id, frameskip=1),
14
+
),
15
+
stack_size=stack_num,
16
+
padding_type="zero",
17
+
)
18
+
for _ in range(num_envs)
19
+
],
20
+
)
21
+
ale_envs = gym.make_vec(
22
+
env_id,
23
+
num_envs,
24
+
use_fire_reset=False,
25
+
reward_clipping=False,
26
+
repeat_action_probability=0.0,
27
+
)
28
+
```
8
29
9
30
## Key Features
10
31
@@ -19,7 +40,7 @@ The vector environment is equivalent to `FrameStackObservation(AtariPreprocessin
19
40
- Episodic life modes
20
41
-**Performance Optimizations**:
21
42
- Native C++ implementation
22
-
- Next-step autoreset (see [blog](https://farama.org/Vector-Autoreset-Mode) for more detail)
43
+
-Same-step and Next-step autoreset (see [blog](https://farama.org/Vector-Autoreset-Mode) for more detail)
23
44
- Multi-threading for parallel execution
24
45
- Thread affinity options for better performance on multi-core systems
25
46
- Batch processing capabilities
@@ -41,7 +62,7 @@ from ale_py.vector_env import VectorAtariEnv
41
62
42
63
# Create a vector environment with 4 parallel instances of Breakout
43
64
envs = VectorAtariEnv(
44
-
game="breakout", # The ROM id not name, i.e., camel case compared to Gymnasium.make name versions
65
+
game="breakout", # The ROM id not name, i.e., camel case compared to `gymnasium.make` name versions
45
66
num_envs=4,
46
67
)
47
68
@@ -73,20 +94,24 @@ envs = VectorAtariEnv(
73
94
img_height=84, # Height to resize frames to
74
95
img_width=84, # Width to resize frames to
75
96
maxpool=True, # If to maxpool sequential frames
97
+
reward_clipping=True, # If to clip environment step rewards between -1 and 1
76
98
77
99
# Environment behavior
78
100
noop_max=30, # Maximum number of no-ops at reset
79
101
fire_reset=True, # Press FIRE on reset for games that require it
80
102
episodic_life=False, # End episodes on life loss
103
+
life_loss_info=False, # Return termination signal on life loss but don't reset the environment until all lives are alot. If used, this MUST be indicated as has a significant impact on training performance.
81
104
max_episode_steps=108000, # Max frames per episode (27000 steps * 4 frame skip)
82
105
repeat_action_probability=0.0, # Sticky actions probability
83
106
full_action_space=False, # Use full action space (not minimal)
107
+
continuous=False, # If to use continuous actions
108
+
continuous_action_threshold=0.5, # The threshold at which to use continuous actions
84
109
85
110
# Performance options
86
111
batch_size=0, # Number of environments to process at once (default=0 is the `num_envs`)
112
+
autoreset_mode=gym.vector.AutoresetMode.NEXT_STEP, # How reset sub-environments when they terminated (https://farama.org/Vector-Autoreset-Mode)
87
113
num_threads=0, # Number of worker threads (0=auto)
88
114
thread_affinity_offset=-1,# CPU core offset for thread affinity (-1=no affinity)
0 commit comments