Edit on GitHub

Environment Wrappers

Wrappers provide a convenient way to alter the behavior of an environment. Wrappers.QuickWrapper provides the simplest way to override the behavior of one or more CommonRLInterface functions. For example,

QuickWrapper(env, actions=[-1, 1])

will behave just like environment env, except that the action space will only consist of -1 and 1 instead of the original action space of env. The keyword arguments may be functions or static objects.

It is also possible to easily create custom wrapper types by subtyping Wrappers.AbstractWrapper.

CommonRLInterface.Wrappers.QuickWrapper — Type
QuickWrapper(env; kwargs...)

Create a wrapper to override specific behavior of the environment with keywords.

Each keyword argument corresponds to a CommonRLInterface function to be overridden. The keyword arguments can either be static objects or functions. If a keyword argument is a function, the arguments will be the wrapped environment and any other arguments. provided is automatically handled.

Examples

Override the action space statically:

w = QuickWrapper(env; actions=[-1, 1])
observe(w) # returns the observation from env
actions(w) # returns [-1, 1]

Override the act! function to return the reward squared:

w = QuickWrapper(env; act! = (env, a) -> act!(env, a).^2)
act!(w, a) # returns the squared reward for taking action a in env
source
CommonRLInterface.Wrappers.AbstractWrapper — Type
AbstractWrapper

Abstract base class for environment wrappers. For a subtype of AbstractWrapper, all CommonRLInterface functions will be forwarded to the wrapped environment defined by wrapped_env.

Interface functions can be selectively overridden for the new wrapper type. provided and optional functions will be handled correctly by default.

Example

struct MyActionWrapper{E} <: AbstractWrapper
    env::E
end
    
# Any subclass of AbstractWrapper MUST implement wrapped_env
Wrappers.wrapped_env(w::MyActionWrapper) = w.env

# Now all CommonRLFunctions functions are forwarded
w = MyActionWrapper(env)
observe(w) # will return an observation from env
actions(w) # will return the action space from env

# The actions function for the wrapper can be overridden
CommonRLInterface.actions(w::MyActionWrapper) = [-1, 1]
actions(w) # will now return [-1, 1]
source