Interacting with Environments

Users who wish to run simulations or implement reinforcement learning algorithms with environments that others have defined should interact with environments using functions from the Required Interface and Optional Interface.

In order to achieve compatibility with the widest possible set of environments, it is critical to handle environments with incomplete implementations of the Optional Interface gracefully. The AutomaticDefault module provides functions that have default implementations based on other functions when possible, so users can call them without worrying about which optional functions the environment implements. For this reason, RL algorithm implementers should use AutomaticDefault unless there is a compelling reason not to.

CommonRLInterface.AutomaticDefault — Module

The CommonRLInterface.AutomaticDefault module contains a complete copy of all of the functions in CommonRLInterface, with the crucial difference that each of the functions will try as hard as possible to return a default value.

For example, if the environment does not have CommonRLInterface.clone implemented for it, AutomaticDefault.clone will fall back to deepcopy, and if CommonRLInterface.valid_actions is implemented, then AutomaticDefault.valid_action_mask will automatically work consistently.

Environment implementers should implement new methods for functions in CommonRLInterface and should not deal with AutomaticDefault; environment users (e.g. RL algorithm implementers) should use AutomaticDefault for the widest compatibility possible with different environments (or use CommonRLInterface.provided to manually adapt to the environment's capabilities).

Example

import CommonRLInterface
using CommonRLInterface.AutomaticDefault
using Test

struct ExampleEnv <: CommonRLInterface.AbstractEnv end

CommonRLInterface.actions(::ExampleEnv) = 1:2

# CommonRLInterface.valid_actions will not work, because valid_actions was not defined 
@test_throws MethodError CommonRLInterface.valid_actions(ExampleEnv())

# But, this version from AutomaticDefault will work by falling back to `actions`:
valid_actions(ExampleEnv())

# output

1:2

source