Edit on GitHub

Optional Interface

Some environments can provide additional information or behavior beyond what can be expressed with the Required Interface. For this reason, CommonRLInterface has an interface of optional functions.

Determining what an environment has: provided

Tip

The AutomaticDefault module provides a convenient way to access optional functions that is an alternative to calling provided manually.

When other code interacts with an environment, it is common to adjust behavior based on the capabilities that environment provides. The provided function is a programmatic way to determine whether the environment author has implemented certain optional behavior. For instance, an algorithm author might only want to consider the valid subset of actions at the current state if the valid_actions optional function is implemented. This can be accomplished with the following code:

if provided(valid_actions, env)
    acts = valid_actions(env)
else
    acts = actions(env)
end

(or by using AutomaticDefault.valid_actions). In most cases, provided can be statically optimized out, so it will have no performance impact.

Optional Function List

The optional interface currently contains the following functions:

Additional optional functions for multiplayer environments are contained in the Multiplayer Interface

To propose adding a new function to the interface, please file an issue with the "candidate interface function" label.

CommonRLInterface.provided — Function
provided(f, args...)

Test whether an implementation for f(args...) has been provided.

If this returns false, you should assume that the environment does not support the function f.

Usage is identical to Base.applicable. The default implementation should be correct for most cases. It should only be implemented by the user in exceptional cases where very fine control is needed.

provided(f, types::Type{<:Tuple})

Alternate version of provided with syntax similar to Base.hasmethod.

If this returns true, it means that the function is provided for any set of arguments with the given types. If it returns false, it may be provided for certain objects. For this reason, algorithm writers should call the provided(f, args...) version when possible.

source
CommonRLInterface.clone — Function
clone(env)

Create a clone of CommonEnv env at the current state.

Two clones are assumed to be completely independent of each other - no action applied to one will affect the other.

source
CommonRLInterface.render — Function
render(env)

Return a showable object that visualizes the environment at the current state.

Implementing this function will facilitate visualization through the Julia Multimedia I/O system, for example, calling display(render(env)) will cause the visualization to pop up in a Jupyter notebook, the Juno plot pane, or an ElectronDisplay.jl window. render should return an object that has Base.show methods for many MIME types. Good examples include a Plots.jl plot, a Compose.jl Context graphic produced by Cairo, or a custom type with show methods.

Example

using CommonRLInterface
using Plots

struct MyEnv <: AbstractEnv
    state::Tuple{Int, Int}
end

CommonRLInterface.render(env::MyEnv) = scatter(env.state[1], env.state[2])
source
CommonRLInterface.state — Function
state(env::AbstractEnv)

Return the state of the environment.

See setstate! for more information about the state.

source
CommonRLInterface.setstate! — Function
setstate!(env::AbstractEnv, s)

Set the state of the environment to s.

If two environments have the same state, future outputs (observations and rewards) should be statistically identical given the same sequence of actions.

source
CommonRLInterface.valid_actions — Function
valid_actions(env)

Return a collection of actions that are appropriate for the current state and player. This should be a subset of actions(env).

source
CommonRLInterface.valid_action_mask — Function
valid_action_mask(env)

Return a mask (any AbstractArray{Bool}) indicating which actions are valid at the current state and player.

This function only applies to environments with finite action spaces. If both valid_actions and valid_action_mask are provided, valid_actions(env) should return the same set as actions(env)[valid_action_mask(env)].

source
CommonRLInterface.observations — Function
observations(env)

Return a collection of all observations that might be returned by observe(env).

This function is a static property of the environment; the value it returns should not change based on the state.

source