Flow Module

baconian.core.flow.train_test_flow.Flow

class baconian.core.flow.train_test_flow.Flow(func_dict)

Interface of experiment flow module, it defines the workflow of the reinforcement learning experiments.

__init__(func_dict)

Constructor for Flow.

Parameters:func_dict (dict) – the function and its arguments that will be called in the Flow
_call_func(key, **extra_kwargs)

Call a function that is pre-defined in self.func_dict

Parameters:
  • key (str) – name of the function, e.g., train, test, sample.
  • extra_kwargs – some extra kwargs you may want to be passed in the function calling
Returns:

actual return value of the called function if self.func_dict has such function otherwise None.
Return type:
_launch() → bool

Abstract method to be implemented by subclass for a certain workflow.

Returns:True if the flow correctly executed and finished
Return type:bool
launch() → bool

Launch the flow until it finished or catch a system-allowed errors (e.g., out of GPU memory, to ensure the log will be saved safely).

Returns:True if the flow correctly executed and finished
Return type:bool
required_func = ()
required_key_dict = {}
class baconian.core.flow.train_test_flow.TrainTestFlow(train_sample_count_func, config_or_config_dict: (<class 'baconian.config.dict_config.DictConfig'>, <class 'dict'>), func_dict: dict)

A typical sampling-trainning and testing workflow, that used by most of model-free/model-based reinforcement learning method. Typically, it repeat the sampling(saving to memory if off policy)->training(from memory if off-policy, from samples if on-policy)->test

__init__(train_sample_count_func, config_or_config_dict: (<class 'baconian.config.dict_config.DictConfig'>, <class 'dict'>), func_dict: dict)

Constructor of TrainTestFlow

Parameters:
  • train_sample_count_func (method) – a function indicates how much training samples the agent has collected currently.
  • config_or_config_dict (Config or dict) – a Config or a dict should have the keys: (TEST_EVERY_SAMPLE_COUNT, TRAIN_EVERY_SAMPLE_COUNT, START_TRAIN_AFTER_SAMPLE_COUNT, START_TEST_AFTER_SAMPLE_COUNT)
  • func_dict (dict) – function dict, holds the keys: ‘sample’, ‘train’, ‘test’. each item in the dict as also should be a dict, holds the keys ‘func’, ‘args’, ‘kwargs’
_is_ended()
Returns:True if an experiment is ended
Return type:bool
_launch() → bool

Launch the flow until it finished or catch a system-allowed errors (e.g., out of GPU memory, to ensure the log will be saved safely).

Returns:True if the flow correctly executed and finished
Return type:bool
required_func = ('train', 'test', 'sample')
required_key_dict = {'START_TEST_AFTER_SAMPLE_COUNT': 1, 'START_TRAIN_AFTER_SAMPLE_COUNT': 1, 'TEST_EVERY_SAMPLE_COUNT': 1000, 'TRAIN_EVERY_SAMPLE_COUNT': 1000}

baconian.core.flow.dyna_flow.DynaFlow

class baconian.core.flow.dyna_flow.DynaFlow(train_sample_count_func, config_or_config_dict: (<class 'baconian.config.dict_config.DictConfig'>, <class 'dict'>), func_dict: dict)

A typical flow for utilizing the model-based algo, it is not restricted to Dyna algorithms, but can be utilized by others.

__init__(train_sample_count_func, config_or_config_dict: (<class 'baconian.config.dict_config.DictConfig'>, <class 'dict'>), func_dict: dict)
Parameters:
  • train_sample_count_func (method) – a function indicates how much training samples the agent has collected currently.
  • config_or_config_dict (Config or dict) – a Config or a dict should have the keys: (TEST_EVERY_SAMPLE_COUNT, TRAIN_EVERY_SAMPLE_COUNT, START_TRAIN_AFTER_SAMPLE_COUNT, START_TEST_AFTER_SAMPLE_COUNT)
  • func_dict (dict) – function dict, holds the keys: ‘sample’, ‘train’, ‘test’. each item in the dict as also should be a dict, holds the keys ‘func’, ‘args’, ‘kwargs’
_is_ended()
Returns:True if an experiment is ended
Return type:bool
_launch() → bool

Launch the flow until it finished or catch a system-allowed errors (e.g., out of GPU memory, to ensure the log will be saved safely).

Returns:True if the flow correctly executed and finished
Return type:bool
required_func = ('train_algo', 'train_algo_from_synthesized_data', 'train_dynamics', 'test_algo', 'test_dynamics', 'sample_from_real_env', 'sample_from_dynamics_env')
required_key_dict = {'START_TEST_ALGO_AFTER_SAMPLE_COUNT': 1, 'START_TEST_DYNAMICS_AFTER_SAMPLE_COUNT': 1, 'START_TRAIN_ALGO_AFTER_SAMPLE_COUNT': 1, 'START_TRAIN_DYNAMICS_AFTER_SAMPLE_COUNT': 1, 'TEST_ALGO_EVERY_REAL_SAMPLE_COUNT': 1000, 'TEST_DYNAMICS_EVERY_REAL_SAMPLE_COUNT': 1000, 'TRAIN_ALGO_EVERY_REAL_SAMPLE_COUNT_FROM_DYNAMICS_ENV': 1000, 'TRAIN_ALGO_EVERY_REAL_SAMPLE_COUNT_FROM_REAL_ENV': 1000, 'TRAIN_DYNAMICS_EVERY_REAL_SAMPLE_COUNT': 1000, 'WARM_UP_DYNAMICS_SAMPLES': 1000}