smartstart.utilities package¶

Submodules¶

smartstart.utilities.datacontainers module¶

Datacontainers module

Defines datacontainer classes training data.

class Episode¶

Bases: object

Datacontainer for episode data

obs¶: list of np.ndarray – observations

action¶: list of int – actions

reward¶: list of float – rewards

obs_tp1¶: list of np.ndarray – next observations

done¶: list of bool – dones

append(obs, action, reward, obs_tp1, done)¶

Add transition to episode

Parameters:	obs (`np.ndarray`) – action (`int`) – reward (`float`) – obs_tp1 (`np.ndarray`) – done (`bool`) –

average_reward()¶

Average episode reward

Returns:	average reward
Return type:	`float`

classmethod from_json(data)¶

Coverts JSON string into Episode object

Parameters:	data (`str`) – JSON string
Returns:	new episode object
Return type:	`Episode`

to_json()¶

Convert episode data to json string

Returns:	JSON string of episode data
Return type:	`str`

total_reward()¶

Total episode reward

Returns:	total reward
Return type:	`float`

class Summary(name=None)¶

Bases: object

Datacontainer for complete training session

Parameters:	name (`str`) – name of the summary, used for saving the data (Default = None)

name¶: str – name of the summary, used for saving the data (Default = None)

episodes¶: list of tuple with episode data

append(episode)¶

Adds the length and total reward of episode to summary

Parameters:	episode (`Episode`) – episode object

average_episode_reward()¶

Average reward per episode

Returns:	average reward per episode
Return type:	`list` of `float`

average_reward()¶

Average reward of all episodes

Returns:	average reward
Return type:	`float`

classmethod from_json(data)¶

Convert JSON string into Summary object

Parameters:	data (`str`) – JSON string with summary data
Returns:	new Summary object
Return type:	`Summary`

classmethod load(fp)¶

Loads a summary from the defined filepath

Parameters:	fp (`str`) – filepath to the summary
Returns:	new summary object
Return type:	`Summary`

save(directory='/home/bartkeulen/repositories/smartstart/smartstart/utilities/../../data/tmp', post_fix=None)¶

Save summary as json file

The summary name is used as filename, an optional postfix can be added to the end of the summary name.

Parameters:	directory (`str`) – directory to save the summary (Default value = DIR) post_fix (`str`) – post_fix to add to the end of the summary name (Default value = None)
Returns:	full filepath to the saved json summary
Return type:	`str`

save_to_gcloud(bucket_name, directory, post_fix=None)¶

Save summary in a Google Cloud Bucket

Note

Google cloud SDK has to be installed and initialized before function can be used. Click here for more information.

Parameters:	directory (`str`) – directory to save the summary (Default value = DIR) post_fix (`str`) – post_fix to add to the end of the summary name (Default value = None)

steps_episode()¶

Number of steps per episode

Returns:	steps per episode
Return type:	`list` of `int`

to_json()¶

Convert summary data to JSON string

Returns:	JSON string of summary data
Return type:	`str`

total_episode_reward()¶

Total reward per episode

Returns:	total reward per episode
Return type:	`list` of `float`

total_reward()¶

Total reward of all episodes

Returns:	total reward
Return type:	`float`

smartstart.utilities.experimenter module¶

Module for making it easy to run experiments

This methods defined in this module make it easy to run multiple experiments in parallel and using different sets of parameters.

process_task(params)¶

Helper method for executing the task defined in params

Parameters:	params (`dict`) – dictionary with the parameters to be used in the experiment

run_experiment(param_grid, n_processes=-1)¶

Method for running experiments

This method is used for making it easy to run experiments in parallel and use different combinations of parameters. The param_grid dictionary describes what task or tasks must be executed and what parameters and/or parameter combinations must be used. Each experiment receives its own unique id.

The param_grid consists of the following entries:

task : function that has to be executed
num_exp : number of time each experiment has to be executed (optional)
params : lists holding the params that have to be used

Example of a param_grid:

param_grid = {‘task’: task_func, ‘num_exp’: 5, ‘alpha’: [0.05, 0.1]}

and the task func should then be in the following format:

def task_func(params):

# Define the code that has to be executed params defined can be # accessed like this alpha = params[‘alpha’]

# a run parameter, useful when num_exp > 1 run = param[‘run’]

# and the unique id id = params[‘id’]

In total the task function will be executed 10 times. 5 times with each alpha parameter. See tutorial or examples for a full example.

Parameters:	param_grid (`dict`) – dictionary containing the parameters to be used during the experiments n_processes (`int`) – number of processes to run in parallel (Default value = -1, all the cpu_cores in the system are used)
Raises:	`Exception` – Please define a task function in the param_grid to execute.

smartstart.utilities.numerical module¶

Module for numerical operations

moving_average(values, window=10)¶

Moving average filter

Parameters:	values (`np.ndarray`) – array the moving average filter has to applied on window (`int`) – window size for the moving average filter (Default value = 10)
Returns:	values after applying moving average filter (length = orignal_length - window + 1)
Return type:	`np.ndarray`

smartstart.utilities.plot module¶

Module for plotting results

This module describes methods for generating a few predefined plots. The methods make it easy to plot multiple different experiments and average the results of experiments with the same parameters.

The results are Summary objects saved as JSON strings.

mean_reward_episode(summaries, ma_window=1, color=None, linestyle=None)¶

Plot mean reward per episode

Parameters:	summaries (`list` of `Summary`) – summaries to average and plot ma_window (`int`) – moving average window size (Default value = 1) color – color (Default value = None) linestyle – linestyle (Default value = None)

mean_reward_std_episode(summaries, ma_window=1, color=None, linestyle=None)¶

Plot mean reward with standard deviation per episode

Parameters:	summaries (`list` of `Summary`) – summaries to average and plot ma_window (`int`) – moving average window size (Default value = 1) color – color (Default value = None) linestyle – linestyle (Default value = None)

plot_summary(files, plot_type, ma_window=1, title=None, legend=None, output_dir=None, colors=None, linestyles=None, format='eps', baseline=None)¶

Main plot function to be used

The files parameter can be a list of files or a list of Summary objects that you want to compare in a single plot. A single file or single Summary can also be provided. Please read the instructions below when supplying a list of files.

The files list provided must contain filenames without the .json extension. For example: ['file/path/to/experiment'] is correct but [ 'file/path/to/experiment.json'] not! The reason for this is when the folder contains multiple summary files from the same experiment (same parameters) it will use all the files and average them. For example when the folder contains the following three files [ 'file/path/to/experiment_1.json', 'file/path/to/experiment_2.json', 'file/path/to/experiment_3.json']. By providing [ 'file/path/to/experiment'] all three summaries will be loaded and averaged.

Note

The entries in files have to be defined without .json at the end.

Note

Don’t forget to run the show_plot() function after initializing the plots. Else nothing will be rendered on screen

Parameters:	files (`list` of `str` or `list` of) –

:param Summary: Option 1: each entry is the filepath to a saved summary without: .json at the end. Option 2: each entry is a Summary object.

Parameters:

plot_type – one of the plot functions defined in this module
ma_window (int) – moving average filter window size (Default value = 10)
title (str) – title of the plot, is also used as filename (Default value = None)
legend (list of str) – one entry per entry in files (Default value = None)
output_dir (str) – if not None the plot will be saved in this directory (Default value = None)
colors (list) – one entry per entry in files (Default value = None)
linestyles (list) – one entry per entry in files (Default value = None)
format (str) – output format when saving plot (Default value = “eps”)
baseline (float) – plotting a dotted horizontal line as baseline (Default value = None)

save_plot(output_dir, title, format='eps')¶

Helper method for saving plots

Parameters:	output_dir (`str`) – directory where the plot is saved title (`str`) – filename of saved plot format (`str`) – file format (Default value = “eps”)
Raises:	`Exception` – Please give a title when saving a figure.

show_plot()¶

Render the plots on screen

Must be run after initializing the plots to actually show them on screen.

steps_episode(summaries, ma_window=1, color=None, linestyle=None)¶

Plot number of steps per episode

Parameters:	summaries (`list` of `Summary`) – summaries to average and plot ma_window (`int`) – moving average window size (Default value = 1) color – color (Default value = None) linestyle – linestyle (Default value = None)

smartstart.utilities.scheduler module¶

Module for schedulers

Schedulers can be used to gradually decrease a variable over time. The scheduler will return a scale parameter that can be multiplied with the original value.

class LinearScheduler(start, end)¶

Bases: smartstart.utilities.scheduler.Scheduler

Returns a scale parameter that linearly decreases from start to end

Every time the sample method is called the count is increased. When the count = start the scale will decrease from one to zero between count = start and count = end.

Parameters:	start (`int`) – start index of the scheduler end (`int`) – final index of the scheduler

start¶: int – start index of the scheduler

end¶: int – final index of the scheduler

count¶: int – count

scale¶: float – current scale factor

step¶: float – step-size by which scale is decreased

sample()¶: Returns the scale factor

class Scheduler¶

Bases: object

Base implementation of a Scheduler

sample()¶: Returns the scale factor

smartstart.utilities.utilities module¶

Module that contains random utility functions used in the smartstart package

get_data_directory(file)¶

Creates and returns a data directory at the file’s location

Parameters:	file – python file
Returns:	filepath to the data directory
Return type:	`str`

smartstart.utilities package¶

Submodules¶

smartstart.utilities.datacontainers module¶

smartstart.utilities.experimenter module¶

smartstart.utilities.numerical module¶

smartstart.utilities.plot module¶

smartstart.utilities.scheduler module¶

smartstart.utilities.utilities module¶

Module contents¶