# Markov Decision Processes (MDPs)

## Documentation for package ‘MDP2’ version 3.0.0

- DESCRIPTION file.
- User guides, package vignettes and other documentation.
- Package NEWS.

## Help Pages

binary_action_writer | Function for writing actions of a HMDP model to binary files. The function defines sub-functions which can be used to define actions saved in a set of binary files. It is assumed that the states have been defined using 'binary_mdp_writer' and that the id of the states is known (can be retrieved using e.g. 'state_idx_df').
binary_mdp_writer | Function for writing an HMDP model to binary files. The function defines sub-functions which can be used to define an HMDP model saved in a set of binary files.
convert_binary_to_hmp | Convert a HMDP model stored in binary format to a 'hmp' (XML) file. The function simply parse the binary files and create 'hmp' files using the 'hmp_mdp_writer()'.
convert_hmp_to_binary | Convert a HMDP model stored in a hmp (xml) file to binary file format.
get_bin_info_actions | Info about the actions in the HMDP model under consideration.
get_bin_info_states | Info about the states in the binary files of the HMDP model under consideration.
get_hypergraph | Return the (parts of) state-expanded hypergraph
get_info | Information about the MDP
get_policy | Get parts of the optimal policy.
get_rpo | Calculate the retention pay-off (RPO) or opportunity cost for some states.
get_steady_state_pr | Calculate the steady state transition probabilities for the founder process (level 0).
get_w_idx | Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.
hmp_mdp_writer | Function for writing an HMDP model to a hmp file (XML). The function define sub-functions which can be used to define an HMDP model stored in a hmp file.
load_mdp | Load the HMDP model defined in the binary files. The model are created in memory using the external C++ library.
memory_mdp_writer | Function for building an HMDP model directly in memory.
plot.HMDP | Plot the state-expanded hypergraph of the MDP.
plot_hypergraph | Plot parts of the state expanded hypergraph.
random_hmdp | Generate a "random" HMDP stored in a set of binary files.
run_calc_weights | Calculate weights based on current policy. Normally run after an optimal policy has been found.
run_policy_ite_ave | Perform policy iteration using the average expected-weight Bellman operator on the MDP.
run_policy_ite_discount | Perform policy iteration using the discounted expected-weight Bellman operator on the MDP.
run_value_ite | Perform value iteration on the MDP.
save_mdp | Save the MDP to binary files
set_policy | Modify the current policy by setting policy action of states.