\n“How can machines learn by trial and error? Reinforcement learning opens up ways to automate complex decisions in dynamic environments.”<\/em><\/p>\n<\/blockquote>\n<\/span>Where is reinforcement learning relevant?<\/strong><\/span><\/h2>\nRL has applications in robotics, autonomous driving, game development (e.g. AlphaGo, OpenAI Five), financial optimization and industrial processes.<\/p>\n\n<\/div>
<\/span>How does reinforcement learning work?<\/strong><\/span><\/h2>\nReinforcement learning is based on a trial-and-error approach<\/strong> in which the agent performs actions, receives feedback from the environment and learns from it.<\/p>\n<\/span>The four iterative steps in the RL process<\/strong><\/span><\/h3>\nA compact overview of the iterative process of reinforcement learning:<\/p>\n
\n- Interaction:<\/strong> The agent interacts with the environment by selecting an action.<\/li>\n
- Reward:<\/strong> The environment gives the agent feedback in the form of rewards or punishments.<\/li>\n
- State transition:<\/strong> The state of the environment changes based on the agent’s action.<\/li>\n
- Learning:<\/strong> The agent adapts its strategy (policy) to maximize future rewards.<\/li>\n<\/ol>\n
<\/span>Basic principles of reinforcement learning<\/strong><\/span><\/h4>\n\n- Agent<\/strong>: The learning system that makes decisions.<\/li>\n
- Environment<\/strong>: The system or world with which the agent interacts.<\/li>\n
- Actions<\/strong>: The agent’s options for influencing the environment.<\/li>\n
- State<\/strong>: The current state of the environment that provides information to the agent.<\/li>\n
- Reward<\/strong>: Feedback from the environment that returns positive or negative values to the agent based on the action performed.<\/li>\n<\/ol>\n
<\/span>Mathematical basis<\/strong><\/span><\/h3>\nComputer science always has a mathematical background; after all, algorithms are based on mathematical, logical rules.<\/p>\n
1. markov decision process (MDP)<\/strong><\/p>\n\n- \n
\n- RL is based on modeling the environment as an MDP, which includes the following elements:\n
\n- State space (S)<\/strong>: All possible states of the environment.<\/li>\n
- Action space (A)<\/strong>: All possible actions that the agent can perform.<\/li>\n
- Transition probability (P)<\/strong>: The probability that an action will change the state.<\/li>\n
- Reward function (R)<\/strong>: The value that is issued for a specific action in the current state.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
2 Bellman equation<\/strong>
\nThe Bellman equation serves as the basis for optimizing the policy. It describes the relationship between the current reward and the expected future rewards: Q<\/span>(<\/span>s<\/span>,<\/span>a<\/span>)<\/span>=<\/span><\/span>R<\/span>(<\/span>s<\/span>,<\/span>a<\/span>)<\/span>+<\/span><\/span>γ<\/span>s<\/span>′<\/span><\/span><\/span><\/span><\/span>∑<\/span><\/span><\/span><\/span><\/span>P<\/span>(<\/span>s<\/span>′<\/span><\/span><\/span><\/span><\/span><\/span><\/span>∣<\/span>s<\/span>,<\/span>a<\/span>)<\/span>a<\/span><\/span>max<\/span><\/span><\/span><\/span><\/span>Q<\/span>(<\/span>s<\/span>′<\/span><\/span><\/span><\/span><\/span><\/span><\/span>,<\/span>a<\/span>)<\/span><\/span><\/p>\n