What Is Reinforcement Learning?
Get an idea of reinforcement learning, the process of training through rewards, and the approaches used in it.
Series: Attention because it's all you need. (11 articles)
- …
- 11What Is Reinforcement Learning? This article
What Is Reinforcement Learning?
In supervised learning, the machine learning model learns from labeled samples, whereas, in unsupervised learning, unlabeled data samples are used to find some patterns. Reinforcement learning is different. It teaches the machine through interaction with the environment and getting some feedback.
The purpose of reinforcement learning is to find out what actions bring the maximum benefits in the long term, rather than the biggest immediate reward. For instance, imagine you train an artificial intelligence for playing a game. At first, it doesn't know what moves are good and what are not. But through trying different actions, observing the result, and getting some rewards, it learns the way of playing that helps it perform better. Reinforcement learning is used in robotics, creating game-playing AI, resource allocation, and other areas of decision-making.
How Does Reinforcement Learning Work?
To use reinforcement learning, you need to learn about its parts first. The agent is the learner itself, and the environment is the place where it operates. In each step, the agent gets the information about the current state, selects an action, and gets some reward depending on its performance.
Then the agent uses its experience to improve the policy, the set of rules according to which it chooses actions. Sometimes it has to balance exploration, the process of trying new actions, and exploitation, the usage of already working actions. Cumulative reward is an important thing to consider. The action that brings a small immediate reward may lead to a worse situation later. Good reinforcement learning algorithm considers both kinds of rewards.
What Are the Types of Reinforcement Learning?
There are many types of reinforcement learning. We will look at two major groups of them.
1. Model-Based Reinforcement Learning
In model-based reinforcement learning, the agent creates the model of the environment to predict how actions influence the state and what rewards they provide. The model can be learned from the experience or be already given. For example, a robot learns how moving its arm changes the state of the object and how it affects its actions later. The creation of this model helps the agent to select actions in advance, though an incorrect model leads to wrong decisions.
2. Model-Free Reinforcement Learning
Model-free reinforcement learning does not require the creation of the explicit model of the environment work. In it, the agent learns which actions are good through interaction and experience. For instance, an agent that learns to play a game improves its strategy by repeating its actions and analyzing the results. Q-learning and SARSA are the examples of this kind of reinforcement learning. The model-free reinforcement learning can be divided into two parts.
- Value-based methods evaluate the goodness of the state or action regarding the future rewards. The agent uses this evaluation to choose actions. Q-learning and DQN are the examples.
- Policy-based methods learn the policy, which is the direct way of choosing actions. They can be useful when there are a lot of possible actions like controlling speed or direction of the robotic arm.
- Actor-critic methods combine both ideas. The actor selects actions, and the critic evaluates the goodness of the action or resulting state. This combination helps to improve decision-making of the agent.
These types are not isolated categories at once. The value-based, policy-based, and actor-critic methods refer to the learning approach, while the model-based and model-free refer to the usage of the explicit model of the environment.
Why Is Reinforcement Learning Important?
The reinforcement learning method is especially useful when decisions affect the future and the best strategy is unknown yet. Using reinforcement learning, the agent can create the strategy for the complex task without having a labeled sample of each possible situation. But there are some disadvantages. For instance, reinforcement learning requires a lot of iterations for training, the reward should be created correctly, and sometimes the agent can learn the wrong strategy if its reward system does not reflect the actual goal. Reinforcement learning completes another important part of the machine learning basis.
Series: Attention because it's all you need. (11 articles)
- …
- 11What Is Reinforcement Learning? This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article