Reinforcement Learning Explained
Learning From Rewards, Step by Step
- Pages
- 81
- Chapters
- 59
- Level
- Intermediate
- Format
- E-book
About this book
Reinforcement learning taught computers to play games at superhuman level and now helps shape the behaviour of language models. This book explains how it works, clearly and with worked numbers.
You will start with the exploration–exploitation dilemma in multi-armed bandits, then learn Markov decision processes, value functions, dynamic programming, Monte Carlo and temporal-difference learning, Q-learning, deep Q-networks and policy-gradient methods. Final chapters cover reinforcement learning from human feedback, safety concerns such as reward hacking, and when RL is the wrong tool.
Small grid-world examples make every algorithm concrete.
What you will learn
- Exploration versus exploitation with bandits
- States, actions, rewards and value functions
- Dynamic programming, Monte Carlo and TD learning
- Q-learning and deep Q-networks
- Policy gradients and actor–critic methods
- RL from human feedback, reward hacking and safety
Table of contents
- IntroductionIntroduction
- Chapter 1Learning From Rewards
- Chapter 2How RL Differs From Other Learning
- Chapter 3The Multi-Armed Bandit
- Chapter 4Estimating Action Values
- Chapter 5Exploration Strategies
- Chapter 6Regret
- Chapter 7Contextual Bandits
- Chapter 8Designing Rewards
- Chapter 9Markov Decision Processes
- Chapter 10A Grid World
- Chapter 11Returns and Discounting
- Chapter 12Policies
- Chapter 13Value Functions
- Chapter 14The Bellman Equation
- Chapter 15Optimal Values
- Chapter 16Dynamic Programming
- Chapter 17When the Model Is Unknown
- Chapter 18Monte Carlo Learning
- Chapter 19Temporal-Difference Learning
- Chapter 20SARSA
- Chapter 21Q-Learning
- Chapter 22On-Policy and Off-Policy Learning
- Chapter 23n-Step Methods and Eligibility Traces
- Chapter 24Function Approximation
- Chapter 25The Deadly Triad
- Chapter 26Model-Based RL and Planning
- Chapter 27Search and Learning Together
- Chapter 28Deep Q-Networks
- Chapter 29Policy Gradient Methods
- Chapter 30Baselines and Advantage
- Chapter 31Actor–Critic Methods
- Chapter 32Trust Regions and PPO
- Chapter 33Continuous Control
- Chapter 34Simulation and Sim-to-Real
- Chapter 35Offline Reinforcement Learning
- Chapter 36Imitation Learning
- Chapter 37Reinforcement Learning From Human Feedback
- Chapter 38Reward Hacking
- Chapter 39Safe Exploration
- Chapter 40Applications in Practice
- Chapter 41When RL Is the Wrong Tool
- Chapter 42Practical Tips
- Chapter 43A Worked Q-Learning Episode
- Chapter 44Q-Learning in Code
- Chapter 45Evaluating RL Agents
- Chapter 46Value Iteration, Worked Through
- Chapter 47Reward Shaping
- Chapter 48Partial Observability
- Chapter 49Hierarchical Reinforcement Learning
- Chapter 50Multi-Agent Reinforcement Learning
- Chapter 51Curiosity and Intrinsic Motivation
- Chapter 52Rewards From Verifiable Outcomes
- Chapter 53Hyperparameters That Matter
- Chapter 54Debugging RL
- Chapter 55RL for Long-Term Recommendations
- Chapter 56The Ethics of Optimising Behaviour
- Chapter 57Case Study: Energy Management
- Chapter 58Case Study: Adaptive Website Experiments
- Chapter 59A Project Checklist
- AppendixExercises
- GlossaryGlossary
Who this book is for
Students and developers with basic algebra and some programming experience who want to understand reinforcement learning.
Details
| Format | PDF + web edition |
|---|---|
| Length | 81 pages (approx.) |
| Chapters | 59 |
| Level | Intermediate |
| Language | English |
| Topic | Fundamentals |
| Author | Latent Library Editorial |
Keep reading