In this post we will discuss a way of representing a state that has exciting connections with how our brain seems to work. First, we’ll briefly look at some foundational ideas (representation and generalisation). Next, we’ll introduce the Successor Representation (SR), which is motivated by finding a representation that generalises across states, and that might be useful in reinforcement learning. … Read More

## AlphaGo Zero: An overview of the algorithm

In this post I go through the algorithms presented in the groundbreaking AlphaGo Zero paper using pseudocode. The objective is to provide a high-level idea of what the model does. Why AlphaGo Zero matters Last week, Google DeepMind published their final iteration of AlphaGo, AlphaGo Zero. To say its performance is remarkable is an understatement. AlphaGo Zero made two breakthroughs: … Read More

## Questions to ask when deciding how to approach predictive problems

Is the situation stochastic or deterministic? Is it time-inhomogeneous? (Different across time?) How much data do you have available? What limitations are there with respect to computational cost (compute and time), both for training and predicting? Do you need to try actions to learn about situations? (If so, consider Reinforcement Learning.) Do your actions have an impact on the environment? … Read More