CV
or: How I learned to stop worrying and love the AI

You can download a pdf of my CV by clicking here

• AI Consultant
• PhD in Computational Neuroscience
• Master in Cognitive Science

I got my PhD from Goldsmiths, University of London. My thesis is called:

On State Representations and Behavioural Modelling Methods in Reinforcement Learning

What does it mean to have a PhD?

It's common that people outside academia are not aware of the diverse set of skills you aquire during a PhD. Some examples of such skills that I now have:

• Learn new things quickly
• Scope projects to get them done
• Adapt information for the intended audience
• Write; both technically and creatively
• Teach
• Present/Speak in public
• Communicate across fields/departments

My research in a nutshell

To build a robot that learns and makes decisions, what algorithms would we need?

My research in a coconut

Reinforcement learning (RL) - algorithms for learning from rewards - has proved successful in the cognitive sciences for explaining both neuronal signals and behaviour in animals, and for producing impressive results in artificial intelligence.

Essential to RL models are state representations. Based on what current state an animal or artificial agent is in, they learn optimal actions by maximizing future expected reward. But how are humans able to learn and create representations of states?

This thesis approaches this quesetion from two fronts. First, we thoroughly investigate methods for fitting behavioural models to human lab data. In contrast to recent proposals, we find that the best methods for model selection - determining what model most likely generated some data - are based on maximum likelihood estimation, rather than Bayesian inference. We also demonstrate the importance of considering individual differences in model fitting: the model which best fits the performance of one participant may not fit the behaviour of another participant.

Second, we introduce Shapetask - a novel learning and decision-making task where participants must find hidden structure in a sequence, without the task explicitly rewarding the appropriate actions. We show that some humans can find this pattern, while RL cannot, unless equipped with appropriate state representations. We then show how previously proposed models that integrate RL with complex state representations can account for individual human behaviour in the Shapetask.

We argue our results add to the growing literature indicating a broader role for dopamine as one involving general sensory prediction errors, not just reward prediction errors. Further, we argue Shapetask holds promise for use in further research on the topic of state representation and task structure. Such research may illuminate the workings of animal brains, and contribute to artificial intelligence, where enhanced models of state representations could improve data efficiency and generalisability over current generation systems.

My research in papers

• The Effect of State Representations in Sequential Sensory Prediction: Introducing the Shape Sequence Task
  (2020, CogSci Conference, Peer-reviewed)
• Towards human-like artificial intelligence using StarCraft II
  (2018, FDG Conference, Peer-reviewed)

My PhD studies were fully funded by the EPSRC via IGGI.

I'm a Cognitive Scientist

I was trained (potentially brainwashed) by the brilliant people at Lund University Cognitive Science to see connections between all forms of thinking systems and how they interact.

This image borrowed from Wikipedia is the best I've found that shows how cognitive science is a mix of many fields. However, it left out animal behaviour and biology; ethology. So I added a bumblebee. Obviously a spider would have made more sense sitting in this web. But bumblebees are cuter.