My PhD project involves the examination of the dynamics of the habitual and deliberative (goal-directed) decision systems within the brain. In particular, how these two systems interact when learning about novel and changing environments and how arbitration between these two systems is supported. My approach to this topic is a combination of human subject behavioural experiments and computational modelling (using Bayesian and reinforcement learning methods). My goal is to create computational models of healthy subjects and then determine how different parameter changes could account for the maladaptive decision making found in subjects with psychiatric disorders.
Predicting actions using an adaptive probabilistic model of human decision behaviours
Cruickshank, AH, Shillcock, R & Ramamoorthy, S 2015, Predicting actions using an adaptive probabilistic model of human decision behaviours. in Proceedings of the 23rd Conference on User Modelling, Adaptation and Personalization (UMAP-15). CEUR Workshop Proceedings, vol. 1388, CEUR Workshop Proceedings, Dublin, Ireland.
Computer interfaces provide an environment that allows for multiple objectively optimal solutions but individuals will, over time, use a smaller number of subjectively optimal solutions, developed as habits that have been formed and tuned by repetition. Designing an interface agent to provide assistance in this environment thus requires not only knowledge of the objectively optimal solutions, but also recognition that users act from habit and that adaptation to an individual’s subjectively optimal solutions is required. We present a dynamic Bayesian network model for predicting a user’s actions by inferring whether a decision is being made by deliberation or through habit. The model adapts to individuals in a principled manner by incorporating observed actions using Bayesian probabilistic techniques. We demonstrate the model’s effectiveness using specific implementations of deliberation and habitual decision making, that are simple enough to transparently expose the mechanisms of our estimation procedure. We show that this implementation achieves> 90% prediction accuracy in a task with a large number of optimal solutions and a high degree of freedom in selecting actions.
Organisations: School of Informatics.
Authors: Cruickshank, A.H., Shillcock, R. & Ramamoorthy, S..
Number of pages: 4
Publication Date: 1 Jun 2015
Category: Conference contribution
Original Language: English
Reward-Based Learning, Model-Based and Model-Free
Huys, QJM, Cruickshank, A & Seriès, P
2014, Reward-Based Learning, Model-Based and Model-Free
. in D Jaeger & R Jung (eds), Encyclopedia of Computational Neuroscience.
Springer New York, pp. 1-10. DOI: 10.1007/978-1-4614-7320-6_674-1
Reinforcement learning (RL) techniques are a set of solutions for optimal long-term action choice such that actions take into account both immediate and delayed consequences. They fall into two broad classes. Model-based approaches assume an explicit model of the environment and the agent.
The model describes the consequences of actions and the associated returns. From this, optimal policies can be inferred. Psychologically, model-based descriptions apply to goal-directed decisions, in which choices reflect current preferences over outcomes. Model-free approaches forgo any explicit knowledge of the dynamics of the environment or the consequences of actions and evaluate how good actions are through trial-and-error learning. Model-free values underlie habitual and Pavlovian conditioned responses that are emitted reflexively when faced with certain stimuli. While model-based techniques have substantial computational demands, model-free techniques require extensive experience.General Information
Organisations: Institute for Adaptive and Neural Computation .
Authors: Huys, Quentin J.M., Cruickshank, Anthony & Seriès, Peggy.
Number of pages: 10
Publication Date: 2014Publication Information
Category: Entry for encyclopedia/dictionary
Original Language: English