The 24th Annual Workshop on Economic Science with Heterogenous Interacting Agents (WEHIA) took place last week, where we had the pleasure of hosting a roundtable on unlocking business value in financial markets using agent-based simulation.
Participants
- Chair – Justin Lyon, Founder and CEO of Simudyne
- Panelist – Dr. Amir Sani, Head of Research at Simudyne
- Panelist – Dr. Krishnen Vytelingum, Senior Quantitative Modeller at Simudyne
Justin: Several people at WEHIA have talked about how high-fidelity agent-based models (ABMs) give a more robust view of risk and should therefore replace some of the other standardized modeling techniques used by banks. What are your thoughts on this?
Amir: You need to remember that some banks survived the crash precisely because they weren’t using anything sophisticated. A lot of banks were, and still are, using excel spreadsheets and basic tools to make complex decisions. And those that tried to use complex methodologies suffered because they didn’t properly understand them or didn’t have the proper validation procedures in place.
Krishnen: Exactly. The function of model validation is a huge one if you want to make something fit for purpose. Yes, ABMs are going to provide a more accurate and realistic view of risk but validating them is much more complicated.
Amir: Absolutely – it’s not just the incumbent models that are linear but the current validation and calibration methods too.
Krishnen: The good thing is that more research is coming out of academia on model validation. Now, it’s a case of convincing those in the bank that ABMs will give a better view of risk.
Justin: Take the housing market model from the Bank of England for example; you’d think it’s a no brainer that every commercial bank with a significant mortgage portfolio would be using these models to understand the market, but numerous commercial banks aren’t using them. Why is this?
Krishnen: Banks are still using the same models that failed in 2008 – they continue to stress test the same functional model that’s not fit for purpose. But building new ones would require a team of quants with machine learning skills and experience in ABM, and this isn’t an insignificant challenge.
Amir: Ultimately, unless regulators globally mandate that banks start using this approach it will remain a nice to have. From what I’ve seen, banks are very much wanting to do ABM, but internal inertia is making it difficult to deploy. Regulators need to start thinking about prescriptive guidance on ABM for risk management.
Justin: On the flipside, how can banks be sure that ABMs don’t become the next thing that fails in bad times?
Krishnen: Stress testing plays a big part in this. Any model must be stress tested in a variety of different environments.
Amir: We need to take the view that all models are broken. If you only rely on a single model you run the risk of failure. Banks need to consider using multiple calibrations of multiple models along with a model selection policy as part of their decision-making workflow, so that in a crisis they aren’t on the back foot.
Krishnen: One clear benefit to ABM is that it’s a white box model – when it breaks you can dive into the model and analyse every single interaction to identify the fault.
Amir: If an organization is only using models to make predictions then they’re missing a lateral advantage. Models should be used to explore intuitions. They should help form the narrative that drives the decisions.
Justin: Let’s move away from risk and talk about investment banking. Investment banks want to move the needle and make money, but regulators want to see an orderly market. What role does ABM have to play in this?
Krishnen: At the end of the day, what algo traders want is a model that will help them generate alpha – a model that’s realistic and proven to work. And there are a lot of steps to take from taking a validated ABM built on sound theory to something that can be put into production. This is exactly what we are working on at Simudyne.
Amir: Calibration is a lot harder than people realize. A lot of assumptions drive traditional models. You can calibrate these dynamics directly from the data when using an ABM. They no longer need to be assumed.
Krishnen: But when it does work it’s incredibly powerful. We’ve been working with an exchange to model how algos behave in stressed environments. Regulations like RTS 6 and 7 require firms to record exactly why they posted the trades that they did. But with ABM you can create a digital twin: a calibrated, perfectly replicated version of an order book. In this environment they can easily create stressed scenarios and test their algos.
Amir: Clients love the fact that this is an environment that they can interact with; they can actually open it up and see what’s happening with their agents, giving them a narrative for their clients and regulators. They can justify their insights with explainable dynamics that prove their algos aren’t contributing to a disorderly market. You simply can’t do this with traditional methods without a lot of hand waving.
Krishnen: It’s such an obvious win and one that banks don’t seem to be exploring.
Amir: Again, it may be that banks are more willing to adapt to a new approach when its anchored to something concrete like a mandate from a regulator.
Krishnen: For me, the future lies in the culmination of machine learning and ABM, making modeling more realistic and useful in production. This is what will drive more rapid adoption in banks – the combination of computational simulation and data science.
Justin: I fundamentally believe that we as a species need help making radically better decisions. The way forward shouldn’t be based on a retreat to gut instincts but on a fact based, model driven approach – we need to embrace testing, and in order to make real change we need to build something that can be used by real people. That’s ultimately what we are trying to do at Simudyne.
Note: The round-table summary of the discussion has been edited for clarity and brevity.
