Technology has historically impacted society in discrete waves. Digital has assumed such significance because this is the first time that a number of technology waves (cloud, IoT, AI, etc) have broken simultaneously and at a price point that makes enormous functionality and processing power available relatively cheaply. In commodity assets, many producers have been at the bleeding edge of IoT and analytics in trying to improve their operating costs. But on the trading floors, the revolution that was assumed to be inevitable five years ago is stalling.
There are no universally applicable reasons, but we do see some common inhibitors to progress:
- Often fragmented and heavily customised CTRM systems, whether off the shelf or bespoke, inhibiting the ability to reorient the business’s core transaction processing technology at speed.
- Data remains trapped in process silos stymied by poor integration and is often of variable quality reflecting a loose definition and adherence to data standards.
- Companies often lead with investing in a technology stating that ‘we’re doing something in AI, blockchain, OCR, analytics …..’ instead of having a clear articulation of a specific business problem or opportunity and setting about developing a solution.
- Leadership and capability challenges with senior business leaders often lacking interest in and an understanding of the strategic importance of technology to the future of their business.
- Cultural change as embedded ways of working fight against the challenges and threat posed by new technologies and new ways of doing business.
Let’s touch briefly on the question of culture and the difficulties we all have when faced with change. The development of algorithmic trading and the money flow into commodities contracts, as a result, is well documented. But that’s not what physical traders have historically done – managing a trade flow is very different from managing a model. In the past few years, this has begun to change and many commodities traders have been hiring the brightest and best data scientists and setting them to work. The new profile of trading talent is becoming increasingly technical.
But the approach has often taken the form of hiring and deploying such people into desks and expecting great things to result. Aligning the old and the new and setting the new off in profitable directions takes focused effort, an open mind, and patience. The hypothesis that using structured and unstructured internal and external data can create a renewed trading edge in an age of information transparency makes good sense. However, to succeed this exploration needs to be framed and directed and it is this which anecdotally at least, I think is often missing. The scientific method goes back to Aristotle and scientific knowledge develops by means of observations leading to hypotheses that can both be tested and potentially be falsifiable. Knowledge and insight are not magically created in the data scientist’s code. It comes from careful thought and the joint framing and testing of effective hypotheses. Note the need for hypotheses plural. It is questions developed by the traders with the data scientists that should be utilising data science to develop a portfolio of insights, some of which may generate an edge but where many will not.
Instead of investing in developing in building an enduring organisational capability what seems to be typically happening today is, a team of data scientists is hired, loosely connected to the physical traders and asked to produce. The capability may mature over time but why would people with highly marketable skills stay where they are not appreciated? Maturity can be accelerated by focus and leadership and a wise choice of where to start. Internal supply chain and process inefficiencies are always good places to start and generate both dollars and credibility. Focusing for example on traditionally painful areas in the execution process to generate insights about where process changes can generate value would be an example.
Trading business leaders should not think their role is complete by signing off hiring and buying new technology and people. Only they have the knowledge and experience to be generating the hypotheses, manage the analytical process and then finally apply their judgment to what is being presented.
This will of course result in conscious and unconscious bias, conscious and unconscious agendas, and good old-fashioned office politics. In fact, we are back to the future and the ongoing evolution in the role of the mark 1 human being. It has been observed that war is too important to be left to the generals. I will humbly suggest that data science is too important to be left to the data scientists.
