How to Utilise and Leverage Big Data and Analytics

12 February 2021 | 4 minutes

KX recently took part in two panel discussions at the “Transform Finance: Big Tech Event” held on 3rd and 4th February, both focussing on the adoption of the cloud to leverage big data analytics.

One of the topics was “How to utilise and leverage big data and analytics while adopting a new technology” and was moderated by technology journalist Madeline Bennet. The panel members were David Masters, Chief Data Officer at Société Générale, Jason Maude, Chief Technology Advocate at Starling Bank and Peter Moylan, Head of European Software Revenue at KX

The opening discussion focussed on how banks can best prepare for extracting insights from the vast amounts of data they currently have. The advice fell broadly into two categories: data preparation and connectivity. 

On the preparation side, it was agreed that there is a large, but crucially important, task in ensuring that the data is clean, accurate and consistent, so that insights generated by downstream analytics are similarly consistent and accurate. That entails validating, normalizing and ensuring the integrity of data, including reference data, as early as possible in the lifecycle so that it is done once, and once only, for the benefit of the rest of the organization. It also means having strong governance in place to ensure that the discipline is maintained, bearing in mind that ever more data will flood into the organization and that in extracting value, its quality is just as important as its quantity.

On the connectivity side, the guiding principle is that data when combined is greater than data insolation, as it is in the correlations, patterns and interlinks across sources that advanced analytics can reveal insights. Indeed, it is in the absence of such integration that silos prevail, and in a comment that resonated with the other panel discussion, it was reemphasized that tactical short-term decisions, sometimes driven by departmental PnL, may often be what prolongs their existence. For that reason, it often requires CIO and cross-departmental oversight to eliminate them.

The opportunities from using Machine Learning and AI techniques to extract insights were then discussed. The ability to look at and explore data in completely different ways using these techniques relies on the skillsets of data scientists, and it was counseled those starting out on their ML journey should focus initially on small well-defined outcomes that deliver a business benefit. For more advanced usage, like finding alpha and fraud detection for example, cloud adoption (in full or in part) may often be required to benefit from both the storage capacity and the compute elasticity it offers. 

That led to a discussion on cloud security where the consensus was that thinking has matured in this area. There is now a clear distinction between the security of the cloud, which is the provider’s responsibility and where they are now acknowledged as experts, and security in the cloud which is the responsibility of the data owners and the applications supporting them. The latter can be achieved by leveraging best practices in cloud-native technologies in areas like authentication, access control and encryption, both at rest and in transit.

In closing, the participants cited examples of where they see the potential of cloud and analytics having been clearly proven already. Examples ranged from the FCA’s adoption of cloud for MiFID reporting to its use in fraud detection where, as well as uncovering market manipulation, it is being asked to solve the most elusive of all problems – those that are ill-defined. In fraud detection vernacular that translates into requests like  “There’s something strange going on here, Can you find it?”, which only something like an AI engine could begin to answer.

KX will shortly be announcing an Azure-based ML project it has been working on with a client to analyze equities, fixed income and rates data to generate buying and selling signals for its customer seeking alpha. 

Click on this link to hear the full recording of the panel discussion.

To read about the other panel discussion on “Transitioning FSI Big Data Workloads to the Cloud” please click on this link

Demo kdb, the fastest time-series data analytics engine in the cloud


    For information on how we collect and use your data, please see our privacy notice. By clicking “Download Now” you understand and accept the terms of the License Agreement and the Acceptable Use Policy.

    This site is registered on as a development site. Switch to a production site key to remove this banner.