Every industry has a different reason for a different problem that relates back to the database. This is because companies need to extract data but can only use old technology to do this with. Businesses are also in need of time and money to stay one step ahead and prevent being fined, but 80% of time is wasted by data scientists just wrangling data.
The importance of innovation strategy
$36 billion was spent on creating data silos with legacy technology last year, according to MarkLogic, and this does show that data integration innovation is being taken seriously in the financial services. The extract, transform and load process, or ETL, cannot be lost because businesses cannot lost any data so as David Gorbet, senior vice president, engineering, at MarkLogic said, harmony between these two practices needs to be maintained so that source content can be preserved and queried. “It’s only when you bring these things together with the meaning intact that the magic happens,” Gorbet said.
A problem occurs when new technology is implemented but the data that has been stored in isolated systems was not designed to work with the new system. This can be solved with a simple data strategy and we spoke to Sinan Baskan, solutions director, financial services CTO solutions at MarkLogic about what should be taken into consideration. Baskan highlighted that the end user’s needs is what is most important and firms should work backwards to the data sources.
“Mapping out which data reaches end users and how it is enriched and/or modified during the process helps to craft not only the requirements and specifications but also shapes the implementation strategy. This should include evaluation of technology and tools, skills assessment and cost-benefit analyses. The key is to collect and manage data for a specific business purpose; first establish what the purpose is and how you will achieve it and then focus on the technical details to optimise the process,” Baskan said.
Baskan provides a good basis for planning the data strategy but J.R. Lowry, EMEA Head of State Street Global Exchange, explored how at first, it is important to get control over your existing data that is locked up in silos and replace outdated systems, but it is not just about the tech. “They need to strengthen data governance, stewardship, and quality processes – it’s definitely not only about the technology. Along the way, they need to make sure regulators’ expectations are met in how and what should be reported to them, so that they’re not stuck just playing ‘data defense’,” Lowry said.
According to the State Street 2014 Data Survey, ‘The Innovator’s Journey: Pathways to Data Dexterity’, nine in 10 investment organisations expect regulatory reporting requirements to increase over the next three years, but 50% of investment organisations say their data capabilities will struggle with regulatory changes. The survey also found that 39% of innovators, those companies that have advanced data infrastructure, expertise and high-quality data governance, are highly confident about evaluating risk and performance across the entire portfolio. This is compared to 4% of data starters, those companies who are at an early stage in their data journey and 17% of companies that are actively moving toward better data capabilities, the “movers”, as the survey described them.
Alongside this, the survey also found that 52% of respondents said that investment in data and analytics has increased each year by between 5% and 10%, but 77% believe that data security is a key strategic issue for the investment industry. However, David Csiki, President of INDATA, highlighted that for many investment firms, securing the necessary IT infrastructure required to maintain an effective data strategy is costly and time consuming. “Firms should consider partnering with solution providers that offer a holistic approach to data management. Additionally, providers that take advantage of cloud-based technologies offer real advantages, since the cloud is often the best delivery mechanism for the various applications and technologies that are required for an effective data strategy.”
James Blake, CEO of big data analytics firm Hello Soda, had a similar view and explained that before a data strategy is implemented, clear goals and key areas of concern need to be highlighted to data is fit for purpose. Blake went on to say that while a range of data analytics techniques are available, “in-house, third parties, unstructured and structured data can all offer a diverse way to tackle the problem with the challenges at hand. When considering data storage solutions, try to go for a data lake as opposed to a data warehouse where possible. While a data warehouse provides a structure that is easier for business professionals to utilise, data lakes hold a vast amount of data in its natural format; the structure and requirements of which are not defined until the point that the data is needed. This format is best suited to data scientists but the cost of storage is low and the data is far more agile.”
The big deal with big data
As technology advances, everyone seems to want everything to happen in real-time and that is an issue that many firms are dealing with at the moment. As Steve Colwill, CEO of Velocimetrics explored, this is so that when there is an incident, the problem can be accurately and quickly dealt with and in turn, minimise the impact on cost, resources and client experience. However, many companies are coming to the realisation that there are many differences between what big data can deliver and what real-time actually seeks to achieve.
“Take for example the fact that although big data enables firms to pump huge quantities of information into their big data lakes, query this data and then subsequently surface valuable business analytics, big data tools fail to provide an understanding of exactly what’s happening here and now to the transactions being processed – the core objective of real-time transaction tracking. Or that the firms wanting to employ real-time transaction tracking usually understand the way in which data should be being processed and specifically want to check whether this is actually what’s happening. Whereas, big data solutions are usually built using generic tools designed to help firms interpret data and tell them what they don’t already know,” Colwill said.
However, is big data an effective way to improve services? Csiki said yes. “From a buy-side perspective, investment managers are constantly looking at ways to be more responsive to their clients. From a client servicing perspective, being able to answer questions clients pose quickly can be a challenge as the data often resides in many different silo-based systems and formats. Big data technologies can be used to aggregate this data and provide a more effective approach and thereby improve client service. If an asset manager has access to accurate, high quality data which is up to date and produces actionable insights, then they also have the ability to gain real advantages over competitors.”
From this, we’ve learnt that the implementation of real time can better the user experience for the customer, which is an important aspect of managing such big volumes of data. Blake also commented on the benefits of big data for the customer and for the company to understand consumer behaviour as it rapidly changing in the digital age. “Traditionally, consumers had a personal, often face-to-face relationship with businesses but technology has eroded that. This means that there’s a danger that customers become lost in the vast sea of consumers. Mining big data for insights into customers allows businesses to understand them as individuals once again. Knowing what customers want, what services are relevant to them and even what puts them off, can completely revolutionise a company’s strategy.”
Daniel Heck, senior director of marketing, EMEA, SugarCRM made reference to the partnership between Lloyds and Google this year which is a great example of this. “The deal sees the pair prototype systems to analyse Lloyd’s customer behaviour in real time, this analysis is then used to aid the bank improving its user experience and digital marketing offering. Ultimately, big data and analytics is a necessity for any business looking to deliver a personal, unified and memorable service to its customers,” Heck said. However, as discussed above, big data tech is not the answer to all of your company’s problems as newer and better systems are emerging, Colwill explored.
“Recent advancements in technology are now enabling firms to gain the insight they need to accurately comprehend what’s happening, as it’s happening to the transactions being processed whilst incorporating the persistence and post-event analysis capabilities that they know and love big data for,” Colwill said.
The emergence of machine learning
In answer to being questioned about the impact of big data on companies, Lowry mentioned how the term big data means different things to different industries, but the human element is becoming much less necessary. “Machine learning algorithms are being overlaid on the data to help surface patterns and reduce the degree to which human analysis is required. As data volumes increase, the man-machine mix will invariably have to tilt further toward the machines. Overall, leading players are increasingly aware of the value data provides, which is worth as much as, if not more than the transactions being performed.
“This is driving a massive mindset shift for an industry that has largely been transaction-centric. For the organisations investing in their internal data infrastructure, it’s translating into greater confidence across a range of areas, including scenario-generation and stress testing, managing multiple external and internal data streams, optimising electronic trading, and cross-portfolio analysis of risk and performance. They see data as a business topic, not one for the IT department, and they are gaining a competitive edge as a result,” Lowry said. It is difficult to get the right balance between human and machine led services for your organisation, but many believe that in the near future, a mixture of both, or a hybrid model is essential.
Baskan from MarkLogic comments on how there is a difference between machine learning and behaviour analytics; the former is more focused on codifying practices for automation and scale and learning from it, the latter is about understanding why humans make the decisions they do and making predictions as a result of that information. Lowry continued to say that fintech startups are seeing behavioural analytics as a product area for fraud, market abuse and investment prevention, so an interest for investment and compliance professionals. “Critical to the success of these tools is the minimisation of ‘false positives’ that require a person to evaluate the results. Too many false positives can erode the confidence in these tools or make them too expensive to use. Hence coupling such tools with machine learning capabilities offers to potential to strengthen their accuracy more quickly, but these tools require time and focus to ‘train’,’ Lowry said.
Csiki does not believe that the buy side can benefit from behaviour analytics or machine learning. “For the buy-side, machine learning and behaviour analytics are in their early phases and have somewhat of a limited application for investment managers with active strategies. For other segments of the market that focus on passive investing, such as robo advisors, this is certainly an area where these emerging technologies will have an application.” Susan Dargan, Head of State Street Global Services Offshore, however, says that “machine learning will help financial institutions detect cyber-attacks more quickly than ever before. As a result, they will be able to stop attacks early and prevent costly data loss.”