The weakness of rule based systems in Middle and Back Office risk analysis data solutions

6 September 2010

By Mark Geffryes,
business development director,
Sinus Iridum

Let’s hear it for the operations team

Typically middle and back office systems ensure that all trading and transactional information is recorded correctly, all governance issues are complied with and all processes behind the scenes run smoothly. In short, this is the function that supports the ‘talent’ who do their best to earn vast sums for their employers.

One of the key issues middle and back office systems face is the volume of data they handle. As trading systems become more sophisticated, and the volume of transactions increase, the more pressure support systems face to keep up. As a result, increasingly powerful computer systems are adopted to make these systems as automatic as possible. Paradoxically this can result in mistakes being hidden as there is less human interaction and more reliance on the system checking that everything is both correct and legal.

In addition, there is a wealth of useful knowledge buried in the information concerning trading patterns, market knowledge and data that could be useful to improve systems in order to gain a competitive advantage or reduce risk.

So what’s the problem?

The limitations of computers

Computers are excellent at handling large data spaces where the rules are well known. However, they are unable to handle large data spaces where the rules are either not fully understood or are constantly changing, as is the case in virtually all the more complex business data systems.

For conventional computing to attempt these data scenarios it is necessary for the problem to be simplified by ‘throwing away’ ‘irrelevant’ data. This ‘irrelevant’ data can often subsequently become quite significant. A good example would be in financial risk management where probability distributions are simplified in order to analyse the vast bulk of the existing risk. However, in extreme market scenarios, such as a default of a major country, these discarded data points can become particularly relevant.

The data is thrown away because computers are unable to handle large numbers of variables in a reasonable time regardless of the computer power deployed. This is amply demonstrated in the ‘travelling salesman’ conundrum. This is where a typical desktop computer can solve a ten variable problem in five seconds but to solve a 20 variable problem will take the same computer 100,000 years to solve. The conundrum is that the salesman can still find the shortest route around 20 towns in a few seconds.

The difference between the approach of the computer and the human is simply that the computer has no initial knowledge and therefore has to compute every possibility. However, the human has acquired knowledge over time and is simply comparing the problem against that existing knowledge and is able to make an instant and correct decision. The human can no longer do this when presented with too many variables.

The Store and Throw-away Data Society

Regulatory requirements demand that organizations store their data in case it is required at a later date. As a result firms around the world are accumulating data faster than there is any meaningful way of analysing it. Eric Schmidt, chairman and chief executive officer at Google, recently said that “information created by humankind since the dawn of the civilization until 2003 weighs in at about five exabytes of data. But in 2010, that amount of information is created every two days”.

So when we come to analyse our historical data in order to better understand the future what do we do? Because of the sheer complexity and volume we throw away, we ignore data that we do not consider necessary to analyse.

Does this mean data that was irrelevant five minutes ago will never be relevant? We know that is not true.

If our computers are looking for direct matches they can trawl huge amounts of data quickly and find the direct match. However, as anyone who has ever received two mailings from the same company will know, just one letter in the wrong place can make an address a separate logical entity as far as a computer is concerned.

The programmes that throw away this data have a vast array of impressive sounding names. Let’s look at a few of them from a common sense perspective.

Neural Nets needs huge amounts of specific training data to learn and then ‘sets’. From then on it can make predictions but unfortunately without all the input data and the information from the time of setting. The result is these programmes are about five per cent to ten per cent accurate on prediction and slow at processing. It makes tossing a coin look like a good option.

Genetic algorithms start out by running some ‘experiments’ and produce a spectrum of bad to good results. Then the best results are taken for the next experiments. The problem is that if the results are selected, the end result is already known. In reality the overall best result might be derived from the initial worst outcomes.

Fuzzy Logic is extremely useful and fast when it comes to balancing just a few variables such as in a car ABS system. However, if the problem space is expanded this approach suffers immediately from the computer variable problem and then takes a long time to find any solution.

Simulated Annealing is able to come up with a solution in a reasonable amount of time by seeding a start point in a multidimensional problem landscape and looking for a minimum or ‘deepest valley’ that represents a solution. The limitation is in terms of time or energy to find alternative ‘deeper valleys’ that represent the best or real solution. So once again most of the problem space is ignored.

Bayesian Networks produce some good results in very controlled environments. Thomas Bayes produced a variant of probability theory. He noticed that not all probability outcomes made sense from a human perspective and some outcomes could be discarded. It suffers with too much information and as the reference data points are diluted with additional data, it loses its accuracy.

All the above software technologies have been produced to try and overcome the limitations of the computer and allow large numbers of variables to be compared against each other.

Knowing what we know now and being surrounded by limitless amounts of digital stored data would we design a computer today the same way as before or would we do it differently?

How good are traditional data mining/analysis methods?

In a nutshell – traditional data mining and analytical tools are great at finding anything that can be described well with rules. They will diligently work through all the data looking for those pieces of information that match the criteria rules set at the beginning. In other words, they are excellent at finding what is known to be buried in the searched data.

Consider ‘looking for a needle in haystack’. We know what the needle looks like – straight, shiny, metal, less than 10cm in length, etc. And we know what hay looks like. But what if the needle has been deliberately disguised? We suspect that it is there but we are not certain what it now looks like or how it has been camouflaged.

So how do traditional methods cope with the usual questions? :
• What if the needs change? No problem, you just need to write new rules.
• What if the data volumes increase? We just buy a bigger box to process the extra.
• If we do not have a bigger box available? We will just take out all the unimportant data – the parts we ‘know’ do not have any relevance to the bigger picture or what we are looking for.
• What if you do not know the rules or do not know what you are looking for? We can carry out a very thorough analysis to find all the likely options so we can write the necessary rules.
• What if the only way to safeguard corporate assets is to do this checking in real-time? We do not recommend real-time processing...

The bottom line is that traditional data mining and analytical tools are only as good as the rules programmed into them, providing you have sufficient computing power.

A different way to analyse data

Knowing the problems computers face coping with large, complex databases from disparate sources, we believe that there is an alternative and better approach - if a computer could be made to work in a way that is more like a human by searching for patterns in data, then working out rules that should be applied.

Design

A solution which would work in this way would be able to:

• learn complex data and make meaningful predictions about possible outcomes.
• not require pre-defined rules (as in conventional analytics software).
• be able to find rules in complex data structures (in much the same way as a human learns and continuously applies that knowledge) on a massive scale.
• be able to receive input in binary.
• be able to process 4,000 binary elements in up to 4,000 different dimensions.

A solution such as this, like Sinus Iridum’s the Uncertainty Engine, would be able to read the data, learn about data patterns and remember ‘pattern rules’ in computer memory in a very compact form. The next piece of data to arrive would update all the previous knowledge and give the engine its real time performance and highly efficient, high 90’s predictive capability - even with small amounts of input data. Additional data would improve the accuracy of prediction. In decision making it would retain information in an uncertain state until such time as it is certain.

Depending on the industry that the data is coming from, such a solution could make predictions for customer behaviour (CRM), fraud, real time Basel II financial risk analysis, enterprise and internet security.

So what would a solution like this mean for the middle and back office function?

The data the operational team handle can yield valuable knowledge that should be fed back into the business. A solution like the Uncertainty Engine would be able to extract transactional data including:

• Exposure of fraudulent trades that might cause financial and reputational loss
• Identification of beneficial trading activities enabling success to be replicated
• Early warning of unusual market movements/trends to create profit opportunity or minimise exposure to loss (particularly in Algo trading scenarios)
• Cross platform analysis focusing on client activity to improve counterparty service/profitability
• More accurate risk/capital adequacy profiling

One of the key points of contention for the middle office is the speed of systems involving risk and liquidity. Being able to analyse in real-time can dramatically improve reaction times to atypical trading patterns.

With increasing dependence on outsourcing there is even less oversight by responsible officers as they have to wait for the outsourced function to provide reports. Ultimately the internal factor is reduced.

And of course any additional knowledge gained from data helps the business overall to make better decisions, spot future profit opportunities, protect current returns from anomalous activity and to more accurately determine risk.

Summary

The Uncertainty Engine can process data as quickly as it can be passed to it so the throughput of data is normally only limited by the size of the ‘pipe’ connecting it to the source data. This means that massive volumes of data can be processed in real-time giving the immediate system feedback that is often lacking in traditional analytical and data mining solutions.

•There is a better way to analyse data
•A solution that gets around the limitations of many computer systems
•A technique that doesn’t require data to be discarded before it starts
•Can run alongside any existing programme or system
•Run in real-time or against historical data feeds
•Future proofed against changes in requirements or architecture


Become a bobsguide member to access the following

1. Unrestricted access to bobsguide
2. Send a proposal request
3. Insights delivered daily to your inbox
4. Career development