Andre Yee’s latest blog on Privacy, Information Theft and Security is prompted by yet more information disclosures, and makes the point that its sloppy process that is allowing this to happen. When will we get more serious about security? He’s absolutely right.

In many cases an insider is involved, colluding with accomplices on the outside. This recent case involved a member of staff in HSBC’s back office operations in India, leading allegedly to losses of $425,000. HSBC say that existing operational procedures (unspecified) identified the fraud. While we have to congratulate HSBC for ‘fessing up and for prosecuting the alleged offender, I’m alarmed that their existing procedures don’t kick in until $425k has been lost.

What is astounding is that while all financial institutions have a comprehensive audit trail, very few of them actually do anything proactive with it.

Bizarre as it may sound, the audit trail is rarely looked at – in fact it’s often only examined after a breach has come to light, and then only to size the problem. Note here that I didn’t say that it is used for gathering evidence, simply because these incidents are often dealt with more discretely. Despite terminating thousands of staff for dishonesty each year, as an industry, financial institutions (HSBC aside) are among the most secretive when it comes to prosecuting insider theft. After all, who wants to trust your hard earned savings to banks with a dishonesty problem?

Of course this issue is all the more topical due to the current fad of outsourcing customer service functions to cheaper locations globally. Who’s policing these remote sites? Who’s checking the audit log for inappropriate access to confidential data?

Government might ultimately weigh in here. While driven by mass data thefts, new federal legislation, in the form of the US Data Accountability and Trust Act (DATA), is undoubtedly coming that will require disclosure of any compromise of personal confidential data. That’s the theory at least, though DATA looks likely to be so watered down it will be ineffective when it eventually arrives.

So I suspect that while legislation will force disclosure under some circumstances, it’s a sorry state of affairs that we need legislation in the first place. What’s really needed is better security – yes we need to look at those audit logs every day, preferably automatically to validate the transactions that are being done on our systems. Validating the transactions will enable financial institutions to find data breaches when they are still minor, and before they grow into a public spectacle.

It’s a simple choice – fix the problem at source, or wait for a big data theft to hit you and then you too can have your fifteen minutes of fame.

I just got off the phone with Rick Whiting’s Information Week after a great discussion around real time BI. The call was prompted in part by his article “Businesses Mine Data To Predict What Happens Next” which touces on real time BI, but mainly covers the rise of predictive analytics. The article starts:

“Real-time information, once a competitive differentiator that produced more timely and relevant business decisions, is now a commodity. Even midsize companies process transactions as fast as the New York Stock Exchange, while decision makers communicate and collaborate over broadband networks as if they were in the same office. Sheer speed isn’t the advantage it once was.”

Of course, predictive analytics and real time BI are converging, but while many organizations are considering real time BI, real time predictive analytics are some way further out into the future. The analysis of real time information is actually still beyond most organizations – real time data is not. Processing transactions ‘as fast as the stock exchange’ is one thing, making sense of them to drive real time intelligent decisions is another matter. The difference is what it takes to turn huge quantities of real time data into useful, actionable information. There’s a big difference.

Going back to BI basics, from data we get information, which once analyzed (hopefully) gives us some insight. The process of getting from data to information requires some preparation, aggregation, and context. Getting to insight requires interpretation of the data (either manually or using embedded analytics) to then identify opportunities, costs or risks that the business can then act upon. This interpretation of data can involve using predictive analytics to predict a score to enable a problem to be identified before it occurs. Equally it could be a simple calculation to predict whether the shipment will occur on time or not.

What Rick misses in his article is that Predictive analytics today is generally an offline process, analyzing batches of data after the fact to identify patterns. This has lots of issues. It’s very manual, and it’s not fast. If fact it’s a very definitely not real time – it’s typically performed by highly skilled analysts. Once the analysis is complete, it’s almost always out of date. And the results may not actually be all that useful.

Somebody once described data mining to me as the ‘art of telling you the bleeding obvious’ because it might just prove that there is a statistically significant relationship between ice cream sales and the month of the year. Clearly it is obvious that ice cream sales will go up in the summer!

So let’s assume that you’ve mined your historical data, and found something useful. The challenge now is how to take the knowledge and make an impact on day to day operations. Traditionally there is a big disconnect here – the analyst writes a report, presents a PowerPoint, and maybe one or two key take out points get implemented. But the results are often not deployed live to make smarter operational decisions. There are always exceptions, in particular in the financial services industries, but generally there is a lack of automation of analysis processes. This of course is set to change.

Don’t get me wrong, predictive analytics definitively brings value. The challenge is how to take the insight gained and make it actionable. This brings you back full circle to real time data and business processes.

In an ideal world your predictive model can be deployed directly into the transaction stream within minutes, provide real time scores on massive quantities of data, which can change the outcome in real time of customer interactions. Oh, and the model will then self update itself as the data changes so that the model doesn’t go out of tune.
Real time data meets predictive analytics, enabling smarter in process decisions.

I need convincing that this is a commodity.