Robust Business Intelligence Solutions

Working in this domain for nearly half a decade, I have been trying to figure out some of the key challenges when it comes to designing a robust Business Intelligence solution, from text books to real business case studies as such there is no single thread you can pick up and say here it goes. The question becomes even more tough to answer when it requires a cross geographical team and cross platform integration

The answer to this question is actually subjective and depends on the domain you are working on be it retail, banking, supply chain etc etc., however there are bunch check list that you need to mark before you actually commit .

1) Scalable Data Integration
    I would call this as one of the major challenge teams are facing, every integration design is ought to be outdated at some point of time, the question is what time do we set as threshold, for industry like e-commerce, banking online retail this will be a continued area of research because of cut-throat competition and business requirement changes with a blink of an eye, however if the design sustain a period of 3-4 years, I would probably call that an excellent design. One of the key thing to incorporate when we design a system is we try to capture all business scenario and integrate data at the most granular level, so that we take care of scalability into consideration. Another dimension to an element of scalability is how flexible is the design is, if a new business case needs to be developed rather than starting from square 1 its better to complement the current system with some additional work around, but yes this should not be made a habit else this might create a web of data which would be hard to maintain.

2) Long Term Solutions 
Most of the time folks are trying to solve a business problem which is 2-3 years in their line of sight, technically a new business launched at least needs 5-6 years to come into break even, providing a solution for such a short term, might break down the whole structure and would incur extra cost. The key to this problem is, the participants need to be prudent enough so that they design solutions which have long lasting life.

3) Cross Geographical Team Integration
With the advent of internet, the world has become flat, to make business operations running 24x7 its always a good idea to make sure teams are coupled tightly among all geographies. Knowledge flow should be proper and the division of labor is well defined. Providing a good business solution is less of technology and more of understanding the business process, so if the knowledge flow is not correct, all the hard work done above are of no use.

4) Remove Social Barriers
Since we are working as global team, and encompass folks from different region it is a key requirement that we remove all the social barriers, the team should not be working with a local goal but should strive on a global scale, driving business is like connecting all the parts of the machine and making sure that each part is functional, with social barriers the drive may not be smooth.

5) Regular Audit
Above all, the most critical point that needs to be plugged in each stage is an element of audit, no design and solution will go perfect in single shot, however by auditing it we can make sure we are sticking to quality standard so the business users have a confidence in the solutions we provide to them.

The above checklist is certainly not perfect and its the outcome of my past experience, however if we take these things into consideration my guess is we would be able to make a good and robust Business Intelligence solution.
I would appreciate feedback and comments on this article, in case I have missed out any key aspect of this discussion.

Best
Siddharth

XML To Be The New Revolution in Business Intelligence Reporting

Hi, Its been long break from my last post here, but yes I don't like the idea of just writing anything crap on the web, after completing my school, I joined Amazon and life became hectic. The good thing about working at Amazon is you really work at the grass root level of the technology for me it was data. data and data.

The very idea that no business can run without informed decisions makes the domain Business Intelligence very interesting, I know a lot more new terms have been coined these days, especially DATA SCIENTIST, well I am not here to comment on what the salutation should be :-), and with internet being the new mode of communication capturing each customer movement and arriving at the correct decision is a critical activity.

Since internet is such a versatile customer experience that it can possibly capture customer information in any form from, DB Storage to flat file, the question is how do we process all these information and collectively arrive at a consolidated decision, since your information could be on several pieces of storage, technically individual sources should add up to the common source, commonly known as POC ( Proof Of Concept ).

There are many vendors which provide us with state of art to Extract, Transform and Load ( ETL ) solution, so capturing the raw data and storing it is not a problem, also storage these days are cheap so eliminating the problem of storing massive amount of data. The real problem is how do we integrate all pieces of block and build a wall so that we are able to retain our PnL statement.

Till date with my knowledge of handling Data Quality issues, there is only one Vendor who has tried to solve this problem is Oracle by coining a yet another term called Real Time Heterogeneous Data Base integration using Oracle Golden Gate, to be very honest two different database servers 10000 miles a part it takes 5-10 seconds to synchronize them, but still the question is maintaining expensive servers and then buying the license to integrate them, is the expense worth extracting such information ? OGG is primarily used for DRS system ( Disaster Recovery System ) and less of data integration with few exceptions.

The solution to the above problem where we can arrive at a consolidated solution by combining all the data points stored at different system without physically synchronizing them is XML.For many readers it might be a surprise but YES.. XML is a strong tool for data exchange primarily used as communication channel between web and back end data server, but this very fact can be used as communication channel between different data points on a common platform eg web. We may use the power of XML data transfer technique and build a robust reporting system that can interact with almost any data source and display the result, and all this without spending a penny.

The concept is really simple to understand but equally tough to implement. Would appreciate any suggestions, reading comments on this.

hope you might have enjoyed my idea, although its just a summary of what I have been reading and thinking.

Best
Sid