Hi, Its been long break from my last post here, but yes I don't like the idea of just writing anything crap on the web, after completing my school, I joined Amazon and life became hectic. The good thing about working at Amazon is you really work at the grass root level of the technology for me it was data. data and data.
The very idea that no business can run without informed decisions makes the domain Business Intelligence very interesting, I know a lot more new terms have been coined these days, especially DATA SCIENTIST, well I am not here to comment on what the salutation should be :-), and with internet being the new mode of communication capturing each customer movement and arriving at the correct decision is a critical activity.
Since internet is such a versatile customer experience that it can possibly capture customer information in any form from, DB Storage to flat file, the question is how do we process all these information and collectively arrive at a consolidated decision, since your information could be on several pieces of storage, technically individual sources should add up to the common source, commonly known as POC ( Proof Of Concept ).
There are many vendors which provide us with state of art to Extract, Transform and Load ( ETL ) solution, so capturing the raw data and storing it is not a problem, also storage these days are cheap so eliminating the problem of storing massive amount of data. The real problem is how do we integrate all pieces of block and build a wall so that we are able to retain our PnL statement.
Till date with my knowledge of handling Data Quality issues, there is only one Vendor who has tried to solve this problem is Oracle by coining a yet another term called Real Time Heterogeneous Data Base integration using Oracle Golden Gate, to be very honest two different database servers 10000 miles a part it takes 5-10 seconds to synchronize them, but still the question is maintaining expensive servers and then buying the license to integrate them, is the expense worth extracting such information ? OGG is primarily used for DRS system ( Disaster Recovery System ) and less of data integration with few exceptions.
The solution to the above problem where we can arrive at a consolidated solution by combining all the data points stored at different system without physically synchronizing them is XML.For many readers it might be a surprise but YES.. XML is a strong tool for data exchange primarily used as communication channel between web and back end data server, but this very fact can be used as communication channel between different data points on a common platform eg web. We may use the power of XML data transfer technique and build a robust reporting system that can interact with almost any data source and display the result, and all this without spending a penny.
The concept is really simple to understand but equally tough to implement. Would appreciate any suggestions, reading comments on this.
hope you might have enjoyed my idea, although its just a summary of what I have been reading and thinking.
Best
Sid
The very idea that no business can run without informed decisions makes the domain Business Intelligence very interesting, I know a lot more new terms have been coined these days, especially DATA SCIENTIST, well I am not here to comment on what the salutation should be :-), and with internet being the new mode of communication capturing each customer movement and arriving at the correct decision is a critical activity.
Since internet is such a versatile customer experience that it can possibly capture customer information in any form from, DB Storage to flat file, the question is how do we process all these information and collectively arrive at a consolidated decision, since your information could be on several pieces of storage, technically individual sources should add up to the common source, commonly known as POC ( Proof Of Concept ).
There are many vendors which provide us with state of art to Extract, Transform and Load ( ETL ) solution, so capturing the raw data and storing it is not a problem, also storage these days are cheap so eliminating the problem of storing massive amount of data. The real problem is how do we integrate all pieces of block and build a wall so that we are able to retain our PnL statement.
Till date with my knowledge of handling Data Quality issues, there is only one Vendor who has tried to solve this problem is Oracle by coining a yet another term called Real Time Heterogeneous Data Base integration using Oracle Golden Gate, to be very honest two different database servers 10000 miles a part it takes 5-10 seconds to synchronize them, but still the question is maintaining expensive servers and then buying the license to integrate them, is the expense worth extracting such information ? OGG is primarily used for DRS system ( Disaster Recovery System ) and less of data integration with few exceptions.
The solution to the above problem where we can arrive at a consolidated solution by combining all the data points stored at different system without physically synchronizing them is XML.For many readers it might be a surprise but YES.. XML is a strong tool for data exchange primarily used as communication channel between web and back end data server, but this very fact can be used as communication channel between different data points on a common platform eg web. We may use the power of XML data transfer technique and build a robust reporting system that can interact with almost any data source and display the result, and all this without spending a penny.
The concept is really simple to understand but equally tough to implement. Would appreciate any suggestions, reading comments on this.
hope you might have enjoyed my idea, although its just a summary of what I have been reading and thinking.
Best
Sid
