From Unstructured to Structured Data

In today’s age of technology-driven industry, data is what can be termed as our competitive currency. Buried deep within the immeasurable volumes of raw data generated by social media, transactional systems, search engines and other technologies, lie critical strategic and operational insights, which when harnessed by analytics have the power to validate/clarify assumptions, facilitate informed decision making and pave new roads to the future. When talking about the mass categorization that is central to most data analytics, there are two types of relevant data that impact the speed of assimilation and information recall – structured and unstructured data. Until quite recently, the standard approach to data and analytics by organizations was quite passive, with the ultimate goal being of ‘report generation’, wherein analytical capabilities were applied to limited volumes of structured data that were siloed within a system/company function. Quality issues with master data, lack of user sophistication and the inability to integrate data across various enterprise systems, often resulted in a limited scope of insights, which at worst would be misleading. The paradigm shift in analytics today has led to companies understanding the importance of harnessing distributed data architecture, in-memory processing, machine learning, natural language processing and cognitive analysis to identify valuable patterns and insights. This means, looking beyond traditional problem-solving methods and creatively exploring the vast volumes of unstructured data to unearth highly nuanced business/customer/operational insights that structured data assets in an organization’s possession, may not be able to reveal. Many organizations have a large collection of both structured and unstructured data, sitting idle. In the case of traditional unstructured data like emails, notes, messages documents, logs, notification etc. the text-based data wasn’t usually within a relational database, until quite recently. Within these unstructured assets, lie buried valuable information on customer behavior, pricing, competitors etc., this holds true especially for multinational companies, which may contain potentially valuable data, yet to be translated created/generated for the overseas markets. While working on a recent project for a background verification agency – converting unstructured data into structured data sets, we were able to look beyond the traditional analytics route of problem-solving and curate business insights.; using a combination of deep modeling and statistical techniques etc., with industry/function specific insights and problem framing, veering beyond technical IT into strategy, while solving core issues. Hence, it is time to recognize analytics as an augmentation to overall business strategy, as opposed to an IT function. By understanding an organization’s goals, the value to be delivered can be determined along with the questions to be asked which can help in harnessing the available data and generate answers. Data analytics can be an insight-driven advantage for an organization, by focusing on harnessing powerful strategic/customer/operational insights, hidden within unstructured and untraditional data assets; to be fully realized by employing a business-wide analytics strategy.

Author: Rajiv Diwan, Practice Head – Advanced Analytics

Rajiv Diwan heads the Advanced Analytics Practice at ITL and is responsible for both – customer acquisition and defining solution offerings of the Practice. He has setup the CoE on Machine Learning from scratch at ITL and has been instrumental in penetrating into new verticals for ITL; including BFSI. Rajiv is engineering graduate from BIT, Bangalore with specialization in Computer Science; having over 18 years of experience in Analytics, Data warehousing, BI and Large Program Management.

Post a Comment