Tackling Variety in Big Data Implementation

By Pawan Mehta June 1, 2017 Big Data Management No comments yet

Big Data Management at a high level consists of managing Velocity, Volume and Variety of data. While most integrated platforms tackle volume & velocity, very few pay attention to managing variety of data.

We have tackled the problem of 'Variety' by integrating handling of various data sources like twitter streams, Databases, block chain data etc. using a single integrated platform which uses Open Source Technologies as the backbone.

Below are some of the use cases tackled by the integrated solution.

 1. CRM Analytics based on various databases ( Oracle, SQL, Mariadb)

We analyzed campaign effectiveness, lead, opportunity & sales order data from ERP systems to produce intelligent reports.


Data Source: Maria Db



Data Source: Oracle Db

 2. Real Time Twitter feed & Facebook sentiment analysis

Using publicly available Twitter & Facebook feeds, real time analysis of feed data is possible.

Architecture for Social Media Analytics

Below is interesting sentiment analysis of twitter & Facebook feeds of various politicians during the UP elections.

Data Source: Twitter & Facebook

3. Block Chain Data

Block chain has many applications, the most famous one being bit coin. As a case study, we have parsed publicly available bit coin data using a block chain parser integrated with HDFS.

This data can be further analyzed to detect fraudulent patterns.

Data Source: Publicly available bitcoin data

If you have any interesting use cases that you would like to implement for your organization using an integrated big data platform, do write in to us at: hello@openetech.com

Pawan Mehta

Pawan is an independent consultant who helps medium sized companies implement ERPNext and BI solutions. During his free time he likes to jog, play carrom and spend time with his family.

No comments yet

No comments yet. Start a new discussion.

Add Comment