SERVICES

Data Analisys

Data Analysis

 

Are you able to look at your data at 360°?

Today large enterprises are living the struggle of dealing with a variety and a volume of data that it's exploding.

 

Log files, webcams, sensors, social network data are invading enterprise life at different level. Moreover, the speed of decision on the business changes due to the fact we read in that data has become another crucial factor of the enterprise performances on the market.

 

This has lead to new generation of data analysis technology called : Big Data.

Nextrace is going to prepare a team to offer solutions in this area actually refining its knowledge skills on:

  • Hadoop platform software
  • Hadoop distributions (like Hortonworks)
  • Data Analysis softwares like : Qlikview, Oracle Big Data Discovery


We are evaluating Big Data solution in the area of security but also open to other areas of interest.


Our first application of Big Data is in the area of log files. Applications, databases, webservers, etc. Thousands of this files are produced every day by every kind of servers in the enterprises. Analysis of this files and cross check on different kind of log files is quite an issue.

HortonWorks

Leveraging the Hortonworks distribution of Hadoop, we have set up a demo working with the following components:

  • Hadoop/HDFS for the storage of log file content
  • Kafka for the message broker receiving log files content
  • Storm (integrated with Kafka) to emit log tuples consumed by Storm bolts populating Hbase tables and HDFS files
  • Hive to access HDFS files leveraging SQL language
  • HBase to experiment the usage of big tables with log content

The main principle of this solution is to get the log content as soon as it is produced and push it to the BigData infrastructure for immediate analysis. To do so we need to capture the log fle content online. This is possible with a "tail" shell command redirect to a simple tcp connection from the server where the log is produced to the BigData infrastructure.

Once this content is captured it can be manipulated to fit into a relational table or even in a "Big" table. Analysis tools like Qlik, Tableau, Excel, Oracle Big Data Discovery can be used to analyse the data and run even complex algorithms.

The following picture gives an idea of the solution.

LogDemoArchtecture

Follow this link to see details of the demo

Share
e-max.it: your social media marketing partner
Category: Focus 2

Your Next Step...