Big Data In Action

A working setup of docker which runs Hadoop and other big data components are very useful for development and testing of a big data project. When I was in need, I couldn't found a simple and a working docker setup and here would like to bring together it for you. Please download the files from here .

I searched a lot on google and struggled a lot to brought some data points in order to learn the basics of data analytics. That is why I decided to share the same with you for not waste much of your time. These are some general analytical data points collected about the United States. Gasoline Price  Interest Rate Unemployment Rate Currency Strength  GDP Inflation House Price Index I will try to enrich the data and will be adding more metrics to it. If anybody interested in contributing data is welcomed. ( My official email jobs.thomas1@yahoo.co m )  You can download the data from git hub repository. https://github.com/jobmthomas/DataForAnalytics.git I hope this is really helpful for many.

HIPI: Hipi is Hadoop's Image Processing interface. This provides a set of tools and Input format to process a bulk amount of images using Hadoop's Distributes File System (HDFS) and MapReduce . STEPS INVOLVED: In hipi, the entire process can be categorized into 2 parts. 1) Converting all images into a bulk file(HIPI Image Bundle). 2) Processing the created bulk file of an image using HIPI's image input formats.     The cull (culler class) is used to filter out images with low clarity or defects ISSUES WITH HIPI: To simulate my bulk image processing scenario, I used a java program to create multiple copies of the same image with different names in a single directory. then by using  hipi's  utility, I converted all images into a bulk file (known as the hip file). To check whether all images exist in the bulk file, I have done the reverse process (Converted HIP file into multiple images). There is a utility of hip...

What is Big Data?            We are living in an era where a huge volume of data has been generated. The below points indicates the rate at which data has been generated in the world. Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. According to IBM, 80% of data captured today is unstructured, from sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals, to name a few. All of this unstructured data is Big Data. Definition : Gartner defines Big Data as high volume, velocity, and variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.  If your company is dealing with a set of data that makes you difficult to process and make information out of it, then you are f...