The Wayback Machine - https://web.archive.org/web/20200812010954/https://github.com/topics/hadoop-docker
Skip to content
#

hadoop-docker

Star

Here are 31 public repositories matching this topic...

Language: All
Filter by language
All 31 Shell 15 Dockerfile 4 Python 3 CSS 1 Java 1 Jupyter Notebook 1 PigLatin 1 TSQL 1
Sort: Best match
Sort options
Best match Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

Mgosi / Big-Data-Analysis-using-MapReduce-in-Hadoop

Star

We explore data by using Big Data Analysis and Visualization skills. To obtain this, we perform 3 main operations. i.e. i)Data Aggregation through different sources. ii) Big Data Analysis using MapReduce and iii) Visualization through Tableau. Data Analysis is very critical in understanding the data, and what we can do with the data. For small datasets it is easier to process and obtain the results. But as for big companies, it becomes crucial for them to obtain the trends of the company for any changes need to be made. Hence we introduce Big Data Analysis to solve this problem. In this lab, we collect close to 20000 tweets, 500 articles on New York Times and 500 articles on Common Crawl Data about Entertainment, which is our main topic of discussion. Using this data, we perform preprocessing and feed it to a MapReduce to find the Word Count and Word Co-Occurrence. Using this, we find the trend of the data collected in this topic. We have used Python to perform Data Analysis.Data Analysis is very critical in understanding the data, and what we can do with the data. For small datasets it is easier to process and obtain the results. But as for big companies, it becomes crucial for them to obtain the trends of the company for any changes need to be made. Hence we introduce Big Data Analysis to solve this problem. In this lab, we collect close to 20000 tweets, 500 articles on New York Times and 500 articles on Common Crawl Data about Entertainment, which is our main topic of discussion. Using this data, we perform preprocessing and feed it to a MapReduce to find the Word Count and Word Co-Occurrence. Using this, we find the trend of the data collected in this topic. We have used Python to perform Data Analysis.
docker big-data twitter-api hdfs tableau data-processing data-pipeline hadoop-docker common-crawl big-data-analytics tweet-collector
  • Updated Oct 5, 2019
  • Jupyter Notebook

Improve this page

Add a description, image, and links to the hadoop-docker topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the hadoop-docker topic, visit your repo's landing page and select "manage topics."

Learn more

You can’t perform that action at this time.