hadoop-hdfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, local tiering, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, Hadoop, WebDAV, encryption, Erasure Coding.

MorphL Community Edition uses big data and machine learning to predict user behaviors in digital products and services with the end goal of increasing KPIs (click-through rates, conversion rates, etc.) through personalization

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Learn how to use Spark SQL and HSpark connector package to create / query data tables that reside in HBase region servers

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

A Java Hdfs client example and full Kerberos example for call hadoop commands directly in java code or on your local machine.

Big data projects implemented by Maniram yadav

Toy Hadoop cluster combining various SQL-on-Hadoop variants

Marathon on yarn

HokStack - Run Hadoop Stack on Kubernetes

Twitter + Flume + Hadoop (HDFS, MapReduce) + Neo4j + Pyhton

Instructions on setting up Hadoop, HDFS, java, sbt, kafka, scala, spark and flume on Ubuntu 18.04

Mammoth is a container based hadoop distributed system log analyzer. Sponsed by Mantech and Naver Cloud Platform.

In this task, we had to write a MapReduce program to analyze the sentiment of a keyword from a list of comments. This was done using Hadoop HDFS.

A MapReduce program to conduct sentiment analysis of a keyword from a list of comments.

In this task, we had to calculate the average temperature for each year from the given dataset using Hadoop HDFS. We had to create a MapReduce function to perform this task.

Vehicle Fuel Hadoop MapReduce

Neat and Handy Place for all Hadoop codes

Analysis of YouTube Data using Hadoop Mapreduce framework in Java.

Install Hadoop, HDFS, Yarn and Spark on 3 Ubuntu 18.04 Machines

Repositório criado para armazenar anotações e atividades desempenhadas no treinamento na plataforma da Digital Inovattion One (DIO) para o Processo seletivo de Engenheiros de Dados pela empresa Everis.

Python automation in linux

Data engineer interview questions.

Python scripts to assist setting up Hadoop v1 in Linux and starting a NameNode, DataNodes and Client.

The ultimate aim of this Hadoop starter-kit Git repository is to help you deploy and manage Hadoop ecosystem components on AWS cloud using Docker, Kubernetes, and Chef.

A MapReduce program to calculate the average length of comments.

Python scripts for automating the hadoop dfs setup

API to interact with HDFS

Projects using S3, Amazon SageMaker, AWS Lambda Function, Amazon Forecast; Projects related to SQL, Hadoop, Flink (Java), and Google Map API (Jun 2019 - Jul 2019)