hadoop-hdfs

SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast! Blob store has O(1) disk seek, local tiering, cloud tiering. Filer supports cross-cluster active-active replication, Kubernetes, POSIX, S3 API, encryption, Erasure Coding for warm storage, FUSE mount, Hadoop, WebDAV.

MorphL Community Edition uses big data and machine learning to predict user behaviors in digital products and services with the end goal of increasing KPIs (click-through rates, conversion rates, etc.) through personalization

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Learn how to use Spark SQL and HSpark connector package to create / query data tables that reside in HBase region servers

A framework for easy development of projects with ETL and ELT processes for working with data sources and data warehouses on the Micro Focus Vertica platform

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

A Java Hdfs client example and full Kerberos example for call hadoop commands directly in java code or on your local machine.

Big data projects implemented by Maniram yadav

Toy Hadoop cluster combining various SQL-on-Hadoop variants

Marathon on yarn

Twitter + Flume + Hadoop (HDFS, MapReduce) + Neo4j + Pyhton

HokStack - Run Hadoop Stack on Kubernetes

Instructions on setting up Hadoop, HDFS, java, sbt, kafka, scala, spark and flume on Ubuntu 18.04

Vehicle Fuel Hadoop MapReduce

Neat and Handy Place for all Hadoop codes

Analysis of YouTube Data using Hadoop Mapreduce framework in Java.

Install Hadoop, HDFS, Yarn and Spark on 3 Ubuntu 18.04 Machines

Python scripts to assist setting up Hadoop v1 in Linux and starting a NameNode, DataNodes and Client.

The ultimate aim of this Hadoop starter-kit Git repository is to help you deploy and manage Hadoop ecosystem components on Docker containers and AWS cloud. You need user manual to operate the starter-kit. Contact us to get the user manual and start learning Hadoop today.

Python scripts for automating the hadoop dfs setup

API to interact with HDFS

Projects using S3, Amazon SageMaker, AWS Lambda Function, Amazon Forecast; Projects related to SQL, Hadoop, Flink (Java), and Google Map API (Jun 2019 - Jul 2019)

EMR 5.25.0 cluster single node Hadoop docker image. With Amazon Linux, Hadoop 2.8.5 and Hive 2.3.5

🐘Yet another Hadoop playground

Hadoop hdfs mapreduce hive spark使用案例

In this project we will fetch tweets using Apache Flume. We will also use the memory channel to buffer these tweets and HDFS sink to push these tweets into the HDFS.

Machine Learning for Forest Fire Prediction using Hadoop ecosystems and Spark Tools (Pyspark)

Repositório criado para armazenar anotações e atividades desempenhadas no treinamento na plataforma da Digital Inovattion One (DIO) para o Processo seletivo de Engenheiros de Dados pela empresa Everis.