Scribd
Upload
4 views2 pages

17. Using Map Reduce Concept, Implement a Java Pro...

The document provides a basic implementation of a MapReduce job for adding two numbers using Hadoop, detailing the Mapper and Reducer classes. It also includes instructions for installing Thrift, generating HBase Thrift bindings, and interacting with HBase, as well as HQL commands for creating and managing a bank account table. Additionally, it mentions Hadoop commands for deleting files and notes on best practices for handling exceptions and using HBase efficiently.

Uploaded by

Sneha.j Sneha.j
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views2 pages

17. Using Map Reduce Concept, Implement a Java Pro...

The document provides a basic implementation of a MapReduce job for adding two numbers using Hadoop, detailing the Mapper and Reducer classes. It also includes instructions for installing Thrift, generating HBase Thrift bindings, and interacting with HBase, as well as HQL commands for creating and managing a bank account table. Additionally, it mentions Hadoop commands for deleting files and notes on best practices for handling exceptions and using HBase efficiently.

Uploaded by

Sneha.j Sneha.j
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

17.

MapReduce for Addition of Two Numbers


While this might seem trivial for a MapReduce job, here's a basic implementation:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

import java.io.IOException;

public class AddTwoNumbers {

public static class Map extends Mapper<Object, Text, IntWritable,


IntWritable> {
public void map(Object key, Text value, Context context)
throws IOException, InterruptedException {
String[] numbers = value.toString().split(",");
int num1 = Integer.parseInt(numbers[0]);
int num2 = Integer.parseInt(numbers[1]);
context.write(new IntWritable(1), new IntWritable(num1 +
num2));
}
}

public static class Reduce extends Reducer<IntWritable,


IntWritable, IntWritable, IntWritable> {
public void reduce(IntWritable key, Iterable<IntWritable>
values, Context context) throws IOException, InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get();
}
context.write(key, new IntWritable(sum));
}
}

public static void main(String[] args) throws Exception {


Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "add two numbers");
job.setJarByClass(AddTwoNumbers.class);
job.setMapperClass(Map.class);
job.setReducerClass(Reduce.class);
job.setOutputKeyClass(IntWritable.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}

18. Thrift and HBase


1. Install Thrift: Follow the instructions for your specific operating system and programming
language.
2. Generate HBase Thrift Bindings: Use the thrift command-line tool to generate the
necessary code.
3. Interact with HBase: Use the generated code to create HBase clients and perform
operations like creating tables, putting and getting data.
19. HQL Commands
-- Create a table
CREATE TABLE bankaccount (
cid INT,
cname STRING,
cage INT,
acctype STRING
);

-- Load data (assuming a CSV file)


LOAD DATA LOCAL INPATH '/path/to/your/data.csv' INTO TABLE
bankaccount;

-- Display table content


SELECT * FROM bankaccount;

20. Hadoop Command for Deleting Files


hadoop fs -rm -r <hdfs_directory>

Note:
● The MapReduce job for addition is a simplified example. In real-world scenarios, you
would likely use a more efficient approach.
● For more complex HBase interactions, consider using the HBase Java API or other
higher-level libraries.
● Always handle exceptions and error conditions in your MapReduce and HBase code.
● Consider using HBase's built-in mechanisms for data loading and querying, as they can
be more efficient than using MapReduce.
● https://blog.actorsfit.com/a?ID=01650-02aca481-7de4-47a3-858d-20cd50875b63
● https://github.com/camsas/Musketeer
● https://github.com/kallurivenkatesh/oddman
● https://github.com/alexcpsec/cscie185-hw
● https://www.projectpro.io/article/mapreduce-vs-pig-vs-hive/163
● https://www.cnblogs.com/ylxb2539989915/p/16329246.html

You might also like