All Your HBase Are Belong to Clojure
I’m sure you’ve heard a variation on this story before… So I have this web crawler and it generates these super-detailed log files, which is great ‘cause then we know what it's doing but it’s also kind of bad ‘cause when someone wants to know why the crawler did this thing but not that thing I have, like, literally gajigabytes of log files and I’m using grep and awk and, well, it’s not working out
Programming Hadoop with Clojure
This article is a short introduction into programming Hadoop using Clojure language. Introduction The Hadoop is free implementation of infrastructure for scalable, distributed computing. It was started as implementation of ideas of MapReduce and GFS, that was introduced by Google, but later, many different projects were included into it. Hadoop is actively used by many projects, including commerci
Hadoop Streaming 㧠Locality-Sensitive Hashing を実装ã—ã¦ã¿ã‚‹ | Atsushi TATSUMA Web Page
ã¯ã˜ã‚ã« ç ”ç©¶å®¤ã§ã¯ã€å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚’対象ã¨ã—ãŸæ¤œç´¢ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã«ã¤ã„ã¦ç ”究ã—ã¦ã„ã¾ã™ã€‚ 分散処ç†ã«ã‚ˆã‚‹ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ä½œæˆã‚’考ãˆã¦ã„ã¦ã€ä»Šå›ž Hadoop Streaming ã§ã©ã“ã¾ã§ã§ãã‚‹ã‹ã‚’試ã™ã¹ã〠Locality-Sensitive Hashing (LSH) を実装ã—ã¦ã¿ã¾ã—ãŸã€‚ 実装ã—ãŸã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã«ã¤ã„㦠LSH ã«ã¯ã„ãã¤ã‹ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®ãƒãƒªã‚¨ãƒ¼ã‚·ãƒ§ãƒ³ãŒã‚ã‚Šã¾ã™ã€‚ LSH ã®è©³ç´°ã¯ã€ãƒ–ãƒã‚°ãªã‚Šè«–æ–‡ãªã‚Šæœ¬ãªã‚Šã‚ã‚Šã¾ã™ã®ã§ã€ã“ã“ã§ã¯çœç•¥ã—ã¾ã™ãŒã€ é¡žä¼¼ã—ãŸãƒ‡ãƒ¼ã‚¿ã«åŒã˜ãƒãƒƒã‚·ãƒ¥å€¤ã‚’与ãˆã‚‹ã“ã¨ã§ã€æ¤œç´¢ã‚’高速化ã—よã†ã¨ã„ã†ã‚¢ã‚¤ãƒ‡ã‚£ã‚¢ã§ã™ã€‚ ã“ã®ãƒãƒƒã‚·ãƒ¥å€¤ã«ã¯ã€0101 ã¨ã‹ã®çŸã„ãƒã‚¤ãƒŠãƒªç¬¦å·ãŒå¥½ã¾ã‚Œã¾ã™ã€‚ 今回ã¯ã€ã„ãã¤ã‹ã‚ã‚‹ LSH ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®ã†ã¡ã€ Charikar,M., Similarity estimation techniques from ro
Twitter to open source MySQL-to-Hadoop tool
Hadoop Summit Twitter intends to open source an additional piece of the Hadoop-happy infrastructure it uses for internal data analysis. Known as Crane, this is a tool for moving data from MySQL into Hadoop, the open source data-crunching platform based on Google's proprietary infrastructure. Twitter uses Hadoop for ad hoc analysis of data collected from its famous microblogging service, but the pl
Cascading
Please note that all new project news and releases have moved to https://cascading.wensel.net The Cascading Ecosystem is a collection of applications, languages, and APIs for developing data-intensive applications. At the ecosystem core is Cascading, a Java API for defining complex data flows and integrating those flows with back-end systems, and a query planner for mapping and executing logical f
Cloudera Hadoop Training: MapReduce and HDFS
Verify to continue To continue, please confirm that you're a human (and not a spambot).
Quick Start
The purpose of this document is to help you get a single-node Hadoop installation up and running very quickly so that you can get a flavour of the Hadoop Distributed File System (see HDFS Architecture) and the Map/Reduce framework; that is, perform simple operations on HDFS and run example jobs. Supported Platforms GNU/Linux is supported as a development and production platform. Hadoop has been de
Map/Reduce Tutorial
This document comprehensively describes all user-facing facets of the Hadoop Map/Reduce framework and serves as a tutorial. Ensure that Hadoop is installed, configured and is running. More details: Hadoop Quick Start for first-time users. Hadoop Cluster Setup for large, distributed clusters. Hadoop Map/Reduce is a software framework for easily writing applications which process vast amounts of dat
Cornell Web Lab: Hadoop
Quick Guide to Developing and Running Hadoop Jobs (Mac OS X 10.6) This guide is written to help Cornell students using Mac OS X 10.6 with setting up a development environment for working with Hadoop and running Hadoop jobs on the Cornell Center for Advanced Computing (CAC) Hadoop cluster. This guide will walk you through compiling and running a simple example Hadoop job. More information is availa
HowToDebugMapReducePrograms - HADOOP2 - Apache Software Foundation
How to Debug Map/Reduce Programs Debugging distributed programs is always difficult, because very few debuggers will let you connect to a remote program that wasn't run with the proper command line arguments. Start by getting everything running (likely on a small input) in the local runner. You do this by setting your job tracker to "local" in your config. The local runner can run under the debugg
)
Running Hadoop On Ubuntu Linux (Single-Node Cluster) - Michael G. Noll
What we want to do In this short tutorial, I will describe the required steps for setting up a single-node Hadoop cluster using the Hadoop Distributed File System (HDFS) on Ubuntu Linux. Hadoop is a framework written in Java for running applications on large clusters of commodity hardware and incorporates features similar to those of the Google File System and of MapReduce. HDFS is a highly fault
HadoopãŒã‚¹ã‚±ãƒ¼ãƒ«ãƒ»ã‚¢ã‚¦ãƒˆã™ã‚‹ä»•çµ„ã¿
å‰å›žã®è¨˜äº‹ã§ã¯ã€HadoopãŒè†¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã‚’ãƒãƒƒãƒå‡¦ç†ã™ã‚‹ãŸã‚ã®ã€Œã‚¤ãƒ³ãƒ•ãƒ©ã€ã¨ã—ã¦ã®æ€§è³ªã‚’å‚™ãˆã¦ã„ã‚‹ã“ã¨ã€æƒ…å ±çˆ†ç™ºæ™‚ä»£ã®æ–°ãŸãªã‚¤ãƒ³ãƒ•ãƒ©ã¨ã—ã¦æ™®åŠã—ã¤ã¤ã‚ã‚‹ã“ã¨ã‚’説明ã—ã¾ã—ãŸã€‚ãã®ä¸ã§ã€æƒ…å ±çˆ†ç™ºæ™‚ä»£ã«å¿…è¦ã¨ã•ã‚Œã‚‹ã‚¤ãƒ³ãƒ•ãƒ©ã¯ã€ã€Œã‚¹ã‚±ãƒ¼ãƒ«ãƒ»ã‚¢ã‚¦ãƒˆãŒå¯èƒ½ã§ã‚ã‚‹ã“ã¨ã€ã¨ã„ã†æ¡ä»¶ã‚’å‚™ãˆã¦ã„ãªã‘ã‚Œã°ãªã‚‰ãªã„ã“ã¨ã‚’示ã—ã¾ã—ãŸã€‚サーãƒãƒ¼ã®å°æ•°ã‚’増やã™ã“ã¨ã§å®¹æ˜“ã«ã‚·ã‚¹ãƒ†ãƒ ã®å‡¦ç†æ€§èƒ½ãŒå¢—ã‚„ã›ã‚‹ã“ã¨ã¯ã€Hadoopã®é‡è¦ãªç‰¹å¾´ã§ã™ã€‚今回ã¯ã€HadoopãŒã©ã®ã‚ˆã†ã«ã—ã¦ã‚¹ã‚±ãƒ¼ãƒ«ãƒ»ã‚¢ã‚¦ãƒˆã‚’å¯èƒ½ã«ã—ã¦ã„ã‚‹ã‹ã‚’ã€ã€Œåˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ãƒ»ã‚·ã‚¹ãƒ†ãƒ ã€ã¨ã€ŒMapReduceフレームワークã€ã®2ã¤ã®è¦³ç‚¹ã‹ã‚‰è§£èª¬ã—ã¾ã™ã€‚ スケール・アウトã¨ã¯ï¼Ÿ コンピュータ・システムを新ãŸã«æ§‹ç¯‰ã™ã‚‹ã¨ãã¯ã€å¿…ãšã€Œé‹ç”¨ã€ã®ã“ã¨ã‚’考ãˆã¦ãŠãå¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚システム・トラブルãŒç™ºç”Ÿã—ãŸå ´åˆã®å¯¾å¿œç–を決ã‚ãŸã‚Šã€å°†æ¥ã®ä»•æ§˜å¤‰æ›´ã«å‚™ãˆã¦ãƒ—ãƒã‚°ãƒ©ãƒ ã«æ‹¡å¼µæ€§ã‚’æŒãŸã›ãŸã‚Š
Welcome to Apacheâ„¢ Hadoopâ„¢!
Apache Hadoop The Apache® Hadoop® project develops open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation an
Hadoop Training: Introduction To Pig « Cloudera » Apache Hadoop for the Enterprise
Date Thursday, January 21st, 2010 Description Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. This lecture includes a work-along exercise. The next video is a screencast of a user performing this exercise. Duration 50 Minutes Materials Slides: PDF Next Steps Pig
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
GitHub - cloudera/sqoop: Sqoop has moved to Apache!
Open Source & Open Standards | Cloudera
Product Downloads | Cloudera
Hadoop for MySQL people
Pig Latin Reference Manual 1
Overview
{{#tags}}- {{label}}
{{/tags}}