Hadoop Tutorial for Beginners: Hadoop Basics

What is Hadoop?

Hadoop (the full proper name is Apache^TM Hadoop^®) is an open-source framework that was created to make it easier to work with big data. It provides a method to access data that is distributed among multiple clustered computers, process the data, and manage resources across the computing and network resources that are involved. “Hadoop” commonly refers to the core technology that consists of the four main components described below, but is also frequently used in reference to the entire ecosystem of supporting technologies and applications.

“Hadoop” also is often used interchangeably with “big data,” but it shouldn’t be. Hadoop is a framework for working with big data. It is part of the big data ecosystem, which consists of much more than Hadoop itself.

Hadoop is a distributed framework that makes it easier to process large data sets that reside in clusters of computers. Because it is a framework, Hadoop is not a single technology or product. Instead, Hadoop is made up of four core modules that are supported by a large ecosystem of supporting technologies and products. The modules are:

• Hadoop Distributed File System (HDFS^TM) –Provides access to application data. Hadoop can also work with other file systems, including FTP, Amazon S3 and Windows Azure Storage Blobs (WASB), among others.
• Hadoop YARN – Provides the framework to schedule jobs and manage resources across the cluster that holds the data
• Hadoop MapReduce – A YARN-based parallel processing system for large data sets.
• Hadoop Common – A set of utilities that supports the three other core modules.

Some of the well-known Hadoop ecosystem components include Oozie, Spark, Sqoop, Hive and Pig.

(This article is part of our Hadoop Guide. Use the right-hand menu to navigate.)

What Hadoop isn’t

In this tutorial for beginners, it’s helpful to understand what Hadoop is by knowing what it is not.

• Hadoop is not “big data” – the terms are sometimes used interchangeably, but they shouldn’t be. Hadoop is a framework for processing big data.
• Hadoop is not an operating system (OS) or packaged software application.
• Hadoop is not a brand name. It is an open source project, although “Hadoop” may be used as part of registered brand names.