Hadoop Interview Questions and Answers
Question - 1 : - What is the difference between Hadoop and Traditional RDBMS?
Answer - 1 : -
Criteria | Hadoop | RDBMS |
| Datatypes | Processes semi-structured and unstructured data. | Processes structured data. |
| Schema | Schema on Read | Schema on Write |
| Best Fit for Applications | Data discovery and Massive Storage/Processing of Unstructured data. | Best suited for OLTP and complex ACID transactions. |
| Speed | Writes are Fast | Reads are Fast |
Question - 2 : - What do you mean by the term or concept of Big Data?
Answer - 2 : -
Big Data means a set or collection of large datasets that keeps on growing exponentially. It is difficult to manage Big Data with traditional data management tools. Examples of Big Data include the amount of data generated by Facebook or Stock Exchange Board of India on a daily basis. There are three types of Big Data:
- Structured Big Data
- Unstructured Big Data
- Semi-structured Big Data
Question - 3 : - What are the characteristics of Big Data?
Answer - 3 : -
The characteristics of Big Data are as follows:
- Volume
- Variety
- Velocity
- Variability
- Where,
Volume means the size of the data, as this feature is of utmost importance while handling Big Data solutions. The volume of Big Data is usually high and complex.
Variety refers to the various sources from which data is collected. Basically, it refers to the types, structured, unstructured, and semi-structured, and heterogeneity of Big Data.
Velocity means how fast or slow the data is getting generated. Basically, Big Data velocity deals with the speed at which the data is generated from business processes, operations, application logs, etc.
Variability, as the name suggests, means how differently the data behaves in different situations or scenarios in a given period of time.
Question - 4 : - What are the various steps involved in deploying a Big Data solution?
Answer - 4 : -
Deploying a Big Data solution includes the following steps:
Data Ingestion: As a first step, the data is drawn out or extracted from various sources so as to feed it to the system.
Data Storage: Once data ingestion is completed, the data is stored in either HDFS or NoSQL database.
Data Processing: In the final step, the data is processed through frameworks and tools such as Spark, MapReduce, Pig, etc.
Question - 5 : - What are the core components of Hadoop?
Answer - 5 : -
The core components of Hadoop are HDFS and YARN.
HDFS: The components of HDFS are name node, secondary name node, and data node.
YARN: The components of YARN are resource manager and node manager.
Question - 6 : - What is the reason behind using Hadoop in Big Data analytics?
Answer - 6 : -
Businesses generate a lot of data in a single day and the data generated is unstructured in nature. Data analysis with unstructured data is difficult as it renders traditional big data solutions ineffective. Hadoop comes into the picture when the data is complex, large and especially unstructured. Hadoop is important in Big Data analytics because of its characteristics:
- Data storage
- Data processing
- Collection plus extraction of data
Question - 7 : - What do you understand by fsck in Hadoop?
Answer - 7 : -
fsck stands for file system check in Hadoop, and is a command that is used in HDFS. fsck checks any and all data inconsistencies. If the command detects any inconsistency, HDFS is notified regarding the same.
Question - 8 : - Can you explain some of the important features of Hadoop?
Answer - 8 : -
Some of the important features of Hadoop are:
Fault Tolerance: Hadoop has a high-level of fault tolerance. To tackle faults, Hadoop, by default, creates three replicas for each block at different nodes. This number can be modified as per the requirements. This helps to recover the data from another node if one node has failed. Hadoop also facilitates automatic recovery of data and node detection.
Open Source: One of the best features of Hadoop is that it is an open-source framework and is available free of cost. Hadoop also allows its users to change the source code as per their requirements.
Distributed Processing: Hadoop stores the data in a distributed manner in HDFS. Distributed processing implies fast data processing. Hadoop also uses MapReduce for the parallel processing of the data.
Reliability: One of the benefits of Hadoop is that the data stored in Hadoop is not affected by any kind of machine failure, which makes Hadoop a reliable tool.
Scalability: Scalability is another important feature of Hadoop. Hadoop’s compatibility with other hardware makes it a preferred tool. You can also easily add new hardware to the nodes in Hadoop.
High Availability: Easy access to the data stored in Hadoop makes it a highly preferred Big Data management solution. Not only this, the data stored in Hadoop can be accessed even if there is a hardware failure as it can be accessed from a different path.
Question - 9 : - What is Hadoop and what are its components?
Answer - 9 : -
Apache Hadoop is the solution for dealing with Big Data. Hadoop is an open-source framework that offers several tools and services to store, manage, process, and analyze Big Data. This allows organizations to make significant business decisions in an effective and efficient manner, which was not possible with traditional methods and systems.
Question - 10 : - In what all modes can Hadoop be run?
Answer - 10 : -
Hadoop can be run in three modes:
Standalone Mode: The default mode of Hadoop, standalone mode uses a local file system for input and output operations. This mode is mainly used for debugging purposes, and it does not support the use of HDFS. Further, in this mode, there is no custom configuration required for mapred-site.xml, core-site.xml, and hdfs-site.xml files. This mode works much faster when compared to other modes.
Pseudo-distributed Mode (Single-node Cluster): In the case of pseudo-distributed mode, you need the configuration for all the three files mentioned above. All daemons are running on one node; thus, both master and slave nodes are the same.
Fully distributed mode (Multi-node Cluster): This is the production phase of Hadoop, what it is known for, where data is used and distributed across several nodes on a Hadoop cluster. Separate nodes are allotted as master and slave nodes.