Skip to main content

Data Locality in Hadoop

What is data Locality in Hadoop?

We can say that, Data locality improves the overall execution of the system and makes Hadoop faster. It reduces the network congestion.

There are two benefits of data Locality in Hadoop.

i. Faster Execution

In data locality, the program is moved to the node where data resides instead of moving large data to the node, this makes Hadoop faster. Because the size of the program is always lesser than the size of data, so moving data is a bottleneck of network transfer.

ii. High Throughput

Data locality increases the overall throughput of the system.

Hope this helps.
Labels:

Comments

Post a Comment

Popular posts from this blog

JPS in Hadoop

What is JPS in Hadoop? For checking running process in our Hadoop cluster we use JPS command. JPS  stands for Java  Virtual Machine Process Status Tool  or [JVM Process Status tool]. Below are some important points which one should remember at the time of using JPS command. To check all running nodes on the host via jps, you need to run the command as r oot. Otherwise, jps will only show nodes which you have currently logged-in user as. Example of JPS What-is-JPS-in-Hadoop By Prajjwal
3 comments
Read more

NOSQL Databases.

NOSQL databases: NoSQL databases are the one which comes in the picture when we no longer work with our traditional databases. "NoSQL" : stand for [NOT ONLY SQL] is an replacement to traditional relational databases in big data world. it is basically schema-free and scalable database, best suitable for solving big data challenges. Properties of NOSQL databases. Schema-free: A SQL database needs to be pre-defined with schema before we can add data to it. A NOSQL database on the other hand allows data insertion dynamically. i.e. we can insert data into NOSQL database without any predefined schema. Scalable : Usually SQL databases scale-up vertically. i.e. when the load is high , the system will be individually upgraded. where as in NOSQL databases it is done horizontally, since these are distributed in nature.  Types and Example of NOSQL databases: Key-Value Stores:                ex : Redis and Amazon S3 Column-Value Stor...
Post a Comment
Read more

How write operation done in HDFS?

How write operation done in HDFS?  HDFS follows Write once Read many model, s o we can't edit files which are already present in HDFS. Syntax to write data in HDFS:  hdfs dfs -put <local/file/path> <HDFS/location/where file needs to write>  exmple:         hdfs dfs -put /home/prajjwal/file1.txt  /landing_location/ To write a file in HDFS, a client needs to interact with master i.e. namenode (master). Namenode provides the address of the datanodes (slaves) on which client will start writing the data. Client can directly write data on the datanodes, now datanode will create data write pipeline. The first D ataNode will copy the block to another datanode, which intern copy it to the third datanode. Once it creates the replicas of blocks, it sends back the acknowledgment. We can understand with the help of below cartoon diagram. Thanks All.
Post a Comment
Read more