hadoop fs -du gives two data columns

Question

I ran the following command on my test hadoop instance :

hadoop fs -du /test/data/ 51179082 153537246 /test/data/9875/2016/02/03 46949272 140847816 /test/data/9875/2016/02/04

I understand du gives the size of files and directories, but for hadoop fs -du it is giving two sizes. Can some one help me understand what are these?

its is similar to the unix du , since you have 2 folders , it showed the folder specific size — K S Nidhin
– K S Nidhin, Commented Feb 24, 2016 at 23:18
I dont think so, because i now have three folders and it is giving 2 columns for each of those — Pradeep
– Pradeep, Commented Feb 25, 2016 at 22:55

chrisaycock · Accepted Answer · 2017-04-18 18:58:40Z

35

One column is size of the file and the other column is size of the file after replication.

edited Apr 18, 2017 at 18:58

chrisaycock

38.1k15 gold badges94 silver badges128 bronze badges

answered Mar 28, 2016 at 0:24

Pradeep

8702 gold badges14 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

sheldonkreger Over a year ago

which is which?

chrisaycock Over a year ago

@sheldonkreger The first is the size and the second is the space consumed, i.e. the post-replication size. So a replication of 3 would cause three times the size to be consumed in space.

user 923227 Over a year ago

I have a 1 node cluster. The cluster complains of under-replicated blocks. However the 2nd value shows nearly 3x of the 1st value!

Collectives™ on Stack Overflow

hadoop fs -du gives two data columns

1 Answer 1

3 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Related