HDFS Commands
- jps
HDFS Command to print Hadoop
processes.
[root@quickstart Desktop]# jps
- fsck
HDFS Command to check the health of
the Hadoop file system.
[cloudera@quickstart
training]$ hdfs fsck /
·
ls
HDFS Command to display the list of
Files and Directories in HDFS.
[cloudera@quickstart
training]$ hdfs dfs -ls /
- mkdir
HDFS Command to create the directory
in HDFS.
[cloudera@quickstart
training]$ hdfs dfs -mkdir /bigdatatesting
[cloudera@quickstart
training]$ hdfs dfs -ls /
drwxr-xr-x - cloudera supergroup 0 2018-05-23 00:46 /bigdatatesting
Note: Here we are trying to create a directory
named “Bigdatatesting” in HDFS.
- touchz
HDFS Command to create a file in
HDFS with file size 0 bytes.
[cloudera@quickstart
training]$ hdfs dfs -touchz /bigdatatesting/test.dat
[cloudera@quickstart
training]$ hdfs dfs -ls /bigdatatesting/
Found 1 items
-rw-r--r-- 1 cloudera supergroup 0 2018-05-23 00:48
/bigdatatesting/test.tx
Note: Here we are trying to create a file named “test.dat” in the directory “bigdatatesting” of hdfs with file size 0 bytes.
- du
HDFS Command to check the file
size.
[cloudera@quickstart
training]$ hdfs dfs -du -s /bigdatatesting/test.dat
0
0 /bigdatatesting/test.dat
·
appendToFile
Appends the contents to the given
destination file on HDFS. The destination file will be created if it does not
exist.
[cloudera@quickstart
training]$ hdfs dfs -appendToFile - /bigdatatesting/test.dat
- cat
HDFS Command that reads a file
on HDFS and prints the content of that file to the standard output.
[cloudera@quickstart training]$ hdfs dfs -cat
/bigdatatesting/test.dat
- copyFromLocal
HDFS Command to copy the file from a
Local file system to HDFS.
Step1:
Create a file in Local File System.
[cloudera@quickstart
training]$ cat>> test1.dat
[cloudera@quickstart
training]$ ls test1.dat
test1.dat
Step2:
Copy file from Local File system to HDFS
[cloudera@quickstart
training]$ hdfs dfs -copyFromLocal test1.dat /bigdatatesting/
Note: Here the test is the file present in the local
directory /home/cloudera/training and after the command gets executed the test
file will be copied in /bigdatatesting directory of HDFS.
- copyToLocal
HDFS Command to copy the file from
HDFS to Local File System.
Step1:
Check test.dat file present in local file system.
[cloudera@quickstart
training]$ ls test.dat
ls: cannot access test.dat: No such
file or directory
Step2:
Copy test.dat file from HDFS to local file system.
[cloudera@quickstart
training]$ hdfs dfs -copyToLocal /bigdatatesting/test.dat
/home/cloudera/training
Step3:
Check again test.dat file present in local file system.
[cloudera@quickstart
training]$ ls test.dat
test.dat
Note: Here test.dat is a file present in the bigdatatesting
directory of HDFS and after the command gets executed the test.dat file will be
copied to local directory /home/Cloudera/training
- put
HDFS Command to copy single source
or multiple sources from local file system to the destination file system.
Step1:
Create a file in Local File System.
[cloudera@quickstart
training]$ cat>> test2.dat
[cloudera@quickstart
training]$ ls test2.dat
test1.dat
Step2:
Copy file from Local File system to HDFS
[cloudera@quickstart
training]$ hdfs dfs -put test2.dat /bigdatatesting/
Note: Here the test2.dat is the file present in the local
directory /home/cloudera/training and after the command gets executed the
test2.dat file will be copied in /bigdatatesting directory of HDFS.
Note: The command put is similar to copyFromLocal
command.
- · get
HDFS Command to copy files from hdfs
to the local file system.
Step1:
Create a new file test3.dat on HDFS.
[cloudera@quickstart
training]$ hdfs dfs -touchz /bigdatatesting/test3.dat
Step2:
Copy test3.dat file from HDFS to local file system.
[cloudera@quickstart
training]$ hdfs dfs -get /bigdatatesting/test3.dat /home/cloudera/training
Step3:
Check again test3.dat file present in local file system.
[cloudera@quickstart
training]$ ls test3.dat
Test3.dat
Note1: Here test3.dat is a file present in the
bigdatatesting directory of HDFS and after the command gets executed the test.dat
file will be copied to local directory /home/Cloudera/training
Note2: The command get is similar to copyToLocal command
- cp
HDFS Command to copy files from
source to destination. This command allows multiple sources as well, in which
case the destination must be a directory.
[cloudera@quickstart
training]$ hdfs dfs -mkdir /hadooptesting/
[cloudera@quickstart
training]$ hdfs dfs -cp /bigdatatesting/test.dat /hadooptesting
- mv
HDFS Command to move files from
source to destination. This command allows multiple sources as well, in which
case the destination needs to be a directory.
[cloudera@quickstart
training]$ hdfs dfs -mv /bigdatatesting/test1.dat /hadooptesting/
- rm
HDFS Command to remove the file from
HDFS.
[cloudera@quickstart
training]$ hdfs dfs -rm /bigdatatesting/test2.dat
Deleted /bigdatatesting/test2.dat
- rm -r
HDFS Command to remove the entire
directory and all of its content from HDFS.
[cloudera@quickstart
training]$ hdfs dfs -rm -r /hadooptesting
Deleted /hadooptesting
- rmdir
HDFS Command to remove the directory
if it is empty.
[cloudera@quickstart
training]$ hdfs dfs -rmdir /bigdatatesting
- usage
HDFS Command that returns the help
for an individual command.
[cloudera@quickstart
training]$ hdfs dfs -usage mkdir
Note: By using usage command you can get
information about any command.
- help
HDFS Command that displays help for
given command or all commands if none is specified.
[cloudera@quickstart
training]$ hdfs dfs -help
