Low Orbit Flux Logo 2 F

HDFS How to Create a Directory

Use the dfs mkdir command to create a directory in HDFS. Here is a quick example:

hdfs dfs -mkdir hdfs://nn1/user/hadoop/dir1

If you want to create a directory and subdirectory but neither exists you can use the “-p” switch. This automatically creates the parent dir. See the following example:

hdfs dfs -mkdir -p hdfs://nn1/user/hadoop/dir1/subdir1

You can also create more than one directory if you want. This can even be done on different hosts. See this example that creates a directory on both hosts nn1 and nn2:

hdfs dfs -mkdir hdfs://nn1/user/hadoop/dir1 hdfs://nn2/user/hadoop/dir1

See Our Hadoop HDFS Commands Guide for more information on other Hadoop/HDFS commands.

HDFS How to Create a Directory Using Java Code?

You could do this using a code snippet similar to the following. Change any paths to match your system.

public void myMethod()
{ 
    Configuration config = new Configuration();
    config.addResource(new Path("/etc/hadoop/conf/core-site.xml"));
    config.addResource(new Path("/etc/hadoop/conf/hdfs-site.xml"));
    config.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
    config.set("fs.file.impl", org.apache.hadoop.fs.LocalFileSystem.class.getName());

    FileSystem dfs = FileSystem.get(config);
    String newDir1 = "myNewDirectory1";
    System.out.println("Creating new directory here: " + dfs.getWorkingDirectory() + "/" + newDir1 + "/n/n");
    Path src = new Path(dfs.getWorkingDirectory() + "/" + newDir1);
    dfs.mkdirs(src); 

} }

Other Issues

People have reported not being able to create directories due to being in safe mode. The following command should let you leave safe mode:

hdfs dfsadmin -safemode leave

References