Pathnames on clusters with federated and non-federated NameNodes
The pathnames of the namespace URIs used in clusters with federated and non-federated NameNodes depend on the value configured for the default file system.
In cluster without federated NameNodes, the default file system is set to the NameNode of
the cluster. For example, fs.defaultFS= hdfs://nnOfThatCluster:port/
.
In a federation with ViewFs, the default file system for each cluster is set to the name
of the cluster. For example, fs.defaultFS=viewfs://clusterX/
.
The value configured for the default file system determines the pathnames of the various URIs used in clusters.
The following table contains examples of pathnames in URIs used in clusters without federated NameNodes:
Pathname Example | Description | |||
---|---|---|---|---|
/foo/bar |
The relative path equivalent to
hdfs://nnOfClusterX:port/foo/bar . |
|||
hdfs://nnOfClusterX:port/foo/bar |
The fully qualified path to the foo/bar directory on
cluster X.
|
|||
hdfs://nnOfClusterY:port/foo/bar |
The URI for referring to the foo/bar directory on cluster
Y. For example, the command for copying files from one cluster to another can be
as
follows:distcp hdfs://namenodeClusterY:port/pathSrc hdfs://namenodeClusterZ:port/pathDest |
|||
hftp://nnClusterX:port/foo/bar |
The URI for accessing the foo/bar directory on cluster X
through the HFTP file system. |
|||
webhdfs://nnClusterX:port/foo/bar |
The URI for accessing the foo/bar directory on cluster X
through the WebHDFS file system. |
|||
http://namenodeClusterX:http_port/webhdfs/v1/foo/bar and
http://proxyClusterX:http_port/foo/bar |
The HTTP URLs respectively for accessing files through WebHDFS REST API and HDFS proxy. |
The following table contains examples of pathnames in URIs used in clusters with ViewFs:
Pathname Example | Description | |||
---|---|---|---|---|
/foo/bar |
The relative path equivalent to
viewfs://clusterX/foo/bar . |
|||
viewfs://clusterX/foo/bar |
The fully qualified path to the foo/bar directory on
cluster X. Unlike a non-federated environment where you must specify the
NameNode in the path to each namespace volume, you can use the mount table name
in a federated environment to qualify the paths to all the namespace volumes of
the cluster.
|
|||
viewfs://clusterY/foo/bar |
The URI for referring to the foo/bar directory on cluster
Y. For example, the command for copying files from one cluster to another can be
as
follows:distcp viewfs://clusterY/pathSrc viewfs://clusterZ/pathDest |
|||
viewfs://clusterX-hftp/foo/bar |
The URI for accessing the foo/bar directory on cluster X
through the HFTP file system. |
|||
viewfs://clusterX-webhdfs/foo/bar |
The URI for accessing the foo/bar directory on cluster X
through the WebHDFS file system. |
|||
http://namenodeClusterX:http_port/webhdfs/v1/foo/bar and
http://proxyClusterX:http_port/foo/bar |
The HTTP URLs respectively for accessing files through WebHDFS REST API and HDFS proxy. |
Note | |
---|---|
You cannot rename files or directories across NameNodes or clusters in both federated
and non-federated environments. For example, while the following renaming operation can
be performed within the scope of a NameNode, the same is not possible if the source and
destination paths belong to different NameNodes:
|