HDP uses multiple ports for communication with clients and between service components. For example, the Hive Metastore port represents the port that the metastore database can use to connect and communicate. To enable HDP communication, open the specific ports that HDP uses.
To open specific ports, you can set the access rules in Windows. For example, the following command will open up port 80 in the active Windows Firewall:
netsh advfirewall firewall add rule name=AllowRPCCommunication dir=in action=allow protocol=TCP localport=80
The following command will open up ports 49152-65535 in the active Windows Firewall:
netsh advfirewall firewall add rule name=AllowRPCCommunication dir=in action=allow protocol=TCP localport=49152-65535
The following tables specify which ports must be opened for specific ecosystem components to communicate with each other. Open the appropriate ports before you install HDP.
Table 1.3. HDFS Ports
|
Service |
Servers |
Default Ports Used |
Protocol |
Description |
Needs End-user Access? |
Configuration Parameters |
|---|---|---|---|---|---|---|
|
NameNode WebUI |
MasterNodes (NameNode and any backup NameNodes) |
50070 |
http |
WebUI to look at current status |
Yes (typically admins, Development/Support teams) |
self-addressed |
|
NameNode metadata service |
Master Nodes (NameNode and any backup NameNodes) |
8020/9000 |
IPC |
File system metadata operations |
Yes (all clients who need to interact directly with the HDFS) |
Embedded in URI specified by |
|
DataNode |
All Slave Nodes |
50076 |
http |
DataNode WebUI to access the status, logs, etc. |
Yes (typically admins, Development/Support teams) |
dfs.datanode. http.address |
|
DataNode |
All Slave Nodes |
50010 |
|
Data transfer |
|
dfs.datanode. address |
|
DataNode |
All Slave Nodes |
50020 |
IPC |
Metadata operations |
No |
dfs.datanode. address |
|
Secondary NameNode |
Secondary NameNode and any backup Secondary NameNodes |
50090 |
http |
Checkpoint for NameNode metadata |
No |
dfs.secondary. http.address |
Table 1.4. YARN Ports
|
Service |
Servers |
Default Ports Used |
Protocol |
>Description |
Needs End-user Access? |
Configuration Parameters |
|---|---|---|---|---|---|---|
|
Resource Manager WebUI |
Master Nodes (Resource Manager and any back-up Resource Manager node) |
8088 |
http |
WebUI for Resource Manager |
Yes |
yarn.resource- manager.webapp. address |
|
Resource Manager WebUI |
Master Nodes (Resource Manager and any back-up Resource Manager node) |
8090 |
https |
WebUI for Resource Manager |
Yes |
yarn.resource- manager.webapp. https.address |
|
Resource Manager Admin Interface |
Master Nodes (Resource Manager and any back-up Resource Manager node) |
8032 |
IPC |
For application submissions |
Yes (All clients who need to submit the YARN applications including Hive, Hive server, Pig) |
yarn.resource- manager.admin. address |
|
Resource Manager Scheduler |
Master Nodes (Resource Manager and any back-up Resource Manager node) |
8033 |
|
Administrative interface |
Yes (Typically admins and support teams) |
yarn.resource-manager. scheduler.address |
|
NodeManager Web UI |
All Slave Nodes |
8031 |
http |
Resource Manager interface |
Yes (Typically admins, Development/Support teams) |
yarn. nodemanager. webapp.address |
Table 1.5. Hive Ports
|
Service |
Servers |
Default Ports Used |
Protocol |
Description |
Needs End-user Access? |
Configuration Parameters |
|---|---|---|---|---|---|---|
|
HiveServer2 |
HiveServer 2 machine (usually a utility machine) |
10001 |
thrift |
Service for programmatically (Thrift/JDBC) connecting to Hive |
Yes |
ENV Variable HIVE_PORT |
|
HiveServer |
HiveServer machine (usually a utility machine) |
10000 |
thrift |
Service for programmatically (Thrift/JDBC) connecting to Hive |
Yes (Clients who need to connect to Hive either programmatically or through UI SQL tools that use JDBC) |
ENV Variable HIVE_PORT |
|
Hive Metastore |
|
9083 |
thrift |
Service for programmatically (Thrift/JDBC) connecting to Hive metadata |
Yes (Clients that run Hive, Pig and potentially M/R jobs that use HCatalog) |
hive.metastore. uris |
Table 1.6. WebHCat Ports
|
Service |
Servers |
Default Ports Used |
Protocol |
Description |
Need End User Access? |
Configuration Parameters |
|---|---|---|---|---|---|---|
|
WebHCat Server |
Any utility machine |
50111 |
http |
Web API on top of HCatalog and other Hadoop services |
Yes |
templeton.port |
Table 1.7. HBase Ports
|
Service |
Servers |
Default Ports Used |
Protocol |
Description |
Need End User Access? |
Configuration Parameters |
|---|---|---|---|---|---|---|
|
HMaster |
Master Nodes (HBase Master Node and any back- up HBase Master node) |
60000 |
|
|
Yes |
hbase.master. port |
|
HMaster Info WebUI |
Master Nodes (HBase master Node and backup HBase Master node if any) |
60010 |
http |
The port for the HBaseMaster WebUI. Set to -1 if you do not want the info server to run. |
Yes |
hbase.master. info.port |
|
Region Server |
All Slave Nodes |
60020 |
|
|
Yes (typically admins, development/support teams) |
hbase. regionserver. port |
|
ZooKeeper |
All ZooKeeper Nodes |
2888 |
|
Port used by ZooKeeper peers to talk to each other. For further information, see the Apache website. |
No |
hbase.zookeeper. peerport |
|
ZooKeeper |
All ZooKeeper Nodes |
3888 |
|
Port used by ZooKeeper for leader election. For further information, see the Apache website. |
|
hbase.zookeeper. leaderport |
|
|
|
2181 |
|
Property from ZooKeeper's configuration file,
|
|
hbase.zookeeper. property. clientPort |
