This topic describes how to identify the Azure locations of Impala logs for the different Impala components.
The Cloudera Data Warehouse service collects logs from Impala Virtual Warehouses and
uploads them to the Azure storage account that was provided while registering the
Environment. This ABFS log location is configured under an external warehouse
directory so that the logs are preserved even if the Virtual Warehouse they are
collected from is destroyed.
To identify the location of the logs in Azure, you must have the environment_ID,
database_catalog_ID, and impala_ID identifiers and to access the logs from the Azure
Portal you must know your storage account name.
Finding the environment_ID, database_catalog_ID, and impala_ID
identifiers
In the Data Warehouse service, expand the Environments column by
clicking More….
From the Overview page, note down the environment_ID, database_catalog_ID,
and impala_ID identifiers.
Retrieving your storage account name
In the Management Console navigate to the
Environments page.
On the Environments page, click on your Environment and
click on the Summary tab.
Scroll down to the Logs Storage and Audits section.
Note down your storage account name.
Accessing the different directories in the Azure Portal
Log in to the Azure Portal and search for your storage
account name using the Search bar.
On the Overview page of your storage account, click on
the Containers menu.
Click on the file system you used during the Environment registration.
Log locations in ABFS
Use the environment_ID,
database_catalog_ID, and impala_ID identifiers, in the following prefix to find
the logs generated by specific components in the following directories. Use the different directories listed here to view Impala/Hue logs
Impala query profiles are written in thrift encoded format in this location:
Impala component
S3 directory location
Impala query profiles
PREFIX +
“app=impala-profiles”
Use the binary tool to decode thrift to text. This binary tool is provided
with the upstream runtime Impala 4.0 as a docker image. Run the following
command to use this tool.
docker run -i apache/impala:4.0.0-impala_profile_tool < name of the thrift encoded file to decode
You can use the docker image available here to use this decoding tool.