Updating instance metadata to IMDSv2

CDP can use IMDSv2 or IMDSv1 for accessing EC2 instance metadata from a running instance.

CDP currently uses IMDSv2 for accessing EC2 instance metadata on all newly created Data Lakes, FreeIPA clusters, and Data Hubs, as long as an IMDSv2-compatible image is used. Prior to CDP supporting IMDSv2, Data Lakes, FreeIPA clusters, and Data Hubs used IMDSv1; These clusters created with IMDSv1 can now be updated to IMDSv2 as long as an IMDSv2-compatible image was used to create the cluster.

Update an existing cluster to IMDSv2

You can update an existing Data Lake, FreeIPA cluster and Data Hub that is currently using IMDSv1 to IMDSv2. This is a zero downtime operation and does not disrupt any existing processes or jobs

Prerequisites

The following prerequisites needs to be met:

Required roles: EnvironmentAdmin or Owner of the environment

Steps

  1. In the Management Console, navigate to Data Lake, FreeIPA or Data Hub details.

  2. Navigate to Nodes.

  3. The option to Update to IMDSv2 is available as follows:
  4. The update should happen within a few seconds. You can track the status in event history.

If you would like to verify that the update happened correctly, navigate to one of the EC2 instances in your AWS console.

Once the update is complete, the Update to IMDSv2 button is grayed out.

Steps CLI

Use the following commands to update a cluster from IMDSv1 ro IMDSv2:

Data Lake:

cdp datalake update-to-aws-imds-v2 --crn <SPECIFY_CRN>

FreeIPA:

cdp environments update-freeipa-to-aws-imds-v2 --environment-crn <SPECIFY_CRN>

Data Hub:

cdp datahub update-to-aws-imds-v2 --crn <SPECIFY_CRN>

The update should happen within a few seconds.

Checking if cluster image is compatible with IMDSv2

To check if the image is compatible, follow these steps:

Steps

  1. In the Management Console, navigate to your cluster.

  2. Navigate to the Image details tab.

  3. Click on the image ID.

  4. Scroll down to Package Versions details of the image:
  5. Package versions include a variable called “imds”. Find this variable and ensure that its value is “v2

  1. You can use the describe CDP CLI command of the given cluster (FreeIPA, Data Lake, Data Hub. The commands are as follows:

  2. The response should contain image and image catalog JSON details.

    Here is an example from image catalog where an image is compatible (note the "package-versions" section, which includes “imds” version. If the value is “v2”, the image is compatible. For example, see the highlighted “imds” section in the following image catalog file:
    {
            "created": 1709942743,
            "published": 1709951597,
            "date": "2024-03-09",
            "description": "Official Cloudbreak image",
            "images": {
              "aws": {
                "eu-central-1": "ami-0faa3e25092764091",
                "us-west-1": "ami-043b94655fb95edf1",
                "us-west-2": "ami-0f9ed7da8f775d00e"
              }
            },
            "os": "redhat8",
            "os_type": "redhat8",
            "uuid": "9b84a914-a15a-4856-94a9-3cfda722b0b8",
            "package-versions": {
              "blackbox-exporter": "0.19.0",
              "cdp-logging-agent": "1.3.2_b1",
              "cdp-minifi-agent": "1.22.07",
              "cdp-prometheus": "2.36.2",
              "cdp-request-signer": "0.2.4",
              "cdp-telemetry": "1.3.2_b1",
              "cloudbreak_images": "8cf7cd165b58343011091c4908f9f796d4dceb92",
           "imds": "v2",
              "inverting-proxy-agent": "3.0.7-b1",
              "inverting-proxy-agent_gbn": "41924420",
              "java": "8",
              "java11": "11.0.21",
              "java17": "17.0.9",
              "java21": "21.0.2",
              "java8": "1.8.0_392",
              "metering_agent": "2.0.0",
              "node-exporter": "1.3.1",
              "psql": "14",
              "psql11": "11.22",
              "psql14": "14.11",
              "python36": "3.6.8-38.module+el8.5.0+12207+5c5719bc",
              "python38": "3.8.16-1.module+el8.8.0+18967+20d359ae.1",
              "python39": "3.9.16-1.module+el8.8.0+20025+f2100191.2",
              "salt": "3001.8",
              "salt-bootstrap": "0.13.6-2022-05-20T08:57:17",
              "source-image": "ami-039ce2eddc1949546"
            },
            "tags": {
              "fips-mode": "disabled",
              "hardening": "cis_server_l1"
            }
          },