Cloudera Docs

Examples of using the Amazon Web Services command-line interface for S3 Gateway

You can use the Amazon Web Services (AWS) command-line interface (CLI) to interact with S3 Gateway and work with various Ozone storage elements.

Defining an alias for the S3 Gateway endpoint

Defining an alias for the S3 Gateway endpoint helps you in using a simplified form of the AWS CLI. The following example shows how you can define an alias for the S3 Gateway endpoint URL:
alias ozones3api='aws s3api --endpoint http://localhost:9878'

Examples of using the AWS CLI to work with the Ozone storage elements

The following examples show how you can use the AWS CLI to perform various operations on the Ozone storage elements. All the examples specify the alias ozones3api:

Operations Examples
Creating a bucket 
ozones3api create-bucket --bucket b1

This command creates a bucket b1.
Adding objects to a bucket 
ozones3api put-object --bucket b1 --key Doc1 --body ./Doc1.md

This command adds the key Doc1 containing data from Doc1.md to the bucket b1.
Listing objects in a bucket 
ozones3api list-objects --bucket b1

This command lists the objects in the bucket b1. An example output of the command is as follows:

{
    "Contents": [
        {
            "LastModified": "2018-11-02T21:57:40.875Z", 
            "ETag": "1541195860875", 
            "StorageClass": "STANDARD", 
            "Key": "Doc1", 
            "Size": 2845
        }, 
        {
            "LastModified": "2018-11-02T22:36:23.358Z", 
            "ETag": "1541198183358", 
            "StorageClass": "STANDARD", 
            "Key": "Doc2", 
            "Size": 5615
        }, 
        {
            "LastModified": "2018-11-02T21:56:47.370Z", 
            "ETag": "1541195807370", 
            "StorageClass": "STANDARD", 
            "Key": "Doc3", 
            "Size": 1780
        }
    ]
}
Downloading an object from a bucket 
ozones3api get-object --bucket b1 --key Doc1 ./Dpc1
This command downloads the key Doc1 from the bucket b1 as a file Dpc1. An example output of the command is as follows:
{
    "ContentType": "application/octet-stream", 
    "ContentLength": 2845, 
    "Expires": "Fri, 02 Nov 2018 22:39:00 GMT", 
    "CacheControl": "no-cache", 
    "Metadata": {}
}
Verifying access to a bucket 
ozones3api head-bucket --bucket b1
This command verifies whether the bucket b1 exists and whether the current user has access to b1. If both the requirements are satisfied, the command returns no output. Otherwise, it displays an error message.
Returning object metadata 
ozones3api head-object --bucket b1 --key Doc1
This command returns the metadata of key Doc1 present in bucket b1. An example output of the command is as follows:
{
    "ContentType": "binary/octet-stream", 
    "LastModified": "Fri, 2 Nov 2018 21:57:40 GMT", 
    "ContentLength": 2845, 
    "Expires": "Fri, 02 Nov 2018 22:41:55 GMT", 
    "ETag": "1541195860875", 
    "CacheControl": "no-cache", 
    "Metadata": {}
}
Copy a key from one bucket to another 
ozones3api copy-object --bucket b2 --key Doc1 --copy-source b1/Doc1
This command copies the key Doc1 from bucket b1 to b2. The following example shows the result of a copy operation:
{
    "CopyObjectResult": {
        "LastModified": "2018-11-02T22:49:20.061Z", 
        "ETag": "21df0aee-26a9-464c-9a81-620f7cd1fc13"
    }
}
To verify whether the specified object is copied, you can run the list-object command on the destination bucket.
Deleting an object from a bucket 
ozones3api delete-object --bucket b1 --key Doc1

This command deletes the key Doc1 from bucket b1.
Deleting multiple objects from a bucket 
ozones3api delete-objects --bucket b1 --delete 'Objects=[{Key=Doc1},{Key=Doc2},{Key=Doc3}]

This command deletes the keys Doc1, Doc2, and Doc3 from bucket b1.