Securing S3 Presigned URLs for Uploads

In this post I give a break down of how to secure S3 presigned upload URLs for untrusted third parties. As a quick refresher, unsigned URLs can be generated and used by third parties to upload and download files to and from S3. In the case of an upload the S3 URL is time bound and used by the third party to add content to an S3 bucket via either a HTTP PUT or POST.

In this example we'll focus on the HTTP PUT request, for untrusted third parties, we'll mitigate the risk by adding the following additional attributes to the presigned upload URL.

1. Set the expiry to the minimum viable time for the presigned upload URL to be consumed to mitigate the upload URL from being used maliciously.
2. Set a limit on the file upload size so that a malicious user does not upload a large file to mitigate against ingress fees for the bucket owner.
3. Add a checksum to guard against man in the middle attacks where the file contents is replaced but more importantly ensure that the file is not corrupted on file upload.
4. Specify the content type of the file which will be uploaded.

This approach requires that a server handle the presigned generation request, where some validation can be performed before the presigned URL is generated. This server has access the necessary permissions to generated the presigned URL. It's assumed that the request to generate the presigned URL is from an authenticated user and that the necessary authn and authz operations have already occurred.

In this example we'll use the Java S3 SDK to generate a PUT presigned URL as example, with following key points:

• An md5 and sha256 checksum are shown as examples although just one approach should be used. It seems md5 was the original implementation and so this is handled slightly differently.
• The checksums should be base64 encoded when generating the presigned URL and when setting the value in the PUT request headers.
• The required headers are printed to the console, these must be added to the put request exactly as shown otherwise S3 will fail the upload.

 public void generatePresignedUrl() throws Exception {
        String bucketName = "MyBucketName";
        String bucketKey = "test/image.png";

        File file = new File("/Users/test/image.png");

        // Validate the file exists
        if (!file.exists()) {
            System.out.println("File doesn't exist");
            System.exit(1);
        }

        final String accessKey = ""; // Set the Access Key
        final String secret = ""; // Set the secret
        final String sessionToken = ""; // Set the session token

        // Generate the md5 checksum of this
        byte[] md5Checksum;
        byte[] sha256checksum;

        try (InputStream is1 = Files.newInputStream(file.toPath());
             InputStream is2 = Files.newInputStream(file.toPath())) {
            md5Checksum = DigestUtils.md5(is1);
            sha256checksum = DigestUtils.sha256(is2);
        }


        Base64.Encoder encoder = Base64.getEncoder();
        String md5Base64 = encoder.encodeToString(md5Checksum);
        String sha256Base64 = encoder.encodeToString(sha256checksum);
        sha256Base64 = "jzgF/RcT0c/BQbt920gQ6SqcGQZ7PMAjjUuZ1/iTr7d=";

        try (S3Presigner presigner = S3Presigner.create()) {

            final AwsCredentials creds = AwsSessionCredentials.create(accessKey, secret, sessionToken);
            final AwsCredentialsProvider credentialProvider = StaticCredentialsProvider.create(creds);
            final AwsRequestOverrideConfiguration awsOverrideConfiguration = AwsRequestOverrideConfiguration.builder().credentialsProvider(credentialProvider).build();
            PutObjectRequest objectRequest = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(bucketKey)
                    .contentLength(file.length())
                    .contentType("image/png")
                    .checksumAlgorithm(ChecksumAlgorithm.SHA256)
                    .checksumSHA256(sha256Base64)
                    .contentMD5(md5Base64)
                    .overrideConfiguration(awsOverrideConfiguration)
                    .build();


            PutObjectPresignRequest presignRequest = PutObjectPresignRequest.builder()
                            .signatureDuration(Duration.ofMinutes(10))
                            .putObjectRequest(objectRequest)
                            .build();

            PresignedPutObjectRequest presignedRequest = presigner.presignPutObject(presignRequest);

            System.out.printf("Presigned URL:\n%s%n", presignedRequest.url());
            System.out.printf("HTTP method: [%s]%n", presignedRequest.httpRequest().method());
            System.out.println("REQUIRED HEADERS:");
            presignedRequest.httpRequest().headers().forEach((name, values) ->
                    System.out.println(name + ": " + String.join(",", values)));
        }

    }

Once the URL is generated we need to create a PUT request using a cURL request:

curl --location --request PUT '<<PRESIGNED_URL>>' \
--header 'Content-MD5: RBNM8DiLPc4+8JyDuJGzgg==' \
--header 'x-amz-checksum-sha256: jzgF/RcT0c/BQbt920gQ6SqcGQZ7PMAjjUuZ1/iTr7d=' \
--header 'x-amz-sdk-checksum-algorithm: SHA256' \
--header 'Content-Type: image/png' \
--data-binary '@/Users/james.johnson/james.png'

In the case of cURL the content size is sent automatically. Note the difference in the md5 and sha256 checksum HTTP headers.

Caveats

• The bucket name is exposed and is visible in the presigned URL.
• Validation of the content type is not handled, on the validation that the PUT request header matches the signed URL, the magic header bytes of the file should be validated to ensure that file is the content type.
• Interestingly in this example if the md5 or the SHA256 checksums do not match the PUT request fails, hence if more than one checksum is provided they are always validated