标签： AWS

By default, accessing S3 resources from any instance or Kubernetes pod within a VPC involves outbound traffic via NAT or IGW. Not only this is less efficient, it also incurs a service fee due to the traffic. The cost can be significant if traffic is huge.

Setup

To keep the traffic within VPC, an S3 accesspoint for the specific S3 resources and a VPC endpoint can be created by following the general instruction.

After that, double check the policies at two places.

1. Policy for IAM

Go to AWS Console → IAM → User. It is supposed to be a service account that has following policy attached

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:ReplicateObject",
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": "*"
        }
    ]
}

As a security best practice, DO NOT grant Allow to all actions, i.e. "Action": ["*"].

2. Policy for the endpoint

Go to AWS Console → VPC → Endpoints → the endpoint → Policy

{
    "Version": "2012-10-17",
    "Id": "Policy1637977229005",
    "Statement": [
        {
            "Sid": "Stmt1637977226759",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": "*"
        }
    ]
}

This endpoint policy is less constrained than the policy for user, because want the policy for user to be more specific.

After all, run traceroute s3.ap-southeast-1.amazonaws.com from any of the instances within the VPC to verify.

• Following output shows it is successful

$ traceroute s3.ap-southeast-1.amazonaws.com
traceroute to s3.ap-southeast-1.amazonaws.com (52.219.129.42), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
    ...
28  * * *
29  * * *
30  * * *

• Following output shows it is still going thru NAT/IGW

$ traceroute s3.ap-southeast-1.amazonaws.com
traceroute to s3.ap-southeast-1.amazonaws.com (52.219.40.74), 30 hops max, 60 byte packets
 1  ec2-175-41-128-191.ap-southeast-1.compute.amazonaws.com (175.41.128.191)  42.290 ms ec2-175-41-128-195.ap-southeast-1.compute.amazonaws.com (175.41.128.195)  3.103 ms ec2-18-141-171-23.ap-southeast-1.compute.amazonaws.com (18.141.171.23)  36.088 ms
 2  100.65.32.176 (100.65.32.176)  1.149 ms 100.65.32.160 (100.65.32.160)  1.132 ms 100.65.33.144 (100.65.33.144)  4.671 ms
 3  100.66.16.248 (100.66.16.248)  3.400 ms 100.66.16.26 (100.66.16.26)  8.787 ms 100.66.16.244 (100.66.16.244)  4.695 ms
 4  100.66.19.122 (100.66.19.122)  3.511 ms 100.66.19.204 (100.66.19.204)  17.491 ms 100.66.18.104 (100.66.18.104)  18.193 ms
 5  100.66.3.241 (100.66.3.241)  15.377 ms 100.66.3.137 (100.66.3.137)  154.101 ms 100.66.3.61 (100.66.3.61)  19.897 ms
 6  100.66.0.135 (100.66.0.135)  10.907 ms 100.66.0.165 (100.66.0.165)  9.064 ms 100.66.0.201 (100.66.0.201)  11.826 ms
 7  100.65.2.41 (100.65.2.41)  3.090 ms 100.65.3.41 (100.65.3.41)  2.821 ms 100.65.2.41 (100.65.2.41)  2.648 ms
 8  s3-ap-southeast-1.amazonaws.com (52.219.40.74)  0.518 ms  0.569 ms  0.578 ms

Follow the troubleshooting steps if anything does not work.

Usage

Now we have both S3 access point and the VPC endpoint for S3 setup.

To access the S3 resource within VPC, instead of using s3://<bucket name> directly, use

s3://arn:aws:s3:ap-southeast-1:<account number>:accesspoint/<bucket name>

Some 3rd party library built for AWS S3 might not recognize the S3 access point URL and throw exceptions like

Caused by: java.lang.NullPointerException: null uri host.
        at java.util.Objects.requireNonNull(Objects.java:228)
        at org.apache.hadoop.fs.s3native.S3xLoginHelper.buildFSURI(S3xLoginHelper.java:71)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.setUri(S3AFileSystem.java:486)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:246)
        at org.apache.flink.fs.s3.common.AbstractS3FileSystemFactory.create(AbstractS3FileSystemFactory.java:123)
        ... 24 more

In that case, S3 access point alias could be used. It can be found under S3 Bucket → Access Points. It looks something like

s3://<bucket name>-1ks47nsk5hyxi845kebsen1nyf11caps1b-s3alias