S3 - Cross-region bucket replication

CRR - Cross-region bucket replication
-         https://docs.aws.amazon.com/AmazonS3/latest/dev/replication.html

-         Replication enables automatic, asynchronous copying of objects across Amazon S3 buckets. Buckets that are configured for object replication can be owned by the same AWS account or by different accounts. You can copy objects between different AWS Regions or within the same Region.

-         Automatic, asynchronous. Mostly for compliance or low-latency access
-         1 bucket source -> 1 destination
-         Requirements for replication – objects:
          §  must be in different regions
          §  must have versioning enabled
          §  S3 must have IAM role to read from source / write into destination
-         Objects share metadata: creation time, version id, ACL, Storage Class, user-defined metadata
-         Can define object prefixes
-         Can change storage class @ the destination
-         SSL in transit
-         Transfer is internet-based
-         Can do cross-account – need policy on destination bucker to permit writes by source
-         Replication trigger: object Create, Delete, Update Metadata on Source
-         Once replication turned on – only new or updated objects are copied, not existing untouched. Can use Copy API to copy existing
-         Only objects owner has access to the replicated objects
-         Only SSE-S3 (S3 encrypted) objects are replicated, not SSE-C or SSE-KMS – keys not present at destination, can’t decrypt
-         DELETEs:
          §  Delete on source w/out specifying Version (add Delete Marker) – replicate
          §  Delete on a specific version on Source – not replicated

-         What does NOT get replicated:
          §  Objects that existed before replication was enabled (use Copy API)
          §  SSC-KMS and SSC-C encrypted objects
          §  Objects w owner different that the bucket owner AND bucket owner has no access permissions for
          §  Updates to bucket-level sub-resources
          §  Life-cycle actions - only user actions replicated
          §  Objects that are copied in as part of another cross-region replication are not re-replicated. A->B!->C

-         Pay for:
          §  Request for replication
          §  Data transfer
          §  Storage at destination

SRR – Same Region Replication
-         Same just in one region

CORS – Cross-Origin Resource Sharing
-         https://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html

-         Cross-origin resource sharing (CORS) defines a way for client web applications that are loaded in one domain to interact with resources in a different domain. With CORS support, you can build rich client-side web applications with Amazon S3 and selectively allow cross-origin access to your Amazon S3 resources

-         Web-application can request resource from another web application

Transfer acceleration
-         For uploads over distance
-         Drop files into the nearest CloudFront access point
-         If AWS achieves transfer speed better than the internet – charge is applied. No guarantees.
-         Gest enabled on a bucket. Take up to 30 min to kick in.
-         Feature not HIPPA compliant
-         Buckets shouldn’t have periods in names
-         Multi-part upload available
-         Speed comparison tool shows – accelerated vs anon-accelerated uploads

-         https://s3-accelerate-speedtest.s3-accelerate.amazonaws.com/en/accelerate-speed-comparsion.html

 
Best Practices
-         Caching for frequently needed data – ElasticCache, CloudFront – single digit ms latencies
-         Build-in retry routines for latency sensitive apps
-         Horizontal scaling and Request Parallelization for High Throughput loads (process multiple PUT/GET’s in parallel)
-         Transfer acceleration for multiregional data transfers
-         Use Byte-range fetches – higher throughput due to parallel operations; in case of failure – re-download only a sub-set of the ranges
-         AWS KMS has limits for the number of encrypt/decrypt requests. Can be a bottleneck.

Event Notification

-         https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
-         SNS (email), SQS, AWS Lambda

-         Create, Delete Object. Delete Many

S3 Fees
-         Storage Gb/Month
-         Data transfer to/from – only across regions
-         Upload requests – both PUT and GET’s
-         S3-IA and Glacier – retrieval
-         Requester pays:
          §  Owner pays for storage only
          §  Can’t be used for anonymous requests and BitTorrent
          §  At bucket level only – not object
-         CloudWatch can monitor S3 metrics (one at a time): S3 Requests, S3 Storage, All Requests, etc.
-         Can user CW filters (CloudDimentions) to filter by bucket name, tag, prefix, etc.
-         Daily CW bucket-level CW metrics on by default - no cost. Can configure 1 min metrics – charge.

-         CloudTrail logs bucket-level API-requests by default. Can configure per-object (DELETE, PUT, GET, etc.)

-         Create, Delete Object. Delete Many