FSx for Lustre

FSx for Lustre
-         Lustre:

-         https://docs.aws.amazon.com/fsx/latest/LustreGuide/what-is.html

-         Amazon FSx for Lustre makes it easy and cost-effective to launch and run the popular, high-performance Lustre file system. You use Lustre for workloads where speed matters, such as machine learning, high performance computing (HPC), video processing, and financial modeling.

-         Distributed parallel data storage for massive scalability, high performance, high- availability
-         Horizontally scalable IO for DC of all sizes
-         Runs on Linux based OS only
-         Can scale to interact with thousands of hosts
-         FSx for Lustre
          §  Fully managed Lustre file system
          §  High performance computing (HPC), Machine learning, Video processing
          §  Hundreds of gig / sec throughput, millions of IOPS, sub ms latency
          §  High perf storage short-term @ low cost – no replication. Meant for data needed for the duration of a test
          §  Lustre client – Linux systems, needs to run Lustre s/w. can be onsite or on AWS
          §  CloudWatch/Trail integrated
          §     S3 integrated – can read data and write back test / computation results

          §  Access via ENI – make sure to set up Sec Group
          §  Data automatically encrypted at rest w FSx-managed KMS key. No support for SSE-KMS and SSE-C

          §  Sec Group needs to allow incoming traffic on Lustre LNET port 988

-         Scratch file systems - ideal for temporary storage and shorter-term processing of data. Data is not replicated and does not persist if a file server fails

-         Persistent file systems - ideal for longer-term storage and workloads. In persistent file systems, data is replicated, and file servers are replaced if they fail

-         Source: https://docs.aws.amazon.com/fsx/latest/LustreGuide/performance.html