Wednesday, July 19, 2023

Snowflake on AWS - Quick Summary

 Snowflake - an SQL Compatible database:

  • Originally redeveloped on AWS, 2012-2014
  • Added other cloud providers
  • AWS's customer AND partner
  • Designed to be a cloud data warehouse - not a data lake
  • Not aimed an analytics use - doesn't have pipeline features
  • Lives on S2, Azure Blob storage
  • Virtual warehouse - doesn’t mean a w/house, just compute
  • Its own meta data store

Consists of:

  • Database Layer - Lives on S2, Azure Blob storage
  • Query Processing Layer - a virtual warehouse - doesn’t mean a w/house, just compute
  • Cloud Services Layer - its own meta data store


Key features:

  • Multi-tenant - all customers live in the snowflake VPC
  • Not available in EVERY region for EVERY cloud provider
  • Data resides in .fdn format - proprietary, columnar, micro-partitions
  • Any data loaded - get converted into this format; gets split into columnar micro-partitions
  • This is done for query optimization purposes - to scan on related portions and get results faster
  • Multiple Editions: Standard, Enterprise, Business Critical, Virtual Private Snowflake
  • Higher level editions include all feature of the lower
  • Pay As You go on demand - credits; there are discounts for pay-upfront
  • Storage - temp tables, transient, permanent - incur cost
  • Credits are T-Shirt sized according to the size of DB - M, XL, etc.

No comments:

Post a Comment