Amazon Web Services

or What The Heck Is A Cloud Anyway?

Tristan Seligmann

Slipgate

Evolution of the Web

• Started as a medium for publishing information to humans
• Applicable to automated systems too
• Screen scraping
• Web services

Web services for everything

• Google APIs for search
• Yahoo pipes
• Google Reader API
  • Feedly

Scalable Computing

• Need to scale to hundreds, thousands, or millions of X
  • Users
  • Operations
  • Documents
  • ...
• Options:
  • Do it yourself
  • Have someone else do it for you

Computing Services

• Colocation hosting
• Managed hosting
• Virtual hosting
• "Outsourcing" (Web services)

Amazon Web Services: Lightweight

• Amazon Associates Web Service (formerly E-Commerce Service, or ECS)
• Alexa Web Search
• Alexa Web Information Service
• Alexa Top Sites
• Alexa Site Thumbnail

Amazon Web Services: Getting Fancy

• Amazon Mechanical Turk
• Amazon Fulfillment Web Service
• Amazon Flexible Payments Service

Amazon Web Services: Infrastructure

• Amazon Simple Storage Service (S3)
• Amazon Simple Queue Service (SQS)
• Amazon Elastic Compute Cloud (EC2)
• Amazon SimpleDB
• Amazon DevPay

Amazon Web Services: S3 features

• Storage of binary blobs
• Buckets (global namespace)
• Eventual consistency
• Semi-arbitrary key/value metadata
• Content-Type, Content-MD5, etc. from HTTP

Amazon Web Services: S3 features (cont.)

• Content ranges
• BitTorrent
• Access control by Amazon accounts
• Virtual hosting: bucket name
• Location constraint / S3 Europe
• REST and SOAP APIs

Amazon Web Services: S3 pricing

• $0.15 per GB-Month of storage used
• $0.100 per GB - all data transfer in
• $0.170 per GB - first 10 TB / month data transfer out
• $0.130 per GB - next 40 TB / month data transfer out
• $0.110 per GB - next 100 TB / month data transfer out
• $0.100 per GB - data transfer out / month over 150 TB
• Higher rates for S3 Europe

Amazon Web Services: SQS features

• Simple queues of small messages: up to 8kB
• Messages are binary blobs
• Lock message while processing, then delete
• "Unlimited" consumers and messages
• Not really a queue
• "Query" and SOAP APIs

Amazon Web Services: SQS pricing

• $0.01 per 10,000 Amazon SQS Requests ($0.000001 per Request)
• Data transfer rates same as S3

Amazon Web Services: EC2 features

• The big one!
• Xen on demand
• Build a disk image ("AMI") with your system and store it in S3
• Can also build an AMI by modifying an existing system and bundling
• Provide metadata when launching instance: SSH keys + user data
• Firewalling / security groups
• Commodity hardware
• EC2 Compute Unit (ECU) - 1.0-1.2 GHz 2007 Opteron or Xeon
• SOAP API

Amazon Web Services: EC2 instance types: Small

• 1 EC2 Compute unit (1 virtual core x 1 ECU)
• 1.7 GB memory
• 160 GB instance storage (150 GB + 10 GB root)
• "Moderate" I/O performance
• 32-bit
• $0.10 / instance-hour

Amazon Web Services: EC2 instance types: Large

• 4 EC2 Compute Units (2 virtual cores x 2 ECUs)
• 7.5 GB memory
• 850 GB instance storage (2 x 420 GB + 10 GB root)
• "High" I/O performance
• 64-bit
• $0.40 / instance-hour

Amazon Web Services: EC2 instance types: Extra Large

• 8 EC2 Compute Units (4 virtual cores x 2 ECUs)
• 15 GB memory
• 1690 GB instance storage (4 x 420 GB + 10 GB root)
• "High" I/O performance
• 64-bit
• $0.80 / instance-hour

Amazon Web Services: EC2 instance types: High-CPU Medium

• 5 EC2 Compute Units (2 virtual cores x 2.5 ECUs)
• 1.7 GB memory
• 350 GB instance storage (340 GB + 10 GB root)
• "Moderate" I/O performance
• 32-bit
• $0.20 / instance-hour

Amazon Web Services: EC2 instance types: High-CPU Extra Large

• 20 EC2 Compute Units (8 virtual cores x 2.5 ECUs)
• 7 GB memory
• 1690 GB instance storage (4 x 420 GB + 10 GB root)
• "High" I/O performance
• 64-bit
• $0.80 / instance-hour

Amazon Web Services: EC2 features (cont.)

• Each instance has dynamic public and private address
• Availability zones
• Elastic IP addresses (public)

Amazon Web Services: EC2 pricing

• Free data transfer with private address in zone
• Free data transfer to S3 (US), SQS, and SimpleDB
• Everything else same as S3
• $0.01 / GB in + out between zones
• No cost for Elastic IP addresses while in use
• $0.01 per non-attached Elastic IP address per complete hour
• $0.00 per Elastic IP address remap - first 100 remaps / month
• $0.10 per Elastic IP address remap - additional remap / month over 100
• Running a small instance for a month ~ $70

Amazon Web Services: SimpleDB features

• Schemaless database
• Automatic indexing
• *NOT* SQL
• Queries restricted to domains
• Domains contain items
• Items are attribute-value pairs
• Multiple values per attribute
• Attributes vary across items

Amazon Web Services: SimpleDB features (cont.)

• Max 256 attribute values
• Attribute values limited to 1024 bytes
• Query operators: =, !=, <, >, <=, >=
• STARTS-WITH, AND, OR, NOT, INTERSECTION, UNION
• Query execution limited to five seconds

Amazon Web Services: SimpleDB pricing

• Data transfer same as S3
• $0.14 per Amazon SimpleDB Machine Hour consumed
• "normalized to the hourly capacity of a circa 2007 1.7 GHz Xeon processor"
• $1.50 per GB-month
• Raw data size + 45 bytes per item, attribute name, and attribute-value pair

Amazon Web Services: DevPay

• Built on Amazon FPS
• Bill customers for AWS usage
• Add your profit margin on top of base cost
• Also allows negative pricing: you pay the difference

Amazon Web Services: EC2/SQS demo