AWS Fundamentals

Little disclaimer, this is not an introductory blog post or meant to be a guide. These are my notes, raw, and may continue to be updated. I have included video links to AWS re:Invent Videos. Though these are not the ones I used to study, I went through the pluralsight Amazon Web Services (AWS) Fundamentals for Systems Administrators course as a basis. The intention is to work my way up to the Solutions Architect Professional certification.

Maybe this is useful to you, if not oh well, tough. 😎

At the end you’ll find links to the actual markdown file with with notes, as the iThoughtsX Mindmap and OPML versions.

AWS SYSOPS FUNDAMENTALS

CERTIFICATION TRACKS

  • Associate
    • Solutions Architect
    • Developer
    • SysOps Administrator
  • Professional
    • Solutions Architect
    • DevOps Engineer
    • Combines SysOps and Developer

WHAT IS CLOUD COMPUTING

The NIST definition of Cloud Computing.

A model for enabling ubiquitous, convenient, on-demand network access to shared pool of configurable resources (e.g., networks, servers, storage, applications, and services) that-can be rapidly provisioned and released with minimal management effort or service provider interaction.

ESSENTIAL CHARACTERISTICS

  • On-demand
  • Broad Network access
  • Resource Pooling
  • Rapid Elasticity
  • Measured service

SERVICE MODELS

  • IaaS - Infrastructure-as-a-Service
    • Server, network and security abstractions
    • On-demand and elastic
  • PaaS - Platform-as-a-Service
    • Code based against specific platforms not infrastructure
    • Elastic
    • Not servers
    • A.K.A - .NET as a service
  • SaaS - Software-as-a-Service
    • Based on leveraging applications
    • Sample is Salesforce

DEPLOYMENT MODELS

  • Private
    • On-premises only
  • Hybrid
    • On-premises and Public Cloud
    • Ability to stretch services across private and public infrastructures
  • Community
    • Institution based cloud
    • Verticalized cloud infrastructure
    • Financial
    • Government
  • Public
    • Resource pooling at a public service
    • AWS / AZURE / GOOGLE CLOUD …

INTRODUCTION TO AWS

ON-PREMISES COMPONENTS

  • Security
  • Cooling
  • Connectivity
  • Networking Hardware
  • Servers
  • Cabling
  • Storage
  • Expertise
  • Facilities
  • More…

UNDERSTANDING IaaS

  • Components
    • Self-service portal
    • Service catalog
    • Management layer
    • capacity management
    • change management
    • life-cycle management
    • charge-back
    • policy-based resource allocation
    • performance optimizations
    • Virtual Infrastructure
    • Physical Infrastructure
    • Identity (Access Management)
    • Centralized management
    • Integrated and Operation
    • Public Cloud Connector
    • Orchestration
    • Infrastructure Authority

UNDERSTANDING AWS GLOBAL INFRASTRUCTURE

The AWS global infrastructure is broken out into regions and availability zones. These are both geographical components.

REGION

  • A region is a:
    • Geographical area where a data center exist

An example of a region would be US-EAST, this in turn has multiple zones of availability.

AVAILABILITY ZONE

  • Area within a region where a data center exists
    • EXAMPLE
      • US-EAST REGION
        • N. Virginia DC
        • Ohio DC
    • Not all services may be available in all regions or availability zones
    • 10 Regions available + GovCloud

You can find AWS Global Infrastructure here and take a deeper look at services and where they’re available.

EDGE LOCATIONS

  • Complementary services
  • Where infrastructure cannot be configured
    • These are things such as
      • Route 53
      • CloudFront

UNDERSTANDING AWS SECURITY MEASURES

  • PHYSICAL ACCESS
    • Locations are not advertised
    • Controlled physical access
    • Best in class datacenter security
    • multifactor (thumbprint, retina scanners)
    • video surveillance
  • SERVERS AND NETWORK INFRASTRUCTURE

SECURITY CERTIFICATIONS AND COMPLIANCE

  • HIPPA
    • SOCK1/SSAE 16/ISAE 3402
    • SOC 2
    • SOC 3
    • PCI DSS LEVEL 1
    • ISO 27001
    • FedRAMP(SM)
    • DIACAP and FISMA
    • ITAR
    • FIPS 140-2
    • CSA
    • MPAA

SHARED SECURITY RESPONSIBILITY

  • AWS RESPONSIBILITY
    • virtual host security
    • storage security
    • network security
    • data center security
    • database security
    • OUR RESPONSIBILITY
      • AWS account security (MFA, API)
      • operating systems
      • database
      • applications
      • data encryption
      • authentication
      • network integrity

SECURITY METHODS AND CONNECTIVITY

  • Security
    • Security Groups
    • Virtual Private clouds virtual networking
    • Direct Connect
    • Connectivity
      • Import / Export
      • Physically (drive shipping, array delivery)
      • VPN Access
      • Dedicated Server
    • Identity and Access Management (IAM)
      • User and service management
      • Controls access to AWS resources
      • Multi-factor authentication
      • API access (keys)

THE AWS FREE TIER

  • Easy and fast sign up
  • Excellent for training on concepts
  • Allows limited workloads and resources
  • Offers basic support and access to resources
  • Only credit card and phone number is needed
  • A monthly recurring program
  • Some services are only allowed for 12 months (EC2, ELB, EBS, S3)
  • DETAILS

WORKING WITH AWS STORAGE: EPHEMERAL AND S3

EPHEMERAL STORAGE

  • Understanding Ephemeral Storage
    • Instance store
    • Storage on instance (e.g,. drive on EC2 instance)
    • Temporary block-level storage

AWS SIMPLE STORAGE SERVICE (S3)

  • Amazon Simple Storage Service (S3)
    • First AWS service introduced in 2006
    • Internet accessible storage via HTTP/HTTPS
    • Audio, video, images, backup etc…
    • Unlimited bucket size
    • UP to 5 TB object size
    • Priced on storage used and transfer out
    • It’s not a file system
    • Two types of S3 Storage
    • Standard Storage
      • 99.99999999% durability
      • 99.99% uptime
      • first 1TB $0.0300 / GB
    • Reduced Redundancy Storage (RRS)
      • First 1TB $0.0240 / GB
      • Reduced durability - 99.99%
    • Granular storage type selection
    • Replication
      • S3 stores data in multiple facilities on multiple devices within each facility
      • S3 RRS (Reduced Redundancy Storage) Amazon S3 does not replicate objects as many times as standard S3
      • Synchronously stores data across multiple facilities before confirming that the data has been successfully stored
      • Calculates checksums on all network traffic to detect corruption of data packets
    • Features
      • Versioning capable
      • Cross-region replication (CRR)
      • MFA delete (via API)
      • Time-limited access to objects
    • Security
      • IAM policies
        • User-level security
        • Granular security configuration
      • Bucket policies
        • Bucket-level security
      • ACLs
        • Legacy access control mechanism
        • Bucket and object-level security
      • Query string authentication (Presigned URLs)
        • Grant temporary access to your Amazon S3 resources
    • PRICING

S3 VIDEO

WORKING WITH AWS STORAGE: EBS AND GLACIER

UNDERSTANDING ELASTIC BLOCK STORAGE (EBS)

Think of EBS as a drive you attach to a virtual machine or server. In this case, it’s attached to an EC2 instance and can only be accessible by that instance.

  • Not internet accessible
  • Persistent file system for EC2
  • Does not need to be attached to an instance
  • Can be transferred between availability zones
  • Supports incremental snapshots
  • EBS leverages S3 for snapshot storage
  • EBS is consumed from EC2 instances and is not a separate service
  • Billed on storage capacity and I/O
  • Does not need to be attached to an instance
  • Can be transferred between availability zones
  • EBS volumes are designed for an annual failure rate (AFR) of between 0.1% - 0.2%
  • EBS volume date is replicated across multiple servers in an availability zone

INCREASING IOPS PERFORMANCE

  • Multiple stripped gp2 or standard volumes (typically RAID 0)
  • Multiple stripped PIOPS volumes (typically RAID 0)
  • Function of the guest OS

EBS-OPTIMIZED INSTANCES

  • Dedicated capacity for Amazon EBS I/O
  • 500 Mbps - 4,000 Mbps
  • Not available for all instance Types
    • for example not available for Micro instances
  • GP-SSD within 10% of baseline and burst performance 99.9% of the time
  • PIOPS within 10% of provisioned performance 99.9% of the time
  • EBS-optimized instances are designed for use with all EBS volume types
  • Additional hourly fee

Find information on EBS-Optimized instances here… -> EBS-Optimized Instances Docs

EBS SNAPSHOTS CHARACTERISTICS

  • Point-in-time snapshots
  • Supports incremental snapshots
    • cannot recover specific files
    • best option really is third-party backup
  • Billed only for the changed blocks
  • Deleting a snapshot removes only the data not needed by any other snapshots
  • Leverages S3 for snapshot storage
  • Features
    • Allows resizing EBS volumes
    • Allows sharing EBS snapshots
    • Copying EBS snapshots across regions
    • Lazy loading (not necessarily a good Feature)
  • Pre-warming EBS Volumes

EBS VIDEO

UNDERSTANDING GLACIER

Glacier is made up of mostly slow drives, can be used to store long-term data that is not access frequently.

  • Very cheap storage
  • Useful for infrequently used data
  • Ideal for backups
  • Slow retrieval times (4 - 6 hours
  • High durability
  • Cost for restore
  • AES 256 bit data encryption
  • 0.01 per GB

AWS GLACIER VIDEO

AWS COMPUTE OPTIONS

ELASTIC COMPUTE CLOUD (EC2)

EC2 are analogous to traditional virtual machines, such as VMware or Hyper-V.

  • Pay-per-use, scalable platform for VMs
  • Supports Windows / Linux instances
  • Amazon Machine Image (AMI) refers to a virtual disk template (eg., OVA, OVF)
  • You can import / export your own AMIs
    • You cannot export Amazon created AMIs
  • AWS Terminology
    • AWS
      • EC Instances
      • Amazon Machine Image (AMI)
      • Elastic Block Storage (EBS)
      • Ephemeral Storage
      • Simple Storage Service (S3)
      • EC2 Compute Unit (ECU)
      • vCPU
      • Identity and Access Management (IAM)
      • Elastic Load Balancer (ELB)
      • Route 53
    • Industry
      • Virtual Machines / Servers
      • Template (OVA, OVF)
      • Volume / Hard Disk / Virtual Disk
      • Temporary Storage
      • Object-based Storage
      • Measure of EC2 Processor
      • vCPU
      • Delegated Administration
      • Load Balancer
      • DNS
  • EC2 Compute Unit (ECU)
    • Obsolete
    • A relative measure of processing power
    • Abstracts CPU hardware changes
    • One ECU is equivalent to 1.0-1.2 GHZ 2007 Intel Opteron or 2007 Xeon (Old)
    • Consistent amount of CPU capacity regardless of hardware
    • Each instance type may be based on different physical processors
    • Manage consistency through benchmarks and tests
  • EC2 Instance Types
    • Micro Instances
      • limited resources
      • small ram/cpu
    • General purpose
    • Compute optimized
      • CPU intensive
    • GPU Instances

UNDERSTANDING EC2 PRICING

Pricing varies for many things, size of instance, instance type, usage, commitment term if any.

ON-DEMAND INSTANCES

  • The default option
    • The most expensive option
    • No commitment
    • Prices vary by AWS Region
    • Billed on an hourly basis
      • no less than 1 hour is billed when powered on

RESERVED INSTANCES (RI)

  • Less expensive (commitment based)
    • Requires a commitment (1 or 3 years)
      • avoid 3 years if possible
    • Has an upfront cost
    • Lower hourly rate
    • RI can be sold on the AWS Marketplace
    • You commit to utilization
    • Light Utilization RIs
      • Ideal for couple of h/day or couple days/week
    • Medium Utilization RIs
      • Ideal for couple of h/day or couple days/week
    • Heavy Utilization RIs
      • Committing to running 100%
      • Highest savings

SPOT INSTANCES

  • Unused AWS capacity
    • Very cheap hourly rate
    • Not guaranteed
    • Based on a bid
      • if outbid, you lose the instance immediately
    • ideal for raw processing power, grid-like applications

EC2 VIDEO

UNDERSTANDING AWS NETWORKING, CLOUDWATCH, AND AUTO-SCALING

VIRTUAL PRIVATE CLOUD (VPC)

  • Logically isolated network in the AWS cloud
  • Control of network architecture
  • Enhanced security
    • egress and ingress security policy
    • routing rules
  • Internetwork with other organizations
  • Elastic IP addresses (public IPs)
  • Enable hybrid cloud (site-to-site VPN)
    • VPN
    • Direct-connect
  • VPN cost is $0.05/hr (varies always check prices)

IP ADDRESS BLOCKS

  • AWS reserves 5 addresses per Subnet
  • Single region, multi-AZ
  • CIDR 16-28
  • Selected IP prefix
    • This means you can select your own private subnet

CREATING A VPC

Quick and dirty on creating a VPC for AWS, the next few steps will show how to edit. Adding subnets, routing rules, attaching EC2 instances and more.

Create VPCCreating a VPC

VPC ACCESS

  • Gateway

    • Internet Gateway (IGW)
      • Ingress and Egress traffic
    • Virtual Private Gateway (VPG)
      • AWS side of Secure VPN
    • Customer Gateway (CG)
      • Customer Side of Secure VPN
  • Direct Connect

    • Dedicated and isolated
    • No internet
    • HA connectivity supported

Direct connect is a through a physical connection, fiber or colo/peering-point. These connections are typically 1Gbps or higher and allow fast direct access to AWS resources.

  • Hardware-based VPN
    • On-premises to AWS over internet
    • HA connectivity supported
    • 3rd party brands supported

Though called “hardware based”, these connections can also be from a virtual VPN appliance on-premises such as a Cisco ASAv running an a VMware environment.

VPC NETWORK SECURITY

  • Security Groups
    • Resource level traffic firewalls
      • Instance, ELB, etc…
      • Instance level
    • Egress and Ingress
    • Stateful
    • Maximums
      • Up to 100 security groups per VPC
      • Up to 50 lines in each SG
      • Up to 5 SG per instance
    • Understanding SGs (Security Groups)
      • Instances can’t communicate unless allowed
      • Default SG allows communications from other instances in the same SG group
      • Destination port filtering only (no source port filtering)
      • default permit all outbound
      • default deny all inbound
      • only allow permit rules

Security groups by default have a deny all for inbound traffic. Deny statements cannot be entered in policies, only allowed policies. Means this follows the white-list model. Deny until explicitly allowed by rule in security groups.

Security groups allow all outbound traffic by default, until an allowed rule is applied. At this point, security group becomes deny all except what’s explicitly allowed.

  • Access control lists
    • Source and Protocol filtering
    • Subnet level firewall
    • Stateless
    • Can have permit and deny rules
    • Only one NACL per subnet allowed
    • They are a list of rules (numbered)
      • lower numbers are processed first
      • first match stop
      • Separate inbound / outbound rules
  • Connecting multiple VPCs is possible
    • Inter-VPC routing
    • Same or different AWS account
    • No overlapping IP addresses permitted

Network Access Control Lists (NACls) default to deny all both inbound and outbound. Can have permit and deny rules (statements) in them. They are more granular than SGs and are applied at the VPC level not instance level.

VPC VIDEO

ELASTIC LOAD BALANCING (ELB)

Elastic load-balancers are Amazon’s version of distributed load-balancers. They are an edge service which means that they are not instance base and are configured on-demand.

  • Region wide load balancer
  • Can be used internally or externally
  • SSL Termination and processing
    • SSL Offloading
  • ELB EC2 health checks
  • Integrates with auto-scaling
  • Route 53 performs ELB health checks / CloudWatch

ELB VIDEO

ROUTE 53 (DNS SERVICE0)

  • DNS is crucial for any environment
  • Worldwide distributed DNS
  • Has a 100% SLA uptime
  • It has an available API
  • Performs server health checks

ROUTE 53 VIDEO

AWS CLOUDWATCH

  • Basic monitoring for free (7 metrics, 5 min)
  • Detailed monitoring (10 alarms, 1 million API requests, 1m)
  • Set alarms and alerts
  • Notification via SES, SNS
  • Billing notifications (set thresholds)
  • Custom monitoring through API
  • Integrate with Auto Scaling
  • Mobile app for basic monitoring and management

CLOUDWATCH VIDEO

AUTO SCALING

This service allows you to grow or shrink your workloads as demands increase or decrease respectively.

  • Expand or Shrink EC2 instances on-demand
  • CloudWatch or manual schedule configuration
  • Notifications
  • It’s free
  • Launch configuration
  • Select gold image (AMI)

AUTO-SCALING VIDEO

AMAZON DATABASE OPTIONS, APIs, AND LAMBDA

AMAZON RDS OPTIONS

  • MySQL
  • Oracle
  • Microsoft SQL
  • PostgreSQL
  • Aurora

NON-RDS OPTIONS

  • SimpleDB
  • DynamoDB
  • MongoDB
  • Couchbase

AMAZON APIs

  • Application Programming Interface
  • Application-to-application communication method
  • Dropbox uses S3 API for storage
  • Almost every AWS service is API capable
  • API Wrappers
    • Android / iOS / .NET / Java / PHP / Ruby / Node.JS / Python
  • API authentication

AMAZON LAMBDA

I’m a big fan of Lambdas or functions as service enablers. They are the cloud of the future along side SaaS solutions in my opinion. While IaaS is not going anywhere, we’ll see Lambda type services increase.

  • Event-driven compute service
    • Stateless, request-driven code called Lambda functions
    • Triggered by events:
      • PUT in S3
      • Write to a DynamoDB table
      • Transition in an EC2 instance
      • Message in an SQS queue or Kinesis stream
      • Any API call or resource transition
    • connective tissue for AWS services
  • Runs code triggered by events
  • Does not require instance
  • Does not require an infrastructure
  • Rapid response to events
  • Thousands of functions can run
  • Run only when needed
  • Before Lambda
    • Provision a fleet of proxy machines to capture uploads
    • For each upload, enqueuer a job to process
    • Provision a second fleet of machines to read and process jobs
    • Pick a deployment solution
    • Plan capacity, accounting for FT, long-term utilization and burst capabilities
    • Monitoring and patching
    • Migrating to new instance types over time

LAMBDA VIDEO

AMAZON SIMPLE SERVICES

SIMPLE EMAIL SERVICE (SES)

  • Cost effective bulk email service
  • Cost based on number of emails sent
  • Outbound-only email-sending service
  • Leverages Amazon email reputation
  • Initially limited to 10,000 emails/day

SIMPLE QUEUEING SERVICE (SQS)

  • Fast, reliable, and scalable
  • Unlimited messages and queue size
  • Payload up to 256KB
  • Billed in chunks of 64KB payloads
  • First 1 million requests are free
  • $0.50 / million SQS requests

SIMPLE NOTIFICATION SERVICE (SNS)

  • Push messaging service (supports):
    • HTTP/HTTPS
    • Email
    • Email-JSON
    • SMS
    • Amazon SQS queues

CLOUDFRONT, CLOUDFORMATION, ELASTIC BEANSTALK, AND CLOUDTRAIL

CLOUDFRONT

  • Global content delivery network (CDN)
  • Leverages all AWS edge locations
  • Cache static content
  • Proxy dynamic information
  • Works with AWS and non-AWS services

CLOUDFRONT VIDEO

CLOUDFORMATION

  • Automates AWS resource provisioning
  • Free service
  • Deleting the CloudFormation deletes all instances except data

CLOUDFORMATION VIDEO

ELASTIC BEANSTALK

  • Automates development environments (APPS)
  • More for developers
  • Supported platforms to automate
    • Node.JS
    • Ruby
    • PHP
    • Python
    • IIS
    • Tomcat
    • More…
  • Leverages CloudFormation services

ELASTIC BEANSTALK VIDEO

CLOUDTRAIL

  • Auditing Service
  • Records API calls
  • Log file includes:
    • Identity of API Caller
    • Time
    • Source IP address
    • Request parameter
    • Response elements returned
  • Maintains history
    • Management console
    • SDKs
    • command-line-tools
    • other AWS services
  • Uses CloudFormation as a service

DOWNLOAD FILES

AWS Fundamentals Markdown Notes

AWS Fundamentals iThoughts File

AWS Fundamentals OPML File

comments powered by Disqus