AWS CloudFormation

Kalyan Reddy Daida

StackSimplify
Course Objectives
• AWS CloudFormation
• Continuous Integration
• AWS Code Commit
• AWS Code Build
• Continuous Delivery
• AWS CodeDeploy
• AWS CodePipeline
• Infrastructure as Code with CI/CD Tools.

Kalyan Reddy Daida StackSimplify

Template Anatomy
5 Examples and 18
Resources resources overall

Parameters 4 Examples

Mappings 2 Examples

Conditions 6 Examples
AWS CloudFormation
Outputs 7 Examples

Metadata Designer &

Interface
cfn-init
Metadata

Kalyan Reddy Daida StackSimplify

Resources EC2 Instance CodeBuild

CodeBuild IAM
Security Group
Role
CodeDeploy
Elastic IP
Application
CodeDeploy
VPC
DeploymentGro
Resources CodeDeploy
Subnet
Deploymnet
CodeDeploy IAM
AWS CloudFormation Route Table
Role
SubnetRoute CodePipeline
TableAssociation IAM Role
CodePipeline
InternetGateway
Pipeline
VPCGatewayAtta
SNS Topic
chment

Kalyan Reddy Daida StackSimplify

Templates Written

Kalyan Reddy Daida StackSimplify

Metadata - cfn-init
Base Template Metadata Format services

packages aws-cfn-bootstrap

groups cfn-init

users cfn-signal

Create Stack &

sources outputs
Test

Create Stack &

files Creation Policy
Test

Update Stack –
commands cfn-hup
Deploy v2 App

Kalyan Reddy Daida StackSimplify

Config Sets
configSets (Single)
Sample:

App1AndApp2

App1 App2

Kalyan Reddy Daida StackSimplify

configSets (Multiple)

SingleAppCS DualAppCS default

App1 SingleAppCS App2 DualAppCS

App1 SingleAppCS App2

Sample: App1

Kalyan Reddy Daida StackSimplify

Nested Stacks
Root Stack

VPC Nested Security Group

Stack Nested Stack

Kalyan Reddy Daida StackSimplify

Templates Written

Kalyan Reddy Daida StackSimplify

Intrinsic Functions & Pseudo Parameters
Intrinsic Functions Condition Functions Pseudo Parameters
• Fn::Ref • Fn::And • AWS::Region
• Fn::Base64
• Fn::Equals • AWS::AccountId
• Fn::FindInMap
• Fn::GetAtt • Fn::If • AWS::StackName
• Fn::GetAzs • Fn::Not • AWS::NoValue (very
• Fn::ImportValue • Fn::Or important when using
• Fn::Join
conditions)
• Fn::Select
• Fn::Sub

Kalyan Reddy Daida StackSimplify

Stages in Release Process

Source Build Test Production

Kalyan Reddy Daida StackSimplify

AWS Developer Tools or Code Services
Source Build Test Deploy Monitor

AWS CodeCommit AWS CodeBuild AWS CodeBuild + AWS CodeDeploy Amazon

AWS X-Ray
Third Party CloudWatch

AWS CodePipeline
Kalyan Reddy Daida StackSimplify
AWS Developer Tools or Code Services
Source Build Test Deploy Monitor

AWS CodeCommit AWS CodeBuild AWS CodeBuild + AWS CodeDeploy Amazon

AWS X-Ray
Third Party CloudWatch

AWS CodePipeline
Kalyan Reddy Daida StackSimplify
AWS Developer Tools or AWS Code Services
Source Build Test Deploy Monitor

CodeCommit CodeBuild CodeBuild + Third Party CodeDeploy CloudWatch

AWS X-Ray
• Version control • Fully managed build service, Compiles • Automates code deployments • Monitors Source
service source code, Runs tests and produces to any instance and Lambda check-ins and triggers
• We can privately software packages • Avoids downtime during builds
store and manage • Scales continuously and processes multiple application deployment • Monitors builds
source code builds concurrently. • Roll back automatically if • Monitors
• Secure & highly • No build servers to manage. failure detected Infrastructure
available • Pay by minute, only for compute resources • Deploy to Amazon EC2, • Collects logs
we use. Lambda, or on-premises servers
• Monitor builds through CloudWatch events.
• Supports following programming language
• Continuous delivery service for fast and reliable
runtimes Ruby, Python, PHP, Node, Java,
application updates
Golang, .Net Core, Docker and Android • Model and visualize your software release process
• Builds, tests, and deploys your code every time there
is a code change
CodePipeline • Integrates with third-party tools and AWS

Kalyan Reddy Daida StackSimplify

CodeCommit
• Build a simple rest service using Java Spring Developer

Boot.
• Check-in code to Local Repo and push to Local Git
CodeCommit. Repo

push

AWS Cloud

AWS CodeCommit

Kalyan Reddy Daida StackSimplify

CodeBuild – AWS Web Console CodeBuild – AWS CloudFormation

Developer Developer

Local Git
Local Git Repo
Repo push
AWS Cloud
push
CodeBuild Stack
AWS Cloud

CodeBuild
CodeCommit CodeCommit Service Role

CodeBuild Simple Storage CodeBuild Simple Storage

Service (S3) Service (S3)

Kalyan Reddy Daida StackSimplify

CodeDeploy – AWS Web Console CodeDeploy – AWS CloudFormation

User
Developer Developer accessing
User Rest service
accessing
Rest service

Local Git Local Git Internet

Repo Repo

Internet push
push
AWS Cloud

CodeDeploy Stack EC2 Apps Stack

AWS Cloud

CodeCommit CodeBuild CodeDeploy Role for EC2

CodeCommit
Service Role Service Role Instance ProfileStaging EC2 Instance

CodeBuild Simple Storage CodeDeploy EC2 Instance CodeBuild Simple Storage CodeDeploy Production EC2
Service (S3) Service (S3) Instance

Kalyan Reddy Daida StackSimplify

CodePipeline – AWS Web Console
AWS Cloud

CodePipeline
Developer CloudWatch

Local Git push

Repo
CodeCommit CodeBuild Simple Storage CodeDeploy EC2
Service (S3) Instance

Kalyan Reddy Daida StackSimplify

CodePipeline – AWS CloudFormation
AWS Cloud

CodePipeline CI CD CloudFormation Stack

Developer

Local Git push

Repo
CodeCommit CodeBuild Simple Storage CodeDeploy Simple Notification CodeDeploy
Service (S3) Service

Staging EC2 Role for EC2 Production EC2

Instance Profile Authorized
Instance Instance Approver
EC2 Apps CloudFormation Stack

Kalyan Reddy Daida StackSimplify

Infrastructure as Code

Source Build Test Production

Source Stage Build Stage Test Stage Prod Stage

AWS CodeCommit AWS CodeBuild AWS CodePipeline AWS CodePipeline

Kalyan Reddy Daida StackSimplify

Infrastructure as Code

Source Build Test Production

Source Stage Build Stage Test Stage Prod Stage

Master Branch Prepare or Validate Create & Execute Create & Execute
Template Change set Change set

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – Manual AWS Web Console
AWS Cloud

Staging VPC Stack

CodePipeline Action-1:
VPC Subnet
Test Stage Create
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set
CodeBuild CodePipeline CloudFormation Internet
Service Role Role Role Route table gateway

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – CFN Template creation Flow
AWS Cloud

CI CD IAC Pipeline Stack Staging VPC Stack

CodePipeline Action-1:
Create VPC Subnet
Test Stage
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set

CodeBuild CodePipeline CloudFormation Internet

Role Route table gateway
Service Role Role

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – Execution Flow
AWS Cloud

CI CD IAC Pipeline Stack Staging VPC Stack

CodePipeline Action-1:
Test Stage Create VPC Subnet
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set
CodeBuild CodePipeline CloudFormation Internet
Service Role Role Role Route table gateway

Kalyan Reddy Daida StackSimplify

Templates Written

Kalyan Reddy Daida StackSimplify

YAML
Kalyan Reddy Daida StackSimplify
YAML
• YAML Key Value pairs
• YAML Lists
• YAML Dictionary
• YAML Lists containing Dictionaries
• YAML Lists containing Dictionaries containing Lists
• YAML Pipe
• YAML Greater than Sign
• YAML Comments

Kalyan Reddy Daida StackSimplify

YAML – Key Value Pairs
• YAML documents will be full of key value pairs.
• Key and Value are separated by colon.
• We must have a space after colon differentiating the Key Value Pairs
value. Name: Dave
• YAML Supports different data types. Age: 29
• Integer Gpa: 4.2
Occupation: Engineer
• Floating point Numbers
State: 'New Jersey'
• Strings AboutMe: "I am a software engineer"
• Boolean Male: true
• Dates - Format: ISO 8601 DateOfBith: 1990-09-15T15:53:00
• Null values PoliceCases: null
• Important Note for Strings: Quote strings when they
have special characters like colons :, braces {}, pipes
|, brackets []

Kalyan Reddy Daida StackSimplify

YAML – List / Array
• YAML List indented with opening dash. List / Array
• Dash indicates that it’s a element of an Block Sequence
array. Persons:
- Dave
• All members of a list are lines beginning at - John
the same indentation level starting with - Mike
- Sam
a ”-” (a dash and a space)
• Block Sequence indicate each entry with a Flow Sequence
Persons: [Dave, John, Mike, Sam]
dash and space
• Flow Sequence is written as a comma
separated list within square brackets.

Kalyan Reddy Daida StackSimplify

YAML Dictionary / Map
• YAML Dictionaries are set of properties Dictionary
grouped together under an item.
Dave:
• YAML Dictionaries contain key value Age: 25
pairs. Occupation: Engineer
State: New Jersey
gpa: 4.5
male: true

Kalyan Reddy Daida StackSimplify

YAML Lists containing Dictionaries

Kalyan Reddy Daida StackSimplify

YAML Lists containing Dictionaries containing Lists

Kalyan Reddy Daida StackSimplify

YAML Pipe
• The pipe notation, also referred to as
literal block
• All new lines, indentation, extra spaces
everything preserved as is.

Kalyan Reddy Daida StackSimplify

YAML Greater than Sign
• The greater than sign
notation, also referred to as
folded block.
• Renders the text as a single
line.
• All new lines will be replaced
with a single space.
• Blank lines are converted to
new line character.

Kalyan Reddy Daida StackSimplify

YAML Comments
• We can have comments in YAML with # sign. Below is an example.

Kalyan Reddy Daida StackSimplify

Stack Features

Kalyan Reddy Daida StackSimplify

AWS CloudFormation
• Simplifies our Infrastructure Management.
• Quickly replicates our infrastructure.
• Easily controls and tracks changes to our
infrastructure.

Kalyan Reddy Daida StackSimplify

How does CloudFormation works?

Kalyan Reddy Daida StackSimplify

AWS CloudFormation
• Stack Core Features
• Create Stack
• Update Stack
• Create Change Set
• Roll back
• Stack
• Managing collection of AWS resources as a single unit is
called stack.
• We can create, update, delete the collection of AWS
resources by creating, updating and deleting stacks.
• To create AWS resources, we create a stack by submitting
the template that we created, AWS CloudFormation
provisions all those resources automatically for us.

Kalyan Reddy Daida StackSimplify

AWS CloudFormation
• Change Set
• If we want to make changes to our stack, we can
update the stack.
• Before making changes to resources, we can generate
a change set, which is summary of proposed changes.
• Change sets allow us to see how our changes might
impact current running resources in a stack especially
for critical resources, before implementing them we
get an idea about the impact.
• For example: If we associate a new keypair to ec2
instance, AWS will delete the current ec2 instance
and replaces it with new ec2 instance by adding new
keypair to it.

Kalyan Reddy Daida StackSimplify

Stack Features
• Step 00: Pre-requisites
• Create Default VPC (if not present)
• Create Key pairs
• cfn-key-1
• cfn-key-2
• Gather AMI ID
• Step 01: Stack Features
• Create Stack
• Update Stack
• Create Change Sets
• Rollback

Kalyan Reddy Daida StackSimplify

Resources

Kalyan Reddy Daida StackSimplify

Resources
• Resources are key components of a stack.
• Resources section is a required section that need to be defined in
cloud formation template.
• Syntax

• Resources Documentation:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-
template-resource-type-ref.html
Kalyan Reddy Daida StackSimplify
Resources
• Step 01: Create resource - EC2 Instance
• Step 02: Add Second Resource - New
security group and Intrinsic Function Ref
• Step 03: Update Resource Properties -
Add new rule to Security group
• Step 04: Add third Resource - Elastic IP
• Step 05: Perform case sensitive test with
resource properties

Kalyan Reddy Daida StackSimplify

Intrinsic Function: Ref
• The intrinsic function Ref returns the value of the specified
parameter or resource.
• Resource Case: When we specify a resource logical name, it returns a
value that we can typically use to refer to that resource.
• Parameter Case: When we specify a parameter logical name, it
returns the value of that parameter.
• Syntax:
• Long Form
• Ref: logicalName
• Short Form
• !Ref logicalName

Kalyan Reddy Daida StackSimplify

Parameters

Kalyan Reddy Daida StackSimplify

Parameters
• Parameters: Parameters enable us to input custom values to our template
each time when we create or update stack.
• We can have maximum of 60 parameters in a cfn template.
• Each parameter must be given a logical name (logical id) which must be
alphanumeric and unique among all logical names within the template.
• Each parameter must be assigned a parameter type that is supported by
AWS CloudFormation.
• Each parameter must be assigned a value at runtime for AWS
CloudFormation to successfully provision the stack. We can optionally
specify a default value for AWS CloudFormation to use unless another value
is provided.

Kalyan Reddy Daida StackSimplify

Parameters
• Parameters must be declared and referenced within the same
template.
• We can reference parameters from the Resources and Outputs
sections of the template.
• Syntax

Kalyan Reddy Daida StackSimplify

Parameter Properties Parameter Types

• AllowedPattern • Type (Mandatory) • Type (Mandatory)

• AllowedValues • String • SSM Parameter Type
• Number • AWS::SSM::Parameter::Name
• ConstraintDescription • AWS::SSM::Parameter::Value
• List<Number>
• Default <String>
• CommaDelimitedList • AWS::SSM::Parameter::Value
• Description • AWS Specific <List<String>>
• AWS::EC2::Instance::Id
• MaxLength • AWS::EC2::VPC::Id
• MaxValue • List<AWS::EC2::Subnet::Id>

• MinLength
• MinValue
• NoEcho

Kalyan Reddy Daida StackSimplify

Parameters - Practice
• Step 01: Create a parameter type of AWS for
KeyName property of ec2 instance.
• Step 02: Create a parameter type of string for
AvailabilityZone property of ec2 instance.
• Step 03: Create a parameter type of string for
InstanceType property of ec2 instance.
• Step 04: Create a parameter type of SSM for
InstanceType property of ec2 instance.
• Pre-requisite: Create a SSM Parameter in parameter
store.

Kalyan Reddy Daida StackSimplify

Mappings

Kalyan Reddy Daida StackSimplify

Mappings
• Mappings section matches a
key to a corresponding set of
named values.
• For example, if we want to set
values based on a region, we
can create a mapping that uses
region name as a key and
contains the values we want to
specify for each region
• We can use Fn::FindInMap
intrinsic function to retrieve
values in map.

Kalyan Reddy Daida StackSimplify

Intrinsic Function: FindInMap
• The intrinsic function FindInMap
returns the value corresponding to
keys in a two-level map that is
declared in Mappings section.
• Parameters
• Map Name
• Top Level Key
• Second Level Key
• Return Value

Kalyan Reddy Daida StackSimplify

Mappings - Practice
• Step 01: Create a Mapping to select the AMI ID for
ec2 instance property – ImageId based on region.
• Top Level Key: Region (us-east-2, us-west-1)
• Second Level Key: HVM64
• Step 02: Create a Mapping to select the instance
type based on environments (dev or prod) for ec2
instance property - InstanceType
• Top Level Key: Environment (dev, prod)
• Second Level Key: Instance Type

Kalyan Reddy Daida StackSimplify

Pseudo Parameters
• AWS::AccountId
• Pseudo parameters are
parameters that are • AWS::NotificationARNs
predefined by AWS • AWS::NoValue
CloudFormation. • AWS::Partition
• We don’t need to declare • AWS::Region
them in our template. • AWS::StackId
• AWS::StackName
• We can use them the same
way as we use parameters as • AWS::URLSuffix
an argument for Ref function.
• Usage:

Kalyan Reddy Daida StackSimplify

Conditions

Kalyan Reddy Daida StackSimplify

Conditions
• Conditions section contains statements that define the circumstances under
which entities are created or configured.
• Example: 1 - We can create a condition and then associate it with a
resource or output so that AWS CloudFormation only creates the resource
or output if the condition is true.
• Example:2 - We can associate the condition with a property so that AWS
CloudFormation only sets the property to a specific value if the condition is
true, if the condition is false, AWS CloudFormation sets the property to a
different value that we specify.
• We will use conditions, when we want to re-use the template in different
contexts like dev and prod environments.
• Synatx:

Kalyan Reddy Daida StackSimplify

Conditions
• Conditions are evaluated based on predefined Psuedo parameters or input
parameter values that we specify when we create or update stack.
• Within each condition we can reference the other condition.
• We can associate these conditions in three places.
• Resources
• Resource Properties
• Outputs
• At stack creation or stack update, AWS CloudFormation evaluates all
conditions in our template. During stack update, Resources that are now
associated with a false condition are deleted.
• Important Note: During stack update, we cannot update conditions by
themselves. We can update conditions only when we include changes that
add, modify or delete resources.

Kalyan Reddy Daida StackSimplify

Conditions - Intrinsic Functions
• We can use the below listed intrinsic functions to define conditions in
cloud formation template.
• Fn::And
• Fn::Equals
• Fn::If
• Fn::Not
• Fn::Or
• We will be covering all these functions in our practice exercises.

Kalyan Reddy Daida StackSimplify

Conditions - Practice
• Step 01: Create an EIP when environment is prod,
use intrinsic function Fn::Equals
• Step 02: Create a security group for dev environment
when condition is met and demonstrate Pseudo
parameter “AWS::NoValue” for when environment is
prod. Use Intrinsic function Fn::If
• Step 03: Create a security group for prod env with
prod related condition added. Use Intrinsic function
Fn::If
• Step 04: Demonstrate Intrinsic function Fn::Not
• Step 05: Demonstrate Intrinsic function Fn::Or
• Step 06: Demonstrate Intrinsic function Fn::And

Kalyan Reddy Daida StackSimplify

Outputs

Kalyan Reddy Daida StackSimplify

Outputs
• Outputs section declares output values that we can
• Import in to other stacks (to create cross-stack references)
• When using Nested stacks, we can see how outputs of a nested stack are
used in Root Stack.
• We can view outputs on the CloudFormation console
• We can declare maximum of 60 outputs in a cfn template.
• Syntax:

Kalyan Reddy Daida StackSimplify

Outputs
• Export (Optional)
• Exports contain resource output used for cross-stack reference.
• For each AWS account, Export name must be unique with in the region. As it
should be unique we can use the export name as “AWS::StackName”-ExportName
• We can’t create cross-stack references across regions.
• We can use the intrinsic function Fn::ImportValue to import values that have been
exported within the same region. We will see this practically.
• In simple terms, export availability zone in stack1 and use it stack2
• For outputs, the value of the Name property of an Export can't use Ref or GetAtt
functions that depend on a resource.
• We can’t delete a stack if another stack references one of its outputs.
• We can’t modify or remove an output value that is referenced by another stack.
• We can use Outputs in combination with Conditions. We will see that in our
practice sessions for Outputs.

Kalyan Reddy Daida StackSimplify

Outputs Practice
• Step 01: Create a very basic output using intrinsic function
Fn::Ref - InstanceId.
• Step 02: We will use Fn::GetAtt intrinsic function to create
outputs.
• Step 03: We will use Fn::Sub intrinsic function to create
outputs and we will use Pseudo Parameter
AWS::StackName. In addition, we will export the Security
Group and Availability Zone.
• Step 04: We will create a new stack by referencing the
Security Group and Availability Zone export value from
previous stack. We will use Fn::ImportValue intrinsic
function to import those exports.
• Step 05: We will use Conditions in Outputs section to
demonstrate their combination.
• Step 06: We will demonstrate Fn::Join intrinsic function.

Kalyan Reddy Daida StackSimplify

Metadata

Kalyan Reddy Daida StackSimplify

Metadata
• Metadata provides details about the cfn template.
• Syntax:

• We have three types of metadata keys which are listed below.

• Metadata Keys
• AWS::CloudFormation::Designer
• AWS::CloudFormation::Interface
• AWS::CloudFormation::Init

Kalyan Reddy Daida StackSimplify

Metadata Keys
• AWS::CloudFormation::Designer
• Auto generated during resources drag and drop to canvas.
• AWS::CloudFormation::Interface
• Used for parameter grouping.
• AWS::CloudFormation::Init
• Used for application installation and configurations on our aws compute (EC2
instances).
• This is core and important feature of CloudFormation.
• We have one complete section outlining the end to end details of init.

Kalyan Reddy Daida StackSimplify

Metadata
AWS::CloudFormation::Designer

Kalyan Reddy Daida StackSimplify

AWS::CloudFormation::Designer
• Designer, Visually depicts how our
resources are laid out
• Designer automatically add this
information when we use it to
create view and update templates.
Its a system generated metadata.
• It defines the information about our
resources such as their size and
relative position in template
metadata. All layout information is
stored in designer metadata.
Kalyan Reddy Daida StackSimplify
AWS::CloudFormation::Designer
• In designer we drag and drop the resources.
• When we create templates in Designer, it enforces some basic
relationships between resources to help us create valid template.
• Example: We cannot directly add EC2 instance in a VPC, we must add
a subnet in a VPC.
• We can also validate template directly in designer.
• We can bring our template which we have written manually and
validate in designer using validate template.

Kalyan Reddy Daida StackSimplify

AWS::CloudFormation::Designer
• Integrated Editor:
• We can make all our template modifications with this editor.
• It also provides the auto-complete feature that lists all property names for a
resource so we don’t need to memorize all the properties of a resource or
refer documentation.
• We can use integrated editor to convert from JSON to YAML and vice versa.

Kalyan Reddy Daida StackSimplify

Kalyan Reddy Daida StackSimplify
How I use Designer?
• Firstly, I write my cfn templates manually in editor by referring
documentation
• Which gives me greater confidence on that particular resource for which I am
writing template.
• I use visual studio code as my editor due to the fact that dealing with YAML
spaces is simplified in this editor. I just use tabs and VS code editor takes care
of yaml spaces.
• Copy template to Integrated Editor and Validate Template.
• Convert template from JSON to YAML or YAML to JSON.
• Drag resources to canvas and see their properties (some times).
• Copy template to Integrated Editor and review template visually on
canvas.
Kalyan Reddy Daida StackSimplify
CloudFormation Designer - Demo

Kalyan Reddy Daida StackSimplify

Metadata
AWS::CloudFormation::Interface

Kalyan Reddy Daida StackSimplify

AWS::CloudFormation::Interface
Syntax:
• When we create or update stacks in
the console, the console lists input
parameters in alphabetical order by
their logical IDs.
• By using this key, we can define our
own parameter grouping and ordering
so that users can efficiently specify
parameter values.
• We can also define labels for
parameters.
• A label is a friendly name or
description that the console displays
instead of a parameter's logical ID
which helps users understand the
values to specify for each parameter.

Kalyan Reddy Daida StackSimplify

EC2 UserData

Kalyan Reddy Daida StackSimplify

CloudFormation & UserData
• We can use UserData in CloudFormation
template for ec2.
• We need to use a intrinsic function Fn::Base64 Sample:
with UserData in CFN templates. This function
returns the Base64 representation of input
string. It passes encoded data to ec2 Instance.
• YAML Pipe (|): Any indented text that follows
should be interpreted as a multi-line scalar value
which means value should be interpreted literally
in such a way that preserves newlines.
• UserData Cons
• By default, user data scripts and cloud-init
directives run only during the boot cycle when
we first launch an instance.
• We can update our configuration to ensure that
our user data scripts and cloud-init directives run
every time we restart our instance. (Reboot of
server required)

Kalyan Reddy Daida StackSimplify

Helper Scripts
cfn-init, cfn-hup and cfn-signal

Kalyan Reddy Daida StackSimplify

Helper Scripts
• AWS CloudFormation provides the following Python helper scripts
that we can use to install software and start services on Amazon EC2
that we create as part of stack.
• cfn-init
• cfn-signal
• cfn-get-metadata
• cfn-hup

Kalyan Reddy Daida StackSimplify

Metadata
AWS::CloudFormation::Init

Kalyan Reddy Daida StackSimplify

Base Template Metadata Format services

packages aws-cfn-bootstrap

groups cfn-init

users cfn-signal

Create Stack &

sources outputs
Test

Create Stack &

files Creation Policy
Test

Update Stack –
commands cfn-hup
Deploy v2 App

Kalyan Reddy Daida StackSimplify

Step 00 – Base Template
• Resources
• Security Group
• VM Instnaces
• Parameters
• We will Parameterize KeyName parameter

Kalyan Reddy Daida StackSimplify

Step-01: Metadata: AWS::CloudFormation::Init
• Type AWS::CloudFormation::Init will be used to
include metadata section on an ec2 instance for cfn-
init helper script.
• Configuration is separated in to sections.
• Metadata is organized in to config keys, which we can
even group in configsets.
• By default cfn-init calls and processes the metadata
section when it has single config key (No configsets
defined).
• We can even specify configsets as input to cfn-init
script so that it can process the entire configset with
all its configkeys. We will see it in detail in configsets
section.
• The cfn-init helper script processes the configuration
sections in the order specified in syntax section.

Kalyan Reddy Daida StackSimplify

Step-01: Metadata: Structure
• If we want to process it in different order, we need to
separate them into different config keys and then use the
order of execution for config keys in a configset.
• In this step we will just add the metadata section with
structure.
• We will incrementally build the metadata sections in
upcoming steps.
• Metadata Structure:

Kalyan Reddy Daida StackSimplify

Step-02: Metadata: packages
• We can use packages key to • Packages with Versions:
download and install pre-
packaged applications.
• On windows systems packages
key supports only the MSI
Installer.
• Supported Package Formats:
• apt
• msi • Our Example:
• python
• rpm
• rubygems
• yum

Kalyan Reddy Daida StackSimplify

Step-03: Metadata: groups
• We can use groups to create Linux/Unix groups and assign to group
id’s.
• Groups key is not supported for windows systems.
• We can create multiple groups as required.
• We can create without group id or create with a desired group id.
• Syntax:

Kalyan Reddy Daida StackSimplify

Step-04: Metadata: users
• We can use the users key to create • Syntax
Linux/Unix users in EC2 Instance.
• Users key is not supported for windows
systems.
• The following are the supported keys
• uid
• groups
• homeDir
• Users are created as non-interactive
system users with a shell of /sbin/nologin.
• This is by design and cannot be modified

Kalyan Reddy Daida StackSimplify

Step-05: Metadata: sources
• We can use the sources key to download an archive file and unpack it
in a target directory on EC2 Instance.
• This key is fully supported for both Linux and Windows systems.
• Supported Archive formats
• tar
• tar + gzip
• tar + bz2
• zip
• Syntax / Example:

Kalyan Reddy Daida StackSimplify

Step-05: Metadata: sources
• Create S3 bucket
• Disable block public access to bucket.
• Create cfn folder
• Upload the zip files demo1.zip, demo2.zip which contains demo.war (two
versions v1 and v2)
• Unzip AWS-CloudFormation.zip to local directory
• Navigate to 11-cfn-init/WAR-Files folder
• Upload the demo1.zip, demo2.zip to S3 bucket cfn folder.
• Path: /AWS-CloudFormation/11-cfn-init/WAR-files
• Make the demo1.zip, demo2.zip as public file.
• Copy the S3 http url for both files and perform public access test.
• Update demo1.zip url in sources section of template.

Kalyan Reddy Daida StackSimplify

Step-06: Metadata: files
• We can use the files key to create files on EC2 Instance.
• The content can be either inline in the template or the content can be
pulled from a URL.
• The files are written to disk in alphabetical order.
• Supported Keys
• content
• source
• Encoding (plain or base64)
• group
• owner
• mode
• authentication
• context

Kalyan Reddy Daida StackSimplify

Step-06: Metadata: files
Syntax / Sample:

Kalyan Reddy Daida StackSimplify

Step-07: Metadata: commands
• We can use commands key to execute
commands on EC2 Instance.
• The commands are processed in Syntax / Example:
alphabetical order by name.
• Supported Keys
• command
• env
• cwd
• test
• ignoreErrors
• waitAfterCompletion

Kalyan Reddy Daida StackSimplify

Step-08: Metadata: services
• We can use services key to define which services should be
enabled or disabled when the instance is launched.
• On Linux systems this key is supported by using sysvinit.
• On Windows systems, it is supported by using Windows Service
Manager.
• Services key also allows us to specify dependencies on sources,
packages and files so that if a restart is needed due to files being
installed, cfn-init will take care of the service restart.
• Supported Keys
• ensureRunning
• enabled
• files
• sources
• packages
• commands

Kalyan Reddy Daida StackSimplify

Step-08: Metadata: services
• The nginx service will be restarted if either
/etc/nginx/nginx.conf or /var/www/html are
modified by cfn-init.
• The php-fastcgi service will be restarted if cfn-
init installs or updates php or spawn-fcgi using
yum.
• The sendmail service will be stopped and
disabled.

Kalyan Reddy Daida StackSimplify

UserData

Kalyan Reddy Daida StackSimplify

Step-09: UserData: aws-cfn-bootstrap
• Helper Scripts are updated periodically.
• We need to ensure that the below listed command is included in
UserData of our template before we call the helper scripts to ensure
that our launched instances get the latest helper scripts.

Kalyan Reddy Daida StackSimplify

Step-10: UserData: cfn-init
• The cfn-init helper script reads template
metadata from the AWS::CloudFormation::Init
key and acts accordingly to:
• Fetch and parse metadata from AWS
CloudFormation
• Install packages Command Syntax:
• Write files to disk
• Enable/disable and start/stop services
• If we use cfn-init to update an existing file, it
creates a backup copy of the original file in the
same directory with a .bak extension.
• cfn-init does not require credentials. However,
if no credentials are specified, AWS
CloudFormation checks for stack membership
and limits the scope of the call to the stack
that the instance belongs to.
Command Usage in UserData:

Kalyan Reddy Daida StackSimplify

Step-11: UserData: cfn-signal
• The cfn-signal helper script signals AWS Command Syntax:
CloudFormation to indicate whether
Amazon EC2 instances have been
successfully created or updated.
• If we install and configure software
applications on instances, we can signal
AWS CloudFormation when those software
applications are ready.
• We can use the cfn-signal script in
conjunction with a CreationPolicy.

Kalyan Reddy Daida StackSimplify

Step-11: UserData: cfn-hup
• Important Note: From here on we will start creating the stack using
v12 template file, we will add cfn-hup command also to template
UserData section even though we discuss that section in step 14.
Reason for doing that is UserData related changes should be included
during instance creation time only.
• Final Look of UserData:

Kalyan Reddy Daida StackSimplify

Step 12 - Outputs
• Add outputs in the template.
• We will add AppURL output for easily accessing the application after
stack creation.
• Sample

Kalyan Reddy Daida StackSimplify

Step 12: Create Stack using template
11-12-cfn-init-v12-Outputs.yml
• Observations
• CloudFormation gets the signal as soon as VM Instance resource gets
created.
• In other words, we will see that stack status “CREATE_COMPLETE” even
though in the back ground application installations are going on in the EC2
Instance.
• With this approach we have problems like
• Applications installs fails and we see the stack status as “CREATE_COMPLETE” in green.
• We will not know what happened to our application installs or configurations until we
login to instance.
• To overcome such type of issues, we need to use “Creation Policy” which we
will see in next step (step 13).

Kalyan Reddy Daida StackSimplify

Step-13: Creation Policy
• Associate the CreationPolicy attribute with a resource to prevent its
status from reaching create complete until AWS CloudFormation
receives a specified number of success signals or the timeout period
is exceeded.
• To signal a resource we can use cfn-signal helper script.
• The creation policy is invoked only when AWS CloudFormation
creates the associated resource.
• Currently, the only AWS CloudFormation resources that support
creation policies are
• AWS::AutoScaling::AutoScalingGroup
• AWS::EC2::Instance
• AWS::CloudFormation::WaitCondition

Kalyan Reddy Daida StackSimplify

Step-13: Creation Policy
• Use the CreationPolicy attribute when you want to wait on resource
configuration actions before stack creation proceeds.
• For example, if we install and configure software applications on an
EC2 instance, we might want those applications to be running before
proceeding. In such cases, we can add a CreationPolicy attribute to
the instance, and then send a success signal to the instance after the
applications are installed and configured.
• Syntax:

Kalyan Reddy Daida StackSimplify

Step 13: Create Stack using template
11-13-cfn-init-v13-CreationPolicy.yml
• Observations
• CloudFormation waits for the status until application installs are completed
for that particular resource (in our case its VM Instance).
• Either it waits for success signal and if within specified time (time specified in
creation policy) if it didn’t get success signal it roll backs the entire stack.

Kalyan Reddy Daida StackSimplify

Step-14: UserData: cfn-hup
• cfn-hup helper is a daemon that detects changes in resource • Format of cfn-hup.conf
metadata and runs user-specified actions when a change is
detected.
• This allows us to make configuration updates on our running
EC2 Instance through the Update Stack feature.
• cfn-hup.conf
• cfn-hup.conf file stores the name of the stack and the AWS
credentials that the cfn-hup daemon targets.
• Format of cfn-hup.conf
• We are creating this file using our Metadata Key named files in
our template.

Kalyan Reddy Daida StackSimplify

Step-14: UserData: cfn-hup
• cfn-hup.conf file content
• stack
• credential-file
• role
• region
• umask (default: 022)
• Interval (default: 15)
• Verbose
• hooks.d Directory
• To support composition of several applications deploying change notification
hooks, cfn-hup supports a directory named hooks.d that is located in the hooks
configuration directory.
• We can place one or more additional hooks configuration files in the hooks.d
directory.

Kalyan Reddy Daida StackSimplify

Step-14: UserData: cfn-hup - hooks.conf
• User actions that cfn-hup daemon calls periodically are defined in
hooks.conf.
• Syntax:

Kalyan Reddy Daida StackSimplify

Step-14: UserData: cfn-hup - hooks.conf
• When the action is run, it is run in a copy of the current environment
(that cfn-hup is in), with CFN_OLD_METADATA set to the previous
value of path, and CFN_NEW_METADATA set to the current value.
• The hooks configuration file is loaded at cfn-hup daemon startup
only, so new hooks will require the daemon to be restarted.
• A cache of previous metadata values is stored at /var/lib/cfn-
hup/data/metadata_db
• We can delete this cache to force cfn-hup to run all post.add actions
again.

Kalyan Reddy Daida StackSimplify

Step 14: Create Stack using template
11-14-cfn-init-v14-Update-App.yml
• Observations
• Old war file will be removed
• New war file will be deployed successfully.
• When we access the app new version of application content will be
displayed.

Kalyan Reddy Daida StackSimplify

Configsets

Kalyan Reddy Daida StackSimplify

Configsets
• We can create more than one config key and have cfn-init process
them in a specific order.
• Single Configset
• Multiple Configset

Kalyan Reddy Daida StackSimplify

configSets (Single)
Sample:

App1AndApp2

App1 App2

Kalyan Reddy Daida StackSimplify

configSets (Multiple)

SingleAppCS DualAppCS default

App1 SingleAppCS App2 DualAppCS

App1 SingleAppCS App2

Sample: App1

Kalyan Reddy Daida StackSimplify

Step#1: Single Configset
• We will define two config keys App1 and Sample:
App2
• We will create a configSet with name as
App1AndApp2.
• First App1 config key will get executed.
• Next App2 config key will get executed.
• Order of execution will be based on how
we define them in configSets.
• Observation
• Both applications should be accessible

Kalyan Reddy Daida StackSimplify

Step#2: Multiple configSets
• We have created 3 configSets
Sample:
• SingleAppCS
• DualAppCS
• default
• SingleAppCS: Only App1 should be
deployed.

Kalyan Reddy Daida StackSimplify

Step#3: Multiple configSets
• We have created 3 configSets
Sample:
• SingleAppCS
• DualAppCS
• default
• DualAppCS: Both App1 and App2 should
be deployed

Kalyan Reddy Daida StackSimplify

Step#4: Multiple configSets
• We have created 3 configSets
Sample:
• SingleAppCS
• DualAppCS
• default
• default: default contains ConfigSet
DualAppCS so both apps should be
deployed. For default we don’t need to
specify “--configSets default” it will pick
automatically.

Kalyan Reddy Daida StackSimplify

Nested Stacks

Kalyan Reddy Daida StackSimplify

Nested Stacks
• The AWS::CloudFormation::Stack type nests a stack as a resource in a
top-level template.
• We can add output values from a nested stack within the root stack.
• We use Fn::GetAtt function with nested stacks logical name and the
name of the output value in nested stack
• Syntax:

Kalyan Reddy Daida StackSimplify

Nested Stacks – Practice – Create Templates
Step#0: Step#1: Step#2:

S3 Bucket VPC Nested Stack Template

Root Stack Template
• Create S3 bucket • Create Parameters
• Create Parameters
• This is required for • Create Metadata
• Create Resources
uploading the Nested • Create Resources
• Create VPC Stack
stack templates to S3 • Create VPC
• Create EC2 Instance
• Create Subnets
• Create Outputs
• Create Route Table
• Associate Subnet &
Route Table
• Create IGW
• Associate IGW to
VPC
• Create Route
• Create Outputs
• Test Template
• Upload to S3

Kalyan Reddy Daida StackSimplify

Nested Stacks – Practice – Create Stack
Root Stack
Step#3: For VPC Nested Stack
• Create Root Stack
• It automatically creates
the vpc nested stack

VPC Nested
Stack

Kalyan Reddy Daida StackSimplify

Nested Stacks – Practice – Create Templates
Step#4 Step#5
Security Group Nested Root Stack Template
Stack Template
• Create Parameters
• Create Parameters • Create Resources
• Create Resources • Create VPC Stack
• Create Security • Create EC2 Instance
Group • Create Outputs
• Create Outputs • Create Resource
• Create Security
Group Stack
• Update VM
Instance resource
with security group

Kalyan Reddy Daida StackSimplify

Nested Stacks – Practice – Update Stack
Root Stack
Step#6: For VPC Nested Stack For Security Group Nested Stack
• Create Root Stack • Update Root Stack with new
• It automatically creates template.
the vpc nested stack • It automatically creates the security
group nested stack

VPC Nested Security Group

Stack Nested Stack

Kalyan Reddy Daida StackSimplify

Nested Stacks – Practice – Update Stack 2
Root Stack
Step#7: For VPC Nested Stack For Security Group Nested Stack
• Create Root Stack • Update Root Stack with new
• It automatically creates template.
the vpc nested stack • It automatically creates the security
group nested stack
• Highly Recommended Approach
• Always perform updates from Root
Stack
• Never update nested stacks directly. For Nested Stack Updates
• Update SG nested
stack with new
security rule.
• Upload the new
template to S3
VPC Nested Security Group • Update Root Stack
Stack Nested Stack with existing
Update Stack template.

Kalyan Reddy Daida StackSimplify

Nested Stacks – Practice – Delete Stack
Root Stack
Step#8:
Always Delete Root Stack
• Recommendations
• Always delete root stack.
• Never delete nested stacks directly.
• Whenever we delete the root stack
associated nested stacks will get
deleted automatically.

VPC Nested Security Group

Stack Nested Stack

Kalyan Reddy Daida StackSimplify

Nested Stacks vs Outputs - Pending
• A nested stack is a stack that you create within another stack by using the
AWS::CloudFormation::Stack (p. 954) resource. With nested stacks, you
deploy and manage all resources from a single stack.
• You can use outputs from one stack in the nested stack group as inputs to
another stack in the group. This differs from exporting values.
• If you want to isolate information sharing to within a nested stack group,
we suggest that you use nested stacks. To share information with other
stacks (not just within the group of nested stacks), export values.
• For example, you can create a single stack with a subnet and then export its
ID. Other stacks can use that subnet by importing its ID; each stack doesn't
need to create its own subnet. Note that as long as stacks are importing the
subnet ID, you can't change or delete it.

Kalyan Reddy Daida StackSimplify

CodeCommit CodeBuild CodeDeploy CodePipeline CloudWatch Simple Notification Service Amazon EC2

Continuous Integration
&
Continuous Delivery

Kalyan Reddy Daida StackSimplify

Stages in Release Process

Source Build Test Production

• Check-in source • Compile Code & • Integration tests • Deployment to

• Deployment to
code build artifacts (war with other production
production
• Peer review new files) systems. environments
environments
code • Unit Tests • Load Testing • Monitor code in
• Monitor code in
• UI Tests production
productiontoto
• Security Tests quickly detect
quickly detect
errors
errors

Kalyan Reddy Daida StackSimplify

Stages in Release Process

Source Build Test Production

Kalyan Reddy Daida StackSimplify

Continuous Integration

Source Build Test Production

• Automatically kick off a new release when new code is checked-in

• Build and test code in a consistent, repeatable environment
• Continually have an artifact ready for deployment

Kalyan Reddy Daida StackSimplify

Continuous Delivery
Source Build Test Production

• Automatically deploy new changes to staging environments for testing

• Deploy to production safely without affecting customers
• Deliver to customers faster
• Increase deployment frequency, and reduce change lead time and change failure
rate
Kalyan Reddy Daida StackSimplify
AWS Developer Tools or Code Services
Source Build Test Deploy Monitor

AWS CodeCommit AWS CodeBuild AWS CodeBuild + AWS CodeDeploy Amazon

AWS X-Ray
Third Party CloudWatch

AWS CodePipeline
Kalyan Reddy Daida StackSimplify
AWS Developer Tools or Code Services
Source Build Test Deploy Monitor

AWS CodeCommit AWS CodeBuild AWS CodeBuild + AWS CodeDeploy Amazon

AWS X-Ray
Third Party CloudWatch

AWS CodePipeline
Kalyan Reddy Daida StackSimplify
AWS Developer Tools or AWS Code Services
Source Build Test Deploy Monitor

CodeCommit CodeBuild CodeBuild + Third Party CodeDeploy CloudWatch

AWS X-Ray
• Version control • Fully managed build service, Compiles • Automates code deployments • Monitors Source
service source code, Runs tests and produces to any instance and Lambda check-ins and triggers
• We can privately software packages • Avoids downtime during builds
store and manage • Scales continuously and processes multiple application deployment • Monitors builds
source code builds concurrently. • Roll back automatically if • Monitors
• Secure & highly • No build servers to manage. failure detected Infrastructure
available • Pay by minute, only for compute resources • Deploy to Amazon EC2, • Collects logs
we use. Lambda, or on-premises servers
• Monitor builds through CloudWatch events.
• Supports following programming language
• Continuous delivery service for fast and reliable
runtimes Ruby, Python, PHP, Node, Java,
application updates
Golang, .Net Core, Docker and Android • Model and visualize your software release process
• Builds, tests, and deploys your code every time there
is a code change
CodePipeline • Integrates with third-party tools and AWS

Kalyan Reddy Daida StackSimplify

CodeCommit
• Build a simple rest service using Java Spring Developer

Boot.
• Check-in code to Local Repo and push to Local Git
CodeCommit. Repo

push

AWS Cloud

AWS CodeCommit

Kalyan Reddy Daida StackSimplify

CodeBuild – AWS Web Console CodeBuild – AWS CloudFormation

Developer Developer

Local Git
Local Git Repo
Repo push
AWS Cloud
push
CodeBuild Stack
AWS Cloud

CodeBuild
CodeCommit CodeCommit Service Role

CodeBuild Simple Storage CodeBuild Simple Storage

Service (S3) Service (S3)

Kalyan Reddy Daida StackSimplify

CodeDeploy – AWS Web Console CodeDeploy – AWS CloudFormation

User
Developer Developer accessing
User Rest service
accessing
Rest service

Local Git Local Git Internet

Repo Repo

Internet push
push
AWS Cloud

CodeDeploy Stack EC2 Apps Stack

AWS Cloud

CodeCommit CodeBuild CodeDeploy Role for EC2

CodeCommit
Service Role Service Role Instance ProfileStaging EC2 Instance

CodeBuild Simple Storage CodeDeploy EC2 Instance CodeBuild Simple Storage CodeDeploy Production EC2
Service (S3) Service (S3) Instance

Kalyan Reddy Daida StackSimplify

CodePipeline – AWS Web Console
AWS Cloud

CodePipeline
Developer CloudWatch

Local Git push

Repo
CodeCommit CodeBuild Simple Storage CodeDeploy EC2
Service (S3) Instance

Kalyan Reddy Daida StackSimplify

CodePipeline – AWS CloudFormation
AWS Cloud

CodePipeline CI CD CloudFormation Stack

Developer

Local Git push

Repo
CodeCommit CodeBuild Simple Storage CodeDeploy Simple Notification CodeDeploy
Service (S3) Service

Staging EC2 Role for EC2 Production EC2

Instance Profile Authorized
Instance Instance Approver
EC2 Apps CloudFormation Stack

Kalyan Reddy Daida StackSimplify

Pre-requisites
• Region: us-east-2 (ohio)
• In templates, EC2 Instnace ImageID is hardcoded to this region (Amazon
Linux AMI). If you want to test in other regions, please update the templates
with ImageId equivalent to that respective region.
• Default VPC
• Ensure we have the default VPC created in the region where we are using
these templates.

Kalyan Reddy Daida StackSimplify

EC2 CloudFormation
Stack

Kalyan Reddy Daida StackSimplify

EC2 CloudFormation Stack
• Step 1: Create Security Group
with port 22 and 8080 rules for
AWS Cloud
inbound access.
VPC Default VPC
• Step 2: Create two EC2 Instances Security group Port 22 & 8080
• Staging
• Production
• Update UserData Staging EC2 Role for EC2 Production EC2

• Step 3: Create Instance Profile Instance Instance Profile Instance

Role and Instance Profile for EC2 EC2 Apps CloudFormation Stack
Instances to access S3 Buckets.
• Step 4: Create stack and verify.

Kalyan Reddy Daida StackSimplify

AWS CodeCommit

Kalyan Reddy Daida StackSimplify

AWS CodeCommit - Introduction
• Version Control Service hosted by AWS
• We can privately store and manage documents, source code, and
binary files
• Secure & highly scalable
• Supports standard functionality of Git (CodeCommit supports Git
versions 1.7.9 and later.)
• Uses a static user name and password in addition to standard SSH..

Kalyan Reddy Daida StackSimplify

CodeCommit – Integration with AWS Services
AWS CodeStar AWS CodeBuild AWS CodePipeline AWS Cloud9 AWS Amplify AWS CloudFormation

AWS CodeCommit

Amazon Simple Notification AWS Key Management AWS Elastic Beanstalk Amazon CloudWatch
AWS CloudTrail
Service Service
Kalyan Reddy Daida StackSimplify
CodeCommit - Steps
• Step#1: Sample Spring Boot Rest Application Developer

• Pre-requisites
• Install STS IDE
• Create Spring boot rest application. Local
• Test it. Git
• Step#2: GIT Repository Repo
• Create a local git repository and check-in code.
• Create a remote git repository in AWS Code Commit. push
• Create Code Commit git credentials to connect.
• Push the code to remote git repository.
• Verify code in AWS Code Commit.
AWS Cloud
• Step#3: CodeCommit Features
• Code, Commits, Branches
• Settings: Notifications, Triggers
• Pull Requests
AWS CodeCommit

Kalyan Reddy Daida StackSimplify

AWS CodeBuild

Kalyan Reddy Daida StackSimplify

CodeBuild – AWS Web Console CodeBuild – AWS CloudFormation

Developer Developer

Local Git
Local Git Repo
Repo push
AWS Cloud
push
CodeBuild Stack
AWS Cloud

CodeBuild
CodeCommit CodeCommit Service Role

CodeBuild Simple Storage CodeBuild Simple Storage

Service (S3) Service (S3)

Kalyan Reddy Daida StackSimplify

CodeBuild - Introduction
• CodeBuild is a fully managed build service in the cloud.
• Compiles your source code, runs unit tests, and produces artifacts
that are ready to deploy.
• Eliminates the need to provision, manage, and scale your own build
servers.
• It provides prepackaged build environments for the most popular
programming languages and build tools such as Apache Maven,
Gradle, and more.
• We can also customize build environments in CodeBuild to use
ourown build tools.
• Scales automatically to meet peak build requests.

Kalyan Reddy Daida StackSimplify

How to run CodeBuild? How CodeBuild works?

Kalyan Reddy Daida StackSimplify

Source
Amazon Simple Storage
AWS CodeCommit
Service (S3) GitHub GitHub Enterprise Bitbucket

Managed Image

AWS CodeBuild Amazon CloudWatch

Logs

Amazon EC2 Container

Registry

Amazon Simple Notification

Service

External Amazon Simple Storage Notifications

Container Service (S3)
Registry
Environment Artifacts AWS CodeBuild Architecture
Kalyan Reddy Daida StackSimplify
CodeBuild - Steps Developer

Local Git
• Step#1: Create CodeBuild Project Repo
• Create a S3 bucket and folder
push
• Create CodeBuild project
• Start build, Verify build logs, Verify build
phase details
• Step#2: buildspec.yml & Start Build
• Create buildspec.yml and check-in code AWS Cloud

• Start build, Verify build logs, Verify build

phase details
• Download the artifacts from S3, unzip and AWS CodeCommit
review
• Run one more build and see versioning in S3.
• Step#3: Create Build Notifications
• Create state change notification
• Create Phase change notification AWS CodeBuild Simple Storage
Service (S3)

Kalyan Reddy Daida StackSimplify

AWS CodeBuild
using
CloudFormation

Kalyan Reddy Daida StackSimplify

CodeBuild – CloudFormation Steps
Developer

• Step 1 : Create S3 bucket and enable versioning

or use existing bucket. Local Git
Repo
• Step 2: Create buildspec.yml in our rest push
application and check-in code AWS Cloud

• Step 3: Create CodeBuild Stack Template CodeBuild Stack

• Create CodeBuild Role.
• Create CodeBuild project.
• Parameters CodeBuild
CodeCommit
• Step 4: Create Stack and Test the build. Service Role

• Click on Start Build

• Verify logs
• Verify artifacts in S3. CodeBuild Simple Storage
Service (S3)

Kalyan Reddy Daida StackSimplify

AWS CodeDeploy

Kalyan Reddy Daida StackSimplify

Compute Platform
CodeDeploy - Introduction
• CodeDeploy is a deployment service that Amazon EC2
automates application deployments to
• EC2 instances
• On-premises instances
• AWS Lambda
EC2 Auto Scaling
• AWS ECS
• We can deploy unlimited variety of application
content
• code AWS CodeDeploy
On-Premise
• serverless AWS Lambda functions
• web and configuration files
• executables
• packages AWS Lambda
• scripts
• multimedia files

Amazon ECS

Kalyan Reddy Daida StackSimplify

CodeDeploy - Introduction
• Benefits
• We can rapidly release new features.
• Update AWS Lambda function versions.
• Avoid downtime during application deployment.
• Reduces the complexity of updating applications when compared to error-
prone manual deployments.
• Service scales with our infrastructure so we can easily deploy to one instance
or thousands.

Kalyan Reddy Daida StackSimplify

CodeDeploy - When compute is EC2/On-Premise
Compute Platform

Revision Type / Source

Amazon EC2

Amazon Simple Storage

Service (S3) EC2 Auto
Scaling
AWS CodeDeploy

GitHub On-Premise

Elastic Load
Balancing (ELB)

Kalyan Reddy Daida StackSimplify

CodeDeploy - Steps Developer
• Step#1: Create CodeDeploy pre-requisite roles
• Create a service role for codeDeploy.
• Create an IAM Instance profile. User
• Step#2: Create a EC2 VM Local Git accessing
• Create EC2 VM Repo Rest service
• During creation associate IAM instance profile.
• Discuss about “Userdata” containing tomcat and codeDeploy Agent push
• Step#3: Create codeDeploy objects
• Create Application Internet
• Create Deployment Group
• Create Deployment
• Step#4: Create codeDeploy files and scripts AWS Cloud
• Create appspec.yml
• Create scripts (before_install script, after_install script, Start up
script, Shutdown script) and check-in
• Step#5: Run CodeBuild and Create Deployment AWS CodeCommit
• Step#6: Verify Deployment
• Verify the deployment Events
• Verify the tomcat deployment
• Verify the codeDeploy agent log
• Verify by accessing app
AWS CodeBuild Simple Storage AWS CodeDeploy Amazon EC2
• Step#7: New App Release: Make change to Application and re-
deploy Service (S3) Instance

Kalyan Reddy Daida StackSimplify

AWS CodeDeploy
using
CloudFormation

Kalyan Reddy Daida StackSimplify

CodeDeploy – CloudFormation Steps User
accessing
Rest service
Developer
• Step 1 : Discuss about appspec.yml and
scripts.
Local Git Internet
• Step 2: Create CodeDeploy service role Repo
• Step 3: Create CodeDeploy Application push
• Step 4: Create CodeDeploy Deployment AWS Cloud
Group and also change CodeBuild packaging
to ZIP. CodeDeploy Stack EC2 Apps Stack
• Step 5:Create stack and verify the following
• Application
• Deployment Group
• Step 6: Create CodeDeploy Deployment CodeCommit
CodeBuild CodeDeploy Role for EC2
• Create Deployment Object
Service Role Service Role Instance ProfileStaging EC2 Instance

• Run CodeBuild and Verify S3 for ZIP

• Update Stack
• Verify Deployment
• Access Application CodeBuild Simple Storage CodeDeploy Production EC2
Service (S3) Instance

Kalyan Reddy Daida StackSimplify

AWS CodePipeline

Kalyan Reddy Daida StackSimplify

AWS CodePipeline

AWS CodeCommit AWS CloudFormation Amazon Elastic Container

Service

AWS CodeBuild Amazon CloudWatch

Amazon EC2 Container
Registry AWS CodeDeploy
Amazon Elastic Container
Service (Blue/Green)

Simple Storage AWS Elastic Beanstalk

Service (S3)
Jenkins GitHub Webhooks
Simple Storage
Service (S3)
AWS Service Catalog
GitHub

Source Build Deploy Monitor Source Changes

Kalyan Reddy Daida StackSimplify
Continuous Delivery

Kalyan Reddy Daida ©Amazon StackSimplify

CodePipeline - Introduction
• AWS CodePipeline is a continuous delivery service to model,
visualize, and automate the steps required to release your software.
• Benefits
• Automate your release processes.
• Establish a consistent release process.
• Speed up delivery while improving quality.
• Supports external tools integration for source, build and deploy.
• View progress at a glance
• View pipeline history details.

Kalyan Reddy Daida StackSimplify

CodePipeline - Steps
• Step#1: Create
Pipeline AWS Cloud
• Artifacts: S3 AWS CodePipeline
• Source: CodeCommit Developer Amazon CloudWatch
• Build: CodeBuild
• Deploy: CodeDeploy
• Server: EC2 Instance
• Step#2: Make
changes & Check-In
Code
• Make changes to rest Local Git push
app and check-in Repo
• Pipeline should AWS CodeCommit AWS CodeBuild Simple Storage AWS CodeDeploy Amazon EC2
trigger the build Service (S3) Instance
automatically.

Kalyan Reddy Daida StackSimplify

CodePipeline – Manual Approval & Prod Deployment

• Step#1: Create new EC2 Instance with tag name as prod

• Step#2: Create new deployment group for prod
• Step#3: Create Manual Approval stage in CodePipeline
• Step#4: Create Prod Deployment stage in CodePipeline .
• Step#5: Check-in changed code to trigger pipeline and monitor the
pipeline process.

Kalyan Reddy Daida StackSimplify

AWS CodePipeline
using
CloudFormation

Kalyan Reddy Daida StackSimplify

Continuous Integration & Continuous Delivery using CloudFormation
AWS Cloud

CodePipeline CI CD CloudFormation Stack

Developer

Local Git push

Repo
CodeCommit CodeBuild Simple Storage CodeDeploy Simple Notification CodeDeploy
Service (S3) Service

Staging EC2 Production EC2 Authorized

Instance Instance Approver
EC2 Apps CloudFormation Stack

Kalyan Reddy Daida StackSimplify

CodePipeline – CloudFormation Steps
• Step 1: Create CodePipeline role
• Step 2: Create Pipeline stages for staging deployment
• Stage 1: Source Stage
• Stage 2: Build Stage
• Stage 3: Deploy To Staging
• Step 3: Create stack and verify the following
• Stages: Source, Build, Deploy to Staging
• Access Application in staging
• Step 4: From IDE make changes to rest app and check-in code and verify the
following
• Stages: Source, Build, Deploy to Staging
• Access Application in staging

Kalyan Reddy Daida StackSimplify

CodePipeline – CloudFormation Steps
• Step 5: Create SNS Topic and its equivalent parameter and add Production DeploymentGroup
• Step 6: Create Pipeline stages for Production deployment
• Stage 4: Production email Approval
• Stage 5: Deploy To Production
• Step 7: Create stack and verify the following
• Confirm SNS Subscription in email
• Stages: Source, Build, Deploy to Staging, Production email approval and Deploy to
production.
• Access Application in staging and production
• Step 8: From IDE make changes to rest app and check-in code and verify the
following
• Stages: Source, Build, Deploy to Staging, Production email approval and Deploy to
production.
• Access Application in staging and production

Kalyan Reddy Daida StackSimplify

Infrastructure as Code
using
AWS CloudFormation
and
AWS Web Console

Kalyan Reddy Daida StackSimplify

Infrastructure as Code

Source Build Test Production

Source Stage Build Stage Test Stage Prod Stage

AWS CodeCommit AWS CodeBuild AWS CodePipeline AWS CodePipeline

Kalyan Reddy Daida StackSimplify

Infrastructure as Code

Source Build Test Production

Source Stage Build Stage Test Stage Prod Stage

Master Branch Prepare or Validate Create & Execute Create & Execute
Template Change set Change set

Kalyan Reddy Daida StackSimplify

Infrastructure as Code
Source Build Test Production

Benefits
• Track Infrastructure changes using version control system like AWS CodeCommit.
• Releaseinfrastructure changes using the same tools ascode changes (AWSCodeCommit, CodeBuild and CodePipeline).
• Replicateproduction environment in any environment as desired for continuous testing.
• Make infrastructure changes repeatable.
• Minimize infrastructure buildout time.
• Seamless provisioning and de-provisioning of infrastructure resources in minutes or even reduced to seconds.
Kalyan Reddy Daida StackSimplify
Infrastructure as Code – Manual AWS Web Console
AWS Cloud

Staging VPC Stack

CodePipeline Action-1:
VPC Subnet
Test Stage Create
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set
CodeBuild CodePipeline CloudFormation Internet
Service Role Role Role Route table gateway

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – CFN Template creation Flow
AWS Cloud

CI CD IAC Pipeline Stack Staging VPC Stack

CodePipeline Action-1:
Create VPC Subnet
Test Stage
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set

CodeBuild CodePipeline CloudFormation Internet

Role Route table gateway
Service Role Role

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – Execution Flow
AWS Cloud

CI CD IAC Pipeline Stack Staging VPC Stack

CodePipeline Action-1:
Test Stage Create VPC Subnet
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set
CodeBuild CodePipeline CloudFormation Internet
Service Role Role Role Route table gateway

Kalyan Reddy Daida StackSimplify

Infrastructure as Code
using
AWS Developer Tools
on
AWS Web Console

Kalyan Reddy Daida StackSimplify

Usecase
• We will create a VPC CloudFormation stack using AWS CodePipeline.
• Manage the AWS VPC infrastructure components like Subnets, Routes,
Route Tables everything with AWS Developer Tools and CloudFormation.
• Any changes to vpc infra, we will change the vpc.yml cfn template and
check-in the code to AWS CodeCommit.
• CodePipeline will trigger pipeline and push the changes to staging VPC
Stack.
• CodePipeline creates a Change Set for production.
• Approve the Change set using SNS notification
• Changes will be pushed to production VPC Stack after approval.
• Finally we will achieve Continuous Integration, Continuous Delivery &
Infrastructure as code after this usecase implementation.

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – Manual AWS Web Console
AWS Cloud

Staging VPC Stack

CodePipeline Action-1:
VPC Subnet
Test Stage Create
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set
CodeBuild CodePipeline CloudFormation Internet
Service Role Role Role Route table gateway

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – Manual AWS Web Console
• Step 1: Understand about the VPC source files.
• vpc.yml
• vpc-config.json
• buildspec.yml
• Step 2: Create CodeCommit repository vpcrepo and check-in vpc source files.
• Step 3: Create a pipeline with source and build stages.
• Build stage fails and we will understand the causes of failure and fix it.
• Create a CFN policy with validate CFN template permission and associate to CodeBuildRole.
• Rerun the pipeline.
• Build stage should pass now.
• Step 4: Create a Test Stage which creates TestVPCStack
• Role#1: Create CFN full access policy and associate it with CodePipeline role
• Role#2: Create CFN VPC full access Role by associating “VPC Full Access policy” which is required by
CloudFormation to create the VPC Stack.
• Create Pipeline stage named VPCTest
• Click “Release Change” after stage creation and verify the stack got created in CFN Console.

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – Manual AWS Web Console
• Step 5: Create Prod Stage
• Pre-requisite: Create SNS Topic
• Action#1: Create Prod Change Set
• Action#2: Create Prod Approval
• Action#3: Create Execute Change Set
• Verify the stack got created in CloudFormation console.
• Step 8: Update vpc.yml with new subnet (subnet02) and check-in file to
CodeCommit
• Verify the pipeline stages
• Source, Build, VPCStage
• VPCProd
• Create Change Set
• Prod Approval
• Execute Change Set
• Verify the same in VPC

Kalyan Reddy Daida StackSimplify

Infrastructure as Code
using
AWS CloudFormation

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – CFN Template creation Flow
AWS Cloud

CI CD IAC Pipeline Stack Staging VPC Stack

CodePipeline Action-1:
Create VPC Subnet
Test Stage
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set

CodeBuild CodePipeline CloudFormation Internet

Role Route table gateway
Service Role Role

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – Execution Flow
AWS Cloud

CI CD IAC Pipeline Stack Staging VPC Stack

CodePipeline Action-1:
Test Stage Create VPC Subnet
Stack
Developer
Internet
Prod Stage Route table gateway

Source Stage
Prod VPC Stack
Build Stage Action-1: Create
Change set
Local Git push Prod
Action-2:
Repo Approval

CodeCommit CodeBuild Simple Storage Simple Notification

Service (S3) Authorized
Service Action-3: Approver
Execute VPC Subnet
Change set
CodeBuild CodePipeline CloudFormation Internet
Service Role Role Role Route table gateway

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – CloudFormation Pipeline
• Step 1: Understand about the VPC source files.
• vpc.yml
• vpc-config.json
• buildspec.yml
• Step 2: Create CodeCommit repository vpcrepo and check-in vpc source
files. (Note: check-in base vpc.yml and vpc-config.json)
• Step 3: Create a CodeBuild related template objects
• Create Parameters (Repo Name, Artifact storage bucket)
• Create CodeBuild Role
• Create CodeBuild Project
• Step 4: Create Other roles
• Create CodePipeline Role
• Create CloudFormation Role

Kalyan Reddy Daida StackSimplify

Infrastructure as Code – CloudFormation Pipeline

• Step 5: Crete Pipeline stages

• Source Stage
• Build Stage
• Test Stage
• Step 6: Create Pipeline stage and actions for production
• Create SNS Topic Resource
• Create Parameter for email notifications.
• Prod Stage
• Action-1: Create Change Set
• Action-2: Prod Approval
• Action-3: Execute Change Set
• Step 7: Add subnet02 in vpc.yml and verify the pipeline end to end.

Kalyan Reddy Daida StackSimplify

Thank You

Kalyan Reddy Daida StackSimplify