UNIT – V

Cloud Application Development: Amazon Web Services : EC2 – instances, connecting

clients, security rules, launching, usage of S3 in Java, Installing Simple Notification Service
on Ubuntu 10.04, Installing Hadoop on Eclipse, Cloud based simulation of a Distributed trust
algorithm, Cloud service for adaptive data streaming ( Text Book 1) Google: Google App
Engine, Google Web Toolkit (Text Book 2)

Microsoft: Azure Services Platform, Windows live, Exchange Online, Share Point
Services, Microsoft Dynamics CRM (Text Book 2).

 Amazon Web Services: EC2 Instances

Figure 11.2 displays the Amazon Management Console (AMC) window listing the
Amazon Web Services offered at the time of this writing.
The services are grouped into several categories: computing and networking,
storage and content delivery, deployment and management, databases, and
application services.

In spite of the wealth of information available from the providers of cloud services,
the learning curve of an application developer is still relatively steep.
To access AWS one must first create an account at http://aws.amazon.com/.
Once the account is created, the AMC allows the user to select one of the services,
e.g., EC2, and then start an instance.
Recall that an AWS EC2 instance is a virtual server started in a region and the
availability zone is selected by the user.

1
Instances are grouped into a few classes, and each class has available to it a
specific amount of resources, such as: CPU cycles, main memory, secondary
storage, and communication and I/O bandwidth.
Several operating systems are supported by AWS, including
o Amazon Linux
o Red Hat Enterprise Linux, 6.3
o SUSE Linux Enterprise Server 11
o Ubuntu Server 12.04.1
o Several versions of Microsoft Windows.
The next step is to create an (AMI) on one of the platforms supported by AWS and
start an instance using the RunInstance API.
If the application needs more than 20 instances, a special form must be filled out.
The local instance store persists only for the duration of an instance; the data will
persist if an instance is started using the Amazon Elastic Block Storage (EBS) and
then the instance can be restarted at a later time.
Once an instance is created, the user can perform several actions – for example,
connect to the instance, launch more instances identical to the current one, or
create an EBS AMI.
The user can also
o Terminate
o Reboot, or
o Stop the instance
The Network & Security panel allows the creation of Security Groups, Elastic IP
addresses, Placement Groups, Load Balancers, and Key Pairs, whereas the EBS
panel allows the specification of volumes and the creation of snapshots.

Connecting Clients to Cloud Instances through Firewalls :

A firewall is a software system based on a set of rules for filtering network traffic.
Its function is to protect a computer in a local area network from unauthorized
access.
The first generation of firewalls,

2
o Deployed in the late 1980s, carried out packet filtering; they discarded
individual packets that did not match a set of acceptance rules.
o Such firewalls operated below the transport layer and discarded packets
based on the information in the headers of physical, data link, and transport
layer protocols.
The second generation of firewalls,
o Operate at the transport layer and maintain the state of all connections
passing through them.
o Unfortunately, this traffic-filtering solution opened the possibility of
denial-of-service (DoS) attacks.
o A DoS attack targets a widely used network service and forces the
operating system of the host to fill the connection tables with illegitimate
entries.
o DoS attacks prevent legitimate access to the service.
The third generation of firewalls,
o “understand” widely used application layer protocols such as FTP, HTTP,
TELNET, SSH, and DNS.
o These firewalls examine the header of application layer protocols and
support Intrusion Detection Systems (IDSs).
Firewalls screen incoming traffic and sometimes filter outgoing traffic as well.
A first filter encountered by the incoming traffic in a typical network is a firewall
provided by the operating system of the router; the second filter is a firewall
provided by the operating system running on the local computer.
Typically, the Local Area Network (LAN) of an organization is connected to the
Internet via a router.
A router firewall often hides the true address of hosts in the local network using the
Network Address Translation (NAT) mechanism. The hosts behind a firewall are
assigned addresses in a “private address range,” and the router uses the NAT tables
to filter the incoming traffic and translate external IP addresses to private ones.

How to launch an EC2 Linux instance and connect to it

A step-by-step process to launch an EC2 Linux instance from a Linux platform.
Launch an instance

3
1. From the AWS Management Console, select EC2 and, once signed in, go to
Launch Instance Tab.
2. To determine the processor architecture when you want to match the
instance with the hardware, enter the command
uname –m
and choose an appropriate Amazon Linux AMI by pressing Select.
3. Choose Instance Details to control the number, size, and other settings for
instances.
4. To learn how the system works, press Continue to select the default
settings.
5. Define the instance’s security, as discussed in Section 11.3: In the Create
Key Pair page enter a name for the pair and then press Create and
Download Key Pair.
6. The key-pair file downloaded in the previous step is a .pem file, and it must
be hidden to prevent unauthorized access. If the file is in the directory
awcdir/dada.pem enter the commands
cd awcdir
chmod 400 dada.pem
7. Configure the firewall. Go to the page Configure firewall, select the option
Create a New Security Group, and provide a Group Name. Normally we
use ssh to communicate with the instance; the default port for
communication is port 8080, and we can change the port and other rules by
creating a new rule.
8. Press Continue and examine the review page, which gives a summary of
the instance.
9. Press Launch and examine the confirmation page, then press Close to end
the examination of the confirmation page.
10. Press the Instances tab on the navigation panel to view the instance.
11. Look for your Public DNS name. Because by default some details of the
instance are hidden, click on the Show/Hide tab on the top of the console
and select Public DNS.
12. Record the Public DNS as PublicDNSname; it is needed to connect to the
instance from the Linux terminal.

4
13. Use the ElasticIP panel to assign an Elastic IP address if a permanent IP
address is required.

Connect to the instance using ssh and the TCP transport protocol.
1. Add a rule to the iptables to allow ssh traffic using the TCP protocol. Without
this step, either an access denied or permission denied error message appears
when you’re trying to connect to the instance.
sudo iptables -A iptables -p -tcp -dport ssh -j ACCEPT
2. Enter the Linux command:
ssh -i abc.pem ec2-user@PublicDNSname
If you get the prompt You want to continue connecting? respond Yes. A
warning that
the DNS name was added to the list of known hosts will appear.
3. An icon of the Amazon Linux AMI will be displayed.

Gain root access to the instance

 By default the user does not have root access to the instance; thus, the user
cannot install any software. Once connected to the EC2 instance, use the
following command to gain root privileges:
sudo -i
Then use yum install commands to install software, e.g., gcc to compile C
programs on the cloud.

Run the service ServiceName

• If the instance runs under Linux or Unix, the service is terminated when the
ssh connection is closed. To avoid the early termination, use the command
nohup ServiceName
• To run the service in the background and redirect stdout and stderr to files
p.out and p.err, respectively, execute the command
nohup ServiceName > p.out 2 > p.err &

How to use S3 in Java

The Java API for Amazon Web Services is provided by the AWS SDK.3

5
Create an S3 client. S3 access is handled by the class AmazonS3Client instantiated
with the account credentials of the AWS user:
AmazonS3Client s3 = new AmazonS3Client(
new BasicAWSCredentials("your_access_key",
"your_secret_key"));

The access and the secret keys can be found on the user’s AWS account homepage.

Buckets. An S3 bucket is analogous to a file folder or directory, and it is used to

store S3 objects. Bucket names must be globally unique; hence, it is advisable to
check first to see whether the name exists:
s3.doesBucketExist("bucket_name");
This function returns “true” if the name exists and “false” otherwise. Buckets can
be created and deleted either directly from the AWS Management Console or
programmatically as follows:
s3.createBucket("bucket_name");
s3.deleteBucket("bucket_name");
S3 objects. An S3 object stores the actual data and it is indexed by a key string. A
single key points to only one S3 object in one bucket. Key names do not have to be
globally unique, but if an existing key is assigned to a new object, the original
object indexed by that key is lost. To upload an object in a bucket, we can use the
AWS Management Console or, programmatically, a file local_ f ile_name can be
uploaded from the local machine to the bucket bucket_name under the key key
using
File f = new File("local_file_name");
s3.putObject("bucket_name", "key", f);
A versioning feature for the objects in S3 was made available recently; it allows us
to preserve, retrieve, and restore every version of an S3 object. To avoid problems
in uploading large files, e.g., dropped connections, use the
.initiateMultipartUpload() with an API described at the AmazonS3Client. To
access this object with key key from the bucket bucket_name use:
S3Object myFile = s3.getObject("bucket_name", "key");
To read this file, you must use the S3Object’s InputStream:
InputStream in = myFile.getObjectContent();

6
The InputStream can be accessed using Scanner, BufferedReader, or any other
supported method. Amazon recommends closing the stream as early as possible,
since the content is not buffered and it is streamed directly from the S3. An open
InputStream means an open connection to S3. For example, the following code will
read an entire object and print the contents to the screen:
AmazonS3Client s3 = new AmazonS3Client(
new BasicAWSCredentials("access_key", "secret_key"));
InputStream input = s3.getObject("bucket_name", "key")
.getObjectContent();
Scanner in = new Scanner(input);
while (in.hasNextLine())

{
System.out.println(in.nextLine());
}
in.close();
input.close();
Batch upload/download. Batch upload requires repeated calls of s3.putObject()
while iterating over local files.
To view the keys of all objects in a specific bucket, use
ObjectListing listing = s3.listObjects("bucket_name");
ObjectListing supports several useful methods, including getObjectSummaries().
S3ObjectSummary encapsulates most of an S3 object properties (excluding the
actual data), including the key to access the object directly,
List<S3ObjectSummary> summaries =
listing.getObjectSummaries();

How to install the Simple Notification Service on Ubuntu 10.04

Ubuntu is an open-source operating system for personal computers based on
Debian Linux distribution; the desktop version of Ubuntu4 supports the Intel x86
32-bit and 64-bit architectures.
The Simple Notification Service (SNS) is a Web service for: monitoring
applications, workflow systems, time-sensitive information updates, mobile

7
applications, and other event-driven applications that require a simple and efficient
mechanism for message delivery.
SNS “pushes” messages to clients rather than requiring a user to periodically poll a
mailbox or another site for messages.
SNS is based on the publish/subscribe paradigm; it allows a user to define the
topics, the transport protocol used (HTTP/HTTPS, email, SMS, SQS), and the
endpoint (URL, email address, phone number, SQS queue) for notifications to be
delivered. It supports the following actions:
 Add/Remove Permission.
 Confirm Subscription.
 Create/Delete Topic.
 Get/Set Topic Attributes.
 List Subscriptions/Topics/Subscriptions by Topic.
 Publish/Subscribe/Unsubscribe.
To install the SNS client the following steps must be taken:
1. Install Java in the root directory and then execute the commands:
deb http://archive.canonical.com/lucidpartner
update
install sun-java6-jdk
Then change the default Java settings:
update-alternatives -config java
2. Download the SNS client, unzip the file, and change permissions:
wget http://sns-public-resources.s3.amazonaws.com/
SimpleNotificationServiceCli-2010-03-31.zip
chmod 775 /root/ SimpleNotificationServiceCli-1.0.2.3/bin
3. Start the AWS Management Console and go to Security Credentials. Check
the Access Key ID and the Secret Access Key and create a text file
/root/credential.txt with the following content:
AWSAccessKeyId= your_Access_Key_ID
AWSSecretKey= your_Secret_Access_Key
4. Edit the .bashrc file and add:
export AWS_SNS_HOME=˜/SimpleNotificationServiceCli-1.0.2.3/
export AWS_CREDENTIAL_FILE=$HOME/credential.txt

8
export PATH=$AWS_SNS_HOME/bin
export JAVA_HOME=/usr/lib/jvm/java-6-sun/
5. Reboot the system.
6. Enter on the command line:
sns.cmd
If the installation was successful, the list of SNS commands will be displayed.

How to install Hadoop on Eclipse on a Windows system

The software packages used are:
• Eclipse (www.eclipse.org) is a software development environment that
consists of an integrateddevelop ment environment (IDE) and an extensible
plug-in system. It is written mostly in Java and can be used to develop
applications in Java and, by means of various plug-ins, in C, C++, Perl,
PHP, Python, R, Ruby, and several other languages. The IDE is often called
Eclipse CDT for C/C++, Eclipse JDT for Java, and Eclipse PDT for PHP.
• Apache Hadoop is a software framework that supports data-intensive
distributed applications under a free license. Hadoop was inspired by
Google’s MapReduce.
• Cygwin is a Unix-like environment for MicrosoftWindows. It is open-
source software released under the GNU General Public License version 2.
The cygwin environment consists of a Dynamic-Link Library (DLL) as an
API compatibility layer providing a substantial part of the POSIX API
functionality; and an extensive collection of software tools and
applications that provide a Unix-like look and feel.
Prerequisites
• Java 1.6; set JAVA_Home = path where JDK is installed
• Eclipse Europa 3.3.2
SSH Installation
1. Install cygwin using the installer downloaded from www.cygwin.com.
From the Select Packages window, select the openssh and openssl under
Net.
Note: Create a desktop icon when asked during installation.
2. Display the “Environment Variables” panel:

9
Computer -> System Properties -> Advanced System Settings ->
Environment Variables
Click on the variable named Path and press Edit; append the
following value to the path variable:
;c:\cygwin\bin;c:\cygwin\usr\bin
3. Configure the ssh daemon using cygwin. Left-click on the cygwin icon on
the desktop and click “Run as Administrator.” Type in the command
window of cygwin:
ssh-host-config.

4. Answer “Yes” when prompted with sshd should be installed as a service;

answer “No” to all other questions.
5. Start the cygwin service by navigating to:
Control Panel -> Administrative Tools -> Services
Look for cygwin sshd and start the service.
6. Open the cygwin command prompt and execute the following command to
generate keys:
ssh-keygen
7. When prompted for filenames and passphrases, press Enter to accept
default values. After the command has finished generating keys, enter the
following command to change into your .ssh directory:
cd˜.ssh
8. Check to see whether the keys were indeed generated:
ls -l
9. The two files id_rsa.pub and id_rsa with recent creation dates contain
authorization keys.
10. To register the new authorization keys, enter the following command (note:
the sharply-angled double brackets are very important):
cat id_rsa.pub _ authorized_keys
11. Check to see whether the keys were set up correctly:
ssh localhost
12. Since it is a new ssh installation, you will be warned that authenticity of the
host could not be established and will be asked whether you really want to

10
connect. Answer Yes and press Enter. You should see the cygwin prompt
again, which means that you have successfully connected.
13. Now execute again the command:
ssh localhost
This time no prompt should appear.

Download Hadoop
• Download Hadoop 0.20.1 and place it in a directory such as:
C:Java
• Open the cygwin command prompt and execute:
cd
• Enable the home directory folder to be shown in the Windows Explorer
window:
Explorer

• Open another Windows Explorer window and navigate to the folder that
contains the downloaded
Hadoop archive.
• Copy the Hadoop archive into the home directory folder.

Unpack Hadoop
• Open a new cygwin window and execute:
tar -xzf hadoop-0.20.1.tar.gz
• List the contents of the home directory:
ls -l
• You should see a newly created directory called Hadoop-0.20.1. Execute:
cd hadoop-0.20.1
ls -l
Cloud-Based Simulation of A Distributed Trust Algorithm

The actual meaning of trust is domain and context specific.

Consider, for example, networking; at the MAC layer the multiple-access protocols
assume that all senders follow the channel access policy, e.g., in Carrier Sense
Multiple Access with Collision Detection (CSMA-CD) a sender senses the channel
and then attempts to transmit if no one else does. In a store-and-forward network,
11
trust assumes that all routers follow a best-effort policy to forward packets toward
their destination.
Cognitive Radio Networks. Research over the last decade reveals a significant
temporal and spatial underutilization of the allocated spectrum. Thus, there is a
motivation to opportunistically harness the vacancies of spectrum at a given time
and place.
The original goal of cognitive radio, first proposed at Bell Labs, was to develop a
software based radio platform that allows a reconfigurable wireless transceiver to
automatically adapt its communication parameters to network availability and to
user demands. Today the focus of cognitive radio is on spectrum sensing.
Two types of devices connected to a CRN:
 Primary
 Secondary.
Primary nodes/devices have exclusive rights to specific regions of the spectrum;
Secondary nodes/devices enjoy dynamic spectrum access and are able to use a
channel, provided that the primary, licensed to use that channel, is not
communicating.
Once a primary starts its transmission, the secondary using the channel is required
to relinquish it and identify another free channel to continue its operation. This
mode of operation is called an overlay mode.

Simulation of the Distributed Trust Algorithm.

The cloud application is a simulation of a CRN to assess the effectiveness of a
particular trust assessment algorithm.
Multiple instances of the algorithm run concurrently on an AWS cloud. The area
where the secondary nodes are located is partitioned into several overlapping
subareas, as shown in below equation.
The secondary nodes are identified by an instance Id, iId, as well as a global Id,
gId.
The simulation assumes that the primary nodes cover the entire area; thus, their
position is immaterial.

12
A Cloud Service for Adaptive Data Streaming

Data streaming is the name given to the transfer of data at a high rate with real-
time constraints.
Multimedia applications such as music and video streaming, high-definition
television (HDTV), scientific applications that process a continuous stream of data
collected by sensors, the continuous backup copying to a storage medium of the
data flow within a computer, and many other applications require the transfer of
real-time data at a high rate.
For example, to support real-time human perception of the data, multimedia
applications have to make sure that enough data is being continuously received
without any noticeable time lag.
Concerned with the case when data streaming involves a multimedia application
connected to a service running on a computer cloud.
The stream could originate from the cloud, as is the case of the iCloud service
provided by Apple, or could be directed toward the cloud, as in the case of a real-
time data collection and analysis system.
Data streaming involves three entities: the sender, a communication network, and a
receiver.
The resources necessary to guarantee the timing constraints include CPU cycles
and buffer space at the sender and the receiver, as well as network bandwidth.
Adaptive data streaming determines the data rate based on the available resources.
Lower data rates imply lower quality, but they reduce the demands for system
resources.
Adaptive data streaming is possible only if the application permits tradeoffs
between quantity and quality.
Such tradeoffs are feasible for audio and video streaming, which allow lossy
compression, but are not acceptable for many applications that process a
continuous stream of data collected by sensors.

13
Data streaming requires accurate information about all resources involved, and this
implies that the network bandwidth has to be constantly monitored; at the same
time, the scheduling algorithms should be coordinated with memory management
to guarantee the timing constraints.
Adaptive data streaming poses additional constraints because the data flow is
dynamic.
Indeed, once we detect that the network cannot accommodate the data rate required
by an audio or video stream, we have to reduce the data rate; thus, to convert to a
lower quality audio or video.
Data conversion can be done on the fly and, in this case, the data flow on the cloud
has to be changed.

Google App Engine

Google App Engine enables developers to build their web apps on the same
infrastructure that powers Google’s own applications.
Features
 Leveraging Google App Engine, developers can accomplish the following
tasks:
• Write code once and deploy
 Provisioning and configuring multiple machines for
web serving and data storage can be expensive and
time-consuming.
 Google App Engine makes it easier to deploy web
applications by dynamically providing computing
resources as they are needed.
 Developers write the code, and Google App Engine
takes care of the rest.
• Absorb spikes in traffic
When a web app surges in popularity, the sudden increase in traffic
can be overwhelming for applications of all sizes, from startups to
large companies that find themselves re-architecting their databases
and entire systems several times a year. With automatic replication
and load balancing, Google App Engine makes it easier to scale

14
from one user to one million by taking advantage of Bigtable and
other components of Google’s scalable infrastructure.
• Easily integrate with other Google services It’s unnecessary and
inefficient for developers to write components like authentication
and email from scratch for each new application. Developers using
Google App Engine can make use of built-in components and
Google’s broader library of APIs that provide plug-and-play
functionality for simple but important features.

 Cost
 Google enticed developers by offering the App Engine for free, when it
launched, but after a few months slapped on some fees.
Google App Engine is available at http://code.google.com/
appengine/.

Google Web Toolkit

With Google Web Toolkit, developers can develop and debug web applications in
the familiar Java programming language, and then deploy them as highly
optimized JavaScript.
In doing so, developers sidestep common AJAX headaches like browser
compatibility and enjoy significant performance and productivity gains. Google
Health is one recently launched application to use Google Web Toolkit.
Google Web Toolkit includes Java 5 language support so that developers can enjoy
using the full capabilities of the Java 5 syntax.
These capabilities include Java generics, enumerated types, annotations, auto-
boxing, variable parameter lists, and more.
The compiler in Google Web Toolkit 1.5 produces faster code than ever, delivering
performance gains big enough for end users to notice.
Indeed, often the compiler produces faster JavaScript than a person would write
by hand in JavaScript.

Google Web Toolkit 1.5

15
Accomplishes this by performing deep inlining, better dead-code elimination, and
other forms of enhanced static analysis.
Google Web Toolkit also continues to provide a rich and growing set of libraries
that help developers build world-class AJAX, including thoroughly tested, reusable
libraries for implementing user interfaces, data structures, client/server
communication, internationalization, testing, and accessibility. More information
about Google Web Toolkit is available at http://code.google.com/webtoolkit/.

16