Categories
Amazon Web Services AWS CLI cloud computing Command Line EC2 FTP Grep Linux Passwords PuTTY PuTTYgen SSH Windows winscp

Connecting to Amazon AWS from Windows to a Linux AMI

Connecting from Windows to a Linux Server poses problems for the average person to connect, and use their Windows OS to connect to the Linux Server. For some this is enough to put off people from jumping in, or banging their head against the wall wondering why you cannot simply RDP into the server and where is the Admin Password for the Linux box. We are all used to what we are used to, and for many using a public/private key pair is part of the problem. We are not exposed to this in our daily lives, although with cloud computing via AWS you have to become smart in this process. The instructions at Amazon are not written in plain English and it’s hard to find the details.

Here are some handy steps that you can take to get access to your Amazon AWS server (Amazon Web Services) that takes place after you have set up your instance and downloaded your keys. You might want to read this article by Dave Winer to get through the set up of the AMI, downloading your keys and otherwise getting your first AWS instance set up and booted to the point where it is running. This is where this article steps in to help people connect to their Linux server from a Windows Box. For this process, we used the new Windows 7 as our Windows box, and all the software we recommend you download worked great.

When you are building out your system you want to have your instance up and running. You can use any AMI (Amazon Machine Image) that you want to use; in this case we are using the simple LAMP Linux version. You can find the LAMP AMI by searching for “LAMP”. There are some quick GotCha’s that you want to know about before you put a lot of time into an AMI getting it all prepped for what you want to do.

Windows or Linux does not matter, if you terminate the instance there is no way to restart it you will get the no valid actions error. When you terminate an instance you cannot restart it, it is gone, shutdown, lost forever. If you click on Launch, you will be prompted for a new image, regardless of what image you want to restart, so be very careful when you terminate an instance. If you terminate, it will be DOA for life. The image below shows what options you have using the AWS console provided by Amazon (via the Web). Remember, never click on Terminate unless you are completely done with the image and you never want to use it again.

A running Linux instance does not have the “get administrative password” option that Dave talks about in his EC2 for Poets article. With Linux there are some things you have to do to get access. If you search Amazon you will see a ton of words on this subject that again for the average person is a rambling diatribe written by computer geeks for computer geeks. They even offer you a set of tools to download, but if you are running windows many of these tools will not work properly. Here are the tools you need to get started.

Putty – Putty is an SSH terminal client that will tie in with another program that you will download here in a minute. Download PuTTY here.

PuttyGen – You can download this software from the same place you downloaded putty. You will want this program to convert your Amazon Key from Amazon format to PPK format that the next bit of software you will download needs to make the connection to your new Linux server.

WinSCP – WinSCP is a SFTP client and SSH client for Windows. Its main function is the secure file transfer between a local and a remote computer. It uses Secure Shell (SSH) and supports, in addition to Secure FTP, also legacy SCP protocol. You can download the software here.

You should have your private key from Amazon when you first started up your AMI. Find the Private key you downloaded from Amazon, or make another key if you have to. Making a key is very easy, you can use Dave’s article, or follow the directions from Amazon which are less cogent but still follow able.

Start the PuttyGen program as shown below.

[PuttyGen1.png]

Find the key that you made when you made your initial image and click on load. You will get a dialog box that wants you to load your key file that you got from Amazon. Find where you dropped your key file (again probably in downloads). Remember to set your file type to * to see all your files (otherwise it will default to looking for a PPK file, which you do not have yet).

Click on OK when you find and enter your file name. You should see the dialog below that shows the Amazon key was successfully converted from the Amazon format to the PPK format that WinSCP needs to work properly.

Click on OK.

Click on Save Private Key. You really need to save your private key at this point. Remember where you saved your brand new PPK extension key (if you forget where you saved it, search your computer for *.ppk). You have made it through the first hurdle, and now you need to fire up WinSCP.

WinSCP

If you skipped everything else and ended up here, WinSCP is a SFTP client and SSH client for Windows. Its main function is the secure file transfer between a local and a remote computer. It uses Secure Shell (SSH) and supports, in addition to Secure FTP, also legacy SCP protocol. You can download it from Source Forge.

Start (You should find it under start/all programs/WinSCP) WinSCP and you should see this after you have installed it

You only have to enter minimal data here, your host name is your EC2 instance, your user name is Root, leave the password BLANK, and click on the three dots for your private key file. If you do not know what your AWS instance is, you can go back to the AWS console and click on connect, you will get a connect help Secure Shell (SSH) help screen. This screen might look bad, but it has all the information you need for the Amazon instance you want to connect to. The screen below shows you what the help screen looks like.

The “enter the following command line” has a lot of information that you need. You can forget about everything before the “@”, you will not need that. Everything after the @ though is the public DNS that you will need to connect to your Amazon instance. You should copy everything after the @ symbol so that what you have looks like this

ec2-72-44-46-XXX.compute-1.amazonaws.com

This is your host name that you want to put into your WinSCP line for host name. The image below show the WinSCP entry screen with all the data filled out.

You want your WinSCP Login screen to look exactly like this, short of the instance name and the PPK key; yours will be different in terms of what is in the Host Name entry, and where you stored your own Private Key file. Remember that your Private Key file is the key you converted in PuttyGen. Click on Login and you will connect to your Amazon EC2 instance.

The good part is now you can tool around the Linux AMI using a graphical interface, and drag and drop files from your computer to the Amazon AMI that you are using.

In WinSCP on the top command ribbon bar you should see two icons as circled below. To get to the Command Line in Linux (remember this is why you downloaded Putty) you want to click on the two PC’s on the top command ribbon shown below.

This will launch the Putty command shell that will allow you to tool around the Amazon AMI. You need to be familiar with the Linux command line because you can do things here that you cannot do in the graphical interface, like bring up the update programs, tool around the OS to get an idea of where things are, use ps –ef |grep PROGRAM to see what programs are running (great for debugging MySQL), and to launch specific MySQL commands. This is what you should see when you click on the two computers graphic in the screen above.

Here are some discovered issues with the AMI that I built out.

MySQL has no password, it is root followed quickly by a blank password. You need to use the MySQL command line to set a password for this as soon as you can. The MySQL website can help you out setting the password.

The AMI in this instance when you look at the security group in the Amazon Web Services Console automatically opened up port 3306 for world wide access to your MySQL database. You want to take this rule out of the configuration, especially if you do not set a password. This is one excellent and easy way for hackers to get into your MySQL database; you need to fix this before you have the world hit your web site.

If you are interested in just web services you will find the root for HTTP under the home directory.

If you are interested in making sure that your Linux AMI is up to date, you need to use YUM, type in YUM update to update your whole computer or type in YUM update php if you want to make sure you are working with the latest version of PHP. Here is the catch, the YUM repo’s that come with the AMI we were using could not find any updates, and started throwing errors that it could not find any available mirrors. The problem with this is that you cannot update your software, which from a security viewpoint is very bad. If YUM is not finding updates, there are many resources on the internet to help you through this, the problem is that the repo’s that come with the image we were using goes right back to the Fedora repository, which is distressingly empty of updates for known vulnerable software. There are options on where you can go to get updates, but that will be another topic to talk about, updating your shiny new Amazon AMI for security when support is lacking from the people who make the software you are using.

If your program has dependencies on the version of PHP you are using (in my case the version was 5.0.4, and needed 5.1.X) you will have to hack your installation script if it checks the environmental variables for your PHP server. That means stepping through the entire install, and finding the call where it tests the PHP environment and changing that part of the install script. You can edit the file in WinSCP and save it back to the server, just remember what files you had to change (hack) when you get done.

Keywords:
windows amazon keyfile winscp
windows amazon key file winscp
winscp aws command line
winscp to linux ami

Categories
Amazon Web Services AWS CLI Command Line EC2 Encryption Linux S3 SSL Windows

Glossary of Amazon EC2 terms

Amazon machine image (AMI)
An Amazon Machine Image (AMI) is an encrypted machine image stored in Amazon S3. It contains all the information necessary to boot instances of your software.

Amazon EBS
A type of storage that enables you to create volumes that can be mounted as devices by Amazon EC2 instances. Amazon EBS volumes behave like raw unformatted external block devices. They have user supplied device names and provide a block device interface. You can load a file system on top of Amazon EBS volumes, or use them just as you would use a block device.

Availability Zone
A distinct location within a region that is engineered to be insulated from failures in other Availability Zones and provides inexpensive, low latency network connectivity to other Availability Zones in the same region.

compute unit
An Amazon-generated measure that enables you to evaluate the CPU capacity of different Amazon EC2 instance types.

EBS
See Amazon EBS.

Elastic Block Store
See Amazon EBS.

elastic IP address
A static public IP address designed for dynamic cloud computing. Elastic IP addresses are associated with your account, not specific instances. Any elastic IP addresses that you associate with your account remain associated with your account until you explicitly release them. Unlike traditional static IP addresses, however, elastic IP addresses allow you to mask instance or Availability Zone failures by rapidly remapping your public IP addresses to any instance in your account.

ephemeral store
See instance store.

explicit launch permission
Launch permission granted to a specific user.

group
See security group.

instance store
Every instance includes a fixed amount of storage space on which you can store data. This is not designed to be a permanent storage solution. If you need a permanent storage system, use Amazon EBS.

instance type
A specification that defines the memory, CPU, storage capacity, and hourly cost for an instance. Some instance types are designed for standard applications while others are designed for CPU-intensive applications.

gibibyte (GiB)
a contraction of giga binary byte, a gibibyte is 2^30 bytes or 1,073,741,824 bytes. A gigabyte is 10^9 or 1,000,000,000 bytes. So yes, Amazon has bigger bytes.

image
See Amazon machine image.

instance
Once an AMI has been launched, the resulting running system is referred to as an instance. All instances based on the same AMI start out identical and any information on them is lost when the instances are terminated or fail.

instance store
The disk storage associated with an instance. In the event an instance fails or is terminated (not simply rebooted), all content on the instance store is deleted.

group
Also known as a security group, groups define firewall rules that can be shared among a group of instances that have similar security requirements. The group is specified at instance launch.

launch permission
AMI attribute allowing users to launch an AMI

Linux
Amazon EC2 instances are available for many operating platforms, including Linux, Solaris, Windows, and others.

paid AMI
An AMI that you sell to other Amazon EC2 users. For more information, refer to the Amazon DevPay Developer Guide.

private IP address
All Amazon EC2 instances are assigned two IP addresses at launch: a private address (RFC 1918) and a public address that are directly mapped to each other through Network Address Translation (NAT).

public AMI
An AMI that all users have launch permissions for.

public data sets
Sets of large public data sets that can be seamlessly integrated into AWS cloud-based applications. Amazon stores the data sets at no charge to the community and, like all AWS services, users pay only for the compute and storage they use for their own applications. These data sets currently include data from the Human Genome Project, the U.S. Census, Wikipedia, and other sources.

public IP address
All Amazon EC2 instances are assigned two IP addresses at launch: a private address (RFC 1918) and a public address that are directly mapped to each other through Network Address Translation (NAT).

region
A geographical area in which you can launch instances (e.g., US, EU).

reservation
A collection of instances started as part of the same launch request.

Reserved Instance
An additional Amazon EC2 pricing option. With Reserved Instances, you can make a low one-time payment for each instance to reserve and receive a significant discount on the hourly usage charge for that instance.

security group
A security group is a named collection of access rules. These access rules specify which ingress (i.e., incoming) network traffic should be delivered to your instance. All other ingress traffic will be discarded.

shared AMI
AMIs that developers build and make available for other AWS developers to use.

Solaris
Amazon EC2 instances are available for many operating platforms, including Linux, Solaris, Windows, and others.

snapshot
Amazon EBS provides the ability to create snapshots or backups of your Amazon EBS volumes and store them in Amazon S3. You can use these snapshots as the starting point for new Amazon EBS volumes and to protect your data for long term durability.

supported AMIs
These AMIs are similar to paid AMIs, except that you charge for software or a service that customers use with their own AMIs.

tebibyte (TiB)
a contraction of tera binary byte, a tebibyte is 2^40 bytes or 1,099,511,627,776 bytes. A terabyte is 10^12 or 1,000,000,000,000 bytes. So yes, Amazon has bigger bytes.

UNIX
Amazon EC2 instances are available for many operating platforms, including Linux, Solaris, Windows, and others.

Windows
Amazon EC2 instances are available for many operating platforms, including Linux, Solaris, Windows, and others.

Categories
Amazon Web Services CLI Command Line EC2 Linux S3 Windows

Amazon Elastic Compute Cloud (EC2) Command Line Tools Reference

The Amazon Elastic Compute Cloud Command Line Tools Reference Guide provides the syntax, a description, options, and usage examples for each command line tool. This section describes who should read this guide, how the guide is organized, and other resources related to Amazon Elastic Compute Cloud.

The Amazon Elastic Compute Cloud is occasionally referred to within this guide as simply “Amazon EC2”; all copyrights and legal protections still apply.

View guide here.

Amazon Elastic Compute Cloud
Command Line Tools Reference (straight to the meat and potatoes)

Categories
Amazon Web Services CLI Command Line EC2 Linux S3 Windows

How to run Bucket Commander: A command line interface for Amazon S3

Bucket commander is a command line tool for Amazon S3.

Bucket Commander needs a configuration file, which can be created using Bucket Explorer’s UI.

Bucket Commander takes three arguments. ” -action ” , ” -authenticate ” and ” -emailprofile ”

“ -emailprofile” is optional argument , you need to specify it only when you have configured the Email profile for getting report of Bucket Commander operations (Upload, Download and Copy) via Email.
Valid values for ” -action ” are:

  • upload
  • download
  • copy

To run Bucket Commander at least one credential should be saved.

In case of Single credential saved the authentication argument is optional.

For ” -authenticate ” specify the nick name that you see in “quick connect” drop down from Bucket Explorer’s UI.
For Bucket Commander to work it needs config folder and .Lic file, i.e. bucketcommander.xml and bucketexplorer.xml . Upload /Download/Copy details are picked from the commander xml and authentication details are picked from bucketexplorer xml.

If BucketCommander.exe runs on different machines then it will not be able to decrypt credentials so it will prompt to update credentials, now you need to update credentials by giving Access Key and Secret Key .

For ” -emailprofile ” specify the profile name that you have saved in Email profile configuration from Bucket Explorer’s UI.

How to send report with Bucket Commander

You can specify more than one Email Profile by comma separator for getting report of Bucket Commander operations via email to each specified profile.
An example of working command looks like:
Command on Windows

Bucketcommander.exe -action:upload/download/copy [-authenticate:nick-name][[-emailprofile:profilename1,profilename2]

Command on Linux

BucketExplorer.sh -action:upload/download/copy [-authenticate:nick-name][-emailprofile:profilename1,profilename2]

Note: On Linux you can open terminal from Applications->Accessories->Terminal in Finder
Command on Mac OSX

java -jar BucketExplorer.jar -action:upload/download/copy [-authenticate:nick-name][-emailprofile:profilename1,profilename2]

Note: On Mac OSX you can open terminal from Applications->Utilities->Terminal in Finder.

Download bucket explorer for windows, linux and mac osx

Categories
CLI Command Line Language Bar Linux regsvr32.exe Unistall Windows

Uninstall the Annoying Windows Language Bar from the Windows Command Line

The lanuage bar can be uninstalled / removed / deleted / disabled completely by running the following command:

regsvr32.exe /u /s msutb.dll

Categories
CLI Command Line FTP GNU HTTP HTTPS Linux Open Source Passwords SourceForge WGET wget examples Windows

GNU WGet for Windows (Windows 7, Vista, XP, etc.)

Whether you need a quick-and-dirty way to download a file via HTTP, HTTPS or FTP; or test a web page or recursively download a whole site, WGET is a great tool for the task.

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc., on both Windows- or *nix-based systems.

GNU Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy, including:

  • Resume aborted downloads, using REST and RANGE
  • Use filename wild cards and recursively mirror directories
  • NLS-based message files for many different languages
  • Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
  • Runs on most UNIX-like operating systems as well as Microsoft Windows
  • Supports HTTP proxies
  • Supports HTTP cookies
  • Supports persistent HTTP connections
  • Unattended / background operation
  • Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring
  • GNU Wget is distributed under the GNU General Public License.

Wget has an extensive set of options, the full list of which can be viewed from the command line with “wget –help.”  Here are a few useful examples

Example 1 – Download the default page for given site to your current directory:

wget powercram.com

Example 2 – Recursively download the default page plus an additional level based on links from the default page:

wget -r -l 2 powercram.com

Example 3 – This will do as in example 2, additionally specifying username and password if required by site:

wget -r -l 2 --random-wait --http-user=powercram --http-password=powercram powercram.com

Example 4 – WGET can be used to recursively mirror your site, including download all the images, css and javascript, etc., and localize all of the URLS (so the site works on your local machine).  You can even save all the pages as .html files.

– To mirror your site:

wget -r http://www.powercram.com

– To mirror the site and localize all of the URLs:

wget --convert-links -r http://www.powercram.com

– To mirror the site and save the files as .html:

wget --html-extension -r http://www.powercram.com

Download WGet for Windows, install it, play with it and have some fun.

GNU wget runs on any version of Windows, including 2000, 2003, XP, Vista, Windows 7.

More

Categories
CLI Command Line Linux Time Ubuntu Windows

Manually sync clock on Ubuntu Linux

You may manually sync the clock using the following

sudo ntpdate servername

where servername can be any public or private time server.

Example

sudo ntpdate pool.ntp.org

Categories
CLI Command Line Linux Mac OSX SourceForge Windows

Synergy – desktop sharing for Windows, Linux and Mac OS

Synergy lets you easily share a single mouse and keyboard between multiple computers with different operating systems without special hardware. It’s intended for users with multiple computers on their desk since each system uses its own display.

sourceforge.net/projects/synergy2/

Categories
CLI Command Line Control Panel Linux Windows Windows Components Wizard

Command line shortcut to Windows Components Wizard

Quick and easy way to launch the Windows Components Wizard. You could either run the following command from the Start / Run line or create a shortcut.

%windir%system32sysocmgr.exe /i:%windir%infsysoc.inf

This will launch the Windows Components Wizard so you don’t have to go the traditional route through Control Panel / Add/Remove Programs.

Categories
CLI Command Line Time Win 7 Win7 Windows Windows 7 Windows7

Windows 7 Command Line Utility for Timezone Management – tzutil.exe

tzutil.exe” is a command line utility introduced in Windows Vista (also in Windows 7 and Windows 2008) which can be used to change Time Zone or to view the current time zone information. To use this command (from a command prompt as administrator):

tzutil /g – To view current time zone
tzutil /l – To get a list of all available time zones
tzutil /s <time_zone_ID> – To change time zone

time_zone_ID is the new time zone which you want to set. For example, if you want to set the time zone to Eastern, Central, Mountain or Pacific Daylight Time, you would use any of the following:

tzutil /s "Eastern Daylight Time"
tzutil /s "Central Daylight Time"
tzutil /s "Mountain Daylight Time"
tzutil /s "Pacific Daylight Time"

Windows change timezone command.