Categories
CLI Command Line FTP Linux Windows Windows 2003

Configuring FTP in Isolation Mode in IIS 6

IIS FTP isolation mode enables you to have a separate directory per FTP user. For example if you had a dozen different users that all need access to your FTP server but you did not want them to see any directories but their own you could setup Isolation Mode. Isolation Mode supports users either on the local computer, or if you are a member of a domain it supports domain users.

We will begin by assuming you have FTP (and IIS) installed. Launch IIS Manager (IIS Manager 6.0 in Windows 2008). NOTE: in Windows 2008 make sure you install the IIS Management Console role so you can manage your FTP sites.

  • Delete or disable the default ftp site
  • Create a new FTP site with desired values – at the FTP User Isolation screen select Isolate Users or Isolate Users using Active Directory (which ever works better for your situation)
  • Specify path for your new FTP site
  • Set FTP Site Access Permissions to both Read and Write
Within the root FTP directory you must create either a directory named localuser or <your_domain_name>, depending on which isolation mode you are using.  In my case I am NOT using Active Directory so I created the directory localuser.
Within the localuser or <your_domain_name> directory create the desired user directories and assign each user sufficient permissions to their respective directory.

Example – creating directories:

md ftproot
md ftprootlocaluser
md ftprootlocaluserlocaluser1
md ftprootlocaluserlocaluser2
md ftprootlocaluserlocaluser3

Example – setting Create NTFS permissions on each respective directory:

cacls ftprootlocaluserlocaluser1 /E /G localuser1:C
cacls ftprootlocaluserlocaluser2 /E /G localuser2:C
cacls ftprootlocaluserlocaluser3 /E /G localuser3:C

Categories
Command Line FTP howto Linux Passwords WGET wget examples Windows

How To Use Wget – Includes Several Examples Using Wget

wget is a great command line utility that is natively available in Linux and can be downloaded for Windows (see also GNU WGet for Windows (Windows 7, Vista, XP, etc.)). wget can be used for many download situations including large files, recursive downloads, non-interactive downloads, multiple file downloads, etc.

Note: options ARE case sensitive.

1. Download a single file with wget using no options.

wget http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz

While downloading, wget will display a progress bar with the following information:

  • % of download completion
  • Download progress in bytes
  • Current download speed
  • Estimated time remaining

Download in progress

Completed download

2. Download a file saving with a different name using wget -O

wget http://www.vim.org/scripts/download_script.php?src_id=7701

Even though the downloaded file is in zip format, it will be saved with the name download_script.php?src_id=7701 without the -O switch.

To modify this behavior specify the output file name using the -O option.

wget -O taglist.zip http://www.vim.org/scripts/download_script.php?src_id=7701

3. Specify download speed / download rate Using wget –limit-rate

While executing the wget, by default it will try to use all possible bandwidth. You can limit the download speed using the –limit-rate switch.

wget --limit-rate=200k http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz

4. Restart a download which stopped in the middle using wget -c.

wget -c http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz

5. Download in the background with wget -b

wget -b http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz

The download will begin and give back the shell prompt to you. You can always check the status of the download using tail -f  (Linux only) .

tail -f wget-log

6. Mask user agent and display wget like browser using wget –user-agent

Some websites can disallow you to download its page by identifying that the user agent is not a browser. So you can mask the user agent by using –user-agent options and show wget like a browser.

wget --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3" http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz

7. Test URL using wget –spider.  This will test that the file exists, but not perform the download.

wget --spider http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz

8. Increase total number of retry attempts using wget –tries.

wget --tries=75 http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz

9. Download multiple files / URLs using wget -i

First, store all the download files or URLs in a text file:
URL1
URL2
URL3
URL4

Next, give the download-file-list.txt as argument to wget using -i option.

wget -i download-file-list.txt

10. Download a full website using wget –mirror

wget --mirror -p --convert-links -P ./LOCAL-DIR WEBSITE-URL

  • –mirror: enable mirroring
  • -p: download all files that are necessary to properly display a given HTML page
  • –convert-link: after the download, convert the links in document for local viewing
  • -P ./LOCAL-DIR: save all the files and directories to the specified directory

11. Skip certain file types while downloading using wget –reject.  In order to download all content except .gif images use the following.

wget --reject=gif WEBSITE-TO-BE-DOWNLOADED

12. Log messages to a log file instead of stderr using wget -o.  To redirect output to a log file instead of the terminal.

wget -o download.log DOWNLOAD-URL

13. Quit downloading when certain size is exceeded using wget -Q.

wget -Q5m -i FILE-WHICH-HAS-URLS

14. Download only certain file types using wget -r -A

You can use this for the following situations

  • Download all images from a website
  • Download all videos from a website
  • Download all PDF files from a website

wget -r -A.pdf http://url-to-webpage-with-pdfs/

15. You can use wget to perform FTP downloads.

wget ftp-url

FTP download using wget with username and password authentication.

wget --ftp-user=USERNAME --ftp-password=PASSWORD DOWNLOAD-URL

Note: username and password can be used for HTTP and HTTPS downloads as well using –http-user=USER, –http-password=PASS respectively.

More

Categories
FTP IIS 7 Linux SSL Win 7 Win7 Windows Windows 2008 Windows 7 Windows7

How to Install and Configure IIS 7 FTP Publishing Service

The new FTP Publishing Service 7.5 has been rewritten for Windows Server 2008 and lot of new futures and additions included. (Integration with IIS 7, FTP over SSL (from v7.0), Improved Logging, etc..)

Prerequisites
1. Internet Information Services 7.0 installed.
2. Windows Server 2008 or Vista.
3. Download FTP Publishing Service 7.5

4. Create a folder for FTP Publishing Service and allow Full access for Administrators

  • mkdir “c:inetpubftprootftp.mydomain.com”
  • cacls “c:inetpubftprootftp.mydomain.com” /G administrators:F /T /E

In case we want to add access to specific backup user we can use the following command”

  • cacls “c:inetpubftprootftp.mydomain.com” /G username:F /T /E

Permissions:

R  Read
W  Write
C  Change (write)
F  Full control

Installation
Before installing FTP Publishing Service 7.5 over IIS 7.0 be sure that you uninstall any previous versions of FTP Publishing Service.

  1. When the installation program of Microsoft FTP Service for IIS 7.0 begins, click Next.
  2. Accept the End-User License Agreement and click Next.
  3. Select the features you want to be installed and click Next.
  4. Click Install to begin the installation.
  5. At the end click Finish.

Creating the Certificate
After the installation finishes we open Internet Information Services 7.0  Manager and we select Server Certificates to create a self-signed SSL certificate. Another option is to create a certificate request to process with a 3rd party SSL provider which is recommended for production systems as they are from a trusted root.

Server Certificates

Click on Create Self-Signed Certificate…

Create a self-signed certificate

Specify a certificate name and click OK:

Specify certificate name


Creating the FTP Site
Right click the Sites node in the tree and click Add FTP Site…

Add FTP Site...

On the Add FTP Site wizard add the FTP site name and select the path we created in the prerequisites and click Next.

Add FTP Site Wizard

On the next page of the wizard we click the Require SSL option and we choose our SSL Certificate then we click Next.

Binding and SSL Settings

On the next page of the wizard we select Basic for Authentication and on Allow access to we select specified users and we define our backup ftp account with read and write permissions then we click on Finish. In case we want to allow anonymous ftp connections select Anonymous authentication and on Permissions we select the Read.

Authentication and Authorization Information


Configure the Firewall
If we are behind a firewall we should configure the FTP Firewall Support under Internet Information Services Manager and configure our firewall ports to accept passive connections on the ports we specify. In case we want to use dynamic port range under Data Channel Port Range we enter port range “0-0?.

FTP Firewall Support


Configure the FTP Client (FileZilla for example)
Configure FTP Client to connect to our FTP Site using FTP over explicit TLS/SSL.

FTP Client Settings


Troubleshooting
In case you get “534 Local policy on server does not allow TLS secure connections.” error this is because we need to select an SSL certificate at the Server Level.

FTP SSL Settings Server Level
FTP SSL Settings Certificate Selection

Categories
FTP Linux Passwords Private Key Public Key SFTP SSH Windows

How to create and use Public Keys with SSH

SSH and SFTP Public Key Authentication requires that you create a public/private key pair. In this howto we will look at how to create then use those keys.

SSH (Secure Shell) and SFTP (Secure FTP) support a very strong security model that can be used instead of the normal username and password authentication scheme. It uses public key cryptography to create a different, and more secure approach to authenticating your identity and rights to access a server or resource.

Essentially you will generate a public and private key pair. The public key will be placed on the server by your system administrator, giving you access. You will keep the file containing the private key in a safe place. You’ll login by simply by providing that private key file to your SSH or SFTP supporting client.

The private key is just that – private. You may put a password on it, but you don’t have to. Without a password, all you need is the file in order to login. Or, to put it more clearly, all anyone needs is that file to login as you. Obviously if you password protect the file, then you’ll need both the file, and the password to unlock it. In that case, logging in is very similar to what you do today: specify a user name, and a password to unlock your key file.

Instructions are included here for the following tools, which are known to work with this technique:

SSH Clients                                    SFTP Clients
PuTTY
SecureCRT
                                   PSFTP
                                   Webdrive
                                   WS_FTP

Instructions for these tools are not provided here, but they either claim or have been confirmed to have the appropriate support:

SSH Clients                            SFTP Clients
Tera Term Pro                            CuteFTP Professional
                           WinSCP

Other tools may also work. The key terminology to look for is “SSH” or “SFTP” and “Public Key Authentication”.

Generating Your Keys

In general it’s best to create your own key. That way you control what happens to your private key.

PuTTYgen

PuTTY is a free SSH client that includes a tool for generating keys, called PuTTYgen. PuTTY is my preferred SSH client.

Run PuTTYgen and click the “Generate” button.

Follow the directive to move the mouse around to generate randomness, which is a key component of public key cryptography. Once that’s done, you should do the following:

  • Specify a passphrase. Technically this is optional, but if you omit the passphrase, then anyone who happens to get ahold of your private key file can login as you. You may have enough security in place where this is not an issue. If you do specify a passphrase, you’ll need to enter it when you login, pretty much as a normal login.
  • Press the Save Public Key button to save the public key. I recommend saving as your name “.pub”. For example I would save “powercram.pub”.
  • Press the Save Private Key button to save your private key. This saves the private key in PuTTY’s own format, a “.ppk” file. So, “name.ppk” might be appropriate.
  • I also recommend hitting the Conversions menu, and then Export Openssh key, and saving that to “name.key”. This format will allow you to use your private key with other applications besides PuTTY.

SecureCRT

SecureCRT is a stand-alone SSH client.
To create a public key with SecureCRT, click the Tools menu, Create Public Key… option to begin the wizard.  Select RSA as the key type. Enter (or not) an appropriate passphrase to protect your private key. A default key length of 1024 is sufficient. Allow SecureCRT to save the key, noting the location. It may ask if you want to use this as your global Public Key, and you can safely say “yes”.

WS_FTP

In WS_FTP, click Tools, Options, and then click on SSH, Client Keys:

Press Create, and step through the wizard. The key type should be RSA, and the default size of 1024 is sufficient.  Once the key has been created and shows up in the list, click on it, and then click on Export, to export your public key.

Using Your Keys – SSH

Once your keys are generated, and the public key installed on the server, you’ll need to specify the private key to your SSH client in order to log in.

PuTTY

There are at least two approaches to using Public/Private keys with PuTTY. When you launch PuTTY without any arguments, you get its standard configuration dialog, into which you can enter the name of the server you want to connect to:

On the left hand side is a tree view of various options. Underneath Connection, SSH, click on Auth and the dialog will include a field “Private key file for authentication“:

Specify the location of the “.ppk” file that you generated with PuTTYgen. When you connect, if your private key is passphrase protected, you’ll be asked for the passphrase.  The other approach is to simply create shortcuts for the various servers I connect to regularly, and specify the location of the private key on the command line. For example:

C:pathPUTTY.EXE -i c:adminpowercram.ppk admin@server.com

That, as a desktop shortcut, or item on a Windows menu, connects to the named server using the specified account name “admin”, and uses the private key found in “c:adminpowercram.ppk” to authenticate.

SecureCRT

SecureCRT has several paths to a connection dialog, but we’ll use “Quick Connect” for our example. Click the Quick Connect Icon:

Make sure that protocol is set to SSH2, and enter your host and username. In Authentication, UNcheck everything except PublicKey. Then click on that, and click Properties.

Typically you don’t need to do anything, but this dialog specifies the location of your identity file (aka Private Key).  Assuming that your public key has been placed on the server for your account, you should now be able to connect.

Using Your Keys – SFTP

Secure FTP, or FTP, is really just using SSH technology to provide FTP-like functionality. Since it’s using SSH, the keys you’ve generated and are using for your SSH authentication work with many SFTP applications as well.

WebDrive

Webdrive is an FTP/SFTP service for Windows that allows you to treat an FTP or SFTP connection like another drive mounted on your system. Uploading and downloading then become simple Windows file copy operations.  In Webdrive, you’ll need to load your private key, and then specify it in the configuration for a specific SFTP connection.  The Certificates tab of Webdrive’s Settings dialog, has a Hostkey Managemet button.  Push that, and you’ll get the host key management dialog, and on that you’ll find an Import button. Press that to import your public and private keys:

Specify the “.pub” key for the public key you generated earlier. The private key should also be specified, and would be the “.key” file. If you passphrase protected your key file, you can specify that here as well. Give it a recognizable name.  The second step, then, takes us back to the Webdrive main window.

Click on a connection (or create a new one). In the Properties for that connection, on the SFTP tab will be a setting Enable client hostkey support for this site:

Here you’ll find a dropdown list of the keys you imported above, and a place to enter the password, if any, to access that key.  Once completed, Webdrive should now be able to connect to your public key authenticated site.

WS_FTP

Having created a key pair already in WS_FTP, using it is simply a matter of defining your connection to use it.
When you create a site, specify its connection type as SFTP/SSH. Specify a user name, but leave your password blank. At the end of the wizard, click on the Advanced button, this will allow you to edit the connection, and is the equivalent to editing an existing connection.

Click on the SSH item on the left, and the dropdown list that results should allow you to select the key pair that you created earlier.  Assuming that the public key you exported and sent to your system administrator has been installed on the server, you should now be able to connect.

psftp

PSFTP is command line FTP program that is distributed with PuTTY. More importantly, it supports public key SFTP by using the “.ppk” file that you created for PuTTY above. Connecting using a public key is simply a different set of comment line options:

psftp -l username -2 -i keys.ppk remotehost

-l username specifies your username on the remote host; -2 indicates that PSFTP should use SSH protocol version 2; -i keys.ppk specifies the location of your private key as created with PuTTYgen; remotehost is the name of the remote host you’re connecting to.

Categories
ethernet FREE FTP Grep Linux Networking NGrep Packet Sniffing SSH Utility Windows WinDump WinPCap WireShark

FREE Windows Utilities for Sniffing


WinDump
WinDump is an exceptionally powerful command-line packet sniffer. Ported over from the source of the Tcpdump utility available for UNIX, WinDump carries over the same power and flexibility to the Windows world, all in a lightweight executable.
WinDump is a helpful utility when you need to do some quick and easy packet capturing to diagnose a network problem. It’s flexible, and it can capture and display details about every packet going across the network interface. It can filter the output results based on protocol (e.g., Address Resolution Protocol—ARP, IP, TCP, UDP), source network or host, destination network or host, source port, destination port, and many other criteria.
To use WinDump, you’ll first need to download and install the Windows Packet Capture Library (WinPcap), the Windows port of the open-source packet-capture and network-analysis library libpcap for UNIX. WinPcap runs on all versions of Windows.
After you install WinPcap, you can download the standalone WinDump executable. To launch WinDump, simply run it from the command line with the appropriate options for what you’d like to capture or read. You’ll find the online manual for WinDump here.
The first command you might want to execute is Win-Dump -D, which will display a list of interfaces available on your computer, as well as a corresponding number for each interface, so that you can determine which interface to use for your sniffing activities. After you know which interface to run, you can simply use that number with the -i option (i.e., WinDump -i 3, if number 3 is the interface you’d like to use) to start viewing packet data in real time. (Because these are ports of UNIX utilities, the command-line switches are case-sensitive.)
Ngrep
Although WinDump is a tremendous utility, sometimes it requires a considerable amount of overhead or knowledge to determine what you’re looking for. For example, suppose you’re trying to look up whether a DNS query is making it across your network, but you aren’t familiar with the protocols and that DNS uses by default. Or, suppose a lot of traffic is coming across a network connection, and you’re finding it too cumbersome to work through all the packets just to find the one particular packet you’re looking for. For such situations enter ngrep, the network-aware grep utility.
If you aren’t familiar with grep, it’s probably one of the most widely known and oft-utilized UNIX utilities. Grep finds matching text strings (through a mechanism known as regular expressions) in files on a file system, then outputs the lines to the display. You might compare grep with the Windows command-line Find utility, but grep differs by providing an exceptional amount of power in its search for text strings.
By applying these capabilities to the network layer, ngrep provides the same level of functionality for packet sniffing. As a result, you don’t need to know what protocols, ports, network, or IP addresses that two devices are using to transfer data. You just need to know something about the packet’s payload, and ngrep will find it for you—regardless of how it’s transmitted.
Ngrep is great for troubleshooting DNS query problems. In a large Active Directory (AD) environment, dozens of DNS queries are typically occurring across the network per second. If I’m trying to troubleshoot a specific set of problems, searching each packet to find the one I’m looking for is cumbersome at best. Instead of relying on a straight packet capture of all DNS traffic, I can simply use ngrep to find the text string I’m looking for because DNS queries and responses are performed in plain text.
Ngrep currently recognizes ICMP, IGMP, Raw, TCP, and UDP protocols across 802.11, Ethernet, FDDI, PPP, SLIP, Token Ring, and null interfaces. Like WinDump it requires the WinPcap library to operate properly.
WireShark (formerly Ethereal)
When you face a situation in which you need to roll up your sleeves and dive as deeply as possible into network capture and analysis, one utility needs to come to mind: the world’s most popular network analyzer, WireShark. Network experts around the world use WireShark because it has all the standard features you’ll find in most protocol analyzers, in addition to some you won’t find in any other product. More than 400 developers around the world have made contributions to this open-source application. A decade ago, you would have to pay thousands of dollars for software that had the same capabilities, but WireShark offers it all free.
WireShark can capture data off your network connection, filter the data, dive into the details of each packet, save the packet capture for detailed analysis, send packet captures to other network engineers (or vendors) to help with debugging, and open packet captures from many other leading packet-capture utilities. WireShark can capture data off of various network transports, such as Classical IP over ATM (CIP), Ethernet, Fiber Distributed Data Interface (FDDI), Point-to-Point Protocol (PPP), Token Ring, 802.11, and loopback interfaces (although it doesn’t support all types on every platform). Across all those network transports, WireShark can “dissect” more than 750 protocol types, including FTP, HTTP, NetBIOS, POP3, remote procedure call (RPC), SNMP, SSH, SMTP, and Telnet, just to name a few.
Like the other sniffing utilities I’ve mentioned, WireShark depends on an installation of WinPcap to function properly, so you’ll need to install that first. Then, after you install the latest Ethereal distribution for Windows, simply access the Capture, Interfaces menu and select the interface you want to start using for capture. Then, you’re ready to start analyzing your traffic.
Categories
Administration BareTail Encryption FREE FTP Grep Linux Notepad++ SSH Utility Windows

32 FREE Windows Administration Utilities

  1. BareTail. A free real-time log file monitoring tool.
  2. CamStudio. Free screen recording software.
  3. CDBurnerXP. Burns CD-ROMs, DVDs, audio CDs, and ISO images.
  4. Comodo Firewall Pro. Is a firewall and antivirus application.
  5. DriveImage XML. Is a program for imaging and backing up partitions and logical drives.
  6. FileZilla. GUI FTP client.
  7. GParted LiveCD. Manages partitions on systems.
  8. InfraRecorder. Burns ISO images and creates data and audio CDs and DVDs.
  9. Lansweeper. Is a network inventory tool that performs hardware scanning, software scanning, and Active Directory (AD) reporting.
  10. LocatePC. Emails you whenever any private or public IP address in your system changes – great for tracking a stolen computer.
  11. MyDefrag (formerly JkDefrag). Defragments and optimizes disks.
  12. Nessus (formerly NeWT). Network/computer vulnerability scanner.
  13. Ngrep. Is a packet sniffer based on finding matching text strings.
  14. Notepad++. Is a text and code editor (more info).
  15. NTFS Undelete. Recovers deleted files that are no longer in the recycle bin.
  16. Open Computers and Software Inventory (OCS Inventory NG). Provides detailed inventory data for an entire network of computers as well as deploys packages.
  17. OpenSSH. Creates secure, encrypted shell sessions.
  18. PageDefrag. Determines how fragmented your paging files and registry hives are, and defragments them.
  19. Paint.NET. Free image and photo editing software for Windows.
  20. PING (Partimage Is Not Ghost) — Backup and Restore Disk Partitions.
  21. PRTG Traffic Grapher. Is a powerful network monitor.
  22. System Information for Windows (SIW). Gathers detailed information about a computer’s system properties, settings, and displays.
  23. TestDisk. Recovers damaged partitions, makes non-bootable disks bootable again, and repairs damaged boot sectors.
  24. TrueCrypt.  Free open-source disk encryption software for Windows 7/Vista/XP, Mac OS X, and Linux.
  25. WinDirStat. Determines how space is being utilized across disks and visually represents the results in multiple ways.
  26. WinPE (Windows Preinstallation Environment). Lets you make a Windows command-line boot recovery DVD.
  27. WinDump. WinDump is the Windows version of tcpdump, the command line network analyzer for UNIX / Linux.
  28. Winfingerprint. Is a network scanner.
  29. Wink. Builds screencast recordings.
  30. WireShark (formerly Ethereal). Network protocol analyzer.
  31. XML Notepad. Is a specialized XML editor.
  32. ZoomIt. Magnifies portions of a screen and lets you draw on and annotate the screen.
Categories
ethernet FTP Linux Passwords Windows

Linux Commands for (Beginning and Forgetful) Administrators





Command
Summary Use
arp
Command mostly used for checking existing Ethernet connectivity and IP address
Most common use: arp

This command should be used in conjunction with the ifconfig and route commands. It is mostly useful for me to check a network card and get the IP address quick. Obviously there are many more parameters, but I am trying to share the basics of server administration, not the whole book of commands.

df
Display filesystem information
Most common use: df -h

Great way to keep tabs on how much hard disk space you have on each mounted file system.

du
Display usage
Most common use, under a specific directory: du -a

Easily and quickly identify the size of files/programs in certain directories. A word of caution is that you should not run this command from the / directory. It will actually display size for every file on the entire Linux harddisk.

find
Find locations of files/directories quickly across entire filesystem
Most common use: find / -name appname -type d -xdev

(replace the word appname with the name of a file or application like gimp)This is a very powerful command and is best used when running as root or superuser. The danger is that you will potentially look across every single file on every filesystem, so the syntax is very important. The example shown allows you to search against all directories below / for the appname found in directories but only on the existing filesystem. It may sound complex but the example shown allows you to find a program you may need within seconds!
Other uses and more complex but beneficial functions include using the -exec or execute a command.
You may also try the commands: locate or try slocate

ifconfig
Command line tool to configure or check all network cards/interfaces
Most common uses: ifconfig and also ifconfig eth0 10.1.1.1

Using the plain ifconfig command will show you the details of all the already configured network cards or interfaces. This is a great way to get a check that your network hardware is working properly. You may also benefit from this review of server configuration. Using the many other options of ifconfig such as the one listed allows you to assign a particular interface a static IP address. I only show an example and not a real world command above. Also review some commands for file permissions here.. Your best bet, if you want to configure your network card using this command is to first read the manual pages. You access them by typing: man ifconfig

init
Allows you to change the server bootup on a specific runlevel
Most common use: init 5

This is a useful command, when for instance a servers fails to identify video type, and ends up dropping to the non-graphical boot-up mode (also called runlevel 3).

The server runlevels rely on scripts to basically start up a server with specific processes and tools upon bootup.Runlevel 5 is the default graphical runlevel for Linux servers. But sometimes you get stuck in a different mode and need to force a level. For those rare cases, the init command is a simple way to force the mode without having to edit the inittab file.

Of course, this command does not fix the underlying problem, it just provides a fast way to change levels as needed. For a more permanent correction to the runlevel, edit your /etc/inittab file to state:id:5:initdefault:

joe or nano
Easy to use command line editors that are often included with the major Linux flavors
Most common uses:
joe filename 
nano filename

A real world example for you to get a better sense on how this works:
nano /etc/dhcp3/dhcpd.conf
This allows you to edit using nano the dhcpd.conf configuration file from the command line.
Maybe you are not up to speed on vi, or never learned how to use emacs? On most Linux flavors the text editor named joe or one named nano are available. These basic but easy to use editors are useful for those who need a text editor on the command line but don’t know vi or emacs. Although, I do highly recommend that you learn and use Vi and Emacs editors as well. Regardless, you will need to use a command line editor from time to time. You can also use cat and more commands to list contents of files, but this is basic stuff found under the basic linux commands listing. Try: more filename to list contents of the filename.

netstat
Summary of network connections and status of sockets
Most common uses: netstat and also netstat |head and also netstat -r

Netstat command simply displays all sockets and server connections. The top few lines are usually most helpful regarding webserver administration. Therefore if you are doing basic webserver work, you can quickly read the top lines of the netstat output by including the |head (pipe and head commands). Using the –option gives you a very good look at the network routing addresses. This is directly linked to the route command.

nslookup
Checks the domain name and IP information of a server
Most common use: nslookup www.hostname.com

You are bound to need this command for one reason or another. When performing server installation and configuration this command gives you the existing root server IP and DNS information and can also provide details from other remote servers. Therefore, it is also a very useful security command where you can lookup DNS information regarding a particular host IP that you may see showing up on your server access logs. Note there are some other commands like file permissions that may also help. There is a lot more to this command and using the man pages will get you the details by typing: man nslookup

ping
Sends test packets to a specified server to check if it is responding properly
Most common use: ping 10.0.0.0 (replace the 10.0.0.0 with a true IP address)
This is an extremely useful command that is necessary to test network connectivity and response of servers. It creates a series of test packets of data that are then bounced to the server and back giving an indication whether the server is operating properly.

It is the first line of testing if a network failure occurs. If ping works but for instance FTP does not, then chances are that the server is configured correctly, but the FTP daemon or service is not. However, if even ping does not work there is a more significant server connectivity issue… like maybe the wires are not connected or the server is turned off! The outcome of this command is pretty much one of two things. Either it works, or you get the message destination host unreachable. It is a very fast way to check even remote servers.

ps
Lists all existing processes on the server
Most common uses: ps and also ps -A |more

The simple command will list every process associated with the specific user running on the server. This is helpful in case you run into problems and need to for instance kill a particular process that is stuck in memory. On the other hand, as a system administrator, I tend to use the -A with the |more option. This will list every process running on the server one screen at a time. Read more of our commands on our reallylinux.com help page. I use ps to quickly check what others are goofing with on my servers and often find that I’m the one doing the dangerous goofing!

rm
Removes/deletes directories and files
Most common use: rm -r name (replace name with your file or directory name)

The –r option forces the command to also apply to each subdirectory within the directory. This will work for even non-empty directories. For instance if you are trying to delete the entire contents of the directory x which includes directories y and z this command will do it in one quick process. That is much more useful than trying to use the rmdir command after deleting files! Instead use the rm -r command and you will save time and effort. You may already have known this but since server administrators end up spending a lot of time making and deleting I included this tip!

route
Lists the routing tables for your server
Most common use: route -v

This is pretty much the exact same output as the command netstat -r. You can suit yourself which you prefer to run. I tend to type netstat commands a lot more than just route and so it applies less to my situation, but who knows, maybe you are going to love and use route the most!

shred
Deletes a file securely by overwriting its contents
Most common use: shred -v filename (replace filename with your specific file)

The -v option is useful since it provides extra view of what exactly the shred tool is doing while you wait. On especially BIG files this could take a bit of time. The result is that your file is so thoroughly deleted it is very unlikely to ever be retrieved again. This is especially useful when trying to zap important server related files that may include confidential information like user names or hidden processes. It is also useful for deleting those hundreds of love notes you get from some of the users on your server, another bonus of being a server administrator. 🙂

sudo
The super-user do command that allows you to run specific commands that require root access.
Most common use: sudo command (replace command with your specific one)

This command is useful when you are logged into a server and attempt a command that requires super-user or root privileges. In most cases, you can simply run the command through sudo, without having to log in as root. In fact, this is a very beneficial way to administer your server without daily use of the root login, which is potentially dangerous.

Note there are other commands for file permissions here. Below is a simple example of the sudo capabilities:
sudo cd /root
This command allows you to change directories to the /root without having to login as root. Note that you must enter the root password once, when running a sudo command.

top
Displays many system statistics and details regarding active processes
Most common use: top

This is a very useful system administrator tool that basically gives you a summary view of the system including number of users, memory usage, CPU usage, and active processes. Often during the course of a day when running multiple servers, one of my Xwindows workstations just displays the top command from each of the servers as a very quick check of their status and stability.

touch
Allows you to change the timestamp on a file.
Most common use: touch filename

Using the basic touch command, as above, will simply force the current date and time upon the specified file. This is helpful, but not often used.

For instance, to force a specific date and time upon all files in a directory, type:
touch *

You can also force a specific date/time stamp using the -t option like this: touch -t200103041200.00 *
The command above will change all files in the current directory to take on the new date of March 4th, 2001 at noon. The syntax follows this pattern: YYYYMMDDhhmm.ss

YYYY represents the four digit year, then the two digit month, day, hour and minutes. You can even specify seconds as noted above. In any case, this is a useful way to control timestamps on any files on your server.

traceroute
Traces the existing network routing for a remote or local server
Most common use: traceroute hostname
(replace hostname with the name of your server such as reallylinux.com)
This is a very powerful network command that basically gives the exact route between your machine and a server. In some cases you can actually watch the network hops from country to country across an ocean, through data centers, etc. Read more of our commands on our reallylinux.com help page.

This comes in handy when trying to fix a network problem, such as when someone on the network can not get access to your server while others can. This can help identify the break or error along the network line. One strong note to you is not to misuse this command! When you run the traceroute everyone of those systems you see listed also sees YOU doing the traceroute and therefore as a matter of etiquette and respect this command should be used when necessary not for entertainment purposes. A key characteristic of gainfully employed server administrators: knowing when to use commands and when not to use them!

w
An extension of the who command that displays details of all users currently on the server
Most common uses: w
This is a very important system admin tool I use commonly to track who is on the server and what processes they are running. It is obviously most useful when run as a superuser.

The default setting for the w command is to show the long list of process details. You can also run the commandw -s to review a shorter process listing, which is helpful when you have a lot of users on the server doing a lot of things! Remember that this is different than the who command that can only display users not their processes.

who
Tool used to monitor who is on the system and many other server related characteristics
Most common uses: who and also who -q and also who -b

The plain command just lists the names of users currently on the server. Using the -q option allows you to quickly view just the total number of users on the system. Using the -b option reminds you how long it has been since you rebooted that stable Linux server! One of my servers had a -b of almost three years! Yes, that’s why we at reallylinux.com call it really Linux!


Thanks to Mark Rais, senior editor ReallyLinux.com for this guide.

Categories
CLI Command Line Encryption FTP Linux Passwords Windows winscp

FTP – File Transfer Protocol

FTP

File Transfer Protocol

Syntax
FTP [-options] [-s:filename] [-w:buffer] [host]

key
-s:filename Run a text file containing FTP commands.

host Host name or IP address of the remote host.

-g Disable filename wildcards.

-n No auto-login.

-i No interactive prompts during ftp.

-v Hide remote server responses.

-w:buffer Set buffer size to buffer
(default=4096)

-d Debug

-a Use any local interface when binding data connection.


Commands to run at the FTP: prompt

append local-file [remote-file]
Append a local file to a file on the remote computer.

ascii Set the file transfer type to ASCII, the default.
In ASCII text mode, character-set and end-of-line
characters are converted as necessary.

bell Toggle a bell to ring after each command.
By default, the bell is off.

binary Set the file transfer type to binary.
Use `Binary' for transferring executable program
files or binary data files e.g. Oracle

bye End the FTP session and exit ftp

cd Change the working directory on the remote host.

close End the FTP session and return to the cmd prompt.

debug Toggle debugging. When debug is on, FTP will display
every command.

delete remote-file
Delete file on remote host.

dir [remote-directory] [local-file]
List a remote directory's files and subdirectories.
(or save the listing to local-file)

disconnect Disconnect from the remote host, retaining the ftp prompt.

get remote-file [local-file]
Copy a remote file to the local PC.

glob Toggle the use of wildcard characters in local pathnames.
By default, globbing is on.

hash Toggle printing a hash (#) for each 2K data block transferred.
By default, hash mark printing is off.

help [command]
Display help for ftp command.

lcd [directory]
Change the working directory on the local PC.
By default, the working directory is the directory in which ftp was started.

literal argument [ ...]
Send arguments, as-is, to the remote FTP host.

ls [remote-directory] [local-file]
List a remote directory's files and folders.
(short format)

mdelete remote-files [ ...]
Delete files on remote host.

mdir remote-files [ ...] local-file
Display a list of a remote directory's files and subdirectories.
(or save the listing to local-file)
Mdir allows you to specify multiple files.

mget remote-files [ ...]
Copy multiple remote files to the local PC.

mkdir directory
Create a directory on the remote host.

mls remote-files [ ...] local-file
List a remote directory's files and folders.
(short format)

mput local-files [ ...]
Copy multiple local files to the remote host.

open computer [port]
Connects to the specified FTP server.

prompt Toggle prompting. Ftp prompts during multiple file transfers to
allow you to selectively retrieve or store files;
mget and mput transfer all files if prompting is turned off.
By default, prompting is on.

put local-file [remote-file]
Copy a local file to the remote host.

pwd Print Working Directory
(current directory on the remote host)

quit End the FTP session with the remote host and exit ftp.

quote argument [ ...]
Send arguments, as-is, to the remote FTP host.

recv remote-file [local-file]
Copy a remote file to the local PC.

remotehelp [command]
Display help for remote commands.

rename filename newfilename
Rename remote files.

rmdir directory
Delete a remote directory.

send local-file [remote-file]
Copy a local file to the remote host.

status Display the current status of FTP connections and toggles.

trace Toggles packet tracing; trace displays the route of each packet

type [type-name]
Set or display the file transfer type:
`binary' or `ASCII' (the default)

If type-name is not specified, the current type is displayed.
ASCII should be used when transferring text files.

In ASCII text mode, character-set and end-of-line
characters are converted as necessary.

Use `Binary' for transferring executable files.

user user-name [password] [account]
Specifes a user to the remote host.

verbose Toggle verbose mode. By default, verbose is on.

! command Run command on the local PC.

? [command] Display help for ftp command.

Examples
an example FTP Script to retrieve files in binary and then ascii mode:

::GetFiles.ftp

[User_id]
[ftp_password]
binary
get /usr/file1.exe
get file2.html
mget *.jpeg
ascii
mget *.txt
quit

To run the above script:
FTP -s:GetFiles.ftp [hostname]
This will connect as the user:User_id with password:ftp_password

An FTP Script to publish files in binary mode:

::PutFiles.ftp

[User_id]
[ftp_password]
binary
mput *.html
cd images
mput *.gif
quit

To run the above script:
FTP -s:PutFiles.ftp [hostname]
This will connect as the user:User_id with password:ftp_password


Using the Windows GUI for FTP
Windows Explorer (not Internet Explorer) also has a built in FTP client.
Type in the address bar:

ftp://username@ftpserver.address.com

you will be prompted for the password.
You can also use

ftp://username:password@ftpserver.address.com

This is not recommended as anyone can read the password.

Secure FTP
Standard FTP does not encrypt passwords – they are sent across the network in plain text. A more secure method is to use SecureFTP (SFTP) or SecureCopy (SCP) Freeware clients are available such as WinSCP.

Categories
Amazon Web Services AWS CLI cloud computing Command Line EC2 FTP Grep Linux Passwords PuTTY PuTTYgen SSH Windows winscp

Connecting to Amazon AWS from Windows to a Linux AMI

Connecting from Windows to a Linux Server poses problems for the average person to connect, and use their Windows OS to connect to the Linux Server. For some this is enough to put off people from jumping in, or banging their head against the wall wondering why you cannot simply RDP into the server and where is the Admin Password for the Linux box. We are all used to what we are used to, and for many using a public/private key pair is part of the problem. We are not exposed to this in our daily lives, although with cloud computing via AWS you have to become smart in this process. The instructions at Amazon are not written in plain English and it’s hard to find the details.

Here are some handy steps that you can take to get access to your Amazon AWS server (Amazon Web Services) that takes place after you have set up your instance and downloaded your keys. You might want to read this article by Dave Winer to get through the set up of the AMI, downloading your keys and otherwise getting your first AWS instance set up and booted to the point where it is running. This is where this article steps in to help people connect to their Linux server from a Windows Box. For this process, we used the new Windows 7 as our Windows box, and all the software we recommend you download worked great.

When you are building out your system you want to have your instance up and running. You can use any AMI (Amazon Machine Image) that you want to use; in this case we are using the simple LAMP Linux version. You can find the LAMP AMI by searching for “LAMP”. There are some quick GotCha’s that you want to know about before you put a lot of time into an AMI getting it all prepped for what you want to do.

Windows or Linux does not matter, if you terminate the instance there is no way to restart it you will get the no valid actions error. When you terminate an instance you cannot restart it, it is gone, shutdown, lost forever. If you click on Launch, you will be prompted for a new image, regardless of what image you want to restart, so be very careful when you terminate an instance. If you terminate, it will be DOA for life. The image below shows what options you have using the AWS console provided by Amazon (via the Web). Remember, never click on Terminate unless you are completely done with the image and you never want to use it again.

A running Linux instance does not have the “get administrative password” option that Dave talks about in his EC2 for Poets article. With Linux there are some things you have to do to get access. If you search Amazon you will see a ton of words on this subject that again for the average person is a rambling diatribe written by computer geeks for computer geeks. They even offer you a set of tools to download, but if you are running windows many of these tools will not work properly. Here are the tools you need to get started.

Putty – Putty is an SSH terminal client that will tie in with another program that you will download here in a minute. Download PuTTY here.

PuttyGen – You can download this software from the same place you downloaded putty. You will want this program to convert your Amazon Key from Amazon format to PPK format that the next bit of software you will download needs to make the connection to your new Linux server.

WinSCP – WinSCP is a SFTP client and SSH client for Windows. Its main function is the secure file transfer between a local and a remote computer. It uses Secure Shell (SSH) and supports, in addition to Secure FTP, also legacy SCP protocol. You can download the software here.

You should have your private key from Amazon when you first started up your AMI. Find the Private key you downloaded from Amazon, or make another key if you have to. Making a key is very easy, you can use Dave’s article, or follow the directions from Amazon which are less cogent but still follow able.

Start the PuttyGen program as shown below.

[PuttyGen1.png]

Find the key that you made when you made your initial image and click on load. You will get a dialog box that wants you to load your key file that you got from Amazon. Find where you dropped your key file (again probably in downloads). Remember to set your file type to * to see all your files (otherwise it will default to looking for a PPK file, which you do not have yet).

Click on OK when you find and enter your file name. You should see the dialog below that shows the Amazon key was successfully converted from the Amazon format to the PPK format that WinSCP needs to work properly.

Click on OK.

Click on Save Private Key. You really need to save your private key at this point. Remember where you saved your brand new PPK extension key (if you forget where you saved it, search your computer for *.ppk). You have made it through the first hurdle, and now you need to fire up WinSCP.

WinSCP

If you skipped everything else and ended up here, WinSCP is a SFTP client and SSH client for Windows. Its main function is the secure file transfer between a local and a remote computer. It uses Secure Shell (SSH) and supports, in addition to Secure FTP, also legacy SCP protocol. You can download it from Source Forge.

Start (You should find it under start/all programs/WinSCP) WinSCP and you should see this after you have installed it

You only have to enter minimal data here, your host name is your EC2 instance, your user name is Root, leave the password BLANK, and click on the three dots for your private key file. If you do not know what your AWS instance is, you can go back to the AWS console and click on connect, you will get a connect help Secure Shell (SSH) help screen. This screen might look bad, but it has all the information you need for the Amazon instance you want to connect to. The screen below shows you what the help screen looks like.

The “enter the following command line” has a lot of information that you need. You can forget about everything before the “@”, you will not need that. Everything after the @ though is the public DNS that you will need to connect to your Amazon instance. You should copy everything after the @ symbol so that what you have looks like this

ec2-72-44-46-XXX.compute-1.amazonaws.com

This is your host name that you want to put into your WinSCP line for host name. The image below show the WinSCP entry screen with all the data filled out.

You want your WinSCP Login screen to look exactly like this, short of the instance name and the PPK key; yours will be different in terms of what is in the Host Name entry, and where you stored your own Private Key file. Remember that your Private Key file is the key you converted in PuttyGen. Click on Login and you will connect to your Amazon EC2 instance.

The good part is now you can tool around the Linux AMI using a graphical interface, and drag and drop files from your computer to the Amazon AMI that you are using.

In WinSCP on the top command ribbon bar you should see two icons as circled below. To get to the Command Line in Linux (remember this is why you downloaded Putty) you want to click on the two PC’s on the top command ribbon shown below.

This will launch the Putty command shell that will allow you to tool around the Amazon AMI. You need to be familiar with the Linux command line because you can do things here that you cannot do in the graphical interface, like bring up the update programs, tool around the OS to get an idea of where things are, use ps –ef |grep PROGRAM to see what programs are running (great for debugging MySQL), and to launch specific MySQL commands. This is what you should see when you click on the two computers graphic in the screen above.

Here are some discovered issues with the AMI that I built out.

MySQL has no password, it is root followed quickly by a blank password. You need to use the MySQL command line to set a password for this as soon as you can. The MySQL website can help you out setting the password.

The AMI in this instance when you look at the security group in the Amazon Web Services Console automatically opened up port 3306 for world wide access to your MySQL database. You want to take this rule out of the configuration, especially if you do not set a password. This is one excellent and easy way for hackers to get into your MySQL database; you need to fix this before you have the world hit your web site.

If you are interested in just web services you will find the root for HTTP under the home directory.

If you are interested in making sure that your Linux AMI is up to date, you need to use YUM, type in YUM update to update your whole computer or type in YUM update php if you want to make sure you are working with the latest version of PHP. Here is the catch, the YUM repo’s that come with the AMI we were using could not find any updates, and started throwing errors that it could not find any available mirrors. The problem with this is that you cannot update your software, which from a security viewpoint is very bad. If YUM is not finding updates, there are many resources on the internet to help you through this, the problem is that the repo’s that come with the image we were using goes right back to the Fedora repository, which is distressingly empty of updates for known vulnerable software. There are options on where you can go to get updates, but that will be another topic to talk about, updating your shiny new Amazon AMI for security when support is lacking from the people who make the software you are using.

If your program has dependencies on the version of PHP you are using (in my case the version was 5.0.4, and needed 5.1.X) you will have to hack your installation script if it checks the environmental variables for your PHP server. That means stepping through the entire install, and finding the call where it tests the PHP environment and changing that part of the install script. You can edit the file in WinSCP and save it back to the server, just remember what files you had to change (hack) when you get done.

Keywords:
windows amazon keyfile winscp
windows amazon key file winscp
winscp aws command line
winscp to linux ami

Categories
Freeware FTP Linux Notepad Replacement Notepad++ Utility Windows

Notepad++ – source code editor for Windows (FreeWare)

Notepad++ is a source code editor for Windows. It is distributed as free software. The project is hosted on SourceForge.net, from where it has been downloaded over sixteen million times.[1]

This project, based on the Scintilla editor component, is written in C++ with only Win32 API calls using only the STL in order to increase performance and reduce program size. The aim of Notepad++ is to reduce overall power consumption by using efficient binaries that require less CPU power.[2]

Notepad++ is available only for the Microsoft Windows operating system. However, users can still get Notepad++ to work on other platforms like Linux and Mac OS X, using software such as Wine.

Programming languages
Notepad++ supports syntax highlighting and syntax folding for 48 programming, scripting, and markup languages.[2] It attempts to automatically detect the language that a given file uses, using a modifiable list of file extension bindings. Users may also manually set the current language, overriding the extension’s default language. The program also supports autocompletion for a subset of the API of some programming languages.

Users can also define their own language (for syntax highlighting) and its respective API (for autocompletion) by using the built-in User Language Define System.[2] Users may configure the syntax highlighting’s font styles per element, per language, and the resulting formatted script may be printed in full-color (WYSIWYG). Additionally, Notepad++ displays indent guidelines when source code is indented with tab characters, and highlights closing braces, brackets and tags.

Some of its features include:

  • Auto-completion (language and file)
  • Bookmarks
  • Syntax highlighting (and brace and indent highlighting)
  • Drag-and-drop
  • File status auto-detection
  • Regular expression find and replace
  • Split screen editing (“View/Clone”) and synchronized scrolling
  • Zooming
  • Spell checker (requires Aspell)
  • Tabbed document interface
  • FTP Browser (plug-in included in standard installation)
  • Support for various file formats including unicode.
  • Find and replace text or text strings of text over multiple documents simultaneously.

Although Scintilla does not support multiline regular expression search and replace, Notepad++ does contain support for plugins which help mitigate this fact.

Notepad++ home page
Notepad++ download