Category: Grep

Categories
Grep grep Examples Linux

How To Use grep Command In Linux

The grep command searches the given file for lines containing a match to the given strings or words. By default, grep prints the matching lines. grep can be used to search for lines of text that match one or many regular expressions, and outputs only the matching lines.

grep Syntax

grep 'word' filename
grep 'string1 string2' filename
cat otherfile | grep 'something'
command | grep 'something'

grep Examples

  • Use grep to search a file
    Search /etc/passwd for user admin:

grep admin /etc/passwd

You can force grep to ignore word case i.e. match admin, Admin, ADMIN or any other combination with the -i option:

grep -i "admin" /etc/passwd

  • Use grep recursively
    You can search recursively with grep, i.e. read all files under each directory for a string “10.10.1.25”

grep -r "10.10.1.25" /etc

  • Use grep to search two different words

grep -w 'word1|word2' /path

  • Count number of lines where word(s) is/are matched
    grep can report the number of times that the pattern has been matched for each file using -c (count) option:

grep -c 'word' /path

Also note that you can use -n option, which causes grep to precede each line of output with the number of the line in the text file from which it was obtained:

grep -n 'word' /path

  • Grep invert match
    The -v option can be used to print inverts of the match; that is, it matches only those lines that do not contain the given word. For example print all lines that do not contain the word invert:

grep -v invert /path

  • Display CPU Model Name With grep

grep -i 'model' /proc/cpuinfo

  • Use grep to just list the names of matching files

grep -l 'word' *.c*

  • Set grep to display output in colors

grep --color admin /etc/passwd

Categories
BackTrack Grep Linux SSL Ubuntu USB VMWare

Building the Perfect Backtrack 4 USB Thumb Drive

This post will show you how to build a USB thumb drive with the following features:

  • Persistent Changes
  • Nessus and NessusClient installed
  • Encryption configured for storing data

Tools and Supplies

  1. USB thumbdrive – minimum capacity 4GB
  2. BackTrack 3 CDROM, BackTrack 4 DVD or an additional USB thumbdrive (minimum 2GB) – Used to partition the thumbdrive.
  3. Optional: UNetbootin – A tool to transfer an iso image to a USB drive.

Download the BackTrack ISO (BackTrack 4 Pre Release is the latest as of this posting)

This tutorial is based commands executed from BackTrack, so you will have to boot BackTrack 4 first. The easiest way to do this is to boot from the BackTrack 4 DVD, which is a live CD.

Partition the USB thumbdrive
First, boot to BackTrack 4. You will have to figure out which drive is the target drive. The following command will show the drives available and you can determine from that which is the new USB drive:

dmesg | egrep hd.|sd.

Partition and format the drive as follows:

  1. The first partition needs to be a primary partition of at least 1.5 GB and set to type vfat. Also remember to make this partition active when you are creating it. Otherwise you might have some boot problems.
  2. The second Partition can be the rest of the thumb drive.

Below are the steps to take to get the drive partitioned and formatted, and were taken from this video on the Offensive Security website. A ‘#‘ indicates a comment and is not part of the command and user typed commands are blue bold. Note, make sure to delete any existing partitions on the drive first.

fdisk /dev/sda # use the appropriate drive letter for your system
# delete existing partitions, of which there may be more than one. 

Command (m for help): d
Partition number (1-4): 1 

# create the first partition 

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-522, default 1): <enter>
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-522, default 522): +1500M 

#create the second partition 

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (193-522, default 193): <enter>
Using default value 193
Last cylinder, +cylinders or +size{K,M,G} (193-522, default 522): <enter>
Using default value 522

# Setting the partition type for the first partition to vfat/fat32 

Command (m for help): t
Partition number (1-4): 1
Hex code (type L to list codes): b
Changed system type of partition 1 to b (W95 FAT32) 

# Setting the partition type for the second partition to Linux 

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 83 

# Setting the first partition active 

Command (m for help): a
Partition number (1-4): 1 

Command (m for help): w 

# now it is time to format the partitions 

mkfs.vfat /dev/sdb1
mkfs.ext3 -b 4096 -L casper-rw /dev/sdb2

Two things to notice above in the format commands:

  1. We are using ext3 instead of ext2
  2. You must include the -L casper-rw portion of the command.

Being able to use ext3 is great because of journaling. The -L casper-rw option helps to get around the problem where we had to enter the partition name in order to get persistence working.

Partition and format the drive according the layout above.

Make it a bootable BackTrack 4 USB thumb drive
These steps are also taken from the Offensive Security video mentioned above.  They are:

  1. Mount the first partition
  2. Copy the BackTrack files to it
  3. Install grub

Execute the following commands.

# mount the first partition, sda1 in my case. 

mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1 

# copy the files, you will need to find where the ISO is mounted on your system. 

cd /mnt/sda1
rsync -r /media/cdrom0/* . 

# install grub 

grub-install --no-floppy --root-directory=/mnt/sda1 /dev/sda

Now you have a bootable BackTrack 4 USB thumb drive. Next let’s configure persistent changes.

Persistent Changes
This is done much differently and more easily than it was in BackTrack 4 Beta or BackTrack 3. First of all, for basic persistence, no configuration is required. There is already a menu option for this, however, it is only for console mode so you will have to make a couple changes:

  1. Change the default boot selection to persistent
  2. Set the resolution for the GUI

cd /mnt/sda1/boot/grub
vi menu.lst 

# change the default line below to ‘default 4' and append ‘vga=0×317' (that’s a zero) to the kernel line to set the resolution to 1024×768 

# By default, boot the first entry.
default 4
.
.
.
title Start Persistent Live CD
kernel /boot/vmlinuz BOOT=casper boot=casper persistent rw quiet vga=0×317
initrd /boot/initrd.gz

:wq

Reboot and either select “Start Persistent Live CD” or just wait since we set it to auto-boot to persistent mode. To testit, create a file and reboot again. If your file is still there, everything is working.

Install Nessus
Download the Ubuntu Nessus and NessusClient packages from nessus.org.

Again, with BackTrack 4 things are little easier. To install the Nessus server, simply execute the following command to install the package.

dpkg –install Nessus-4.0.2-ubuntu810_i386.deb

Things used to be a little bit more complicated for the client, but with the release of the pre-final version, it is just as easy as installing as the server.

dpkg –install NessusClient-4.0.2-ubuntu810_i386.deb

Finally it’s time to configure Nessus. Execute each of the following and follow the prompts.

#create server certificate 

/opt/nessus/sbin/nessus-mkcert

This script will now ask you the relevant information to create the SSL
certificate of Nessus. Note that this information will *NOT* be sent to
anybody (everything stays local), but anyone with the ability to connect to your Nessus daemon will be able to retrieve this information.
CA certificate life time in days [1460]:<enter>
Server certificate life time in days [365]:<enter>
Your country (two letter code) [FR]:US
Your state or province name [none]:State
Your location (e.g. town) [Paris]:City
Your organization [Nessus Users United]:<enter>
.
.
.
Congratulations. Your server certificate was properly created.
.
. 

# add user 

/opt/nessus/sbin/nessus-adduser 

Login :Admin
Authentication (pass/cert) : [pass]<enter>
Login password :
Login password (again) :
Do you want this user to be a Nessus ‘admin’ user ? (can upload plugins, etc…) (y/n) [n]:y
User rules
———-
nessusd has a rules system which allows you to restrict the hosts
that Me has the right to test. For instance, you may want
him to be able to scan his own host only.
Please see the nessus-adduser manual for the rules syntax
Enter the rules for this user, and enter a BLANK LINE once you are done :
(the user can have an empty rules set)
Login : Admin
Password : ***********
This user will have ‘admin’ privileges within the Nessus server
Rules :
Is that ok ? (y/n) [y]y
User added
We want to disable Nessus starting at boot. We are going to do some things a little later than require that Nessus not be running at boot. 

/usr/sbin/update-rc.d -f nessusd remove

This command does not remove the Nessus start scripts. It only removes the links that cause Nessus to start at boot time.

The next thing to do is register the installation so you can get the plugin feed. You need to go here and request
a key.

Once you have your key. Execute the following to update your plugins. Please note that there are two dashes before register in the nessus-fetch line below. They can display as one sometimes.

/opt/nessus/bin/nessus-fetch –register [your feed code here]

When that is done, and it is going to take a few minutes, you are ready to start the server and client. Be aware that with version 4.0, while the command to start returns quickly, the actual starting of the service may take a minute or two. You may have to reboot before Nessus will work. You can use netstat -na to check that the
server is listening on port 1241.

/etc/init.d/nessusd start
/opt/nessus/bin/NessusClient

Time to find those vulnerabilities.

Configure Encryption
Since BackTrack will be used to poke at peoples networks and systems, with permission of course, it is very important that the information we find be protected. To do this, we are going to setup an encrypted volume that will eventually become our home directory.

This can be done with the GUI or via command line. We will be using the gui because we need to be able to format the volume with ext3 and, as yet, I have not been able to figure out how to do that via the command line on linux.

  • Launch truecrypt from a terminal window.  
  • When truecrypt opens click the “Create Volume” button.  
  • In the Volume Location field enter the path to your volume, like /work_dir, click next.  
  • Leave the default Encryption Options & click next.  
  • Enter the volume size, say 1GB or so.  
  • Enter and confirm your desired password. 
  •  Select ext3 for the file system type, click next.  
  • Click next on the Cross-Platform Support page leaving the default values.  
  • Click format – you should move your mouse to create randomness for higher security.

You will get a message that the volume was successful created. Click on the ‘OK’ button, then exit the Truecrypt gui, both the ‘Create Volume’ windows and the main windows, going back to your terminal (command line) window.

To test the filesystem, execute the following, note the -k ” is two single quotes, not a double quote:

truecrypt -t -k ” --protect-hidden=no /work_dir /media/truecrypt1
mount
cd /media/truecrypt1
df .

This will show that the volume is mounted and the amount of disk space you have left. The next step is to have this volume mounted when you log in. This can be done by editing the root user’s .profile file. Add the truecrypt command above to root’s .profile so it looks like this:

# ~/.profile: executed by Bourne-compatible login shells.
if [ "$BASH" ]; then
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
fi
truecrypt -t -k '' --protect-hidden=no /my_secret_stuff /media/truecrypt1
mesg n

The next time you reboot you will be asked for the password for the volume and it will be mounted for you.

Now let’s tweak a few tings
The first thing we are going to do is configure networking to start at boot time. It’s convenient and easy to
disable later if necessary. All you have to do is execute the following command.

/usr/sbin/update-rc.d networking defaults

Next, make sure all the BackTrack tools and the system itself is up-to-date. First execute the following:

apt-get update

This is update the software repository information. Next, execute:

apt-get upgrade

The system will determine if there is anything that needs to be updated and then prompt you to continue. Individual packages can be updated by including the package name after upgrade.

Next reset the root user’s home directory during the login process to the mounted truecrypt volume. This will ensure that anything written to the home directory will be encrypted. Enter the following commands:

cd /media/truecrypt1
rsync -r –links /root/ . 

# add the bold lines below 

vi /root/.profile 

# ~/.profile: executed by Bourne-compatible login shells. 

if [ "$BASH" ]; then
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
fi
truecrypt -t -k '' --protect-hidden=no /my_secret_stuff /media/truecrypt1 

export HOME=/media/truecrypt1
export HISTFILE=/media/truecrypt1/.bash_history
cd 

mesg n 

:wq

The next time you reboot, when you are finally in the system, your home directory will be /media/truecrypt1.
There is one last thing we should do – change nessus to log to the encrypted volume. This is very easy. The
file that controls this is /opt/nessus/etc/nessus/nessusd.conf. We need to create a place for the log files to go. Execute the following:

cd /media/truecrypt1
mkdir -p nessus/logs 

Once you have done that, edit the /opt/nessus/etc/nessus/nessusd.conf file and change this:
.
.
.
# Log file :
logfile = /opt/nessus/var/nessus/logs/nessusd.messages
# Shall we log every details of the attack ? (disk intensive)
log_whole_attack = no
# Dump file for debugging output
dumpfile = /opt/nessus/var/nessus/logs/nessusd.dump
.
.
.

to this:

.
.
.
# Log file :
logfile = /media/truecrypt1/nessus/logs/nessusd.messages
# Shall we log every details of the attack ? (disk intensive)
log_whole_attack = no
# Dump file for debugging output
dumpfile = /media/truecrypt1/nessus/logs/nessusd.dump
.
.
.

That’s it. Now you have the Perfect Backtrack 4 USB Thumb Drive.

More BackTrack:

Categories
Command Line Grep Linux Windows

How to kill and logout a user in Linux

You can check who is logged in to a Linux system by executing

who or w commands

and then you can get the user PID

ps aux | grep username or tty

and kill it

kill -KILL PID

or you can use pkill command and kill the user by its username

pkill -KILL -u username

It would be nice to send the users a warning message before logging them out.

Categories
ethernet FREE FTP Grep Linux Networking NGrep Packet Sniffing SSH Utility Windows WinDump WinPCap WireShark

FREE Windows Utilities for Sniffing


WinDump
WinDump is an exceptionally powerful command-line packet sniffer. Ported over from the source of the Tcpdump utility available for UNIX, WinDump carries over the same power and flexibility to the Windows world, all in a lightweight executable.
WinDump is a helpful utility when you need to do some quick and easy packet capturing to diagnose a network problem. It’s flexible, and it can capture and display details about every packet going across the network interface. It can filter the output results based on protocol (e.g., Address Resolution Protocol—ARP, IP, TCP, UDP), source network or host, destination network or host, source port, destination port, and many other criteria.
To use WinDump, you’ll first need to download and install the Windows Packet Capture Library (WinPcap), the Windows port of the open-source packet-capture and network-analysis library libpcap for UNIX. WinPcap runs on all versions of Windows.
After you install WinPcap, you can download the standalone WinDump executable. To launch WinDump, simply run it from the command line with the appropriate options for what you’d like to capture or read. You’ll find the online manual for WinDump here.
The first command you might want to execute is Win-Dump -D, which will display a list of interfaces available on your computer, as well as a corresponding number for each interface, so that you can determine which interface to use for your sniffing activities. After you know which interface to run, you can simply use that number with the -i option (i.e., WinDump -i 3, if number 3 is the interface you’d like to use) to start viewing packet data in real time. (Because these are ports of UNIX utilities, the command-line switches are case-sensitive.)
Ngrep
Although WinDump is a tremendous utility, sometimes it requires a considerable amount of overhead or knowledge to determine what you’re looking for. For example, suppose you’re trying to look up whether a DNS query is making it across your network, but you aren’t familiar with the protocols and that DNS uses by default. Or, suppose a lot of traffic is coming across a network connection, and you’re finding it too cumbersome to work through all the packets just to find the one particular packet you’re looking for. For such situations enter ngrep, the network-aware grep utility.
If you aren’t familiar with grep, it’s probably one of the most widely known and oft-utilized UNIX utilities. Grep finds matching text strings (through a mechanism known as regular expressions) in files on a file system, then outputs the lines to the display. You might compare grep with the Windows command-line Find utility, but grep differs by providing an exceptional amount of power in its search for text strings.
By applying these capabilities to the network layer, ngrep provides the same level of functionality for packet sniffing. As a result, you don’t need to know what protocols, ports, network, or IP addresses that two devices are using to transfer data. You just need to know something about the packet’s payload, and ngrep will find it for you—regardless of how it’s transmitted.
Ngrep is great for troubleshooting DNS query problems. In a large Active Directory (AD) environment, dozens of DNS queries are typically occurring across the network per second. If I’m trying to troubleshoot a specific set of problems, searching each packet to find the one I’m looking for is cumbersome at best. Instead of relying on a straight packet capture of all DNS traffic, I can simply use ngrep to find the text string I’m looking for because DNS queries and responses are performed in plain text.
Ngrep currently recognizes ICMP, IGMP, Raw, TCP, and UDP protocols across 802.11, Ethernet, FDDI, PPP, SLIP, Token Ring, and null interfaces. Like WinDump it requires the WinPcap library to operate properly.
WireShark (formerly Ethereal)
When you face a situation in which you need to roll up your sleeves and dive as deeply as possible into network capture and analysis, one utility needs to come to mind: the world’s most popular network analyzer, WireShark. Network experts around the world use WireShark because it has all the standard features you’ll find in most protocol analyzers, in addition to some you won’t find in any other product. More than 400 developers around the world have made contributions to this open-source application. A decade ago, you would have to pay thousands of dollars for software that had the same capabilities, but WireShark offers it all free.
WireShark can capture data off your network connection, filter the data, dive into the details of each packet, save the packet capture for detailed analysis, send packet captures to other network engineers (or vendors) to help with debugging, and open packet captures from many other leading packet-capture utilities. WireShark can capture data off of various network transports, such as Classical IP over ATM (CIP), Ethernet, Fiber Distributed Data Interface (FDDI), Point-to-Point Protocol (PPP), Token Ring, 802.11, and loopback interfaces (although it doesn’t support all types on every platform). Across all those network transports, WireShark can “dissect” more than 750 protocol types, including FTP, HTTP, NetBIOS, POP3, remote procedure call (RPC), SNMP, SSH, SMTP, and Telnet, just to name a few.
Like the other sniffing utilities I’ve mentioned, WireShark depends on an installation of WinPcap to function properly, so you’ll need to install that first. Then, after you install the latest Ethereal distribution for Windows, simply access the Capture, Interfaces menu and select the interface you want to start using for capture. Then, you’re ready to start analyzing your traffic.
Categories
Administration BareTail Encryption FREE FTP Grep Linux Notepad++ SSH Utility Windows

32 FREE Windows Administration Utilities

  1. BareTail. A free real-time log file monitoring tool.
  2. CamStudio. Free screen recording software.
  3. CDBurnerXP. Burns CD-ROMs, DVDs, audio CDs, and ISO images.
  4. Comodo Firewall Pro. Is a firewall and antivirus application.
  5. DriveImage XML. Is a program for imaging and backing up partitions and logical drives.
  6. FileZilla. GUI FTP client.
  7. GParted LiveCD. Manages partitions on systems.
  8. InfraRecorder. Burns ISO images and creates data and audio CDs and DVDs.
  9. Lansweeper. Is a network inventory tool that performs hardware scanning, software scanning, and Active Directory (AD) reporting.
  10. LocatePC. Emails you whenever any private or public IP address in your system changes – great for tracking a stolen computer.
  11. MyDefrag (formerly JkDefrag). Defragments and optimizes disks.
  12. Nessus (formerly NeWT). Network/computer vulnerability scanner.
  13. Ngrep. Is a packet sniffer based on finding matching text strings.
  14. Notepad++. Is a text and code editor (more info).
  15. NTFS Undelete. Recovers deleted files that are no longer in the recycle bin.
  16. Open Computers and Software Inventory (OCS Inventory NG). Provides detailed inventory data for an entire network of computers as well as deploys packages.
  17. OpenSSH. Creates secure, encrypted shell sessions.
  18. PageDefrag. Determines how fragmented your paging files and registry hives are, and defragments them.
  19. Paint.NET. Free image and photo editing software for Windows.
  20. PING (Partimage Is Not Ghost) — Backup and Restore Disk Partitions.
  21. PRTG Traffic Grapher. Is a powerful network monitor.
  22. System Information for Windows (SIW). Gathers detailed information about a computer’s system properties, settings, and displays.
  23. TestDisk. Recovers damaged partitions, makes non-bootable disks bootable again, and repairs damaged boot sectors.
  24. TrueCrypt.  Free open-source disk encryption software for Windows 7/Vista/XP, Mac OS X, and Linux.
  25. WinDirStat. Determines how space is being utilized across disks and visually represents the results in multiple ways.
  26. WinPE (Windows Preinstallation Environment). Lets you make a Windows command-line boot recovery DVD.
  27. WinDump. WinDump is the Windows version of tcpdump, the command line network analyzer for UNIX / Linux.
  28. Winfingerprint. Is a network scanner.
  29. Wink. Builds screencast recordings.
  30. WireShark (formerly Ethereal). Network protocol analyzer.
  31. XML Notepad. Is a specialized XML editor.
  32. ZoomIt. Magnifies portions of a screen and lets you draw on and annotate the screen.
Categories
Grep Linux netstat TCP/IP

Get Detailed Information About Particular IP address Connections Using netstat

netstat command and shell pipe feature can be used to dig out more information about particular IP address connection. You can find out total established connections, closing connection, SYN and FIN bits and much more. You can also display summary statistics for each protocol using netstat.

This is useful to find out if your server is under attack or not. You can also list abusive IP address using this method.
# netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n

Output:

1 CLOSE_WAIT
      1 established)
      1 Foreign
      3 FIN_WAIT1
      3 LAST_ACK
     13 ESTABLISHED
     17 LISTEN
    154 FIN_WAIT2
    327 TIME_WAIT

Dig out more information about a specific ip address:
# netstat -nat |grep {IP-address} | awk '{print $6}' | sort | uniq -c | sort -n

2 LAST_ACK
      2 LISTEN
      4 FIN_WAIT1
     14 ESTABLISHED
     91 TIME_WAIT
    130 FIN_WAIT2

Busy server can give out more information:
# netstat -nat |grep 202.54.1.10 | awk '{print $6}' | sort | uniq -c | sort -n

Output:

15 CLOSE_WAIT
  37 LAST_ACK
  64 FIN_WAIT_1
  65 FIN_WAIT_2
1251 TIME_WAIT
3597 SYN_SENT
5124 ESTABLISHED

Get List Of All Unique IP Address

To print list of all unique IP address connected to server, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq

To print total of all unique IP address, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq | wc -l

Output:

449

Find Out If Box is Under DoS Attack or Not

If you think your Linux box is under attack, print out a list of open connections on your box and sorts them by according to IP address, enter:
# netstat -atun | awk '{print $5}' | cut -d: -f1 | sed -e '/^$/d' |sort | uniq -c | sort -n

Output:

1 10.0.77.52
      2 10.1.11.3
      4 12.109.42.21
      6 12.191.136.3
.....
...
....
    13 202.155.209.202
     18 208.67.222.222
     28 0.0.0.0
    233 127.0.0.1

You can simply block all abusive IPs using iptables or just null route them.

Get Live View of TCP Connections

You can use tcptrack command to display the status of TCP connections that it sees on a given network interface. tcptrack monitors their state and displays information such as state, source/destination addresses and bandwidth usage in a sorted, updated list very much like the top command.

Display Summary Statistics for Each Protocol

Simply use netstat -s:
# netstat -s | less
# netstat -t -s | less
# netstat -u -s | less
# netstat -w -s | less
# netstat -s

Output:

Ip:
    88354557 total packets received
    0 forwarded
    0 incoming packets discarded
    88104061 incoming packets delivered
    96037391 requests sent out
    13 outgoing packets dropped
    66 fragments dropped after timeout
    295 reassemblies required
    106 packets reassembled ok
    66 packet reassembles failed
    34 fragments failed
Icmp:
    18108 ICMP messages received
    58 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 7173
        timeout in transit: 472
        redirects: 353
        echo requests: 10096
    28977 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 18881
        echo replies: 10096
Tcp:
    1202226 active connections openings
    2706802 passive connection openings
    7394 failed connection attempts
    47018 connection resets received
    23 connections established
    87975383 segments received
    95235730 segments send out
    681174 segments retransmited
    2044 bad segments received.
    80805 resets sent
Udp:
    92689 packets received
    14611 packets to unknown port received.
    0 packet receive errors
    96755 packets sent
TcpExt:
    48452 invalid SYN cookies received
    7357 resets received for embryonic SYN_RECV sockets
    43 ICMP packets dropped because they were out-of-window
    5 ICMP packets dropped because socket was locked
    2672073 TCP sockets finished time wait in fast timer
    441 time wait sockets recycled by time stamp
    368562 delayed acks sent
    430 delayed acks further delayed because of locked socket
    Quick ack mode was activated 36127 times
    32318597 packets directly queued to recvmsg prequeue.
    741479256 packets directly received from backlog
    1502338990 packets directly received from prequeue
    18343750 packets header predicted
    10220683 packets header predicted and directly queued to user
    17516622 acknowledgments not containing data received
    36549771 predicted acknowledgments
    102672 times recovered from packet loss due to fast retransmit
    Detected reordering 1596 times using reno fast retransmit
    Detected reordering 1 times using time stamp
    8 congestion windows fully recovered
    32 congestion windows partially recovered using Hoe heuristic
    19 congestion windows recovered after partial ack
    0 TCP data loss events
    39951 timeouts after reno fast retransmit
    29653 timeouts in loss state
    197005 fast retransmits
    186937 retransmits in slow start
    131433 other TCP timeouts
    TCPRenoRecoveryFail: 20217
    147 times receiver scheduled too late for direct processing
    29010 connections reset due to unexpected data
    365 connections reset due to early user close
    6979 connections aborted due to timeout

Display Interface Table

You can easily display dropped and total transmitted packets with netstat for eth0:
# netstat --interfaces eth0

Output:

Kernel Interface table
Iface       MTU Met    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0       1500   0  2040929      0      0      0  3850539      0      0      0 BMRU

Other netstat related articles / tips:

  1. Get Information about All Running Services Remotely
  2. Linux / UNIX Find Out What Program / Service is Listening on a Specific TCP Port

Read following man pages for the details:
$ man netstat
$ man cut
$ man awk
$ man sed
$ man grep

Categories
Grep Linux Passwords

Linux Upgrade Password Hashing Algorithm to SHA-512

The default algorithm for storing password hashes in /etc/shadow is MD5. I was told to use SHA-512 hashing algorithm. How do I set password hashing using the SHA-256 and SHA-512 under CentOS or Redhat Enterprise Linux 5.4?

You need to use authconfig command to setup SHA-256/512 hashing. This command provides a simple method of configuring /etc/sysconfig/network to handle NIS, as well as /etc/passwd and /etc/shadow, the files used for shadow password support. Basic LDAP, Kerberos 5, and SMB (authentication) client configuration is also provided.

Display Current Hashing Algorithm

Type the following command:
# authconfig --test | grep hashing

Sample outputs:

password hashing algorithm is md5

Configure Linux Server To Use The SHA-512

To configure the Linux system to use the SHA-512 algorithm, enter:
# authconfig --passalgo=sha512 --update

Note users need to change their passwords in order to generate hashes using SHA-512. You can force users to change their password on next login:
# chage -d 0 userName

Categories
Grep Linux SSH

Linux / UNIX Find Out What Program / Service is Listening on a Specific TCP Port

How do I find out which service is listening on a specific port? How do I find out what program is listening on a specific TCP Port?

Under Linux and UNIX you can use any one of the following command to get listing on a specific TCP port:

  • lsof : list open files including ports.
  • netstat : The netstat command symbolically displays the contents of various network-related data and information.

lsof command example
Type the following command to see IPv4 port(s), enter:
# lsof -Pnl +M -i4

Type the following command to see IPv6 listing port(s), enter:
# lsof -Pnl +M -i6

Sample output:

COMMAND    PID     USER   FD   TYPE DEVICE SIZE NODE NAME
gweather- 6591     1000   17u  IPv4 106812       TCP 192.168.1.100:57179->140.90.128.70:80 (ESTABLISHED)
firefox-b 6613     1000   29u  IPv4 106268       TCP 127.0.0.1:60439->127.0.0.1:3128 (ESTABLISHED)
firefox-b 6613     1000   31u  IPv4 106321       TCP 127.0.0.1:60440->127.0.0.1:3128 (ESTABLISHED)
firefox-b 6613     1000   44u  IPv4 106325       TCP 127.0.0.1:60441->127.0.0.1:3128 (ESTABLISHED)
firefox-b 6613     1000   50u  IPv4 106201       TCP 127.0.0.1:60437->127.0.0.1:3128 (ESTABLISHED)
deluge    6908     1000    8u  IPv4  23179       TCP *:6881 (LISTEN)
deluge    6908     1000   30u  IPv4  23185       UDP *:6881
deluge    6908     1000   45u  IPv4 106740       TCP 192.168.1.100:50584->217.169.223.161:38406 (SYN_SENT)
deluge    6908     1000   57u  IPv4  70529       TCP 192.168.1.100:57325->24.67.82.222:21250 (ESTABLISHED)
deluge    6908     1000   58u  IPv4 106105       TCP 192.168.1.100:38073->24.16.233.1:48479 (ESTABLISHED)
..........
......
ssh       6917     1000    3u  IPv4  23430       TCP 10.1.11.3:42658->10.10.29.66:22 (ESTABLISHED)

First column COMMAND – gives out information about program name. Please see output header for details. For example, gweather* command gets the weather report weather information from the U.S National Weather Service (NWS) servers (140.90.128.70), including the Interactive Weather Information Network (IWIN) and other weather services.
Where,

  1. -P : This option inhibits the conversion of port numbers to port names for network files. Inhibiting the conver-
    sion may make lsof run a little faster. It is also useful when port name lookup is not working properly.
  2. -n : This option inhibits the conversion of network numbers to host names for network files. Inhibiting conversion may make lsof run faster. It is also useful when host name lookup is not working properly.
  3. -l : This option inhibits the conversion of user ID numbers to login names. It is also useful when login name lookup is working improperly or slowly.
  4. +M : Enables the reporting of portmapper registrations for local TCP and UDP ports.
  5. -i4 : IPv4 listing only
  6. -i6 : IPv6 listing only

netstat command example

Type the command as follows:
# netstat -tulpn

OR
# netstat -npl

Output:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:6881            0.0.0.0:*               LISTEN     6908/python
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN     5562/cupsd
tcp        0      0 127.0.0.1:3128          0.0.0.0:*               LISTEN     6278/(squid)
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN     5854/exim4
udp        0      0 0.0.0.0:32769           0.0.0.0:*                          6278/(squid)
udp        0      0 0.0.0.0:3130            0.0.0.0:*                          6278/(squid)
udp        0      0 0.0.0.0:68              0.0.0.0:*                          4583/dhclient3
udp        0      0 0.0.0.0:6881            0.0.0.0:*                          6908/python

Last column PID/Program name gives out information regarding program name and port.
Where,

  • -t : TCP port
  • -u : UDP port
  • -l : Show only listening sockets.
  • -p : Show the PID and name of the program to which each socket / port belongs
  • -n : No DNS lookup (speed up operation)

/etc/services file

/etc/services is a plain ASCII file providing a mapping between friendly textual names for internet services, and their underlying assigned port numbers and protocol types. Every networking program should look into this file to get the port number (and protocol) for its service. You can view this file with the help of cat or less command:
$ cat /etc/services
$ grep 110 /etc/services
$ less /etc/services

Categories
Grep Linux Security

Understanding Linux /etc/passwd File Format

/etc/passwd file stores essential information, which is required during login i.e. user account information.
/etc/passwd is a text file, that contains a list of the system’s accounts, giving for each account some useful information like user ID, group ID, home directory, shell, etc. It should have general read permission as many utilities, like ls use it to map user IDs to user names, but write access only for the superuser (root).

Understanding fields in /etc/passwd

The /etc/passwd contains one entry per line for each user (or user account) of the system. All fields are separated by a colon (:) symbol. Total seven fields as follows.
Generally, passwd file entry looks as follows:

  1. Username: It is used when user logs in. It should be between 1 and 32 characters in length.
  2. Password: An x character indicates that encrypted password is stored in /etc/shadow file.
  3. User ID (UID): Each user must be assigned a user ID (UID). UID 0 (zero) is reserved for root and UIDs 1-99 are reserved for other predefined accounts. Further UID 100-999 are reserved by system for administrative and system accounts/groups.
  4. Group ID (GID): The primary group ID (stored in /etc/group file)
  5. User ID Info: The comment field. It allow you to add extra information about the users such as user’s full name, phone number etc. This field use by finger command.
  6. Home directory: The absolute path to the directory the user will be in when they log in. If this directory does not exists then users directory becomes /
  7. Command/shell: The absolute path of a command or shell (/bin/bash). Typically, this is a shell. Please note that it does not have to be a shell.

/etc/passwd is only used for local users only. To see list of all users, enter:
$ cat /etc/passwd

To search for a username called tom, enter:
$ grep tom /etc/passwd

/etc/passwd file permission

The permission on the /etc/passwd file should be read only to users (-rw-r–r–) and the owner must be root:
$ ls -l /etc/passwd
Output:

-rw-r--r-- 1 root root 2659 Sep 17 01:46 /etc/passwd

Your password is stored in /etc/shadow file

Your encrpted password is not stored in /etc/passwd file. It is stored in /etc/shadow file. In the good old days there was no great problem with this general read permission. Everybody could read the encrypted passwords, but the hardware was too slow to crack a well-chosen password, and moreover, the basic assumption used to be that of a friendly user-community.

Almost, all modern Linux / UNIX line operating systems use some sort of the shadow password suite, where /etc/passwd has asterisks (*) instead of encrypted passwords, and the encrypted passwords are in /etc/shadow which is readable by the superuser only.

Categories
Amazon Web Services AWS CLI cloud computing Command Line EC2 FTP Grep Linux Passwords PuTTY PuTTYgen SSH Windows winscp

Connecting to Amazon AWS from Windows to a Linux AMI

Connecting from Windows to a Linux Server poses problems for the average person to connect, and use their Windows OS to connect to the Linux Server. For some this is enough to put off people from jumping in, or banging their head against the wall wondering why you cannot simply RDP into the server and where is the Admin Password for the Linux box. We are all used to what we are used to, and for many using a public/private key pair is part of the problem. We are not exposed to this in our daily lives, although with cloud computing via AWS you have to become smart in this process. The instructions at Amazon are not written in plain English and it’s hard to find the details.

Here are some handy steps that you can take to get access to your Amazon AWS server (Amazon Web Services) that takes place after you have set up your instance and downloaded your keys. You might want to read this article by Dave Winer to get through the set up of the AMI, downloading your keys and otherwise getting your first AWS instance set up and booted to the point where it is running. This is where this article steps in to help people connect to their Linux server from a Windows Box. For this process, we used the new Windows 7 as our Windows box, and all the software we recommend you download worked great.

When you are building out your system you want to have your instance up and running. You can use any AMI (Amazon Machine Image) that you want to use; in this case we are using the simple LAMP Linux version. You can find the LAMP AMI by searching for “LAMP”. There are some quick GotCha’s that you want to know about before you put a lot of time into an AMI getting it all prepped for what you want to do.

Windows or Linux does not matter, if you terminate the instance there is no way to restart it you will get the no valid actions error. When you terminate an instance you cannot restart it, it is gone, shutdown, lost forever. If you click on Launch, you will be prompted for a new image, regardless of what image you want to restart, so be very careful when you terminate an instance. If you terminate, it will be DOA for life. The image below shows what options you have using the AWS console provided by Amazon (via the Web). Remember, never click on Terminate unless you are completely done with the image and you never want to use it again.

A running Linux instance does not have the “get administrative password” option that Dave talks about in his EC2 for Poets article. With Linux there are some things you have to do to get access. If you search Amazon you will see a ton of words on this subject that again for the average person is a rambling diatribe written by computer geeks for computer geeks. They even offer you a set of tools to download, but if you are running windows many of these tools will not work properly. Here are the tools you need to get started.

Putty – Putty is an SSH terminal client that will tie in with another program that you will download here in a minute. Download PuTTY here.

PuttyGen – You can download this software from the same place you downloaded putty. You will want this program to convert your Amazon Key from Amazon format to PPK format that the next bit of software you will download needs to make the connection to your new Linux server.

WinSCP – WinSCP is a SFTP client and SSH client for Windows. Its main function is the secure file transfer between a local and a remote computer. It uses Secure Shell (SSH) and supports, in addition to Secure FTP, also legacy SCP protocol. You can download the software here.

You should have your private key from Amazon when you first started up your AMI. Find the Private key you downloaded from Amazon, or make another key if you have to. Making a key is very easy, you can use Dave’s article, or follow the directions from Amazon which are less cogent but still follow able.

Start the PuttyGen program as shown below.

[PuttyGen1.png]

Find the key that you made when you made your initial image and click on load. You will get a dialog box that wants you to load your key file that you got from Amazon. Find where you dropped your key file (again probably in downloads). Remember to set your file type to * to see all your files (otherwise it will default to looking for a PPK file, which you do not have yet).

Click on OK when you find and enter your file name. You should see the dialog below that shows the Amazon key was successfully converted from the Amazon format to the PPK format that WinSCP needs to work properly.

Click on OK.

Click on Save Private Key. You really need to save your private key at this point. Remember where you saved your brand new PPK extension key (if you forget where you saved it, search your computer for *.ppk). You have made it through the first hurdle, and now you need to fire up WinSCP.

WinSCP

If you skipped everything else and ended up here, WinSCP is a SFTP client and SSH client for Windows. Its main function is the secure file transfer between a local and a remote computer. It uses Secure Shell (SSH) and supports, in addition to Secure FTP, also legacy SCP protocol. You can download it from Source Forge.

Start (You should find it under start/all programs/WinSCP) WinSCP and you should see this after you have installed it

You only have to enter minimal data here, your host name is your EC2 instance, your user name is Root, leave the password BLANK, and click on the three dots for your private key file. If you do not know what your AWS instance is, you can go back to the AWS console and click on connect, you will get a connect help Secure Shell (SSH) help screen. This screen might look bad, but it has all the information you need for the Amazon instance you want to connect to. The screen below shows you what the help screen looks like.

The “enter the following command line” has a lot of information that you need. You can forget about everything before the “@”, you will not need that. Everything after the @ though is the public DNS that you will need to connect to your Amazon instance. You should copy everything after the @ symbol so that what you have looks like this

ec2-72-44-46-XXX.compute-1.amazonaws.com

This is your host name that you want to put into your WinSCP line for host name. The image below show the WinSCP entry screen with all the data filled out.

You want your WinSCP Login screen to look exactly like this, short of the instance name and the PPK key; yours will be different in terms of what is in the Host Name entry, and where you stored your own Private Key file. Remember that your Private Key file is the key you converted in PuttyGen. Click on Login and you will connect to your Amazon EC2 instance.

The good part is now you can tool around the Linux AMI using a graphical interface, and drag and drop files from your computer to the Amazon AMI that you are using.

In WinSCP on the top command ribbon bar you should see two icons as circled below. To get to the Command Line in Linux (remember this is why you downloaded Putty) you want to click on the two PC’s on the top command ribbon shown below.

This will launch the Putty command shell that will allow you to tool around the Amazon AMI. You need to be familiar with the Linux command line because you can do things here that you cannot do in the graphical interface, like bring up the update programs, tool around the OS to get an idea of where things are, use ps –ef |grep PROGRAM to see what programs are running (great for debugging MySQL), and to launch specific MySQL commands. This is what you should see when you click on the two computers graphic in the screen above.

Here are some discovered issues with the AMI that I built out.

MySQL has no password, it is root followed quickly by a blank password. You need to use the MySQL command line to set a password for this as soon as you can. The MySQL website can help you out setting the password.

The AMI in this instance when you look at the security group in the Amazon Web Services Console automatically opened up port 3306 for world wide access to your MySQL database. You want to take this rule out of the configuration, especially if you do not set a password. This is one excellent and easy way for hackers to get into your MySQL database; you need to fix this before you have the world hit your web site.

If you are interested in just web services you will find the root for HTTP under the home directory.

If you are interested in making sure that your Linux AMI is up to date, you need to use YUM, type in YUM update to update your whole computer or type in YUM update php if you want to make sure you are working with the latest version of PHP. Here is the catch, the YUM repo’s that come with the AMI we were using could not find any updates, and started throwing errors that it could not find any available mirrors. The problem with this is that you cannot update your software, which from a security viewpoint is very bad. If YUM is not finding updates, there are many resources on the internet to help you through this, the problem is that the repo’s that come with the image we were using goes right back to the Fedora repository, which is distressingly empty of updates for known vulnerable software. There are options on where you can go to get updates, but that will be another topic to talk about, updating your shiny new Amazon AMI for security when support is lacking from the people who make the software you are using.

If your program has dependencies on the version of PHP you are using (in my case the version was 5.0.4, and needed 5.1.X) you will have to hack your installation script if it checks the environmental variables for your PHP server. That means stepping through the entire install, and finding the call where it tests the PHP environment and changing that part of the install script. You can edit the file in WinSCP and save it back to the server, just remember what files you had to change (hack) when you get done.

Keywords:
windows amazon keyfile winscp
windows amazon key file winscp
winscp aws command line
winscp to linux ami