IOMeter on CIFS share

If you are trying to run IOMeter on network mounted drives, you might find that IOMeter disk targets doesn't show network drives. And most search results will tell you that IOMeter cannot be run on CIFS share

After playing with it for few hours, I found that older version of IOMeter (2006) does work with CIFS shares. Here is the download link - http://www.iometer.org/doc/downloads.html

1. Downloand IOMeter 2006 and install
2. Mount your network drive
3. Provide necessary permissions
4. Launch IOMeter after finishing Steps 2 and 3
5. Now you should see network drives in disk targets on IOMeter


Logrotate Windows files from Linux

I could not find a simple and free application to rotate log files on Windows. So, I used Linux to rotate log files on my Windows machine.

1. Configure the Windows Folder which has the log files, for sharing

2. Enable Read/Write Permissions
3. Mount the Windows Folder on Linux. Check out my post on how to Share Windows files with Linux - Share Windows Files with Linux

4. Now, create a file in /etc/logrotate.d to do the rotation. 

    In the below config, I am rotating the log files daily and saving the previous five copies

    root@localhost# more /etc/logrotate.d/winfiles

    /mnt/win/files.txt {
        daily
        rotate 5
        missingok
        compress
    }

Share Windows Files with Linux

- Install mount.cifs

        yum install cifs-utils -y

- Create a directory where you want the Windows folder 

root@localhost#mkdir /mnt/win

- Use mount.cifs  with the windows machine IP, folder and credentials to mount

root@localhost#sudo mount.cifs //10.4.23.67/Syslogs/ /mnt/win/ -o user=administrator,pass=password

root@localhost#
root@localhost#ls /mnt/win/
Documents Downloads
root@localhost#


ipv4: Neighbour table overflow


I was noticing the below error message in /var/log/messages. I also noticed that my linux DHCP server was not serving leases. Again, I modified the sysctl file to fix the issue.

Problem:
Jun 21 07:56:03 localhost kernel: [36050.926734] ipv4: Neighbour table overflow.
Jun 21 07:56:03 localhost kernel: [36050.930354] ipv4: Neighbour table overflow.
Jun 21 08:01:02 localhost kernel: [36349.592127] ipv4: Neighbour table overflow.

Solution:
1. Edit /etc/sysctl.conf
 vi /etc/sysctl.conf2. Change to have the following values

net.ipv4.neigh.default.gc_thresh1 = 8192
net.ipv4.neigh.default.gc_thresh2 = 16384
net.ipv4.neigh.default.gc_thresh3 = 16384

2. To load new changes type the following command:

  sysctl -p

What are these 3 parameters?

These parameters are defined in the Linux Kernel Code in /include/net/neighbour.h as integer, which suggests that maximal accepted value is (2^32 - 1)

gc_thresh1 - The minimum number of entries to keep in the ARP cache.

gc_thresh2 - The soft maximum number of entries to keep in the ARP cache. 

gc_thresh3 - The hard maximum number of entries to keep in the ARP cache. 



No buffer space available

I was noticing the below error message when I try to ping or telnet/ssh from my Linux machine. I also noticed similar error messages in /var/log/messages.

I modified the sysctl file to fix the issue.


Problem:
---------
May 29 16:27:01 localhost dhcpd: icmp_echorequest 155.1.19.178: No buffer space available
May 29 16:27:01 localhost kernel: [11032.635447] ipv4: Neighbour table overflow.

Solution:
---------
1. Edit /etc/sysctl.conf

 vi /etc/sysctl.conf

2. Change to have the following values

net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.core.wmem_max = 12582912
net.core.rmem_max = 12582912
net.ipv4.tcp_rmem = 10240 87380 12582912
net.ipv4.tcp_wmem = 10240 87380 12582912
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.neigh.default.gc_thresh1 = 4096
net.ipv4.neigh.default.gc_thresh2 = 8192
net.ipv4.neigh.default.gc_thresh3 = 8192
net.ipv4.neigh.default.base_reachable_time = 86400
net.ipv4.neigh.default.gc_stale_time = 86400

3. To load new changes type the following command:

  sysctl -p

Linux Networking Tips


How to grant privileged access to specific users?
You can grant specific users full access to all privileged commands, with this sudoers entry. (add it to the file: /etc/sudoers).
kranti  ALL=(ALL) ALL
It is not recommend because this allows user kranti to use the su command to get permanent root privileges thereby bypassing the command logging features of sudo.

How to disable/enable firewall in Linux (Fedora)?
Disable -
    # service iptables stop
Enable -
    # service iptables start
Disable at boot time
     # chkconfig iptables off
Enable at boot time
     # chkconfig iptables on

Configuring Static IP address on Linux from the Command Line (Fedora)
1. Assign static IP Address
vi /etc/sysconfig/network-scripts/ifcfg-em1
DEVICE="em1"
ONBOOT=yes
NM_CONTROLLED="yes"
TYPE=Ethernet
BOOTPROTO=none
IPADDR0=172.19.3.1
PREFIX0=24
GATEWAY0=172.19.3.254
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System em1"
UUID=1dad842d-1912-ef5a-a43a-bc238fb267e7
HWADDR=00:C0:9F:BA:C5:32

2. Restart the NIC
service network restart

3. Add the Default Route:
Default Route:
route add default gw ip_address_of_your_default_gateway em1
route add default gw 172.16.3.1 em1

To make gw/route permanent:
vi /etc/sysconfig/network

change to look like this:
NETWORKING=yes
HOSTNAME= localhost.localdomain
GATEWAY=172.19.3.254

Add default route through an interface (Fedora)
-> route add -net 172.0.0.0 netmask 255.0.0.0 dev em1
-> route -n
Kernel IP routing table
Destination     Gateway      Genmask      Flags Metric Ref   Use  Iface
172.0.0.0          0.0.0.0         255.0.0.0       U       0          0       0     em1

4. Add Permanent Route

Set up static routes for certain network interface (for example eth1) by editing file /etc/sysconfig/network-scripts/route-eth1.

For example, if you have to save static route added by the following command:
route add -net 192.168.0.0 netmask 255.255.255.0 gw 192.168.100.1 dev eth1

To do it, just add the following line to /etc/sysconfig/network-scripts/route-eth1:
ADDRESS0=192.168.0.0
NETMASK0=255.255.255.0
GATE
WAY0=192.168.100.1


Configure DHCP Server

To start, stop, restart and check status of dhcpd:
- service dhcpd start
- service dhcpd stop
- service dhcpd restart
- service dhcpd status

To ensure dhcpd starts automatically on your next reboot:
- chkconfig dhcpd on

DHCP server config file:
- Find sample config file @ /usr/share/doc/dhcp*/dhcpd.conf.sample
- The actual config file @ /etc/dhcp/dhcpd.conf

Test config file for errors:
- /usr/sbin/dhcpd –f

To check log messages:
- cat /var/log/messages

To check the assigned leases:
- cat /var/lib/dhcpd/dhcpd.leases

To configure to listen on an interface:
- vi /etc/sysconfig/dhcpd
        DHCPDARGS="em1"
- If not configured the following error will be thrown in the error log
        Not configured to listen on any interfaces!

Other errors:
1. No subnet declaration for em1 (172.19.3.1).
** Ignoring requests on em1.  If this is not what you want, please write a subnet declaration in your dhcpd.conf file for the network segment to which interface em1 is attached. **
- Add subnet declaration for 172.19.3.0 in /etc/dhcp/dhcpd.conf file

2. "DHCPDISCOVER from 00:00:aa:aa:aa:aa via 172.19.5.1: unknown network segment"
- Mostly because the subnet definition is missing for the particular subnet that the DHCP request is coming from (Most of the time because of DHCP helpers on switches). Add  subnet declaration to your dhcpd.conf file for this subnet

To Change the MTU of an interface on Linux:

 - sudo ifconfig eth0 mtu 2000

IP Addressing


IP addresses are described as consisting of two groups of bits in the address:
  - the most significant part is the network address which identifies a whole network or subnet
  - and the least significant portion is the host identifier, which specifies a particular host interface on that network.
This division is used as the basis of traffic routing between IP networks and for address allocation policies.

Classful Addressing -

 - Classful network design for IPv4 sized the network address as one or more 8-bit groups, resulting in the blocks of Class A, B, or C addresses.

Class
First Byte
Network ID / Host ID (Bytes)
Theoretical IP Address Range
Class A
0xxx xxxx
1 / 3
1.0.0.0 to 126.255.255.255
Class B
10xx xxxx
2 / 2
128.0.0.0 to 191.255.255.255
Class C
110x xxxx
3 / 1
192.0.0.0 to 223.255.255.255
Class D
1110 xxxx
224.0.0.0 to 239.255.255.255
Class E
1111 xxxx
240.0.0.0 to 255.255.255.255

Classless Inter-Domain Routing -

 - CIDR allocates address space to Internet service providers and end users on any address bit boundary, instead of on 8-bit segments
 - CIDR specifies an IP address range using a combination of an IP address and its associated network mask. CIDR notation uses the following format - xxx.xxx.xxx.xxx/n, where n is the number of (leftmost) '1' bits in the mask.
 - For example, 192.168.12.0/23 applies the network mask 255.255.254.0 to the 192.168 network, starting at 192.168.12.0. This notation represents the address range 192.168.12.0 - 192.168.13.255.


IP Address FAQs

1. In general, the first address (ending with 0) and last address (ending with 255) in a subnet are used as the network identifier and broadcast address, respectively. All other addresses in the subnet can be assigned to hosts on that subnet.

2. Private IP address ranges (RFC 1918) 
    a.  Class A: 10.0.0.0 through 10.255.255.255
b. Class B: 172.16.0.0 through 172.31.255.255
c. Class C: 192.168.0.0 through 192.168.255.255

3. Special IP addresses 
    a.  127.0.0.1 – Loopback address
    b.  224.0.0.0 – 239.255.255.255 – Multicast addresses
    c.  0.0.0.0 – Wildcard address

Interframe Gap and Throughput

Ethernet devices must allow a minimum idle period between transmission of frames known as the interframe gap (IFG) or interpacket gap (IPG). It provides a brief recovery time between frames to allow devices to prepare for reception of the next frame. The minimum interframe gap is 96 bit times, which is 9.6 microseconds for 10 Mb/s Ethernet, 960 nanoseconds for 100 Mb/s Ethernet, and 96 nanoseconds for 1 Gb/s Ethernet.

Let's assume a Gigabit port that can transfer raw data up to 125,000,000 bytes per second with minimum frame size of 64 bytes. The minimum inter frame gap period is 96 bits or 12 bytes which amounts to 96 nano seconds + 7 byte of preamble and 1 byte of delimiter consequence 84 bytes. 


Maximum Frame Rate and Throughput Calculations For a 1-Gb/s Ethernet Link
Frame PartMinimum Frame SizeMaximum Frame Size
Inter Frame Gap (9.6 ms)
12 bytes
12 bytes
MAC Preamble (+ SFD)
8 bytes
8 bytes
MAC Destination Address
6 bytes
6 bytes
MAC Source Address
6 bytes
6 bytes
MAC Type (or length)
2 bytes
2 bytes
Payload (Network PDU)
46 bytes
1,500 bytes
Check Sequence (CRC)
4 bytes
4 bytes
Total Frame Physical Size
84 bytes
1, 538 bytes

[1,000,000,000 b/s / (84 B * 8 b/B)] == 1,488,096 f/s (maximum rate)
[1,000,000,000 b/s / (1,538 B * 8 b/B)] == 81,274 f/s (minimum rate)

Example Formula for Ixia  or other Test Device to find the exact linerate in bps-

=((Bytes Received Rate * 8) + (Frames Received Rate * 160))
where 160 bits (20 bytes) is the inter frame gap

Regular Expressions

The Anchor Characters: ^ and $

Pattern
Matches
^A
"A" at the beginning of a line
A$
"A" at the end of a line
A^
"A^" anywhere on a line
$A
"$A" anywhere on a line
^^
"^" at the beginning of a line
$$
"$" at the end of a line

Match any character with .

The character "." is one of those special meta-characters. By itself it will match any character, except the end-of-line character.
The pattern that will match a line with a single characters is ^.$

Specifying a Range of Characters with [...]

If you want to match specific characters, you can use the square brackets to identify the exact characters you are searching for
The pattern that will match any line of text that contains exactly one number is ^[0123456789]$
This is verbose. You can use the hyphen between two characters to specify a range: ^[0-9]$
You can intermix explicit characters with character ranges. This pattern will match a single character that is a letter, number, or underscore: [A-Za-z0-9_]

Exceptions in a character set

You can easily search for all characters except those in square brackets by putting a "^" as the first character after the "["
To match all characters except vowels use "[^aeiou]".
Like the anchors in places that can't be considered an anchor, the characters "]" and "-" do not have a special meaning if they directly follow "[". Here are some examples:

Regular Expression
Matches
[]
The characters "[]"
[0]
The character "0"
[0-9]
Any number
[^0-9]
Any character other than a number
[-0-9]
Any number or a "-"
[0-9-]
Any number or a "-"
[^-0-9]
Any character except a number or a "-"
[]0-9]
Any number or a "]"
[0-9]]
Any number followed by a "]"
[0-9-z]
Any number,
or any character between "9" and "z".
[0-9\-a\]]
Any number, or
a "-", a "a", or a "]"

Repeating character sets with *

The special character "*" matches zero or more copies. That is, the regular expression "0*" matches zero or more zeros, while the expression "[0-9]*" matches zero or more numbers
This explains why the pattern "^#*" is useless, as it matches any number of "#'s" at the beginning of the line, including zero. Therefore this will match every line, because every line starts with zero or more "#'s"
Just use "^ *" to match zero or more spaces at the beginning of the line. If you need to match one or more, just repeat the character set. That is, "[0-9]*" matches zero or more numbers, and "[0-9][0-9]*" matches one or more numbers

Matching a specific number of sets with \{ and \}

You can specify the minimum and maximum number of repeats by putting those two numbers between "\{" and "\}"
The backslashes deserve a special discussion. Normally a backslash turns off the special meaning for a character. A period is matched by a "\." and an asterisk is matched by a "\*"
If a backslash is placed before a "<," ">," "{," "}," "(," ")," or before a digit, the backslash turns on a special meaning
The regular expression to match 4, 5, 6, 7 or 8 lower case letters is [a-z]\{4,8\}
Any numbers between 0 and 255 can be used. The second number may be omitted, which removes the upper limit. If the comma and the second number are omitted, the pattern must be duplicated the exact number of times specified by the first number
Regular Expression
Matches
_
*
Any line with an asterisk
\*
Any line with an asterisk
\\
Any line with a backslash
^*
Any line starting with an asterisk
^A*
Any line
^A\*
Any line starting with an "A*"
^AA*
Any line if it starts with one "A"
^AA*B
Any line with one or more "A"'s followed by a "B"
^A\{4,8\}B
Any line starting with 4, 5, 6, 7 or 8 "A"'s
followed by a "B"
^A\{4,\}B
Any line starting with 4 or more "A"'s
followed by a "B"
^A\{4\}B
Any line starting with "AAAAB"
\{4,8\}
Any line with "{4,8}"
A{4,8}
Any line with "A{4,8}"

Matching words with \< and \>

Searching for a word isn't quite as simple as it at first appears. The string "the" will match the word "other". You can put spaces before and after the letters and use this regular expression: " the ". However, this does not match words at the beginning or end of the line. And it does not match the case where there is a punctuation mark after the word.
There is an easy solution. The characters "\<" and "\>" are similar to the "^" and "$" anchors, as they don't occupy a position of a character. They do "anchor" the expression between to only match if it is on a word boundary.
The pattern to search for the word "the" would be "\<[tT]he\>". The character before the "t" must be either a new line character, or anything except a letter, number, or underscore. The character after the "e" must also be a character other than a number, letter, or underscore or it could be the end of line character.