superMaru

Tag Archives: linux

verify dhcpd.conf for syntax errors

Leave a comment / data center, linux, tips and tricks / , ,

In CentOS you can check to see if your dhcpd.conf file has any syntax errors before you restart DHCP and it fails to come up.

You’ll need to run a command called dhcpd and point it to your configuration file like this:

[root@dhcp-server ~]# dhcpd -t -cf /etc/dhcp/dhcpd.conf

 
If you run into any syntax errors, it will point it out for you like so:

[root@dhcp-server ~]# dhcpd -t -cf /etc/dhcp/dhcpd.conf
Internet Systems Consortium DHCP Server 4.1.1-P1
Copyright 2004-2010 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
/etc/dhcp/dhcpd.conf line 474: semicolon expected.
server-a {
^
/etc/dhcp/dhcpd.conf line 476: fixed-address parameter not allowed here.
fixed-address 192.168.1.50;
^
/etc/dhcp/dhcpd.conf line 476: expecting a parameter or declaration

 

If your configuration is good, it should come up clean like so:

Internet Systems Consortium DHCP Server 4.1.1-P1
Copyright 2004-2010 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

PI-HOLE: Adblocking for home

Leave a comment / Home DIY, linux, Raspberry Pi / , , ,

I’ve came across a yet another easy and cool home project that is intended for Rapsberry Pis, but can run on any flavor of Linux (Requirements here). Its called Pi-Hole and you can set it up in 5 minutes or less.

Website: https://pi-hole.net/

The idea is simple: it blocks all add traffic in your network as it acts as a DNS server. While you should have browser extensions to protect your privacy (a post for later), mobile devices, smart TVs, your mom’s laptop and your younger siblings computer full off questionable “free” software can benefit as they do not have to install or configure anything. Its all dynamically configured with some effort on the network level and the result is there is no user-level interaction required.

 

If you are familiar with OpenDNS, the service is similar with the following differences:

  1. You need to install PiHole locally in your network, then configuring your DNS server to send queries to PiHole versus setting the DNS server IP to OpenDNS
  2. PiHole takes care of blocking advertisements only
  3. OpenDNS does content filtering and protects you from malware, botnets, phishing and targeted online attacks

 

To get the best of both worlds,

  1. Configure your local LAN to resolve DNS queries to PiHole
  2. Configure PiHole to forward DNS queries to OpenDNS
  3. Use a firewall (pfsense works great)

Installation

I’ll go over the basic installation on a Ubuntu 16.04 LTS server VM instead of a Raspberry Pi. The configuration would be the same, except I already have openssh-server, updates, etc setup on a VM that I’ve setup and treat as my template. I just simply did a link-clone in VMware Workstation and my environment was up in seconds.

Things you need to get started:

  1. Access to your DHCP server
  2. Provide a static IP address that is not within the range of your DHCP scope
  3. A Raspberry pi or Linux system (A VM would suffice for testing it out)

DHCP configuration

You will need to configure your DHCP server (if its home, then its most likely your router) so when a client requests for a DHCP IP, you can push the DNS IP address as well. I won’t touch on this configuration step as everyones home configuration would be different.

Once you have terminal access to your server, just type: curl -sSL https://install.pi-hole.net | bash

The installation process is straight forward and quick.

Shortly after, a pink install dialogue will open:

Once the packages have loaded, you’ll be greeted with the automated installer message below. You will need to hit enter to proceed with the installation

Just like any open-source project, its free. However donations help and go a long way.

A message will appear stating that you will need a static IP address (regardless if you have already done so)

This is where you can select your upstream DNS provider. Don’t worry, you can configure this later in the nice UI dashboard

Your next choice is to decide which protocol to select (IPv4 or IPv6, or both)

The installer will detect your IP address and ask if you really want to proceed. Go ahead and hit enter.

One last warning stating the static IP address may be within a DHCP scope.

Next menu is regarding the web admin interface. Leave as default and continue

Last item would be log queries. I left it default.

 

Assuming your install went fine, you should see this final screen. Make note of your login password!

 

 

Once the install is done, open up a web browser and navigate to http://[your ip address]. You’ll need that password to proceed from here.

Once you are in, you are presented with a nice admin web dashboard:

I overwrote the DHCP assigned DNS server on my desktop with my PiHole server and started browsing the web. So far, it managed to block a couple advertisements already, regardless if i have AddBlock Plus

If you want to change the Pi-Hole configuration such as making it a DHCP server, changing the upstream DNS server, etc – its all do-able under Settings:

Now you’re all setup! Now start browsing websites and check back on the dashboard in 30 minutes. I guarantee how surprised you will be to see how much content you are filtering and how many things are working in the background on websites. Once cool idea is to run tcpdump (WireShark) and see how much calls to ad-related servers are going on before and after switching to PiHole.

Ubuntu ZFS Samba share

Leave a comment / linux / , , , ,

In my previous posts regarding setting up ZFS on Ubuntu 16.04, I had referred to Aaron Toponce’s website on doing additional configuration to ZFS such as sharing via NFS or SMB.

In the portion for SMB here: https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/, it seems that there is a couple more things to do.

After running zfs set sharesmb=on pool/srv  and zfs share pool/srv, the share will be visible on the network from a Windows computer. However, if you try to access it, you will be prompted by an user authentication prompt. If you try to access as guest, access will be denied.

When running the zfs share command, a file will get created in /var/lib/samba/usershares/[zpool name]

In the file, you will need to look for “guest_ok=n” and change it to: “guest_ok=y”

example:

#VERSION 2
path=/opt/share
comment=Comment: /opt/share
usershare_acl=S-1-1-0:F
guest_ok=y
sharename=tank_export

And finally, to make the directory read/writable: sudo chmod 0777 [path to your ZFS filesystem]

ZFS on ubuntu 16.04

Leave a comment / linux / , ,

On my previous post, ZFS on Ubuntu Linux, I had setup ZFS on Ubuntu 14.04 by adding a ZFS repository, installing multiple packages and such. Now on 16.04 and 16.10, it is available by simply typing:

sudo apt-get install zfs

My previous post covers how to setup a ZFS Pool, creating ZIL, and sharing the ZFS filesystem. For additional documentation on setting up ZFS, i’ll refer to this document by Aaron Toponce.

A quick TL;DR on ZFS if you are unfamiliar with it:

  1. ZFS is a combined filesystem and logical volume manager designed by Sun Microsystems.
  2. Protects against data corruption
  3. Support for high storage capacity
  4. Efficient data compression
  5. Volume management
  6. Snapshots
  7. And many more features to list

Fun fact: Originally, ZFS was an acronym for Zettabyte File System. Its capable of storing 256ZB. 1ZB is equivalent of 1 million Petabytes (PB), 1 billion terabytes (TB) or 1 trillion gigabytes (GB). Thats a lot of cat videos!

Why use ZFS?

For me, personally I like the idea of eliminating hardware RAID. I’ve personally ran into one too many dead RAID controllers, snapshot capabilities, online array construction and reassembly and the ability to use ZIL (ZFS Intent Log), which allows me to use an SSD in increase the write throughput of the system. By no means i’m an expert on storage, but ZFS is great for a home-brew NAS if you have a spare computer laying around.

ZFS on Ubuntu Linux

Leave a comment / linux / , ,

This guide discusses how to setup ZFS on Ubuntu 14.04.3 LTS, based from Aaron Toponce’s guide.

Environment settings:

Operating System: Ubuntu 14.04 LTS

Setting up your host:

Installation of the ZFS repository

# sudo add-apt-repository ppa:zfs-native/stable

 

Installation of ZFS dependencies

# apt-get install spl-dkms

 

Installation of Ubuntu ZFS

# apt-get install -y ubuntu-zfs

 

Finding out disk mapping

# apt-get install -y lsscsi

 

# lsscsi

[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0

[2:0:0:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sda

[2:0:1:0]    disk    ATA      INTEL SSDSC2BX01 CS01  /dev/sdb

[2:0:2:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdc

[2:0:3:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdd

[2:0:4:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sde

[2:0:5:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdf

[2:0:6:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdg

[2:0:7:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdh

[2:0:8:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdi

[2:0:9:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdj

[2:0:10:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdk

[2:0:11:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdl

[2:0:12:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdm

[2:0:13:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdn

[2:0:14:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdo

[2:0:15:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdp

[2:0:16:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdq

[2:0:17:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdr

[2:0:18:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sds

[2:0:19:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdt

[2:0:20:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdu

[2:0:21:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdv

[2:0:22:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdw

[2:0:23:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdx

[2:0:24:0]   enclosu Cisco    UCS-C240-M4      0224  -

[3:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sdy

 

Creating a ZFS pool

ZFS pool is a group of spinning hard drives. In the case below, the ZFS pool tank is a group of 1TB HDDs with no RAID protection:

# zpool create tank sdc sdd sde sdf sdg sdh –f

Once you create your pool, you can view the status:

# zpool status

pool: tank

state: ONLINE

scan: none requested

config:

NAME        STATE     READ WRITE CKSUM

tank        ONLINE       0     0     0

sdc       ONLINE       0     0     0

sdd       ONLINE       0     0     0

sde       ONLINE       0     0     0

sdf       ONLINE       0     0     0

sdg       ONLINE       0     0     0

sdh       ONLINE       0     0     0

 

You can list all your ZFS pools:

# zpool list

NAME   SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT

tank  6.52T  68.5K  6.52T         -     0%     0%  1.00x  ONLINE  -

 

Adding the ZFS Intent Log (ZIL)

You want to make sure the dynamic system mapping of the disk (sda, sdb) does not mess you up next time you reboot your server. You want to map your ZIL drive by disk-by-id. In this case, I have a SSD, sdb, that I want to use.

# ls -l /dev/disk/by-id/ | grep sdb

lrwxrwxrwx 1 root root  9 May 14 12:55 ata-INTEL_SSDSC2BX016T4K_BTHC536502FZ1P6PGN -> ../../sdb

lrwxrwxrwx 1 root root  9 May 14 12:55 wwn-0x55cd2e404c0d9cb4 -> ../../sdb

Now that I have the disk-by-id, i can add the log device to my existing ZFS pool: tank:

# zpool add tank log /dev/disk/by-id/ata-INTEL_SSDSC2BX016T4K_BTHC536502FZ1P6PGN  -f

# zpool status

pool: tank

state: ONLINE

scan: none requested

config:

NAME                                           STATE     READ WRITE CKSUM

tank                                           ONLINE       0     0     0

sdc                                          ONLINE       0     0     0

sdd                                          ONLINE       0     0     0

sde                                          ONLINE       0     0     0

sdf                                          ONLINE       0     0     0

sdg                                          ONLINE       0     0     0

sdh                                          ONLINE       0     0     0

logs

              ata-INTEL_SSDSC2BX016T4K_BTHC536502FZ1P6PGN  ONLINE       0     0     0

 

Sharing your ZFS filesystem

# zfs create tank/export

If you have a preferred mountpoint, you can mount the ZFS filesystem to your path of choice:

# zfs set mountpoint=/opt/zfs tank/export

 

# zfs list

NAME          USED  AVAIL  REFER  MOUNTPOINT

tank         99.5K  6.31T    19K  /tank

tank/export    19K  6.31T    19K  /opt/zfs

 

Sharing via NFS:

You want to install NFS and start the services:

# apt-get install -y nfs-kernel-server

# /etc/init.d/nfs-kernel-server start

Enable NFS share via ZFS:

# zfs set sharenfs=on tank/export

# zfs set sharenfs=”rw=@10.0.0.0/8″ tank/export

# zfs share tank/export

 

Verifying:

# showmount -e 10.0.11.211

Export list for 10.0.11.211:

/opt/zfs 10.0.0.0/8

Script to monitor disk space usage

Leave a comment / linux /

Here is a quick and easy script that you can add to your crontab to monitor disk space consumption. Once you hit 85% disk utilization, you automatically send an email to yourself

# cat space-monitor.sh
#!/bin/bash
#This script runs nightly to check disk space usage
ADMIN="macky@supermaru.com"
df -h | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{print $5 " " $1 }' | while read output;
do
 usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
 partition=$(echo $output | awk '{ print $2 }' )
 if [ $usep -ge 85 ]; then
 echo "Running out of space \"$partition ($usep%)\" as on $(date)" > /tmp/email.txt
 mail -s "Alert: $partition Almost out of disk space" $ADMIN < /tmp/email.txt ;
 fi
done

 

Email example:

Subject: Alert: /ftp Almost out of disk space

Running out of space “/ftp (93%)” as on Fri Oct  7 00:00:01 PDT 2016

 

hot-add CPU and memory to Ubuntu guest in VMware

2 Comments / ESXi, linux, VMware / , , ,

Virtualization has made our lives easier. We can spin up new resources for applications, dynamically add more disk space, migrate them, etc. In vSphere, we can dynamically add additional memory and vCPU to a guest VM without any downtime (assuming its been enabled prior to powering on).

Now in Ubuntu 12.04, I had two production virtual machines that I needed to increase resources, but could not afford downtime (of course). Being the good sysadmin that I am, I spun up a test VM to ensure there is no downtime and last-minute gotchas. To my surprise, I was able to dynamically add more resources and vSphere recognized it, but the operating system did not. I stumbled across this Ubuntu forum and found the answer I was looking for: https://askubuntu.com/questions/764620/how-do-you-hotplug-enable-new-cpu-and-ram-in-a-virtual-machine

TL;DR: You need to “turn on” the resources. Change some 0s to 1s to bring them online. Once you add more resources, copy this script over to your guest VM and fire away.

I initially created my test VM with 4GB of RAM and dynamically added 8GB more (total of 12GB). Works like a charm .

root@ubuntu-template:~# cat /proc/meminfo | more
MemTotal: 4040264 kB
root@ubuntu-template:~# ./hotadd.sh
Found cpu: "/sys/devices/system/cpu/cpu0" ...
 cpu0 already configured prior to hot-add
Found cpu: "/sys/devices/system/cpu/cpu1" ...
 cpu1 already online
Found ram: /sys/devices/system/memory/memory0/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory100/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory101/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory102/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory103/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory10/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory11/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory12/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory13/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory14/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory15/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory16/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory17/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory18/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory19/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory1/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory20/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory21/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory22/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory23/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory2/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory32/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory33/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory34/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory35/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory36/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory37/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory38/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory39/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory3/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory40/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory41/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory42/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory43/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory44/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory45/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory46/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory47/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory48/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory49/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory4/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory50/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory51/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory52/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory53/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory54/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory55/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory56/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory57/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory58/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory59/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory5/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory60/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory61/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory62/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory63/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory64/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory65/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory66/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory67/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory68/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory69/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory6/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory70/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory71/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory72/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory73/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory74/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory75/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory76/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory77/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory78/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory79/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory7/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory80/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory81/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory82/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory83/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory84/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory85/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory86/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory87/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory88/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory89/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory8/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory90/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory91/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory92/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory93/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory94/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory95/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory96/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory97/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory98/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory99/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory9/state:online ...
Already online
root@ubuntu-template:~# cat /proc/meminfo
MemTotal: 12429660 kB

 

 

Script:

#!/bin/bash
# Based on script by William Lam - http://engineering.ucsb.edu/~duonglt/vmware/

# Bring CPUs online
for CPU in $(ls /sys/devices/system/cpu/ |grep -E '(cpu[0-9])')
do
        CPU_DIR="/sys/devices/system/cpu/${CPU}"
        echo "Found cpu: \"${CPU_DIR}\" ..."
        CPU_STATE_FILE="${CPU_DIR}/online"
        if [ -f "${CPU_STATE_FILE}" ]; then
                STATE=$(cat "${CPU_STATE_FILE}" | grep 1)
                if [ "${STATE}" == "1" ]; then
                        echo -e "\t${CPU} already online"
                else
                         echo -e "\t${CPU} is new cpu, onlining cpu ..."
                         echo 1 > "${CPU_STATE_FILE}"
                fi
        else 
                echo -e "\t${CPU} already configured prior to hot-add"
        fi
done

# Bring all new Memory online
for RAM in $(grep line /sys/devices/system/memory/*/state)
do
        echo "Found ram: ${RAM} ..."
        if [[ "${RAM}" == *":offline" ]]; then
                echo "Bringing online"
                echo $RAM | sed "s/:offline$//"|sed "s/^/echo online > /"|source /dev/stdin
        else
                echo "Already online"
        fi
done

Speedtest-cli on Ubuntu

Leave a comment / linux / , , , , , ,

Speedtest.net is one of my de-facto go-to websites when doing a sanity check on my network for throughput. Unfortunately it requires the Adobe Flash Player plugin, so in certain circumstances its too much of a hassle to get Flash installed or poor graphics on a VM, etc.

So good thing is that there is a CLI option to run speedtest!

Note:

I don’t recommend solely relying on speedtest as your benchmark tool. There are others you can run through a browser like Shoretel Sky’s VOIP test, which has a nice quick in-depth test on your circuit link consistency, jitter, latency, etc. I’d also recommend doing an FTP test to a known server to get your tests. There are also many other tools out there if you do a quick google-search, but speedtest is typically the first to come to mind for bandwitdh testing

Getting started:

You’ll need pip in order to install speedtest-cli. If you don’t have it, go ahead and run:
sudo apt-get install python-pip

Then:
sudo pip install speedtest-cli

 

Running speedtest-cli

If you just run “speedtest-cli” you instantly kick off a speed-test. It will automatically pick a server with the best latency and execute the test:

# speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from [My ISP] (1.2.3.4 IP adddress)...
Selecting best server based on latency...
Hosted by SoftLayer Technologies, Inc. (San Jose, CA) [1.46 km]: 17.049 ms
Testing download speed........................................
Download: 50.84 Mbit/s
Testing upload speed..................................................
Upload: 49.45 Mbit/s

Using certain servers

When debugging network issues, its good to stay consistent with test servers, so you can list out your options by running speedtest-cli -list

Once you pick a specific server by calling out its specific ID such as: speedtest-cli –server [ID]

root@macky-vm1:~# speedtest-cli --list
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
4042) SoftLayer Technologies, Inc. (San Jose, CA, United States) [1.46 km]
5479) Sonic.net, Inc (San Jose, CA, United States) [4.18 km]
5086) Google Fiber (Palo Alto, Ca, United States) [26.62 km]
2761) SingTel (Palo Alto, CA, United States) [27.70 km]
1418) Fiber Internet Center LLC (Palo Alto, CA, United States) [27.70 km]

If I want to test with SoftLayer, ill go ahead and run: speedtest-cli --server 4042

root@macky-vm1:~# speedtest-cli --server 4042
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from [My ISP] (1.2.3.4 IP adddress)...
Hosted by SoftLayer Technologies, Inc. (San Jose, CA) [1.46 km]: 16.227 ms
Testing download speed........................................
Download: 49.68 Mbit/s
Testing upload speed..................................................
Upload: 47.80 Mbit/s

Happy speed-testing

vCSA 6.0 automated deployment

Leave a comment / linux, VMware / , , , , , , , ,

Doing the first batches of vCenter Server Appliance (vCSA) 6.0 deployment was kinda…rough. Playing around with the OVA and getting annoyed at the web-based deployment option kind of slowed us down. At my current job we prefer vCSA’s for our Test and Dev environments and dont want to deal with Windows Server. Fortunately VMware has given us the option of deploying through the command line!

Where?!

You can deploy a vCSA to any ESX server on your network. Its in the VCSA ISO and its called “vcsa-cli-installer”

How does it work?

The cli installer uses a JSON file to help populate the necessary parameters, which makes it easy for us to automate deployment and turn around vCSAs faster.

Once you have extracted out the directory to your *nix system, you should see the following directories to quickly get you started: “lin64” and “templates”.

# ls
dbchecker lin64 templates

Files you care about

In the lin64 directory, there is a file called “vcsa-deploy”.

In the templates directory, there are JSON templates you can modify to quickly deploy our VCSA.

Once you have gotten your JSON file populated with your enviornment information, you deploy by running:

 ./vcsa-deploy path-to-your-json.file.json

Automating deployments

Since we generate vCSAs frequently, having an option to use JSON files is perfect. You can take a provided template and modify it even more to suit your needs.

JSON templates:

vCSA 6.0 and 6.0b: Templates are the same. For stand-alone vCenters we found that the embedded.example.json is good enough for our needs.

vCSA 6.0U1: The template has changed the format and requires version 1.1, which my scripted deployments I am sharing revolves around.

Scripted deployments:

You can write a simple wrapper bash script to take care of deployments for you. Since we provision vCSAs and they are for different testbeds, I decided to automate the creation of the JSON template, verify that the DNS entries for the vCenter is correct by doing a simple nslookup and sending an email notification with log outputs once its completed (or failed).

Here is the script you can use yourself and modify for 6.0U1.

Requirements:

I expect the following directories: json and vc6.0u1

file directory tree that I care about:
.
|-deploy6u1.sh
|-json
|    |–6.0u1
|        |_template.json
|-vc6.0u1
|-vcsa-cli-installer
|-lin64

json file and directory:
I used the embedded_vCSA_on_VC.json and renamed it to template.json in the json direcory
Since I have deployment scripts for other versions, I dump 6.0U1 JSONs in 6.0u1. During deployment, I copy template.json and name it whatever the name of the vCSA is.

Here is a modified version of my template.json. I took out items that we hard-code such as DNS servers and default vCSA passwords:

# cat json/6.0u1/template.json
{
 "__version": "1.1",
 "__comments": "Sample template to deploy a vCenter Server with an embedded Platform Services Controller to an ESXi host 6.0U1 09/22/15 MR",
 "target.vcsa": {
 "appliance": {
 "deployment.network": "VMNW",
 "deployment.option": "SIZE",
 "name": "VCSANAME",
 "thin.disk.mode": true
 },
 "esx": {
 "hostname": "EHOSTNAME",
 "username": "root",
 "password": "ESXPW",
 "datastore": "EDATASTORE"
 },
 "network": {
 "hostname": "VCSANAME",
 "dns.servers": [
 "1.2.3.4",
 "5.6.7.8"
 ],
 "gateway": "GW",
 "ip": "IPADDR",
 "ip.family": "ipv4",
 "mode": "static",
 "prefix": "CIDR"
 },
 "os": {
 "password": "Password1!",
 "ssh.enable": true,
 "ntp.servers": "9.8.7.6"
 },
 "sso": {
 "password": "Password1!",
 "domain-name": "vsphere.local",
 "site-name": "VCSANAME"
 }
 }
}

Here are lines you should change to work in your environment:

Under the “esx” section:

 "dns.servers": [
 "1.2.3.4", <- This is your primary DNS server
 "5.6.7.8"  <- This is your secondary DNS server

Under “os” section:

 "ntp.servers": "9.8.7.6" <- Change this IP address

Under the “os” and “sso” section:

 "password": "Password1!", <- Change the password to your liking.

Here is my wrapper script I use. I also have removed hard-coded items here.

# cat deploy6u1.sh
#!/bin/bash
#092215
#Script for VCSA 6.0 Update 1 deployment
#MR
cd json/6.0u1
itmail='you@company.com'
#Steps below creates a new JSON file for deployment
echo "Enter the new VCSA name: "
 read name
 cp template.json $name.json
 sed -i "s/VCSANAME/$name/g" $name.json
 date=`date +"%m%d%y"`
 vlog=/var/log/$name-$date
echo "Enter the VCSA's IP address: "
 read ip
 sed -i "s/IPADDR/$ip/g" $name.json
#DNS check
fwd=`nslookup $name | grep $name -A2 | grep Address | awk '{print $2}'`
rev=`nslookup $ip | grep name | awk '{print $4}'`
if [ $name.your.domain != $rev ]
then
 echo "Forward and reverse DNS lookup FAILED. Aborting deployment."
 exit
fi
#End DNS check
echo "Enter the vCenter deployment size. Default is small (<1000 VMs). For: "
echo "< 100 VMs, type 'tiny'"
echo "< 4,000 VMs, type 'medium'"
echo "< 1,000 VMs, type 'large'"
 read envsize
 if [[ -z $envsize ]];
 then
 sed -i "s/SIZE/small/g" $name.json
 echo "Setting default environment to 'small'."
 else
 sed -i "s/SIZE/$envsize/g" $name.json
 fi
echo "Enter the VCSA's netmask in CIDR notation (Press Enter for default: 21): "
 read cidr
 if [[ $cidr -eq 0 ]];
 then
 sed -i "s/CIDR/21/g" $name.json
 echo "Setting /21"
 else
 sed -i "s/CIDR/$cidr/g" $name.json
 fi
echo "Enter the VCSA's default gateway address: "
 read gw
 sed -i "s/GW/$gw/g" $name.json
echo "Enter the target ESX host to install: "
 read esx
 sed -i "s/EHOSTNAME/$esx/g" $name.json
echo "Enter the target ESX password (warning: in cleartext!): "
 read pw
 sed -i "s/ESXPW/$pw/g" $name.json
echo "Enter the target ESX datastore to install: "
 read datastore
 sed -i "s/EDATASTORE/$datastore/g" $name.json
echo "Enter the ESX deployment network: (Press Enter for default "VM Network") "
 read nw
 if [[ -z $nw ]];
 then
 sed -i "s/VMNW/VM Network/g" $name.json
 echo "Setting default 'VM Network'."
 else
 sed -i "s/VMNW/$nw/g" $name.json
 fi
echo "$name.json created."
cd ../../vc6.0u1/vcsa-cli-installer/lin64/
./vcsa-deploy ../../../json/6.0u1/$name.json --accept-eula -v 2>&1 | tee /var/log/$name.deploy
cat /var/log/$name.deploy | mail -s "VCSA $name completed" $itmail

Items you have to manually change:

Email recepient, line 8:

itmail='you@company.com'

Under the #DNS check section, you should change “your.domain”.

if [ $name.your.domain != $rev ]

Email notification and logs

The emails contains the deployment progress and log location incase things go south. So when you go and get your cup of coffee as you kick off deployment, you’ll get an email notification if deployment done (successful or failed).

Future improvements and things to note:

I’m planning on adding a field during deployment if you want a second person to get notified with a simple email that deployment was successful. I’ll add this to my public git repo, one day…

For deploying on a ESX host that does not have the rsa-key in your *nix system, you might have to type “yes” to continue and the OVF deployment starts.

There are no pre-checks of host names and IP addresses when deploying the vCSA – other than my forward and reverse lookup for the vCSA hostname. Make sure your target vCSA netmask, gateway,  ESX server information and credentials are correct.

And finally: I am not a developer! A skilled person who writes code for a living can probably write it in less lines of code, or/and add more intelligence.

More reading:

VMware has a technical doc on CLI based deployments: http://www.vmware.com/files/pdf/products/vsphere/VMware-vsphere-60-vcenter-server-appliance-cmdline-install.pdf

 

Add drives without reboot on VMware Ubuntu guest

Leave a comment / linux, Uncategorized / , ,

This post is on deploying a new Ubuntu VM, dynamically adding more disks, having the guest OS recognize the new drives and increasing its capacity without rebooting. This is really useful for critical production servers that cannot go down for reboots like a NFS server. Lets get started.

OS: I’m using Ubuntu 12.02.4 LTS server
OS drive: 20GB, thin provision.
I want to add another drive to export via NFS.

Lets see what lsscsi shows connected to the VM:

root@nfs-1:~# lsscsi
[1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0
[2:0:0:0] disk VMware Virtual disk 1.0 /dev/sda

On your ESX host, add the new VMDK to the VM, then install scsitools on the guest Linux VM. Scsitools will scan the SCSI bus and detect the drives without rebooting.

root@nfs-1:~# apt-get install -y install scsitools

Since lsscsi showed me that I only have one drive, sda, I’ll rescan.

root@nfs-1:~# rescan-scsi-bus
/sbin/rescan-scsi-bus: line 592: [: 1.03: integer expression expected
Host adapter 0 (ata_piix) found.
Host adapter 1 (ata_piix) found.
Host adapter 2 (mptspi) found.
Host adapter 3 (mptspi) found.
Scanning SCSI subsystem for new devices
Scanning host 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 1 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 1 0 0 0 ...
OLD: Host: scsi1 Channel: 00 Id: 00 Lun: 00
 Vendor: NECVMWar Model: VMware IDE CDR10 Rev: 1.00
 Type: CD-ROM ANSI SCSI revision: 05
Scanning host 2 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 2 0 0 0 ...
OLD: Host: scsi2 Channel: 00 Id: 00 Lun: 00
 Vendor: VMware Model: Virtual disk Rev: 1.0
 Type: Direct-Access ANSI SCSI revision: 02
Scanning host 3 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 3 0 0 0 ...
OLD: Host: scsi3 Channel: 00 Id: 00 Lun: 00
 Vendor: VMware Model: Virtual disk Rev: 1.0
 Type: Direct-Access ANSI SCSI revision: 02
1 new device(s) found.
0 device(s) removed.

You can see above that scsitools found a new drive. Lets verify below:

root@nfs-1:~# lsscsi
[1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0
[2:0:0:0] disk VMware Virtual disk 1.0 /dev/sda
[3:0:0:0] disk VMware Virtual disk 1.0 /dev/sdb

From here we can do whatever we want with the drive (sdb). In my case, i’ll create a LVM and can grow it whenever I need additional space.

First I’ll need to format the newly drive, /dev/sdb via fdisk.

root@nfs-1:~# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xd219bc7f.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Partition type:
 p primary (0 primary, 0 extended, 4 free)
 e extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
Using default value 1
First sector (2048-209715199, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-209715199, default 209715199):
Using default value 209715199
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)
Command (m for help): p
Disk /dev/sdb: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders, total 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xd219bc7f
Device Boot Start End Blocks Id System
/dev/sdb1 2048 209715199 104856576 8e Linux LVM
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Now I’ll create the Volume Group

root@nfs-1:~# vgcreate vgpool /dev/sdb1
 No physical volume label read from /dev/sdb1
 Physical volume "/dev/sdb1" successfully created
 Volume group "vgpool" successfully created

Create the logical volume

root@nfs-1:~# lvcreate -L 90G -n lvstuff vgpool
 Logical volume "lvstuff" created

Format and mount the logical volume. I’ll format it as a ext3 filesystem.

root@nfs-1:~# mkfs -t ext3 /dev/vgpool/lvstuff
mke2fs 1.42 (29-Nov-2011)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
5898240 inodes, 23592960 blocks
1179648 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
720 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
 4096000, 7962624, 11239424, 20480000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
root@nfs-1:~# mkdir /mnt/data
root@nfs-1:~# mount -t ext3 /dev/vgpool/lvstuff /mnt/data/

Verify that it is indeed mounted and all the space is there

root@nfs-1:~# mount | grep data
/dev/mapper/vgpool-lvstuff on /mnt/data type ext3 (rw)
root@nfs-1:~# df -h | grep data
/dev/mapper/vgpool-lvstuff 89G 184M 84G 1% /mnt/data

To have a persistent mount during bootup, you can add this in /etc/fstab. First you’ll need to find out the drives UUID

root@nfs-1:~# blkid /dev/mapper/vgpool-lvstuff
/dev/mapper/vgpool-lvstuff: UUID="7b1a695d-9688-4b98-866f-946d6634674b" TYPE="ext3"

Add this format into /etc/fstab like so:

UUID=7b1a695d-9688-4b98-866f-946d6634674b /mnt/data     ext3    defaults        0       2


Now that we have the data path setup, all thats left is setting up a NFS server.


root@nfs-1:~# apt-get install nfs-kernel-server


edit /etc/exports and add the mountpoint


/mnt/data        *(rw,sync,no_root_squash)


Restart the NFS server


root@nfs-1:~# /etc/init.d/nfs-kernel-server restart


Verify that the NFS export is working. I’ll do this on a separate VM:


root@macky-vm1:~# showmount -e 10.64.1.12
Export list for 10.64.1.12:
/mnt/data (everyone)




Putting everything together: Now we have a Linux VM with a 20GB OS drive and a 100GB drive in a LVM, which is being exported via NFS. Now lets go through the motions of adding additional space to the LVM without bringing down the server. Add the new vmdk to the VM. run rescan-scsi-bus


root@nfs-1:~# rescan-scsi-bus
/sbin/rescan-scsi-bus: line 592: [: 1.03: integer expression expected
Host adapter 0 (ata_piix) found.
Host adapter 1 (ata_piix) found.
Host adapter 2 (mptspi) found.
Host adapter 3 (mptspi) found.
Scanning SCSI subsystem for new devices
Scanning host 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 1 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 1 0 0 0 ...
OLD: Host: scsi1 Channel: 00 Id: 00 Lun: 00
 Vendor: NECVMWar Model: VMware IDE CDR10 Rev: 1.00
 Type: CD-ROM ANSI SCSI revision: 05
Scanning host 2 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 2 0 0 0 ...
OLD: Host: scsi2 Channel: 00 Id: 00 Lun: 00
 Vendor: VMware Model: Virtual disk Rev: 1.0
 Type: Direct-Access ANSI SCSI revision: 02
Scanning host 3 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 3 0 0 0 ...
OLD: Host: scsi3 Channel: 00 Id: 00 Lun: 00
 Vendor: VMware Model: Virtual disk Rev: 1.0
 Type: Direct-Access ANSI SCSI revision: 02
Scanning for device 3 0 1 0 ...
NEW: Host: scsi3 Channel: 00 Id: 01 Lun: 00
 Vendor: VMware Model: Virtual disk Rev: 1.0
 Type: Direct-Access ANSI SCSI revision: 02
1 new device(s) found.
0 device(s) removed.


root@nfs-1:~# ls /dev/sd?
/dev/sda /dev/sdb /dev/sdc


I see that now sdc is available. Lets format the drive with a LVM partition. See above on how to create a partition via fdisk and pvcreate so that LVM can recognize it.


Add the new drive to the volume group


root@nfs-1:~# vgextend vgpool /dev/sdc1
 Volume group "vgpool" successfully extended


Extend the Logical Volume


root@nfs-1:~# lvextend -L+90G /dev/vgpool/lvstuff
 Extending logical volume lvstuff to 183.00 GiB
 Logical volume lvstuff successfully resized


Extend the File System


root@nfs-1:~# resize2fs /dev/vgpool/lvstuff
resize2fs 1.42 (29-Nov-2011)
Filesystem at /dev/vgpool/lvstuff is mounted on /mnt/data; on-line resizing required
old_desc_blocks = 6, new_desc_blocks = 12
Performing an on-line resize of /dev/vgpool/lvstuff to 47972352 (4k) blocks.
The filesystem on /dev/vgpool/lvstuff is now 47972352 blocks long.



Then verify on the NFS server that the partition has grown:

From the NFS server:


root@nfs-1:~# df -h | grep data
/dev/mapper/vgpool-lvstuff 181G 188M 171G 1% /mnt/data


From another VM mounting the NFS export:


10.64.1.12:/mnt/data 181G 188M 171G 1% /mnt/nfs1