superMaru

Tag Archives: linux

Windows Subsystem for Linux – Ubuntu GUI with xfce

Leave a comment / tips and tricks, Uncategorized / , , , ,

I’ve finally moved on from my Macbook Pro, late 2013 model and onto a Dell XPS13. I have a work-issued MBP with the touch-bar gimmick and absolutely dislike the keyboard. I decided to dual boot the device with Ubuntu 18.04 but what to do so many workarounds due to the NVMe disk in my XPS13 that made switching between OS inconvenient. I was going to wipe the Windows partition and run it as a VM until I updated the USBC firmware then something happened: my GRUB menu vanished and it booted straight to Windows.

Out of pure laziness I decided to try Windows 10 out and the new Windows Subsystem for Linux (more info here: https://docs.microsoft.com/en-us/windows/wsl/faq). Much to my amazement, it does a lot of basic things. There are some things that do not work (tcpdump is one example), but I think overtime it improve. As I was searching on what I can and cannot do, I found this cool party trick: Getting a GUI for WSL.

Pre-reqs:

  1. Internet connectivity to download packages and such
  2. A Windows 10 device
  3. Enable WSL and install Ubuntu – https://docs.microsoft.com/en-us/windows/wsl/install-win10
  4. Download a Windows X-server such as VcXsrv (https://sourceforge.net/projects/vcxsrv/)

Once you get your Ubuntu instance running, you will need to install a desktop environment. In this case, its xfce.  We will also need to tell Ubuntu where to display xfce.

Ubuntu setup:

To install xfce, you will need to run: sudo apt install xfce4

And to tell Ubuntu where to display xfce, run: sudo echo "export DISPLAY=:0.0" >> ~/.bashrc

Windows setup:

Install VcXsrv and run XLaunch.

I select “One window without titlebar”

I leave with default settings as shown here

I check “Disable access control”

Don’t be scared when you just see a blank screen. You can do a ALT + TAB back to your Bash terminal and run: xfce4-session and watch the magic happen. Now you have a Linux system without running a VM!

My personal preferences:

  1. I hide my Windows task bar. Just hit the Windows key, type “Windows task bar” and select “Automatically hide the taskbar in desktop mode”
  2. I run the X-server in a different virtual desktop. You can do this by pressing the CTRL + WIN + D to start new desktop. You can also use the CTRL + WIN + LEFT/RIGHT arrow keys to switch between.
    1. Here is a view of my virtual desktops: 
  3. In xfce, I noticed Windows takes precedence for key combinations such as ALT + TAB. I made CTRL + TAB my combination to switch applications for Linux

Post installation:

I would like to try and get the following going:

  1. A one-liner script to open a new virtual desktop, start Xlaunch and start xfce
  2. Try to get GNOME working since at the time of this writing, the version you get on the Microsoft store is 18.04.1 LTS

For now, i’ll try running some of my daily-use programs with WSL. I’m also quite glad that I’m not eating up too many hardware resources. I’m currently at 37% CPU and 50% memory (my biggest resource hog is Google Chrome). This leaves me with plenty of space to spin up a VM if I need to.

HTML5 GUI management for KVM hosts

Leave a comment / data center, Home DIY, linux / , , ,

I came across this cool HTML5-based management interface for KVM hosts. Its called kimchi-project (https://github.com/kimchi-project/kimchi/). In this guide, i’ll show you how to install it on an Ubuntu 16.04 host and manage your guest VMs.

Pre-installation steps

If you haven’t installed quemu, go ahead and do that now.

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install qemu qemu-kvm libvirt-bin

Downloading Kimchi

wget http://kimchi-project.github.io/wok/downloads/latest/wok.noarch.deb &&\
wget http://kimchi-project.github.io/gingerbase/downloads/latest/ginger-base.noarch.deb &&\
wget http://kimchi-project.github.io/kimchi/downloads/latest/kimchi.noarch.deb

Installing Kimchi

Its best you follow the steps below in sequential order:

sudo dpkg -i wok-*.deb
sudo apt-get install -f
sudo service wokd start
sudo dpkg -i ginger-*.deb
sudo apt-get install -f
sudo service wokd restart
# reboot here
sudo reboot
sudo dpkg -i kimchi-*.deb
sudo apt-get install -f
#last reboot
sudo reboot

Once you have installed Kimchi, go ahead and open your favorite web browser and point it to: https://127.0.0.1:8001

Loging in

Kimchi uses PAM to authenticate, so you can use your default Linux login credentials at the login page

Once you are in, you are defaulted to the Activity log.

Creating a VM

First order of business, you will need to create a template. Head over to Virtualization > Templates > Add template

You can either pick a file path to your local ISO or pick any available ISO that Kimchi shows. I’ve kept mine local and created it there.

You can modify your template with the usual storage, network and CPU settings:

General settings covers basic information of the VM:

Storage options: You can add a directory, NFS, iSCSI, local or SCSI FC storage option

Network options: You can create your usual isolate,d NAT, passthrough or bridged networking options. They also allow something called VEPA, which the VMs are connected to a VEPA-enabled switch

 

Once you got a basic template created, you can head over to Virtualization > Guests > Add guests

Your template that was created earlier will show up. Go ahead and click “create” to spawn off that VM

Managing your VMs

Once your VM has been created, you can click on the Actions drop down menu to do the usual VM operations such as view console, edit, migrate, stop/start/restart/nuke.

  

 

KVM is also decent. It uses noVNC and doesn’t require any additional browser plugins.

Snapshots can also be managed through Kimchi. Its hidden a couple more clicks away instead of being directly accessible on the Action drop down menu. You’ll need to go to Actions > Edit Guest > Snapshot:

Guest VM View

You have two options, the default AWS-ish view where you have the Guest ID name, OS Type, and VM stats called Table View.

The other view is called Gallery View, which shows a KVM screenshot of the VM. Very useful in case your guest VM has a kernel panic. No need to open a KVM and you can just reboot right away.

Misc

You can also see live stats of the hypervisor under the Host tab

 

Final thoughts:

Since its an HTML5 UI, I like the snappy response, nice colors and layout. If VMware’s vCenter is this nice when it GA’s, I wouldn’t mind giving up my C# client. Its a decent KVM UI manager with a lot of potential.  There are 3 things lacking that I would love to see in the future. I’d like to think of these as must-have’s:

  1. Guest VM performance history
  2. Host hypervisor performance history
  3. Integrated guest VM logging and history (ELK, maybe?)

Future support is multiple KVM hosts, HA and DRS!

verify dhcpd.conf for syntax errors

Leave a comment / data center, linux, tips and tricks / , ,

In CentOS you can check to see if your dhcpd.conf file has any syntax errors before you restart DHCP and it fails to come up.

You’ll need to run a command called dhcpd and point it to your configuration file like this:

[root@dhcp-server ~]# dhcpd -t -cf /etc/dhcp/dhcpd.conf

 
If you run into any syntax errors, it will point it out for you like so:

[root@dhcp-server ~]# dhcpd -t -cf /etc/dhcp/dhcpd.conf
Internet Systems Consortium DHCP Server 4.1.1-P1
Copyright 2004-2010 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
/etc/dhcp/dhcpd.conf line 474: semicolon expected.
server-a {
^
/etc/dhcp/dhcpd.conf line 476: fixed-address parameter not allowed here.
fixed-address 192.168.1.50;
^
/etc/dhcp/dhcpd.conf line 476: expecting a parameter or declaration

 

If your configuration is good, it should come up clean like so:

Internet Systems Consortium DHCP Server 4.1.1-P1
Copyright 2004-2010 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

PI-HOLE: Adblocking for home

Leave a comment / Home DIY, linux, Raspberry Pi / , , ,

I’ve came across a yet another easy and cool home project that is intended for Rapsberry Pis, but can run on any flavor of Linux (Requirements here). Its called Pi-Hole and you can set it up in 5 minutes or less.

Website: https://pi-hole.net/

The idea is simple: it blocks all add traffic in your network as it acts as a DNS server. While you should have browser extensions to protect your privacy (a post for later), mobile devices, smart TVs, your mom’s laptop and your younger siblings computer full off questionable “free” software can benefit as they do not have to install or configure anything. Its all dynamically configured with some effort on the network level and the result is there is no user-level interaction required.

 

If you are familiar with OpenDNS, the service is similar with the following differences:

  1. You need to install PiHole locally in your network, then configuring your DNS server to send queries to PiHole versus setting the DNS server IP to OpenDNS
  2. PiHole takes care of blocking advertisements only
  3. OpenDNS does content filtering and protects you from malware, botnets, phishing and targeted online attacks

 

To get the best of both worlds,

  1. Configure your local LAN to resolve DNS queries to PiHole
  2. Configure PiHole to forward DNS queries to OpenDNS
  3. Use a firewall (pfsense works great)

Installation

I’ll go over the basic installation on a Ubuntu 16.04 LTS server VM instead of a Raspberry Pi. The configuration would be the same, except I already have openssh-server, updates, etc setup on a VM that I’ve setup and treat as my template. I just simply did a link-clone in VMware Workstation and my environment was up in seconds.

Things you need to get started:

  1. Access to your DHCP server
  2. Provide a static IP address that is not within the range of your DHCP scope
  3. A Raspberry pi or Linux system (A VM would suffice for testing it out)

DHCP configuration

You will need to configure your DHCP server (if its home, then its most likely your router) so when a client requests for a DHCP IP, you can push the DNS IP address as well. I won’t touch on this configuration step as everyones home configuration would be different.

Once you have terminal access to your server, just type: curl -sSL https://install.pi-hole.net | bash

The installation process is straight forward and quick.

Shortly after, a pink install dialogue will open:

Once the packages have loaded, you’ll be greeted with the automated installer message below. You will need to hit enter to proceed with the installation

Just like any open-source project, its free. However donations help and go a long way.

A message will appear stating that you will need a static IP address (regardless if you have already done so)

This is where you can select your upstream DNS provider. Don’t worry, you can configure this later in the nice UI dashboard

Your next choice is to decide which protocol to select (IPv4 or IPv6, or both)

The installer will detect your IP address and ask if you really want to proceed. Go ahead and hit enter.

One last warning stating the static IP address may be within a DHCP scope.

Next menu is regarding the web admin interface. Leave as default and continue

Last item would be log queries. I left it default.

 

Assuming your install went fine, you should see this final screen. Make note of your login password!

 

 

Once the install is done, open up a web browser and navigate to http://[your ip address]. You’ll need that password to proceed from here.

Once you are in, you are presented with a nice admin web dashboard:

I overwrote the DHCP assigned DNS server on my desktop with my PiHole server and started browsing the web. So far, it managed to block a couple advertisements already, regardless if i have AddBlock Plus

If you want to change the Pi-Hole configuration such as making it a DHCP server, changing the upstream DNS server, etc – its all do-able under Settings:

Now you’re all setup! Now start browsing websites and check back on the dashboard in 30 minutes. I guarantee how surprised you will be to see how much content you are filtering and how many things are working in the background on websites. Once cool idea is to run tcpdump (WireShark) and see how much calls to ad-related servers are going on before and after switching to PiHole.

Ubuntu ZFS Samba share

1 Comment / linux / , , , ,

In my previous posts regarding setting up ZFS on Ubuntu 16.04, I had referred to Aaron Toponce’s website on doing additional configuration to ZFS such as sharing via NFS or SMB.

In the portion for SMB here: https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/, it seems that there is a couple more things to do.

After running zfs set sharesmb=on pool/srv  and zfs share pool/srv, the share will be visible on the network from a Windows computer. However, if you try to access it, you will be prompted by an user authentication prompt. If you try to access as guest, access will be denied.

When running the zfs share command, a file will get created in /var/lib/samba/usershares/[zpool name]

In the file, you will need to look for “guest_ok=n” and change it to: “guest_ok=y”

example:

#VERSION 2
path=/opt/share
comment=Comment: /opt/share
usershare_acl=S-1-1-0:F
guest_ok=y
sharename=tank_export

And finally, to make the directory read/writable: sudo chmod 0777 [path to your ZFS filesystem]

ZFS on ubuntu 16.04

Leave a comment / linux / , ,

On my previous post, ZFS on Ubuntu Linux, I had setup ZFS on Ubuntu 14.04 by adding a ZFS repository, installing multiple packages and such. Now on 16.04 and 16.10, it is available by simply typing:

sudo apt-get install zfs

My previous post covers how to setup a ZFS Pool, creating ZIL, and sharing the ZFS filesystem. For additional documentation on setting up ZFS, i’ll refer to this document by Aaron Toponce.

A quick TL;DR on ZFS if you are unfamiliar with it:

  1. ZFS is a combined filesystem and logical volume manager designed by Sun Microsystems.
  2. Protects against data corruption
  3. Support for high storage capacity
  4. Efficient data compression
  5. Volume management
  6. Snapshots
  7. And many more features to list

Fun fact: Originally, ZFS was an acronym for Zettabyte File System. Its capable of storing 256ZB. 1ZB is equivalent of 1 million Petabytes (PB), 1 billion terabytes (TB) or 1 trillion gigabytes (GB). Thats a lot of cat videos!

Why use ZFS?

For me, personally I like the idea of eliminating hardware RAID. I’ve personally ran into one too many dead RAID controllers, snapshot capabilities, online array construction and reassembly and the ability to use ZIL (ZFS Intent Log), which allows me to use an SSD in increase the write throughput of the system. By no means i’m an expert on storage, but ZFS is great for a home-brew NAS if you have a spare computer laying around.

ZFS on Ubuntu Linux

Leave a comment / linux / , ,

This guide discusses how to setup ZFS on Ubuntu 14.04.3 LTS, based from Aaron Toponce’s guide.

Environment settings:

Operating System: Ubuntu 14.04 LTS

Setting up your host:

Installation of the ZFS repository

# sudo add-apt-repository ppa:zfs-native/stable

 

Installation of ZFS dependencies

# apt-get install spl-dkms

 

Installation of Ubuntu ZFS

# apt-get install -y ubuntu-zfs

 

Finding out disk mapping

# apt-get install -y lsscsi

 

# lsscsi

[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0

[2:0:0:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sda

[2:0:1:0]    disk    ATA      INTEL SSDSC2BX01 CS01  /dev/sdb

[2:0:2:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdc

[2:0:3:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdd

[2:0:4:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sde

[2:0:5:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdf

[2:0:6:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdg

[2:0:7:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdh

[2:0:8:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdi

[2:0:9:0]    disk    SEAGATE  ST1200MM0007     0003  /dev/sdj

[2:0:10:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdk

[2:0:11:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdl

[2:0:12:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdm

[2:0:13:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdn

[2:0:14:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdo

[2:0:15:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdp

[2:0:16:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdq

[2:0:17:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdr

[2:0:18:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sds

[2:0:19:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdt

[2:0:20:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdu

[2:0:21:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdv

[2:0:22:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdw

[2:0:23:0]   disk    SEAGATE  ST1200MM0007     0003  /dev/sdx

[2:0:24:0]   enclosu Cisco    UCS-C240-M4      0224  -

[3:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sdy

 

Creating a ZFS pool

ZFS pool is a group of spinning hard drives. In the case below, the ZFS pool tank is a group of 1TB HDDs with no RAID protection:

# zpool create tank sdc sdd sde sdf sdg sdh –f

Once you create your pool, you can view the status:

# zpool status

pool: tank

state: ONLINE

scan: none requested

config:

NAME        STATE     READ WRITE CKSUM

tank        ONLINE       0     0     0

sdc       ONLINE       0     0     0

sdd       ONLINE       0     0     0

sde       ONLINE       0     0     0

sdf       ONLINE       0     0     0

sdg       ONLINE       0     0     0

sdh       ONLINE       0     0     0

 

You can list all your ZFS pools:

# zpool list

NAME   SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT

tank  6.52T  68.5K  6.52T         -     0%     0%  1.00x  ONLINE  -

 

Adding the ZFS Intent Log (ZIL)

You want to make sure the dynamic system mapping of the disk (sda, sdb) does not mess you up next time you reboot your server. You want to map your ZIL drive by disk-by-id. In this case, I have a SSD, sdb, that I want to use.

# ls -l /dev/disk/by-id/ | grep sdb

lrwxrwxrwx 1 root root  9 May 14 12:55 ata-INTEL_SSDSC2BX016T4K_BTHC536502FZ1P6PGN -> ../../sdb

lrwxrwxrwx 1 root root  9 May 14 12:55 wwn-0x55cd2e404c0d9cb4 -> ../../sdb

Now that I have the disk-by-id, i can add the log device to my existing ZFS pool: tank:

# zpool add tank log /dev/disk/by-id/ata-INTEL_SSDSC2BX016T4K_BTHC536502FZ1P6PGN  -f

# zpool status

pool: tank

state: ONLINE

scan: none requested

config:

NAME                                           STATE     READ WRITE CKSUM

tank                                           ONLINE       0     0     0

sdc                                          ONLINE       0     0     0

sdd                                          ONLINE       0     0     0

sde                                          ONLINE       0     0     0

sdf                                          ONLINE       0     0     0

sdg                                          ONLINE       0     0     0

sdh                                          ONLINE       0     0     0

logs

              ata-INTEL_SSDSC2BX016T4K_BTHC536502FZ1P6PGN  ONLINE       0     0     0

 

Sharing your ZFS filesystem

# zfs create tank/export

If you have a preferred mountpoint, you can mount the ZFS filesystem to your path of choice:

# zfs set mountpoint=/opt/zfs tank/export

 

# zfs list

NAME          USED  AVAIL  REFER  MOUNTPOINT

tank         99.5K  6.31T    19K  /tank

tank/export    19K  6.31T    19K  /opt/zfs

 

Sharing via NFS:

You want to install NFS and start the services:

# apt-get install -y nfs-kernel-server

# /etc/init.d/nfs-kernel-server start

Enable NFS share via ZFS:

# zfs set sharenfs=on tank/export

# zfs set sharenfs=”rw=@10.0.0.0/8″ tank/export

# zfs share tank/export

 

Verifying:

# showmount -e 10.0.11.211

Export list for 10.0.11.211:

/opt/zfs 10.0.0.0/8

Script to monitor disk space usage

Leave a comment / linux /

Here is a quick and easy script that you can add to your crontab to monitor disk space consumption. Once you hit 85% disk utilization, you automatically send an email to yourself

# cat space-monitor.sh
#!/bin/bash
#This script runs nightly to check disk space usage
ADMIN="macky@supermaru.com"
df -h | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{print $5 " " $1 }' | while read output;
do
 usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
 partition=$(echo $output | awk '{ print $2 }' )
 if [ $usep -ge 85 ]; then
 echo "Running out of space \"$partition ($usep%)\" as on $(date)" > /tmp/email.txt
 mail -s "Alert: $partition Almost out of disk space" $ADMIN < /tmp/email.txt ;
 fi
done

 

Email example:

Subject: Alert: /ftp Almost out of disk space

Running out of space “/ftp (93%)” as on Fri Oct  7 00:00:01 PDT 2016

 

hot-add CPU and memory to Ubuntu guest in VMware

4 Comments / ESXi, linux, VMware / , , ,

Virtualization has made our lives easier. We can spin up new resources for applications, dynamically add more disk space, migrate them, etc. In vSphere, we can dynamically add additional memory and vCPU to a guest VM without any downtime (assuming its been enabled prior to powering on).

Now in Ubuntu 12.04, I had two production virtual machines that I needed to increase resources, but could not afford downtime (of course). Being the good sysadmin that I am, I spun up a test VM to ensure there is no downtime and last-minute gotchas. To my surprise, I was able to dynamically add more resources and vSphere recognized it, but the operating system did not. I stumbled across this Ubuntu forum and found the answer I was looking for: https://askubuntu.com/questions/764620/how-do-you-hotplug-enable-new-cpu-and-ram-in-a-virtual-machine

TL;DR: You need to “turn on” the resources. Change some 0s to 1s to bring them online. Once you add more resources, copy this script over to your guest VM and fire away.

I initially created my test VM with 4GB of RAM and dynamically added 8GB more (total of 12GB). Works like a charm .

root@ubuntu-template:~# cat /proc/meminfo | more
MemTotal: 4040264 kB
root@ubuntu-template:~# ./hotadd.sh
Found cpu: "/sys/devices/system/cpu/cpu0" ...
 cpu0 already configured prior to hot-add
Found cpu: "/sys/devices/system/cpu/cpu1" ...
 cpu1 already online
Found ram: /sys/devices/system/memory/memory0/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory100/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory101/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory102/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory103/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory10/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory11/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory12/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory13/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory14/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory15/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory16/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory17/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory18/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory19/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory1/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory20/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory21/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory22/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory23/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory2/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory32/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory33/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory34/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory35/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory36/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory37/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory38/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory39/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory3/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory40/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory41/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory42/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory43/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory44/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory45/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory46/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory47/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory48/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory49/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory4/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory50/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory51/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory52/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory53/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory54/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory55/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory56/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory57/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory58/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory59/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory5/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory60/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory61/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory62/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory63/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory64/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory65/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory66/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory67/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory68/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory69/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory6/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory70/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory71/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory72/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory73/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory74/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory75/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory76/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory77/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory78/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory79/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory7/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory80/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory81/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory82/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory83/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory84/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory85/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory86/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory87/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory88/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory89/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory8/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory90/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory91/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory92/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory93/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory94/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory95/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory96/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory97/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory98/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory99/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory9/state:online ...
Already online
root@ubuntu-template:~# cat /proc/meminfo
MemTotal: 12429660 kB

 

 

Script:

#!/bin/bash
# Based on script by William Lam - http://engineering.ucsb.edu/~duonglt/vmware/

# Bring CPUs online
for CPU in $(ls /sys/devices/system/cpu/ |grep -E '(cpu[0-9])')
do
        CPU_DIR="/sys/devices/system/cpu/${CPU}"
        echo "Found cpu: \"${CPU_DIR}\" ..."
        CPU_STATE_FILE="${CPU_DIR}/online"
        if [ -f "${CPU_STATE_FILE}" ]; then
                STATE=$(cat "${CPU_STATE_FILE}" | grep 1)
                if [ "${STATE}" == "1" ]; then
                        echo -e "\t${CPU} already online"
                else
                         echo -e "\t${CPU} is new cpu, onlining cpu ..."
                         echo 1 > "${CPU_STATE_FILE}"
                fi
        else 
                echo -e "\t${CPU} already configured prior to hot-add"
        fi
done

# Bring all new Memory online
for RAM in $(grep line /sys/devices/system/memory/*/state)
do
        echo "Found ram: ${RAM} ..."
        if [[ "${RAM}" == *":offline" ]]; then
                echo "Bringing online"
                echo $RAM | sed "s/:offline$//"|sed "s/^/echo online > /"|source /dev/stdin
        else
                echo "Already online"
        fi
done

Speedtest-cli on Ubuntu

Leave a comment / linux / , , , , , ,

Speedtest.net is one of my de-facto go-to websites when doing a sanity check on my network for throughput. Unfortunately it requires the Adobe Flash Player plugin, so in certain circumstances its too much of a hassle to get Flash installed or poor graphics on a VM, etc.

So good thing is that there is a CLI option to run speedtest!

Note:

I don’t recommend solely relying on speedtest as your benchmark tool. There are others you can run through a browser like Shoretel Sky’s VOIP test, which has a nice quick in-depth test on your circuit link consistency, jitter, latency, etc. I’d also recommend doing an FTP test to a known server to get your tests. There are also many other tools out there if you do a quick google-search, but speedtest is typically the first to come to mind for bandwitdh testing

Getting started:

You’ll need pip in order to install speedtest-cli. If you don’t have it, go ahead and run:
sudo apt-get install python-pip

Then:
sudo pip install speedtest-cli

 

Running speedtest-cli

If you just run “speedtest-cli” you instantly kick off a speed-test. It will automatically pick a server with the best latency and execute the test:

# speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from [My ISP] (1.2.3.4 IP adddress)...
Selecting best server based on latency...
Hosted by SoftLayer Technologies, Inc. (San Jose, CA) [1.46 km]: 17.049 ms
Testing download speed........................................
Download: 50.84 Mbit/s
Testing upload speed..................................................
Upload: 49.45 Mbit/s

Using certain servers

When debugging network issues, its good to stay consistent with test servers, so you can list out your options by running speedtest-cli -list

Once you pick a specific server by calling out its specific ID such as: speedtest-cli –server [ID]

root@macky-vm1:~# speedtest-cli --list
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
4042) SoftLayer Technologies, Inc. (San Jose, CA, United States) [1.46 km]
5479) Sonic.net, Inc (San Jose, CA, United States) [4.18 km]
5086) Google Fiber (Palo Alto, Ca, United States) [26.62 km]
2761) SingTel (Palo Alto, CA, United States) [27.70 km]
1418) Fiber Internet Center LLC (Palo Alto, CA, United States) [27.70 km]

If I want to test with SoftLayer, ill go ahead and run: speedtest-cli --server 4042

root@macky-vm1:~# speedtest-cli --server 4042
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from [My ISP] (1.2.3.4 IP adddress)...
Hosted by SoftLayer Technologies, Inc. (San Jose, CA) [1.46 km]: 16.227 ms
Testing download speed........................................
Download: 49.68 Mbit/s
Testing upload speed..................................................
Upload: 47.80 Mbit/s

Happy speed-testing