ip and ss: better than ifconfig and netstat

I’ve been using Linux for a while now, so typing certain commands is fairly ingrained, like ‘ifconfig’ and ‘netstat’. I know about “ip addr”, which is more modern than ifconfig, and I use it sometimes.

This week, I learned about ‘ss’, which is faster than ‘netstat’, and does more. My favorite invocation is “ss -tlp” to show programs listening on tcp sockets.

Ubuntu, ecryptfs, and changing password

I changed my password on my Ubuntu system this week, and then found that I couldn’t log in, except on a virtual terminal.

My home directory is encrypted, and apparently, it’s better to change a password using the graphical utilities, rather than the command line utilities. The following article was quite helpful in recovering:


OpenWest notes

This past weekend, I attended the excellent #OpenWest conference, and I presented Scaling RabbitMQ.

The volunteers that organized the conference deserve a huge amount of thanks. I can’t imagine how much work it was. I should also thank the conference sponsors.

A local group of hardware engineers designed an amazing conference badge, built from a circuit board. They deserve a big “high-five”. There was a soldering lab where I soldered surface mount components for the first time in my life – holding the components in place with tweezers. I bought the add-on kit for $35 that included a color LCD screen and Parallax Propeller chip. It took me 45 minutes to do the base kit, and two hours to do the add-on kit. I breathed a sigh of relief when I turned on the power, and it all worked.

The speakers did a great job, and I appreciate the hours they spent preparing. I wish I could have attended more of the sessions.

Among others, I attended sessions on C++11, Rust, Go lang, Erlang, MongoDB schema design, .NET core, wireless security, Salt Stack, and digital privacy.

I’m going to keep my eye on Rust, want to learn and use Go, and use the new beacon feature of Salt Stack. Sometime in the future, I’d like to use the new features of C++11.

The conference was an excellent place to have useful side-conversations with vendors, speakers, and past colleagues. It was a great experience.

Linux, time and the year 2038

Software tends to live longer than we expect, as do embedded devices running Linux. Those that want to accurately handle time through the year 2038 and beyond will need to be updated.

Fifteen years after Y2K, Linux kernel developers continue to refine support for time values that will get us past 2038. Jonathan Corbet, editor of LWN.net, explains the recent work in his typical lucid style: https://lwn.net/Articles/643234

It sounds like ext3 and NFSv3 filesystems will need to go the way of the dodo, due to lack of support for 64 bit time values, while XFS developers are working on adding support to get us past 2038. By that time, many of us will have moved on to newer file systems.

One comment linked to this useful bit of information on time programming on Linux systems: http://www.catb.org/esr/time-programming/, the summary is:

To stay out of trouble, convert dates to Unix UTC on input, do all your calculations in that, and convert back to localtime as rare as possible. This reduces your odds of introducing a misconversion and spurious timezone skew.

It’s also excellent advice for any back-end system that deals with data stored from devices that span a continent or the world, although it doesn’t necessarily eliminate daylight savings bugs.

Containerization – the beginning of a long journey

I read this today, and thought it’s worth sharing:

The impact of containerization in redefining the enterprise OS is still vastly underestimated by most; it is a departure from the traditional model of a single-instance, monolithic, UNIX user space in favor of a multi-instance, multi-version environment using containers and aggregate packaging. We are talking about nothing less than changing some of the core paradigms on which the software industry has been working for the last 20 – if not 40 – years.”

And yet it is tempered with reality:

we are really only at the beginning of a long journey…”


Ubuntu and .local hostnames in a corporate network

In the past, I’ve had trouble getting my Ubuntu machine to resolve the .local hostnames at work. I didn’t know why Ubuntu had this problem while other machines did not.

When I did a DNS lookup, it failed, and ping of host.something.local failed. Yet ping of the hostname without the .something.local extension worked. Odd. I googled various terms, but nothing useful came up. I tried watching the DNS lookup with tcpdump, but it didn’t capture anything.

Eventually, I thought of using ‘strace ping host.something.local’ to see what was happening, and it turns out that DNS was never being queried — it was talking to something called avahi.

I googled “avahi”, and was reminded that hostname resolution is configured in /etc/nsswitch.conf. In the case of Ubuntu, it’s configured to send *.local requests to Avahi (mdns4_minimal), and no further — i.e. if Avahi doesn’t resolve it, it doesn’t try DNS.

In my case, I want corporate DNS to resolve .local addresses. So I changed my /etc/nsswitch.conf from this:

hosts: files mdns4_minimal [NOTFOUND=return] wins dns mdns4

to this:

hosts: files wins dns mdns4_minimal mdns4

And now my Ubuntu development machine can communicate with our internal .local machines without having to resort to using IP addresses, short names, or having to place the mapping in /etc/hosts.

Grepping archived, rotated log files — in order

Say you’ve got the following log files with the oldest entries in myapi.log.3.gz:


If you want to ‘grep’ them for a string, in order of date, oldest to newest, there’s no need to extract them one at a time, and there’s no need to concatenate the files first. Use sort to put the files in the proper order, and zgrep to search though the compressed files.

Here’s how to order the file list:

ls myapi.log.*.gz | sort -nr -t . -k 3,3

Here’s how to ‘zgrep’ them in the proper order:

ls myapi.log.*.gz | sort -nr -t . -k 3,3 | xargs zgrep “404”

Ubuntu Unity application launchers for IntelliJ IDEA and PyCharm

Here’s how I got Ubuntu Unity to show application launchers for >IntelliJ IDEA and PyCharm

In $HOME/.local/share/applications, add an IDEA.desktop file with these contents:

#!/usr/bin/env xdg-open
[Desktop Entry]
Name=IntelliJ IDEA
GenericName=Text Editor

And a PyCharm.desktop file:

#!/usr/bin/env xdg-open
[Desktop Entry]

I use the default keyboard shortcuts, and on Linux, CTRL-ALT left arrow doesn’t work with PyCharm or IDEA (jumps back to where I was before I followed a symbol with CTRL-B). I’ve found that CTRL-WINDOWS-ALT left arrow does work. Same thing for many other shortcuts that use CTRL-ALT.

vCloud Director CentOS Guest OS Customization Script

I needed to customize a custom Linux VM image with a static IP address in vCloud Director. Here’s the script I came up with. It doesn’t work unless VMWare tools is installed and running in the VM (kernel upgrades tend to break it). The VM should be stopped, then paste the script into the “Guest OS Customization” script area. Boot the VM with the “Power On and Force Recustomization” option.


echo "------------------------------------------------------------------------------"
if [ x$1 == x"precustomization" ]; then
  set -e
  echo "Running Pre-Customization tasks..."
  if [ x$HOSTNAME != x ]; then
    echo "Configuring hostname as $HOSTNAME"
    sed -i "/HOSTNAME/d" /etc/sysconfig/network
    echo "HOSTNAME=$HOSTNAME" >> /etc/sysconfig/network 
    hostname $HOSTNAME
    # Set it so it can be used below
  if [ x$IPADDR != x ]; then
    echo "Configuring network IP=$IPADDR NETMASK=$NETMASK"
    rm /etc/udev/rule.d/70-persistent-net.rules
    sed -i "/IPADDR/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/NETMASK/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/HWADDR/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/BOOTPROTO/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/DHCPHOSTNAME/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    cat >> /etc/sysconfig/network-scripts/ifcfg-eth0 <<ENDCFG
  echo "Done with Pre-Customization"
elif [ x$1 == x"postcustomization" ]; then
  echo "Running Post-Customization tasks..."
) | tee -a /root/customize.log


Update: It turns out that having VMWare Tools functioning properly is enough for vCloud Director to configure the network settings all by itself, without the help of a script.

Tools and trade-offs

Ten or more years ago, I purchased a Gerber multi-tool. It was less expensive than a Leatherman, and it was painted in my favorite color. Since then, I saw my brother’s Leatherman Wave, and wanted one. For Father’s Day this year, Janice gave me a Wave. I like it — it’s a sleek piece of engineering in every way. The screwdriver fits more of the screws that I encounter compared to the bulky phillips head on the Gerber. The saw works far better. It’s easier to access the Leatherman’s knives, saw and file.

Leatherman Wave vs Gerber multi-tools

However, I’ve found that when I’m out and about in the yard, the Gerber is easier to work with. It’s belt-holster is secured with velcro — easy not only to open, but to close. And opening the pliers is an easy one-handed flick-of-the wrist, which means I don’t have to put the tool in my left hand down in order to open the pliers like I do with the Leatherman.

The Gerber’s pliers can be used to pry things apart. The Leatherman folds up when I try the same thing. The Leatherman is a more dangerous tool to open and use — more likely to pinch or draw blood (as it has done on a few occasions).

The tools definitely have trade-offs. I admire the Leatherman most, but for regular use, the Gerber wins.

The same thing is true of operating systems. I’ve used Linux and Windows for years, and have wanted to try out Mac OS X and a MacBook Pro.

My wish came true. For the past two months, I’ve been using a MacBook Pro 17″ with Retina display. Not only is the engineering and design gorgeous, the screen is eye-poppingly clear and crisp. The battery lasts and lasts, and the laptop doesn’t get too warm. The track-pad is the best I’ve ever used, and I love the gesture support to go forward and back in the web browser, and the gestures to switch desktop workspaces, activate Mission Control and Launch Pad.

Yet the keyboard shortcuts have taken some getting used to compared to the ones I know from both Gnome (Linux) and Windows 7. I find that it’s easy to accidentally hit Command-Q when I meant to hit Command-W. Control-Left and Control-Right don’t work — it’s Option-Left and Option-Right to jump forward and backward by a word. Home and End go to the beginning and end of a document instead of the beginning and end of a line. There’s no built-in window-snapping like Windows 7 or Gnome-Shell.

When I switch back and forth between Mac, Linux, and Windows, I hit the wrong keyboard shortcuts. Is it Command-C or Control-C for copy? On-and-on it goes — sharp edges. But I’m getting used to context switching. I love the MacBook Pro.

Last week, a Gallego UltraPro arrived from System76. It’s a compact powerhouse, and best of all, the keyboard shortcuts work like I expect, and I can get work done faster because I’m used to the underlying OS and tools. When programs break, I know where to look and what utilities to use to solve the problems (I love having /proc available). But the UltraPro with Ubuntu has sharp edges as well. The screen is a bit small, and it’s not as beautiful as the Retina display. It doesn’t have a backlit keyboard, and so far, I haven’t been able to get Evolution to talk to our Exchange server (which isn’t a problem on Mac because it has Outlook).

I admire the refinement, the visual beauty of OS X and the design of the MacBook Pro. For regular use in getting my work done, the UltraPro and Ubuntu currently win.