When your USB devices can be used against you

Interesting: “about half of your devices, including chargers, storage, cameras, CD-ROM drives, SD card adapters, keyboards, mice, phones, and so on, are all likely to be proven easily reprogrammable and trivially used to… attack software. Unfortunately, the only current solution on the horizon is to not share any USB devices between computers.” — Dragos Ruiu

Grepping archived, rotated log files — in order

Say you’ve got the following log files with the oldest entries in myapi.log.3.gz:

myapi.log.1.gz
myapi.log.2.gz
myapi.log.3.gz

If you want to ‘grep’ them for a string, in order of date, oldest to newest, there’s no need to extract them one at a time, and there’s no need to concatenate the files first. Use sort to put the files in the proper order, and zgrep to search though the compressed files.

Here’s how to order the file list:

ls myapi.log.*.gz | sort -nr -t . -k 3,3

Here’s how to ‘zgrep’ them in the proper order:

ls myapi.log.*.gz | sort -nr -t . -k 3,3 | xargs zgrep “404”

SaltStack RPM archive

SaltStack is a great infrastructure management tool. I don’t always want to use the latest version, so it’s useful to know where to download older releases. The problem is that using EPEL to install it on CentOS/RHEL systems only offers the latest version.

Fortunately, it’s possible to download old RPM packages from the following URLs — thanks to “forrest” on the SaltStack IRC channel for digging up this information.

http://koji.fedoraproject.org/koji/packageinfo?packageID=13129

https://kojipkgs.fedoraproject.org/packages/salt/

—-

Here’s how I’ve upgrade my salt infrastructure:

– Make a backup of the salt-master
– Upgrade the salt master
– Install the ‘at’ package on all minions (my minions are linux)
– salt ‘*’ pkg.install at
– Enable and start the at service
– salt ‘*’ service.enable atd
– salt ‘*’ service.start atd
– Run a script
– Copy and paste the code below into your salt directory, typically /srv/salt/upgrade_minion.sh
– salt ‘*’ cmd.script salt://upgrade_minion.sh timeout=15

Here’s the script. Note: It would be safer (and faster) to download from a trusted internal HTTP server.

#!/bin/bash
VERSION=$(salt-minion –version | awk ‘{print $2}’)

cd /root
if [ X$VERSION != ‘2014.1.7’ ] ; then
echo “upgrading `date`”
set -e
curl https://kojipkgs.fedoraproject.org/packages/salt/2014.1.7/3.el6/noarch/salt-2014.1.7-3.el6.noarch.rpm -o salt-2014.1.7-3.el6.noarch.rpm
curl https://kojipkgs.fedoraproject.org/packages/salt/2014.1.7/3.el6/noarch/salt-minion-2014.1.7-3.el6.noarch.rpm -o salt-minion-2014.1.7-3.el6.noarch.rpm
sha1sum salt-2014.1.7-3.el6.noarch.rpm
sha1sum salt-minion-2014.1.7-3.el6.noarch.rpm
echo ‘/etc/init.d/salt-minion stop ; /bin/rm -rf /var/cache/salt ; truncate -s 0 /var/log/salt/minion ; rpm -Uhv /root/salt*2014.1.7-3*rpm ; /etc/init.d/salt-minion start’ | at now
else
echo “Already upgraded to $VERSION”
fi

Smartphones and Privacy

A cautionary note from Arstechnica:

Given how much of what is on smartphones is now automatically backed up to the cloud, anyone should take pause before disrobing before their smartphone camera—regardless of the phone operating system or how that image will be delivered to its intended audience. The security of all of these services is only as secure as the obscurity of the mother’s maiden name of the person you sent that picture to—or of the next zero-day flaw.

I don’t think smartphones belong in bedrooms or bathrooms, but since most people want the convenience of having them there, it may be a good idea to keep the phone in the drawer while changing, or covered while showering, etc.

I think it’s a good idea to assume that what one’s smartphone can hear, see, or the data it contains could be made public someday — and perhaps sooner than we think. The same is true for any data we store “in the cloud”.

jq: sed/grep-like tool for processing JSON data

The other day I was exporting json data from mongodb using mongoexport, and needed a way to exclude certain subdocument fields. mongoexport doesn’t support excluding fields.

jq saved the day.

I piped the output of mongoexport through jq, and piped the output of that to gzip. My pipeline looks like this:

nice mongoexport –db mydb –collection accounts –out – | nice jq -c -M ‘del(.phash) | del(.h[].phash)’ | nice gzip > accounts.json.gz

Whoever wrote jq, thank you.

Introduction to SaltStack

I had the opportunity to give an introduction to SaltStack at the OpenWest conference this week (Video on YouTube).

As I was setting up for the presentation, I realized I needed an HDMI cable, and there wasn’t one in the room. A conference helper found a cable at the last minute, and we started on time (thank you!).

From what the audience said, many people are interested in using SaltStack, but haven’t started yet. It’s a fantastic tool for remote execution and configuration management. The documentation is initially a bit intimidating, but it’s easier to get started than it appears — especially using one of he many online tutorials. And there are many examples available from saltstarters.org.

SaltStack releases for CentOS/RHEL have become more stable over the past 10 months, and I’m excited about it’s future. It has a vibrant development community, and they’re implementing cool new features to extend its capabilities, including:

– RAET, an optional new, secure UDP communications protocol, which will easily support more than 10,000 minions. https://github.com/saltstack/raet
– Proxy minions will eventually allow the configuration of load balancers and networking equipment.

And advanced features, already present, sound like they could be useful:

– The orchestration framework, based around the reactor, can allow configuration when certain events are triggered. For example, Jenkins, upon building successfully, could send an event from the Salt minion to the master, and the Salt Master could automatically deploy the build to a test environment.
– Custom module, state and proxy support. In /srv/salt, create a _modules directory, etc. They can either override the existing module, or add new ones.

Ubuntu Unity application launchers for IntelliJ IDEA and PyCharm

Here’s how I got Ubuntu Unity to show application launchers for >IntelliJ IDEA and PyCharm

In $HOME/.local/share/applications, add an IDEA.desktop file with these contents:

#!/usr/bin/env xdg-open
[Desktop Entry]
Version=13
Name=IntelliJ IDEA
GenericName=Text Editor
Exec=/home/YourHomeDirectory/idea-IC-135.480/bin/idea.sh
Terminal=false
Icon=/home/YourHomeDirectory/idea-IC-135.480/bin/idea.png
Type=Application
Categories=TextEditor;IDE;Development
X-Ayatana-Desktop-Shortcuts=NewWindow
Icon[en_US]=/home/YourHomeDirectory/idea-IC-135.480/bin/idea.png

And a PyCharm.desktop file:

#!/usr/bin/env xdg-open
[Desktop Entry]
Version=3
Name=PyCharm
Exec=/home/YourHomeDirectory/pycharm-3.0.1/bin/pycharm.sh
Terminal=false
Icon=/home/YourHomeDirectory/pycharm-3.0.1/bin/pycharm.png
Type=Application
Categories=TextEditor;IDE;Development
X-Ayatana-Desktop-Shortcuts=NewWindow
Icon[en_US]=/home/YourHomeDirectory/pycharm-3.0.1/bin/pycharm.png

I use the default keyboard shortcuts, and on Linux, CTRL-ALT left arrow doesn’t work with PyCharm or IDEA (jumps back to where I was before I followed a symbol with CTRL-B). I’ve found that CTRL-WINDOWS-ALT left arrow does work. Same thing for many other shortcuts that use CTRL-ALT.