LWN.net has an interesting article about Easier Python string formatting. Summary: For Python 3.6, they’re going to introduce yet another way to format strings, called f-strings.
answer = 42
f’The answer is {answer}’
This past weekend, I attended the excellent #OpenWest conference, and I presented Scaling RabbitMQ.
The volunteers that organized the conference deserve a huge amount of thanks. I can’t imagine how much work it was. I should also thank the conference sponsors.
A local group of hardware engineers designed an amazing conference badge, built from a circuit board. They deserve a big “high-five”. There was a soldering lab where I soldered surface mount components for the first time in my life – holding the components in place with tweezers. I bought the add-on kit for $35 that included a color LCD screen and Parallax Propeller chip. It took me 45 minutes to do the base kit, and two hours to do the add-on kit. I breathed a sigh of relief when I turned on the power, and it all worked.
The speakers did a great job, and I appreciate the hours they spent preparing. I wish I could have attended more of the sessions.
Among others, I attended sessions on C++11, Rust, Go lang, Erlang, MongoDB schema design, .NET core, wireless security, Salt Stack, and digital privacy.
I’m going to keep my eye on Rust, want to learn and use Go, and use the new beacon feature of Salt Stack. Sometime in the future, I’d like to use the new features of C++11.
The conference was an excellent place to have useful side-conversations with vendors, speakers, and past colleagues. It was a great experience.
Say you’ve got the following log files with the oldest entries in myapi.log.3.gz:
myapi.log.1.gz
myapi.log.2.gz
myapi.log.3.gz
If you want to ‘grep’ them for a string, in order of date, oldest to newest, there’s no need to extract them one at a time, and there’s no need to concatenate the files first. Use sort to put the files in the proper order, and zgrep to search though the compressed files.
Here’s how to order the file list:
ls myapi.log.*.gz | sort -nr -t . -k 3,3
Here’s how to ‘zgrep’ them in the proper order:
ls myapi.log.*.gz | sort -nr -t . -k 3,3 | xargs zgrep “404”
SaltStack is a great infrastructure management tool. I don’t always want to use the latest version, so it’s useful to know where to download older releases. The problem is that using EPEL to install it on CentOS/RHEL systems only offers the latest version.
Fortunately, it’s possible to download old RPM packages from the following URLs — thanks to “forrest” on the SaltStack IRC channel for digging up this information.
http://koji.fedoraproject.org/koji/packageinfo?packageID=13129
https://kojipkgs.fedoraproject.org/packages/salt/
—-
Here’s how I’ve upgrade my salt infrastructure:
– Make a backup of the salt-master
– Upgrade the salt master
– Install the ‘at’ package on all minions (my minions are linux)
– salt ‘*’ pkg.install at
– Enable and start the at service
– salt ‘*’ service.enable atd
– salt ‘*’ service.start atd
– Run a script
– Copy and paste the code below into your salt directory, typically /srv/salt/upgrade_minion.sh
– salt ‘*’ cmd.script salt://upgrade_minion.sh timeout=15
Here’s the script. Note: It would be safer (and faster) to download from a trusted internal HTTP server.
#!/bin/bash
VERSION=$(salt-minion –version | awk ‘{print $2}’)
cd /root
if [ X$VERSION != ‘2014.1.7’ ] ; then
echo “upgrading `date`”
set -e
curl https://kojipkgs.fedoraproject.org/packages/salt/2014.1.7/3.el6/noarch/salt-2014.1.7-3.el6.noarch.rpm -o salt-2014.1.7-3.el6.noarch.rpm
curl https://kojipkgs.fedoraproject.org/packages/salt/2014.1.7/3.el6/noarch/salt-minion-2014.1.7-3.el6.noarch.rpm -o salt-minion-2014.1.7-3.el6.noarch.rpm
sha1sum salt-2014.1.7-3.el6.noarch.rpm
sha1sum salt-minion-2014.1.7-3.el6.noarch.rpm
echo ‘/etc/init.d/salt-minion stop ; /bin/rm -rf /var/cache/salt ; truncate -s 0 /var/log/salt/minion ; rpm -Uhv /root/salt*2014.1.7-3*rpm ; /etc/init.d/salt-minion start’ | at now
else
echo “Already upgraded to $VERSION”
fi
Here’s how I got Ubuntu Unity to show application launchers for >IntelliJ IDEA and PyCharm
In $HOME/.local/share/applications, add an IDEA.desktop file with these contents:
#!/usr/bin/env xdg-open
[Desktop Entry]
Version=13
Name=IntelliJ IDEA
GenericName=Text Editor
Exec=/home/YourHomeDirectory/idea-IC-135.480/bin/idea.sh
Terminal=false
Icon=/home/YourHomeDirectory/idea-IC-135.480/bin/idea.png
Type=Application
Categories=TextEditor;IDE;Development
X-Ayatana-Desktop-Shortcuts=NewWindow
Icon[en_US]=/home/YourHomeDirectory/idea-IC-135.480/bin/idea.png
And a PyCharm.desktop file:
#!/usr/bin/env xdg-open [Desktop Entry] Version=3 Name=PyCharm Exec=/home/YourHomeDirectory/pycharm-3.0.1/bin/pycharm.sh Terminal=false Icon=/home/YourHomeDirectory/pycharm-3.0.1/bin/pycharm.png Type=Application Categories=TextEditor;IDE;Development X-Ayatana-Desktop-Shortcuts=NewWindow Icon[en_US]=/home/YourHomeDirectory/pycharm-3.0.1/bin/pycharm.png
I use the default keyboard shortcuts, and on Linux, CTRL-ALT left arrow doesn’t work with PyCharm or IDEA (jumps back to where I was before I followed a symbol with CTRL-B). I’ve found that CTRL-WINDOWS-ALT left arrow does work. Same thing for many other shortcuts that use CTRL-ALT.
Software doesn’t exist in a vacuum — the environment and its inputs and outputs change over time. So it’s likely to break at some point and will require maintenance.
I’ve got a now-ancient static webgallery generator that I’ve tweaked and used for more than a decade. I enhanced it so that it creates animated gif thumbnails for movie files using a combination of transcode and mencoder. Recently, I added an MP4 movie file along with my photos. The web gallery generator chugged away longer than usual, and I gave it no notice — until it filled up my hard drive.
It wasn’t expecting to encounter an additional movie file format, and when it did, it went with the default of using ImageMagick to generate the thumbnail, instead of using my alternate solution for movie files — and ImageMagick filled up my hard drive with a giant temporary file.
So I edited the Perl-based web gallery program and added .mp4 and .m4v files to the list of special cases to be handled separately. It will work until the next time another new movie file format is encountered, and then I’ll need to maintain it again.
Nearly all software is that way — it must be maintained, or else it rots.
Last October, I learned about “key stretching” from Solar Designer’s “How to manage a PHP application’s users and passwords” — a concept that is applicable well beyond PHP. Combined with salting, it is effective at slowing down brute forcing attacks, and even against rainbow tables.
According to The Linux Programming Interface, a well-behaved multi-threaded UNIX program should use sigwait() or sigwaitinfo() instead of signal() or sigaction(). A linux-only program could even use signalfd().
Unfortunately, Linux/UNIX programs using sigwait() are hard to interrupt in the debugger. Instead of interrupting the program, it terminates the program. How does one work around this problem? See my answer on stackoverflow.
One of the cool new features in C++ is initializer lists. Here’s an example from Wikipedia:
struct Object
{
float first;
int second;
};
Object scalar = {0.43f, 10}; // One Object, with first=0.43f and second=10
Object anArray[] = {{13.4f, 3}, {43.28f, 29}, {5.934f, 17}}; // An array of three Objects
There’s another example over at stackoverflow.
IBM has published the useful C++0x feature support in GCC 4.5.
Here’s a cool tool: autojump, written by Joel Schaerer (thanks, Joel). I spend much of my day as a programmer navigating around in the linux filesystem. Built-in tools like ‘pushd’ and ‘popd’ are nice, as are subprocesses — e.g.
(cd ~/Download && wget http://somewhere.com/path/to/file)
… and when it finishes downloading, I’m still in the directory I was in before the download was started.
Now there’s autojump to add in to the mix. After I ‘cd’ to various directories, later, I can type ‘j Down’ to cd to my Downloads directory. Very convenient. I just wish it were built into every distribution of linux.