Idioms facilitate communication

No matter what you think of a computer language, you ought to respect its idioms for the same reason one has to know idioms in a human language—they facilitate communication, which is the true purpose of all languages, programming or otherwise.

George V. Neville-Neil

George also explains that “a single cache miss is more expensive than many instructions, so optimizing away a few instructions is not really going to win your software any speed tests”.

HTML Subresource Integrity

LWN covers the new W3C spec for HTML subresource integrity (SRI):

SRI is designed to combat injection attacks that come through third-party content. The originating site can include cryptographic hashes of third-party script and image files, enabling the user’s browser to hash the corresponding files it receives from the third-party servers and verify that the hashes match.

Most browsers already support SRI, including Firefox, Chrome and Opera.

How to store passwords: Use Argon2

If you’re designing a service that requires passwords for authentication, store them using the Argon2 or bcrypt password hashing functions. Don’t use MD5, SHA-1, SHA-2 or SHA-3 — they’re not designed to keep passwords secure against attackers that gain access to your password database.

Reference article: How LinkedIn’s password sloppiness hurts us all by Jeremi M. Gosney

If [online services] aren’t using something like bcrypt or Argon2 for password storage, then they’re doing things very, very wrong. But slow hashing is no longer as effective of a solution as it could have once been had it only been adopted sooner.

When you suspect a password database has been compromised, even just in part, you cash in on that insurance policy [of using forced password resets] immediately by activating your incident response team and your public relations team.

What is Argon2? It’s the winning algorithm from the Password Hashing Competition. Argon2 has been added to recent versions of libsodium.

URL shorteners can compromise security

It’s useful to shorten long URLs, especially when sending them in tweets and in text messages. An LWN.net article helped me learn that they can be a security risk:

URL shorteners such as bit.ly and goo.gl perform a straightforward task: they turn long URLs into short ones, consisting of a domain name followed by a 5-, 6-, or 7-character token. This simple convenience feature turns out to have an unintended consequence. The tokens are so short that the entire set of URLs can be scanned by brute force. The actual, long URLs are thus effectively public and can be discovered by anyone with a little patience and a few machines at her disposal.

Around 7% of the OneDrive folders discovered in this fashion allow writing. This means that anyone who randomly scans bit.ly URLs will find thousands of unlocked OneDrive folders and can modify existing files in them or upload arbitrary content

— VITALY SHMATIKOV

KeyCzar: Encryption made easy

Encrypting sensitive data-at-rest (i.e. in a database) is a good idea, but how does one manage the encryption keys, and rotate keys or start using a new algorithm down the road without orphaning or migrating the old data? Use KeyCzar

Cryptography is easy to get wrong. Developers can choose improper
cipher modes, use obsolete algorithms, compose primitives in an unsafe
manner, or fail to anticipate the need for key rotation. Keyczar
abstracts some of these details by choosing safe defaults,
automatically tagging outputs with key version information, and
providing a simple programming interface.

Keyczar is designed to be open, extensible, and cross-platform
compatible. It is not intended to replace existing cryptographic
libraries like OpenSSL, PyCrypto, or the Java JCE, and in fact is
built on these libraries.

Or learn from what Google did with KeyCzar, and implement the same ideas (key rotation and key version info) using a more modern encryption library, like libsodium.

RabbitMQ, memcache, and too many socket connections

What happens when you have hundreds of services connected to RabbitMQ and memcache, and those services have a bug that causes them to keep their previous socket connections open, and repeatedly reconnect to RabbitMQ and memcache?

They crash.

It occurred to me that one can prevent too many connections using iptables on the RabbitMQ and memcache machines. Here’s how:

http://www.cyberciti.biz/faq/iptables-connection-limits-howto/

The corollary is that setting the per-ip connection limit too low can also cause problems.

I’d guess that more commonly public-facing servers like NGINX and Apache don’t have the problem of crashing. Hopefully, they degrade gracefully, and refuse additional connections while continuing to service the connections they already have open.

Great tools: ag and rlwrap

It’s fun to learn about new command line tools from coworkers. Here are two.

  1. rlwrap can be used to wrap anything in a realine command history. It’s useful to preserve command history, including the commands typed in remote ssh sessions. Just wrap ssh in rlwrap.
  2. ag, the silver searcher, is a super-fast recursive grep tool. I enjoy using it, and am pleased at how quickly it returns search results on my source code trees.

OpenWest notes

This past weekend, I attended the excellent #OpenWest conference, and I presented Scaling RabbitMQ.

The volunteers that organized the conference deserve a huge amount of thanks. I can’t imagine how much work it was. I should also thank the conference sponsors.

A local group of hardware engineers designed an amazing conference badge, built from a circuit board. They deserve a big “high-five”. There was a soldering lab where I soldered surface mount components for the first time in my life – holding the components in place with tweezers. I bought the add-on kit for $35 that included a color LCD screen and Parallax Propeller chip. It took me 45 minutes to do the base kit, and two hours to do the add-on kit. I breathed a sigh of relief when I turned on the power, and it all worked.

The speakers did a great job, and I appreciate the hours they spent preparing. I wish I could have attended more of the sessions.

Among others, I attended sessions on C++11, Rust, Go lang, Erlang, MongoDB schema design, .NET core, wireless security, Salt Stack, and digital privacy.

I’m going to keep my eye on Rust, want to learn and use Go, and use the new beacon feature of Salt Stack. Sometime in the future, I’d like to use the new features of C++11.

The conference was an excellent place to have useful side-conversations with vendors, speakers, and past colleagues. It was a great experience.

Grepping archived, rotated log files — in order

Say you’ve got the following log files with the oldest entries in myapi.log.3.gz:

myapi.log.1.gz
myapi.log.2.gz
myapi.log.3.gz

If you want to ‘grep’ them for a string, in order of date, oldest to newest, there’s no need to extract them one at a time, and there’s no need to concatenate the files first. Use sort to put the files in the proper order, and zgrep to search though the compressed files.

Here’s how to order the file list:

ls myapi.log.*.gz | sort -nr -t . -k 3,3

Here’s how to ‘zgrep’ them in the proper order:

ls myapi.log.*.gz | sort -nr -t . -k 3,3 | xargs zgrep “404”