Programmer Productivity

Twenty years ago, an extended family relation, a patent lawyer, expressed his opinion that there’s not that much variance between engineers — at least, not as much as people suppose. Companies draw from the same pool of talent, and the idea that one company has the bulk of talent is a misconception.

This article by Bill Nichols confirms that idea in the realm of programmers.

Programmer Moneyball: Challenging the Myth of Individual Programmer Productivity

My view is that hard work, good health, persistence, consistency, the ability to work with others make a big difference. On the other hand, poor health, inconsistency and confusion of priorities lead to mediocre results.

Key takeaways from the article, with my commentary below

  1. Keep tasks small.
    • Reworking the design or the infrastructure (e.g. build system) is rarely small, but it can often be done in parallel with the existing solution.
  2. Plan for uncertainty by leaving adequate margins.
    • Planning with adequate margins often comes from similar experience, especially within an organization.
  3. Start critical work early since almost half the time it will take longer than expected, sometimes much longer.
    • There are edge cases that must be handled, and we discover things that we didn’t know up-front: new difficulties, new requirements, etc.
  4. Don’t be fooled by short-term progress.
  5. Provide a quiet work place so that programmers can focus.
    • Carve out times of day where there are no meetings.
  6. Design to control the complexity and size of solutions.
    • Peer feedback is a good way to simplify design, especially when a first attempt has started to be overly complex, or not work as intended.
  7. Encourage frequent peer review.
    • Agreed. Note that not all code changes need the same level of peer feedback, so saddling certain kinds of changes with too much process can be the antithesis of progress and quality in things like readme files, fixing spelling mistakes, adding a comment to clarify code, etc. Find a balance that works well.
  8. Automate routine tasks such as regression test and deployment.
    • I mostly agree. Beware of automating everything, especially one-off-temporary-solutions. Manual testing has its place.
  9. Develop talent with training, such as for design, review, and test.
    • Not all training is of equal value.
  10. Since quality can be taught and benefits apply to the total lifecycle cost, emphasize quality rather than speed.
    • For a revenue producing product, I tend to agree. When there’s a critical time-to-market component that has been adequately quantified, it can make sense to initially prioritize speed over quality — with the understanding that it may require all-hands-on-deck to handle quality failures.

He concludes, “the most motivating and humane way to improve average performance is to find ways to improve everyone’s performance.”

When management doesn’t prioritize items on the above list, it’s important for a software engineer to mange themselves and make them a priority.

Ubuntu on Windows: Refreshing & Fast

Microsoft has been doing interesting things with Windows, such as adding Windows Subsystem for Linux, which allowed me to install and run Ubuntu from the Windows app store.

I love having a full and familiar Linux bash command shell at my fingertips, with the utilities I know and love, including ssh. It’s fast.

https://www.microsoft.com/en-us/p/ubuntu/9nblggh4msv6

Prior to this, I used cygwin on Windows, which was also good. However, I prefer Ubuntu, mostly.

There are some caveats:

  • The home directory is in a different place from the Windows home. So for easy access, I symlink Documents, Downloads, Pictures and Videos to my Ubuntu homedir.
  • Still need to keep the Ubuntu software up-to-date. Microsoft’s app store doesn’t do it for you. Run sudo apt-get update ; sudo apt-get upgrade
  • Removable drives aren’t mapped into the filesystem automatically — cygwin was better in this regard.
  • It’s a subsystem — a container, so it doesn’t manage Windows. E.g.
    — Can’t reboot from the command line
    — Can’t manage Windows processes or users

Worth the read: Metrics That Matter

There’s a book titled “Measure What Matters”, and it’s fascinating and worth the read. This article from acmqueue with nearly the same title is quite interesting, because it turns some of my previous thinking on it’s head.

Metrics That Matter

Summary: “Speed matters”; Instrument client code to measure user experience; Measure “long-tail” latency at the 95th and 99th percentiles; even minor changes to code or user behavior can effect things; build bench-marking into release testing procedures.

OpenWRT + SafeSearch

I’ve got an OpenWRT router, and here’s how I configured it to enable safesearch on my home network.

uci add dhcp cname
uci set dhcp.@cname[-1].cname="www.google.com"
uci set dhcp.@cname[-1].target="forcesafesearch.google.com"
uci commit dhcp

uci add dhcp cname
uci set dhcp.@cname[-1].cname="www.bing.com"
uci set dhcp.@cname[-1].target="strict.bing.com"
uci commit dhcp

uci add dhcp cname
uci set dhcp.@cname[-1].cname="duckduckgo.com"
uci set dhcp.@cname[-1].target="safe.duckduckgo.com"
uci commit dhcp

for name in www.youtube.com m.youtube.com youtubei.googleapis.com youtube.googleapis.com www.youtube-nocookie.com ; do
    uci add dhcp cname
    uci set dhcp.@cname[-1].cname="$name"
    uci set dhcp.@cname[-1].target="restrict.youtube.com"
    uci commit dhcp
done

service dnsmasq restart

See the configuration:

grep -A2 cname /etc/config/dhcp

Python: How to reduce memory usage

Useful information for reducing memory usage of Python programs:

  • https://m.habr.com/en/post/458518/
  • https://stackoverflow.com/questions/472000/usage-of-slots

TLDR: Dictionaries use a lot of memory. Possible solutions include using a class and slots, namedtuple, recordclass, cython, or numpy.

Screen brightness: Linux + Lenovo P50

I run Ubuntu on my Lenovo P50, and the backlight keys haven’t ever worked. Here’s how I got it working.


sudo apt install xbacklight

Then I mapped the keys using Settings > Devices > Keyboard and added mappings for the following:

Windows-F5: xbacklight -inc 5 -time 1 -steps 1
Windows-F6: xbacklight -dec 5 -time 0 -steps 1

shellcheck

If you write Linux shell scripts (bash), you should use https://www.shellcheck.net/ to improve the quality of the script.

Code reviews: Benefits and Counterindications

Microsoft has a long article in ACM Queue on what people think they’re getting out of code reviews, what they’re actually getting, as well as a list of benefits a code review tool should provide.

My main takeaways:

  • Requiring two sign-offs is too many for low-risk changes such as renaming internal (not API) methods or local variables — only need one reviewer
  • Tag the files/changes that are at the heart of the change
  • Small reviews get better feedback. More than 20 files, and a code review isn’t going to provide much, if any, value.
  • If code reviews are important, doing reviews should be tracked and rewarded, just like anything else that has value.
  • Reviews tend to focus on: comments about maintainability, documentation, alternative solutions, validation, and API usage
  • Reviews only identify bugs ~15% of the time, so some other form of validation is important
  • A good code review tool can help greatly by recommending reviewers — lightening the burden and getting knowledgeable people involved
  • Show entire file to give reviewers context
  • There’s no one-size-fits-all solution for code reviews — i.e. each team and code base has different needs and different culture.

Lightning Memory Mapped Database

My Ubuntu box decided to update this library today, so I learned about a very fast and cool key-value database. It replaces BerkleyDB on Linux systems and is also used to persist data for Redis, InfluxDB, and has even been adapted for use with SQLite — called SQLightning (20x faster).

https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

LibreOffice Spell Checking on Ubuntu 18.04

Spell checking wasn’t working for me in LibreOffice Writer. I followed the instructions here and got it working:

https://askubuntu.com/a/954608

In writer, I used this menu, and it finally worked:

Tools > Language > For all Text > English (USA)