Category: Tech

Posted on

Which programming language?

During my career, I’ve written code in C, C++, GNU Make, Perl, Java, Groovy, Ruby, Python and bash. I’ve often wondered what new programming language (or framework) I should learn so that my skills remain relevant. The number of languages and frameworks has blossomed over time, and the size of the industry has increased to support and keep them vibrant.

In the old days, I read books to learn new languages, but in modern Internet times with online tutorials and stackoverflow.com, I haven’t used them as much. Recently however, a coworker loaned me “The Go Programming Language,” published by Addison Wesley and written by intelligent, seasoned software engineers. Studying it reminds me that I enjoy books. They often contain a distillation of knowledge and wisdom that is harder to come by in online mediums.

For example, the Go tutorials explain how to use slices, but they don’t explain the mechanism of how they work, so they were a bit confusing. This book rectifies that gap in my knowledge, and briefly covers other interesting side-topics such as how UTF-8 was designed to be self synchronizing and directly searchable without having to convert it first.

Go isn’t as familiar to me as what I’m used to, so initially I didn’t like Go. I remember feeling the same way about learning Ruby, which is even more foreign than Go. In the end, there are many considerations with using a programming language that matter more than whether it feels familiar or foreign, elegant or boring. John Arundel summed it up well:

…every programming language represents a set of trade-offs. Each language is optimised for different things, so your choice of language should be determined by what suits you and the problems you want to solve with it.

Tradeoffs include, but are not limited to…

  • Integration — the ability to integrate with existing systems at various levels — the operating system, foreign libraries and programming languages, network systems and APIs
  • Maintainability (simplicity and type checking are great benefits on large projects)
  • Speed of compilation (important on large projects where frequent change occurs)
  • Performance in production — speed, scalability, memory usage
  • Tooling — build system, editors (IDEs), refactoring support, linters, formatters, packaging, deployment, profiling, testing, etc.
  • Ecosystem: libraries (availability and quality of those libraries), community
  • Backward compatibility
  • Industry momentum — pool of engineers, the time to come up to speed, trust, longevity.

Engineers don’t often get to pick the programming language for a project — it’s already been chosen by the time they arrive on the scene. Instead, they get to pick the project or the company they work on or for.

Posted on

Linux: Remap the menu key as a secondary windows key

My MS Natural Ergonomic Keyboard 4000 died after seven years of use — the underscore key stopped working. So I ordered a Microsoft Ergonomic Keyboard LXM-00001 to replace it, and found that the menu key is in the WRONG PLACE. It gets frustrating to hit the menu key when I actually wanted the windows key.

How to solve? Edit /usr/share/X11/xkb/symbols/pc and replace this
key <MENU> { [ Menu ] };

with this
key <MENU> { [ Super_L ] };

And restart the computer.

Originally, I tried a solution where I used xmodmap to remap the key at login time, but it wasn’t reliable and didn’t work across laptop sleeps. Editing the file in /etc works better.

Posted on

Record a screencast with audio on linux

Here’s how I recorded my screen, with audio from the microphone, from my Ubuntu 20.04 machine:


sleep 10 && ffmpeg -video_size 1920x1080 -framerate 25 -f x11grab -i :0.0+0,0 -f pulse -ac 2 -i default ~/Videos/screencast.mp4

Reference: https://trac.ffmpeg.org/wiki/Capture/Desktop

Posted on

How lightbulbs and TVs can do HTTPS without certificate warnings

When your web browser visits a secure website, the experience is seamless. Many in-home appliances also have web servers built in. If you were to point your browser to a WiFi-enabled lightbulb running its own web server at https://192.168.1.123, the browser would most likely give you a big scary warning.

What to do? This writeup explains most of the technical details of how plex did it. I think it’s a fascinating read. There are at least a couple of CAs that offer services to make this possible.

https://blog.filippo.io/how-plex-is-doing-https-for-all-its-users

“… they partnered with Digicert to issue a wildcard certificate for *.HASH.plex.direct to each user…”

“the client, instead of connecting to http://1.2.3.4:32400, connects to https://1-2-3-4.625d406a00ac415b978ddb368c0d1289.plex.direct:32400 which resolves to the same IP, but with a domain name that matches the certificate that the server (and only that server, because of the hash) holds.”

Posted on

Ubuntu 20.04 + btrfs + mirrored drives

I run a Linux machine at home. It’s mostly a server, although sometimes I use it as a desktop. I use it for SSH, samba, and I have apache hosting family photos. The photos sit on a secondary drive, and lately it manifest sector errors. Some of photos were no longer accessible.

Fortunately, I have a backup on an external hard drive. Thank goodness for backups!

I decided that it was time to upgrade to Ubuntu 20.04. This time, I wanted mirrored drives with a file system that detects silent data corruption 2.

I started with btrfs, partitioned my first drive and installed Ubuntu. But wait? Should I have chosen ZFS?

ZFS is a first class citizen in Ubuntu, and is reportedly rock solid. The documentation is good.

In contrast, it takes some searching to find good btrfs documentation, and some of the documentation refers to scary features that might eat your data — not comforting. Yet Facebook uses btrfs, so that’s good, right? On the other hand, they have people whose job it is to use it for the right use cases, and configure it appropriately. I don’t.

Oh, and there’s bcachefs! It sounds great, and is reportedly faster, more modern than btrfs or ZFS, and more reliable that btrfs. Unfortunately, it’s not part of most Linux distributions yet, nor is it feature complete. I’ll look forward to using it in three to five years.

Ultimately, I chose btrfs because it doesn’t require that I dedicate the entire hard drive, and because I already had some momentum having started my install with it.

I found the following guide to be helpful to set up btrfs in mirrored mode: https://work-work.work/blog/2018/12/01/ubuntu-1804-btrfs.html

Arch has good documentation for btrfs: https://wiki.archlinux.org/index.php/Btrfs

Update Sept, 2021: Ars Technica has a review of the many sharp edges (i.e. dangers) of using Btrfs. Among other things, the author, Jim Salter, says:

Btrfs’ refusal to mount degraded, automatic mounting of stale disks, and lack of automatic stale disk repair/recovery do not add up to a sane way to manage a “redundant” storage system.

Regardless, it’s what I’ve been using, and the data healing part of btrfs reportedly works well. I’d rather not go through a “tortuous” recovery in a future day, but at least I’ve linked to the article, and will reference it when that day comes.

Once Ubuntu was installed, it took hours to add missing packages, reconfigure software and restore from backup. Here’s some of what I installed:

sudo apt-get install meld screen jhead socat apache2 zsh tree tcpdump zip xz-utils zsh vlc ffmpeg samba fail2ban mutt curl strace git ruby rsync python3-virtualenv fetchmail procmail postfix openssh-server netcat-openbsd lsof gwenview dnsutils dosfstools build-essential htop silversearcher-ag mailutils vim-gtk3 at fetchmail

Posted on

COVID-19 experience working from home

Like tens or hundreds of thousands of others, I am working from home while the coronavirus pandemic sweeps the United States and the rest of the world.

Here are a few observations about my experience working from home. I’m a software engineer, and I realize that my experience is different from that of hardware engineers, QA folks, customer solutions agents, teachers, and even from other software engineers.

Sleep. The first several days, I worked from my bedroom, which has a great view of the outside world, but I found it difficult to sleep restfully at night. Once my wife shut down her in-home preschool, I moved to working from one of the two preschool rooms, and now I sleep quite well at night. It’s good to have a separate work place.

Exercise. Working from home, I feel more fidgety… maybe I move around less, because there are fewer meetings, and fewer impromptu discussions. So I feel a strong need to get out of the house and work in the yard (pruning, since it’s spring), take a walk or ride a bike.

Convenience. The kitchen and bathroom are closer. If I want to take a nap, no problem — I’ve got a comfortable bed nearby. Taking a walk around the block is a piece of cake — the front door isn’t that far away compared to when I’m working in the office.

Interruptions. Most of my children are teenagers, and believe it or not, I have fewer interruptions at home than I do at work. When my children are doing their remote school, two of them are in the same room as me, with headphones in their ears, and it’s so quiet that I can hear a pin drop.

Social. I enjoy the more frequent face to face interactions with my wife and children. I miss the face to face interactions with my coworkers. Video conferencing is a great invention, and yet it’s not the same as being there.

Communication. There are a lot fewer impromptu hallway discussions with coworkers, and more with family members. I’m not sure if we’re better at communicating with slack now that we’re all remote or not.

Internet. We have excellent wireless internet service. Since our entire family is working/schooling from home, we notice much more quickly when there’s network latency or poor quality.

Up to now, we’ve used a Disney Circle for parental control, and we knew it caused problems on a nearly weekly basis (it does ARP poisoning of a local network), especially for managed devices. With remote school, Circle started breaking things on a daily basis, so I turned Circle off.

I found that 5 Ghz WiFi on the main floor of our house doesn’t penetrate the floor and walls to the basement, and so I pulled my old ASUS WiFi router out of storage, put it in access point mode in our basement, installed a firmware update, and wired it to the router upstairs. Now I get better speeds and connectivity.

Commute. I had a great commute before, but it’s even better now. With so many people staying home, the air is cleaner.

Productivity. My productivity ebbs and flows at work, and the same is true while working from home. Overall, I feel a bit less productive working from home. I’m more inclined to put my Slack app in “do not disturb” mode when I finish working.

Update: It took three weeks before I settled into a good flow of working from home.

Posted on

Captive portal detection

I did a wireshark dump on my Ubuntu 18.04 laptop and noticed that both Firefox and Ubuntu do captive portal detection. Of the two, I think the Firefox method is simpler to implement and use.

Firefox does an HTTP GET on http://detectportal.firefox.com/success.txt
Responds with HTTP 200 OK with a Content Type of text/plain and a body of “success\n”

Ubuntu does an HTTP GET on http://connectivity-check.ubuntu.com
Responds with HTTP 204 and a header of X-NetworkManager-Status: online\r\n

Notice that captive portal detection uses an unencrypted transport — http, and not https.

Posted on

Simulate dropped or latent packets in Linux

I’m documenting this more for my own reference than anything, partly because I’ve used ‘tc’ off and on over the years.

  • https://stackoverflow.com/questions/614795/simulate-delayed-and-dropped-packets-on-linux
  • https://wiki.linuxfoundation.org/networking/netem

    
tc qdisc add dev tun0 root netem loss 30%
tc qdisc show dev tun0
tc qdisc change dev tun0 root netem loss 0.1%

    When finished:

    tc qdisc del dev tun0 root
    • Posted on

    Programmer Productivity

    Twenty years ago, an extended family relation, a patent lawyer, expressed his opinion that there’s not that much variance between engineers — at least, not as much as people suppose. Companies draw from the same pool of talent, and the idea that one company has the bulk of talent is a misconception.

    This article by Bill Nichols confirms that idea in the realm of programmers.

    Programmer Moneyball: Challenging the Myth of Individual Programmer Productivity

    My view is that hard work, good health, persistence, consistency, the ability to work with others make a big difference. On the other hand, poor health, inconsistency and confusion of priorities lead to mediocre results.

    Key takeaways from the article, with my commentary below

    1. Keep tasks small.
      • Reworking the design or the infrastructure (e.g. build system) is rarely small, but it can often be done in parallel with the existing solution.
    2. Plan for uncertainty by leaving adequate margins.
      • Planning with adequate margins often comes from similar experience, especially within an organization.
    3. Start critical work early since almost half the time it will take longer than expected, sometimes much longer.
      • There are edge cases that must be handled, and we discover things that we didn’t know up-front: new difficulties, new requirements, etc.
    4. Don’t be fooled by short-term progress.
    5. Provide a quiet work place so that programmers can focus.
      • Carve out times of day where there are no meetings.
    6. Design to control the complexity and size of solutions.
      • Peer feedback is a good way to simplify design, especially when a first attempt has started to be overly complex, or not work as intended.
    7. Encourage frequent peer review.
      • Agreed. Note that not all code changes need the same level of peer feedback, so saddling certain kinds of changes with too much process can be the antithesis of progress and quality in things like readme files, fixing spelling mistakes, adding a comment to clarify code, etc. Find a balance that works well.
    8. Automate routine tasks such as regression test and deployment.
      • I mostly agree. Beware of automating everything, especially one-off-temporary-solutions. Manual testing has its place.
    9. Develop talent with training, such as for design, review, and test.
      • Not all training is of equal value.
    10. Since quality can be taught and benefits apply to the total lifecycle cost, emphasize quality rather than speed.
      • For a revenue producing product, I tend to agree. When there’s a critical time-to-market component that has been adequately quantified, it can make sense to initially prioritize speed over quality — with the understanding that it may require all-hands-on-deck to handle quality failures.

    He concludes, “the most motivating and humane way to improve average performance is to find ways to improve everyone’s performance.”

    When management doesn’t prioritize items on the above list, it’s important for a software engineer to mange themselves and make them a priority.