Python: How to reduce memory usage

Useful information for reducing memory usage of Python programs:

  • https://m.habr.com/en/post/458518/
  • https://stackoverflow.com/questions/472000/usage-of-slots

TLDR: Dictionaries use a lot of memory. Possible solutions include using a class and slots, namedtuple, recordclass, cython, or numpy.

Screen brightness: Linux + Lenovo P50

I run Ubuntu on my Lenovo P50, and the backlight keys haven’t ever worked. Here’s how I got it working.


sudo apt install xbacklight

Then I mapped the keys using Settings > Devices > Keyboard and added mappings for the following:

Windows-F5: xbacklight -inc 5 -time 1 -steps 1
Windows-F6: xbacklight -dec 5 -time 0 -steps 1

shellcheck

If you write Linux shell scripts (bash), you should use https://www.shellcheck.net/ to improve the quality of the script.

Code reviews: Benefits and Counterindications

Microsoft has a long article in ACM Queue on what people think they’re getting out of code reviews, what they’re actually getting, as well as a list of benefits a code review tool should provide.

My main takeaways:

  • Requiring two sign-offs is too many for low-risk changes such as renaming internal (not API) methods or local variables — only need one reviewer
  • Tag the files/changes that are at the heart of the change
  • Small reviews get better feedback. More than 20 files, and a code review isn’t going to provide much, if any, value.
  • If code reviews are important, doing reviews should be tracked and rewarded, just like anything else that has value.
  • Reviews tend to focus on: comments about maintainability, documentation, alternative solutions, validation, and API usage
  • Reviews only identify bugs ~15% of the time, so some other form of validation is important
  • A good code review tool can help greatly by recommending reviewers — lightening the burden and getting knowledgeable people involved
  • Show entire file to give reviewers context
  • There’s no one-size-fits-all solution for code reviews — i.e. each team and code base has different needs and different culture.

Lightning Memory Mapped Database

My Ubuntu box decided to update this library today, so I learned about a very fast and cool key-value database. It replaces BerkleyDB on Linux systems and is also used to persist data for Redis, InfluxDB, and has even been adapted for use with SQLite — called SQLightning (20x faster).

https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

LibreOffice Spell Checking on Ubuntu 18.04

Spell checking wasn’t working for me in LibreOffice Writer. I followed the instructions here and got it working:

https://askubuntu.com/a/954608

In writer, I used this menu, and it finally worked:

Tools > Language > For all Text > English (USA)

Convert hiec to jpg

I downloaded some photos from Google Photos, only to find that they were in heic format (https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format). [When I originally wrote this blog post,] none of my linux programs know how to open that format — I needed them in jpg format. I found a utility to do the job: https://launchpad.net/~xiota/+archive/ubuntu/stuff-3

sudo add-apt-repository ppa:xiota/stuff-3
sudo apt-get install libheif-tools

sudo apt-get install libheif-examples
for f in *heic ; do heif-convert -q 90 $f ${f%.heic}.jpg ; done

organize photos with exiftool

I organize images in a custom hierarchy of directories. Lately, I’ve been using Google Photos and its shared albums to share photos with family. When I download the photos from the album that others have contributed, I want to organize those photos based on who contributed them. Google photos doesn’t make that easy.

Fortunately, cameras embed their make and model into each picture that they take, and since nearly everyone uses a different model of smartphone or camera, it is easy to separate out the images.

Windows Explorer allows me to add the model of the camera for an image in the details view. Then I can sort on the model, and break out the photos into a separate subdirectory for each contributor.

On Linux, I haven’t found an easy way to do the same thing — i.e. Gnome’s file manager, Nautilus, is anemic in comparison, as are other Linux alternatives.

Recently, I found exiftool, and it solves the problem for me, at a command-line level (it works on Windows, MacOS, Linux, BSD, and others).

Test adding the model of camera to image (e.g. Canon EOS Rebel XIV)

exiftool -p -v1 '-testname<$model %f.%e' *.jpg

Do it for real:

exiftool -p -v1 '-filename<$model %f.%e' *.jpg

The final task is to remove the model name from the image. E.g.

rename 's/Canon EOS Rebel XIV //' *.jpg

Exiftool can do so much more, including organize images into subdirectories. This examples organizes images in the current directory into year-month-date directories:

exiftool -preserve -d "%Y-%m-%d" "-directory<filemodifydate" "-directory<createdate" "-directory<datetimeoriginal" .

Models of code ownership

As I’ve worked as a software engineer, I’ve noticed that individuals, teams and companies approach things differently with regard to code ownership. Most engineers and teams take pride in quality code, in solving real problems, and in shipping software.

The model of code ownership affects the dynamics of a team, of a company, and of the software that is created and maintained. I’m strongly in favor of collaborative ownership, with the understanding that individual engineers are stronger in some areas than others. I appreciate collaborative ownership because it engenders a culture of inclusion, participation, cross-functional learning, and openness to a diversity of ideas and experience.

On the other hand, I’ve seen it work for individual ownership of certain components of a software stack — as long as there’s a way for others to give feedback in constructive ways — i.e. the owner takes pride in his/her code, wants to learn and accept feedback so that their code can be the best that it can be.

It drives a wedge in working relationships when a code owner says “hand’s off!” or “how dare you touch my code without consulting me first!”. It also creates problems when a contributor says, “I’ll make changes when I want to, whether you like it or not!”. Such attitudes indicate a lack of trust and respect. Perhaps this is why distributed version control (with pull requests or patch submission) works well compared to the anyone-can-commit model more common with Subversion. E.g. with github or gitlab, anyone can contribute, but the code stewards get to decide whether or not to accept the request.

The same principles apply when outside team members attempt to contribute code to another team. Ideally, the recipient team documents code standards, design decisions, and if nothing else, during code review, they communicate the ideals to the team attempting to contribute the code.

Here are some articles that talk about the styles of code ownership, and the pros and cons: