Monday, June 26, 2017

re: hyperproductive development and debugging microservices

Story time.
At one company I've worked with, two guys knew the code completely cold. The other 5 devs only worked on bits and pieces, so it was difficult for them to have substantial discussions let alone valuable changes to the codebase. Visibility was further limited by there being no "end to end, production-like" stack. Each dev would run one component, developing and testing it always in isolation from the other components.
When I Am King(tm), all devs (and qa) will work on independent, production-like environments. They can beat the shit out of the entire system and get high-quality feedback on changes before it lands in production, where visibility is more awkward, and there are consequences.
For those people working with services, I mean microservices: check out Linkerd from the Cloud Native Computing Foundation (they host Kubernetes, Prometheus, and other valuable software). Linkerd allows services to be assembled from other services, with additional requirements. For example in production let's say service A calls service B. In environment "Staging", it would be configured to use (production) service A, but instead of A calling B, it would call the new version, B'. This allows "services" to be stitched up together easily. Another advantage is that B can be wrapped by another service C, allowing C to emit before/after messages and enabling further tracing. This allows service components to be safely debugged, even in production! This "remapping" feature can even be done on a per-request basis, allowing you to route and test services on the fly! Check it out.

Monday, June 12, 2017

practical use of GNU Makefile for testing

Recently I reviewed a bunch of code from students. The projects are in Java. To test each one, we:

  1. compile Java source file into class file
  2. run the class file
  3. examine output

Over time I'll be getting more code, thus I want a simple workflow that will automatically adapt to having more source in the same directory. I can't just list all the source files then write out a Bash script to compile + run everything, for example. As new code comes in, I want it to be included in the overall test run.

How would you accomplish this?

GNU Make is the bomb

I used my good old friend, GNU Make! This simple tool runs programs to turn files into other files. It understands dependencies, and won't do work unless it needs to. For example, it'll compile C files into objects, then at the end will link all the objects into a single executable. The input language has rules like "to make X into Y, run program Z". You say "make Y", the system will automatically go find file X, run program Z on it, giving you the result.  If the file Y already exists, it won't re-make it, and will exit.

Make understands dependencies. For example if it knows Y depends on X, and X has been edited after Y was created, Make understands that it has to re-build Y given the updated source X.

I often use Make for holding lots of tiny 1-2 line scripts, example make backup will copy all my important files into a compressed, timestamped tarball for archival purposes. I tell Make that source code is important, but object and executable files are not -- thus my archive is small but contains all my valuable work.

Simple example: use Make to build a Java class file

Create this file, name it "Makefile" in the current directory. NOTE: the second line has a tab character -- don't use spaces!  Make is ornery in this regard.

javac $<

The % character means "anything", and $< means "dependent file".  The above stanza says "to make a class file from a java file, run the "javac" command on the dependent Java file".

To create a class file from Java source, type make myfile.class (if your source is in

Sample run:

$ make JenExercise2.class

Make automatically outputs each command as it's running it, thus you can see example what Make is doing.  For example, after the class file is built, Make doesn't need to, err, make it again. If we re-run the above command, the class file will not be rebuilt.

$ make JenExercise2.class
make: 'JenExercise2.class' is up to date.

Use Make to compile and test a single program

In my case, I want to type one command and have all my programs compiled and run so I can see everyone's output all at once. For this to happen we'll make all the class files like above. Then, for each class file we'll run it to get the output.

How do we do this?

We use a pseudo suffix. The java files and class files exist on disk, and can be manipulated directly. However to say "run test on file X", the output just goes to the screen, there's nothing stored.  To resolve this I tell Make: "hey, when I ask for X.test, build the X.class file if needed, then run it with the java command".  Make is happy and runs my commands. Since the "X.test" file isn't created, when I ask Make to do it again, it'll just re-run the commands. When we make X.class files, they live on disk, so Make won't rebuild them.  Test files don't exist -- ".test" is a pseudo-suffix -- thus Make will always run my testing commands, which is what we want.

Here's the Makefile which allows compiling and testing of Java programs:

javac $<

%.test: %.class
java $(subst .class,,$<)

The "$(subst...)" bit says "run the Makefile function subst to do string substitution". In this case, it uses the dependent file ("$<"), searches it for the text ".class", then substitutes it with the empty string.  That is, it converts "beer.class" into "beer". We use this so that when we specify "make beer.test", it'll first build the class file beer.class, then run it, stripping the ".class" to run the command "java beer". Our test runs.

And a sample run:

$ make JenExercise2.test
java JenExercise2


The bit after "java JenExercise2" is the program's output -- we can now type "make x.test" to automatically compile and run our program!

I no longer specify "make the class file", Make is smart enough to build it if necessary.  As you use Make more, it gets smarter, so it automatically does what you want!

Real example: use Make to test all Java programs

Most Makefiles are not much more complex that the example above.  However I'm a wiseass and I want a single command to run all the tests. I don't want to manually know the file names.  So, I'll define a Make variable to figure all the test names for me! To do this we'll use lots of Makefile functions.

Here's the first:

echo $(wildcard *.java)

$ make zoot

Make runs the function "wildcard" to find all Java files in the current directory.  It then runs the "zoot" command (which we're using for testing the Makefile), running a shell command to echo the function result to the screen. This is the easiest (and silliest) way to test Makefile functions.

Okay, we have a list of the Java files. We want the list of tests. That is, given "", we want "beer.test".  Here's our second testing Makefile:

echo $(patsubst,%.test,$(wildcard *.java))

$ make zoot
echo JenExercise2.test

It worked!

Now, we want to tell Make that when we type "make" with no arguments, it'll go find out which tests should be run, then run them.  To do that, we'll define a simple named verb in the top of our Makefile:

test: $(patsubst,%.test,$(wildcard *.java))

This says "generate a list of tests from the list of Java files.  The "test" command depends on running all the tests". Thus when we type "make test", or just "make", it'll compile all our programs, and run them!

Whole Makefile:

test: $(patsubst,%.test,$(wildcard *.java))

javac $<

%.test: %.class
java $(subst .class,,$<)

-$(RM) *.class


$ make test
java JenExercise2

We're done!

( The "clean" verb is an example of my little 1-2 line helper scripts.  It zaps all class files, thus cleaning my directory up. )

Tuesday, April 11, 2017

Ego is the enemy of great work

Today coincidentally I listened to two strong podcasts that highlight the enemy of great work. Not fear, but EGO.

In the first talk Andy Molinsky talks persuasively about how to move forward in your career when everything is overwhelming and/or you're an introvert. Example: you have a small business and you feel uncomfortable 1) meeting new people, and 2) promoting yourself. Of course, with a small business, a huge way to get your name out there and to get new clients is to go to "business mixers", where talking to strangers about how great you are is all you do.  His solution is simple and direct: define "success" before you go into a challenge, and repeat to cement incremental progress into large gains over time. Example: define "success" as making one contact in small business mixer, so you can talk with that person later.  Then, schedule yourself for another mixer. By doing these two things over time you learn quickly, expand your comfort level, and adapt yourself to having a running business with lots of contacts.

The second talk very much overlaps with the essential book "War of Art". It's too easy for us to say "I'm great, but I won't bring everything I have to this project until X", where X is something arbitrary. For example in one project I disengaged because I expected someone else to integrate my work into the overall system, whereas management expected I'd learn how to deploy my own work. My ego slowed down my growth in that job.  Todd points out that this behavior is pure ego and self-defeating.  By bringing everything you have to each project, you have the capability and resilience to do great work at all times.

Both of these podcasts are highly recommended!

Radio Free Leader: Stepping Outside Your Comfort Zone with Andy Molinsky

Accidental Creative with Todd Henry: The 3 Letter Word That’s Robbing You Of Great Work

Wednesday, March 22, 2017

podcast: advantages of meditation

lots of great ideas and info in this podcast on the advantages of meditation. Sam Harris interviews Joseph Goldstein.

Tuesday, March 21, 2017

new short talk: Distributed Processing

My new talk is about different processing tradeoffs, using good old Unix utility xargs, Python multiprocessing, gevent, and some distributed processing tips.  Also: tonight there's a seekrit announcement -- watch this space, and/or follow me at @JohnTellsAll

Talk URL is (this page, which will receive updates)

Slides are in Google Docs

Sunday, January 22, 2017

Favorite podcasts

I listen to a lot of podcasts. Some are moderately heavy and technical, some are lighter stories and just for fun. Many encourage me to be a better person, or help me to move forward to achieve my goals.

Here are a few favorites:

- 99% Invisible: this podcast is fun and interesting!  Their specialty is "intriguing design of everyday things", often architecture. Recently they had an episode of mini-stories, one on the design of the standard ("Snelling") eye chart. It turns out everyone uses it because it's easy, but not because it's the best. It's easier for the doctors, but not necessarily for the patients, it's not as accurate.
Another mini-story was about "copyright traps" in maps. Map makers spend a great deal of time and effort on their work, so they insert fake towns into their maps.  If another map copies their work, they'll copy the fake town, so they'll get busted for copyright violation. In one case people set up a few businesses and homes in a new area. They looked up on the map what they should name their town, they saw a name, so they used it.  A "fake town" became a real one!

- New Stack Makers: As an "expert on quality dev and devops"*, I keep up to speed with what people are doing in terms of web server systems. The trend in the last few years is for containers, specifically Docker containers, and also microservices.  These tools give us power and flexibility at the expense of making things explosively more complex. This podcast helps me stay up-to-date on what practical steps people are making to create, manage, update, and simplify their server infrastructure.

- Entre Leadership Podcast: Again as an expert, commonly I find myself in a leadership position. Generally I'm the most senior person, so people look to me for guidance and perspective. This is a challenge for my enlightenment and humility, and I take this role seriously. By listening to this and other leadership podcasts, I learn how to side step my ego and help people come up with their own plans, execute them, and get feedback.

Tuesday, December 20, 2016

Using multiple worktrees with git

Modern development with Git is great. It's really easy to work on a feature branch, make changes, undo changes (oops), and share work with colleagues. However, there are challenges. If I'm running tests on my local "beer" branch, I can't easily switch over to the "gin" branch to edit code. The local environment only has one feature branch checked out at a time. I can run tests, or add code, or switch branches, but I can't do two things on two different branches at once...

But now I can! Thanks to Manoj Mahalingam S, I know understand the "git worktrees" feature which lets you easily use multiple branches at the same time. Thanks, Manoj!

Using multiple worktrees with git