10,000 Foot Developer

I’m not sure who remembers Yngwie Malmsteen. He was a hair rock guitar player from the 1980’s who was quite honestly the best technical guitarist I have ever seen.  If you get a chance look up a video of him playing arpeggios.  Fast and ON.

He started a band   They played around the United States and of course Europe and Japan.  Um, the world, if you will.

His songs sucked.  Bad.

So here you have a guy with unbelievable talent who can’t write a song anyone will ever remember (anyone except maybe patrons of The Zoo in Winnipeg ).

You of course know what I am getting at.  A lot of times the ability to write a solid, supportable, operational piece of software is more important than the micro-correctness of its parts.

I started to think about this because I was reading Joel S. blogs on hiring, retaining,  and on the so-called best developers there are.  Also because a company I was at had migration after migration of  talent, and kept hiring in the BEST people (so they thought) using the same technique to get them and lose them again.

One for example — was a code rock star they hired who blew the top off their entrance exams; you know, fizz buzz problems, what are the different kinds of state, why override an equals — all those things many of us do not do on a daily basis.  Not that it was important, these things are, but the guy couldn’t code.  He had never used any meaningful tools or had any experience with any real Java architecture, but *had* written a lot with vim and plain old swing type stuff.  In a nutshell, he knew everything about wood but didn’t know how to use a hammer and saw.  The outcome was a great struggle, and a lot of bad rewrites of code because of an inflated ego.  Code reviews would consist of arguments about where to put brackets and the role of “final” in local variable declarations. Never about how to make a better overall application that did what it was supposed to do.

I consider myself an plain old programmer because I have some things I am really great at and some things I am not — good at things like doing data, solving problems in adverse situations, writing web services stacks, thorough testing and getting along with co-workers.   I also have my shortcomings like having not worked on  many service busses, been a while since I’ve touched EJB, UI sometimes bores me (and I have written MORE than my share of Java and JavaScript UI code), and although can do config management (I have set up Hudson/Jenkins servers and integrated them with repositories, build jobs, management tools like Rally etc.) it’s just not my passion.  Plus my memory for everything pedantic just isn’t there.  I only can access so much information.

The few projects I have worked on that have failed ended up having a very opinionated Yngwie’s taking over the teams near the end game.   But they simply cannot tolerate other people’s non-perfection for their own (non)perfection.  One startup I worked on had a Flex front end some years ago, and while we needed a new customer site the Yngwie on that team shot off his mouth and had people expelled to build an unneeded rewrite of the admin piece.    Many of use believe   Code Rewrite is Company Suicide:

The Siren Song to CEO’s Who Aren’t Technical
CEO’s face the “rewrite” problem at least once in their tenure. If they’re an operating exec brought in to replace a founding technical CEO, then it looks like an easy decision – just listen to your engineering VP compare the schedule for a rewrite (short) against the schedule of adapting the old code to the new purpose (long.) In reality this is a fools choice. The engineering team may know the difficulty and problems adapting the old code, but has no idea what difficulties and problems it will face writing a new code base.

It’s almost like being with someone who is OCD.  Another gig I did had a dude who literally did Mark Wahlberg type karate kata in the bathroom mirror (aka the film Boogie Nights) and would come out and insult everyone into submission.  Code did not get written so well and that environment had a terrible blamestorming attitude to our daily existence.  We all cringed (or laughed).

How about a n Yngwie who writes his own build tool when things like Maven, Gradle, and Buildr exist already and are more than adequate?  Seen it.  How about an Yngwie who abstracted classes down so far they hierarchy was 12 deep needlessly?  Seen it. technical skill is fantastic — when used in the correct manner then  technical skill is awesome, when used properly.  But I think there is a bias in our industry to measure something incorrectly — that technical skill is the same as resourcefulness and creativity.

You ever see this video, about the Netscape Rewrite that supposedly put it in the can?  The one thing that sticks out to me in that film is that the people who created it were not the greatest coders, but they CREATED it.  Then later all the uber techno geeks come in and say “this is crap” even though they themselves didn’t invent a Netscape.

It takes all kinds of developers.  But one thing I watch out for is not getting enough opinions on a team.  Many times the Yngwie’s are the loudest and seamingly best choice to ask about something; but if they have no proof of being able to write the song then they might even be the worst coder on the team, the cowboy every manager so fears.

Personally I enjoy a team with a diverse set of skills.  Also, most people I know who produce code for businesses and make money at it are not Linus Torvalds stellar.    How could anyone be?  Heck would most places even pay for a Linus?  And isn’t our industry incremental — a few people making small contributions like a pizza ordering app here, a cool SQL algorithm there; with a few great leaps and bounds — much like Evolution?

It is my opinion that a good dose  of humility makes us all better.  So what if code isn’t perfect, writing code like writing a book is a genesis of getting better; in the application and everything else.

And like most business owners — I appreciate an Yngwie Malmsteen, but I love a George Harrison.

A great little exercise is implementing a sort comparator.

The problem is this:

1. Goal:  Write a concrete implementation to an interface “Comparator”  that sorts an object Zombie by Zombie’s last name property.  Comparator has a method “compare” that you will override with your implementation.
2. The  output is a new list sorted in alphabetical order by last name, ascending.
3. Provided are the Zombie object and the Service, and any information about Comparator.

I fired up my Eclipse editor and wrote it in about 20 minutes. I always use JUnit as a way to run the code — it’s a much better technique I feel than writing a main for this fun little test.

//First the main entity class
package main;

public class Zombie {

	private String firstName;
	private String lastName;

	public Zombie(String firstName, String lastName) {
		this.firstName = firstName;
		this.lastName = lastName;
	}

	public String getFirstName() {
		return firstName;
	}

	public void setFirstName(String firstName) {
		this.firstName = firstName;
	}

	public String getLastName() {
		return lastName;
	}

	public void setLastName(String lastName) {
		this.lastName = lastName;
	}

}

//Here's the service you will use to test your concrete comparator
package main;

import java.util.Collections;
import java.util.Comparator;
import java.util.List;

public class ZombieService {
	@SuppressWarnings({ "unchecked", "rawtypes" })
	public void sortByLastName(List zombies) {
		Comparator comp = new ZombieComparator();
		Collections.sort(zombies, comp);
		printZombie(zombies);
	}

	private void printZombie(List zombies) {
		for (Zombie z : zombies) {
			System.out.println(z.getLastName());
		}
	}
}

//here's the unit test
package test;

import java.util.ArrayList;
import java.util.List;

import main.Zombie;
import main.ZombieService;

import org.junit.Test;

public class ZombieServiceTest {

	@Test
	public void test() {
		List zombieList = new ArrayList();
		zombieList.add(new Zombie("Fred", "Munster"));
		zombieList.add(new Zombie("Count", "Dracula"));
		zombieList.add(new Zombie("Lilly", "Adams"));

		ZombieService zombieService = new ZombieService();
		zombieService.sortByLastName(zombieList);
	}

}


//Here's the comparator implementation  THIS IS THE SOLUTION!
package main;

import java.util.Comparator;

@SuppressWarnings("rawtypes")
public class ZombieComparator implements Comparator {

	@Override
	public int compare(Object arg0, Object arg1) {
		Zombie o0 = (Zombie) arg0;
		Zombie o1 = (Zombie) arg1;
		return o0.getLastName().compareTo(o1.getLastName());
	}
}

The output:

Adams
Dracula
Munster

I am thinking of doing more algorithm studies at a code meetup group I started, which would focus on the fundamental problem solving skills which we may or may not use at work. The idea would be to bring any compiler (maybe groovy, or clojure, ruby, pearl etc.). How about solving the fizzbuzz problem in SQL? What about Fibonacci with QBasic? Many languages, many people, many approaches, many funs! I’ll give it some thought …

I added the project to Bitbucket in the Zombie folder and its Eclipse files:

Zombie

Working on a legacy Logging class that uses Thread and a custom Log class singleton (that doesn’t even extend slf4j.Logger, but uses it. ummm . . . future refactor); the task was givern to me to test it. Looked in the pom and saw JMockit AND Mockito referenced, and no PowerMock. There were very few tests, less than 1% for the codebase and neither mock libraries had been implemented in any tests. Knowing there were a lot of final and static classes and other nasty code in this old java app JMockit seemed a good choice because Mockito won’t test these without Powermock.

The CustomLogger class in question, that has Thread and the Log singleton was something like this:

public final class CustomLogger {
	public static void logStack()
	{
		try
		{
			Map traces = Thread.getAllStackTraces();
			printStack(traces);
		}
		catch(Error e)  
		{
			//<-- HEADACHE!!!! catches junit assertion errors, 
			// and all stays green even when things are red
		}
	}

	public static String toString(String name, StackTraceElement[] trace)
	{
		StringBuffer buffer = new StringBuffer();
		buffer.append("Thread "+name);
		buffer.append("\n");
		for(StackTraceElement element:trace)
		{
			buffer.append("\tat "+element);
			buffer.append("\n");
		}
		return buffer.toString();
	}

	private static void printStack(Map traces)
	{
		StringBuffer buffer = new StringBuffer();
		for(Thread thread:traces.keySet())
		{
			buffer.append(toString(thread.getName(), traces.get(thread)));
		}
		Log.getInstance().error(buffer.toString()); // the singleton instance
	}
}

A LOT of things going on in here. I decided to mock the Thread and also the underlaying Log singleton. Things get a little complicated. On the Log mock I used the validate() technique — but of course there was a caveat: logStack() has an empty catch block. This means that method catches any JUnit failures. I needed a way around it so I decided to throw an actual high level Exception with a message for a failed condition.

I wrote the tests with JUnit/JMockit on this as such:

public class CustomLoggerTest
{

	private final StackTraceElement[] traces = new StackTraceElement[2];

	private final Map stackTraces = new HashMap();

	private Thread localThread;

	@Before
	public void setUp() throws Exception
	{
		traces[0] = new StackTraceElement("TestClass34", "testMethod34", "testFile34", 34);
		traces[1] = new StackTraceElement("TestClass51", "testMethod51", "testFile51", 51);

		Thread localThread = new Thread();

		stackTraces.put(localThread, traces);
	}

	@After
	public void tearDown() throws Exception
	{
		localThread = null;
	}

	@Test
	public void testLogStack()
	{
		new Expectations()
		{
			@Mocked({ "getAllStackTraces" })
			final Thread unused = null;
			{
				Thread.getAllStackTraces();
				result = stackTraces;
			}
		};

		new Expectations()
		{
			Log log;
			{

				Log.getInstance();  //<-- mocking the singleton
				returns(log);
				log.error(anyString);
				times = 1;
				forEachInvocation = new Object()
				{
					void validate(Invocation inv, String buffer) throws Exception
					{	
						String expected =
								"Thread Thread-1\n\tat TestClass34.testMethod34(testFile34:34)\n\tat " + "TestClass51.testMethod51(testFile51:51)\n";
						System.out.println(expected);	
						System.out.println(buffer);		
					
						if (!expected.equalsIgnoreCase(buffer)) {
							//Assert.fail();  //can't work due to error trapping in method!
							throw new Exception("Assertion Error: Log Output Not As Expected");
						}
					}
				};
			}
		};
		CustomLogger.logStack(); //<-- Custom Logger is static
	}

}

Side note: the test coverage in this article isn't complete this is just the most interesting stuff.

Now about that empty empty catch block; that thing cause me some head scratching. Everytime it went through the assert would throw but the test wouldn't report as RED in a fail situation. The class can't fail properly because it catches a failed assertEquals. The mechanism in JUnit throws a java.lang.AssertionError or a junit.framework.AssertionFailedError -- trapped by the empty catch block. The best thing to do would be to fix the code (haha), but those weren't the marching orders. Please don't get me started.

This goes to show you -- when you are writing codde do NOT ever make *any* assumptions about seemingly harmless methods. You can get caught inheriting a methoud you may never think gets called (like an equals or hash method_ and it does.

Now about JMockit. It solved my problem, but I find the documentation not too friendly and the syntax is barely readable to me. I much, much prefer Mockito. I am not sure what I think about a mixed JMockit/Mockito application; I made the decision not to do so on that code base. But in the future might rethink that -- unit tests should be terse and readable in my opinion, not so cleaver as to require a lot of refactoring. Self contained per each test is possible, with few harnasses if possible. This makes things like onboarding and code maintenance MUCH easier. Who cares, as long as the tests run quickly and they give you meaningful, readable coverage? They are, after all, the business requirements especially in TDD is a shop's chosen method.

Sometimes I’m on several projects with differing and complicated configs  for Maven, and use a few different Eclipse/IDE installs to manage all the preferences between the projects.  With many different leads can be different Check Style files, PMD rules sets, formatting rules, naming conventions etc.  Why not just the same standards universally?  Well, different projects need different things and have different people.

For just the Maven environments, here is a simple Windows batch script I use to switch up the settings.  Assuming there are two complicated settings.xml files, it prompts for either and copies onto the file name “settings.xml” in your .M2 directory.

@echo off &setlocal

:COMMANDLOOP
echo.
echo 1 = Maven settings for Project 1
echo 2 = Maven settings for Project 2
set "TEMPCMD=%CD%"
set /P "TEMPCMD=%CD% :"

IF "%TEMPCMD%"=="1" (
 del settings.xml
 copy settings-1.xml settings.xml
 ECHO "settings for Project 1 complete"
) ELSE If "%TEMPCMD%"=="2" (
 del settings.xml
 copy settings-2.xml settings.xml
 ECHO "settings for Project 2 complete"
) ELSE (
 echo "Please enter 1 or 2. Try again."
 GOTO COMMANDLOOP
)

pause

I set up a quick multi-thread query test for a complex model entity made of several hibernate table entities. What I was trying to accomplish was a faster retrieval by running the queries in parallel. Spring JPARepository is the underlying querying mechanism. Here’s the higher level entity:

/*
* LookupEntity is a composite pojo of TableX instances,
* TableX instances are just any jpa/hibernate entities
* that share the same id (for simplicity)
*/
public LookupEntity {
private Integer id;
private Table1 table1;
private Table2 table2;
private Table3 table3;

public LookupEntity(){}

public Table1 getTable1() {return table1;}
public void setTable1(Table1 table1) {this.table1 = table1;}

public Table2 getTable2() {return table2;}
public void setTable2(Table2 table1) {this.table2 = table2;}

public Table3 getTable3() {return table3;}
public void setTable3(Table3 table3) {this.table3 = table3;}
}

In my service method is the threading test. I wrote a quick @TimeMethod annotation and point cut to time the method:

/*
* getLookupThreaded will run TableX repository methods to populate
* the TableX objects in a LookupEntity
*/
@TimeMethod
public LookupEntity getLookupThreaded(final Integer id) {
final LookupEntity lookupEntity = new LookupEntity(id);

Runnable r1 = new Runnable() {
public void run() {
lookupEntity.setTable1(table1Repository.findOne(id));
};
};
Runnable r2 = new Runnable() {
public void run() {
lookupEntity.setTable2(table2Repository.findOne(id));
};
};
Runnable r3 = new Runnable() {
public void run() {
lookupEntity.setTable3(table2Repository.findOne(id));
};
};

Thread t1 = new Thread(r1);
Thread t2 = new Thread(r2);
Thread t3 = new Thread(r3);

t1.start();
t2.start();
t3.start();

return lookupEntity;
}

This worked pretty well but would still need more scaling management with the threads if a server was going to hit the method 10k times. But what I found was more weaknesses in my table designs and indexing as opposed to coding deficiencies after turning on jpa and hibernate logging. Personally, I avoid threading and non injection container singletons like the plague unless absolutely necessary because they introduce, many times, unnecessary complexity. But if I were to write some low connection count analysis procedures; maybe doing big data style calculations in a job, I might consider threading to save time.

Well it happened — a third party project team decided they’d configure a complicated settings.xml file, which resides in the .m2 folder on your local.  But I already have projects that don’t like what they wrote in that file, so I had to create two:

• settings-common.xml for our projects
• settings-thirdparty.xml for their projects

Now when I open a workspace, I just configure the Maven plugin in Eclipse to point at the correct settings file:

Of course, to run from the command line you should specify the proper settings file as a parameter:

C:\<pom directory> mvn install --settings c:\User\.m2\settings-common.xml

About reusable UI.  I have a hard time believing it.

The backend seems much simpler to me; conducive to patterns, more easily solvedd in some ways for representing domains.  The complexity of the back and middle ware IMHO comes in with engineering, capability, scalability and new technologies.

But UI?  Oh my my my . . .

Interesting that there are only a few MVCs and persistence frameworks to choose from, comparatively.  But a billion calednar widgets.

Here’s a library of GWT widget extensions, one of many.

http://www.gwt-ext.com/demo/

Here’s the PrimeFaces Showcase:

http://www.primefaces.org/showcase/ui/home.jsf

Some JS/HTML5 Widgets, they haven’t adapted the word “showcase” for Javascript yet (that I’ve seen), but they will:

http://wijmo.com/demo/explore/

Flex/Flash. Of course Adobe. They’ve made 15 trillion of each type of widget and still no one likes Flash. Imagine — the platform destroyed its goal — useability:

http://www.adobe.com/cfusion/exchange/index.cfm?l=-1&o=desc&cat=186&event=productHome&s=5&exc=15

Flash has soooo many private sites full of widgets:

http://www.coolwidgetsgadgets.com/

So, there are for instance a billion calendar widgets.  And every single gig I had to do UI for, not one of them was good enough for what the business wanted.  Barely a one.

On almost every gig I did UI on, a few things happened:

1. Copy-paste UI code. The biggest nightmare, usually due top-down developers who didn’t have programming fundamentals (like a BA who became a “developer.”) Write 100 times, deploy 100 times. Maintenance hell.
2. Inversion of control nightmares — usually from hardcore backend/administrator types who think the command line solves everything. One widget that does everything — and very brittle at that in the end. More points of entry, higher cyclomatic index, more likelihood to break.
3.Happy medium — one widget does 80% of the lifting, write custom ones the other 20%. I think this is the best we could do with UX.

That happy medium is the *craftsmanship* — picking the point of scale or pattern.  It’s not something easily learned or taught; it may take some experience to figure out when a customer needs 1000 calendar widgets . . . or one.  Or even to convince them they need just one.

———————-

On the good side of this, think of all those engineers with this knowledge of UX, developing and honing these controls.  Makes one wonder — is this a search for a perfect implementation of a control, or for us to come to terms with the limitations of what a calendar control, for instance, might be?