“Why doesn’t Poker Copilot have feature X that some Windows products have?”
There are two reasons.
The first reason is this: Poker Copilot is a one-person operation. I have a finite amount of time and need to prioritise.
Secondly, I don’t pay much attention to competitors. Even if Poker Tracker finally releases their Mac OS X version, I won’t give it much attention. If I follow my competitors closely, I’ll become preoccupied with the wrong things. I’ll start obsessing over the features they have. I believe I should be obsessing over the features my customers ask for instead of the features my competitors have.
Why re-invent the wheel, you may ask? Why not use the most popular Windows products as a spec, and copy what they do? The little I have seen of the Windows products are a usability disaster. They use what I call Matroshka tabs: tabs within tabs within tabs. Buttons and icons and check-boxes are flung everywhere. I try hard to add important features without making the user interface overwhelming. It’s a tough balance.
I’m finding that people tell me what features Poker Copilot is missing. I try to focus on the most highly requested features.
Sometimes I’ll suddenly get an avalanche of requests for a specific feature. I interpret that as a sign that a) a Windows poker analysis product has added that feature; and b) people like it. And therefore it becomes high priority.
Warning: This is a highly technical post. If phrases like “garbage collection algorithm” and “hash map” make you bored already, don’t read on.
The unofficial 2.17 version of Poker Copilot fixed all major known issues, except one. Sometimes Poker Copilot crashes complaining that it has run of memory. The exact error message is “java.lang.OutOfMemoryError: Java heap space”. In Java parlance, this is the feared OOME problem.
There are two ways to deal with a Java “out of memory” error:
You can increase the amount of memory available to the Java program, or
you can track down what part of your program is hogging memory.
The first approach is the easiest. But it creates other problems, such as excess memory paging on computers with modest amounts of RAM. It’s also avoiding the real issue.
The second approach takes discipline, and I’ve avoided it for some time. But finally these last few days I got to it. With the help of JVisualVM and loyal Poker Copilot customer Andy’s 100 Megabyte hand history file, I discovered a couple of problems with the Poker Copilot code.
First, I had an almost pointless hash map in memory that contained 900,000 items. Was it coincidence that I had 900,000 hands in my hand history database? No. Problem identified. Problem eliminated.
The second memory gorging problem was caused by Poker Copilot’s parser. This is vaguely how parsing works in Poker Copilot:
Step 1: Obtain the chronological latest hand history file that needs parsing Step 2: Turn the hand history file into a list of hands Step 3: Insert the hands in that list into the database one by one Step 4: Lather and repeat
This works fine with lots of small hand history files. But add a big hand history file, say Andy’s 100 Megabyte monster, and things grind to a halt. Step 2 results in a list containing thousands, tens of thousands of hands, eating precious memory.
I changed the parsing as follows:
Step 1: Obtain the next hand history file that needs parsing Step 2: Parse a hand from the hand history file Step 3: Insert the hand into the database Step 4: If the hand history file still has more data, repeat from step 2 Step 5: There is no step 5 Step 6: Repeat from step 1
This is a gross simplification. There’s multi-threading and queues in there to keep things humming along at a brisk pace.
The result? The nasty “java.lang.OutOfMemoryError: Java heap space” problems are gone. Poker Copilot runs well now in its allocated 256 Megabytes.
The unexpected additional result? Poker Copilot is faster. The thread that parses hand history files and the thread that inserts hands into the database co-operate on a “just in time” basis that would make Toyota proud.
Something even better? Better use of memory leads to better garbage collection performance in Java, speeding up the overall Poker Copilot show. The engineers at Sun who build the Java Virtual Machine’s garbage collector noticed something a few years back. Most Java objects are short-lived. So they optimised for short-lived objects, using an algorithm called “Generational garbage collection”. A result of this is that if a high proportion of your Java objects have a short life (that is, mere milliseconds), the garbage collection works more efficiently. The new Poker Copilot approach to parsing the hand history files has only short-lived objects, and therefore works even faster.
How fast? My 2009 iMac gets through a million hands in two hours.
I find it enormously satisfying to fix a problem such as this.
These improvements will be included in the next update.
I thought the answer was 24 hours give or take a second or so to accommodate a slight wobble in the earth’s rotation. The correct answer is of course however long the online poker rooms say it is.
Today, Full Tilt Poker and PokerStars seem to have defined the day as being 25 hours long. I guess this is because USA went from summer time to winter time. This caused Poker Copilot’s HUD to not work for one of those 25 hours.
This Poker Copilot issue is going in the “I ain’t gonna fix it” category. Because reproducing this problem is something I can only do for one hour a year. That’s a long time to wait between test sessions.
In my most recent (and perhaps final) consulting gig, I had to implement a complex “standard” for statistical data interchange. It was a horrible task. The standard had many problems and even some impossibilities. Eventually I discovered that no-one had ever fully implemented this standard before, including the people who created it.
Keep the standard as simple and stupid as possible. My standard nightmare was specified in hundreds of pages of XML schemas.
The data being exchanged should be human readable and easy to understand. My standard nightmare made the data being exchanged complicated. So complicated that two people versed in the standard and the data needed to ponder it and debate it to understand it.
Standards work best when they are focused. My standard nightmare tried to use the same syntax to share wildly different types of data, such as the contact info of a person, and the historical modifications to the reported monthly unemployment data for Sweden.
Standards should have precise encodings. My standard nightmare got this right.
Always have real implementations that are actually being used as part of design of any standard. The creators of my standard nightmare claimed to have done this. But eventually they had to admit that there was no such thing. They were satisfied merely to assume they got it right. But they didn’t get it right – there were problems which made some parts of the standard impossible to implement. It was as if you wanted to request a person’s phone number, but could only do so by specifying the person’s favourite colour.
Put in hysteresis for the unexpected. I had to look up what this means. It means, allow the system to cope with situations you haven’t thought of. My standard nightmare was highly rigorous in what it allowed, and therefore made it impossible for us to return some data, or work around the problems in the standard.
Make the spec itself free, public on the web, and include lots of simple examples on the web site. My standard nightmare was free and public on the web. It included only one example which – although simple – was incorrect.
If you are ever offered the chance to work with my standard nightmare, decline. Refuse. Escape. Quickly. Normally I like a challenge. But I like it to have a realistic chance of being solvable.