Be Wary of Poker Statistics: Part 2

Yesterday I demonstrated that a poker statistic generated from a small amount of data is unreliable. More data = more reliable statistics.

Just how much data do you need to use the statistics reliably? I found the answer through a Monte Carlo simulation showing how likely it is that a statistic is within +/- 5% of the opponent’s real playing style:

After 275 hands, 90% of stats based upon # of times seen are within +/- 5%. After 400 hands, 95% of the stats are pretty accurate.

[Note: Take this as a guide only. I made some simplifying assumptions]

Be really, really wary

95% chance of pretty good accuracy sounds good right? Does that mean that if you have played all your current opponents 400 times, then you can trust the statistics? No, for two reasons:

1) This only includes statistics based on “# of times seen”. Many statistics are based on much less frequency occurring situations.

2) Poker Copilot has 18 HUD statistics. If you have 8 opponents, that’s 18 times 8 = 144 statistics available. If each of these have a 95% chance of being accurate, then it is very likely at any given time that several of these stats do not realistically represent some aspect of an opponent’s playing style.

To conclude, as I wrote in the part 1: Statistics are only reliable if they are based on a large enough sample space. The larger the sample space, the more reliable the stats.