So I noticed that no matter what I designed, the holes were always too small and the post were too big. Since the holes and pegs are really not round but are approximations made by straight lines, it was understandable, but how much to compensate? I decided to see by how much and designed a fixture so that when I need a particular size, I can just measure what I came up with so that I can get a perfect fit every time.
Except for the smaller sizes, turns out that they were a constant offset. The 0.36" dia hole fit the 0.30" dia peg, the 0.30"dia hole fit on the 0.24" dia peg, etc., so if I offset the test pieces by 0.06" the holes and pegs all fit. Now I know if I want to have my 0.25" rod, I have to design it with a 0.31" hole. The peg will end up 0.24" dia and the hole will end up 0.26" dia and a perfect fit.

The holes and posts are 0.100" to 0.36" dia in 0.20" increments. I've included Inventor files so you can vary the sizes to suit.