Sounds like the holes are fine. Riv nuts are easy to install with the puller, but they can be tricky. If you don't pull them tight enough, they can spin in the hole, and if you pull them too tight, you can pull the threads out. I would suggest a couple practice ones on a scrap first before installing them if you decide to use them.