Faith, its the torque thats not enough at the low speeds so you end up having to run it too fast to get it to turn which then wears the cutting edges out really quickly.
If you are just doing a few though it may not be a big issue, using plenty of lubricant on the burrs may help.
Oh and also make sure you measure the girdle of each stone and each burr before burring the hole as they won't be the size stated and they need to be exactly right.
That means either using a burr very slightly smaller than the stone or using one very slightly bigger but not pushing it all the way in to the widest point.