If the cam was/is moving as much as .060/.080 i don't think the change would be more than 1 degree. But if you have the cover off put a button on the cam even useing a stock steel or alum cover, i would do it. I use the nylon button that Comp Cams sells (P/N CC205) and furnishes with their cam conversion kits, cost under $5.00. You may have to remove some mat'l from the front of the button to fit it, but it's the quick/cheap way to stop cam walk.
As mentioned above NO NEEDLE BEARING buttons, they do come apart.
JMO
Zip.