Is this fuel injected or does it have carbs ?
Assuming it is FI, then it is likely that one injector is stuck open.
But NO, I am NOT suggesting that you continue guessing and buy new injectors.
They can be tested and sometimes cleaned.
If you haven't installed the regulator yet you should be able to take it back. I would never change that without first testing the pressure.