Yes, unplug the speed sensor as it you get the cart going, you will get a violent reaction when you let of the gas pedal as the dynamic brake solenoid is bypassed. And yes, you can completely remove that solenoid if you like.
If you have voltage coming from the Potentiometer, then see what you have coming out the Fwd switch. And make sure no wires are broken at the sonic weld the red/white wire is connected too. Be sure to use your drawing to trace all this out and seek the problem.....