When I push the start button it won't stay down. The only way I can vaccuum is if I keep my finger on the start button and manually hold it down.

Sounds like the switch needs replacing or the switch cover isn't pushing the switch down enough to latch it. Take the switch cover off and see if the switch latches ok when you press it (with the machine unplugged of course!). Replacing it should be cheap and easy if you're confident with these sort of things. There's a useful tip here which will make removing the switch cover a bit easier:

http://www.fixya.com/support/t7785183-trying_remove