As for cost.. Merlin has made a good point. A lot of cheap guitars are cheap. But to put off playing while amassing enough for a $700 Fender Strat plus another $700 for a amp is probably not the best way to go right now. There are plenty of guitars that are inexpensive yet not cheap.
The trick is to find one. There your best bet is to have someone that knows how to play, how to tell if a guitar is in good condition, and then have a go at the local papers and Craig's List. In this economy a lot of people are getting rid of unwanted gear and deals can be had. Loads of decent Fender Squires can be found, Yamaha Pacifica is also a nice entry level guitar. It's what I started on and I never felt like I wasted my money.
As for the amp, I suggest a small amp with digital modeling. It's not a tube amp but you'll have a lot of different sounds and tones to play around with and it can substitute nicely for $100's in pedals while you're in your learning phase.
Then again I could be wrong and money might not even be an issue, in which case I'm up for adoption.