I've been in those shoes before....it sucks.
Luckily I inspected everything before I went home...and ended up leaving without the car, and I still had the green in my wallet.
My case was one in which the seller lied over the net and phone, saying the car was in good shape, had lots of new body panels, and JUST needed the radiator support pulled back.
It ended up needing bodywork on the roof (basketball sized dent), a whole new Cowl Panel (Trim and actually metal piece), 2 new fenders, a whole passenger side Unibody front horn, 4 new tires, and a rear cover, not to mention all the other little dents and dings. I didn't lose any cash, but wasted about 8 hours of driving, and gas...
It sux, but it's a risk you have to take.
Now, for not inspecting the shit...I can't help ya there, the seller should've been more than willing to wait however, especially if he had set a time with you.