Housewashing, to me, is the 2nd easiest part of what I do. Concrete is the easiest (and most boring). Roof cleaning can be some of the hardest work I've done, in that it is damned hot up there and you're constantly walking the pitch.
I imagine if you're doing multiple stories and are using an extension wand quite a bit, then it would rank up there as one of the harder parts of the job.
As far as dealing with different people, I don't find that a problem...Most are nice, happy you're there, and more than pleased with the results. I've had a few customers who are pretty stand-offish, but none that are really difficult to deal with. You just find out what they want and what they expect, and give it to them.
My favorite type of work is wood restoration, we just don't get a lot of it in this area.