It's kind of easy to forget how all this affects out DHs/DPs/DFs. I know mine was struggling with the whole IF thing very quietly, and then just before our last specialist appointment pretty much spontaneously combusted one night when we were at the park, and that's when I realised how stressed he's been

Then, just after the appointment he got really sick with the flu and took a few days off work - and he hasn't had the flu in all the time I've known him, so I think it's a stress thing as well.
It's so hard, this IF ride, but as you go you find ways to cope, like the other girls have said, and you find ways to make the most of what you have, and appreciate each other as well. It doesn't mean that things become easy, you just find your groove a bit, and it helps you get from one step to the next.
I also remember saying to DH exactly what your DH said to you too, and I remember how upset he was when I said it. I think the thing is that as time goes by, and you keep doing battle, it gets harder to see the end of it. Keep your chin up though chicky babe - you've got a long way to go before your options are out. You're on the very first step with IF treatment and there are lots of other things to try if this one doesn't work out, and each one increases your chances that much more. Sometimes I feel terrified that we are starting out at the end (ICSI IVF), but when you think how good the chance are, there really is no need to be scared at this point
Have a fab weekend!!