Mining is hard work; often dangerous because of fires and explosions and cave-ins. The mines are cold (especially in winter) and there's limited fresh air. You have to be agile, not afraid of confined places, and I suppose every day that you go to work you don't always know if you'll be coming home at night. It's not work that I would want to do.
I imagine that a miners life would be very complicated because he would never know weather he would be able to provide for his family like buy them food or take them to the doctors if they were sick if you didnt find gold for money.