It used to be the weekend was my build time, but that has become a little harder lately. For me, I make that time now after work most nights. I come in, shower etc, eat dinner, then into the model room until bed time. Even if I only get an hour a night on the weeknights, that is still 5 hours a week. And that is what I need to end my day anyway, that relaxing hobby time.
If you have the little ones, do not sacrifice that time with them, but maybe start to include them in the hobby with you. Otherwise, I would trade in the TV time, and then only watch that on the weekends with the spouse.