As a country music star with six No. 1 hits, Joe Nichols is on the road a lot. And while he loves touring, there's one thing that tears his heart out each time he walks out the door.

The 'Hard to Be Cool' hitmaker and his wife, Heather, have two little girls together (Dylan, 2, and Georgia Blue, 4 months), but the couple struggled with years of infertility struggles, which also brought five miscarriages. So when Nichols has to catch a plane or load up on the bus, he's saddened by leaving his girls -- especially now that Dylan has begun understanding what happens when Daddy says goodbye.

"One of her first sentences was 'no Daddy bye,'" says the singer, adding, "It's so hard to leave her. She's my little buddy."

Nichols' wife recalls a particularly heartbreaking moment between Dylan and her daddy.

"The last time she said, 'No, wait Daddy!' she ran and got her shoes, like she was going too. It broke his heart, he had tears in his eye," Heathe shares. "It's tough, but they're so little, the best place for us is at home, and when he comes home, it's all about family time."

Even though it's incredibly tough for Nichols to leave home, he knows it's best for his career -- and his career is what supports his family. Of course, he's truly grateful for the opportunity, concluding, "The first ride to the top came so fast and was a new experience for me. It was kind of a whirlwind. This one I can enjoy more and I have a lot of people around me I can enjoy it with. I'm in a different place and it's a deeper experience. I'm grateful to still have No. 1s 12 years after the first one."

 How Joe Nichols Is a Different Father Since His First Daughter 15 Years Ago

More From Taste of Country