Hillary Scott was the first from Lady Antebellum to have a baby. Dave Haywood came next, and inevitably, fans, media and friends started asking Charles Kelley when he and wife Cassie would have children. Answering that question with a smile became difficult for Kelley, as the couple had been struggling to conceive for years.

The 34-year-old singer says people didn’t seem to be sensitive to the fact that when you’re married for six years, there’s a good chance that you’re trying to make a baby, but haven't been able to.

“It was frustrating,” the singer tells Taste of Country. “You want to be happy for everybody else. I have friends and everybody is having kids. We’re 33, 34 and yeah they’re asking ‘Why not you, why not you?’”

“And you’re like ‘Okay, so you want me to just tell you my struggles?’”

The Kelleys did share their struggles upon revealing they were pregnant. They went to see a fertility doctor last November, and were told they had a one percent chance of conceiving naturally. They decided to relax about it for a few months this summer before renewing their efforts in the fall.

“We were lucky. We were going to do the IVF (In Vitro Fertilization) thing the next month and it happened,” Kelley spills.

Baby Boy Kelley is due in February. Lady Antebellum are currently taking a break from the road, while Kelley remains focused on music — his own music. He recently revealed he'll release a six-song EP in 2016. The first single is "The Driver," and it features Eric Paslay and Dierks Bentley.

Everything You Need to Know About Charles Kelley: Solo

See All Babies Born in 2015

More From Taste of Country