How old are your children?
I met my wife when she asked me if I knew anything about camping. In the next ten years, we never camped once, and our longest hike together was probably to Angel Falls and back, about two miles round trip.
Just after our son's fourth birthday, while she was at work, I took Cameron up to Mount Monadnock and we climbed it together. We did a few more mountains that September; Watatic, North Pack, and Crotched. In October the three of us took a trip to Vermont to look at covered bridges. In November I took Cam up to Pinkham Notch after showing him some of my old slides and he said that he wanted to see the mountains.
Just after New Year's, 1996, I moved out (her decision). No, it wasn't attributable to my hiking, which I hadn't done much of at all. We were divorced later that year.
The next two Mays Cam and I went up to Maine to hike. In the meantime, I met Susan, who was willing to do just about anything to spend time with me. She had hardly hiked at all in her life, now she's climbed 74 Northeast 4000-footers and she's just three ascents shy of completing the Adirondack 46. Cam has 37/30 under his belt. Susan and I can get away while Cam is in school.
Susan has asthma, though, so she can't hike in the winter. I'd like to try it, but not so much that I want to leave her behind to go do it.
You have to decide if your children are old enough to take along, if that's a possible solution, or how selfish you want to be if it's not. As Ann Landers used to say, are you better with her or without her?