I've been down that path, and it was a whole lot of fun, until it wasn't.
I was seeing a Sugar Babe for 6 years and we got really tight. Knew pretty much everything about her, we always used our real names from day 1, I still now her birth date, her child's birth date, her middle name etc
We both got caught by various family members, during our time together, once her sister threatened to tell child services about her, and my wife was very close to leaving me, when I left my laptop open with my hotmail mail account, and yet despite all of this, we still continued to see each other for another 4 years after these events took place. Hell she even got engaged at one point and we continued to see each other. (that didn't last long, the engagement that is)
But I still remember the feeling when she defriended me from her facebook account. Felt like I'd be kicked in the gut..honestly it was physically painful for hours. Then she moved out to Maple Ridge, which might as well be Calgary so what use to take an hour or an hour and a half, started taking 3-4 hours. The time itself wasn't the problem, it was explaining where I was for hours on end that wasn't easy, especially given that I'd been busted once already.
But ideally if it could have worked, I would have wanted the same thing you did. Keep my wife and child, and keep her too. But it never happened, and looking back on it, it would have be a disaster had I left my wife for her. She certainly provided me with intimacy no one else ever has, but at the same time, my wife was very loyal, predictable, a good mother etc. And while I don't think marriage is really all that natural in this day and age, you're either married or your not, you can't be half way married. And as fun as she was, she wasn't all that reliable, she had her own friends and I was never going to meet them as equals, as I was old enough to be their dad's. (She was 26 when we met and I was 45). She looked like Julie Roberts, and I don't look like George Clooney or Brad Pitt. etc. So yes it's a hard juggling act but if you go down this path, be prepared for two things. One is to end up alone if you leave your wife and it doesn't work out, and two is to be damn sure that if this does happen and you are alone with 1/2 your net worth given away to your ex spouse, you have no regrets, and if at all possible, try your best not to hurt anymore more than is absolutely necessary.
Otherwise good luck to you. If it were me, I'd opt for the status quo for as long as I could.