Male / Female Business Friendships

Hello business ladies, I was wondering how you react in the following situation: you meet some new people either at a networking event or at a party and start talking business with a male guest whose wife / girlfriend is also present. Well, I have not managed a single time to make friends and continue talking business with the male part of the couple.
I love my job and love to talk about new opportunities with women but also with men! I always end up having to go for lunch with the wife / girlfriend. And it makes me want to scream! I have lots of female friends with whom I talk about women's stories and lives and worries...
And in the best case, if I manage to see the couple, it always feels like I have to end up talking more to the lady even if I am more interested in sharing about his job / his view.

Has it happened to you before? What do you do in this case? Does it just happen to me?