You do have a point, but I don’t believe that as a man you should censor yourself because you might hurt other’s people feelings, I think it’s a cowardly behavior.
And for me, my family speaks with a foul language too, it’s not too bad but he include cursing in our sentences (not directly to people but just as a method to replace words like amazing or grand or very good)