The very fact that you have people in this forum attests to a need to bond with others of similar experiences or likes. While we have rejected the idea of a God, that the Torah is True, and that the stories are nothing more than literature, it does not mean that everyone feels equally comfortable with rejecting that connection entirely. For example, in Israel there is an atheist kibbutz made up, mostly, of Russian immigrants. They are proud of their bond, as only Russians can be, and proud of their atheism.
But there is a difference between being part of the club because you got the lifetime membership (Jew) and actually believing in any of that (Jewish). For example, if you are invited to a Purim party, which is basically a costume party (costumes were a a much later addition) with a lot of drinking and dancing and singing, based on a story that cannot be placed historically - do you denounce such things, or accept the invite and have a good time? I agree with @Loren who brings up the social nature that we humans crave.
I had a friend who was Scottish (in the USA) who, every month, put on his kilt and got together with other Scots and did whatever Scots do (I am still not sure). It was a cultural thing, and he could name all kinds of Scottish history that I was ignorant of. It reminded me of the Flintstones where Fred and Barney would go to the "Loyal Order of Water Buffaloes Lodge No. 26". But in the case of finding such a social outlet that is devoid of any reference to a deity, is going to be tough. And if a religious reference is disturbing to you, then perhaps a different social outlet or connection would be more appropriate. It doesn't have to be Jewish, but it could be. In other words, find your own comfort zone, and be honest.
According to Jewish law, you will always be a Jew albeit a "naughty" one.
There is a Yiddish expression: If you're eating pork, let it drip from your beard.
You are a Jew. Period.
It's an expression, for Christ's sake! LOL!
Tell the frummer Yiden you're an apikoros!
I was raised Jewish. So let us ask, Am I still Jewish???
I am NOT Jewish in the sense that if you define Jewish as a religion, the the fact that I do not do Judaism (or any religion) makes me NOT Jewish!! And "Jewish" is just a label I was taught to have in order for my parents to avoid feeling too guilty due to the possibility of not teaching me to have the label they were taught to teach their kids to have. You do not do any part of Judaism, so you are not Jewish!!! Now you need to find a life and find fun activities and hobbies and stop obsessing over religion and culture and the fact that you are an atheist and you used to do Judaism and catholicism and so on.
I do not have time to care what labels other people need to call me by (in order for them to avoid feeling too guilty or bad due to the fact that I don't participate in the religion/ culture I was taught I have to participate in and have my life revolve around) and you shouldn't care either.
At least two people at atheist events asked me whether I identify as a Jewish atheist and I replied, no, I do not identify as a Jewish atheist, I identify as Michael.
You got your sons circumcised? That was a very evil thing to do to your sons. You did not not know any better. You do now.