There is a bit of schism, I believe, in the non-religious community. There are those who believe that the worst part of religions is the organization of them, the so-called "organized" religion. And then, there are people like me who believe that the worst part about religions is that they harbor irrational thought.
In my opinion, when you say that organized religion is bad, but that unorganized religion is just fine, or worse, good, then you are missing the point of what is at the root of all religion, irrationality. Irrational thinking is the enemy, not people organizing their life and/ or morals around a central idea. In fact, that is what they probably should be doing! But, it's just that it should be a rational idea.
To tell you the truth, I believe that the organization of religion is actually the best part of religion. In fact, it may very well be its only redeeming quality. It organizes people for political movements, charity, community activities, and a discussion of morality. And all of these are good things.... As long as they are based on rational thinking. Which is why, as I've said, it's the irrational part of religion which non-believers should rally against, not the organizational aspect of it.