That isn't a fair assessment. In cultures where everyone shares the same religion there is a standard of moral behavior. If there was one worldwide religion there would be one moral standard. The religious factor in morals is completely moot. It is inconsequential what the source of a moral code is. The only factor of consequence is how many people share a given moral code. It doesn't even seem to matter what the code is, only that it is shared.
Personally, I don't even think people need to agree with a moral code in order for it to be binding. Plus, morals can be taught, therefore theoretically one universal moral code is only a forced reeducation session away from reality.