James Joyce is one of my very favorite writers.
I'm about 300 pages into Finnegans Wake right now, and it is very tough sledding. But I'm finding it worth the effort. This is my third attempt to read it. I seem to have finally broken through and it is making more "sense" to me finally.
I read Ulysses about ten years ago and loved it.
From an atheist perspective, I enjoy his work because I find it very humanistic. He makes a strong connection between mythology and everyday events and feelings -- which is why Joseph Campbell was such a great admirer of his.