"I'm not saying that Nietzsche believed in the literal existence of deities."
Good, then you don't deny that he was an atheist, because that's literally all that the word means. -- Oh, but you do deny it:
"I wouldn't call him an atheist."
One thing's for sure: he's safely dead and buried and unable to directly contradict any words that anybody puts into his mouth, or complain about what people call him. The fact is that he did refer to himself as an atheist, and never objected to being described that way. I can't imagine him denying he was an atheist any more than I can imagine him objecting to someone saying that his eyes were whatever color they were.
-- Wait: actually, I can imagine Nietzsche objecting to someone referring to the color of his eyes. Nietzsche detested antisemites, and spent some time and energy disassociating himself from some of them, including antisemitic politicians such as his sister's husband. Let's say for the sake of argument that Nietzsche's eyes were blue, and that his sister or brother-in-law was trying for the umpteenth time to associate him with their antisemitic crusade, and mentioned his blue eyes in the context of some tripe about racial types -- yes, Nietzsche might well have objected to that.
I have heard, although I haven't been able to confirm it, that Nietzsche sometimes denied that he was German, and asserted that he was Polish. That's easy to believe, because he detested the nationalism of Bismarck's newly-united Germany.
I emphasized that Nietzsche was an atheist, not because of anything in Nietzsche's time, but because of the current phenomenon of atheists denying that they are atheists because they are disgusted by the New Atheists. I too am disgusted by New Atheists, I just don't feel inclined just yet to surrender a perfectly-good adjective to Dawkins, Harris & Co, and let them make it exclusively their own.
But I can imagine such a time coming, if the New Atheists succeed in making the term "atheist" synonymous with themselves to a sufficient degree.
One big reason why I don't feel inclined to grant them that success without a fight is because of all the safely-dead-and-buried atheists, like Russell, Sartre, Schopenhauer, Twain and Nietzsche, who would have been just as disgusted with the New Atheists as I and the atheists who these days prefer to call themselves non-believers or skeptics. But back then they called themselves atheists, loudly and proudly, which is where we came in. If we who actually are the intellectual heirs of Russell, Sartre, et al, give up the adjective atheist to New Atheists, it will lend support to the impression that they, not we, are the intellectual heirs of those bygone thinkers. And the New Atheists are already spreading more than enough confusion and nonsense as it is.
PS: I almost got through an entire blog post without remembering to mention how much Nietzsche hated Christian theology, and would've objected to the way that some theologians since his death have used his name as though he didn't detest it all all -- a misusage which, of course, is aided by not calling Nietzsche an atheist.