Thanks for taking an interest, @Carl Love and @dharr. I'm no mathematician, so I am struggling to explain and notate this problem.

I don't personally find the physics example helpful, but it reminds me that I should clarify that I mean Shannon entropy, not thermodynamic entropy.

Carl Love understands what I'm getting at and has the right idea by pointing out, what if n = ∞ in the equation for (Shannon) entropy? Is it as simple as that? Essentially, defining a discrete random variable of countably infinite size, would that drv represent the characteristic entropy of the underlying probability distribution?

The kind of work that I'm reading to try to find an answer are Wilf's generatingfunctionology (free to download, if you're interested) and Flajolet & Sedgewick's Analytic Combinatorics (also free to download). Wilf shows examples in Maple for solving generating functions, which is what led me here. So maybe now my question is, can I get a probability-generating function of a countably infinite discrete random variable? And if so, can I get the entropy from that pgf?

Back to the books, I think.