I'm experimenting with supporting avatar icons for comments for my blog. Instead of rolling my own avatar system, I'm using Gravatar: Globally Recognized Avatars. You register your avatar with Gravatar and it appears on a variety of blogs, including mine.
Note that you can enter your email address into the comments page as your Gravatar ID, then get your account later. Whenever you get around to getting a Gravatar account (be it days, weeks, or months later) your avatar will start showing up.
For your avatar to appear, you'll need to include the email address you registered with Gravatar. I don't store your email address at all, but if you're concerned you can also enter your Gravatar ID hash (the 32 character MD5 hash).
I don't keep your email address at all, I generate the Gravatar ID hash and only save that. Your email address is not saved and certainly not displayed in the comments.