Although it was slated for WP 1.6, WordPress still is using the deprecated Atom 0.3 output.
Time to update atom xml to 1.0 ! Find the fix instructions here:
http://nlindblad.org/2006/08/05/wordpress-atom-10-support-revisited/

(the original php5 fix is here but I figure most people are still using php4)

I also just found this in trac, Atom 1.0 template from July 2005 and
someone posted an Atom 1.0 comments template back in October 2005.

I discovered the PHP4 fix was not quite complete so I had to do a few minor tweaks.

Here are the two templates I edited, zipped up for the lazy or non-programmers.
http://www.savefile.com/files/4808860

For the ATOM comments to work properly with rewrite urls you have to go into wp-includes/functions.php and find function do_feed { around line 750
comment out the forced rss2 for comments like this (around line 761):

if ( is_single() || (get_query_var('withcomments') == 1) ) {
// $feed = 'rss2'

and then enhance/replace the atom function like this (around line 785):

function do_feed_atom($for_comments) {
if ( $for_comments ) {
load_template(ABSPATH . 'wp-commentsatom.php');
} else {
load_template(ABSPATH . 'wp-atom.php');
}
}


A few things were missed in the PHP4 fixup.
1. the comments link should be changed from wp-commentsatom.php to comments/feed/atom/
(in both .php files)
2. all the feed urls should end in trailing slash to match WP’s default ie. feed/atom/ not feed/atom (even though both work)
(in both .php files)
3. the PHP4 time patch needs to be applied to wp-commentsatom.php as well (there is one reference to ‘c’)

Someone needs to put this into the trac bug system if not already so it can get out in the next 2.1 alpha build.