One of the big annoyances I always had with Java was how bad rich client / desktop applications looked. They can be ugly, and even with Swing, the look and feel often never quite matched the native desktop... this was particularly true on Linux, and I was also disappointed with the lack of antialiased (smoothed) fonts for Swing GUIs on Windows until recently.
Ensode.net has some screenshots of the new GTK look and feel that is due to come along in Mustang (aka Java 1.6), and it does seem that it is likely to be a much better match for my Linux GNOME desktop.
Of course, this isn't enough to sway me away from SWT, which always looks completely native, and as a programming framework it gets richer by the day. developerWorks has an article comparing AWT, Swing and SWT, and a tutorial on how to migrate your applications from Swing to SWT.