Sometimes the problem with Swing apps is that the people making them don't know how to make them perform well...

( I'm reading the book "Java Platform Performance" right now, and it covers some things to look out for when using Swing... ) But yes, Swing does tend to take up a lot of memory while running... well more than VB, etc.
AWT can still be used, if you are doing a simple GUI it's fine. Also, for some types of things, AWT is preferred.
Applets for one. Personal Java devices also. (I recently had to re-remember AWT when I started programming on my Zaurus...)
I *think* you may be able to mix SWT and Swing... however I have heard the results sometimes work and sometimes don't...
I am trying to get up to speed on how to use SWT and looking for tutorials and such. From the small programs I have worked with it seems that SWT is harder to use(manual de-allocation of resources, etc.), but I am used to programming in Swing more, so maybe it's only hard because I am 'un'-learning Swing...