IBM has some nice articles on its website, here's one which explains how you can reduce your memory footprint in Linux. Here's a short snippet:
A much-touted benefit of Linux is that it is more efficient than Microsoft® Windows®, and will therefore perform better on less than cutting-edge hardware. This performance makes Linux a very attractive upgrade for the many people who have old Windows 98-era boxes still sitting around that are no longer being treated to the latest and greatest software (particularly security patches).
The truth of the matter, however, is that while the Linux kernel can still be configured to be reasonably small and efficient, as new computers have increased in power, many Linux desktop environments (such as KDE and GNOME) have added lots of features. Consequently, the default install of most distributions offer a less than stellar level of performance when installed on older hardware. The same is true of many modern applications also -- Web browsers such as Firefox and office suites such as OpenOffice are fully featured, but trying to run them on a machine with 128MB of RAM can be a painful experience!