Just build your own, since you already have some of the parts. The "skills" required are quite minimal as in, a CPU and RAM will only plug into a monther board one way, kind of like a 3 prong extension cord will only plug into a outlet one way. It will take you about 30 minutes to an hour for most people.
Since you have a DVD drive and hard drive, if you have about 10 GB of free space, i would just get a "bare bones" system such as a Shuttle, then add your CPU and RAM to your hard drive and DVD drive.
Then order your free Ubuntu CD's (or pay $50 for Suse 10 if you want support) and the installer will automatically make it dual boot on your existing hard drive so you still have windooze, but can use linux as well.
If you want a second Hard drive, I bought a 500 gb external USB 2.0 for about $160 a month ago. If you don't play video games you don't need a very high end graphics card even if you want to use XGL. After using XGL for 6 months, I can't stop using it and just find Vista Aero and the new OS X Aqua "Spaces" a big step down from a productivity standpoint.
If you can use Windooze or OSX, new versions of Linux (SUSE and Ubuntu inc) will be not be a problem, all your applications will open files from MS Office and best of all, they will be free from viruses/worms and free $$$. You should be able to get everything for easily under $500, maybe $700 for a Core Duo with lots of RAM.
A co-worker and I just finished building a 4 Shuttle cluster, each with a Core Duo processor (so that's 8 cores) and 8 GB or ram total for just over 2k. The thing smokes the 7k Mac Pro machines for 1/4 the price.