I haven't seen any Dell P4 1.7-1.8 GHz machines that couldn't take up to 1GB of RAM (that doesn't mean that there aren't any, but I would put a good bet that you are ok). I have seen them cost a lot of money, however - it depends on the type of RAM they use. That was right about the time we had three major types of memory: DDR, SDRAM, and RDRAM (I think). DDR and SDRAM are cheap today. RDRAM is not - it is extremely expensive.
As for your file server, I would suggest investing money here. 4GB still seems high, but its cheap to get these days. Make sure you have some sort of RAID system. I recommend a hardware RAID system (where the technology is provided by a RAID controller), but software RAID would be acceptable. I'd also recommend external hard drives as a backup - depending on which backup strategy you implement, it could save you money. DAT tape drive is acceptable and is a proven technology.
Also consider that if your network guy is the one supporting this, you want to make sure its a system he's comfortable supporting. You can get entry-level servers from Dell that have hardware RAID-1 SATA drives, Core 2 Duo processor and 4GB of RAM for a lot less than you might think.