Yes, it's an excellent solution. Move, say, 1GB of your MP3s from your hard drive to your memory card, then increase the virtual memory setting for your hard drive.
Don't use the memory card itself as virtual memory. You can only write to a memory card about 100,000 times before it starts to fail. Virtual memory is constantly being overwritten - each time your computer copies something from virtual memory on your disk drive to RAM, it has to copy something from RAM to the disk drive.
Today on this computer that's been happening about 2 million times an hour. (Add a column for "Page Faults" in the "Processes" tab of Windows Task Manager to see this on your computer.) At best, a 4GB memory card would last less than 2-1/2 years if you used the whole thing for virtual memory; in practice, you can expect it to die in 8 months or less if it's being used for a swap file. Please don't do it.
Memory cards are cheap, but if you can swing $60 or so, a second hard drive, portable or not, is a much better value ($/GB) than a memory card. Of course, that means $60, doesn't it?