I now have an iPod, which I love because it hooks into my stereo system in my apartment and in my car, and I can take it to the gym and anywhere else, and have all my music with me. I usually put it on the treadmill while I run, rather than clip it to my shorts.
My first MP3 player, which I got about 2 yrs ago, was an RCA K@zoo! It held 64meg internally, and had a memory card slot (where I had a card with an additional 128megs). Very small...about 2"x2.5", an inch or so thick, very light. Cost $77 at Wal-Mart. Worked very well. That's probably something like you're looking for. I still use it on occasion when I go on long outdoor runs, since it is so tiny and light.