I've used this Creative Muvo TX FM for 10+ years and still don't see anything that can replace it's functionality.
It has a built in USB connector so I don't need to fiddle with or carry a cable. It takes regular AAA batteries so it's easy to swap and not worry about charging. It lasts forever on a single battery and I don't need to worry about draining my phone which I need for other purposes. It's tiny so I don't have to hold it in my hand when running. I have it for workouts (comes with workout clip) and travelling. Physical controls are also nice. I don't like having to take my phone out, open the case, unlock my phone, open the app, etc. to change playlists, etc. Also - it has a built in FM radio which phones have omitted for years. Phones allow me to carry and access more music than ever before - but make it damned difficult to use and consume.
The only viable alternative to this is the Huawei Android Wear Watch I bought off lucky1 here on CP. 3rd party apps let me sideload Mp3s onto the watch which I can then sync to a Bluetooth headset to play music without carrying a phone. It kinda works but then the battery gets drained ultra fast and then I have a non-functional watch strapped to my arm.
The biggest problem now with technology is I don't really listen to pre-recorded commercial music anymore. Most of the music I listen to are indie releases or DJ mixes on Soundcloud, Mixcloud, and other streaming services in addition to bootlegs of live performances and other video based compilations (again, streaming, Youtube, Vimeo, etc.) so I have to bother ripping streams into audio files if I want to use traditional portable media formats like Mp3. Arrgh. It's actually easier to hit play on a stream and record it straight to cassette tape...