This is a problem with "single-purpose" devices for kids, too. Drawing tablets, music players. They're all actually full Android phones (sans cell modem) and tablets. It sucks.
The issue isn't really Android, it's the touchscreen and the way the UX is a regression from many analog single-purpose devices.
If you gonna have a single-purpose device - make it analog (or close to analog)!
Don't give it a perceptible boot-time and all the other flaws that come with general-purpose computing. Don't make the user have to "wake up the device", let alone have to visually confirm that it is woken-up, before they can switch to the next song.
It represents a general purpose computer on your network which will accumulate vulnerabilities and never be patched or otherwise secured, making it a persistent insider threat as a launchpad for attacks on your network