The media servers work fine for other assets when you use them, they just do not cache css and javascript if CCC is not enabled. This provides you the ability to locally debug without bringing the CDN down for your whole site.
If your CSS is having issues, I would run it through a validator, the only time I have ever seen the CCC library have issues is when people short close statements at the end of files. If you are using properly formed CSS and there is truly a bug, report it with an example, it will help it get fixed.
I don't think you follow. Its all an ease of use thing. It keeps the development flow where users can clear their cache and changes are shown. With what you are suggesting, users would have to load a minified file, try to find the code they want to change, change it, save it on the server, then go to their cdn and void the caches. That is not considered user friendly for novice users. I cannot think of any software package that ships with pre-minified code other than PrestaShop 1.7, and how is that working out for them?