It sounds like these files may be public resources. If you don't want the content accelerated, have you thought about putting them on a dedicated static server outside of Access Manager control? There are a few advantages here:
1) Server can be optimized for static content delivery
2) MAGs add some latency to requests for policy evaluation (cached or not) that can be avoided for this content
3) Browsers normally open only so many concurrent connections to a particular server. An additional server means more concurrent connections, likely leading to faster content acquisition. How much this helps depends greatly on how the site.