tar, gzip, bzip2, et al.
There are many different compression formats available, and thankfully OS X recognizes most of them natively. However, Safari is not able to peak inside all of them. This is especially true if you decide to use multiple compression schemes as described above.
Even if you wisely decide to pick one compression format, do not use any of the esoteric ones. Safari does not love them like it does ZIP, and none of them offers the easiest/best experience for your user. In some cases, Safari will expand only part of the file (in the case of tar.gz), or the file will just be dumped onto the user's computer with no hint of what to do next.
The tar, gzip, bzip2, et al., formats are more "geeky" than what the standard user needs to see. Most people have never heard of these formats and have better things to do (such as using your application) than learn about fancy compression formats. Sure—the compression might be 0.00001% better, but just avoid them. Most people do understand ZIP just fine.
There are a large number of offenders in this category.