One common cause of speed problems in Java programs is the misuse of arrays. Java was intended to look like C++, and arrays in C++ are the most primitive compound type.
What's wrong with arrays in Java? An array is a low-level idea exposed in a high-level language. In a low-level language such as C, an array is just a blob of memory. Accessing elements is very fast, because you're just multiplying the size of an element (typically a power of two) by the index and adding the result to the start.
By contrast, the memory model in Java is intended to prevent unsafe accesses, so no pointer arithmetic is used. Activities that do pointer arithmetic in the background are subjected to additional tests. Any lookup of an array value requires a range check. A naïve Java compiler inserts a test around each array access to check whether the index is in the permitted range. A modern compiler or VM does a lot of static analysis to try to remove these checks.
How is this problem caused by choosing the wrong abstraction? A few years ago, a group at IBM tried modifying Java to require arrays to be accessed via index sets. The only operations you could perform on index sets were those from set theory; therefore, the bounds needed to be checked only when you performed a union operation or shrunk an array. Any other operation was guaranteed to be in bounds. For example, if you created an index set from an array, all of those indexes were valid. If you intersected that index set with any other index set, you got a subset of those indexes, so they were still valid. Java code ran faster, simply as a result of choosing a higher-level abstraction for a high-level language.
This problem is very common in language design, but that's not the only place where it's a problem. Picking an abstraction that's too close to what you have or what you want, rather than being easy to map to both, can cause a lot of problems later.