Since Java 5, Java graphics APIs support OpenGL acceleration, but they are disabled by default. To enable them, set property sun.java2d.opengl to True. For example:
java -Dsun.java2d.opengl=True -jar Java2Demo.jar
Detailed introduction to Java 5 OpenGL features can be found in the article Behind the Graphics2D: The OpenGL-based Pipeline on java.net.
Java 3D Open GL Bindings are being developed as a separate project, JOGL: