For most components and containers, setting the opaque property to false will prevent painting the background (and effectively make them transparent). Use setOpaque() method to change the opaque property of most components.
Some containers (JDesktopPane for example) override getOpaque() method and always return true. If you find a container that still paints the background after setting opaque to false, look at the source of that class1), and see if getOpaque is overridden. To the best of my knowledge, there is absolutely no reason to hard-code the opaque property, so override the method again and return true if you want it transparent - and add the component to the list below.
To get complete transparency on some components and containers, you will have to remove borders. Setting the border to null usually does the trick.
Here is a list of workarounds and fixes for some components that are harder to make transparent: