Content that is published will have a little globe icon shown on top of the page or component icons.
The image above is zoomed in on, the regular sized icon is quite small and can be hard to see. This is what the globe icon normally looks like:
Because the globe icon is so small and hard to see, and because it doesn't tell you which environment (Staging or Live) the content is published to, another option you'll want to consider using is the Where Used feature.
To see which environment (Staging and/or Live) a page or component has been published to, right-click on the page or component and select Where Used from the menu that appears. This will open the Where Used pop-up window.
Click the Published to tab. This will display which environments the content has been published to. In the example below, the content has only been published to the Staging environment. Once it has been published to Live, this window would show both Staging and Live under the Published to tab.