Basically, there are three kinds of updates…
Major Releases
These are done every 3-4 months as described in the WordPress roadmap. These releases would have a tag like 3.2, 3.3, 3.4, etc. These releases do not include any security updates. These would only contain new features. Personally, I don’t recommend my clients to update their WordPress core at this point. Because it may contain a few bugs that may break a WordPress site.
First Bug Fix Releases
These are mostly done just after the major release. For example, 3.2.1, 3.3.1, 3.4.1, etc. It is usually safe to update the WordPress core at this stage. Because most of the critical bugs (that were not previously known in the major release) would have been addressed by now. Again, these bug fix releases do not usually contain security updates. It is recommended to update WordPress core at this point. As it may not contain any security updates, you may ignore to update here too. Personally, I recommend everyone to give it a go when I see the first bug-fix release.
Minor Releases
These releases contain security patches and/or major bug fixes (that were not caught during the major release and during the first bug fix release). These releases may also contain some minor bug fixes too. It is highly recommended to update WordPress core at this stage. These releases usually have a tag like 3.1.2, 3.1.3, 3.1.4, etc.
Still unclear? Please feel free to ask in the comment section below.