AoA is the angle between the line that extends from your nose, and your velocity vector (i.e. where you're actually going). The FPM in the HUD is a visualization of the latter, which explains why you can place it on an ant on the ground and be 100% sure you're going to give that particular ant a very bad time.
When you climb or turn, the direction you're actually going will "lag behind" where you're pointing: the slower you are, the more it lags (i.e. the higher the AoA). At certain angles, the airflow on the wings becomes disrupted (basically vortices start to form), and you start losing lift: that's buffetting. If you keep pulling the stick, increasing AoA, one or both wings will completely loose the ability to generate lift, stalling the aircraft. The "wing-drop" you experience during tight turns when riding the chopped tone is exactly that.
It's also not true that the faster you go the higher AoA angles you can pull: an aircraft can stall at any speed, you can check by yourself by yanking the stick back hard while in a 300+ knots dive.
Just to be clear, although it's easier to think that AoA is controlled through the pitch channel alone, the fact that it's a function of speed means that thrust is relevant as well. To understand how this work, slow down to 160 knots and place your FPM on the horizon line: note how your nose is pointing up, but your velocity vector is straight forward; now add thrust while applying some forward stick to keep the FPM stable, and notice how your nose lowers as you gain speed; if you hadn't applied forward stick, the nose would have tended to stay in place, and the FPM would have risen instead.
This ties to pretty much everything when it comes to flying: trimming and landings for example. The latter is especially interesting, because it explains how switched controls work with landing (i.e. controlling speed with pitch and slope with thrust).
To confuse matters a bit, the AoA gauge in the cockpit is not in ° but rather in unspecified "units".