One of the key elements of balance in any yoga balance pose, assuming that stability is taking care of, is keeping the bodies center of gravity over the foundation.
I'll define here the foundation of a yoga pose as any part of the body that is in contact with the floor and that is supporting some portion of the body's center of gravity.
So in side plank, above, the bottom arm and leg are part of my foundation.
Part of staying balanced can include choosing where to position your center relative to your foundation, i.e. which part of the foundation are you trying to keep your center over.
Another part of staying balanced is being able to sense where you center of gravity is relative to your foundation.
This means not only sensing when your center is over your target area but also sensing when your center is moving away from the target area.
Sensitivity without the ability to effectively respond is an exercise in frustration (unless you like frustration) and so another part of balance is being able to control your body in such a way that you can position your center and move it as required.
You could think of each part of the body: the forearms, the upper arms, the head, the ribcage, the pelvis, the thighs, the calves and feet, as each having their own center of gravity.
The position of the body's overall center of gravity depends on the position of all of these elements relative to each other, relative to your foundation and relative to the force of gravity.
Standing upright with your weight over one foot, then your center of gravity is in the region of your pelvis. (Bottom left.)
Pushing your hips left while keeping your center of gravity over your foot moves your pelvis relative to your center of gravity. Now your center of gravity ends up to the side of your pelvis instead of being centered within it. (Above right.)
Bending forwards at the hips while balancing on your forefeet, with your arms reaching back, your center has to be over your forefeet for you to stay balanced. (Below left.)
Move your arms forwards, then your hips have to shift backwards for you to stay balanced. This means that your center shifts away from your pelvis and towards your ribcage. (Above right.)
Moving into an arm balancing yoga pose like Crow Pose (bakasana) you start with your center of gravity over your feet.
For more on Crow Pose you can read:
To get your center over your hands so that you can lift your feet in crow pose, you have to move your body forwards.
This same is true in most arm balances where you start with your weight on your feet behind your hands.
For more Yoga Arm Balances read:
To balance on your hands in these yoga balance poses you have to get your center of gravity over your hands.
To stay balanced, you have to keep it there.
In a pose like half moon pose it can be easy to shift your weight towards your hand so that your center is actually over a point between your hand and foot.
How do you lift the hand?
By shifting your body so that your center of gravity is over your foot.
For more on balancing in Half Moon Yoga Pose:
read Half Moon.
To balance in side plank pose, you have to have your center of gravity somewhere over the line that connects your supporting hand and foot.
To make it easier to keep your center of gravity over this Foundation Line you can make your foundation wider by placing one foot in front of the other.
To balance with only one foot on the floor make sure that your bottom foot is stable.
One of the easier ways to stay balanced in this pose is to move the hips.
Avoid overcompensating. That's easier to do if you move slowly and smoothly.
For more on learning to balance in Side Plank Yoga Pose:
Balancing In Side Plank.
For some fun (and strengthening) Side Plank options:
A pose that is similar to side plank in that you are balancing on one shin and one hand (the same side foot and hand ) is Balancing cat pose.
In Headstand yoga pose you can keep your center over the line midway between elbows and crown.
For a step-by-step guide read
How to Do A Headstand.
If you are trying to lift into headstand with legs straight then it helps to understand that with your feet an inch off of the floor you have to move your hips back to counter balance the weight of the legs.
Moving the legs to horizontal the hips have to move back even more.
From here approaching the legs vertical position the hips can move forwards so that you keep your center of gravity over your foundation.
For more on lifting into headstand with knees straight read
To avoid excess stress on your neck when lifting your legs from plough pose, move your hips forwards (away from your head) while keeping your feet on the floor. This moves your center of gravity over your shoulders. Your neck should feel relaxed.
With your weight over your shoulders, and not somewhere between your head and your shoulders, you can lift your legs without hurting your neck.
As your legs lift higher, move your hips back, towards your head, to keep your center of gravity over a point between shoulders and elbows.
An alternative to shifting your hips prior to lifting your legs is to prebrace the neck prior to lifting the feet. One way to do this is by pressing the back of the head into the floor so that your neck feels strong.
Even with your neck "prebraced", lift your legs slowly and smoothly off of the floor so that you don't subject your neck to a sudden impulse of force.
For balancing in handstands the intent can be to keep your center over your hands. When jumping up you'll probably have to angle your arms so that your shoulders move forwards. This then gets your center over your hands.
As you bring the legs up past the horizontal then you can bring your shoulders back over your hands so that your arms are vertical.
In both headstand and handstand to reduce the need for your center of gravity to go forwards, bend the knees or move the legs out to the sides as you lift them
Or scissor the legs, bringing one leg behind the body to help balance the weight of the leg that is in front of the body.
If you are doing a handstand against a wall then you can use a similiar understanding to get your feet off of the wall.
Either pull your pelvis forwards first to get your center over your hands or scissor one leg forwards to get your weight over your hands.
If you pelvis is forwards, then as you bring your feet off of the wall move your pelvis backwards to maintain balance.
If scissoring the legs, then after removing your other leg from the wall bring both legs together to keep your center of gravity over your hands.
For more on stability and balance check out improving balance.
For a slightly more detailed discussion of balance, along with exercises, check out the ebook Balance Basics.