Even Ringo started somewhere...
How young is too young? In practice, youre never too young to embark on your musical journey. Nursing infants gurgle happily to themselves and bash their rattles with gusto, while nursery rhymes introduce toddlers to the principles of melody and rhythm. If you want your child to develop a love of music, then fill their world with it - play music throughout the day, sing and clap along with them, and share the fun of music with them. There is even some research that suggests it is beneficial to play music to babies in the womb - the same music appears to provide comfort after they are born.
As far as a more formal musical education is concerned, children will inevitably develop at their own individual pace. An introduction through the Orff approach with melodic percussion, dance and singing provides a strong foundation for future learning, but a very general rule of thumb suggests that children of between five and eight years old are ready for more formal teaching. As to whether they are ready to learn a specific instrument, this is a judgement that must be based on each individuals capabilities. Forcing children to learn instruments before they are ready can put them off learning altogether if they find it too demanding, or simply dont like the instrument chosen for them.