Programming is a discipline that should be an essential part of the school curriculum, not because everyone has the talent to be a professional programmer, but because working within the rules of structured systems (and occasionally extending or working around them) develops your brain in ways that prepare you for real life, which is also dominated by systems. The systems you encounter in real life are generally less formal, but learning to navigate them and developing creative approaches to work around their restrictions (or extend them) is no less important for a successful social and economic life.
How young is too young to learn programming? Better to ask, “What sort of programming is appropriate for a toddler, a 5th grader, a teenager?” Obviously, it will be frustrating to teach (or try to teach) most 5th graders classical C programming, but 2 and 3 year olds are already capable of playing with legos and other systems with “formal” albeit physical rules. It remains only to abstract these systems to allow the student to develop an understanding on how non-physical systems can be created and extended to produce new objects that are fun to interact with and sometimes solve real world problems. Software is of course the means that enables this.
KQED has an excellent article on introducing children to Scratch, a programming tool usable by children and available free from MIT. There are similar tools (ALICE from Carnegie Mellon for instance) that make programming approachable and rewarding for most children. In an increasingly digital world made possible by the rapid evolution of computing tools, we owe it to our children to expose them to programming regardless of their individual strengths.