The short answer is Stanford University. And a healthy heaping of history.
Yes, New York and Los Angeles have more people but what you also have to consider is the demographics, makeup, and skillsets of the population.
Stanford has always had a strong engineering (and eventually computer) program. A lot people attribute the start of Silicon Valley to Fredrick Terman, who was the dean of School of Engineering. He pushed new graduates to start their own businesses. He also helped set up Stanford Research Park, which leased out space to high-tech companies that moved in.
Many of these early companies were doing work in silicon chips, hence the name, but it has become much broader now to include all forms of high-techery.
Of course, once you set up the seed like this, and a lot of cool things are happening in one area, other companies are naturally going to gravitate to there. New companies will come in around this to pick up the engineers that are tired of their current job or boss, to collaborate with these companies, to just be in the company of people who are innovating like you, or just to get noticed by association. And of course add on to that the fact that some engineers feel they can outdo the giants, so they quit and move down the block to build their own company.
Additionally, where there's innovation there's success (sometimes), and where there's success there's money. Where there's money there's investors. Lots of VC firms, angel investors, and incubator programs like YCombinator form there because that's where the tech scene is, and so lots of startups relocate to there to have easy access to the investors.
In effect, it's there now because something similar to it was there yesterday. And it was there yesterday because something similar to it was there the day before. It's a magnet that keeps on attracting and growing.It does make you wonder though; Is it repeatable? If you wanted to create your own Silicon Valley counterpart elsewhere, would these same steps work again today?