When it comes to picking stocks, why is it hard to beat the indexes like S&P 500 (USA) or Nifty 50 (India)?

(a thread)
2/ Like anything about the markets, there are no definitive answers as it's an emergent phenomena but I have a few guesses on why.

First clue comes from this long range study that suggests that MOST of the index returns come from a minority of stocks.
3/ Just 4 percent of all US stocks are the reason behind long term equity out-performance over risk-free deposits (FDs).

See this study: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2900447
5/ So remember that stocks follow a power-law in terms of their returns: majority of gains come from minority of stocks.

How does this impact building portfolios by picking stocks?
6/ Most people give equal weights to stocks that they pick e.g. if they invest in 20 stocks, each gets 4% weightage.

In comparison, indexes usually weight companies by their (free floating) market capitalization.
8/ So the trick is to give higher weightage to best performing stocks in your portfolio.

Which is what an index does automatically!
9/ The stocks listed in an index get weights depending on their market capitalization AND market capitalization is a function of historical business performance (EARNINGS) and expectations of business performance (PRICE people are willing to pay for those EARNINGS)
10/ As either earnings or expectations grow, an index automatically gives a higher weight to that particular stock.

So in a way, index investing is sort of like betting the winners and cutting out the losers (which is a classic investing maxim).
11/ Another reason index outperform is because usually the companies that have demonstrated growth for a long time grow big enough to become a part of those indexes.

A company big enough to be part of the index usually has GOOD FUNDAMENTALS behind it, which take time to change
12/ This is in contrast to stock picking where you have to guess and study management, market prospects, moats, etc.

A random company included index is likely to have better prospects than a random company on the market.
13/ So by investing in the index, you're effectively outsourcing research to other investors who take bets on non-indexed companies. Most of them turn out to be lemons, the few ones that succeed enter the index

And once they enter the index, you automatically buy good businesses
14/ So imagine indexes as a filter to provide you with companies that have managed to grow from 0 to really big and you're backing only those companies.

Rather than trying to guess future winners by stock picking, you're betting on past winners (which is less risky).
15/ This indexes are less risky than active stock picking is relevant bit because MOST of us do not want excessive risk with our savings.
16/ This is what causes pension schemes and even individual investors prefer large, established stocks to allocate majority of their assets marked for equity

So the money keeps flowing into equity markets and most of it goes to large cap stocks (which make up the index)
17/ Especially after low interest rates across the world (including India), net new money in equity mutual funds isn't stopping and more money is chasing the same 50 stocks in Nifty 50

https://tradingqna.com/t/interesting-statistics-about-the-indian-mutual-fund-industry/46185
18/ So imagine a situation where people aren't getting enough interest in FDs but don't want excessive risk. Where do they invest?

You guessed it right.

Large cap, blue chip stocks.
19/ So money is literally chasing few stocks.

This means when you trade a stock that makes up an index, the BUY decision is already made because of new cash coming into the markets.
20/ But when you trade a stock not in index, you're trading against a sophisticated investor who might be selling it to you because s/he knows something that you don't.

In contrast, seller for a stock in index fund might be selling because of liquidity or rebalancing needs.
22/ That's it. Hope you liked it.

Let me know if you disagree anywhere or have comments.

In case you want to brush up fundamentals of investing, check out my other thread: https://twitter.com/paraschopra/status/1112619960507887616
23/ Indexing is really a type of insurance.

Instead of investing in particular companies with their own idiosyncratic risks, you pool money with other investors and invest in a universe of companies.

It's just like life insurance: probability of death is low but fatal.
You can follow @paraschopra.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled: