This is a good question, and since I've studied a lot about the Olympics in undergrad (my advisor/close friend is an Olympic scholar), I actually feel like I have some small expertise to answer. https://twitter.com/Ishikawa_Sachi/status/1391907955931422720

Many countries' Olympic regimes can be broadly categorized within the 2x2 matrix "More Free vs. Less Free" and "Publicly Sponsored vs. Independently Sponsored."

We can think of these as categories A, B, C, and D.

(Note: the letters/colors don't imply normative value)

This is somewhat of a false dichotomy. For many, preparation for the games is a full-time job. And aside from self-funders, athletes in many countries receive a combination of government and private sponsorship. Few (though US is an example) are all-or-nothing.

Nevertheless…

Athletes in A or B societies may live in housing provided by their governments (along with their families). Their living conditions may be better than they would otherwise, and they may lose these benefits for failure to compete. https://apnews.com/article/0eba52a75e334dd29dc8d4339d641300

In the case of athletes from B or D societies, their sanction for failure to compete may be far worse than economic. And their governments likely have de facto leverage, through social pressure or other means, over more than just their individual person.

Athletes from C societies (a prominent example being the US) may have some legally protected right to protest, but are nonetheless beholden to their private sponsors (many of which are themselves major sponsors of the Games). https://www.teamusa.org/us-olympic-and-paralympic-foundation/team-usa-fund

This was a very broad explanation, and people from other countries (I'm from the US, living in Japan) will likely have a much more detailed understanding of the pressures (the industries in) their societies place on athletes.

However as many Olympic events are considered "prestige" sports, rather than massively popular ones like soccer, the vast majority of Olympic athletes are likewise not independently wealthy.

Many may not have a source of income outside their participation in the Games.

To say nothing of the information available to athletes, both with regard to availability in their own languages, as well as the ideological slant of its source.

Many athletes are likely completely unaware of the conditions on the ground in Japan.

It's incumbent on the rest of us to raise awareness of the current situation as broadly, and in as many languages, as possible.

We may see more athletes voice their concern as we ramp up to the Games, but we have to provide them ample support to do so. https://www.abc.net.au/news/2021-05-11/japan-tennis-kei-nishikori-naomi-osaka-tokyo-olympics-covid/100130120

I’ll try to attach some more articles about the relationship between Olympic athleticism and poverty below:

USA Today—Survey finds Olympic, elite athletes struggling financially (2020)

https://www.usatoday.com/story/sports/olympics/2020/02/24/survey-finds-olympic-elite-athletes-struggling-financially/111365842/

Calgary Journal—Elite athletes struggle for financial support (2016)
https://calgaryjournal.ca/2016/07/06/elite-athletes-struggle-for-financial-support/

CNN—Olympic athletes face financial hardship (2012)
https://money.cnn.com/2012/07/10/news/economy/olympic-athletes-financial/index.htm

The Washington Post—Olympic executives cash in on a ‘Movement’ that keeps athletes poor (2016)
https://www.washingtonpost.com/sports/olympics/olympic-executives-cash-in-on-a-movement-that-keeps-athletes-poor/2016/07/30/ed18c206-5346-11e6-88eb-7dda4e2f2aec_story.html

And lastly, related to the Olympic Movement, we have Olympic Charter Rule 40:

LawInSport—Navigating Olympic advertising: Rule 40 (2016)
https://www.lawinsport.com/topics/features/item/navigating-olympic-advertising-rule-40-a-global-perspective#:~:text=%E2%80%9CRule%2040%E2%80%9D%20is%20the%20somewhat,the%20permission%20of%20the%20IOC.