Robert Downey Jr. (RDJ) was the only person on The Avengers cast (excluding perhaps Samuel L. Jackson) who can consistently put butts in seats. None of the other Avengers are A-list stars and theirside projects (excluding Chris Evans turns as Captain America, which is technically an ancillary project of the Avengers franchise and Jeremy Renner's role in the most recent Jason Bourne sequel) don't earn enough money for them to be able to negotiate an RDJ-sized share of the pie.
Robert Downey Jr. is the star of The Avengers. Period. Without him (and the next sequel may discover this to its detriment) the franchise really doesn't hold its own and the grosses begin to drop. As such, the producers are faced with the very real possibility that if they didn't pay him what he wants, he could easily walk away and they'd potentially lose HUNDREDS of millions (Yes...Hundreds) of dollars.
If or when Jeremy Renner and Scarlet Johansson (the only two in the Avengers pantheon who can do it) earn RDJ levels of return on their side projects, then they can cash Downey Jr.-sized paychecks. Until then, they are talented character actors who occasionally get minor leading roles.
Answered by Jon Mixon via Quora