That's a good point, bur with climbing, skateboarding and surfing in (and likely, baseball/softball) it doesn't make for many quotas to spare (an hundred or so)
EDIT:
With Boxing, MP and Weightlifting out, we are saving 444 quotas from Paris program, plus the 32 from breaking not guaranteed to be at LA. Assuming we will have Baseball/Softball with the same format as in Tokyo, it left us with 242 quotas for any other additional sports and/or any of the three sports left out to make a come back. That's really not a lot...
EDIT2:
With these numbers, it hard to see how boxing could feature in LA.