If it was so easy, why isn't it being done already? Your last point rings true. If the IOC were more responsible, they'd choose the more sensible and sustainable bid rather than the ones making lavish promises, but having no legacy plan. But that would require the IOC to put the needs of host cities ahead of their own wants and desires. Good luck getting that to happen.
So yes, the question is how to balance the needs of a large capacity arena for competitions and use the rest of the time? Any city could use a community swimming pool, but how do they balance that need with having a full-sized venue to host competitions? A swimming venue can be used for other things. Or they put a pool at an existing venue that already has the capacity. If it is planned in advance, it can be successfully pulled off.
As for the hotels, the issue isn't just capacity. You can't trust the athletes as if they were tourists. Not only do you need beds for them, but they need dining, transportation, security, medical services, etc. Most hotels can't provide that. And even if they could, who is covering it at all? If I'm a hotel operator, I want my rooms filled by paying customers, not giving them away free to athletes and then having to take care of them. If that's the arrangement, then why would a city want to bid for the Olympics if their tourism business is going to be marginalized by the IOC needing their resources?