Sorry, anyone who says baseball/softball is not popular around the world really means it's not popular in Europe. It's extremely popular in the Americas and East Asia (Japan, Korea and China). I think that probably makes it more popular than several team sports already included, who only have European medal threats, along with the USA. Softball just had their world championships in Canada and 31 countries participated, including ones from Kenya and Uganda.
I prefer the idea of rotating sports depending on the host city. Baseball/softball makes sense in Japan, but it probably wouldn't have in London. Baseball fields are expensive so I wouldn't expect every host city to build them. It's easier to build venues, or just re-use them, for the smaller events.