There are too many sports/too few days of competition/ too few venues for the actual Olympics, choose which one fits better with your vision. Fencing is not the main problem in my opinion, just think to baseball/softball recently added: 6 (SIX!!!) teams tournaments! When men's football is given 16 and it is an U23 competition, while other team sports qualify 12 teams with 3 group matches that are generally useless.
The whole Olympic Games format needs revision, with possibly another week of competition (in my opinion) and spanning at least team sports in multiple cities, in order to dilute the cost of making a huge olympic village, that usually is then considered as a weight for the host city.
In contrast to Agger, I like team events, especially mixed: they're useful for making a competition between "schools" of sport rather than individuals, and I think IOC should invest in them. In addition, that would be a way to exploit more the venues built for the Games. As I already reported in other posts, track cycling requires a velodrome that is generally used for less than a week and results in 14 medals, if I'm not wrong: according to me, this is a waste.