I would not be thrilled with that, no
I mean, in terms of atmosphere, there is absolutely not going to be any better place in the world for speed skating - from national championships to Olympics - than Thialf. Hamar once rivalled it, but not anymore. That being said: I'd be sad to have Olympics in one country, and one sport pretty far away in another country.
Last week a Dutch former speed skater (Olympic champion Mark Tuitert) was talking about it during the World Cup broadcast, and he said he'd have been sad to have that in his career: according to him, a majority of athletes of course wants to go to the Olympics, but with part of that being immersed in the entire Olympic feeling, from a nation's culture to so many athletes from different sports being in the same area, which normally just doesn't happen. Olympics in France or Switzerland or so, with speed skating in Thialf, would mean the speed skating is basically just another championship instead of something special like the Olympics.