Well, you have already been informed about Iceland. Iceland really lacks any real city, even Reykjavik is basically a large town. Reykjavik became the seat of government after Iceland declared independence because, well, the government was already there.
Denmark had Roskilde and before the official unification in the 900's by Sveyn Forkbeard, the seat of the king was mainly located in Jelling, a small and insignificant place (which is why it was moved to the more central Roskilde). In the mid-1400's Copenhagen became the capital.
Sweden originally had many different temporary seats of government due to the tumultous situation there in the Middle Ages. The most stable one was probably the royal castle at Visingsö. After they exited the Kalmar Union Stockholm was made the capital in 1523.
Norway had the town of Trondheim as capital from roughly 997 to 1217. In 1217 it was moved to the city of Bergen, before it was again moved to the small trading town of Oslo in 1299, where it has remained since (unless you count Copenhagen, the capital of the union, as capital of Norway during the union with Denmark). Oslo must have seemed a strange choice to many, being a tiny trade town of only a few thousand inhabitants. But the king had problems with the loyalty of the East Coast leaders, and wanted to move to more central location where he could keep them in check personally.