Most of Siberia was only colonized in the 17th and 18th centuries. Even then, it was (and to some degree, still is) a peripheral region of Russia - most of the population (80%) is in European Russia.

Siberia also isn’t where Napoleon or Hitler were stopped at.