I think you lose a lot of neuroplasticity once you become an adult though, which you would need to reroute the neural pathways. Although I guess that there have been cases of adults who lost half their brain matter in accidents and were able to develop normal cognitive functions again. Actually, even babies must already have the visual cortex all connected up if they can see so maybe some things just develop too soon.