According to his tweet he wore it to a party in 2012. And the tweet is still there, so the part about removing it from the internet is just a meme.

Edit: https://twitter.com/elonmusk/status/214636339474345984