“We do have to have some austerity,” Smith told podcaster