You could argue that it’s because it has a milky texture, therefore still correct to call it almond milk despite not matching the traditional definition if milk.

I don’t see people being irrationally bothered about peanut butter using the word butter, despite not being dairy-based.