As mentioned by foggy, jazz harmony (which I frankly suck at) or counterpoint are both the things which will give a formal understanding of this sort of thing.

That said I picked up a lot of it more from playing regularly with people who are much better than me at music. In the end if you immerse yourself in music that uses these ideas more regularly you start encountering strange chord notations and seeing patterns in why they are as they are. Finally it isn’t really a prescriptive thing, there will always be many ways to write the same chord, and it will usually be much of a muchness what is written vs what you actually play.

In the case above I’d probably always write it as a D because for someone trying to learn it quickly they’ll know what a D is more instinctively than a weird augmented minor.