How can the act of merely visiting a Temple and looking at an idol can be haram in any way if you aren’t worshipping said idol?
From what I know about Islam, there shouldn’t be anything wrong with attending a non-Muslim festival as long as you refrain from taking part in any religious ritual that takes place there.
Its the same as Christians who believe that reading or watching Harry Pitter is worshipping the occult, and ban their congregation from doing so. I.e. stupid