Absolutely 100% agreed. They might assume you’re stoned when you’re just sleepy, make you take drug tests, maybe not trust you to operate equipment in case you might be on weed, etc. Landlords may suspect you of smoking in non-smoking areas of the building, if it’s a non-smoking building (sidenote, please don’t do that if you do that), which they care about because of the smell sinking into the building or other tenant complaints, though if you smoke enough for there to be a lingering smell they might find out anyway when they check fire alarms or whatever. It’s also still federally illegal, so doubly don’t confess to it if you work for the government or the law. It could also be an issue if you ever need to dispute termination or worker’s comp with an employer, if they might accuse you of being high when you were doing xyz thing that got you hurt at work, or say they fired you because you were high at work, or suchlike.
Every situation is different but I’d recommend keeping it mum unless you really trust your employer/landlord and/or don’t care about the potential consequences if they react badly like this. At least for regular or frequent use. Admitting to trying it once or twice might be fine, though for some people even that might be enough for them to start thinking of you as someone who “does drugs”.
Landlord and boss are usually not your friends. No need to tell them.
I’m not really an expert on the subject, but yeah, probably better to fly under the radar.