There might be other reasons, but here’s one good one:

It’d be really really hard to.

At 120m (the legal ceiling for “amateurs”), they’re barely even visible in the sky. At 500m (which even cheap drones can achieve), it might as well be invisible. Shooting it down would be next to impossible.

But I agree strongly with the sentiment.