Some things that are very sensitive won’t work because as above the voltage is lower.
In things that take a reasonable amount of power the voltage of alkalines will drop to a similar level to that of rechargeables when the power is on (because of the high internal resistance, because physics) so they end up delivering more or less similar voltage.
However with low currents, the voltage difference is greater. So if something is designed specifically with 1.5V requirement it wouldn’t necessarily work with 1.2V. Battery powered clocks with a single cell don’t work on rechargeables, afaik, not sure about remotes either.
If it’s just a light though, well, it might not work – but it might. It’s definitely safe though and definitely won’t damage it.