Smart meters do not connect to your home wifi.
The following is from Smart Energy GB
Inside your home, smart meters use their own secure, wireless network using radio waves, just like mobile phones or TVs do. (It won’t use your wifi and you don’t need wifi in your home for it to work).
Your smart meters will link to a similar wireless network outside your home. This network is run by the new Data and Communications Company, which is overseen by the energy regulator Ofgem.
There are strict new regulations and codes of practice to keep smart meter data private and secure.
I really can’t see the objection to knowing how and when you are using energy from either the consumer of providers point of view.