It’s either a smart phone, or not a smart phone.
You just need to find a phone that fits your budget & not use the other capabilities it has to offer.
Regarding the ‘small device’ requirement. How small? Most phones nowadays are pretty large, but the only way you can get around that is to buy an smaller old phone second hand like a Samsung Galaxy S5 Mini, but I wouldn’t really recommend that as it will be running an old version of Android that at some point won’t be supported by Strava.
I know this because I was using my old Sony Xperia T for strava while running & all of a sudden it stopped working because strava updated the app. I then nicked my Wife’s old Galaxy S5 Mini which worked well, but it’ll only be a matter of time before it doesn’t work.
Maybe you’d be better off with a budget GPS device and a non-smart phone?