Grow some and tell him no. When he asks why just tell him you don’t like him.
If you do end up doing it, make sure your speach includes the following…
“Usually the bestman speach includes a bit about how the groom is a great guy…well I don’t particularly like him so this bit was a challenge.”
Actually, maybe that could be your entire speach.