I've seen it work fine.
Might be more likely on a steel frame with skinny stays, and you can always give them a tap with a hammer for more room!
If you can then go for the bigger chainset, as you'll have better jumps between the rear gears then.. If you're on the 11-12-13 sprockets all the time, it'll wear out faster.. Go bigger and you'll be able to pick a nice range cassette to suit.
EDIT.. Sorry, ignore that bit about gears.. You said singlespeed! It's always worth running bigger rings/sprockets though for a lower wear factor.