I'd say you've got a case for refusing them because they were described as Kiss Off hubs and they patently aren't.
You've got no comeback on the 36h thing. Don't mention it at all otherwise the seller can claim you're just rejecting them because of that.
Just explain to the seller you came in late to the auction, saw them described as Kiss Off and didn't have time to do any investigation on Google. Frankly I don't think he'd have a leg to stand on if it came down to a dispute. Ebay won't look at the picture and go "that's obviously not a xxxxx", they'll look at the description. That pic may have been the 10th of 15 pics and you never noticed it.
You won't get negative feedback as sellers can't give that these days, though they can (and should) report non-payment as this is the only way they can get their eBay selling fees back (which could be a fair amount to him/her). You'll get a black mark against you (not literally) and some sellers can set up that they won't accept bids from non-payers.
Whether it's morally wrong is up to you.