I’d think that if banana skins were a part of the natural environment of the place, then how long they would take to rot would be irrelevant, as the location is already optimised for dealing with them.
My POV on taking them home is that pure wild environments are precious and rare, it’s good to leave them be as much as reasonably possible. A tiny bit of contamination is still contamination.
I was reading about Scottish Wildcats recently, there are less than 400 pure-breeds left. The biggest threat is not environment but interbreeding with ferals/domestics. Every time that happens all descendants of that litter, even if raised wild by a Wildcat mother, are not Wildcats and that bloodline is lost forever and they compete with the pure-breeds too.