Never mind all the people who say 'oh your life is over you'll be dead on your feet forever'. It's not necessarily like that (as long as you don't end up seriously anaemic like my Mrs).
Don't admit defeat. Your life is to be made to the satisfaction of all involved. This means NOT giving up what you love. You need to carefully plan exactly how you are going to do things and make damn sure you're not doing any more work than is absolutely necessary. This means NOT, I repeat N.O.T ironing baby clothes – amongst other things. Yes, people do do this – talk about making work for yourself!
You could get a Chariot trailer with the baby sling and suspension, but that limits you to fire-roads so won't be that much fun. My suggestion would be to either ride near your house and get a babysitter in, or ride near the house of someone who can look after your baby 🙂