If going down the PC route you’ll need a decent soundcard or USB interface to plug the midi keyboard into and act as a pre-amp for the microphone. It will also allow you to use software instruments (of which there are many free ones) without unusably long latency (ie. the time between you pressing the keyboard and a sound coming out).
Behringer kit is cheap and cheerful. £75 is gonna be tight though!
You’ll need interface, mic, cable, midi keyboard.
Edit: I see you stated USB mic and keyboard. While that would solve the connectivity, you will probably still have latency problems if you’re using a bog standard in-built PC soundcard. You might possibly get away by installing something called Asio4all which replaces the default sound driver with something more geared for real time recording.