Metro to Las Arenas, have a look at the Puente Colgante (hanging bridge), cross over to Portugaltete. If they dont have a fiesta on when you are there you must be the unluckiest person in the world, non-stop during the summer.
Go up the Funicular to the top of Artxada, great views and some nice places to eat.
As Bob said, the coast is great. Bilbao isnt all that touristy really. Close by places worth visiting – Gernika (for the Basque parliament). San Sebastian, cos it’s nice. Bakio, nice coastal town, not far from San Juan Gaztelegatxe, you could combine with trip to Bermeo. Some walking/cunning bus timetabling needed here though.