Food - breast milk exclusively until 6 months, then they can join in with family meals, and don't eat a lot before one anyway so shouldn't have a great impact on your pocket
Nappies - get reusable! I've spent less than £100 on nappies, wipes, nappy bin and wet bags. It's just an extra couple of washes a week.
Sleep - my youngest baby has shared my bed from newborn. I didn't plan it this way but with a feisty toddler to entertain in the day it was either co sleep or no sleep! I imagine he will be in with us until at least one so no need for a cot yet, even though he has his big brother's waiting for him.
Transport - car seat wise, I feel this is a necessary expense. Get the best you can afford and never buy second hand. But there's no need for a fancy pram/pushchair/travel system. Get a sling! Start with a stretchy wrap for your teeny newborn, then move on to a woven, ring sling or SSC as baby grows. My youngest has barely looked at the pushchair! Though, slings can be an expensive hobby so it's debatable whether it will save you money. But you can get more slings to the pound than pushchairs, and they're prettier!
Bathing - no need for plastic baby baths, babies don't need a daily bath anyway, and when they do it's much lovelier to just climb in with them, easier on the back too! Do invest in a snuggly soft towel though, I can recommend Cuddledry.
Clothing and toys - yes babies need clothes, but avoid tiny outfits. Much comfier for baby to live in a onesie in those early months, and they grow so quick. Buy a few simple vests and onesies, everyone will buy outfits and toys as gifts anyway, and most babies prefer a good old box and a wooden spoon to expensive toys!
I wish I'd thought more about this first time round, but the excitement just catches up with you and you want everything! And the benefit of hindsight is a great thing. Have I missed anything?