You lost a ton of cossacks for free. While you were busy raiding (losing a lot of cossacks), the rattan shields killed 5-6 cossacks for free and you didn't even notice. And your macro was pretty off: you basically didn't herd at all. I think with 10 more cossacks you would have been fine that last fight, so those 10 cossacks you lost for free (maybe more/less than 10, I didn't actually count) probably cost you the game.
Furthermore, he pushed you just at the right time. Your next shipment would have been Boyars, and if he knew that and decided to push at that precise time, it was a very good read by him.
Disclaimer: I am shit at this game, so take everything I say with a grain of salt, a slice of lime, and a shot of tequila.