He has fighted and won against so much more powerful than him : some who got blinding speed, some who got power. Guts is able to saw his own arm to survive or to kill someone, or to being a very callous and cold bastard, so...
Considering he has faced worst adversaries like an endless army of Apostles in the realm of the God Hand and lived to tell about it, I have to give this match to Guts. While the guy's human, he's able to take inhuman amounts of punishment and donning the Berserker Armor would just make things much more problematic for the Fuhrer, especially if Guts goes Hellhound on the guy. Guts didn't get the names of Black Swordsman and 100-Man Slayer for nothing.
That would be Rasenshuriken that does that. Guts was at least sonic to super-sonic by vol2. He got considerably faster later with all the magical armor upgrades. Even I'm only trying to catch up. On vol6 (The Golden Age, though I have the whole Golden Age on DVD with the Berserk anime).
I can't give scans atm since Mangafox has none and onemanga had to remove the whole Berserk series from their site at the request of the company. besides, as stated, he was slaughtering an endless horde of Demons within the Godhand's own realm. Hell he was fighting them to get out even before he had all the magical upgrades and such. I'm snaggin vol 7-8 tommorow, but that doesn't mean much since there's like 30 volumes.