I gotcha. I can agree with that. I also think he's the #2 Warlord. But for haki users being immune to his manipulation, I think he's proven otherwise. Maybe the degree he can control differs dependkng on the strength or haki of the intended victim, but In Marineford he managed to control Diamond Jozu who's physical strength is among the top and he has haki. He also recently stopped Sanji dead in his tracks and we know he also has haki. I think even at his introduction he was manipulating Vice Admirals to make them fight and we know that all Vice Admirals and Admirals have haki.