Them not controlling every single island there is doesnt change the fact that the world at large is under their command, and Spandan wasnt exaggerating, no one,  not even Luffy, treated his speech that way
	
	
		
		
	
	
Robin stated that her enemy was the world i.e the WG
	
	
		
		
	
	
The empty throne is supposed to represent how the is no single king of the world in the WG
	
	
		
		
	
	
And even Whitebeard himself saw the WG as the ones that rule the world
	
	
		
		
	
	
Its not open to interpretation dude. Challenging the world=Challenging the WG, and that will be done by whoever finds the one piece and BB isnt that guy, he will fall during the race for the one piece