It won't take that long.
Whitebeard won't take the detainment of Ace lying down. I bet he is en route to Impel Down at this point.
Shanks is probably with him, as he warned Whitebeard about this. His secondary goal might be to go after Blackbeard, who after becoming a Shichibukai should be going after Whitebeard.
Currently, Kuma, Garp, and Sengoku are at the Navy Headquarters near Impel Down, and Aokiji has to be around there somewhere as well.
Since Impel Down is located underwater, there is a very high possibility that it is located near Merman Island. In that case, we might as well throw in the Straw Hats and Jimbei into the mix, as there is no way the plot would allow them to do otherwise.
It's also great time for Buggy to come back out and confront Shanks, for some long needed backstory on Gol D. Roger. Smoker also promised to meet the Strawhats in the New World, so why not throw him in there too.
Add Croc, Mr. 1, Mr. 2, and Mr. 3 who are all imprisoned in Impel Down, several more Navy Admirals, and a lot of Mermen, and you have one massive epic clusterfuck of awesome.