It wouldn't require too much luck at all for the Raptors to get the #2 pick and Portland to end up with #4 and something around #10-12.
Going back to last year's trade deadline, I've read several rumors that had Portland being quite high on both DeRozan and Calderon as well, so it's not entirely out of the realm of impossibility. DeRozan and Batum would be great on the wing together and Calderon would be fantastic running the P&R with Aldridge (and Amir if they wanted him for the 2nd unit). Giving up 3 good rotation players, as well as 2 2nd round picks, seems like a pretty good haul for their 2 lottery picks. Hell, Toronto could even through in a future top-3 protected 1st round pick to sweeten the deal.
Signing Nash and Mayo would be the most unlikely parts of the scenario, but money could talk for Mayo and Nash almost seems destined for a Canadian reunion, so who knows... we can dream in the wake of this post-NCAA tourney, draft pick kool-aid drinking bball fest, right?!?!