Even if the three players who have fueled the last two wins cool down and the Raptors suffer setbacks, they will not likely have one of the 5 worst records in the league. There are too many other bad teams out there.
Washington: By now the consensus pick for 30th. They have no frontcourt, and John Wall (who is still not playing that well) is their best player. They have no wins, and their losses are on AVERAGE by 13 points.
New Jersey: Kris Humphries is their third best player. Their second best player has a broken foot. They lost to Atlanta by 36. Their only win is over Washington.
Charlotte: They are playing Boris Diaw at centre. Diaw is 6'8". Miami beat them by 39 points. They are being outscored by 11 points per game.
Sacramento: They are stuck in the strong western conference, and John Salmons might be their best player. Tyreke Evans is playing terribly, and his backup is Jimmer Fredette. Their average game is a double-digit loss.
Cleveland: They are 0-2 against the Raptors. Their three wins are against Charlotte, New Jersey, and Detroit. Anderson Varejao is their best player.
Detroit: Tayshaun Prince and Rodney Stuckey are their best players. Their offense is the 26th best in the league. They are being outscored by 6 points a game, and lost to Cleveland by 16 points.
And that's only the teams that look definitely weaker than the Raptors. Golden State, Phoenix (imagine if they trade Nash!), New Orleans, Minnesota and Milwaukee all have a chance of being worse as well, if their lineups don't gel or they suffer a key injury.
It looks to me as if the Raptors will have the 21-25 worst record even if their play regresses from the last few games.