You said this "get workhorses who know how to bang, take contact and finish at the rim." While Kleiza certainly isn't a stellar defender, he is a workhorse who knows how to bang and take contact. If it's his finishing at the rim you have a problem with, why not include Reggie Evans? The guy is currently shooting 24%, and pretty much all from under the basket. And Andersen? He's shown to be good at banging down low. A lot better than I expected.
As for the fan base, I couldn't care less what they wanted. If the fan base had any say in how things went then the team would change the starting lineup every other game, make trades once a week and have a new coach once a month. They're like a bunch of bi-polar adolescents with attention deficit disorder. Making decisions based on their whims is probably not the best way to go.
Building through the draft is a gamble, but you never, ever win Championships by playing it safe. Ever. That's the route to mediocrity, which I want absolutely no part of. Currently, there are 3 players which look like possible franchise players projected to be in the draft next season: Barnes, Jones and Irving. That's pretty good odds, to me. Any one of them will probably immediately become the best player on the team.
And I'm certainly not against trading some players, but definitely not for veterans who might give the Raptors a couple of more wins. The Raptors have a number of pieces that they can use to better themselves. There shouldn't be any hurry. That's the smart thing to do.