Dene, correct me if I'm wrong, but your arguments seem to stem from two major points: that pot users tend to be unproductive (precisely because of pot), and that they can negatively affect those around them.
To the first point: surely you don't think that all pot users are useless to society? I'd agree that a higher percentage of them are in comparison to the average population, but what if that's the result of other factors? For instance, pot users by definition subvert the law as things stand now, because pot is illegal in the United States (is it also in New Zealand? I assume so). The pot-user demographic is therefore limited to people comfortable with breaking the law. This is largely the problem with the whole "gateway drug" argument as well. People who use pot might be likely to use other drugs, but it wasn't necessarily the pot that made them that way. People who use pot might be more likely to be useless bums, but it wasn't necessarily the pot that made them that way. Legalizing pot wouldn't, therefore, necessarily worsen the problem.
To your second point: I think that the sad reality is that irrespective of whatever restrictions governments place on drugs like pot, they'll still be used. Just look at alcohol. Yes, I gather from your post that you believe that alcohol should be illegalized as well, but as I'm sure you know this has been attempted in the past and it failed. Rather epically. Why? Because people will do whatever they need to buy alcohol, or pot, regardless of the law.
The problem is that when it's illegal, the government can't regulate it. Can't establish minimum ages, driving laws, etc. The problematic affects of pot on "third parties" is, I suspect, due in large part to the inability of government to intervene because they've illegalized it.