Try using BitTorrent on a Linux machine.

I use KTorrent on my Ubuntu box and have not had any problems.
I think the main problem is that you're opening ports on your computer and there's no way around that. But having an OS with less things exposed may help a little bit.
You can run a VMware Server (it's free from vmware.com) on your Windows machine and create a virtual computer with Linux or Windows from which you could run your BitTorrent client. That way, if things get screwed up, you can just erase the virtual machine without harming your main computer.