A couple of things here.
First of all, we do have a system to hold our elected Representative (Presidents included) accountable. On one level we have re-elections and we have the power of recall (take them out before the term ends). With the three branches, there are checks and balances and the ability for criminal charges and/or impeachment. A quick scan of history will highlight a few, e.g. Bill Clinton, Ronald Reagan, Richard Nixon, and Andrew Johnson.
I think the case of Nixon is most pertinent here - one of the things (among many) that he was in trouble for (and most likely would have been removed from office for), was the "illegal" invasion of Cambodia.
Although the President is the Coofftopicnder-In-Chief of the military (to keep the military in civilian hands), he is limited on the duration that he can send troops into a combat area without Congressional approval (checks and balances again). Long story short, when we got involved in Vietnam, we did not want a formal declaration of "war" and were limited on what we could really do until Congress passed the "Tonkin Gulf Resolution", giving the President the ability to deploy additional forces to protect US forces in Vietnam from attack - this basically upset the balance of power and allowed us to conduct a 'war' without a formal declaration. Despite this, he still didn't have Congressional approval to extend it into Cambodia and he would have faced prison time for it (but Ford pardoned him). In 1973, Congress took back this power with the "War Powers Act" (quite a fascinating read some of you might enjoy).
Fast forward to today and we find a similar problem, only worse in my opinion. After the attacks on 2001, President Bush was able to use fear/media/nationalism/etc to encourage Congress to declare "War on Terror", giving the President the power to use force ANYWHERE in the war against terrorism (instead of declaring war on a specific country). These broad powers would seem to remove one of the checks on the power of the President in not requiring further Congressional approval to attack various places. So, in essence, the border area issue (lack of rights in that area) and the use of drones in various countries (as well as other military options) are legal under US law.
Now, as we all know, the US has a lot of influence and power in the world and the chances of some other country or power successfully being able to charge our President with an international "crime" (albeit legal under US law) really isn't going to happen. Who would enforce it? The UK, Canada, Russia, China, France? In some cases, might does make right...