Usually you pay taxes in the country in which you earn them. If you work and live in Germany you pay taxes here, even when you deal with an US customer. There might be a difference on VAT. US customers do not need to pay VAT in Germany, they seem to either pay VAT themselves in the US or the service provider needs to pay the VAT. I am unsure of that, because all of my customers are from my country.
Anyway I would recommend you to ask a tax consultant for the exact regulations. As a business man you should have one - I really recommend that to you.
This is my second, and probably-final version of what I've learned.
I called the UK tax department, and was told that they use the location of the individual (not the server, or customer) to decide whether to tax. So given what I've written above, I wouldn't be taxed for earnings from UK customers.
For the US, I've had no answer from the IRS help desk, but I have read parts of several papers, and the US Code.
First, US has Withholding tax of 30% on all profits sent outside the country. (See the US Code) I expect the practicalities of this is very important, because the emphasis is on the US resident to apply the tax.
Then there's the state of the US tax system. From what I've read of 'Taxing International Income: Inadequate Principles, Outdated Concepts, and Unsatisfactory Policies' (NYU paper), I think the US is quite happy with the performance of relatively old taxing policies (pre-Internet), and considering the necessity of deep justifications for any change, and perceived general unimportance to politicians, I expect the US won't change any time soon to tax Internet companies more aggressively.
While reading, I heard anecdotes that most companies can't avoid taxes by relocating because they tend to have interest in having physical presence in the US.
And from one lawyer I spoke to (informally), physical business/property in the US seems to be essential to the US claiming tax, even though I think I've read broader definitions of what can be taxed on the IRS website.
So actually it's not as clear-cut as I first imagined it might be, but that seems to be historically caused.
I then read the non-speculative part of the OECD's report 'Are the current treaty rules for taxing business profits appropriate for e-commerce?'. (The OECD provides model treaties for member countries, including Japan, France, and USA.)
The report explained various things, including the idea of 'separate accounting', which is quite interesting relating generally to international taxation.
Importantly for my question, they describe the practical aspects of the law, and define 'permanent establishment', and specifically mention their suggestions for specific cases.
I won't try to summarise the report, but if you're looking for the answer to the question I asked, and even want to know more, I suggest you read that OECD paper.