As a founder of a few apps, I always get caught up in redesigning or recoding an entire app from scratch.
Sometimes it's because I'm not satisfied with the first iteration (or have been staring at it too long). Other times it's because a newer update of a technology came out.
For example, Ruby on Rails often has a new release with new programming features every few months. And I end up updating my existing apps to use the latest release of the framework.
How do other founders avoid getting stuck in this loop?
We used to call this "creeping elegance" - where as a developer you know about, and are bothered by, bits of code that would benefit from a polish.
So first, recognise that the impulse is a good one. You stay familiar with the code, you refactor for good reasons, you deliver better features or you deliver features better. All good. The problem is that you know you should be putting your time elsewhere.
The key discipline you need to keep this in check is simple: make changes because a customer needs them.
Not fancy. Just true.
I think Jeremy Parsons is right on the money. Everything you're talking about is good. It helps in eliminating technical debt. If you've got time to deal with it, by all means, deal with it. You just don't want to do something good, when there's something better or best to work on.
If you use Scrum or some other agile methodology, you might consider picking a certain percentage (in points) of your stories that are cleanup/maintenance/upgrades that aren't visible to the user, and leave the rest to things that will immediately add value to your users. This allows you to do this kind of behind the scenes polishing continually as a part of a deliberate practice, so you don't get overwhelmed by it later, while still forcing yourself to not get buried in it.