I'm looking to build something similar for our in-house marketing campaigns and having a tough time figuring out how they do it.
Basically, companies like WhatRunsWhere, AdBeat, and MixRank let you do:
The first thing that comes to mind is that they crawl all the web many times a day to keep a track of these ads -- but that seems nearly impossible as each refresh loads a different advertiser.
Are there any public/paid APIs they likely use to run their services?
Any idea on this would really help us build out our own internal service.