The best way to filter out responses caused by weird keyword matches is, I think, to work hard at the wording of the ads so that they are as unambiguous as possible, and then to only count clickthoughs – not page views.

As you say, there are always going to be a lot of people who click pretty much randomly despite that, but perhaps the trick is then to use some other way of filtering them – perhaps the ad takes them to a “sign up for the beta” page, and you only count the people who actually do sign up?

Essentially you need to make certain assumptions about the data provider here, and you are then interpreting the results based on those assumptions.

For example, perhaps you’re assuming that the 300,000 views are suitably matched by Google? You’re giving them keywords that help Google determine if you’re of interest to the search, but you’re assuming that the correlation Google creates is relevant. In other words, it’s something of an unknown. For example, I know that the ads I see both in my e-mail and in my google results often are very very tenuously related to the trigger text. You frequently have to think for some time to even work out the correlation, and even then it might be very tenuous.

Of course, the 350 people that clicked on the results probably narrow down the margin for error here, in that they’re probably looking for something Resolverish. But they could be bored, drunk, nutjobs, or unable to understand what they’re reading.

Again, that’s unlikely in the case of Resolver, but nonetheless worth bearing in mind.

I guess that could all be entirely boiled down to: Identify the unknowns here, which in itself would be an interesting exercise and probably quite valuable.

Why do you think that the ad text makes more difference than the keywords?

Sorry, I don’t quite understand where you’re getting that from. They both vary together – that is, if you enter keywords that match a given feature then you get the ad that describes just that feature.

I imagine that one set of keywords might give better results than another thus making the user less likely to even look at the adds in the first place.

You may be onto something there – but the click-through ratio was pretty constant. Some ads just got more clicks because people searched for the relevant keywords more frequently. Certainly something to look out for, though.