As a follow-up to our status update last night, we wanted to send an additional update talking about next steps and the new reality of SERP data after this latest round of Google actions.

We resolved / closed the previous incident because that had been open for more than a month and was initially scoped around the total blockage that Google had initiated.

Since the last week of September, however, we have been providing daily data for all accounts, to varying degrees of success depending on market (Japan was the most difficult).

The main difference was that we have been providing it for SERP page 1 in most cases except for Desktop (outside of Japan), where we have been continuing to provide 5 pages.

The technical difficulties are largely dealt with, although there are some ongoing situations we continue to monitor, such as the spurious / poisoned SERP data, empty AIOs for about 30% of the cases, and so forth. We will continue to work on these.

The main factor now that has changed, for the whole industry is cost relative to overall throughput. It's not about cost only and not about throughput only, but the two factors combined. This means that fetching teams are focused on optimizing for the most throughput for the most accounts at the best possible cost, in order to continue offering this data without impacting customer costs.

We will be reaching out to customers to explain things in more detail but the main effect of all of this is that we are planning to reduce the number of pages in the SERPs we fetch, in order to keep overall fetching throughput and success rates high.

We have spoken with a number of other leaders in the space and we believe that this is both the direction that the industry is heading and also where the most actionable data lives. The 5 page depth was more or less an artifact of the previous 10 page depth that everybody got for free.

This also means that our Top 20 data, both via API and in Search Intelligence (BigQuery) will be limited to the first page, so it's more like Top 10 data until this is resolved.

Further details will be provided in calls with clients directly, but please feel free to reach out with any questions.

The primary purpose of this incident report is to make it clear that we are still fetching page 1 only for most sites and accounts but are taking steps to go to 2 pages as quickly as possible.

We also noted this in our previous update but felt that it warranted an incident report of its own.