How do I record exactly what changed in a refresh so I can prove it worked?
Short answer
Record five things at the moment you publish: the before-state as frozen figures, the specific changes made, what you expected to happen, the date you will check, and who approved it. The before-state and the expectation are the two that people skip, and without them a refresh can never be judged, only argued about.
Freeze the before-state as numbers
Write the actual figures into the record rather than linking to a dashboard.
Capture the trailing quarter for clicks, impressions, average position on the target query, citations in AI answers, and sales conversations attributed to the page. Then stop them from moving by writing them down.
A link to a report is not a baseline, because the report will show a different window by the time anyone checks. Half the disputes about whether a refresh worked come down to two people looking at different date ranges.
Analyze AI supplies these from one place: GSC nodes for clicks, impressions and position, Citation Pages for the AI side, and HubSpot for outcomes.
Log the changes as specifics
"Updated and improved" tells you nothing three months later. List what actually changed.
| Change type | Record it as |
|---|---|
| Facts | Which figure, old value, new value, source |
| Structure | Sections added, removed, reordered |
| Format | New table, calculator, screenshots |
| Intent | Target query changed from X to Y |
| Links | Internal links added or repointed |
| Meta | Title and description before and after |
The reason this matters is attribution. If a page recovers and you changed six things, a specific list lets you form a view about which one did it and try that first on the next page. A vague note leaves you with a page that worked and no transferable lesson.
Keeping the original and the optimized version together is what makes a change log a diff rather than a description.
Write down what you expected
This is the field almost nobody keeps and it is the one that makes the record useful.
State the expectation before publishing: "we expect position to return from 8 to 4 within eight weeks, and click-through at held position to recover from 1.9% to 3%."
Two things follow. You can be wrong in a way you learn from, which is the entire value of keeping records. And it stops the retrospective redefinition of success, where a refresh that was supposed to recover rankings gets reported as a win because time on page went up.
Expectations should be specific enough to fail.
Note what you deliberately did not change
The counterpart to the change list, and it protects against the most expensive refresh failure.
If the page was earning citations or links from a particular passage, record that you preserved it. If the page recovers, you know the preserved element was not the problem. If it declines further, you know the removal was not the cause.
This matters because our own state of AI search research found only a weak link between a site's strength and how widely its pages were cited, across the 4,824 cached pages examined. Citations follow the wording, not the domain. A refresh that deletes the cited passage can lose citations while the ranking looks fine for months, and without the record you will never connect the two. Google's AI features documentation confirms eligibility follows ordinary indexing, so nothing flags a page that quietly stopped being useful as a source.
Drafting against the existing page rather than replacing it is what makes the do-not-change list enforceable.
Keep the records in one place, appended
Individual records are useful. The series is where the value is.
Keep them in one running file with one row per refresh, so after twenty refreshes you can ask which change types actually correlated with recovery on your site. That is a far better guide to what to do next than anyone's general advice, including ours.
Start (webhook, fired when a page is republished in your CMS) → Web Page Scrape capturing the new version → GSC Page-Keyword Breakdown for the frozen trailing quarter → Citation Pages for the AI baseline → HubSpot Search Deals for attributed conversations → Prompt LLM diffing the previous captured version against the new one and classifying each change by type → Code node writing the row, including the expectation and check date from the brief → Export Excel appended to the running log.
Firing on republish rather than on a schedule is what makes the record complete. A monthly sweep misses the pages that were edited and reverted, and those are often the interesting ones.
FAQ
Related answers
- What should a content-refresh brief contain?
- How long after a refresh should I measure results?
- How do I prove maintenance created more value than new content?
- How do I maintain hundreds of pages without giving AI full publishing access?
Make every refresh provable
Analyze AI captures the before-state, diffs the published version, and appends the record automatically when a page goes live.
Start your free trial