> ## Content Index
> Fetch the complete content index at: https://cascader.io/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Before you budget from a search coefficient
- URL: https://cascader.io/blog/before-you-budget-from-a-search-coefficient/
- Published: 2026-10-01T18:00:18.000Z
- Updated: 2026-10-01T18:00:18.000Z
- Description: One retailer's model fits transaction value better as its paid-search coefficient falls. The same-case comparison shows what query controls must accomplish before a coefficient can support a budget claim.
- Author: Rick Hecker
- Tags: Cascader Research

A retailer's daily search spending and offline transaction value had a Pearson correlation of 0.91\. A regression using spending alone had an adjusted R² of 0.83\. The paid-search coefficient estimated the change in offline transaction value per unit of search spending. That estimate was also 14.7 times the point estimate from a randomized geo experiment for that retailer.

Both facts belong in the budget discussion. The model fitted the history well; that did not establish how much of the history paid search caused.

The retailer is an unnamed US business in [Chen and colleagues' 2018 study of bias correction in media mix modeling](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=14). The authors analyzed 65 daily observations from 2015\. Search was its only major advertising channel, and the outcome was **offline transaction value**. This is an unusually useful comparison because the authors change the query information in the model while keeping the advertiser and modeled outcome fixed, then compare the estimates with an experimental reference.

It is also an imperfectly reported comparison. The paper's prose and figure disagree about the richest query-adjusted estimate. That disagreement limits the exact result we can carry forward. It does not erase the reason to examine what the coefficient is following.

## What the model learns when demand rises

People who want something search for it. More relevant searches create more opportunities to serve paid ads, which can increase spending. The same demand can generate transactions through organic search and other routes.

Spending and transaction value can therefore rise together even when some of those transactions would have happened without the ads. A spend-only regression gives the coefficient work that belongs to both advertising and demand. In this retailer's data, target-favoring query volume correlated 0.91 with search spending and 0.97 with offline transaction value. Those correlations make the problem concrete; they do not establish the causal effect of either queries or ads. [The paper's original comparison](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=15) shows these relationships alongside the estimates.

The authors add query controls in two steps. First, they fit a flexible response to the **total volume of relevant category queries**. Then they retain three separate daily volumes: queries favoring the advertiser, queries favoring competitors, and queries expressing general category interest. The richer model fits a separate flexible response to each series.

These are different representations of demand. A category total can stay the same while its composition shifts toward the advertiser or its competitors. Adding the three series together loses that distinction. The richer model is designed to retain it.

**Same retailer, different query controls**The reported coefficient falls as query information increases, even as fit improves.

Case 1: unnamed US retailer; 65 daily observations in 2015; outcome: offline transaction value.

Dimensionless index: every estimate and standard error is divided by the same confidential experimental point estimate. Experimental point = 1.0\. Bars show point estimates; black whiskers show ± one standard error (SE). Dashed line marks 1.0.

**Spending alone**Point 14.7 · SE 0.83Adjusted R² 0.83

0481216 

**Spending + category total**Point 7.1 · SE 1.51Adjusted R² 0.90

0481216 

**Spending + three query series\***Prose point 3.0 · prose SE 1.02Adjusted R² 0.95

0481216 

**Randomized geo reference**Point 1.0 · SE 0.66Experimental estimate, not a fitted-model R²

0481216 

**\*Unresolved source disagreement:** page 16 reports 3.0 (SE 1.02) for the three-series model; Figure 6.2(a) plots a lower bar, below 2\. The hatched row uses the prose values. It does not reproduce that original bar or resolve which estimate is correct.

Source: [Chen et al., arXiv v1, p. 16](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=16); index and one-SE definition in [Figure 6.2, p. 15](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=15). One-SE whiskers are not 95% confidence intervals or tests of differences between methods. [Download the values and qualifications (CSV)](https://cdn.mymidnight.blog/ced556cd9f9c0c8315cfbe0744a3baf0/files/2026/09/coefficient-comparison.csv?ref=cascader.io).

The spend-only model assigns a much larger coefficient to search than the experimental reference does. Adding the category total lowers the reported point from 14.7 to 7.1; retaining query composition lowers it further in both the prose and figure, although the exact rich-control point is unresolved. Meanwhile, adjusted R² rises from 0.83 to 0.90 to 0.95\. Fit improves while the advertising coefficient shrinks. A budget defense based on fit alone cannot explain that movement.

Nor can these numbers tell you the retailer earned $14.70, $7.10 or $3.00 per advertising dollar. The underlying coefficient relates offline transaction value to search spending, but the authors hide its absolute scale by dividing every estimate and SE by the same undisclosed experimental point. The displayed values are an index. They cannot recover dollar returns, incremental profit or a sensible budget cut.

The experimental point's position at 1 is also a normalization, not certainty about the true return. Its indexed SE is 0.66\. The experiment ran in the second month of the retailer's observational period, so the two estimates concern related but nonidentical windows. The reported marginal SEs do not supply the covariance or full ratio uncertainty needed for a joint comparison. A lower point, proximity to 1, or overlapping whiskers is not a test of agreement. [The numeric account is on page 16](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=16).

## What gives the adjusted coefficient a causal meaning

The query adjustment has a specific job: account for the demand-related reasons spending and transactions move together, leaving spending variation that can identify the advertising effect.

Conceptually, the coefficient asks how offline transaction value changes with spending after the modeled query volumes and composition have been held fixed. This describes the model's intended comparison; the paper did not create matched groups of otherwise identical days. For a causal interpretation, the query measurements must account for the relevant common causes of spending and transaction value. The spending variation that remains must not still be connected to other causes of transaction value.

In [the paper's simple causal model](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=7), the demand-related routes into search spending pass through the measured queries. Controlling those queries can then separate the advertising route from the demand and organic-search routes. The model also needs spending variation beyond what the query variables perfectly predict, suitable functional assumptions and adequate measurement. Sixty-five observations and a high R² cannot establish those conditions by themselves.

The empirical result is evidence consistent with the proposed correction: changing the representation of demand materially changes the same retailer's estimate, toward the experimental reference. It does not prove that the query controls removed all bias. In particular, the paper's fitted query-response curves are adjustments for demand and other contributions; they are not estimates of the causal value of creating an extra query.

There is a practical access boundary, too. The authors used internal Google query data and classified queries using appearances of advertiser and competitor URLs in organic results. A paid-search terms report is not the same measurement. The raw series, classifier implementation and fitted model code are unavailable here. Substituting an accessible search metric would introduce a new measurement assumption; the historical comparison does not validate that substitution. [Query construction and the three-series model appear on pages 10–13](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=10).

## Change the budget policy, then follow what still moves spending

Suppose a proposed budget now binds. That changes why search spending varies. The useful question is what sets the cap and whether that decision is connected to transaction value beyond the queries already measured.

If managers raise the cap when they expect strong sales for reasons absent from the query controls, the remaining spending variation can still carry those expectations. The adjustment has not isolated advertising merely because queries were included. But a binding budget does not automatically invalidate the coefficient: [the paper explicitly gives randomized budget variation as a counterexample](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=8). Independent variation can identify an effect even when the unconstrained-budget condition fails. The budget's assignment matters.

Now suppose a shared media plan raises search and television spending together. Television may generate transactions directly as well as stimulate queries. Holding queries fixed can account for the demand reflected in searches while leaving search spending connected to television's direct contribution through the common budget decision. The query-only coefficient can absorb that remaining contribution. The paper's shared-budget graph therefore calls for a different adjustment than its simple search case.

In the paper's alternative graph, the direct connection from the budget decision to search spending is removed. That change permits a different identification result. **The presence of other media alone does not decide which graph fits.** [Figures 4.2–4.3 and Theorem 2](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=8) make the distinction explicit.

Or suppose transactions respond to ads with a material delay. Today's transaction value then reflects earlier exposure, while today's spending and queries describe a different part of the history. A contemporaneous query adjustment cannot be assumed to identify that delayed effect. The authors excluded one candidate with a strong lag effect from the benchmarked cases and list lag among the limitations of a straightforward application. An immediate daily coefficient should not become a claim about a later budget horizon without addressing that timing. See [case selection on page 13](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=13) and [limitations on page 20](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=20).

Each change requires the same positive explanation: what generated the spending variation being used, which non-advertising causes of the outcome have been accounted for, and which connection remains. A coefficient that falls after a control is added has changed. Whether it now measures an advertising effect depends on that explanation.

The policy question adds another link. Bids and budgets help cause spending; spending is an intermediate outcome of the auction. The authors themselves flag this distinction in [the appendix's expanded causal model](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=21). Even a causally identified same-period spending coefficient does not, by itself, tell you how much a proposed budget change will alter spending or later transactions. That use needs a defensible connection from the intervention to actual spending and the outcome over the decision's horizon.

The retailer comparison gives a paid-search lead a concrete response to a well-fitting model: demand can help the model reproduce transaction value while inflating the search coefficient; richer query information can change the estimate substantially; an experimental comparison provides a separate, uncertain check. To carry the coefficient into a budget decision, the analyst must explain why the spending variation left after adjustment identifies the effect under the proposed policy and time horizon. The fit statistic cannot supply that explanation.

Scope of the evidence and source locations 

This article uses *Bias Correction For Paid Search In Media Mix Modeling* by Aiyou Chen, David Chan, Mike Perry, Yuxue Jin, Yunting Sun, Yueqing Wang and Jim Koehler, Google Inc., [arXiv:1807.03292v1](https://arxiv.org/abs/1807.03292v1?ref=cascader.io), submitted July 9, 2018; the PDF title page says last update July 10, 2018\. The controlling evidence is this historical PDF edition, rather than a later HTML conversion's display date. Printed and PDF page numbers coincide.

The retailer had annual revenue in the tens of millions of dollars. The paper describes the query-classification procedure; the retailer's case-specific implementation, source query/spend/outcome data and fitted model code are unavailable here. Its identity and unindexed effect are not disclosed. Page 14 supplies the retailer context; page 16 supplies the points, SEs and fit values. Figure 6.1 on page 14 is a simulated illustration, not the retailer's raw daily data.

The paper's other two selected simple cases use different outcomes. Each case's points and standard errors are divided by its own confidential experimental point, producing a dimensionless index. These cases are not a common-dollar sample and cannot provide a correction factor for another advertiser.

**Case 2 — inquiries over 135 days.** The experiment ran in the last six weeks. The spend-only point is 8.4 (SE 1.30); category-total adjustment, 7.3 (SE 1.14); three-query-series adjustment, 1.9 (SE 0.71); randomized experimental reference, 1.0 (SE 0.14). See [page 16](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=16).

**Case 3 — site visits over 88 days.** The experiment ran in the last six weeks. The spend-only point is 2.9 (SE 0.23); category-total adjustment, 1.4 (SE 0.33); three-query-series adjustment, 0.8 (SE 0.28); randomized experimental reference, 1.0, with its precise SE unstated in the prose. The rich-control point is descriptively below the experimental point; that is not a tested difference or equivalence. See [page 16](https://arxiv.org/pdf/1807.03292v1?ref=cascader.io#page=16).

The reported marginal SEs do not supply joint tests of these comparisons. The three selected benchmarked cases are not a representative survey of media mix models.

The excluded lag-heavy candidate is distinct from the paper's later multi-channel Case 4\. Case 4 has more than a dozen media channels and no randomized reference. Its table is normalized to a 2013 model estimate, not an experiment. Different coefficients there do not establish experimentally verified improvement. See pages 13 and 18–19.

The paper describes geo experiments and cites a [time-based regression framework](https://research.google.com/pubs/pub45950.html?ref=cascader.io). That methodological reference does not supply the retailer's actual assignment records, treatment dose, cooldown, model diagnostics or covariance between estimates. The case comparison is reported evidence; those missing records limit an independent assessment of its experimental precision and comparability.

Further failure conditions include missing relevant queries or competitors, coordinated price and bid changes, substantial media synergy and abrupt changes outside the model. Organic clicks require particular care: they can be affected by paid ads through substitution, so controlling them can remove part of the effect being studied. The paper discusses these limits on pages 20–22\. Its same-period coefficient and simplified causal graphs do not, by themselves, define the marginal return to a new bid or budget policy.