The core idea of the strategy evaluation method in non-experimental scenarios is to artificially create a virtual control group to compare with the strategy online data to estimate the real effect of the strategy .
Effects regression is essentially a causal inference problem in statistics. In statistical science, the essence of the problem to be solved by causal inference is to strip away the influence of external variables that we do not care about on the results , so as to mobile number list accurately estimate the single impact of the strategic factors we care about on the results. In scenarios where AB experiments cannot be performed, there are usually two ways to accomplish this:
Constructing Similar Groups (Matching) : This idea assumes that there are some samples that are not affected by the experimental strategy and that there is homogeneity in the samples that are affected by the experimental strategy. As long as we figure out a way to find these similar samples as a dummy control group, we can control for exogenous factors. The most classic method of this idea is PSM (propensity score matching method);
Constructing virtual reality (Synthetic Control): This kind of thinking believes that the impact of the strategy is actually the difference between the performance of the indicators after the strategy is applied and the performance of the indicators in parallel time and space when the strategy is not applied. Therefore, as long as the index level of the virtual space-time that the hypothetical strategy is not covered by the modeling method is constructed, the benefits of the experimental strategy can be evaluated. Typical methods include synthetic control method, Causal Impact, etc.;