Ad Yield Ops 105: Yield Tests Choosing Which Tests to Run

<< Previous Lesson

Next Lesson >>

Read the Transcript:

Let’s jump in and talk about how to choose tests to run.

First, knowing what makes a good test is incredibly important.

Any test you devise should include clear control and test conditions. This may take the form of a true a/b test, but it also might mean that you run the test on 100% of your traffic for a predetermined amount of time and then do a pre- and post-analysis of what changed from your standard results.

All of your tests should be time-bound. We’ll talk a little bit in the next module about how to determine the length of time needed for a test, but you should have a predetermined end time for the test at which point you return and review the results.

Your experiments should be run with a clear hypothesis, in which you are seeking to test a measurable KPI to determine whether the test was a success or not.

You’ll need to ensure that your tests are run with statistical significance in mind. You need to collect enough data to be confident that your results aren’t based on chance, but imply a true cause and effect relationship.

Now, let’s talk about how to get ideas for tests. Our recommendation is to run a recurring brainstorm session with your team.

Great yield optimization ideas start with a meeting of the minds. Typically the best way to come up with a backlog of yield tests to run is to do a brainstorming session with your team members.

Get your yield optimization professionals in a room at least once a month and have them each present ideas to be torn apart by the rest of the group. At the end of the session, you’ll walk out with a list of potential tests that survived the slaughter to work on implementing over the course of the following month.