Split testing with Netlify with Phil Hawksworth

Opt-in betas with branches and split testing

You can use branch deploys and split testing to create opt-in versions of your site. Ideal for giving people access to features which are in beta, or other special items.

This video shows how to allow users to accept an invitation to opt-in to use a specific branch of your site, with just a little JavaScript to control cookies.

[00:00:00] In another video, we looked at how split testing can shape the traffic to your site so that it's shared across different branches that you've nominated. Now you can do some clever things with this as well. I wanted to show you a way that you can use this to allow people to opt-in to a private beta of your site. Let's imagine the scenario again for the Vapourium. Imagine that we're about to roll out a new section of the site that we wanted to give some people a chance to opt-in to preview that like you might do with a new feature or something along those lines. What I've done is I've created a page here called VIP. This is just an informational page. It gives the visitors some way to understand what the context is and what we're going to do. It gives them a call to action, a button to opt-in to now enter that private beta.

[00:00:50] What this button is going to do is it's going to give them the chance to nominate to use the branch that you specified. Let's go and look at behind the scenes about how this works and then we'll get this all set up. Over in our admin, let's go and look again at this split testing. At the moment, we don't have an active split test, but what we'll do is we'll choose to shape traffic between the main branch and our VIP trial branch. This is the hidden beta, it's actually that we want people a chance to get to. Now if we run this test as is, half the visitors will arrive at our standard experience and half will arrive at our exclusive VIP experience, this beta that we want people to get a chance to choose to enter.

[00:01:40] What we could do, is if we change the way that we are shaping that traffic to tell Netlify to direct all of the traffic to the main branch and none of it to the VIP branch, what we can do instead of leaving it to chance with the randomly generated cookie, what we can do is set the value of the cookie to bethe branch name itself so that people can nominate to arrive at this branch. We can do that with a little bit of Javascript. Let's save this first of all, and we'll start the test.

[00:02:10] Now, if I go back over here and I refresh this page, I'm going to go exactly where I would be expected. I'm going still to the same branch, the production branch. Nothing different here, but I have come to this VIP page that I've set up. Now this button, what it would do is, is it will set a cookie. It will set Netlify's split-testing cookie but instead of it being a random number, it will actually be set to the value of the branch we want to serve people from, and then it will reload the page.

If I click that, we're still on the same URL. We're still on the production URL. We are on the VIP page, but now, we are being served from the different branch. This is the VIP test branch. It's the same page, the VIP entryway if you like but now, we are saying, "Welcome to the new area." If you prefer to opt-out, you can do that.

[00:03:00] Again, there is a call to action. This button will clear that cookie. It will delete it and reload the page. Let's go back and forth a couple of times. There we are, we've gone back to the other main production branch. Let's opt-in one more time and see what other changes there are. Well, yes, we are serving things with the gold VIP theme. Of course, we are but then also our homepage has a different product on it now. Now, the coffee cup is $1000, a little steep but if you are a VIP, you probably want this, "Be a flamingo in a flock of pigeons," mug instead.

I'm not going to click Buy Now. I don't think. But let's go back and opt-out of this. I think I can get my mugs a little bit cheaper, although still $300. There it is. There is our split test now running in a way that allows users to nominate to enter a private beta.

[00:04:00] Let's just look quickly at just that one bit of code that really needs to exist for this to happen. Since we already have the split testing set-up to shape the traffic, now is the question of making sure that we give people a chance to specify that cookie. If we go and look at the code for this, this is the code on the VIP trial branch.

I also have the same code running on the master branch. We've added a bit of Javascript which has a few things here.

[00:04:20] The first thing we have in our Javascript is this is a utility function. This is just a button handler that I often use as a utility to help me attach, click events to different elements, and specify a callback function. We can gloss past this. This is just a utility function. There are really more interesting thing happens here. We've got a couple of button handlers that we've added. The first one adds an event handler to an element on the page which is button opt-in. That is, hop over to take a look at it. That is this button. That's this button. That's the ID of this button.

[00:05:10] When we click that button, what happens is we create an expiration date a year in advance with this bit of code and then we set a cookie. You may have not remembered from a previous video that the split testing feature of Netlify drops a cookie. We are doing this explicitly here instead so we are setting the Netlify A/B testing cookie instead of it being a random number between 0 and 1, now we are specifying the branch that we'd like to go to and set a new expiry.

Then once we've done that, we simply reload the page. Now, the traffic will be shaped by Netlify in the background to go to these branches instead of the master branch. If we skip ahead down to the other button handler that we're creating here. This is our opt-out handler. This does the opposite. This attaches to the button that we are showing people to give them a chance to opt-out of the private beta.

[00:06:00] There, all we are doing is expiring the cookie, setting the expiry date well in the past, and then reloading the page. That's all that's going on. We do have some different content in each branch but really this is the mechanics that we've seen in the split testing before, and the branch deploys before then.

There we are, that's the way that you can use split testing to instead of shaping traffic at random, sending all of your traffic to one specified branch unless users explicitly opts to go to another branch that you are exposing but not shaping traffic to in the future. Lots of interesting things to explore with split testing. This is a good one to get your teeth into.


your progress