30. MONTAGS-Interview auf Web Analytics Europa – Heute: Dan Siroker

Dan Siroker ist Co-Founder von Optimizely, einem Optimierungs-Unternehmen – in diesem Interview verrät er Web Analytics Europa einige Einblicke zu Optimization und seinem neuen Buch

Dan Siroker Optimizely

Das Interview wurde auf Englisch geführt, um die Inhalte so genau wie möglich wiederzugeben.

Dan is the CEO and co-founder of Optimizely. The original inspiration for Optimizely came from his experience as the Director of Analytics on the Obama 2008 campaign. Prior to Optimizely, Dan served as Product Manager for Google Chrome and AdWords. Dan Siroker has also been featured on Bloomberg, the Wall Street Journal, New York Times and Wired and recognized by Forbes as Top 30 under 30 in Technology in 2012.

Web Analytics Europa: A book about optimization. Why?

Dan Siroker: We had an idea for our book “A/B Testing” – The Most Powerful Way to Turn Clicks into Customers“ – for a while. At the time, we had thousands of customers, advocates and partners that made the project possible. The book highlights key takeaways and best practices from hundreds of thousands of tests, which have been run on our platform and make them accessible to the public. It aims to educate people who don’t yet know about the value A/B testing can bring to organizations. We also want to inspire marketers to take the testing process into their own hands. Over the last few years, web site optimization through A/B testing has also become available to anyone who is interested in the impact of data-driven marketing, but without the need for technical resources.

The book is also a way to address the following questions, which we have been asked over the last years:
– What is A/B testing?
– Why will it add value to my company?
– Where do I begin testing?
– How do I build a testing culture at my company?

Today, the average conversion rate of the websites out there is roughly 2%, meaning that 98% of visitors to a website do not become customers. A/B testing enables businesses to make the leap from “we think” to “we know” data-driven decisions. Data has enabled teams to let the data be the decision maker. This is of course one reason we wrote the book. We firmly believe in the power of A/B testing because we have seen more than 6.000 of our customers use it to their advantage. Another reason we wrote this book was because we were not able to find a guide to A/B testing that summarized our learnings over the last years in a practical way that would at the same time inspire others. The book is meant to be an actionable guide for everyone who wants to learn more about testing and how to create a culture of testing in their organization.

AB Testing Book

Web Analytics Europa: Can you give a detailed example of a successful case study you were able to achieve?

Dan Siroker: I think if we talk about testing in general, then this would be the Case Study from the Obama campaign. It is the case study that inspired us to start Optimizely and may be the most successful experiment I can think of. I was Director of Analytics at the Obama campaign in 2008. Our overall goal was to increase online donations for the campaign. To increase signups and donations, we decided to test different buttons and different media elements in order to see which ones performed best. We tested 4 different buttons saying: “Sign Up”, “Learn more”, “Join us now”, “Sign up now”. We also tested three images and three videos. The results were eye opening and very counterintuitive. We found out that the combination of the button “Learn More” and the family image of the Obama’s performed the best. All the videos performed worse than the images, which was very surprising to the entire team. By running this test, we increased e-mail subscriptions by 40.6 percent. This led to an increase of 280,000 volunteers, which in turn led to $57 million Dollars in donations. Without having established quantifiable success metrics and testing them, we would never had known that visitors would respond best to the combination of “Learn more” and the family image.

This case can teach us many things about testing. First of all, it shows us that small changes in elements such as buttons or images can lead to amazing results. Second of all, the Obama case study teaches us to challenge the status quo. We shouldn’t base our decisions on assumptions.

Web Analytics Europa: Being the Director of Analytics at the Obama Campaign sounds extremely exciting and on the other hand very confidential. Can you let us know the set up around those campaigns and the team (size, responsibility, feedback by Obama etc.)?

Dan Siroker: For outsiders, it’s hard to imagine the size of a team that is involved in running a presidential campaign. On top the food chain is of course the candidate himself, and directly below him is the campaign manager. He oversees a number of different departments such as field, correspondence, paid media, which was TV and radio, operations, finance and New Media. I was part of the New Media team which was basically everything technology-related that the people from the other departments didn’t understand. Video, marketing, design, blog as well as my team, the analytics team – all belonged to the New Media team. I was leading the analytics team, a team of 6 members. We were situated kind of in the middle of the whole funnel. Online marketing would be the top of the funnel and tried to get people to visit the site. The same with the blog team. They were constantly writing and telling stories which were not about the candidate himself, but more about the supporters. Once people had come to the site they would receive e-mails from the e-mail team. A lot of the mails were on a local or state level and customized to the state in which people were living. The design team was responsible for all the design of the hand-outs and the website. Our analytics team was composed of engineers and analysts.

Web Analytics Europa: Can you also explain the missed opportunities when not analyzing your website via Optimization?

Dan Siroker: As mentioned, conversion rates for websites average 2%. This means that 98% of your visitors are not turning into customers. A lot of companies spend the largest part of their marketing budgets on driving traffic to a website. Since the website or the landing pages are not optimized, many of these visitors will not convert and a lot of money spent on these campaigns is wasted. This could be drastically improved by focusing more attention on optimization of these pages. A/B testing is a very low risk tool that is extremely powerful. Companies that do not test give away competitive advantages to competitors. Why is that? A/B tests help make decisions based on data. With every test we run, we learn more about user experience. Decisions based on user data can offer the best experience to our visitors and turn more of them into customers.

AB Website Testing

Web Analytics Europa: Where do you see testing “going”? What will change in 2014? What will be different in five years from now? Will it be a commodity to do tests before publishing any kind of content?

Dan Siroker: We have seen that in 2013, only around 14% among the top 10,000 websites worldwide have a testing solution in place. We are at a tipping point with organizations starting to understand the benefit of optimization to deliver targeted web experiences based on customer insights such as browsing history, geography or particular interests.

With online retailers delivering dynamic experiences to customers and publishers optimizing their paywalls to maximize revenue, 2014 is going to be a big year for harnessing the power of online insight to drive revenue. In the future, optimization and website testing will become more engrained in the organization. We will also see the rise of the ‚Chief Testing Officer‚, a person that will oversee testing, coordinate testing in different departments and help steer the strategy and knowledge sharing around testing. In the next couple of years we will also be able to observe that some industries which have not been very serious about testing will engage and discover the benefits that testing can bring to their organizations as well. Mobile and the optimization of experiences across devices will be a trend that is going to continue and grow in the years to come. As competition online rises, focus will shift to efficiency in order to gain competitive advantages.

Das letzte Interview finden Sie hier: 29. MONTAGS-Interview auf Web Analytics Europa

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.