You Should Test That!

You Should Test That! Plot Summary

Introduction

Have you ever launched a marketing campaign that seemed perfect on paper, only to watch it fall flat in the real world? Or perhaps you've redesigned your website based on the latest trends, but your conversion rates actually dropped instead of soaring? These frustrating experiences happen to marketers every day. The truth is, when we rely solely on intuition, industry "best practices," or creative genius to drive our marketing decisions, we're essentially gambling with our business resources. What if there was a more reliable approach? A systematic method that could dramatically improve your conversion rates, increase revenue, and provide valuable insights into your customers' behavior—all without increasing your advertising spend. This approach exists, and it's called scientific marketing testing. By implementing a structured testing strategy, you can move beyond guesswork and discover exactly what motivates your audience to take action. The pages that follow will equip you with the frameworks, tools, and mindset needed to transform your marketing from an art of luck into a science of predictable success.

Chapter 1: Embrace Data-Driven Decision Making

Marketing has traditionally been viewed as primarily a creative discipline, where success often hinges on brilliant ideas and intuitive understanding of consumer psychology. While creativity remains essential, today's most successful marketers recognize that behind every effective campaign lies a scientific process of testing, measuring, and optimizing. This shift from opinion-based to evidence-based marketing represents one of the most significant transformations in the field. The scientific method in marketing isn't about eliminating creativity—it's about validating it. When Electronic Arts wanted to improve registration rates for The Sims 3 game, they didn't simply redesign their registration page based on what they thought would work. Instead, they analyzed their data, formed specific hypotheses about what might improve conversion, and systematically tested these ideas. This methodical approach led them to discover that offering a free town as an incentive increased registrations by an astounding 128%, far outperforming other incentives they had assumed would be more appealing. What makes this approach so powerful is that it eliminates the guesswork from marketing decisions. Rather than debating whose opinion about a website design or email subject line is correct, teams can simply test different approaches and let customer behavior determine the winner. This not only improves results but also reduces internal friction and speeds up decision-making processes. As one marketing director put it, "Testing has completely changed our team dynamics—we spend less time arguing and more time learning." To implement data-driven decision making in your organization, start by questioning assumptions. When someone says "our customers prefer video content" or "blue buttons convert better than green ones," ask what evidence supports these claims. Create a culture where "let's test that" becomes the standard response to marketing assertions. Begin with small, simple tests to build momentum and demonstrate the value of this approach before tackling more complex challenges. Remember that the goal isn't to test everything, but to prioritize testing opportunities that have the greatest potential impact on your business outcomes. Focus on high-traffic pages, critical conversion points in your funnel, or expensive marketing campaigns where even small improvements can yield significant returns. By systematically testing these high-value areas, you can achieve remarkable results without overwhelming your team or resources. The transition to data-driven marketing doesn't happen overnight. It requires patience, persistence, and a willingness to be proven wrong. But the rewards—higher conversion rates, increased revenue, and deeper customer insights—make this journey well worth the effort. As you embrace this approach, you'll find that marketing decisions become less stressful and more successful, guided by evidence rather than opinions.

Chapter 2: Identify High-Impact Testing Opportunities

Not all testing opportunities are created equal. The most successful optimization programs focus their efforts on areas that will deliver the greatest business impact. This requires a systematic approach to identifying and prioritizing testing opportunities based on data rather than random hunches or the latest design trends. When the marketing team at Electronic Arts wanted to improve conversion rates for their game The Sims 3, they didn't randomly test elements across their website. Instead, they analyzed their funnel data to identify that game registration was a critical bottleneck. By examining their analytics, they discovered that while millions of copies had been sold, a significant percentage of players only used the game launcher to start the game without registering. This represented a massive untapped opportunity to convert casual players into registered users who could be marketed additional content. The team used the PIE framework—Potential, Importance, and Ease—to prioritize various testing opportunities. Game registration scored high on all three dimensions: it had significant room for improvement (Potential), directly impacted revenue opportunities (Importance), and could be tested relatively quickly (Ease). This data-driven prioritization led them to focus their testing efforts on the game launcher, which ultimately resulted in a 128% increase in registrations and millions in additional revenue. To identify high-impact testing opportunities in your own business, start by gathering data from multiple sources. Web analytics can reveal pages with high traffic but poor conversion rates. Customer feedback and support tickets might highlight friction points in your user experience. Heatmaps and session recordings can show where visitors are getting stuck or confused. Look specifically for pages that serve as critical junctions in your conversion funnel—landing pages, product pages, checkout flows—where even small improvements can have outsized impacts. Once you've identified potential testing candidates, evaluate each using the PIE framework. Rate each opportunity from 1-10 on Potential (how much room for improvement exists), Importance (the page's value to your business), and Ease (how quickly you can implement and complete tests). The opportunities with the highest combined scores should become your testing priorities. Remember that prioritization isn't a one-time exercise but an ongoing process. As you complete tests and gather more data, continually reassess your priorities to ensure you're always working on what matters most. This disciplined approach prevents the common pitfall of getting distracted by low-value tests that might be interesting but don't significantly impact your business outcomes. The most successful testing programs maintain a balanced portfolio of opportunities: some quick wins that can build momentum and demonstrate value, alongside more substantial tests that address fundamental aspects of your value proposition or user experience. By strategically identifying and prioritizing these opportunities, you can maximize the return on your testing investment and drive meaningful business results.

Chapter 3: Craft Powerful Hypotheses That Drive Results

A hypothesis is more than just a guess—it's a structured prediction about how a specific change might improve your conversion rate. The most effective testing programs are built on well-crafted hypotheses that connect observed problems to potential solutions and expected outcomes. This approach transforms random experimentation into strategic learning. When WiderFunnel worked with Singular Software to improve downloads of their PluralEyes audio synchronization software, they began with a thorough analysis using the LIFT Model—a framework examining six conversion factors: Value Proposition, Relevance, Clarity, Anxiety, Distraction, and Urgency. The team identified several issues: competing calls to action created confusion, the value proposition was unclear, and the visuals were puzzling. One particularly problematic element was an image of goldfish that was meant to represent audio streams converging—clever, but confusing to visitors who might wonder if the site was selling fish food rather than software. Based on this analysis, the team developed specific hypotheses. They predicted that focusing on a single call-to-action (download), clarifying the value proposition with a new headline ("Skip Endless Hours of Tedious Manual Syncing"), and replacing the goldfish image with either a screenshot or video demonstration would increase conversion rates. The test confirmed these hypotheses, resulting in a 17.4% boost in trial downloads. Interestingly, while everyone involved expected the video version to perform best, it was actually the screenshot variation that won—highlighting why testing hypotheses is so crucial. To create your own powerful hypotheses, start by analyzing your pages through the LIFT lens. Identify specific problems in each of the six conversion factors. For example, you might notice that your value proposition is buried below the fold (Clarity issue), or that you have multiple competing calls to action (Distraction issue). Then, formulate hypotheses that address these problems using the structure: "Because we observed [problem], we believe that [proposed change] will result in [expected outcome]." The most valuable hypotheses often challenge conventional wisdom or address fundamental aspects of your value proposition rather than superficial design elements. While testing button colors might yield incremental improvements, testing different ways of communicating your core value proposition or addressing major customer concerns can produce dramatic results. Focus on hypotheses that not only might improve metrics but will also generate insights about your customers regardless of the outcome. Document your hypotheses carefully, including your rationale and expected results. This documentation creates an invaluable learning repository over time, allowing you to identify patterns across tests and develop deeper insights about what motivates your audience. Remember that even "failed" tests—those that don't produce the expected improvement—provide valuable information that refines your understanding and informs future hypotheses.

Chapter 4: Optimize Your Value Proposition First

Your value proposition is the foundation of your conversion potential—it's the full set of perceived benefits and costs in your prospect's mind. While many marketers focus on optimizing design elements or call-to-action buttons, the most dramatic improvements often come from clarifying, strengthening, or repositioning your value proposition. When WiderFunnel worked with a higher education institution, they identified that the school's value proposition wasn't resonating with potential applicants. Rather than making minor tweaks to design elements, they decided to test four fundamentally different positioning statements. Each landing page maintained the same layout and design, varying only the headline and supporting copy to emphasize different aspects of the school's value: academic excellence, career outcomes, campus experience, or affordability. The results were remarkable—the winning value proposition increased new applicant conversion rates by 40%. More importantly, the test revealed profound insights about what motivated their prospects. The school discovered that emphasizing career outcomes resonated far more strongly than academic prestige, campus life, or affordability. This insight didn't just improve their landing page; it transformed their entire marketing approach, influencing everything from advertising messages to recruitment events. To optimize your own value proposition, start by identifying all its components: tangible features (what your product does), intangible benefits (how it makes customers feel), and costs (price, effort, risk). Then test different ways of emphasizing these elements to discover what matters most to your audience. Remember that perceived value is subjective—what you think is most valuable about your offering may not match what your customers actually care about. When crafting value proposition tests, focus on addressing the fundamental questions in your prospect's mind: "What's in it for me?" "Why should I choose you over alternatives?" "Is this worth the cost?" Create variations that emphasize different answers to these questions, then let your audience tell you through their behavior which resonates most strongly. This approach often reveals surprising insights about customer priorities that wouldn't emerge from surveys or focus groups. The most effective value proposition tests maintain consistency in design elements while varying the core message. This isolation ensures that you're measuring the impact of the value proposition itself, not confounding factors like layout or imagery. While this approach might seem less creative than comprehensive redesigns, it produces clearer insights that can guide not just your website optimization but your broader marketing strategy. Remember that optimizing your value proposition isn't a one-time exercise but an ongoing process of refinement. As your market evolves, competitors emerge, and customer needs change, continue testing different aspects of your value proposition to ensure it remains compelling and differentiated. The insights gained from these tests represent some of the most valuable marketing intelligence you can gather.

Chapter 5: Eliminate Anxiety and Distraction

Even with a compelling value proposition and relevant messaging, your conversion rates can suffer from two powerful conversion killers: anxiety and distraction. These psychological barriers prevent otherwise interested prospects from taking action, and addressing them can produce dramatic improvements in your results. AllPopArt, a company that creates custom pop art portraits, faced this challenge on their product pages. Customer surveys revealed that many visitors didn't understand that real artists created the artwork (rather than automated filters), and consequently felt the prices were too high. Additionally, the complex product customization form at the top of the page created immediate anxiety about the ordering process. These issues were creating significant friction in the conversion path. Working with WiderFunnel, AllPopArt tested several page variations to address these anxiety points. The winning design made a critical change: it moved the product description and value proposition to the top of the page, while relegating the customization form below the fold. This simple restructuring allowed visitors to understand the product's value before confronting the potentially overwhelming customization options. The redesign also emphasized that "Your portrait will be hand-illustrated by a U.S.-based artist," directly addressing the value perception concern. The results were dramatic: a 28% increase in purchase conversions and a 42% increase in revenue per visitor. By reducing the initial anxiety triggered by complex forms and clearly communicating their unique value, AllPopArt significantly improved their conversion performance without changing their product or pricing. To reduce anxiety on your own pages, start by identifying elements that might cause uncertainty or hesitation. Common anxiety triggers include complex forms, unclear pricing, vague shipping information, or missing privacy policies. Consider adding reassurance elements like guarantees, security badges, or testimonials strategically placed near points of potential anxiety. However, be careful not to overdo it—sometimes adding too many trust symbols can actually increase anxiety by drawing attention to concerns visitors didn't previously have. Distraction is equally damaging to conversion. When Hair Club tested their lead generation landing page, they discovered that aspirational images of beautiful women with flowing hair—which conventional wisdom suggested would be motivating—actually distracted visitors from the core message about hair restoration options. A redesigned page that replaced these images with more focused content about service options increased conversions by 20%. To minimize distraction, ruthlessly evaluate every element on your page and ask: "Does this help visitors take the desired action?" Test removing unnecessary navigation links, competing calls to action, or irrelevant content that might divert attention from your primary conversion goal. Remember that in conversion optimization, less is often more—the clearest path to conversion is usually the one with the fewest distractions. The most effective approach combines addressing both anxiety and distraction simultaneously, creating pages that not only remove barriers to conversion but also provide a clear, focused path to action. By testing different approaches to these psychological factors, you can discover exactly what makes your specific audience comfortable enough to convert.

Chapter 6: Transform Test Results into Strategic Insights

Testing isn't just about finding winning page variations—it's about uncovering deeper insights about your customers that can inform your broader marketing strategy and even your business decisions. The most valuable outcome of a testing program often isn't the immediate conversion lift but the strategic understanding you gain. WineExpress.com, the exclusive wine shop partner of Wine Enthusiast, discovered this when testing their "Wine of the Day" page. This page featured a daily wine selection promoted through email to their subscriber list, with a special shipping offer of just 99 cents. The page already converted well, but the team believed it could perform even better. Working with WiderFunnel, they tested three variations against their control page, exploring different layouts, call-to-action placements, and content emphasis. The winning design increased the sales conversion rate by 5%, which was valuable—but the real revelation came when they analyzed the revenue data. That same variation increased revenue per visitor by a remarkable 41%. This disparity between conversion rate and revenue highlighted a crucial insight: the new design wasn't just convincing more people to buy; it was influencing them to make larger purchases by emphasizing wine quality and expert ratings more prominently. This strategic insight extended far beyond a single page optimization. It fundamentally changed how WineExpress approached their marketing, shifting emphasis from price and shipping offers to quality indicators and expert endorsements across all channels. What began as a tactical conversion test became a driver of business strategy that influenced product selection, email marketing, and even how they trained their customer service representatives. To extract maximum value from your own testing program, look beyond simple win/loss metrics. Analyze patterns across multiple tests to identify recurring themes. For example, if tests consistently show that emphasizing certain product features outperforms others, this reveals valuable information about your customers' priorities that should influence product development, not just marketing. Create a systematic process for documenting and sharing insights across your organization. After each test, conduct a formal analysis session asking questions like: "What does this result tell us about our customers' preferences?" "How might this insight apply to other marketing channels?" "Should this change how we develop or position our products?" Document these insights in a central repository accessible to stakeholders across marketing, product, and executive teams. Remember that the goal of testing isn't just to optimize individual pages but to develop a deeper understanding of your customers that drives continuous improvement across your entire business. By approaching testing with this strategic mindset, you transform it from a tactical marketing function into a powerful engine for business growth and competitive advantage.

Chapter 7: Build a Culture of Continuous Optimization

The most successful organizations don't view conversion optimization as a one-time project or isolated marketing tactic—they integrate it into their organizational DNA, creating a culture where testing and data-driven decision-making become standard practice. This cultural transformation can be challenging but delivers enormous long-term value. SAP, the global enterprise software leader, exemplifies this transformation. Initially, SAP's landing page for Crystal Reports trial downloads was underperforming. Christine Mykota, Director of Business Analytics Marketing, faced several challenges in implementing a testing program: convincing skeptical senior management, working within strict corporate branding guidelines, and lacking internal resources for conversion optimization. Despite these obstacles, Mykota partnered with WiderFunnel to develop and execute a testing strategy. The initial test produced a 32.5% lift in trial downloads—far exceeding management's expectations. This success did more than improve a single landing page; it catalyzed a cultural shift within the organization. As the case study reports, "Now she's a champion within the organization to look at landing-page testing and to do more of it." Within six weeks, Mykota's team implemented similar changes across other landing pages, achieving a 26% lift in conversion rates site-wide. More importantly, the success gave Mykota leverage to negotiate greater flexibility within branding guidelines for future tests, as the data clearly demonstrated the business impact. What began as a single test evolved into a program of continuous optimization that spread throughout the organization, changing how decisions were made across multiple departments. To build a similar testing culture in your organization, start by securing senior-level buy-in. Focus on the business impact of testing rather than technical details—executives care about revenue and ROI, not button colors or headline variations. Create a tangible opportunity by identifying high-value, low-risk testing candidates where you can demonstrate quick wins that build credibility and momentum. Share results widely throughout the organization, celebrating both successes and learning opportunities. Many successful testing programs include regular results presentations where team members from different departments are invited to see outcomes and guess which variations won. This creates engagement, educates colleagues about the testing process, and positions you as a leader driving measurable results. Be prepared for resistance. "You should test that" can be threatening to those accustomed to making decisions based on opinion or authority. Counter this by emphasizing that testing isn't about proving people wrong—it's about discovering what works best for customers. As WiderFunnel's "Optimization Manifesto" states: "We listen to our gut, and then test what it says. We gather marketing research, and then test it." The most successful testing cultures maintain a balance between creativity and discipline, intuition and evidence. They recognize that brilliant ideas can come from anywhere in the organization, but that these ideas should be validated through testing before significant resources are invested. This balanced approach respects both the art and science of marketing, creating an environment where innovation flourishes within a framework of accountability.

Summary

Throughout these pages, we've explored how systematic testing can transform your marketing from guesswork to science. From identifying high-impact testing opportunities to crafting powerful hypotheses, optimizing your value proposition, and building a data-driven culture, each element builds upon the last to create a framework for continuous improvement. As the Optimization Manifesto states: "We believe in art and science, creativity and discipline, intuition and evidence, and continuous improvement." Your journey to marketing success through testing begins with a single step: identify one high-value page on your website and create a hypothesis about how it could be improved. Then, test that hypothesis with a controlled experiment. Don't worry about getting it perfect—the beauty of testing is that it's self-correcting. Each test, whether it produces a winner or not, generates valuable insights that inform your next experiment. The path to marketing excellence isn't found in following best practices or copying competitors—it's discovered through your own systematic testing program, tailored to your unique audience and business goals.

About Author

Loading...

Chris Goward

Read more