Evaluating AI in marketing: The four metrics that matter in assessing impact

Navigating through the transformative world of generative AI in marketing requires a fine balance between experimentation and strategic application. It’s not just about embracing technological advancements, but instead, finding use cases where implementing them will enhance your marketing efforts. So, how can you tell when AI is simply fun to use versus when it truly improves the impact of your work? Let’s look at the four metrics and some practical use cases to understand when and how to integrate AI effectively into your marketing initiatives.

Metrics that matter: Cost, Time, Performance, and Creativity

When evaluating AI's efficacy and impact on marketing, I focus on four metrics. These metrics help you evaluate the tangible and intangible returns on AI investments in your marketing campaigns.

1. Cost: Evaluate the financial implications of incorporating AI versus traditional methods. Ask yourself: Does using AI cost more or less than how we previously accomplished this task?

2. Time: Analyze the temporal efficiency gained or lost. Ask yourself: Does using AI take more or less time than how we previously accomplished this task?

3. Performance: Examine the tangible and quantifiable results derived from campaigns. Ask yourself: Does using AI get you better or worse results than the previous performance for the task?

4. Creativity: Assess the innovative quotient and aesthetic appeal brought in by AI. Ask yourself: Does using AI deliver better or worse creative results than how we previously accomplished this task?*

Some use cases will excel in one metric—some will exceed in all four. Others will be worse in some areas and better in others. And some . . . well, AI is not good for everything, yet to be fair, I haven’t found a use case in this bucket yet! It’s important to remember that a successful AI use case does not need to excel across all four metrics to have a significant impact.

* A quick side note: Creativity is subjective and hard to quantify. However, I have found examining it through variables like audience engagement, brand resonance and consistency, and innovative appeal provides a more tangible approach to assessing its impact.

Evaluating two use cases against these metrics

Over the past year, I experimented with and implemented AI across many different marketing use cases at Google and other organizations. While not all experiments resulted in tangible ROI, each benefited from the use of the tools.

Use case 1: Using AI to create derivative assets, such as promotional materials, for a white paper in a campaign.

• Without AI: Traditional methods entailed an agency creation process that spanned 2-4 weeks and cost between $3,000 and $7,000, with FTE involvement of 90-120 minutes. Alternative methods relied on an FTE to do all of the work, taking 6-8 dedicated working hours.

• With AI: An FTE, utilizing content-focused AI tools like Jasper, Anyword, or Writer, generated and edited all of the assets within 90-120 minutes, incurring only the tool’s cost, which is between roughly $50 and $100 per month.

• Result: Let’s look at each metric: 1) Cost—Positive: with AI, our costs went down. Instead of paying an agency, we used an AI tool that gave us more options for significantly less money. 2) Time—Positive: with AI, the time to complete these assets went down in both cases. 3) Performance—Neutral: we did not do a true A/B test as we only used the AI-generated assets, however, the assets performed similarly to previous campaigns. 4) Creativity—Neutral: in my assessment, the assets we created with AI in this example were no more or less creative than previous campaigns.

• Bottom line: AI facilitated a notable reduction in both time and costs, while maintaining comparable campaign performance and creativity levels, demonstrating a great use case for AI in content creation of derivative assets.

Use case 2: Using AI to generate consistent and stylized speaker illustrations for an event.

• Without AI: Existing photos were provided to the team by the speakers themselves—each of these photos varied in their formality and professionalism.

• With AI: AI tools were utilized to create stylized headshots, ensuring visual consistency across all speaker slides, albeit with an investment of 90 minutes and a $60 tool cost.

• Result: Let’s look at each metric: 1) Cost—Positive / Neutral: with AI, our cost went up (minimally). Now, if your team had taken new headshots—or used a designer to touch up existing headshots—for an event so they would have a similar look and feel, that would have cost more than our AI tool. 2) Time—Negative / Neutral: with AI, the time dedicated to this task went up. Again, if you had taken headshots for this, it would have taken the time of the photographer, designer, and speakers (who would need to be in one place). 3) Performance—Neutral: in this example, there was no tangible performance benefit from using AI. Attendees didn’t see the photos until they got to the location, so it didn’t impact registrations. 4) Creativity—Positive: at this event, some attendees commented on the “cool factor” of the headshots, which I put in the audience engagement bucket noted above. Given this, we saw an increase in audience engagement as well as an improvement in the overall innovative appeal of the headshots.

• Bottom line: Despite the additional time and cost, the consistent visual appeal and innovative touch enriched the event’s aesthetic and brand representation. If your events use custom or touched-up headshots, the ROI would be tangible.

Assessing ROI doesn't need to be difficult. Simply assign positive/neutral/negative impact to the four metrics I've defined. You will see immediately if it's worth experimenting. Over time, you can sub out the qualitative assessment with quantitative results, demonstrating true ROI to your business.

Other considerations that marketers should assess

Beyond the four metrics covered above, there are a myriad of other considerations that you should account for when determining the value AI plays in your use cases. Some of these tie indirectly to the metrics above, and some of them don’t. While each of these topics warrant an entire post, I will quickly discuss three here, but know there are many, many more.

• Employee satisfaction. Integrating AI can streamline work processes, reducing manual and mundane tasks. Oftentimes, this can boost job satisfaction and productivity, especially for more seasoned professionals who get bogged down with the more tedious aspects of their jobs. It’s important to find the use cases where the use of AI assists and empowers your team, rather than alienates or overwhelms them, thereby fostering a supportive work environment that can enhance employee retention.

• Skill level of marketers. AI enables marketers at all skill levels to achieve what previously might have required a seasoned professional. Through the strategic use of AI tools, even those with less experience or specialized skills can execute complex marketing tasks, create engaging content, and analyze data effectively. This democratization of marketing capabilities not only broadens the scope of what your team can achieve but also allows for the optimization of talents across various facets of your marketing strategy.

• Ethical guidelines. Using AI in marketing should adhere to ethical guidelines that protect consumer data, prevent biases, and ensure fair and transparent practices. Marketers must ensure that use cases that leverage AI, from data collection to content creation, uphold moral and legal standards, safeguarding both consumer interests and brand reputation.

It’s time to start using AI in your marketing

AI opens up a realm of possibilities in marketing. Strategic implementation, backed by thoughtful analysis and ethical considerations, is key to harnessing its true potential. By evaluating your various use cases through well-defined metrics (as well as softer “intangibles”) and by integrating AI into use cases where it provides tangible value, marketers can truly leverage AI’s transformative potential to supercharge their work.

This blog was also published on my LinkedIn profile.