Ad Ops is constantly evolving—the industry isn’t what it was a year or even two years ago. Big data is a large driver of this evolution with its increasingly important role in the latest technologies available. Ad ops professionals are now responsible for overseeing data and providing insight into measuring ad effectiveness and recommending strategies for ad optimizations. Data has become vital to Ad Ops.
The power of analytics enables us to make smart business decisions, have far more efficient operations, and see happy publishers. In this webinar, Lila Hunt, Head of Publishing Solutions at Sortable and James Murphy, VP of Monetization at TextNow discuss why using deeper analytics is critical to maximizing earnings, which metrics to watch and why, and looking at real examples.
If you're interested in learning about Sortable Analytics or any slides in this presentation, please contact us at Sortable.
Craig Ling: Hey everybody, thanks for attending. We're just going to give people a few more minutes to join and then we'll start webinar.
Craig Ling: [00:02:06] Okay, hello everybody, and welcome to today's webinar on analytics and day-to-day ad ops. My name is Craig Ling, and I'll be your moderator. I work on the marketing team here at Sortable, and I'm really excited to be hosting a session today. Co-moderating will be Jaime Murphy, VP of product here at Sortable. I'm pleased to introduce today's speakers, Lila Hunt, head of publishing solutions here at Sortable, and James Murphy, senior vice president of monetization at TextNow. Lila has almost a decade of senior technical experience in the ad ops industry, and is often a guest speaker at conferences and sits on many PreBid.org committees, helping to shape and guide the future of the tech ecosystem. James' entire career has centered around digital media, having worked in sales, operations, and tech-focused roles. James has over a decade of industry knowledge and is currently responsible for all advertising for TextNow's properties. We are so honored to have James participate and share his knowledge with us today.
Craig Ling: Before I hand the mic over to Lila, I have a few housekeeping items to cover about this presentation. First, we will be recording today's webinar, and everyone registered will receive an email within a few days with a link to the recorded version. We'd love to hear from you during this webinar. If you have any questions for our speakers, please feel free to send it through the question tab in the webinar software, and we will try to answer as many questions as possible at the end of the presentation. So, without further ado, I'd like to kick things off by welcoming our head of publisher solutions, Lila. Lila, over to you.
Lila Hunt: [00:04:01] Thanks Craig, and thanks very much James for joining us today. I'm going to start off by talking a little bit about who we are at Sortable and what we do. So, we serve a global portfolio of publishers which represents over 20 billion monthly ad requests across more than 300 customers. We're primarily a tech company headquartered in Kitchener-Waterloo, Canada. We often describe ourselves as the Silicon Valley of the north. It's a tech hub with some of the world's best engineering talent. 70% of our employees are technical and focus on using technology to solve our industry's greatest challenges. We find this approach really helps our publishers focus on their business, creating content and driving growth through user retention and by acquiring new audiences.
Lila Hunt: So, just a little bit about our product suite. One of the common themes our publishers often share with us about what we do is they say Sortable does a lot, and that's very true because we started as publishers ourselves. We grew really frustrated by the obstacles and monetizing our own websites. Some of the challenges included lack of transparency into how advertisers who were spending on our sites, and the convolution of technologies that were required to monetize a digital business. We started building tools to solve these challenges for ourselves, and then we began sharing our platform with peers in the industry who were voicing similar frustrations. Today, we help publishers with monetization through managed solutions that execute optimizations across their businesses, and we offer server- and client-side header bidding and fully transparent analytics platform. We also work to bring ad services for fraud, discrepancy, malicious ad monitoring into a single platform.
Lila Hunt: TextNow and Sortable partnered last year, and so I'm going to pass it over to James so he can tell us a bit about his business.
James Murphy: Thanks, Lila. It's nice to chat with all of you today. So here at TextNow, we're a ten year old company, also based in the Waterloo, Canada area. Our free calling and texting app is the leading app in that category in the United States. On the Android platform, we're a top 5 utility app. On the iOS platform, we're a top 20 social networking app. So for that part of our business, we really focus obviously on monetizing our free customers through advertising and in-app purchases. We also have a fully-built out wireless business, and we act as a telco, offering unlimited LTE plans, texting and calling plans, and competing with the MetroPCS's or Boost Mobiles of the world. What we're working on this year is having those two businesses converge, so looking at offering our customers free calling and texting not only when they're on a wifi connection, but also through cellular data. So look out for more updates from us in that category moving forward.
James Murphy: You can see the metrics here, but we're a sizable app across kind of all of our properties, Android, iOS, and desktop. We have about 6 billion monthly ad impressions. Our monthly active users are about 20 million, and our daily actives are 2.5 million. We'll continue to focus obviously on growing our user base. We've benefited from really having a lot of our users coming to us organically. What differentiates us on the free side of our business is we always give our users a dedicated phone number. We're always free calling and texting as long as you're on a wifi connection, which is a differentiator to our competitive set, which you have to earn calling points or obviously pay for that. So that's our business in a nutshell; if you could go to the next slide...
Lila Hunt: Sure. So last year we also did a case study, and this is available on Sortable's website if anybody is interested in digging into the more broader content of the case study. James, could you share some of what your challenges were before you partnered with us, and what were some of the results once you transitioned to our platform?
James Murphy: Sure. Yeah, so working on the desktop side of things, we had a lot of challenges as far as optimizing and maximizing revenue, specifically with the larger partners like Google, and looking at ways to really offer the most viewable inventory, and that's obviously very important from a monetization standpoint. And sort of what really helped us by iterating on the things we were doing in those categories and really looking at things to offer more ad requests when we have viewable inventory, which would translate obviously to the higher CPMs, more fill and overall more revenue for our companies. So we've seen a lot of success.
James Murphy: We've moved pretty much all of our desktop inventory over to Sortable. The logic that they have and the algorithms as far as maximizing the things I mentioned and passing on viewable inventory and looking at the most optimal refresh rates going through the larger partners, we've seen a ton of success. So, moving forward, we're going to be working with Sortable to monetize more of our in-app inventory where we have a bigger opportunity, being a utility for texting and calling there. So given that we've seen so much success on the desktop side, and the algorithms and logic that's within the platforms that they manage, we're very optimistic that we're going to see very positive results expanding into the in-app space.
Lila Hunt: Thanks, James. I think another really important thing to focus on is, I mean nobody doubts desktops is hard; app is so much harder to build for because of challenges with SDKs and app weight and app speed, and so that's where I'm really excited about our opportunity to work together to build really groundbreaking solutions for the industry in that space.
James Murphy: Exactly.
Lila Hunt: All right. The focus of our webinar today is on how James and I work through, on a regular basis, using analytics and data from our platform to really monitor trends, talk about performance, and look at opportunities to optimize their business. The place where I thought it made most sense to start would be at the beginning, which is monitoring top-line goals. Here you can see a chart that is just basic revenue CPM and impressions by day, day over day, and we chose Q4 into Q1 as a visual. And then you can see that infamous drop on January 1st when the advertiser's budgets reset.
Lila Hunt: James, would you describe how you measure your business, and how do you know you're on track with your goals?
James Murphy: The key metric that we look at is really the ARPU (Average Revenue Per User), the average revenue per user, so given that text now, we control a user base. People come to us for the product that we offer and the service that we provide around texting and calling, so given that we control users, we look at a user level, so we're not looking at monetizing our website or app, it's really at the specific user level and how do we maximize efforts at that user level. Given that we have the ability to monetize those users through advertising, we also have options to push them over to in-app purposes, or obviously convert them to a wireless customer. So we're really looking at maximizing total revenue at the user level and obviously key metrics that are important to us also things like retention and making sure that the ad experience isn't too intrusive that we make our users leave. That's really the key metric that we look at, and everything's centered around ARPU or [inaudible 00:12:15].
Lila Hunt: Very interesting. The next slide focuses on taking some of those top-line KPIs and looking at benchmarking through seasons or across different events. Here we surfaced sessions and session RPM, and it would make sense because you guys have a utility service that session volumes look really consistent through Q4 into January, and you can see how session RPM has changed with the change in the demand landscape. I like focusing on session RPM because it helps us understand that value of a user's journey and I think it complements what you're describing with understanding the revenue per user that you're looking at from your business.
Lila Hunt: Can you talk a little bit about what session value means to your business. I know you started talking about retention and maybe you can get in a little bit more about how you allocate inventory for marketing and how you really drive to convert and measure those users.
James Murphy: Yeah, of course. Obviously, there's seasonality that affects any business, and obviously Q4 is much bigger than Q1. The luxury that we have, given that we have obviously a product where we offer free texting and calling, and we have a way to upgrade those users to a paid wireless customer or subscriber.
James Murphy: When there's less demand in Q1, we'll allocate more of our inventory towards internal products and services to drive up the value of the inventory that we're offering in the open market. If you have the same amount of inventory available when demand's low, obviously that's going to yield lower CPMs, less fill. During times like that, we promote our own internal products more and we, obviously like I said, have fully built out LTE plans that we can offer users. We have texting and calling packages for as low as $9.99. By making less inventory available, and people want, obviously our user base performs well for certain brands and agencies, that'll drive up CPMs in times when demand's lower. So that's something we're always focused on, and speaking to the [inaudible 00:14:49] side of things, given that we're a utility for texting and calling, if you make the ad experience too intrusive, like I said, too many ads in places where people are texting, for example, it's going to cause lower session times and obviously hurt retention.
James Murphy: So probably even more important to us, believe it or not, than maximizing revenue is keeping our users happy and making sure we have high retention and highlight time value of our users. That's the whole business, right? When you're in a business like ours, if you're turning users and people are leaving at a high rate, you don't have a business, so the first metric we're always looking at is how do we retain our users, how do we make them happy, and then how do we make money off of them, whether it be through advertising, or like I said, we could promote obviously internal products and convert them to a subscriber-based plan.
Lila Hunt: Yeah, great. That makes a lot of sense. So how do you balance the value of the user through advertising compared to the value of the user through a subscription model?
James Murphy: It really goes back to that ARPU. What is our average revenue that we're earning through our advertising for a given user? We then have different offerings. It could be obviously converting them to a wireless customer where it could range anywhere from $9.99 a month for texting and calling to $39.99 for unlimited LTE. But then we have the ability for users to turn off advertising for $2.99, and that's something we look at based on your point, like how much money are we earning on average per user through ads, and does it makes sense to give them the ability to turn off ads for X price. It really just goes back to the economics of things. Again, if we're able to make a higher ARPU by giving the ability to turn off ads, that's great, and then they're happy and then we just obviously drive them to that in-app purchase funnel. As you can imagine, a lot of our revenue comes from advertising though on the free app side, but there's constant economics and metrics that we're look at in that regard.
Lila Hunt: Okay. On the next slide is where we start to dig into some of the dimensions more granularly. Generally, we work together on looking at trends and then identifying changes to those trends, and then digging into where maybe those trends have room to optimize or where something has changed and why. In this chart, we're looking at really high level changes in fill rate. Fill generally helps us understand demand density, or if you have the right amount of supply based on the market or the right amount of inventory based on the market. I think that really aligns with what you were describing and understanding do we need more inventory because we have really great demand performance, or should we be spending some of that inventory on internal marketing. I think CPM is another metric that helps us articulate that inventory value as well so we can track CPMs over time and understand again how that's impacting session RPMs and to tie back to your ARPU metric.
Lila Hunt: Talk to us a little bit about inventory scarcity and how you manage inventory based on changes to the demand density like we see at the beginning of Q1.
James Murphy: I think that's a common mistake a lot of publishers make. It's always more equals more, right? The more inventory I make available, the better chance I have to monetize it even though fill might be less. But in our opinion, that couldn't be further from the truth. What Sortable's helped us do, which has been great, is when we have high value inventory, like when it's viewable, for example, at a high percentage, you make more of that inventory available, even when there's less demand. People always want inventory, especially on desktop, that's a viewability rate over 70% for example. For less desired inventory types or formats, we have the luxury to promote, like I said, internal products and so forth.
James Murphy: I think it's just a common mistake that we see, and I'm from the platform side of the business. Now being on the publisher side, I understand that we see what it looks like to be at the edge, but I also understand what it looks like to be at intermediary level. It's a common mistake that you see in the industry; just make more available, flood the market with more of your inventory, and we have a better chance of monetizing it, but what's happening at the buyer side, or the DSP side, is they're just going to start randomly hunting more of your requests because they see so much of it. You want to make sure that you're maximizing your efforts around the inventory that the demand side actually wants. Obviously change the structure of things if it's not desired at all, or when demand's lower, like I said, promote internal products. Or even, when you don't have that luxury as a publisher, just don't make that inventory available because you'll drive higher CPMs and more fill from inventory that people actually want.
Lila Hunt: Exactly. An example that we implemented at the start of Q1 was we actually changed the refresh rates because we saw CPMs drop, and so we turned down the refresh rates, which helped encourage the CPMs to lift back up. What we saw was a more aggressive recovery, I think, than normal because there wasn't this, like you said, flood of inventory based off how much demand was actually available at that time. Then once the budget started to come back in, that CPM stayed high and we were able to increase revenue and maintain that upward performance.
Lila Hunt: I know [crosstalk 00:21:14]. I know platform's also really important dimension for you because you run app and also a web app. Talk to us a little bit about the trends you observe on different platforms.
James Murphy: It's very different. The Android versus iOS, even on the app side, it's a different mix of users. People on the iOS platforms tend to have newer iPhones; it's a different demographic than you see on Android platform. For example, obviously you see different behavior as a result. So it's really looking at all different metrics. We have internal AB frameworks where we can test all these different things, even specific within one platform like the Android platform. Certain devices and how they perform from a monetization standpoint. We're constantly looking at these things. Both of those users that then could use either our desktop application or browser based service, so then we have the ability to look at those metrics as those people visit on the desktop side. You really have to look at just going back to your business at a user level.
James Murphy: Given that we offer products where we give people a free utility, we get a lot of information about those users so we know, in many cases, age and gender. We can infer things like ethnicity because we have a lot of content around texting. We could obviously look at the behaviors based on that, and then make deterministic decisions on how to monetize these people better, how to make that inventory of a higher value to the demand side. We're constantly looking at these things, and it's a very a tech-driven business., getting deterministic data to make decisions and not just doing on a hunch or feel, if you will.
Lila Hunt: Great. There's a few questions that are coming in about the inventory optimizations, actually that segues really well into the next slide, which is specifically on optimizing inventory. I'll try and maybe touch on those questions a little bit as we go through here. Essentially, philosophically at Sortable we believe that machines really should do a lot of the heavy lifting when it comes to granular optimizations. There are things that we work on together that have to do with more manual implementations, changes to sizes, formats, ad units, and refresh rates. We typically look at by different dimensions and then make decisions about how to set them for the site or against different sections of inventory. Then there are facets that are better left to machines at a really granular impression-level decision, and that's where flooring and timeouts generally happen in our platform informed by statistical modeling per impression. For the higher level metrics that we look at, we can look out performance against the dimensions and then make recommendations, and then A/B test different configurations to understand where performance really is better.
Lila Hunt: One of the questions that's come up so far is what kind of analysis goes into matching of inventory to market demand, and how do you know when more inventory is overall harmful. I think really once CPMs come down really aggressively, that's where I would say there's a red flag on both our sides. Would you be able to comment a little bit on...I guess maybe a little more elaborate on streamlining these optimizations and how we make decisions together about when that performance is critical to react-to.
James Murphy: Sure. There's constant optimizations that you do obviously around the things you see here. We talked about the device side, or around sizes, we're constantly looking at do we have the right mix in front of our users? Are those formats viewable? All these things that matter. What we love about Sortable is the transparency and the communication we have all the way down to the actual buyers and understanding what they want, in a feedback loop from one edge to the other is how we like to say it, us being on one side as the publisher and Sortable representing the demand side with full transparency and visibility to what they actually want and giving us that feedback to say, hey, these units have a higher demand density; we should run more of those. Or, you have these ad units we can make available, for example. Making refresh rates around refresh rates that people care about, not just doing it in a kind of random way; let's have a higher refresh rate because it creates more ad inventory. No, let's make a higher refresh rate when those ads are viewable, and make that available to the demand side. That's constantly things that we're looking at, and we continue to iterate in that regard, and obviously timeouts and setting floor pricing is a natural thing to find that right mix of the pricing that people are going to buy at based on the things they want to buy, right?
Lila Hunt: Right. In addition to our algorithmic flooring, you also do use manual minimum floors to help control how inventory gets exposed to buyers. Can you describe a little bit how you use minimum floors?
James Murphy: Like I talked about earlier, we have our own internal products and services, i.e. our wireless business that we're saying below X price for this format, for this geo, for this portion of our app or this screen on desktop, we're going to make this the floor based on data that we have with seasonality and all the different metrics we look at like I talked about earlier. We have that luxury to say, okay, if we can't monetize about X price for the different matrix of things that I mentioned, we're going to run our own internal promotions.
Lila Hunt: All right. If we go to the next slide, we have an example here that we're currently working on; we haven't finished implementing this, but we cooked it up together. Here you can see we've got really obvious higher CPM on the 300 by 600, which makes sense because it's a large format unit. Even though the 300 by 250s have lower CPMs, we're look at potentially breaking up the 300 by 600 into two 300 by 250s, and again we can look at A/B testing this to see which configuration actually drives more revenue and more demand density. This is an area where a 300 by 250 may...there may just be more campaigns available, so there could be stronger fill; there could be maybe not necessarily stronger CPMs, but more top-line revenue.
Lila Hunt: I think that's an example of how we look at demand density. We may look at participation rate as well from different bidders on those units to see how they're performing. Do you have anything to add about format sizes and testing different ad layouts?
James Murphy: No, this is a great example. We're looking at exactly that. The relationship between CPM and fill, obviously equates to the revenue that we're going to earn. So how do we maximize these things? You mentioned that we see units, you're going to have higher CPMs, most likely less fill, so what's that rate balance? I think that's what Sortable provides. They give us a kind of outlook on that. Like you mentioned, Lila, through A/B testing and understanding, great, of course we want the highest CPMs, but we also want to monetize more of our inventory, so what do the actual buyers want? We want to fill our inventory at the highest rate, but they're going to fill more at X rate, so should we focus more of our efforts there? It's really about that balance; it's not one extreme versus the other. It's finding that right mix or right recipe, if you will.
Lila Hunt: Right. That's a great segue into the next section of the webinar. We have done a lot of work after we partnered through the end of the year into Q1 to really get the inventory optimized and performing well, and now we're looking at ways to, again, influence the demand landscape. I think, to start the conversation off with respect to demand, I wanted to look at transparency because it's a huge theme in our industry. It's a really, really often used buzzword by a lot of different types of companies. Everyone wants it. Many claim that they're transparent, but what does that really mean?
Lila Hunt: Would you be able to talk about what transparency means to you and why it's important when you're looking at demand?
James Murphy: Yes, of course. It's extremely important, and the two key things that we look at around transparency is, first and foremost, the fee structure. What we've learned, myself coming from the platform side, is, even though your rev share may be 85-15 or 80-20, whatever, what they give in exchange, what is that based off of? What is revenue, and how is it defined? We, first and foremost, get clear definition around what gross revenue means and have a rev share based off of that that we both agree on, and then really around the demand base. Who's monetizing our inventory? Gone are the days where it's a black box of this exchange and we don't know what's behind it. We obviously mandate that we get full transparency in the demand mix, meaning the DSPs behind the exchange and even further beyond that, the actual agencies and brands. What we love about Sortable is they provide that whole chain. You hear the big buzzword about SPO and how do we maximize our efforts on the supply side and so forth, but it's okay obviously working with all the different players in the industry, but we need that transparency. As long as you have, going from our edge being the publisher, all the way to the demand side, and knowing what fees are being taken in the middle and who's actually monetizing our inventory, that's what we're comfortable with and that's what we mandate. To us, that's what transparency means.
Lila Hunt: Yep, beautiful. We definitely are huge advocates of transparency. I think one of the things we believe really strongly in here is that log-level data is really important to figuring out different pieces of the puzzle of how dollars are flowing, and that's one of the services we provide is we make log-level data accessible and easy for publishers to access.
Lila Hunt: I think the other thing that often comes up with respect to demand and the demand landscape are market fluctuations. CPMs change, and it's often really hard to explain why, especially if there hasn't been changes on the publisher side in terms of inventory or user base or general products for their users. This graph is looking at the exchanges and how they're bidding based off of viewability. Viewabilities are really important metric to the DSPs. Matching what a publisher can offer to a buyer helps encourage the spend from the DSPs for that publisher. Often it's hard to understand what KPIs the buyers are trying to track against because they're tracking towards clicks or conversions, things that aren't always very easy to measure through a programmatic transaction. Here you can see the viewability by partner, you can see which partners are really spending based off of viewability and which partners, again this would tie back into which partners are the largest revenue partners. Do they care about viewability? If they do, then it's really important to focus on. That starts with...a user can't click on the ad if they can't see it. A user's not going to satisfy a view through conversion if they can't see it.
Lila Hunt: Can you talk to us a little bit about your conversations with buyers, the metrics they care about and how you present the value of your inventory to buyers who may be interested in spending directly on your site?
James Murphy: Obviously viewability is a big one, but beyond that, given that we control large number of users, I mentioned 20 million monthly active users, we know a lot information about those users. We have a highly sought-after audience that people want and it's very monetizable in the advertising community and we have the ability to segment that out to offer that to the demand side by a private market place deals, direct deals that we just plug into an ad server that we use, for example. Really what we like is obviously moving those connections to server side type connections, which Sortable helps us with and gives us that connection to talk directly to buyers to set up these private deals.
James Murphy: The obvious things are setting up buys based on demographic data. Like I mentioned earlier, we know simple things like age and gender, we can infer ethnicity; we have lifestyle data on our users and so forth. Coupling that with viewability metrics is very powerful, and that's how we're able to get high CPMs. Having that transparency and the ability to talk directly to the demand side, know what they want, and we have the luxury of having a lot of information about our users and have data to make our inventory available on the KPI side of things, like viewability and so forth, to make it very powerful.
Lila Hunt: Great. When you have these conversations with buyers, or you're engaging buyers for these unique opportunities to spend on your site, are you finding more consistent CPM performance or more consistent spend that you can depend on as a result of having those strong agency or strong relationships with the buy side?
James Murphy: Yes, absolutely. Obviously if it performs, people need inventory that index at a high rate or whatever demo they're looking for. Our inventory skews heavy on the millennial side; on the ethnicity side we have a lot of inventory that's African American and Hispanic that's very sought-after in the advertising community. We see consistently high CPMs in those categories. When you index high in areas that matter to the demand side, they're going to continue to buy at high CPMs and at a consistent basis.
Lila Hunt: Right. Makes a lot of sense. And I think, through our platform, we can see through different demand sources, who's spending, what campaigns are spending, what the price points are that they're bidding on, and use, again, data to help understand what might be attractive to a buyer.
Lila Hunt: All right. We talked quite a bit about the buy side. In your experience...and I know this question comes up a lot to me, but how do you get started with the buy side? How do you approach a buyer? How do you open the door for yourself as a publisher to start having those conversations?
James Murphy: It's a great question. How we start those conversations. As you know, for the most part, things are changing consistently in our industry, but for the most part everyone is very good monetized by exchanges. Again, that's shifting a bit as of late. What we do, like I said, first get transparency as to who's buying our inventory and who's not, and then what we've done is we go directly to the demand side and really start with the DSPs who obviously control agency demand that represent brands. We just go in and educate them. In the spirit of being transparent, we share with them what we're seeing as a publisher and how duplication is happening, and how do we eliminate that. Quite frankly, Sortable helps us do that in a great way where a clean path from the publisher directly to the demand side, and make sure they know exactly what they're getting and there's transparent fees in the middle and all that good stuff.
James Murphy: To answer your question, it just starts with education. We're not trying to sell them anything. Here's what we're seeing; here's how we can help you. Did you know this is happening? Here's what our user mix looks like. Here are the things that we think they matter to you. And then it naturally evolves into a commercial partnership from there. It's really about education and offering that transparency and giving them visibility into things that they may not have seen on the demand side. We have that luxury obviously on the pub side to give them that data.
Lila Hunt: Yeah, amazing. That's super valuable insight and I think that's where having strong data as a publisher, and the ability to present that data in ways that buyers can understand, is really, really helpful to support those conversations. That's something that we like to help publishers with so that, again, we can just help increase their access to demand generally once we've finished working with them on optimizing inventory.
Lila Hunt: All right. I think we can move through to the recap. To summarize, we really look at starting off at setting very high level goals and looking at the trends at a very high level. We look for changes that diverge from what is expected or what may be necessary to track towards a broader corporate goal. As we dig into those trends more granularly across different dimensions, we can identify opportunities to optimize inventory as either traffic changes or as the demand landscape changes to preserve that demand density. Once we understand that the inventory's performing optimally, there's a lot of different opportunities to start looking at different packages for partners. We can look at how the partners are spending across different segments of inventory or types of buy side KPIs that the publisher can measure and then package that so that publishers can start having conversations with their agencies, and then from there the publishers and the agencies can start transacting on direct deals and measure the direct deals, talk about the performance of those deals, and then that leads to natural conversations about heavy up budget. As you get towards the ends of the month or the quarter, you can go back to the agency and say, hey, you've got these campaigns and they're spending, they're doing great. You have budget to spend.
Lila Hunt: I think one of the conversations I often hear on the buy side is, they're saying, we have money to spend, we're just looking to spend it. Publishers are like, hey, I want money to spend with me too. There seems to be this huge gap between publishers who are looking for agency relationships or direct relationships and buyers who are looking for opportunities to spend their budgets. I think look at the whole package of why we put a lot of effort into optimizing inventory, why we look at having a really well-optimized and transparent demand landscape leads to having really well-supported conversations for really stable, consistent, and high CPM budgets down the line.
Lila Hunt: James, do you have anything else to add before we open it up to questions?
James Murphy: No, I think that was a good summary. It's really, again, I think where the industry is going is just sharing that transparency. Everybody has the different [inaudible 00:43:20] things. Us as the publisher, we know exactly how people are offering our inventory up into the open market, where the demand side may not know that because things like publisher IDs are obfuscated, or what have you. I think just sharing that information and educating one another is key. I think the industry is going to continue to evolve. Everyone thinks there's transparency around things, but I think there's big problem with duplication and all these things. Obviously I'll summarize by just saying we love working with Sortable because the things that are important to us, around fee transparency, demand transparency, we absolutely get that. The logic that they've has just been phenomenal. We've obviously seen great results and we're just looking to expand the partnership. That's really it from me.
Lila Hunt: Sure. So we have a couple of questions. One of the questions is from our conversation on refresh rate, and they're asking if we decreased or increased refresh rate to improve CPMs.
Lila Hunt: We definitely decreased the refresh rate, and what ended up happening was we saw CPMs increase, inventory decrease, and revenues stay relatively stable. But because we had lifted the CPMs up, as demand density started to increase as the quarter started to gain momentum, we were able to then increase inventory back up to increase total revenue. Decreasing inventory did not actually negatively impact revenue.
Lila Hunt: Do you have anything to add to that, James?
James Murphy: Just going back to the point I made earlier. Everyone's under the assumption that more equals more, and that's just not the case. That proves that. We're obviously a firm believer of looking at data and A/B testing things and basing your decisions off of deterministic data and not just on a hunch.
Lila Hunt: There's another question that I think would be a great question for you because of all your experience with the buy side. Somebody asked what is an agency heavy op? Maybe you can talk a little bit about how agencies plan budgets.
James Murphy: It's usually a quarter out and they're looking for a statistic target. From our experience, and my experience from working on the platform side, there's always some kind of metric they're looking around when it comes to video or even the sway around things like viewability or obviously CPR and whatever metric that they're looking for for that specific campaign. There's always a certain user set that they want to reach. So if it's a Axe deodorant campaign, they're obviously going to target to males, for example. We have the ability to segment out our audience. I think that's key. Really going back to the planning side, there's always discretionary budget. What tends to happen, using that Ax deodorant example, they can't find enough of the inventory to satisfy that campaign based on the viewability metric that they have to hit, for example, and the demographic that they have to show it to. You're always going to find that if you have inventory for whatever campaign that's not delivering in full, there's always discretionary budget, so having that communication and that connection to the demand side is very important and just constantly sharing information of what's available, what's changed, and so on.
Lila Hunt: Great. There's another question that I think I'll cover because it's a platform question, which is how do we determine the viewability of an ad unit.
Lila Hunt: Maybe I'll start by talking about how our platform determines viewability, and then maybe you can describe a little bit about your viewable refresh solution.
James Murphy: Sure.
Lila Hunt: We have our own proprietary viewability measurement that's based off of the IAB standard for viewability. The reason we did this, quite honestly, because we didn't want to pay extra money for viewability solutions. Our viewability solutions in the market, which you can pay for, but it's just another extra cost that the publisher has to cover. The publisher usually has to cover it, again, to satisfy relationships with buyers. What we can do, based off our viewability data, is surface deals through the DSPs that are based off of a threshold of viewability, so we know when the impression loads if it's viewable or not.
Lila Hunt: We work with publishers on techniques like lazy loading that waits until the ad unit is going to come in view before we start making more requests for bids from the exchanges. All of these techniques help lift the top-line viewability. Generally what I find is there is a difference between exchanges or demand sources who are looking at site level viewability because that's what their capability is as a buyer. They may be looking at it based off how their IDs are broken out, and that varies from publisher configuration to publisher configuration.
Lila Hunt: And then you have partners like AdEx, who actually are looking at ad level viewability, and Google, because they are the ad server, who knows what the viewability is of each individual ad unit. So there's different levels of granularity, and I think really to maximize opportunity for publisher and really push CPMs as far as they can go, looking at the top-line viewability covers all of the bases and make sure all the demand partners are getting what they need, regardless of what their platforms or what granularity they're able to evaluate viewability by.
Lila Hunt: James, can you talk a little bit about your viewable refresh, because we kind of hinted at it but didn't really go into detail. I think it was a feature of the case study, and it's really cool.
James Murphy: Of course. It just goes back to the point I was making. You have to be smart about how you make inventory available. Make more inventory available to buyers that they actually want. All buyers want viewable inventory, so if you have inventory that's above X percent viewability, that's typically above 60%, you make more of that available through refreshing of ads at a higher rate obviously within the IAB standard. That's what we've done successfully as a grade solution for that, and we really maximize our efforts by doing just that and then really using the guidance from their team based on the logic that they have within their platform to say, here's what makes sense. Obviously it has a high probability of being viewable, we've seen that through data, and that becomes deterministic data to know. Okay, these ad units are at a higher viewability rate, make more of that available to the buy side, and they're obviously going to buy it and we sell just that.
Lila Hunt: Great. There's another really great question that follows this well I think, which is how much time is long enough to let a test run before evaluating if it had a positive or negative impact.
Lila Hunt: I think this question's really important because sometimes we do see a dip or a neutral performance trend before we actually see a lift. Generally, I look at the bidders or exchanges as having 3- to 5-day optimization cycles. It takes time for their algorithms to adjust to changes, whatever it is. There's a few really important things when executing new strategies. One is to only execute one strategy at a time because if you're changing a bunch of things all at the same time, you don't really know which test had an impact.
Lila Hunt: Another approach that we take is we segment the traffic usually at 50-50 because at 50-50 it's really agnostic. We'll measure the results of a change on 50% of traffic with it on and 50% of traffic with it off, and then we can see across the different dimensions in metrics how the performances on side A and side B. From there, sometimes when we scale up that change to 100%, it takes time again for the bidders to re-pace and adjust, and then we see these gradual lift over time across the different sources. They all start influencing each other. Once one of them re-paces and starts bidding more, the other ones may lift and start bidding more, and that all snowballs over an amount of time.
Lila Hunt: Generally I try and leave tests running for at least 7 days before making a decision. Sometimes if it's neutral, I'll let it go for a little bit longer. If it's negative, I usually give it about 3 days at the very least before deciding that it's not going to work and turning it off.
Lila Hunt: Do you have any feedback, James, on the risk associated with running these tests and bracing yourself for the unknown when you're testing out a new strategy?
James Murphy: Very well said. Definitely don't have overlapping experiments. That's the common mistake. Test one thing at a time. Really how long do you test for, it depends. It depends how sensitive you are around certain things. Like you were saying Lila, if you do a 50-50 test, that's very different if you have the control of 90% of your users and then only 10% as a variance and you want to be a bit more careful around testing things. I couldn't have said it better. Just make sure you're not testing too many things at once, understand the impact, and then obviously base your decisions off of that output. That's my feedback there.
Lila Hunt: All right, I think we've covered all of the questions. We're coming up to time, and I think we can probably look at closing.
Lila Hunt: I just wanted to thank you so much for participating today. It was really, really nice to have you, and I hope this was a interesting conversation for everyone on the line. I'm really excited for this year, and all of the next optimizations we have planned.
Lila Hunt: I'm going to kick it back to Craig now to...
Craig Ling: I wanted to thank all of the attendees for joining us today. I thought it was a really awesome, really some great information in there. I wanted to let everybody know again that we have recorded this and we'll send a link out to everybody that registered so they can come and view the webinar on demand.
Craig Ling: James, do you have any final thoughts you wanted to close with on?
James Murphy: Thank you for having me. I hope this was useful. I enjoyed doing this. Please reach out to me if I could help with anything really and provide transparency and a view from the publisher side. Thank you again, all.
Craig Ling: Thanks everybody.