Digital publishers are always striving to optimize their web properties. In this webinar, we discuss real examples and industry trends that publishers should be paying attention to in order to maximize their display ad revenue and UX.
You can hear from Chris Reid, CEO at Sortable.com, and Joshua Mendelssohn and Jeff Myers, Co-Founders of Factinate.com as they share their learnings and insights in the complicated and ever-changing ad tech industry. You can learn from industry veterans on challenges facing digital publishers in the current age, like
If you need help implementing any (or all!) of the ideas discussed in this webinar, contact us at Sortable.
Subscribe to our monthly newsletter or book a demo with us to learn how Sortable can help.
Craig: Hello everyone, and welcome to today's webinar on optimizing revenue and user experience. My name is Craig Ling and I'll be your moderator. I work on the monitoring team here at Sortable and I'm really excited to be hosting this session today. Co-moderating will be Jamie Murphy, VP of product here at Sortable.
Jamie: Hi everyone.
Craig: I'm pleased to introduce today's speakers, Chris Reid, Jeff Myers, and Josh Mendelssohn. Chris is our CEO at Sortable, and today he's going to share trends, themes, and insights from working with hundreds of publishers and having access to massive amount of data. ... Sorry. Jeff and Josh are the co-founders of Factinate. Currently in their second wildly successful digital publishing venture. Jeff and Josh are going to share their learnings as publishers in the industry and things they pay attention to to ensure they are maximizing their earnings.
Craig: Before I hand the mic over to Chris, I have a few housekeeping items to cover about this presentation. First we will be recording today's webinar, and everyone registered will receive an email within a few days with a link to the recorded version. We'd love to hear from you. If you have questions for our speakers, please feel free to send it through the Q&A tab in the webinar's software and we will try to answer as many questions as possible at the end of the presentation. So without further ado, I'd like to kick things off by welcoming our CEO, Chris Reid. Chris, over to you.
Chris: Thanks, Craig. That's great. So, I'm really excited about today. I'm just amped that Factinate can be here with us in our office. I think growing any business is really hard, and the guys at Factinate have really done an exemplary job. And so, as a bit of a serial entrepreneur myself, Sortable's my fifth startup, I just have a lot of respect, but I have sort of twice as much respect because my last business was a publishing business and really, being a publisher myself, really formed the impetus for why I wanted to start Sortable. And I'll give you a little bit of information about Sortable.
Chris: Again, I started because of the pain I felt as a publisher, and I really wanted to go out and build an agnostic and transparent company that really empowered publishers and had a core focus on helping them grow revenue, and there's really three key things that make up what Sortable does. And take a look at the next slide here. So, the three core focuses of Sortable. The first one is our client-side container, and the whole point of the client-side container is publishers need to do a lot of things on-page. They need to do real-time flooring, they need to handle bad ads, they need to be able to execute A/B experiments, there's a lot of data capture that needs to happen, you need to monitor viewability, you need to monitor bid-level data, UTMs, URLs. So, there's all that data capture, there's things like manage refresh, native [inaudible 00:03:15]. And so really, what we wanted to do was build a client-side container that made these things unified, they work all well together, and it works well with all the other platforms like Amazon.
Chris: The next thing we wanted to do was we wanted to have a data warehouse where all this really important data can be stored because we're huge fans of data and we don't really think we can run a business as optimally without the data. And so, to us, storing all this bid-level data and providing publishers with query tools, advanced analytics, allowing them to do yield and header bidding analysis, monitor of viewability, report on key-value pairs. And we'll get more into analytics as we go through this presentation, but data and analytics are the key part of what Sortable wanted to deliver to publishers. And then, the last thing that really makes up the third leg of the stool so to speak, is Sortable working directly with agencies and DSDs to deliver significant and different shaded demand to our partners. ... So, now I'd like to get the guys from Factinate, please ... Again, welcome. Thanks for coming today guys, and would love you just to share a little bit about your business.
Josh: Thanks, Chris. Josh here from Factinate. Just to kick things off, we're really, really happy to be here. Special thanks to Sortable for asking us to participate in this webinar. It's flattering. For the folks listening, probably useful to know a little about us. So, Factinate is, as the name suggests, a publisher of facts. We write on a variety of topics. Humanity, such as history, articles like, "46 Facts About Ancient Rome." We also cover a lot of entertainment and pop culture, whether that be facts about Friends or facts about The Sopranos. We're fairly new. Factinate launched January of 2017, and we've grown over the past two years. Two quite volatile years I'd say in the online publishing space. Now, a bit about us by the numbers. We're up to 14 employees. We do six million monthly visitors, 200 million ad impressions, and we've also had quite the journey on the Adups side, throughout which Sortable has been an incredible partner, and definitely get into some of the details there on the next slide. ...
Jeff: Thanks, Josh. So, I think by wildly successful, Craig's referring to the team that we've been able to build and lucky to work [inaudible 00:05:42] Factinate. But Sortable certainly helped us get there. We started working with Sortable as we were beginning to pick up steam, and we certainly drew the attention of malicious advertisers. It became a huge problem for us. Another hurdle we were facing is getting in touch with premium advertisers. Really hard for a small site, but since working with Sortable we've seen a [inaudible 00:06:02] increase in session value, a 60% increase in eCPM, and essentially doubled our page RPM. And over the next 30 minutes we're gonna talk more about how Sortable's been an extremely important ad partner and also strategic partner for Factinate. ...
Chris: Okay great, so the first thing we wanted to go through today is we wanted to talk about viewability, and I think viewability is something that became a hugely hot topic in 2017, that's continued through '18. It's still there and it's very much relevant. I think it's important to help everyone understand that viewability is all about what advertisers are doing, right? So, on the backend, the buy-side, they have requirements for certain levels of viewability, and if that's how they're buying, then the inventory you're selling is going to be impacted by that, those buying behaviors. And so, if you fall below these advertisers' sort of requirements, you're not going to be eligible for these higher paying campaigns. And there's a lot of things you can do, right? You can look at lazy loading, you can look at A/B testing, [inaudible 00:07:19], and you can really analyze your data and analyze your site performance and your UX to drive through these things, but let's take a look at some more details around viewability. ...
Chris: So, I think what we've seen is ... Again, the last slide sort of mentioned that there's cliffs, right? So, at some point you start to unlock an advertiser's dollars, and it really depends on site, it depends on ad unit. And so, there's no magic formula here, but there's generalization, and some of the generalizations are that definitely for certain sites and for certain ad units we can unlock higher CPM almost like a [inaudible 00:08:02]. It's not perfectly true, but we've definitely seen it. And what we're looking at now is a graph of a publisher where the step function was absolutely true, and as you can see, as viewability increased due to UX changes onsite, which we sort of A/B tested and worked through with them, you saw CPM lift right away, and then as bidders responded to that increased CPM or that increased viewability, CPM's continued to rise.
Chris: And the next slide, we can take a quick look at a bit of a scatter plot. Scatter plots are not the best way to curve fit a step curve, but this gives you an idea of distribution, of how are different viewability levels affecting CPMs? And one thing that I think you can see is that between sort of 55 ... You can start to see there's a lot of impressions that, say that low CPMs, and then they start to jump up, and so you can sort of see this viewability threshold, the sweet spot is between 55 and 70% where you can unlock these higher paying campaigns. And what Sortable really wants to do ... The reason why we track viewability at a bid-level, is so that you can really unpack and analyze viewability from any facet. And by being able to analyze viewability across any facet, you can then optimize and test across any facet, and we think that's really important. But I think what's best here is ... Let's talk to Factinate about this because you guys have some real-time experience, you've done a lot of testing around viewability, and you have a great story about how viewability unlocks additional sort of revenue opportunities for you.
Jeff: Yeah, we certainly do. In the last year viewability's become the metric we're hearing the most about from advertisers. So, advertisers don't just want to see a high volume of impressions, they want to see a high volume of highly viewable impressions. And in our conversations with our ad partners, we've heard that 70% is that level where you're really gonna unlock the PMP campaigns or maybe that higher tier of demand. In some cases, you'll see almost a point for point correlation between increased viewability and increased eCPM, but in many cases, as you sort of showed in the last two slides ago, there might be an initial increase, but then it takes time for advertisers to get used to seeing your ad units at that level, and then align their buys towards your better performing units. For us, when we switched, we switched from a page native layout to an only single page layout where every ad is lazy loaded, we saw a massive increase in sitewide viewability. We jumped up from around 50% to everything being over 70%.
Jeff: But for us, our eCPM's actually decreased as a whole when we moved to single page. We have some theories as why that might be, but we think this provides the best user experience, and clearly displays ads, increases intentional clicks, which actually provides more value to our advertisers. So for us, we're gonna keep investing in higher viewability, and over time we hope that our advertisers will reward us for that. Our articles are also extremely long, so we have some articles that run 8,000 words, so lazy loading for us was a must. I mean, you can imagine how if you load an ad at the bottom of an 8,000 word article, it's probably not that likely to be viewable. But for us working with Sortable, it's helped a ton using their analytics, and running traffic to different layouts let us choose the layout that we found the best balance between user experience and viewability. And you have to make some sacrifices, but we think overall this has been the best for us.
Jamie: I think guys it's interesting, you mentioned UX and user experience and balancing that. So, it's not just about viewability, but it's also about how you drive longer sessions, how you keep users engaged throughout those sessions. Can you talk about some of the trade-offs you made between UX, viewability, and also sort of session dynamics?
Josh: [crosstalk 00:12:10]. Oh.
Jeff: We certainly can. I think what we saw, which I should've mentioned probably, is that as a result of the better user experience in running a single page layout is users are actually seeing more ads, and as a result our session RPMs are actually increasing. So, although eCPM might have taken a little bit of a dip, it was a higher session value for us, and that's really what we're looking for. We're looking to drop the most value [inaudible 00:12:36] on the advertising side, and also provide users with the best experience, and we're able to nail that.
Josh: Yeah, absolutely. And there's also things ... In terms of the UX viewability trade-off, there's common tactics that get used that perhaps are not the best user experience. And so, for example, as users are scrolling down our website, our ads do not stick. But if you did make them stick in viewport, for say 200 pixels, that probably would increase viewability. How much would that increase our CPMs? Well, it's tough to say, but these are sort of the things that we wrestle with, right? So, we might make that change, see how it works for a week or so, and if we see a substantial change, well we have a decision to make, and if not, we would roll it back.
Chris: Yeah, no it's definitely interesting, and it's hard to make these decisions without the data, right? And so-
Chris: Yeah, we would definitely say impossible, so I'm glad you said that. So-
Josh: [crosstalk 00:13:33].
Chris: You can try.
Jeff: [crosstalk 00:13:35] successful.
Chris: Yeah, I mean this is the thing, right? We're trying to remove guesswork out of the revenue generation process. And so, another error where we've applied a lot of process to is bad ads, it's ad quality, right? This is another opportunity for data drip and insights. And so, first I'm gonna talk a little bit about ad quality that's subjective, right? One publisher's bad ad is another publisher's gold mine. In-banner video might fit that narrative. But every publisher has different needs, they have different audiences, they have different preferences on what they want their users to experience. And so, this is the area of subjectivity, and we view our area in the role of subjectivity is to really set up, maintain, and enforce.
Chris: So, if we're providing a managed service, then we're going to really help set up the blocks, whether they're URL level or advertiser level, set them up across the DSP, set them up across exchanges, and then monitor and enforce. And that's an ongoing effort. And we also work on letting users submit when they see a bad ad, right? So, getting user signals from the site level that, "Hey I didn't like this ad," and gathering some data around that so that you can really address that subjectivity. Because maybe it wasn't a malicious ad, but it was ... Maybe you run a clean eating site, and your user is really offended that McDonald's was serving on it. Fair enough. If that's where your preferences fit, then we're going to support you then.
Chris: And so, that's a little bit about the premise, but maybe we can talk a little bit now about how do you deal with that? Because it's really tough in an environment. You have control of your website, but you don't have control over the serve environment as a publisher. You don't have control over the creatives. So, one of the recent introductions in the industry is ad monitoring, and we're huge fans of ad monitoring. At Sortable, we've always had a ticketing system, right? So every bad ad, every ad quality complaint goes into our ticketing system, and so we've had this really good ... We've had data around what bad ad complaints look like. What's the prevalence of them, or how often are they occurring? And Confiant who is probably one of the first companies to market with a product, we got them live, and we started implementing their software where they would block bad ads, and we were able to see a 46% reduction in tickets into our system for bad ads.
Chris: So for us, this is great, right? For the publishers, it's great. We're seeing a much lower prevalence. But to take this further, right? To go back to data, if you log every single one of these bad ad instances, you can do more. And so, that's what I want to talk about next, is how do you do more? ... So, right now this is data's looking at the malicious IBV and audio. So, instances of different types of bad ads and the block rates, right? So, block rates indicates the prevalence of them, prevalence of attacks on your site. And you can see that over time two things are going down. The base loads, so the base amount of blocking is going down, but also the severity of the attacks. Any time you see a spike, that's 'cause someone's attacked.
Chris: A good example is on Labor Day weekend, when the bad guys think everyone's left work, right? Everyone's at home drinking a beer, boom, that's when they come with the attacks. So, you see these spikes on Sundays, on Saturdays, on holidays when everyone's meant to be at home. And so, those are the spikes. But what you can do is if you're collecting all this data, and what we do at Sortable, is we use this data to track down bad creative IDs. We use this to institute blocks, we look at the underlying SSPs, we look at problematic exchanges, and we accurately initiate blocks. We talk to partners, we block bad creatives, bad advertisers, and we continually push down this route, and that's what's contributed to our accounts' continual decline in bad ad instances. But you guys have Factinate, and I really want to let you talk now 'cause I've talked way too long. You've had your own really interesting experiences with bad ads, and from a publisher perspective, can you speak to how that's affected you and your business?
Josh: Definitely. I mean, we used to get absolutely killed by bad ads. There was a time where we were seeing three to four attacks per month, and honestly it's hard to even explain just how damaging these attacks were. I mean, just in term of earning potential, we'd see our revenue per user cut in half while we were being attacked, we'd see time onsite cut in half, but there were also some other major concerns. I mean, it would alienate users, we'd start seeing Facebook comments, people saying, "It's a stupid spam site." And by the way, what I'm talking about here are really the malicious ads. These are mobile redirects where users gonna land on factinate.com, and then they're going to be told that they won a million dollars or an Amazon gift card, but newsflash, they had not. But they were certainly very upset. And it also creates a policy risk for us.
Josh: So we do Facebook marketing, we're expanding our audience to the Facebook ads platform. The Facebook policy team ... Even though we're not trying to redirect anyone, we are held accountable. If people are getting redirected when they're hitting our site, it's our ads that will get disapproved, and it's ultimately our accounts that are at risk of being in bad standing. This is one of the main reasons we switched over to Sortable. We know that they've had an ad quality team that was very dedicated. And that is true, we used to see attacks, and we would submit tickets, and their ad quality team would block these bad actors. But it's sort of fighting a losing battle still. The manual process, it just isn't enough because these creatives can continue to come through, you block it from one DSP and it finds your site through another DSP, or these bad actors just find a new way to continue to attack you.
Josh: It was really ad blocking scripts that started to make a huge, huge difference. We work with Confiant through Sortable's partnership with them, and they've been awesome. We're also working with a company called Clean Creative and really, since we started working with them, these guys are blocking these scripts and these bad ads in real time, and we've seen a decrease from maybe three or four attacks per month to getting attacked once every two months and for a much shorter duration because we're able to ... Let's say I email the Clean Creative team, and [inaudible 00:22:01], "Hey we're getting attacked right now," and they'll find the vulnerability, they'll block it, and things will return to normal. And this is critical, getting this under control. 'Cause not only is it so damaging for all the reasons I discussed, basically it also can destroy any tests you're running. If you're seeing what improves viewability, then suddenly you get attacked by a bunch of bad ads, well all your metrics are gonna plummet and basically your data is no good. ...
Jamie: Cool. There may be ... So, I think next ... We showed that graph with spikes, right? And talked a little bit about when a bad actor is initiating an attack, you get these huge spikes in volume. And I was wondering, why don't we take a look at the anatomy of one of these bad ad attacks? Do you guys wanna share your thoughts on what you see as publishers when one of these attacks sort of initiates?
Josh: Absolutely. So, you'll see here this spike in blocks. And what this is, is somebody deciding, "We're gonna target your site, we're gonna start redirecting users." And now, Confiant or Clean Creative steps in and they start blocking these attacks. Well, now the person who's attacking your site needs to adjust their strategy, they see that they're not actually making money attacking your site, and so they start to change and tweak things on your end. And then, they'll try and basically continue to attack you, and you call these "probing attacks", let's say from 13 to 22 on the graph there. And then eventually, they give up and they go and attack someone else, which is what you want. ...
Chris: Yeah, and hopefully everyone has a strong solution and the bad guys give up, or they have to regroup and come up with their next move.
Jeff: Yeah, I think that's the problem, is that they're always regrouping, coming out with the next move. So, it's on the guys like Confiant and Clean Creative to constantly be innovating, coming up with new ways to block those scripts, and it's extremely challenging 'cause the guys that are doing this are also very sophisticated. So, it's a constant battle, and you just have to route it out sometimes and hope for the best result.
Chris: Yeah, it's really fascinating. I mean, this sort of cat and mouse, it's never-ending. We were having lunch with the Google malware [inaudible 00:24:18] team and just talking about the different attack vectors and what things look like. And it's really fascinating, the bad actors are just continually getting more and more sophisticated, and as a publisher, I guess ... I think this is just a general theme, is how do you do all this yourself as a publisher, right? Are you gonna be an ad quality expert? Are you gonna be a big data expert? Because things are so hard, right? Because marketers are using such sophisticated tools, you now have to understand how marketers work because bad actors are using such sophisticated tools, you have to ... It becomes difficult, and I'm glad that at Sortable we've been able to help take some of that burden off of you guys' shoulders and let you focus on your core business.
Jeff: So do we. I mean, there's so many things to focus on. There's so many different players in ad tech, and they're all experts at what they do. And so, for a small publisher like us to be able to lean on our partners and not have to bring those people in-house, I mean we couldn't have done it, we couldn't have afforded all of those different specialists, so it's so important for us to have partners like this that can help us out.
Chris: So, I think the next thing we want ... We've been through viewability, we've been through bad ads. We want to talk about demand. We want to talk about sort of advertisers and why having advertisers is important. I think when you say it like that it sounds pretty obvious, right? If you have no advertisers, then there's no ads, and so what you need to do is you need to then think about, okay how do you have sufficient number of advertisers? How do you add demand density and unique and high quality demands so that your audience is being monetized at an appropriate level, right? Commensurate with the work you've put in as a publisher.
Chris: And so, I think a huge part of driving revenue is that demand density and making sure publishers have access to advertisers to bid on their inventories is critically important to us at Sortable, and so we always took this approach where we're gonna be an agnostic platform whose sole purpose is to transparently connect publishers to advertising dollars. And that sort of agnostic mission is why we're Google certified publishing partner, why we are one of Facebook's launch partners for Audience Network. It's why Amazon works closely with us and why their tech's integrated with ours. It's because we don't have a horse in the race. We're not demand biased. We really just want the best outcome. And I think part of that outcome, again, it comes back to technology, to generate really good revenues that results ... Even from a demand standpoint, there's a lot of technology involved. Part of that technology is your header bidding option, right?
Chris: So, at Sortable we became a member of Prebid.org because being a member of Prebid.org backs up our agnostic approach, right? It means that our publishers can work with anyone because everyone works with Prebid. It's the dominant auction ware in the world right now. And this is why we share on the mobile committee for Prebid, and it's why we do work directly with agencies. But if you think about the technology, one of the reasons why we built our Container was because our Container really helps unlock more revenue. It's how do you price to demand partners? How do you push them? How do you make sure that they're integrated well? How do you make sure that the cookie syncs are optimized? We talked about viewability. Viewability is this other thing where you're literally having a conversation with your advertising partner, and that conversation is, "Here's my inventory, here's what it looks like. Do you want to buy it?" And so, this sort of dance you're doing back and forth with demand partners requires a lot of tech for you to sort of execute well. And I'd really like to hear from you guys, how do you view demand? What's performing well for you? Where do you see really big impacts? Yeah.
Josh: Absolutely. Firstly, when it comes to demand sources, I recognize that what I'm about to say is not necessarily the conventional wisdom. A lot of experts will talk about how having four or five really premium demand partners can perform better than having a stack that hides, let's say 10 to 12 demand partners. For us, that hasn't really been our experience. For us, I would say having more premium demand bidding on our inventory has generally resulted in a higher eCPM. And Sortable's certainly been instrumental in bringing on these demand sources. We didn't have relationships with some of the people who are bidding on us now, Index and Amazon, some of them can be pretty tough to get, and Sortable certainly helped us there.
Josh: What we've seen is that a single source can really move the needle, and this can be good or bad. When we brought on Index for example, we saw our CPMs increase by 10% basically overnight. However, this can also go the other way where if you, for whatever reason, a DSP starts taking less interest in your inventory, you can see your numbers drop off quite significantly as well. And that's not to say that we're not selective with who we work with. We certainly are. But we typically would only remove a demand source if we're seeing all of the below. So, low participation rate, low win rate, low eCPMs. And latency is actually a little bit different than that. It's possible that there are latency issues. That could be a reason to remove demand even if the metrics are strong, but basically what I'm saying is you need to be seeing quite poor performance from our perspective for it to make sense to remove the demand.
Josh: And just the last piece to touch on, demand partner approval can be a bit of a slow process, and especially if you're a newer site, you wind up running a little bit of a rejection gauntlet where you're saying, "Hey please work with me," and various DSPs are coming back to you saying, "No you don't have enough traffic history." And that's certainly an area where Sortable really helped us. ...
Chris: That's great. Thanks guys. So, I think ... What I want to go into next is I want to talk about analytics. And analytics and data has really ... It's really been the sort of backbone to the conversation to date, right? A lot of these ... Everything we discussed comes back to data. Talking about demand partners, if you're monitoring participation rates, you need to have the data to do that. You can also look at how does a partner's first bid compare against the second best bid in the auction to understand how much yield are they bringing to the auction. You can also do revenue lift analyses. But again, you can't do this without data, and we live in this data driven world.
Chris: There's no such thing as marketing without data anymore. Marketing is all about conversions and performance and attribution, and so if you're selling your inventory to marketers, you need to understand how they're buying it, and you need to have data to help you refine your strategies. So again, obviously Sortable, a big fan of data. We log a lot of data. It's our job to log a lot of data and to surface it to publishers really easily. We warehouse petabytes of data, so thousands and thousands of terabytes of our publishers' data sit in our data warehouse, and then it's our job to make that recognizable and serviceable to them. And so, we really take three ... There's three core approaches.
Chris: So, one is we have quick tools and dashboards, and those are meant to give you the high level answers quickly, the things that you need to know today, get you those answers quickly. Then we have query tools that allow you to dig down into lower level dimensions, right? So, if you want to look at authors, or URLs, ad sizes, UTM device browsers, you can run arbitrary queries and create arbitrary reports with our query tool. But then there's the next level, and the next level is really where you engage our data science tests, our technical services teams, where we're gonna help you answer ad hoc ... Do ad hoc investigations and answer complex questions against the raw bit data. And I think we can go through a ton of use cases, and I really want to talk to Factinate about their use cases. I just want to talk to just a couple of use cases that we're seeing a bunch of publishers find useful.
Chris: So, one is section value. Publishers really like this idea of, how much are my users worth? What's the lifetime value of my user? And that's a pretty interesting one. We talked about measuring exchanges by the lift they're providing. I was talking to a publisher recently, we were doing some dashboard training with them, and they had a really interesting use case where they deal with a lot of automotive content, and they wanted to look at CPMs for direct versus programmatic by brand. So, "For my Ford content, how much am I earning off direct?," which is probably directly through Ford, "And how much is programmatic willing to pay on that content?" And doing those analyses is really interesting because this data is useful to marketing teams, it's useful to direct sales teams, it's useful to programmatic teams, and also content teams. If you're tracking authors, and content sections, and which content is most lucrative or most engaged, it all comes back to data. And so, guys can you share with us some of your big use cases for data?
Jeff: Yeah, we certainly can. I mean, we use Sortable's analytics for a variety of things. Something that stands out for us is the UTM report, which has been extremely important for us in making decisions in a variety of different areas in Factinate. If you're not familiar with UTMs, there's an example on the slide there where you can see the initial URL, and then the UTM is gonna be appendant to the end of any URL. There you've got some [inaudible 00:35:08] source medium and campaign, but there's a couple others you can add in there and get really granular.
Jeff: Sortable has a complete UTM report which allows us to see our advertising metrics by any one of those parameters or by parameters in conjunction with each other. For example, when we promote articles on social media, we want to see how the different platforms perform, we want to see how different devices on those platforms perform for different geos, even different audience segments, and we can do that all using UTMs and Sortable's report. Tying that back into the viewability, this report is what allows us to see how our various layouts also performed on these different platforms and allowed us to optimize for the best advertising metrics. The reason why this has been so useful is because Google also works really well with UTMs. I mean, under the acquisition tab in Google Analytics you can see there's a source and medium report right there. So, for us it was super easy to match up what we were seeing in Google and the user behavior side along with the advertising that you'd see in Sortable, and then that's where we're able to make those optimal balance decisions between user experience and revenue.
Jeff: One of the drawbacks of UTMs though is that once somebody comes in on a session, you can't change the UTM ... Or sorry, once someone starts a session on a certain UTM, you can't change that UTM mid-session. You run into issues. So, for this we've been using Sortable's page segmenting tool, which we've been using to evaluate what happens after a certain action takes place onsite. And we've used that to evaluate a number of things on our site. Most prominently is probably our internal content REC which we've been building over the last couple months, and Josh can tell you a little bit more about that right now.
Josh: Absolutely. Yeah, so I mean we had some issues with external content REC, those widgets ... Just to have the context, I'm talking about the widgets below articles that typically say, "More from the web," and users click them and link out to other websites. We ran into a few issues. Some of the vendors that we tested put fairly restrictive contracts in front of us saying, "In order to run us, you need to not work with X, Y, and Z," which might not be the biggest deal, but also it might. If a company invents something that's game-changing and something that can really help our website, we don't want to be contractually forbidden from working with that vendor.
Josh: And then, the other piece that we ran into issue with was with Facebook policy. We basically found that our articles that we're running at these content recommendation widgets were getting disapproved more than the articles that were not. So, we set out to build our own and see how close we can come in terms of how much money they make to these external vendors. There were some challenges, it definitely took a little bit of dev resources, but what we were able to do, I think, was pretty positive overall. So, there are a few things that we basically set out to do. And the first of which was to improve the click-through rate of our content recommendation widget. So, we ran these standard widgets that you might see on most vendors, and then on top of it you'll see sort of this more social style thumbnail. And our widget onsite now looks like a feed of this, so it'll do the standard thumbnails, and then two social thumbnails, and then back to the standard thumbnails. And we were able to increase the click-through rate on our content REC really, really significantly.
Josh: And then, the next thing that we did is we embraced the ZergNet style two click model, where rather than users clicking on the article and going right to the article they clicked on, they go to this page which is designed to multiply clicks where they have to click again to continue in a new tab, and then the hope is that when they're finished reading they come back here and they see another article and they wind up reading two articles instead of just one. Overall it was fairly successful. We actually didn't think that we were gonna be able to match some of the external vendors, but we did. Of the three that we tested, our internal REC actually earned more money than two of them. The other ones still had a little bit of an edge on us, but it was close, so overall we were pretty happy and decided to go with this for the qualitative [inaudible 00:39:10]. ...
Jamie: Great, so just ... That's the end of the core content of the presentation. We just want to give you a bit of a recap, and then we're gonna jump into questions. I think that one of my key takeaways, and this is really sort of a philosophical one, is that continuous improvement is really, really important, and that you can move a mountain one block at a time. And I think you guys have experienced that in real life, right? You sort of worked at this every day in your business, continually improving, improving, improving. Where has that landed for you? What's the takeaway you would give a publisher in terms of, how do you go about doing that?
Josh: Totally. There are a lot of moving parts. I mean, I think if you just think about all the stuff we talked about today, viewability, lazy loading, ad quality, content REC, it's a lot and it can be a bit overwhelming. But it's key to remember that you're really just testing and trying things. Nothing is permanent. Each week if you just push one initiative forward, say this week we're gonna try and make our lazy loading ads load 700 pixels in advance instead of 500 pixels, then you just look at the data and if there was an improvement, great. Otherwise roll it back. And if you keep making these improvements one step at a time, you're gonna see a huge improvement over time.
Chris: That's greet. I mean, I think ... We just love working with publishers like you guys. It's quite a joy for us as a company who obviously is a big believer in data and a company who thinks you can't ... It's really hard to do these things without data. To have a customer like you guys where we can partner together and really work with you to run experiments and make a difference, that's pretty awesome for us. And ultimately, I think that to me, data is about transparency and it's about openness, it's about dealing with the facts, and then iterating. There's a lot of tricks being played in this industry, and I think that just continuous improvement eats tricks for lunch. It might take a week, but you get there, right? And then, those things have a lot of permanency to them. There's a lot of value in these sort of aggregate learnings. And so, we're just big fans.
Chris: We're glad you could come here today. We're running a lot of tests on our side with agencies and with other partners that we think are gonna generate really interesting outcomes, and those outcomes are gonna have very big results for our publishers. And hope to share more in 2019 what that looks like, but just again, I want to thank you guys for coming today. We're gonna take some time for questions now, and just to the audience, give us just 20 seconds while we get set up for the questions and then we'll jump in. ...
Jamie: Thanks guys. So, we've got a bunch of questions. I want to start with flooring. We have a few questions around flooring. Does flooring still matter in ... Do they make sense, as most networks now follow first-price auctions? So, I'll open this up to both Sortable and Factinate.
Chris: Sure. So yes, floors definitely matter. They still matter in a first price world. Bidders' behavior, it changes based on the floors passed to them. You think that they should just be bidding what they want to bid, but they are definitely influenced by floor. So, because that is part of their calculation and how they bid, an effective flooring strategy is really important. Now, just a word of caution, doing flooring well takes effort. You really want to dedicate a lot of time. At Sortable, the approach we take is we do per impression level flooring, where we're using historical and real-time auction data to craft our flooring strategy. We do a per impression per user, but there's a lot of approaches you can take that might be less sophisticated. Guys, any comments you have on flooring?
Josh: Well to be honest, from our perspective, I mean we don't control much of it on our end. The only thing ... We rely largely on what Sortable does. The only thing that we have done is we've implemented a hard floor where no matter what, people can't really bid lower than five cents on our ads. To be honest, this was really designed to prevent malicious ads. Back in the day, the thought that people who were attacking our site were likely bidding low. One of the things we're examining is actually removing this because now with the scripts that we're running we're able to block those ads anyways, and we're not sure it's actually resulting in any CPM lift.
Jeff: Yeah, I don't think ... When we talk about that floor, some people are like, "Well that's just not a flooring strategy." And we sort of understand that, and so what we've been doing is looking at internal initiatives, seeing what sort of ... And trying to value those post-initiatives, and then we're gonna put a value on that, and try to run through Sortable self-serve ad platform will run some of our own in-house campaigns. So, that's probably where we'll handle the flooring. And then other than that, it's largely you guys. Something we've heard from other publishers is that if you don't start off with a flooring strategy right away, as soon as you start flooring publishers ... Sorry, your advertisers, you might see that their participation dips a bit, but in the long term it's probably a good thing to have those floors. Once they take it into account and they decide that your inventory's worth what you're flooring it at, I'm sure they'll come back.
Chris: Yeah, we prospect different strategies all the time where we're able to measure, how does this flooring strategy work? 'Cause we'll have the flooring strategy on X% of the time, we'll have it off Y%, and so we always have a control where we can measure, and we're always looking to improve that strategy with different controls. So, we're gonna go to the next question Jamie?
Jamie: The next question is how do you implement lazy load ads on your site?
Josh: So, I think there are a number of things to consider here. I mean, basically the two variables that I think we've played with most are, how far in advance do you want to be calling the ads? And what we've found is that it's actually more than we expected. We initially started off, I think, calling them just 250 pixels in advance, now we're back up to 1,200 pixels which gives the ads time to load. And then, the other piece of the puzzle that's pretty key is making sure that you don't basically keep the ads that are way up high on the page on the page forever. So, we do what's called "unloading our ads", where if the user has scrolled three minutes past an ad, we're actually going to remove that ad from the page now, the user's not gonna see it anyways, and that prevents you having, let's say, six IB views sitting on the page slowing down the user's browser [crosstalk 00:46:41]].
Jeff: Right, especially for our extremely long content that was extremely important for us. Yeah, the other thing is every publisher's different. You need to play with how you're lazy loading in content. How are you lazy loading ads? Are you lazy loading content and ads, or just ads, and all the content loads? These are things that we tested, and for us ... And then, how are you inserting the ads? Is it by word count? Is it by amount of time? Is it by a number of points due to listicles? And so, these things will all effect your viewability, how quickly your users are reading through your articles. And so, for each publisher it is different and you'll have to test these things and find what works best for you.
Chris: Yeah, I think one thing I can echo there is that every publisher is different. We manage about 20 billion ad impressions a month, and that's across a lot of different publishers, and they're all different, right? The behaviors are never consistently the same, so no one can come to you with a playbook and say, "This is the playbook that will work," because I mean outside of chance, they're wrong, right? And so, what you really need to do is dig down and figure out what works for you, what works for your site, your users, advertisers, advertisers' bid history, for you.
Jamie: Next question is around ad refresh. Opinions? Is it helpful? Is it not helpful?
Chris: I'll jump in there. So ad refresh, it can be both, right? It can be quite damaging, it can be quite helpful. Depends on your approach. And so, the approach we sort of espouse is we promote engage refresh, right? Where a user is engaged in the content, so they're currently live, they're engaged, and the tab is live, and the current ad they have not interacted with it for a sufficient amount of time. And so you refresh that, which means you're now loading an ad into view, so it's 100% viewable. The user's gonna see that change, and typically CTR on that event is much higher than the first look, right? So, it's very valuable for the advertiser, it's valuable for the publisher, and if you're doing a 20 or 30 second wait period, the first advertiser had a great experience. I mean, if their ad was in view for 20 seconds, then they also got a great experience. So, I think it can be a win-win.
Chris: I think there's also really awful versions where not only are you loading ads outside of view, but you're refreshing them outside of view, and this, to me, is a great way to get your inventory added to blacklists. And a really important point there is that when you get added to a blacklist, you don't know, and no one's calling you to add you back in. Unless you're a Comscore 100 publisher or something. Yeah, so you got to be careful with what you do to yourself, and again, there's a lot of tricks in this industry, there's a lot of unscrupulous players, and like I said, transparency, openness, eats tricks for lunch. Hopefully you didn't eat a bad trick and kill yourself, but I don't know. [crosstalk 00:50:08] publishers.
Jamie: Next question's for Factinate. So, you mentioned that you measure partner latency. Are you doing this entirely yourself? Do you use any third party tech?
Jeff: Yeah, we use a third party. It's Sortable. We call up our refs, they're happy to help us. As I think Chris mentioned, some of the stuff that you dive a little deeper into, they have the ability to pull a lot of reports on their end, so we called our rep, he's always around to help us out. He can pull those reports we can look at, and then we can look at by day, by hour, we can see ... I mean, on our end we're keeping a lot of the changes that we made and we can see how the changes we're making onsite are effecting latency, and Sortable has all that data for us.
Josh: It's also worth saying that managing latency is another thing in itself. We've noticed that sometimes it's fairly inconsistent who is causing latency. So, pulled reports one day, we'll see we're having an issue with this partner, and the next day it's entirely different. And for us, I mean really what managing it means is just usually having sort of a reach out to the advertiser and say, "Hey guys, what's going on?," and they take a look and usually they fix it.
Jamie: Next question is for Sortable. Is it possible to pass tracking parameters to Sortable via Sortable JS or some other mechanism that does not involve primers using the page URL?
Josh: Thank you [crosstalk 00:52:03].
Jamie: Next question is around how you define premium demand partners. Is that just descriptive? I guess this is really answering, how do you decide which demand partners you want to work with, and which ones you want to exclude?
Jeff: I think most people ... I don't want to start listing off who the premium demand partners ... I don't want to miss somebody who's a premium demand partner 'cause there is a lot of premium demand out there. You know who they are, but for us it's around when we start getting ... We run those, like Confiant and Clean Creative, malicious advertising blocking scripts. We tend to see who's sending the malicious ads, and if the problem persists for too long, we'll block that advertiser.
Josh: Yeah, and I think labeling them premium or not premium is not actually that helpful. The better advice is probably to test ... I mean, I guess the only thing would be if you absolutely haven't heard of this exchange, you know nothing about them, then maybe they're not [inaudible 00:53:06], but all of the big names in the industry, it's worth testing, it's worth seeing how they respond to your inventory, and you never know. You might have somebody who you don't consider premium who suddenly is buying more of your inventory at a higher CPM than any other [inaudible 00:53:21].
Jeff: Any exchange can have premium demand within that exchange. I think what I was referring to was blocking an entire exchange, so as you were saying, you have to test each one.
Chris: Yeah, and again, we see across enough publishers, both their seats and our seats, we see really a breadth of experiences, and again, it's the same thing. It's not the same for every publisher. One person's premium partner might be another publisher's worst partner, which sounds crazy, but it's true. It also changes by the season, right? We see certain partners really rise up and outperform and then drop off, and that might be that they got some really interesting different [inaudible 00:54:07] demand, they've burned through those campaigns, maybe they lost an advertiser. It's hard to keep track of the sort of ... What's happening on the buy-side, and so to your guys' point, one way you can protect yourself is have everyone available. And there's some interesting approaches you can take to capture that, but I'm not gonna get into advanced demand partner throttle. We'll do another webinar.
Jamie: I'm gonna finish with one final question. You guys talked a lot about the importance of data, you talked about experimenting, and optimization, and how every publisher's scenario and experiences is different. Talk to me about how can you get started with the experimentation, right? If you're a publisher that's not doing a lot of experimentation today, how do you get started? What are the first baby steps along that practice?
Josh: It's an interesting question because I think it really, really depends. I think it would be tough to say everyone listening, every publisher should start here. It really depends ... There's a saying in business where they say, "Where is the gaping neck wound?," which is a gruesome saying, but it's true. It's, what are you really struggling with today? If your viewability is really, really low, say 40% or something, than probably start with lazy loading ads. That would be a good place. If you have horrible problems with malicious ads, then start trying to figure out how to block those. It really is gonna depend on what the publisher is facing, I think.
Jeff: Yes. Also know what you're trying ... As Josh was saying, focus on one thing. Know what outcome you want and test one thing at a time. Control your variables. You don't want to be running 15 tests at one time 'cause you just want to start testing everything. It can get very overwhelming and things might affect each other in ways that you didn't consider, and something that you thought was not gonna help might help a lot. We recently changed the font on our site that had just ripples that were so unforeseen. So, focus on one thing at a time, and you'll get there. I mean, testing is a trial and error process, so just trial and error.
Chris: I totally agree with you guys. It runs the gambit, and so it can help get fresh eyes on your site. Obviously if you're working with Sortable, your account manager's gonna help get their fresh eyes on your site, and they're gonna help work through your UX, how are you paginating content? What do your layouts look like? What is going to drive longer sessions? What is going to drive better viewability? And we can look at engage refresh strategies. If your users have a certain behavior where they have time in viewport. You really have to look at your site, your user behaviors, and what outcomes you want to achieve. Maybe you have some really high value session content, and you should start doing some audience development against it. Maybe certain articles are generating you way more money than others, and so you should double down on that. It comes back down to your business, and the analysis of your business, and looking at what's working, what's not, do the gap analysis, and to the guys' point here, pick one thing, have a goal in mind, execute, and then start over. ...
Craig: Okay, that concludes the webinar for today. I'd really like to thank Josh and Jeff and Chris for their time and all their thoughts on this. Just a reminder, we will be sending out a link to all the attendees with a link to the webinar in a few days. And thanks everybody, and have a good day. ...