Large corporations are now utilizing brain scans, digital tracking, big data, and even facial recognition software to script and target their advertisements. In Adcreep: The Case Against Modern Marketing, Mark Bartholomew documents how advertisers are creeping into our lives in ever new ways, compiling detailed dossiers on ever more elements of our lives, and at the same entering ever more geographical spaces. A fascinating look at how large corporations are using the latest technology to both understand and sell to consumers, Adcreep raises the alarm about these hidden practices, arguing they are fundamentally altering the consumer-corporation balance.      

Kelly Sarabyn: Thank you for joining us. Your book details how new technologies have allowed companies, especially large companies, to fundamentally alter their advertising relationship with consumers. Can you give us an overview of these developments?

Mark Bartholomew: The biggest issue is one of advertisements overtaking new spaces: spaces in public areas, like parks or schools, as well as spaces in the home. We’re doing things we thought were private, like using the computer or watching TV, but now advertisers can track those activities. They can scrutinize our computer and TV habits; they can record what we ask Alexa to do. Businesses now try to sell to us on trails in public parklands or on our kids’ report cards, spaces we once thought of as off-limits for the commercial hard sell. So one key area of change is companies, for the purpose of market research and advertising, entering spaces or activities that they weren’t able to before.

The other part of it is the ability to know people on a more individualized level. If you look at the history of advertising, it has been composed of broad-based appeals that are directed to a large number of people at the same time. Now more and more there is an ability to micro-target us with ads. Companies can know what our individual moods are, what our preferences are, where we are located, and what we are saying online, and they can use that information to determine what type of ad to show us, as well as where and when to show it to us. I think that makes advertising a lot more effective.

KS: I reviewed a book recently, The End of Advertising, and the author, a veteran of the advertising industry, argued almost the exact opposite, that because of ad blockers, and streaming TV, this generation of children is growing up exposed to less advertising than ever. He claimed that, with the exception of social media and live sports, this generation will come of age with little exposure to ads, and thus advertisers will have to work harder to provide quality content. Do you disagree with that interpretation of where we are?

MB: I do. In doing my research, I found that people in the advertising industry have always said that people are becoming blind or turned off to ads, that they are finding new ways to avoid advertising. Advertisers have been saying this since at least the 1700s. It’s simply not true that advertising is dead or even dying.

It is true that technology today affords some ways for us to avoid advertising. You can pay for HBO, fast forward through the commercials on TV, and use adblockers online. But advertisers have ways around this. They use product placement instead of traditional 30-second television ads. The force us to watch the ads that show before feature films. Or even with the advertisements we fast forward through on our DVRs, corporations have figured out how to generate an impression in consumers’ minds even in that short window when they see a blip of an image during fast forwarding. Companies are also fighting back against ad-blocking technologies, sometimes refusing to allow access to websites when an ad blocker is in place, sometimes making deals with ad-blocking companies to allow their particular ads to pass through the ad-blocking filters.

And the other side of the coin is that even if it was true that our TV-watching and online experiences aren’t as filled with commercials as they used to be, advertising has also moved to other spaces. If I fly on a plane, there’s often an ad on the back of my tray table, or if I am at the ATM, there is a commercial playing. My kids are seeing advertisements at school, and in the online learning platforms they use for school. If you go to a state parkland, you’ll see signs for Odwalla or REI. On balance, I think we’re exposed to more advertising today, not less.

KS: One of things I found fascinating about your book was learning more about how companies are using large data collection and neuromarketing to shape their advertising content and strategy. Can you give an overview of what companies are doing with these new technologies?

MB: First, companies have greatly increased their abilities to record information about you as an individual. There is a constant tracking that takes place through our computers, our smartphones, and our social media. What people don’t seem to realize, though I think they are starting to, is that all this information is bundled together. I think we understand when we interact with an individual website, that that company will have some information on us. But I don’t think people realize the scope of the whole digital advertising ecosystem. There are massive brokers of data that bundle all these digital transactions together. It’s not like they have a few pages of data about us, they have thousands of pages of data about us. That’s part of what is different.

Related to that point is the phenomena of big data. It’s not just that they have information about me, or you: they have information about everybody. And when you have information about everybody, it’s a lot easier to identify individual idiosyncrasies. They can discern a behavior that is out of place, or that makes a person special, or vulnerable or susceptible, because they have a baseline on everyone to compare it to.

KS: How has the rise of neuromarketing affected advertising?

Businesses have a much better sense now of how advertising works, and how it affects us, which has traditionally been a problem for advertisers, than they did in the past. There is a famous quote by John Wannamaker, who started the first big department store in this country. He said, “Half of all the money I spend on advertising is wasted. I just don’t know which half.” And that uncertainty always held things back.

In the past, advertisers used surveys and focus groups to analyze advertising effectiveness. But the problem with those methods, from the advertisers’ perspective, is that people aren’t always in touch with themselves, so they will say they like this, but they don’t really know. Or maybe they are faking. Maybe they are embarrassed to say they like something. There are some examples of big brand flops. For example, when New Coke was introduced in the 1980s, they focus-grouped the hell out of it, but it still failed. Part of the problem was that people weren’t giving the researchers quite the full answer on how they felt about this new product.

So what neuromarketing proposes to do is to prevent people from mediating their own responses. Instead, it tracks how a person’s brain reacts in real-time to advertising stimuli.

There was an important neuromarketing study about a decade ago where they compared consumers’ reactions to Pepsi and Coke. Consumers’ brains were being scanned as they drunk both beverages from a long straw. When the researchers did not tell the consumers what they were drinking, the part of the brain that signals pleasure had a stronger response to the drink that was Pepsi. This also matched people’s oral responses as to which beverage they preferred. But when the researchers conducted the same study, but showed consumers the logos for the beverages before they took their sips, everything switched. Both the brain scans and the oral responses then showed a preference for Coke.

By today’s standards, this study looks crude. Now we look at many parts of the brain operating together, not just one, because the way we process advertising is complicated. This early study is important, however, because it started advertisers down the path of neuromarketing, trying to find a way to determine how people “really” feel about different brands and different ads.

Of course, there is a lot of charlatanism in the neuromarketing field because the people who are talking about this are usually advertisers. When they are talking amongst themselves, or in marketing circles, they will pump up these abilities, claiming to have found the “buy button” in the brain. Nobody has found the buy button in the brain. And nobody can as there is not one area that lights up in the brain for effective advertising. It is a much more complex process. So I don’t want to imply this stuff always works, that there’s not any subjective interpretation involved in the reading of brain images. But the technology is getting better and better. Today, you can see on multiple levels people’s neurological reactions to things that we couldn’t before and advertisers are investing more and more money in neural imaging, which suggests there is something to it. All the big advertising agencies today have a neuromarketing department.

KS: What potential dangers do you see for society with corporations using these new technologies in this way? Are you concerned they will end up appealing to our baser instincts?

MB: What is disturbing to me is the bigger picture of what this information is being used for. One, if they can use these technologies to figure out which ads appeal to us and which don’t, that is going to make advertising much more effective. I think of the advertiser-consumer relationship as a kind of a battle between parties that needs to be somewhat equally matched. To the extent we are worried about a power imbalance, I think these new capabilities — big data, individualized tracking, neuromarketing — all tip that power imbalance further in favor of the advertisers.

Beyond that, there is also an appeal to basic instincts in the way consumer neuroscience is applied that should give us pause. The benefit of the survey or the focus group is there is something democratic about it: you go in, and Procter & Gamble shows you an ad, and you tell them what you think about it, what you like and what you don’t like. There is a dialogue there between you and the company. By contrast, the whole reason to invest in neuroscience is to avoid talking to, or listening to the consumer. Instead, the company just needs these instinctual, involuntary responses as revealed in an fMRI of your brain — they don’t have to ask you any questions.

The example I give in the book is Frito-Lay. They wanted to develop a new advertising campaign for Cheetos. They had some ideas, and they used brain scans of potential consumers to figure out what resonated. When asked directly, consumers reported that the only thing they didn’t like about Cheetos was the orange dust that sticks to their fingers. They claimed it was gross and messy. But the brain scans revealed that consumers actually felt pleasure upon seeing the orange residue on other people’s fingers.

I don’t know why consumers had the favorable response that was revealed in the brain scans, but the people who ran the ad campaign for Frito-Lay thought viewers must be getting a pleasurable sense of subversion when they saw the cheese-coated fingers. These are adults so perhaps being messy is breaking the rules, and Cheetos are a guilty pleasure to begin with. The ad agency took this information and developed a whole marketing scheme based on the Cheetos mascot, Chester the Cheetah, egging people on to acts of vandalism and anti-social behavior.

In one ad, a person is in the laundromat — the person is a kind of everywoman — and Chester says, ‘oh, do it, you know you want to do it,’ and then the woman dumps a bag of Cheetos into someone’s pristine load of white laundry. There’s a similar ad in an office, and the protagonist is in a cubicle next to very neat and tidy coworker. When the tidy coworker leaves to go to the bathroom, Chester says to the protagonist, ‘go on, do it, you know you want to it,’ and the protagonist mashes Cheetos into their coworker’s keyboard.

When people were asked about these commercials, whether they were people working in marketing or just people on the street, they said the commercials were terrible, and cynical. They didn’t like them. But because of the brain scans, Frito-Lay thought people were actually feeling differently than what they were reporting. Whether this is cause and effect, we can’t say for sure, but following the release of the ad campaign, Cheetos sales spiked by something like 47 million dollars.

This bothers me because it might lead to more and more advertising that plays on our worst impulses. If we dialogue with someone, we can hide our worst impulses, or at least tamp them down. But if ads are just based on our instinctual, emotional responses, that we don’t have a chance to mediate, then perhaps we will see more ads that glorify anti-social behaviors, or more ads that appeal to racial prejudice, or sexism (and there are enough of those already). But if neurological research says that those kinds of appeals are the way to get people interested in, or attracted to our product, we will see more of it.

KS: That seems like a distinct possibility, but doesn’t the success of brands like Patagonia, REI, and Whole Foods, and this general trend of millennials (ostensibly) wanting to buy from companies that “do good,” suggest that neuromarketing studies might also reveal that, while people are driven by emotion, it might not be primarily emotions defined by our baser instincts, but rather emotions that are shaped by more elevated, and inspiring concerns?

MB: Sure, I don’t mean to say that every campaign will rely on the worst part of us, and not the best. Patagonia, and companies like that, do those campaigns for a reason. Some people are drawn to that. What I worry about is that most all of this is done clandestinely. Advertisers don’t always volunteer how they arrive at their campaigns this way, and there are trade secret protections involved, and they don’t want to divulge it. But as a result, we don’t really know what is happening behind closed doors. It would help to know more, for this to be more public, and perhaps even have regulatory agencies take a look at what is happening. Ultimately, we don’t know where neuromarketing will lead us, and how it will affect the advertising environment we are exposed to.

I’d also say that all this can be done very subtly. There can be images and storylines in an ad meant to suggest social responsibility from a company. This might be exactly what the prototypical millennial consumer wants if you ask them. But involuntary neural responses might also indicate that this same consumer will be motivated by visceral appeals to their own prejudices. And these appeals can be encoded in the ad in ways that we might not realize — an ad can appeal to social responsibility and sexist stereotypes at the same time. So the consumer is really flying in the dark here.

KS: Would transparency help the consumer, if there was awareness that a particular ad campaign was developed based on neuromarketing? Would it help them look at it more critically?

MB: I’m skeptical of that. It’s better than nothing, but I think, and this is a big part of the book, there’s a project here where advertisers and businesses try to condition us to expect certain things, to get used to them. “OK, I guess I will have to give up my privacy to get this online service,” or us just accepting advertisements in an ever growing number of new places. I’m a fan of society trying to formulate standards for what is acceptable, and what is not, and perhaps even having a regulatory agency decide what is allowed and what is not when it comes to the ads that we are exposed to. When you think about it, there are few forces that take up our daily environment more than advertising. It makes sense to think about how that environment should best be curated, not just for advertisers, but for everyone.

One more point. I don’t mean to imply that the free market has no role to play here or that the government should approve every commercial communication before it goes out. There’s a difference between persuasion – where you provide a list of reasons for something, and I can evaluate them and use my rational faculties — and manipulation, where I don’t know what’s going on, and I am being influenced and I don’t know why. I think people are a lot more comfortable with having the government step in and stop manipulation than with the regulation of commercial persuasion. I want more of an active regulatory role when manipulation is involved.

KS: What do you think about companies like Facebook trying to put restrictions on companies’ ads and who those companies can target? Do you look at that as an optimistic sign that there will be some self-regulation in the market?

MB: No, I’m pessimistic. There’s some good things there, and there are some positive things that companies like Facebook and Google do. The recent controversy over the political ads linked to Russia suggests that perhaps there is a growing movement to restrict what kind of ads can be shown, but I think Facebook is one of the worst offenders. They have some standards for what kind of ads companies can show, but they are the ones that are compiling all the data on us and leveraging it in the service of advertising. There was a leaked memo in May where FB executives said they can use this data to target people’s moods in real time, and they were targeting teenagers, not even adults. This FB memo showed they had the capacity to diagnose people’s moods, to know when a person was feeling “silly” or “sad” or “feeling like a failure,” and there was the possibility of selling this to a business who could say, ‘Oh, it’s 2 pm, in Sydney, and Mark is feeling like a failure; this is the perfect time to put an ad for Ben and Jerry’s in his newsfeed because I know he likes to eat Chunky Monkey when he’s feeling bad about himself.’ I think we should worry about something like that. It’s not a fair fight.

I don’t want to say that the market will never self-correct, but one of the problems is there is a monopoly in a lot of these spaces. If you’re talking about digital advertising, Facebook and Google control 85% of that market. That makes it hard to find an alternative that doesn’t engage in these practices. These businesses also have a history of buying up popular platforms without ads, and then proceeding to use the previously ad-free platform to mine your data and show you ads.

Structurally, I don’t have much faith in letting the digital advertising market fix this problem on its own.

KS: You seemed to be saying, in the book, that the growth of social media influencers – or micro-celebrities – cuts against consumer autonomy, which I thought was really interesting because I think the prevailing narrative is the opposite, which is that, everyone being able to have a blog and their own little group of followers has moved power toward the consumer. That, in the past, power was centralized in large companies, and celebrities were being boosted by a small number of studios, and a small number of publications. Now, “everyone can be a star.” But you seemed to be arguing that this is not actually empowering to people, or consumers.

MB: I like the way you framed that, and I think part of what I am pushing against is the narrative that social media is so democratic and empowering. I don’t mean to say that it never is empowering, or that we can’t have great exchanges online, or that we can’t find like-minded people online. In some ways, social media is wonderful for that kind of connection, and we didn’t have it before.

But I think the narrative we get sold about this hides some flaws in the system. When I wrote the book, I felt like I had to have a chapter about celebrity because celebrity is just so interwoven with advertising. Something like 20 to 25 percent of ads in the United States feature a celebrity. A lot of advertising money is spent targeting celebrities’ online followings.

Today, some of that money is trickling down to average people with smaller followings, and like you say, in some ways that can be good, that can be empowering, it can be a good way for a regular person to make a little money. But part of the problem is that we see a parallel with the celebrities on Twitter and Facebook and Instagram when we go online and it’s not quite a parallel. When celebrities use Twitter and Facebook, they get all these VIP capabilities from the social media sites that the rest of us don’t get. They are allowed to track how their posts flow, see who responds to their posts, and given advice on how and when to respond back. Celebrities have a lot more vision and understanding of who their online audience is than we do. One of the tricky things about social media is we don’t always know who we are talking to. I give some stories in the book where people posted things on social media without realizing where the post was going to end up, and that had catastrophic personal consequences even though it was a rather innocent mistake.

So one of my concerns is it’s a false dichotomy. We don’t have the same resources as celebrities, so social media is not quite as empowering for us.

The other thing is, on the one hand we can use Twitter and Facebook to connect with people, but we are always being encouraged to disclose, disclose, disclose, to be as personal as you can. That’s how you get more followers, that’s the way to attract more of an audience, and get more engaged viewers. And all that disclosure is put in our digital dossiers to sell more things to us, and to attract more people to us, and to sell things to them. Perhaps it’s empowering in some ways, but we’re also doing the bidding of advertisers when we try to maintain online followings. That’s something to keep in mind.

KS: Traditional publications have always had advertisements, but this era is distinct in merging the personal and the public, and people are sort of selling themselves online. Do you think that is this more of a cultural or legal issue?

MB: It’s mostly cultural, and about evolving norms. The way we self-present has changed over the years, and we’re constantly evolving new ideas of what is acceptable. I don’t think everyone realizes what these underlying architectures are like, though, and what information is being collected when we self-present online. What is disturbing is we often don’t know the micro-celebrity is selling to us. It’s much better when we know an online post is being instigated by McDonald’s, that they gave my friend a free hamburger, or one hundred dollars, to tweet something about how great McDonald’s is. It’s more disturbing, more like outright manipulation, when our online friends pitch a product to us without revealing that they received compensation for doing so.

The problem is the federal agency in charge of this, the Federal Trade Commission, has limited resources so there is only so much they can do. They have their hands full with the shenanigans of the celebrity influencers, who are pretty bad, in general, about putting #ad or #spon next to their posts. There are some legal things we can do to require social media influencing to be more transparent, but I also think it’s a cultural thing where we imitate celebrities, who aren’t always disclosing, and we copy those behaviors. This can be problematic from the perspective of consumer welfare as we buy things under false pretenses.

And if it gets bad enough, it can be problematic in another way. We will lose a lot of trust in these online conversations, because we won’t know whether a person is selling to us, or genuinely recommending a product.

KS: Do you feel like, historically speaking, this is a time of particular overreach in advertising?

MB: This struggle has always occurred. There have always been examples of advertiser overreach. The snake oil salesmen of the early 1900s, that was bad time, and eventually their behavior was restrained by legal action. In the 1950s, there was a scandal over subliminal advertising – during a movie, a theater flashed the words “Hungry, Eat Popcorn” and “Drink Coca-Cola” for less than a tenth of a second, and supposedly everyone went out to the snack stand after being exposed to these subliminal ads. The public was outraged when they heard about it, and the regulators pushed back.

I think this is part of a reoccurring cycle in history where there is a new technology, and advertisers try to use it to their advantage. That is natural, but at a certain point, there needs to be a response. There needs to be a hard and fast boundary set by society, saying you can’t do this, you’re going too far. That hasn’t happened yet, and the danger is that if it doesn’t happen, we just get used to it. With feature films, for example, people were initially outraged when commercials started playing prior to the film, but there wasn’t enough traction, and now we’ve gotten used to going to the movie theater and being forced to sit through a barrage of annoying ads. I worry that if there isn’t a response soon to these more invasive practices we’ve been talking about, then these things are here to stay.

Mark Bartholomew is Professor of Law at University at Buffalo School of Law. He has provided commentary on intellectual property and privacy issues in The New York TimesThe Washington PostLos Angeles TimesUSA TodayThe Wall Street Journal, and other news outlets. You can buy his book here.