Joining me on this week’s episode is Johna Burke, the Global Managing Director for AMEC, the Association of the Measurement and Evaluation of Communications.
In this engaging discussion, Johna gives insightful commentary on:
- The ethics of AVE
- How PR agencies are using ROI Incorrectly
- The importance of a SMARTER measurement framework
- Where PR pros can get ethical measurement frameworks
Why don’t you start off by telling our listeners a little bit more about yourself and your career?
I will go back a little way, but not too far because nobody likes anybody’s CV read. It’s about the most boring thing in the world, but I started out my career in technical writing, then moved onto PR and communications, and then ended as a PR and IR Director for a global Fortune 500. I made the transition into the PR services where I decided it would be interesting to see the other half and how they were delivering information. After doing that for 18 years, I recently made the change over to association management with AMEC, and am its Global Managing Director.
It makes ethics and learning and data that much more interesting to see the maturity of markets and the interpretation and the importance of words. We look at things in a very almost Puritan sense sometimes and in first world countries; it’s really not the case in some of these other developing markets. We have to be, I think, linguistically and culturally and definitively understanding of what things mean before we take a really hard stance on something. It’s a great learning process and I’m excited for the journey ahead now.
Can you give a specific example about what some of those differences are between the different markets?
Sure. I think one of the biggest things that people will talk about with some regularity are AVEs. It’s in the Barcelona Principles that AVEs are not the value of public relations. I think that’s undeniable in some of the first world markets. In some of the developing markets though, where they are still literally counting clips as part of a measure of the type of impact, having any other index and data point to give some validity and credibility is really important for them. As they mature their markets will evolve into some of the other data points that aren’t so reliant on a methodology that’s questioned, flawed, and not of true relevance.
So I think it’s a matter of understanding the markets, understanding those audiences, and then at AMEC, the great thing is that we facilitate the education process. That’s how we’re able to break down and figure out why is this so important to you? Why is this so not important over here? And then bridge that gap to really be the tide that lifts all around the measurement and analysis of communication.
Can you tell us about the most difficult ethical challenge you ever confronted at work?
Interestingly enough, at this time in history where we are, the most difficult thing ethically that I confronted was when I was in PR and a customer service issue seemed to bubble to the surface because it was now a reporter asking the question in a B2C environment, wanting those on the record answers that, historically, bubble up to the media relations department.
It was innocuous enough. But this was especially difficult because this particular journalist was doing the, I hate to use this, but was doing a hatchet job. It was a personal matter; it wasn’t a matter of informing the public. It was a matter of grinding their own ax and it felt absolutely that way from the beginning. Unfortunately, as it progressed, we were able to confirm it. Unfortunately, not before they published a piece that was inflammatory and out of context.
It was one of those rare issues where I ended up going to the editor of publication and demanding that they produce the situation, otherwise I was going to out them as writing a fiction piece in an editorial publication. It came back and the editor, all of a sudden, started to not be available and, “Yeah, I’m going to get back to you.” It was because they had realized that it was their reporter grinding a personal ax and I only found out because within some of the conversation, the reporter refused to give me any specifics that we could actually respond to.
Knowing their geography and some of those simple things that they were talking about in the question, I went to find the contract. This was at the time where it was a manual search of agreements and contracts. I was determined I had a manual search done. We were able to unearth someone with their same last name. When the editor then went totally dark on me, I sent a copy of the contract and I said, “I’m guessing you already know this. We need to talk about it.”
They did offer a more substantive correction to the story as opposed to notorious A16, “We used a misspelling of something.” But it was difficult because it was something where if I’m charged with protecting a brand and a reputation, and I knew that this wasn’t a normal PR and journalist interaction, it took a very negative, a very personal feel to it that was pseudo irrational. That’s always more difficult to navigate because in the overall relationship, if you’re wrong, you can potentially do long-term damage with the relationship with the publication, with the journalist, and wherever they would go at that time, but it was a matter of the things that were being questioned were not the great wrongs that had been done.
It is the labor in defense and in preserving their branding and their reputation and promoting the good things that they do to make sure that we had a fair and balanced response to something that was very personal.
How did you work through the situation?
When I wasn’t able to get information from them, by taking a couple of data points that I had and doing the research. I looked to see if we have any trends with this same location that are consistent with this claim? When all of that information came back invalidated, it proved my gut of the situation that it just didn’t seem right.
Gut instinct is part of it, but really, as we learn with everything else, following the data is essential. Evidence is such an important component of finding the truth and unearthing where those opportunities might lie. When you have data to support that, you can create not disciplinary moments, but I do believe in teaching moments.
I say this quite frequently in my presentations- one of the biggest hurdles for public relations practitioners is so wanting to focus on the positive part of the story that they sometimes miss the plot. They sometimes miss that hey, negative things happen, charts go down, that doesn’t mean the end of the world. If you’re only preventing, if you’re manipulating data in any way to only present the positive part of the story, it can impact your overall credibility. Really good leadership, if you’re reporting up and you have things where you see a dip, you see a decline, you see something, but you’re able to use that to identify a trend, a weakness, something bigger about the organization, then that gives you more credibility. It doesn’t taint your representation of being able to deliver results, but it means that you’re not afraid and that you are looking at a holistic picture to then go in and help diagnose a problem and show value to the organization by learning from mistakes or learning from shortcomings where you’ve experienced them. Great leaders never hit a home run 100% of the time and they understand that.
What was the key lesson that you learned from that instance?
“To thine own self be true,” thank you Shakespeare. Just that you need to have an alignment and a fine tuning of your ethical compass, and then follow it and pursue it, and then be true about the evidence that you find at the end of that because this happened very early in my career. I was young, taking on a very big, credible publication, but I just knew, based on some of the things that were being said, that it just wasn’t a true practice.
You mentioned you were earlier in your career. Did you talk to your managers about your plan of attack before you did this? Did you have to secure some buy-in? And if so, how did you go about doing that?
I didn’t really have to secure any buy-in. I did have to secure budgeting from the location because it was going to require overtime for this manual search that I was requesting over the course of a two-week period of going through every invoice. Fortunately, I had built up enough of a track record and credibility with my organization that when I made this request, they knew it wasn’t, “Ugh, here’s another request,” but they knew, “Okay, this is someone who thinks about the business, who thinks about the budget, who thinks about the organization and doesn’t overstep those boundaries,” so I really did have a wide range to be able to take care of that to do the investigation necessary.
Let’s change gears. As the Global Manager Director at AMEC, what are some of the ethical issues you’re seeing about public relations measurement?
I would say the biggest thing in public relations measurement is putting forward a number as ROI. I think we have diluted the financial term of ROI so much to twist it onto the P&L that we need to report into, that that desperation has, over the years, whether it’s an AVE or a reach or an impression or some other number, we try to get tied to some single metric that isn’t ROI.
Time and time again, I think you can probably attest to this from your years at being at an agency, even in working with a client, they didn’t tell you, “Okay, this is all of the M&D costs that we have. This is all of our fully loaded … You didn’t value in your account execs, their fully loaded weight costs. You had their hourly rate, but that, in compared with …” So you aren’t really talking about ROI; you’re talking about return on relationships, return on those efforts. You can talk about the correlation.
Now, it’s definitely evolved because edge computing has created a means for a lot of organizations to build their own data stack so that it’s getting them to a number that is their own net promoter internal scorecard, but it’s using multiple data inputs that they’re able to get to that point. But if you’re going to a single source and if you aren’t giving that other costing data and they’re presenting you something as ROI, it should be questioned at every turn.
Without that costing data, what should agencies focus on?
I think what they need to focus on is what are the objectives that they’re looking to impact for the client? An agency should not take an objective of increase awareness. They should really focus on the SMART objectives that I like to promote, and you’ll be happy to hear this, as the SMARTER objectives.
SMARTER OBJECTIVES
- Specific
- Measurable
- Attainable
- Realistic
- Time-bound
- Ethical
- Revolutionizing
There’s a whole love affair with the biggest numbers, and as the data mix continues to evolve, those numbers are shifting all over the place. We’ve all heard the story of 11 billion impressions, yet you couldn’t walk outside of your door and ask one person about it and they would know what happened. Is that realistic? Is that something that you want to have your name attached to? And that answer is no.
So making sure that you are taking a measured and strategic approach, and I think there’s a lot of good conversation now because there’s a frustration with PR professionals and breaking through some of those barriers of getting to those corporate objectives. I think the more that they ask the why and so what to exhaustion, then they do become a better, more trusted counselor and consultant to their clients across the board.
I’m intrigued by the E R for the ethical and revolutionary, as you mentioned, in terms of framework there. What do you consider to be ethical measurement? What makes up a truly ethical measurement report?
Well, I think it’s really around ethical goals. An ethical report would be a report that includes all data that can be replicated and validated. The minute that you go into a black box of some kind and that information is not replicable, and that the methodology is not transparent, I think you have a problem. I think you have an ethical dilemma that you have to overcome to be able to demonstrate the validity of the impact and the value. I think it can call into question those reports.
Are you seeing that sort of approach as widespread or is that something that you’re just seeing infrequently?
I think it’s definitely a widespread issue and I think right, wrong, or indifferent, an accurate statistical sampling is probably a little further down the path for some of our PR peers than just being able to question those basics and the foundational efficacy of data and what they’re looking at. Like I say, it’s a time and tested. And as the profile and the prescriptive of the communication teams is changing, there’s still a lot of that, “Math is hard. That’s why I went into communications.” That’s when the data scientist in the cube next to him pokes his head up over and gives a big smile of, “My job is secure.”
The R, revolutionizing. What type of revolutionizing measurement should organizations be doing?
Are you doing something that matters? Are you doing something that affects your bottom line? Are you doing something that grows your brand, your organization in the hearts and minds of your constituents?
Because again, I see time and time again, there are still RFPs going out there that say, “We would like 12 press releases and we want two events.” It’s so difficult and I think that’s what we, as an association, can help bring to light and start to develop some of the nuances because some of the practitioners are fighting back against those things, but then at the same time, good practitioners are losing that business because someone else will just tick the boxes, fill in the form online, and have the lowest price. There is a suppression of progress as a result of that, but it’s capitalism as well.
If Johna and AMEC could wave their magic wand, what are some of the fundamental things that you’d want people to do ethically when it comes to measurement?
I think because we’ve seen some of these challenges and some of these gaps, we’ve created some aides, some tools, some resources that help remove that gap from the individual and place them on the profession. I think the first of those is the Barcelona Principles. Now they have a third party-endorsed and validated with many other peer groups across the world of, “These are the basics.” If you’re only asking me for AVEs, we adhere to these principles and we can tell you that the AVEs are not the value of communication. I think in the work that we’ve done through the Barcelona Principles is one of the first and foremost ways.
One of the other things that we’ve done is we’ve created a Measurement Maturity Mapper. That allows organizations to go through and fill out a survey and they have the opportunity to map themselves against all of the survey respondents or up against their specific peer groups of which our joint peer group of PRSA is one of those, and then they can plot themselves against that peer group to see where they are in their maturity journey. We give them then a prescriptive at the end of that and say, “Okay, based on your answers, here’s where you are. Here are the things that you can do skill up and level up your program,” so that people don’t feel so desperate of, “This isn’t my expertise. I’m just going to do what I’ve always been doing because I don’t want to learn something new or I don’t want to do something new.” When they have a very clear prescriptive, PR professionals are very good at executing a plan. That’s what keeps them profitable and keeps them going.
We really have, through the measurement maturity map, given them a plan that they can go through with each of their individual clients collectively to say, “Okay, client A, you need this to skill up. You need this to skill up,” or, “This is where you are. You’re at the very basic level. Do you want to go to that next level?” Or, “Are we providing just the basic information because this is going into a much bigger system and program that we don’t know about?” Which in either case, makes the PR practitioner smarter. If they can figure out a way to dig in and have a better understanding of what all of the assets that the client might be using.
And then the other piece is we have the Integrated Evaluation Framework, which again, is an interactive tool that helps people that are putting together objectives. It helps them plot so that they can get from those outputs to outcomes. I’m very happy to say that, on a weekly basis, I talk to organizations who have modified the framework to meet their own needs and are adopting it with their overall integrated marketing mix.
It wasn’t meant to be a one-size-fits-all; it was meant to be a real conversation starter of how do people then use this resource to make themselves smarter, to go through the taxonomy of the meaning of things as they’ve been defined and confirmed by our education committee to say, “This is the true and valid definition of this,” and then work through that with their teams as they are planning so at the end of the campaign, someone doesn’t come up with some ghost metric. I call them ghost metrics instead of vanity metrics because they’re scary to use and I would be terrified to present them to my senior management team. But if someone doesn’t come out with something from left field that then doesn’t work within the structure and maintain that integrity and credibility of your reporting.
Where can they find the measurement maturity metric? Are they on the AMEC site or is there some place else they should go for those resources?
AMECorg.com. All of the resources are available and are free to members and non-members right now at AMECorg.com.
Are there trends in Europe or in Asia that are different than the US that people in the US should be aware of?
We have four regions within AMEC. We have our North America region, we have our European region, Latin America, and Asian Pacific. Each of our regions are going out and looking at some regional programming this year that I think will help us unearth some of those things that you’re talking about. Instead of doing something that is the more generic, we’re talking about having them go out and find things that are very regionally specific, and then troubleshooting those accordingly. I think that will, by its design, gives some oxygen to what those trends are that we might see that could be different from region to region and how those could separate the different areas.
Are there new areas that you’re seeing that you’re concerned about when it comes to communications and ethics?
The first in general is just civil public discourse. That seems to be totally abandoned in society and churned by politics, but that’s globally. Everywhere, there just seems to be a lack of civility in having a conversation where people may not agree and really looking towards evidence of things.
I think, unfortunately, one of the main accelerants to that is technology. We live in an age where there are things like the celebrity voice changer. The evidence, as we normally thought about it, where unless I see it with my own eyes or hear it with my own voice, but now you can hear the voice of a celebrity say anything. How do you know if it’s something that’s been created or if it’s something that’s been manufactured?
With video and editing being so advanced now, you really have to be an expert to be able to see some of these editing things that are done. That can create a whole different pseudo reality to an evidence-based reality where it’s no longer if I can see it and hear it with my own eyes and ears, but what are the other pieces of evidence that back that up so I’m not operating and escalating on a feeling or a single point of data that could be flawed?
What do businesses do to navigate those waters?
Well, I think that’s the challenge for all of us. I think greater adoption of AI will both stimulate and quell that because I think it will create somewhat of a balance in how people work with data, but I think there will be a growing demand for additional data points that are evidentiary as opposed to single data points that we were able to historically look towards. That then brings into the validation of looking at an integrated marketing mix to where you have that stack and now you can see, as a trend starts to develop, is the arc in the curve the same as it was historically? What are the same other common factors that had been fed into a database that create more of a predictive model that people can get to?
As much as it creates anarchy, it can also create a good balance to where companies and organizations can start to make predictively in-kind decisions based on the use of data.
It sounds like, from what you’re saying, companies are going to need to make sure they’re investing both in data scientists as well as in that technology and the AI to help differentiate what’s accurate from what’s the fake, especially as bad actors start having more tools at their disposal.
Absolutely. Invest in and be very knowledgeable about those things that are out there that could influence their market. I was recently at a copyright discussion talking about copyright and publishers and PR practices and how those fees are all assessed. I’ve said that if the publishers figure out blockchain, the whole thought of free market and, “Nope, I just copied and pasted my Google hit into my report” goes away. Nope, it was on Google, so it’s okay. I don’t have to pay for this.
There are things in technology that can level balance all of those conversations if they’re applied.
What is the best piece of ethics advice you were ever given?
I think it’s a couple of things, really. I think the top would probably be keep confidences and act responsibly. The other is listen with intent to understand and then act. I probably draw upon the latter so much more in this day and age as everything is timely and everybody wants to be the first with some information or something. It still makes me a little nervous, being given that advice long, long ago, that it really was great advice to not be easily swayed and when we’re listening with the intent to understand and not listening as an exercise, it does change that active listening sensitivity and depth that you have.
It’s a good piece of advice and it’s always that rush to be first. It’s more important to be right in some cases. A lot of cases.
Is there anything else, Johna, that you wanted to share with our listeners?
Really seek knowledge and test your theory against observation. Look to find and practice scientific methods in your everyday rigor so that you can get closer to the truth.
Some people might call that a justified belief, but I think we, being word people and wanting to be good advisors, we come up with very scary phrases like “my truth” instead of “the truth.” We will just talk about my truth when you’re responding to the media. You don’t want to tell the truth because we know that it could be wrong, so you want to talk about my truth. I think as we always steer people towards clearer and away from ethically gray language, we are doing a much better job for our profession and for our clients in the long run, even if it’s difficult at first.
Obviously, have a tie and a connection to a strong association as AMEC – shameless plug – to be there to help you validate and to help you understand some of those changes you might be seeing in your organization. Have an established peer group that’s working with data, that’s working with technology, that’s working with changes who you can bounce those ideas off of and really be able to present with data confidence, but also, with that brings an ethical confidence of being able to have the conversation and provide evidence to any naysayers that you might come across of what’s valid and what’s really going on when you’re presenting those things that are facts.
Listen to the full interview, with bonus content, here:
- How to best counsel your client when they want to respond unethically to an unethical competitor – Tatevik Simonyan - October 28, 2024
- Why it’s important to know many codes of ethics – Erin Kennedy - September 9, 2024
- The Importance of Really Small Things – Capt. Barbara Bell, USN (ret) - May 28, 2024
3 Comments