How To Get The Best AMT & Callout Research Results

Commissioning music testing should be based on format requirements, rather than financial possibilities, radio researcher Stephen Ryan says in this guest post.

In this second part of our series, we cover how to use your research budget effectively, looking at the characteristics, advantages and disadvantages of radio’s most-used methods for music testing, AMT and callout. Plus: how to get a high-quality sample (ensuring reliable results), and how to limit the cost of music research.

 

“Larger operations can still benefit from scale”

 

 

 

 

Choose relevant research methods

We continue our series on radio research (with a focus on music research), written by Stephen Ryan for or Radio))) ILOVEIT. Previously, he argued that while Big Data, social media and streaming trends are useful tools in a program director’s armory, legacy radio research is a significant cornerstone in competitive music scheduling. In future articles, we’ll cover specific methods of music research (and how to extract the best value and information from them) in great detail. But we’ll start with a broad view on how to allocate a tight music research budget in the most effective way, and what to consider when making strategic decisions on music testing. Over to Stephen!

 

 

Play enough ‘tasty’ songs

Music research is often accused of producing bland and formulaic radio. I see it differently. If they’re correctly used and integrated in your scheduling, music tests provide reference and structure to ensure that your listeners receive a diet of popular, familiar songs that they like on a consistent basis. Playing well-tested songs allows a music programmer to be creative with other music content within a controlled environment.

 

 

Consider doing inhouse research

Radio stations that are part of a large group or brand family may benefit from economies of scale. I have worked with operations where the amount of research across multiple stations got them to set up internal call centers and research departments. Traditionally, there would be format mappings, cluster studies, callout research and multiple auditorium music tests (AMTs) across a year. These days, there’s a much more conservative approach to budgets, and cost-benefit analysis becomes more important. This applies to both groups and independents. While larger operations can still benefit from scale, the focus of our discussion is how smaller, individual stations could approach music research within a restricted budget.

 

 

“Choose the most effective research method”

 

 

 

 

Schedule new music carefully

Whichever form of music research you consider, the goals are the same. In today’s competitive markets, you really need to know why each song has been included in your playlist and why it’s being scheduled at a specific position in the hour. With music research, you are testing a group of songs that you believe to be relevant and of appeal. Ranking them lets you identify the strongest songs in your repertoire. These have a dual purpose. They have strength in themselves, because of their familiarity and popularity, but they can also protect potentially lesser appealing songs and non-researched songs within your playlist. This is where the music programmer can broaden the repertoire within a controlled environment.

 

 

Notice music preference changes

In a recent interview here at Radio))) ILOVEIT, music content strategist Chris Price stated that “familiarity trumps discovery at scale”, a concept that should be etched in every programmer’s mind. Music research provides a benchmark for defining both popularity and familiarity. Any music station will benefit from regular callout and 2 auditorium music tests a year. Of course, that’s the ideal situation.

 

 

Think format, not money

What’s the best course of action if your budget is restricted? While your budget range may dictate which type of research is possible right now, you’re better off if you choose the most effective method for your format. The more CHR or Top 40 lean your format has, the more relevant callout becomes; the more AC and Classic Hits lean, the more relevant AMTs becomes (as your back catalogue will be larger). Looking at your format clocks and music logs, if most positions are Current and Recurrent, callout is the most useful reference; if most positions are older Recurrent and Gold, auditorium music testing is the better choice.

 

 

“Expose a carefully chosen new song, while avoiding the risk of alienating listeners”

 

 

 

 

Embed unfamiliar within familiar

If you program a CHR station, callout can enable you to program weaker or un-tested songs between two strong, tested songs. The familiarity and strength of song 1 and 3 can help you to expose a carefully chosen new song, while avoiding the risk of alienating listeners. New songs should be showcased, thus production should be used to explain why they are being played. In a 3-song sweep, it should be: Strength, Experiment, Strength. On an AC station, AMT results could allow you to use ‘spice’ songs to broaden your repertoire, while providing control over risk. Here, the experiment is sandwiching untested or lower-testing track between songs with a higher popularity and familiarity.

 

 

Invest in large(r) samples

The primary cost for research is driven by the fieldwork. In order for a cross-tab to have any chance of providing a robust and dependable result, you want to ensure an absolute minimum of 30 people. All of them should be familiar with your radio station and have listened to it in the last week, and half of them should be part of your P1 music core audience (meaning: people who listen to you mostly for the music).

 

 

Include format competition’s P1s

Furthermore, you want to make sure that the second half of your sample is consisting of your direct competition’s audience, of which 25% should be listeners of your biggest format competitor, and 25% of your other format competitors. They need to be format or close-format competitors; not just competitors. There is no point in asking a fundamentalist Classic Rock fan’s opinion on a Beyoncé song. The overall cost for conducting music research will vary from station to station, from market to market, and from country to country, for reasons explained below.

 

 

“Recruiting a sufficient sample of your own music core may be expensive”

 

 

 

 

Save some research budget

I’ve seen the cost of a callout project in the same market quoted by two different companies with a € 35.000 difference. There are so many variables; it’s difficult to provide cost specifics. However, as a ballpark guide, if we looked at a sample of 120 people 20-29 years for an AMT recruit in a developing market, you would see a cost of € 10.000 – € 15.000, and in a developed market more like € 15.000 – € 20.000. For callout, there are even more variables. But if we look at the same filters and quotas across 120 people in case we do 26 weeks of (bi-weekly) callout, you’d see around € 30.000 in a developing market and € 35.000+ in a developed market. Later, I will give you ideas on how to reduce these costs.

 

 

Focus on regular listeners

The cost will depend on your ratings, and how specific you are on quotas and filters. Your ability to recruit music core listeners will depend on your station’s reach. It’s good to mainly interview people from your daily reach (yesterday listenership), which is the nearest thing to a loyalty rating, providing a higher likelihood that these people are also P1 music core listeners (in other words: your biggest music fans).

 

 

Interview your (format) fans

Let’s assume our research is based on a 20-29 demographic, and we require 50% to be our station’s P1 music core. It means 30 respondents 20-24, and 30 respondents 25-29. If these represent 50%, our total sample will be 120. Keep in mind: we’re still working on the minimum principal. If you have a small station, music core quotas more likely favor your main format competitor (if there is one). Recruiting a sufficient sample of your own music core may be prohibitively expensive. By using a larger competitor as the main filter, at least you’re getting feedback from fans of your format. New stations can use a combination of existing format competitors, and apply music genre filters to include or exclude certain types of people.

 

 

“Much time and effort is required for a reliable, quota-based sample”

 

 

 

 

Find the right participants

The cost of any Computer-Assisted Telephone Interviewing (CATI) research is primarily driven by just how difficult it is to recruit people to take part in your survey. The difficulty noticeably varies from country to country, and is influenced by many factors:

• The prevalence of nuisance and marketing calls, reducing people’s tolerance (also for research calls)
• The move away from traditional 9-to-5 lives, making it difficult to contact people at home
• The ubiquitous use of mobile phones, reaching single users (instead of households)
• Caller ID, allowing people to easily screen unwanted calls

 

 

Reflect your audience composition

But there are more factors affecting music research. As explained earlier, we need to have a suitably filtered, quota-based sample. Let’s say our Station X broadcasts to a city of 2,000,000 people, our target audience is 20-29, which forms 20% of the total population in our market, and we want a sample of 100 people aged 20-29, of which (in this example) only 30% should be part of our P1 music core.

 

 

Budget enough recruitment calls

To find one 20-29 year-old person, we will (in theory) be required to make a minimum of 5 calls. However, if we have a daily reach of 18% in this demographic, we will (in theory) need to make circa 28 phone calls to find one Station X music core listener who is 20-29 years of age. We also need to factor in telephone survey response rates. In this example, we’ll say it’s 9% (normally 8%-12%). This will require a minimum of 308 calls to find one Station X music core listener who is 20-29 years of age and is willing to participate in this survey. If we require 30 Station X music core listeners, it’s a total of 9,240 phone calls! This is still a very simplistic example. Factor in ‘females only’ and filter by a music genre, and you can see how much time and effort is required for a reliable, quota-based sample. If you’re a new or a smaller station (having a low ‘yesterday listenership’ rating), you’ll have to relax the filters accordingly.

 

 

“After hearing 30 songs this way, the respondent’s tolerance also starts to become tested”

 

 

 

 

Track popularity & burn development

Let’s look at the advantages (and disadvantages) of auditorium music test and callout. The plus of an AMT is a chance to test many recurrent and gold songs (and some new ones) on your playlist, up to 600 songs per test. The downside: it’s a snapshot with a limited lifespan. You’ll be given a list of familiar Recurrents and Golds that have a high popularity at the time of the AMT, and rotate songs on a frequency that’s based on their individual test results. Regardless how beautiful your rotation graphs are, these songs will be on a high and medium rotation, and therefore will ‘burn’ at some point in the future. The ‘tightness’ of your rotations will affect how quickly they’ll burn, but they will burn! One annual AMT is better than no music research at all, but the results become less and less valid after 4-5 months (and potentially before).

 

 

Switch playlist segments regularly

Two AMTs per annum is a good strategy. If you’re restricted to a single AMT, you can platoon a part of your database; temporarily warehousing that part of your playlist; constantly replacing a certain number of active songs by an equal number of rested songs at regular intervals. Platooning helps to retain a perception of musical freshness and to slow the burn rate of your station’s entire music database.

 

 

Test more than currents

Callout brings the advantage of both analyzing music trends and monitoring life cycles. It is normally used to test currents and (recent) recurrents, and most relevant to Hot AC and CHR formats. The limiting factor is the number of songs tested in each survey or wave. A telephone is not ideal to listen to audio. After hearing 30 songs this way, the respondent’s tolerance also starts to become tested. The number of active currents and (recent) recurrents on a CHR or Hot AC format will easily fill up 30 slots. However, you can allocate a number of older recurrents and even some gold tracks on an ad hoc basis to test if and how much they’re burned. Now, let’s see how you can reduce the cost of music research.

 

 

“Consider setting up a P1 panel online”

 

 

 

 

Demand fresh enough samples

A callout panel or AMT sample is often used in a blind test, where people are unaware why they’ve been recruited. To retain this anonymity, the ability to reduce costs comes down to the choice of the fieldwork company. It’s not about how cheap or expensive they are; more about their reputation and experience. In some countries, the organization that provides independent radio ratings often offers fieldwork services for other surveys. They have built large databases, and can target calls far more effectively based on your quotas and filters. This efficiency can save you a lot of money for recruiting. If you do use such a company, ensure that you stipulate the conditions of how often and for what a respondent may have been used for in previous surveys. You need to avoid having semi-professional respondents in your sample.

 

 

Contact your listener database

If you are willing to forgo some anonymity, you can provide the fieldwork company with station fans from your own database (e.g. social followers and competition winners). Keep in mind any data protection issues. It’s better to contact your database members directly, and explain that they can help you improve the music you play. Include a link to the fieldwork company’s site for them to take it further.

 

 

Set up audience panels

If the costs of auditorium music testing and callout research are really out of the question, consider setting up a P1 panel online to conduct your own research. While not as effective and/or as anonymous as pure callout (as it’s more difficult to get views from people who mainly listen to your format competitors), modern music radio programmers can still use it. P1 music panel research is a subject we will return to in a future post. In the meantime, you can also read the previous article.

 

Stephen Ryan can be contacted through the website of his company Ryan Research.