Do 1 in 5 British Muslims really have sympathy for ‘jihadis’?

However, when many observers began to scrutinize the poll, they found that the Sun appeared to have exaggerated the results. Worse still, there were serious doubts about the manner in which the survey was conducted.

Polling is a notoriously tricky thing to get right, and polls about controversial subjects such as terrorism and extremism can be even harder. One widely shared survey last year suggested that 1 in 6 French citizens supported the Islamic State militant group — a figure that would seem to suggest that every French Muslim (and a variety of non-Muslims, too) supported the most extreme of extremists. Estimates of the number of Muslims living in France vary from about 5 percent to 12 percent.

The results of that poll were difficult to explain. However, the problems with the Sun's poll are pretty easy to spot.

I ran the poll past Scott Clement, The Washington Post's resident polling expert, who offered his thoughts on some of the problems with the Sun's survey:

1. The Sun's headline states that 1 in 5 British Muslims have sympathy for jihadis. However, the results of the poll actually show that only 5 percent of respondents said they had a "lot of" sympathy, while 14.5 percent said they had "some."
2. Respondents were asked whether they agreed with the statement “I have a lot of sympathy with young Muslims who leave the UK to join fighters in Syria." The wording of the question is vague — not all young Muslims who travel to Syria fight with the Islamic State or another jihadist group — and it is unclear what sympathy means in this context. "It’s possible people think idealistic opponents of [Syrian President Bashar al-]Assad went to Syria and were unaware of ISIS’s harsh rule and are, thus, sympathetic, for instance," Clement says, using an alternative acronym for the Islamic State. "Because a clearer term was not used, and the Islamic State not mentioned explicitly, it’s tough to tell."
   (Side note: A poll conducted by the same company for Sky News in March found that 4.3 percent of non-Muslims expressed “a lot of sympathy with young Muslims who leave the UK to join fighters in Syria.”)
3. The Sun's usual polling company, YouGov, refused to conduct the poll, concerned about how it could ensure accuracy given the short time frame. A different company, Survation, was used. YouGov's decision may reflect the difficulty of polling Britain's Muslim community. According to Clement, the most accurate way of polling Britain's Muslim community would be to call households until you reached a Muslim one. Because Muslims make up only about 4 percent of Britain's population, you would expect to have to make about 25 calls before reaching a Muslim household. "Taking this approach typically requires multiple months of interviewing to obtain a sufficient sample of the population," Clement says. "Survation’s three-day field period is extraordinarily short for such an effort, limiting the ability to make callback attempts."
4. To find British Muslims to poll, Survation called people with "Muslim names." This was probably an attempt to speed up the survey so it could be published quickly, but Clement says problems are likely with this method. "What this means is that only Muslims with typically Muslim surnames could potentially be interviewed," he says. "It’s impossible to assess what percentage of the U.K. Muslim population has typically Muslim surnames and how many do not."
5. It's not clear whether other socioeconomic and demographic factors were included to ensure that those who responded to the poll were a representative sample of British Muslims. Clement also notes that it is unclear whether cellphones were called, which means that only households with home phones may have been called. It is also unclear whether the survey was conducted in English. "More heavily immigrant communities may not speak English effectively, which can either make it impossible to conduct interviews with some respondents or reduce quality of data," Clement says.

"In general, I would expect that reports for support for extremist groups would be underestimated in surveys due to social desirability bias [against admitting such support] in interviewer-administered surveys," he says, adding that there are problems with the sampling and that the question is too vague to assume any support for the Islamic State or the Paris attacks in particular.

Survation themselves have put out a statement that distances their organization from the Sun's interpretation of the poll.

Online, many have responded to the poll with mockery. Check the hashtag #1In5Muslims for some of that:

However, not everyone can laugh away the poll. At least 450 people have complained about the poll to the Independent Press Standards Organization, the most complaints related to a single story since the watchdog was founded in September 2014. "The grand strategy of Daesh is to divide our communities and stoke fear between communities," Shuja Shafi, secretary general of the Muslim Council of Britain, said in a statement, using an Arabic acronym for the Islamic State. "We should not play their game.”

Even Maajid Nawaz, the head of the counter-extremism group Quilliam Foundation, who is often critical of those who underplay the link between mainstream Islam and extremism, was quick to criticize the poll. "The Sun has made it harder to have vital conversation around worrying levels of support for Islamism," he wrote on Twitter. "All for a cheap headline."

More on WorldViews

Do 1 in 6 French citizens really support Islamic State?

The Islamic State wants you to hate refugees