September 2023
Fathom Journal
By Alexandra Fishman and Max Samarov
The Brookings Institute, Washington DC
Alexandra Fishman and Max Samarov take issue with what they see as ‘leading’ questions used in a recent and much-publicised opinion poll about US attitudes to Israel’s future. The editors have invited Professor Telhami to respond.
Since 2016, when he established the University of Maryland Critical Issues Poll, Professor Shibley Telhami and his colleagues have published a series of studies about the perspective of Americans on the Israeli-Palestinian conflict and other political issues. Their work has been covered by the Washington Post, CNN and NPR, and appeared in academic journals such as Middle East Journal, Journal of Conflict Resolution, and Political Science Quarterly. The polls are also promoted by the Brookings Institution, a powerful thinktank where Professor Telhami is a fellow. Given their widespread influence, it is important to analyse how reliable these studies are, starting with the most recent one conducted in June, 2023.
There are some key factors to consider when looking into whether or not a poll is accurate. For starters, a reliable poll is not something that any layperson can conduct. It needs to be done by a trained researcher. This is because even seemingly simple tasks like asking neutral questions and performing data analytics are nuanced and complicated. Additionally, having verifiable credentials and the skills to perform quality research does not eliminate bias, because even experts are not entirely objective. It is important to look into what the researcher’s angle(s) may be before reading the research. Finally, it is crucial to understand the methodology of every study. If the methods are flawed, the research is flawed.
In terms of credentials, Professor Telhami is an academic with the qualifications needed to conduct reliable research. However, he has a strong opinion on matters related to Israel. In April 2023 he published an article accusing Israel of choosing Jewish identity over democracy. He also urged the U.S. government to dramatically reduce its support for Israel and stop advocating for peace deals between Israel and Arab states. Additionally, he may have ties to the government of Qatar as an academic advisory board member of the Arab Center Washington DC. This thinktank is affiliated with the Arab Center for Research and Policy Studies in Doha, which according to the AP is, ‘funded by the Qatari government.’ Its offices are in City Centre DC, a building owned by the Qatari government.
Professor Telhami’s biases are evident in the latest ‘American Public Attitudes on Israel/ Palestine’ study published by the Critical Issues Poll. One of the critical tasks of a researcher creating surveys is ensuring that questions are not worded in a way that is leading. For example, to ask about the condition of a glass bowl after it was dropped on a marble floor, a question might be phrased: ‘What was the condition of the glass bowl after it was dropped on the marble floor?’. However, it could also be phrased: ‘How badly did the glass bowl shatter into endless shards of glass after it was dropped on the marble floor?’
The first option is more neutral, allowing people to come to their own conclusion before answering. In contrast, the second option is a leading question because it paints a vivid image of the glass bowl’s condition after being dropped, pushing people towards certain answers.
The centerpiece of Professor Telhami’s recent survey is also clearly a leading question. It asks: If a two-state solution to the conflict between Israel and the Palestinians were not possible, meaning the West Bank and Gaza had to be under Israeli control indefinitely, which of the following would be closer to your view?
I would favor Israel’s democracy over its Jewishness: I would support a single democratic state in which Jews and non-Jews would be equal, even if that meant Israel would no longer be a politically Jewish state.
I would favor the Jewishness of Israel over its democracy: I would support preserving Israel as a politically Jewish state, even if that meant that millions of indigenous non-Jews living under its authority would not have citizenship and equal rights.
Notice the stark difference between the first and second options.
The consequences of option #1 are described in positive terms (‘Jews and non-Jews would be equal…’). A portion of participants might have the background knowledge to understand that dismantling the world’s only Jewish state and trying to replace it with a ‘single democratic state’ could have many negative consequences. Most Israelis and Palestinians oppose this idea for a variety of reasons, including concerns that it would result in even more tensions, violence, and suffering in the region. However, most Americans do not pay close attention to the Israeli-Palestinian conflict and are unlikely to know such details.
In contrast, the consequences of option 2 are described in a way that many survey participants would very clearly see as negative (millions of indigenous people being denied citizenship and equal rights). The wording may have also misled participants by implying that Jews in Israel are non-indigenous foreigners. This is ahistorical, given the undeniable evidence that the land of Israel is the ancestral home of the Jewish people.
Unsurprisingly, when given the options above, 73 per cent of survey participants said they would ‘favor Israel’s democracy over its Jewishness.’
The way these ‘findings’ were advertised to the public made matters even worse. The Brookings Institution tweeted a graphic with responses to the question described above, without acknowledging its obvious methodological flaws. This was followed by an article asserting that, ‘there does not seem to be a strong American public attachment to Zionism, which entails a Jewish character of Israel.’ At least one major media outlet bought this misleading narrative, running a headline saying, ‘Most Americans Would Choose a Democratic Over a Jewish Israel, Poll Finds.’
In reality, what the poll found is that when Americans are asked to consider Israel’s ‘Jewish character’ in a deeply negative context alongside a much better-sounding alternative, they’re likely to choose the alternative. Indeed, one would expect most participants in any survey to pick a relatively beneficial-sounding scenario over a very harmful-sounding one. The lesson from this ‘Critical Issues Poll’ is not about American public opinion, but about how researchers with an agenda can use leading questions to get the results they want. Disturbingly, it is also about how seemingly easy it is to get mainstream organisations and publications to uncritically promote bad data.
This is especially puzzling in the case of the Brookings Institution, which is considered by some to be the best thinktank in the world and has historically provided analysis about Israel that reflects a diverse set of viewpoints. One would hope that an organisation dedicated to quality research and ‘the highest standards of scholastic rigor’ has the integrity to acknowledge mistakes, correct the record, and review publications more carefully going forward.
Dr Alexandra Fishman is Director of Data & Analytics at StandWithUs, an international, non-partisan education organisation that supports Israel and fights antisemitism. Max Samarov is its Executive Director of Research & Strategy.
Read the full article here.
