From data to insight
Combining different disciplines, Leiden University researchers work together to formulate innovative solutions to societal problems. Below is an example from the field of methodology in social and behavioural sciences.
The importance of sound research methods
Social science research helps us understand human behaviour and social structures. These are determined by various factors, which makes the research complex and increases the likelihood of drawing the wrong conclusions. The choice of research method and analysis is therefore extremely important. It is also essential to indicate the limitations and assumptions of research when publishing results. This makes it clear what exactly the researchers have and have not studied.
Recent discussions in academia and society on the reliability of social science research highlight the importance of looking at research methods and how to improve them.
Leiden University has a centre that studies and evaluates the conduct of scientific research: the Centre for Science and Technology Studies (CWTS). The social scientists who work there conduct research into the dynamics of academic research around the world. Their research increases our knowledge of the international academic system and the evaluation of academic research.
The unit of Professor of Methodology and Statistics of Psychological Research Mark de Rooij focuses on improving statistical methodology. His researchers use computer simulations to evaluate which statistical methods are applicable to which types of research. The results enable them to draw up guidelines that help psychologists select the right method for their research.
New insight into society
The focus on methodology delivers powerful insights in various academic disciplines. For instance, political scientists Tom Louwerse and Simon Otjes discovered why one MP is much more active than another by examining the number of motions, amendments and parliamentary questions that MPs tabled.
Careful choice of method also enabled researchers from Leiden in Education and Child Studies to gain insight into the wellbeing of small children in noisy and quieter day care centres. Another example is economic anthropologist Erik Bähre, who chose to use the extended case study in his research into the causes of inequality and violence in South Africa. He wanted to learn about the effect of insurance companies on daily life in South Africa. His research included talking to people, participating in daily life in the townships and analysing websites and documentation.
Faculty of Social and Behavioural Sciences
Centre for Science and Technology Studies
Improving psychological research
When psychologists repeated a hundred studies in 2015, their results differed in two-thirds of the studies. ‘Research into research is not a luxury but a necessity,’ says Professor of Methodology and Statistics of Psychological Research Mark de Rooij. ‘My aim is to improve psychological research, to
think carefully about each step of the research to ensure that the ensuing research is better: from research question to publication.’
Computer simulation: a fictional Netherlands
De Rooij’s research group uses computer simulations to improve statistical research. They simulate the real world in a computer programme. Here they create fictional populations with specific characteristics. ‘I can create a fictional Netherlands, where I know the relationship between my predictor and my result. I therefore know what the truth is.’
What is true knowledge?
In these verifiable populations, De Rooij tests whether the statistical methods that psychologists use for research into, say, neuroticism or depression are correct. This can result in complex calculations. ‘Sometimes my computer takes one-and-a-half hours to complete 5,000 analyses where I take 100 samples from each of 50 populations. Other times such research entails leaving all the computers in the Faculty running from closing time on Friday evening to early on Monday morning.’
The results of such research tell De Rooij if the statistical technique really does find what it should find. This provides insight into whether the sampling and analysis methods used allow one to make general conclusions about the population as a whole. This helps nudge the results of psychological research into the realms of reliable knowledge.
Complexity of social sciences
Every study does have a margin of error. ‘What is special about psychology and the social sciences is that a lot of things can’t be directly observed. I can measure a person’s height, but I can’t see the extent to which that person is depressed. A psychologist uses a particular psychological instrument to measure this, a test or a questionnaire for instance, but these are never 100% accurate. There are always measurement errors, because everyone is different in every situation. That is what makes social sciences research so tricky and complex. No one person is exactly average, but in research we draw conclusions about characteristics of the supposed average person.’
The computer simulation gives De Rooij access to a stable population. Here he can repeat the technique until there is almost no margin of error. ‘The research that I do is 100% reproducible. Someone in Bangladesh or New York can do the same thing and get exactly the same results. The knowledge that this yields is therefore true knowledge.’
De Rooij’s research group uses this knowledge to draw up guidelines that tell psychologists whether they can use the statistical technique that the group tested in their research and whether the analysis of their data will actually tell them what they want to find out. This improves psychological research and produces knowledge upon which we can continue to build.
Research into the academic system
Research into research improves the academic system. CWTS studies and evaluates the academic system in various ways. One way is to provide information on the productivity and impact of academics by documenting how many publications they or their universities produce and how often
these publications are cited.
Apples and oranges
Last year, the UK Research Excellence Framework reported that British biomedical researchers had written twice as many top publications in 2014 as in 2008. Doubling the number of top publications in the space of six years would be a fantastic achievement. CWTS researcher Thed van Leeuwen and a number of British colleagues delved into this remarkable study. When he examined the data on which the claim was based, Van Leeuwen saw that the 2014 study did not include publications in journals with lower impact factors, whereas the 2008 one did. This is like looking at the average of your whole school report one year and at the average of only your higher marks the next. It looks like you have massively improved, but you are essentially comparing apples and oranges. Van Leeuwen worked out that the British biomedical research had really improved ‘only’ to the order of 10 to 25%.
This incident highlights the importance of transparent research. The correct definition and calculation of indicators is important, but so too is high-quality data, the cornerstone of research. Van Leeuwen: ‘The quality of data collection is hugely important. This part of the research process is often underestimated and undervalued. Collecting research data is often seen as boring, routine and definitely less “exciting” than its analysis. But data and all that relates to it is actually of crucial importance in conducting academic research. ‘The old chestnut “Garbage in, garbage out” couldn’t be more true here.’
Measuring academic productivity
CWTS specialises in bibliometrics, one aspect of which involves producing statistics on the number of publications that universities and academics publish and the number of times that these publications are cited. Bibliometrics provide information on the academic output and impact of universities and academics. The figures are often used in university rankings that are published all over the world. As quantitative information is of increasing importance to policymakers and administrators in higher education, the figures must be reliable and it must be possible to compare them (over time too).
When conducting bibliometric research, CWTS researchers stumble upon errors and imprecise data. Van Leeuwen: ‘When we analyse universities, we often find that researchers have been careless with the university’s address. Then an institution appears in different formats in our data files.’ The CWTS ensures that the different formats used for an institution or author are brought under a single header and also draws up standards for whether university hospitals are counted as part of a university.
‘Another problem, if we look at individuals, is with homonyms and synonyms: there can be more than one John Brown, so different researchers can share the same name. But there are also different ways to spell John Brown. Researchers often publish under different variants of their full name.’
The findings of the bibliometric research conducted by the CWTS are used in the CWTS Leiden Ranking, which provides an overview of the performance of a large number of universities worldwide and is based on bibliometric statistics such as number of frequently cited publications. What makes the Leiden Ranking unique is that alongside a transparent methodology, CWTS uses advanced indicators to give an idea of the scientific impact of universities. CWTS thus tries to give the most precise possible picture of the scientific performance of one university compared with another.
Some MPs are very active, while others are not. The number of proposals and questions that MPs in the Netherlands table is determined in part by the level of activity of their fellow committee members rather than by electoral incentive, which is the case in other countries. This is what
political scientist Tom Louwerse discovered with the aid of statistical analysis.
An understanding of MPs’ behaviour can help the electorate decide who to vote for. If politicians who are lower on the party list are less active, this does not necessarily correlate with the effort that they put in but rather to their surroundings. The research also shows that MPs do not put in an effort just to win votes but that the social norms of those around them determine their level of activity.
From Kok II to Rutte I
For the Personalised parliamentary behaviour without electoral incentives: the case of the Netherlands (2016) study, Louwerse and his fellow political scientist Simon Otjes defined the level of activity of an MP as the number of motions, amendments and parliamentary questions that the
MP tables. They collated the data for each MP during the Kok II, Balkenende I to IV and Rutte I cabinets and used this to establish whether there was a relationship between the activity level of MPs and their position on the party list, their level of specialisation or the activity level of their fellow committee members.
Louwerse and Otjes used regression analysis to establish which of their explanations were most plausible. The activity level of a committee proved to be contagious: the more active the members of the committee were, the more motions, amendments and questions were tabled. Louwerse: ‘Politicians appear to conform to what is customary for their party and the parliamentary committee on which they sit. A committee such as Foreign Affairs tables few motions, whereas other committees make frequent use of this tool. MPs follow what is customary for the committee.’
It was also possible to conclude from the results that specialised MPs are less active and that a higher position on the party list leads to the tabling of more motions and questions. However, list position proved to have no effect on the tabling of amendments.
The researchers had thought that the likelihood of promotion within the party could provide a further impetus for MPs to put in more effort, but the analysis did not support this. Louwerse: ‘This could be because we can’t draw a good distinction between MPs who are pursuing promotion, MPs who want to consolidate their leadership position and MPs who prefer to remain in the background. That is an interesting area for further research.’
The wellbeing of babies and toddlers
Do babies and toddlers thrive more in a noisy or quiet day care centre? Research by specialist in Education and Child Studies Claudia Werner shows that children feel most comfortable in an environment where there is a little noise. With this kind of research, the method of analysis is very
important. If not enough thought is given to different alternatives, particular correlations can be wrongly interpreted or even completely ignored.
Not too noisy and not too quiet
Children feel less at ease when noise levels are very high or very low, and in situations where there is a lot or very little variability in the level of noise (peaks and troughs). This is what Claudia Werner discovered. She studied the link between noise and the emotional wellbeing of children aged 0 to 4 in 64 day care centres.
As it is difficult to get young children to talk about their behaviour, Werner used systematic observations to measure the children’s wellbeing. She observed whether the children were relaxed, open to contact with others and enjoying themselves. On the study Noise in center-based child care: Associations with quality of care and child emotional well-being Werner worked with fellow Education and Child Studies researcher Mariëlle Linting. Linting teaches methods and techniques at Leiden University.
Choice of method
In order to establish links between the data, it is essential first to stop and think about what possible kind of relationships there could be between the data. In the above research one might expect a linear link, with more noise being associated with decreased wellbeing. However, the link could also be non-linear. If the researcher – on the basis of the expected relationship – had opted for the method of analysis most common for this type of data, she would only have been able to detect linear relations in the data. In that case, the outcome of the research would have been that there is no relation between noise and wellbeing in child care centres. In this study, the choice was made in favour of an analysis method that is also able to detect non-linear relations, which show that babies and toddlers not only feel less at ease when there is a lot of noise, but also in a very quiet environment. Their wellbeing is highest in situations where there is a low level of noise. This relationship would not have been discovered if only standard analysis methods had been applied.
Understanding social issues
Inequality and violence are the two biggest issues in South Africa. What is the underlying cause? Could they be linked to rivalling groups, economic factors, government policy or all of the above? To understand social issues, it is important to study them within their context. The extended case study
is a good method for this.
The effect of insurance companies on everyday life
The effect of financial products on everyday life is the subject of the research of economic anthropologist and developmental sociologist Erik Bähre. The policy pursued by insurance companies, for instance, can have unexpected consequences on insured persons and those around them.
Erik Bähre is particularly interested in the effect of economic change on personal relationships. He
specialises in South Africa. In September 2016, he began the study Moralising Misfortune: A Comparative Anthropology of Commercial Insurance, for which he received a five-year grant from the European Research Council. With this project, he is expanding his research – which he has conducted thus far in his country of specialisation South Africa – to Brazil, France, India, the Netherlands and the US.
The research considers which issues arise when insurance companies define social aspects such as responsibility and solidarity. Here Bähre and his team are looking at matters such as how relationships in a neighbourhood or family change if an insurance company intervenes in everyday life.
Participating in daily life
The extended case study is the main method used in this project. It enables the researchers to gain the fullest possible picture of specific events. Extended here means extensive and inclusive. Anyone who is the slightest bit involved with the research topic is an object of research. Conversations often yield names of other people who are involved in the subject, allowing the researcher to follow the line of research. Of course, the researcher is dependent on people’s willingness to cooperate. This generally depends on the nature of the topic that the researcher is studying. It is fairly easy to conduct research into some topics, but others such as funerals, are more difficult.
Bähre: ‘There are various tensions between policy and practice, between neighbours and between insurance companies. We look at whether we can explain why different people have a different experience of an event.’ One of the methods he uses for this is participant observation in the townships of South Africa. He therefore participates in daily life. He also spends a lot of time with people who are in some way linked to the topic. ‘I visit them, preferably at home, if at all possible. I go to meetings and conduct open interviews and sometimes surveys. I also interview insurance experts about their considerations, and analyse websites and the policy documents of insurance companies. The aim of an extended case study is to study as many aspects of a phenomenon as possible.’
Scientists working in this multidisciplinary research area
Mark de RooijProfessor Psychology
Topics: Statistical analysis for measurement in Psychology
Thed van LeeuwenSenior Researcher Centre for Science and Technology Studies
Topics: Organization of research, science policy, bibliometric indicators
Mariëlle LintingAssociate Professor Education and Child Studies
Topics: Methodology, child- and familiy studies, sensitivity and noise in home-based and centre-based care
Tom LouwerseAssociate Professor Political Science
Topics: Political representation, parliaments, political parties, elections, polls and voting advice applications, research methods
Erik BähreAssociate Professor Cultural Anthropology and Development Sociology
Topics: Economic anthropology, mixed methods, Africa
Brenda Van Coppenolle Associate Professor Political Science
Topics: Identities of politicians, changes in representation changes with implementation of institutional changes, political inequality and political careers in the UK and France
Elise DusseldorpAssociate Professor Psychology
Topics: Modelling of interaction effects in prediction problems, machine learning techniques, meta-analysis
Marjolein FokkemaAssociate Professor Psychology
Topics: Application of algorithms and models for improving the efficiency and accuracy of assessments in clinical psychology
Joost van GinkelAssociate Professor Education and Child Studies
Topics: Statistics, child- and familiy studies
Zane Kripe Lecturer Cultural Anthropology and Deleopment Sociology
Topics: Startup companies Singapore, open interviews, participating anticipating
Marianne Maeckelbergh Associate Professor Cultural Anthropology and Deleopment Sociology
Topics: Democracy and radical politics within social movements and citizen projects, use of new digital technologies within these movements
Michael MeffertAssociate Professor Political Science
Topics: Political psychology and political communication, esp. motivated information processing, selective exposure, and strategic voting
Huib Pellikaan Associate Professor Political Science
Topics: Research methods, rational choice theory and political philosophy
Sarah de RijckeAssociate Professor Centre for Science and Technology Studies
Topics: Interplay between knowledge production and research governance, accountability, and performance measurement
Ralph RippeAssociate Professor Education and Child Studies
Topics: Statistical issues in obesity research, statistical consultation for researchers, PhD candidates and students, software development
Rebekah TrombleAssociate Professor Political Science
Topics: Political communication, international relations, social movement studies, and Muslim politics
Anja van der VoortAssociate Professor Education and Child Studies
Topics: Sensitive parenting, adoption, problem behavior, temperament, methods and statistics
Mark WestmorelandSenior University Lecturer Cultural Anthropology and Deleopment Sociology
Topics: Middle East, visual ethnography
Tom WilderjansAssociate Professor Psychology
Topics: Novel data analytical methods in the social and behavioral sciences
Preparation for leading positions in a complex society
- BA Cultural Anthropology and Development Sociology (in Dutch) http://www.studereninleiden.nl/studies/info/culturele-antropologie-en-ontwikkelingssociologie/
- BSc Political Science (in Dutch) http://www.studereninleiden.nl/studies/info/politicologie/
- BSc Psychology (International) http://www.bachelors.leiden.edu/studies/info/psychology
- BSc Education and Child Studies (in Dutch) http://www.studereninleiden.nl/studies/info/pedagogische-wetenschappen/
- MA Cultural Anthropology and Development Sociology http://www.mastersinleiden.nl/programmes/cultural-anthropology-and-development-sociology/en/introduction
- MSc Political Science http://www.mastersinleiden.nl/programmes/political-science-leiden/en/introduction
- MSc Psychology http://www.mastersinleiden.nl/programmes/psychology/en/introduction
- MSc Education and Child Studies http://www.mastersinleiden.nl/programmes/education-and-child-studies/nl/introduction
- MSc Political Science and Public Administration (research) http://www.mastersinleiden.nl/programmes/political-science-and-public-administration-research/en/introduction
- MSc Psychology (research) http://www.mastersinleiden.nl/programmes/psychology-research/en/introduction
Leiden University has a long tradition of teaching in the area of the social sciences. The Faculty of Social and Behavioural Sciences offers students a thorough social science training in the disciplines of cultural anthropology, education and child studies, political science and psychology. The bachelor’s programmes provide students with a strong scientific basis to prepare them for leading positions in a complex society. They are trained as researchers, and learn skills that are invaluable for careers both within and outside the academic world.
Outreach & News
7 November - Research Seminar: Bähre, De Maaker and Samuels about their new projectshttps://www.universiteitleiden.nl/en/events/2016/11/caos-research-seminar
30 September - Emotion and cognition in humanshttps://www.universiteitleiden.nl/agenda/2016/09/emotion-and-cognition-in-humans
19 September - Inaugural lecture prof.dr. Janneke Wesselinghttps://www.universiteitleiden.nl/en/events/2016/09/inaugural-lecture-prof.-janneke-wesseling