Generative AI for Social Research: Going Native with Artificial Intelligence

Federico Pilati; Anders Kristian Munk; Tommaso Venturini

doi:10.6092/issn.1971-8853/20378

Authors

Federico Pilati Department of Political and Social Sciences, University of Bologna https://orcid.org/0000-0001-5526-1011

Federico Pilati is a Postdoctoral Researcher at the University of Milano-Bicocca (Italy) and Research Associate at the Medialab of the University of Geneva (Switzerland). As an Adjunct Professor, he teaches “Qualitative Methods in Digital Media Research” at the University of Bologna (Italy) and “Machine Learning and Generative AI for Social Research” at the University of Milano-Bicocca. He has been a member of the Horizon 2020 projects “inDICEs” and “EUMEPLAT” and a research fellow of the Future Artificial Intelligence Research Foundation.
Anders Kristian Munk Department of Technology, Management and Economics, Technical University of Denmark https://orcid.org/0000-0002-5542-3065

Anders Kristian Munk is a Professor of Computational Anthropology in the Section for Human-Centered Innovation at DTU Management (Denmark). His research focuses on controversies about emerging technologies, artificial intelligence and the green transition. Over the past decade, he has worked to integrate computational methods into qualitative traditions. He has co-founded the Public Data Lab, The Techno-Anthropology Lab, and MASSHINE (Aalborg University’s hub for computational social science and humanities), the latter two of which he has also directed. He holds a DPhil in Geography from the University of Oxford (UK) and has been a visiting researcher at SciencesPo (France).
Tommaso Venturini Medialab, University of Geneva https://orcid.org/0000-0003-0004-5308

Tommaso Venturini is a Researcher at the CNRS Centre for Internet and Society (France), Associate Professor at the Medialab of the University of Geneva (Switzerland), and founder of the Public Data Lab. In 2017 and 2018, Tommaso has been a researcher at the École Normale Supérieure of Lyon (France) and recipient of the Advanced Research fellowship of the French Institute for Research in Computer Science and Automation. In 2016, he was Digital Methods Lecturer at the Department of Digital Humanities of King’s College London (UK). From 2009 to 2015, he coordinated the research activities of the médialab of SciencesPo Paris (France).

DOI:

https://doi.org/10.6092/issn.1971-8853/20378

Keywords:

Artificial intelligence, generative AI, digital methods, repurposing, social research

Abstract

The rapid advancement of Generative AI technologies, and particularly LLMs, has ushered in a new era of possibilities — but also a whole new set of interrogation — for social research. This symposium brings together a set of contributions that collectively explore the diverse ways in which Generative AI could be “repurposed” in a digital methods fashion.

References

Atari, M., Xue, M.J., Park, P.S., Blasi, D.E., & Henrich, J. (2023). Which Humans? (Culture, Cognition, Coevolution Lab Working Paper). Department of Human Evolutionary Biology, Harvard University. https://doi.org/10.31234/osf.io/5b26t

Bail, C.A. (2024). Can Generative AI Improve Social Science?. Proceedings of the National Academy of Sciences of the United States of America, 121(21), e2314021121. https://doi.org/10.1073/pnas.2314021121

Buttrick, N. (2024). Studying Large Language Models as Compression Algorithms for Human Culture. Trends in Cognitive Sciences, 28(3), 187–189. https://doi.org/10.1016/j.tics.2024.01.001

Chang, K.K., Cramer, M.H., Soni, S., & Bamman, D. Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4. In H. Bouamor, J. Pino, & K. Bali (Eds.) Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 7312–7327). Singapore: Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.453

Chopra, F., & Haaland, I. (2023). Conducting Qualitative Interviews with AI. (CESifo Working Paper No. 10666). Munich Society for the Promotion of Economic Research. https://doi.org/10.2139/ssrn.4583756

Cioni, D., Berlincioni, L., Becattini, F., & Del Bimbo, A. (2023). Diffusion Based Augmentation for Captioning and Retrieval in Cultural Heritage. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1699–1708). Paris: IEEE Press. https://doi.ieeecomputersociety.org/10.1109/ICCVW60793.2023.00186

de Seta, G. (2024). Synthetic Probes: A Qualitative Experiment in Latent Space Exploration. Sociologica, 18(2), 9–23. https://doi.org/10.6092/issn.1971-8853/19512

Do, S., Ollion, É., & Shen, R. (2024). The Augmented Social Scientist: Using Sequential Transfer Learning to Annotate Millions of Texts with Human-Level Accuracy. Sociological Methods & Research, 53(3), 1167–1200. https://doi.org/10.1177/00491241221134526

Ellis, A.R., & Slade, E. (2023). A New Era of Learning: Considerations for ChatGPT as a Tool to Enhance Statistics and Data Science Education. Journal of Statistics and Data Science Education, 31(2), 128–133. https://doi.org/10.1080/26939169.2023.2223609

Esposito, E. (2022). Artificial Communication: How Algorithms Produce Social Intelligence. Cambridge, MA: MIT Press.

Gilardi, F., Alizadeh, M., & Kubli, M. (2023). ChatGPT Outperforms Crowd Workers for Text-Annotation Tasks. Proceedings of the National Academy of Sciences, 120(30), e2305016120. https://doi.org/10.1073/pnas.2305016120

Götz, F.M., Maertens, R., Loomba, S., & van der Linden, S. (2023). Let the Algorithm Speak: How to Use Neural Networks for Automatic Item Generation in Psychological Scale Development. Psychological Methods, 29(3), 494–518. https://doi.org/10.1037/met0000540

Jacomy, M., & Borra, E. (2024). Measuring LLM Self-consistency: Unknown Unknowns in Knowing Machines. Sociologica, 18(2), 25–65. https://doi.org/10.6092/issn.1971-8853/19488

Khandelwal, K., Tonneau, M., Bean, A.M., Kirk, H.R., & Hale, S.A. (2024). Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Models. In GoodIT ’24: Proceedings of the 2024 International Conference on Information Technology for Social Good (pp. 231–239). New York, NY: Association for Computing Machinery. https://doi.org/10.1145/3677525.3678666

Marino, G., & Giglietto, F. (2024). Integrating Large Language Models in Political Discourse Studies on Social Media: Challenges of Validating an LLMs-in-the-loop Pipeline. Sociologica, 18(2), 87–107. https://doi.org/10.6092/issn.1971-8853/19524

Marres, N. (2015). Why Map issues? On Controversy Analysis as a Digital Method. Science, Technology, & Human Values, 40(5), 655–686. https://doi.org/10.1177/0162243915574602

Masoud, R.I., Liu, Z., Ferianc, M., Treleaven, P., & Rodrigues, M. (2023). Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede’s Cultural Dimensions. arXiv. https://doi.org/10.48550/arXiv.2309.12342

Munk, A.K. (2023). Coming of age in Stable Diffusion. Anthropology News, 64(2). https://www.anthropology-news.org/articles/coming-of-age-in-stable-diffusion/

Munk, A.K., Jacomy, M., Ficozzi, M., & Jensen, T.E. (2024). Beyond Artificial Intelligence Controversies: What Are Algorithms Doing in the Scientific Literature? Big Data & Society, 11(3), 1–20. https://doi.org/10.1177/20539517241255107

Nelson, L.K. (2021). Leveraging the Alignment Between Machine Learning and Intersectionality: Using Word Embeddings to Measure Intersectional Experiences of the Nineteenth Century US South. Poetics, 88, 101539. https://doi.org/10.1016/j.poetic.2021.101539

Omena, J.J. (2024). AI Methodology Map. Practical and Theoretical Approach to Engage with GenAI for Digital Methods-led Research. Sociologica, 18(2), 109–144. https://doi.org/10.6092/issn.1971-8853/19566

Rogers, R. (2009). The End of the Virtual: Digital Methods. Amsterdam: Amsterdam University Press.

Rogers, R. (2015). Digital Methods for Web Research. In R. Scott & S. Kosslyn (Eds.) Emerging Trends in the Social and Behavioral Sciences. Hoboken, NJ: Wiley. https://doi.org/10.1002/9781118900772.etrds0076

Rogers, R., & Marres, N. (2000). Landscaping Climate Change: A Mapping Technique for Understanding Science and Technology Debates on the World Wide Web. Public Understanding of Science, 9(2), 141–163. https://doi.org/10.1088/0963-6625/9/2/304

Rossi, L., Shklovski, I., & Harrison, K. (2024). Applications of LLM-generated Data in Social Science Research. Sociologica, 18(2), 145–168. https://doi.org/10.6092/issn.1971-8853/19576

Suchman, L. (2023). The Controversial ‘Thingness’ of AI. Big Data & Society, 10(2), 1–5. https://doi.org/10.1177/20539517231206794

Taylor, Z W. (2024). Using Chat GPT to Clean Interview Transcriptions: A Usability and Feasibility Analysis. American Journal of Qualitative Research, 8(2), 153–160. https://doi.org/10.29333/ajqr/14487

Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Tamkin, A., Durmus, E., Hume, T., Mosconi, F., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., Jermyn, A., Carter, S., Olah, C., Henighan, T. (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Anthropic. https://transformer-circuits.pub/2024/scaling-monosemanticity

Törnberg, P. (2024). Best Practices for Text Annotation with Large Language Models. Sociologica, 18(2), 67–85. https://doi.org/10.6092/issn.1971-8853/19461

Venturini, T. (2023). Bruno Latour and Artificial Intelligence. Tecnoscienza – Italian Journal of Science & Technology Studies, 14(2), 101–114. https://doi.org/10.6092/issn.2038-3460/18359

Venturini, T., Bounegru, L., Gray, J., & Rogers, R. (2018). A Reality Check (list) for Digital Methods. New Media & Society, 20(11), 4195–4217. https://doi.org/10.1177/1461444818769236

Weltevrede, E., & Borra, E. (2016). Platform Affordances and Data Practices: The Value of Dispute on Wikipedia. Big Data & Society, 3(1). https://doi.org/10.1177/2053951716653418

Zhu, L., Mou, W., Lai, Y., Lin, J., & Luo, P. (2024). Language and Cultural Bias in AI: Comparing the Performance of Large Language Models Developed in Different Countries on Traditional Chinese Medicine Highlights the Need for Localized Models. Journal of Translational Medicine, 22(1). https://doi.org/10.1186/s12967-024-05128-4

Generative AI for Social Research: Going Native with Artificial Intelligence

Authors

DOI:

Keywords:

Abstract

References

Downloads

Published

How to Cite

Issue

Section

License

Information

Make a Submission

Keywords

Current Issue