The great Transformer: Examining the role of large language models in the political economy of AI

In recent years, AI research has become more and more computationally demanding. In natural language processing (NLP), this tendency is reflected in the emergence of large language models (LLMs) like GPT-3. These powerful neural network-based models can be used for a range of NLP tasks and their lan...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Dieuwertje Luitse, Wiebke Denkena
Formato:	article
Lenguaje:	EN
Publicado:	SAGE Publishing 2021
Materias:	General Works A
Acceso en línea:	https://doaj.org/article/d452201d701947dc85a5502a0c6122ed
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:d452201d701947dc85a5502a0c6122ed
record_format	dspace
spelling	oai:doaj.org-article:d452201d701947dc85a5502a0c6122ed2021-12-01T08:33:27ZThe great Transformer: Examining the role of large language models in the political economy of AI2053-951710.1177/20539517211047734https://doaj.org/article/d452201d701947dc85a5502a0c6122ed2021-07-01T00:00:00Zhttps://doi.org/10.1177/20539517211047734https://doaj.org/toc/2053-9517In recent years, AI research has become more and more computationally demanding. In natural language processing (NLP), this tendency is reflected in the emergence of large language models (LLMs) like GPT-3. These powerful neural network-based models can be used for a range of NLP tasks and their language generation capacities have become so sophisticated that it can be very difficult to distinguish their outputs from human language. LLMs have raised concerns over their demonstrable biases, heavy environmental footprints, and future social ramifications. In December 2020, critical research on LLMs led Google to fire Timnit Gebru, co-lead of the company’s AI Ethics team, which sparked a major public controversy around LLMs and the growing corporate influence over AI research. This article explores the role LLMs play in the political economy of AI as infrastructural components for AI research and development. Retracing the technical developments that have led to the emergence of LLMs, we point out how they are intertwined with the business model of big tech companies and further shift power relations in their favour. This becomes visible through the Transformer, which is the underlying architecture of most LLMs today and started the race for ever bigger models when it was introduced by Google in 2017. Using the example of GPT-3, we shed light on recent corporate efforts to commodify LLMs through paid API access and exclusive licensing, raising questions around monopolization and dependency in a field that is increasingly divided by access to large-scale computing power.Dieuwertje LuitseWiebke DenkenaSAGE PublishingarticleGeneral WorksAENBig Data & Society, Vol 8 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	General Works A
spellingShingle	General Works A Dieuwertje Luitse Wiebke Denkena The great Transformer: Examining the role of large language models in the political economy of AI
description	In recent years, AI research has become more and more computationally demanding. In natural language processing (NLP), this tendency is reflected in the emergence of large language models (LLMs) like GPT-3. These powerful neural network-based models can be used for a range of NLP tasks and their language generation capacities have become so sophisticated that it can be very difficult to distinguish their outputs from human language. LLMs have raised concerns over their demonstrable biases, heavy environmental footprints, and future social ramifications. In December 2020, critical research on LLMs led Google to fire Timnit Gebru, co-lead of the company’s AI Ethics team, which sparked a major public controversy around LLMs and the growing corporate influence over AI research. This article explores the role LLMs play in the political economy of AI as infrastructural components for AI research and development. Retracing the technical developments that have led to the emergence of LLMs, we point out how they are intertwined with the business model of big tech companies and further shift power relations in their favour. This becomes visible through the Transformer, which is the underlying architecture of most LLMs today and started the race for ever bigger models when it was introduced by Google in 2017. Using the example of GPT-3, we shed light on recent corporate efforts to commodify LLMs through paid API access and exclusive licensing, raising questions around monopolization and dependency in a field that is increasingly divided by access to large-scale computing power.
format	article
author	Dieuwertje Luitse Wiebke Denkena
author_facet	Dieuwertje Luitse Wiebke Denkena
author_sort	Dieuwertje Luitse
title	The great Transformer: Examining the role of large language models in the political economy of AI
title_short	The great Transformer: Examining the role of large language models in the political economy of AI
title_full	The great Transformer: Examining the role of large language models in the political economy of AI
title_fullStr	The great Transformer: Examining the role of large language models in the political economy of AI
title_full_unstemmed	The great Transformer: Examining the role of large language models in the political economy of AI
title_sort	great transformer: examining the role of large language models in the political economy of ai
publisher	SAGE Publishing
publishDate	2021
url	https://doaj.org/article/d452201d701947dc85a5502a0c6122ed
work_keys_str_mv	AT dieuwertjeluitse thegreattransformerexaminingtheroleoflargelanguagemodelsinthepoliticaleconomyofai AT wiebkedenkena thegreattransformerexaminingtheroleoflargelanguagemodelsinthepoliticaleconomyofai AT dieuwertjeluitse greattransformerexaminingtheroleoflargelanguagemodelsinthepoliticaleconomyofai AT wiebkedenkena greattransformerexaminingtheroleoflargelanguagemodelsinthepoliticaleconomyofai
_version_	1718405382589644800

The great Transformer: Examining the role of large language models in the political economy of AI

Ejemplares similares