Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments
This article discusses the usefulness of Toulmin’s model of arguments as structuring an assessment of different types of wrongness in an argument. We discuss the usability of the model within a conversational agent that aims to support users to develop a good argument. Within the article, we present...
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
Frontiers Media S.A.
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/cc6a553b2afb427f81b715aea8786f97 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:cc6a553b2afb427f81b715aea8786f97 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:cc6a553b2afb427f81b715aea8786f972021-12-03T15:31:21ZDeveloping a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments2624-821210.3389/frai.2021.645516https://doaj.org/article/cc6a553b2afb427f81b715aea8786f972021-11-01T00:00:00Zhttps://www.frontiersin.org/articles/10.3389/frai.2021.645516/fullhttps://doaj.org/toc/2624-8212This article discusses the usefulness of Toulmin’s model of arguments as structuring an assessment of different types of wrongness in an argument. We discuss the usability of the model within a conversational agent that aims to support users to develop a good argument. Within the article, we present a study and the development of classifiers that identify the existence of structural components in a good argument, namely a claim, a warrant (underlying understanding), and evidence. Based on a dataset (three sub-datasets with 100, 1,026, 211 responses in each) in which users argue about the intelligence or non-intelligence of entities, we have developed classifiers for these components: The existence and direction (positive/negative) of claims can be detected a weighted average F1 score over all classes (positive/negative/unknown) of 0.91. The existence of a warrant (with warrant/without warrant) can be detected with a weighted F1 score over all classes of 0.88. The existence of evidence (with evidence/without evidence) can be detected with a weighted average F1 score of 0.80. We argue that these scores are high enough to be of use within a conditional dialogue structure based on Bloom’s taxonomy of learning; and show by argument an example conditional dialogue structure that allows us to conduct coherent learning conversations. While in our described experiments, we show how Toulmin’s model of arguments can be used to identify structural problems with argumentation, we also discuss how Toulmin’s model of arguments could be used in conjunction with content-wise assessment of the correctness especially of the evidence component to identify more complex types of wrongness in arguments, where argument components are not well aligned. Owing to having progress in argument mining and conversational agents, the next challenges could be the developing agents that support learning argumentation. These agents could identify more complex type of wrongness in arguments that result from wrong connections between argumentation components.Behzad MirzababaeiViktoria Pammer-SchindlerViktoria Pammer-SchindlerFrontiers Media S.A.articleToulmin’s model of argumentargument miningargument quality detectioneducational technologyeducational conversational agentElectronic computers. Computer scienceQA75.5-76.95ENFrontiers in Artificial Intelligence, Vol 4 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
Toulmin’s model of argument argument mining argument quality detection educational technology educational conversational agent Electronic computers. Computer science QA75.5-76.95 |
| spellingShingle |
Toulmin’s model of argument argument mining argument quality detection educational technology educational conversational agent Electronic computers. Computer science QA75.5-76.95 Behzad Mirzababaei Viktoria Pammer-Schindler Viktoria Pammer-Schindler Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments |
| description |
This article discusses the usefulness of Toulmin’s model of arguments as structuring an assessment of different types of wrongness in an argument. We discuss the usability of the model within a conversational agent that aims to support users to develop a good argument. Within the article, we present a study and the development of classifiers that identify the existence of structural components in a good argument, namely a claim, a warrant (underlying understanding), and evidence. Based on a dataset (three sub-datasets with 100, 1,026, 211 responses in each) in which users argue about the intelligence or non-intelligence of entities, we have developed classifiers for these components: The existence and direction (positive/negative) of claims can be detected a weighted average F1 score over all classes (positive/negative/unknown) of 0.91. The existence of a warrant (with warrant/without warrant) can be detected with a weighted F1 score over all classes of 0.88. The existence of evidence (with evidence/without evidence) can be detected with a weighted average F1 score of 0.80. We argue that these scores are high enough to be of use within a conditional dialogue structure based on Bloom’s taxonomy of learning; and show by argument an example conditional dialogue structure that allows us to conduct coherent learning conversations. While in our described experiments, we show how Toulmin’s model of arguments can be used to identify structural problems with argumentation, we also discuss how Toulmin’s model of arguments could be used in conjunction with content-wise assessment of the correctness especially of the evidence component to identify more complex types of wrongness in arguments, where argument components are not well aligned. Owing to having progress in argument mining and conversational agents, the next challenges could be the developing agents that support learning argumentation. These agents could identify more complex type of wrongness in arguments that result from wrong connections between argumentation components. |
| format |
article |
| author |
Behzad Mirzababaei Viktoria Pammer-Schindler Viktoria Pammer-Schindler |
| author_facet |
Behzad Mirzababaei Viktoria Pammer-Schindler Viktoria Pammer-Schindler |
| author_sort |
Behzad Mirzababaei |
| title |
Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments |
| title_short |
Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments |
| title_full |
Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments |
| title_fullStr |
Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments |
| title_full_unstemmed |
Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments |
| title_sort |
developing a conversational agent’s capability to identify structural wrongness in arguments based on toulmin’s model of arguments |
| publisher |
Frontiers Media S.A. |
| publishDate |
2021 |
| url |
https://doaj.org/article/cc6a553b2afb427f81b715aea8786f97 |
| work_keys_str_mv |
AT behzadmirzababaei developingaconversationalagentscapabilitytoidentifystructuralwrongnessinargumentsbasedontoulminsmodelofarguments AT viktoriapammerschindler developingaconversationalagentscapabilitytoidentifystructuralwrongnessinargumentsbasedontoulminsmodelofarguments AT viktoriapammerschindler developingaconversationalagentscapabilitytoidentifystructuralwrongnessinargumentsbasedontoulminsmodelofarguments |
| _version_ |
1718373175232823296 |