Conversational recommendation based on end-to-end learning: How far are we?
Conversational recommender systems (CRS) are software agents that support users in their decision-making process in an interactive way. While such systems were traditionally mostly manually engineered, recent works increasingly rely on machine learning models that are trained on larger corpora of re...
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
Elsevier
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/3ec5dc2810de48008678313db4da4c0a |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:3ec5dc2810de48008678313db4da4c0a |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:3ec5dc2810de48008678313db4da4c0a2021-12-01T05:04:52ZConversational recommendation based on end-to-end learning: How far are we?2451-958810.1016/j.chbr.2021.100139https://doaj.org/article/3ec5dc2810de48008678313db4da4c0a2021-08-01T00:00:00Zhttp://www.sciencedirect.com/science/article/pii/S2451958821000877https://doaj.org/toc/2451-9588Conversational recommender systems (CRS) are software agents that support users in their decision-making process in an interactive way. While such systems were traditionally mostly manually engineered, recent works increasingly rely on machine learning models that are trained on larger corpora of recorded recommendation dialogues between humans. One promise of such end-to-end learning approaches therefore is that they avoid the knowledge-engineering bottlenecks of traditional systems. Recent empirical evaluations of such learning-based systems sometimes demonstrate continuous progress relative to previous systems. Therefore, it may not be entirely clear how useable these systems are on an absolute scale. To address this research question, we evaluated two recent end-to-end learning approaches presented at top-tier scientific conferences with the help of human judges. A first study showed that in both investigated systems about one third of the system responses were not considered meaningful in the given dialogue context, which questions the applicability of these systems in practice. In a second study, we benchmarked the two systems against a trivial rule-based approach, again with human judges. In this second study, the participants considered the quality of the responses of the rule-based approach significantly better on average than those of the learning-based systems. Overall, besides pointing to open challenges of state-of-the-art learning-based approaches, our studies indicate that we must improve our evaluation methodology for CRS to ensure progress in this field.1Ahtsham ManzoorDietmar JannachElsevierarticleConversational recommender systemsEvaluationEnd-to-end learningElectronic computers. Computer scienceQA75.5-76.95PsychologyBF1-990ENComputers in Human Behavior Reports, Vol 4, Iss , Pp 100139- (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
Conversational recommender systems Evaluation End-to-end learning Electronic computers. Computer science QA75.5-76.95 Psychology BF1-990 |
| spellingShingle |
Conversational recommender systems Evaluation End-to-end learning Electronic computers. Computer science QA75.5-76.95 Psychology BF1-990 Ahtsham Manzoor Dietmar Jannach Conversational recommendation based on end-to-end learning: How far are we? |
| description |
Conversational recommender systems (CRS) are software agents that support users in their decision-making process in an interactive way. While such systems were traditionally mostly manually engineered, recent works increasingly rely on machine learning models that are trained on larger corpora of recorded recommendation dialogues between humans. One promise of such end-to-end learning approaches therefore is that they avoid the knowledge-engineering bottlenecks of traditional systems. Recent empirical evaluations of such learning-based systems sometimes demonstrate continuous progress relative to previous systems. Therefore, it may not be entirely clear how useable these systems are on an absolute scale. To address this research question, we evaluated two recent end-to-end learning approaches presented at top-tier scientific conferences with the help of human judges. A first study showed that in both investigated systems about one third of the system responses were not considered meaningful in the given dialogue context, which questions the applicability of these systems in practice. In a second study, we benchmarked the two systems against a trivial rule-based approach, again with human judges. In this second study, the participants considered the quality of the responses of the rule-based approach significantly better on average than those of the learning-based systems. Overall, besides pointing to open challenges of state-of-the-art learning-based approaches, our studies indicate that we must improve our evaluation methodology for CRS to ensure progress in this field.1 |
| format |
article |
| author |
Ahtsham Manzoor Dietmar Jannach |
| author_facet |
Ahtsham Manzoor Dietmar Jannach |
| author_sort |
Ahtsham Manzoor |
| title |
Conversational recommendation based on end-to-end learning: How far are we? |
| title_short |
Conversational recommendation based on end-to-end learning: How far are we? |
| title_full |
Conversational recommendation based on end-to-end learning: How far are we? |
| title_fullStr |
Conversational recommendation based on end-to-end learning: How far are we? |
| title_full_unstemmed |
Conversational recommendation based on end-to-end learning: How far are we? |
| title_sort |
conversational recommendation based on end-to-end learning: how far are we? |
| publisher |
Elsevier |
| publishDate |
2021 |
| url |
https://doaj.org/article/3ec5dc2810de48008678313db4da4c0a |
| work_keys_str_mv |
AT ahtshammanzoor conversationalrecommendationbasedonendtoendlearninghowfararewe AT dietmarjannach conversationalrecommendationbasedonendtoendlearninghowfararewe |
| _version_ |
1718405536695713792 |