Overview

Developing intelligent dialog systems has been one of the longest running goals in AI. Significant progress has been made in building dialog systems with deep learning methods and more recently with large language model. However, there are still full of challenges toward building future dialog systems. The 1st challenge FutureDial-SereTOD focused on building semi-supervised and reinforced task-oriented dialog systems, which was successfully held at EMNLP 2022 SereTOD workshop. Retrieval augmented generation (RAG) has been introduced to enhance dialog systems with higher accuracy and factuality, providing more informative and grounded responses. However, there remain challenges for RAG-based dialog systems such as retrieving knowledge from multiple knowledge sources, building RAG-based dialog systems with available tools and API-calls, and etc. To further promote the study of how to empower dialog systems with RAG, we organize the 2nd challenge FutureDial-RAG. This challenge aims to benchmark and stimulate research in building dialog systems with RAG, with the newly released dialog dataset, MobileCS2 (Mobile Customer Service), in both Chinese and English versions. The challenge includes two tracks: Information retrieval based on knowledge bases and dialog context (Track 1), and Dialog systems with retrieval augmented generation (Track 2).

Check out FutureDial-RAG

News

2025/01/07  To facilitate future research, we re-open the challenge, and people who are interested in building systems for the challenge task can register and send email to FutureDialRAG (at) gmail (dot) com to obtain the data. The end date is open and the organizers reserve the right as to when to end the challenge.

2024/12/01  The program for the FutureDial-RAG Challenge Special Session at SLT 2024 is posted here.

2024/07/23  The test data with labels for both Track 1 and Track 2 have been released via email.

2024/07/20  We are pleased to announce the evaluation results for both Track 1 and Track 2, which can be accessed via this link. Congratulations to the teams who submitted effective results for FutureDial-RAG Challenge!

2024/05/10  Registration deadline is extended by one month to June 9; Accordingly, the due dates for entry submission and evaluation announcement are also extended.

2024/05/07  The code repository for baseline systems and evaluation scripts for both Track 1 and Track 2 is released.

2024/04/29  The MobileCS2 dataset for the challenge has been released to registered teams. Challenge description paper is available here.

2024/04/09  Registration opening for challenge! To register for the challenge, please fulfill the form (English form, Chinese form) and send to FutureDialRAG (at) gmail (dot) com to complete the registration. Registration deadline is May 9, 2024. Welcome to join FutureDial Google Group, where we will distribute messages relevant to the Challenge.

Important Dates (AOE)

April 9, 2024 Registration opening for challenge
April 29, 2024 Training data release
June 20, 2024 SLT paper submission due
June 27, 2024 SLT paper update due
July 10, 2024 Entry submission deadline
July 20, 2024 Evaluation results announced
August 30, 2024 Notification of paper acceptance
December 4, 2024 SLT 2024 Special Session (in-person)

Contact

FutureDialRAG (at) gmail (dot) com

Copyright © FutureDial-RAG Organizers. All rights reserved.