To improve QoE for users in RTC systems, the ACM MMSys 2024 grand challenge focuses on learning a fully data-driven bandwidth estimator using offline reinforcement learning based on a real-world dataset of packet traces with objective metrics that reflect user-perceived audio/video quality in Microsoft Teams.
Offline reinforcement learning (RL) is a variant of RL where the agent learns from a fixed dataset of previously collected experiences, without interacting with the environment during training. In offline RL, the goal is to learn a policy that maximizes the expected cumulative reward based on the data. Offline RL is different from online RL where the agent can interact with the environment using its updated policy and learn from the feedback it receives online.
In this challenge, participants are provided with a dataset of real-world trajectories for Microsoft Teams audio/video calls. Each trajectory corresponds to the sequence of high-dimensional observation vector computed based on packet information received by the client in one audio/video call along with the bandwidth estimates. In addition, objective signals which capture the user-perceived audio/video quality during the call are provided. This dataset is based on calls with different bandwidth estimators (behaviour policies), including traditional and ML (machine learning) policies. The task of the challenge is to train a policy model (receiver side bandwidth estimator) which maps observations (observed network statistics) to actions (bandwidth estimates) to improve QoE for users. To this end, participants are free to define the agent state-action space and reward function based on the provided data and use offline RL techniques, such as imitation learning, conservative Q-learning, inverse reinforcement learning, and constrained policy optimization, to train a deep learning-based model for bandwidth estimation.
Participants with queries related to this grand challenge can either contact Sami Khairy by email or create an issue on the Github repository.
Please refer to the important dates page.