Lab Members

Haizhou LI
School of Data Science, The Chinese University of Hong Kong (Shenzhen)
X.Q. Deng Presidential Chair Professor
Executive Dean
SAEng Fellow
Chief Scientist of Shenzhen Big Data Research Institute

Zhizheng Wu
School of Data Science, The Chinese University of Hong Kong (Shenzhen)
Associate Professor
Research Field: Speech Processing, Speech Synthesis, DeepFake Detection

Shuai Wang
SRIBD Research Scientist
Research Field: Speech processing, Speaker recognition, Voice conversion and Speech synthesis

Siqi Cai
National University of Singapore
Postdoc research fellow
Research Field: Brain-like coding and decoding theory and application of auditory cognition

Feng Jiang
The Chinese University of Hong Kong (Shenzhen)
Postdoc research fellow
Research Field: Artificial intelligence, Natural language processing, Large language models, Discourse parsing, Dialogue systems, Summarization

Mingyang Zhang
The Chinese University of Hong Kong (Shenzhen)
Postdoc research fellow
Research Field: Speech synthesis and voice conversion

Event Review

EEG Project: E-ASD (EEG-based Speaker Detection)

The human brain features a unique "speech separation system" that enables it to discern and concentrate on specific voices within complex acoustic scenes. However, current artificial intelligence has not yet achieved precise isolation of targeted speech. The Brain-like Auditory Attention Platform* aims to develop intelligent auditory devices by investigating how the brain processes sounds, thereby emulating human auditory attention and accomplishing speech separation.

*Brain-like Auditory Attention Platform: Developed independently by the Human Language Technology Lab team, the platform utilises multimodal signal interactions involving speech, brainwaves, and eye movements. This platform is designed to enhance auditory attention detection in hearing aids, improving the speech quality output and enabling those with hearing impairments to listen and communicate effectively in complex auditory environments. This advancement significantly enhances the quality of life for those with hearing loss and is anticipated to yield substantial economic benefits upon commercialisation.

The experiment utilises EEG equipment to identify and analyse the level of attention listeners pay to specific speakers. E-ASD technology, through the precise interpretation of EEG signals, can capture the listener's focus on different speaking voices in real-time. This technology has vast potential applications, from assisting individuals with hearing impairments to improving efficiency in multitasking environments.

Lab personnel detailed the experimental design process and the usage of EEG equipment, demonstrating real-time monitoring of brain wave fluctuations. They also highlighted challenges such as data collection and the difficulty of interdisciplinary collaboration. During interactions with students, one researcher shared insights from his scientific journey: "In the era of deep learning, much knowledge is interconnected, yet each field has its unique characteristics. The more a field is related to basic science, the more challenging it is to achieve digitisation." They also encouraged students to actively explore and cultivate their research interests by making full use of the rich experimental resources available.

Eye Tracker Experiment: The Impact of Visual Information on Auditory Attention

Does the familiar timbre of a voice affect auditory attention? Do different volumes and pitches influence our selective hearing? How might the interplay between human vision and hearing be applied in the field of artificial intelligence?

Through the eye tracker experiment, lab personnel guided students in exploring how visual information affects auditory attention. They clearly articulated the experimental design and preparation process, demonstrating the operation of the eye tracker. As the focal point of subjects was adjusted, the eye tracker precisely recorded and marked data changes, leading to conclusions drawn from both objective and subjective analyses.

Experimental Equipment: Eye Tracker

The laboratory staff introduced that the preparation of materials for the eye-tracking experiment has evolved from "extracting information from a paragraph" to the current method of having a broadcaster announce numerical dates, which has undergone repeated formal experiments, communication with subjects, and adjustments. What appears to be a simple experimental process requires months of refinement by the experimenters. From experimental design to material selection, instrument use, and data analysis, every detail reflects the researchers' passion for their work, their meticulous attention to detail, and the powerful, albeit subtle, emergence of new voices in the field.

Event Impressions

"The laboratory visit was incredibly enlightening. As I delved into the experimental process, the use of equipment, and the development of the research field from the perspective of an experimenter, my understanding of 'auditory attention' deepened. Starting from the study of human functions, it advances the deep imitation of human hearing by artificial intelligence. The innovative spirit and passion for research of the senior students deeply moved me, and I hope to have the opportunity to participate in such experiments."

——Liu

"I am very grateful to the SSC for organising this event, allowing us to engage with interesting experiments and patient, responsible senior students. I am thankful for the exposure to interesting disciplines outside of my major and to projects like the eye-tracking device, which were developed by the senior students themselves. The experience was profound. I also hope that such events will continue in the future."

——Pan