【导师基本情况】

姓名: 余婷 副教授 硕士生导师

邮箱：yut@hznu.edu.cn

指导专业：计算机科学与技术、网络空间安全、密码、电子信息

课题组网站：https://openmicg.github.io/OpenMICG

【研究领域】

多模态内容理解：面向图像、视频、文本、语音和三维场景等多模态数据，研究跨模态感知、语义对齐、表示学习与知识增强方法，重点关注多模态大模型、视觉语言模型、视频理解、视觉问答、跨媒体检索和三维场景理解与生成。

智能体与具身交互：面向开放环境中的自主感知、交互与决策，研究视觉、语言、动作和环境状态之间的协同建模机制，重点关注具身智能、视觉-语言-动作模型、智能体导航、任务规划、物理常识建模和长期记忆机制。

跨媒体可信智能： 面向多模态大模型和跨媒体智能系统的实际部署，研究模型可信性、安全性、隐私保护与可解释性，提升智能系统在开放环境中的可靠性、透明性与可控性。

【主讲课程】

计算机网络、Python程序设计、移动应用开发、数据结构等。

【学术简介】

主要从事多模态内容理解、跨媒体智能、视觉问答、视觉语言导航、具身智能与可信人工智能等方向的研究，重点关注多模态大模型驱动的视觉-语言理解、跨模态推理与智能体交互。现担任 CCF 多媒体技术执行委员、CCF 自然语言处理专委委员、CCF 智能机器人专委委员、CCF 计算机视觉专委委员及 CCF YOCSEF 委员。长期担任 IEEE TIP、TNNLS、TCSVT 等国际期刊审稿人，以及 CVPR、ICCV、AAAI、ACM MM 等国际会议 PC 或审稿人。近年来，在 IEEE TIP、TMM、TMI、IJCV 等国际期刊及 CVPR、AAAI 等 CCF A 类会议上发表论文 20 余篇。主持国家自然科学基金面上项目、青年项目、浙江省自然科学基金面上项目等科研项目，并作为课题骨干参与多项国家级和省部级科研项目。

【主持科研项目】

[1]国家自然科学基金面上项目：开放域下长时视频跨模态可信问答技术研究(62572164)，2026.01-2029.12

[2]国家自然科学基金青年项目：基于跨媒体层次深度推理的视频问答技术研究(62002314)，2020.01-2023.12

[3]浙江省自然科学基金面上项目：跨媒体“数据-知识”联合增强的视频问答技术研究(LY23F020005), 2023.01-2025.12

【代表性论著】

[1] T. Yu, Y. Lin, J. Yu, Z. Lou, Q. Cui, “Vision-Guided Action: Enhancing 3D Human Motion Prediction with Gaze-informed Affordance in 3D Scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. (CCF A)

[2]  T. Yu, Z. Tong, J. Yu, K. Zhang, “Fine-grained Adaptive Visual Prompt for Generative Medical Visual Question Answering,” in Proceeding of the 39th Annual AAAI Conference on Artificial Intelligence (AAAI), 2025. (CCF A)

[3]  T. Yu, Y. Wu, Q. Cui, Q. Huang, J. Yu, “MossVLN: Memory-Observation Synergistic System for Continuous Vision-Language Navigation,” in IEEE Transactions on Multimedia (TMM), 2025. (CCF A, SCI 中科院一区 TOP)

[4]  T. Yu, W. Lu, Y. Yang, W. Han, Q. Huang, J. Yu, " Adapter-Enhanced Hierarchical Cross-Modal Pre-training for Lightweight Medical Report Generation," in IEEE Journal of Biomedical and Health Informatics (JBHI), 2025. (SCI 中科院一区 TOP)

[5]  T. Yu, K. Fu, J. Zhang, Q. Huang, J. Yu, "Multi-Granularity Contrastive Cross-Modal Collaborative Generation for End-to-End Long-Term Video Question Answering," in IEEE Transactions on Image Processing (TIP), vol. 33, pp. 3115-3129, 2024. (CCF A, SCI 中科院一区 TOP)

[6]  T. Yu, K. Fu, S. Wang, Q. Huang, J. Yu, "Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering," in IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2024. (SCI 中科院一区 TOP)

[7]  T. Yu, X. Lin, S. Wang, W. Sheng, Q. Huang, J. Yu, "A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes," in IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), vol. 34, no. 3, pp. 1322-1338, 2024. (SCI 中科院一区 TOP)

[8]  T. Yu, B. Ge, S. Wang, Y. Yang, Q. Huang, J. Yu, "Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering," in IEEE Journal of Biomedical and Health Informatics (JBHI), 2024. (SCI 中科院一区 TOP)

[9]  T. Yu; J. Yu; Z. Yu; Q. Huang; Q. Tian; Long-Term Video Question Answering via Multimodal Hierarchical Memory Attentive Networks, IEEE Transactions on Circuits and Systems for Video Technology (TCSVT),2021,31(3):1051-8215. (SCI 中科院一区 TOP)

[10]  T. Yu; J. Yu; Z. Yu; D. Tao; Compositional Attention Networks with Two-Stream Fusion for Video Question Answering, IEEE Transactions on Image Processing (TIP), 2019, 29(1): 1204-1218. (CCF A, SCI 中科院一区 TOP)

[11] W. Yang, K. Zhang, Z. Kuang, T. Yu*, “Aegis: Reconciling Global Path Navigation with Local Fine-Grained Interaction in 3D Environments,” in Proceedings of the 34rd ACM International Conference on Multimedia (ACM MM), 2026. (CCF A)

[12]  Z. Tang, J. Lu, Z. Kuang*, T. Yu*, A. Liew, X. Yin, Y. Zhu, “3DGFA: 3D Gaze-consistent Face Anonymization,” in Proceedings of the 34rd ACM International Conference on Multimedia (ACM MM), 2026. (CCF A)

[13]  Y Lin, W Xia, T Yu*; HuMo3D: Intention-Driven Dual-Branch Multimodal Human Motion Prediction in 3D Scenes, IEEE Signal Processing Letters (SPL), 2026. (SCI, JCR Q1)

[14]  Z Dong, T Yu*; SwiftCraft3D: semantic-enhanced multi-view prompting for efficient and high-fidelity text-to-3D generation, The Visual Computer, 2026. (SCI, JCR Q1)

[15]  S Fu, Y Wu, T Yu*; WP-CMA: Waypoint Prediction for Cross-modal Alignment of Vision-and-Language Navigation in Continuous Environments, ACM International Conference on Multimedia in Asia, 2026.

[16]  Z Niu, Z Dong, T Liu, Y Yang, T. Yu*; CloudCap3D: enhancing 3D in-scene descriptions via point cloud integration and efficient text filtering. Multimedia Systems, 2026. (SCI, JCR Q1)

[17]  X. Dong, J. Zhang, J. Yu, T. Yu*; 3D human pose estimation with multi-hypotheses gated transformer. Multimedia Systems 30, 309, 2024. (SCI, JCR Q1)

[18]  J. Zhang, K. He, T. Yu, J. Yu, Z. Yuan; Semi-supervised RGB-D hand gesture recognition via mutual learning of self-supervised models. ACM Transactions on Multimedia Computing, Communications and Applications, 2025. (SCI, JCR Q1)

[19]  Y. Yang, J. Yu, Z. Fu, K. Zhang, T. Yu, X. Wang, H. Jiang, J. Lv, Q. Huang, W. Han, "Token-Mixer: Bind Image and Text in One Embedding Space for Medical Image Reporting," in IEEE Transactions on Medical Imaging (TMI), vol. 43, no. 11, pp. 4017-4028, 2024. (SCI 中科院一区 TOP)

[20]  Y. Zhan; J. Yu; T. Yu; D. Tao; Multi-task Compositional Network for Visual Relationship Detection, International Journal of Computer Vision (IJCV), 2020,128:2146-2165. (SCI中科院一区 TOP）

[21]  Y. Zhan, J. Yu, T. Yu, D. Tao, On Exploring Undetermined Relationships for Visual Relationship Detection, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, 2019. (CCF A)

[22]  Z. Yu, D. Xu, J. Yu, T. Yu, Z. Zhao, Y. Zhuang; D. Tao; ActivityNet-QA: A dataset for understanding complex Web videos via question answering, AAAI Conference on Artificial Intelligence (AAAI), Hawaii, USA, 2019. (CCF A)

【授权专利及转化】

[1]一种跨媒体层次深度视频问答推理框架,202011499931.2

[2]一种面向长跨度视频问答的多粒度对比学习协同生成方法,CN202410280286.7  

[3]一种多层次跨媒体融合的视觉语言导航方法,CN202410915863.5  

[4]一种基于语义增强混合重建的三维生成方法,CN202411384389.4  

[5]一种基于细粒度视觉提示的医学视觉问答推理方法,CN202411384380.3

[6]一种特定域细粒度启发提示的视频问答方法,CN202411373766.4 

[7]一致性约束下的记忆动态化医学图像问答分类系统及方法,CN202410462542.4

[8]一种轻量级空间适配器增强的医学报告生成方法,CN202411007351.5 

[9]一种视线引导可供性感知的三维场景人类运动预测方法, CN120726090A

[10]一种通过利用点云增强3D场景描述的文本生成方法, CN120146055A

【欢迎加入】

欢迎对多模态大模型、视觉语言理解、视频问答、具身智能、三维视觉与可信人工智能感兴趣的同学加入课题组。希望你积极主动、自律踏实、基础扎实、乐于合作，并对人工智能前沿研究保持热情。课题组将提供系统科研训练，支持学生参与科研项目、发表高水平论文、参加学科竞赛，并鼓励继续赴国内外高水平高校和科研机构深造。