Robot Perception & IoT - Research Project

Research Project

Heterogeneous Robot Teamwork Through VR Control

This subproject focuses on remote control, task collaboration, and wireless communication for multi-robot teams, establishing a VR operation system that supports collaborative work among heterogeneous robots. The system enables the operator to control different platforms through intuitive hand gestures in an immersive interface, while using a digital twin to understand each robot’s position, status, and task relationships in real time. The overall architecture emphasizes functional task allocation and communication capability, allowing the robot team to exchange control commands, image streams, and status information within the same network, while each platform retains its own computation and execution capabilities. The project has been validated on multiple robotic platforms and has completed load testing of compressed RGB-D streams from multiple robots, confirming that the system can support multi-robot data transmission. This system is expected to serve as a foundation for robot collaboration, allowing additional robots to be integrated into the same robot team at lower cost in the future.

Project Details

Principal Investigator

Ching-I Huang Assistant Professor

Research Field: robot-learning methodologies and virtual reality technologies aimed at enhancing human-robot cooperation

PI Details

Research Project

Collective Collaborative Power Saving

This sub-project develops a resource allocation and communication scheduling mechanism centered on "extreme power saving" for Robotic IoT. By analyzing environmental and task requirements, the system intelligently dispatches workloads, transferring power-intensive tasks to fully powered devices. This significantly reduces the overall energy consumption of mobile robots, resolves the pain point of premature battery depletion, and extends the operational time and lifecycle of the system.

Project Details

Principal Investigator

Kuang-Hsun Lin Assistant Professor

Research Field: 6G Mobile Network, MAC Protocols, Energy Efficiency, Mobility Management, Non-terrestrial Network

PI Details

Research Project

Collaborative Perception And Semantic Occupancy Construction For Intelligent Team Robots

This subproject focuses on collaborative perception and semantic occupancy construction for intelligent team robots. The goal is to improve object detection, spatial understanding, and global semantic map construction in multi-robot, multi-view, and communication-constrained environments. In 2025, we established a multi-robot, multi-view collaborative object detection capability by integrating primary and auxiliary robot imagery through a shared encoder and multi-view feature fusion, thereby improving perception coverage under occlusion and limited field-of-view conditions. We also built a multi-agent semantic occupancy construction pipeline, where each agent first predicts local occupancy and then aligns the local prediction to the world frame for fusion into a global semantic map. In 2026, we will further move from late fusion to feature-level collaborative perception with adaptive transmission, bandwidth-aware alignment, uncertainty modeling, and delay-robust fusion. The project aims to build a unified framework that jointly supports communication-efficient object detection and semantic occupancy prediction for downstream planning, collaborative control, and future field deployment.

Project Details

Principal Investigator

Hong-Han Shuai Professor

Research Field: National Yang Ming Chiao Tung University Department of Electrical and Computer Engineering

PI Details

Research Project

Multi-Modal Wireless-Based Cooperative Sensing And Learning Framework For Mobile Robots

This sub-project aims to develop a multi-modal wireless-based cooperative sensing and learning framework for mobile robots, applied to human state detection in smart healthcare environments. The system integrates mobile robot-assisted automatic data labeling and federated learning to achieve camera-free, 24/7 human anomaly monitoring while preserving privacy.

Project Details

Principal Investigator

Kai-Ten Feng Professor

Research Field: Wireless Comunication and Networking, Wireless Localization, Wireless Sensing

PI Details

Research Project

Intelligent Collaborative Integrated Sensing And Communication System

Dedicated to developing an intelligent collaborative integrated sensing and communication system that combines multimodal sensing, Wi-Fi and mobile communications, high-speed transmission interfaces, and dynamic resource allocation schemes. The system provides real-time, full-view sensing information required by collaborative robot teams and realizes a digital twin system with dynamically updated full-view perception. The outputs and functions can provide high-quality 3D modeling information for VLA models (Vision-Language-Action models).

Project Details

Principal Investigator

Ming-Chun Lee Associate Professor

Research Field: B5G and 6G mobile communications and networks、Edge AI、Integrated sensing and communication (ISAC) systems、AI-enhanced wireless communications and networking、Sustainable wireless systems and network、Intelligent sensing systems and signal processing

PI Details