Abstract:

Industrial assembly is shifting toward human-robot collaboration (HRC) to leverage the complementary strengths of both agents. However, traditional task allocation referred to as the Robotic Assembly Line Balancing Problem (RALBP) remains labor-intensive and often lacks transparency. We introduce DELEGACT, a framework designed to produce workable, intelligible human-cobot task allocations. The framework uses a Vision-Language Model (VLM) to extract atomic operations from expert demonstration videos, then employs a Large Language Model (LLM) to delegate these tasks based on robot specifications, operator competencies, and material definitions. We provide a proof-of-concept prototype and preliminary testing on illustrative cases. Results demonstrate the system's ability to reason about complex constraints such as precision, weight, and ergonomics. This paper illustrates how off-the-shelf foundation models can automate HRC decision-making via a human-in-the-loop paradigm while preserving operator agency and understanding.

Cite (BibTeX):

@inproceedings{verstappen2026delegact,
  author    = {Verstappen, Bram and Cardinaels, Dries and Leen, Danny and Luyten, Kris and Ramakers, Raf},
  title     = {Learning to delegate and act with DELEGACT: Multimodal language models for task-level human--cobot planning in industrial assembly},
  booktitle = {Extended abstracts of the 2026 CHI conference on human factors in computing systems},
  series    = {CHI EA '26},
  year      = {2026},
  publisher = {ACM},
  address   = {New York, NY, USA},
  doi       = {10.1145/3772363.3798803},
  url       = {https://driescardinaels.be/papers/delegact/index.html}
}