Posts tagged: AI/ML

Teaching as training: Iterative and incremental AI skill development

Higher education must equip students with skills for complex, multidisciplinary challenges. Traditional approaches relying on fixed deadlines and traditional exams often limit opportunities for growth and continuous skill development. This contribution presents an iterative and incremental teaching method, applied for five years in a row in a master-level Computer Science course on Human–AI Interaction. Our approach emphasizes formative feedback, collaborative learning, and individual progression. Students work on group assignments and an individual project, with no strict deadlines and unlimited opportunities during the semester to resubmit until a "pass" is achieved. Compact feedback sessions after each iteration serve both as assessment moments and teaching opportunities, clarifying expectations and guiding improvement. The method is grounded in mastery learning, formative assessment, and the High Impact Learning that Lasts model, fostering motivation and self-determination. Survey data and performance analysis of a study conducted two years ago, show positive effects on learning outcomes and student motivation: students valued the clarity of assessment, the removal of "one chance" exams, and the freedom to iteratively improve. Over five years of teaching, this approach has proven effective in balancing diverse prior knowledge, building applicable skills, and sustaining motivation during the semester. We conclude that incremental and iterative teaching constitutes a viable model for skill-oriented higher education, adaptable across contexts where collaboration, feedback, and progression are central.

Learning to delegate and act with DELEGACT: Multimodal language models for task-level human--cobot planning in industrial assembly

Industrial assembly is shifting toward human-robot collaboration (HRC) to leverage the complementary strengths of both agents. However, traditional task allocation referred to as the Robotic Assembly Line Balancing Problem (RALBP) remains labor-intensive and often lacks transparency. We introduce DELEGACT, a framework designed to produce workable, intelligible human-cobot task allocations. The framework uses a Vision-Language Model (VLM) to extract atomic operations from expert demonstration videos, then employs a Large Language Model (LLM) to delegate these tasks based on robot specifications, operator competencies, and material definitions. We provide a proof-of-concept prototype and preliminary testing on illustrative cases. Results demonstrate the system's ability to reason about complex constraints such as precision, weight, and ergonomics. This paper illustrates how off-the-shelf foundation models can automate HRC decision-making via a human-in-the-loop paradigm while preserving operator agency and understanding.

DIVERSE: Disagreement-inducing vector evolution for rashomon set exploration

We propose DIVERSE, a framework for systematically exploring the Rashomon set of deep neural networks, the collection of models that match a reference model's accuracy while differing in their predictive behavior. DIVERSE augments a pretrained model with Feature-wise Linear Modulation (FiLM) layers and uses Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search a latent modulation space, generating diverse model variants without retraining or gradient access. Across MNIST, PneumoniaMNIST, and CIFAR-10, DIVERSE uncovers multiple high-performing yet functionally distinct models. Our experiments show that DIVERSE offers a competitive and efficient exploration of the Rashomon set, making it feasible to construct diverse sets that maintain robustness and performance while supporting well-balanced model multiplicity. While retraining remains the baseline for generating Rashomon sets, DIVERSE achieves comparable diversity at reduced computational cost.

Challenges and opportunities for delay-invariant telerobotic interactions (short paper)

Effective operation in direct-control telerobotics relies heavily on real-time communication between the operator and the robot, as the operator retains full control over the robot's actions. However, in scenarios involving long distances, communication delays disrupt this feedback loop, creating significant challenges for precise control. To investigate these challenges, we conducted a user study where participants operated a TurtleBot3 Waffle Pi under varying delay conditions. Post-experiment brainstorming and analysis revealed recurring challenges, including over-correction, unpredictable robot behavior, and reduced situational awareness. Potential solutions identified include improving robot behavior predictability, integrating feedforward mechanisms, and enhancing visual feedback. These findings underscore the importance of designing intelligent interfaces to mitigate the impact of delays on telerobotic performance.

AI-spectra: A visual dashboard for model multiplicity to enhance informed and transparent decision-making

We present an approach, AI-Spectra, to leverage model multiplicity for interactive systems. Model multiplicity means using slightly different AI models yielding equally valid outcomes or predictions for the same task, thus relying on many simultaneous "expert advisors" that can have different opinions. Dealing with multiple AI models that generate potentially divergent results for the same task is challenging for users to deal with. It helps users understand and identify AI models are not always correct and might differ, but it can also result in an information overload when being confronted with multiple results instead of one. AI-Spectra leverages model multiplicity by using a visual dashboard designed for conveying what AI models generate which results while minimizing the cognitive effort to detect consensus among models and what type of models might have different opinions. We use a custom adaptation of Chernoff faces for AI-Spectra; Chernoff Bots. This visualization technique lets users quickly interpret complex, multivariate model configurations and compare predictions across multiple models. Our design is informed by building on established Human-AI Interaction guidelines and well know practices in information visualization. We validated our approach through a series of experiments training a wide variation of models with the MNIST dataset to perform number recognition. Our work contributes to the growing discourse on making AI systems more transparent, trustworthy, and effective through the strategic use of multiple models.

Opportunities and challenges of model multiplicity in interactive software systems

The proliferation of artificial intelligence (AI) in interactive systems has led to significant challenges in model integration, but also end-user-related aspects such as over- and undertrust. This paper explores how multiple AI models with the same performance and behavior but different internal workings –a phenomenon called model multiplicity– affect system integration and user interaction. We discuss the implications of model multiplicity for transparency, trust, and operational effectiveness in interactive software systems.

Anthropomorphic user interfaces: Past, present and future of anthropomorphic aspects for sustainable digital interface design

Interactions with computing systems and conversational services such as ChatGPT have become an inherent part of our daily lives. It is surprising that user interfaces, the gateways through which we communicate with an interactive intelligent system, are still predominantly devoid from hedonic aspects. There is little attempt to make communication through user interfaces intentionally more like communication with humans. Anthropomorphic user interfaces can transform interactions with intelligent software into more pleasant experiences by integrating human-like attributes. Anthropomorphic user interfaces expose human-like attributes that enable people to perceive, connect and interact with the interfaces as social actors. This integration of human-like aspects not only enhances user experience but also holds the potential to make interfaces more sustainable, as they rely on familiar human interaction patterns, thus potentially reducing the learning curve and increasing user adoption rates. However, there is little consensus on how to build these anthropomorphic user interfaces. We conducted an extensive literature review on existing anthropomorphic user interfaces for software systems (past), in order to map and connect existing definitions and interpretations in an overarching taxonomy (present). The taxonomy is used to organize and structure examples of anthropomorphic user interfaces into an accessible collection. The taxonomy and an accompanying web tool provides designers with a reference framework for analyzing and dissecting existing anthropomorphic user interfaces, and for designing new anthropomorphic user interfaces (future).

AI-spectra: A visual dashboard for model multiplicity to enhance informed and transparent decision-making

We present an approach, AI-Spectra, to leverage model multiplicity for interactive systems. Model multiplicity means using slightly different AI models yielding equally valid outcomes or predictions for the same task, thus relying on many simultaneous "expert advisors" that can have different opinions. Dealing with multiple AI models that generate potentially divergent results for the same task is challenging for users to deal with. It helps users understand and identify AI models are not always correct and might differ, but it can also result in an information overload when being confronted with multiple results instead of one. AI-Spectra leverages model multiplicity by using a visual dashboard designed for conveying what AI models generate which results while minimizing the cognitive effort to detect consensus among models and what type of models might have different opinions. We use a custom adaptation of Chernoff faces for AI-Spectra; Chernoff Bots. This visualization technique lets users quickly interpret complex, multivariate model configurations and compare predictions across multiple models. Our design is informed by building on established Human-AI Interaction guidelines and well know practices in information visualization. We validated our approach through a series of experiments training a wide variation of models with the MNIST dataset to perform number recognition. Our work contributes to the growing discourse on making AI systems more transparent, trustworthy, and effective through the strategic use of multiple models.

FortClash: Predicting and mediating unintended behavior in home automation

Smart home inhabitants can specify trigger-condition-action rules to control the home's behavior. As the number of rules and their complexity grow, however, so does the probability of issues such as inconsistencies and redundancies. These can lead to unintended behavior, including security vulnerabilities and wasted resources, which harms the inhabitants' trust in the system. Existing approaches to handle unintended behavior typically require inhabitants to define all-encompassing, permanent solutions by modifying the rules. Although this is fitting in certain situations, some unforeseen situations might occur. We argue that the user always must have the last word to avoid unwanted behaviors, without altering the overall behavior. With FortClash, we present an approach to predict many different types of unintended behavior, and contribute four novel mechanisms to mediate them that rely on making one-time exceptions. With FortClash, inhabitants gain a new tool to deal with unintended behavior in the short-term that is compatible with existing long-term approaches such as editing rules.

Model-based engineering of feedforward usability function for GUI widgets

Feedback and feedforward are two fundamental mechanisms that support users' activities while interacting with computing devices. While feedback can be easily solved by providing information to the users following the triggering of an action, feedforward is much more complex as it must provide information before an action is performed. For interactive applications where making a mistake has more impact than just reduced user comfort, correct feedforward is an essential step toward correctly informed, and thus safe, usage. Our approach, Fortunettes, is a generic mechanism providing a systematic way of designing feedforward addressing both action and presentation problems. Including a feedforward mechanism significantly increases the complexity of the interactive application hardening developers' tasks to detect and correct defects. We build upon an existing formal notation based on Petri Nets for describing the behavior of interactive applications and present an approach that allows for adding correct and consistent feedforward.

All Posts by Category or Tags.