Open ended problems – A robot programming experiment to compare and test different development and design approaches

Evidence-based research in engineering design is in need of well-documented experimental setups that are replicable by other researchers. We describe an experiment setup that involves programming a LEGO Mindstorms robot to solve a complicated open-ended task. This task has a quantifiable performance outcome. The setup is adaptable to investigate different development and design approaches related to engineering performance. It is designed such that direct personal interactions between the participant and the experimenter are minimized to keep the conditions repeatable. We provide a description of the robot and the dedicated software library that makes it easy to control the robot, the open-ended task, the physical setup and the interactions with the participant. We illustrate how this setup can be used in two study design examples.