How Evaluation Techniques Influence the RE-Tool Evaluation: An Experiment

Requirements engineering (RE) tools could be compared by analyzing documentation and performing explorations, but the evaluation process is complex and time consuming. This work reports on a two-session experiment which compares evaluation techniques for RE-tools. First a theoretical framework and test scenarios are used, next tool tutorials and a feature-based questionnaire are applied. The findings show that the framework provides a detailed evalua-tion, but it is more time consuming than the questionnaire-based technique. The results also support the use of test scenarios instead of tool tutorials. Although the scenario preparation is relatively long and requires expert involvement, the scenarios provide comparison of the same tool functions in the same problem domain.