Challenges of evaluating a living lab in South Africa

Abstract

The purpose of this paper is to discuss certain challenges that emerge when evaluating a Living Lab (LL). These challenges are linked to the choice of evaluation methods as well as whether to measure, when, what and how to measure. A LL in itself is a complex context that provides a successful mixture of ICT-based collaborative environments, open innovation platforms, and user centred product/service development methods and Public Private Partnerships. All of these can have effects on regional economies, industries and societal landscapes. This article shares the combined methods which were applied in evaluating a LL and also reflects on why, how and what were evaluated. A combination of three evaluation frameworks were applied, which involved: a) Outcome Mapping (OM) methodology to track user/stakeholder behavioural changes and lessons learnt, b) a framework developed by the University of Madrid, known as OO/UC3M/63 to provide the innovation strategy, process, technical development, competitive advantage and management, c) PACE which focuses on project assets, core competencies and exploitable items. The interpretavist philosophy was applied with a qualitative methodology to use a project’s LL as a case study to collect feedback on a continuous basis from stakeholders through semi-structured interviews, questionnaires and observation and to interpret the results. Measurement over a specific period of time within the specific complex context of a LL with different stakeholders resulted in an improvement in LL processes, product and services.