Framework for evaluating information technology benefits in local communities

Abstract

Governments of the developing world, including South Africa, have a strong commitment and resolve to accelerate the rollout of Information and Communication Technology (ICT) to achieve developmental benefits in communities. Consequently both government and the private sector are delivering a number of interventions in South Africa based on Information and Communication Technologies for Development (ICT4D). Hard evidence regarding the development benefits of ICT4D interventions is lacking and there is little agreement on measures to evaluate the benefits of such projects. One possible reason for this is that there are no established evaluation frameworks to assess the benefits. Frameworks used in ICT4D evaluation are investigated in this study to ascertain their usefulness to identify benefits of ICT4D initiatives. Sen’s Capability Approach defines development as freedom. An example of such development can be the process of providing opportunities through ICT and meaningful ways to use these opportunities to realise various benefits. Tangible benefits are simple to identify (e.g. number of people using the public access centre, or number of people finding employment). Intangible benefits include the real ‘wins’—capabilities garnered through access and meaningful use of ICT, leading to the recognition of new opportunities for the users of the public access centres. Sen’s Capability Approach is operationalised, demonstrating the inclusion of a person’s agency and conversion factors that inhibit or enhance utilisation of opportunities and choices in realising benefits. The SmartCape initiative is a 2002 ICT4D intervention established in the libraries of Cape Town, South Africa, and is used as a case in this research study. The libraries act as public access centres that provide free ICT and Internet access to library members in the community. Surveys completed by users of these centres provided useful quantitative data. A broad spectrum of qualitative data was gleaned from interviews and focus groups with users of the ICT centre at a recently established library and with focus groups from other centres in two underserved areas of Cape Town. Quantitative data analysis techniques applied to qualitative content data was used to investigate users’ diverse perceptions. An initial framework guided the analysis of data to identify the benefits realised by the users of the public access centre. Perceptions of a changed life, aspirations for a better life, and ‘hope’ emerged as intangible benefits. On the intangible side, the effect of keyboard proficiency, greater ease in finding information, and a preference for accessing the Internet at a public access centre emerged as having a significant effect on the hopefulness of PAC users. Two theme-groups were identified through using co- occurrences of themes and the statistical techniques of cluster analysis and multidimensional scaling. The Benefits-framework, produced by this study, based on Amartya Sen’s Capability Approach, represents the relationship between all the themes, includes emergent intangible benefits and can be used to identify the benefits of ICT4D interventions in public access centres. This study also produces empirical evidence of the developmental impact of the SmartCape ICT4D programme in Cape Town and thus provides evidence of its value.