The project (coordination: Prognos AG) evaluates a large-scale research programme of the Federal Ministry of Research and Education (BMBF) called „ICT 2020 – Research for Innovation“. Between 2007-2017, the programme funded more than 6,400 individual projects allocating about €3.4b of public funds by employing four strategic instruments (lead innovations, technology collaborations, service platforms, ICT specific funding of SMEs) related to various base technologies-(electronics and micro systems, software technologies and knowledge processing, communication technologies and networks).

The evaluation focuses on three main topics: the extent to which programme goals have been achieved, the causal impact of the programme on the funded organisations (firms, universities, research institutes), and the effectiveness in terms of administrative procedures and cost-benefit relations. The evaluation consists of seven steps:

  • In a first step, logic models based on the theory of change are developed for each of the four strategic instruments.
  • Logic models are used to derive indicators on the programme's goals, activities (inputs) and results (outputs and outcomes). This step includes the development of a data collection procedure which also considers data for a control group approach.
  • The third step focuses on the processing of existing data (programme monitoring data) and own data collection activities.
  • In a fourth step, the evaluation looks at the degree to which the programme's goals have been achieved and to what extent the observed results can be attributed to the programme's activities.
  • The fifth step deals with a more detailed analysis of target achievement and impacts at the level of individual funding measures. In addition to a quantitative analysis, this step puts a strong emphasis on qualitative aspects, including the technology and market context of the funding activities.
  • The sixth step analyses the effectiveness of the programme in terms of administrative procedures and the relation between the cost of the programme and the effects resulting from programme activities.
  • The seventh step evaluates the process of programme steering.