Dropping the pilot

By Dr. Alicia Martínez de Yuso, ZLC Project Manager and Carolina Ciprés, ZLC Director of Research.

Leer versión en español


Why do so many apparently promising pilot schemes in fields such as logistics, distribution and freight transport fail to gain wider traction? Why indeed are they often abandoned or allowed to fade away by the very partners that have claimed tangible, measurable success? Does this mean that not insignificant amounts of public and private research and development investment is being wasted? And if so, what can we do about it?

Back in 2021 ZLC announced our participation in the European Commission sponsored BOOSTLOG project (see https://www.zlc.edu.es/news/uncovering-the-real-value-in-logistics-research/)which has set out to examine these issues, and results are now coming in.

Over the past two decades the EC has funded, in whole or in part and largely through the Framework Programs budget stream, over 160 major research projects aimed at improving the sustainability and competitiveness of companies, and of the European economy more generally, in their logistics and related operations. Most of these have gone beyond mere theoretical research to demonstrate their validity through ‘Living Labs’ – demonstration projects by real firms and public authorities involving real assets, facilities, goods and of course money. Naturally not all of these were successful, for a variety of reasons (often unforeseen external circumstances rather than any deficiencies in the innovation itself) but many showed real promise, if for example they could be scaled up, and quite a few showed genuine benefits even at pilot scale. Nonetheless, the overall impact on wider logistics practice has been limited, to say the least, and as noted even successful projects have sometimes died the death.

BOOSTLOG has been seeking to increase visibility and support valorisation of project results, outcomes and implementation cases; develop actionable reports to share and highlight progress and impact (actual and potential); and identify the barriers to wider and swifter deployment of successful R&D. That means identifying which areas of research have achieved the highest impacts (and how further public investment could take these impacts further). At the same time the work has tried to identify the gaps in research and innovation efforts which need to be bridged if work already done or still in progress is to deliver its full potential – again as a guide to the European Commission and other funding organizations.

And ultimately the aim is to boost the impact of R&I by bringing in the end users of freight and logistics systems and other stakeholders and involving them in collaborative ‘best practice’ networks at every level from the regional to EU-wide (and indeed beyond).

Implementation cases from the corpus of R&I activity has been divided across ten broad subject areas or clouds (obviously there is a fair degree of overlap). In a recent exercise, the team has been analysing 15 implementation cases across four areas  – five looking at logistics nodes (such as ports), four on freight and logistics data sharing, five based around logistics networks (for example multimodal operations or hub and spoke architectures) and one case involving Physical Internet concepts. The approach is two-fold: analysing the links and relationships between the policies and R&I strategies of authorities (from municipalities to the EC itself), corporate objectives, and sector-specific performance indicators. Secondly, BOOSTLOG has sought to identify the key actions, characteristics and framework conditions (such as programme definitions and possible cross-programme synergies) that have tended to facilitate impact, or as it may be whose absence has tended to constrain the potential impact of the R&I investment.

As you can see in the ZLC-led report [here] this becomes quite complex, not least because EU policy objectives, and company/sector priorities, have not remained static over the years – most notably, in the greatly increased policy emphasis on sustainability. That has inevitably created some misalignments, while expected impacts as shown in KPIs may have changed. Additionally, some implementation operations are quite mature whilst others are still not out of the ‘proof of concept’ stage. And of course, each case may have its own highly specific circumstances.

Nonetheless our work has served to verify conclusions which had already been tentatively reached earlier in the project.

Firstly, the EU’s Sustainable Development Goals do indeed provide a viable common framework for alignment between European Union, companies, service providers, and other stakeholders (which may include local authorities, consumer representatives and others. But while recent R&I programmes do explicitly address Sustainable Development Goals and their related KPIs, these KPIs may need to be developed and refined.

Similarly, we confirmed important correlations between most of the social objectives of policymakers on the one hand, and businesses on the other. However, translating those objectives into concrete KPIs for R&I is not straightforward as there are too many Impact Indicators and KPIs. We think it is necessary to prioritize the most important indicators to address particular policy and company objectives rather than trying to cover every environmental, social and governance issue.

Thirdly, for the market-oriented implementation cases (which not all are – R&I doesn’t necessarily lead to marketable goods and services; a better way of doing things may be invaluable without representing patentable Intellectual Property), our study of the pathway towards wider impact (or the lack thereof) clearly confirms the existence of the ‘valley of death’. This can be thought of as the time delay between the research activity, and its actual implementation in operation and as commercially available goods and services. Time really matters – not least because innovations often depend on internal ‘champions’ – if these move on, interest fades.

Nonetheless we have described in the report how eleven of the implementation cases have made it through the valley of death. Here we find that, although there are undoubtedly causal links between the nature of the research and its impact (it would be strange if there weren’t!), often the more critical aspects were around regulation, markets, and the social, commercial and industrial readiness of the solution. So, we need a way of defining the ‘readiness level’ for different R&I projects if we are to predict the potential, and assess the actual, impact beyond the initial Living Lab pilots.

The headline question that everyone will ask is ‘which R&I cloud is delivering most bang for the EU buck?’. This is slightly unfair. Of the implementation cases in our analysis, those based in networks and in data sharing are most likely to be operating in the market, while those concerned with logistics nodes are more likely to still be at proof of concept stage, with full impacts only evident in years to come. That should come as no surprise – it takes a long time to make physical changes to something like a port – much longer than improving data flow with a swift bit of coding!

To read the full report visit https://www.etp-logistics.eu/boostlog/, or for more information on BOOSTLOG contact Dr. Alicia Martinez de Yuso at [email protected]