Uploaded December 2022 | Updated September 2026, 18 hours ago
In the time it takes you to watch this video, hundreds of millions of bytes of data will have been collected online. All of this data provides insight about our world, and what we are doing, when and where, in real time.
With the huge surge of data collected, big data and data analytics have become a near necessity in understanding the complex and dynamic world around us. The popularisation of big data and data analytics has initiated a shift, the magnitude of which has not been observed since the industrial revolution.
Nowadays, industries embrace big data and data analytics, but public policy seems to be a few steps behind in terms of leveraging it. The project BIGPROD used the information stored on company websites to shed light on company activity and the impact company's have on our economies.
In order to better understand industrial activity in Europe and the UK, the project's team has web scraped big data from close to 100,000 companies' websites creating novel measures on innovation, digitalization, ecosystem of firms, and the structure of the industry.
Engaging with stakeholders was of the utmost importance, and the data gathered is online for anyone to use. We even accompany the data with video tutorials and code examples on how to leverage the data.
Through stakeholder workshops, we have shared the data and tools with policy-makers and researchers ensuring the adoption of the tools.
Not only did our results show how digitalization impacts firms differently, but also have demonstrated how big data measures can give us a much more granular picture of firms, innovations and their impact to our society.
In the time it takes you to watch this video, hundreds of millions of bytes of data will have been collected online. All of this data provides insight about our world, and what we are doing, when and where, in real time.
With the huge surge of data collected, big data and data analytics have become a near necessity in understanding the complex and dynamic world around us. The popularisation of big data and data analytics has initiated a shift, the magnitude of which has not been observed since the industrial revolution.
Nowadays, industries embrace big data and data analytics, but public policy seems to be a few steps behind in terms of leveraging it. The project BIGPROD used the information stored on company websites to shed light on company activity and the impact company's have on our economies.
In order to better understand industrial activity in Europe and the UK, the project's team has web scraped big data from close to 100,000 companies' websites creating novel measures on innovation, digitalization, ecosystem of firms, and the structure of the industry.
Engaging with stakeholders was of the utmost importance, and the data gathered is online for anyone to use. We even accompany the data with video tutorials and code examples on how to leverage the data.
Through stakeholder workshops, we have shared the data and tools with policy-makers and researchers ensuring the adoption of the tools.
Not only did our results show how digitalization impacts firms differently, but also have demonstrated how big data measures can give us a much more granular picture of firms, innovations and their impact to our society.










