Text and Data Mining (TDM) has a vital role to play to ensure that Big Data exerts an economic impact. Gleaning value from vast informational resources has become an indispensable activity in an ever-expanding world of information. Private and public organizations are increasingly more interested in understanding how to make decisions based on sufficient and sound information to succeed in the global economy that is becoming more and more knowledge-based. By all means, the ability to glean proper and actionable intelligence from vast information sources is essential for market success, public policy impact and economic growth.
We have recently submitted Deliverable 5.2 “Trend analysis, future applications and economics of TDM”, where we present the economic perspective of Text and Data Mining (TDM). The purpose of the study is to:
- portray current and future trends in TDM’s development and usage as an economic asset;
- depict and clarify market dynamics surrounding TDM solutions (global and European);
- estimate TDM’s global and European economic impact.
What are the findings?
TDM is one of the most prominent techniques for extracting value from huge amounts of data. However, in order for it to work it has to be able to produce actionable intelligence on a constant basis for decision-makers to use. However, TDM in and of itself does not necessarily lead to actionable intelligence; its success greatly depends on human skills and capacities. Data miners should have an understanding not only of the data they compile, but also – even more importantly – of the context in which they operate to determine the right pieces of information to be mined and analyzed.
In business, data miners have to learn about their company and its operations – the more in-depth their knowledge is, the greater the chances for discovering useful patterns and making useful predictions. In this sense, the process of translating data into economic value needs to be strongly supported by the business side of a given organization – namely the people who are the most knowledgeable about how an organization creates, delivers and sells value. When crucial departments and decision-makers fail to engage in information interchange, this undoubtedly leads to one of the greatest organizational bottlenecks in the quest to develop a consistent analytical value chain.
Big (and Open) Data will be responsible for a 1.9% increase in European GDP by 2020
Companies may innovate by tapping into Big Data to identify the many economic opportunities that lay ahead. Improvements in their response capacity and operations on the micro level can affect whole economies. According to Buchholtz et al. (2014), Big (and Open) Data will be responsible for a 1.9% increase in European GDP by 2020 and economic gains can be reaped through three types of micro level improvements, namely (1) better resource efficiency, (2) better decision-making through data-driven solutions and (3) improved products and processes as a result of R&D activities, monitoring and feedback.
Companies are struggling to derive value from TDM and Big Data. Companies’ specific information shows that their success is largely driven by their ability to mobilize the right skills and talent, financial resources and leadership expertise to translate data into business and economic value. As Big Data is quickly becoming a mainstream process, the number of leaders who are struggling with the first stage of Big Data development, i.e. infrastructure building, is rapidly diminishing. As the challenge posed by building proper infrastructure becomes less pronounced, another challenge becomes more urgent, namely, talent acquisition because it is crucial for developing suitable TDM tools and methods. Without this magic ingredient, data (Big Data, too) continues to remain in a raw and unusable form.
Although data analysis is becoming more widespread, the shortage of talent is preventing companies from translating analytical insights into business actions. Even business giants have recognized that they have failed to exploit the full potential of their data due to a shortage of internal capabilities and their access to skilled analysts. This says much about the market’s level of sophistication in integrating data analysis into the business decision-making process. Thus, it would be fair to claim that investments in the foreseeable future will focus on talent acquisition for TDM. Another key factor for companies to address is transforming organizational culture to make them more “data savvy” and to enable them to implement the work-products of data miners and analysts throughout their structures in a speedy way.
48% of all companies that experiment with Big Data reported using text based data for their Big Data projects
Increasing numbers of companies are experimenting with Big Data and TDM solutions, especially in the retail, communications and utilities sectors. Although the conversion deployment rate is equal to 60%, there is extensive room for improvement. The ability to harness infrastructure to tackle business problems and produce outcomes depends on using TDM tools appropriately so as to maximize their economic impact.
48% of all companies reported using text based data for their Big Data projects. Text mining is not as widely recognized as a source of business value compared to more structured, numerical, data. However, its value grows when it is processed in (Big) Data & Analytics projects.
2016 estimates suggest that a total of $23.8 billion will be spent globally on the Big Data Market
TDM’s financial value is growing rapidly even though companies, generally speaking, continue to invest more in infrastructure and database maintenance than they do in developing analytical talents, tools and methods. However, they are starting to feel considerable demand for talent and a need to alter their organizational culture, a purpose that also demands proper training. 2016 estimates suggest that a total of $23.8 billion will be spent globally on the Big Data Market, with $6.4 billion of that figure for TDM (software purchases, support and training). In the European Big Data Market, it is estimated that TDM will be worth c.a. $2.5 billion in 2016 juxtaposed against the total expected market value of $9.37 billion for the same year. European TDM may be expected to grow rapidly to $10.3 billion in 2021.
Data value extraction using TDM may exert a significant impact on the economy as a whole, depending on the capacity of countries, sectors and more specifically, companies to deploy ICT innovations and create internal processes to facilitate wider business efficiency, decision-making improvements and entry costs reduction. According to the BOUDICA model €1 spent on Big Data products and solutions translates into €10.7 of overall economic value. This also implies that the economic impact exerted by Big Data is estimated to be over twice as powerful as the economic impact exerted by more traditional data (whose ratio ranges from 1 to 5.2). Taking all this into account one could claim that TDM’s impact on the European Economy may range from $13 billion (conservative calculation) to $26.7 billion (optimum scenario) in 2016. This impact may rise to $110.1 billion in 2020.
To see all our findings and conclusions, please have look at the deliverable1.
1 Deliverable 5.2 Trend analysis, future applications and economics of TDM
// All blog posts are the personal opinion of the bloggers. For more information see FutureTDM's DISCLAIMER on how we handle the blog. //