In a previous post, we addressed Hadoop ecosystem and a set of tools that reside and operate near the two core components of Hadoop (i.e. MapReduce and HDFS) to help them store and manage data, and perform various analytic tasks. However, Big Data landscape is more than Hadoop alone.
In this post, we will expand the circle a little bit and address the many technologies that are involved in Big Data processes. The Big Data landscape can be daunting. The vast proliferation of technologies in this competitive market means there’s no single go-to solution. However, it is possible to group the different tools and frameworks based on similarity in goal and functionality into a number of main components:
- Distributed file systems: file systems that run on multiple servers and allow access to files from multiple hosts, which means the ability to share files and storage resources by multiple users.
… Click to read the full post
The Wageningen University and Research Centre (WUR) is a Dutch public university in Wageningen, Netherlands, which consists of the Wageningen University and the former agricultural research institutes (Dienst Landbouwkundig Onderzoek – DLO). WUR is one of the top institutes at a global level in the field of agri-food and environmental research.
As expected, such a large and active group of research institutes produces huge amounts of data – and WUR has developed the expertise throughout the years to make good use of big data in the agri-food research context. It has been almost one century (98 years, to be more precise) since WUR started collecting research data of various types, using various means and managing all this information and data so that it can be easily reused.… Click to read the full post
The European Agricultural Research Initiative (EURAGRI) is a European platform for the political and executive organizations in the fields of Food and Agricultural Policy and Research. By providing a forum for informal exchange on opinions and views on developments in science, society and industry, thus facilitating the design, implementation and planning of agri-food research, EURAGRI plays a key role in agricultural research at an EU level. From an organizational point of view, EURAGRI consists of representatives of Ministries of Agriculture and Research in various EU countries, a fact that allows EURAGRI to be in the position to advocate and work at a high level, involving EU countries as members and have an impact on the directions of agri-food research at national and EU level.… Click to read the full post
BigDataEurope is a project funded by the European Commission’s Horizon 2020 Research and Innovation programme, aiming to facilitate European companies in building innovative multilingual products, applications and services based on semantically interoperable, large-scale, multi-lingual data assets and knowledge, available under a variety of licenses and business models. The project aims to provide a generic approach meeting a wide variety of needs, and in this context it encompasses all seven (7) societal challenges defined by the European Commission. Agroknow is a consortium member of the BigDataEurope project, representing the agri-food community in the Societal Challenge 2.
The BigDataEurope project is organizing a webinar titled “A Big Data platform for the future: Technical insights into the BDE Project“.… Click to read the full post
Trying to emphasize on the importance of food safety information and data is probably meaningless; we are discussing about data that can be used for saving millions of lives each year so its management and sharing is of highest importance at a global level. And when we are talking about related information aggregated through various sources, we are talking about big data at the service of the food safety sector.
In this context, Agroknow CEO Nikos Manouselis made a trip to Cranfield, Bedfordshire, UK in order to participate to an event simply titled “Using Big Data“, organised by the Cranfield University. … Click to read the full post
Big Data is the data that, in addition to being massive in size, is of a greater variety and complexity, and is generated at a high velocity. Collectively, these are referred to as the three Vs of Big Data. However, the concept is relative and highly dependent: the organizations that lack the ability to handle, store or analyze their own sets of data are in fact experiencing the Big Data phenomenon.
Big Data analytics, on the other hand, is a business strategy that uses technology to gain deeper insights into customers, partners, and businesses, and hence achieving competitive advantage. It involves working with data that, because of its size and variety, lie beyond the ability of typical database management systems to store, manage and analyze.… Click to read the full post
One aspect of Agroknow’s work in the context of the BigDataEurope Horizon 2020 project (and as the project is moving forward entering its second year) is to develop a pilot addressing a specific Horizon 2020 Societal Challenge and from a domain specific perspective (in our case that of Societal Challenge 2: Food security, sustainable agriculture and forestry, marine, maritime and inland water research and the bioeconomy). Our pilot is specific to agriculture and more specifically to viticulture and the diversity of grapevine varieties. What we want to do is to explore how we can “support advanced crop data discovery, processing, combining and visualization from distributed and heterogeneous data repositories”.… Click to read the full post
One of the components of Agroknow’s work in the context of the BigDataEurope Horizon 2020 project is to identify and promote big data initiatives and existing work in the agri-food sector and at a global level and see how these could be mapped to the technical work that the project will design and deploy at a later stage. At the same time, existing efforts, work and outcomes are studied and good practices are identified, in order to be adapted and adopted in different cases. In this context, we were happy to see some month ago CGIAR, a global network of fifteen (15) agricultural research centers all over the world, show activity towards the exploitation of big data for addressing food security.… Click to read the full post
Data types involved in Big Data analytics are many: structured, unstructured, geographic, real-time media, natural language, time series, event, network and linked. It is necessary here to distinguish between human-generated data and device-generated data since human data is often less trustworthy, noisy and unclean.
A brief description of each type is given below.
Types of data used in big data analytics
- Structured data: data stored in rows and columns, mostly numerical, where the meaning of each data item is defined. This type of data constitutes about 10% of the today’s total data and is accessible through database management systems.
… Click to read the full post
Businesses have recognized the benefit rooted in data and, therefore, put together people, technologies, and processes to collect, store and analyze huge amounts of data. A key element for deriving value from data is by using analytics.
Big Data analytics is a business strategy that uses technology to gain deeper insights into customers, partners and businesses and so achieving competitive advantage. It includes working with huge data that, because of its size and variety, lie beyond the ability of typical database management systems to store, manage and analyze.
The use of “Big Data analytics” involves two dimensions: one is Big Data, which is annotated with the three ‘Vs’: data volume, velocity, and variety.… Click to read the full post