Apache Hadoop is an emerging technology that was designed to address the specific requirements of Big Data. It can deal with petabytes of structured and unstructured data. The technology was developed by Yahoo! in 2005 and it got its name from a toy elephant. However, Hadoop does not work alone. Rather, it is part of an increasing number of associated technologies such as HBase, Hive, Pig, Oozie, and Zookeeper.
Apache Hadoop Ecosystem (source: quantfarm.com)
- Is Fault-tolerance open-source software framework that can deal with software and hardware failures.
- Scales well to any increase in processors, memory or storage devices.
… Click to read the full post
Because customer relationship constitutes an important part of any strategic decision-making process, shifting towards Big Data technologies would enable executives to keep up with customer service expectations. A top concern for them is how to achieve faster access to data in order to overcome the many obstacles they would encounter.
Typically, data in organizations can be in the following three forms:
- Structured Data. Such data is stored in databases (in tables) and can be accessed by using database management systems such as Oracle, DB2 and MySQL. This data constitutes only 10% of the universal data today.
- Unstructured Data. Such data cannot be stored using traditional relational databases.
… Click to read the full post
Are you working with Big Data in the agri-food sector? Share the information of your research project & institution and be a part of the European map of big data in agriculture and food.
agINFRA, the community-specific research hub for agriculture and food, the BigDataEurope coordination action, and a number of supporting networks and initiatives, are creating a European map of big data in agricultural and food research.
In order to identify and document the key stakeholders and their relevant initiatives, UN FAO, Agro-Know, GODAN , Alterra WUR and other partners have joined forces in order to map the current status and future challenges on big data sources and technologies in the domains of Food Security, Sustainable Agriculture and Forestry, Marine and Maritime and Inland Water Research, and the Bioeconomy.… Click to read the full post
Today, there are more than 4.6 billion mobile-phone subscribers; more than 2.4 billion people with access to the Internet; and more than a billion Facebook subscribers. All of them are producing large amount of data.
It was estimated that the amount of data produced from the dawn of civilization to 2003 is 5 exabytes, at a time that every two days, we produce the same volume of data. It is even expected that by this year, the volume of digital universe of data will reach 8 zettabytes. This flood of data, which is commonly referred to as Big Data information overload or data deluge has become a challenge for many businesses.… Click to read the full post
he term “Big Data” is relative and highly dependent. For example, the organizations that lack the ability to handle, store and analyze their own sets of data, are in fact experiencing the Big Data “phenomenon”. Nevertheless, this is not what Big Data is all about. Besides of being by order of magnitudes in terms of Volume, data has to be of greater Variety and complexity, and generated at a high Velocity, which are usually referred to as the three Vs of Big Data.
The three Vs of Big Data (source: http://blog.softwareinsider.org/2012/02/27/mondays-musings-beyond-the-three-vs-of-big-data-viscosity-and-virality/)
A better definition of Big Data might be: the processing, interpretation and representation of large volumes of data (typically, petabytes or zettabytes) originating from different sources in a way that makes the data meaningful and usable.… Click to read the full post
Less than one year ago, we were really happy to feature a guest blog post on using social networks for disaster recovery, focusing on the Malaysian Airlines Flight MH370 which disappeared on March, 8, 2014. The post was authored by Mohammed Z. Al-Taie, a PhD candidate at the Universiti Teknologi Malaysia (UTM), Faculty of Computing (FC) – Soft Computing Research Group (SCRG).
Mr. Al-Taie was kind enough to propose authoring a series of blog post on open, linked and big data in the following months through the Agro-Know blog. We gladly accepted this kind gesture as these topics are really close to what we are working on in Agro-Know, focusing on the agri-food sector.… Click to read the full post
We were recently informed about the 2015 Copernicus Masters competition, an annual competition that aims to make good use of open satellite data, raising at the same time awareness about environmental issues (and how they can be solved through apps making use of satellite data). However, this time the competition looked even more appealing to the Agro-Know team due to the fact that one of the nine (9) in total challenges of this year’s competition is a Big Data Challenge, a fact that would be of specific interest to the Big Data Europe project that we participate in, focusing on the Food & Agriculture societal challenges.… Click to read the full post
The members of the 7 societal challenges of the Big Data Europe Project gathered in Vienna on April 1st, 2015 to communicate their next steps in terms of requirements elicitation and workshops’ planning. Agro-Know in a joint partnership with the Food & Agriculture Organization of the United Nations (FAO) is responsible for the societal challenge of “Food security, sustainable agriculture and forestry, marine and maritime and inland water research, and the Bioeconomy” and went to Vienna to present the requirements elicitation methodology and also discuss with other consortium members the next steps of this promising project.
A good meeting starts from the night before!
… Click to read the full post
Is this the time for big data technologies to be streamlined in Europe? Are we going to see a wave of big data powered tools and applications in different application domains and industry sectors? This is the ambitious goal of the recently launched initiative Big Data Europe and its is a major opportunity (but also challenge) for the agri-food sector as well.
The European Commission decided to bring together some of the best European minds researching and developing big data technologies, and ask them to work with representatives of the six key Societal Challenges that Europe is facing
(and the European Commission is channeling its funding towards) so that we explore the capabilities and limitations of existing technologies.… Click to read the full post
New European project develops a platform to facilitate big data usage
The growing digitization and networking process within our society has a large influence on all aspects of everyday life. Large amounts of data are being produced permanently, and when they are analyzed and interlinked they have the potential to create new knowledge and intelligent solutions for both the economy and society as a whole. To facilitate the use of Big Data technologies, the European Union funds an interdisciplinary research and innovation project called “BigDataEurope”. A European consortium, led by the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, is developing a platform that will simplify the creation and access to Big Data applications and their integration into existing workflows.… Click to read the full post