“It’s the variety, stupid”

14493234103_8d65485b5f_o

Well, I should have suspected it. But it was good to see more than 40 experts from around the world highlighting and explaining it: the special thing about big data in agriculture is its extreme variety.

This is what you get, if you contrast the four (as IBM suggests) V’s of big data to the data types and sources that are typically used in agricultural, food and environmental research. We are not talking about an extremely large Volume; other domains have much more voluminous data. It is not that they come with a high Velocity, especially compared to other domains.… Click to read the full post

Big data in Europe calls on Agro-Know

paris

We have been following Big Data Europe since its early stages, learning about the recent advances and trends in big data by prestigious partners like Fraunhofer (yes, the guys that invented MP3). We got more and more involved in this flagship big data initiative for Europe, sharing our understanding of data-related challenges in the agri-food sector, what kind of big data our communities work with, and how cutting edge solutions using big data analytics may be developed to serve their needs.

This is the time to take an important step forward: and beautiful Paris is the place where this will happen.… Click to read the full post

Fighting hunger with big data: doing it the CGIAR way

hungergames

Get ready for some serious numbers – by 2030, the CGIAR wants its action to result in 150 million fewer hungry people, 100 million fewer poor people – at least 50% of whom are women, and 190 million ha less degraded land. They have mobilised a tremendous amount of money from their donors to achieve it. And they are now designing the way in which they will make it happen.

Taking a closer look to their recently published progress of work, I was intrigued by two things:

  • They follow a truly transparent process, since they have published quite elaborate and detailed pre-proposals that are still under evaluation.
Click to read the full post

Big Data as Part of Internet of Things Solutions

Internet Of Things

The Internet of Things or IoT is basically a complex network that seamlessly connects people and things together through the Internet. Theoretically, anything that can be connected (smart watches, cars, homes, thermostats, vending machines, servers…) will be connected in the near future using sensors and RFID tags. This allows connected objects to continuously send data over the Web and from anywhere. The first time the term was used was in 1999 by Kevin Ashton, the creator of the RFID standard.

Internet Of Things

Source: http://inoviagroup.se

IoT will have the advantage of bringing us smart cities with smart cars, secure and efficient buildings, and smart traffic management systems.… Click to read the full post

The Future is for NoSQL Data Storage model!

NOSQL

Databases come in a variety of tastes, such as relational (e.g. Postgres, Oracle and MySQL), document-oriented (eg. MongoDB, CouchDB and SimpleDB), columnar (e.g. BigTable and HBase), key-value (e.g. MemcacheDB, Redis and Riak) XML (e.g. MarkLogic, BaseX and eXist) and graph (e.g. Neo4J, GraphDB and Giraph). All data stores support writing and retrieving data but with some differences in terms of database indexing, database schema, query format, data sharding, replication, scalability and others.

Although the relational model and the Structured Query Language (SQL) were for decades the de facto for storing data, it has become established that relational databases are no more the winners when it comes to flexibility and scalability.… Click to read the full post

Hadoop Ecosystem: an Integrated Environment for Big Data

Hadoop_Ecosystem

Hadoop is currently the most common single Big Data platform. However, still other techniques play a role in the scene. While there are proprietary distributions for Hadoop which are developed by giant Big Data companies, such commercial products rely heavily on open source projects.

Hadoop ecosystem includes a set of tools that function near MapReduce and HDFS (the two main Hadoop core components) and help the two store and manage data, as well as perform the analytic tasks. As there is an increasing number of new technologies that encircle Hadoop, it is important to realize that certain products maybe more appropriate to fulfill certain requirements than others.… Click to read the full post

Hadoop as the Backbone of Big Data Technologies

Hadoop_Ecosystem

Apache Hadoop is an emerging technology that was designed to address the specific requirements of Big Data. It can deal with petabytes of structured and unstructured data. The technology was developed by Yahoo! in 2005 and it got its name from a toy elephant. However, Hadoop does not work alone. Rather, it is part of an increasing number of associated technologies such as HBase, Hive, Pig, Oozie, and Zookeeper.

Hadoop_Ecosystem

Apache Hadoop Ecosystem (source: quantfarm.com)

Hadoop:

  • Is Fault-tolerance open-source software framework that can deal with software and hardware failures.
  • Scales well to any increase in processors, memory or storage devices.
Click to read the full post

Obstacles to the Adoption of Big Data

Obstacles to Big Data Implementation (Source: http://www.eweek.com/)

Because customer relationship constitutes an important part of any strategic decision-making process, shifting towards Big Data technologies would enable executives to keep up with customer service expectations. A top concern for them is how to achieve faster access to data in order to overcome the many obstacles they would encounter.

Typically, data in organizations can be in the following three forms:

  • Structured Data. Such data is stored in databases (in tables) and can be accessed by using database management systems such as Oracle, DB2 and MySQL. This data constitutes only 10% of the universal data today.
  • Unstructured Data. Such data cannot be stored using traditional relational databases.
Click to read the full post

Call to Action: be part of the EU map of big data in agriculture and food sector

BDE_survey

Are you working with Big Data in the agri-food sector? Share the information of your research project & institution and be a part of the European map of big data in agriculture and food.

agINFRA_logo_new2agINFRA, the community-specific research hub for agriculture and food, the BigDataEurope coordination action, and a number of supporting networks and initiatives, are creating a European map of big data in agricultural and food research.

In order to identify and document the key stakeholders and their relevant initiatives, UN FAO, Agro-Know, GODAN , Alterra WUR and other partners have joined forces in order to map the current status and future challenges on big data sources and technologies in the domains of Food Security, Sustainable Agriculture and Forestry, Marine and Maritime and Inland Water Research, and the Bioeconomy.… Click to read the full post

The Three Vs of Big Data

the three vs of big data

Today, there are more than 4.6 billion mobile-phone subscribers; more than 2.4 billion people with access to the Internet; and more than a billion Facebook subscribers. All of them are producing large amount of data.

It was estimated that the amount of data produced from the dawn of civilization to 2003 is 5 exabytes, at a time that every two days, we produce the same volume of data. It is even expected that by this year, the volume of digital universe of data will reach 8 zettabytes. This flood of data, which is commonly referred to as Big Data information overload or data deluge has become a challenge for many businesses.… Click to read the full post