Big Data Analytics: A Closer Look

Businesses have recognized the benefit rooted in data and, therefore, put together people, technologies, and processes to collect, store and analyze huge amounts of data. A key element for deriving value from data is by using analytics.

Big Data analytics is a business strategy that uses technology to gain deeper insights into customers, partners and businesses and so achieving competitive advantage. It includes working with huge data that, because of its size and variety, lie beyond the ability of typical database management systems to store, manage and analyze.

The use of “Big Data analytics” involves two dimensions: one is Big Data, which is annotated with the three ‘Vs’: data volume, velocity, and variety.… Click to read the full post

“I do big data for the food & beverage industry” – OK, but for which problem?

milk

I was reading through the 2013 report on “Formula for growth: Innovation, big data and analytics” that the US Grocery Manufacturers Association (GMA) and Deloitte Consulting L.L.P. produced. This is a very interesting report, since it outlines the opportunities big data offers to food and beverage manufacturers, bringing the industry perspective to the big data discussion. The report also discusses how data mining technologies are starting to transform the consumer packaged goods marketplace, and outlines what companies may do to use the technologies to improve performance.

A GMA video summarizing the report findings (find it here: http://www.gmaonline.org/issues-policy/collaborating-with-retailers/big-data-analytics/welcome)

A very interesting Deloitte video with Prof. Tom Davenport summarises the report findings

There are a couple of recommendations in the report that rang a bell:

  • Business context is required to operationalize big data, analytics, and innovation
  • The majority of the consumer packaged goods (CPG) industry is lagging in data and analytical capabilities
  • Rapid-fire pace of innovation requires data & analytics competency

Yes, I agree that context is king.… Click to read the full post

Big data in Europe calls on Agro-Know

paris

We have been following Big Data Europe since its early stages, learning about the recent advances and trends in big data by prestigious partners like Fraunhofer (yes, the guys that invented MP3). We got more and more involved in this flagship big data initiative for Europe, sharing our understanding of data-related challenges in the agri-food sector, what kind of big data our communities work with, and how cutting edge solutions using big data analytics may be developed to serve their needs.

This is the time to take an important step forward: and beautiful Paris is the place where this will happen.… Click to read the full post

Fighting hunger with big data: doing it the CGIAR way

hungergames

Get ready for some serious numbers – by 2030, the CGIAR wants its action to result in 150 million fewer hungry people, 100 million fewer poor people – at least 50% of whom are women, and 190 million ha less degraded land. They have mobilised a tremendous amount of money from their donors to achieve it. And they are now designing the way in which they will make it happen.

Taking a closer look to their recently published progress of work, I was intrigued by two things:

  • They follow a truly transparent process, since they have published quite elaborate and detailed pre-proposals that are still under evaluation.
Click to read the full post

Big Data as Part of Internet of Things Solutions

Internet Of Things

The Internet of Things or IoT is basically a complex network that seamlessly connects people and things together through the Internet. Theoretically, anything that can be connected (smart watches, cars, homes, thermostats, vending machines, servers…) will be connected in the near future using sensors and RFID tags. This allows connected objects to continuously send data over the Web and from anywhere. The first time the term was used was in 1999 by Kevin Ashton, the creator of the RFID standard.

Internet Of Things

Source: http://inoviagroup.se

IoT will have the advantage of bringing us smart cities with smart cars, secure and efficient buildings, and smart traffic management systems.… Click to read the full post

The Future is for NoSQL Data Storage model!

NOSQL

Databases come in a variety of tastes, such as relational (e.g. Postgres, Oracle and MySQL), document-oriented (eg. MongoDB, CouchDB and SimpleDB), columnar (e.g. BigTable and HBase), key-value (e.g. MemcacheDB, Redis and Riak) XML (e.g. MarkLogic, BaseX and eXist) and graph (e.g. Neo4J, GraphDB and Giraph). All data stores support writing and retrieving data but with some differences in terms of database indexing, database schema, query format, data sharding, replication, scalability and others.

Although the relational model and the Structured Query Language (SQL) were for decades the de facto for storing data, it has become established that relational databases are no more the winners when it comes to flexibility and scalability.… Click to read the full post

Hadoop Ecosystem: an Integrated Environment for Big Data

Hadoop_Ecosystem

Hadoop is currently the most common single Big Data platform. However, still other techniques play a role in the scene. While there are proprietary distributions for Hadoop which are developed by giant Big Data companies, such commercial products rely heavily on open source projects.

Hadoop ecosystem includes a set of tools that function near MapReduce and HDFS (the two main Hadoop core components) and help the two store and manage data, as well as perform the analytic tasks. As there is an increasing number of new technologies that encircle Hadoop, it is important to realize that certain products maybe more appropriate to fulfill certain requirements than others.… Click to read the full post

Hadoop as the Backbone of Big Data Technologies

Hadoop_Ecosystem

Apache Hadoop is an emerging technology that was designed to address the specific requirements of Big Data. It can deal with petabytes of structured and unstructured data. The technology was developed by Yahoo! in 2005 and it got its name from a toy elephant. However, Hadoop does not work alone. Rather, it is part of an increasing number of associated technologies such as HBase, Hive, Pig, Oozie, and Zookeeper.

Hadoop_Ecosystem

Apache Hadoop Ecosystem (source: quantfarm.com)

Hadoop:

  • Is Fault-tolerance open-source software framework that can deal with software and hardware failures.
  • Scales well to any increase in processors, memory or storage devices.
Click to read the full post

Obstacles to the Adoption of Big Data

Obstacles to Big Data Implementation (Source: http://www.eweek.com/)

Because customer relationship constitutes an important part of any strategic decision-making process, shifting towards Big Data technologies would enable executives to keep up with customer service expectations. A top concern for them is how to achieve faster access to data in order to overcome the many obstacles they would encounter.

Typically, data in organizations can be in the following three forms:

  • Structured Data. Such data is stored in databases (in tables) and can be accessed by using database management systems such as Oracle, DB2 and MySQL. This data constitutes only 10% of the universal data today.
  • Unstructured Data. Such data cannot be stored using traditional relational databases.
Click to read the full post

The Three Vs of Big Data

the three vs of big data

Today, there are more than 4.6 billion mobile-phone subscribers; more than 2.4 billion people with access to the Internet; and more than a billion Facebook subscribers. All of them are producing large amount of data.

It was estimated that the amount of data produced from the dawn of civilization to 2003 is 5 exabytes, at a time that every two days, we produce the same volume of data. It is even expected that by this year, the volume of digital universe of data will reach 8 zettabytes. This flood of data, which is commonly referred to as Big Data information overload or data deluge has become a challenge for many businesses.… Click to read the full post