Data analytics, as defined in “Competing on Analytics: The New Science of Winning” by Thomas H. Davenport and Jeanne G. Harris, refers to the extensive use of data, statistical and quantitative analysis, explanatory and predictive models, and fact-based management to derive decisions and actions. It has the meaning of discovery and communication of meaningful patterns in data.
Dealing with business analytics implies the efficient use of quantitative analysis, statistics, as well as information modelling to shape business decisions. In this context, people dealing with business analytics can be classified into three levels: analytics scientists who build complex models to extract insights from data, analytics experts who apply the models from the first level to real business problems, and analytics specialists who can build insights based on the output of the previous steps.… Click to read the full post
The Internet of Things or IoT is basically a complex network that seamlessly connects people and things together through the Internet. Theoretically, anything that can be connected (smart watches, cars, homes, thermostats, vending machines, servers…) will be connected in the near future using sensors and RFID tags. This allows connected objects to continuously send data over the Web and from anywhere. The first time the term was used was in 1999 by Kevin Ashton, the creator of the RFID standard.
IoT will have the advantage of bringing us smart cities with smart cars, secure and efficient buildings, and smart traffic management systems.… Click to read the full post
Databases come in a variety of tastes, such as relational (e.g. Postgres, Oracle and MySQL), document-oriented (eg. MongoDB, CouchDB and SimpleDB), columnar (e.g. BigTable and HBase), key-value (e.g. MemcacheDB, Redis and Riak) XML (e.g. MarkLogic, BaseX and eXist) and graph (e.g. Neo4J, GraphDB and Giraph). All data stores support writing and retrieving data but with some differences in terms of database indexing, database schema, query format, data sharding, replication, scalability and others.
Although the relational model and the Structured Query Language (SQL) were for decades the de facto for storing data, it has become established that relational databases are no more the winners when it comes to flexibility and scalability.… Click to read the full post
Hadoop is currently the most common single Big Data platform. However, still other techniques play a role in the scene. While there are proprietary distributions for Hadoop which are developed by giant Big Data companies, such commercial products rely heavily on open source projects.
Hadoop ecosystem includes a set of tools that function near MapReduce and HDFS (the two main Hadoop core components) and help the two store and manage data, as well as perform the analytic tasks. As there is an increasing number of new technologies that encircle Hadoop, it is important to realize that certain products maybe more appropriate to fulfill certain requirements than others.… Click to read the full post
Apache Hadoop is an emerging technology that was designed to address the specific requirements of Big Data. It can deal with petabytes of structured and unstructured data. The technology was developed by Yahoo! in 2005 and it got its name from a toy elephant. However, Hadoop does not work alone. Rather, it is part of an increasing number of associated technologies such as HBase, Hive, Pig, Oozie, and Zookeeper.
Apache Hadoop Ecosystem (source: quantfarm.com)
- Is Fault-tolerance open-source software framework that can deal with software and hardware failures.
- Scales well to any increase in processors, memory or storage devices.
… Click to read the full post
Because customer relationship constitutes an important part of any strategic decision-making process, shifting towards Big Data technologies would enable executives to keep up with customer service expectations. A top concern for them is how to achieve faster access to data in order to overcome the many obstacles they would encounter.
Typically, data in organizations can be in the following three forms:
- Structured Data. Such data is stored in databases (in tables) and can be accessed by using database management systems such as Oracle, DB2 and MySQL. This data constitutes only 10% of the universal data today.
- Unstructured Data. Such data cannot be stored using traditional relational databases.
… Click to read the full post
Today, there are more than 4.6 billion mobile-phone subscribers; more than 2.4 billion people with access to the Internet; and more than a billion Facebook subscribers. All of them are producing large amount of data.
It was estimated that the amount of data produced from the dawn of civilization to 2003 is 5 exabytes, at a time that every two days, we produce the same volume of data. It is even expected that by this year, the volume of digital universe of data will reach 8 zettabytes. This flood of data, which is commonly referred to as Big Data information overload or data deluge has become a challenge for many businesses.… Click to read the full post
he term “Big Data” is relative and highly dependent. For example, the organizations that lack the ability to handle, store and analyze their own sets of data, are in fact experiencing the Big Data “phenomenon”. Nevertheless, this is not what Big Data is all about. Besides of being by order of magnitudes in terms of Volume, data has to be of greater Variety and complexity, and generated at a high Velocity, which are usually referred to as the three Vs of Big Data.
The three Vs of Big Data (source: http://blog.softwareinsider.org/2012/02/27/mondays-musings-beyond-the-three-vs-of-big-data-viscosity-and-virality/)
A better definition of Big Data might be: the processing, interpretation and representation of large volumes of data (typically, petabytes or zettabytes) originating from different sources in a way that makes the data meaningful and usable.… Click to read the full post
We recently had the opportunity to discuss about a new idea that allows sharing of agricultural tools & machinery among farmers, with Thomi Giotopoulou, one of the people working on this concept. The idea seems to be really promising, especially in the case of Greece (but also for similar countries), where farms are limited in size and farmers do not always have the funds required for actually buying an expensive piece of agricultural machinery; instead they are willing to share the equipment they already have and in return borrow a tool that they do not own and use it for as long as needed before returning it to its owner.… Click to read the full post
We are back from a short summer break with a special blog post; special because it is the first one that comes from a person outside the Agro-Know team. In fact, the author of this blog post is Mohammed Z. Al-Taie, a PhD candidate at the Universiti Teknologi Malaysia (UTM), Faculty of Computing (FC) – Soft Computing Research Group (SCRG). Even though the topic of the post is not related to agriculture nor any other green topic, the post provides an analysis on how the social media can be used in cases of disasters. Mr. Taie was kind enough to contact us and offered to publish some parts of his research through the AK blog; an offer that we could not refuse, as the analysis seems really interesting and thorough and on top of that, it focuses on the use of social networks (which we frequently use mostly for dissemination purposes) for alternative but useful purposes.… Click to read the full post