Big Data as Part of Internet of Things Solutions

Internet Of Things

The Internet of Things or IoT is basically a complex network that seamlessly connects people and things together through the Internet. Theoretically, anything that can be connected (smart watches, cars, homes, thermostats, vending machines, servers…) will be connected in the near future using sensors and RFID tags. This allows connected objects to continuously send data over the Web and from anywhere. The first time the term was used was in 1999 by Kevin Ashton, the creator of the RFID standard.

Internet Of Things


IoT will have the advantage of bringing us smart cities with smart cars, secure and efficient buildings, and smart traffic management systems. It will achieve major efficiency in industry, healthcare and retail and will save millions of dollars. Today, attention is being given by governments, industry players, and manufacturers.

The IoT is not a single technology but rather a holistic view of how objects, devices and structures are increasingly being interconnected, with the help of a number of enablers: machine-to-machine communications, passive and active radio frequency ID (RFID) that connect physical objects, expansion of Internet networks and their new underlying technologies, mobile devices, cloud computing services, Big Data technologies, and the new paradigms in software design that incline towards building loosely coupled distributed applications.

Sensor devices include any device that generate information about itself or about the environment such as heat and humidity sensors, pressures components, cameras, microphones, etc. Although in a way or another they overlap, IoT and sensor networks are not the same because in addition to sensors, IoT networks encompass other “things” besides sensors, at a time sensor networks should not necessarily be connected to the Internet.

Additionally, to enable the communication over long distances between different systems, a range of communication protocols are involved in IoT processes such as Wi-Fi, Bluetooth, GPRS, 3G, LTE, ZigBee networking protocol for very low-power environments, Z-Wave home automation communication protocol, Near Field Communication or NFC which is an ensemble of protocols that allow electronic devices to establish radio communication either by touching them together or by bringing them into proximity, and many other forms of data connectivity.

Companies such as Google, LG, Intel, Samsung, Texas Instruments, Freescale Semiconductor, NXP Semiconductors, GreenPeak Technologies, Anaren Bosch and Schneider, Vodafone, Verizon and Sigfox are involved providing the infrastructure (both hardware and software) for the generated data.

Business forecasters expect that by 2019, 35 billion things will be connected through the Internet, and the wealth of data will be 40,000 exabytes of machine-generated content by 2020 [PDF]. These large amounts of streaming data need to be stored, organized, analysed and harnessed. From business intelligence point of view, such data is useful to improve customer service, invest in new products, optimize operations and reuse in other analytics operations.


Just analyzing the data will not give advantage. Companies need to: innovate their business processes, run smarter and to deliver new value to customers. The data collected through IoT becomes more useful if collected from different types of devices and then combined in a creative way. This should be followed by building connections and correlations between data units that lead to intelligent decision making processes.

With estimates like the above, the hardest challenge is how to dive into diverse and ever growing data streaming sources and to extract meaningful information out of it. Fortunately, software development paths (Hadoop and its incorporated technologies such as HDFS, MapReduce, Hive, Pig and others that support Big Data paradigms) will embrace a large part of the IoT functionality. The structured, semi-structured and unstructured huge volumes of data need Big Data solutions to be combined, integrated and analyzed regardless of their source, format or size. This task is not easy, especially that data volumes double every few months.

Three challenges are involved in processing the data gathered from IoT: first, data collection, which includes gathering data from different sources. Second: data storage. There are several possible directions in this regard such as storing the data in a relational database, in the cloud or in a NoSQL database (e.g. MongoDB and CouchDB). Finally, data analysis. This the most challenging part. How to extract value from the huge amount of collected data? This requires the development of applications that can analyze the data for patterns, trends and critical points.. a time consuming mission particularly for real-time data processing.

The main advantage of Big Data to IoT is that predictive analytics is provided over all the data, not only a small part of it. This allows to dig up for patterns, correlations, and build insights from data stored in Big Data databases in a way never expected before. For IoT, the use of Big Data will create more accurate market strategies and enhanced Return on Investment (ROI) in the future sales. This is because data analysis will be implemented over the complete product lifecycle, and we will get feedback from devices as well as from customers.

Big Data and Internet of Things


With IoT, more devices are connected to the Internet, which means more amounts of data that are sources of Big Data. Further, better use of Big Data can improve the efficiency of data in IoT. One final thing: the Big Data that appeals to the IoT is the type of data that is collected from different sources and anazlyed to reveal trends, patterns and associations. A company that does not have the financial sources, time, or experience to deal with the new huge amount of incoming data will end up losing all its storage resources, money, effort and time.


Leave a Comment.

Can you prove that you are not a bot? * Time limit is exhausted. Please reload CAPTCHA.