{"id":51982,"date":"2019-09-17T13:11:07","date_gmt":"2019-09-17T11:11:07","guid":{"rendered":"https:\/\/www.cubeserv.com\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/"},"modified":"2023-12-25T21:12:54","modified_gmt":"2023-12-25T20:12:54","slug":"the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster","status":"publish","type":"post","link":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/","title":{"rendered":"The Internet of Things, Big Data and a Fischertechnik factory &#8211; Part 3: The Big Data Cluster"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"51982\" class=\"elementor elementor-51982 elementor-804\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-758afbc elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"758afbc\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-dd2e744\" data-id=\"dd2e744\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3457262 elementor-widget elementor-widget-text-editor\" data-id=\"3457262\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>In the first two parts of this blog, we presented the Fischertechnik factory simulation. This factory simulation should now continuously generate data that should end up in a big data cluster on the one hand and be reportable using SAP BW or SAP Analytics for Cloud on the other.<\/p>\n<p>We set up a Cloudera installation as a big data cluster. Similar to Linux, the free open-source operating system that can be purchased in preconfigured and then paid versions (e.g. Suse Linux, RedHat,&#8230;), there are also preconfigured installations of Apache Hadoop, e.g. from Hortonworks or Cloudera. We opted for Cloudera and installed a 60-day free Enterprise Edition. As this is only a demo application, it has also become a simple cluster with just one node.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d60367f elementor-widget elementor-widget-image\" data-id=\"d60367f\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"551\" src=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1-1024x551.png\" class=\"attachment-large size-large wp-image-812\" alt=\"\" srcset=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1-1024x551.png 1024w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1-300x161.png 300w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1-768x413.png 768w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png 1199w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-df4bb1a elementor-widget elementor-widget-text-editor\" data-id=\"df4bb1a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Hadoop comes with a whole range of services that initially mean little to the layperson and are initially very confusing in terms of their number and meaning.<\/p>\n<ul>\n<li>HBase is the system&#8217;s database.<\/li>\n<li>HDFS is the Hadoop Distributed File System, which stores data redundantly and thus guarantees high reliability.<\/li>\n<li>Hive is a service that allows SQL-like queries against the system&#8217;s database.<\/li>\n<li>Impala enables faster SQL queries than Hive thanks to better parallel algorithms. Impala is primarily used for fast reading and less for other operations such as creating, writing, or changing.<\/li>\n<li>Hue is a service that offers an SQL query workbench and visualization. Here you can try out SELECT statements, for example (and choose whether you want to query via the Hive or Impala engine).<\/li>\n<li>Kafka allows you to load and export data streams.<\/li>\n<li>Oozie is the service used to schedule batch jobs.<\/li>\n<li>YARN is the resource management service that also controls the allocation of queries to servers.<\/li>\n<li>Flume is a service for streaming logs to Hadoop.<\/li>\n<\/ul>\n<p>The Fischertechnik factory is controlled via the Siemens Logo control units. The control units and their programming interface now offer the option of writing the states of all existing inputs and outputs at periodic intervals (e.g. 1s) to a log file such as Log.csv. All inputs and outputs are always written, but we have initially limited ourselves to just three, namely the controls for the sorting chutes that separate white, red, and blue plastic cylinders from each other, depending on the color detected in the photocell.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-813dd45 elementor-widget elementor-widget-image\" data-id=\"813dd45\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1024\" height=\"652\" src=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT2-1024x652.png\" class=\"attachment-large size-large wp-image-809\" alt=\"\" srcset=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT2-1024x652.png 1024w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT2-300x191.png 300w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT2-768x489.png 768w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT2.png 1200w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-aacf70a elementor-widget elementor-widget-text-editor\" data-id=\"aacf70a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This created the simplest possible report, namely a simple counter that counted how often a 1 was reported by motor Q2 (white), Q3 (red), or Q4 (blue). This then corresponded to the corresponding number of plastic goods passed through.<\/p>\n<p>The Siemens control interface is therefore constantly writing to a local CSV file on the laptop, which is used to control the small system. This is therefore the source of our sensor data. The directory in which this log file is written has now been mounted on the Cloudera system via Samba. This meant that a CSV file was available locally from the point of view of the Cloudera system. At the same time, there was no risk that a brief interruption of the connection would lead to the writing process being aborted. Writing continued permanently and if the connection via the Internet was interrupted, the new data could be retrieved the next time the connection was re-established.<\/p>\n<p>This locally visible CSV file, the LogFile.csv, is now permanently written to a target in Kafka with the help of a Flume agent. The corresponding configuration instruction, which defines a source, a destination (sink), and a channel, is as follows:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-af49458 elementor-blockquote--skin-boxed elementor-blockquote--button-color-official elementor-widget elementor-widget-blockquote\" data-id=\"af49458\" data-element_type=\"widget\" data-widget_type=\"blockquote.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<blockquote class=\"elementor-blockquote\">\n\t\t\t<p class=\"elementor-blockquote__content\">\n\t\t\t\t#------------------------------------<br>\r\n<br>\r\ntier1.sources = r1<br>\r\n<br>\r\ntier1.sinks = k1<br>\r\n<br>tier1.channels = c1<br>\r\n<br>\r\n# Describe\/configure the source<br>\r\n<br>\r\ntier1.sources.r1.type = TAILDIR<br>\r\n<br>\r\ntier1.sources.r1.filegroups = f1<br>\r\n<br>\r\ntier1.sources.r1.filegroups.f1 = \/samba\/FischerTechnik\/LOG\/LogFile.csv<br>\r\n<br>\r\ntier1.sources.r1.positionFile = \/tmp\/flume-position_3.json<br>\r\n<br>\r\n# Describe the sink<br>\r\n<br>tier1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink<br>\r\n<br>\r\ntier1.sinks.k1.topic = sensor_csv<br>\r\n<br>\r\ntier1.sinks.k1.brokerList = quickstart.cloudera:9092<br>\r\n<br>\r\ntier1.sinks.k1.batchSize = 1<br>\r\n<br>\r\n# Use a channel which buffers events in memory<br>\r\n<br>\r\ntier1.channels.c1.type = memory<br>\r\n<br>\r\ntier1.channels.c1.capacity = 100000<br>\r\n<br>\r\ntier1.channels.c1.transactionCapacity = 10000<br>\r\n<br>\r\n# Bind the source and sink to the channel<br>\r\n<br>\r\ntier1.sources.r1.channels = c1<br>\r\n<br>\r\ntier1.sinks.k1.channel = c1<br>\r\n<br>\r\n#------------------------------------\t\t\t<\/p>\n\t\t\t\t\t<\/blockquote>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-01163e7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"01163e7\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-90d75c6\" data-id=\"90d75c6\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-493dcfb elementor-widget elementor-widget-text-editor\" data-id=\"493dcfb\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>For setting up this configuration, many thanks once again to the experts at our partner company for Hadoop systems, Ultra Tendency GmbH (www.ultratendency.com), and especially to Matthias Baumann, who has already helped us several times with his Hadoop expertise.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2905a8c elementor-widget elementor-widget-image\" data-id=\"2905a8c\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1024\" height=\"447\" src=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_3-1024x447.png\" class=\"attachment-large size-large wp-image-815\" alt=\"\" srcset=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_3-1024x447.png 1024w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_3-300x131.png 300w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_3-768x335.png 768w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_3.png 1200w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-baa39f5 elementor-widget elementor-widget-text-editor\" data-id=\"baa39f5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The sensor_csv file is now available in Cloudera in the HDFS file system and receives new data every second. To make this CSV file available for SQL queries, the following command is issued in Hive:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-46875dd elementor-blockquote--skin-boxed elementor-blockquote--button-color-official elementor-widget elementor-widget-blockquote\" data-id=\"46875dd\" data-element_type=\"widget\" data-widget_type=\"blockquote.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<blockquote class=\"elementor-blockquote\">\n\t\t\t<p class=\"elementor-blockquote__content\">\n\t\t\t\tCREATE EXTERNAL TABLE sapt90.zcssensorq (<br>\r\n<br>\r\n      key varchar(6),<br>\r\n<br>\r\n      time VARCHAR(8),<br>\r\n<br>\r\n      value VARCHAR(4),<br>\r\n<br>\r\n      sensor VARCHAR(20)<br>\r\n<br>\r\n)\r\n<br><br>\r\nSTORED BY \"org.apache.hadoop.hive.hbase.HBaseStorageHandler\"<br>\r\n<br>\r\nWITH SERDEPROPERTIES (<br>\r\n<br>\r\n       \"hbase.columns.mapping\" =<br>\r\n<br>\r\n       \":key,default:time,default:value,default:sensor\"<br>\r\n<br>    )<br>\r\n<br>\r\n    TBLPROPERTIES(\"hbase.table.name\" = \"sensor_csv\")\t\t\t<\/p>\n\t\t\t\t\t<\/blockquote>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-08975af elementor-widget elementor-widget-text-editor\" data-id=\"08975af\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This command creates an empty table shell zcssensorq (in a schema with the name sapt90) with 4 columns (key, time, value, sensor) and informs the system that the content of this table is located in an HBase table with the name sensor_csv. In this way, simple Hive or Impala queries can now be started in the Hue service, e.g. select * from zcssensorq;:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a2f643e elementor-widget elementor-widget-image\" data-id=\"a2f643e\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"518\" src=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_4-1024x518.png\" class=\"attachment-large size-large wp-image-818\" alt=\"\" srcset=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_4-1024x518.png 1024w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_4-300x152.png 300w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_4-768x388.png 768w, https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_4.png 1199w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cf54cd3 elementor-widget elementor-widget-text-editor\" data-id=\"cf54cd3\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The sensor data from the Fischertechnik factory is thus written continuously and is available as a file or table in the Hadoop system. How can this data now be made available for reporting? This is the subject of the next part of this blog.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>In the first two parts of this blog, we presented the Fischertechnik factory simulation. This factory simulation should now continuously generate data that should end up in a big data cluster on the one hand and be reportable using SAP BW or SAP Analytics for Cloud on the other. We set up a Cloudera installation &#8230; <a title=\"The Internet of Things, Big Data and a Fischertechnik factory &#8211; Part 3: The Big Data Cluster\" class=\"read-more\" href=\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\" aria-label=\"Read more about The Internet of Things, Big Data and a Fischertechnik factory &#8211; Part 3: The Big Data Cluster\">Read more<\/a><\/p>\n","protected":false},"author":16,"featured_media":814,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[41],"tags":[45,44,43],"class_list":["post-51982","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-iot","tag-business-analytics-platform","tag-hadoop","tag-hana"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v23.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The Internet of Things, Big Data and a Fischertechnik factory - Part 3: The Big Data Cluster - CubeServ<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Internet of Things, Big Data and a Fischertechnik factory - Part 3: The Big Data Cluster\" \/>\n<meta property=\"og:description\" content=\"In the first two parts of this blog, we presented the Fischertechnik factory simulation. This factory simulation should now continuously generate data that should end up in a big data cluster on the one hand and be reportable using SAP BW or SAP Analytics for Cloud on the other. We set up a Cloudera installation ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\" \/>\n<meta property=\"og:site_name\" content=\"CubeServ\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/CubeServ\" \/>\n<meta property=\"article:published_time\" content=\"2019-09-17T11:11:07+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-25T20:12:54+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1199\" \/>\n\t<meta property=\"og:image:height\" content=\"645\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Thorsten Stossmeister\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CubeServ\" \/>\n<meta name=\"twitter:site\" content=\"@CubeServ\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Thorsten Stossmeister\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\"},\"author\":{\"name\":\"Thorsten Stossmeister\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/#\/schema\/person\/2a15f26fb37d4ff3c8fb491ac4ce66d9\"},\"headline\":\"The Internet of Things, Big Data and a Fischertechnik factory &#8211; Part 3: The Big Data Cluster\",\"datePublished\":\"2019-09-17T11:11:07+00:00\",\"dateModified\":\"2023-12-25T20:12:54+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\"},\"wordCount\":928,\"publisher\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png\",\"keywords\":[\"Business Analytics Platform\",\"Hadoop\",\"HANA\"],\"articleSection\":[\"IoT\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\",\"url\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\",\"name\":\"The Internet of Things, Big Data and a Fischertechnik factory - Part 3: The Big Data Cluster - CubeServ\",\"isPartOf\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png\",\"datePublished\":\"2019-09-17T11:11:07+00:00\",\"dateModified\":\"2023-12-25T20:12:54+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage\",\"url\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png\",\"contentUrl\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png\",\"width\":1199,\"height\":645},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/www.cubeserv.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Internet of Things, Big Data and a Fischertechnik factory &#8211; Part 3: The Big Data Cluster\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/#website\",\"url\":\"https:\/\/www.cubeserv.com\/en\/\",\"name\":\"CubeServ\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.cubeserv.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/#organization\",\"name\":\"CubeServ Group\",\"url\":\"https:\/\/www.cubeserv.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2020\/07\/CubeServ_Web_Logo-768x372-1.png\",\"contentUrl\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2020\/07\/CubeServ_Web_Logo-768x372-1.png\",\"width\":768,\"height\":372,\"caption\":\"CubeServ Group\"},\"image\":{\"@id\":\"https:\/\/www.cubeserv.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/CubeServ\",\"https:\/\/x.com\/CubeServ\",\"https:\/\/www.instagram.com\/cubeservgroup\/\",\"https:\/\/www.linkedin.com\/company\/112961\/\",\"https:\/\/www.youtube.com\/user\/CubeServGroup\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/#\/schema\/person\/2a15f26fb37d4ff3c8fb491ac4ce66d9\",\"name\":\"Thorsten Stossmeister\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.cubeserv.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2022\/12\/stossmeister_thorsten-backup-150x150.jpg\",\"contentUrl\":\"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2022\/12\/stossmeister_thorsten-backup-150x150.jpg\",\"caption\":\"Thorsten Stossmeister\"},\"description\":\"Competency Lead EDW\/HANA with sometimes old-fashioned preferences for backend and ABAP\",\"sameAs\":[\"https:\/\/www.linkedin.com\/in\/stossmeister-thorsten-4b2427117\/\"],\"url\":\"https:\/\/www.cubeserv.com\/en\/author\/stossmeistert\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"The Internet of Things, Big Data and a Fischertechnik factory - Part 3: The Big Data Cluster - CubeServ","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/","og_locale":"en_US","og_type":"article","og_title":"The Internet of Things, Big Data and a Fischertechnik factory - Part 3: The Big Data Cluster","og_description":"In the first two parts of this blog, we presented the Fischertechnik factory simulation. This factory simulation should now continuously generate data that should end up in a big data cluster on the one hand and be reportable using SAP BW or SAP Analytics for Cloud on the other. We set up a Cloudera installation ... Read more","og_url":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/","og_site_name":"CubeServ","article_publisher":"https:\/\/www.facebook.com\/CubeServ","article_published_time":"2019-09-17T11:11:07+00:00","article_modified_time":"2023-12-25T20:12:54+00:00","og_image":[{"width":1199,"height":645,"url":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png","type":"image\/png"}],"author":"Thorsten Stossmeister","twitter_card":"summary_large_image","twitter_creator":"@CubeServ","twitter_site":"@CubeServ","twitter_misc":{"Written by":"Thorsten Stossmeister","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#article","isPartOf":{"@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/"},"author":{"name":"Thorsten Stossmeister","@id":"https:\/\/www.cubeserv.com\/en\/#\/schema\/person\/2a15f26fb37d4ff3c8fb491ac4ce66d9"},"headline":"The Internet of Things, Big Data and a Fischertechnik factory &#8211; Part 3: The Big Data Cluster","datePublished":"2019-09-17T11:11:07+00:00","dateModified":"2023-12-25T20:12:54+00:00","mainEntityOfPage":{"@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/"},"wordCount":928,"publisher":{"@id":"https:\/\/www.cubeserv.com\/en\/#organization"},"image":{"@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage"},"thumbnailUrl":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png","keywords":["Business Analytics Platform","Hadoop","HANA"],"articleSection":["IoT"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/","url":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/","name":"The Internet of Things, Big Data and a Fischertechnik factory - Part 3: The Big Data Cluster - CubeServ","isPartOf":{"@id":"https:\/\/www.cubeserv.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage"},"image":{"@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage"},"thumbnailUrl":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png","datePublished":"2019-09-17T11:11:07+00:00","dateModified":"2023-12-25T20:12:54+00:00","breadcrumb":{"@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#primaryimage","url":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png","contentUrl":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2019\/09\/IoT_1.png","width":1199,"height":645},{"@type":"BreadcrumbList","@id":"https:\/\/www.cubeserv.com\/en\/the-internet-of-things-big-data-and-a-fischertechnik-factory-part-3-the-big-data-cluster\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/www.cubeserv.com\/en\/"},{"@type":"ListItem","position":2,"name":"The Internet of Things, Big Data and a Fischertechnik factory &#8211; Part 3: The Big Data Cluster"}]},{"@type":"WebSite","@id":"https:\/\/www.cubeserv.com\/en\/#website","url":"https:\/\/www.cubeserv.com\/en\/","name":"CubeServ","description":"","publisher":{"@id":"https:\/\/www.cubeserv.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.cubeserv.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.cubeserv.com\/en\/#organization","name":"CubeServ Group","url":"https:\/\/www.cubeserv.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.cubeserv.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2020\/07\/CubeServ_Web_Logo-768x372-1.png","contentUrl":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2020\/07\/CubeServ_Web_Logo-768x372-1.png","width":768,"height":372,"caption":"CubeServ Group"},"image":{"@id":"https:\/\/www.cubeserv.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/CubeServ","https:\/\/x.com\/CubeServ","https:\/\/www.instagram.com\/cubeservgroup\/","https:\/\/www.linkedin.com\/company\/112961\/","https:\/\/www.youtube.com\/user\/CubeServGroup"]},{"@type":"Person","@id":"https:\/\/www.cubeserv.com\/en\/#\/schema\/person\/2a15f26fb37d4ff3c8fb491ac4ce66d9","name":"Thorsten Stossmeister","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.cubeserv.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2022\/12\/stossmeister_thorsten-backup-150x150.jpg","contentUrl":"https:\/\/www.cubeserv.com\/wp-content\/uploads\/2022\/12\/stossmeister_thorsten-backup-150x150.jpg","caption":"Thorsten Stossmeister"},"description":"Competency Lead EDW\/HANA with sometimes old-fashioned preferences for backend and ABAP","sameAs":["https:\/\/www.linkedin.com\/in\/stossmeister-thorsten-4b2427117\/"],"url":"https:\/\/www.cubeserv.com\/en\/author\/stossmeistert\/"}]}},"_links":{"self":[{"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/posts\/51982","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/comments?post=51982"}],"version-history":[{"count":0,"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/posts\/51982\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/media\/814"}],"wp:attachment":[{"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/media?parent=51982"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/categories?post=51982"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cubeserv.com\/en\/wp-json\/wp\/v2\/tags?post=51982"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}