January 2015 | Dios Kurniawan
Last week was pretty busy for IBM support here. The Netezza TwinFin12 machine, purchased in 2013, suddenly refused to work normally. It was still running, lights were still blinking green but somehow it was very slow to respond to any user query – if it responded at all. Business operations were affected. After a lengthy troubleshooting, it was decided that the cause was a known bug in the firmware, and as the time of writing, the machine has been restored and back to normal operation.
The IBM Netezza – now rebranded “IBM PureData System” – is a data warehouse appliance, just like Teradata (that we love and hate). It is a database specifically designed for data warehousing. At its heart, Netezza is powered by a set of blade servers, each running Intel CPUs and a specialized FPGA (field-programmable gate array), an electronic circuitry whose role is to help the main CPUs.
The FPGA is the secret sauce of Netezza system; it minimizes data movement by performing a lot of filtering and compression/decompression jobs on the storage level. These FPGAs are housed in S-Blades enclosures (12 of them are installed in each rack). These blades are controlled by a pair of host servers, which coordinate the execution of user queries. Storage is provided by disk array enclosures totalling 96TB of raw capacity per rack. Netezza is a well-designed architecture and offers a solid data warehousing solution.
How does Netezza compare with other DWH appliance and what are actually the strong points of Netezza? For most, it is the total costs of ownership. To some extent, it is sold at significantly lower prices than the competition such as Teradata, most probably because of the use of many off-the-shelf components. DBA activities are also simpler in Netezza, which makes hiring a dedicated DBA unnecessary in most situations, again, lowering costs. Because of its smart approach of filtering data close to the disks, some say Netezza also outperforms Teradata in many cases as well. Netezza has sold well and today there are hundreds of installations in the world.
The question now is, in Big Data era such as today, does Netezza still hold a future in the data warehousing business? Hadoop and its true cost-saving advantages has been driving people away from buying “enterprise data warehouse” products. Vendor lock-in is the pitfall of appliance approach. Hardware and software need to be sourced from the same vendor, and each time users would like to expand their data warehouses they would need to go to the same vendor again. To make things worse, most of the time older models cannot co-exist with the new model, rendering the old hardware useless.
Hadoop, in the other hand, allows users to purchase commodity servers from any manufacturer and can always co-exist. Hadoop runs on open-source software and on cheap commodity servers, saves money and at the same time offers more flexible solution. Hey, when company owners hear you say ‘saving money’, they will start listening to you. That is why Hadoop is grabbing a lot of attention nowadays.
Proponents would say IBM is now expanding its PureData product offering to include Hadoop integration which makes it more technically capable, however the economics will eventually prevail: businesses will simply select Hadoop platforms for the huge price/TB advantage over traditional DWH appliance. Even if large companies still require traditional data warehouse appliance and are willing to accept vendor lock-in, they would likely go playing safe; they would buy from vendors with the least risk such as Teradata and Oracle, which offer long-time, more mature products. FYI, Netezza is only 10-12 years old while Oracle and Teradata have been in the market for 30+ years.
Positioned in a less advantageous place, IBM Netezza/PureData System will be much less attractive to businesses in the days to come. I am predicting that IBM will run into a hard time selling Netezza and sales revenue will be hardly enough to finance further R&D efforts. Netezza will sooner or later lose the market and eventually go the way of Dodo bird.