Parallel Data Warehouse (PDW) AU1 released

Microsoft’s PDW has seen a big boost in visibility and sales over the past year, and part of the reason is due to frequent upgrades to the hardware and software.  About every six months there is an appliance update, which is sort of like a service pack, except that in addition to new features it usually also includes upgrades in hardware (since the PDW is a hardware and software appliance).  Just released is Appliance Update 1 (AU1) to version 2 of PDW.  Details below, including improvements with HDInsight, Hadoop, and Polybase,

First on-premises HDInsight region

  • Enables customers to load, query and analyze structured and unstructured data within a single appliance.
  • T-SQL compatibility over Hadoop via Polybase.
  • Windows failover clustering and full hardware redundancy.
  • Unified management and monitoring experience across HDInsight and PDW.

Polybase Enhancements

  • Better Polybase performance
    • Compute push-down enables 2x improvement.
    • Ability to create statistics over external tables via fullscan or sampling.
  • Polybase T-SQL semantics – Close alignment with T-SQL semantics for data types & conversions.
  • RCFile format support.
  • Compressed and uncompressed Hadoop data support.
  • Better manageability and user experience with new catalog views, DMVs, EXPLAIN, and SSDT support.

Hybrid Cloud Support

  • Import and export to Azure Blob storage – WASB/ASV.
  • Seamless query across WASB and HDInsight region.

PDW Performance Improvements

  • Parallel data load enables 2-8x better load performance.
  • Native Type Conversion for Loader improves load time of single load by 75%, with multiple concurrent loads achieving 2x improvements, validated up to 9.4TB/Hr.

Integrated Authentication (PDW)

  • Enables single sign-on for simplified user management and easier authentication.
  • Support for Kerberos and NTLM.
  • Windows authentication supported through SqlClient, ODBC, and OLE-DB.

Transparent Data Encryption (PDW):

  • Protects data at rest by encrypting data pages on disk, transactional logs, and database backups.

Add Capacity (PDW):

  • Enables customers to expand to any supported topology.
  • All operations except data redistribution are online operations, reducing downtime.

Add Region (HDInsight):

  • Enables customers to add HDI region to existing PDW appliance.

Better upgrade experience:

  • Upgrades can now be run remotely.
  • Reduces time to upgrade appliances by simplifying and minimizing manual steps.

About James Serra

James is a big data and data warehousing solution architect at Microsoft. Previously he was an independent consultant working as a Data Warehouse/Business Intelligence architect and developer. He is a prior SQL Server MVP with over 25 years of IT experience.
This entry was posted in PDW/APS, SQLServerPedia Syndication. Bookmark the permalink.