James works at Microsoft as a big data and data warehousing solution architect where he has been for most of the last eleven years. Before that he was an independent consultant working as a Data Warehouse/Business Intelligence architect and developer. He is a prior SQL Server MVP with over 40 years of IT experience. Check out his book Deciphering Data Architectures: Choosing Between a Modern Data Warehouse, Data Fabric, Data Lakehouse, and Data Mesh.
The Azure Data Architecture Guide has just been released! Check it out: http://aka.ms/ADAG Think of it as a menu or syllabus for data professionals. What service should you use, why, and when would you use it. I had a small involvement … Continue reading →
In this podcast I talk with Mike Rabinovici of Dimodelo Solutions about data being the new currency, the importance of showing customers the art of the possible, and last but not least my go to TV show. Click here to listen. Also … Continue reading →
I have blogged about Data Virtualization vs Data Warehouse and wanted to blog on a similar topic: Data Virtualization vs. Data Movement. Data virtualization integrates data from disparate sources, locations and formats, without replicating or moving the data, to create a single … Continue reading →
As I mentioned in my recent blog Use cases of various products for a big data cloud solution, with so many products it can be difficult to know the best products to use when building a solution. When it comes to … Continue reading →
There have been a number of enhancements to Hadoop recently when it comes to fast interactive querying with such products as Hive LLAP and Spark SQL which are being used over slower interactive querying options such as Tez/Yarn and batch … Continue reading →
Azure Databricks (documentation and user guide) was announced at Microsoft Connect, and with this post I’ll try to explain its use case. At a high level, think of it as a tool for curating and processing massive amounts of data and … Continue reading →
Microsoft Connect(); is a developer event from Nov 15-17, where plenty of announcements are made. Here is a summary of the data platform related announcements: Azure Databricks: In preview, this is a fast, easy, and collaborative Apache Spark based analytics platform optimized … Continue reading →
Better late than never: The Analytics Platform System (APS), which is a renaming of the Parallel Data Warehouse (PDW), released an appliance update (AU6) about a year ago, and I missed the announcement. Below is what is new in this release, also called … Continue reading →
Azure SQL Database Managed Instance is a new flavor of Azure SQL Database that is a game changer. It offers near-complete SQL Server compatibility and network isolation to easily lift and shift databases to Azure (you can literally backup an … Continue reading →
There were so many good presentations at Microsoft Ignite, all of which can be viewed on-demand. I wanted to list the big data related presentations that I found the most useful. It’s a lot of stuff to watch and with … Continue reading →
There are a tremendous amount of Microsoft products that are cloud-based for building big data solutions. It’s great that there are so many products to choose from, but it does lead to confusion on what are the best products to … Continue reading →
I just uploaded a new presentation called “What’s new in SQL Server 2017”. It covers all the new features in SQL Server 2017 (which went GA this past Monday), as well as details on upgrading and migrating to SQL Server … Continue reading →
Many product announcements were made this week at Microsoft Ignite, and I wanted to give a quick overview of all the data platform related announcements: SQL Server 2017 on Linux, Windows, and Docker, generally available on October 2nd. SQL Server … Continue reading →
In SQL Server, scaling out reads (i.e. using Active secondary replicas via AlwaysOn Availability Groups) is a lot easier than scaling out writes. So what are your options when you have a tremendous amount of writes that scaling up will not handle, … Continue reading →
Last week Microsoft released a public preview of a new service called Azure Archive Blob Storage, offering customers a lower-cost cloud storage solution for rarely accessed data. This allows for storage tiering, where organizations can place their critical data on expensive, … Continue reading →
Data virtualization goes by a lot of different names: logical data warehouse, data federation, virtual database, and decentralized data warehouse. Data virtualization allows you to integrate data from various sources, keeping the data in-place, so that you can generate reports … Continue reading →
I previously blogged about Microsoft certification changes, and since then there have been some new Microsoft Big Data certifications exams released: Already live: 70-773: Analyzing Big Data with Microsoft R 70-774: Perform Cloud Data Science with Azure Machine Learning 70-775: Perform … Continue reading →
Microsoft has released a preview of the Azure Analysis Services web designer. This is a browser-based experience that will allow developers to start creating and managing Azure Analysis Services (AAS) semantic models quickly and easily. SQL Server Data Tools (SSDT) … Continue reading →
Azure SQL Data Sync has been ignored for quite some time, but has finally gotten an update (it’s in public preview). This release includes several major improvements to the service including new Azure portal support, PowerShell and REST API support, … Continue reading →
Microsoft has introduced two new disk sizes for Azure IaaS VMs in P40 (2TB) and P50 (4TB) for both managed and unmanaged Premium Disks and S40 (2TB) and S50 (4TB) for both managed and unmanaged Standard Disks. This enables customers … Continue reading →