Category Archives: Business Intelligence

The Power BI Premium Pricing Model – The Good, The Bad and the Ugly

On May 3, Microsoft announced sweeping changes to the pricing of Power BI by introducing a new “Power BI Premium” SKU. The announcement itself can be found here, and there a number of other related resources worth that I am listing here for convenience:

Distribute to large audiences with Power BI apps
Changes to Power BI embedded
Power BI Premium White Paper
Power BI Pricing
Premium pricing calculator
Introducing Power BI Report Server

Power BI Premium is intended to address deficiencies in the current pricing model primarily with respect to sharing content. In my opinion, the new model succeeds in this goal for the most part, but it leaves a significant number of customers behind, and it also leaves many unanswered questions. These problems need to be addressed for Power BI to succeed in its goal of bringing BI to the masses. Overall, I like what Microsoft is trying to do with this new pricing model, and with a few tweaks, I think that it can resonate.

First, we need to understand the new model, and to do that, we need to understand the former model and the need for Premium. Given that the former model (consisting of free and Pro licenses) has not been replaced (although it is changing significantly), we will refer to it as the original model, and when Premium is added to it, we will refer to that as the Premium model. The original model is still completely relevant moving forward.

The original model and the need for change

The original model is relatively simple, and relatively unique to the industry. Power BI users are licensed for either free or Pro features. If a report contains any Pro capabilities, any consuming user requires a Pro license. A free user can create a report that uses Pro features, but that same user will not be able to consume that report in the free service. This is a very important distinction to understand. The author of a report (using Power BI Desktop) could be a free user, use a Pro feature, and after deploying the report to the service, be unable to use it in the service.

Difference between free and Pro from a feature standpoint is no longer (as of this writing) available on the Power BI pricing page, however, prior to June 1, 2017 it is the list below.

Therefore, if a report is configured to be refreshed more than once per day, or even if the time of day is specified, or if the report uses on-premises data, then all users accessing that report require a Pro license. Given that Power BI is all about bringing Business Intelligence to the masses, when each one of those masses needs to pay $10/month, it tends to constrain adoption, particularly if a report’s audience is organization wide, and you are in a very large organization.

Report sharing is also relatively limited. Reports can be shared anonymously, which is insecure. Dashboards and their constituent reports can be shared either internally or externally, but they are read only. Finally, both dashboards and reports can be shared through Group workspaces (now app workspaces). Currently, Group workspaces do not allow for external sharing, but they are the preferred means of sharing. However, they too require Pro licenses, which constrains adoption. For the free user, anonymous and dashboard sharing are the only real options.

New model

The introduction of Power BI Premium aims to solve some of the sharing issues listed above, and therefore to help drive adoption. Premium capacity is an add on to a Power BI tenant (organization), and is different that free or Pro licenses which are assigned to users. An organization can purchase Premium capacity, and then a Pro user (this is restricted to Pro) can move or publish content to the Premium capacity. Once the content is in Premium storage, all users can utilize all the features in the dashboards and reports. Premium effectively removes all feature barriers from the reports.

Premium storage also brings many performance enhancements, such as the ability to refresh data up to 48 times per day (vs the previous 8), and the effective removal of data model size limits.

Without Premium, there are also several changes to the original licensing model. According to the May 3 Announcement FAQ on the Power BI community site:

Beginning June 1, the free service will have capabilities equivalent to Power BI Pro. This includes the same 1 GB workbook size limit, up to 8 daily scheduled refreshes for datasets, and maximum 1 million rows/hour streaming data rate. We’re also providing access to all data sources, including those available through the on-premises data gateway. Peer-to-peer dashboard sharing, group workspaces (now called app workspaces), export to PowerPoint, export to CSV/Excel, and analyze in Excel with Power BI apps are capabilities limited to Power BI Pro.”

Therefore, after June 1, 2017, Pro features are effectively an addition to the free features, and the feature differences should be as below:

From the May 3, 2017 announcement:

“Going forward, we will improve the free service to have the same functionality as Power BI Pro, but will limit sharing and collaboration features to only Power BI Pro users.”

The only features that Pro will have that free will not are those that are related to sharing. The above feature list reflects that.

Power BI Embedded

Power BI Embedded is the way that developers can embed Power BI in their applications. Using Power BI Embedded, until now, developers build reports, deploy them to their Azure instance, and call them from their applications. End users do not need any sort of Power BI licenses, and the developers are charged per report “render session”. This charging model has been one of the criticisms of Power BI embedded in that it is very difficult to predict costs. ISVs are at the mercy of the end users viewing reports, and any measure that is put into place to curb these render sessions is by definition a disincentive to adoption.

The fact that Embedded runs in a different namespace than the core Power BI service is another, leading to differences between the capabilities of Power BI Embedded and the core Power BI service. For example, the current iteration of Power BI Embedded cannot use the On-Premises Data Gateway, which can be quite restrictive.

Power BI Embedded is changing to use the new Premium capacity model. ISVs will purchase Premium capacity, and serve reports to their end users from that space. There will only be a single namespace for all Power BI content.

What’s Good

Power BI Premium solves to sharing problem for organizations that want to distribute their BI assets across the organization. If organizations would be accessing on premises data, a key feature of Power BI for enterprises, the Pro license requirement has discouraged adoption. With Premium capacity, an report publisher can share content with as many users as necessary without worrying about licensing the target users. Even better, those target users can be external, further extending the reach of that content.

For large enterprises, this has the potential to turn Power BI from a niche solution to something that is used by everyone.

The changes to the original model also makes things clearer for report designers and publishers. These publishers can work with the full range of Power BI features while the report is being built, and while they are themselves using it. When it comes time to share the report to a wider audience, they can publish it to Premium capacity where anyone can access it. If the organization has not purchased Premium, then the original model applies, and all recipients will still require a Pro license.

On the Power BI Embedded side, switching to Premium capacity completely eliminates the unpredictability of the current model. The fact that the reports will be rendered from the core Power BI service means that it will be fully on par with other Power BI reports, and developers will be able to take advantage of the full spectrum of Power BI features as they appear in the service.

What’s not so good

If you are a large company, there is very little not to like with this new model. It was large organizations that felt most of the pain with the original model, and it is they that benefit most from the Premium model. In fact, in my opinion, they are the only ones that benefit from the Premium model. Well, they and organizations that have no sharing requirements. The issue here is cost.

The Premium pricing estimator can be found online, but at present, it boils down to this. The smallest block of capacity that can be purchased by an organization is “P1”. To publish content to Premium capacity, you must also have a Power BI Pro license. Therefore, the minimum cost of entry is $4,995 (P1) plus $9.99 (Pro) for a total of $5,004.99 per month. This is well out of the reach of most small to medium sized organizations. In fact, an organization needs to be larger that 500 users (and those would be active Power BI users) for Premium to begin to make sense from a licensing perspective. The model size limit removal and the increased refresh frequency are also compelling reasons to move to Premium, but it’s easy to see that Premium is only for larger organizations.

Compounding this issue for small to medium sized organizations is the fact that as of June 30, dashboard sharing has been removed from the free SKU of the original pricing model. Any dashboards that had previously been shared broadly to free users will cease to function as of the cut-off date. If Premium does not make sense for these organizations, then they do have the option of purchasing Pro for the consumer. To help ease this transition, Microsoft is offering a year’s worth of Power BI Pro to all active free users that signed up prior to May 3, 2017.

However, dashboards can be shared with external users, and it’s a pretty tall order to expect an external user to subscribe to Pro just to be able to read your report.

With Power BI Embedded switching to the Premium model, the ISV now needs to buy Premium capacity. Given that the entry price for Premium is so high, it is (in my opinion) out of reach of most of the services that would rely on it, not to mention those developers that simply want to get up to speed on it or do some testing. There has recently been some indications on the forums that the barrier to entry won’t be as high for developers, but even a figure as low at $600/month may still be too high for many to swallow.

Conclusions

Overall, I think that the Premium pricing model solves a problem that desperately needed to be solved. This approach opens the door to Power BI truly democratizing Business Intelligence and becoming almost as ubiquitous as Excel. The opening up of features to the free SKU and focusing the Pro SKU on sharing means less confusion for report designers.

Unfortunately, for the moment price stands in the way of that goal of many small-medium sized businesses. These businesses may be small in stature, but they are many in number. The removal of sharing from the free SKU actually represents a step backward for them. The floodgates have been opened for large businesses, but the stream has been dammed for smaller ones.

Fortunately, pricing is a simple problem to solve. My hope is that the entry point for Premium comes down to something that would make sense for even a 10-person company, and that the cost for developers using Embedded could scale with far more elasticity, starting at $0 to encourage investment. These changes would, in my opinion, truly set the stage for Power BI dominance.

The Difference Between Reporting and Analytics is 42

In his novel “The Hitchhiker’s Guide to the Galaxy”, Douglas Adams envisioned a giant supercomputer named “Deep Thought” that was built to solve the answer to the ultimate question of life, the universe and everything. For the 5 people out there that are unfamiliar with the story, I’ll relate the important bits here. Deep Thought was commissioned by a race of pan-dimensional beings and required seven and a half million years to complete its calculations. When it was finally complete, Deep Thought informed the ancestors of the original creators that the answer was 42. The receivers were understandably disappointed with this response, and when they questioned Deep Thought further, the computer postulated that perhaps the problem was that they never really knew what the question was.

Undeterred, the race then commissioned a second computer (which happened to be the Earth) that would calculate the ultimate question. After a couple of 10 million year attempts, the ultimate question was determined to be “What do you get when you multiply six by nine”. Of course, Adams never claimed that the universe made sense.

To my mind, this is an excellent demonstration of the difference between reporting and analytics. The accurate answer (report) provided a result, but not meaning. Further analytics were necessary to determine context.

Like many information technology terms (Big Data, machine learning, CRM) Business Intelligence (BI) is one of those umbrella terms that many people use regularly without fully understanding its meaning. BI is comprised of many tools that help to glean information and insights from raw data. Thus, an ETL package that moves data from one location to another is just as much a BI tool as is a fancy looking infographic. Combine this lack of clarity with the overloading of the term “reporting, and we wind up with some real confusion in this space.

Reporting is the process of using data to highlight things or trends that have already happened. This can be contrasted with monitoring, which does the same for things that are happening now, and predictive analytics, which tries to predict what will happen in the future based on the same data. The difference between reporting and monitoring is only one of data latency, and as such, monitoring is often referred to as real time reporting, which further muddies the water. However, for the purposes of this article, I want to focus on historical reporting.

Reports are typically one of two types, either operational or analytical. Tools that are good at producing one type are typically not so good at producing the other. What’s the difference? Operational reports are designed to provide information that we know we need, and analytical reports are designed to help us discover things that we didn’t know, or to help answer unanticipated questions. Operational reports are typically designed to be printed. They are typically well paginated, pixel perfect, and provide a single view of the data within any given report. Analytical reports are just the opposite. They are designed with visuals as a starting point, but allow for the ability to pivot on or drill down into the data as appropriate to answer ad-hoc questions. Printing is typically a weakness for analytical reports, whereas drilldown is a weakness for operational reports.

Both report types have their place but they both have very different design point. The data that backs an operational report should ideally be relatively flat, as that best reflects the report layout and helps with performance. Conversely, cubes and data models exist simply because a flat data structure does not adequately support analytical reporting. With analytical reporting, a user may at any point decide to view quantitative data (a measure) through the lens of a different facet (dimension). This difference is so great, that we need a different type of engine to support it. OLAP cubes and tabular models are both examples of this.

Another difference is the data that is necessary to support both report types. Operational reports tend to concern themselves with various levels of subtotals per the predefined facets. In a case like that, the data mart that backs the report only needs to store those subtotals. The granularity, or resolution of the data stored in the data mart does not need to exceed that of report that references it. Analytical reporting is different. Since users will be expected to drill down on data, from on dimension to another, or to filter the data according to increasingly granular facets, it is critical to store all of the data in the data mart backing the data model. We don’t know the level of resolution the analyst will need; therefore, all detail is required.

As a simple example of this, consider the case where we want to analyze some server log data over a period of time. We can pre-aggregate the data in the data model such that it stores the total of the log entries of various entries on a daily basis. There would need to be a total based on each dimension, but the overall data storage would be less than for the raw data. Such data would allow an analyst to spot trends over several days, but the decrease in resolution means that it will be impossible to spot any usage trends within a given day. If daily trends will never be necessary, then this doesn’t matter, but the nature of analytical reports means that the designer can never be sure.

The more that the source data for the report is pre-aggregated, the less that report becomes analytical in nature, and the more it approaches operational. This is regardless of the tool used; you can build either report type with any tool, it’s just that it may not be optimal.

The issue here is one of semantics. Semantics however are important in knowing what you are getting if reports are being provided to you. Calling something “Analytics” does not make it so. If you spin up a content pack in Power BI, and find that the underlying data model provides just enough dimensions and measure to construct the provided report, and that you can’t deconstruct the data in any meaningful way, what you have is a report, not analytics, no matter what the platform. As with anything, there is a trade-off between complexity and power. Given the nuances of this topic, it’s important to look under the hood to know what you are getting.

The answer “42” is perfectly acceptable if you already knew that the question was “what is 6×9?”. But if you want to know why, that takes a little more digging. You’d also know that there might be a data problem…

Completing the Microsoft Reporting Roadmap

In the recent announcement outlining the SharePoint integration strategy on the SQL Server Reporting Services Team’s blog, there was a statement that was almost hidden that I think deserves more attention. The statement was:

“….in time, we aim to support web-based viewing of Excel workbooks in Native mode…”

This may not sound like a big deal – after all, we’ve been able to serve up Excel workbooks in a browser since Excel Services was initially introduced with SharePoint 2007. However, as per Microsoft’s Reporting Roadmap from October 2015, Reporting Services is their on-premises solution for BI report delivery. If an Excel workbook is to be considered a report, the SSRS absolutely should be able to do it. The roadmap defined four types of reports:

  • Paginated
  • Interactive
  • Analytical
  • Mobile

I tend to see there being two types of reports, Structured and Analytical. In the list above, Structured corresponds with Paginated, and the other 3 types are different subtypes of Analytical. They four categories do, however line up well with the different reporting technologies available.

Report Type File Type
Paginated RDL (Classic SSRS)
Interactive PBIX (Power BI)
Analytical XLSX (Excel)
Mobile RSMOBILE (SSRS Mobile aka Datazen)

The roadmap was primarily concerned with the future of SSRS, but SSRS is Microsoft’s stated report delivery platform for on-premises reporting. The platform for cloud reporting is of course Power BI. There is a third platform for the delivery of “Analytical”, or Excel based reports, and that’s Excel Online. On premises, it’s called Office Online Server, but it is the same technology. The three platforms and their capabilities are shown below.

SSRS

Excel Online/OOS

Power BI

Paginated

Yes

N/A

No

Interactive

Preview

N/A

Yes

Analytical

Announced

Yes

Yes

Mobile

yes

N/A

Yes

The technical preview of Power BI reports in Reporting Services is available for testing, which covers Interactive reports in SSRS, and the above statement indicates that there is a solution to support Analytic reports in SSRS as well. The Power BI platform does this already, and it is done by leveraging the capabilities of Excel Online. Given the fact that Office Online server provides the same capabilities on premises, it makes sense that it would be used by SSRS when the Excel workbook support is added.

It should also be noted that the above comparison shows Mobile reports being supported by Power BI. To be clear, Power BI does not support RSMOBILE files, but regular Power BI reports are inherently mobile and available through the Power BI mobile client. which is also how RSMOBILE reports are delivered to end users.

The Reporting Services team is clearly very close to completing the vision laid out over a year ago, in the Reporting Roadmap for on-premises users. If the goal is to have parity between on-premises and cloud platforms, the only thing remaining (apart from possible support of the RSMOBILE format) is support for Paginated reports in Power BI. There have been no statements made regarding this capability, but its absence is certainly notable.

The Future of Report Integration with SharePoint

Yesterday, Microsoft made official what many, including myself had been suspecting ever since the release of SQL Server 2016 – that SQL Server Reporting Services Integrated mode would not exist in the future. With the announcement, we now know the timeline of when that will happen. SSRS Integrated Mode will not be included with the next version of SQL Server. Instead, SSRS Native mode will be more tightly integrated with SharePoint for those organizations that use both products. As someone that has always approached Business Intelligence from the SharePoint point of view, I see this is a good thing.

This change is another step in the process of de-cluttering and uncomplicating SharePoint. This process started arguably with the move away from fully trusted code running on SharePoint, to the newer app, add-in and now SPFx development models that run with SharePoint. When Excel Services was removed from SharePoint in SharePoint 2016, with its capabilities moved to Office Online Server this process became obvious. A decreased dependency on SharePoint allows for simpler, more streamlined architectures, better options for upgrade management, and better, more targeted performance management.

SSRS SharePoint Integrated mode has been with us in various forms since it was first introduced in SQL Server 2005 SP2. The original goal was to simplify the integration of the two products, and to take advantage of SharePoint’s storage and authorization capabilities. The integration has always worked well, but the very fact that these two products were delivered by two different product teams on different media, often on different release schedules has typically led to a great deal of confusion. The SharePoint prerequisite for Integrated Mode leads to far too many SQL servers having SharePoint installed on them.

Managing SSRS in SharePoint Integrated mode requires a combined skill set to some degree as well, with knowledge of both SSRS and SharePoint. SharePoint administrators tend to be intimidated by SSRS, and SharePoint simply mystifies SQL DBAs.

The fact that the two different modes did not always maintain feature parity is another problem. PowerView and several other features are only available through SharePoint integrated mode. This results in entire SharePoint farms being created for the sole purpose of providing reporting features. Since those performing these installations are typically not familiar with SharePoint best practices, these farms tend to be unreliable. The latest release of SSRS 2016 contains a massive number of new features, but many of them in Native mode only, leaving the SharePoint integrated folks with a decision whether to favour features or integration.

A strategy that reduces the dependency of one platform on the other is therefore to everyone’s advantage.

The two operating modes also represent two different code streams for Microsoft to maintain. Given the finite set of resources that is any development team, resources must be spent on maintaining both of these streams that could otherwise be applied to features. A single code stream is simply more efficient.

Preventing the wholesale move to Native mode are several SharePoint integration features that have been employed over the years that are only available in SharePoint Integrated mode.

There has been a SharePoint Report viewer web part for SSRS Native mode since SharePoint 2003. The trouble is that while it does work, it is deprecated, and hasn’t been updated since SQL Server 2008 R2. It also doesn’t allow for parameter binding, or interface control. For all intents and purposes, it is an iFrame embed of a report. The web part that is available through integrated mode allows for parameter interactivity, and significant control of the look and feel. It has been widely deployed. Integrated mode also allows for the reporting of SharePoint list data, and the ability to publish reports to a SharePoint library on a schedule. These features are well utilized in the market today.

Power View reports (RDLX) built on top of SSAS tabular models, or Excel Power Pivot models also require Integrated mode. Compounding this is the fact that Power View requires Silverlight, which does not work in either the Chrome nor the Microsoft Edge browsers.

These integration features will need to be added to Native mode before it will be possible to fully abandon Integrated mode. The good news is that the announcement commits to doing just that. Report embed, Report viewer web part, and SharePoint library destination capabilities will all be added. For Power View reports, a conversion tool will be provided to convert from RDLX into Power BI Desktop (PBIX). A technical preview is already availably that demonstrated PBIX rendering in SSRS.

This announcement signals the end of SSRS SharePoint Integrated mode, but it does not spell the end of SSRS SharePoint integration. The single mode architecture should be more approachable, simpler, and more efficient. It’s a win all the way around.

SharePoint and Power BI – Better Together


Ever since 2007, SharePoint has included Business Intelligence amongst its core workloads. There have been a variety of approaches to the workload over the years, but today those core workloads include Excel Services/Excel Online, PerformancePoint, SQL Server Reporting Services Integrated Mode, and Power Pivot for SharePoint.

Power Pivot for SharePoint and Excel Services go hand in hand and can really be considered as one of the main pillars, leaving us with three. If we quickly examine these three pillars in SharePoint 2016, it’s pretty easy to spot an emerging trend. Excel Services is gone from SharePoint 2016, its capabilities being added to Excel Online. Excel Online connects to, but does not run on SharePoint. PerformancePoint still exists in 2016, but it has received precisely 0 new features – it is identical to the version in SharePoint 2013, and remains a part of product for legacy reasons. For all intents and purposes, I consider PerformancePoint to be deprecated. SSRS Integrated mode has been greatly improved in 2016, but contains nowhere near the improvements that the Native Mode version of SSRS has in 2016.

At the same time, the past year has witnessed the spectacular rise of Power BI. Power BI is clearly the focus area for Business Intelligence within Microsoft for cloud based BI delivery. Last fall the SQL team announced that on-premises customers were not being ignored, and that SSRS was the platform for on premises BI delivery They also sketched out a roadmap that showed both platforms being able to deliver the same type of reports. In June 2016, the team delivered on a portion of this vision with SQL Server 2016 Reporting Service.

So where does this leave SharePoint in the Business Intelligence ecosystem?

In my opinion, it leaves it right where it should be – as an integrating platform, and NOT as a runtime platform as it has been in the past. SharePoint provides in context BI by connecting content to reports, and providing dashboards connected to multiple sources. In 2016, SharePoint connects to Excel Online to deliver Analytical reports. Excel runs with SharePoint now, not on it. SSRS Integrated mode still runs on SharePoint, but the investments in Native mode are a clear indication to me that this will be the direction here as well. Unfortunately, Sharepoint has been lacking tight integration with Power BI.

The recent Ignite 2016 conference was the first public appearance of the Power BI web part.

Figure 1: Insert web part dialog with Power BI web part

The Power BI web part works with Modern Sharepoint pages and is based on the new SharePoint Framework (SPFx), which means that it is completely client-side. Why does this matter to us? The fact that it is completely client side means that it will work both in SharePoint Online and on premises. Initially, it will only work with SharePoint Online, but that is because the SharePoint Framework is currently unavailable on premises.

To use the new web part, first create or edit a Modern SharePoint page. The Modern pages support the new Modern web parts. Click on a “+” icon to open the insert part control (Figure 1). Once inserted, add the report URL, and the page. The report page should immediately render within the context of the SharePoint page.

Figure 2: Power BI Report page rendered within a SharePoint page

Since the web part is rendering client side, the consuming user obviously needs to have access to the report. This means that the source report must have been shared with them through Power BI dashboard sharing, or the report is in a group within which the consuming user is a member. This latter case makes the most sense given that all Office 365 Groups will have a corresponding Modern Team site. Embedding the report within group pages should “just work”.

The devil is of course in the details, and all of these details are not yet available, but Given the number of questions that I have received over the past year about Sharepoint/Power BI integration, I expect that its existence will come as welcome news. Over time I would expect to see it picking up support for parameters and the ability to work with individual report items (this is speculation, but it makes sense). It’s also not much of a stretch to see how SSRS could make available a Modern web part that worked in the same fashion with on premises SSRSs. That web part could conceivably work both on premises an Online, bringing SSRS to SharePoint Online for the first time.

SharePoint is still very much a platform for integration and for Business Intelligence content delivery. SSRS and Power BI will be the de facto reporting engines for on-premises and the cloud respectively, and Sharepoint will be the dashboarding/integrating platform for both environments.