Power BI as a Product Today

Recently, I have come across several situations where people are confused about where Power BI fits in a solution scenario. There is a fair bit of confusion as to precisely what the product is and what it does. The problem is that Power BI isn’t really a product at all, but instead a collection of different products and services. Adding to the confusion is the fact that some of these products require a Power BI license, while others do not. In fact some of these products are actually embedded in other products.

Power BI is Microsoft’s cloud based Business Intelligence solution billed as “Self service analytics for all of your data”. In reality, it’s a little more than self service, it also is a great solution for team BI as it’s based on Office 365. That’s all well and good, but what is it really? What does it consist of, and how does it work? If you look at the main product site for Power BI, it’s not immediately obvious at what you get when you purchase it, or what you need to run it. This post is an attempt to demystify the product.

To start, let’s break it down by its constituent components. Today Power BI consists of the following parts.

image

Unfortunately, this can be rather confusing from a product perspective. Looking first at the on-premises components, Power Query, Power View, and Power Map are all Excel plug ins. Excel is therefore a prerequisite for Power BI. All of these add ins also require (or in the case of Power Query, support) the embedded xVelocity data model, and therefore Power Pivot is a prerequisite. Power Pivot is included in Excel 2013 (Professional Plus), but it can also be downloaded for free for Excel 2010.

Also included in Excel 2013 is Power View, and, with Office 2013 SP1, Power Map. Power Query is downloaded separately, but is free. This is where much of the confusion arises. Due to the fact that these three add ins are included in the product definition of Power BI, it is often assumed that a Power BI license is required to use them. It is not. These products have a life of their own, and can be fully (or almost fully) used within Excel without any association with a Power BI license.

Power Query contains a few features that will only work with a Power BI tenant, mostly involved around the creation and maintenance of shared queries. Since this is part of the cloud service, this makes complete sense, but none of the other features of the product are in any way reduced in the absence of a license. Power View is enhanced through a Power BI license, but only because this makes Power View reports available within the mobile client(s). Indeed, Power Map has no use whatsoever of a Power BI license. Power Maps cannot be viewed at all within a browser – they are a client side feature only. In my opinion, they shouldn’t even be included under the Power BI umbrella, but that’s just my opinion.

Thus far, I have been talking about the modelling and visualization creation aspects of the tools, but what about pure consumption clients? The whole idea of power BI is that designers can create these models and users can interact with them. The workbooks containing these models are stored within Office 365, so do casual users need a license?

The answer is of course maybe. If these users are going to take advantage of any of the services specifically offered by Power BI, then the answer is yes. For example, any user can open a workbook in a browser in Office 365. However, if they want to interact with that model, by using a slicer, pivot table, etc, and that model is larger than 10 MB, then the answer is yes. Obviously, if the user wants to use the Power Q&A features, then the answer is also yes.

For the record, I don’t like this answer. To my mind, designers and content creators should require a license, but consumers should not. This would greatly encourage adoption of the product, so I do hope for some changes in this area.

So, precisely what do you get when you purchase a Power BI license? These are the things that you will absolutely need a Power BI license for.

  • Opening workbooks in a browser with models larger than 30 MB on Office 365
  • Interacting with (slicers, pivot tables, etc) workbooks in a browser with models larger that 10 MB on Office 365
  • Automatic refresh of on premises data
  • Sharing of Power Query queries
  • Refresh of Power Query queries
  • Power Q&A – Natural language queries
  • Power BI mobile application

and that’s it.

In fact, if you check out my earlier article “Whither Power Pivot for SharePoint”, you’ll see that many of the features of Power BI are already available in Power Pivot for SharePoint.

To my mind, the product “Power BI” should not include the Excel add ins, but only list them as a requirement, much like Excel itself is a requirement. This would help to reduce confusion. The next version of Power will support their inclusion. If you’re interested in this new version, you can sign up for the preview when it’s ready here. I’ll be writing more about that shortly.

How to Use a Wildcard Certificate With Office Web Apps 2013

 

There is a lot of guidance out there that indicates that it isn’t possible to use wildcard certificates with Office Web Apps 2013. Much of this stems from one of the certificate requirements listed in this Technet document:

“The FQDN in the SAN field can’t begin with an asterisk (*).”

On first blush, this would in fact rule out wildcard certificated since their SAN (Subject Alt Name) or friendly name is the FQDN of the wildcard (*.domain.com). Indeed, there is no way in the IIS manager to alter the name of the certificate . Issuing a PowerShell command the create a new Office Web Apps server farm with the wildcard name results in the certificate being invisible to the command, and you receive the following error:

Office Web Apps was unable to find the specified certificate

However, in my experience, wildcard certificates work just fine with Office web apps. There is just a trick to getting them to work.

The problem isn’t the wildcard certificate per se, it’s the fact that the friendly name can’t contain a wildcard. All that we really need to do is to change it.  It’s not immediately obvious how that’s done, but it can be done through the MMC snap in.

If the certificate is already on the server, then great, but if not, you’ll need to import it. In this example, we’re importing a certificate that’s been exported from another server, a common enough scenario for wildcard certificates. However, the origin doesn’t matter.

The first thing you’ll need to do is to run MMC (Microsoft Management Console). To do this, from a command prompt, type MMC and hit enter. Then load the Certificates snap in, selecting the Local Computer store when prompted.

The location of the certificate is important. If it’s not in the right place, the new farm command won’t see it, and you’ll receive exactly the same error as above. The certificate needs to be imported into the Personal folder of Local Computer. Right click on the Certificates folder, hover over All Tasks, and select Import. Then, go through the prompts and select the certificate to import.

image

Once imported, we want to change its Friendly Name. We can do that by selecting the certificate, right clicking it, and selecting Properties.

image

From here, it’s a simple matter of changing the friendly name from the wildcard address, to something with significantly fewer asterisks, in our case xxxxxxwildcard

SNAGHTML37b8afc

Once done, click OK, and close the MMC. Your certificate now has a friendly name that you can use to create your Office Web Apps farm.

New-OfficeWebAppsFarm -InternalUrl "https://xxx.xxxxxxxx.com" -ExternalUrl "https://xxx.xxxxxxxx.com" -CertificateName "xxxxxxWildcard" –EditingEnabled

Once created, you can continue to configure your Office Web Apps farm, and then bind SharePoint to it.

While I’m on the topic, and because it comes up frequently, once you have bound your SharePoint farm to the Office Web Apps server, it’s important to turn off view rendering for Excel files if your farm uses Excel Services. This is because the Office web apps don’t support data connections in Excel files, or PowerPivot models, and data interactivity won’t work. By issuing the following command:

New-SPWOPISuppressionSetting -extension xlsx -action view

we tell the SharePoint farm to use Excel Services (which supports data connections) when viewing xlsx files, not Office Web Apps.

Finally, if this is being added to an existing farm, you’ll want to run a full crawl to repopulate your search index with the new rendering mechanism for Office documents.

Using Power Query with SharePoint Lists and Lookup Fields

As I’ve explained many times before, querying SharePoint data directly is a bad idea. The SharePoint data storage mechanisms simply aren’t designed for querying of any scale, hence the lookup limitations that have been imposed upon it. The best approach to querying SharePoint list data is to first load it into a data warehouse or data mart of some sort. However, both Reporting Services (SSRS) and Power Query support direct access to SharePoint lists. While I try to strongly dissuade people from doing this with Reporting Services, properly used, Power Query is a totally viable means of querying SharePoint list data.

Why is this? With SSRS, every query goes back to the data source for retrieval.  Power Query is different – it’s analogous to SQL Server Integration Services, which is an ETL management product. It loads source data into a repository, in this case, an embedded xVelocity, or Power Pivot model which can be considered a “personal data warehouse”. Queries against this mini data warehouse are fast, and don’t rely on SharePoint  retrieval mechanisms, and can be used quite effectively in reports.

There are a couple of subtleties to querying SharePoint list items with Power Query, and I will briefly walk through the process below.

With Excel open, click the Power Query tab, select “From Other Sources” and the select “From SharePoint List”.

image

Next, enter the URL for the SharePoint site (or subsite) that contains the list you wish to query.

image

If it is the first time accessing this site, you will be prompted for credentials. If your site is Office365, be sure to enter organizational credentials. If it is on premise, use Windows credentials.

Once entered, you will be presented with a list of SharePoint lists in the Power Query Navigator window. Select the list that you wish to query, in our case, Announcements. When selected, click the edit button to edit the query.

image

The data, or a subset will load into the query editor window. You will see all of the list item fields expressed as columns, and for the most part, using the correct data type. At this point you can remove any columns that are unnecessary, or filter any undesired rows. There are a couple of SharePoint field types that bear special mention.

Lookup fields are a lookup into another SharePoint list. Internally, the SharePoint item stores this as an ID and display value, but Power Query gives you access to all of the properties of the related item as a one-to-one relationship. Essentially, what you can do is to flatten that relationship by incorporating the related item’s attributes.

If you scroll to a column of this particular type, you will see the value expressed as a hyperlink with the value “Record”. Clicking on it will drill down to one related record, but that’s not what we want to do. We want to expand the properties for all items in the list. The way that you do this is  to click on the expand icon in the column header. In our case, we want to expand the “CreatedBy” field. CreatedBy is a standard list field, of the Person type. Person fields are actually a special case of a lookup field, so it exhibits this behaviour.

image

Here, we are interested in retrieving the user’s name and mobile phone, so we deselect all of the other fields. A new column will be created for every expanded field in the format sourcefieldname.attributename .

image

Attachments are another special case. There can be multiple attachments for a single list item, a one to many relationship. The hyperlink is therefore “Table”. Clicking on the column header expand for this column looks similar, but with an important difference. Options are available to either expand or aggregate the related items.

image

Selecting expand will create a new source record for each related item, and the only columns that will differ will be the items selected from the related table (Name in our case). Aggregate will not create any new records, but will summarize the related fields. For numeric fields, they can be totalled or averaged, and for text fields they can be counted.

Once ready, click “Close and Load” from the Query Editor ribbon, and the list data will load to either your model, or your workbook, depending on what your preferences are. Of course, I always recommend that you load to the model only.

Once loaded, any visualizations and queries will work against the model. The data can be refreshed at any point either manually, or automatically if using the Data Management Gateway. Keep in mind however that refreshes will operate against the source list.

Reporting Services Web Part Won’t Save Parameters with SharePoint 2013 SP1

We recently performed an installation of SharePoint 2013 with SP1 for a customer using the slipstreamed ISO available on MSDN. Part of the rollout in this case involved migrating a number of reports from a standalone instance of SQL Server Reporting Services (SSRS) to SSRS running in SharePoint integrated mode. However, during the creation of some new reports, we observed something odd. The custom parameter values were not being saved for web parts that were using these reports.

SSRS web parts in SharePoint allow for a report to be displayed within a web part, a therefore in part of the page. Report parameters created in the report can be exposed as web part properties. Therefore, you can have multiple instances of the same report on different pages using different parameters. You can also connect the web part to other web parts, such as SharePoint filters, and have the parameters driven by these filter values. However, in this particular case, none of it was working. The parameter could be saved, and applied, but when the page was saved, the parameter values would revert to their defaults.

After doing some digging, I came across this discussion on the MSDN forums. The discussion indicated that there may be a regression bug in SharePoint 2013 Service Pack 1 that exhibited this behaviour, and that the application of the May 2014 Cumulative Update had helped them somewhat. We had never seen this issue with builds prior to SP1. Given that our farm had not yet gone live, we quickly acquired and applied the July 2014 CU in the hopes that it would help.

It did.

In our case, it fixed the problem completely, however there was one small negative side effect. All of the web parts that had been added to pages before the update completely lost all of their saved properties, including the report that they were pointing to. In our case, this was no big deal, but if you have a lot of these, it could be an annoyance. I also don’t know if this would happen generally with the July 2014 CU, or it was only because that the original system was affected (as I suspect).

In any event, this apparently was a regression bug introduced with SP1, and applying the July 2014 CU clearly fixed this problem for us. Our new baseline for SharePoint 2013 will not be sP1, but SP1 with the July 2014 CU.

Power BI Data Management Gateway 1.2 Changes the Game

Last week, a new version of the Power BI Data Management Gateway was released. If you’re unfamiliar with it, it is the Power BI component that allows for workbooks stored in the cloud to be refreshed on  a regular basis with data that exists on-premises, or outside of the hosting center.

I’ve been using the gateway since its initial availability in preview form, and in my opinion, this is the most significant functionality change yet. Until this release there were a grand total of three possible data source types that could be refreshed. With this release, the total increases to 18 by my count (you could argue 22, but that’s plenty).

With past versions, I would write up a quick post on how it is configured, but that has been done, along with the complete list of supported data sources and a helpful video on this blog post by the Power BI team. In addition, an very comprehensive (although amazingly already in need of update) white paper on hybrid data scenarios has just been published by Microsoft here.

The big change here is that this multitude of data sources is supported for data retrieved by Power Query, and NOT by Power Pivot natively. The catch is that they’re only supported for Power Query queries. There is absolutely nothing wrong with this, but it does require us to change our approach a bit to using the data management gateway.

As I first mentioned in a post almost a year ago on Using the Data Management Gateway, and in a number of posts since, data connection strings needed to line up with Power Pivot connections. At the time, the only supported sources were SQL Server and Oracle (for the gateway) and Power Query wasn’t supported at all. Version 1.1 of the Gateway brought Power Query support, but only for those 2 supported data sources. With this release, the Power Query support includes not only all of the new data sources, but also the three original Power Pivot data sources (note: Power Pivot data connections can be found in the Power Pivot add-in UI, while Power Query connections are available on the data tab in Excel).

image

image

As of this release of the Data Management Gateway, there is almost no reason to use native Power Pivot connections any longer. My recommendation is therefore that unless there is a good reason for not doing so, you should try to use Power Query for all data acquisition tasks. It is quite clearly the way forward, and will only gain in supported capabilities. My suspicion is that Power Pivot connections will be retained for backward compatibility reasons only.

With that said, there are a couple of good reasons for using Power Pivot connections directly. One of these reasons is if your data source is online, whether it is SQL Azure, SharePoint Online, or Project Server online. With these data sources, a Data Management Gateway is not required for refresh to work from an embedded Power Pivot connection. However, if Power Query is used to access these sources, it is.

What this means is that for Power Pivot connections to these sources, a refresh allows the Power BI service in the cloud to directly access these data sources in the cloud. However, because ALL Power Query connections require the Data Management Gateway for refresh operations, a Power Query refresh operation will require all of the data to be first downloaded to the on-premises gateway, and then sent back up to the Power BI service in the cloud. While functional, this is hardly the most efficient approach.

Apart from this one small caveat, this version of the Data Management Gateway spells the way forward. Additional data source support should come fast a furious, and the Power Query focus means that we can start to rely its powerful transformational capabilities without having to sacrifice refreshability (if that’s even a word….).