Microsoft Lists debuted in 2020 and they are a (yet another) great way to organize list-based data. Now, when someone says data, I think Power BI. Obviously, we’ll want to report on this data, but how do we do that? There is no Power BI connector for Microsoft Lists. The answer is quite simple, if not completely obvious. You need to use the SharePoint Online List connector in Power BI.
Microsoft Lists are the same thing as SharePoint lists. In fact, they ARE SharePoint lists. The Microsoft Lists service is just a new user interface for interacting with them. You can use Lists all you want and never see SharePoint at all, unless you need to change list settings, but I digress. Given this fact, as far as Power BI is concerned, everything that applies to SharePoint lists applies to Microsoft Lists lists (the grammar here gets awfully awkward).
For reference, I wrote a series of articles some time ago about the idiosyncrasies of working with SharePoint data in Power BI, and these articles are still valid today (2021). The most recent of these articles can be found here and includes links to the others.
There is one thing that is worth mentioning about Microsoft Lists. When a new list is created using the Lists interface, the user can save it to any of their SharePoint sites, but another option is to same it to “My lists”.
When you use the SharePoint Online list connector in SharePoint, it prompts you to enter the URL for the SharePoint site that contains the list that you want to report on. That is straightforward when your list is stored in a SharePoint site, but what if your list is stored in “My lists”? Where are “My lists” stored?
They are stored in a “personal” SharePoint site. We SharePoint old timers would know it as the MySite, and while usage of MySite has been de-emphasized in Microsoft 365, it is very much still there. Each user has one. In fact, this is where the “personal” OneDrive for Business content is stored – in the Documents library of the very same MySite. By storing personal lists in the MySite, Microsoft Lists is just following the same pattern used by OneDrive for Business, which makes perfect sense.
Given this, what URL should you use in Power BI to connect to your lists stored in “My Lists”? You’ll find it in the Microsoft Lists web interface in the URL bar. It’s that portion of the URL up to “/Lists/.
Power BI licensing has changed again. This week at Microsoft Ignite, Microsoft introduced a new capacity based SKU for Power BI Embedded, intended for ISVs and developers: The A SKU. This brings the number of capacity based SKUs to 3, with each category having numerous sub categories. This means that there are a number of ways to embed content by using Power BI Pro, Power BI Embedded, or Power BI Premium. The trick is to know what will be needed for what circumstances. This post will attempt to help with the distinctions.
The SKUs are additive in nature, with A (Power BI Embedded) providing a set of APIs for developers, EM (Power BI Premium) additional ad-hoc embedding features for organizations, and P (Power BI Premium)providing a SaaS application that contains everything that the Power BI service offers. For some background, the EM SKU was initially introduced to serve the needs of both Independent Software Vendors (ISVs) and of organizations that needed to do some simple sharing within the organization, and give them access to the latest Power BI features. However, ISVs have a different business model than enterprises, which is why the A series was introduced.
Power BI Embedded A SKUs
The A SKU (A is for Azure) is a Platform-as-a-Service and set of APIs for those ISVs who are developing an application to take to market. These ISVs choose to use Power BI as the data visualization layer of that application to add value to their own application. As such, Power BI assets contained in Power BI Embedded capacities cannot be accessed by a licensed Power BI user, but are only accessed by customers of the ISV’s application.
Power BI Embedded capacity is billed hourly, can be purchased hourly, and can be paused – meaning no long-term commitments to a specific capacity. This pausing capability is critical for small ISVs that don’t yet have the revenue stream to support monthly commitments, and it addressed one of my largest concerns over moving Power BI Embedded over to the Premium model. Power BI Embedded is purchased through Microsoft Azure. Additionally, Power BI Embedded can scale up and down as needed to accommodate the requirements of the ISV business model as the vendor’s application grows.
Running the entry level A1 capacity for a full month equates to approximately $750/month, so while the capacity of the Power BI Embedded A1 SKU is equivalent to the Power BI Premium EM SKU, ISVs pay a slightly higher effective monthly price for the flexibility mentioned above.
There are 6 sizes of Power BI Embedded available, each capacity mapping to an existing Power BI Premium capacity so ISVs can grow their business as needed. Pricing starts at about $1/hour:
Power BI Premium EM SKUs
The EM SKU (EM is for embedding – NOT Embedded) covers off everything contained in the Power BI Embedded A SKU, but also offers the ability to share Power BI reports within an organization through content embedding. Currently, this can be accomplished through the use of the SharePoint Power BI web part for modern pages, or the through tabs using Microsoft Teams.
There are three EM SKUs, and while the largest, EM3, can be purchased through Office 365 monthly, the smaller 2 (EM1 and EM2) must be purchased through Volume Licensing. Volume licensing represents an annual commitment, and may be an incentive for ISVs to remain on the A SKU even if they are not pausing their service. EM SKUs cannot be paused – a month is the smallest available billing unit. Additionally, scaling on EM SKUs requires that you retain your monthly or annual commitment to the initial SKU purchased until the end of the contract term.
Details on the EM SKUs are below:
Power BI Premium P SKUs
The P SKU (P is for Premium, but it helps to think of it as “Power BI Service”) is the “all in” version of Power BI licensed through capacity. It offers everything that is available with Power BI, which includes everything available in the A and EM SKUs. It also offers the ability to share Power BI assets in the Power BI service through apps, or if personal workspaces are in a Premium capacity, through dashboard sharing.
The entry point of the P SKU is significantly higher than EM as well, but you’re getting a business application vs a set of APIs. It also comes with significantly more resources attached to it. For example, P1 comes with 8 virtual cores and 25 GB of RAM, whereas the largest EM offering is EM3, with 4 cores and 10 GB RAM.
All the P SKUs can be purchased through the Office 365 administration center, and can be billed monthly. Details are below:
What to use when
PBI Embedded A
PBI Premium EM
PBI Premium P
Embed PBI Reports in your own application
Embed PBI Reports in your own application
Embed PBI Reports in your own application
Embed PBI Reports in SaaS applications (SharePoint, Teams)
Embed PBI Reports in SaaS applications (SharePoint, Teams)
Share Power Reports, dashboards and datasets through Power BI Apps (workspaces)
Ad hoc dashboard sharing from personal workspaces
With the addition of the Power BI Embedded capacity based SKUs, many of the concerns around Premium pricing have been addressed. I would still like to see all EM SKUs available monthly, and to see a “P0” premium SKU, but it’s fairly clear as to which scenarios call out for which licenses.
An ISV that is embarking on the use of Power BI embedded will at the very least need a Power BI Pro license. When development gets to the point where sharing with a team is necessary, a Power BI Embedded A SKU can be purchased from Azure. Once 24/7 availability is required, the ISV may wish to switch to an Premium EM capacity. An ISV should never require a P SKU unless capacity demands it, or they have additional requirements.
An organization that has a few data analysts or Power Users that need to share reports with a broader audience would likely be well served with one of the EM SKUs. This scenario assumes that the organization is also using SharePoint Online, Microsoft Teams, or both. This approach will allow the power users (who will require a Pro license as well) to embed Power BI content within a SharePoint page or a Microsoft Teams tab where it can be accessed by users without a Pro license. This organization would need to include more than 63 users accessing the reports to be financially viable.
Finally, larger organizations with a significant investment in Power BI, or organizations that don’t currently utilize SharePoint Online or Microsoft Teams would benefit from a Premium P capacity. With this, the Power BI interface could be utilized by end users to access shared content without a Pro license. Given it’s monthly cost, compared to the monthly cost of Pro, the organization would need to have at least 500 active report consumers for this approach to practically considered.
Since there are two major SKU types for Power BI premium, and that there was (and is) a fair amount of confusion around this area, I also published another article attempting to clear up the confusion. While this article was based on all the information publicly available at the time, new information has pointed out that it is incorrect.
The two major SKU types are P and EM, with P standing for Premium and EM for Embedded. This matters significantly because the two SKU types have significantly different entry points and therefore costs.
The P SKU was the only one introduced when Premium was originally announced. It gives organizations the ability to place Power BI assets in a premium capacity container (a Power BI “app”), and once this is done, anyone can consume these assets whether or not they have a license.
Subsequent to this, an additional SK (EM) was introduced to address Power BI Embedded. Power BI embedded allows an ISV to use Power BI to add reports to their own applications. In this scenario, the reports run from the ISV’s tenant. Originally the assets were housed in Azure, but with the availability of Premium capacity, the decision was made to shift Power BI Embedded to use this new model. Given that the requirements of an ISV are not the same as a general organization, this new SKU was introduced. The EM SKU comes with a significantly lower entry point and cost, but also with significant restrictions. This is where the confusion sets in.
The wording around the restrictions on the EM SKU indicated that it could not be used for the SharePoint web part, and that a P SKU, or a Pro license would be required for that use case. This is where my earlier article is incorrect. I have since had conversations with the product team, and have been informed in no uncertain terms that the EM SKU CAN be used for both SharePoint web part, and Teams tab embedding of Power BI reports.
This is a very significant difference. An organization that is using Power BI casually, but has a few reports that they want to share with a broad audience was looking at a cost of almost $5,000 per month to do this. Given that the cost of a Pro license is $10/user/month, this meant that the organization needed to have at least 500 frequent report consumers before Premium was even worth considering. Also given the fact that the embedding features available in both SharePoint and Teams require that Pro or Premium SKU, this could be a real disincentive to its use. However, given that the EM SKU start at approximately $650/month for the entire organization, this becomes much more approachable, and it lowers the bar to entry significantly. This should result in significantly greater adoption of these Power BI embedding features, and consequently, of Power BI as a whole.
To be clear, there are still restrictions around the EM SKU. You cannot share Power BI apps with this SKU, but you CAN use it to embed reports in both SharePoint and Microsoft Teams.
After my recent post “You Can’t Disable Office 365 Groups”, I received feedback from a few people, specifically Elaine Van Bergen, Martina Grom and Joe Stocker that some editing controls have been added in through the tenant that allows Group creation to be disabled in the Office 365 tenant, and that these controls affect all of the user interfaces that can create groups. The procedure is outlined here, and Martina offers her insight on it here . I was a little disappointed that I had missed these newer controls earlier, but quite happy about the discussion that the original article started. It brought to light some of the confusion around this feature. In addition, it also highlights the fact that Office 365 Groups are about far more than just conversations, they are the bedrock of all Office 365 services moving forward.
Having said that, and having tested these new controls, I have a few observations to make about disabling Groups.
Much of the feedback that I received of my original article was “Good, they shouldn’t be disabled anyway, they’re too important”. To be sure the other side of that argument was heard from as well, but I tend to side with the former. For me at least, the group construct represents real value. It is a trade-off between ease of use and control to be sure, but as a container, it’s easy to understand, and relatively easy to work with for end users. The concerns around Groups are focused on governance, and those concerns are valid. If anyone can create a group anytime, and there is not process for organizing or classifying them in place, they can quickly get out of hand, producing islands of information all over.
The new management controls allow for a single security group (not an Office 365 group) to define those that can create Groups. Groups created by these members are available to all, but only these members can create new Groups. Only one security group can be flagged for group creation, so it’s an all or nothing proposition for these group members.
The article above walks through the process of creating these controls through PowerShell with the Microsoft Azure Active Directory Module for Windows. There are a couple of quirks when walking through this process. I found that the article itself contains a typo, the PowerShell command “Get-MsolCompanyInfo” should actually be “Get-MsolCompanyInformation”. In addition, when downloading the module itself, the 18.104.22.168 Preview version is required.
One would think that the GA version (22.214.171.124) would include everything necessary, but one would be wrong. I made the mistake of trying to use that version and I hit a wall. You need the preview version.
The Azure Active Directory management area in the new Azure portal also allows for the management of group creation rights. I was unable to use the interface to initially set these controls, but once set, the controls were reflected in the user interface, and it’s possible to manage them. Azure Active Directory management is still in preview in the new portal, so presumably this will improve at GA. The controls can be found in the Azure Active Directory blade under Users and Groups – Group Settings.
Like their predecessor, these controls don’t remove the option to create a group from the client interfaces. Once the “Create” option is selected, the user is usually notified that this will not be possible. In one case, it simply fails. The following are the different messages that users will receive when they try to create a new Group but are prevented from doing so.
Outlook Web Access
Ideally, the create option would simply be removed from the user interface, but at least these interfaces prevent the user from filling out details before failing with one notable exception. When a new Group Workspace is created in Power BI, the operation simply fails, and the user isn’t notified as to why. It almost seems as if the Power BI team wasn’t notified that these new controls exist.
The remaining workload that is (ok – will be) integrated with Groups is Yammer. With Yammer, when a Yammer Group is created, a corresponding Office 365 group will be created, and kept in sync with the Yammer group construct. This will ultimately be where Yammer notes and files are stored (via OneNote and OneDrive – basically SharePoint) as well as the group calendar (in Exchange). However, according to this Microsoft support article, if Office 365 Group creation is disabled, then the Yammer groups will not be Office 365 connected.
It therefore is now possible to prevent users from creating Office 365 Groups. This will be important to large organizations while they formulate an adoption strategy for Groups, but formulate it they should. Just because Groups can be disabled, it doesn’t mean that they should. Groups are by their very nature a compromise between usability and manageability, and centralizing creation tips the scales on the side of manageability. We’ve had this for a long time with classic SharePoint, and the usability of Groups is what’s so exciting from an adoption standpoint. Almost all innovations in the Office 365 space are now centered on Groups – they are the new foundational unit, and by ignoring them, you miss out on much of the future enhancements.
Caution is certainly advised, but it’s a good idea to move forward with a Groups strategy. Now.
Note – Since originally publishing this post, I have been made aware of some new management tools that will allow the ability to disable group creation by default. As opposed to modifying this post, which contains other observations, I have published a new one dealing with these new tools here.
As I’ve discussed before, Office 365 Groups are a very important feature in Office 365, and one that all organizations using Office 365 should fully understand as soon as possible. Groups are either required or they provide important capabilities for every product in the Office 365 stack. However, every organization has a different tolerance for change, and some have no tolerance for it at all. In addition, there are many aspects of Groups that are still a work in progress (navigation for example). A frequently asked question is “how do we turn off Groups?”. There’s nothing in the Office 365 Administration interface in either the Groups, or the Services & Add-ins sections.
What these approaches do is to adjust the Outlook Web Access policy that controls the creation of Office 365 Groups. At its core, an Office 365 Groups is just a type of Azure Active Directory Group, one with multiple services attached to it. When Groups were first introduced, the only way of creating them was through the Azure Active Directory interface, PowerShell and through Outlook Web Access (OWA). The first two methods require an administrative level of access, so enabling and disabling this feature in OWA effectively disabled it for end users. An end user can still see the Group creation controls, but any attempt to create a new group is met with a dialog informing them that this feature is disabled.
Since Groups were first introduced, there have been several significant changes as more Office 365 services embraced the Groups structure, and others have been introduced that rely on it.
When the “new” Power BI was introduced in mid 2015, its Sharing story relied heavily on Office 365 Groups. Each Group receives a Power BI workspace, and conversely each new Power BI workspace is a Group. Given that end users can create and to some extent manage the workspace directly in the Power BI user interface, it represents an alternate Groups management tool focused on the end user.
Creating a new Group in Power BI
Microsoft Planner, launched in mid-2016 is another product that relies on the availability of Groups. For the most part Planner stands on its own, with minimal ties to the rest of the Office 365 stack. Each Plan contains multiple tasks, but under the covers, each Plan is backed by an Office 365 Group, with all the rest of the available services. Creating a new plan in Planner creates a new Group, and everything that goes with it, even though the interface doesn’t make it very clear. You’re getting far more than just a plan.
Creating an Office 365 Groups (aka Plan) in Planner
With the release of Modern Team Sites in SharePoint, SharePoint is also very tightly bound to Office 365 Groups. Before this release, creating a new team site through the SharePoint interface or through the SharePoint administration interface created a classic SharePoint site collection. Doing so now also creates a group to go along with it (again, with everything that goes along with that) and all the access to the new Team Site (a site collection) is controlled through membership to that Group. The SharePoint interface for this makes it very clear as to what is happening – “Lets create a new team site and group”.
Creating an Office 365 Group from SharePoint
It is still possible to create a SharePoint site collection that is not bound to a group through the SharePoint administration interface. Modern team sites (the site collections created through the SharePoint user interface) don’t appear in the SharePoint administration interface at all.
The Outlook 2016 rich client also has a comprehensive set of group management features. A group can be created by right clicking on the “Groups” node in the Outlook mailbox, and once created can be fully managed by the “Home” tab in the ribbon.
Creating a new group in Outlook 2016
Managing a group in Outlook 2016
There are now 5 different way for end users to create and in some cases, manage their Office 365 groups. The original Outlook Web Access interface, and now Outlook 2016, Planner, SharePoint and Power BI. The processes outlined above for disabling group creation prevent group creation from Outlook Web Access, but what effect do they have on these new interfaces? The answer is, no effect whatsoever. Whether the “GroupCreationEnabled” OWA policy has been set to false or not, these other interfaces will still be able to create and work with Office 365 Groups. This may not be surprising as Power BI, Planner, and now even features of SharePoint are dependent on the Groups infrastructure.
I have not called out Microsoft Teams above. It is true that Teams is also dependent on the Groups infrastructure, and that creating a new team will create a new group. Where Teams differs from the other dependent services is that the creation of a new Group in one of the other interfaces does not automatically create a new Team. In addition, Teams itself must be enabled by an administrator, meaning that for this additional service, Groups creation can be controlled centrally.
In the very near future, Yammer will also become Groups dependent. Creation of a new group in Yammer will spin up a corresponding Office 365 group, which will be used to store the files and notes available in Yammer. These groups will be flagged as “Yammer managed” meaning that they will not appear in the Outlook interfaces, but they will be available to all the other services that utilize groups.
The bottom line of all this is that even if you use Office 365, and you think that you have disabled Groups in your tenant, the chances are that you could be in for a surprise. If any of these dependent services are in use, the chances are that you already have several created.
Groups are the bedrock of all new features in Office 365 moving forward – it is therefore a good idea that your organization understand them as soon as possible. Their inevitability is also another strong argument for paying close attention to them. If you are currently discussing whether or not they should be used, I would strongly encourage you to shifting that discussion to how they should best be used.