Tag: Cloud

Power BI Analyze in Excel – The beginning of a beautiful thing?

Published April 8, 2016

One of the announcements made at the Microsoft Data Summit in this past March 2016 was the availability of Analyze in Excel. This feature allows an Excel workbook to connect to a data model that is stored in the Power BI service, and to use it to analyze the data contained within. With this approach Excel is not importing data, or at least it is not importing any more data than the query results. It is exactly like connecting data to SQL Server Analysis Services data sources, something that Excel users have been doing for years. Well, to be completely accurate, it’s not LIKE connecting to SSAS, it IS connecting to SSAS. The only difference is that in this case SSAS is in the cloud. This feature significantly enhances the utility of the Power BI Service, and is important for several reasons that may not be all that obvious. I’d like to walk through a few of them, but let’s start with the obvious.

Excel is a very powerful analytical tool

As nice, and as attractive as Power BI visuals are, Excel still rules the roost when it comes to doing advanced analytics. Excel has been doing this for years and is very mature. It supports features such as pivot tables, pivot charts, and drill through to data, where Power BI reports still do not. The lack of these features can be a blocker for Power BI on its own, but if the data models in Power BI can be analyzed with Excel, suddenly a move to Power BI is not an either/or decision – you can have it both ways. You can deploy models and reports into Power BI and take advantage of all the goodness there, but you can also connect with Excel when the deep analysis is needed. With Analyze in Excel, you can have it both ways.

A wider audience for your data models

Very often, the person that builds the data model is the same person that does the analysis. This is the nature of self-service analytics. In the past when the only multidimensional analysis tools were OLAP cubes and connected Excel workbooks, cube design was a specialized skill. The cubes were published and users would use them as is. The advent of the data model (Power Pivot) and self service analytics lowered the skills bar so that analysts could acquire data, model it and analyze it, making the entire process much quicker and responsive. However, this still puts the model out of reach for those with no data modeling skills or interest.

Analyze in Excel provides the best of both worlds. Analysts can build models and reports in Power BI, and users that need more in depth analysis can connect to them with Excel without having to reinvent the wheel. This in effect provides the same capability that PowerPivot for SharePoint does on premises. One model can now reach a much wider audience of users. This has many of the benefits of an on-premises SSAS deployments without all of the organizational overhead of getting them up and running.

Uses the Analysis Services OLEDB Driver

The connection that is made from Excel to the Power BI services uses the latest version of the classic Analysis Services OLEDB driver. This is the driver that has always been used to allow Excel (and other tools) to communicate with Analysis Services, and this new version has been updated in order to work with the cloud based SSAS service. In fact, in order to use the feature, you must first download and install the updated driver. Therefore, in theory, any tool that uses this driver should be able to communicate with Power BI models as if Power BI was one great big SQL Server Analysis Services server (because it is).

It really is Analysis Services in the cloud

The Power BI service itself is backed by tabular mode SSAS. Until now, it was necessary to go through the service to access it. Analyze in Excel is the first instance that I know of that a client application communicating directly with that SSAS instance. While this connection is really using the Power BI API, it does beg the question – can a fully Platform as a Service version of Analysis Services be very far away?

Claims based authentication and Power BI API

None of the products in the SQL Server suite currently supports claims authentication. This is true even for the yet unreleased SQL Server 2016. Even SQL Azure, a cloud based version of SQL Server, requires SQL authentication only (although Azure Active Directory authenticated databases are currently in preview). However, looking at the connection string contained in the ODC file used by the Analyze in Excel feature reveals some interesting things. Here’s one connection string:

<odc:ConnectionString>Provider=MSOLAP.7;Integrated Security=ClaimsToken;Identity Provider=AAD;Data Source=https://analysis.windows.net/powerbi/api;;Initial Catalog=xxxxxxx; ……..

The value MSOLAP.7 for the provider indicates that this is the next version of the SSAS OLEDB Driver. No surprises there, but this does hint at future compatibility (see SharePoint below). The value for Integrated Security, and Identity Provider (ClaimsToken and AAD) indicate that it is leveraging Azure Active Directory Claims authentication. We therefore have a version of SSAS that can use Claims based authentication. This isn’t available to on-premises installations, but given that the capability has been built, I imagine that it is not all that far away.

Finally, the Data source indicates that the Power BI API is being used to marshal all communication with the back end API service. I think that it is reasonably to conclude that any API for a PaaS based version of SSAS would be based on, or strongly resemble the Power BI API. They may even be one and the same.

Excel Online in SharePoint

As anyone that has set up PowerPivot for SharePoint can tell you, SharePoint supports the configuration of new OLEDB drivers. This support carries forward into the Office Online Server in the world of SharePoint 2016. Given that both SharePoint and OOS utilize claims based authentication, it should theoretically possible to create a workbook that uses the Analyze in Excel feature, store it in SharePoint, and have it work for multiple users from within a browser. I imagine that more plumbing is needed at this point, but it would be an interesting way of integrating Power BI in the cloud with SharePoint both on premises and Online.

Reusing the Excel Files, and Limitations

In the same vein as discussed with SharePoint, Power BI itself allows Excel files to be interacted with in the service in exactly the same manner that Excel Online does. Theoretically, one should be able to use Analyze in Excel to build a workbook, then connect it to Power BI and have it work for interaction. While it is possible to connect it, all interactions fail at the moment. It appears that the Power BI service (or the backing Office Online service) does not yet support the new OLEDB driver.

Another current limitation of this feature is that data sources using Direct Query (this includes SSAS sources) or sources created by the Power BI API cannot be used with Analyze in Excel. At least not yet.

Analyze in Excel is another useful tool in the Power BI arsenal, but as outlined above, I think that it’s a harbinger of even greater things to come.

7 Comments

The White Pages are now Running on Windows and SQL Azure

Published January 24, 2011

Ever since I started this blog, I’ve hosted it internally on our premises. Part of the reason for this was that I wanted to have full control over what was going on with it, and I wanted to work in a familiar environment. For me, that was of course the Microsoft stack. While SharePoint has excellent blogging features, made even better by the Community Kit for SharePoint: Enhanced Blog Edition, my feeling is that its feature set is more applicable to an inside the firewall deployment. Also, if I were to use SharePoint for this purpose, I’d be constantly distracted by the desire to improve upon it.

What I needed was a platform that was focused on blogging, and that I wouldn’t wind up tinkering with too much. I settled on WordPress, which seemed to be very well supported, and quite good at what it did. WordPress had direct integration with Windows Live Writer, and had apps for the iPhone, Blackberry, Android, and now Windows Phone 7.

WordPress natively runs on PHP and MySQL, and typically runs in Linux environments. However, since IIS supports PHP and MySQL runs on Windows, it is possible to get it running in my “familiar environment”. Normally doing this sort of thing is a bear, but by using the Web Platform Installer from Microsoft, the installation was a breeze. All that was necessary was to run it, and select WordPress as a desired application. The installer then took care of downloading PHP, MySQL, WordPress, and integrating them all together. After answering a few account and password questions, I was up and running, and have been ever since.

The one drawback of this approach was that I was hosting it myself, and therefore always concerned with reliability and uptime. More importantly it has been sharing a server with other applications, and more than once has gone down because another system needed a reboot, crashed, or something. A hosted environment was obvious, and since I’ve been exploring the Azure platform lately, I thought I’d see what was involved. One of the advantages of the MVP program, which I’m newly a part of is that you are allocated a certain amount of Azure computing hours, so off I went experimenting.

Happily, one weekend later, this blog has been transitioned to a high speed, and highly available platform, that most importantly, I don’t have to maintain. Not only that, but I’ve been able to take MySQL out of the picture completely, and I’m using a SQL Azure database as my data store. I had several false starts right away, and I’m going to document the approach that I took and post it here shortly, but for now, I’m pretty happy with the results.

Hello Azure!

Integrating SharePoint On Premises With BPOS and Exchange Online: Part 1–Outbound

Published September 21, 2010

If you’ve ever set up a SharePoint Farm, you’ll know that one of the first things that you need to configure is Outgoing Email. The way that you typically do this is to specify the name of an Exchange (or any other SMTP server) in your organization, a from and reply to address, and you’re done. Occasionally there are problems because the Exchange (or other) server does not allow relaying, and that’s easily remedied with a quick server configuration.

If you’re using BPOS, you not only have Exchange online, but you have SharePoint online. Outgoing email is already automatically set up for you. Just set an email alert for yourself and your will receive an email when anything changes. However, what about organizations that have elected to use hosted Exchange or BPOS, but also still maintain an on-premises SharePoint farm?

The good news is that it can be done. The bad news is that it’s not as simple as before when everything was on premises. This article will attempt to walk through the required steps.

This example uses SharePoint 2010 but the same is true (I think) for SharePoint 2007.

1. Setup SharePoint Outgoing Mail

As before, go to Central Administration, navigate to system settings, and click the “Outgoing E-Mail Settings” link.

There are two major things to note here. Firstly,the Outbound SMTP server is NOT one of the Online Services servers. What is it? Well,it doesn’t exist yet – we’ll get to that in step 3. The reason for this is that to deliver mail to any of the Online Services servers, you need to authenticate, use SSL, and use a non standard SMTP port. Unless I’m missing something, I don’t see any of those options here. (You can find complete instructions on relaying messages to BPOS and Exchange Online here). What we therefore need to do is to set up our very own SMTP server that can relay these messages for us.

The other thing to take note of here is the “From address”. When working internally, this doesn’t normally matter, you can give it any old fake name and off it goes. This is not true here, if this email address is not valid online, mail will not be delivered. We will add this address later.

2. Update DNS

If you haven’t already done so, you’ll need to add DNS entries for the SMTP server that will use the same IP address as this SharePoint server. In this example I’m using an internal domain, but you’ll also want to add another one externally if you’ll be configuring incoming email. all that will matter is that it resolve to the same machine.

3. Add the SMTP Service

Complete instructions on how to do this (as well as configuring mail for SharePoint generally) here. I will therefore not go into any detail except to point out that if you’re running on Windows versions prior to 2008, the installation procedure is slightly different. The end result is pretty well the same. In fact, I don’t think that this feature has been updated in about 10 years, it still requires the IIS6 admin interface.

4. Add Proxy User to Online Services

Unless you want all of your automated emails appearing as if they originated with an actual user, you’ll want to use a proxy user. The down side is – you’re going to pay a license for this user. Of course, given the cost of hosted Exchange, that’s not a big deal, but it would be nice if this wasn’t required.

If you’re reading this because you have BPOS or Hosted Exchange, you already know how to do this, so I won’t spell it out here. Just remember that this user needs to be the same as that specified in the first step.

5. Configure the SMTP Service

Open up the “Internet Information Services (IIS) 6.0 Manager” from the administrative tools group on the server. You should see your server as a node – open the node, and you’ll see the SMTP server. If it’s not already started, start it. Then right click on the server and select properties. When the properties box comes up, select the delivery tab. It should appear like the following:

If you haven’t already guessed it, we will be using all 3 numbered buttons.

1 – Outbound Security:

This is where we enter the credentials for the proxy user. Exchange Online checks to see if the “From” user and the authenticated user are one and the same. If they’re not, it rejects the message, which is why we need to have the proxy user, and to be careful about the “From” field in step 1.

TLS encryption is basically SMTP’s way of saying SSL, so this screen covers that off as well.

2 – Outbound Connections

This is the screen where we get to specify the non standard SMTP port used by Exchange Online. Use 587.

3 – Advanced

I have no idea why this is any more advanced than either of the other two screens, but no matter. This is where you enter the Online Services SMTP server as a Smart Host. You can also enter a masquerade domain (the “real” mail domain), but it’s optional.

When you’re done, click OK. Then, click the access tab. Click the Authentication button to ensure that Anonymous access is selected (it is by default). Then, click the relay button.

13 Comments

Moving To Cloud Based Email–My BPOS Story

Published June 29, 2010

When I first stuck out on my own (OK…some time before I struck out on my own..), I knew that I was going to need to come up with a good email solution. My requirements extended beyond those of the consumer market, and ultimately I needed the power and control that commercial email system would offer. I really didn’t know Exchange very well, and I wasn’t about to set up a Domino server (which I knew very well) as it was no longer the direction I was heading in.

I signed up with a hosted Exchange provider. This worked quite well, and was very reliable, but I quickly bumped into size limitations and integration problems. I think that at the time the maximum size mailbox was 25 MB. I also wanted to gain experience with Exchange, so I bit the bullet and setup up a full domain with Exchange 2003 (including a Blackberry BES server) in my basement. That setup ran (in various guises) from mid 2006 to this past weekend. Initially it was comprised of multiple Exchange servers on virtual machines (required for remote Exchange access with 2003) to a single Exchange server without the BES after upgrading to Exchange 2007.

Hosting my own Exchange server was instructive, but ultimately a pain. My home internet connection is a consumer plan, and my service provider implemented multiple approaches to prevent any server hosting. This initially included blocking SMTP traffic inbound and ultimately (at a particularly bad time) blocking outbound SMTP. I quickly found workarounds to these problems (if you’re interested, I’ve used DynDNS for years, and I find their service to be exceptional. I’d recommend them in a heartbeat), but each one of these represented a significant drag on my time,and I’m not getting any younger.

In addition to the active blocking attempts,consumer ISV service isn’t exactly industrial grade. To be fair, they don’t claim that it is. In fact, ISPs typically go out of their way to not promise uptime reliability. Far too frequently after an outage, communication or power, my automatic DNS synchronizer wouldn’t update quickly enough and mail flow would be interrupted. Backup was another maintenance headache – yes it was getting done, but I had to have the infrastructure to support it, etc. All of this, and a few other things have prompted me to keep an eye open for alternatives.

My company is a Microsoft Online partner. We initially signed up to this program in the early days because of our extensive work with SharePoint, and recently, we have targeted online services as a significant growth area. One of the packages offered in Online Services is BPOS – The Business Productivity Online Suite. Simply put, this is hosted Exchange, SharePoint, Unified Messaging, and Live Meeting. All of this is offered at a very reasonable rate – $12.50 per user per month.

I decided last week to take my home Exchange system and migrate it to BPOS. The process went incredibly smoothly. The BPOS portal lays out all of the steps, but it can be a little confusing. I’ll quickly summarize them below.

1. Sync the Active Directory with BPOS

This sets up a one way synchronization between your Active Directory, and your BPOS Active directory. To be sure these are 2 different directories, and this just allows for simple user maintenance in the cloud. This step is not required for operation, but it is required for mailbox migration. One annoyance here – the synchronization tool must run on a domain joined Windows server running a 32 bit (!!!) OS. Since I only have 64 bit server set up, I had to spin up a new one. Ultimately, I would hope this was replaced by some sort of claims based model.

2. Set up your domain records

There are a number of steps here that are well documented in the setup section. These steps will allow your Outlook clients to auto discover your hosted Exchange mailboxes.

3. Migrate mailboxes

There is a tool that sets all of the appropriate user records, migrates mailbox content, and sets up email forwarding for the migrated users. It’s a VERY good idea to clean up all of your old junk before migrating. I, of course didn’t. That said, my largest mailbox (~2GB) took only about 6 hours to migrate. During the migration period, mail is still delivered to the on premises server, and it is kept both locally and in the cloud for migrated users. If a migration fails, it can be rerun and will pick up from where it left off. Once a user is migrated, and tested to be working, you use the tool to remove the mailbox from the on premises server, which will also remove forwarding. All mail will be delivered to the hosted mailbox.

3.5. Optionally, set up handheld connections to the hosted mailboxes.

4. Set Domain Records

Once all mailboxes have been migrated, set your domain’s MX record to now point to the hosted server, and use the administration portal to set it as authoritative, and to allow incoming mail. Once this is done there will be a lag while the changes propagate through the internet. Mail will not flow for a period of time, so don’t be alarmed.

5. Shut down your on premises Exchange server

…and rest peacefully.

Performance on the BPOS system has been great, and there appear to be no capacity issues. The per user mailbox limit can be set on a per person basis and the maximum is 25 GB. My mailbox is less than 2GB, and I do next to nothing to keep it cleaned out.

The only potential problem I see with it is integration. The Hosted server IS out in the cloud in a different domain, and therefore can’t reach back into the internal systems when necessary. For example, if running in a coexistence mode, free/busy time searches won’t work between the two groups of users. Also, on premises servers that need to send email won’t be able to use the hosted server to do so. Again, I hope that the promise of claims based authentication will help to alleviate these issues going forward.

BPOS is still using the 2007 Suite of products… Exchange 2007 and SharePoint 2007. They are slated to be moved to 2010 this fall, and I’m anxious to see what that will bring. When I know, I’ll certainly be posting back here.

I’m very happy with the results I’ve achieve, and heartily recommend it to any small-medium sized business. In fact, given the cost savings that can be achieved, I can’t see any reason why you wouldn’t want to go this route.

Storing Data In The Cloud

Published June 21, 2010

Last week, my colleague Ed Senez posted a very good article about cloud computing, and it’s benefits.Our company has been making moves toward the cloud for a couple of years now, with both Microsoft’s BPOS offering, and our own SharePoint Extranet Accelerator. While companies struggle with the benefits and risks of moving pieces of their business to the cloud, I can see a huge role for the cloud in the consumer space, primarily because it is so cost effective. I have been moving a lot of my personal data to the cloud for the past little while, and I thought that I would share my current observations.

Photos and Videos

Almost any Facebook user is familiar with posting pictures. The social functionality is great – tagging people lets all their friends know that they are in a new picture (maybe not so great if you don’t like the picture, but I digress….). YouTube is of course great for uploading and sharing videos, but both of these services have one drawback – they convert the files on upload resulting in a loss of fidelity. If you care about the quality of your source content, you can’t rely on these services for backup.

This fact led me a few months back to Flickr. At first look, Flickr had a lot of limitations too – a maximum file size,and a maximum upload rate per month,which initially caused me to dismiss it. What I found out was that with the subscriber version there are no limits at all – you can upload to your hearts content, and it will store the images in their true source format. I have been doing just that when I could for the past few weeks, and currently have over 2000 pictures in my photostream. Just 8000 or so to go.

Flickr also allows you to share your pictures publicly, with family and friends, or just keep them private. However, Flickr doesn’t have Facebook’s ubiquity, so I use it for purely public pictures only, and continue to rely on Facebook primarily for sharing and people tagging. Flickr does allow for videos as well, but it does have some size limits, so I will be relying on YouTube for sharing my videos, along with a separate backup strategy (see below) as I get my videos organized.

So how much does this cost? For $25 per year, I know that all of my personal pictures are backed up. Pictures are quite literally irreplaceable. Documents can be recreated, but you’ll never have a chance to capture those precise moments again. The fact that I can use the services to share picture (in full source quality) is really just a bonus.

Simple Storage with SkyDrive

Did you know that you have 25 GB of storage in the cloud that you can use free of charge? If you have a Windows Live ID (also free..) then you do. It’s called Sky Drive, and it’s extremely handy. Simply upload the files you wish to private, shared, or public folders and they’re safely secured away and accessible from any machine with a web browser. Because SkyDrive also uses WebDAV, you can map your SkyDrive folders directly to folders on your computer.

When you are navigating through your SkyDrive, you also have access to the recently released Office Web Applications. These are light, browser only versions of Microsoft Word, Excel, PowerPoint, and One Note, and they’re completely free of charge. You can create a new document using these apps, or edit anything that you upload. These apps are very handy for occasional use, for viewing purposes, or just for accessing an Office document that may have been sent to you when you don’t have the Office applications readily available.

Sky drive should pretty much eliminate the need for FTP servers, certainly for personal use. Given the cost of the service ($0.00), I really don’t see why someone wouldn’t want to take advantage of it.

Backup

I think that everyone that has used a computer for any amount of time has at some point lost data. Afterwards, there is a mad rush to back up the systems, and then make sure that there is a system in place to back everything up. Corporations typically have solid backup strategies in place (that aren’t tested frequently enough, in my opinion), but personal users are often too busy to ensure that their data is backed up in a timely fashion. There are a ton of consumer backup product out there, but they all often have one fatal flaw. They require the user to actually do something to make it work.

This is where the cloud can be of great help. If we can assume that the machine will typically have a connection to the internet, then for all intents and purposes, our backup destination is always available. All that is needed is a good service to make this painless and automatic for the end user. There are a number of such providers out there, and I’m going to briefly discuss the one that I’ve settled on – Carbonite.

With Carbonite, you download a small application that runs in the background, and is constantly ensuring that your files are being backed up. For most users it is as simple as a next – next install, which will backup all standard data folders. If you want to back up a non standard folder, just right click on it and choose to add it to the backup. You can always see what the backup status is from the console, but carbonite also (optionally) places a small indicator over the icon for each file that you have to let you know its backup status. The backed up files are also browser accessible from any internet connected PC, allowing you to access your files in a pinch, and one of the nicest features is that it not only keeps a mirror image of your system off site, it maintains file versioning, so when you make a change to a file and later decide that it wasn’t such a good idea, you can retrieve a previous version.

Given most end users’ bandwidth constraints, the initial backup can take a little while. Mine took two weeks, but that’s me. After initial backup, it all goes very rapidly. So what’s the cost of all of this storage? You can back up as much as you want from a single machine for $55 US per year. To me, that’s a no-brainer.

I spend about 5-10% of my time inside my company firewall. Tools to help with remote connectivity are crucial, and I really see a place for cloud based services to provide a lot of these tools. They’re safe, they’re easy, they’re useful, and they’re highly cost effective. In storage alone, I now back up all of my important personal data (redundantly I might add) and enhance my convenience in accessing it. All for less than $100/year.

I’m sold.