The White Pages are now Running on Windows and SQL Azure

Jan 242011
 

Ever since I started this blog, I’ve hosted it internally on our premises. Part of the reason for this was that I wanted to have full control over what was going on with it, and I wanted to work in a familiar environment. For me, that was of course the Microsoft stack. While SharePoint has excellent blogging features, made even better by the Community Kit for SharePoint: Enhanced Blog Edition, my feeling is that its feature set is more applicable to an inside the firewall deployment. Also, if I were to use SharePoint for this purpose, I’d be constantly distracted by the desire to improve upon it.

What I needed was a platform that was focused on blogging, and that I wouldn’t wind up tinkering with too much. I settled on WordPress, which seemed to be very well supported, and quite good at what it did. WordPress had direct integration with Windows Live Writer, and had apps for the iPhone, Blackberry, Android, and now Windows Phone 7.

WordPress natively runs on PHP and MySQL, and typically runs in Linux environments. However, since IIS supports PHP and MySQL runs on Windows, it is possible to get it running in my “familiar environment”. Normally doing this sort of thing is a bear, but by using the Web Platform Installer from Microsoft, the installation was a breeze. All that was necessary was to run it, and select WordPress as a desired application. The installer then took care of downloading PHP, MySQL, WordPress, and integrating them all together. After answering a few account and password questions, I was up and running, and have been ever since.

The one drawback of this approach was that I was hosting it myself, and therefore always concerned with reliability and uptime. More importantly it has been sharing a server with other applications, and more than once has gone down because another system needed a reboot, crashed, or something. A hosted environment was obvious, and since I’ve been exploring the Azure platform lately, I thought I’d see what was involved.  One of the advantages of the MVP program, which I’m newly a part of is that you are allocated a certain amount of Azure computing hours, so off I went experimenting.

Happily, one weekend later, this blog has been transitioned to a high speed, and highly available platform, that most importantly, I don’t have to maintain. Not only that, but I’ve been able to take MySQL out of the picture completely, and I’m using a SQL Azure database as my data store. I had several false starts right away, and I’m going to document the approach  that I took and post it here shortly, but for now, I’m pretty happy with the results.

Hello Azure!

Integrating SharePoint On Premises With BPOS and Exchange Online: Part 1–Outbound

Sep 212010
 

If you’ve ever set up a SharePoint Farm, you’ll know that one of the first things that you need to configure is Outgoing Email. The way that you typically do this is to specify the name of an Exchange (or any other SMTP server) in your organization, a from and reply to address, and you’re done. Occasionally there are problems because the Exchange (or other) server does not allow relaying, and that’s easily remedied with a quick server configuration.

If you’re using BPOS, you not only have Exchange online, but you have SharePoint online. Outgoing email is already automatically set up for you. Just set an email alert for yourself and your will receive an email when anything changes. However, what about organizations that have elected to use hosted Exchange or BPOS, but also still maintain an on-premises SharePoint farm?

The good news is that it can be done. The bad news is that it’s not as simple as before when everything was on premises. This article will attempt to walk through the required steps.

This example uses SharePoint 2010 but the same is true (I think) for SharePoint 2007.

1. Setup SharePoint Outgoing Mail

As before, go to Central Administration, navigate to system settings, and click the “Outgoing E-Mail Settings” link.

image

There are two major things to note here. Firstly,the Outbound SMTP server is NOT one of the Online Services servers. What is it? Well,it doesn’t exist yet – we’ll get to that in step 3. The reason for this is that to deliver mail to any of the Online Services servers, you need to authenticate, use SSL, and use a non standard SMTP port. Unless I’m missing something, I don’t see any of those options here. (You can find complete instructions on relaying messages to BPOS and Exchange Online here). What we therefore need to do is to set up our very own SMTP server that can relay these messages for us.

The other thing to take note of here is the “From address”. When working internally, this doesn’t normally matter, you can give it any old fake name and off it goes. This is not true here, if this email address is not valid online, mail will not be delivered. We will add this address later.

2. Update DNS

If you haven’t already done so, you’ll need to add DNS entries for the SMTP server that will use the same IP address as this SharePoint server. In this example I’m using an internal domain, but you’ll also want to add another one externally if you’ll be configuring incoming email. all that will matter is that it resolve to the same machine.

3. Add the SMTP Service

Complete instructions on how to do this (as well as configuring mail for SharePoint generally) here. I will therefore not go into any detail except to point out that if you’re running on Windows versions prior to 2008, the installation procedure is slightly different. The end result is pretty well the same. In fact, I don’t think that this feature has been updated in about 10 years, it still requires the IIS6 admin interface.

4. Add Proxy User to Online Services

Unless you want all of your automated emails appearing as if they originated with an actual user, you’ll want to use a proxy user. The down side is – you’re going to pay a license for this user. Of course, given the cost of hosted Exchange, that’s not a big deal, but it would be nice if this wasn’t required.

If you’re reading this because you have BPOS or Hosted Exchange, you already know how to do this, so I won’t spell it out here. Just remember that this user needs to be the same as that specified in the first step.

5. Configure the SMTP Service

Open up the “Internet Information Services (IIS) 6.0 Manager” from the administrative tools group on the server. You should see your server as a node – open the node, and you’ll see the SMTP server. If it’s not already started, start it. Then right click on the server and select properties. When the properties box comes up, select the delivery tab. It should appear like the following:

image

If you haven’t already guessed it, we will be using all 3 numbered buttons.

1 – Outbound Security:

image

This is where we enter the credentials for the proxy user. Exchange Online checks to see if the “From” user and the authenticated user are one and the same. If they’re not, it rejects the message, which is why we need to have the proxy user, and to be careful about the “From” field in step 1.

TLS encryption is basically SMTP’s way of saying SSL, so this screen covers that off as well.

2 – Outbound Connections

image

This is the screen where we get to specify the non standard SMTP port used by Exchange Online. Use 587.

3 – Advanced

image

I have no idea why this is any more advanced than either of the other two screens, but no matter. This is where you enter the Online Services SMTP server as a Smart Host. You can also enter a masquerade domain (the “real” mail domain), but it’s optional.

When you’re done, click OK. Then, click the access tab. Click the Authentication button to ensure that Anonymous  access is selected (it is by default). Then, click the relay button.

image

This one is very important. Get it wrong on the strict side, and mail won’t flow. Make it too permissive, and you become a spam relay, and likely get your domain blacklisted (not a good idea). Don’t mess around with this one. In this case, the SharePoint server needs to relay, so I’ve allowed its IP address. You can allow an entire range if that’s more appropriate. When we get to inbound mail, it will be delivered here, so that’s not a problem.

If you like go back to the General tab, and enable logging. It’s the only way you’ll be able to trace what’s going on with mail. Once you know it’s all working, then you’ll want to disable logging.

You will need to stop and restart the SMTP service, and I suggest an IISReset. If you can get away with it, just go ahead and reboot the server. It helps. I should point out that debugging this is very hard. There are slight time lags everywhere, but one to be aware of if you’re using SharePoint alerts to test this is that alerts are NOT immediate – they run on a timer job. Don’t assume it’s failing because new item alerts aren’t arriving. However, new subscriptions ARE immediate, so the best way to test is to add a new alert to a list.

That should just about do it. I’ll post on my experiences getting incoming email to work with BPOS as soon as I do….

Moving To Cloud Based Email–My BPOS Story

Jun 292010
 

When I first stuck out on my own (OK…some time before I struck out on my own..), I knew that I was going to need to come up with a good email solution. My requirements extended beyond those of the consumer market, and ultimately I needed the power and control that commercial email system would offer. I really didn’t know Exchange very well, and I wasn’t about to set up a Domino server (which I knew very well) as it was no longer the direction I was heading in.

I signed up with a hosted Exchange provider. This worked quite well, and was very reliable, but I quickly bumped into size limitations and integration problems. I think that at the time the maximum size mailbox was 25 MB.  I also wanted to gain experience with Exchange, so I bit the bullet and setup up a full domain with Exchange 2003 (including a Blackberry BES server) in my basement. That setup ran (in various guises) from mid 2006 to this past weekend. Initially it was comprised of multiple Exchange servers on virtual machines (required for remote Exchange access with 2003) to a single Exchange server without the BES after upgrading to Exchange 2007.

Hosting my own Exchange server was instructive, but ultimately a pain. My home internet connection is a consumer plan, and my service provider implemented multiple approaches to prevent any server hosting. This initially included blocking SMTP traffic inbound and ultimately (at a particularly bad time) blocking outbound SMTP. I quickly found workarounds to these problems (if you’re interested, I’ve used DynDNS for years, and I find their service to be exceptional. I’d recommend them in a heartbeat), but each one of these represented a significant drag on my time,and I’m not getting any younger.

In addition to the active blocking attempts,consumer ISV service isn’t exactly industrial grade. To be fair, they don’t claim that it is. In fact, ISPs typically go out of their way to not promise uptime reliability. Far too frequently after an outage, communication or power, my automatic DNS synchronizer wouldn’t update quickly enough and mail flow would be interrupted. Backup was another maintenance headache – yes it was getting done, but I had to have the infrastructure to support it, etc. All of this, and a few other things have prompted me to keep an eye open for alternatives.

My company is a Microsoft Online partner. We initially signed up to this program in the early days because of our extensive work with SharePoint, and recently, we have targeted online services as a significant growth area. One of the packages offered in Online Services is BPOS – The Business Productivity Online Suite. Simply put, this is hosted Exchange, SharePoint, Unified Messaging, and Live Meeting. All of this is offered at a very reasonable rate – $12.50 per user per month.

I decided last week to take my home Exchange system and migrate it to BPOS. The process went incredibly smoothly. The BPOS portal lays out all of the steps, but it can be a little confusing. I’ll quickly summarize them below.

1. Sync the Active Directory with BPOS

This sets up a one way synchronization between your Active Directory, and your BPOS Active directory. To be sure these are 2 different directories, and this just allows for simple user maintenance in the cloud. This step is not required for operation, but it is required for mailbox migration. One annoyance here – the synchronization tool must run on a domain joined Windows server running a 32 bit (!!!) OS. Since I only have 64 bit server set up, I had to spin up a new one. Ultimately, I would hope this was replaced by some sort of claims based model.

2. Set up your domain records

There are a number of steps here that are well documented in the setup section. These steps will allow your Outlook clients to auto discover your hosted Exchange mailboxes.

3. Migrate mailboxes

There is a tool that sets all of the appropriate user records, migrates mailbox content, and sets up email forwarding for the migrated users. It’s a VERY good idea to clean up all of your old junk before migrating. I, of course didn’t. That said, my largest mailbox (~2GB) took only about 6 hours to migrate. During the migration period, mail is still delivered to the on premises server, and it is kept both locally and in the cloud for migrated users. If a migration fails, it can be rerun and will pick up from where it left off. Once a user is migrated, and tested to be working, you use the tool to remove the mailbox from the on premises server, which will also remove forwarding. All mail will be delivered to the hosted mailbox.

3.5. Optionally, set up handheld connections to the hosted mailboxes.

4. Set Domain Records

Once all mailboxes have been migrated, set your domain’s MX record to now point to the hosted server, and use the administration portal to set it as authoritative, and to allow incoming mail. Once this is done there will be a lag while the changes propagate through the internet. Mail will not flow for a period of time, so don’t be alarmed.

5. Shut down your on premises Exchange server

…and rest peacefully.

Performance on the BPOS system has been great, and there appear to be no capacity issues. The per user mailbox limit can be set on a per person basis and the maximum is 25 GB. My mailbox is less than 2GB, and I do next to nothing to keep it cleaned out.

The only potential problem I see with it is integration. The Hosted server IS out in the cloud in a different domain, and therefore can’t reach back into the internal systems when necessary. For example, if running in a coexistence mode, free/busy time searches won’t work between the two groups of users. Also, on premises servers that need to send email won’t be able to use the hosted server to do so. Again, I hope that the promise of claims based authentication will help to alleviate these issues going forward.

BPOS is still using the 2007 Suite of products… Exchange 2007 and SharePoint 2007. They are slated to be moved to 2010 this fall, and I’m anxious to see what that will bring. When I know, I’ll certainly be posting back here.

I’m very happy with the results I’ve achieve, and heartily recommend it to any small-medium sized business. In fact, given the cost savings that can be achieved, I can’t see any reason why you wouldn’t want to go this route.

Storing Data In The Cloud

Jun 212010
 

Last week, my colleague Ed Senez posted a very good article about cloud computing, and it’s benefits.Our company has been making moves toward the cloud for a couple of years now, with both Microsoft’s BPOS offering, and our own SharePoint Extranet Accelerator. While companies struggle with the benefits and risks of moving pieces of their business to the cloud, I can see a huge role for the cloud in the consumer space, primarily because it is so cost effective. I have been moving a lot of my personal data to the cloud for the past little while, and I thought that I would share my current observations.

Photos and Videos

Almost any Facebook user is familiar with posting pictures. The social functionality is great – tagging people lets all their friends know that they are in a new picture (maybe not so great if you don’t like the picture, but I digress….). YouTube is of course great for uploading and sharing videos, but both of these services have one drawback – they convert the files on upload resulting in a loss of fidelity. If you care about the quality of your source content, you can’t rely on these services for backup.

This fact led me a few months back to Flickr. At first look, Flickr had a lot of limitations too – a maximum file size,and a maximum upload rate per month,which initially caused me to dismiss it. What I found out was that with the subscriber version there are no limits at all – you can upload to your hearts content, and it will store the images in their true source format. I have been doing just that when I could for the past few weeks, and currently have over 2000 pictures in my photostream. Just 8000 or so to go.

Flickr also allows you to share your pictures publicly, with family and friends, or just keep them private. However, Flickr doesn’t have Facebook’s ubiquity, so I use it for purely public pictures only, and continue to rely on Facebook primarily for sharing and people tagging. Flickr does allow for videos as well, but it does have some size limits, so I will be relying on YouTube for sharing my videos, along with a separate backup strategy (see below) as I get my videos organized.

So how much does this cost? For $25 per year, I know that all of my personal pictures are backed up. Pictures are quite literally irreplaceable. Documents can be recreated, but you’ll never have a chance to capture those precise moments again. The fact that I can use the services to share picture (in full source quality) is really just a bonus.

Simple Storage with SkyDrive

Did you know that you have 25 GB of storage in the cloud that you can use free of charge? If you have a Windows Live ID (also free..) then you do. It’s called Sky Drive, and it’s extremely handy. Simply upload the files you wish to private, shared, or public folders and they’re safely secured away and accessible from any machine with a web browser. Because SkyDrive also uses WebDAV, you can map your SkyDrive folders directly to folders on your computer.

When you are navigating through your SkyDrive, you also have access to the recently released Office Web Applications. These are light, browser only versions of Microsoft Word, Excel, PowerPoint, and One Note, and they’re completely free of charge. You can create a new document using these apps, or edit anything that you upload. These apps are very handy for occasional use, for viewing purposes, or just for accessing an Office document that may have been sent to you when you don’t have the Office applications readily available.

Sky drive should pretty much eliminate the need for FTP servers, certainly for personal use. Given the cost of the service ($0.00), I really don’t see why someone wouldn’t want to take advantage of it.

Backup

I think that everyone that has used a computer for any amount of time has at some point lost data. Afterwards, there is a mad rush to back up the systems, and then make sure that there is a system in place to back everything up. Corporations typically have solid backup strategies in place (that aren’t tested frequently enough, in my opinion), but personal users are often too busy to ensure that their data is backed up in a timely fashion. There are a ton of consumer backup product out there, but they all often have one fatal flaw. They require the user to actually do something to make it work.

This is where the cloud can be of great help. If we can assume that the machine will typically have a connection to the internet, then for all intents and purposes, our backup destination is always available. All that is needed is a good service to make this painless and automatic for the end user. There are a number of such providers out there, and I’m going to briefly discuss the one that I’ve settled on – Carbonite.

With Carbonite, you download a small application that runs in the background, and is constantly ensuring that your files are being backed up. For most users it is as simple as a next – next install, which will backup all standard data folders. If you want to back up a non standard folder, just right click on it and choose to add it to the backup. You can always see what the backup status is from the console, but carbonite also (optionally) places a small indicator over the icon for each file that you have to let you know its backup status. The backed up files are also browser accessible from any internet connected PC, allowing you to access your files in a pinch, and one of the nicest features is that it not only keeps a mirror image of your system off site, it maintains file versioning, so when you make a change to a file and later decide that it wasn’t such a good idea, you can retrieve a previous version.

Given most end users’ bandwidth constraints, the initial backup can take a little while. Mine took two weeks, but that’s me. After initial backup, it all goes very rapidly. So what’s the cost of all of this storage? You can back up as much as you want from a single machine for $55 US per year. To me, that’s a no-brainer.

 

I spend about 5-10% of my time inside my company firewall. Tools to help with remote connectivity are crucial, and I really see a place for cloud based services to provide a lot of these tools. They’re safe, they’re easy, they’re useful, and they’re highly cost effective. In storage alone, I now back up all of my important personal data (redundantly I might add) and enhance my convenience in accessing it. All for less than $100/year.

I’m sold.