De-duplication

One thing that I think is missing in DPM is the ability to use the de-duplication functions that other backup software uses. For those who aren’t familiar with the terminology there are two similar definitions for the de-duplication:

1. Backup only one instance of a file. If 500 users read and save a monthly report only one instance will be backup and therefor use the disk-pool more sufficient.
   
2. The ability to copy information between two storage-pools.
   

The definition I write about is the first. Yes, Microsoft has this function and it’s called SIS and stands for Single Instance Storage.

This is a great feature but you will only use this by default on your exchange databases. Regarding your file servers you will still backup every single file that is the same if you don’t have the Storeage Server installed that has the SIS function built-in.

But there’s hope if you got large file servers that isn't Windows Storage Server. Microsoft will (I’m hopeful) make the SIS function work on a standard server operating system. But until then there is the possibility to use a third party software that’s called BitWackr from the company Exar

Look at this You Tube clip for more information: http://www.youtube.com/watch?v=_mwff2BKrR8

I do want to emphasize my point that Microsoft uses the SIS component for larger server software but the software from Exar is a nice feature for the DPM storage-pool.

Good DPM links to TechNet

Here are some good links that is useful regarding the preformance of your DPM server.

DPM Managing Performance: http://technet.microsoft.com/en-us/library/ff399471.aspx

How DPM Operations Affect Performance: http://technet.microsoft.com/en-us/library/ff399315.aspx

DPM and Memory: http://technet.microsoft.com/en-us/library/ff399244.aspx

Improving Performance: http://technet.microsoft.com/en-us/library/ff399571.aspx

Managing DPM Performance on a WAN: http://technet.microsoft.com/en-us/library/ff634226.aspx

How Protection Groups Changes Affect Jobs: http://technet.microsoft.com/en-us/library/ff399316.aspx

KB 2250444

The first QFE for DPM2010 was released at the 15th of November. This contains fixes for:


• Blocks a backup of an Exchange DAG to a secondary DPM 2010 server.
• You are prompted to restart a client computer after you install an agent on the client.
• DPM services crash and you receive the error: Unable to connect to the database because of a fatal database error.
• MSDPM crashes with event ID 945 in the event log.
• When you change a Protection Group, add a very large database, change the disk allocation and then commit the Protection Group, DPM 2010 does not honor the user intent and set the sizes of replica and shadow copy volumes to the default sizes.

You can download the QFE here.

The backup storage location is invalid. You cannot use a volume that is included in the backup as a storage location.

A customer of mine did an in-place-upgrade of their domain controllers to 2008R2. The backup of the domain controllers system state went flawless until the in-place-upgrade took place. After the in-place-upgrade the backup of the domain controllers system state stop working.

The error encountered in the domain controllers logs was:

The backup operation attempted at '‎2010‎-‎12‎-‎01T19:39:32.315578200Z' has failed to start, error code '2155348032' (The backup storage location is invalid. You cannot use a volume that is included in the backup as a storage location. ). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.

To solve this problem you have to enable system state backups to be targeted to critical volumes.

To do this go to the registrykey at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\wbengine.
Create a key called "SystemStateBackup"
Set the value of this entry as follows:
Name: AllowSSBToAnyVolume
Data type: DWORD
Value data: 1

Reboot the server and case closed

Can’t perform an item level restore on a SharePoint 2010 farm

I’m teaching at CornerStone this week and running DPM in a lab environment. This can give you more insight in the problem solving scenarios since you always run into different interesting possibilities to solve problems.

After protecting a SharePoint 2010 farm I couldn’t perform an item level restore. Everything looked good in the console but double-clicking the content database didn’t populate any items to restore.

Since DPM must run the cataloguing function of the SharePoint content database you can be stranded up to 24h before you can preform an item level restore.

If you don’t want to wait 24h do this in the DPM2010 server DPM PowerShell:

Get-ProtectionGroup yourdpmservername | get-datasource | where-object {$_.type -like "*sharepoint*"} | start-createcatalog

Close your console and start it up again, problem solved.

DPM 2010 training

On the 25 of november I will teach the DPM 2010 training at Cornerstone. There are a few places left so call Cornerstone and book a seat.

Remove-ProductionServer.ps1

During the last two weeks I have been upgrading a large DPM 2007 installation to DPM 2010. After the upgrade was done I noticed that some agents appeared that the customer didn’t protect ant longer.

The agents information were present on the secondary DPM server, on the primary everything looked ok.

To remove those agent records I used the new PowerShell script, Remove-ProductionServer.ps1 and it worked like a charm!

Just run the script, use FQDN for the DPM server and the production server.

Can't delete Recovery Points Error ID: 32008 and 31224

This week I got a problem regarding a SharePoint solution. I couldn’t create a recovery point and I couldn’t get the DPM server GUI to remove the SharePoint Recovery Points. I got the error 32008 and 31224 .


To resolve this problem I needed to identify the Recovery Points associated with the SharePoint farm and delete those with the DPM PowerShell.


I got a list of all data sources on the DPM server
$ds = Get-Datasource –DPMServerName [dpmservername]


Identify the data source that you want to protect in the list. Using $ds returns a zero-based array. So if the data source you want is the sixth one in the list, you would use $ds[5] to identify it in the next step.

Get all the recovery points associated with the data source.
$rp = Get-RecoveryPoint $ds[5]

Select a recovery point that was taken when the data source was in the first protection group (for example, $rp[2]). Remove the recovery point.

Remove-RecoveryPoint $rp[2]

Repeat the last two steps until you have removed all recovery points associated with the data source when it was in the first protection group.

How much will the workloads allocate in my disk pool?

In the planning stage of a DPM installation you may want to calculate the amount of disk space your workloads will use. DPM will create two different volumes in the DPM disk pool, the replica- and recovery point volume.

You calculate your replica volume with the following formulas:

For Exchange data = Data source size x (1 + log change) / (alert threshold - .05)
For SQL Server data = Data source size x (1 + log change) / (alert threshold - .05)
For Windows SharePoint Services data = Total size of all databases/ (alert threshold - .05)
For Virtual Server data = Data source size x 1.5
For Windows Server system state = (Data source size x 3) / 2
For file data = (Data source size x 3) / 2

You calculate your Recovery point volume with the following formulas:

For Exchange data = 4.0 x retention range in days x log change x data source size + 1600 MB
For SQL Server data = 2.5 x retention range in days x log change x data source size + 1600 MB
For Windows SharePoint Services data = 1.5 x retention range in days x log change x total size of all databases + 1600 MB
For Virtual Server data = (Data source size x retention range in days x 0.02) + 1600 MB
For system state = (Data source size x retention range in days x 2) / 100 + 1600 MB
For files = (Data source size x retention range in days x 2) / 100 + 1600 MB

The log change is:

• 6% for Exchange
• 6% for SQL
• 10% for SharePoint

CornerStone's DPM 2010 course

The official DPM 2010 course from CornerStone is now available. Welcome to attend my class!


CornerStone's DPM 2010 course, click here!

Manually Install DPM Agents

If you want to install a DPM agent manually on a domain server do this:

1. Share the RA catalouge under the %systemdrive%:\Program Files\Microsoft DPM\DPM\ProtectionAgents
   
   
2. From the server, run from START //dpmservername/RA
   
   
3. Right click the DPMAgentInstaller_x86 or DPMAgentInstaller_x64 depending in the version on the operating system and choose run as administrator
   
   
4. After the installation is finished open a command prompt in an elevated mode
   


5. Go to %systemdrive%\Program Files\Microsoft Data Protection Manager\DPM\bin
   
   
6. Run the SetDpmServer.exe executable with the switch -DpmServerName
   
   
7. On the DPM server attatch the agent in the console using the installation wizard or the PowerShell command Attach-ProductionServer.ps1

Best Of MMS

The Best Of MMS event went of in Stockholm in the end of September. A great event planned and arranged by CornerStone. Thank you to all the participant on the DPM session.

Mr. Jonny Andersson from Microsoft Sweden

MigrateDatasourceDataFromDPM.ps1

This week I ran into a little trouble regarding one of the DPM’s disk in the diskpool. The ISCSI storage I had connected to the DPM server went faulty and went offline and started to rebuild its RAID now and then, no good...

I needed to migrate the data from one disk to another disk in the diskpool but I had to migrate to protection groups one by one. To do so I did this.

1. Connect the new disk to the DPM server
2. Open DPM’s PowerShell console
3. Extract the Different protection groups on the DPM server to a variable with the following command $pg = Get-ProtectionGroup DPMTestServer
4. Next I needed to get all the data source for a protection group $ds = Get-Datasource $pg[0]
5. Then I needed to know what disks was connected to the DPM server $disk = Get-DPMDisk –DPMServerName
6. Now I had all the information I needed to start my migration using the MigrateDatasourceDataFromDPM.PS1 script
   ./MigrateDatasourceDataFromDPM.ps1 –DPMServerName –Source $ds –Destination $disk[4]
   

Note that the old volumes on the old disk must be left active until the retention time has run out. DPM will then automatically unallocated those old volumes. After the script has run successfully DPM will start to write the new data on the new disk.

Basics Of Backups, my friend BOB =)

Every now and then I’m sitting next to an IT department or another IT person on a company and talk about backup strategies. The scariest part is that 75% of companies and IT personnel don’t do any test restores of the backed-up data. And if the need to restore something else than just flat-files the depend on consultant that will cost the companies a lot of money just down the drain.

WHY?

A good thing is to do those test restores, set up a restore strategy, train the IT personnel, inform the CEO about the strategy, let the CEO sign the written strategy.
Another part is that many companies use a mixed environment of backup software. This is just begging for trouble. Use a clean strategy, one software. I recommend DPM to the companies I visit or buy my services for creating backup strategies for a simple reason, it works!

Disaster Recovery of a DPM server made easy

Using a primary and a secondary DPM server is a great feature for a disaster recovery scenario. The advance of using this approach is that you will be safer if the primary DPM server will fail in somehow, switching the agents so they communicate with the secondary DPM server directly is made easy with a PowerShell script or a CMDLET.

Some companies doesn’t got the economy to have a primary and a secondary DPM server. So how can we design a disaster and recovery function for companies that runs only one DPM server? This is made easy with the executable DpmBackup.exe.

This batchfile will dump the local DPM database (DPMDB) to the location:
%systemdrive%\Program Files\Microsoft DPM\DPM\Volumes\ShadowCopy\Database Backups.

After running the batchfile you will see a file named DPMDB.BAK. This file contains all the important information regarding DPM and its agents, disks etc.

A good idea is to run a schedule task that dumps this DPMDB and copy it to another physical location or hardware. An example of a BAT-file could look like this:

cd "c:\Program Files\Microsoft DPM\DPM\bin"
dpmbackup.exe –db

net use x: \\SERVERNAME\SHARENAME
cd "C:\Program Files\Microsoft DPM\DPM\Volumes\ShadowCopy\Database Backups"
xcopy dpmdb.bak X: /Y

Secondary DPM server and the “Modify link”

DPM2010 is made better in many ways. One of those ways is working with primary and secondary protection of DPM servers. In DPM 2007 the DPM administrator had to manually activate the secondary protection for the DPM agent attached to the primary DPM server for the workloads on the primary DPM server.

In DPM2010 the DPM administrator just have to attach the primary DPM server to the secondary DPM and voilà the secondary DCOM+ configuration for the protected members of the primary DPM server is fixed.

Great job guys!!

Shrink Volumes & Auto-Grow

A new feature in DPM2010 is the Shrink Volume feature that will verify the possibility of shrinking an allocated volume in DPM’s disk pool.

In DPM2007 the disk allocation was an issue regarding administrators allocated to much disk space from the disk pool. This resulted in very many holes never to be used in DPM’s disk pool.

With the ability of shrinking a volume to a correct size and activating auto-grow, a new feature that gives the administrator a more relaxed word day, the DPM administrator has more time over for administration tasks instead of allocating more disk space to different volumes in DPM’s disk pool.

To activate auto-grow you mark the checkbox in the Review Disk Allocation window when you create a new Protection Group.

If you have upgraded your DPM server from 2007 to 2010 you can afterwards activate the auto-grow function. Right click on the Protection Group, choose Modify Disk Allocation… and check the box Automatically grow the volumes .

BMR can protect more

BMR of Bare Metal Recovery is a ”new” feature in DPM 2010. This will give the administrator the possibility to recover the server from a dreadful state like corruption in the registry or total HW failure.

One important note is that the BMR function ONLY supports 2008 OS that has the WSB component installed. The WSB is the local backup feature on the server if it’s not installed BMR won’t work, pretty simple.

Regarding to Microsoft BMR will only protect the System State and C drive on the server you are protecting. You CAN modify the BMR function so it will protect all disks on the server! BMR functions with a CMDLET named Bmrbackup.cmd. To alter this configuration do this:

On the protected server, open the C:\Program Files\Microsoft Data Protection Manager\DPM\bin\Bmrbackup.cmd file in notepad. Modify this line and add an -include: statement to add the drive letter of the additional volumes to be included in the BMR backup.

FROM
rem Start a wbadmin.exe to create a BMR backup
start /WAIT %SystemRoot%\system32\wbadmin.exe start backup -allcritical -quiet -backuptarget:%1

TO
rem Start a wbadmin.exe to create a BMR backup
start /WAIT %SystemRoot%\system32\wbadmin.exe start backup -allcritical -include:D: -quiet -backuptarget:%1
You will need to adjust the replica volume and recovery point volume sizes on the DPM server to account for the additional space required for the extra volume(s) being included in future BMR backups.

DPM RA Rights

Since DPM is an agent based solution the agents is of great dependence for DPM’s functionality. In some cases when I have deployed agents in a domain the ACL rights for some reason went wrong.

I you got problem starting your DPM RA service it’s a good thing to verify that your ACL for the “Launch and Activation Permission” is correctly configured.

To verify this click START and type dcomcnfg, this will open the Component Services.

Expand the Component Services / Computer / My Computer /DCOM Config. Right click the DPM RA Services and choose Properties.

Under the Security tab click the Edit… button in the Launch and Activation Permissions to open the ACL.


Verify that the DPM Server has all the Allow checkboxes marked.

Best Of MSS 2010

At the 28 of september and 29 of september I’m asked to be the key speaker for DPM 2010 at the Best of MMS Summit in Sweden. At this seminar you will be able to get the latest updates regarding DPM, Operations Manager etc. Don’t miss this great opportunity to meet colleges in your line of work and ask the experts.

http://www.cornerstone.se/sv/Event/Best-of-MMS/

CU THERE!

=)

Enabling Serialized backup of Hyper-V CSV Cluster

One feature that is really great about the Hyper-V role in Windows Server 2008 R2 is the CSV cluster service. DPM2010 has the function of backup this cluster type. By design DPM2010 will use the function of VSS hardware providers, this can simply be explained as letting the hardware (SAN) manage the VSS snapshot instead of the operating system. The advantage is that DPM2010 can make a parallel backup of, by default, 3 virtual servers.



A few week ago I ran into a little challenge regarding the VSS hardware provider. The customer bought a ISCSI storage solution that didn’t have any VSS hardware provider. So when I ran the backup DPM2010 alerted me that there was an issue with the VSS hardware provider.


Failed to prepare a Cluster Shared Volume (CSV) for backup as another backup using the same CSV is in progress. (ID 32612 Details: Back up is in progress. Please wait for backup completion before trying this operation again (0×8007173D)



To solve this problem I had to enable serialized backup of the Hyper-V cluster. This is a four-step process. First you must modify a registry entry and second you must share a folder and third modify a script called ListVMsOnClusterForDPM and finally you must create a scheduled task. You can download the script here.


1. Modify the registry of the DPM server
a. Go to Start > Run and type in regedit
b. Navigate to HKLM\SOFTWARE\Microsoft\Microsoft Data Protection Manager\2.0\Configuration\MaxAllowedParallelBackups
c. Edit the Microsoft Hyper-V key and change the Value from 3 to 1
d. You must reboot your DPM server for the new registry entries to take effect


2. Share the config folder on the DPM server
The next part we must do is to share the config folder on the DPM server. From a security perspective it’s recommended that you use alternative accounts also called service accounts that is used for one purpose only in your domain.


You will locate the folder at the C:\Program Files\Microsoft DPM\DPM\Config. Share this folder with the shared name DPMConfig. In this step you can determine if you want to use your service account or for test purpose use the domain admin, not recommended for a permanent solution.


3. ListVMsOnClusterForDPM.ps1
Download the PowerShell script called ListVMsOnClusterForDPM, you must extract it to a folder on one of your CSV nodes, for example “C:\Scritps”. Replace the value $DPMServerName with a FQDN name of your DPM server. Now you must configure the execution policy of the node in the CSV cluster. To do so open PowerShell in administrative rights and type Set-ExecutionPolicy RemoteSigned. Now your CSV node will be able to execute the PowerShell script. The last thing you must do is to execute the ListVMsOnClusterForDPM.ps1 script from the folder you formerly extracted to the CSV node. After executing the PowerShell script an XML file called DataSourceGroups.XML will be created in the config catalogue at your DPM server. If the XML file isn’t created verify your NTFS and shared rights on the share and the config catalogue.


4. Create a scheduled task
The last thing we must fix is to keep the information in the DataSourceGroups.XML updated. We will do this by making a scheduled task that runs the ListVMsOnClusterForDPM.ps1 script. Go to Start\Administrative Tools\Task Scheduler. Click on the Create Task… link. On the General Tab. Name the task ListVMsOnClusterForDPM, enable the security feature of letting the task run whether the user is logged on or not by clicking the “Run Whether user is logged on or not” radio button. Also check the check box “Run with highest privileges”. Go now to the Triggers tab. Click New… and make your configuration as the picture below.







On the Action tab click New… make your configuration as the picture below. Note that you should use your script library =)





When you click OK you will be prompted for credentials for the account. After entering the password and clicking OK run the scheduled and verify that the file DataSourceGroups.XML is created on the DPM server.

Now you can create a protection group for your CSV resources or if you already have one just modify it.

Protecting Hyper-V with DPM 2010

Since the new great feature of CSV in Microsoft Clusters the availability of a virtual Microsoft Workload is significantly higher. Though there are some prerequisites that must be fulfilled before you can protect the Microsoft Workloads that uses the CSV function in the cluster.

There are two hotfixes that you must apply on the host. After applying those hotfixes you must update your Hyper-V Integration Services on the guests.
Apply the hotfix KB975921 and KB975354 on your Hyper-V hosts.

Reboot is required.

Upgrading DPM 2007 to DPM 2010

I have just perfomed the first customer upgrade of a DPM 2007 server to DPM 2010. This went really well. Just as I did with the DPM 2010 RC upgrade I dumped the DPM database with the CMDLET “dpmbackup.exe –db” just in case something went wrong.

When the upgrade was finished I reviced a nice notice regarding the next steps for completing the upgrade. Restart the DPM server, upgrade the DPM agents, run consistency checks on all protected members.

/Robban

DPM 2010 RC to RTM upgrade

Microsoft made this a simple task. To perform a upgrade from DPM 2010 RC to DPM 2010 RTM insert the media an run the setup wizard for DPM 2010 RTM. Before I did the upgrade I perform a database dump with the DPM cmdlet “dpmbackup.exe –db”, just in case...

After performing the upgrade you must update your clients, no restart needed. You must also run a consistency check on your protected members alternately run the modify wizard for some Microsoft Workloads.

MMS 2010

The MMS 2010, arranged by Cornerstone, will be held i Stockholm at the 28 and 29 of September. I will be the key speaker for DPM 2010.

Click on the MMS 2010 image on the right side of the blog to get more info about the event and book a seat in the front row!

DPM Upgrade Advisor

Since there been a lot of questions regarding the compability of different upgrade scenarios regarding upgrade from DPM 2007 SP1 to DPM 2010 RTM or DPM 2010 RC to DPM 2010 RTM I thought it would be a good idea to post some information regarding this.

Microsoft has released DPM Upgrade Advisor that will help you with the steps when you perform an upgrade from your version to DPM 2010 RTM.

Download the DPM Upgrade Advisor here and try it out!

/Robban

Report Access Problem

A few weeks ago I ran into a problem regarding the access to the reports that is available on a DPM server. After I did my authentication I got the the following error:

An error has occurred during report processing. (rsProcessingAborted)
Cannot create a connection to data source 'DLS'. (rsErrorOpeningConnection)
Cannot open database "DPMDB" requested by the login. The login failed. Login failed for user 'SERVERNAME\DPMR$-SERVERNAME'.

This error occurred because the users mapped to this login pointing the tha DPMDB was missing. To solve this error do this.

Open Microsoft SQL Server Management Studio and verify %DPMServerName%\MS$DPM2007$ is selected on the “Connect to Server” screen and click Connect.

Expand %DPMServerName%\MS$DPM2007$ > Security > Logins then right click %DPMServerName%\DPMDBReaders$%DPMServerName% and select Properties. On the Login Properties screen select “User Mapping” on the left pane and under “Users mapped to
this login:” verify DPMDB is selected then you will notice DPMDBReaders$%DPMServerName% will be added under the “Users” column.

Under “Database role membership for: DPMDB” verify db_datareader and public are both checked and select “OK”.

/Robban

PruneShadowCopies.ps1

A good thing is to not make the DPM server work heavily during midnight. This is because the DPM server has a PowerShell script that runs every night at midnight.

The purpose of the PruneShadowCopies.ps1 is to determine which recovery points that are out of range of the retention time and delete those from the DPM storage pool.

Do not run a express full backup during midnight if you don’t want to choke your DPM server.

/Robban

Missing WSS Writer for the system state

Hi
I have ran into a remarkable problem regarding the backup of a System State. The thing that was so interesting was that the WSS writer that manages the System State
(System Writer)of a server was missing when I ran the vssadmin list writers cmdlet.

This problem occurs if the NTFS permissions have been removed to the C:\WINDOWS\WINSXS\FILEMAPS directory and / or files.
To resolve this issue run the following command from an administrative CMD
prompt:

Takeown /f %windir%\winsxs\filemaps\* /a
icacls %windir%\winsxs\filemaps\*.* /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\winsxs\filemaps\*.* /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\winsxs\filemaps\*.* /grant “BUILTIN\Users:(RX)”

C:\Windows\system32>icacls C:\windows\winsxs\temp\PendingRenames

C:\windows\winsxs\temp\PendingRenames NT SERVICE\TrustedInstaller:(OI)(CI)(F)
BUILTIN\Administrators:(OI)(CI)(RX)
NT AUTHORITY\SYSTEM:(OI)(CI)(RX)
BUILTIN\Users:(OI)(CI)(RX)

Tape script

Hi

Here’s a good powershell script that will help you to determine the content of a tape used by DPM.

$Lib = Get-DPMLibrary "Server-name"
$tpList = Get-Tape $Lib

foreach ($tp in $tpList)
{
$tp | fl label,barcode
Get-RecoveryPoint $Tp | fl
}

I can’t take fully credit for the design of this script, thanx M.

Important fix

Due to disk management issues in Windows Server 2008 a fix was published to fix this problem on the following URL: http://support.microsoft.com/kb/962975

Windows Server 2008 can run into problem with the disk management if the disks consist of more than 99 volumes. This could also result into a system failure.

Since DPM works with dynamic disks and creates volumes very frequently on those disks it’s a very good idea to install the fix.

/Robban

Let the DPM server backup itself

Recently a customer of mine invested in the DPM backup solution. Due to economical issues the customer waited to invest in a secondary DPM server, instead I configured the primary DPM server to backup it self. Use this command in the DPM PowerShell.

Set-DPMGlobalProperty -DPMServerName DpmServerName -AllowLocalDataProtection $true