Post
Ask Question

Automating SharePoint Online Migration Using Sharegate

Introduction

Migrating to M365 cloud is an ongoing task and most organizations are moving forward with online approach for following benefits.

  • No need to maintain physical infrastructure
  • No need to configure and update the security and cumulative updates
  • Take advantage of pay per use/subscription model
  • And for other obvious reasons ๐Ÿ˜Š

There are some organizations that want to stick with on-prem for obvious reasons such as GDPR, local security, governance, etc.

This article focuses on migrating to SharePoint online using Sharegate Desktop tool. Sharegate is industry standard licensed tool to migrate to M365 cloud from on-premises environment.

Sharegate Desktop gives rich and intuitive UI for migrating to M365 cloud. It also has a built-in PowerShell module aka Sharegate Desktop Shell (PowerShell with Sharegate Desktop libraries loaded) which helps to automate the migration jobs using PowerShell.

Advantages

There are a few advantages such as

  • The source and destination sites can be loaded from CSV. Following is the screen capture of the template.

  • If you are trying to migrate more than one site collection, you can automate the migration using the Sharegate PowerShell module.
  • No need to navigate to Sharegate UI screens for multiple times.

Points to Note

  • Microsoft recommends performing the migration to M365 cloud off the business hours and weekends.
  • There is no option to set the migration speed and mode in Sharegate PowerShell. However, the settings from the UI will be applied to Sharegate PowerShell scripts too.
  • Throttling is normal when performing migrations to cloud. Though the Sharegate tool intelligently handles the throttling below are the recommendations from the Sharegate SME team

Have the migration performance settings from the Sharegate tool to ‘Normal’ mode.

Automating SharePoint online migration using Sharegate

You may need to switch between ‘Normal’ and ‘Low’ modes. If you are getting throttled frequently and try using ‘Low’ mode. Even though it takes time, the success rate is much better.

In the migration tab, set the migration speed to ‘Insane Mode’.

Automating SharePoint online migration using Sharegate

  • Microsoft encourages a flat structure of sites in SharePoint online. For that reason, it recently deprecated the subsite creation in the newly created tenants. MSFT recommends using hub site features to connect the various sites under particular category.
  • The detailed logging is available in Sharegate UI. You can observe the migration status from the tool UI once the PowerShell script starts executing.

Automating SharePoint online migration using Sharegate

More information about migration speeds and observations can be found in references section.

Steps

Below are the steps at a glance

  • At first, the source and destination sites are imported from the CSV file
  • The script checks the destination file if it is already created and active
  • If destination file exists, it performs the incremental migration
  • Once the migration completed it logs the message in console as well as in the log file.

PowerShell Script

As a pre-requisite, the Sharegate Desktop needs to be purchased and installed on the Windows server or client machine with valid production keys. The Trial key may not support the larger migrations.

Step 1

Load the CSV files from the local folder.

$SiteInfo=Import-Csv-Path"C:\M365Migration\ToBeMigrated.csv"

Step 2

Update the copy settings to Incremental mode. I selected incremental mode since it can be reused for doing incremental migration. For the incremental, it checks if the content exists in destination if not it copies the content from the source.

$copySettings=New-CopySettings-OnContentItemExistsIncrementalUpdate

Step 3

For each record, check if the destination site is active, if so then perform the migration.

foreach($recordin$SiteInfo) {
    $SiteCheck = CheckUrl($record.DestinationSite)
    if ($SiteCheck - contains "Success") {
        $srcSite = Connect - Site - Url$record.SourceSite
        $dstSite = Connect - Site - Url$record.DestinationSite
        $MigrationResult = Copy - Site - Site$srcSite - DestinationSite$dstSite - Merge - NoCustomizedListForms - CopySettings$copySettings - ErrorActionStop
    }
}

Please note that ‘CheckUrl’ is the custom function that is integrated in this Migration Script, this function is to check whether the destination site is active and has proper permissions to the account that is running the migration.

  • Merge – to merge the source site into destination site
  • NoCustomizedListForms – since SharePoint online comes with modern UI by default, using this option will have the lists and libraries that are migrated from source to have modern look and feel.
  • CopySettings – this is used to tell the migration to be done in incremental model

ErrorAction – this parameter is used here to stop the migration and log the error when some unexpected error occurs.

Update: Updated the below 'Complete Script' to use the existing connection. 

Complete Script

# Load the sites from CSV
# check for incremental - Completed
# check for migration speed control - Need to check with SG team 
# check for monitoring the running jobs, if gets throttled, is there option to pause 
#Script Updated to use existing connection
# The script is based on moving On-Prem SP2013 to SharePoint online
function CheckUrl($urlparam) {
  <#
    .Description
    This function checks the destination URL after loaded from CSV
  #>
    try {
        Write-Host "verifying the url $urlparam" -ForegroundColor Yellow
        $CheckConnection = Invoke-WebRequest -Uri $urlparam
        if($CheckConnection.StatusCode -eq 200) {
        Write-Host "Connection Verified" -ForegroundColor Green
        $status="Success"
    }
}
catch [System.Net.WebException] {
    $ExceptionMessage = $Error[0].Exception
    if ($ExceptionMessage -match "403") {
    Write-Host "URL exists, but you are not authorized" -ForegroundColor Yellow
    Write-LogWarning -Message "URL $urlparam exists, but you are not authorized" -TimeStamp -LogPath $ErrorLogFile
    }
    elseif ($ExceptionMessage -match "503"){
    Write-Host "Error: Server Busy" -ForegroundColor Red
    Write-LogWarning -Message "URL $urlparam exists, but server is busy" -TimeStamp -LogPath $ErrorLogFile
    }
    elseif ($ExceptionMessage -match "404"){
    Write-Host "Error: URL doesn't exists" -ForegroundColor Red
    Write-LogError -Message "URL $urlparam doesn't exists" -TimeStamp -LogPath $ErrorLogFile
    
    }
    else{
    Write-Host "Error: There is an unknown error" -ForegroundColor Red
    Write-LogError -Message "URL $urlparam unknown error" -TimeStamp -LogPath $ErrorLogFile
    }
    $status="Error Occured"
}
return $status
}
 

Import-Module PSLogging
$CurrentDate = Get-Date
$DateFormat = $CurrentDate.ToString('MM-dd-yyyy_hh-mm-ss')
#Update the path to the operation logs. 
$LogPath = "C:\Users\vayina1\Temp\OpsLogs"
#update the path to error logs
$ErrorLogFile= "C:\Users\vayina1\Temp\ErrorLogs\MigraionError_" + $DateFormat + ".txt"
$LogName = "MigraionLogs_" + $DateFormat + ".log"
$LogFullPath = $LogPath + "\" + $LogName
#Change the path to CSV file where the source and destination sites are updated according to template. 
$SitesInfo = Import-Csv -Path "C:\Users\vayina1\Desktop\VinayWorkingDocs\Migration\ToBeMigrated3.csv"

Start-Log -LogPath $LogPath -LogName $LogName -ScriptVersion "1.0" | Out-Null  
# setting to incremental update. 
$copySettings = New-CopySettings -OnContentItemExists IncrementalUpdate
#use any of existing site where the account has Site Collection Admin rights and store in variable to use it for btach migrations
$m365Connection = Connect-Site -Url "https://contoso.sharepoint.com/teams/qavinay" -Browser -DisableSSO

foreach($record in $SitesInfo){
    
    $SiteCheck = CheckUrl($record.DestinationSite)
    if($SiteCheck -contains "Success") {
        try {
            $srcSite = Connect-Site -Url $record.SourceSite
            # for destination to connect use cached credentials from the previous connection $m365Connection.
            $dstSite = Connect-Site -Url $record.DestinationSite -UseCredentialsFrom $m365Connection
            Write-Host "Migration started from $srcSite to $dstSite" -ForegroundColor Yellow
            Write-LogInfo -Message "**************************************************************************************************************" -LogPath $LogFullPath
            Write-LogInfo -Message "Source Site: $srcSite" -LogPath $LogFullPath
            Write-LogInfo -Message "Destination Site: $dstSite" -LogPath $LogFullPath
            $MigrationResult = Copy-Site -Site $srcSite -DestinationSite $dstSite -Merge -NoCustomizedListForms  -CopySettings $copySettings -ErrorAction Stop 
            Write-Host $MigrationResult.Errors
            Write-Host $MigrationResult.Result

    
            if($MigrationResult.Errors -ne $null){
                Write-Host "The migration completed with errors" -ForegroundColor Yellow
                
                Write-LogInfo -Message $MigrationResult.Result -TimeStamp -LogPath $LogFullPath        
                Write-LogInfo -Message "**************************************************************************************************************`r`n" -LogPath $LogFullPath
            }

        }
        catch {
            $ErrorMessage = $_
            Write-Host "An Exception occured...$ErrorMessage"
            Write-LogError -Message $ErrorMessage -TimeStamp -LogPath $ErrorLogFile
        }
    }
    else {
    Write-Host "Error Occured... $SiteCheck" -ForegroundColor Red
    
    }
}

Conclusion

Thus, in this article we have seen how to perform the migration for multiple sites at one shot, using Sharegate Desktop Shell.

References


Similar Articles