Migrate SharePoint List From On-Premises To SharePoint Online

Introduction 

Hi guys. Sometimes we want to migrate a list from SharePoint on-premises to SharePoint online without using any third-party tools, by retaining the ID, Created, Modified, CreatedBy and ModifiedBy column values along with their versions. This article will help you to accomplish the above requirement.

We will use the Sharepoint migration API PowerShell commands for migrating the list from on-premises to online. To learn more, visit this link.

Prerequisites

• You should have an Azure account. If you don't have one, go and create the trial version for 30 days here
• Source List
• Destination List with the similar column as of source list 

These are the steps required to migrate an on-premises list online.

• Create a storage account in Azure.
• Export the on-premises list
• Import the list on SharePoint online.

Create a storage account in azure

Log in to the Azure portal by using this URL: http://portal.azure.com/ with valid credentials. 

Go to Storage and Click on add create link.

Enter the following value in required fields and click on the review + create button.

Now click on the create button once the validation has passed.

Now wait for some time until the deployment completes.

Once the storage account is created, navigate to newly created storageaccount listmigrationapiarvind-> Access key, and copied the key and account name and save it somewhere else. The storage account is listmigrationapiarvind and the key will be required to upload the source list content here.

Export the on-premises list

To demonstrate the export and import using migration API, I created the list 'MigrationList' with versioning enabled on SharePoint on-premises site.

I had also created the list on the SharePoint online site using save as list template functionality with version enabled.

A total of 5 items exist in the on-premises 'MigrationList' list.

The version history of Item ID 1 is:

Open the SharePoint PowerShell window and paste the following code:

 Export-SPWeb -Identity "Your Site Url" -ItemUrl "Lists/MigrationList" -Path "D:\Arvind\MigrationListContent" -NoFileCompression -IncludeVersions ALL

This command will export all the necessary files required for the SharePoint online site. The folder D:\Arvind\MigrationListContent will have the following files.

Import the list on SharePoint Online

We will import all the items of source list into SharePoint online using migration API. 

Firstly, we will convert the package into the required format for a SharePoint online site.

Note

I copied all the exported files to this path D:\Arvind\MigrateContent\MigartionList\sourceFile

Open SharePoint online management shell window on target site and paste this script. It will prompt for password, provide the password and press enter.

$userName = "Your target site user name"  
$cred = Get-Credential $userName  
$sourceFilePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
$sourcePackagePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
$outPutPackagePath ="D:\Arvind\MigrateContent\MigartionList\outputPackage"  
$targetWebUrl ="Your target webUrl"  
$targetListName ="Lists/MigrationList"  
  
ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath $sourceFilePath -SourcePackagePath $sourcePackagePath -OutputPackagePath $outPutPackagePath -TargetWebUrl $targetWebUrl -TargetListPath $targetListName -Credentials $cred -Verbose

where,

• SourceFilesPath : The content you want to migrate.
• SourcePackagePath : The content you want to migrate.
• OutPutPackagePath : Your tempory folder path.
• TargetWebUrl : Your target web url
• TargetListPath : Your targetList Name
• Credentials: Valid Credentials of your target site.

Once the package has been converted successfully, it will look like:

After converting the source files into required output package files, upload the files to an Azure storage account with these scripts.

#Upload the files and package to azure.  
$fileContainerName ="migratelistcontainer" #Not Necessary  
$packageContainerName ="migratelistpackage" #Not Necessary  
$azureAccountName ="listmigrationapiarvind"  
$azureAccountKey = "Your azure account key"  
$azureQueueName = "migratelistqueue" #Not Required  
  
$azureLocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFilePath  -SourcePackagePath $outPutPackagePath -FileContainerName $fileContainerName -PackageContainerName $packageContainerName -AccountName $azureAccountName -AccountKey $azureAccountKey -Overwrite -AzureQueueName $azureQueueName 

 Where:

• SourceFilesPath: Path of Source file i.e. SharePoint on-premises export files path.
• SourcePackagePath: Path of convert file i.e. Path of convert package.
• FileContainerName: (Optional) It is an Azure storage container name.
• PackageContainerName: (Optional) It is an Azure storage package container name.
• AzureQueueName: (Optional) It is an Azure queue name.
• AccountName: It is an Azure account name i.e. listmigrationapiarvind
• AccountKey: It is an Azure account storage key access i.e. "Your azure account key"

It will upload the package into Azure storage container. To check it, log into the azure site with valid credentials, and navigate to your storage account(listmigrationapiarvind) Containers.

The migratelistpackage contains the following files:

The command $azureLocations | flis optional but it help us to check the following file uri i.e. FileContainerUri, PackageContainerUri, ReportingQueueUri and FileContainerUploadUri 

Now submit the migration job with below command, so that it starts importing our on-premises list items into the targeted SharePoint online list.

Submit-SPOMigrationJob -TargetWebUrl $targetWebUrl -MigrationPackageAzureLocations $azureLocations -Credentials $cred 

Where:

• TargetWebUrl: Your target site Url i.e. Already created the variable in the above command which holds the target web URL value.
• MigrationPackageAzureLocations: Above, set the variable value.

It is an optional command, but helps us to monitor the migration job status. 

Get-SPOMigrationJobStatus -TargetWebUrl $targetWebUrl -Credentials $cred

Once the migration job processing completes, verify the target list items and their version history.

 

Note

You can check the job success or any job-related errors of submit a migration job on the Azure site queue.

To check the Azure queue, login to the azure site and navigate to your storage account (listmigrationapiarvind) then queue (migratelistqueue).

The whole script will look like:

# Export the onpremises list content  
Export-SPWeb -Identity "Your On Premises Site Url" -ItemUrl "Lists/MigrationList" -Path "D:\Arvind\MigrationListContent" -NoFileCompression -IncludeVersions ALL  
  
# create variable to hold value which requried to convert the source files  
########################################  
# Convert the .dat files into proper packages.  
  
$userName = "Your targeted site user Name"  
$cred = Get-Credential $userName  
$sourceFilePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
$sourcePackagePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
$outPutPackagePath ="D:\Arvind\MigrateContent\MigartionList\outputPackage"  
$targetWebUrl ="Your targeted site Url"  
$targetListName ="Lists/MigrationList"  
  
# Convert the Source package into target site.  
  
 ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath $sourceFilePath -SourcePackagePath $sourcePackagePath -OutputPackagePath $outPutPackagePath -TargetWebUrl $targetWebUrl -TargetListPath $targetListName -Credentials $cred  
  
# Create azure variables and upload the packages on azure.  
####################################  
#Upload the files and package to azure.  
$fileContainerName ="migratelistcontainer" #Not Necessary  
$packageContainerName ="migratelistpackage" #Not Necessary  
$azureAccountName ="listmigrationapiarvind"  
$azureAccountKey = "Your azure account key"  
$azureQueueName = "migratelistqueue" #Not Required  
  
$azureLocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFilePath  -SourcePackagePath $outPutPackagePath -FileContainerName $fileContainerName -PackageContainerName $packageContainerName -AccountName $azureAccountName -AccountKey $azureAccountKey -Overwrite -AzureQueueName $azureQueueName  
  
# Check the uploaded files Uri  
$azureLocations | fl  
  
############################################  
  
#Submit the job  
  
Submit-SPOMigrationJob -TargetWebUrl $targetWebUrl -MigrationPackageAzureLocations $azureLocations -Credentials $cred  
  
#Check the job status with this command  
Get-SPOMigrationJobStatus -TargetWebUrl $targetWebUrl -Credentials $cred  

You can also download the script from the attachment.

I hope you liked this article.