Not able to Unfollow sites that were followed prior to ADFS

We have SP 2013 on Prem, One of the user was following few sites prior to ADFS migration and after ADFS migration they are not able to unfollow them. However they are able to follow and unfollow new sites now. 

When I checked the LOGS I Found this issue “Could not stop following the item https://------.com/sites/xxxx: Exception: ItemDoesNotExist : Item does not exist”

I ran the below PowerShell script but it was able to unfollow few of them.

if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null) {

Add-PSSnapin "Microsoft.SharePoint.PowerShell"

}

$siteUrl = "https://xx/applications/Test"

$userName = "contosso\admin"

# Load site and user profile service

$site = Get-SPWeb $siteUrl

$serviceContext = Get-SPServiceContext($site.site)

$profileManager = New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager($serviceContext)

if ($profileManager.UserExists($userName)) {

# Load user profile

$userProfile = $profileManager.GetUserProfile($userName)

# Load social following manager

$followingManager = New-Object Microsoft.Office.Server.Social.SPSocialFollowingManager($userProfile, $serviceContext)

# Prepare following object

$actorInfo = New-Object Microsoft.Office.Server.Social.SPSocialActorInfo

$actorInfo.ContentUri = $site.Url

$actorInfo.AccountName = $userProfile.AccountName

$actorInfo.ActorType = "Site"

# Add to followed sites

$followingManager.StopFollowing($actorInfo)

}

Could you help me what else can be done.

Thanks in advance

Regards,

Shravan