Issue
Recently we started getting an error in CRM to BC integration where we use SSIS Integration toolkit for Microsoft Dynamics 365 from KingswaySoft and we have hosted this in Azure Data Factory. We were getting the following error:
- “SsisDBOperationGuid”: “0ea2f07f-56ee-404d-8762-8d38851f7cb7″,
- “SsisDBOperationId”: 64741,
- “OperationErrorMessages”: “2/9/2021 7:33:27 AM +00:00 : Data Flow Task:Error: Dynamics CRM Destination.Inputs[Input].Columns[invoiceid] has lineage ID 1611 that was not previously used in the Data Flow task.\r\n\n2/9/2021 7:33:27 AM +00:00 : Data Flow Task:Error: \”Dynamics CRM Destination\” failed validation and returned validation status \”VS_NEEDSNEWMETADATA\”.\r\n\n2/9/2021 7:33:27 AM +00:00 : Data Flow Task:Error: One or more component failed validation.\r\n\n”,
- “effectiveIntegrationRuntime”: “KingswaySSIS (North Europe)”,
- “executionDuration”: 16,
- “durationInQueue”: {
- “integrationRuntimeQueue”: 1
- }
Solution
While troubleshooting we found this type of error mostly comes when there are any metadata changes, so we updated all the packages to make sure they are using latest metadata from entity and rechecked all the mapping between CRM and BC. There are no errors reported in the data flow for the mapping earlier but we still remapped all fields but that did not work. Another step was to recreate package but that also did not help.
While troubleshooting we noticed that while running packages from VS it was working fine; all the data was integrated correctly without any issue but we were getting the same error again after deploying to Azure.
The next step was to remove some data flow and check on which particular data flow it was failing. While doing further troubleshooting, we deleted data flow which was updating data into CRM field, and only tried with the data flow which was transferring data from CRM to BC. After doing these changes we found that data was following correctly to BC from VS so we deployed the latest packages to Azure and executed triggers there as well to see if data was following correctly or not, and it worked!. Now from kingswaySoft toolkit point of view we were only using KingswaySoft’s Dynamics 365 Source component and NAV destination component, which was responsible to fetch data from CRM in our and pass it to BC.
Now the next step was to use another component which was a CRM destination component. But as soon as we added Dynamics 365 Destination component we started getting an error.
We discussed our issue with the KingswaySoft team and finally we were able to resolve this issue with their help by updating our KingswaySoft toolkit in development machine and KingswaySoft Azure Runtime to the latest release.
Summary
We solved this issue by trying different things which also helped us to narrow down this issue specific to Dynamics 365 Destination component.
I hope it will help someone!
Keep learning, Keep sharing !!