Mission Impossible - Migrating .NET Core 1.x To 2.0

Introduction

Everybody is praising the new edition of .NET Core framework, and everybody is so interested in publishing new articles on .NET Core, ASP.NET Core, or the Entity Framework Core. Yeah, I get that but what about migration from the older editions? I personally have had a very tough time with the migration of my ASP.NET Core 1.1 application to ASP.NET Core 2.0 application. There are, of course, a lot of changes, improvements to the system but, they don't even matter if my application cannot be upgraded to utilize them, or if it breaks the system itself. I was annoyed, so I asked myself, instead of doing an article for, "How to create an ASP.NET Core 2 application", why should I not do the, "Upgrade your existing ASP.NET Core 1.1 applications to ASP.NET Core 2". This seems more legit, and there's a need for this sort of article. 


Figure 1: Afzaal Ahmad Zeeshan reporting for duty — btw, my belly is not this much out of order. :D

In this article, I will summarize everything that you might require to update in your ASP.NET Core 1.1 applications, to bring the improvements of ASP.NET Core 2 applications. I won't focus on ASP.NET Core itself, but the .NET Core framework, which means that I will talk about Entity Framework Core as well and a few of the packages that must be targeted through NuGet galleries. 

There were some common changes made to the API set, which I am going to talk about here, but note that I will be talking specifically about a specific few changes made, if your application has a lot of changes applied and requires some manual configuration, you may want to dig a bit deeper, but this article will intend to solve 95% of problems faced during migrations. 

1. .NET Core packages upgrade
   
   
   1. Upgrading the packages from .NET Core 1.1 to .NET Core 2.
   2. NuGet support here
   3. Understanding what and how changed
      
      
2. NET Core upgrade
   
   
   1. Changing the authentication system
   2. Understanding new elements in ASP.NET Core 2
   3. Testing performance — I will not do this here, I have a separate post for that, load testing thing.
   4. Startup script etc.
      
      
3. Entity Framework Core tweaks 
   
   
   1. Understanding Entity Framework Core 2, and how it changed
   2. Applying changes to the system
   3. Adding migrations to the database through DbContext
      1. There is a change in the service calls, which makes it tough.
      2. I will walk you through those changes
   4. Applying migrations, understanding where changes were made.

The default application that I have now is expected to demonstrate the typical changes in the ASP.NET Core authentication system, Entity Framework Core tips and tricks and how to upgrade the .NET Core 1.1 to .NET Core 2.0. So, by the end of this post, you will know what are the tools you require to take control of this fast ride. 

.NET Core upgrade

.NET Core itself can be programmed through command-line tools as well, but if you use Visual Studio then the project upgrades are simple and straightforward enough, however for other cases you might have to upgrade some of the csproj file content because there was a statement that the csproj files no longer have gibberish stuff in them and instead are really packed with what the project contains. Thus, for those who are using terminal based environments, it might be a bit tough. Or maybe you may need to reconfigure a lot there, however since I am going to use Visual Studio, in my case the only thing that I must update is the Properties tab. Before that, I want to show the csproj file content, 

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netcoreapp1.1</TargetFramework>
    <PackageTargetFallback>$(PackageTargetFallback);portable-net45+win8+wp8+wpa81;</PackageTargetFallback>
    <UserSecretsId>aspnet-SampleAspNetCore11-A98F289E-EEAA-438F-A1D7-5C3A8E54DB2A</UserSecretsId>
  </PropertyGroup>


  <ItemGroup>
    <PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.0.0" />
    <PackageReference Include="Microsoft.AspNetCore" Version="1.1.2" />
    <PackageReference Include="Microsoft.AspNetCore.Authentication.Cookies" Version="1.1.2" />
    <PackageReference Include="Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore" Version="1.1.2" />
    <PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="1.1.2" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="1.1.3" />
    <PackageReference Include="Microsoft.AspNetCore.StaticFiles" Version="1.1.2" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="1.1.2" PrivateAssets="All" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="1.1.2" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer.Design" Version="1.1.2" PrivateAssets="All" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="1.1.1" PrivateAssets="All" />
    <PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="1.1.2" />
    <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="1.1.2" />
    <PackageReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Design" Version="1.1.1" PrivateAssets="All" />
    <PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="1.1.2" />
  </ItemGroup>

  <ItemGroup>
    <DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="1.0.1" />
    <DotNetCliToolReference Include="Microsoft.Extensions.SecretManager.Tools" Version="1.0.1" />
    <DotNetCliToolReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Tools" Version="1.0.1" />
  </ItemGroup>

</Project>

In most cases, you would have to modify the field, TargetFramework and set the value to target .NET Core 2.0 and then upgrade the package version (see the packages above), which would refresh the .NET Core targeting for you. But do not forget, a lot of things have changed here, thus I recommend reading the csproj file reference to understand how to do this yourself. 

<TargetFramework>netcoreapp2.0</TargetFramework>

After this, a simple dotnet restore or dotnet build or dotnet run would do the thing for you. Universally, you can execute the following command and it would do everything for you.

$ dotnet run

Package restoring is implicit now, and is done for you by .NET Core command-line tools — same as it is done by Visual Studio — and then the project is built, and started for you right away. Now let me update the project from .NET Core 1.x to .NET Core 2.0 and see what has changed, 

Figure 2: Changing .NET Core edition in Visual Studio Properties tab.

After changing the type, you will notice (and I will not show here in a duplicated way), that now the csproj file still targets our older binaries and the indexers have not been upgraded. But, we have NuGet package manager for our help, and thus we can use that tooling to upgrade the project packages. Before this, you will see that your NuGet packages now show a warning message, 

Figure 3: Dependencies showing warnings after changing .NET Core edition.

Just follow the following steps, and your .NET Core would have been migrated from 1.x to .NET Core 2.0, 

1. Open the NuGet package manager
2. Go to the Updates section
3. Select all of the packages (I am not providing any support for non-.NET Core packages at the moment, you may remove them from upgrades)
4. Update them all

It takes a while, as most packages need to be removed and other packages need to be installed, thus be patient. 

Figure 4: Updating the NuGet packages to upgrade .NET Core from 1.x to 2.0 in Visual Studio.

Congratulations, by now your application will be targeting .NET Core 2.0. You can then later check the runtime information and other related stuff as needed. Now the framework, which you are targeting has been updated and if you were only using .NET Core and not the rest of the stuff such as ASP.NET Core, then your problem has been solved as the rest of the things are related to ASP.NET Core, but the problem doesn't get solved right there for web developers, so those of you folks please continue reading below. 

ASP.NET Core upgrade

Now coming down to a bigger picture here, ASP.NET Core requires some more changes, as compared to the .NET Core upgrade, there are a lot of breaking changes in the API of ASP.NET Core. Some changes are the renaming of the components, then there are some changes which highlight that some services are deprecated and will be removed soon, some are minor performance improvements, but the change in the function signature makes it break the build system. 

Thus, this section will primarily talk about the changes in ASP.NET Core 2, that are required to be reflected in ASP.NET Core 1.1 applications, to properly run them, let me summarize the changes into sections, 

API changes 

There have been several changes to the ASP.NET Core framework as itself, a lot of improvements to the way your application's pipeline was designed to bring more performance, and there were new ways that you can build and deploy your web applications. In the announcement page for ASP.NET Core 2, all of the improvements were mentioned which you can review from there. Let me just give you a warm-up, and then we can continue with what I was doing. 

Razor Pages are back

I get it, Razor was always here, MVC uses Razor and all... But, here you get to skip the controllers and all. The thing is, those who used to develop using the Microsoft WebMatrix tool, would remember the days of ASP.NET Web Pages, the framework which did not require most of the complex business logic and security abstractions of a web application. That framework kind of looks like is getting back now. 

1. Pages will be short, compact and would contain only their code.
2. Pages section is defined for this purpose.
3. Pages would contain integrated functions to handle GETand POST requests, etc. 

I still must figure out these pages and will write about them once I am done with some other important things at the moment. 

C# 7 features support in Razor Pages

It was also mentioned that all the recent updates of C# 7 (7.1 was named in that official post), as a language are also integrated in the Razor Pages, which means that you can use and utilize all the features right inside those pages. The reason for that was, that since there is no controller or other business-logic managing class backend, you must write the code in the page itself, thus providing these tools would be better, and thanks to Roslyn, we have it. 

However, I'd like you know... This post won't be covering anything new, but just how to make sure your older applications are upgraded — if you want to upgrade them at all!

Solving the problems 

Now let us take a brief look at the problems, which we can solve here. The following image shows the main problems, which you are going to face... Of course, there are several coming when we have added these fixes and patches since the API once integrated requires a lot of further changes. 

Figure 5: Build system breaking after application of changes to the project.

I know you must be thinking, that it only requires adding some using statements, but it is not as simple as it sounds. 

There are a lot of changes in the API, I was lucky enough that I was using the defaults in my own web application, and I did not need to make many modifications, which meant that I knew what was going wrong and where, thus applying changes was an easy task. But what you would be doing would depend entirely upon how far you have taken your web application. The IdentityCookieOptions, which were used in the external logins for ASP.NET Core have been changed, and instead they are now IdentityConstants, exposing the values of the types of cookie storage or login mechanism and principals and much more. So, in cases where you have written, 

public AccountController(
            UserManager<ApplicationUser> userManager,
            SignInManager<ApplicationUser> signInManager,
            IOptions<IdentityCookieOptions> identityCookieOptions,
            IEmailSender emailSender,
            ISmsSender smsSender,
            ILoggerFactory loggerFactory)
{
    _userManager = userManager;
    _signInManager = signInManager;
    _externalCookieScheme = identityCookieOptions.Value.ExternalCookieAuthenticationScheme;
    _emailSender = emailSender;
    _smsSender = smsSender;
    _logger = loggerFactory.CreateLogger<AccountController>();
}

It becomes, 

public AccountController(
            UserManager<ApplicationUser> userManager,
            SignInManager<ApplicationUser> signInManager,
            IOptions<IdentityConstants> identityCookieOptions, // <-- Here
            IEmailSender emailSender,
            ISmsSender smsSender,
            ILoggerFactory loggerFactory)
{
    _userManager = userManager;
    _signInManager = signInManager;
    _externalCookieScheme = IdentityConstants.ExternalScheme;  // <-- Here
    _emailSender = emailSender;
    _smsSender = smsSender;
    _logger = loggerFactory.CreateLogger<AccountController>();
}

Even better can be, that you remove the parameter from constructor because we are now passing a value from the IdentityConstants. But that is just because I have not used it, you may have used the parameter for other purposes which is why I did not remove it. 

There were some changes in the Entity Framework Core API as well, for ASP.NET Core. Previously, the following, 

public class ApplicationUser : IdentityUser

Which came from Microsoft.AspNetCore.Identity.EntityFrameworkCore, now it has been changed and now it comes from Microsoft.AspNetCore.Identity. Not just these, but there are several other objects that got migrated to this namespace, such as IdentityRole. I believe, it was a decision to bring everything under Microsoft ASP.NET Core Identity framework and remove the concept of Entity Framework Core backing up the feature or service, a good decision I would say because why would IdentityRole name depend on EntityFrameworkCore at all, right? So, making this change fixes the problem. That said, now the final problems are that most of the APIs have either been deprecated or have been improved. I am nobody to say anything, in this context because I have not yet run a full test on a number of performance improvements, which ASP.NET Core 2 brings, but of course, there are major solutions and even their verbal explanations give you a notion of the fact that they are improvements to the framework. Same goes for deprecation, we can see why the team wanted to make such changes. 

First, let me fix the improvement thing, the improvement thing is the one where your function returned a type, and not it returns a Task<type>. 

_signInManager.GetExternalAuthenticationSchemes()...

What I did was I tried to look into the definition of the class, SignInManager and there I figured the definition had changed just a bit, 

namespace Microsoft.AspNetCore.Identity
{
    //
    // Summary:
    //     Provides the APIs for user sign in.
    //
    // Type parameters:
    //   TUser:
    //     The type encapsulating a user.
    public class SignInManager<TUser> where TUser : class
    {
        // Rest of the stuff...
        //
        // Summary:
        //     Gets a collection of Microsoft.AspNetCore.Authentication.AuthenticationSchemes
        //     for the known external login providers.
        //
        // Returns:
        //     A collection of Microsoft.AspNetCore.Authentication.AuthenticationSchemes for
        //     the known external login providers.
        [AsyncStateMachine(typeof(SignInManager<>.<GetExternalAuthenticationSchemesAsync>d__46))]
        public virtual Task<IEnumerable<AuthenticationScheme>> GetExternalAuthenticationSchemesAsync();

        // Rest of the stuff...
}

Thus, just the way we know what happens here is that we apply async/await pattern to get the result in an asynchronous way, so what I did here was, 

(await _signInManager.GetExternalAuthenticationSchemesAsync())...

The rest of the code also required some changes, such as the fields AuthenticationScheme required to be changed, as it had been removed. In the default ASP.NET Core code, there is a change in the ViewModel object itself and it accepts IList<AuthenticationDescription> and not the List<AuthenticationDescription>, so that also requires a casting of types. After these changes, the only change required left would be the deprecation. 

In most cases, the compilers are configured to consider deprecation as errors instead of warnings. Because, developers are not afraid of warnings, but they are terrified by an error... No matter how major or how small it might be. Thus, in cases where a team has configured deprecation to be considered as an error, the following would solve it. What happens is, that, HttpContext.Authentication is about to go deprecated and would be removed in future updates, 

await HttpContext.Authentication.SignOutAsync(_externalCookieScheme);

This shows the warning, and would soon show an error and later a build error if not taken seriously. The warning message states clearly, that you should consider the extension methods provided to take care of the signing out process for a user. 

// It has been taken back from HttpContext, and instead goes to Microsoft.AspNetCore.Authentication
// HttpContext gets passed as a parameter
await Microsoft.AspNetCore.Authentication.AuthenticationHttpContextExtensions.SignOutAsync(HttpContext);

// Or more concisely it can be rewritten as,
using Microsoft.AspNetCore.Authentication;

// Function call
await AuthenticationHttpContextExtensions.SignOutAsync(HttpContext);

This solves that, and the previous line can be commented out if not removed. You should consider updating this everywhere your application was using the code to sign out the user and make the changes based on authentication. After this, the Startup class configuration also requires some updates, and like we have been watching, the app.UseIdentity() has gone deprecated as well, and the recommended alternate to that one is, app.UseAuthentication(). Such other minor changes, but API changing changes, and build breaking changes were implemented in ASP.NET Core 2.0. 

In this sample, only this change was detected, your application might have huge scales of changes and they would all be like this one. The purpose was just to give you an idea of what to expect to change and break. Thus, after these changes — and provided I did not have other changes in the application, these were the only one required. Now I can continue to the final change in this post, the Entity Framework Core 2.0. 

Entity Framework Core upgrade

In this section, I want to talk about the improvements in Entity Framework Core, as well as the improvements and their effects on other frameworks, such as the case of ASP.NET Core pooling for DbContext objects. I will tell you about that, but again I did not load test any of this so I cannot provide the accurate metrics for any of this now, but soon I will. 

Improvements

First let us talk about the improvements, the major one that I like was the ability to add DbContext objects as a pool object, I skipped it from ASP.NET Core section because that section already had a lot to talk about, anyways, what this upgrade states is that you can now add DbContext, as an object to DbContext pool. This way you would be provided with an object which exists in the pool, instead of recreating the object each time a request comes. 

services.AddDbContext<DbContextName>(options => options.UseSqlServer(connString));

This was used to register your DbContext to the DI, and upon each request ASP.NET Core would itself create the object, and then pass it to your constructor. That was fine and worked very excellent, but as you might have guessed for a thousand requests, thousand times object instantiation and Dispose calls unless you configured an alternate way, such as a Singleton or what-so-ever. However, now ASP.NET Core 2 supports a pooling for DbContext, which means it creates the object instance once and reuses the object from pool, and your code becomes, 

services.AddDbContextPool<DbContextName>(options => options.UseSqlServer(connString));

How much performance boost it might give you, is not mentioned because it depends on several factors and thus stating any of them would be useless. 

Also, there was a statement about LIKE clause of SQL to be provided and supported in Entity Framework LINQ queries, something like this would be possible now, 

var posts = from p in dbContext.Blogs.Posts
            where EF.Functions.Like(p.Name.ToLower(), "How to%"); // Lower case, pattern matching.
            select p;

But that could have been done by the older ways of string management in .NET framework, like

var people = from p in dbContext.Blogs.Posts
             where p.Name.ToLower().StartsWith("How to"); // Lower case, pattern matching.
             select p;

The older versions were doing all of this in memory, and hopefully were never converted to SQL LIKE clauses (I might be wrong here), whereas the new EF.Functions.Like(...) gets translated to SQL LIKE clause and the work is done by database server instead of in-memory processing anymore, which would definitely improve the performance. 

Problems and breaking changes

Anyways, moving onwards to the changes in Entity Framework Core pipelines, I am going to talk about the problems that you will face while upgrading ASP.NET Core 1.x to ASP.NET Core 2.0. If you have been following up with .NET Core team on GitHub, you will notice that they have mentioned clearly in their "breaking-change" issue label, that there are a lot of changes to the API which would break the build system and to fix them you would have to rewire a lot of things every here and there. For example, the currently open breaking changes in Entity Framework Core repository are, Issues - aspnet/EntityFrameworkCore. Feel free to revisit them to learn more, and to find out what else you were using and which broke during this upgrade. 

First and foremost of the changes, which requires being fixed is that now Entity Framework Core requires a default contractor. It was mentioned in one of the issues that Entity Framework Core ignores the DbContextOptions in the constructor because it no longer follows DI chain and expects a Factory pattern to be implemented, too awkward! 

#8328 issue

The most noticeable change is that we're going to stop attempting to call Startup.ConfigureServices(). This means that if your DbContext derived type doesn't have a default constructor or isn't in either your startup or target project, you'll need to add an implementation of IDbContextFactory<TContext>.

I could have followed the new design and gone onwards with the flow but my existing application was based on the older version. Most of you would be tempted to create an IDbContextFactory<TContext> class and implement it from there, but what I did was something else, I provided a compile-time and design-time implementation of a default constructor so that my code compiles, the migrations get applied and then I remove that to go back to the same DI service. 

Initial state of BlogContext : DbContext

I did not use a sample to demonstrate, but the following was the context which I just came up with, 

namespace ApplicationToMigrate.Data
{
    // DbContext class
    public class BlogContext : DbContext
    {
        // DbContextOptions based constructor, for DI through
        // services.AddDbContext<BlogContext>(...);
        public BlogContext(DbContextOptions options) : base(options) { }

        public DbSet<Blog> Blogs { get; set; }
    }

    // Type declaration
    public class Blog
    {
        public string Id { get; set; }
        public string Title { get; set; }

        public List<Post> Posts { get; set; }
    }

    public class Post
    {
        public string Id { get; set; }
        public string Title { get; set; }
        public string Content { get; set; }

        // Navigations
        public Blog Blog { get; set; }
        public string BlogId { get; set; }
    }
}

I was going to apply a migration, because I wanted to add another field, PublishedOn to the post (you understand this is just a demo, right?) and that would require a migration, but since Entity Framework Core does not call the ConfigureServices function, dependency injection will no longer be used and it would complain that you must provide a default constructor. The following was the line, 

public DateTime PublishedOn { get; set; }

Now, what I searched and found was that you can add a factory pattern, which returns an instance of the DbContext, on the official documentation pages, there is a sample provided, I captured following code from there, 

namespace MyProject
{
    public class BloggingContextFactory : IDesignTimeDbContextFactory<BloggingContext>
    {
        public BloggingContext Create(string[] args)
        {
            var optionsBuilder = new DbContextOptionsBuilder<BloggingContext>();
            optionsBuilder.UseSqlite("Data Source=blog.db");

            return new BloggingContext(optionsBuilder.Options);
        }
    }
}

As you can see, this works. However, this requires that in case of continuous integration toolchain, you must skip the DI and start using this pattern. If you consider it to be a useful method, stick to it, however, I did not like it and so I wanted to apply a quick hack and get my system up and running as it was, and to make sure that Entity Framework Core gets a grip of instance too. So, I made some changes, which are discussed below. 

Updating the BlogContext

What I wanted to do, was to maintain the same DI chain, and to make sure that my Entity Framework Core thing works, I just provided a default constructor and by overriding the OnConfiguring constructor added the same service of adding migrations through Add-Migration and Update-Database command. Following was my solution, 

namespace ApplicationToMigrate.Data
{
    public class BlogContext : DbContext
    {
        public BlogContext(DbContextOptions options) : base(options) { }
        public BlogContext() { } // Default constructor

        // Needed by EF tools
        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlServer("Server=(localdb)\\mssqllocaldb;Database=aspnet-ApplicationToMigrate-3C3829CB-5A83-4BD3-8141-288E09E3FF88;Trusted_Connection=True;MultipleActiveResultSets=true");
        }

        public DbSet<Blog> Blogs { get; set; }
    }

    // Types are similar, as shown above.
}

Before you start to bash me up, I want to state that you do not need to add this code here, if you are using version control this code can be removed without breaking anything because it is not required by application. Let me enlist the points, most of you would not get the point and start to bash, 

1. The default constructor and overriding the function is not required by ASP.NET Core 2.0. 
2. They can be removed as soon as you have applied migrations. 
   1. Same is the case for your connection strings, they are not needed in the code.
   2. You can still follow the jsonway of storing settings, or consider other means of storing connection string easily (such as Microsoft Azure App Service default values, for connection strings etc.), because on the runtime it is your Startup class maintaining everything. 
3. The problem was just with Entity Framework Core not being able to capture this instance from Startup class and in ASP.NET Core 1.x only.
4. In the ASP.NET Core 2.0, you can still use the same code and you are not required to provide a default constructor, because Entity Framework Core team updated their toolchains and manage how the services are created in pipeline and act as needed. 

Which brings us back, to the claim that in most cases you should consider migrating the pipeline of functions to ASP.NET Core 2.0 as well, nonetheless, if you are so much backed up with ASP.NET Core 1.x then there is less hope for you, let’s hope soon .NET Core team publishes a script to upgrade .NET Core 1.x apps to .NET Core 2 applications, hierarchically moving down to ASP.NET Core and Entity Framework Core to provide a wonderful experience for users. 

Final words

The upgrade suggestions which I had found are shared in this post, and hopefully, if you ever want to apply these changes this post will help you, I will keep an eye on other problems and issues related to .NET Core migration, I will share them here. In case you find a few areas which I missed to cover, please do let me know and the changes which I have told here are opinions, these may not be the best approaches — such as that default constructor thing in DbContext. 

If your application still does not work or does not build itself, follow the steps, 

1. Try to Google for the problem, chances are that you will find a discussion thread on GitHub. 
2. Apply those changes and see what happens. 
3. Comment here and I will try finding the solution as well and post it here. 
4. If nothing works out, you may be required to create a new project with ASP.NET Core 2.0 template and migrate everything manually, unless they come up with a script for migration. 

Pardon me for a while, I will present a separate test-based alternative soon, which will talk about how much of improvements benefit you get from these changes. ASP.NET Core load testing, Entity Framework Core SQL profiler and much more can be covered but in this post my focus was to make sure that you can know how to solve these common problems, as soon as possible. I understand this  because I also had to undergo this.