Native Assembly Programming in .NET

Introduction

MASM is maintained by Microsoft and is an x86 assembler that consumes Windows and Intel syntax to produce COFF executables. It is compatible with both 16-bit and 32-bit sources. Fortunately, Microsoft's Visual Studio IDE supports MASM programming just by making a couple of project property changes. The prime objective behind it is to introduce the power of assembly code in terms of speed and full control over programs that are typically not seen in other programming languages. There are even numerous editors and software available for such tasks in a standalone manner but the aspirant system or security programmers who are only limited to .NET software IDE so far can enter into the real system programming world by using, none other than the Visual Studio IDE.

Prerequisites

In this article, we will get an understanding of creating both an EXE and DLL using MASM with Visual Studio for the newbies that are supposed to have a brief knowledge of these technologies.

• Visual Studio 2010 or Later Version
• MASM (Microsoft Macro Assembler) SDK Library
• Basic Assembly Coding Competency
• VC++

Developing EXE using MASM

We shall show assembly programming by creating a simple Windows executable that typically shows “Hello World!” in a message box, the moment it is initiated likewise another example. It is very tricky to do such an implementation because the Visual Studio 2010 IDE doesn't offer any explicit templates for writing assembly code like the C#, VC++, and VB.NET programming languages. It has a built-in option to compile or run assembly programs.

Creating a New Project

We shall create a VC++ project solution that is later accompanied by an assembly code file. Hence, open Visual Studio and choose a VC++ Empty Project template type. There is no need to create a sub-directory for this empty solution, so uncheck the corresponding check box as in the following.

Once the test_masm project of VC++ is created, go to the Solution Explorer and right-click to choose the Build Customizations command as in the following.

The Build Customizations command opens the MASM compiler options that are unchecked by default. This is the key option that must be enabled to edit and compile native assembly code files.

Assembly coding

As we have stated earlier, Visual Studio 2010 doesn't provide assembly file templates, however, choose the project from Solution Explorer and right-click to add a text file that will provide a *.ASM extension as in the following.

Now, a blank text.asm file is added to our test_masm solution. Open it and paste the following assembly code that is responsible for displaying a message box as in the following.

The assembly code file is written but remains patient, this is not ready to be compiled or executed, because some important project settings are remaining.

Mandatory project configurations

Successful execution of an assembly code file with the Visual Studio IDE depends on an external library file that will be available from the MASM SDK. Hence, choose Project Properties by right-clicking it from the Solution Explorer. Here, choose General by expanding the Linker, and in the Additional Library Directories, insert the path of the include, lib, and macros directories as in the following.

Next, go to the Input section in the Linker and provide a reference for the masm32.lib file as additional dependencies.

It is not required to generate a manifest file for such manipulation, hence disable it as in the following.

Now, go to System from the Linker and set Windows in the subsystem section as in the following.

Finally configure the code entry point as a start from the Advanced option in the Linker, which determines the code execution flow. We can identify the entry point of the ASM file from the .code section.

Now go to the Microsoft Macro Assembly section from the solution properties that appear the moment we add an assembly file to the solution directory, otherwise it shall be hidden. Here, set the directory name where the MASM SDK is installed earlier as in the following.

Finally, everything is ready to compile the solution. If the entire configuration is correct, then the test_masm.exe file is created in the Debug folder of the solution.

Testing and Debugging

It is time to test the executable, the moment the exe is clicked, a “Hello World!” The message box would appear as in the following.

We can even debug the assembly code by inserting a breakpoint at a specific location and using the Register window in the Debug menu, we can observe all the CPU registers with corresponding flags as in the following.

We shall cover the advanced debugging of an application in the later articles. The following image shows the assembly code in debug mode which assists in understanding what is happening behind the scenes.

Although this section is not relevant to this article, just from a knowledge point of view, we can disassemble any C++ file to its corresponding ASM code. The Visual Studio IDE has a built-in Disassembly option that is very helpful to detect runtime bugs such as buffer overflow in the code via converting the source code file to an assembly code file as in the following.

Developing DLL using MASM

In the previous section, we saw how to create an EXE file using MASM with Visual Studio 2010. We can also develop a library (DLL) using MASM programming much like other technologies such as C#, VB, and C++. Therefore, the method can be utilized in the other client application that created DLL. The procedure for generating a DLL is nearly the same as for an EXE but requires some subtle configuration. First, we need to set the Configuration Type as DLL in the General section because now we are dealing with a DLL. Such modification can be done from the solution properties as in the following.

And as we all know, DLL files are libraries that contain methods. An entry point is typically absent from the DLL file. Hence we need to change this setting as in the following.

Finally, add a text file as masmlib with ASM extension in the solution such as earlier, and use the following code that typically contains a method testingMethod that will show some alert during load and unload of the DLL in the client program as in the following.

Finally, compile this program and a test_masm_dll.dll file will be created in the Debug folder that can be referenced in a C++ program or in the MASM client program itself.

Final note

So, we saw how to create both an EXE and a DLL file using the MASM programming language with the Visual Studio IDE. In fact, such a task could be done by hard-core MASM SDK but .NET programmers are typically afraid of assembly programming due to the strange syntax and platform. Assembly language programming opens a new horizon of advanced coding in terms of faster code execution, exploit writing and shell coding. Programmers are often comfortable with Visual Studio due to having numerous built-in features and functionality. Hence, this article is dedicated to those professionals planning to shift towards system programming by leaving the .NET Framework.