Configuration Files In Python

Do you even need config files?

If a developer has a module in their project that connects with an FTP server to download some files, then the developer would write a method to connect to FTP via FTP URL and use credentials like username and password for successful connection. If the developer uses these credentials in the code and hardcodes them in their code file and deploy the application, it could work fine, job done! Now, imagine that after two months the password for that FTP site is changed and the developer must again update that password in your application to make sure it does not break the existing functionality of FTP connection. Now in this scenario, the developer would again change the code and redeploy it. Here, a developer must for a small configuration change, take the latest of the code, do the required change, make sure nothing else breaks, re-deploy the application and then test it. And this could be a repetitive task after 2~3 months when some configuration again changes. What if the application has a configuration file that has key value pairs as “User”: “<username>”, “Password”: “password” and whenever some change is needed only that file is touched and configurations are updated rather than digging into the actual code.

Why config files?

Coming from a .Net background, I found it a bit challenging in figuring out how a developer can have a configuration file in the Python application which could be used to read setting values and one does not have to even touch the code to update or save settings. Config files are used to store key value pairs or some configurable information that could be read or accessed in the code and at some point, of time. If that configuration changes, developers can just change the configuration in that config file and not worry about changing the code as it may require re-compiling the code and deploying it. Not only this but using config files makes your settings and code more reusable and keep the settings information at a centralized location and segregated. Of course, sensitive information like passwords, secrets, and certificates should be kept more secure, maybe in cloud vaults. But basic settings used in the application could be part of a configuration file.

Microsoft.Net vs Python config files

Microsoft .Net applications sometimes by default provide files like appSettings, web.config, app.config when you start with a project template, based on what kind of application you are creating. These files serve the purpose of storing settings information.

Python applications do not by default provide a settings file, however, you can use an existing one and modify it. But the best way is to create your own from scratch and use it as needed.

Getting Started

In this article, we’ll have a walkthrough of how to create a config file, add a configuration, update a configuration, delete a configuration, and read a configuration in a python application.

We’ll use ConfigParser module to deal with config files and see how easy it could be to generate and read configuration files. Python can have config files with all settings needed by the application dynamically or periodically. Python config files have the extension as .ini.

We’ll use VS Code (Visual Studio Code) to create a main method that uses config file to read the configurations and then print on the console. This could be very new to a developer like me who has just started working on Python, so, we’ll start from scratch.

Pre-requisite

I’ll not go into details of installing Python and configuring VS Code to run a python application and assume that you have it. If not, I can post a separate article based on a request on how to get started with Python in VS Code

Create a Config File

Launch VS Code and create a main.py file

Open VS Code and create a new file and name it main.py

Start from Hardcoding.

Write a small code that reads hardcoded values and print those. To start with and for the sake of understanding I am first using hard coded values to print then later will fetch those from config file.

ftpUrl = "demoftp.codeteddy.com"
userName = "codeteddy"
password = "my#supersecret#password"
print("\nDisplaying FTP details\n")
print("FTP URL: " + ftpUrl)
print("FTP User Name: " + userName)
print("Password: " + password)

Run the code.

Now run the code to print the details. At the corner, there should be an icon to run your code and display the output on terminal.

So, the output is,

Displaying FTP details

FTP URL: demoftp.codeteddy.com
FTP User Name: codeteddy
Password: my#supersecret#password
PS D:\Articles\Python_ConfigFile\Code>

Our goal is to make sure that the output remains same, but the values of FTP settings are read from a config file.

Code to generate config file.

In VS Code, create a new file named generate_config.py and import configparser module as shown below,

import configparser

Now put the following code into the file,

import configparser

# CREATE OBJECT
config_file = configparser.ConfigParser()

# ADD SECTION
config_file.add_section("FTPSettings")
# ADD SETTINGS TO SECTION
config_file.set("FTPSettings", "ftpUrl", "demoftp.codeteddy.com")
config_file.set("FTPSettings", "userName", "codeteddy")
config_file.set("FTPSettings", "password", "my#supersecret#password")

# SAVE CONFIG FILE
with open(r"configurations.ini", 'w') as configfileObj:
    config_file.write(configfileObj)
    configfileObj.flush()
    configfileObj.close()

print("Config file 'configurations.ini' created")

# PRINT FILE CONTENT
read_file = open("configurations.ini", "r")
content = read_file.read()
print("Content of the config file are:\n")
print(content)
read_file.flush()
read_file.close()

So, we import configparser module that contains methods and classes which could be used to define a layout of the config file that we need. We create an object of Configparser and name it config_file. A config file can have sections against which the details would be stored i.e., FTP Settings or Logger settings, etc. So, we created one section named “FTPSettings” and added key value pairs to that section via config_file.set method which takes first argument as section name, second as key and third as value. In our case section name is “FTPSettings” and three keys would be ftpUrl, userName and password.

We then save the layout defined in an ini file named configurations.ini. You can give the file name of your choice where the settings would be saved. Then at last we are retrieving the content of the saved ini file and printing those on the terminal window or console.

Once you save this file and hit on run, it will generate a configurations.ini file.

Generated file.

If we open the configurations.ini file i.e., the newly generated file, we see the following settings in place,

[FTPSettings]
ftpurl = demoftp.codeteddy.com
username = codeteddy
password = my#supersecret#password

It contains the section FTPSettings and our FTP settings as key value pairs under it. Each time the code for generate_config is run, it will create a fresh config file with the layout as defined in the code file.

Let’s add one more section to the file. But this time we’ll add it in a different way. For e.g., if we need to add a section for Logger and its corresponding settings, we can also use the following code to do so in a single code statement,

# ADD NEW SECTION AND SETTINGS
config_file["Logger"]={
        "LogFilePath":"<Path to log file>",
        "LogFileName" : "<Name of log file>",
        "LogLevel" : "Info"
        }

Now, the generate_config.py looks like as follows with the newly added section,

import configparser

# CREATE OBJECT
config_file = configparser.ConfigParser()

# ADD FTPSettings SECTION
config_file.add_section("FTPSettings")
# ADD SETTINGS TO FTPSettings SECTION
config_file.set("FTPSettings", "ftpUrl", "demoftp.codeteddy.com")
config_file.set("FTPSettings", "userName", "codeteddy")
config_file.set("FTPSettings", "password", "my#supersecret#password")

# ADD NEW SECTION AND SETTINGS
config_file["Logger"]={
        "LogFilePath":"<Path to log file>",
        "LogFileName" : "<Name of log file>",
        "LogLevel" : "Info"
        }

# SAVE CONFIG FILE
with open(r"configurations.ini", 'w') as configfileObj:
    config_file.write(configfileObj)
    configfileObj.flush()
    configfileObj.close()

print("Config file 'configurations.ini' created")

# PRINT FILE CONTENT
read_file = open("configurations.ini", "r")
content = read_file.read()
print("Content of the config file are:\n")
print(content)
read_file.flush()
read_file.close()

When we run the code, we see that our config file is updated with these new details,

[FTPSettings]
ftpurl = demoftp.codeteddy.com
username = codeteddy
password = my#supersecret#password

[Logger]
logfilepath = <Path to log file>
logfilename = <Name of log file>
loglevel = Info

So, we can use either of the ways i.e. add_section() and set method or object initializer way to create section and add settings to it.

Adding/Updating/Deleting configuration file settings

We can add or update or delete the settings in the config file via code as well.

Adding/Updating settings in the config file

For e.g. If we need to update the “loglevel” from “Info” to “Debug”, or we need to add a new setting to the Logger section. Either you can update the setting in the same generate_config.py file and run it or you can write code to read the existing configuration file and update new settings. For e.g. I created a new python file named update_config.py and added following code to it,

import configparser

# CREATE OBJECT
config_file = configparser.ConfigParser()

# READ CONFIG FILE
config_file.read("configurations.ini")
 
# UPDATE A FIELD VALUE
config_file["Logger"]["LogLevel"]="Debug"
 
# ADD A NEW FIELD UNDER A SECTION
config_file["Logger"].update({"Format":"(message)"})
 
# SAVE THE SETTINGS TO THE FILE
with open("configurations.ini","w") as file_object:
    config_file.write(file_object)
 
# DISPLAY UPDATED SAVED SETTINGS
print("Config file 'configurations.ini' is updated")
print("Updated file settings are:\n")
file=open("configurations.ini","r")
settings=file.read()
print(settings)

Here in this code, we first import, create the object of configparser. We read the existing configuration file and then set the LogLevel from Logger section to “Debug”. Likewise, we added one more setting under Logger section via update method by this code: 

config_file["Logger"].update({"Format":"(message)"})

 This code adds a new setting named “Format” with value as “(message)” under Logger section. In the end, we again save the file and then again read it to display the saved information.

Now when you run the code, the output is

Config file 'configurations.ini' is updated.

Updated file settings are,

[FTPSettings]
ftpurl = demoftp.codeteddy.com
username = codeteddy
password = my#supersecret#password
[Logger]
logfilepath = <Path to log file>
logfilename = <Name of log file>
loglevel = Debug
format = (message)

So, these are the ways, you can update the ini file.

Deleting the settings

We can remove the settings from config files using remove_option() and remove_section() module in configparser module. remove_option() is used to delete a field (i.e., key value pair) from any section and remove_section() is used to remove a complete section of the config file.

# DELETE A FIELD IN THE SECTION
config_file.remove_option('Logger', 'Format')
 
# DELETE A SECTION
config_file.remove_section('Logger')

Reading the configuration file in Python

Now that the file is created, and we have learned how to generate and update the config file when needed, let’s visit the last section of this article i.e., reading the values from config file in our main class. This is the most simple and straightforward part.

Let’s make it a bit organized and reusable. Let’s add a helper file in our VS Code workspace and name it helper.py and add the following code to it,

import configparser
# Method to read config file settings
def read_config():
    config = configparser.ConfigParser()
    config.read('configurations.ini')
    return config

Now this helper method code will serve as reusable method wherever we need the configurations.

Now. Go to the main.py file and import this helper file as

Now, in the main.py, replace the hardcoded pieces with reading the settings from config file as,

import helper
config = helper.read_config()
ftpUrl = config['FTPSettings']['ftpUrl']
userName = config['FTPSettings']['userName']
password = config['FTPSettings']['password']
print("\nDisplaying FTP details\n")
print("FTP URL: " + ftpUrl)
print("FTP User Name: " + userName)
print("Password: " + password)

And run the code. We’ll see that our output is same as when it was reading the values from the hardcoded code.

Now it reads from the config file.

• Complete Source Code