Post
Ask Question

Building A Questionnaire Site With Node.js

Introduction and Background

This was our first project with Node.js, and seriously, it was a pain. Node.js is an excellent framework with blazing fast speed and an intuitive development framework. For those who are familiar with HTML, CSS, and JavaScript, Node.js is one of the best alternatives to PHP, ASP.NET, or other server-side scripting languages that require a language to be learned. Node.js is light-weight, resilient and community-led and in-development framework. This article is primarily focused on teaching the beginners in Node.js development. You can expect to learn a few basics of Node.js setup, how to manage the databases in the Node.js environment, and a bit more (less too, maybe). 

What we had thought of was an app to build forms, and then have them filled... A simple survey, with a bit of friendly topping. Just to refresh some things, we skipped ASP.NET and tried Python (first) and then moved on to Node.js for simplicity and ease. Our project had some requirements, a few of which we were able to fulfill and some we skipped for future releases. To keep the things on track, the following are a few features of the app.

  1. Templating of the layouts
    1. Pug templating framework was used for this.
    2. Most of the pages are dynamically generated and contain variables.
    3. Bootstrapping was used for responsiveness.
  2. Creation of forms
    1. The forms are simple — no kidding.
    2. Variable controls the number of questions to show on the form, providing an easy interface to change the number of question fields and to configure later. 
    3. Each question holds 4 options (multiple-choice type questions).
      • Support of other types of questions was not worked out in the current release. Maybe someday later. 
    4. Rendering the questionnaire is also done using Pug framework.
  3. js runtime for server
    1. Express framework to support some basic functionality.
    2. SQLite databases (you can port other databases as well).
      • The framework doesn't have ORMs, so you are free to update that content later on.
      • We hardcoded a lot of stuff, and are expected to update this to reflect the ORMs — we didn't find any suitable, Sequelize was tried... But, we are still looking. 
    3. Routing to map the URLs to the functions
    4. We did not use controllers and other patterns because the app was simple. But we have learned that it would be kajillion times better to use some patterns and, yes, leave JavaScript. 
  4. Heroku platform
    1. I am a Microsoft guy, but since we wanted a fresh start, I thought we need to give someone (something) else a try. 
    2. Heroku struck me because I did create an account with Heroku around 4 years ago. 
    3. They are quite good, and since I am on their free platform, this is enough for me. 

Now that you know what this article is going to cover, and what you are going to learn in this article, let us just begin and start to share the insights on the development process and overall learning curve that we had to slide over. 

Lastly, you are however required to have a basic understanding of web standards. It would be amazing if you are familiar with the basics of JavaScript, how JavaScript scripting (or modules, .js files) works. Also, since we are working with SQLite databases it would be nice for you to understand relational databases, and how SQLite works (it is not that much tough), and we will try giving a few basics of the process we are working on. 

Node.js setup

Node.js is a JavaScript framework for server-side programming. The installation and setup of Node.js are just straight-forward, you can do it in a number of ways. I will show you the setup that worked for us, the default VS tools for Node.js didn't work out quite well and had some modifications to be done. 

Installation and setup

If you would like to use Visual Studio (non-Code editions), then you can install the Visual Studio tools for Node.js, and most of the things would be set up for you. Visual Studio will also start to show a few Node.js project templates to get started quickly. 

Visual Studio providing templates for Node.js runtime
Figure 1: Visual Studio providing templates for Node.js runtime.

However, you might have to configure a few other things such as the default location for Node.js runtime (node.exe on Windows). There were other things to take care of, such as the mismatching of the runtimes and conflicts. The simple and easy way of doing this was to just use a separate runtime. 

As a second option, you can install the standalone Node.js runtime from their official website. Typically you would select LTS version and not the in-development and mainstream version. There are several reasons for this, 

  1. LTS versions are battle-tested, for production environments.
  2. They will be providing updates and same features as the mainstream versions.
  3. They, however, contain the stable features for production environments. 

You can, however, select other versions if you know what you doing. Standalone Node.js installer contains the NPM packages and other stuff... Also, note that Node.js installer will also set up the PATH for your Node.js (which is one of the best things a framework installer can contain!). Also note that if you want to develop using Visual Studio Code edition, then you have to consider the second option — because Visual Studio tools for Node.js is not available in that lightweight IDE. The second option is also cross-platform so you can setup the environment on Windows and continue on Linux and macOS. This will be helpful in most cases, such as when you want to target a community-oriented project where Visual Studio (non-Code edition) is not an option. 

Lastly, on Visual Studio Code you can set up the extensions for Node.js runtime and debugging. For that, consider reading this full documentation for Node.js on Visual Studio Code. Also note that this is a community-led environment and thus, you can always extend/modify the platform, you can find some extensions in the extensions library for the VS Code. 

Once you are done with this setup, just run the app to verify if everything is in place. The default project contains the minimal code to run the server and return a Hello World text, 

  1. 'use strict';  
  2. var http = require('http');  
  3.   
  4. var port = process.env.PORT || 1337;  
  5.   
  6. http.createServer(function (req, res) {  
  7.     res.writeHead(200, { 'Content-Type': 'text/plain' });  
  8.     res.end('Hello World\n');  
  9. }).listen(port);  

So just ensure that things are in their proper place, if they are not — such as Node.js runtime not found, or Node.js is unable to connect to the port, or anything else, please recheck that you installed it correctly. One more thing, you need to restart the machine as Node.js runtime requires a restart to set up the PATH and much more. 

If everything is correct, then check that you have the Node.js runtime setup (for Visual Studio non-Code). 

Node.js runtime setup for server configuration in the Visual Studio.
Figure 2: Node.js runtime setup for server configuration in the Visual Studio.

Starting from now on, I will not mention anything about configurations and will assume your Node.js runtime is configured properly. 

Dependencies

Almost everything in the Node.js runtime is based on libraries, which are dependencies in the project. Both the front-end and the back-end of the application is based entirely on the dependencies. The dependencies are JavaScript libraries that you integrate with your app and then run the app for further usage, to enhance the services offered. 

We worked out with the usage of the following dependencies for front-end as well as for back-end, enlisting them as, 

  • Back-end dependencies
    • Express framework for Node.js
    • SQLite 3 for databases
    • Body parser and multer for handling file submissions
  • Front-end dependencies
    • Pug
      • Previously known as Jade.
    • Bootstrap
      • Integration of bootstrap was done via their CDN, and not the package manager.

The dependencies are available in downloads, you can just run the app after downloading (just change a few things such as the Node.js runtime, or other stuff). The dependencies in the project are as follows, 

  1. {   
  2.     "dependencies": {  
  3.         "body-parser": "^1.18.2",  
  4.         "express": "^4.16.2",  
  5.         "multer": "^1.3.0",  
  6.         "pug": "^2.0.0-rc.4",  
  7.         "sqlite3": "^3.1.13"  
  8.     }  
  9. }  

The dependencies are versioned as shown above, For a complete reference for the package.json file, read the reference here. Upon updating the package dependencies you will be able to get everything setup as needed and run the app, 

  1. $ npm update  
  2. $ npm start  

For the npm start command to execute, you also need to add the "start" node to the package.json file. You can see that complete in the material associated and ready for download in the downloads section. 

Note
The package we compiled contains the references and packages, for a smoother experience, you can consider updating the packages again, and also you should update if you make any changes to the package. 

Front-end technologies

First, let me cover up the front-end portion of the app, the front-end portion is easy, straight-forward and not that much of complex as much as the back-end management was. This is why I would like to cover the front-end of the app before, diving deep into the backend management of databases, routing, and other services to manage the web server. 

I have further divided the sections into two categories, 

  1. Generating the HTML content
  2. Stylizing the HTML content

In both ways, Node.js is not integrated and they are standalone frameworks for generation of HTML content and making the web pages responsive. Later on, I will demonstrate how you can pass model variables and other objects to the HTML generation code and render the dynamic content. 

Templates and HTML content

As mentioned, we used Pug framework to render the HTML content on the pages. Pug is a framework for generating HTML, dynamically (or statically as needed). The perks of having Pug used are that, 

  • You can use the Pug files to write the HTML content, is a very easy way.
  • You can generate the HTML, as you would write it — nothing extra gets added, and nothing gets removed. 
  • Pug also allows you to enter some JavaScript-like variables and collections. 
  • You can write some scripts to dynamically modify the HTML content generation. Some code block types are, 
    • Conditionals
    • Iterative
  • Pug can be used to provide templates
  • You can add models to be passed through your Node.js file, and that then generates the content based on those values. 

We need to create 3 pages, where we will add the HTML content for creation of forms, creation of questionnaire and then finally showing a page as a main page — in the sample we have more pages than that, the reason is we had to provide a demo site as well, so just to save ourselves we needed to add those about, terms and privacy pages in the demo. 

The following content is the basics of Pug framework, being used here, not a documentation. We started by creating the layout page for our project, 

  1. // layout.pug  
  2. block variables  
  3. doctype html  
  4. html  
  5.   head  
  6.     title Friend Knower  
  7.     script(src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js")  
  8.     link(rel="stylesheet", href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css")  
  9.     link(rel="stylesheet", type="text/css", href="/style.css")  
  10.     link(href="https://fonts.googleapis.com/css?family=Open+Sans:300", rel="stylesheet", type="text/css")  
  11.   body.container-fixed  
  12.     block heading  
  13.         nav#navigation.navbar.navbar-inverse  
  14.             div.container-fixed  
  15.                 div.navbar-header  
  16.                     a(href="/").navbar-brand Friend Knower  
  17.                 ul#navigation.nav.navbar-nav  
  18.                     li  
  19.                         a(href="/create") Create Form  
  20.                     li  
  21.                         a(href="/about") About & Contact  
  22.                     li  
  23.                         a(href="/policy") Privacy Policy  
  24.   
  25.     div.container  
  26.         block content  
  27.   
  28.     block footer  
  29.         footer.footer  
  30.             div  
  31.                 p Friend Knower  

In Pug files, there are several blocks of content, which act and serve as a placeholder for the content to be passed from various pages down the stream. They are labeled, such as the block variables or block heading or block footer. They can be named as needed or wanted, there is no rule of thumb; just name them what you want. You can also see that we used bootstrap to provide the responsiveness to the web pages. The bootstrap part is in the next section. 

The above code will generate the HTML content as written there, Pug files usually form as element content blocks. 

  1. h3 Heading  
  2. p Hi there, this is a paragraph.  
  3.   
  4. <h3>Heading</h3>  
  5. <p>Hi there, this is a paragraph</p>  
  6.   
  7. p.with-class You can add IDs and classes to the paragraphs too.  
  8. <p class="with-class">You can add IDs and classes to the paragraphs too.</p>  
  9.   
  10. div#container  
  11.     h4 Content holder  
  12.     p Adding indentation to the elements make them a child of the element above.   
  13.   
  14. <div id="container">  
  15.     <h4>Content holder</h4>  
  16.     <p>Adding indentation to the elements make them child of the element above.</p>  
  17. </div>  

These are a few of the ways you can write HTML content with Pug templating framework. Also, you may have guessed that if you do, 

  1. h4 Heading  
  2.    p Paragraph inside h4.  

Pug will not warn you that you are adding the element inside an element, where it should not be — some linter or expression parser might generate warnings, but I did not test against any.

To learn some advanced Pug scripting, consider consulting the documentation. The next file to be created was for the main page, then the create form and so on. The following is the create page, where the user would be able to enter the questions, 

  1. // create.pug  
  2. extends layout.pug  
  3.   
  4. block variables  
  5.   - var questions = 10  
  6.   
  7. block content  
  8.     form(action = "/create", method = "POST").form  
  9.          .content  
  10.          div.span4#headerText  
  11.             h3 Enter the details...  
  12.   
  13.             div#userdetails  
  14.                 div.form-group  
  15.                     label.control-label Your Name:  
  16.                     input(type="text", name="username", placeholder="John Doe" id="username", required).form-control.input-lg  
  17.                     p Fill in the questions below and submit the form to get the shareable link.  
  18.   
  19.             - for (var question = 1; question <= questions; question++) {  
  20.             - var randomOption = Math.floor(Math.random() * (4 - 1) + 1);  
  21.              div.span#questions  
  22.                 h4.bold= "Question #" + question  
  23.                 - var name = "question" + question  
  24.                 input(type="text", name=name, placeholder="Question " + question, required).form-control.input-lg  
  25.                 br   
  26.   
  27.                 - var options = 4  
  28.                 - for (var option = 1; option <= options; option++) {  
  29.                  div.form-group  
  30.                      - var optionName = name + "option" + option  
  31.                      - var selectedOption = "question" + question + "selected"  
  32.   
  33.                       div.radio  
  34.                           label#q1op1label.radio-inline.control-label  
  35.                               input(type="radio", name=selectedOption, value=option, required)  
  36.                               input(type="text", name=optionName, placeholder="Question " + question + " option " + option, id=optionName, required).form-control.expandinginput  
  37.                 - }  
  38.             - }  
  39.   
  40.          input(type="hidden", name="questions", value=questions)  
  41.          div.span4#Submitbutton  
  42.             button(type = "submit")#sbmitBtn.btn.btn-success Generate Form!  
  43.          br   
  44.          em By submitting you confirm that you have read and agree to   
  45.             a(href="/policy") the terms and policies of privacy  
  46.             span.  
  47.                 . Do not share passwords, credit card information or other sensitive information through these forms.  .

In the code above, you notice that I extend the layout page here; extends layout.pug. This is similar to the include in PHP, and other rendering options on other frameworks such as ASP.NET Views. Then, the same case of variables and conditional blocks with iteration comes in. 

Sample form with questions and their options rendered with Pug
Figure 3: Sample form with questions and their options rendered with Pug.

You can notice that I have an attribute, placeholder here, which can be converted to a value attribute if you want to debug the application and do not want to be left with filling the form each time. Also, I have a randomOption value that gets generated for each of the questions to select a random value in the options for each question. Use that as a value to test whether the option is selected for the question or not. These are the variables that can be used during the debugging session in case you are trying out the demo sample that is provided in the downloads section. 

Next page we created was the page where the user would fill in the form. The page is similar to what we have before, the only difference is that now we do not have to render the text boxes, instead we display the labels and those labels have a selection that marks the answer of the user. That way we provide the solutions to the questionnaires. The page code for the Pug to render the HTML is, 

  1. // preview.pug  
  2. extends layout.pug   
  3.   
  4. block variables  
  5.   - var totalQuestions = 10  
  6.   
  7. block content  
  8.     form(method="POST")  
  9.         .content  
  10.         input(type="hidden", value=Id, name="formId")  
  11.         div.span4#headerText  
  12.         h3 How well you know #{userName}?  
  13.         p Start by entering your name, #{userName} can know who knows them best.  
  14.   
  15.         input(type="text", name="responseBy", required).form-control.input-lg  
  16.   
  17.         h4 Questionnaire  
  18.         - for (var q = 0; q < totalQuestions; q++) {  
  19.          - var question = questions[q];  
  20.          - var questionName = "question" + (q + 1);  
  21.          div  
  22.              h4   
  23.                 b Question   
  24.                 span.  
  25.                     !{question.statement}  
  26.              div.span4  
  27.                 div.row  
  28.                 - for (var o = 1; o <= 4; o++) {  
  29.                     div.col-md-3.col-sm-3.col-lg-3  
  30.                         div.form-group  
  31.                             div.radio  
  32.                             label.control-label.radio-inline  
  33.                                 input(type="radio", name=questionName, value=question.id + "," + o, required, id=questionName + o)  
  34.                                 span.   
  35.                                     !{question["option" + (o)]}  
  36.                             br  
  37.                 - }  
  38.         - }  
  39.           
  40.         div.span4#Submitbutton  
  41.         button(type="submit").btn.btn-default Submit  
  42.           
  43.         br  
  44.         br  
  45.         em By submitting you confirm that you have read and agree to   
  46.             a(href="/policy") the terms and policies of privacy  
  47.             span.  
  48.                 . Do not share passwords, credit card information or other sensitive information through these forms.  

Following the same patterns, we wrote down the code for Pug, to render the forms to accept the user inputs. The variable for the total number of questions controls how many questions to render here. Since our form supports 10 questions, we kept a variable holding value of 10 everywhere to store the form and to ask the questions. The variable alteration means that the number of questions on the form would change. 

Questionnaire to be filled, with autogenerated questions and options
Figure 4: Questionnaire to be filled, with autogenerated questions and options.

Note
All of this code is available in the GitHub repository, as well as on the code sample provided here. 

Other minor pages, such as the pages to show how many times a form has been filled, and what are the stats for the page were generated. These pages use the same Pug framework. You can review the pages in the source samples. In the next sections, I will close the front-end portion of the article and write up for the back-end. 

Bootstrapping

As seen above, the bootstrap was used in the layout file to be streamed down to every file. Bootstrap is a responsive framework and the only framework of JavaScript that I can infect my web apps with. The bootstrap happened to serve as the primary and the core JavaScript framework for our front-end. We heavily utilized the framework, to support desktop, mobile, and other screen variants. 

  1. script(src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js")  
  2. link(rel="stylesheet", href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css")  

We used the CDNs services for bootstrap, to load the Bootstrap JavaScript and CSS files. This would end up with removing some load from our server and improving the performance as well. Finally, we did not need to integrate anything, as we were just loading it — they are .min. files, because we do not need to debug their code at all, so why waste KBs?

Bootstrap has some good and useful stuff, that you definitely should look into. Such as the buttons we built, or the forms and the input fields we built. They were developed with bootstrap and the default styling was considered to support responsiveness. To support, we just ran a simple Google test to determine our website is mobile friendly or not, the response was, 

Google test for mobile friendliness of a web page. We used Bootstrap for respnsiveness and some extra ingredients to support the purpose
Figure 5: Google test for the mobile friendliness of a web page. We used Bootstrap for responsiveness and some extra ingredients to support the purpose. 

The problem with the URLs loading is the off-site URLs for other stuff applied on the web page and has no direct relation with the website's responsiveness. This demonstrates that the website is capable of being SEO friendly. Besides, this had the least hard work of ours, because most of the work was done by Bootstrap, so credit to them. 

Back-end stuff

That part was the front-end of our app, the backend stuff is a bit complex and complicated. In this section, I will cover up the stuff, such as database management, routing of the application, error handling and a few other things that need to be mentioned. Node.js runtime provide an excellent way of writing the apps, it is performance efficient, and community-supported, which means that it can provide you with a library for almost anything. 

The overall setup we ended up with after the app was running looks something like this, of course, this is just an overview of the design,

Overall structure of the app, from connection to backend management
Figure 6: Overall structure of the app, from connection to backend management.

There are different components that run, in order to make the app live and bring the functionality. We have looked into the front-end techniques, now it would be a good time to look at the backend stuff. 

Routing

Before I talk about the database stuff, I would like to share the routing of the URLs and mapping of the HTTP verbs. The URLs of the app are the main part, you cannot expect your app to be used efficiently if your URLs are not well-formed. For the same reason, we tried to use as little URLs are possible and then map the rest to the suitable URLs, such as 404-page. The Express framework provides the services, that you can use to easily map the URLs to your functions, and handle the HTTP requests in them. 

We are using the Express framework, which enables us to utilize the HTTP verbs as functions, to map to a URL. To create a server, start with the most minimal approach of a Hello World, 

  1. const express = require("express");  
  2. const app = new express();  
  3.   
  4. // Set up the default route  
  5. app.get("/", function (req, res)) {  
  6.     res.send("Hello world.");  
  7. }  
  8.   
  9. // Start the server  
  10. const port = process.env.PORT || 8080;  
  11. app.listen(port, function () {  
  12.     console.log("Server is listening at localhost:" + port);  
  13. });  

Express Framework supports the HTTP verbs, to attach the functions to each type of URL, based on the request type, such as app.post, app.put etc. You can also use app.all to map to a URL without prejudice for any HTTP verb. In a similar manner, we also configured the routing to support an HTTP GET and HTTP POST for the pages where we had to manage and configure the forms from. 

The structure was something like this, 

  1. /** 
  2.  * All of the commented code is discussed in the model passing section below.  
  3.  */  
  4.   
  5. app.get("/create", function(req, res) {  
  6.     // Code below  
  7. });  
  8.   
  9. app.post("/create", function (req, res) {  
  10.     // Code below  
  11. });  
  12.   
  13. app.get("/preview/:formId", function (req, res) {  
  14.     // Code below  
  15. });  
  16.   
  17. app.post("/preview/:formId", function (req, res) {  
  18.     // Code below  
  19. });  
  20.   
  21. app.get("/stats/:formId", function (req, res) {  
  22.     // Code below  
  23. });  
  24.   
  25. app.get("/formcreated/:formId", function (req, res) {  
  26.     res.render("formcreated", { formId: req.params.formId });  
  27. });  
  28.   
  29. app.get("/", function (req, res) {  
  30.     res.render("index");  
  31. });  
  32.   
  33. app.get("*", function (req, res) {  
  34.     res.redirect("lost");  
  35. });  
  36.   
  37. app.get("/lost", function (req, res) {  
  38.     res.render("lost");  
  39. });  

There are a few things to note here, first of all, the parameters in the URL can be specified like, /:paramName. You can then later on access the parameter in the req.params property. This can some real handy if you want to design the URLs to be, /resource/id, instead of having the URLs like /resource?id=value. We followed the same practice and enabled the URL mapping of parameters and optional values, this lets us manage the form IDs and then provide the dynamic content based on that. 

Routing can be done in other formats as well, there are controllers in Express framework that you must look into if you want to build enterprise-ready apps, controllers can help you bring the functionality in a layered form. 

SQLite database

Node.js runtime supports SQLite 3 databases and provides a library that can be used to work with databases. The npm can be used to install the library. 

Either, install via the npm CLI, 

  1. $ npm install sqlite3  

Otherwise, install via adding a dependency to that package in the package.json file. 

  1. "dependencies": {  
  2.     ....  
  3.     "sqlite3": "^3.1.13"  
  4. }  

At the time of current writing, the latest version is 3.1, so it would be best if you can check against the npm tools and CLI to determine the latest version of the package for your app. If you do this, even then you would need to update the packages by running the npm update command. The SQLite 3 library contains everything from the database drivers, to connection management to query executors. The complete information for the package can be captured from here. This library is also asynchronous, and does not block the Node.js runtime code during execution of database-bound operations. This means that you can use the JavaScript promise API to chain the SQL operations, and then execute them one after the other. 

The topics to cover here in this portion are, 

  • SQL queries in SQLite3 library
  • Parameterizing the queries — again, to best avoid any kind of misuse, consider an ORM wrapper. 
  • Executing queries further, only if the first query executes well
  • Capturing data in different types of boxes (single record, multiple records)
  • Managing when no data row is to be returned.

Starting off, I will assume that you have the basic understanding of SQL queries. You start off with the initialization of the database engine, and then you manage the tables. 

  1. var sqlite3 = require("sqlite3").verbose();  
  2. let db = new sqlite3.Database("friendknower.sqlite", (err) => {  
  3.     if (err) {  
  4.         console.log("Cannot connect to the database.");  
  5.     } else {  
  6.         console.log("Connection established with the database.");  
  7.     }  
  8. });  

If you cannot import the sqlite3 package, make sure there is a dependency logged and then update the package manager. First we move on and define out tables, 

  1. let removeTables = false;  
  2. function setupDb() {  
  3.     if (removeTables) {  
  4.         db.run("DROP TABLE IF EXISTS Users");  
  5.         db.run("DROP TABLE IF EXISTS Forms");  
  6.         db.run("DROP TABLE IF EXISTS Questions");  
  7.         db.run("DROP TABLE IF EXISTS Solutions");  
  8.     }  
  9.   
  10.     db.run("CREATE TABLE IF NOT EXISTS Users (Id INTEGER PRIMARY KEY, Name TEXT, Token TEXT)");  
  11.     db.run("CREATE TABLE IF NOT EXISTS Forms (Id INTEGER PRIMARY KEY, UserId INTEGER, Name TEXT)");  
  12.     db.run("CREATE TABLE IF NOT EXISTS Questions (Id INTEGER PRIMARY KEY, FormId INTEGER, Statement TEXT, Option1 TEXT, Option2 TEXT, Option3 TEXT, Option4 TEXT, SelectedOption TEXT)");  
  13.     db.run("CREATE TABLE IF NOT EXISTS Solutions (Id INTEGER PRIMARY KEY, FormId INTEGER, Answers TEXT, SubmissionBy TEXT)");  
  14. }  

SQLite supports the queries such as IF NOT EXISTS and IF EXISTS, that can be deemed helpful when you are trying to create a table and want to make sure it only gets created if it does not exist. And same for the IF EXISTS, to delete only when there is something to delete. And during debugging, it might be useful to delete the tables on every restart, for that that removeTables variable was used. 

Then, the library also supports different functions that you can execute to select the values. A simple rule of thumb is, 

  • Execute get(), when you want to get only 1 record from the table, such as a top-level record.
  • Execute each(), when you want multiple records but are unsure of how many there'd be. This can help in ignoring a load of data being pushed all of a sudden and the latency to capture multiple records. 
  • Execute all(), to capture all the records from the database in one go. 
  • Execute run(), to insert the records and perform another sort of functions, or to create tables etc. 

To demonstrate this, consider the following function to capture the solutions for a form, 

  1. app.get("/stats/:formId", function (req, res) {  
  2.     // Load the stats  
  3.     var formId = req.params.formId;  
  4.   
  5.     db.each("SELECT * FROM Forms WHERE Id =?", [formId], function (err, form) {  
  6.         if (err) {  
  7.             console.log("Cannot read from Forms table.");  
  8.             console.log(err);  
  9.             res.redirect("/error");  
  10.         } else {  
  11.             db.all("SELECT * FROM Questions WHERE FormId =?", [formId], function (err, retrievedQuestions) {  
  12.                 if (err) {  
  13.                     console.log("Cannot read from Questions table");  
  14.                     console.log(err);  
  15.                     res.redirect("/error");  
  16.                 } else {  
  17.                     // Read the questions  
  18.                     var solutions = [];  
  19.                     db.all("SELECT * FROM Solutions WHERE FormId =?", [formId], function (err, retrievedSolutions) {  
  20.                         if (retrievedSolutions.length == 0) {  
  21.                             res.render("stats", { notSolved: true, formId: formId });  
  22.                         } else {  
  23.                             // .. code further down  

Like you can see here, we started off with the request and captured the formId from the URL itself. Then, the queries move in and provide us with the form. The signature holds, 

  1. db.each(query: String, parameters: [], callback: function (error, returnedObject) { }  

The same can be seen above, thus we check if there were any errors in the transaction of the query. If there are, we return the error message. The object returned contains the columns in the form of properties, you can access the properties in a number of ways. To demonstrate this, let me tell you a few of examples, 

  1. Id | Column1 | column2 | Column 3 | Column-4  

I used different column labels to demonstrate how JavaScript can let you capture the values of that object in different manners.

  1. var column1 = obj.Column1      // Captial C  
  2. var column2 = obj.column2      // Small C  
  3. var column3 = obj["Column 3"]  // Space is not allowed in names  
  4. var column4 = obj["Column-4"]  // Special characters are not allowed as well  

The rule is that if a column name doesn't follow JavaScript naming conventions or that is not a legal JavaScript variable name, then you can use the index notation of capturing the variable names, as in the case of last two. 

Rest of the SQL queries, and how the form works would take a longer content, where the post is already huge. Thus, if you would like to read how I managed to write the SQL scripts, please read the repository code. Also, the SQL code is not that much important because it is recommended to consider an ORM, which is what we recommend. The repository of the project will also upgrade to using an ORM and layered structure for the forms and data orientation. 

The asynchronous approach or writing the code and managing the chaining would take too much of article scope to explain, thus you can try to understand how to write the code, with chaining. The SQLite 3 library that we are using has this asynchronous approach built-in. 

Note
It is recommended to consider an ORM, such as Sequelize. Hardcoding the SQL queries is not a bad practice, it only takes more time to do the same. With ORMs, you also have a safer environment, because you do not write any bad code — unless of course the bad code was written in the library, in which case, you're doomed. 

Delivery of content as model to Pug

Last but not least, the model passing to the Pug template is the part, where you pass the forms and other information to the templated file. I skipped this part, from the databases section and I am going to talk about how you pass the variables to the template file. 

The simplest way is to pass the second parameter to the render function, that would define the properties you can access. 

  1. // "stats" page is the page for solutions of the form submissions  
  2. // var solutions = []  
  3. // solutions contains list of elements such as the solutions for the form.  
  4. res.render("stats", { solutions: solutions, formId: formId });  

Take the page above, for example, we are passing the solutions and the ID of the form here. To render them, you can access the properties directly — without calling any Model, or other objects first. Such as -

  1. div#stats  
  2.     - for (var i = 0; i < solutions.length; i++) {  
  3.      div#item  
  4.         p.bold= solutions[i].by + " — " + solutions[i].corrects + "/15."  
  5.     - }  

This would create a div element, with the ID of stats, and then iterate and add a child-div element for each of the solutions. A few tips to get you started in this would be, 

  • If you want to pass an array of elements, pass them as a property of the object, { elements: list }, then access them via elementsvariable in Pug.
  • For other fields and variables, pass them as a property directly and access via the name. 

There is no need to wrap or write any special model imports in Pug template, the overall object that you pass as a parameter gets exposed as a model and you call the properties and render them. You can also see the same pattern in the code above, solutions[i].by, where solutions were the list, and the by was a field to render who filled the form with this solution. To learn more about this, consider learning more about the res.render() function of the Express framework and how you can pass the model parameters. 

Deployment

Now that our app is up and running, it is time to host it. As mentioned, we wanted to try something else, something other than just-Microsoft. So, we worked out an account on Heroku platform and setup the verifications/other formalities to have our app run smoothly. 

Heroku somehow let us down, provided the speed and performance of Node.js framework. Understood that I am using a free account, but even then it must not be as much bad as it sounds — after all, I would have just used Azure for this and I would have at least x10 times better results than this. Nonetheless, the deployment models of Heroku platform support an overall three models, 

  • GitHub
  • Dropbox — Heroku CLI falls back to this deployment model. 
  • Windows Containers

The following image demonstrates this, 

Heroku dashboard for deployment of apps
Figure 7: Heroku dashboard for deployment of apps.

What I consider a good approach here is the Heroku CLI approach. There are several benefits of using Heroku CLI instead of GitHub or Dropbox. Heroku CLI uses the Dropbox folders that you authenticate against, then it uses the git version control to manage the repository and deployments. I used GitHub, the repository for it was public, and can be accessed at https://github.com/afzaal-ahmad-zeeshan/friend-knower, there were some common problems with this, such as all your code is public, unless you have a premium GitHub account. Which I cannot afford for this. I had some code integrations for other purposes, such as ads, which I cannot afford to share on GitHub — even though it is not a big deal for your ad units to be left in wrong hands, as Google will not bother delivering ads to those websites. 

Heroku CLI is a very simple, intuitive and straight-forward command-line interface for managing your applications. The overall workflow to, 

  • Authenticate your account
  • Connect to the Heroku repository for your project
  • Deploy and publish the app

Would be just the way as follows, to authenticate, you run the following command, 

  1. $ heroku login  

This will ask for your Email and Password to authenticate against. Once that has been done you just need to clone your repository and commit the changes to the same repository, which will publish your application. 

  1. $ # friend-knower was the name of my app, use your own.  
  2. $ heroku git:clone -a friend-knower  
  3.   
  4. $ # friend-knower folder was created  
  5. $ cd friend-knower  
  6.   
  7. $ # Assuming you have not copied the content in this folder  
  8. $ git add .  
  9. $ git commit -m "Publishing the application"  
  10. $ git push heroku master  

Heroku CLI would generate the heroku remote link to git repository, which is why you can push to this repository. It is similar to your own repository on Heroku, either connecting directly or through this method. 

A live demo of the app can be checked out at, https://friend-knower.herokuapp.com/.

What we learnt

Among many things that we discovered the primary thing was: Abandon JavaScript! 

JavaScript is a dynamically typed language, thus it gives you an opportunity to write bad code. If you have programmed in JavaScript, then you know that you are typically prone to errors like, undefined type or not found and much more. Not just this, there is no type safety in JavaScript. You can manually add conditions like, instanceof and then continue writing the code which would be type safe, but other than that, there is no type safety thus you write code with more if...else blocks to check if everything is Okay. 

In this case, we realized the fact that TypeScript can help us in writing better code for Node.js. We will try to write another article about something in in Node.js but with TypeScript in future. 

TypeScript logo
Figure 8: TypeScript logo.

Another major thing we had learnt was, that you should consider using an ORM in Node.js for the database platforms that you are trying to target. We discovered Sequelize framework, which is quite good and provides an excellent support for generating the models and managing the overall DDL and DML experience. Consider visiting their official documentations for learning how to write the ORMs for Node.js. Also, Sequelize supports multiple database dialects, 

  • SQLite
  • PostgreSQL
  • MySQL
  • SQL Server

An ORM can help you out with a lot of stuff. We somehow managed to have the app built with minimal coding and thus integrating the ORM would've not made any difference for our sample. If you are trying to write a large app, we recommend that you use an ORM — you are free to make any choice, Sequelize is just what we felt good with. 

For the platform of Heroku, we learnt that they do provide a well formed service. They provide excellent deployment models, with multiple ways to deploy the app. Since they support GitHub, they support automatic integration (continuous integration happens on GitHub). This was a very easy model for deployment of the apps. See the Deployment section above to learn how the process works. 

What's missing

This application was merely a sample run down for the overall concept of questionnaire app. Though we tried to invest a huge amount of time in building a good starter template, but we were unable to determine whether this concept would be adopted or whether it would be a waste of time. Thus, we did not add more improvements to it, if you feel like this has to be upgraded, let us know and we can start the platform. 

Our focus in the upcoming versions would be to generate templates so that forms are autofilled by others, and you only pass the name to write on the form, we also want to integrate the SQLite ORM library in this, that will help us in removing the problems such as writing/changing the queries each time our model changes. 

Lastly, we want to write handlers for states where a form doesn't exist, a question doesn't exist and much more. Our current version doesn't support that. 

Lastly, we also to support multiple question types, such as checkbox-type and text-box type. More over, the number of questions is a hurdle in making this app useful. We want to work this out, and provide an extendable form that can grow/shrink as needed by the form creator. We could really use some of your help in shaping this library/app to extend and provide more functionality. You can share your insights or some help on the GitHub repository.


Similar Articles