Node.js, overview

Node.js, configuration

npm is in sync with a package.json file for configuration purposes including file location, nature, purpose… library versioning, licensing, etc.

Library versioning:

Other configurations:

Further information on configuration here

Node.js and TypeScript

TypeScript imposes the installation of Node.js types as dev. dependency:

npm i @types/node --save-dev

Library import can then be ruled from Node.js common style, i.e., require or TypeScript pure style (recommended):

const HTTP = require("http"); // 'require' => 'npm i @types/node -S'
import * as HTTP from "http"; // Pure TypeScript style...

Core libraries, the case of assert (see also: here…)

console.assert(Dollar.get('common_symbol') === "$"); // Common style...
const assert = require('assert').strict;
const {message} = new assert.AssertionError({
    actual: Dollar.get('common_symbol'),
    expected: "$",
    operator: "strictEqual"
try {
    assert.strictEqual(Dollar.get('common_symbol'), "$");
} catch (error) {
    assert(error instanceof assert.AssertionError);
    assert.strictEqual(error.message, message);
    assert.strictEqual(, 'AssertionError');
    assert.strictEqual(error.actual, Dollar.get('common_symbol'));
    assert.strictEqual(error.expected, "$");
    assert.strictEqual(error.code, 'ERR_ASSERTION');
    assert.strictEqual(error.operator, 'strictEqual');
    assert.strictEqual(error.generatedMessage, true);

Node.js, sandbox

const server = http.createServer((request, response) => { // const http = require("http");
    response.writeHead(200, {"Content-Type": "text/html; charset=utf-8"});
    file_system.readFile('./index.html', null, function (error, data) { // const file_system = require('fs');
        if (error) {
            response.write("Whoops! './index.html' not found...");
        } else { response.write(data); }
}).listen(port); // const port = process.env['PORT'] || 8080; // Env. variable
console.log("Server ready to accept requests on port %d", port);

Express, sandbox

const my_application = express(); // const express = require('express');
my_application.get('/', (request, response) => {
    response.setHeader("Content-Type", "text/html; charset=utf-8");
    response.statusCode = 200;
    response.sendFile(path.join(__dirname + '/index.html')); // const path = require('path');
const server = my_application.listen(port); // const port = process.env['PORT'] || 8080;
console.log("Server ready to accept requests on port %d", port);

Express, “router”

const router = express.Router();

const my_application = express();
my_application.use('/', router);
router.get('/', (request, response) => {
    response.setHeader("Content-Type", "text/html; charset=utf-8");
    response.statusCode = 200;
    response.sendFile(path.join(__dirname + '/index.html'));
const server = my_application.listen(port);
console.log("Server ready to accept requests on port %d", port);

Serving static files

Serving static files in Node.js relies on the node-static library.

const ns = require('node-static');
const file_server = new ns.Server('./static');
http.createServer(function (request, response) {
    file_server.serve(request, response);

Express, serving static files (see also: here…)

console.log(__dirname + path.sep + 'static'); // Dir. of static files...
barbierdarnal.use(express.static('static')); // This serves resources from 'static' folder...
barbierdarnal.use('/', router);
router.get('/', (request, response) => {
    response.sendFile(path.join(__dirname + '/BarbierDarnal.html')); // '__dirname' resolves to your project folder
router.get('/Genie_logiciel', (request, response) => {
    response.sendFile(path.join(__dirname + '/static/html/Genie_logiciel.html'));
const server = barbierdarnal.listen(port);

Querying the Web

Querying the Web is a common repeated task in Node.js. The fact that the use of the native http and https libraries may become tedious, third-party libraries like request (obsolete) or Axios may do the job in a more seamless way.

Web scraping is a more advanced usage of Web querying with dedicated libraries like Puppeteer.

Querying the Web, request (third-party) library (simple HTTP client)

const request = require("request"); // 'npm install --save request' or 'sudo npm install --save request'

const options = {
    headers: {accept: 'application/json', 'content-type': 'application/json'},
    method: 'GET',
    url: ''

request(options, function (error, response, body) {
    if (error)
        return console.error('Failed: %s', error.message);
    console.log('Success: ', body);

Querying the Web, https (native) library

const https = require('https');
https.get('', (response) => { // Aubergines Cassegrain
    let data = '';
    response.on('data', chunk => { // A chunk of data has been received...
        data += chunk;
    response.on('end', () => { // The whole response has been received...
        let object;
        try {
            object = JSON.parse(data);
            console.log(Object.getOwnPropertyNames(object).join(" - "));
        } catch (exception) { console.log(`${}: ${exception.message}`); }
}).on('error', (error) => { console.log("Error: " + error.message); });


// 'Save_image_from_URL.js' file
const file_system = require('fs'); //
const https = require('https'); //

module.exports.save_image_from_URL = function () {
    https.get('', response => {


// 'Main.js' file
const Save_image_from_URL = require('./Save_image_from_URL.js');
// 'function' is displayed:
console.log(typeof Save_image_from_URL.save_image_from_URL);
 // Simple usage:

Web services, micro-service architecture design (see also: here… and there…)

Express is a natural support for Web service design in the logic of “micro-service architecture”.

import * as Express from 'express';
const GPAO = Express();
// For POST (and PUT) requests (based on 'body-parser' library in Node.js):
const port = process.env['PORT'] || 1963; // 'process' => "@types/node"
const server = GPAO.listen(port);
console.log("Server ready to accept requests on port %d", port);

Case study (in French): here

Web services, micro-service architecture design, GET

GPAO.get('/', (request, response) => {
    response.send('<h1 style="color: green;">"GPAO.Node.js.ts": Restful Web services, test</h1>');
GPAO.get('/api/GPAO/Articles', (request, response) => {
GPAO.get('/api/GPAO/Articles/:reference', (request, response) => {
    const article = Articles.find((a: Article) => a.reference === request.params.reference);
    if (!article) response.status(404).send('<h1 style="color: red;">Not found: ' + request.params.reference + '</h1>');

Reminder on HTTP status codes: here

Web services, micro-service architecture design, POST"/api/GPAO/Nouvel_article", (request, response) => {
    const article: Article = request.body;
    const {error, value} = Articles_.validate({
        reference: article.reference, // Unicity required as well...
        designation: article.designation, // Unicity required as well...
        type_fabrication_achat: article.type_fabrication_achat,
        unite_achat_stock: article.unite_achat_stock,
        delai_en_semaine: article.delai_en_semaine,
        lot_de_reapprovisionnement: article.lot_de_reapprovisionnement,
        stock_maxi: article.stock_maxi,
        PF_ou_MP_ou_Piece_ou_SE: article.PF_ou_MP_ou_Piece_ou_SE

Web services, micro-service architecture test

# Lire tous les articles...
curl http://localhost:1963/api/GPAO/Articles
# Lire un article particulier via sa référence...
curl http://localhost:1963/api/GPAO/Articles/CD100
# Ecrire un article (PowerShell)...
$CC201 = @{
    reference = 'CC201'
    designation = 'Camion citerne rouge'
    type_fabrication_achat = 'Fab. a la commande'
    unite_achat_stock = 'unite'
    delai_en_semaine = 2
    lot_de_reapprovisionnement = 150
    stock_maxi = 600
    PF_ou_MP_ou_Piece_ou_SE = 'PF'
 echo $CC201
 curl http://localhost:1963/api/GPAO/Nouvel_article -Body $CC201 -Method Post

Data validation

The importance of data validation within the middleware tiers requires devoted libraries like express-validator that supports data validation within Express. Type set for TypeScript:

"@types/validator": "^13.1.3",

Joi within Hapi is a competitor.

Middleware-side data validation, the case of Joi within Hapi

Hapi is an Express competitor. Data validation relies on Joi.

import * as Joi from '@hapi/joi';
export enum Article_type { MP = 'MP', PF = 'PF', Piece = 'Piece', SE = 'SE' }
export interface Article { reference: string; designation: string; type_fabrication_achat: string; unite_achat_stock: string; delai_en_semaine: number;
    prix_standard?: number; lot_de_reapprovisionnement?: number; stock_mini?: number; stock_maxi?: number; pourcentage_de_perte?: number; inventaire?: number;
    PF_ou_MP_ou_Piece_ou_SE: Article_type
export const Articles_ = Joi.object({
    reference: Joi.string().alphanum().required().min(4).max(10),
    designation: Joi.string().pattern(new RegExp('^[a-zA-Z0-9 ]*$')).required().max(30),
    PF_ou_MP_ou_Piece_ou_SE: Joi.string().valid(Article_type.MP, Article_type.PF, Article_type.Piece, Article_type.SE)

Middleware-side data validation, the case of Joi within Hapi cont'd

const Articles = new Array<Article>();
const {error, value} = Articles_.validate({
    reference: 'CD100', designation: 'camion demenagement bleu', type_fabrication_achat: 'pf fabr. par lot', unite_achat_stock: 'unite', delai_en_semaine: 2,
    lot_de_reapprovisionnement: 200, stock_maxi: 600, PF_ou_MP_ou_Piece_ou_SE: Article_type.PF
if (error === undefined) {
    console.log("Article validé (succès) : " + value.reference);
        reference: 'CD100', designation: 'Camion demenagement bleu', type_fabrication_achat: 'Fab. par lot', unite_achat_stock: 'unite', delai_en_semaine: 2,
        lot_de_reapprovisionnement: 200, stock_maxi: 600, PF_ou_MP_ou_Piece_ou_SE: Article_type.PF
} else console.log("Article invalidé (échec) : " + error.message);

Database tiers (a.k.a. back-end software)

By principle, Node.js acts as a middleware for front-end software typically based on Angular or React.

The need for database connection depends upon external libraries like TypeORM, Prisma or, Sequelize in the SQL world. TypeORM and Java Persistence API (JPA) are siblings in using annotations, i.e., @, the former acting in the Node.js world while the latter is a key Java standard. Both TypeORM or Prisma target well-known RDBMS like SQLite or MySQL.

MongoDB NoSQL DBMS benefits from directly using the MongoDB Node.js driver even though both TypeORM or Prisma deal with MongoDB as well.

SQL, the case of TypeORM

TypeORM, fundamentals

import {Entity, PrimaryColumn, Column} from "typeorm";
export class CriminalCase {
    @PrimaryColumn({length: 10})
    criminal_case_number: string;
    @PrimaryColumn({length: 30})
    jurisdiction_name: string;
    date_of_criminal_case: Date;


TypeORM, relationships

export class Prisoner {
    @ManyToOne(type => CriminalCase) // '@ManyToOne' does not require '@OneToMany'
        name: "CRIMINAL_CASE_NUMBER",
        referencedColumnName: "criminal_case_number" // Name of attribute in class 'CriminalCase'
    }, {
        name: "JURISDICTION_NAME",
        referencedColumnName: "jurisdiction_name" // Name of attribute in class 'CriminalCase'
    incarceration_main: CriminalCase;
    @ManyToOne(type => Motive) // '@ManyToOne' does not require '@OneToMany'
    @JoinColumn({name: "MOTIVE_NUMBER"})
    incarceration_motive: Motive;
    @ManyToMany(type => CriminalCase)
        joinColumns: [{name: 'PRISON_FILE_NUMBER', referencedColumnName: 'prison_file_number'}],
        inverseJoinColumns: [{name: 'CRIMINAL_CASE_NUMBER', referencedColumnName: 'criminal_case_number'},
            {name: 'JURISDICTION_NAME', referencedColumnName: 'jurisdiction_name'}]
    offense: CriminalCase[];
    @OneToMany(type => JudicialDecision, judicial_decision => judicial_decision.prisoner)
    judicial_decision: JudicialDecision[];


TypeORM, connection configuration

import {ConnectionOptions, createConnection} from "typeorm";
const options: ConnectionOptions = {
    type: "sqlite",
    database: "./New_York_City_Penitentiary_database",
    entities: [ __dirname + "/Entities/*.js" ],
    logging: false,
    synchronize: false // Don't overwrite existing schema (and data)...
createConnection(options).then(connection => { …

TypeORM, queries

web.get('/Under_remand', async (request, response) => { // 'web' is an express app.
    // Ver. 1:
    // const prisoners: Prisoner[] = await database.getRepository(Prisoner).find({relations: ["judicial_decision"]});
    // response.send(prisoners.filter(prisoner => prisoner.judicial_decision.filter(judicial_decision => judicial_decision.decision_type_number === "1").length === 0));
    // Ver. 2 ('SELECT * FROM Prisoner WHERE prison_file_number NOT IN (SELECT prison_file_number FROM Conviction)'):
    const sub_query = await database.getRepository(Conviction).createQueryBuilder('conviction').select("conviction.prison_file_number");
    const result = await database.getRepository(Prisoner).createQueryBuilder('prisoner')
        .where("prisoner.prison_file_number NOT IN (" + sub_query.getQuery() + ")")


NoSQL, the case of MongoDB

MongoDB sample

// TypeScript style instead of 'const MongoDB = require("mongodb");'
import * as MongoDB from "mongodb";
// MongoDB Atlas:
const MongoDB_URL = "mongodb+srv://FB:<password>";

const client = new MongoDB.MongoClient(MongoDB_URL,
                                       {useNewUrlParser: true,
                                        useUnifiedTopology: true});

MongoDB sample cont'd

client.connect((error, client) => {
    // @ts-ignore (direct '.s' access isn't advised!)
    console.log("'client.s.url': " + client.s.url);
    const Web_scraping_database = client.db("Web_scraping");
    Web_scraping_database.admin().listDatabases((error_, dbs) => {
        console.log("'dbs.databases': " + JSON.stringify(dbs.databases));
    const my_data = Web_scraping_database.collection("data");
    my_data.insertOne({forname: "Franck", surname: "Barbier"}).then(() => {
        my_data.countDocuments({}).then(value => { // console.log("'my_data.countDocuments({})': " + value); // '1'
            my_data.drop().then(() => {
                client.close().then(() => { console.log("See you later!"); });

mongoose (see also: here…)

mongoose is a JavaScript/TypeScript library for MongoDB. mongoose promotes disciplined data modeling (Mongoose.Schema) in the spirit of SQL (primary keys, foreign keys and constraints on data in general).

import * as Mongoose from "mongoose";
const currencies_schema: Mongoose.Schema = new Mongoose.Schema({ // USA dollar:
    common_name: Mongoose.Schema.Types.String, // Dollar
    common_symbol: {type: Mongoose.Schema.Types.String}, // $
    substitute: {type: Mongoose.Schema.Types.Mixed},
    substitution_date: {type: Mongoose.Schema.Types.Date, default:}

mongoose, from schema to model and document…

import * as Mongoose from "mongoose";
// mongoose automatically looks for the plural, lowercase version of the model name...
// So, 'currencies' collection is created in the MongoDB database:
const currencies_model = Mongoose.model('currencies', currencies_schema);
// 'Dollar' as model instance is a *document* (storable data) in the 'currencies' collection...
const Dollar = new currencies_model({
    common_name: "Dollar",
    common_symbol: "$",
    substitute: null

mongoose, connection

import * as Mongoose from "mongoose";
// MongoDB Atlas:
const MongoDB_URL_ = "mongodb+srv://FB:<password>";
Mongoose.connect(MongoDB_URL_, {useNewUrlParser: true, useUnifiedTopology: true}).then((value: Mongoose.Mongoose) => {
    console.log("Connected database: '" + (value.connections.length > 0 ? value.connections[0].name : "?")
        + "' on '" + (value.connections.length > 0 ? value.connections[0].host : "?") + "'");
const connection: Mongoose.Connection = Mongoose.connection;
connection.on('error', console.error.bind(console, 'connection error:'));

mongoose, data usage

import * as Mongoose from "mongoose";
connection.once('open', () => { // Remove existing data (for test):
    currencies_model.deleteMany({common_symbol: "$"}, function (error) {
        if (error) return console.error(error.toString());
    }); => { // Then, check if present:
        currencies_model.find({common_symbol: "$"}).exec().then(currency => {
                connection.close().then(() => { console.log("See you later!"); });

Two-tier architecture

mongoose, index as primary key support

export default interface CriminalCase extends Mongoose.Document {
    criminal_case_number: string;
    jurisdiction_name: string;
    date_of_criminal_case: Date;
const schema = new Mongoose.Schema({ // 'create table CRIMINAL_CASE('
        criminal_case_number: { // 'CRIMINAL_CASE_NUMBER varchar(10),'
            type: Mongoose.Schema.Types.String, required: true, maxlength: 10},
        jurisdiction_name: { // 'JURISDICTION_NAME varchar(30),'
            type: Mongoose.Schema.Types.String, required: true, maxlength: 30},
        date_of_criminal_case: { // 'DATE_OF_CRIMINAL_CASE Date,'
            type: Mongoose.Schema.Types.Date, required: true, default:
// 'constraint CRIMINAL_CASE_key primary key(CRIMINAL_CASE_NUMBER,JURISDICTION_NAME));' (performance issues!):
schema.index({'criminal_case_number': 1 /* ascending '1', descending '-1' */, 'jurisdiction_name': 1}, {unique: true});

mongoose, index as foreign key support

const schema = new Mongoose.Schema({
        decision_type_number: {type: Mongoose.Schema.Types.String, required: true,
                                enum: ['1', '2','3'], maxlength: 1},
        prisoner: {type: Mongoose.Schema.Types.ObjectId, required: true,
                                ref: 'PRISONER'},
        date_of_decision: {type: Mongoose.Schema.Types.Date, required: true,
// Primary key (performance issues!):
schema.index({decision_type_number: 1, prison_file_number: 1, date_of_decision: 1}, {unique: true});
export const JudicialDecisionModel = Mongoose.model<JudicialDecision>('JUDICIAL_DECISION', schema);
// By default, Mongoose adds an '_id' property to any document:
console.assert('_id' in new JudicialDecisionModel(),"'_id' in new JudicialDecisionModel()");

mongoose, query (as a service)

router.get('/', (request, response) => { // curl http://localhost:1963/NYCP/API/Under_remand
                    from: Get_collection_name(), // Search in 'judicial_decisions' collection...
                    let: {prisoner_id: "$_id"}, // Get '_id' field in 'prisoners' collection...
                    pipeline: [
                        {$match: {$expr:
                            { // 'prisoner' field in 'judicial_decisions' collection matches with 'prisoner_id'
                                $and: [{$eq: ["$decision_type_number", "1"]}, {$eq: ["$prisoner", "$$prisoner_id"]}]
                    as: "convictions"
        // Only keep those for whom 'convictions' array (added) field is empty, i.e., no conviction yet...
        {$match: {convictions: {$size: 0}}},
        // Only 'prison_file_number' data...
        {$project: {_id: false, prison_file_number: true}}
    ]).then(data => response.status(200).json(data)).catch(error => response.status(400).json({error, message: error.message}));

Dangling references (i.e., SQL ON DELETE CASCADE)

There is no direct support for managing dangling references.

Back-end For Front-end (BFF) design pattern (see also: here…)

Node.js, app. configuration

Any Node.js app. benefits from being configured from environment variables (here…) following one of the Twelve-Factor App. Example (Configuration.ts):

export const corsMethods = process.env.CORS_METHODS;
export const corsUrl = process.env.CORS_URL;
export const database = { // MongoDB Atlas (
    name: process.env.DB_NAME || 'New_York_City_Penitentiary_database',
    password: process.env.DB_PASSWORD || 'YouMayKnowMyPasswordFor$100.000...',
    user: process.env.DB_USER || 'FB'
console.assert(process.env.NODE_ENV === 'DEBUG' || process.env.NODE_ENV === 'RELEASE'); // 'DEBUG' or 'RELEASE' values expected...
export const port = process.env.PORT || 1963; // If 'PORT' env. variable is unset then exposes API at port '1963'...
// console.log(process.env); // Caution: sensitive data may be displayed...

Node.js, app. configuration (.env file)

The .env file must include values for the needed environment variables. Example (.env):



DB_PASSWORD = YouMayKnowMyPasswordFor$100.000...
DB_NAME = New_York_City_Penitentiary_database

PORT = 1963

Node.js, app. configuration with dotenv

dotenv simply loads data from .env file. Example (package.json and next TypeScript):

"dependencies": {…, "dotenv": "^8.2.0", …},
import dotenv from "dotenv"; // Caution: it must be installed in '"dependencies"'!
// Load data from '.env' file:
// console.log(JSON.stringify(process.env)); // Caution: sensitive data may be displayed...

Node.js, app. configuration with dotenv and if-env

dotenv can be preloaded by means of the -r Node.js option. As a specific usage, dotenv acts as a dev. dependency only! In other words, the app. code does not rely on dotenv as a library.

Example (package.json)

"devDependencies": {…, "dotenv": "^8.2.0", …},
node -r dotenv/config.js dist/API.js

Example (package.json)

"devDependencies": {…, "dotenv": "^8.2.0", …, "if-env": "^1.0.4", …},
"scripts": {
    "server:NODE": "node -r dotenv/config dist/API.js",
    "server:NODEMON": "nodemon -r dotenv/config dist/API.js",
    "start": "if-env NODE_ENV=RELEASE && npm run start:RELEASE || npm run start:DEBUG",
    "start:DEBUG": "npm run build && npm run server:NODEMON",
    "start:RELEASE": "npm run build && npm run server:NODE",

App. utility: nodemon

The nodemon command just aims at replacing the node command. Added value is the fact that nodemon automatically reloads JavaScript source files at changing time…

Example (package.json)

"devDependencies": {…, "nodemon": "^2.0.7", …},
"scripts": {
    "server:NODEMON": "nodemon -r dotenv/config dist/API.js",
    "start": "if-env NODE_ENV=RELEASE && npm run start:RELEASE || npm run start:DEBUG",
    "start:DEBUG": "npm run build && npm run server:NODEMON",


Messages (beyond an e-mail, a SMS…) are streamed data whose processing (filtering, aggregating…) calls for real time. Brokers are middleware where messages are published and consumed by subscribers. Publishers and subscribers are “clients”.

Apache Kafka, Mosquitto or, RabbitMQ are popular message brokers. Messaging protocols are varied, but MQTT is an OASIS worldwide standard and, as such, widely supported.

Messaging broker clients require appropriate libraries like KafkaJS (Apache Kafka), MQTT.js (Mosquitto), or amqp.js (RabbitMQ) to allow clients' connection and exchange with messaging brokers.

Intuitive messaging versus broker-based messaging (right-hand side)


The native WebSockets API (example in browser) allows the connection and exchange with a WebSockets server like the one here…. To have a WebSockets server on the top of Node.js calls for appropriate libraries* like ws.

import * as WebSocket from "ws"; // '"@types/ws"'
const Launch_WebSockets_server = (configuration: Object = {host: 'localhost', path: '/FranckBarbier', port: 1963}) => {
    const server = new WebSocket.Server(configuration);
    server.on('connection', (ws: WebSocket) => {
        console.log('OnOpen... status: ', ws.readyState === 1 ? "OPEN" : "Not OPEN");
        ws.on('message', (event: WebSocket.MessageEvent) => { console.log('Message from client: %s', event); });
        ws.send("{\"Connected\": \"Yes\"}");
export default Launch_WebSockets_server;

*Even though uses the WebSockets technology as transportation facility, it is incompatible with the WebSockets API.

Security (authentication) Passport

Passport supports the OAuth protocol, which allows Facebook, Google, Twitter… authentication.


Performance (load balancing) NGINX

NGINX supports…


Performance (load balancing) Cluster API



Nice libraries

Internet of Things (IoT): Node-RED

© Franck Barbier