Memory Leak in Nodejs | Detect the Memory Leak In Your Node App | Profiling Node App

Let's implement memory leak in nodejs, detect the memory leak in node app and profiling node app:

M

emory Leak is a dangerous and big problem for your app. So we need to learn how we can detect the memory leak in your node app. Before doing this you need to know how you can detect the what memory is being used by your application, to learn this you can check out my previous blog:

Profiling Node App: Detect the memory uses of node app | Use of -inspect | Heapdump | Heap Snapshot

In this blog we will achieve our goal doing below stages:

Stage 1: Will create a node app.

Stage 2: Will open our app in chrome dev tool.

Stage 3: Will create the heap snapshots and get the memory size, consumed by our application

Stage 4: Will compare the heap snapshots.

Stage 5: Will change some code in our app so that we can achieve the goal.

Stage 6: Will see the memory leak in our app.

Now, let’s start

Stage 1: Create a node app:

Step 1: Let’s create a basic node.js application, create two files with the name app.js and package.json:

package.json

                        {
                            "name": "memory-leak",
                            "version": "1.0.0",
                            "private": true,
                            "scripts": {
                                   "start": "node --inspect app.js"
                              },
                            "author": "Shubham Verma",
                            "license": "",
                            "dependencies": {
                                             "express": "^4.14.1",
                                             "fast-levenshtein": "^2.0.6"
                                 }
                            }
                    

app.js:

                                const express = require('express');
                                const console = require('console');
                                const levenshtein = require('fast-levenshtein');//When set to 100 it should be the only visible operation.
                                const HOW_OBVIOUS_THE_FLAME_GRAPH_SHOULD_BE_ON_SCALE_1_TO_100 = 10;
                                const someFakeModule = (function someFakeModule () {return {
                                calculateStringDistance (a, b) {
                                //Here's where heavy sunchronous computation happens
                                return levenshtein.get(a, b, {
                                    useCollator: true
                                })
                                }
                                }
                                })()const app = express();
                                app.get('/', (req, res) => {
                                res.send(`
                                            <h2>Take a look at the network tab in devtools</h2>
                                            <script>
                                                function loops(func) {
                                                    return func().then(_ => setTimeout(loops, 20, func))
                                                }
                                                loops(_ => fetch('api/tick'))
                                            </script>
                                        `)
                                });app.get('/api/tick', (req, res) => {
                                                Promise.resolve('asynchronous flow will make our stacktrace more realistic'.repeat(HOW_OBVIOUS_THE_FLAME_GRAPH_SHOULD_BE_ON_SCALE_1_TO_100))
                                .then(text => {
                                                const randomText = Math.random().toString(32).repeat(HOW_OBVIOUS_THE_FLAME_GRAPH_SHOULD_BE_ON_SCALE_1_TO_100)
                                                return someFakeModule.calculateStringDistance(text, randomText)
                                })
                                .then(result => res.end(`result: ${result}, ${arr.length}`))
                                });app.get('/api/end', () => process.exit())
                                app.listen(8080, () => {
                                console.log(`go to http://localhost:8080/ to generate traffic`)
                                })
                    
                    

Step 2: Go to the app location and run below command

In this step, we need to install all the dependencies, so run the below command.

npm install

Detect the Memory Leak In Your Node App | Profiling Node App

Step 3: After the successful installation, run bellow command:

Now we need to run the command "--inspect" with node command.

node --inspect app.js

Detect the Memory Leak In Your Node App | Profiling Node App

Step 4: Create URL:

This step is very important and error prone, so be careful at this step, after running the `node — inspect app.js` command you can see an URL start with ‘ws’, just like in above snapshot ‘ws:127.0.0.1:9229/4ba52d1b-95c8–4b19-a9bf-bfb2b5ef8732’ Now what you need to do is, you need to make an URL like this:

devtools://devtools/bundled/js_app.html?experiments=true&v8only=true&ws=127.0.0.1:9229/4ba52d1b-95c8–4b19-a9bf-bfb2b5ef8732

Stage 2: open our app in chrome dev tool:

We have completed the stage 1 and now we need to follwo the stage 2 from here. In this stage we have some steps also as we have in stage 1.

Step 1: Now, open the above URL:

Now, open the above URL in the browser, make sure the URL is correct, otherwise, you will get the following error:

Detect the Memory Leak In Your Node App | Profiling Node App

Detect the Memory Leak In Your Node App | Profiling Node App

( Suggestion: make your URL into browser’s address bar )
If your URL is correct you can see below the page in your browser:

Detect the Memory Leak In Your Node App | Profiling Node App

Step 2: Click on the link ‘http://localhost:8080/’, it will open a tab with URL ‘http://localhost:8080/’ ( you can directly open the URL `http://localhost:8080/` into browser )

Step 3: After clicking/opening URL `http://localhost:8080/` you can see the below page, also open network tab in dev tools.

Detect the Memory Leak In Your Node App | Profiling Node App

You can see there are many requests are hitting by the browser. if not then check all the steps again ( you are doing wrong somewhere ) If yes then move to the next step.

Stage 3: Create the heap snapshots and get the memory size, consumed by our application

Step 1: Now open the dev tool tab ( which is previously opened in “stage 2 & step 1” ) Click on “Memory”, Select “heap snapshot” under the “select profiling type” and click on the below button “Take snapshot”.

Detect the Memory Leak In Your Node App | Profiling Node App

Step 2: After clicking the “Take snapshot”, you can see there is a snapshot taken under the “profiles-> HEAP SNAPSHOT”. Now click on that “snapshot 1”.

Detect the Memory Leak In Your Node App | Profiling Node App

Detect the Memory Leak In Your Node App | Profiling Node App

Step 3: After clicking on that “Snapshot 1”, You can see that each and every object created for this app. * You can also see the num of that variable which is created to run this app.

Detect the Memory Leak In Your Node App | Profiling Node App

* You can see how much memory is being used for this app.

Detect the Memory Leak In Your Node App | Profiling Node App

You can see in the above snapshot, the app is consuming total of 8.3 MB memory. * You can see the “Shallow size” of that kind of variables.

Detect the Memory Leak In Your Node App | Profiling Node App

* You can see the “Retained Size” of that kind of variables.

Detect the Memory Leak In Your Node App | Profiling Node App

Stage 4: Let’s compare the heap snapshots

To compare the heap snapshots, we need to multiple heap snapshot. You can take multiple snapshots and compare each and everything by clicking on the left button as shown in the below snapshot ( Make sure you are taking snapshot after 3 to 5 seconds )

Detect the Memory Leak In Your Node App | Profiling Node App

I have taken the multiple snapshots as :

Detect the Memory Leak In Your Node App | Profiling Node App

In the above image, we can see the memory size consumed by our app, it shows that it usages limited ( No increment of memory size) memory space and its fixed to 7.9 MB. It means there is no memory leak. Now we will change our code to see the memory leak.

Stage 5: Let’s change some code in our app so that we can achieve the goal.

Step 1:

In this, we will add an array of object and will insert some data so that each time we hit the API ( hitting constantly already in our app ), the size of array will increase and in last it will consume our all memory and app will be crashed ( but it will not be happening here because of good memory in my system).
So just copy and paste below code into your app.js file:
app.js:

                        const express = require('express');
                        const console = require('console');
                        const levenshtein = require('fast-levenshtein');//When set to 100 it should be the only visible operation;
                        const HOW_OBVIOUS_THE_FLAME_GRAPH_SHOULD_BE_ON_SCALE_1_TO_100 = 10;
                        let memory_leak=[];
                        const someFakeModule = (function someFakeModule() {
                        return {
                        calculateStringDistance(a, b) {
                        return levenshtein.get(a, b, {
                        useCollator: true
                        })
                        }
                        }
                        })();
                        const app = express();
                        app.get('/', (req, res) => {
                        res.send(`
                        <h2>Take a look at the network tab in devtools</h2>
                        <script>
                            function loops(func) {
                                return func().then(_ => setTimeout(loops, 20, func))
                            }
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                            loops(_ => fetch('api/tick'));
                        </script>
                        `)
                        });
                        app.get('/api/tick', (req, res) => {
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        memory_leak.push({name:"Shubham Verma"});
                        Promise.resolve('asynchronous flow will make our stacktrace more realistic'.repeat(HOW_OBVIOUS_THE_FLAME_GRAPH_SHOULD_BE_ON_SCALE_1_TO_100))
                        .then(text => {
                        const randomText = Math.random().toString(32).repeat(HOW_OBVIOUS_THE_FLAME_GRAPH_SHOULD_BE_ON_SCALE_1_TO_100)
                        return someFakeModule.calculateStringDistance(text, randomText)
                        })
                        .then(result => res.end(`result: ${result}, ${memory_leak.length}`))
                        }); app.get('/api/end', () => process.exit())
                        app.listen(8080, () => {
                        console.log(`go to http://localhost:8080/ to generate traffic`)
                        });
                    

Step 2: Go to the app location and run the command:

node --inspect app.js

Detect the Memory Leak In Your Node App | Profiling Node App

Step 3: Run the code and open dev tool by creating new URL ( as we described in stage 1+ step 3 & 4 then follow stage 2 ).

Stage 6: Detect the memory leak in our app.

Step 1: Now take the multiple heap snapshots as:

Detect the Memory Leak In Your Node App | Profiling Node App

In the above image, you can see the memory size of each snapshot is increasing. It means something went wrong in your code, You code is storing memory which is not good ( As we did it to see the memory leak ).
You can also see the size of variables is increasing:

So make sure your code should not store the memory at all, and your heap size should be the same after accepting the “n” number of request.
That's it.

The best use of it:

You can take snapshots before hitting an API and after hitting “N” API, Now you can compare the differences between. You can check where you need to improve. In the next blog, I will tell you how you can get the execution time of each function in your node app, also I will tell you how you can create the “flame chart” for your node app.

Congratulations… You are becoming an expert in node.js.