Processing large amounts of data in JavaScript

Have you ever wondered why when you are processing a lot of data (for example thousands of DOM manipulation) browser usually freezes? It’s because of the way JavaScript works. It makes use only of one thread at a time. It can’t multitask. So all the events are being queued – mouse click, animations, data processing and so on. When you do some large computation, JavaScript is solving that problem and only after it is finished with it, then it moves to other events.

How do we solve such kind of a problem?

Well, there is an easy solution. We’ll use timers! setTimeout(function, delay) works in such a way that it postpones the execution of the function by the delay specified. Firstly, let’s look at the slow way of processing data.

Processing a lot of data in the slow way

Create a simple webiste with one div element in the body. It might look like this:

<!DOCTYPE html>
<html>
<head>
    <title>Noo, so sloow</title>
</head>
<body>

    
</body> </html>

Now, it’s time to add JavaScript code into the page. Copy and paste following code under the div element.


        var content = document.getElementById("content"); //get the div in which the data will appear

        for (var i = 0; i 

It’s time to fire up your favorite browser and give it a try. Does it freeze? It took quite a long time to complete on my PC. Really bad, isn’t it.. The question is, how can we make it more responsive and user-friendly? Timers, of course.

A better way of battling with large amounts of data

We will divide the execution of the code into several parts. After the first part is processed, the timer will schedule itself to execute after 1 ms but in reality, it will execute as fast as it can after everything else has been done. Processing data in this way ‘unfreezes’ the browser for a certain period in which it can handle other things such as mouse clicks and animations. Let’s look at the code.


        var numOfDivs = 20000; // number of the main loop
        var divideIntoParts = 5000; // number of processing parts
        var partSize = numOfDivs / divideIntoParts; // number of iterations in one part
        var iteration = 0; // iteration number

        var content = document.getElementById("content"); //get the div in which the data will appear

        setTimeout(function generateData() {
            for (var i = 0; i 

Although the script takes a long time to complete, it doesn’t slow down the browser and user can do other stuff on the website. My specific example is definitely overblown. You probably won’t do so much data processing on one page. But if you would, setTimeout can significantly help you.

In conclusion, I want to thank the authors of the book Secrets of the JavaScript Ninja because I learned it there. I absolutely positively recommend buying that book 🙂

 

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: