Server Side Pagination In NodeJS

In a real-world, we often have to deal with a large number of data. So just imagine a scenario when you have to manipulate 1 million records in the database and you want this data to show on a web page using an API in Node.JS.

It is not a good practice and of course, it is not a feasible. To handle such scenarios, we use pagination. Pagination is what showing records based 0n page numbers, just like a book.

There are mainly two ways to implement pagination which are as:

1. Client Side Pagination
2. Server Side Pagination

We usually do client-side pagination but my focus is on server side pagination. When we have to deal a large volume of data, we use Server side pagination.

Today, I am going to use Server Side pagination in a demo. For this demo, I am using MongoDB for storing the data and Node.JS to paginate using REST API.

Before going to the demo, create some mock data to test.

Generate mock data

Mockaroo is a website where you can generate mock data for testing. Visit http://www.mockaroo.com/

To generate some mock data and start back-end coding.

Select the fields what you want and download this data in any format i.e (CSV, JSON, etc).

Dumping data in MongoDB

MongoDB comes with a command line program which is Mongo Import. It is used to import a large amount of data directly into the MongoDB collection.

Here is the command.

mongoimport -d demo -c users --type csv MOCK_DATA.csv --headline

Note: the demo is the name of my database and users is a collection of demo database.Once above command executed successfully, it means data is imported, Now you can check this database in your Mongo shell using the following command.

Writing Server code for pagination

So now we are all set and ready to write server-side code to paginate this data.

Create a file (user.js) and paste this code. I had created mock data based on the following schema. You can create your own data as per your choice.

       // requiring dependencies
       var mongoose = require('mongoose');
       
       // creating schema
       var Schema = mongoose.Schema;  
       var userSchema = new Schema({
           firstName: String,
           lastName: String,
           email: String,
           gender: String,
           username: String,
           mobile: String
       });
         
       // make this schema available to use in any module
       module.exports = mongoose.model('users', userSchema);
       

Now create a server (server.js) and paste the following code:

var express = require('express');
       var mongoose = require('mongoose');
       var User = require('./user');
       mongoose.connect('mongodb://localhost:27017/pagination', { useMongoClient: true }, function(err, db) {
           if(err) throw err;
           console.log('database connected successfully');
       });
       var app = express();
       app.get('/', function(req, res) {
           res.send('<h1>welcome to server side pagination demo</h1><br><h3>Enter any url containing <b>/users/page_no/per_page</b></h3>');
       });
       app.get('/users/:page/:perPage', function(req, res) {
           console.log('page number : ' + req.params.page); 
           console.log('per page : ' + req.params.perPage);
           var pageNo = req.params.page ; // parseInt(req.query.pageNo)
           var size = req.params.perPage;
           var query = {}
           if (pageNo < 0 || pageNo === 0) {
               response = { "error": true, "message": "invalid page number, should start with 1" };
               return res.json(response)
           }
           query.skip = size * (pageNo - 1)
           query.limit = size
           // Find some documents
           User.find({}, {}, query, function (err, data) {
               // Mongo command to fetch all data from collection.
               if (err) {
                   response = { "error": true, "message": "Error fetching data" };
               } else {
                   response = { "error": false, "message": data };
               }
               res.json(response);
           });
       });
       app.listen(8083, function(req, res) {
           console.log('server started on port : 8083');
       });
       

Once you have written server.js, we are now ready to test it. When we run server.js using command:

$ node server.js

And navigate to http://localhost:8083/  We will see following.

Now add any page number and records per page in the url as:

http://localhost:8083/users/3/10

Where 3 is page number and 10 is the number of records per page. The  response returned from the server is:

Thanks,