MongoDB

MongoDB Basics
Documents
• MongoDB is a document database which means that the equivalent of a record is a
document, or an object. In a relational database, we organize data in terms of rows
and columns, whereas in a document database, an entire object can be written as a
document.
• A document is a data structure composed of field and value pairs. The values of fields
include objects, arrays, and arrays of objects and so on, as deeply nested as you want
it to be.
• MongoDB documents are similar to JSON objects (as JavaScript objects).
• MongoDB document has support for the primitive data types - Boolean, numbers, and
strings and also common data types such as dates, timestamps, regular expressions,
and binary data.
Example:
• In MongoDB, the entire Invoice object would be stored as one document. That is
because a document can contain arrays and other objects in a nested manner and the
contained objects don’t have to be separated out into other documents.
• An Invoice object may look like this:
Collections
• Collection is like a table in a relational database: it is a set of documents.

• Collection can have a primary key and indexes.

➢ A primary key is mandated in MongoDB, and it has the reserved field name _id.
Even if _id field is not supplied when creating a document, MongoDB creates this
field and auto-generates a unique key for every document. MongoDB uses a
special data type called the ObjectId for the primary key.
➢ The _id field is automatically indexed. Apart from this, indexes can be created on
other fields, and this includes fields within embedded documents and array fields.
Indexes are used to efficiently access a subset of documents in a collection.

• MongoDB does not require us to define a schema for a collection.

• All documents in a collection must have a unique _id. All documents in a collection
do have the same fields.

• MongoDB has supported a concept of schema. A schema can enforce allowed and
required fields and their data types. It can also validate other things like string length
and minimum and maximum values for integers.

Databases
• A database is a logical grouping of many collections. Since there are no foreign keys
like in a SQL database, the concept of a database is nothing but a logical partitioning
namespace.
• Most database operations read or write from a single collection.
• $lookup, which is a stage in an aggregation pipeline, is equivalent to a join in SQL
databases. This stage can combine documents within the same database.
• A database connection is restricted to accessing only one database, so to access
multiple databases, multiple connections are required.
• We can keep all the collections of an application in one database, though a database
server can host multiple databases

Query Language
• MongoDB query language is made up of methods to achieve various operations.
• The main methods for read and write operations are the CRUD methods. Other
methods include aggregation, text search, and geospatial queries.
• All methods operate on a collection and take parameters as JavaScript objects that
specify the details of the operation. Each method has its own specification.
Example: To insert a document, the only argument needed is the document itself and
for querying, the parameters are a query filter and a list of fields to return (also called
the projection).
• The query filter is a JavaScript object consisting of zero or more properties where the
property name is the name of the field to match on and the property value consists of
another object with an operator and a value.
Example - To match all documents with the field invoiceNumber that are greater than
1,000, the following query filter can be used:
{"invoiceNumber": {$gt: 1000}}
• MongoDB encourages denormalization, that is, storing related parts of a document as
embedded subdocuments rather than as separate collections (tables)
Example - Consider people (name, gender, etc.) and their contact information
(primary address, secondary address etc.)
• In MongoDB, it can be stored as a list of contacts within the same People document.

Installation
• For installing MongoDB on your computer (it can be installed easily on OS X,
Windows, and most distributions based on Linux), look up the installation
instructions, which are different for each operating system.
• We could install MongoDB by following the instructions at the MongoDB website
(https://docs.mongodb.com/manual/installation/ or search for “mongodb installation”
in your search engine).
➢ Choose MongoDB version 3.6 or higher.
• After a local installation, ensure that we have started MongoDB server.
• Test the installation by running the mongo shell like this
$ mongo
• On a Windows system, we may need to append .exe to the command.
• The command may require a path depending on your installation method. If the shell
starts successfully, it will also connect to the local MongoDB server instance.
MongoDB shell version v4.0.2
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.2
>
The Mongo Shell
• The mongo shell is an interactive JavaScript shell. In the interactive shell, a few non-
JavaScript conveniences are available over and above the full power of JavaScript.
• The commands that we will be typing in the mongo shell is present in a file called
mongo_commands.txt. These commands have been tested to work as is on Atlas or a
local installation.
• To work with MongoDB, we need to connect to a database.
• The command to show the current databases is:
> show databases
• This will list the databases and the storage occupied by them.
• Example: In a fresh local installation of MongoDB, we will see:

• To identify the current database, the command is:

> db
• The default database a mongo shell connects to is called test. To see what collections
exist in this database.
> show collections
• To switch to a database, say called “issuetracker” instead of using the default database
command is:
> use issuetracker
• This should result in output that confirms that the new database is issuetracker:

• To confirm that there are no collections in this database command is:

> show collections
• Create a new collection.
➢ This is done by creating one document in a collection.
➢ A collection is referenced as a property of the global object db, with the same
name as the collection. The collection called employees can be referred to as
db.employees.
➢ Let’s insert a new document in the employees collection using the insertOne()
method.
> db.employees.insertOne({ name: { first: 'John', last: 'Doe' }, age: 44 })
• The result of this command will show you the result of the operation and the ID of the
new document that was created

• Other methods which are available on any collection (mentioned below)

• To check if the document has been created in the collection we can use the find()
method on the collection.
> db.employees.find()
• Above command result in displaying the document and to get a more legible output,
we can use the pretty() method on the result of the find() method:
> db.employees.find().pretty()

• To insert another document in the same collection and deal with multiple documents
in the collection:
> db.employees.insertOne({ name: { first: 'Jane', last: 'Doe' }, age: 54 })
• toArray() method assign its result to an array variable.
> let result = db.employees.find().toArray()
• Use the JavaScript array method forEach() to iterate through them and print the first
names of each employee:
> result.forEach((e) => print('First Name:', e.name.first))

• The mongo shell, provides the print() method for printing objects on the console but
this prints only strings.
• Objects need to be converted to strings before printing, using the utility function
tojson(). This method prints objects as JSON.
> result.forEach((e) => printjson(e.name))
Output

MongoDB CRUD Operations

1. Create
• Code to create our own ID instead of letting MongoDB auto-generate one.

Here,
➢ value of insertedId reflected the value that we supplied for _id means that, instead
of an ObjectID type of value, we were able to supply our own value.
• If we create a new identical document. It will fail with the below error. This shows
that the _id field continues to be a primary key and it is expected to be unique.

• Let’s add another document, but with a new field as part of the name. This works just
fine, and using find(), we can see that two documents exist in the collection

• The schema can be enhanced whenever a new data element that needs to be stored is
discovered, without having to explicitly modify the schema.
• Let’s drop the collection and start creating new documents with a new field called id.

• The collection has a method that can take in multiple documents in one go. This
method is called insertMany(). We need to use this to create a few more documents
in a single command.
2. Read
• The find() method takes in two arguments.
1. The filter to apply to the list.
2. The projection, a specification of which fields to retrieve.
Filter
• The filter specification is an object where the property name is the field to filter on,
and the value is its value that it needs to match.
• The method findOne() is a variation of the method find(), and it returns a single object
rather than a cursor.
> db.employees.findOne({ id: 1 })

• The filter is actually a shorthand for {id: {$eq: 1}},

➢ where $eq is the operator signifying that the value of the field id has to be equal to
1.
• The format of a single element in the filter is fieldname:
{operator: value}
• Other operators for comparison are available, such as
➢ $gt for greater than
➢ $gte for greater than or equal to
Example: For fetching a list of employees aged 30 or older:
> db.employees.find({ age: { $gte: 30 } })
➢ $and operator has to be used, which takes in an array of objects specifying
multiple field-value criteria.
• The createIndex() method on the collection is meant for creating an index on that
field. It takes in an argument specifying the fields that form the index (multiple fields
will form a composite index).
Example: To create an index on the age field
> db.employees.createIndex({ age: 1 })
• With this index, any query that uses a filter that has the field age in it will be
significantly faster because MongoDB will use this index instead of scanning through
all documents in the collection.
• MongoDB automatically creates an index on the _id field.
Example:
➢ Let’s create an index on a field and it has to be unique since it identifies the
employee: no two employees should have the same value for id.
➢ The second argument to createIndex() is an object that contains various attributes
of the index, one of them specifying whether the index is unique.
➢ Let’s use that to create a unique index on id:
> db.employees.createIndex({ id: 1 }, { unique: true })
Projection
• To restrict the fetch to only some fields, the find() method takes a second argument
called the projection. A projection specifies which fields to include or exclude in the
result.
• The format of this specification is an object with one or more field names as the key
and the value as 0 or 1, to indicate exclusion or inclusion.
• The _id field is an exception; it is always included unless you specify a 0.
• The following will fetch all employees but only their first names and age:
>db.employees.find({}, { 'name.first': 1, age: 1 })
Output:

• To suppress the inclusion of this field, it needs to be explicitly excluded, like this
> db.employees.find({}, { _id: 0, 'name.first': 1, age: 1 })
Output
3. Update
• There are two methods updateOne() and updateMany() available for modifying a
document.
• The arguments to both methods are the same, except that updateOne() stops after
finding and updating the first matching document.
➢ The first argument is a query filter, the same as the filter that find() takes.
➢ The second argument is an update specification if only some fields of the object
need to be changed.
• When using updateOne(), the primary key or any unique identifier is normally used in
the filter because the filter can match only one document.
• The update specification is an object with a series of $set properties whose values
indicate another object, which specifies the field and its new value.
Example:
To update the age of the employee identified by the id 2:
db.employees.updateOne({ id: 2 }, { $set: {age: 23 } })

• The matchedCount returns how many documents matched the filter.

• To modify multiple documents in one shot, the updateMany() method has to be used.
• The format is the same as the updateOne() method, but the effect is that all documents
that match will be modified.
> db.employees.updateMany({}, { $set: { organization: 'MyCompany' } })
• There is also a method to replace the complete document called replaceOne()
• It can be used to just replace the existing document with the new one.
Example: Below command will replace the existing document with ID 4, with the
new one
4. Delete
• The delete operation takes a filter and removes the document from the collection.
• Available delete operation are.
1. deleteOne()
2. deleteMany()
• Example: To delete the last document, with ID 4:
> db.employees.deleteOne({ id: 4 })

Aggregate
• Summary or an aggregate, counts documents that match a certain condition.
• aggregate() method performs the function of the GROUP BY clause. It can also
perform other functions such as a join, or even an unwind (expand the documents
based on arrays within).
• The aggregate() method works in a pipeline. Every stage in the pipeline takes the
input from the result of the previous stage and operates as per its specification to
result in a new modified set of documents.
• The initial input to the pipeline is, the entire collection. The pipeline specification is in
the form of an array of objects, each element being an object with one property that
identifies the pipeline stage type and the value specifying the pipeline’s effect.
Example:
• The find() method can be replicated using aggregate() by using the stages $match (the
filter) and $project (the projection)
• To perform an actual aggregation, the $group stage needs to be used.
• The stage’s specification includes the grouping key identified by the property _id and
other fields as keys, whose values are aggregation specifications and fields on which
the aggregation needs to be performed.
• The _id can be null to group on the entire collection.
• Let’s try this by getting the total age of all employees in the entire collection.
• We will need to sum (using the aggregate function $sum) the field age into a new
field called total_age like this:
• The same function, $sum, can be used to get a count of the records by summing the
value 1:

• To group the aggregate by a field, we will need to specify the name of the field
(prefixed by a $) as the value of _id.
MongoDB Node.js Driver
• Node.js driver lets us connect and interact with the MongoDB server. It provides
methods similar to mongo shell. We could use an Object Document Mapper called
Mongoose, which has a higher level of abstraction and more convenient methods.
• To install the driver:
$ npm install mongodb@3
• To make a connection to the database server we need to follow below steps
1. Import the object MongoClient from the driver.
2. Creating a new client object from it using a URL that identifies a database to
connect to.
3. Call the connect method.

• The URL should start with mongodb:// followed by the hostname or the IP address of
the server to connect to.
• For the local installation, the URL will be
mongodb://localhost/ issuetracker
Example: Let’s add the local installation URL to trymongo.js and a commented
version of cloud providers’ URLs

• To avoid a warning in the latest Node.js driver (version 3.1) we need to pass argument
in constructor.

• Connection to the database can be obtained by calling the db method of the client
object.
• The callback and connection to the database can be written like this:

Example: Let’s insert a document and read it back.

• The insertion can be written using the insertOne method, passing in an employee
document and a callback.
• Created ID is returned as part of the result object, in the property called insertedId.

• We could use either the ID we supplied (id) or the auto-generated MongoDB ID (_id)
to read back the inserted document

• To close the connection to the server we use

• If there are any errors, then these can be passed to the callback function like below

• Within each callback as a result of each of the operations, on an error we need to do

the following:
1. Close the connection to the server
2. Call the callback
3. Return from the call, so that no more operations are performed
• The pattern of the error handling is like:

Example:
• trymongo.js: Using Node.js driver, Using the Callbacks Paradigm
• For the local installation, the command will look like this:
$ mongo issuetracker --eval “db.employees.remove({})”
• To test the trial program can be executed like this
$ node scripts/trymongo.js
• All asynchronous calls with a callback can now be replaced by a call to the same
method, but without supplying a callback.
• Using await before the method call will simulate a synchronous call by waiting for the
call to complete and return the results.
Example

• The new function using await before each of the operations connect(), insertOne(),
and find() could be written as below
Schema Initialization
• Using scripts, we can perform schema initialization
• Steps to create a schema initialization script called “init.mongo.js” within the script
directory.
1. Create indexes
2. Initialize the database with some sample documents
Example:
• issuetracker database to store all the collections relevant to the Issue Tracker
application.
• Copy the array of issues from server.js and use the same array to initialize the
collection using insertMany() on a collection called issues
Below program shows contents of the initialization script, init.mongo.js
Reading from MongoDB
• Store the connection to the database in a global variable and call the global database
connection variable db:

• Write a function to connect to the database, which initializes this global variable.

• Change the setup of the server to first connect to the database and then start the
Express application.
• Since connectToDb() is an async function, we can use await to wait for it to finish,
then call app.listen()

• Enclose the contents of this anonymous function within a try block and print any
errors on the console in the catch block:
• Now we have a connection to the database set up in the global variable called db, we
can use it in the List API resolver issueList() to retrieve a list of issues by calling the
find() method on the issues collection.

• Example: Complete program of server.js: Changes for Reading the Issue List from
MongoDB
Writing to MongoDB
• To create a new document is to use the insertOne() method on the collection. Use the
size of the in-memory array to generate the new document’s id field.
• MongoDB support an atomic update operation, which can return the result of the
update. This method is called findOneAndUpdate().
• Create a collection with the counter that holds a value for the latest Issue ID
generated.
• Modify the schema initialization script to include a collection called counters and
populate that with one document for the counter for issues.
• The changes are in init.mongo.js

• A call to findOneAndUpdate() that increments the current field is guaranteed to return

a unique value that is next in the sequence.
• It identifies the counter to use the ID supplied, increments the field called current, and
returns the new value.
• Result of this method returns the original document.
• The arguments to the method findOneAndUpdate() are
1. The filter or match, for which we used _id
2. The update operation, for which we used a $inc operator with value 1
3. The options for the operation.

• Use this function to generate a new ID field and set it in the supplied issue object in
the resolver issueAdd()
➢ We can write to the collection called issues using insertOne()
➢ We can read back the newly created issue using findOne()
• Example: server.js: Changes for Create API to Use the Database
Node JS: Introduction
• A common task for a web server can be to open a file on the server and return the
content to the client.
• Node.js handles a file request in below three steps
1. Sends the task to the computer's file system.
2. Ready to handle the next request.
3. When the file system has opened and read the file, the server returns the
content to the client.
• Node.js eliminates the waiting, and simply continues with the next request.
• Node.js runs single-threaded, non-blocking, asynchronous programming, which is
very memory efficient.
• Node.js can generate dynamic page content.
• Node.js can create, open, read, write, delete, and close files on the server.
• Node.js can collect form data.
• Node.js can add, delete, modify data in your database.

Setting up Node.Js
• We need to follow below steps to install Node.js on windows
1. Download the Node.Js “.msi” installer.
2. Run the Node.Js installer.
❖ Double click on the msi installer. Node.Js setup wizard will open and proceed
with the instructions and click on Finish button.
3. Verify Node.Js is installed or not
❖ It could we performed by running below commands in command prompt or
PowerShell
C:\Users\Admin\node -v
4. Updating the local npm version
❖ Final step in node.js is the updation of your local npm version – the package
manager that comes bundled with Node.Js
❖ To update the npm we need to run below command
npm install npm -global
• Components of Node JS, consists of three components
1. Import Required modules
➢ We use require directive to load the http module and store the returned Http
instance into an Http variable as follows
var http = require(“http”);
2. Create Server
➢ We use the created http instance and call http.CreateServer()method to create a
server instance and then we bind it at port 8081using the listen method associated
with the server instance.
➢ Pass it a function with parameters request and response.
3. Read request and return response
➢ Execute the main.js to start the server
$ node main.js
Output: Server has started.
• Node js main basic commands
➢ To check version of install node js
node -v
➢ To change the node js version
nvm use <version number>

CallBacks and Events

Callback
• A Callback is a function that is called automatically when a particular task is
completed. The Callback function allows the program to run other code until a certain
task is not completed.

• The callback function allows you to perform a large number of I/O operations that can
be handled by your OS without waiting for any I/O operation to finish. It makes
nodeJS highly scalable.

• Example: In Node.js, when a function starts reading a file, it’s obviously going to
take some time so without waiting for that function to complete, it returns control to
the execution environment immediately an executes the succeeding instruction. Once
file I/O gets completed, the callback function will be automatically called hence there
will be no waiting or blocking for file I/O.
• Syntax of the callback function

• Example: Code for reading a file asynchronously in Node.js.

With the help of the readFile() function, we are able to use the Asynchronous
approach here, which is also called a non-blocking function as it never waits for each
instruction to complete, rather it executes all operations in the first go itself.

Output:

Events
• Node.js has an event-driven architecture that can perform asynchronous tasks.

• Node.js has an ‘events’ module that emits named events that can cause corresponding
functions or callbacks to be called.

• Functions (Callbacks) listen or subscribe to a particular event to occur and when that
event triggers, all the callbacks subscribed to that event are fired one by one in order
to which they were registered.
• The EventEmmitter class: All objects that emit events are instances of the
EventEmitter class. The event can be emitted or listened to an event with the help of
EventEmitter.

Syntax:

• Lists all the important methods of the EventEmitter class

1. eventEmitter.addListener(event, listener) && eventEmitter.on(event, listener)

2. eventEmitter.once(event, listener)
3. eventEmitter.emit(event, [arg1], [arg2], […])
4. eventEmitter.removeListener(event, listener)
5. eventEmitter.removeAllListeners()
6. eventEmitter.getMaxListeners(n)
7. EventEmitter.defaultMaxListeners
8. eventEmitter.listeners(event)
9. eventEmitter.listenerCount()
10. eventEmitter.prependOnceListener()
11. eventEmitter.prependListener()
• Example: Code for creating a program of a simple event:
File System
• The fs (File System) module in Node.js provides an API for interacting with the file
system.

• It allows you to perform operations such as reading, writing, updating, and deleting
files and directories, which are essential for server-side applications and scripts.

• All file system operations can have synchronous and asynchronous forms depending
upon user requirements. To use this File System module, use the require() method

const fs = require('fs');

1. Open a File

• The fs.open() method is used to create, read, or write a file.

• Syntax:

fs.open(path, flags, mode, callback)

• Here,

❖ path: It holds the name of the file to read or the entire path.

❖ flags: It indicate the behaviour of the file to be opened.

❖ mode: Sets the mode of file i.e. r-read, w-write, r+ -readwrite.

❖ err: If any error occurs.

❖ data: Contents of the file.

• Example:
2. Reading a File

• The fs.read() method is used to read the file specified by fd.

• This method reads the entire file into the buffer.

• Syntax:

fs.read(fd, buffer, offset, length, position, callback)

• Here,

❖ fd: This is the file descriptor returned by fs.open() method.

❖ buffer: This is the buffer that the data will be written to.

❖ offset: This is the offset in the buffer to start writing at.

❖ length: This is an integer specifying the number of bytes to read.

❖ position: This is an integer specifying where to begin reading from in the file. If
the position is null, data will be read from the current file position.

❖ callback: It is a callback function that is called after reading of the file

• Example:
3. Writing to a File

• The fs.writeFile() method is used to asynchronously write the specified data to a file.

• Syntax:

fs.writeFile(path, data, options, callback)

• Example:

4. Appending to a File

• The fs.appendFile() method is used to synchronously append the data to the file.

• Syntax:

fs.appendFile(filepath, data, options, callback);

// or

fs.appendFileSync(filepath, data, options);

• Example:
5. Closing the File

• The fs.close() method is used to asynchronously close the given file descriptor
thereby clearing the file that is associated with it.

• Syntax:

fs.close(fd, callback)

• Example:

6. Delete a File

• The fs.unlink() method is used to remove a file from the filesystem.

• Syntax:

fs.unlink(path, callback)

• Example:

Buffers and Stream:

Buffers
• Nodejs Buffer is a fixed size area of memory allocated outside of the Javascript
engine. Once created they cannot be resized. Buffers are stored in a sequence of
integers.
• The Buffer class in Nodejs is designed to handle raw binary data.

• To create a new Buffer includes

1. Buffer.from()

2. Buffer.alloc(size)

3. Buffer.allocUnsafe(size)

1. Buffer.from():

❖ When creating a new Buffer from a string or array of integers, the corresponding
response will be encoded.
❖ The numbers in each index of the Buffer indicate the position of the character in
the Buffer.
❖ Example:

❖ To view the content of a string in Buffer, use buf.toString()

2. Buffer.alloc(size, fill)

❖ Buffer.alloc(size, fill) allocates a new Buffer of size bytes.

3. Buffer.allocUnsafe(size, fill)

❖ They are both used to create a new buffer and allocate default byte size and
values.

Streams in NodeJs
• Stream is the method of transferring large amounts of data in an efficient way.
• They are used to read or write input into output sequentially.
• Types of NodeJs Streams
1. Readable stream – A readable stream is an abstraction for a source from which
data can be read, in other words, it lets you read data from a source
2. Writeable stream – A writable stream is an abstraction for a destination to which
data can be written
3. Duplex stream – You can both read and write into it, in other words, it is a
combination of both readable and writeable streams. Example – net.Socket
4. Transform stream – It is similar to a duplex stream, but it can modify the data as it
is being written and read. Example – zlib.createGzip

1. Reading from a stream

• This is a stream you can read from but cannot send data into it.
2. Writing to a stream

• This is a stream used for writing data in chunks, instead of all at once.

3. Piping Streams

• Piping is a mechanism in NodeJs where the output of a stream is fed as an input into
another stream.