MongoDB Overview

Introduction

MongoDB is a powerful, flexible, and scalable general-purpose database system designed for modern applications. ### Usability

MongoDB is a document-oriented database rather than a relational one. This approach primarily aims to achieve better scalability. Compared to relational databases, document-oriented databases eliminate the concept of "rows" in favor of a more flexible "document" model.

By embedding documents and arrays within documents, the document-oriented approach can represent complex hierarchical relationships using a single record, aligning with how modern object-oriented language developers perceive data. Additionally, there's no predefined schema: document keys and values are not fixed types or sizes. This flexibility makes adding or removing fields easier, accelerating development cycles and facilitating experimentation.

Scalability

Application datasets are growing at unprecedented rates. With increasing bandwidth and declining storage costs, even small applications may need to handle massive amounts of data that exceed many databases' processing capabilities.

As data volumes grow, developers face the question of how to scale databases. Vertical scaling is the simplest approach but has limitations as mainframes are expensive and physical limits exist. Horizontal scaling is more appropriate when vertical scaling reaches its limits, but it introduces the challenge of managing multiple servers.

MongoDB is designed for horizontal scaling. Its document data model enables easy data partitioning across multiple servers. MongoDB automatically handles cross-cluster data distribution and load balancing, automatically redistributing documents and routing requests to the appropriate machines. This allows developers to focus on application development without worrying about scaling issues. When a cluster needs more capacity, simply adding new servers causes MongoDB to automatically redistribute existing data to them.

Rich Features

Beyond basic CRUD operations, MongoDB offers a continuously expanding set of unique features:

• Indexing: Supports secondary indexes for various fast queries, including unique, composite, geospatial, and full-text indexes.
• Aggregation: Provides an aggregation pipeline allowing users to create complex operations through simple stages, with database optimization.
• Special Collection Types: Supports time-limited collections for data that expires at a certain time (like sessions) and fixed-size collections for recent data (like logs).
• File Storage: Offers a simple protocol for storing large files and metadata.

Performance

A primary goal of MongoDB is to deliver exceptional performance, which significantly influences its design. MongoDB utilizes as much memory as possible for caching, automatically selects appropriate indexes for queries, and optimizes various aspects to maintain high performance.

MongoDB Fundamentals

Documents

Documents are the core concept in MongoDB. A document is an ordered set of key-value pairs, like {'message': 'hello', 'number': 3}, similar to an ordered dictionary in Python.

Key points about documents:

• Key-value pairs in documents are ordered.
• Values can be strings in double quotes or other data types (including embedded documents).
• MongoDB distinguishes between types and is case-sensitive.
• Documents cannot have duplicate keys.
• Values can be various data types or complete embedded documents. Keys are strings, with few restrictions on which UTF-8 characters can be used.

Document Key Naming Conventions:

• Keys cannot contain \0 (null character).
• Periods (.) and dollar signs ($) have special meanings and should only be used in specific contexts. li>Keys starting with underscores (_) are reserved (though not strictly required).

Collections

A collection is a group of documents. If we compare a MongoDB document to a row in a relational database, a collection is equivalent to a table.

Key points about collections:

• Collections exist within databases. Different formats and types of data can be inserted into different collections for management, though collections don't have fixed structures.
• Subcollections can be organized using periods (.) to separate namespaces. For example, a blog application might have blog.posts and blog.authors collections.
• Collections are created when the first document is inserted. Valid collection names cannot be empty strings, cannot contain \0 characters, cannot start with "system.", and should avoid reserved characters like $.

Databases

In MongoDB, documents form collections, and collections form databases.

Database names can be any UTF-8 string that:

• Is not an empty string.
• Does not contain spaces, ., $, /, \, or \0.
• Should be all lowercase.
• Is limited to 64 bytes.

Some database names are reserved for special purposes:

• admin: The "root" database for authentication. Users added to this database gain access to all databases.
• local: Never replicated, stores all local collections for a server.
• config: Stores sharding information when MongoDB is configured for sharding.

Namespaces

The fully qualified name of a collection is created by prefixing the database name to the collection name, forming a namespace. For example, the namespace for the blog.posts collection in the cms database is cms.blog.posts. Namespaces cannot exceed 121 bytes and should generally be less than 100 bytes in practice.

Installation & Configuration

Installation

To install MongoDB:

1. Download from the official MongoDB website for your platform.
2. Run the installer (typically a straightforward process).
3. The installer automatically creates data directories and log files, and sets up the service.
4. Review the configuration file (mongod.cfg) in the bin directory, which typically includes settings for storage path, logging, and network configuration.

Starting and Stopping

To manage the MongoDB service:

net start MongoDB
net stop MongoDB
mongod --config "mongod.cfg"

Authentication

To enable authentication:

mongod --config "mongod.cfg" --auth

Basic Data Types

MongoDB documents conceptually resemble JavaScript objects and are similar to JSON. JSON is a simple data representation format with six basic types. While JSON is easy to understand and parse, it has limitations for database applications, such as lacking a date type and having only one numeric type.

MongoDB extends JSON with additional data types:

• null: Represents empty or non-existent fields
• Boolean: true and false values
• Number: Both integers and floating-point values
• String: UTF-8 encoded character strings
• Date: Date and time values
• Regular Expression: Pattern matching using regex syntax
• Array: Ordered lists of values
• Embedded Document: Documents nested within other documents
• Object ID: 12-byte unique identifier for documents

_id and ObjectId

Every MongoDB document must have an "_id" field, which can be any type but defaults to an ObjectId. ObjectId is designed for distributed environments and consists of 12 bytes (24 hexadecimal characters):

• 4 bytes: Timestamp
• 3 bytes: Machine identifier
• 2 bytes: Process ID
• 3 bytes: Counter

If no "_id" is provided when inserting a document, MongoDB automatically creates one. This is typically done by the client driver rather than the server, following MongoDB's philosophy of offloading processing to clients when possible.

CRUD Operations

Database Operations

• Create/Use: use database_name creates a database if it doesn't exist or switches to it.
• List: show dbs displays all databases.
• Delete: db.dropDatabase() deletes the current database.

Collection Operations

• Create: Collections are automatically created when the first document is inserted.
• List: show collections or show tables displays all collections in the current database.
• Delete: db.collection_name.drop() deletes a collection.

Document Operations

Inserting Documents

Documents can be inserted individually or in batches:

# Insert single document
db.collection.insert(document)

# Insert multiple documents
db.collection.insertMany([doc1, doc2, doc3])

# Insert or update (upsert)
db.collection.save(document)

Querying Documents

MongoDB offers various query operators:

• Comparison: {field: value} (equality), {field: {$gt: value}} (greater than), etc.
• Logical: {field1: value1, field2: value2} (AND), {$or: [{field1: value1}, {field2: value2}]} (OR)
• Membership: {field: {$in: [value1, value2]}} (in), {field: {$nin: [value1, value2]}} (not in)
• Regular Expression: {field: /pattern/} or {field: /^pattern$/}

Updating Documents

Updates can be performed using various operators:

• $set: Update field values
• $inc: Increment numeric values
• $push: Add elements to arrays
• $pop: Remove elements from arrays
• $pull: Remove specific elements from arrays
• $addToSet: Add elements to arrays if not already present

Deleting Documents

• Delete one: db.collection.deleteOne(filter)
• Delete many: db.collection.deleteMany(filter)
• Delete all: db.collection.deleteMany({})

Aggregation

MongoDB's aggregation framework processes data through a pipeline of stages:

• $match: Filter documents
• $project: Reshape documents, include/exclude fields, add computed fields
• $group: Group documents by specified fields and apply aggregation operations
• $sort: Sort documents
• $limit: Limit the number of documents
• $skip: Skip a number of documents
• $sample: Randomly select documents

PyMongo

PyMongo is the official Python driver for MongoDB. Here's a basic example of using PyMongo:

from pymongo import MongoClient
import datetime

# Connect to MongoDB
client = MongoClient('mongodb://username:password@localhost:27017/')
db = client['database_name']
collection = db['collection_name']

# Insert documents
doc1 = {
    "name": "John",
    "age": 30,
    "join_date": datetime.datetime.now(),
    "interests": ["reading", "hiking", "coding"]
}
collection.insert_one(doc1)

# Find documents
results = collection.find({"age": {"$gt": 25}})
for doc in results:
    print(doc)

# Update documents
collection.update_one(
    {"name": "John"},
    {"$set": {"age": 31}}
)

# Delete documents
collection.delete_one({"name": "John"})

Practice Exercises

1. Query collection names and employee names within each collection
2. Query collection names and the count of employees in each collection
3. Query the count of male and female employees in the organization
4. Query collection names with average, maximum, and minimum salary for each
5. Query average salary for male employees and female employees separately
6. Query collections with fewer than 2 employees, showing collection names, employee names, and counts
7. Query collections with average salary greater than 10,000, showing collection names and average salary
8. Query collections with average salary between 10,000 and 20,000, showing collection names and average salary
9. Query all employee information, sorted by age ascending, then by hire date descending if ages are equal
10. Query collections with average salary greater than 10,000, sorted by average salary ascending
11. Query collections with average salary greater than 10,000, sorted by average salary descending, limited to the top result