# The N+1 Query Problem in Mongoose: A Production War Story
By Govind Bajaj · 2025

Tags: mongodb, mongoose, performance, database, backend
One missing .populate() call brought the Sans Herbals API to its knees at 200 concurrent users. The fix was two lines. The debugging took three hours.
The Sans Herbals API went down at 11:47 AM on a Tuesday.

The database connection pool was exhausted. Response times spiked from 200ms to 12 seconds. Customers couldn't check out. I was pulling my hair out because the code hadn't changed in two weeks.

The culprit was the N+1 query problem. One missing `.populate()` call turned a single database query into 847 sequential queries under load. The fix was two lines. The debugging took three hours.

## What the N+1 Problem Actually Is

You fetch a list of documents, then loop through them and query related data for each one. N documents become N+1 database queries.

```js
// The problematic code
const orders = await Order.find({ status: 'pending' })
  // Query 1: find orders

for (const order of orders) {
  const user = await User.findById(order.userId)
  // Queries 2 through N+1: one per order!
  console.log(user.name, order.total)
}
```

With 200 pending orders, this executes 201 database round-trips. Each round-trip is network latency + database work + response serialization. At 5ms per query, that's a full second of sequential database time. Under load, the connection pool saturates and requests queue.

## How Mongoose Makes This Easy to Do

Mongoose's API is designed for convenience. Every model method returns a Promise. It's natural to `await` inside a loop. The framework doesn't warn you. The queries work fine in development with 10 documents.

The trap: Mongoose queries are lazy by default. They don't execute until you `await` or call `.then()`. This means you can build up complex query chains without realizing how many actual database calls you're making.

## The Fix: Population (and When It Works)

Mongoose's `.populate()` method does a join-like query to fetch related documents in one round-trip.

```js
// Fixed: single query with population
const orders = await Order.find({ status: 'pending' })
  .populate('userId', 'name email')
  // Still just 1 query — Mongoose handles the join

for (const order of orders) {
  console.log(order.userId.name, order.total)
  // No additional queries
}
```

The second argument to `.populate()` specifies which fields to include. Always limit populated fields to what you need — over-fetching creates its own performance problems.

## When Population Isn't Enough

`.populate()` works for simple foreign key relationships. For complex aggregations, use MongoDB's aggregation pipeline directly:

```js
// Aggregation pipeline for complex joins
const result = await Order.aggregate([
  { $match: { status: 'pending' } },
  {
    $lookup: {
      from: 'users',
      localField: 'userId',
      foreignField: '_id',
      as: 'user',
      pipeline: [
        { $project: { name: 1, email: 1 } }
      ]
    }
  },
  { $unwind: '$user' },
  {
    $project: {
      orderTotal: '$total',
      customerName: '$user.name',
      customerEmail: '$user.email'
    }
  }
])
```

`$lookup` is MongoDB's join equivalent. It runs entirely on the database server — no network round-trips between documents. For reports and dashboards, aggregation pipelines are significantly faster than population.

## The Real Fix: Schema Design

Population and aggregation treat the symptom. The root cause is often schema design.

In MongoDB, embedding related data is frequently better than referencing:

```js
// Instead of referencing users
const orderSchema = new Schema({
  userId: { type: Schema.Types.ObjectId, ref: 'User' },
  // Need to populate every time you read an order
})

// Embed the minimal user data you need for reads
const orderSchema = new Schema({
  user: {
    id: { type: Schema.Types.ObjectId, ref: 'User' },
    name: String,  // embedded for fast reads
    email: String
  }
})
```

For the Sans Herbals orders collection, I embedded the customer's name and email directly in the order document. Orders are read 100x more often than customer profiles are updated. The trade-off favors read performance.

## Detecting N+1 in Development

Enable Mongoose's debug mode to see every query:

```js
// In development
mongoose.set('debug', true)
```

Better yet, write a middleware that logs slow query patterns:

```js
// Warn on queries during a single request
let queryCount = 0

orderSchema.pre('findOne', function() {
  queryCount++
  if (queryCount > 10) {
    console.warn(`N+1 detected: ${queryCount} queries in one request`)
  }
})
```

For production monitoring, I use `mongoose-paginate-v2` with query logging and set alerts when any request exceeds 20 database queries.

## Takeaways

- The N+1 problem turns 1 query into N+1 queries and kills performance under load
- `.populate()` fixes simple cases by joining related documents in one query
- Aggregation pipelines (`$lookup`) handle complex joins without round-trips
- Embedding related data in MongoDB schemas often prevents the problem at the source
- Enable `mongoose.set('debug', true)` in development to catch excessive queries early
- Set query count alerts in production — the problem only appears under real load