# pm2 logs Saved My 2 a.m. (Read Them Like This)

> pm2 logs plus pm2 monit plus structured app logs: the 3-layer debug flow that ends 2am guessing. Incident walkthrough inside.

Source: https://boringpush.com/blog/pm2-logs-debug-nextjs-vps

## 2 a.m., alerts firing, logs everywhere

Telegram buzzed at 2:07am: checkout 500s. Termius opened from bed, `pm2 logs` ran, and the reader drowned — three workers interleaving, restart notices, deprecation warnings from a dependency, and somewhere inside, the actual error. Twenty minutes of scrolling before a null dereference that took ninety seconds to fix.

The incident lasted forty minutes. The fix took two.

Pm2 logs debug nextjs skill isn't talent — it's a flow practiced calm so it works panicked. Mine now runs in under five minutes, most of it waiting for commands to print.

## Log chaos vs log flow at 2am

| Minute | Chaos (first incident) | Flow (now) |
|---|---|---|
| 0–2 | SSH in, `pm2 logs`, scroll blindly | Alert context names suspect (orderId) |
| 2–10 | Grep guesses, wrong service | `--lines 50 --nostream` per process |
| 10–20 | Restart "to see if it helps" | `monit`: memory/CPU per worker |
| 20–30 | Find error, guess cause | Structured log: cause + request id |
| 30–40 | Fix + deploy + pray | Fix + gate + reload + verify |

Three things 2am teaches about logs:

1. **Interleaved workers hide sequences.** Three processes writing one stream turns causality into confetti — the error's context (request in, query out) scatters across worker boundaries. Per-process views (`--name` filters) restore narrative; global tails destroy it. Debug one worker's story at a time, always.
2. **Restarts destroy evidence.** A "restart to see" wipes the dying worker's final state and resets the log context needed — treatment before diagnosis, evidence cremated. Read first, restart last (and only with the fix deployed). Logs are crime scenes; secure them before touching anything.
3. **Unstructured logs answer nothing quickly.** "Error: failed" with no request ID, no user, no timestamp context takes ten grep rounds to place. Structured lines (time, level, route, userId, requestId, message) answer in one look. Logging discipline pays exclusively at 2am — which is exactly when debts come due.

The Telegram-alert post starts incidents. This one ends them.

## Pm2 logs debug nextjs flow

Past, present, why — three layers in fixed order, drilled monthly from the couch.

**Step 1 — Read the recent past, per process.**

```bash
pm2 logs boringpush --lines 50 --nostream   # last 50, frozen (no follow yet)
pm2 logs boringpush --lines 200 --nostream | grep -i "error\|500\|orderId"
# per-worker when interleaved: pm2 logs --name "boringpush-1" --lines 100
```

Frozen first (nostream stops the scroll so you can think), filtered second (error/500/context IDs), per-process third when workers interleave. The alert's context (orderId, route) focuses every grep — alerts without context send you reading everything; context turns forty minutes into four. `--lines` before `--follow`, always: history first, live tail after.

**Step 2 — Check the live present with monit.**

```bash
pm2 monit   # CPU + memory per worker, live: leak? loop? dead worker?
# shapes: one worker 100% CPU = infinite loop; climbing memory = leak; 0 workers = crash-loop guard tripped
```

Shapes diagnose instantly: flat-and-low means code bug, spiking means resource bug, dead means crash loop.

**Step 3 — Follow structured app logs to the why.**

```js // app log line shape (winston/pino, or console JSON — structure matters, vendor doesn't)
{"t":"02:07:11","lvl":"error","route":"POST /api/order","user":"u_8821","req":"r_99120","msg":"coupon null","stack":"..."}
# one line holds: when, what, where, who, which request, why — grep any dimension
```

Request IDs tie the whole incident together (alert → route → query → error in one grep chain). Structured lines make 2am reading deterministic: filter by req ID, read top to bottom, find the break. Unstructured logs need interpretation; structured logs need eyes. At 2am, eyes are all you have — feed them structure.

That's the product — incidents that end before the coffee cools.

## Incident ledger, illiterate vs flow

| 2am incident | First (chaos) | Recent (flow) |
|---|---|---|
| Time to SSH + orient | 10 min (scrolling) | 2 min (alert context) |
| Diagnosis method | Guesses + restart | Per-process logs + monit |
| Evidence preserved | No (restarted first) | Yes (read first) |
| Fix + verify | 15 min + prayer | 5 min + health check |
| Total / sleep lost | 40 min | 9 min |

Said without spin: structured logging costs lines of code per route (or one middleware doing it globally — do that) plus log volume (disk, rotation, retention policy). Priced against thirty extra minutes per incident times quarterly frequency: the middleware afternoon pays back in two incidents. Logs are written for the 2am reader; write them like that person matters, because that person is on call.

## FAQ

### Should pm2 logs debug nextjs go to files, journald, or a service?

Files by default (PM2 manages rotation with `pm2-logrotate`), journald on systemd-purist boxes, external service past the volume where grep gets slow (millions of lines/day). Solo: files + logrotate + 90-day retention cron. The medium matters less than the shape (structured) and the flow (past→present→why). Fancy pipelines serving unstructured text are lipstick on chaos.

### How do I keep logs from filling the disk?

Rotate (pm2-logrotate: 10MB files, keep 7), retain 90 days via cron, compress cold logs, alert on disk >80% (Telegram scream, obviously). Disk-full kills apps silently (writes fail, SQLite can't checkpoint) — the retention cron is infrastructure, not housekeeping. Logs should observe incidents, never cause them. Cap everything, alert early.

### What log level belongs in production?

Error + warn always, info for money paths (orders, webhooks, auth events), debug behind a flag (off in prod, on per-incident via env + reload). Verbose prod logs cost disk and drown signal; silent prod logs starve diagnosis. Money-path info is the sweet spot: audit trail plus incident context for free. Levels are budgets; spend on money.

### Can agents read pm2 logs to self-diagnose incidents?

Yes, carefully: paste the frozen `--lines` output (not live tails) into the session with the alert context — agents correlate well with bounded input. Never pipe live logs into long sessions (context flood, token burn, the loop post's warnings apply). Bounded excerpts in, diagnoses out, human verifies. Agents read; humans conclude. Same split as everything.

### How do I correlate logs across app, Nginx, and Cloudflare?

Request IDs propagated end to end (Cloudflare `cf-ray` + app `req` id in one structured line per hop), timestamps in UTC everywhere (timezone math at 2am kills), clock sync via NTP (drift lies about order). Three conventions make distributed debugging linear: shared IDs, shared timezone, synced clocks. Implement once, use every incident forever.

### When do logs graduate to APM/tracing (Sentry, Datadog)?

Past single-box scale (distributed traces matter when requests cross services) or compliance (audit-grade retention with access controls). Solo VPS: structured files plus monit plus alerts cover 95% of incidents at $0. Graduate on distributed pain, not dashboard envy — traces answer cross-service questions you don't have yet. Logs first, traces when lost.

## Keep reading

- [Fix Production From Your Phone: The Grocery-Line Drill](/blog/fix-production-from-phone-telegram) — the alert starting this flow
- [PM2 Reloads Nobody Feels: Next.js With Zero Downtime](/blog/pm2-zero-downtime-nextjs) — the reload ending it safely
- [Tailscale Deploy Trick: Close Port 22 Forever](/blog/tailscale-wireguard-deploy-private) — the road Termius travels