Why my cron job fires at 2am three times in November

On a Monday morning in November, someone on the team noticed a charge had hit our payment processor twice. Then four times. Then eight. The job that ran nightly billing had fired three times during the Sunday DST fall-back, and by the time anyone looked, a handful of customers had been billed anywhere from once to three times for the same invoice. Refunds took a week. The root cause was a single line in a crontab that had been untouched for two years — 30 2 * * * — and a team that, like most teams, hadn't thought about what happens to a 2:30am job when the clock repeats 1am to 2am.

Cron is one of those tools that looks trivial for the first week and increasingly less trivial for the rest of your career. Five fields, a few special characters, and you're done — except you're not, because the actual behavior involves DST, timezones, overlapping executions, subtle syntax traps, and at least three different "cron" dialects that look similar and aren't. The query data from our own site shows people searching for fundamentals ("cron expression", "crontab format") right next to people searching for debugging ("cron expression 6 fields", "cron example"). Everyone who writes a cron job eventually needs both — the syntax to get started and the subtleties to not get paged at 4am.

This post covers both. It walks through the five-field syntax, the common expressions you'll actually write, the traps (the day-of-month/day-of-week OR behavior is the one that bites every developer exactly once), DST and timezone behavior with a real calendar, overlapping executions and how to stop them with flock, the difference between Linux cron and Quartz 6-field cron and systemd timers, and the ten-point debugging checklist for when a job doesn't run and nobody knows why. If you just want to build an expression right now, our cron expression generator and crontab guru handle that. If you want to actually understand what you're looking at, read on.

Cron in 30 seconds

A cron expression is five fields separated by spaces:

* * * * *
│ │ │ │ │
│ │ │ │ └─ day of week  (0-7, where 0 and 7 both mean Sunday)
│ │ │ └─── month        (1-12)
│ │ └───── day of month (1-31)
│ └─────── hour         (0-23)
└───────── minute       (0-59)

A * in any field means "every value." So * * * * * means "every minute of every hour of every day." Specific numbers pin the schedule to those values. 30 2 * * * means "at 2:30 every day" — which is the expression that caused the billing incident.

Four special characters cover most real expressions:

• * — any value
• , — list of specific values (1,15,30)
• - — range (9-17)
• / — step (*/5 means every 5)

Combine them and you have the full syntax vocabulary:

0 */6 * * *          every 6 hours on the hour
15,45 9-17 * * 1-5   at :15 and :45, 9am-5pm, Monday-Friday
0 0 1 * *            midnight on the 1st of every month
*/10 * * * *         every 10 minutes
0 8 * * 0            8am every Sunday

That's enough syntax to read 90% of the cron expressions you'll encounter. The remaining 10% is where the trouble lives.

Reading real expressions

Start with a few you've probably seen and might not have fully decoded:

*/15 * * * * — every 15 minutes. The step operator on the minute field. Fires at :00, :15, :30, :45 of every hour. This is common for polling-style jobs (checking a queue, syncing with an upstream).

0 */2 * * * — every 2 hours on the hour. Fires at 00:00, 02:00, 04:00, ..., 22:00. Note the minute is explicit at 0; without that, * */2 * * * would fire every minute within every even hour, which is sixty times as often.

0 9-17 * * 1-5 — every hour from 9am through 5pm, Monday through Friday. Fires at 9:00, 10:00, 11:00, 12:00, 13:00, 14:00, 15:00, 16:00, 17:00 on weekdays only. Good for business-hours jobs.

0 0 * * 0 — midnight Sunday. 0 in day-of-week is Sunday. 7 also works and means Sunday. Saturday is 6.

*/5 9-17 * * 1-5 — every 5 minutes during business hours on weekdays. Common for "is the app up?" monitoring.

0 0 1,15 * * — midnight on the 1st and 15th of every month. Typical payroll schedule.

0 0 * * 1-5 — midnight on every weekday. Note how "every weekday" differs from "every business day of the month" — cron has no concept of holidays. You'd need application-level logic to skip holidays.

30 2 * * * — 2:30am daily. The expression from the opening anecdote. Looks innocent. Isn't.

Each of these decodes mechanically from the five-field structure. If you're staring at one that doesn't make sense, paste it into our crontab guru for an instant parse and the next few fire times.

The day-of-month / day-of-week trap

This is the syntax bit that surprises every developer at least once. Cron treats the fifth field (day of week) and third field (day of month) in a non-obvious way.

If both fields have wildcards (*) or both are specific, cron behaves as you'd expect. The surprise is when one is specific and the other isn't.

0 0 15 * 1

You read this as "midnight on the 15th, if the 15th is a Monday" — an AND. It's not. Cron evaluates day-of-month and day-of-week as an OR when both are specified with non-wildcard values. The expression actually means "midnight on the 15th of every month OR any Monday, whichever comes first or both."

The logic is documented in the POSIX crontab spec but it's easy to miss. The fix if you need AND logic is to do the check in the script itself:

# crontab
0 0 * * 1 /usr/local/bin/run-if-15th.sh

# run-if-15th.sh
#!/usr/bin/env bash
[ "$(date +%d)" -eq 15 ] || exit 0
# ... the actual job

Cron fires every Monday; the script exits early unless today is also the 15th. Ugly but correct.

This trap is why a lot of experienced developers default to using just one of those fields at a time and handling the rest of the condition in code.

Shortcut strings

Cron also supports named shortcuts for common schedules:

Shortcut	Equivalent
@yearly or @annually	0 0 1 1 * (midnight, Jan 1)
@monthly	0 0 1 * * (midnight, 1st of every month)
@weekly	0 0 * * 0 (midnight Sunday)
@daily or @midnight	0 0 * * * (midnight every day)
@hourly	0 * * * * (top of every hour)
@reboot	at system startup, once

These are readable at a glance and harder to typo than numeric equivalents. @daily is the one I use most — it's obvious to anyone reading the crontab that the job runs once a day without having to parse a five-field expression.

Support varies. GNU cron (the usual default on Linux) supports all of them. Some embedded or minimal cron implementations (dcron, busybox cron) support most but not all. Windows Task Scheduler has its own system and doesn't use these at all. When you move a crontab from one system to another, check what shortcuts the target supports.

@reboot is the odd one — it fires once when cron starts, not on a schedule. Useful for "start this service at boot" in environments without systemd. Not supported in all cron flavors, and of limited use now that systemd handles boot-time services better.

DST and timezones: the two-hour story

This is the big one and the reason cron has a reputation for subtle production incidents.

Cron runs in whatever timezone your system's crontab is configured for. By default that's the server's local timezone. When daylight saving time kicks in, the clock either skips forward (spring) or repeats an hour (fall). Cron's behavior in each case is a specific kind of broken.

Spring forward — some jobs disappear

In the US, spring forward happens at 2:00am on a Sunday in March. The clock goes from 01:59 directly to 03:00. The hour from 2:00am to 2:59am does not exist.

A cron job scheduled for 30 2 * * * (2:30am daily) simply does not run on that Sunday. The minute 2:30am doesn't happen. Vixie cron and most other implementations quietly skip the missing time — no run, no error, no log entry that would alert you. The job missed. If that job is your nightly billing or data sync, you have a gap in your data that someone might not notice until weeks later.

Fall back — some jobs double

In the US, fall back happens at 2:00am on a Sunday in November. The clock goes from 01:59 directly to 01:00. The hour from 1:00am to 1:59am happens twice.

A cron job scheduled for 30 1 * * * (1:30am daily) fires at 1:30am during the first 1am hour, then fires again at 1:30am during the second 1am hour. Two runs, same day. Some implementations try to detect this (Vixie cron catches jobs in the hour that runs twice and suppresses the second execution if the clock moved backward), but behavior varies. macOS cron, busybox cron, various containerized crons behave differently. Testing is the only way to know.

The practical rules

Three habits cover most of the risk:

1. Avoid scheduling anywhere near 1am–3am local time. It's a dead zone for DST-affected systems. If you genuinely need an overnight job, schedule it at 4:30am — outside the window where DST causes problems. The billing job that caused our opening story was at 2:30am; moving it to 4:30am would have prevented everything.

2. Use UTC if you can. The simplest fix is to run cron jobs in UTC, which has no DST. Set the timezone in the crontab itself:

TZ=UTC

30 2 * * * /usr/local/bin/nightly-job

UTC crontabs mean your schedule is stable year-round. The tradeoff is that "2:30 UTC" isn't a meaningful time for your operations team to reason about — it's 9:30pm EST one half of the year and 10:30pm EST the other. For internal jobs where precise clock alignment doesn't matter, UTC is the right default.

3. Use a scheduler that knows about DST. systemd timers handle DST correctly when you use OnCalendar=*-*-* 02:30:00 in the system's local timezone. AWS EventBridge schedules handle DST correctly for timezone-aware schedules. Kubernetes CronJob does not — it inherits whatever kernel time the cluster is running with, typically UTC, which is DST-free by definition. If DST behavior matters for a specific job, pick a scheduler that handles it, not cron.

The Red Hat DST-and-cron documentation covers the specific Vixie cron behavior on RHEL systems in detail. The general principle — cron is naive about DST — applies everywhere.

Overlapping executions

This is the second-most-common cron incident after DST. A job is scheduled every 10 minutes. Sometimes the job takes 8 minutes; usually it takes 2. One Thursday afternoon, the database is slow and the job takes 12 minutes. Cron kicks off the next run anyway, at the 10-minute mark, while the previous run is still going. Now two processes are trying to update the same state simultaneously. The third run starts while both are still going. By 5pm, there are twelve overlapping processes eating database connections.

Cron has no built-in concept of "don't start another one if the previous is still running." Every scheduled time is a new invocation.

The standard fix is flock, a file-locking utility that comes with util-linux on most Linux distributions:

# In crontab
*/10 * * * * /usr/bin/flock -n /tmp/sync.lock /usr/local/bin/sync-job

flock -n /tmp/sync.lock attempts to acquire a lock on /tmp/sync.lock. If the lock is already held by another process (meaning the previous run is still going), flock exits immediately without running the command. When the first run finishes, the lock is released, and the next cron trigger gets through.

A few things to notice:

-n is important. Without it, flock waits for the lock, which means jobs accumulate in a queue. You probably want -n so the new run exits instead of waiting, which is how you'd normally think of "skip if already running."

The lock file path matters. Use /tmp or /var/lock for host-level locking. Inside Docker containers, /tmp is container-scoped and works as expected. For Kubernetes CronJobs with concurrencyPolicy: Forbid, the cluster handles locking and you don't need flock.

Document the lock file. Future you will wonder why there's a random file in /tmp. A comment in the crontab saves an hour of investigation.

# Prevent overlapping sync jobs
*/10 * * * * /usr/bin/flock -n /tmp/sync.lock /usr/local/bin/sync-job

An alternative approach is pgrep in the script itself:

#!/usr/bin/env bash
if pgrep -f "sync-job" | grep -v "^$$$" > /dev/null; then
    exit 0  # already running
fi
# ... the actual job

This is less robust than flock because it depends on process name matching, but it works when flock isn't available (some minimal containers, some BSDs). For most situations flock is the cleaner choice.

If you want to experiment with different expressions to see how often they'll actually fire, our cron expression generator previews the next 10 fire times so you can spot the "this overlaps with itself" problem before shipping.

Five fields, six fields, seven fields — which cron are you writing?

The syntax changes depending on which scheduler is actually parsing the expression. Confusion here is why the same 0 0 * * * can mean different things in different tools.

Linux cron (5 fields) — the dialect we've been using throughout this post. Standard GNU cron, BSD cron, Vixie cron, dcron, busybox cron. Fields are minute, hour, day-of-month, month, day-of-week. This is what you get in /etc/crontab, crontab -e, and most scripts labeled "cron" on a server.

Quartz / Spring cron (6 fields) — adds a seconds field at the beginning. Fields are second, minute, hour, day-of-month, month, day-of-week. Popular in Java-based schedulers like Quartz (used by Spring Scheduler). A Quartz cron expression fed to Linux cron will fail or run at the wrong time because the first field becomes the minute.

Quartz with year (7 fields) — adds a year field at the end. Allows patterns like "2025-2030" for year-restricted schedules. Rarely used.

AWS EventBridge cron (6 fields, different from Quartz) — uses a ? character and slightly different semantics. Year is the 6th field. Day-of-week starts Sunday=1 (Linux is Sunday=0). Enough different to trip people up.

Side by side, "every day at 2:30am":

Linux cron (5):     30 2 * * *
Quartz (6):         0 30 2 * * ?
Quartz +year (7):   0 30 2 * * ? *
AWS EventBridge:    cron(30 2 * * ? *)
Spring @Scheduled:  0 30 2 * * *

The Spring @Scheduled(cron = "...") annotation uses 6 fields but the semantics differ from Quartz — particularly around day-of-week numbering and the ? character. Spring accepts * where Quartz requires ?.

The query data from our site shows people explicitly searching "cron expression 6 fields" — they've encountered Quartz or AWS cron and realized it's not Linux cron. The first thing to do is figure out which scheduler is parsing your expression, because the syntax is not interchangeable.

If you're writing an expression for a tool you don't know, read its documentation on what cron dialect it uses. Don't assume it's Linux cron just because it calls itself "cron."

Cron vs systemd timers

On modern Linux, systemd timers are a serious alternative to cron. They handle DST correctly, integrate with the system's logging (journalctl), support dependencies on other units, and have better handling of missed runs.

A simple comparison — a daily job at 2:30am:

Cron:

30 2 * * * /usr/local/bin/nightly

Systemd timer — two files:

# /etc/systemd/system/nightly.service
[Unit]
Description=Nightly job

[Service]
Type=oneshot
ExecStart=/usr/local/bin/nightly

# /etc/systemd/system/nightly.timer
[Unit]
Description=Run nightly job at 2:30am

[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true

[Install]
WantedBy=timers.target

Enable with systemctl enable --now nightly.timer.

The systemd version is more verbose but buys you:

• Correct DST handling — OnCalendar is DST-aware in the system timezone.
• Journal logging — journalctl -u nightly.service shows every run's output without needing MAILTO= or manual log redirection.
• Missed runs caught up — Persistent=true re-runs the job after downtime if it missed a scheduled fire time.
• Dependencies — After=network-online.target in the .service file makes the job wait for the network to be up before running.
• Restart policies — failed jobs can retry automatically.

Cron wins on simplicity for quick tasks on any Unix. Systemd wins on reliability for long-running production jobs. If you're on a systemd-using distro and the job matters, systemd timers are usually worth the extra configuration.

Why didn't my job run? A checklist

When a cron job quietly doesn't fire, the cause is almost always one of these ten things. Work through them in order:

1. Wrong crontab. System crontab (/etc/crontab, /etc/cron.d/*) and user crontabs (crontab -e for each user) are separate. A user's crontab is installed for that user only. If you ran sudo crontab -e, you edited root's crontab, not the user's. Check both.

2. No newline at end of file. Classic gotcha. If the last line of a crontab doesn't end with a newline, many cron implementations silently ignore it. The crontab command usually adds one, but pasting content directly into /etc/cron.d/ can miss this.

3. PATH is minimal. Cron runs with a very limited PATH (usually /usr/bin:/bin). If your job calls anything in /usr/local/bin, /opt/whatever/bin, or a custom install path, cron won't find it. Either use absolute paths (/usr/local/bin/my-script) or set PATH at the top of the crontab:

PATH=/usr/local/bin:/usr/bin:/bin

30 2 * * * my-script

4. Environment is minimal. Cron doesn't source .bashrc, .profile, or any shell startup file. Environment variables you rely on (API keys, language settings, tool-specific vars) are not present. Either set them at the top of the crontab, source a file explicitly in the job, or use a wrapper script. Our env validator is handy when you're auditing what's supposed to be in the environment before deploying.

5. Wrong working directory. Cron runs jobs from the user's home directory by default. Relative paths in your script break. Use absolute paths or cd to the correct directory at the start of the script.

6. Permissions. The script must be executable (chmod +x) and readable by the user whose crontab it's in. If the script is on an NFS mount or similar, permissions at mount time matter.

7. Missing shebang. A script without #!/usr/bin/env bash (or similar) at the top may fail to execute. The kernel needs to know which interpreter to use.

8. MAILTO is not set. By default, cron emails the output of every job to the user's local mailbox. If you don't have a working mail setup, errors go into a void you never see. Set MAILTO=you@example.com at the top of the crontab to route errors to a real inbox, or redirect output to a file:

30 2 * * * /usr/local/bin/nightly >> /var/log/nightly.log 2>&1

9. Cron daemon isn't running. Rare but possible. Check with systemctl status cron (or crond on some distros). If the daemon isn't running, none of your jobs fire.

10. Log files tell you. Cron logs its scheduling decisions. Common locations:

/var/log/cron             # RHEL, CentOS, Fedora
/var/log/syslog           # Debian, Ubuntu (grep for CRON)
journalctl -u cron        # systemd-based distros
journalctl -u crond

Searching these logs for your job's time is almost always faster than guessing at the cause. grep CRON /var/log/syslog shows what cron actually tried to run.

If after all ten checks you still can't figure it out, run the command by hand as the user whose crontab it's in and see if it works. Most "my cron job failed" problems are actually "my command failed under cron's minimal environment" problems, not cron problems.

Testing a cron expression without waiting

You want to know if 0 */6 * * * will fire when you think it will, without waiting hours to check. A few approaches:

Parse it with a generator. Our cron expression generator accepts an expression and previews the next 10 fire times in a human-readable list. Fastest way to verify a schedule is correct.

Use the cron-utils library if you're in Java. It parses any cron dialect and computes next-fire times programmatically, which is handy for writing tests.

Python's croniter library.

from croniter import croniter
from datetime import datetime

expression = "0 */6 * * *"
base = datetime.now()
iter_ = croniter(expression, base)
for _ in range(5):
    print(iter_.get_next(datetime))

Calculate by hand for simple cases. 0 */6 * * * fires at minute 0 of hour 0, 6, 12, 18 — every 6 hours starting from midnight. 30 9 * * 1-5 fires at 9:30 on weekdays. For complex expressions hand-calculation is tedious but for simple ones it's the fastest check.

Temporarily schedule for one minute ahead. If you can't wait but can afford a test run, set the expression to fire one minute from now and see what happens. Not great for production but acceptable in a staging environment.

The one I use most is the generator/parser preview. Paste the expression, see the next 10 fires, confirm it matches intent. The unix timestamp converter helps when you want to sanity-check that "the next fire at 1736208000" is actually the time you expected.

FAQ

What does */5 * * * * mean?

Every 5 minutes. The step operator on the minute field. Fires at :00, :05, :10, :15, :20, :25, :30, :35, :40, :45, :50, :55 of every hour.

Why did my cron job run twice on one day?

Almost always DST. If the job is scheduled between 1am and 3am local time, the fall-back DST transition in the autumn causes that hour to repeat, which fires the job twice. Spring forward causes jobs in that window to miss entirely. Move the job out of the 1-3am window or run cron in UTC.

How do I prevent cron jobs from overlapping?

Use flock -n /path/to/lockfile as a wrapper in the crontab entry. If the lock is held, the new run exits immediately instead of starting a second concurrent process.

What's the difference between cron and crontab?

cron is the daemon that runs jobs. crontab is both the file that lists jobs (one per user plus the system crontab) and the command you use to edit that file (crontab -e). The relationship: cron reads each user's crontab and runs the listed jobs at the scheduled times.

What does @reboot do?

Runs the job once when cron starts — usually at system boot. Useful for "start this service" in environments without systemd. Not supported on every cron implementation; GNU cron supports it.

How many fields does a cron expression have?

Linux cron has 5 fields (minute, hour, day-of-month, month, day-of-week). Quartz cron has 6 (adds seconds first). Quartz with year has 7. AWS EventBridge uses 6 with different semantics from Quartz. Always check which dialect your scheduler expects.

Why is my Spring cron expression invalid in Linux cron?

Spring @Scheduled uses 6 fields (seconds first). Linux cron uses 5. The first field differs in meaning. Convert by dropping the seconds field, or use a scheduler that accepts 6-field expressions.

How do I run a cron job every 15 minutes?

*/15 * * * * fires at :00, :15, :30, :45 of every hour. If you want "every 15 minutes" starting from some other offset, use comma-separated values: 5,20,35,50 * * * * for :05, :20, :35, :50.

What's the difference between cron and systemd timers?

Cron is universal and simple. Systemd timers are more verbose but handle DST correctly, integrate with systemd's journal for logging, and support dependency management and restart policies. For one-off servers, cron is fine. For production systems that must handle DST and downtime correctly, systemd timers are the safer choice.

What does 0 0 * * 1-5 mean?

Midnight on every weekday (Monday through Friday). 0 0 is minute 0 and hour 0 (midnight). * * matches any day of month and any month. 1-5 is day-of-week range from Monday (1) to Friday (5).

How do I schedule a job on the last day of the month?

Cron has no "last day of month" concept directly. The workaround is to run daily and check in the script:

0 0 28-31 * * [ "$(date -d '+1 day' +%d)" = "01" ] && /path/to/job

This fires on the 28th-31st, then the shell checks if tomorrow is the 1st, meaning today is the last day of the month. Systemd timers support OnCalendar=*-*-~0.. syntax for last-day-of-month more cleanly.

Can cron run jobs every second?

Linux cron does not. The smallest unit is 1 minute. For sub-minute scheduling, use a long-running process with sleep 1 loops, systemd timers with OnUnitActiveSec=1s, or a proper job scheduler like Quartz (which supports seconds).

What's flock for in a cron job?

File-based locking. Prevents a scheduled job from starting a new instance while the previous instance is still running. flock -n /tmp/job.lock /path/to/job — the -n means "don't wait for the lock; exit immediately if it's held."

How do I test a cron expression without waiting?

Paste it into our cron expression generator to see the next 10 fire times. Or use libraries like croniter in Python or cron-utils in Java to compute next-fire times programmatically.

Why does my cron job work when I run the command manually but fail from cron?

Almost always one of: PATH is minimal (cron doesn't see /usr/local/bin), environment variables you rely on aren't set, or the script uses relative paths and cron runs from a different working directory. Add PATH=... at the top of the crontab, set needed environment variables explicitly, and use absolute paths in your script.

The takeaway

Cron is easy for the first cron job and subtle for every production cron job after that. The five-field syntax is the easy part. The parts that cause 3am pages are DST on the 1-3am window, overlapping executions when a job runs longer than its interval, the 5-vs-6-field dialect mismatches when you switch schedulers, and the PATH/environment differences that make a working command silently fail under cron.

Three practical rules cover most real systems:

• Schedule outside 1-3am local time or run cron in UTC. The 30-minute billing job that triggered the opening anecdote would have survived every DST transition if it had been at 4:30am instead of 2:30am.
• Wrap long-running jobs in flock so overlapping executions can't happen. One line in the crontab, no more duplicate runs.
• Set explicit PATH and MAILTO in the crontab. Minimal environment is the most common reason a cron job "doesn't work" when the command clearly does when you run it.

Our cron expression generator builds expressions interactively and previews the next fire times so you can verify before shipping. Crontab guru parses existing expressions into plain English. For the deeper context on debugging tokens, IDs, and hashes that often appear in cron-generated jobs, our JWT decoder guide, UUID v4 vs v7 guide, and hash algorithm guide cover that ground.

Three times in November is once a year. Once a year is often enough to ruin a Monday morning.