Command line

paste vs join, Measured: Fast by Position, Wrong by Key

paste merges two files in 0.087 seconds and join takes 0.155 on the same million lines — but the speed ranking is the wrong reason to choose. paste pairs rows by position, join pairs them by key, and on any input where the two files are not already in the same order, paste's answer is wrong on every line and nothing in the output says so.

Written November 9, 2026 · Tested with Python 3.12.3 · 12 min read

By the end of this guide
  • The table: paste is fastest, and speed is the wrong reason to choose
  • Why paste is faster and join is stricter
  • What the nearest guide gets wrong
  • How to tell which tool fits

What this guide is

Two files, one million lines each, four ways to merge them, three reps each. Both files hold the same keys, k0000001 through k1000000, in the same random order, tab-separated from their values — file A carries a-values, file B carries b-values. The question is which command to reach for when two files need to become one, and the answer turns on what the two files already agree on: their order, or their keys. Every arm's output is verified the same way — the merged lines are sorted and hashed, and all twelve runs hash to the same value — so the table compares merges that arrived at the same answer, not four different answers.

The trap corpora are the second half of the guide. One file is cut short by 1,000 lines, one is shuffled into a different order, one is missing 1,000 keys spread through the file. Each trap is run once per tool, and the output is counted line by line and checked pairing by pairing, so the trap sections report what the tools actually did, not what their manuals say they do.

The table: paste is fastest, and speed is the wrong reason to choose

Wall seconds, medians of three runs. Every rep returned 1,000,000 lines and every rep's sorted output hashed to the same value, so all four arms produced the same merge.

  • paste A B: fresh median 0.087, reps 0.091, 0.086, 0.087.
  • join -t TAB on pre-sorted files: fresh median 0.155, reps 0.160, 0.155, 0.152.
  • join with the sort included, join -t TAB <(sort A) <(sort B): fresh median 0.410, reps 0.424, 0.410, 0.403.
  • Python dict join, the reference arm: fresh median 1.038, reps 1.015, 1.038, 1.045.

The ratios: paste is 1.78 times faster than pre-sorted join, 4.71 times faster than join with the sort included, and 11.93 times faster than Python. Pre-sorted join is 6.71 times faster than Python. The sort itself is the honest cost of join's sorted-input requirement — sorting file A took 0.146 seconds and sorting file B took 0.169, timed once each, and inside the timed arm the two sorts run in parallel, so the 0.410 is roughly the slower sort plus the join. The 0.087-second paste cells are close to the cost of reading a million lines and writing them out again, so the ratios are order-of-magnitude, not precision claims — but the ordering is not close, and it held on all three reps.

Why paste is faster and join is stricter

The manual says what the table shows. paste writes lines consisting of the sequentially corresponding lines of each file, separated by tabs — line 1 with line 1, line 2 with line 2, no questions asked. It never looks at the content of a line, never compares anything, never holds more than one line per file in memory. That is why it is fast, and it is also why it cannot tell a right pairing from a wrong one.

join writes one line for each pair of input lines with identical join fields, and the manual adds the condition this whole guide turns on: the two files must be sorted on the join fields. join walks both files together like a merge, comparing keys as it goes, which costs the comparisons paste skips — 0.155 seconds against 0.087 on the same corpus — and it buys the guarantee that a row in the output means the keys matched. The sort is the price of that guarantee, and the 0.146 plus 0.169 seconds of sorting is part of the honest cost whenever the files arrive unsorted.

The Python arm is the reference, not a contender. It reads all of file B into a dictionary and streams file A, which is the correct answer by construction — the probe checks the other three arms against it — and it costs 1.038 seconds, twelve times paste. Its cost is the dictionary build and the per-line Python loop, and it is the arm to reach for when the merge logic is anything more complicated than one key, not when the files are large and the logic is simple.

The traps: three ways the merge goes wrong

Trap
The table above is the easy case: same keys, same order, same length. The traps are what happens when the two files do not agree, and they split the tools cleanly — paste fails silently in all three, join fails loudly in one and quietly drops rows in the other two.

The unequal-length trap cuts file B to 999,000 lines. paste still emits 1,000,000 lines — it does not stop at the shorter file — and the last 1,000 lines are the key and value from file A followed by a tab and nothing: k0192594, tab, a0192594, tab, newline, verified byte by byte with od. The missing value is a silent empty field, not an error and not a visible marker. join emits 999,000 lines: the inner join drops the tail, because those 1,000 keys have nothing to pair with. Both tools did what their definitions say; only one of them leaves a trace — join's output is 1,000 lines short, which a line count catches, while paste's output has the right line count and 1,000 empty values, which a line count does not catch.

The shuffled trap keeps the same keys in both files but in a different order. paste emits 1,000,000 lines, and all 1,000,000 pairings are wrong — every row pairs a key with another key's value, checked pairing by pairing against the key-to-value map built from file B. There is no warning, no error, no short line count; the output looks exactly like a correct merge. join on the same unsorted files emits 2 lines, exits with status 1, and prints a warning on stderr naming both files as not sorted, then join: input is not in sorted order. Both input files were unsorted here — file A is in the same random order as the original file B — so the warning names both. The join failure is loud, and the manual says the output should not be used when the command exits nonzero. The contrast is the finding: paste's worst failure looks identical to success, and join's worst failure announces itself.

The missing-keys trap removes 1,000 keys from file B, spread through the file, one every thousand. join emits 999,000 lines — the inner join drops exactly the missing keys, no more, no less. paste emits 1,000,000 lines, of which 997,204 carry the wrong value: from the first missing key onward every row is shifted by one, pairing each key with the next key's value, and the last 1,000 lines carry no value at all. The first wrong pairing is at line 1,797, where key k0841000 is paired with value b0663730. The count is exact because the check reads every line: 999,000 four-field lines, 997,204 of them wrong, plus 1,000 lines with no value. A merge that is wrong on 99.7 percent of its rows and right on its line count is the shape of the trap — the line count, the usual sanity check, passes.

What the nearest guide gets wrong

The closest published page on merging two files, the copyprogramming guide, states in its unequal-length section that paste stops at the shortest file's length by default, and repeats the claim in its FAQ. The trap above shows it does not: paste emits all 1,000,000 lines from the longer file, with an empty field where the shorter file ran out. Anyone scripting around paste on the strength of that claim — expecting a short output when a file is short — gets a full-length output with silent empty values instead.

The same page asserts that paste outperforms awk by 2 to 3 times on GB-scale data, and advises testing with time. The assertion carries no numbers, no corpus size, no reps, and no join arm at all — it is not a measurement, and it cannot be checked. Its alternatives table lists join as key-based with sorted files required, which is correct as far as it goes, but the page never measures join, never runs the shuffled case, and never states that paste's positional merge is wrong whenever the two files disagree on order.

How to tell which tool fits

The choice is not speed. paste is the fastest arm on every rep, and it is the wrong tool whenever the two files might disagree. Use paste only when the two files are already in the same order and the same length — generated together, or checked first with a line count and a spot check of the first and last pairings. The check costs less than the merge: wc -l on both files, and one look at a line in the middle.

Use join when the files share a key and may not share an order. Sort both files first, in the same locale join will compare them in, and pay the sort — 0.315 seconds for the two files here, sequential — or fold the sort into the command with process substitution and pay 0.410 total. If join complains the input is not sorted, believe it: the output is not to be used, and the fix is the sort, not a flag. --check-order makes join fail on any out-of-order line even when all lines pair, which is the strict form of the same guarantee.

Use the Python dict join when the merge logic is more than one key — a second condition, a transformation, a default for missing keys — and accept the 1.038 seconds as the price of the logic. It is the only arm of the four that does not care about input order at all, and it is the reference the other three were checked against. No awk arm is measured here: the awk, cut, and sed field-extraction family is killed, and paste, join, and Python already answer the merging question.

Checklist

  • Before merging two files with paste, check they are in the same order and the same length — wc -l on both files, and a spot check of the first, middle, and last pairings, because paste pairs by position and a wrong pairing looks identical to a right one.
  • Before trusting a paste output, count the fields: a file that ran short leaves empty trailing fields, not an error, and the line count stays full.
  • Before running join, sort both files in the same locale join will compare them in — unsorted input gives two lines and a nonzero exit, and the output should not be used.
  • Before choosing on speed, check the table: paste is 1.78 times faster than pre-sorted join and 11.93 times faster than Python on a million lines, and the speed is the wrong reason to pick it.
  • Before quoting a ratio, check the range: the paste arm's reps run 0.086 to 0.091 seconds, so the ratios are order-of-magnitude, and the ordering held on all three reps.
  • Before reusing these numbers, check the setup: WSL2, 32 CPUs, GNU coreutils 9.4, Python 3.12.3, tmpfs, warm page cache, one thread per arm — the ranking held across all three reps, the absolute seconds are this box's.

Sources

  • paste(1), Linux manual page — read 2026-11-09: paste writes lines consisting of the sequentially corresponding lines of each file, separated by tabs; the newline of every line except the line from the last file is replaced with a tab — the mechanism behind the trap's empty trailing field. Box runs coreutils 9.4; the manual text is 9.11, and every cited behavior was exercised by the probe itself.
  • join(1), Linux manual page — read 2026-11-09: join writes one line for each pair of input lines with identical join fields; the default join field is the first; -t sets the separator; the files must be sorted on the join fields, and if the input is not sorted and some lines cannot be joined, a warning is given. Box runs coreutils 9.4; the manual text is 9.11, and every cited behavior was exercised by the probe itself.
  • GNU Coreutils, join invocation — read 2026-11-09: the files should be sorted on the join fields in the LC_COLLATE sequence; if an input file is diagnosed as unsorted, join exits with a nonzero status and the output should not be used; --check-order fails on any out-of-order line.
  • copyprogramming, Join Two Files Horizontally in Unix — read 2026-11-09: the nearest published page — no measured timing table; asserts paste outperforms awk by 2 to 3 times with no numbers, no corpus, and no join arm; states paste stops at the shortest file's length, which the T1 trap disproves; its alternatives table correctly notes join needs sorted files and keys.
  • Hacker Public Radio, UNIX Curio #11 — Merging Files — read 2026-11-09: explains paste, join, and sort -m semantics with no timings — paste pairs lines in order and needs the same order in both files, join merges on a key field and needs sorted input, sort -m merges sorted files.
  • Veldica, Which Lines Are in Both Files? comm vs grep vs sort|uniq vs Python, Measured — the sibling guide: the same two-files question, different operation — intersecting, not merging. Its join arm is comm, which also needs sorted input; its sort|uniq -d trap is the same failure shape as this guide's paste traps, a silent wrong answer.

Keep Exploring