Converting CRLF to LF, Measured: tr vs sed vs perl vs awk
The fastest tool is also the only one that eats carriage returns that are not line endings; the other three produce byte-identical output at different speeds; and one of them adds a newline the file never had.
Written November 10, 2026 · 12 min read
- The table: tr is fastest, and the other three agree byte for byte
- The closest published work asserts without a table
- The carriage return that is not at the end of a line
- The byte-identical trio
What this guide is
One corpus, four arms, three reps each. The corpus is 1,000,000 lines, 98,897,900 bytes, every line terminated with a carriage return plus a line feed, and 1,000 of the lines carry one more carriage return in the middle of the line, so the file holds 1,001,000 carriage returns in total. Every arm converts the file to line-feed endings, and every rep is checked the same way: line count, carriage-return count, byte count, and a sha256 of the whole output, so a timing number can never come from a run that printed something different. The question is the one people ask before converting a file that came from Windows: which tool strips the line endings fastest, and which tools change something else while looking complete.
The answer is a correction in both directions. tr -d '\r' is the fastest arm at 0.12 seconds, and it is also the only arm that removes every carriage return: it deletes the 1,000 mid-line ones along with the 1,000,000 line-end ones, so its output is 1,000 bytes shorter than the other three. sed, perl, and awk each strip only the line-end carriage returns and produce output byte-identical to each other, at 2.33, 3.51, and 4.10 times the cost of tr tonight. The traps decide more real conversion jobs than the speed table does: a file with a carriage return inside a line loses data under tr with no warning, and a file whose last line has no trailing newline gains one under awk.
The table: tr is fastest, and the other three agree byte for byte
Wall seconds, medians of three runs, fresh tonight. Every rep returned rc=0, and every arm's output sha256 was identical across its three reps.
tr -d '\r': fresh median 0.1166, reps 0.1546, 0.1165, 0.1166. Night before: 0.1484, reps 0.1206 to 0.1585. Prints 1,000,000 lines, 0 carriage returns left, 97,896,900 bytes.sed 's/\r$//': fresh median 0.2711, reps 0.2781, 0.2711, 0.2705. Night before: 0.2747, reps 0.2703 to 0.2755. Prints 1,000,000 lines, 1,000 carriage returns left, 97,897,900 bytes.perl -pe 's/\r$//': fresh median 0.4095, reps 0.4095, 0.3978, 0.4414. Night before: 0.4104, reps 0.4059 to 0.4307. Prints 1,000,000 lines, 1,000 carriage returns left, 97,897,900 bytes.awk '{sub(/\r$/,"")}1': fresh median 0.4782, reps 0.4728, 0.4782, 0.5040. Night before: 0.4803, reps 0.4797 to 0.4825. Prints 1,000,000 lines, 1,000 carriage returns left, 97,897,900 bytes.
Against tr at 0.1166 seconds tonight, sed costs 2.33 times, perl 3.51 times, and awk 4.10 times. The night before, the same ratios were 1.85, 2.77, and 3.24, so the ratios against tr moved, and the reason is visible in the table: tr got faster, from a median of 0.1484 to 0.1166, a 21 percent drop, while sed, perl, and awk moved by less than 2 percent each. The ratios among the three did not move: perl is 1.51 times sed tonight against 1.49 the night before, and awk is 1.76 times sed against 1.75. The ranking is identical on both nights, and the exact ratios against tr are one machine's. Throughput on the 99 MB input: tr reads about 848 MB/s, sed about 365 MB/s, perl about 242 MB/s, awk about 207 MB/s.
Two cells are noisy and the guide says so rather than hiding it. The tr first rep is the cold spike both nights: 0.1546 seconds tonight and 0.1585 the night before, against fastest reps of 0.1165 and 0.1206. A one-run timing of tr would report about a third more than the steady cost. The perl third rep tonight is 0.4414 against 0.3978 to 0.4095 on the other two, and the awk third rep is 0.5040 against 0.4728 to 0.4782. The sed first rep on the banking night's first run was 0.3173 against a second-run range of 0.2703 to 0.2755, the same first-run cold cost. The 0.12 to 0.48 second cells sit near the spawn-and-read floor, so treat the exact ratios as order-of-magnitude; the ranking and the byte-identical trio are the stable claims.
The closest published work asserts without a table
The closest published work is a methods guide, not a measurement, and the distinction is the gap this probe fills. The CopyProgramming sed guide covers sed, tr, awk, and dos2unix as methods for the same conversion, with worked examples and a table that rates speed in words only: sed Medium, tr Fastest, awk Slower than sed, dos2unix Fast. It states that tr deletes carriage returns fastest and that dos2unix outperforms sed by 20 to 50 percent on large files, both as assertions with no measured table, no corpus, no reps, and no output verification. The second assertion is not adjudicated here at all, because dos2unix is not installed, so there is no dos2unix arm and the 20 to 50 percent claim stays unmeasured, not confirmed or refuted.
The guide's other methods differ from this probe's in one place that matters. Its awk arm deletes every carriage return, which makes it a second tr in output, while this probe's awk arm strips only the line-end carriage return and preserves the 1,000 mid-line ones, which is why it lands byte-identical to sed and perl here. The same guide also notes the tr limitation in prose, that it removes all carriage returns and not just those at line endings, and then recommends sed for that reason. This probe measures the limitation instead of stating it: exactly 1,000 carriage returns, exactly 1,000 bytes, on a corpus where the mid-line carriage returns are one per 1,000 lines.
The carriage return that is not at the end of a line
tr -d '\r' deletes characters, and the manual says so directly: with -d, it deletes the characters in the set and does not translate, and it reads from standard input only, taking no filename operand. A carriage return is a carriage return to tr wherever it sits in the line, so the 1,000 mid-line carriage returns in this corpus are gone from its output along with the 1,000,000 line-end ones. The other three arms anchor the match to the end of the line, s/\r$// in sed and perl, sub(/\r$/,"") in awk, so a carriage return in the middle of a line never matches and survives. The difference is exactly 1,000 bytes on the main corpus: tr output is 97,896,900 bytes and the other three are 97,897,900, with the same sha256 across the trio on all three reps each, tonight and the night before.
The trap file shows the same behavior at small scale. It is 28 bytes: a line ending in carriage return plus line feed, a line with a carriage return in the middle and a line feed at the end, and a last line ending in a carriage return with no trailing newline. tr returns 25 bytes, with the mid-line carriage return gone and the two words joined. sed and perl return 26 bytes, with the mid-line carriage return kept, the trailing carriage return stripped, and no trailing newline added. awk returns 27 bytes: the same 26 bytes as sed and perl, plus a newline it appends to the unterminated last line. The awk difference comes from its record handling: the manual sets the output record separator to a newline by default, and print writes it after every record, including a last record that arrived without one. A file whose last line is unterminated gains a newline under awk and does not under the other three, and the probe measures that on a 3-line file, so real files hit it only when the last line is unterminated.
The practical rule follows from the counts, not from the speed table. If the file has no carriage return except at line ends, tr is the fastest correct tool and the choice is settled. If any carriage return sits inside a line, as in a data file with embedded control characters, tr silently deletes data and the line-end-only arms are the correct tools, at 2.33 to 4.10 times the cost. Among those three, the output is identical, so the choice is availability and habit, not output: sed is the cheapest of the three, perl is the portable one-liner with the \R alternative for mixed endings, unmeasured here, and awk is the one to avoid when the last line may be unterminated.
The byte-identical trio
sed, perl, and awk outputs share one sha256 on all three reps each, tonight and the night before. Same lines, same 1,000 mid-line carriage returns, same bytes, byte for byte. The three commands take different routes to the same file: sed at a fresh median of 0.2711 seconds, perl at 0.4095, awk at 0.4782, and paying 1.51 or 1.76 times the sed cost buys nothing in the output. That is the same pattern the line-numbering guide found the night before, where nl -ba output was byte-identical to cat -n output at 2.76 times the cost, and the hidden-characters guide found the same night, where cat -vet output was byte-identical to cat -A output at the same cost. Here the pair is a trio, and the cheapest member is the one to reach for when the output is all that matters.
The trio also agrees on what it does not do. None of the three touches the mid-line carriage returns, none of them adds or removes a line, and none of them changes the line count: every rep prints exactly 1,000,000 lines. The only difference among them is speed and the awk trailing-newline append on an unterminated last line, which the main corpus never exercises because every line in it ends with a newline. The trap file is the only place that difference shows, and it shows by exactly one byte.
Checklist
- Before running
tr -d '\r'on a file, check for mid-line carriage returns:trdeletes 1,000 of them on this corpus with no warning, and the line-end-only arms preserve all 1,000. - Before choosing among
sed,perl, andawk, check the output: the three are byte-identical, so pick by availability and habit, and by speed, withsedcheapest at 0.2711 seconds. - Before running
awkon a file, check the last line:awkappends a newline to an unterminated last line, adding 1 byte the other three do not add. - Before quoting the
dos2unixspeed claim, check the measurement:dos2unixis not installed, the 20 to 50 percent claim is unmeasured here, and this guide neither confirms nor refutes it. - Before quoting a ratio against
tr, check both nights:trmoved from 0.1484 to 0.1166 between nights while the other three moved by less than 2 percent, so the ratios againsttrare one machine's and the ranking is the stable claim. - Before reusing these numbers, check the setup: WSL2, 32 CPUs, tr 9.4, sed 4.9, Perl 5.38.2, awk 5.2.1, tmpfs, warm page cache, one thread per arm, one 1,000,000-line CRLF corpus with 1,000 mid-line carriage returns, sha256 recorded in the probe log.
Related guides on this site
- Tabs to Spaces and Back, Measured: expand vs unexpand vs sed vs awk vs perl
- Editing a File In Place, Measured: sed -i vs perl -pi vs awk
- Finding Lines by Prefix: look vs grep vs rg vs awk vs sed, Measured
- Cutting a Million-Line File Into Ten, Measured: split vs csplit vs awk
Sources
- Linux man-pages, tr(1) — read 2026-11-10: translates, squeezes, or deletes characters from standard input, writing to standard output, and
-ddeletes the characters in the set without translating. Box runs GNU coreutils 9.4; the manual text is coreutils 9.11 and every cited behavior was exercised by the probe itself. - Linux man-pages, sed(1) — read 2026-11-10:
s/regexp/replacement/attempts to match the regular expression against the pattern space and replace the matched portion. Box runs GNU sed 4.9; the manual text is GNU sed 4.10 and every cited behavior was exercised by the probe itself. - Perldoc, perlrun — read 2026-11-10:
-passumes a loop around the program that prints each line,-esupplies the program on the command line, and-0sets the input record separator. Box runs Perl 5.38.2; the-peform in this guide was exercised by the probe itself. - Linux Man Pages, gawk(1) — read 2026-11-10:
gawkis the GNU implementation of the AWK language, and the full manual documents the record separator and output record separator the probe relies on. Box runs GNU Awk 5.2.1; the manual text documents gawk 5.2 and every cited behavior was exercised by the probe itself. - CopyProgramming, Mastering sed: Replace CRLF with LF in Shell Scripts — read 2026-11-10: covers
sed,tr,awk, anddos2unixas methods for the same conversion, rates speed in words only withtrFastest andsedMedium, states thattrdeletes carriage returns fastest, and states thatdos2unixoutperformssedby 20 to 50 percent on large files. No timing table, no corpus, no reps, and no output verification; itsawkarm deletes every carriage return, while this probe'sawkarm preserves the mid-line ones. The closest published work, and the gap this probe fills. - Veldica, Seeing the Hidden Characters: cat -A vs sed -n l vs od -c, Measured — the sibling guide: measures the same carriage return as a character to display, with
cat -Ashowing every^Min the corpus, and found the same byte-identical-pair pattern,cat -vetoutput equal tocat -Aoutput. This guide converts the carriage returns; it does not display them. - Veldica, Hex-Dumping a 64 MiB Binary: xxd vs hexdump vs od, Measured — the sibling guide: measures
od -cas the verification tool for byte values, the tool this guide's checklist points at for checking line endings before converting. This guide converts line endings; it does not dump bytes.
Keep Exploring
Paste a [Service] section and get a plain-language verdict on every sandboxing setting it finds.
Paste sshd -G output and get a hardening verdict on the values your server actually runs.
Paste a server block and get WebSocket, TLS, HSTS, and proxy-header verdicts.
Paste a systemd timer expression and get the specific reason it fails, the normalized form, and the past-date traps the built-in check hides.
The full sysadmin hub \u2014 systemd, cgroups, file tools, and measured Linux comparisons.