Python urllib.parse — URLs, Queries, and Quoting
Scripts meet URLs as text and need them as parts: the host to connect to, the query as named values, a relative link resolved against the page it came from. The urllib.parse module does that work with no package to install. A parsed URL exposes each part by name, a query string becomes a dict of lists, and quoting turns arbitrary text into text a URL can carry. This guide takes those parts in turn, then the joining trap, the rebuild path, and the errors a bad port raises.
Written November 16, 2026 · Tested with Python 3.12.3 · 11 min read
- The main program: parse a URL into parts
- urlparse or urlsplit, and the missing scheme
- Joining: urljoin and the trailing slash
- Reading a query string: parseqs and parseqsl
Written November 16, 2026
Tested with Python 3.12.3
What this guide is
The urllib.parse module treats a URL as a row of named parts. Parsing splits the text once, and the parts come back as attributes: scheme, hostname, port, path, query, fragment. The same module runs the process backwards, joining parts into a URL, and handles the two encodings a URL needs: the query string's name-and-value pairs and the percent-encoding that lets a space or a slash travel inside a value. Nothing here fetches anything; this is the text work a script does before and after a request.
The main program: parse a URL into parts
urlparse returns a result with six fields, and the network location breaks down further through properties. The hostname and scheme come back lowercased, the port comes back as an int, and geturl rebuilds the URL from its parts:
from urllib.parse import urlparse
u = urlparse("https://alice:secret@Example.COM:8443/docs/search.html?q=python&page=2#results")
print(u.scheme, u.hostname, u.port)
print(u.username, u.password)
print(u.path, u.query, u.fragment)
print(u.netloc)
print(u.geturl())
The exact output:
https example.com 8443
alice secret
/docs/search.html q=python&page=2 results
alice:secret@Example.COM:8443
https://alice:secret@Example.COM:8443/docs/search.html?q=python&page=2#results
Two details in that output matter in practice. The properties normalize: hostname lowercases the host and port converts the text to a number, while netloc keeps the text as it arrived, capital letters and all. The rebuilt geturl text matches the input here, but that is not a promise: in general the scheme is normalized to lower case and empty components are dropped, while the netloc text, host capitals included, survives the rebuild. And a URL with no port reports port as None, not zero, so a script tests for None before it decides a default. Code that needs only the host and path reads those two attributes and ignores the rest; code that logs a URL should drop the password property's value rather than print the whole netloc.
urlparse or urlsplit, and the missing scheme
The two parsers differ in one place. urlparse splits a ; parameter section off the last path segment into a separate params field; urlsplit leaves it in the path. Most URLs carry no such section, so the two agree until one shows up. The second trap in this program is the missing scheme: text that starts with a host but no scheme:// parses as a path, and the host lands nowhere:
from urllib.parse import urlparse, urlsplit
print(urlparse("https://example.com/a/b;type=edit?x=1").params)
print(urlsplit("https://example.com/a/b;type=edit?x=1").path)
print(urlparse("example.com/path").hostname, urlparse("example.com/path").path)
print(urlparse("//example.com/path").hostname)
The exact output:
type=edit
/a/b;type=edit
None example.com/path
example.com
The fix for the second trap sits in the last line: a network-path reference starting with // does expose the host, so a script that accepts scheme-less input from a file prefixes or checks for that form instead of trusting hostname to find a host in bare text. Between the parsers, urlsplit is the plainer choice for ordinary web URLs, and urlparse earns its keep only when a program actually reads the params field.
Joining: urljoin and the trailing slash
urljoin resolves a link found on a page against the page's own URL, following the same rules a browser applies. An absolute path replaces the whole path, .. climbs one level, a full URL replaces everything, and a link starting with // keeps the base scheme and swaps the host:
from urllib.parse import urljoin
base = "https://example.com/docs/guide/index.html"
for rel in ("other.html", "/top.html", "../up.html", "https://other.net/x", "//cdn.net/lib.js", "?page=3"):
print(rel, "->", urljoin(base, rel))
print(urljoin("https://example.com/docs/guide", "other.html"))
print(urljoin("https://example.com/docs/guide/", "other.html"))
The exact output:
other.html -> https://example.com/docs/guide/other.html
/top.html -> https://example.com/top.html
../up.html -> https://example.com/docs/up.html
https://other.net/x -> https://other.net/x
//cdn.net/lib.js -> https://cdn.net/lib.js
?page=3 -> https://example.com/docs/guide/index.html?page=3
https://example.com/docs/other.html
https://example.com/docs/guide/other.html
The last two lines are the trap. A base URL without a trailing slash treats its last segment as a file name, so the relative link replaces that segment instead of sitting under it: /docs/guide plus other.html lands in /docs, not in /docs/guide. A crawler or a link checker that builds base URLs by hand adds the trailing slash when the base names a folder. A link that is only a query, like ?page=3, keeps the base path and swaps the query, which is the shape pagination links take.
Reading a query string: parse_qs and parse_qsl
A query string is a list of name-and-value pairs, and a name can repeat. parse_qs groups the pairs into a dict of lists, in the order the values arrived. parse_qsl keeps the pairs as a list of tuples instead. Both drop a pair whose value is blank unless told to keep it:
from urllib.parse import parse_qs, parse_qsl
qs = "q=python&page=2&tag=a&tag=b&empty=&flag"
print(parse_qs(qs))
print(parse_qs(qs, keep_blank_values=True))
print(parse_qsl(qs))
The exact output:
{'q': ['python'], 'page': ['2'], 'tag': ['a', 'b']}
{'q': ['python'], 'page': ['2'], 'tag': ['a', 'b'], 'empty': [''], 'flag': ['']}
[('q', 'python'), ('page', '2'), ('tag', 'a'), ('tag', 'b')]
Every value stays a string: page is '2', not 2, and a script converts the values it needs as numbers itself. The dropped pairs are the practical lesson. Both empty=, with an equals sign and nothing after it, and flag, with no equals sign at all, vanish from the default result, so a handler that treats a present-but-blank field differently from an absent one passes keep_blank_values=True. The values are percent-decoded and + reads as a space, the query encoding, before they reach the dict. Since Python 3.10 the only separator is &; a ; inside a value stays part of that value.
Building a query string: urlencode
urlencode runs the reading process backwards. It takes a dict, or a list of pairs when a name repeats, and percent-encodes the values as it joins them. A list value needs doseq=True to become repeated pairs; without it the list encodes as its own text form:
from urllib.parse import urlencode
print(urlencode({"q": "a b", "page": 2}))
print(urlencode([("tag", "a"), ("tag", "b")]))
print(urlencode({"tag": ["a", "b"]}, doseq=True))
print(urlencode({"q": "café"}))
The exact output:
q=a+b&page=2
tag=a&tag=b
tag=a&tag=b
q=caf%C3%A9
The space in the first line becomes +, which is the query-string form; a space inside a path segment encodes differently, as the next section shows. Numbers pass through as their text form, so the int 2 needs no conversion first. Non-ASCII text encodes as its UTF-8 bytes, each written as a percent escape, which is why café grows: the é is two bytes, C3 and A9.
Quoting: quote, quote_plus, and unquote
Percent-encoding is the general mechanism, and quote applies it to any text. Its default leaves / as it is, which suits a whole path; a single path segment or a value passes safe="" so a slash in the text cannot split into fake segments. quote_plus is the query form: it turns a space into + and encodes /:
from urllib.parse import quote, quote_plus, unquote, unquote_plus
print(quote("a b/c?d"))
print(quote("a b/c?d", safe=""))
print(quote_plus("a b/c"))
print(unquote("a%20b%2Fc"))
print(unquote_plus("a+b%2Bc"))
The exact output:
a%20b/c%3Fd
a%20b%2Fc%3Fd
a+b%2Fc
a b/c
a b+c
The pair of decoders shows why the two space forms must not mix. unquote leaves a + alone, because in a path a plus is a literal plus; unquote_plus turns it into a space and decodes %2B into the literal plus, which is the last line's result. Decoding a query value with unquote leaves stray plus signs where the spaces were, and decoding a path with unquote_plus invents spaces the path never had. Pick the pair that matches the part of the URL the text came from.
Rebuilding a URL, and the port trap
_replace returns a copy with named fields changed. urlunsplit joins a result back into text, and urldefrag splits a fragment off a URL a script is about to fetch, since a fragment is never sent to a server. The traps section of this program is the port property: reading it on a URL whose port text is not a usable port raises ValueError at access time, not at parse time:from urllib.parse import urlparse, urlsplit, urlunsplit, urldefrag
parts = urlsplit("https://example.com/a/b?x=1#frag")
changed = parts._replace(path="/a/c", query="y=2")
print(urlunsplit(changed))
print(urldefrag("https://example.com/a#frag"))
for raw in ("https://example.com:99999/", "https://example.com:abc/"):
try:
print(urlparse(raw).port)
except ValueError as e:
print("ValueError:", e)
The exact output:
https://example.com/a/c?y=2#frag
DefragResult(url='https://example.com/a', fragment='frag')
ValueError: Port out of range 0-65535
ValueError: Port could not be cast to integer value as 'abc'
The parse itself succeeded on both bad URLs; the failure waits until something reads port, which can be far from the line that accepted the input. A script that takes URLs from outside input reads port once, early, inside a try block, or checks the text itself, instead of letting the error surface inside a connection routine. The _replace path is the safe way to change one part of a URL: string surgery on the full text breaks the first time a value contains the character the surgery searched for.
Which choice for which job
| Job | Choice | Why |
|---|---|---|
| Split a URL and read its parts | urlsplit |
Leaves the path whole; properties for host and port |
Read a ; parameter section |
urlparse |
The only parser that splits params out of the path |
| Resolve a link against its page | urljoin |
Applies the browser's relative-link rules |
| A query string as named values | parse_qs |
Groups repeated names into lists |
| A query string keeping pair order | parse_qsl |
Returns the pairs as a list of tuples |
| Build a query string | urlencode |
Encodes values; doseq=True repeats a list value |
| Encode text for a path | quote |
Percent-encodes, with safe controlling what passes |
| Encode text for a query value | quote_plus |
Space becomes +, matching the query form |
| Change one part of a URL | _replace plus urlunsplit |
Edits a named field instead of the raw text |
Checklist
- Read
hostnameandportas properties; expect lowercase text and an int, orNonefor a missing port. - Treat scheme-less text as a path until it starts with
//;hostnamefinds no host in bare text. - Add the trailing slash when a
urljoinbase names a folder, or the last segment is replaced. - Pass
keep_blank_values=Truewhen a blank field means something different from an absent one. - Convert query values yourself;
parse_qsreturns strings, in lists, even for a single value. - Match the quoting pair to the URL part:
quoteandunquotefor paths, the_pluspair for queries. - Read
portearly on outside input, inside atryblock; a bad port raises when the property is read.
Related guides on this site
- Python regular expressions — search, match, and split — the pattern tools for the values a query string carries, once they are decoded.
- Python ipaddress — networks, hosts, and subnets — the host half of a connection, once the URL has given up its hostname.
- Python files — open, read, and write — reading the link lists and URL files a script then parses.
Sources
- Python Standard Library,
urllib.parse—urlparseandurlsplit, the result properties,urljoin,parse_qsandparse_qsl,urlencode,quoteandunquoteand their_plusforms,urlunsplitandurldefrag, and the port validation errors.
Keep Exploring
The full Python hub \u2014 language basics, the standard library, and data tools.
Every guide on the site in one list, grouped by topic.
Every browser-only checker and verifier on the site.
The C++, Python, Sysadmin, Rust, Go, and TypeScript hubs.