Python

Python urllib.parse — URLs, Queries, and Quoting

Scripts meet URLs as text and need them as parts: the host to connect to, the query as named values, a relative link resolved against the page it came from. The urllib.parse module does that work with no package to install. A parsed URL exposes each part by name, a query string becomes a dict of lists, and quoting turns arbitrary text into text a URL can carry. This guide takes those parts in turn, then the joining trap, the rebuild path, and the errors a bad port raises.

Written November 16, 2026 · Tested with Python 3.12.3 · 11 min read

By the end of this guide
  • The main program: parse a URL into parts
  • urlparse or urlsplit, and the missing scheme
  • Joining: urljoin and the trailing slash
  • Reading a query string: parseqs and parseqsl

Written November 16, 2026

Tested with Python 3.12.3

What this guide is

The urllib.parse module treats a URL as a row of named parts. Parsing splits the text once, and the parts come back as attributes: scheme, hostname, port, path, query, fragment. The same module runs the process backwards, joining parts into a URL, and handles the two encodings a URL needs: the query string's name-and-value pairs and the percent-encoding that lets a space or a slash travel inside a value. Nothing here fetches anything; this is the text work a script does before and after a request.

The main program: parse a URL into parts

urlparse returns a result with six fields, and the network location breaks down further through properties. The hostname and scheme come back lowercased, the port comes back as an int, and geturl rebuilds the URL from its parts:

Python
from urllib.parse import urlparse
u = urlparse("https://alice:secret@Example.COM:8443/docs/search.html?q=python&page=2#results")
print(u.scheme, u.hostname, u.port)
print(u.username, u.password)
print(u.path, u.query, u.fragment)
print(u.netloc)
print(u.geturl())

The exact output:

Output
https example.com 8443
alice secret
/docs/search.html q=python&page=2 results
alice:secret@Example.COM:8443
https://alice:secret@Example.COM:8443/docs/search.html?q=python&page=2#results

Two details in that output matter in practice. The properties normalize: hostname lowercases the host and port converts the text to a number, while netloc keeps the text as it arrived, capital letters and all. The rebuilt geturl text matches the input here, but that is not a promise: in general the scheme is normalized to lower case and empty components are dropped, while the netloc text, host capitals included, survives the rebuild. And a URL with no port reports port as None, not zero, so a script tests for None before it decides a default. Code that needs only the host and path reads those two attributes and ignores the rest; code that logs a URL should drop the password property's value rather than print the whole netloc.

urlparse or urlsplit, and the missing scheme

The two parsers differ in one place. urlparse splits a ; parameter section off the last path segment into a separate params field; urlsplit leaves it in the path. Most URLs carry no such section, so the two agree until one shows up. The second trap in this program is the missing scheme: text that starts with a host but no scheme:// parses as a path, and the host lands nowhere:

Python
from urllib.parse import urlparse, urlsplit
print(urlparse("https://example.com/a/b;type=edit?x=1").params)
print(urlsplit("https://example.com/a/b;type=edit?x=1").path)
print(urlparse("example.com/path").hostname, urlparse("example.com/path").path)
print(urlparse("//example.com/path").hostname)

The exact output:

Output
type=edit
/a/b;type=edit
None example.com/path
example.com

The fix for the second trap sits in the last line: a network-path reference starting with // does expose the host, so a script that accepts scheme-less input from a file prefixes or checks for that form instead of trusting hostname to find a host in bare text. Between the parsers, urlsplit is the plainer choice for ordinary web URLs, and urlparse earns its keep only when a program actually reads the params field.

Joining: urljoin and the trailing slash

urljoin resolves a link found on a page against the page's own URL, following the same rules a browser applies. An absolute path replaces the whole path, .. climbs one level, a full URL replaces everything, and a link starting with // keeps the base scheme and swaps the host:

Python
from urllib.parse import urljoin
base = "https://example.com/docs/guide/index.html"
for rel in ("other.html", "/top.html", "../up.html", "https://other.net/x", "//cdn.net/lib.js", "?page=3"):
    print(rel, "->", urljoin(base, rel))
print(urljoin("https://example.com/docs/guide", "other.html"))
print(urljoin("https://example.com/docs/guide/", "other.html"))

The exact output:

Output
other.html -> https://example.com/docs/guide/other.html
/top.html -> https://example.com/top.html
../up.html -> https://example.com/docs/up.html
https://other.net/x -> https://other.net/x
//cdn.net/lib.js -> https://cdn.net/lib.js
?page=3 -> https://example.com/docs/guide/index.html?page=3
https://example.com/docs/other.html
https://example.com/docs/guide/other.html

The last two lines are the trap. A base URL without a trailing slash treats its last segment as a file name, so the relative link replaces that segment instead of sitting under it: /docs/guide plus other.html lands in /docs, not in /docs/guide. A crawler or a link checker that builds base URLs by hand adds the trailing slash when the base names a folder. A link that is only a query, like ?page=3, keeps the base path and swaps the query, which is the shape pagination links take.

Reading a query string: parse_qs and parse_qsl

A query string is a list of name-and-value pairs, and a name can repeat. parse_qs groups the pairs into a dict of lists, in the order the values arrived. parse_qsl keeps the pairs as a list of tuples instead. Both drop a pair whose value is blank unless told to keep it:

Python
from urllib.parse import parse_qs, parse_qsl
qs = "q=python&page=2&tag=a&tag=b&empty=&flag"
print(parse_qs(qs))
print(parse_qs(qs, keep_blank_values=True))
print(parse_qsl(qs))

The exact output:

Output
{'q': ['python'], 'page': ['2'], 'tag': ['a', 'b']}
{'q': ['python'], 'page': ['2'], 'tag': ['a', 'b'], 'empty': [''], 'flag': ['']}
[('q', 'python'), ('page', '2'), ('tag', 'a'), ('tag', 'b')]

Every value stays a string: page is '2', not 2, and a script converts the values it needs as numbers itself. The dropped pairs are the practical lesson. Both empty=, with an equals sign and nothing after it, and flag, with no equals sign at all, vanish from the default result, so a handler that treats a present-but-blank field differently from an absent one passes keep_blank_values=True. The values are percent-decoded and + reads as a space, the query encoding, before they reach the dict. Since Python 3.10 the only separator is &; a ; inside a value stays part of that value.

Building a query string: urlencode

urlencode runs the reading process backwards. It takes a dict, or a list of pairs when a name repeats, and percent-encodes the values as it joins them. A list value needs doseq=True to become repeated pairs; without it the list encodes as its own text form:

Python
from urllib.parse import urlencode
print(urlencode({"q": "a b", "page": 2}))
print(urlencode([("tag", "a"), ("tag", "b")]))
print(urlencode({"tag": ["a", "b"]}, doseq=True))
print(urlencode({"q": "café"}))

The exact output:

Output
q=a+b&page=2
tag=a&tag=b
tag=a&tag=b
q=caf%C3%A9

The space in the first line becomes +, which is the query-string form; a space inside a path segment encodes differently, as the next section shows. Numbers pass through as their text form, so the int 2 needs no conversion first. Non-ASCII text encodes as its UTF-8 bytes, each written as a percent escape, which is why café grows: the é is two bytes, C3 and A9.

Quoting: quote, quote_plus, and unquote

Percent-encoding is the general mechanism, and quote applies it to any text. Its default leaves / as it is, which suits a whole path; a single path segment or a value passes safe="" so a slash in the text cannot split into fake segments. quote_plus is the query form: it turns a space into + and encodes /:

Python
from urllib.parse import quote, quote_plus, unquote, unquote_plus
print(quote("a b/c?d"))
print(quote("a b/c?d", safe=""))
print(quote_plus("a b/c"))
print(unquote("a%20b%2Fc"))
print(unquote_plus("a+b%2Bc"))

The exact output:

Output
a%20b/c%3Fd
a%20b%2Fc%3Fd
a+b%2Fc
a b/c
a b+c

The pair of decoders shows why the two space forms must not mix. unquote leaves a + alone, because in a path a plus is a literal plus; unquote_plus turns it into a space and decodes %2B into the literal plus, which is the last line's result. Decoding a query value with unquote leaves stray plus signs where the spaces were, and decoding a path with unquote_plus invents spaces the path never had. Pick the pair that matches the part of the URL the text came from.

Rebuilding a URL, and the port trap

Trap
A parsed result is a tuple with named fields, and _replace returns a copy with named fields changed. urlunsplit joins a result back into text, and urldefrag splits a fragment off a URL a script is about to fetch, since a fragment is never sent to a server. The traps section of this program is the port property: reading it on a URL whose port text is not a usable port raises ValueError at access time, not at parse time:
Python
from urllib.parse import urlparse, urlsplit, urlunsplit, urldefrag
parts = urlsplit("https://example.com/a/b?x=1#frag")
changed = parts._replace(path="/a/c", query="y=2")
print(urlunsplit(changed))
print(urldefrag("https://example.com/a#frag"))
for raw in ("https://example.com:99999/", "https://example.com:abc/"):
    try:
        print(urlparse(raw).port)
    except ValueError as e:
        print("ValueError:", e)

The exact output:

Output
https://example.com/a/c?y=2#frag
DefragResult(url='https://example.com/a', fragment='frag')
ValueError: Port out of range 0-65535
ValueError: Port could not be cast to integer value as 'abc'

The parse itself succeeded on both bad URLs; the failure waits until something reads port, which can be far from the line that accepted the input. A script that takes URLs from outside input reads port once, early, inside a try block, or checks the text itself, instead of letting the error surface inside a connection routine. The _replace path is the safe way to change one part of a URL: string surgery on the full text breaks the first time a value contains the character the surgery searched for.

Which choice for which job

Job Choice Why
Split a URL and read its parts urlsplit Leaves the path whole; properties for host and port
Read a ; parameter section urlparse The only parser that splits params out of the path
Resolve a link against its page urljoin Applies the browser's relative-link rules
A query string as named values parse_qs Groups repeated names into lists
A query string keeping pair order parse_qsl Returns the pairs as a list of tuples
Build a query string urlencode Encodes values; doseq=True repeats a list value
Encode text for a path quote Percent-encodes, with safe controlling what passes
Encode text for a query value quote_plus Space becomes +, matching the query form
Change one part of a URL _replace plus urlunsplit Edits a named field instead of the raw text

Checklist

  • Read hostname and port as properties; expect lowercase text and an int, or None for a missing port.
  • Treat scheme-less text as a path until it starts with //; hostname finds no host in bare text.
  • Add the trailing slash when a urljoin base names a folder, or the last segment is replaced.
  • Pass keep_blank_values=True when a blank field means something different from an absent one.
  • Convert query values yourself; parse_qs returns strings, in lists, even for a single value.
  • Match the quoting pair to the URL part: quote and unquote for paths, the _plus pair for queries.
  • Read port early on outside input, inside a try block; a bad port raises when the property is read.

Sources

  • Python Standard Library, urllib.parse — urlparse and urlsplit, the result properties, urljoin, parse_qs and parse_qsl, urlencode, quote and unquote and their _plus forms, urlunsplit and urldefrag, and the port validation errors.

Keep Exploring