URL

This is the manual for the url Emacs Lisp library.

Copyright © 1993–1999, 2002, 2004–2022 Free Software Foundation, Inc.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover Texts being “A GNU Manual,” and with the Back-Cover Texts as in (a) below. A copy of the license is included in the section entitled “GNU Free Documentation License”.

(a) The FSF’s Back-Cover Text is: “You have the freedom to copy and modify this GNU manual.”

Table of Contents


1 Introduction

A Uniform Resource Identifier (URI) is a specially-formatted name, such as an Internet address, that identifies some name or resource. The format of URIs is described in RFC 3986, which updates and replaces the earlier RFCs 2732, 2396, 1808, and 1738. A Uniform Resource Locator (URL) is an older but still-common term, which basically refers to a URI corresponding to a resource that can be accessed (usually over a network) in a specific way.

Here are some examples of URIs (taken from RFC 3986):

ftp://ftp.is.co.za/rfc/rfc1808.txt
https://www.ietf.org/rfc/rfc2396.txt
ldap://[2001:db8::7]/c=GB?objectClass?one
mailto:John.Doe@example.com
news:comp.infosystems.www.servers.unix
tel:+1-816-555-1212
telnet://192.0.2.16:80/
urn:oasis:names:specification:docbook:dtd:xml:4.1.2

This manual describes the url library, an Emacs Lisp library for parsing URIs and retrieving the resources to which they refer. (The library is so-named for historical reasons; nowadays, the “URI” terminology is regarded as the more general one, and “URL” is technically obsolete despite its widespread vernacular usage.)


2 URI Parsing

A URI consists of several components, each having a different meaning. For example, the URI

https://www.gnu.org/software/emacs/

specifies the scheme component ‘https’, the hostname component ‘www.gnu.org’, and the path component ‘/software/emacs/’.

The format of URIs is specified by RFC 3986. The url library provides the Lisp function url-generic-parse-url, a (mostly) standard-compliant URI parser, as well as function url-recreate-url, which converts a parsed URI back into a URI string.

Function: url-generic-parse-url uri-string

This function returns a parsed version of the string uri-string.

Function: url-recreate-url uri-obj

Given a parsed URI, this function returns the corresponding URI string.

The return value of url-generic-parse-url, and the argument expected by url-recreate-url, is a parsed URI: a CL structure whose slots hold the various components of the URI. See the CL Manual in GNU Emacs Common Lisp Emulation, for details about CL structures. Most of the other functions in the url library act on parsed URIs.


2.1 Parsed URI structures

Each parsed URI structure contains the following slots:

type

The URI scheme (a string, e.g., http). See Supported URL Types, for a list of schemes that the url library knows how to process. This slot can also be nil, if the URI is not fully specified.

user

The user name (a string), or nil.

password

The user password (a string), or nil. The use of this URI component is strongly discouraged; nowadays, passwords are transmitted by other means, not as part of a URI.

host

The host name (a string), or nil. If present, this is typically a domain name or IP address.

port

The port number (an integer), or nil. Omitting this component usually means to use the “standard” port associated with the URI scheme.

filename

The combination of the “path” and “query” components of the URI (a string), or nil. If the query component is present, it is the substring following the first ‘?’ character, and the path component is the substring before the ‘?’. The meaning of these components is scheme-dependent; they do not necessarily refer to a file on a disk.

target

The fragment component (a string), or nil. The fragment component specifies a “secondary resource”, such as a section of a webpage.

fullness

This is t if the URI is fully specified, i.e., the hierarchical components of the URI (the hostname and/or username and/or password) are preceded by ‘//’.

These slots have accessors named url-part, where part is the slot name. For example, the accessor for the host slot is the function url-host. The url-port accessor returns the default port for the URI scheme if the parsed URI’s port slot is nil.

The slots can be set using setf. For example:

(setf (url-port url) 80)

2.2 URI Encoding

The url-generic-parse-url parser does not obey RFC 3986 in one respect: it allows non-ASCII characters in URI strings.

Strictly speaking, RFC 3986 compatible URIs may only consist of ASCII characters; non-ASCII characters are represented by converting them to UTF-8 byte sequences, and performing percent encoding on the bytes. For example, the o-umlaut character is converted to the UTF-8 byte sequence ‘\xD3\xA7’, then percent encoded to ‘%D3%A7’. (Certain “reserved” ASCII characters must also be percent encoded when they appear in URI components.)

The function url-encode-url can be used to convert a URI string containing arbitrary characters to one that is properly percent-encoded in accordance with RFC 3986.

Function: url-encode-url url-string

This function return a properly URI-encoded version of url-string. It also performs URI normalization, e.g., converting the scheme component to lowercase if it was previously uppercase.

To convert between a string containing arbitrary characters and a percent-encoded all-ASCII string, use the functions url-hexify-string and url-unhex-string:

Function: url-hexify-string string &optional allowed-chars

This function performs percent-encoding on string, and returns the result.

If string is multibyte, it is first converted to a UTF-8 byte string. Each byte corresponding to an allowed character is left as-is, while all other bytes are converted to a three-character sequence: ‘%’ followed by two upper-case hex digits.

The allowed characters are specified by allowed-chars. If this argument is nil, the allowed characters are those specified as unreserved characters by RFC 3986 (see the variable url-unreserved-chars). Otherwise, allowed-chars should be either a list of allowed chars, or a vector whose Nth element is non-nil if character N is allowed.

Function: url-unhex-string string &optional allow-newlines

This function replaces percent-encoding sequences in string with their character equivalents, and returns the resulting string.

If allow-newlines is non-nil, it allows the decoding of carriage returns and line feeds, which are normally forbidden in URIs.


3 Retrieving URLs

The url library defines the following three functions for retrieving the data specified by a URL. The actual retrieval protocol depends on the URL’s URI scheme, and is performed by lower-level scheme-specific functions. (Those lower-level functions are not documented here, and generally should not be called directly.)

In each of these functions, the url argument can be either a string or a parsed URL structure. If it is a string, that string is passed through url-encode-url before using it, to ensure that it is properly URI-encoded (see URI Encoding).

Function: url-retrieve-synchronously url &optional silent no-cookies timeout

This function synchronously retrieves the data specified by url, and returns a buffer containing the data. The return value is nil if there is no data associated with the URL (as is the case for dired, info, and mailto URLs).

If the optional argument silent is non-nil, progress messages are suppressed. If the optional argument no-cookies is non-nil, cookies are not stored or sent. If the optional argument timeout is non-nil, it should be a number that says (in seconds) how long to wait for a response before giving up.

Function: url-retrieve url callback &optional cbargs silent no-cookies

This function retrieves url asynchronously, calling the function callback when the object has been completely retrieved. The return value is the buffer into which the data will be inserted, or nil if the process has already completed.

The callback function is called this way:

(apply callback status cbargs)

where status is a plist representing what happened during the retrieval, with most recent events first, or an empty list if no events have occurred. Each pair in the plist is one of:

(:redirect redirected-to)

This means that the request was redirected to the URL redirected-to.

(:error (error-symbol . data))

This means that an error occurred. If so desired, the error can be signaled with (signal error-symbol data).

When the callback function is called, the current buffer is the one containing the retrieved data (if any). The buffer also contains any MIME headers associated with the data retrieval.

If the optional argument silent is non-nil, progress messages are suppressed. If the optional argument no-cookies is non-nil, cookies are not stored or sent.

Function: url-queue-retrieve url callback &optional cbargs silent no-cookies

This function acts like url-retrieve, but with limits on the number of concurrently-running network processes. The option url-queue-parallel-processes controls the number of concurrent processes, and the option url-queue-timeout sets a timeout in seconds.

To use this function, you must (require 'url-queue).

User Option: url-queue-parallel-processes

The value of this option is an integer specifying the maximum number of concurrent url-queue-retrieve network processes. If the number of url-queue-retrieve calls is larger than this number, later ones are queued until earlier ones are finished.

User Option: url-queue-timeout

The value of this option is a number specifying the maximum lifetime of a url-queue-retrieve network process, once it is started. If a process is not finished by then, it is killed and removed from the queue.


4 Supported URL Types

This chapter describes functions and variables affecting URL retrieval for specific schemes.


4.1 http and https

The http scheme refers to the Hypertext Transfer Protocol. The url library supports HTTP version 1.1, specified in RFC 2616. Its default port is 80.

The https scheme is a secure version of http, with transmission via SSL. It is defined in RFC 2069, and its default port is 443. When using https, the url library performs SSL encryption via the ssl library, by forcing the ssl gateway method to be used. See Gateways in General.

User Option: url-honor-refresh-requests

If this option is non-nil (the default), the url library honors the HTTP ‘Refresh’ header, which is used by servers to direct clients to reload documents from the same URL or a different one. If the value is nil, the ‘Refresh’ header is ignored; any other value means to ask the user on each request.


4.1.1 Cookies

Function: url-cookie-list

This command creates a *url cookies* buffer listing the current cookies, if there are any. You can remove a cookie using the C-k (url-cookie-delete) command.

Function: url-cookie-delete-cookies &optional regexp

This function takes a regular expression as its parameters and deletes all cookies from that domain. If regexp is nil, delete all cookies.

User Option: url-cookie-file

The file in which cookies are stored, defaulting to cookies in the directory specified by url-configuration-directory.

User Option: url-cookie-confirmation

Specifies whether confirmation is required to accept cookies.

User Option: url-cookie-multiple-line

Specifies whether to put all cookies for the server on one line in the HTTP request to satisfy broken servers.

User Option: url-cookie-trusted-urls

A list of regular expressions matching URLs from which to accept cookies always.

User Option: url-cookie-untrusted-urls

A list of regular expressions matching URLs from which to reject cookies always.

User Option: url-cookie-save-interval

The number of seconds between automatic saves of cookies to disk. Default is one hour.


4.1.2 Language and Encoding Preferences

HTTP allows clients to express preferences for the language and encoding of documents which servers may honor. For each of these variables, the value is a string; it can specify a single choice, or it can be a comma-separated list.

Normally, this list is ordered by descending preference. However, each element can be followed by ‘;q=priority’ to specify its preference level, a decimal number from 0 to 1; e.g., for url-mime-language-string, "de, en-gb;q=0.8, en;q=0.7". An element that has no ‘;q’ specification has preference level 1.

User Option: url-mime-charset-string

This variable specifies a preference for character sets when documents can be served in more than one encoding.

HTTP allows specifying a series of MIME charsets which indicate your preferred character set encodings, e.g., Latin-9 or Big5, and these can be weighted. The default series is generated automatically from the associated MIME types of all defined coding systems, sorted by the coding system priority specified in Emacs. See Recognizing Coding Systems in The GNU Emacs Manual.

User Option: url-mime-language-string

A string specifying the preferred language when servers can serve files in several languages. Use RFC 1766 abbreviations, e.g., ‘en’ for English, ‘de’ for German.

The string can be "*" to get the first available language (as opposed to the default).


4.1.3 HTTP URL Options

HTTP supports an ‘OPTIONS’ method describing things supported by the URL.

Function: url-http-options url

Returns a property list describing options available for URL. The property list members are:

methods

A list of symbols specifying what HTTP methods the resource supports.

dav

A list of numbers specifying what DAV protocol/schema versions are supported.

dasl

A list of supported DASL search types supported (string form).

ranges

A list of the units available for use in partial document fetches.

p3p

The Platform For Privacy Protection description for the resource. Currently this is just the raw header contents.


4.1.4 Dealing with HTTP documents

HTTP URLs are retrieved into a buffer containing the HTTP headers followed by the body. Since the headers are quasi-MIME, they may be processed using the MIME library. See Emacs MIME in The Emacs MIME Manual.


4.2 file and ftp

The ftp and file schemes are defined in RFC 1808. The url library treats ‘ftp:’ and ‘file:’ as synonymous. Such URLs have the form

ftp://user:password@host:port/file
file://user:password@host:port/file

If the URL specifies a local file, it is retrieved by reading the file contents in the usual way. If it specifies a remote file, it is retrieved using either the Tramp or the Ange-FTP package. See Remote Files in The GNU Emacs Manual.

When retrieving a compressed file, it is automatically uncompressed if it has the file suffix .z, .gz, .Z, .bz2, or .xz. (The list of supported suffixes is hard-coded, and cannot be altered by customizing jka-compr-compression-info-list.)


4.3 info

The info scheme is non-standard. Such URLs have the form

info:file#node

and are retrieved by invoking Info-goto-node with argument ‘(file)node’. If ‘#node’ is omitted, the ‘Top’ node is opened.


4.4 mailto

A mailto URL specifies an email message to be sent to a given email address. For example, ‘mailto:foo@bar.com’ specifies sending a message to ‘foo@bar.com’. The “retrieval method” for such URLs is to open a mail composition buffer in which the appropriate content (e.g., the recipient address) has been filled in.

As defined in RFC 6068, a mailto URL can have the form

mailto:mailbox[?header=contents[&header=contents]]

where an arbitrary number of headers can be added. If the header is ‘body’, then contents is put in the message body; otherwise, a header header field is created with contents as its contents. Note that the url library does not perform any checking of header or contents, so you should check them before sending the message.

User Option: url-mail-command

The value of this variable is the function called whenever url needs to send mail. This should normally be left its default, which is the standard mail-composition command compose-mail. See Sending Mail in The GNU Emacs Manual.

If the document containing the mailto URL itself possessed a known URL, Emacs automatically inserts an ‘X-Url-From’ header field into the mail buffer, specifying that URL.


4.5 news, nntp and snews

The news, nntp, and snews schemes, defined in RFC 1738, are used for reading Usenet newsgroups. For compatibility with non-standard-compliant news clients, the url library allows host and port fields to be included in news URLs, even though they are properly only allowed for nntp and snews.

news and nntp URLs have the following form:

news:newsgroup

Retrieves a list of messages in newsgroup;

news:message-id

Retrieves the message with the given message-id;

news:*

Retrieves a list of all available newsgroups;

nntp://host:port/newsgroup
nntp://host:port/message-id
nntp://host:port/*

Similar to the ‘news’ versions.

The default port for nntp (and news) is 119. The difference between an nntp URL and a news URL is that an nttp URL may specify an article by its number. The ‘snews’ scheme is the same as ‘nntp’, except that it is tunneled through SSL and has default port 563.

These URLs are retrieved via the Gnus package.

User Option: url-news-server

This variable specifies the default news server from which to fetch news, if no server was specified in the URL. The default value, nil, means to use the server specified by the standard environment variable ‘NNTPSERVER’, or ‘news’ if that environment variable is unset.


4.6 rlogin, telnet and tn3270

These URL schemes are defined in RFC 1738, and are used for logging in via a terminal emulator. They have the form

telnet://user:password@host:port

but the password component is ignored. By default, the telnet scheme is handled via Tramp (see URL Types Supported via Tramp).

To handle rlogin, telnet and tn3270 URLs, a rlogin, telnet or tn3270 (the program names and arguments are hardcoded) session is run in a terminal-emulator buffer. Well-known ports are used if the URL does not specify a port.


4.7 irc

The irc scheme is defined in the Internet Draft at https://www.w3.org/Addressing/draft-mirashi-url-irc-01.txt (which was never approved as an RFC). Such URLs have the form

irc://host:port/target,needpass

and are retrieved by opening an IRC session using the function specified by url-irc-function.

User Option: url-irc-function

The value of this option is a function, which is called to open an IRC connection for irc URLs. This function must take five arguments, host, port, channel, user and password. The channel argument specifies the channel to join immediately, and may be nil.

The default is url-irc-rcirc, which uses the Rcirc package. Other options are url-irc-erc (which uses ERC) and url-irc-zenirc (which uses ZenIRC).


4.8 data

The data scheme, defined in RFC 2397, contains MIME data in the URL itself. Such URLs have the form

data:[media-type][;base64],data

media-type is a MIME ‘Content-Type’ string, possibly including parameters. It defaults to ‘text/plain;charset=US-ASCII’. The ‘text/plain’ can be omitted but the charset parameter supplied. If ‘;base64’ is present, the data are base64-encoded.


4.9 nfs

The nfs scheme, defined in RFC 2224, is similar to ftp except that it points to a file on a remote host that is handled by an NFS automounter on the local host. Such URLs have the form

nfs://user:password@host:port/file
Variable: url-nfs-automounter-directory-spec

A string saying how to invoke the NFS automounter. Certain ‘%’ sequences are recognized:

%h

The hostname of the NFS server;

%n

The port number of the NFS server;

%u

The username to use to authenticate;

%p

The password to use to authenticate;

%f

The filename on the remote server;

%%

A literal ‘%’.

Each can be used any number of times.


4.10 ldap

The LDAP scheme is defined in RFC 2255.


4.11 man

The man scheme is a non-standard one. Such URLs have the form

man:page-spec

and are retrieved by passing page-spec to the Lisp function man.


4.12 URL Types Supported via Tramp

Some additional URL types are supported by passing them to Tramp (see The Tramp Manual in The Tramp Manual). These protocols are listed in the url-tramp-protocols variable, which you can customize. The default value includes the following protocols:

ftp

The file transfer protocol. See file and ftp.

ssh

The secure shell protocol. See Inline methods in The Tramp Manual.

scp

The secure file copy protocol. See External methods in The Tramp Manual.

rsync

The remote sync protocol.

telnet

The telnet protocol.


5 General Facilities


5.1 Disk Caching

The disk cache stores retrieved documents locally, whence they can be retrieved more quickly. When requesting a URL that is in the cache, the library checks to see if the page has changed since it was last retrieved from the remote machine. If not, the local copy is used, saving the transmission over the network. Currently the cache isn’t cleared automatically.

User Option: url-automatic-caching

Setting this variable non-nil causes documents to be cached automatically.

User Option: url-cache-directory

This variable specifies the directory to store the cache files. It defaults to sub-directory cache of url-configuration-directory.

User Option: url-cache-creation-function

The cache relies on a scheme for mapping URLs to files in the cache. This variable names a function which sets the type of cache to use. It takes a URL as argument and returns the absolute file name of the corresponding cache file. The two supplied possibilities are url-cache-create-filename-using-md5 and url-cache-create-filename-human-readable.

Function: url-cache-create-filename-using-md5 url

Creates a cache file name from url using MD5 hashing. This is creates entries with very few cache collisions and is fast.

(url-cache-create-filename-using-md5 "http://www.example.com/foo/bar")
  ⇒ "/home/fx/.url/cache/fx/http/com/example/www/b8a35774ad20db71c7c3409a5410e74f"
Function: url-cache-create-filename-human-readable url

Creates a cache file name from url more obviously connected to url than for url-cache-create-filename-using-md5, but more likely to conflict with other files.

(url-cache-create-filename-human-readable "http://www.example.com/foo/bar")
  ⇒ "/home/fx/.url/cache/fx/http/com/example/www/foo/bar"
Function: url-cache-expired

This function returns non-nil if a cache entry has expired (or is absent). The arguments are a URL and optional expiration delay in seconds (default url-cache-expire-time).

User Option: url-cache-expire-time

This variable is the default number of seconds to use for the expire-time argument of the function url-cache-expired.

Function: url-fetch-from-cache

This function takes a URL as its argument and returns a buffer containing the data cached for that URL.


5.2 Proxies and Gatewaying

Proxy servers are commonly used to provide gateways through firewalls or as caches serving some more-or-less local network. Each protocol (HTTP, FTP, etc.) can have a different gateway server. Proxying is conventionally configured commonly amongst different programs through environment variables of the form protocol_proxy, where protocol is one of the supported network protocols (http, ftp etc.). The library recognizes such variables in either upper or lower case. Their values are of one of the forms:

  • host:port
  • A full URL;
  • Simply a host name.

The NO_PROXY environment variable specifies URLs that should be excluded from proxying (on servers that should be contacted directly). This should be a comma-separated list of hostnames, domain names, or a mixture of both. Asterisks can be used as wildcards, but other clients may not support that. Domain names may be indicated by a leading dot. For example:

NO_PROXY="*.aventail.com,home.com,.seanet.com"

says to contact all machines in the ‘aventail.com’ and ‘seanet.com’ domains directly, as well as the machine named ‘home.com’. If NO_PROXY isn’t defined, no_PROXY and no_proxy are also tried, in that order.

Proxies may also be specified directly in Lisp.

User Option: url-proxy-services

This variable is an alist of URL schemes and proxy servers that gateway them. The items are of the form (scheme . host:portnumber), says that the URL scheme is gatewayed through portnumber on the specified host. An exception is the pseudo scheme "no_proxy", which is paired with a regexp matching host names not to be proxied. This variable is initialized from the environment as above.

(setq url-proxy-services
      '(("http"     . "proxy.aventail.com:80")
        ("no_proxy" . "^.*\\(aventail\\|seanet\\)\\.com")))

5.3 Gateways in General

The library provides a general gateway layer through which all networking passes. It can both control access to the network and provide access through gateways in firewalls. This may make direct connections in some cases and pass through some sort of gateway in others.1 The library’s basic function responsible for making connections is url-open-stream.

Function: url-open-stream name buffer host service

Open a stream to host, possibly via a gateway. The other arguments are as for open-network-stream. This will not make a connection if url-gateway-unplugged is non-nil.

Variable: url-gateway-local-host-regexp

This is a regular expression that matches local hosts that do not require the use of a gateway. If nil, all connections are made through the gateway.

Variable: url-gateway-method

This variable controls which gateway method is used. It may be useful to bind it temporarily in some applications. It has values taken from a list of symbols. Possible values are:

telnet

Use this method if you must first telnet and log into a gateway host, and then run telnet from that host to connect to outside machines.

rlogin

This method is identical to telnet, but uses rlogin to log into the remote machine without having to send the username and password over the wire every time.

socks

Use if the firewall has a SOCKS gateway running on it. The SOCKS v5 protocol is defined in RFC 1928.

native

This method uses Emacs’s builtin networking directly. This is the default. It can be used only if there is no firewall blocking access.

The following variables control the gateway methods.

User Option: url-gateway-telnet-host

The gateway host to telnet to. Once logged in there, you then telnet out to the hosts you want to connect to.

User Option: url-gateway-telnet-parameters

This should be a list of parameters to pass to the telnet program.

User Option: url-gateway-telnet-password-prompt

This is a regular expression that matches the password prompt when logging in.

User Option: url-gateway-telnet-login-prompt

This is a regular expression that matches the username prompt when logging in.

User Option: url-gateway-telnet-user-name

The username to log in with.

User Option: url-gateway-telnet-password

The password to send when logging in.

User Option: url-gateway-prompt-pattern

This is a regular expression that matches the shell prompt.

User Option: url-gateway-rlogin-host

Host to ‘rlogin’ to before telnetting out.

User Option: url-gateway-rlogin-parameters

Parameters to pass to ‘rsh’.

User Option: url-gateway-rlogin-user-name

User name to use when logging in to the gateway.

User Option: url-gateway-prompt-pattern

This is a regular expression that matches the shell prompt.

User Option: socks-server

This specifies the default server, it takes the form ("Default server" server port version) where version can be either 4 or 5.

Variable: socks-password

If this is nil then you will be asked for the password, otherwise it will be used as the password for authenticating you to the SOCKS server.

Variable: socks-username

This is the username to use when authenticating yourself to the SOCKS server. By default this is your login name.

Variable: socks-timeout

This controls how long, in seconds, to wait for responses from the SOCKS server; it is 5 by default.

User Option: socks-nslookup-program

This the ‘nslookup’ program. It is "nslookup" by default.


5.3.1 Suppressing Network Connections

In some circumstances it is desirable to suppress making network connections. A typical case is when rendering HTML in a mail user agent, when external URLs should not be activated, particularly to avoid “bugs” which “call home” by fetch single-pixel images and the like. To arrange this, bind the following variable for the duration of such processing.

Variable: url-gateway-unplugged

If this variable is non-nil new network connections are never opened by the URL library.


5.4 History

The library can maintain a global history list tracking URLs accessed. URL completion can be done from it. The history mechanism is set up automatically via url-do-setup when it is configured to be on. Note that the size of the history list is currently not limited.

The history “list” is actually a hash table, url-history-hash-table. It contains access times keyed by URL strings. The times are in the format returned by current-time.

Function: url-history-update-url url time

This function updates the history table with an entry for url accessed at the given time.

User Option: url-history-track

If non-nil, the library will keep track of all the URLs accessed. If it is t, the list is saved to disk at the end of each Emacs session. The default is nil.

User Option: url-history-file

The file storing the history list between sessions. It defaults to history in url-configuration-directory.

User Option: url-history-save-interval

The number of seconds between automatic saves of the history list. Default is one hour. Note that if you change this variable directly, rather than using Custom, after url-do-setup has been run, you need to run the function url-history-setup-save-timer.

Function: url-history-parse-history &optional fname

Parses the history file fname (default url-history-file) and sets up the history list.

Function: url-history-save-history &optional fname

Saves the current history to file fname (default url-history-file).

Function: url-completion-function string predicate function

You can use this function to do completion of URLs from the history.


6 Customization

The following user options affect the general operation of url library.

User Option: url-configuration-directory

The value of this variable specifies the name of the directory where the url library stores its various configuration files, cache files, etc.

The default value specifies a subdirectory named url/ in the standard Emacs user data directory specified by the variable user-emacs-directory (normally ~/.config/emacs or ~/.emacs.d). However, the old default was ~/.url, and this directory is used instead if it exists.

User Option: url-debug

Specifies the types of debug messages which are logged to the *URL-DEBUG* buffer. t means log all messages. A number means log all messages and show them with message. It may also be a list of the types of messages to be logged.

User Option: url-personal-mail-address
User Option: url-privacy-level
User Option: url-lastloc-privacy-level

Provided lastloc is not prohibited by url-privacy-level, this determines who we send our last location to. none means we include our last location in every outgoing request. domain-match means we send it only if the domain of our last location matches the domain of the URI we are requesting. host-match means we only send our last location back to the same host. The default is domain-match.

Using domain-match for this option requires emacs to make one or more DNS requests each time a new host is contacted, to determine the domain of the host. Results of these lookups are cached, so repeated visits do not require repeated domain lookups.

User Option: url-uncompressor-alist
User Option: url-passwd-entry-func
User Option: url-standalone-mode
User Option: url-bad-port-list
User Option: url-max-password-attempts
User Option: url-show-status
User Option: url-confirmation-func

The function to use for asking yes or no functions. This is normally either y-or-n-p or yes-or-no-p, but could be another function taking a single argument (the prompt) and returning t only if an affirmative answer is given.

User Option: url-gateway-method

A symbol specifying the type of gateway support to use for connections from the local machine. The supported methods are:

telnet

Run telnet in a subprocess to connect;

rlogin

Rlogin to another machine to connect;

socks

Connect through a socks server;

ssl

Connect with SSL;

native

Connect directly.

User Option: url-user-agent

The User Agent string used for sending HTTP/HTTPS requests. The value should be nil, which means that no ‘User-Agent’ header is generated, default, which means that a string is generated based on the setting of url-privacy-level, a string or a function of no arguments that returns a string.

The default is default, which means that the ‘User-Agent: package-name URL/Emacs string will be generated, where package-name is the value of url-package-name and its version, if they are non-nil.


Appendix A GNU Free Documentation License

Version 1.3, 3 November 2008
Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc.
https://fsf.org/

Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
  1. PREAMBLE

    The purpose of this License is to make a manual, textbook, or other functional and useful document free in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

    This License is a kind of “copyleft”, which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

    We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

  2. APPLICABILITY AND DEFINITIONS

    This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The “Document”, below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as “you”. You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.

    A “Modified Version” of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

    A “Secondary Section” is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document’s overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

    The “Invariant Sections” are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.

    The “Cover Texts” are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.

    A “Transparent” copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not “Transparent” is called “Opaque”.

    Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.

    The “Title Page” means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, “Title Page” means the text near the most prominent appearance of the work’s title, preceding the beginning of the body of the text.

    The “publisher” means any person or entity that distributes copies of the Document to the public.

    A section “Entitled XYZ” means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as “Acknowledgements”, “Dedications”, “Endorsements”, or “History”.) To “Preserve the Title” of such a section when you modify the Document means that it remains a section “Entitled XYZ” according to this definition.

    The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.

  3. VERBATIM COPYING

    You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

    You may also lend copies, under the same conditions stated above, and you may publicly display copies.

  4. COPYING IN QUANTITY

    If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document’s license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

    If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

    If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

    It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

  5. MODIFICATIONS

    You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

    1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.
    2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement.
    3. State on the Title page the name of the publisher of the Modified Version, as the publisher.
    4. Preserve all the copyright notices of the Document.
    5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.
    6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.
    7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document’s license notice.
    8. Include an unaltered copy of this License.
    9. Preserve the section Entitled “History”, Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled “History” in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.
    10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the “History” section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.
    11. For any section Entitled “Acknowledgements” or “Dedications”, Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.
    12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.
    13. Delete any section Entitled “Endorsements”. Such a section may not be included in the Modified Version.
    14. Do not retitle any existing section to be Entitled “Endorsements” or to conflict in title with any Invariant Section.
    15. Preserve any Warranty Disclaimers.

    If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version’s license notice. These titles must be distinct from any other section titles.

    You may add a section Entitled “Endorsements”, provided it contains nothing but endorsements of your Modified Version by various parties—for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

    You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

    The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

  6. COMBINING DOCUMENTS

    You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.

    The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

    In the combination, you must combine any sections Entitled “History” in the various original documents, forming one section Entitled “History”; likewise combine any sections Entitled “Acknowledgements”, and any sections Entitled “Dedications”. You must delete all sections Entitled “Endorsements.”

  7. COLLECTIONS OF DOCUMENTS

    You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

    You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

  8. AGGREGATION WITH INDEPENDENT WORKS

    A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an “aggregate” if the copyright resulting from the compilation is not used to limit the legal rights of the compilation’s users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.

    If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document’s Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.

  9. TRANSLATION

    Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.

    If a section in the Document is Entitled “Acknowledgements”, “Dedications”, or “History”, the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.

  10. TERMINATION

    You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute it is void, and will automatically terminate your rights under this License.

    However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.

    Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.

    Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it.

  11. FUTURE REVISIONS OF THIS LICENSE

    The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See https://www.gnu.org/licenses/.

    Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License “or any later version” applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used, that proxy’s public statement of acceptance of a version permanently authorizes you to choose that version for the Document.

  12. RELICENSING

    “Massive Multiauthor Collaboration Site” (or “MMC Site”) means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such a server. A “Massive Multiauthor Collaboration” (or “MMC”) contained in the site means any set of copyrightable works thus published on the MMC site.

    “CC-BY-SA” means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-profit corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization.

    “Incorporate” means to publish or republish a Document, in whole or in part, as part of another Document.

    An MMC is “eligible for relicensing” if it is licensed under this License, and if all works that were first published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008.

    The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing.

ADDENDUM: How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

  Copyright (C)  year  your name.
  Permission is granted to copy, distribute and/or modify this document
  under the terms of the GNU Free Documentation License, Version 1.3
  or any later version published by the Free Software Foundation;
  with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
  Texts.  A copy of the license is included in the section entitled ``GNU
  Free Documentation License''.

If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the “with…Texts.” line with this:

    with the Invariant Sections being list their titles, with
    the Front-Cover Texts being list, and with the Back-Cover Texts
    being list.

If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.


Command and Function Index


Variable Index

Jump to:   H   M   N   S   U  
Index Entry  Section

H
HTTP_PROXY: Proxies

M
mail-user-agent: mailto

N
NNTPSERVER: news/nntp/snews
NO_PROXY: Proxies

S
socks-nslookup-program: Gateways in general
socks-password: Gateways in general
socks-server: Gateways in general
socks-timeout: Gateways in general
socks-username: Gateways in general

U
url-automatic-caching: Disk Caching
url-bad-port-list: Customization
url-cache-creation-function: Disk Caching
url-cache-directory: Disk Caching
url-cache-expire-time: Disk Caching
url-configuration-directory: Customization
url-confirmation-func: Customization
url-cookie-confirmation: Cookies
url-cookie-file: Cookies
url-cookie-multiple-line: Cookies
url-cookie-save-interval: Cookies
url-cookie-trusted-urls: Cookies
url-cookie-untrusted-urls: Cookies
url-debug: Customization
url-gateway-local-host-regexp: Gateways in general
url-gateway-method: Gateways in general
url-gateway-method: Customization
url-gateway-prompt-pattern: Gateways in general
url-gateway-prompt-pattern: Gateways in general
url-gateway-rlogin-host: Gateways in general
url-gateway-rlogin-parameters: Gateways in general
url-gateway-rlogin-user-name: Gateways in general
url-gateway-telnet-host: Gateways in general
url-gateway-telnet-login-prompt: Gateways in general
url-gateway-telnet-parameters: Gateways in general
url-gateway-telnet-password: Gateways in general
url-gateway-telnet-password-prompt: Gateways in general
url-gateway-telnet-user-name: Gateways in general
url-gateway-unplugged: Suppressing network connections
url-history-file: History
url-history-hash-table: History
url-history-save-interval: History
url-history-track: History
url-honor-refresh-requests: http/https
url-irc-function: irc
url-lastloc-privacy-level: Customization
url-mail-command: mailto
url-max-password-attempts: Customization
url-mime-charset-string: HTTP language/coding
url-mime-language-string: HTTP language/coding
url-news-server: news/nntp/snews
url-nfs-automounter-directory-spec: nfs
url-passwd-entry-func: Customization
url-personal-mail-address: Customization
url-privacy-level: Customization
url-proxy-services: Proxies
url-queue-parallel-processes: Retrieving URLs
url-queue-parallel-processes: Retrieving URLs
url-queue-timeout: Retrieving URLs
url-queue-timeout: Retrieving URLs
url-show-status: Customization
url-standalone-mode: Customization
url-tramp-protocols: Tramp
url-uncompressor-alist: Customization
url-unreserved-chars: URI Encoding
url-user-agent: Customization


Concept Index

Jump to:   A   B   C   D   E   F   G   H   I   L   M   N   O   P   R   S   T   U   Z  
Index Entry  Section

A
automounter: nfs

B
bugs, HTML: Suppressing network connections

C
Cache cleaning: Disk Caching
Caching: Disk Caching
character sets: HTTP language/coding
Cleaning the cache: Disk Caching
Clearing the cache: Disk Caching
coding systems: HTTP language/coding
compressed files: file/ftp
configuration files: Customization

D
DASL: HTTP URL Options
data URLs: data
DAV: HTTP URL Options
debugging: Customization
dired: file/ftp
Disk Cache: Disk Caching

E
email: mailto
environment variable: news/nntp/snews
environment variables: Proxies
ERC: irc

F
File Transfer Protocol: file/ftp
files: file/ftp
firewalls: Gateways in general
FTP: file/ftp

G
gateways: Gateways in general

H
HTML “bugs”: Suppressing network connections

I
Info: info
Internet Relay Chat: irc
IRC: irc

L
language preferences: HTTP language/coding
LDAP: ldap
Lightweight Directory Access Protocol: ldap

M
mailto: mailto
man: man
MD5: Disk Caching

N
network connections, suppressing: Suppressing network connections
Network File System: nfs
network news: news/nntp/snews
news: news/nntp/snews
NFS: nfs
NNTP: news/nntp/snews
nslookup: Gateways in general

O
opening a stream: Gateways in general

P
P3P: HTTP URL Options
parsed URI: URI Parsing
parsed URIs: URI Parsing
percent encoding: URI Encoding
Persistent Cache: Disk Caching
proxies: Proxies
proxy servers: Proxies

R
rcirc: irc
rlogin: rlogin/telnet/tn3270
rlogin: Gateways in general
rsync: Tramp

S
scp: Tramp
snews: news/nntp/snews
SOCKS: Gateways in general
ssh: Tramp
stream, opening: Gateways in general
suppressing network connections: Suppressing network connections

T
telnet: rlogin/telnet/tn3270
telnet: Gateways in general
terminal emulation: rlogin/telnet/tn3270
Texinfo: info
tn3270: rlogin/telnet/tn3270

U
uniform resource identifier: Introduction
uniform resource locator: Introduction
Unix man pages: man
unparsing URLs: URI Parsing
unreserved characters: URI Encoding
URI: Introduction
URL: Introduction
usenet: news/nntp/snews

Z
ZEN IRC: irc


Footnotes

(1)

Proxies (which only operate over HTTP) are implemented using this.