| www.cronistalascolonias.com.ar {utils} | R Documentation |
Download File from the Internet
Description
This function can be used to download a file from the Internet.
Usage
www.cronistalascolonias.com.ar(url, destfile, method, quiet = FALSE, mode = "w", cacheOK = TRUE, extra = getOption("www.cronistalascolonias.com.ar"), headers = NULL, )Arguments
a string (or longer vector e.g., for the method) naming the URL of a resource to be downloaded. |
a character string (or vector, see ) with the name where the downloaded file is saved. Tilde-expansion is performed. |
Method to be used for downloading files. Current download methods are , (Windows only) , and , and there is a value : see ‘Details’ and ‘Note’. The method can also be set through the option : see . |
If , suppress status messages (if any), and the progress bar. |
character. The mode with which to write the file. Useful values are , (binary), (append) and . Not used for methods and . See also ‘Details’, notably about using for Windows. |
logical. Is a server-side cached value acceptable? |
character vector of additional command-line arguments for the and methods. |
named character vector of HTTP headers to use in HTTP requests. It is ignored for non-HTTP URLs. The header, coming from the option (see ) is used as the first header, automatically. |
allow additional arguments to be passed, unused. |
Details
The function can be used to download a single file as described by from the internet and store it in . The must start with a scheme such as http://, https://, ftp:// or file://.
If is chosen (the default), the behavior depends on the platform:
On a Unix-alike method is used except for file:// URLs, where uses the library of that name (www.cronistalascolonias.com.ar).
On Windows the method is used apart from for ftps:// URLs where is tried. The method uses the WinINet functions (part of the OS).
Support for method is optional on Windows: use to see if it is supported on your build. It uses an external library of that name (www.cronistalascolonias.com.ar) against which R can be compiled.
When method is used, it provides (non-blocking) access to https:// and (usually) ftps:// URLs. There is support for simultaneous downloads, so and can be character vectors of the same length greater than one (but the method has to be specified explicitly and not via). For a single URL and a progress bar is shown in interactive use.
For methods and a system call is made to the tool given by , and the respective program must be installed on your system and be in the search path for executables. They will block all other activity on the R process until they complete: this may make a GUI unresponsive.
is useful for http:// and https:// URLs: it will attempt to get a copy directly from the site rather than from an intermediate cache. It is used by .
The and methods follow http:// and https:// redirections to any scheme they support: the method follows http:// to http:// redirections only. (For method use argument . To disable redirection in , use .) The method supports some redirections but not all. (For method , messages will quote the endpoint of redirections.)
Note that https:// URLs are not supported by the method but are supported by the method and the method on Windows.
See for how file:// URLs are interpreted, especially on Windows. The and methods do not percent-decode file:// URLs, but the and methods do: method does not support them.
Most methods do not percent-encode special characters such as spaces in URLs (see ), but it seems the method does.
The remaining details apply to the , and methods only.
The timeout for many parts of the transfer can be set by the option which defaults to 60 seconds. This is often insufficient for downloads of large files (50MB or more) and so should be increased when is used in packages to do so. Note that the user can set the default timeout by the environment variable R_DEFAULT_INTERNET_TIMEOUT in recent versions of R, so to ensure that this is not decreased packages should use something like
options(timeout = max(, getOption("timeout")))(It is unrealistic to require download times of less than 1s/MB.)
The level of detail provided during transfer can be set by the argument and the option: the details depend on the platform and scheme. For the method setting option to 0 gives all available details, including all server responses. Using 2 (the default) gives only serious messages, and 3 or more suppresses all messages. For the method values of the option less than 2 give verbose output.
A progress bar tracks the transfer platform specifically:
- On Windows
If the file length is known, the full width of the bar is the known length. Otherwise the initial width represents Kbytes and is doubled whenever the current width is exceeded. (In non-interactive use this uses a text version. If the file length is known, an equals sign represents 2% of the transfer completed: otherwise a dot represents 10Kb.)
- On a unix-alike
If the file length is known, an equals sign represents 2% of the transfer completed: otherwise a dot represents 10Kb.
The choice of binary transfer ( or ) is important on Windows, since unlike Unix-alikes it does distinguish between text and binary files and for text transfers changes line endings to (aka ‘CRLF’).
On Windows, if is not supplied () and ends in one of , , , , , , or , is set such that a binary transfer is done to help unwary users.
Code written to download binary files must use (or ), but the problems incurred by a text transfer will only be seen on Windows.
Value
An (invisible) integer code, for success and non-zero for failure. For the and methods this is the status code returned by the external program. The method can return , but will in most cases throw an error.
What happens to the destination file(s) in the case of error depends on the method and R version. Currently the , and methods will remove the file if there the URL is unavailable except when specifies appending when the file should be unchanged.
Setting Proxies
For the Windows-only method , the ‘Internet Options’ of the system are used to choose proxies and so on; these are set in the Control Panel and are those used for system browsers.
The next two paragraphs apply to the internal code only.
Proxies can be specified via environment variables. Setting no_proxy to stops any proxy being tried. Otherwise the setting of http_proxy or ftp_proxy (or failing that, the all upper-case version) is consulted and if non-empty used as a proxy site. For FTP transfers, the username and password on the proxy can be specified by ftp_proxy_user and ftp_proxy_password. The form of http_proxy should be or where the port defaults to and the trailing slash may be omitted. For ftp_proxy use the form where the default port is . These environment variables must be set before the download code is first used: they cannot be altered later by calling .
Usernames and passwords can be set for HTTP proxy transfers via environment variable http_proxy_user in the form . Alternatively, http_proxy can be of the form for compatibility with . Only the HTTP/ basic authentication scheme is supported.
Under Windows, if http_proxy_user is set to then a dialog box will come up for the user to enter the username and password. NB: you will be given only one opportunity to enter this, but if proxy authentication is required and fails there will be one further prompt per download.
Much the same scheme is supported by , including no_proxy, http_proxy and ftp_proxy, and for the last two a contents of where the parts in brackets are optional. See www.cronistalascolonias.com.ar for details.
Secure URLs
Methods which access https:// and ftps:// URLs should try to verify the site certificates. This is usually done using the CA root certificates installed by the OS (although we have seen instances in which these got removed rather than updated). For further information see www.cronistalascolonias.com.ar
This is an issue for on Windows, where the OS does not provide a suitable CA certificate bundle, so by default on Windows certificates are not verified. To turn verification on, set environment variable CURL_CA_BUNDLE to the path to a certificate bundle file, usually named ‘www.cronistalascolonias.com.ar’ or ‘www.cronistalascolonias.com.ar’. (This is normally done for a binary installation of R, which installs ‘/etc/www.cronistalascolonias.com.ar’ and sets CURL_CA_BUNDLE to point to it if that environment variable is not already set.) For an updated certificate bundle, see www.cronistalascolonias.com.ar Currently one can download a copy from www.cronistalascolonias.com.ar and set CURL_CA_BUNDLE to the full path to the downloaded file.
Note that the root certificates used by R may or may not be the same as used in a browser, and indeed different browsers may use different certificate bundles (there is typically a build option to choose either their own or the system ones).
FTP sites
ftp: URLs are accessed using the FTP protocol which has a number of variants. One distinction is between ‘active’ and ‘(extended) passive’ modes: which is used is chosen by the client. The and methods use passive mode, and that is almost universally used by browsers. The method first tries passive and then active.
Good practice
Setting the should be left to the end user. Neither of the nor commands is widely available: you can check if one is available via, and should do so in a package or script.
If you use in a package or script, you must check the return value, since it is possible that the download will fail with a non-zero status but not an R error.
The supported s do change: method was introduced in R and is still optional on Windows – use in a program to see if it is available.
Note
Files of more than 2GB are supported on bit builds of R; they may be truncated on some bit builds.
Methods and are mainly for historical compatibility but provide may provide capabilities not supported by the or methods.
Method can be used with proxy firewalls which require user/password authentication if proper values are stored in the configuration file for .
(www.cronistalascolonias.com.ar) is commonly installed on Unix-alikes (but not macOS). Windows binaries are available from Cygwin, gnuwin32 and elsewhere.
(www.cronistalascolonias.com.ar) is installed on macOS and commonly on Unix-alikes. Windows binaries are available at that URL.
See Also
to set the , and options used by some of the methods.
for a finer-grained way to read data from URLs.
, , for applications.
Contributed packages RCurl and curl provide more comprehensive facilities to download from URLs.
-
-