FTP: Difference between revisions
m (Minor changes, fixed typos) |
(More grammar corrections, typo fixes, rearrangements, enhancements, and a bit of restructuring. Added new banner to notify that content should be restructured) |
||
Line 1: | Line 1: | ||
{{Style|This page has been rampant with typos and grammar errors before, most of which have been corrected. However, this page could probably be restructured so that the content flows more smoothly, although this is currently being worked on.}} | |||
There are also | FTP (or '''F'''ile '''T'''ransfer '''P'''rotocol) is a protocol that allows you to transfer files from a server to a client and vice versa (as FTP uses a client-server architecture). FTP is among the oldest protocols as its origins can be traced as far back as 1971 according to [https://en.wikipedia.org/wiki/Ftp#History_of_FTP_servers Wikipedia]. | ||
There are also variants of FTP. Such include SFTP (SSH FTP, not to be confused with Simple FTP), FTPS (FTP SSL), and others. SFTP, as the name implies, is done over SSH. FTPS is plain FTP with TLS/SSL encryption. | |||
Alpine Linux has various FTP clients and servers that you can install and use: | Alpine Linux has various FTP clients and servers that you can install and use: | ||
Line 11: | Line 13: | ||
There are also plenty of other clients available too, some not available in Alpine's repositories in fact. However, we will only be covering the clients and servers mentioned above. In addition, there are several graphical clients. See [[#FTP Clients]] for more details. | There are also plenty of other clients available too, some not available in Alpine's repositories in fact. However, we will only be covering the clients and servers mentioned above. In addition, there are several graphical clients. See [[#FTP Clients]] for more details. | ||
= | = Servers = | ||
== vsftpd == | |||
In Alpine Linux the default FTP server is {{Pkg|vsftpd}}. However, it's not widely used due to common hosting panels not handling it's configuration management. In contrast {{Pkg|vsftpd}} is more secure and doesn't require many updates. | |||
== About vsftpd == | |||
vsftpd claims to be the "most secure and fastest FTP server for UNIX-like systems". It's the default FTP server in NimbleX, Slackware, and many others. In addition, it's also recommended because of relatively easy configuration. | |||
{| class="wikitable" | {| class="wikitable" | ||
Line 29: | Line 30: | ||
| Manpages and DOCs packages || vsftpd-doc || <code><nowiki>apk add vsftpd-doc</nowiki></code> | | Manpages and DOCs packages || vsftpd-doc || <code><nowiki>apk add vsftpd-doc</nowiki></code> | ||
|- | |- | ||
| Configuration file || {{Path|/etc/vsftpd/vsftpd.conf}} || | | Configuration file || {{Path|/etc/vsftpd/vsftpd.conf}} || Little to no customizations are made by Alpine | ||
|- | |- | ||
| Default path for files || {{Path|/var/lib/ftp}} || cos | | Default path for files || {{Path|/var/lib/ftp}} || cos anonymous are default, is home for daemon user | ||
|- | |- | ||
| Log files (error, access, etc) || {{Path|/var/log/vsftpd.log}} || Defined in vsftp.conf | | Log files (error, access, etc) || {{Path|/var/log/vsftpd.log}} || Defined in vsftp.conf | ||
|- | |- | ||
| User running the service || vsftpd || Others Linux used "ftp" alpine has as a group | | User running the service || vsftpd || Others Linux used "ftp" alpine has as a group | ||
|- | |- | ||
| Group to common to || vsftpd || Used to share things among others daemons or services, like Redis or Apache | | Group to common to || vsftpd || Used to share things among others daemons or services, like Redis or Apache | ||
|- | |- | ||
| Written in || C || Main engine code in C | | Written in || C || Main engine code in C | ||
|} | |} | ||
==== | ==== Limitations ==== | ||
vsftp does not support SFTP, but it does have support for FTPS. | * vsftp does not support SFTP, but it does have support for FTPS. | ||
=== | === Installing vsftpd === | ||
The installation does not work out of the box, cos enable by default anonymous access is enabled and it only works with IPv4 only. | The installation does not work out of the box, cos enable by default anonymous access is enabled and it only works with IPv4 only. | ||
Line 64: | Line 65: | ||
</pre> | </pre> | ||
=== vsftpd | === Configuring vsftpd === | ||
By default, Alpine's vsftpd package does not | By default, Alpine's vsftpd package does not modify the default configuration. If you leave the configuration untouched, then vsftpd will be configured exactly the same as if you downloaded vsftpd from [[https://security.appspot.com/vsftpd.html its site]]. Regardless, you might want to change the server configuration anyways. | ||
The server configuration obviously depends on the your needs. Some common server configuration schemes include allowing anonymous access of files, user system FTP services, and virtual users (on the server end). | |||
For production purposes the [[Production FTP virtual users: vsftpd + MySQL]] wiki page will explain | For production purposes, the [[Production FTP virtual users: vsftpd + MySQL]] wiki page will explain all the requirements for setting up virtual users. The [[Production FTP user service: vsftpd]] wiki page also has further information on how to use vsftpd with system users and Let's Encrypt certificates ['''fixme''': pages doesn't exist]. | ||
Because there are no customizations to the default configuration (in other words no Alpine-specific customizations), the configuration may not mean easy maintenance of the server, among other things. See the [[Production VSFTP system: vsftpd + MySQL + SFTP]] wiki page, which goes in depth about configuring the lighttpd web server and others ['''fixme''': page does not exist]. | |||
==== Controlling vsftpd ==== | ==== Controlling vsftpd ==== | ||
''''' | '''''Starting vsftpd''''': After the installation {{Pkg|vsftpd}} is not running. As we said in first section, was started already but if you want to start {{Pkg|vsftpd}} manually use: | ||
{{Cmd|rc-service vsftpd start}} | {{Cmd|rc-service vsftpd start}} | ||
After you issue this command, OpenRC will tell you the status of vsftpd. | |||
<pre> | <pre> | ||
Line 87: | Line 88: | ||
</pre> | </pre> | ||
''' | '''Stopping vsftpd''': If you want to stop the web server use ''stop'' in the same way of previous command: | ||
{{Cmd|rc-service vsftpd stop}} | {{Cmd|rc-service vsftpd stop}} | ||
''' | '''Restarting or reloading vsftpd''': After changing the configuration file, you must either restart or reload vsftpd in order for the new configuration to take effect. You can issue one of the following commands below to achieve the desired effect: | ||
{{Cmd|rc-service vsftpd restart}} | {{Cmd|rc-service vsftpd restart}} | ||
{{Cmd|rc-service vsftpd reload}} | |||
'''' | The latter command entirely restarts vsftpd, while the former only reloads vsftpd's configuration file. The former command might be more desired as it prevents having to reload vsftpd entirely. | ||
After issuing one of the commands, you might see something similar either to this output: | |||
<pre> | |||
* Stopping vsftpd... [ ok ] | |||
* Starting vsftpd... [ ok ] | |||
</pre> | |||
(for restarting vsftpd) or to this output: | |||
<pre> | |||
* Reloading vsftpd... [ ok ] | |||
</pre> | |||
(for reloading vsftpd). | |||
'''Using the proper runlevel''': By default no services are added to start process. You must know what you want and what will these services do. Furthermore, runlevels aren't present in Docker containers (per se), and Alpine is used mostly in those containers. You must add the service only to the 'default' runlevel, not to the 'boot' runlevel. Networking isn't enabled during the the 'boot' runlevel, and thus it doesn't make much sense to run an FTP server without networking. | |||
{{Cmd|rc-update add vsftpd default}} | {{Cmd|rc-update add vsftpd default}} | ||
Line 102: | Line 120: | ||
=== Testing vsftpd === | === Testing vsftpd === | ||
This section is assuming that vsftpd is running. | This section is assuming that vsftpd is running. At the moment, there might not be a user 'ftp' allowed available but the server is up and running. | ||
{{Cmd|nc -zv server_address}} | {{Cmd|nc -zv server_address}} | ||
Line 119: | Line 137: | ||
= FTP Clients = | = FTP Clients = | ||
As FTP uses a client-server architecture, you will need an FTP client if you want to interact with an FTP server. Alpine | As FTP uses a client-server architecture, you will need an FTP client if you want to interact with an FTP server. Alpine provides two packages for command line and batch processing of files over FTP. The following is a list of some graphical FTP clients: | ||
* {{Pkg|pcmanfm}}. Check out [[#PCManFM with GVfs]] | |||
* {{Pkg|firefox}}, before [https://www.mozilla.org/en-US/firefox/88.0/releasenotes/ Firefox 88], or any other web browser supporting FTP. | |||
The following is a list of some command line FTP clients: | |||
* {{Pkg|lftp}}. Check out [[#lftp]] | |||
* {{Pkg|ncftp}} | |||
You can also use any web browser that supports FTP as an FTP client also, as is obvious. To do so, simply enter <nowiki><code>ftp://server_address</code></nowiki>, where "server_address" is the IP address of the FTP server. | You can also use any web browser that supports FTP as an FTP client also, as is obvious. To do so, simply enter <nowiki><code>ftp://server_address</code></nowiki>, where "server_address" is the IP address of the FTP server. | ||
Line 132: | Line 150: | ||
== lftp == | == lftp == | ||
LFTP is a sophisticated file transfer program that supports a number of network protocols. Such include FTP, HTTP, SFTP, FISH, | LFTP is a sophisticated file transfer program that supports a number of network protocols. Such include FTP, HTTP, SFTP, FISH, BitTorrent, and possibly more. | ||
lftp has the following features: | lftp has the following features: | ||
* | * lftp supports job control | ||
* | * The readline library is used for input | ||
* | * lftp has support for bookmarks | ||
* | * lftp has a built-in mirror command | ||
* | * lftp can transfer several files in parallel | ||
{| class="wikitable" | {| class="wikitable" | ||
Line 150: | Line 168: | ||
| Manpages and DOCs packages || vsftpd-doc || Install: <code><nowiki>apk add lftp-doc</nowiki></code> | | Manpages and DOCs packages || vsftpd-doc || Install: <code><nowiki>apk add lftp-doc</nowiki></code> | ||
|- | |- | ||
| Configuration file (system) || {{Path|/etc/lftp.conf}} || | | Configuration file (system) || {{Path|/etc/lftp.conf}} || Little to no customizations are made by Alpine | ||
|- | |- | ||
| Configuration file (user) || {{Path|~/.lftprc}} || The first user configuration file to read from. | | Configuration file (user) || {{Path|~/.lftprc}} || The first user configuration file to read from. | ||
|- | |- | ||
| Configuration file (user) || {{Path|~/.lftp/rc}} || Second user configuration file to read from | | Configuration file (user) || {{Path|~/.lftp/rc}} || Second user configuration file to read from. | ||
|- | |- | ||
| Written in || C || Main engine code in C | | Written in || C || Main engine code in C | ||
Line 173: | Line 191: | ||
=== Configuring lftp === | === Configuring lftp === | ||
The main configuration file, <code><nowiki>/etc/lftp.conf</nowiki></code> comes with | The main configuration file, <code><nowiki>/etc/lftp.conf</nowiki></code> comes with different examples and comments. On startup, lftp reads <code>/etc/lftp.conf</code>, <code>~/.lftprc</code>, and <code>~/.lftp/rc</code> (in the specified order). You can use those files to set system-wide and user-specific settings. | ||
In your configuration file, simply use the command set followed by the name of the setting, followed by its value. Use on/off for boolean values. | In your configuration file, simply use the command set followed by the name of the setting, followed by its value. Use on/off for boolean values. Here's an example: | ||
<pre> | <pre> | ||
Line 195: | Line 213: | ||
</pre> | </pre> | ||
{{Note|The closure for `dns:', `net:', `ftp:', `http:', `hftp:' domain variables is currently just the host name as you specify it in the `open' command (with some exceptions where closure is meaningless, e.g. dns:cache-size). For some `cmd:' domain variables the closure is current URL without path. For other variables it is not currently used. See examples in the sample lftp.conf.}} | {{Note|The closure for `dns:', `net:', `ftp:', `http:', `hftp:' domain variables is currently just the host name as you specify it in the `open' command (with some exceptions where closure is meaningless, e.g. dns:cache-size). For some `cmd:' domain variables the closure is the current URL without path. For other variables it is not currently used. See examples in the sample lftp.conf.}} | ||
=== | === More on lftp === | ||
By default, lftp is | By default, lftp is not very verbose about operations it performs in the background. If you want more output, e.g. for debugging purposes, you can use the 'debug' command (without the 'set' command): | ||
* debug 5 will display the full debug output. | * debug 5 will display the full debug output. | ||
Line 221: | Line 239: | ||
The last alias (edit) should be all in one line. It allows you to get a remote file, edit it with vim (change as needed), and place the edited file back on the server. This is very convenient! | The last alias (edit) should be all in one line. It allows you to get a remote file, edit it with vim (change as needed), and place the edited file back on the server. This is very convenient! | ||
=== Example usage === | |||
Below is an example of a mult-part download over the SFTP protocol: | Below is an example of a mult-part download over the SFTP protocol: | ||
< | <code>$ lftp -e 'pget -c -n 5 /path/to/file' <nowiki>sftp://username@server</nowiki></code> | ||
Here's a breakdown of the above command: | Here's a breakdown of the above command: | ||
Line 240: | Line 258: | ||
# From the menubar open the "go" menu | # From the menubar open the "go" menu | ||
# Choose "Connect to a server" | # Choose "Connect to a server" | ||
# Choose the FTP protocol and input username (path is optional) | # Choose the FTP protocol and input your username (path is optional) | ||
# After a while you will input the password | # After a while you will input the password for the username you input | ||
Check the picture for further reference | Check the picture for further reference. | ||
[[File:Ftp-gui-pcmanfm_at_2020-09-11_14-13-03.png]] | [[File:Ftp-gui-pcmanfm_at_2020-09-11_14-13-03.png]] | ||
Line 257: | Line 275: | ||
[[Category:Web_Server]] | [[Category:Web_Server]] | ||
[[Category:Development]] | [[Category:Development]] | ||
ddd |
Revision as of 02:20, 13 May 2021
This material needs wiki syntax or style improvements ... This page has been rampant with typos and grammar errors before, most of which have been corrected. However, this page could probably be restructured so that the content flows more smoothly, although this is currently being worked on. |
FTP (or File Transfer Protocol) is a protocol that allows you to transfer files from a server to a client and vice versa (as FTP uses a client-server architecture). FTP is among the oldest protocols as its origins can be traced as far back as 1971 according to Wikipedia.
There are also variants of FTP. Such include SFTP (SSH FTP, not to be confused with Simple FTP), FTPS (FTP SSL), and others. SFTP, as the name implies, is done over SSH. FTPS is plain FTP with TLS/SSL encryption.
Alpine Linux has various FTP clients and servers that you can install and use:
There are also plenty of other clients available too, some not available in Alpine's repositories in fact. However, we will only be covering the clients and servers mentioned above. In addition, there are several graphical clients. See #FTP Clients for more details.
Servers
vsftpd
In Alpine Linux the default FTP server is vsftpd. However, it's not widely used due to common hosting panels not handling it's configuration management. In contrast vsftpd is more secure and doesn't require many updates.
About vsftpd
vsftpd claims to be the "most secure and fastest FTP server for UNIX-like systems". It's the default FTP server in NimbleX, Slackware, and many others. In addition, it's also recommended because of relatively easy configuration.
Feature | Value/Name | Observations |
---|---|---|
Package name | vsftpd | apk add vsftpd
|
Manpages and DOCs packages | vsftpd-doc | apk add vsftpd-doc
|
Configuration file | /etc/vsftpd/vsftpd.conf | Little to no customizations are made by Alpine |
Default path for files | /var/lib/ftp | cos anonymous are default, is home for daemon user |
Log files (error, access, etc) | /var/log/vsftpd.log | Defined in vsftp.conf |
User running the service | vsftpd | Others Linux used "ftp" alpine has as a group |
Group to common to | vsftpd | Used to share things among others daemons or services, like Redis or Apache |
Written in | C | Main engine code in C |
Limitations
- vsftp does not support SFTP, but it does have support for FTPS.
Installing vsftpd
The installation does not work out of the box, cos enable by default anonymous access is enabled and it only works with IPv4 only.
Per user ftp files can be supported by special directive in the vsftp.conf file using user_sub_token
to something like /home/$USER/public_ftp directory by example if we enable it (process are described below in further section "vsftpd configuration".
vsftpd is available in the Alpine Linux repositories. To install and run vsftpd, simply run the commands below:
apk add vsftpd rc-update add vsftpd default rc-service vsftpd restart
Configuring vsftpd
By default, Alpine's vsftpd package does not modify the default configuration. If you leave the configuration untouched, then vsftpd will be configured exactly the same as if you downloaded vsftpd from [its site]. Regardless, you might want to change the server configuration anyways.
The server configuration obviously depends on the your needs. Some common server configuration schemes include allowing anonymous access of files, user system FTP services, and virtual users (on the server end).
For production purposes, the Production FTP virtual users: vsftpd + MySQL wiki page will explain all the requirements for setting up virtual users. The Production FTP user service: vsftpd wiki page also has further information on how to use vsftpd with system users and Let's Encrypt certificates [fixme: pages doesn't exist].
Because there are no customizations to the default configuration (in other words no Alpine-specific customizations), the configuration may not mean easy maintenance of the server, among other things. See the Production VSFTP system: vsftpd + MySQL + SFTP wiki page, which goes in depth about configuring the lighttpd web server and others [fixme: page does not exist].
Controlling vsftpd
Starting vsftpd: After the installation vsftpd is not running. As we said in first section, was started already but if you want to start vsftpd manually use:
rc-service vsftpd start
After you issue this command, OpenRC will tell you the status of vsftpd.
* Caching service dependencies ... [ ok ] * Starting vsftpd... [ ok ]
Stopping vsftpd: If you want to stop the web server use stop in the same way of previous command:
rc-service vsftpd stop
Restarting or reloading vsftpd: After changing the configuration file, you must either restart or reload vsftpd in order for the new configuration to take effect. You can issue one of the following commands below to achieve the desired effect:
rc-service vsftpd restart
rc-service vsftpd reload
The latter command entirely restarts vsftpd, while the former only reloads vsftpd's configuration file. The former command might be more desired as it prevents having to reload vsftpd entirely.
After issuing one of the commands, you might see something similar either to this output:
* Stopping vsftpd... [ ok ] * Starting vsftpd... [ ok ]
(for restarting vsftpd) or to this output:
* Reloading vsftpd... [ ok ]
(for reloading vsftpd).
Using the proper runlevel: By default no services are added to start process. You must know what you want and what will these services do. Furthermore, runlevels aren't present in Docker containers (per se), and Alpine is used mostly in those containers. You must add the service only to the 'default' runlevel, not to the 'boot' runlevel. Networking isn't enabled during the the 'boot' runlevel, and thus it doesn't make much sense to run an FTP server without networking.
rc-update add vsftpd default
Testing vsftpd
This section is assuming that vsftpd is running. At the moment, there might not be a user 'ftp' allowed available but the server is up and running.
nc -zv server_address
Be sure to change server_address
with the actual IP address of the server. The common response will be OPEN if the server is running.
See also about vsftpd
- Production Lets Encrypt: dehydrated
- Production FTP user service: vsftpd
- Production FTP virtual users: vsftpd + MySQL
- Production VSFTP system: vsftpd + MySQL + SFTP
- Alpine newbie developer
- Alpine newbie lammers
FTP Clients
As FTP uses a client-server architecture, you will need an FTP client if you want to interact with an FTP server. Alpine provides two packages for command line and batch processing of files over FTP. The following is a list of some graphical FTP clients:
- pcmanfm. Check out #PCManFM with GVfs
- firefox, before Firefox 88, or any other web browser supporting FTP.
The following is a list of some command line FTP clients:
You can also use any web browser that supports FTP as an FTP client also, as is obvious. To do so, simply enter <code>ftp://server_address</code>, where "server_address" is the IP address of the FTP server.
lftp
LFTP is a sophisticated file transfer program that supports a number of network protocols. Such include FTP, HTTP, SFTP, FISH, BitTorrent, and possibly more.
lftp has the following features:
- lftp supports job control
- The readline library is used for input
- lftp has support for bookmarks
- lftp has a built-in mirror command
- lftp can transfer several files in parallel
Feature | Value/Name | Notes |
---|---|---|
Main package name | lftp | Install: apk add lftp
|
Manpages and DOCs packages | vsftpd-doc | Install: apk add lftp-doc
|
Configuration file (system) | /etc/lftp.conf | Little to no customizations are made by Alpine |
Configuration file (user) | ~/.lftprc | The first user configuration file to read from. |
Configuration file (user) | ~/.lftp/rc | Second user configuration file to read from. |
Written in | C | Main engine code in C |
Installing lftp
To install lftp, run the following command:
apk add lftp
lftp has been provided in the main repository since Alpine 3.6, so it's a supported package that's ready for production use.
Configuring lftp
The main configuration file, /etc/lftp.conf
comes with different examples and comments. On startup, lftp reads /etc/lftp.conf
, ~/.lftprc
, and ~/.lftp/rc
(in the specified order). You can use those files to set system-wide and user-specific settings.
In your configuration file, simply use the command set followed by the name of the setting, followed by its value. Use on/off for boolean values. Here's an example:
set ftp:ssl-force on set ssl:verify-certificate on set ftp:use-feat off set ftp:ssl-protect-data on
You can set specific settings for specific servers only. Simply append a slash (/) and the server name to the setting.
set ftp:use-feat off /example.com set ftp:ssl-force on /example.com
More on lftp
By default, lftp is not very verbose about operations it performs in the background. If you want more output, e.g. for debugging purposes, you can use the 'debug' command (without the 'set' command):
- debug 5 will display the full debug output.
- debug 3 will only display greeting messages and error messages.
Note that if you use the -d switch from the command line, the debug setting will be overruled and the full debug output will be displayed.
If you prefer some commands to be called by another name, you can set aliases:
alias dir ls alias less more alias zless zmore alias bzless bzmore alias reconnect "close; cache flush; cd ." alias edit "eval -f "get $0 -o ~/.lftp/edit.tmp.$$ && shell \\"cp -p ~/.lftp/edit.tmp.$$ ~/.lftp/edit.tmp.$$.orig && vim ~/.lftp/edit.tmp.$$ && test ~/.lftp/edit.tmp.$$ -nt ~/.lftp/edit.tmp.$$.orig\\" && put ~/.lftp/edit.tmp.$$ -o $0; shell rm -f ~/.lftp/edit.tmp.$$*"
The last alias (edit) should be all in one line. It allows you to get a remote file, edit it with vim (change as needed), and place the edited file back on the server. This is very convenient!
Example usage
Below is an example of a mult-part download over the SFTP protocol:
$ lftp -e 'pget -c -n 5 /path/to/file' sftp://username@server
Here's a breakdown of the above command:
- -e: lftp option to execute command
- pget: is the command for partial download
- -c: pget option to resume
- -n: pget option for number of parts
PCManFM with GVfs
While PCManFM is a file manager, installing the gvfs plugin allows you to use it as a graphical FTP client. To connect to an FTP server, you can do the following:
- From the menubar open the "go" menu
- Choose "Connect to a server"
- Choose the FTP protocol and input your username (path is optional)
- After a while you will input the password for the username you input
Check the picture for further reference.
See Also
- Alpine newbie developer
- Alpine newbie lammers
- https://pkgs.alpinelinux.org/packages?name=tftp-hpa related to PXE boot
ddd