INSTALLATION ============ You just received a file named "peer.tar.gz" from "vito", your godfather in the peer system. To join the family, here is how to proceed. Let's assume you are "johnny" and that you have an account named "johnny" on the computer. Ensure that you have the following mandatory tools available on the computer: ssh ssh-keygen ssh-agent ssh-add git iconv awk The following tools are not mandatory, but strongly recommended: gettext, rsync, nc, sendmail, cstream, gzip, gpg, sha256sum, base64 Create a new user account (called "peer" in the following), dedicated to the usage of the peer system. Ensure that user "peer" has permission to connect to this computer from the Internet using ssh. Login as user "peer" on the computer. As user "peer", in $HOME, unpack the distribution archive: $ tar zxf peer.tar.gz It unpacks the "peer" script, a "vito" key file and a hidden ".locale" directory containing message translations. Initialize the system by running: $ ./peer init This will fail if a ~/.ssh or a ~/.share directory exist. If they exist from a previous attempt, you need to remove them manually. If you have never "peer init" before, the script will ask you for some parameter values: Peer ID: this must be unique in the whole network of peers, let's call it "johnny". Peer username: we chose username "peer" in this example. Host address: the public DNS name or IP address of the computer. SSH server port: by default, it is 22. E-mail address: the e-mail address where you want to receive e-mail notifications from the peer system, for example "johnny" for the local user "johnny", or "johnny@example.com". Character set: the name of the character set that is used for file names on the computer. Default download directory: the directory where to put the downloaded files. The "init" command also produces your key file, named "johnny". Transmit this file to "vito" and ask him to add "johnny" to his system. Add the "vito" key. This allows "vito" to connect to your computer. $ ./peer add If "vito" has not yet added "johnny"'s key file in his system, you will get an error message. This is normal. Wait until "vito" has added your key file to his system. When "vito" has done his part, you will receive an e-mail telling you about the files that have been added to the system, if any. Congratulations, you are now a member of the family! A note on e-mail ---------------- If the computer can handle local e-mails, then everything is fine. If it does not handle local e-mail but can send e-mails to the Internet on the condition that the supplied sender's address is valid, you need to add the two following variables to the ~/.peerrc configuration file: - SENDER_ADDRESS which is set to a valid sender's e-mail address - SENDER_NAME which contains the sender's real name (used by the mail server to fill the "From: " field). SHARING FILES ============= The files you want to share must be placed in ~/files (this directory is created during installation). It is possible to place just symbolic links into this directory, while the files the refer to are actually in a different directory. If you want the system to ignore some of the files located in ~/files, you can write into ~/.peerignore regular expression corresponding to the path of these files, relatively to ~/files. E.g., to ignore all the files located in ~/files/A/B/C you need to add the line A/B/C/.* to the ~/.peerignore file (which you may need to create). USING PEER ========== Here are the peer commands that can be used. They are all parameters to the "peer" command, so for example the "help" command must be run as "peer help". help : the list of commands readme : displays the readme init : initializes a blank account to be used in the peer system add : add the file to your network of peers. It will be identified as in the system. del : remove from the network. list : list all the peers and their SSH address. links : list all the available files as rsync/scp URLs. sync : get updates from all the other peers. diff : print the local differences with the previous push. push : tell all the other peers to get your updates. agent start : unlock your ssh key to allow others to get your updates (place the ssh key in the ssh agent). agent stop : stop the ssh agent. get : download (rsync URL) into directory . q [] : enqueue and start downloading in the background. getlist ...: enqueue / from and start dowloading in the background. qstart : start downloading the queue in the background. qstop : stop downloading the queue. qedit : edit the queue. wstart [p h] : start the internal web server (port p, host h). Its URL is printed out on startup. Defaults to port 8080 on host 127.0.0.1. wstop : stop the internal web server. msg : send a message to . msgall : send a message to all the peers. newkey : create a new ssh key. rename : rename this peer as . passwd : change the ssh key's password. test [opt]: test the connection to and pass the [opt] options to ssh. distribute [opt] : distribute the script's files. If opt is an e-mail address, sends it by e-mail. If opt is '-', writes to the standard output. If there is no opt, writes to ~/peer.tar.gz. update_host_key : update this peer's key file from the USERNAME, HOSTNAME and PORT values in ~/.peerrc and the host's public SSH key (use it when connection information or the host's SSH key have changed). Ssh Passphrase -------------- The ssh key is normally stored in the ssh agent. If you reboot the computer, the key will not find its way magically into the agent; you will have to call "peer agent start" to put it there and allow other peers to push updates towards you. Automatic Updates ----------------- When "peer init" is run, the "peer" script is renamed "peer.old" and replaced with a symbolic link to ~/.share/peer/peer". This allows the "peer" command to always execute the latest version of the script; the latter is automatically updated. DOWNLOADING FILES ================= Single files ------------ You can get a list of all the files available at the other peers with "peer links". This commands lists all the files as rsync URLs, which can be directly used as parameter to the "peer get" command. Since the output of the "links" command is text, it can be processed with the usual tools (e.g., grep). Only the files are listed, not the directories. The number between parentheses after the rsync URL is its size in bytes. Once you have chosen a file, copy its rsync URL (the one from the output of "peer links"). The URL is also a valid scp URL, so if rsync is not available, you can download the file by running $ scp If rsync is available, you can pass the URL as the first parameter to "peer get". Enclose it in double quotes in case it contains spaces or other dubious characters. Pass the destination directory as second parameter of "peer get" e.g., $ peer get "vito:dire/cto/ry/fil/ena/me" incoming where "vito:dire/cto/ry/fil/ena/me" is the rsync URL and "incoming" is the destination directory. Rsync will then be called and download the file. You can pass additional parameters to rsync before the URL. For example, if you want to recursively download the directory "vito:dire/cto/ry", you can use $ peer get -r "vito:dire/cto/ry" incoming Commonly used options can be placed in the RSYNC_OPTS variable in the ~/.peerrc file. Using the download queue ------------------------ Instead of using "peer get", you can use "peer q" to add an rsync URL and its corresponding destination directory to the download queue, and let peer download the files in the background: $ peer q "vito:dire/cto/ry/fil/ena/me" incoming Note that passing options to rsync is not possible with this method. If options are needed, they must be placed in the RSYNC_OPTS variable in the ~/.peerrc file. You can control the download queue with the "qstart"/"qstop" commands. It is OK to use "qstop" while a download is in progress. It will resumed the next time you call "qstart". If for some reason you need to edit the download queue, you can use the "qedit" command. Using the Web interface ----------------------- If you find too tiresome to use the command-line interface, you can start the internal web server instead, using $ peer wstart The command displays the URL at which the interface is found, by default it is http://localhost:8080/ . Point your Web browser to this URL and there you are. You can specify a different address with the "HTTP_HOST" variable in the ~/.peerrc configuration file. The same applies to the port number, which can be specified in the "HTTP_PORT" variable in the same configuration file. On the top of the page, there is a list of commands corresponding to the command-line commands with the same name (except "EXIT" which calls "wstop"). Below, the files are listed as tree, representing the corresponding filesystem hierarchy. If Javascript is available, the list is folded and parts of it can be unfolded by clicking the "+" symbol next to the directory names. Re-folding is made by clicking the "-" symbol that replaced the "+" on unfolding. Every file and directory is actually a link. Clicking on the list places it into the download queue, and downloading immediately starts in the background. The size of each file and directory is displayed in small fonts next to its name. In the case of French comic books (i.e., files that have a "BD" parent directory), a link represented as a question mark in square brackets ([?]) starts a search for the comic book (or series of comic books) on http://www.bedetheque.com/ . SHARING FILES ============= Sharing files does not require any particular operation. If you want to limit the upload rate of an incoming connection (scp or rsync), you can indicate the maximum rate with the "THROTTLE" variable in the ~/.peerrc configuration file. Rate limitation requires the "cstream" command, which therefore needs to be installed on the system. For the sake of simplicity, rate limtiation is applied independently to each connection; it is therefore not possible to specify a global rate limit that would apply simultaneously to all incoming connections. USING PEER FROM A NORMAL USER ============================= If you want to use the command-line tool without having to login as "peer", here are a few tips. Put the SSH key from the "johnny" user into "peer"'s ~/.ssh/authorized_keys.default file. Test that you can connect to the computer as user "peer". Add the following function to the ~/.bashrc of "johnny": function peer() { [[ "$1" =~ qedit|msg|sync|push|test ]] && local emulate_tty="-t" ssh -q $emulate_tty peer@localhost .share/peer/peer "$@" } Reload the ~/.bashrc by running $ source ~/.bashrc From now on you can run all the peer commands as user "johnny". Simply remember that relative paths will be based on "peer"'s home directory, not the current directory. You can also make the incoming directory so that it belongs to user "johnny" but is accessible to group "peer". This way, by making "johnny" belong to the group "peer", "johnny" can manipulate the downloaded files. BECOMING THE GODFATHER ====================== If you want to invite a new user into the peer network, you need to become this user's godfather and send him the peer script and your key. You can generate a .tar.gz archive that contains all that, plus this README file by running $ peer distribute This command creates the peer.tar.gz file in "peer"'s home directory. You can then transmit it to your godson. If you are running the peer command from another user, as described in the previous section, you can instead run $ peer distribute - > peer.tar.gz which writes the content of the tar.gz archive to stdout, and you can then redirect it to a file located in any convenient location. Finally, if your computer can send e-mails to the Internet, you can supply an e-mail address to the "distribute" command, as in $ peer distribute godson@example.com and the script will send the e-mail by itself to godson@example.com . At some point, your godson will send you his key file, that you will then have to add to your network with the "peer add" command. BELOW THE HOOD ============== The goal of the peer system is to facilitate ssh connections between computers belonging to the same nework. Each node in the network has a git repository which contains the public RSA keys of each of the peers and listings of the files that are available on each peer. Additionally, the repository is also used for storing the latest version of the peer script. Using git allows any member of the network to update a file in the repository (typically the listing of its files, after adding or removing shared files) and then to "push" these changes to the other peers (this is actually implemented by making a "git push" to a special branch in each remote peer in turn, and then using the "post-update" hook on those peers to run "git merge" on that branch, merging it into the "master" branch). The counterpart "sync" operation, where one node calls "git pull" on every other peer, is also possible. RSA keys being protected by a passphrase, it is necessary to place the key in the node's ssh agent to allow automatic connection from this node to another node (but this is necessary only for "peer push" operations). As a corollary, this means that every member of the network can connect with ssh to any other computer in that network. SECURING THE SYSTEM =================== At SSH access level ------------------- To prevent a compromised computer to be able to connect to every other member of the network and thus use their computers to launch attacks to the Internet, the commands an SSH client connecting to your computer can run are limited: the ~/.ssh/authorized_keys file forces the use of "peer -D" for incoming connections. The latter command verifies which command should be run through the SSH connection and if it is not one of the commands that are needed by peer, the connection will be closed immediately. At code integrity level ----------------------- The script and associated files come with a list of SHA256 checksums ($HOME/.share/peer/sha256sums), the list itself being signed using the GnuPG key 88BB DC17 2A81 1C8D (the signature is in $HOME/.share/peer/sha256sums.asc). This verification is made by the ~/.share/.git/hooks/post-merge script (which is independent from the peer script). If the signature or the checksums are not verified, this script removes all permissions on $HOME/.share, thus making the peer user account inaccessible through SSH. The ~/.share/.git/hooks/post-merge script is generated during the installation of peer and is not modified afterwards. To update this script, you have to run $HOME/.share/peer/peer make_git_post_merge_hook as user peer.