Duality BitTorrent Client

The social BitTorrent client that allows peers in close geographical proximity aiming for the same content to share files effectively.

Is it a new way of sharing files? We believe so.
The Duality project was also discussed on Hacker News. Come, try it out!

The Idea_{What  Why}     |    Installation_{Linux  Windows}    |    Manual_{Using Merging}    |    Code    |    Report Bugs

The Idea

I. What is it?

In case you don't like reading text, here is a visual explanation of what Duality is and how it works.

The client is based on the idea presented in the paper.

Duality is our solution to a commonly observed problem that when peers in close geographical proximity (meaning they can exchange data over sneakernet) are aiming for the same file, either:

- a single peer downloads it and shares it within the group, or
- each peer in the group downloads its own copy of the file.

Both these approaches are inefficient:

- if a single peer is downloading the given file, the entire download process from initiation to completion is dependent upon it.
- if all the peers in the group are downloading their own copy individually, it leads to redundancy.

This is where Duality comes in. We propose that the file is split into parts, with each peer in the group downloading a part of the file. After each peer has downloaded its part, the individual parts are collected on any peer machine and are merged to form the complete content.

Conventional Approach:

The peer with the fastest internet connection downloads the file for the peer group.
    -    The other peers are dependent upon this peer for the download and are mere spectators.

or

All the peers download their own copy of the content.
    -    This leads to redundancy. Why should all peers download the same file individually?

Duality Approach:

The peer group decides among themselves that each peer will download a certain part of the file (decided amongst the group).
Each peer starts the download process and downloads its own part of the file.
When the parts have finished downloading, the peers bring their respective parts to any peer machine and the parts are merged.

II. What makes it better?

- The download is no longer dependent on a single peer as each peer in the group contributes.
- The speed of the download is drastically increased -- higher the number of peers, higher the download speed.
- Duality reduces redundancy and encourages equal participation. The peers which were spectators in the conventional approach can now participate in the download process and contribute their bandwidth.
- The download time is decreased as each peer is downloading a part of the file.
- Each peer in the group, aiming for the same file, contributes to the download process.

III. Why should I use it?

When you and your friends want to download a common file over BitTorrent that all of you want, Duality helps everyone in the group participate in the download process. Download the individual parts and when you are done, merge them to get the complete file.

1. Each peer downloads the torrent file individually on their respective computers.
2. Duality asks each peer what part of the file it would like to download.
The peer group can decide this among themselves, based on factors such as bandwidth, uptime etc.
3. Peers starts the download process; effectively, each peer is downloading a part of the file.
4. The peers then bring the parts over to any machine (using secondary storage devices such as USB flash drives) and the parts are merged.

[Top]

Installation

Requirements:

Python 2.6+        (2.7 is also supported)
libtorrent

We assume that you have Python installed on your system. If not, you can download the installer from python.org.

Then get the Duality source from github.com/sukhbir.
You can either clone the repository (using git: git clone git://github.com/sukhbir/duality.git) or download the tarball.

The following steps will help you to install libtorrent on Linux and Windows.

Linux

The libtorrent library and the Python bindings can be installed on Debian/ Ubuntu by installing the requisite packages:

   $ sudo apt-get install libtorrent-rasterbar6 python-libtorrent

This will install the library, the binding and take care of the dependencies.

Compiling from source is recommended because this way you will have the latest version of libtorrent.

1. Grab the latest release of libtorrent (tarball).
    You can also check out development branches from the SVN repository.

2. Extract the files to a directory and then run:
    
   $ ./configure --enable-python-binding

    Followed by:

   $ make

    And then finally:

   $ sudo make install

Now install the bindings:

   $ sudo python bindings/python/setup.py install

You will of course have to meet the required dependencies.

Windows

Duality runs on Windows also, but getting it to work involves a little extra effort because of the installation of libtorrent dependencies..

1. Download Python 2.6.6 or Python 2.7.2 (if you already have Python installed, any version > 2.6 or 2.7 will work) and the python-libtorrent installer.
2. You will need the OpenSSL library for Windows and the Microsoft Visual C++ runtime components to be installed. To achieve this, check if you have the following DLLs available:

If yes, copy them to C:\Python26\Lib\site-packages. If they are not available, install the corresponding package and then copy them.

If you prefer building OpenSSL yourself, follow the instructions here.

3. Now run Duality as specified in the manual.

If you are unable to run Duality or if you feel there is a better way of handling this installation on Windows, please help us by filling a bug report.

[Top]

Manual

In case you have not already installed Duality and the required dependencies, please do so.
Before we begin, you should know that Duality does not work for torrents that have more than one file (you can check for the number of files in a torrent using torrentinfo.py)
There are two files that make up the client:

Using the client

The client allows you to download a file in parts.
By downloading a file in parts, we are restricting the piece count (the number of pieces) that will be downloaded.

Arguments: client.py <torrent-file>

Sample:

The client asks the peer for the piece count it wants to download. This is entirely up to the peer group and the piece count each peer downloads is decided by the peers themselves.
Once the part download is complete, two files (per download) will be saved in the current working directory.

An example:

The duality file is the part file downloaded while the config file has the required information that is needed to merge the parts later.

Each peer must carry both these files to the computer on which the parts are to be merged.

If ever required:
    You can rename the config file (but don't change the extension), but you should not modify it.
    You should not rename or modify the part file.

Merging files

The joinit.py tool merges the part files to form the complete file.
Parts can be merged on any machine, just the torrent file that was used to download the parts and the parts themselves are required.

Arguments: joinit.py <torrent-file>

How to merge the parts

Let the peers bring over their respective part files (along with the config files as shown above) to any common computer.
Place the config and part files in the same directory as the joinit.py file.

Run: joinit.py <torrent-file>
joinit will automatically load all the config and part files and attempt to merge them.

SHA-1 hashes of the parts are compared with the respective pieces in the torrent file. This may take some time for files of large sizes.
That should be all. The error messages that are shown in case something goes wrong are pretty accurate and should help you diagnose the problem.

Integrity checks on the parts

Before the actual merging takes place, integrity checks are performed on the parts in the following order:

1. Check whether the part files exist.
2. Check whether all the parts are present.
3. Check for duplicate parts.
4. Compare the SHA-1 hash of each piece.
In case of an error for (1 - 4), exit.
5. If all is well, merge the files

[Top]

Written in Python by Sukhbir Singh (sukhbir [dot] in [at] gmail.com), based on the paper co-authored with Piyush Gambhir.
Thanks to Piyush Gambhir and Aseem Tejpal for patiently testing this.
Arvid Norberg wrote the awesome libtorrent library on which Duality runs.