L
Luke Kenneth Casson Leighton
http://gitorious.org/python-libbittorrent/pybtlib
this is to let people know that a first milestone has been reached in
an experiment to combine git with a file-sharing protocol, thus making
it possible to use git for truly distributed software development and
other file-revision-management operations (such as transparently
turning git-configured ikiwiki and moinmoin wikis into peer-to-peer
ones).
the milestone reached is to transfer git commit "pack objects", as if
they were ordinary files, over a bittorrent network, and have them
"unpacked" at the far end. the significance of being able to transfer
git commit pack objects is that this is the core of the "git fetch"
command.
the core of this experiment comprises a python-based VFS layer,
providing alternatives to os.listdir, os.path.exists, open and so on -
sufficient to make an interesting experiment itself by combining that
VFS layer with e.g. python-fuse.
the bittornado library, also available at the above URL, has been
modified to take a VFS module as an argument to all operations, such
that it would be conceivable to share maildir mailboxes, mailing list
archives, .tar.gz archives, .deb and .rpm archives and so on, as if
they were files and directories within a file-sharing network.
as the core code has only existed for under three days, and is only
400 lines long, there are rough edges:
* all existing commit objects are unpacked at startup time and are
stored in-memory (!). this is done so as to avoid significant
modification of the bittorrent library, which will be required.
* all transferred commit objects are again stored in-memory before
being unpacked. so, killing the client will lose all transfers
received up to that point.
on the roadmap:
* make things efficient! requires modification of the bittornado library.
* create some documentation!
* explore how to make git use this code as a new URI type so that it
will be possible to just do "git pull"
* explore how to use PGP/GPG to sign commits(?) or perhaps just
tags(?) in order to allow commits to be pulled only from trusted
parties.
* share all branches and tags as well as just refs/heads/*
* make "git push" re-create the .torrent (make_torrent.py) and work out
how to notify seeders of a new HEAD (name the torrent after the HEAD ref,
and just create a new one rather than delete the old?)
so there is quite a bit to do, with the priority being on making a new
URI type and a new "git-remote-{URI}" command, so that this becomes
actually useable rather than just an experiment, and the project can
be self-hosting as a truly distributed peer-to-peer development
effort.
if anyone would like to assist, you only have to ask and (ironically)
i will happily grant access to the gitorious-hosted repository.
if anyone would like to sponsor this project, that would be very
timely, as if i don't get some money soon i will be unable to pay for
food and rent.
l.
this is to let people know that a first milestone has been reached in
an experiment to combine git with a file-sharing protocol, thus making
it possible to use git for truly distributed software development and
other file-revision-management operations (such as transparently
turning git-configured ikiwiki and moinmoin wikis into peer-to-peer
ones).
the milestone reached is to transfer git commit "pack objects", as if
they were ordinary files, over a bittorrent network, and have them
"unpacked" at the far end. the significance of being able to transfer
git commit pack objects is that this is the core of the "git fetch"
command.
the core of this experiment comprises a python-based VFS layer,
providing alternatives to os.listdir, os.path.exists, open and so on -
sufficient to make an interesting experiment itself by combining that
VFS layer with e.g. python-fuse.
the bittornado library, also available at the above URL, has been
modified to take a VFS module as an argument to all operations, such
that it would be conceivable to share maildir mailboxes, mailing list
archives, .tar.gz archives, .deb and .rpm archives and so on, as if
they were files and directories within a file-sharing network.
as the core code has only existed for under three days, and is only
400 lines long, there are rough edges:
* all existing commit objects are unpacked at startup time and are
stored in-memory (!). this is done so as to avoid significant
modification of the bittorrent library, which will be required.
* all transferred commit objects are again stored in-memory before
being unpacked. so, killing the client will lose all transfers
received up to that point.
on the roadmap:
* make things efficient! requires modification of the bittornado library.
* create some documentation!
* explore how to make git use this code as a new URI type so that it
will be possible to just do "git pull"
* explore how to use PGP/GPG to sign commits(?) or perhaps just
tags(?) in order to allow commits to be pulled only from trusted
parties.
* share all branches and tags as well as just refs/heads/*
* make "git push" re-create the .torrent (make_torrent.py) and work out
how to notify seeders of a new HEAD (name the torrent after the HEAD ref,
and just create a new one rather than delete the old?)
so there is quite a bit to do, with the priority being on making a new
URI type and a new "git-remote-{URI}" command, so that this becomes
actually useable rather than just an experiment, and the project can
be self-hosting as a truly distributed peer-to-peer development
effort.
if anyone would like to assist, you only have to ask and (ironically)
i will happily grant access to the gitorious-hosted repository.
if anyone would like to sponsor this project, that would be very
timely, as if i don't get some money soon i will be unable to pay for
food and rent.
l.