Making FreeBSD magnet links
Updated: December 7, 2022 at 12:20 PM
Originally Posted: July 3, 2018 at 4:49 PM
Note: This was originally posted here, but things have been improved since then and so it has been updated.
For the last few years, I’ve been producing torrents and publishing magnet links, but there is some special work that I do to make these. The first few releases, I inserted a bogus tracker into the torrent, because despite there being plenty of tools out there for producing trackerless (DHT) torrents, they were all GUI and I never found any that were command line based. The other was there was/is no tool for extracting the info hash and building the magnet link. There may be tools now, but I couldn’t find any when I started 3 years ago.
The following steps are based upon the recent release of FreeBSD 11.2‑R, adjust as necessary.
- Fetch the FreeBSD release and release announcement into a directory
(I create a per release directory).
remove the
CHECKSUM.SHA256
,CHECKSUM.SHA512
andindex.html*
files.
This step depends upon snapaid running. If it is no longer running for some reason,$ REL=12.4-RELEASE $ RELABV=12.4R $ curl https://www.funkthat.com/~jmg/FreeBSD-snap/snapshot.idx.xz | xzcat | awk ' $2 == "'"$REL"'" { print $9 }' | xargs -L 1 -P 3 wget --no-verbose -c --limit-rate=7000k $ wget https://www.freebsd.org/releases/$RELABV/announce.asc
addinfo.sh
could be run manually against the release email (not the downloadedannounce.asc
file as the message id is needed) to generate snapshot.idx.xz locally, and use that instead. - Verify the GPG key that signed the above files. This is usually Glen Barber’s key, but not always. I have met and verified his fingerprint in person, If you have verified someone’s key who has signed Glen’s key, that is another good way.
- Verify the release announcement:
$ gpg --verify announce.asc Warning: using insecure memory! gpg: Signature made Mon Dec 5 17:34:59 2022 PST using RSA key ID 478FE293 gpg: Good signature from "Glen Barber <gjb @ FreeBSD.org>" [unknown] gpg: aka "Glen Barber <gjb @ keybase.io>" [unknown] gpg: aka "Glen Barber <gjb @ glenbarber.us>" [unknown] gpg: aka "Glen Barber <glen.j.barber @ gmail.com>" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 78B3 42BA 26C7 B2AC 681E A7BE 524F 0C37 A0B9 46A3 Subkey fingerprint: 8D12 403C 2E6C AB08 6CF6 4DA3 0314 58A5 478F E293
- In the past I have used BitTornado for other things, so I ended up
using it as the basis to make the tool for creating trackerless torrent
files. The modifications were simple. It appears that the original
BitTornado CVS tree is off-line (anyways, it was served insecurely),
but it looks like
effigies/BitTornado is
similar enough that it could be modified and used. I copied
btmakemetafile.py
tobtmaketrackerless.py
and applied the following patch:
If you notice, the only thing that is done is to drop the first argument, and instead of passing it into$ diff -u btmakemetafile.py btmaketrackerless.py --- btmakemetafile.py 2004-05-24 12:54:52.000000000 -0700 +++ btmaketrackerless.py 2016-10-10 17:13:32.742081000 -0700 @@ -23,9 +23,9 @@ def prog(amount): print '%.1f%% complete\r' % (amount * 100), -if len(argv) < 3: +if len(argv) < 2: a,b = split(argv[0]) - print 'Usage: ' + b + ' <trackerurl> <file> [file...] [params...]' + print 'Usage: ' + b + ' <file> [file...] [params...]' print print formatDefinitions(defaults, 80) print_announcelist_details() @@ -33,9 +33,9 @@ exit(2) try: - config, args = parseargs(argv[1:], defaults, 2, None) - for file in args[1:]: - make_meta_file(file, args[0], config, progress = prog) + config, args = parseargs(argv[1:], defaults, 1, None) + for file in args[0:]: + make_meta_file(file, None, config, progress = prog) except ValueError, e: print 'error: ' + str(e) print 'run with no args for parameter explanations'
make_meta_file
, aNone
is passed instead. This will simply not add trackers to the torrent file. - I then run the following script to verify the downloaded files, and
generate the torrent files:
$ cat cmp.sh #!/bin/sh - # REL=12.4-RELEASE # RELABV=12.4R # xzcat ~jmg/public_html/FreeBSD-snap/snapshot.idx.xz | awk ' $2 == "'"$REL"'" { print $9 }' | xargs -L 1 -P 3 wget --no-verbose -c --limit-rate=7000k # XXX - following command 404, manually saved from relase announcement # wget https://www.freebsd.org/releases/$RELABV/announce.asc # wget -c https://download.freebsd.org/ftp/releases/CI-IMAGES/$REL/amd64/Latest/FreeBSD-$REL-amd64-BASIC-CI.raw.xz # wget -c https://download.freebsd.org/ftp/releases/VM-IMAGES/$REL/riscv64/Latest/FreeBSD-$REL-riscv-riscv64.{qcow2,raw,vhd,vmdk}.xz grep -h '^ SHA512' announce.asc | sed -e 's/SHA512 (\(.*\)) = \(.*\)/\2 \1/' | sort -k 2 > sha512.from.asc while read hash fname; do if [ -e "$fname" ]; then if [ -e "$fname".torrent ]; then echo skipping "$fname"... continue fi sigfile=$(grep -l -- "$fname" *.asc | head -n 1) echo checking "$fname", sig in: "$sigfile" #res=`sha512 -q "$fname"` #res=`shasum -a 512 "$fname" | awk '{ print $1 }'` res=$(openssl sha512 < "$fname" | awk '{ print $2 }') echo "File is: $res" if [ x"$res" != x"$hash" ]; then echo missmatch! "$fname" exit 1 fi btmaketrackerless.py "$fname" & else echo missing "$fname" exit 1 fi done < sha512.from.asc
- Once all the torrents have been generated, I then make the magnet
links:
$ cat btmakemagnet.sh #!/bin/sh - # metainfo file.: FreeBSD-10.3-RELEASE-sparc64-bootonly.iso.torrent # info hash.....: 06091dabce1296d11d1758ffd071e7109a92934f # file name.....: FreeBSD-10.3-RELEASE-sparc64-bootonly.iso # file size.....: 203161600 (775 * 262144 + 0) # announce url..: udp://tracker.openbittorrent.com:80 # btshowmetainfo 20030621 - decode BitTorrent metainfo files for i in *.torrent; do btshowmetainfo.py "$i" | awk ' $0 ~ "^info hash" { info = $3 } $0 ~ "^file name" { name = $3 } END { print "magnet:?xt=urn:btih:" info "&dn=" name }' done
- I then create the magnet links file, and update the Torrents wiki page.