How to share a ton of stuff with someone else

If you have the stuff to share:

1. Download and install Resilio Sync
2. After opening the app, click the plus sign in the top left corner and select "Standard Folder"
3. Browse to the folder you want to share with someone else and click "Open"
4. A new entry will appear in your list of folders. At the right end of this entry will appear three dots (when you mouseover the row). Click the three dots and select "Copy Read Only key" or "Copy Read & Write key" depending on whether or not you want the sharer to be able to change what's in the shared folder. They key is now in your clipboard.
5. Send the key to the person you want to share with.

If you have received a key:

1. Download and install Resilio Sync
2. After opening the app, click the plus sign in the top left corner and select "Enter key or link"
3. Paste in the key that was sent to you
4. Browse to the folder you want to synchronize and select Open.
5. Go grab a coffee and chips and wait until the synchronization finishes.

Disclaimer: using any protocol to transmit/receive data that you are not legally allowed to transmit/receive is obviously illegal. I'm not responsible if you use this to do something illegal.

My Wallpaper Collection

I have amassed a fairly large collection of wallpapers over the years.  At one point, I decided to start over since I was tired of all the wallpapers I had.  My new wallpaper collection only uses high resolution photos from themes or other sources that look really good on high resolution monitors.

If you want to have a copy of my wallpaper collection, you can get it using BTSync (more info here), which will also allow you to get any new wallpapers I add to my collection (I usually add 5-8 per week).  If you haven't already, install the app, then add a new sync folder.  Choose where you want my wallpapers to be stored, then put the following in for the secret:

BFVR62FRM2AOZT3TLBISA2MLNZJ4XO5CA

This should get you started downloading.  Whenever I add new wallpapers, you'll get them.  If there are any you don't want, just delete them.

If you need help setting up your wallpaper to cycle through these images, go here.  The only instruction I would add is in step 3, browse to the folder where the pictures are being downloaded.

Dropping Dropbox and Google Drive for BTSync & a Raspberry Pi

I thought this article deserved republication, so I decided to take the easy route with today's post and just link to Jack's blog post.  In his post, Jack describes how he decided to drop his cloud file sync'ing app in favor of something run in house.  The best part about it is that it works just as well as Dropbox or Google Drive, but without putting your file on any device other than your own.  The advantage of this, as I've posted before, is that you can sync an unlimited number and size of files.

I've done something similar to this using Western Digital's My Book Live Duo.  Since the MBLD runs on Linux (Lenny Squeeze), it was pretty easy to load BTSync and fire it up.  So, my NAS participates in the synchronization of my folders from my desktop to my laptop.  So, even if my desktop is offine (or in standby) I can still sync between my laptop and my NAS (over the internet too) and when my desktop comes online it'll sync up with the NAS and laptop.

BTSync: A new alternative to cloud based drive services

In a previous post, I wrote about how to connect your system folders (My Documents, My Pictures, etc.) to a Google Drive/Dropbox/Skydrive account.  The benefit of this is that you always have a backup available on the web.  If my hard drive were to die today, I wouldn't be too bad off since all I need to do is download the Google Drive desktop app and redirect my system folders to my Google Drive folder.  All my stuff would come back like it was never gone.  I'd still have to install applications but that's not too bad.  It's nice every once in a very long while to lose all my programs.  It forces me to trim the fat or look for better versions of the apps I use (Like Paint.Net or Notepad++).

However, I've always been a fan of the Bittorrent protocol.  It's a peer to peer file sharing program that most people use to download illegal movies or music.  While those days are behind me, I've tried to help people understand that P2P is no the same as illegal.  There are perfectly legal uses of P2P protocols like Bittorrent (see this, this, and this).  

And so my becoming a fan of Google Drive (and also Google Apps that allow me to work on a spreadsheet or document with many other people simultaneously) only makes sense.  If you're happy with that, or you only have one computer and no one you would ever want to share anything with, stop reading this blog post now (check out my most popular post instead).

Last week I stumbled upon BTSync (it has since been spun off and renamed to Resilio Sync).  This is a little app created by the same people that designed the Bittorrent protocol.  While there are several uses of BTSync, the main use is to compare it to the features and functionality of products like Google Drive.  Here is my comparison matrix:

Feature	Google Drive	SkyDrive	Dropbox	BTSync
Size Limit1	15GB2	7GB	18GB	∞
Shareable Content	Must be in Google Drive folder	Must be in SkyDrive folder	Must be in Dropbox folder3	Any number of existing folders
Your files stored on a corporate server	Yes	Yes	Yes	No
Online File Editors	Yes	Yes4	Viewer: Yes Editor: No	No
Web Access to Files	Yes	Yes	Yes	No
Sharing	Only with Google users5	Only with MSN Passport users	Only with Dropbox users	Anyone with the app

¹Without paying anything
²Shared with Gmail and Google+ Hi Res Photos
³Although pretty easy
⁴If you have a paid subscription to Office360
⁵Not necessarily Gmail users, but anyone with a Google account

So, let me highlight some of the reasons that BTSync intrigues me.  First of all, there is no limit to the amount of content that can be synchronized.  This is mainly due to the fact that your sync'd files are not stored on some limited corporate server somewhere, they're stored only on the systems where your files are synchronized.  This is a double edged knife, however.  While Cloud based drives can be used for backup, BTSync doesn't back your content up to the internet.  If you add a folder to BTSync only on one computer, the files aren't copied anywhere.  So, unlimited size but no storage on any corporate servers.  That may be two advantages in some peoples' books.

Unlike most of the corporate cloud based drives, BTSync is only about the file transfers.  As such, there are no online editors for your files.  However, since there is no web access to your files (because they're not on any corporate server) you'll only be accessing your files from your desktop.  You can install any office productivity suite locally (or even use google docs in a roundabout way).

Sharing is another feature that is different from other offerings.  While other offerings essentially require you to have an account in order to have RW access to a shared folder, BTSync will allow anyone with the app to access/sync your folder as long as they have the secret, a special, very long, very complicated password.  If you give another person the secret to your folder, they can sync your folder with a folder on their computer.  Any changes either of you make will be reflected in the other's sync'd folder.

You can however, give out two other types of secrets: a read only secret and a one time secret.  The RO secret allows the person to whom you give it to sync your folder to a folder on their system, but they won't be able to make changes to your folder.  This is a good way of distributing files to friends.  For example, you could setup your pictures folder and hand out the RO secret to family members.  They would then get copies of any pictures you put in your pictures folder (think of doing this with your iCloud Photo Stream).

Have you started thinking about the possibilities yet?

Another thing I did right away with BTSync was to synchronize my Dosbox working directory across all my PCs.  I play retro DOS games every once in a while.  By synchronizing the working directory for Dosbox (a DOS emulator) I can access the games, save files, and anything else on any of my computers.  This allows me to play a game on my desktop then save the game, exit Dosbox, go to the living room and launch Dosbox and pick up the saved game right where I left off.

I'm hosting a LAN party this weekend.  I setup a folder where I intend to put all the installers and files needed (including my dosbox folder).  I've added it to BTSync and will hand out the RO key via Facebook and email to everybody coming to the party.  That way they can install the games ahead of time to make sure they work.

Another idea I had was to use BTSync to replace NQSync (which I had originally intended to write using the bittorrent protocol anyway).

One more feature then I'm done.  I promise.
In my previous post, I talked about changing the default location of system folders.  This is necessary since most cloud drives require the sync'd files to be in a particular folder.  So I have to move my system folders to that sync'd folder in order to get them to sync.  With BTSync, I don't have to move the folders.  I can setup each folder in BTSync without moving it.  This means that I don't have to move anything, I don't have to change Windows configurations, or anything.

I haven't done it yet, but I will probably only run Google Drive on one of my PCs.  The rest will use BTSync to stay in sync.

So, that's my initial review of BTSync.  So far, I don't see it replacing Google Drive, but I do see myself using it to distribute pictures and home videos to my family, keeping my games in sync across all my PCs, possibly synchronizing recorded TV shows, making backups, and using it at work.

Legal Uses of the BitTorrent Protocol: Part 3

Alright, so if you've read my previous 2 posts, you should have a good understanding of how torrents work and how you could setup your own torrent.  Now i'll explain how to use HFS (explained in part 2) to turn your torrents into items in an RSS feed.

The concept is fairly simple when compared to the bittorrent protocol itself.  Essentially an RSS feed needs to have a link to the torrent file in the title, description, or enclosure of the item.  HFS allows us to do this with a little tweak.  I found this tweak on http://www.rarst.net/software/hfs-rss/ (original text below) and i got it working in less than 2 minutes.  I currently use this to send home videos to all my family members.  They all have utorrent running and subscribed to my feed.  All i do is create the torrent and dump it into my HFS folder.  They get the torrent from the rss feed and all start downloading it at the same time.

----------Original Article----------
I had previously covered HFS which is excellent software for setting up HTTP file-sharing server. Unfortunately it does not natively provide RSS feeds that would be perfect for notifications about new files available for download.
After some poking and bugging fellow HFS users and developer himself last year I had put together usable workaround.

File list
HFS has native support for file lists. It outputs nice pack of URLs that is easy to feed to download managers instead of clicking each file.
Add some XML markup to that list and we have ourselves RSS feed.
File list template
Create text file named filelist.tpl in same directory as HFS executable and paste following code in it:
<?xml version="1.0" ?>
<rss version="2.0">
<channel>
<title>%host%%folder% directory feed</title>
<description>Files in %host%%folder% directory</description>
<link>http://%host%</link>
%files%
</channel>
</rss>
[files]
%list%
[file]
<item>
<title><![CDATA[%item-name%]]></title>
<description><![CDATA[%item-name% (%item-size%)]]></description>
<guid><![CDATA[http://%host%%folder%%item-url%]]></guid>
</item>
Browser discovery
File list is replaced with RSS feed and it should be obvious as such.
Menu > Edit HTML template…
Insert following line right before </head> and press Ok.
<link rel=”alternate” type=”application/rss+xml” title=”RSS for folder” href=”http://%host%%folder%~files.lst?recursive” />
Now browsers that support feed discovery will show available RSS feed in each folder.  File list link that is shown on page can also be renamed (or removed).

Limitations
Feed is served as plain text instead of XML – should not be a problem but may limit compatibility with some readers; no timestamps on RSS items because of complex time format used, currently impossible to replicate it in HFS; some characters (square brackets for example) are not properly encoded; resulting RSS includes all files instead of few recent ones, no workaround because of way sorting in HFS works.Overall
Method is far from perfect but is easy to implement and doesn’t require additional software. Definitely beats constant explaining why I had deleted something before someone had seen it.

Legal Uses of the BitTorrent Protocol: Part 2

The other day i posted a basic introduction to the bittorrent protocol.  Now i want to write down somewhere how bittorrent can be used in a legal situation.  As i mentioned in yesterday's blog, i have many servers that all need the same file every time there is an update, which could be as frequent as once a quarter for major updates.  The status quo is to remote into every server, log into the vendor website, and download the update.  This can be very time consuming, not to mention very boring. 
That's where bittorrent has helped me.  To continue with the analogy of the ants: each server is an anthill.  The update is the sandwich.  Now all i need are ants at each anthill (a bittorrent client at each server) and an ant with a radio (a bittorrent tracker).  As my torrent client, i chose to use uTorrent.  It's a fairly small torrent client with all the features i need.  Particularly, uTorrent can listen to an RSS feed and (as long as the feed is designed correctly) automatically get the torrents in the feed.  In other words, if i can create an RSS feed which contains the torrent file (the descriptor file that tells torrent clients about the sandwich) then i can have all the anthills subscribe to that feed.  But i get ahead of myself.
So, each anthill (server) has an army of ants (uTorrent client).  Now i need a seeder, that would be someone who has the original sandwich.  So, i install utorrent on my laptop (or i could use on of the servers, it doesn't really matter).  I also need a tracker.  Turns out the uTorrent client has a small basic tracker built in but disabled by default.  So, on the client on my laptop, i go and enable it.  Now all of the pieces are there to start torrenting.
For the first few torrents, i do things manually, just to make sure I've got everything setup properly.

1. I open the uTorrent client on my laptop and start the 'new torrent' wizard.  I point the wizard to the update file i want to share (this could also be a directory with thousands of files, no need for zipping).  I also point the wizard to my laptop's IP address as the tracker.  When i hit finish, uTorrent outputs a *.torrent file (i saved that to my desktop).  If i look in my uTorrent client, I'll see that i'm now seeding the file and the fact that I'm seeding it has been reported to the tracker portion of the uTorrent client.  Now all i need to do is get that torrent file over to the other clients.  (Remember I'm doing this manually, this part will be much less complicated after i automate it.) 
2. I go to each one of the servers and copy the *.torrent file (2-3KB max) to each server. 
3. Then i open the *.torrent file using the uTorrent client on each server.

On a side note, i usually configure each torrent client to use the same port number.  This makes it easy to categorize the traffic in monitoring software.  It also makes it easy for security guys to recognize that it's a legal protocol.  Once i open the torrent file in the torrent client on each of the remote servers, that torrent client contacts the tracker on my laptop and asks for who else has the file.  The tracker says the file is being seeded (the whole file is shared) by my laptop.  uTorrent then starts to request very small pieces of the file from my laptop.  As it downloads the file, it updates the tracker to let it know which parts of the file it made available to other members of the torrent.  Eventually, between the whole copy on my laptop and the partial copies on the other servers, there are 2 copies of each piece of the entire file. At that point the torrent really speeds up and fairly quickly, the file is copied to each one of the servers.
Now, to automate the process.  I'm sure there is probably some perl code to automatically create a torrent from files in a certain directory, but for now, I'm still doing this manually.  It only happens once every time there's an update.  However, i can automate steps 2 and 3, by adding one piece of software to my laptop (obviously everything running on my laptop could actually be done on one of the servers, it's not crucial where it actually happens).  The software to be added is HFS (HTTP file server).  The reason i added this software is that it easily creates an RSS feed from the files in a directory.  So, by adding HFS, all i have to do is save the *.torrent files in a particular directory, and they automatically get added to the RSS feed.  There's some tweaking necessary for the RSS feed, so I'll detail how to do that.  Once the RSS feed is setup, i can go to each of the torrent clients on the remote servers and add it to the feeds list.  The client checks the RSS feed and downloads all torrents in it (there are obviously checks to make sure a torrent isn't downloaded twice).  By adding the RSS functionality, steps 2 and 3 are taken care of automatically.  All i have to do is save the *.torrent file in the HFS directory (saving the *.torrent file to a particular location is part of the torrent creation process, step 1).
So there you have it.  I'll post instructions tomorrow on how to customize HFS to make the RSS feed compatible with uTorrent's RSS feature.

Legal Uses of the BitTorrent Protocol: Part 1

You may have heard of bittorrent and associated it with illegal movie and music downloading.  While admittedly, the bittorrent protocol is mainly used for nefarious purposes, it occurred to me a long time ago that there had to be a legal use for it.  Then it hit me, the main purpose of the bittorrent protocol is to efficiently move one file from one place to many other places.  I manage a set of servers that run a third party software package that gets updated several times a year.  These updates are difficult to distribute because 1) they are fairly large (100-750MB) and 2) I have to distribute the same file or group of files to all my servers.  In steps bittorrent. 

How does bittorrent work?
Think of bittorrent as organized ants eating your sandwich while out for a lovely picnic.   Their ultimate goal is to get the sandwich to the anthill.  So, they break the sandwich up into very small pieces and each ant takes one back to the anthill.  The more ants, the faster the job gets done.  Now, imagine there are 15 anthills that want your sandwich.  (Also imagine the ants have an advanced piece of technology that duplicates small pieces of your sandwich).  With all these anthills trying to get your sandwich and given their duplication technology, each anthill would have a copy of your sandwich within short order.  Now imagine there's a huge ant with a radio.  You let him know that you have a sandwich to share with the ants.  Each anthill asks this ant where the sandwiches are and they get started.
Torrents work the same way.  You have a big file (the sandwich) that you want to distribute to a bunch of destinations (anthills).  You create a file called a torrent file (file extension *.torrent) and announce it to a tracker (the ant with the radio).  Then you email the *.torrent file (a very small file by the way) to the various destinations (anthills) who then open the torrent with their favorite bittorrent client.  The torrent file instructs the client to go to the tracker (the big ant with the radio) and get a list of the sources (or seeders) of the file, aka you.  The ants descend on your file and begin to take back copies of very small chunks of your file.  None of this is very novel (in fact it's quite similar to napster if you remember).  The cool thing is that once an anthill begins to participate in the torrent, the tracker (the big ant with the radio) is notified who then lets all the other anthills know.  From the point of view of one of the anthills, there is a big sandwich out there.  There are also 14 other anthills with part (or soon all) of the sandwich.  The anthills begin to trade with each other.  The more anthills involved, the larger the swarm, the more quickly the file is distributed.  Cool, huh?