10,000 Foot Developer

The time came this week when I decided to replace my aging 500GB drives with something bigger and newer so I purchased two 2TB drives for about $95 a piece and started the task of dropping them into my NAS server planning for my usual mirrored configuration.  I’m concerned with long term redundant storage on my NAS which is the DLink DNS-343.  Can’t say enough about how much I like it.

Currently my DNS-343 has 4 drives, two 1TB drives in a RAID 1 (mirrored) config, and the two 500GB drives mirrored.  I treat the larger array like long-term memory, and the smaller array as short term and it is my more active array.  I do this because the drives spin down on the DNS-343 when not in use so the larger drives are less apt to failure, being less used, protecting that data even more.  And with planned drive upgrades its a nice way to rotate hardware through the NAS.

So my planned upgrade would happen like this:

1. First, you have two choices for the 500 GB data, since you won’t transfer this until last:
   1. You can make a tar.gz of the data,  transfer somewhere safe, and move it over at the end.
   2. You can just put one drive in the end and transfer the data in a telnet session.
2. Remove the 500GB Drives.
3. Add the 2TB drives.
4. Build the 2TB mirror array.
5. Copy the data from the 1TB mirror to the 2TB mirror.
6. Remove the 2TB drives, and replace with the 500GB drives.
7. Rebuild the 1TB mirror, and copy the 500GB array data to the 1TB array.
8. Put the 2TB drives back into place, and use the 500GB drives for something else maybe my surveillance system I am playing with (puppy linux has a nice live CD that recognizes camera hardware and has monit0r apps — ‘nother story).

So I do all that but step 4, copy data, is taking FOREVER.  I figure I have to copy about 700GBs over.  It was taking days.  Till finally – DOH! – I figured it out.

Wired up to my network, I was using Windows from my notebook to do the copying.  On a 100Mbps network!!!!  And considering that the drives have to mirror the data, the network has other traffic on it, etc. its no wonder.

Now at best, a 100Mbps network could copy 1 GB of data in about 82 seconds  (1 gigbyte = 8192 megabits).  That’s about 16 hours.  But this wasn’t happening!  The copy was taking days.

So, I downloaded a cool free program off of CNET called Lan Speed Test to test my local.  It was reporting my LAN speeds to actually be around 60Mbps.  So that’s about 137 seconds to copy gigabyte.  About 27 hours to copy all that data.  But it was taking even longer — because of the read/writes on the mirror on the NAS, I measured manually and the transfer was taking about 4 minutes a gigabyte!!  Easily, it was going to take 47 hours or longer to copy and that’s the time it was taking.

I thought — gee why don’t I just telnet in and do a Linux copy?  I can telnet into my router, my modem.  Well, it was almost trivial, I found out that you need to drop something called a “fun-plug” onto your machine to add telnet functionality.  DLink probably locked it off at some level.  People even root these NAS devices and install Debian, but I don’t want to program the thing I just want to finish installing my drives!!!!!

So here were the steps:

1. Make sure your DNS-343 is upgraded to the most current firmware.  Currently its 1.03, obtained from the DLink site.
2. Search on the net for “DNS-343 telnet” and you will find out about Fonz’s funplug.
3. Download it from his site — two files.  I got version 0.5, fun_plug and fun_plug.tgz.
4. Drop the two files onto Volume_1.  Fortunately, the drives I was keeping (the 1 TB drives) were in that configuration.
5. Restart the DNS-343.  It will unpack and install the telnet files on the device.
6. Telnet into your device from the command line like “telnet 192.168.0.32” — don’t worry you aren’t prompted for a uid or password. (about security in a moment)
7. The mirrored devices are located under /mnt.  What I did was go to the 2TB mirror under /mnt/HD_b2 (cd /mnt/HD_b2) and issued the copy command (use the -a option to preserve the directory timestamps): cp -a /mnt/HD_a2/* .
   
8. The copy ensues.
9. At then end, just remove the fun_plug files and reboot — now the telnet functionality is uninstalled.  There are ways of doing security and leaving it in but, like I said, I’m not interested in playing around I just wanted to copy files.  By removing telnet altogether I don’t have that security hole.
10. OK — to copy the data from the original drives.  The DNS-343 won’t recognize the old RAID if you put it back.  What you can do it insert one drive, mount it, and copy it or — if you tar’d up the filed and saved them just transfer them from wherever you stored them.

So how was the performance?

Well, at least four times faster easily than the network method.  The data was being copied at a rate of about 1GB/minute from one internal drive to another on the DNS-343.  With this method I could copy all my data to the new drives in about 12 hours.  Also it frees up my notebook from having to leave it running or at home while the copy is executing.

Honestly, I was hoping for faster!  The drives are 3 GBit sata drives, and I believe the busses in the device are supposed to be just that.  But even if the busses are the older 1.5 GBit speeds, it should only have to take about, supposedly, 7 seconds to copy over 1 GB (1.5 GBit speed = about 150MB/second) so my 700 GBs of data should have only taken 12 minutes to copy in a perfect world.  🙂