Configuring Raspberry Pi Zero W as NAS/backup server using Unison

"Free" cloud storage: overview 

I built a super cheap linux box for backing up my files remotely. I used the cheapest components, mostly what I had on hand. My costs were less than $50, one time, if you factor in my friend who's willing to let this live at his house permanently ($free/pricele$$). The total power draw is about 3.4W at idle which works out to about $7/year at least in southern California where rates are sky high.  

Parts:

• Raspberry pi Zero W: ~$12
• 2GB Micro SD card ($3?)
• Samsung S7 phone charger
• WD Elements 500GB HD (<$30)
• Old USB hub with power adapter ($10?)

Linux on a raspberry pi

Using Rufus, I wrote 2018-04-18-raspbian-stretch-lite to a microSD card.  This fit nicely on a 2GB SD card with ~400mb free/leftover.

You can install an OS on the Pi without any screen. To summarize, add these files to the FAT partition of the SD card:

1) wpa_supplicant.conf with unix line endings:

country=US
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev 
update_config=1
network={
       ssid="YourNetworkSSID"
       psk="Your Network's Passphrase"
       key_mgmt=WPA-PSK
    }

2) ssh (file is empty).

On my first boot of the Pi didn't manage to log into my network. Checking the SD card is confusing because on boot the Pi deletes the ssh and wpa file from the fat partition. To find out what's going wrong connect your pi to HDMI and run journalctl (dmesg reported wlan0 wasn't ready but not why). I had a typo in wpa_supplicant. I wasted a lot of time trying to find solutions, which involved lots of commands and configuration files which are all no longer used in a modern raspbian-stretch image. Don't believe what you read online.

The Pi's green light flashes quite a bit during boot but once it is solid the machine was ssh'able (about 1 minute). I removed the pi user (userdel) and made one for me (adduser), adding it to /etc/sudoers with sudoedit.

The idle power usage at this point was less than 1w if you can believe my killawatt.

Plugging in a hard drive

In /boot/config.txt I added max_usb_current=1 so that enough power is available to run the spinning HD. Actually, this wasn't enough for my drive, so I had to "backpower" the Pi.  That means that I plugged the drive into a 1A powered hub and the hub into the pi's usb (not power) port. The hub powered both the pi and the drive. Some people say this is a bad idea, but only because it bypasses a fuse that would protect the Pi from a surge. My Pi didn't cost that much. Unfortunately in this configuration the power draw was too high to consistently start up the hard drive, so I ended up forward and backward powering the Pi. Which seems to cause no problems, but probably isn't the most efficient. 

This explains how to mount and format an USB drive (not that different from any other type of drive).

Adding   /dev/sda /mnt ext4 defaults,nofail   to /etc/fstab makes the drive mount automatically at boot. chown user /mnt makes it so that your user can write to the mounted drive.

I wanted my hard drive to spin down while not in use. The first hard drive I tried (coolmax)  did not default to this, either under linux or windows.  So I learned all about how to change that setting, which is preserved here.  I tried 4 packages that didn't work before finding the 5th that did:

1. hdparm is the standard program you see everywhere, but mostly doesn't work with USB drives
2. sdparm is specific to SCSI drives in theory, but apparently many USB mass storage devices emulate SCSI.  Not in my case anyway.
3.  sg3-utils is another way of sending SCSI commands. I'm really starting to doubt this whole USB pretends to be SCSI since it didn't work either
4. udisks2 is another generic disk control program & deamon (not just for SCSI!) that didn't work.
5.  smartmontools from the name you'd think it's only for SMART (drive health) but apparently they threw the kitchen sink in there (total install footprint is 30MB!). This worked.

Cutting to the chase:

apt-get install smartmontools

add this line to /etc/rc.local:

smartctl --set=standby,15 /dev/sda

When spun up the total power usage reached 6W, but more importantly the idle consumption is just 3.4 W. That's around 3W for the HD for those keeping track. I suspect it could be much better but for the low-efficiency powered hub I'm using. The power adapter actually feels warm to the touch even at idle.  When I was using the coolmax drive my idle consumption was just 2.4W, total using a LG phone USB charger, but the drive didn't always spin up on reboot.

Attempting to save more power (made a tiny difference)

https://scribles.net/disabling-bluetooth-on-raspberry-pi/
https://www.jeffgeerling.com/blogs/jeff-geerling/raspberry-pi-zero-conserve-energy

Installing unison

apt-get install unison

Best to read the manual because there's lots of options to consider. Here's the basics of my pi.prf file on my PC. There's really too much to cover here sorry.

# Unison preferences
auto = true
fastcheck = yes
# Skip files ending in Name (mind the /)
ignore = Name *.tmp
ignore = Name Thumbs.db
ignore = Name ZbThumbnail.info
ignore = Name Release/*.pch
ignore = Name Debug/*.pch
ignore = Name Release/*.bsc
ignore = Name Debug/*.bsc
ignore = BelowPath $RECYCLE.BIN
ignore = BelowPath RECYCLER
ignore = Name System Volume Information
ignore = Name lost+found
root = h:/
root = ssh://user@DNS//mnt
times = true
xferbycopying = true
perms = 0
maxthreads = 1

Dynamic DNS if you don't have static IP 

Afraid.org offers free and reliable dynamic DNS so that you can connect to the machine remotely even when it gets assigned a new IP. I used the wget/cron script and crontab -e to install it. Just editing the cron file doesn't work, even though comments in the file suggest it would. 

Conclusion 

Cheap. Reliable? I hope so. My last attempt at this using a netbook wasn't reliable, but I think that was the hardware. Plus the netbook used about 3W more power at idle so in the long term this will be a better option (I guess that depends on who pays for the power though). 

update: worked great for a year and then the hard drive died. I suspect I wasn't powering it well enough since it was driven by the same USB adapter that also powered the pi. I switched to a new hard drive with it's own external brick, described here. While the power consumption was higher when on, I was able to make it overall much lower by hot-unplugging the USB hardware when not in use. 

Choosing a new low-input lag monitor for gaming

I'm looking for a new IPS LCD for my gaming/exercise rig. Because I'm using it for exercise, I need the color to be stable  independent of viewing angle, which means IPS. Which is too bad for gaming, where TN offers the highest speed and the lowest prices. But IPS is non-negotiable.

Modern IPS monitors offer much better response times than what was available a decade ago, which is when my current Samsung LCD was sold. There's no specific measurements for my model, but a very similar one was clocked at 60ms more lag than a CRT.  Even though the measurement methods used in those days were questionable, that's awful.

So far I've found a couple options that don't break the bank:

Dell SE2717HR

27 inches, 75hz max refresh rate, and freesync compatible. Regular lower price is $144, has been as low as $110 with MIR. I have not found a good measure of lag though. Tom's Hardware has some numbers for the same mode minus the R; it looks like very similar specs so???  They measure lag by pushing a button and waiting for the screen to actually change, which is realistic but gives a number much larger than any other measurement method: 47ms. I tried to find some way to translate this to the numbers from prad.de, whom supposedly use an oscilloscope to measure response time.

I looked for a common monitor review. The one I found is the asus mg24uq, which Tom's method rates at 63ms, and prad rates at 6ms! Yikes. How can they be talking about the same thing here? I'd like to trust prad here, since I went to all the work of letting google translate the german review into english :-)

VC279H 27in, 60hz max. Not freesync. Currently $160, has been $130 with MIR. prad rates the lag as 13ms in game mode. And it has vesa mounting. Some issues tho: for DVI inputs it seems that it doesn't do a good job of scaling < 1080p inputs. On HDMI it scales properly, but expects "video" luminance levels, meaning it maps the lower 16 shades of gray all to black (presumably a similar issue for white). Interesting that the two inputs clearly use different different hardware paths. No word on VGA.

VC239h 23in, 60hz, no freesync VESA mountable. $136. Presumably the same as above but smaller by 4 inches. This is actually what I purchased. The low-lag is really quite good, and quite a bit noticeably improved over my acient samsung LCD. Interestingly, it took awhile for me to adjust in some of my more twitch-orgiented games to the lower lag. Which is to say it was actually harder to play with less lag at first. I got over it, though.

The ergonomics of this display are so bad though. Most painful is that there's no button for switching between inputs. You have to mess around with the on-screen menus and it requires 3-5 button presses depending on which mode you are in and where you want to go. Man I'm regretting this purchase for that reason alone.

Does anybody prefer windows 10 to windows 7

I'm serious. I've yet to meet anyone who does. And I can't think of a real world usage case where I would prefer 10. I'm open to new perspectives, so enlighten me!

Meanwhile, how embarrassing for Microsoft. Somewhat reminiscent of the xp to 7 transition, where there were some advantages to 7, but not so many that it made the pain of the transition easy to ignore.  In each case the "killer" freature was that Microsoft literally killed the older OS by stopping security updates.

It's unfair to expect Microsoft to continue to improve or support an os so old that many customers payed for it more than 5 years ago. But forcing an unloved "update" on your users is not the way to maintain market share. If only osx wasn't a crappy alternative or Linux had better q/a the choice would be obvious.

PS. I'm not saying there are zero improvements in 10. Although the only one that comes to mind is the improved file copying status dialog box. I'm just saying none of those mild improvements justifies all the huge regressions in usability.

Lg Stylo 4: a disappointing "upgrade'

I've been saying a lot of positive things about the Stylo 3, and had high hopes for the 4th edition. Sadly LG is not the only one not reading my blog ;-)

Most disappointing is that the Stylo 4 does not have a removable back. No longer can you replace they battery. Or other broken components (which I have actually done on my Stylo 3). In this day and age there isn't such a huge improvement in phones every year. Continuing to make them disposable instead of repairable is morally questionable, and I thought that at least the the Stylo team had this figured out. The Stylo 3 was trivially easy to disassemble, and replacing a broken LCD took 10 minutes. No longer with the Stylo 4, which has a glued on back just all the other phone of this era.

Testing a generic 128gb microSD card: it's junk!

I purchased a 128GB SD card from ebay, for $10. At this price it pretty much had to be a lower capacity card that's been programmed to report greater capacity than it really has. Given how good ebay's buyer protection is I was curious to see how one of these preformed. Man, it's junk.

Delivery was quick, just 3 days. The package is so generic it doesn't even have a capacity printed on it. You have to look at the microSD card itself for that, which as advertised on ebay says 128GB.  No speed listing or anything but I will say it was a very easy to open package. So in that one way only, it was superior to what you would get in the store.

Windows recognized it as having 125GB of space, A somewhat odd compromise between marketing GB (1GB=1000^3) and real GB (1GB = 1024^3). FYI  a real SD card marketed as 128gb has 119 real GB. Since we don't expect it to have all that space anyway, who cares. So I tried writing just 1GB to it. Write speed quickly dropped to 1MB/s, and then  failed completely at about 400MB!  I had to reinsert the card to get windows to see it. Interestingly, it now claims to have a 55 GB file on (I've heard of hardware that employes file compression but never file expansion!).

Unsurprisingly, the 400MB that did get written were highly corrupted. I got tired waiting to see how much was actually good, since read speed was just 65KB/s! I mean, what is that? Class .0001? After reading 280MB, only 60mb of the data "written" was stored.

I tried the same trick I  used with my other fake SD card and added a dummy 500mb partition to the start of the disk so that the real file system would start at a later, perhaps better functioning region of flash memory. But that failed even worse, with only 80MB "written" before the drive became inaccessible.

So in conclusion, this generic SD card is complete junk. I paid $10 for "128"GB, but got perhaps .06GB. At that price ratio, a full 128gb would cost $2000! So that's a pretty bad deal. Of course, I asked for a refund, but even at $0, .06GB isn't worth anything.

Testing a fake 128GB Samsung EVO+ micro SD card

For about $45 you can buy a Samsung 128GB Micro SD card from Amazon. Sellers on Ebay offer them new for a lot less. I selected one for $26, shipped (from S. California), suspect it would be a fake, but curious what I would get. What I got looked reasonably real (shown here after opening).

My phone and laptop both reported full 128GB of space. Interestingly, the case contained the regular disclaimer that 1GB = 1000,000,000 bytes, but presumably in an attempt to avoid suspicion the card reported a full 128GB of space (using 1024^3 = 1GB). I tested it by copying some files to it and immediately had problems. The transfer rate was abysmal, between 1 and 2 MB/s, nothing close to the advertised rate on the package. And Windows kept on saying the card wasn't inserted, or needed to be checked for errors. After checking for errors some of the files were missing. I figured maybe the first part of the disk was bad, corrupting the FAT file system data, so I partitioned the disk in two, starting with a dummy 8mb partition.

The dummy partition helped. The write speed of the disk went up to about 5MB/s and Windows stopped reporting damage to the file system.

I also got more systematic and used a tool called H2testw to verify the actual storage space on the card. h2testw is not the nicest tool, but it got the job done, revealing that a full 24gb of "usable" space on the disk. Well, sort of usable: even keeping within that fraction of space, some of the data written was corrupted, though well below 1%.  So the card had about 20% of the advertised 128gb, which is a pretty bad deal for $26, especially given the random corruption of that 24gb (undetected until read back). For comparison, Amazon currently sells real 32GB Samsung cards for $16, or 50 cents/GB, whereas this card cost about 100 cents/GB.

I suppose I could use it to store my MP3 collection for playing back on a old phone. Time will tell if the usable storage remains usable long enough to make even that worthwhile.

Final note: that SD card adapter was so cheap that it failed after just a couple insertion/removal cycles. See the crack in the lower left corner. Thus, all my tests were conducted with a genuine Samsung adapter.

Repairing Win7 autochk "Cannot open volume for direct Access" error after power loss

My Win7 PC recently lost power while it was doing heavy work (compiling my latest android app). The result was file system corruption that could not be fixed the normal way (by scheduling to run chkdsk during bootup).  Here's the very useful error message Win7 gave me:

Checking file system on C:
Cannot open volume for direct access
Autochk cannot run due to an error caused by a recently installed software package.  Use system restore....
An unspecified error occurred (766f6c756d652e63 3f1)

I doubt a power failure really counts as installing a software package, MicroSoft.

What's going on here is unclear, but the problem persists; no amount of rebooting and running chkdsk at bootup fixes it, you always get this message. Nor could I run chkdsk while the OS is fully loaded (since it's my C drive).

But there is a solution, it seems. Reboot into system repair mode (F8 during boot up), and then select the command prompt. Find you C drive (it almost certainly won't be labeled as C; in my case it was H), and run chkdsk /r on it. For some reason it does not let the file system corruption prevent the scan here. The process is slow: on my mostly full 110GB SSD drive (samsung 840), it took a good 30 minutes. But it found and fixed the errors and now my PC can boot properly. Somewhat oddly the scan concluded with "Failure to transfer logged messages to the event log with status 50." but this seems to be non-fatal.