The Truth about USB Flash Drives

When Kaon (where I work) got into the touchscreen appliance business many years ago, we started by using Hard Disk Drives running Windows. We quickly learned that Hard Disk Drives cannot be shipped cross-country if you need them to work at the other end, and we learned that Windows cannot be properly secured for use as a Kiosk-style operating system. The former of these realizations led us to looking at using USB Flash Drives to hold all the software, and the latter led us to build our own operating system based on the Linux kernel. So a modern Kaon v-OSK boots from a USB Flash Drive, loads our Meson media platform, and then reads all the customer-specific application data, all from that one drive.

This is a tremendously powerful paradigm, because we know that the software environment is completely self-contained and reproducible. If that USB Flash Drive (UFD) works perfectly on one Kaon v-OSK, it’s going to work perfectly on all of them.

The end result is that Kaon is a massive consumer of UFDs. We buy them in huge quantities, because we are shipping frequent updates to Kaon v-OSKs all over the world. And by using them in such quantities, we’ve learned quite a lot about the UFD industry. If you use UFDs (also known as “Drive Keys”, or “Thumb Drives”, or “Jump Drives”, or “that useless piece of plastic that has files I can’t get at any more”), here are some things you should know.

0. Most companies that resell UFDs have no idea what they are doing.

Most companies that sell UFDs in large quantities are actually in the printing business. A UFD is just something they print on, like a T-Shirt or a Coffee Mug. That means that they have no idea what they are selling you, and will not be able to provide any consistency in terms of manufacturer and quality control. It took us quite a while to find a vendor we trust, and they now get all our business.

1. The “limited write cycle” thing is not an issue.

You may have heard that Flash memory can only be written a certain number of times, and therefore your drives will wear out. This is technically true, but you are never, ever going to see it happen because the number of write cycles supported is enormous.

2. Between 5% and 15% of UFDs are garbage.

Recall that we buy these things by the thousands, and so we get to experience all the low-probability events. And every UFD we get has already been tested by our supplier and the manufacturer, so the actual percentage of junk coming off the factory line is probably much higher. The very best manufacturers have the error rate cited. Some manufacturers have junk rates closer to 100%. (We don’t use their drives.)

3. Most Bad UFDs are Bad in very subtle ways.

When you put a bad UFD into your computer, there is only a very small chance that you will notice it is bad. The ones that are electrically inert, or start causing problems for your O/S are pretty rare. Most of them seem to work just fine. But when you go to read the data back, you find that it isn’t right. This is particularly tricky to detect because unless you eject, remove, and replace the UFD into your computer, you probably will just read back a cached copy of whatever you wrote.

4. You should verify everything you write to UFD, just like you do with DVDs.

When you burn a DVD, the software automatically goes back and checks that everything was written correctly. For some reason, this is not the case when you write to a UFD. I can only assume that the people who decided that these things should be treated like Hard Disk Drives had never actually used them. The most common problem that we see goes like this:

  • Write a file to the drive (if I were to unmount/remount and read back the data, I’d see that this file wrote fine)
  • Write a second file to the drive (this one wrote fine, too)
  • The first file is now corrupted

Actually, there are probably hundreds of files written between that first one being good, and it being bad, but the point is that writing a file to a bad Flash Drive changes some other random file on the drive.

Since we are writing a master copy of an operating system, media platform, and customer content, this is no big deal for us. Our mastering process includes a verification step similar to what you’d do with a DVD. But for a regular user, this is pretty hopeless. If you have one of these bad drives, your files are going to be OK for a while, and then suddenly they won’t be.

5. A UFD is either good, or bad. Good drives almost never “go bad.”

This means that if you have a drive that you have ascertained to be of the “not junk” variety, you can use that drive all the time, and you don’t need to worry about it. The only way I know how to make that determination is to completely fill the drive with data, unmount/remount it, and then read back all the data to see if it is exactly what you wrote. For a really big UFD, this is going to take a long time. (And the UFD will get really, really hot.)

6. UFDs and SSDs have nothing in common.

It might seem that since a USB Flash Drive uses NAND Flash memory, and a Solid State Drive uses NAND Flash memory, the two might be similar. They are not. SSDs use a fast, expensive kind of Flash called SLC, and UFDs use a slow, cheap kind of Flash called MLC, and that first letter makes all the difference. Another issue is the micro controller in the drive (called the Flash Translation Layer, see the diagram my daughter Cedulie drew at the top of this article).  It appears that SSDs are made by serious companies like Intel that know what they are doing when they write these micro controllers. And UFDs are, for the most part, not.

Notes

  1. joshuaesmith posted this