...making Linux just a little more fun!
Fri, 27 Oct 2006 17:04:01 -0400
Greetings,
I have Linux (2.6.15.6) booting from a USB drive. My application is machine control and I wanted to use something solid-state. A USB stick seemed like a good idea because it is cheap and readily available and reasonably fast.
My problem is that I get file system corruption and "bad sectors" some times, presumably due to an unclean shut down. I am using an ext2 filesystem with the thinking that a journal would just wear out the drive in a certain spot. Maybe that's not correct -- I am not sure if the drive has a wear-leveling algorithm implemented or not.
Questions:
1. Is ext2 the right choice? Or would reiserfs be better? Vfat??
2. Is their a quick way to detect bad sectors? I tried running e2fsck -c but it has been 45 minutes so far!
I am thinking if I can detect the corruption, I can just reformat the partition and restore it from another read-only partition. The kernel seems to report them:
sd 5:0:0:0: SCSI error: return code = 0x08000002
sdb: Current: sense key=0x3
ASC=0x11 ASCQ=0x0
end_request: I/O error, dev sdb, sector 598730
I guess I could grep the messages log, but I don't think that is a
conclusive check.
Any pointers would be appreciated.
-- Brandon Reynolds Ph: 330-644-3059 Systems Engineer Fax: 330-644-8110 CTI, a Poling Group Company Email: bmr at comtime.com
Rick Moen [rick at linuxmafia.com]
Tue, 31 Oct 2006 10:24:33 -0800
Quoting Brandon M. Reynolds ([email protected]):
> I have Linux (2.6.15.6) booting from a USB drive. My application is > machine control and I wanted to use something solid-state. A USB stick > seemed like a good idea because it is cheap and readily available and > reasonably fast. > > My problem is that I get file system corruption and "bad sectors" some > times, presumably due to an unclean shut down. I am using an ext2 > filesystem with the thinking that a journal would just wear out the > drive in a certain spot. Maybe that's not correct -- I am not sure if > the drive has a wear-leveling algorithm implemented or not. > > Questions: > > 1. Is ext2 the right choice? Or would reiserfs be better? Vfat??
Flash-memory drives have some peculiar physical characteristics, and choice of filesystem for them depends on whether you value multi-OS compatibility (which suggests VFAT) or whether you value drive life and even-ness of wear (which suggests JFFS2 or YAFFS). Essentially, they have a limited number of erase/write cycles per sector, and each sector must be read, erased, and rewritten any time any data in the sector changes. The reasons why VFAT is bad for drive life, not to mention atime updating, follow logically from that, and should be obvious.
I imagine that the wear resulting from ext2/ext3 or ReiserFS would be absolutely excessive.
> 2. Is their a quick way to detect bad sectors? I tried running e2fsck > -c but it has been 45 minutes so far!
I imagine that the wear resulting from e2fsck would be absolutely frightful, sorry to say.
More at: http://www.linuxjournal.com/article/6867 http://en.wikipedia.org/wiki/Flash_memory
-- Cheers, The genius of you Americans is that you never make Rick Moen clear-cut stupid moves, only complicated stupid moves [email protected] that make us wonder at the possibility that there may be something to them that we are missing. --Gamel Abdel Nasser
Brandon M. Reynolds [breynolds at comtime.com]
Tue, 31 Oct 2006 15:31:53 -0500
The original e-mail was top-posted, and contained all of Rick's previous answer. PLEASE do not top-post, and please do snip quoted material down to relevant text! - Kat
Rick,
Thanks a lot for the help! I will look into those other filesystems. The other thing I forgot to mention is that if you write over that sector again it seems to work. So I don't think we are hitting the 10000 writes, but maybe not. I wonder if it is possible that the drive is marking a sector bad because it was in the middle of zeroing the sector or something?
I do have the partitions mounted with noatime. I think part of my problem maybe that I am using sqlite which seems to want to sync everything to disk all the time. What I will probably end up with is using a RAM disk and then syncing that to USB drive on a shutdown.
Regards,
-- Brandon Reynolds Ph: 330-644-3059 Systems Engineer Fax: 330-644-8110 CTI, a Poling Group Company Email: [email protected]
Rick Moen [rick at linuxmafia.com]
Tue, 31 Oct 2006 14:08:51 -0800
Quoting Brandon M. Reynolds ([email protected]):
> Thanks a lot for the help! I will look into those other filesystems. > The other thing I forgot to mention is that if you write over that > sector again it seems to work. So I don't think we are hitting the > 10000 writes, but maybe not. I wonder if it is possible that the > drive is marking a sector bad because it was in the middle of zeroing > the sector or something?
I'm thinking it smells a bit like borderline filesystem corruption, e.g., a given sector has existing coherency problems, which get cured in the act of going through an erase/rewrite cycle. That might be just a SWAG (silly wild-assed guess) on my part, of course.
> I do have the partitions mounted with noatime. I think part of my > problem maybe that I am using sqlite which seems to want to sync > everything to disk all the time. What I will probably end up with is > using a RAM disk and then syncing that to USB drive on a shutdown.
I do think that sounds like a winner. It's so tempting to just treat these NAND flash-memory devices as if they were just tiny, electonic hard disks, but that's really not quite the right model, as it turns out.
-- Cheers, "Transported to a surreal landscape, a young girl kills the first Rick Moen woman she meets, and then teams up with three complete strangers [email protected] to kill again." -- Rick Polito's That TV Guy column, describing the movie _The Wizard of Oz_