[NTLK] flash longevity question

[Dan]

Thanks Andrei.  I knew most of this.  And it is interesting that the NAND in say a Wii console (SLC internal NAND) does have 100,000 cycles.  But the WiiU which is MLC only has 10,000.  So some devices did have the 100,000 write cycles depending.  I assume that is the early to late 2000's era, and later on they started with MLC and 10,000 cycles.  And TLC is even worse at 3,000 write cycles (and I think most high capacity drives/chips are TLC these days).

I never knew the BIOS flash was only 35 on some boards.  True you don't use it much but wow that is low.

I also didn't think of FAT being a writing hog, but once you mentioned I remembered.  A lot of USB flash drives are formatted in FAT as to be the most compatible with the widest range of devices.  Some of them I had left rather than reformat them.  I think I will change that, although I don't use those particular drives much.

With regard to TRIM and SSDs, most people fill them up even when you tell them not to.  Similar to telling people never click a link in a email unless you check it first in detail.  They never do it and click first/ask questions later, or at least this is true of most of the computer using population.  I can't tell you the number of youtube videos I have seen where someone shows their file system clicking through to demonstrate something and show their drives installed.  The SSD boot drive always has almost FULL red bar under it showing at a glance how overfull it is.  Also seen quite a few with one person fixing another's computer with a SSD drive that failed.

The one good thing about SSD's is they usually tell you once their spares are used up or getting there giving you time to move data off before the drive fails completely.  It is much worse with USB thumb drives as they do not tell you and simply go into lock down mode/read only mode.  Sometimes this will still enable you to get the data off it.  Other times they fail completely.  I haven't had one do it yet, but I know people that have.

Write cycles I think most of us heard about (at least on some level), but the aspect of the data fading away without power after a perlonged period of time is a new slant.  Or at least it was to me. Also how greatly the number of write cycles have shrunk in recent years.

-Dan

On 9/16/2022 12:26 AM, Andrei Chichak wrote:
> The flash chips from the Newton era had a typical guaranteed life expectancy of 100,000 cycles.
> 
> What is a cycle?
> 
> The flash chip is broken up into sectors, typically 8 sectors per chip at the time (but it depends). You can read the data in these sectors all you want, reading doesn’t degrade the chips at all. When you write to the chip, you have to erase the sector before you can write it, and the erase operation for a single sector takes about 1 second.(1) The write operation is very fast, it happens in a “flash” (300us). Oh, yes, you can’t read the chip while it is being erased or written. So that’s a drag.
> 
> Now, erasing is where the damage happens, Newton era flash chips can only endure 100,000 erase cycles PER SECTOR before they might not erase properly or write properly (stuck bits). This is a cycle, erase and write (mostly erase). But you can read it as many times as you wish.
> 
> A full chip erase would take 8 seconds (8 sectors, 1 second per sector, 8 seconds).
> 
> We really didn’t get a good read on how long the chips would retain their data back then, or what you had to do to maintain the bit patterns, but 20-30 years were numbers that were thrown around, whether you used the system or not.
> 
> More recent flash chips were being targeted towards digital cameras (they were the big consumer of flash chips before Apple iPhones got popular). The use pattern for a digital camera is writes of a few huge files (images). So the design considerations in camera targeted flash is cheap, large capacity, and not very many erase cycles. It’s a thing, cheap, big, durable - pick two.  Modern flash chips are down to 10,000 cycles or less, you trade off erase cycles for cheap but increased size. If you want to get weird, the flash chips that PC motherboards use are really cheap and have a 35 cycle endurance, but how many times to you reflash the BIOS on your PC? Twice maybe?
> 
> Somewhere between the Newton era and the modern era, flash chips crept up to 1,000,000 cycles. But those chips were not very big, and were quite expensive, but you could still wear them out. Microsoft’s FAT file system has one sector that gets written (erased and written when on an SD card) every time that drive is written to. So if you were to, say, write 400 bytes to a file, once per second, on an SD card, in about 2.5 years you will have erased and written the FAT table about 2 million times and your SD card will no longer read or write properly (first hand experience).
> 
> 
> To get around the limitations inherent to flash, things like TRIM, wear levelling, and spare sectors are now common (but not in the Newton). With wear levelling, the OS keeps tracks of how many times a particular sector has been erased and spreads the pain around so that one particular sector doesn’t get erased to death. After erasing, the OS can check and make sure that all of the bits in the sector got erased (to 1) and, after writing, a checksum calculation can determine if any of the bits didn’t write properly (to 0). If the sector doesn’t erase or write properly, it can be replaced by a sector from the spare sector pool. Once your spare pool is exhausted, you’re going to start getting bad sectors on your SSD. Bad sectors on an SSD is a good indication of imminent failure, since all of the free sectors have been wear levelled their whole lives.
> 
> Enough for now. Enjoy using your Newts, the Newton development team did a bunch of extra work with data compression and full knowledge of the foibles of flash to make the chips last a long time. Don’t worry, but do backups occasionally.
> 
> A
> 
> 
> 
> (1) - think about an SSD, these things also use flash memory which is broken into sectors, and has to erase a sector before writing it. What they do is they have a pool of pre-erased sectors that are ready to write to, so you don’t have to wait for a sector to be erased so you can write it. When a sector of space is released to the free list, it hangs around a while in purgatory, then when nothing is happening on the system, something called TRIM comes around and erases the sectors, one second per sector, and then puts it into the free list, pre-erased so you can write them quickly.
> Think what would happen if you were running your SSD almost full, you might have to start waiting for TRIM to erase some sectors so you can write your file. One second per sector. That would suck. Don’t fill up your SSD.
> 
> 
> 
>> On 2022-September-15, at 15:13, Victor Rehorst <victor at chuma.org> wrote:
>>
>> On 2022-09-14 20:21, Dan wrote:
>>> Wow Victor!  Thank you!  I knew that the chips were NOR but I have heard the phrases used interchangeably with NAND.  A large mistake as they are very different.  I suspected as much but couldn't find the details when I looked.  Apparently I was using the wrong search terms.
>> Hey, no problem, it was a fun exercise for a Tuesday evening for me :)
>>> I did have a hunch the older, less dense chips would be a world of difference with regard to longevity.  What IS fascinating to me is the large push for the flash drives when there is a obvious longevity problem, and no one talks about this.  Sure for the moment it is fine and if you are using it not for long term storage no problem.  But consider how many devices have these chips in them which most people think 'flash is forever' if it is just sitting there which is not the case.
>>
>> Longevity is relative, and let's face it: the vast majority of consumer electronics are made to be disposable, not durable.  When it comes to NAND vs NOR, they have different use cases in computing.  NOR flash is meant to be a flexible replacement for mask ROM; both are byte-addressable, which means code can be executed directly from it, instead of being copied to RAM first. NAND flash is a replacement for hard disks: large capacity, and block-addressable.
>>
>> In terms of preservation, best practices for flash memory seem to boil down to:
>>
>> 1. Keep program/erase cycles as low as possible.
>> 2. Don't store at extremely hot temperatures.
>> 3. Power on every now and then, to refresh the charge in the flash.
>>
>> -V
>>
>> ----------------------------------------------------------------------
>>
>> http://newtontalk.net
>> http://twitter.com/newtontalk
> 
> ----------------------------------------------------------------------
> 
> http://newtontalk.net
> http://twitter.com/newtontalk
>