How can microSD flash memory wear be minimized?


(Sandgroper) #21

You you can setup the Pi to boot from USB. A (minor) downside of this is that it is a one time decision. Once you have selected boot from USB it is not possible to select boot from SD.


(Amedee) #22

I am not sure it is really the case. It changes the boot order, but it will still consider SD.
This boot from USB thing does not work reliably for all devices, and I have RPi’s where I “blew the fuse” that I now use to boot from SD (with just /boot on it, the rest of the OS being on USB drive).


(Ryan Walmsley) #23

You can revert the decision indeed. I’ts more likely that it’d just be a hassle to change the same contents over.


(Sandgroper) #24

It does not appear from the Raspberry Pi documentation that you can revert this decision:

Program USB Boot Mode
Before a Raspberry Pi 3 will boot from a mass storage device, it needs to be booted from an SD card with a config option to enable USB boot mode. This will set a bit in the OTP (One Time Programmable) memory in the Raspberry Pi SoC that will enable booting from a USB mass storage device. Once this bit has been set, the SD card is no longer required. Note that any change you make to the OTP is permanent and cannot be undone.


(Ryan Walmsley) #25

Indeed I was wrong! https://twitter.com/gsholling/status/1021461011864449024

I was thinking of the older Models where you’d still have an SD card in it which told it to use the USB Drive for the files.


(Amedee) #26

Believe me, been there, done that: you can still boot from SD after – but obviously USB becomes the primary boot so if it is bootable it will boot from there; but if not it will still boot from SD.

The problem with this method is that they don’t have enough space for a full blown boot loader, so it is not a guaranteed method. For most of the hardware I have (spinning drives and SSDs) it does not work reliably, so I now always use the old method with a small SD-Card just for the boot.
But you might be more lucky or you could buy hardware known to work – I just have plenty of scavenged drives that I am happy to reuse with my RPi’s…

For the ones who don’t believe:

root@rpi-dev:~# # The following shows USB boot is enabled
root@rpi-dev:~# vcgencmd otp_dump | grep 17:
17:3020000a
root@rpi-dev:~# # My Boot is still on SD-Card:
root@rpi-dev:~# df -h /boot
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p7   65M   21M   45M  33% /boot

(It is true that the change cannot be undone, but it doesn’t mean you can’t boot from SD anymore)


(Bobbie Mac) #27

MicroSD cards wear out when used in place of RAM or instead of an SSD. SSDs have wear-leveling controllers. Transcend makes SD (not micro) cards that fit into MacBooks. These have wear leveling. Memory cards and USB sticks wear out based on the number of writes. A few hundred writes is enough to cause them to fail. I doubt Transcend will make micro cards with wear leveling as extra chips and assembly are needed and probably don’t fit on a micro sized card very easily, but contacting them may help.


#28

really ? that’s not my experience with USB sticks.
did you 'contact them ? and if not… I don’t understand your first post on this board.


(Jens) #29

Hello

Is this the right categorie?
The right Toppic?

Very interestening theme!!! But only for RasPi an microSD`s???

Most SBC`s uses massstorage like flashtechnoligies. Not only SD. USB, SD, eMMC, SSD and so on.
So the right category was Hardware an the right Toppic “How can flash memory wear be minimized”.

So, when we know what SBC, what massstorage, what controller for that,… we must ask for: What FS (File System) we want to use.

Most SBC``s use LINUX as OS. There are many FS`s possible. Fat, FAT16, FAT32, NTFS, ext3, ext4, BTTRFS… and so on. But what is the best FS for that problem of wear on flashdevices? That is the question.

I read a little bit in the web. What do you mean about: F2FS?
F2FS ( Flash-Friendly File System ) is a flash file system initially developed by Samsung Electronics for the Linux kernel.

Have a nice day :slight_smile:


(Mikael Falkvidd) #30

I’ve been using f2fs on my Raspberry Pis for more than a year. I don’t believe in anecdotal ”evidence” but f2fs is designed for flash memory without (sufficient) built-in wear leveling so it should be a good alternative.


(Sandgroper) #31

SD , microSD, miniSD cards and SSDs all implement wear leveling. The comment about SD or uSD cards failing after a few 100 write is incorrect.


#32

Both yes. The topic / question is specifically targeted at microSD flash cards in Raspberry Pi’s - the SBC’s often used for DIY TTN gateways.

No it was not.
While preventing flash memory wear is also of interest in a broader sense than microSD cards in Raspberry Pi’s, that is not what this topic is meant for.
Also, there exists no general ‘Hardware’ category. It only exists as a subcategory of ‘End Devices (Nodes)’.


#33

When USB Boot OTP option is enabled the SD is still the primary boot device.
So in case both a bootable SD and bootable USB device are inserted, the Pi will boot from the SD.
(Unless other specific OTP boot options have explicitly been enabled.)

Note: The USB Boot OTP (one time programmable) option is supported on Rasberry Pi 3 only.


(Bobbie Mac) #34

Sorry, but most basic SD and MicroSD cards DO NOT have wear leveling. SSDs and eMMC memory do have wear leveling. Get into the Raspberry Pi forums and start looking at the topics. All kinds of problems with MicroSD cards failing. Why? Because they don’t have wear leveling. Go look things up on useful websites before publishing something that will cause other people problems.


(Sandgroper) #35

You should not trust everything you read on a hobbyist’s forum. I have not checked every supplier of SD cards. All Sandisk, Toshiba and Kingston microSD cards support wear leveling. It is on their respective web sites. It took me less than 5 minutes to check these three companies.

Your comment about SD cards failing after a few hundred writes also is completely wrong. There is no need for me to research this because it is obvious. If it were true the file allocation tables alone would result in the cards dying within days as these tables are updated every time you modify any file on the media.


(Tim Everitt) #36

I agree with @Sandgroper.

I have a number of RPi systems doing continuous per 1-minute data logging into small SQLite3 databases on the microSD cards. Old data is deleted automatically on a rolling 7-day cycle. Filesystem is regular Linux ext4. I am always careful to disable the paging file to avoid excessive writing to the microSD card.

I always purchase major brand microSD cards, usually SANDisk, and always purchase cards that are much bigger than I need, e.g. 32 or 64 GBytes. I believe that this provides both wear-levelling and 10x or 20x the space I really need to provide space for the wear-levelling to work.

No card failures yet, some systems have been running for 2+ years.


#37

Good idea to use a larger card for minimizing wear. A bigger card however adds a substantial increase to the total price, until recently at least.
Currently SSD’s can be had for less than €25 (Kingston A400 SATA 120GB) which are a better alternative but they make the RPi more bulky (and apparently no RPi cases exist for that yet).

Can that be done without any serious impact on the operation of the RPi?
Any side effects?

FYI, I experienced a Samsung Evo 16GB (major brand) card in RPi that got corrupted/damaged during normal operation.


(Amedee) #38

Yes and no…

Swap was important when memory was scarce. It tend to be a problem from the past.
So if you have “enough” memory you don’t need swap.

If you don’t have a swap partition (or you have one and it is full), then the kernel will kill memory hungry processes to survive, which is usually bad. On the other hand if you have to swap on a device like and RPi, the performance will go through the toilet, so it won’t help either.

On a gateway, memory should not be an issue, so you should be better without swap.

If you absolutely want to play safe, you could have a swap, but reduce the Swappiness to its minimum so that it will only swap when there are no other options.

(I personally run without swap on all my RPis)


(Tim Everitt) #39

Hi @bluejedi, thanks for your post.

I’d like to point out that the RPi deployments that I do are in a commercial/industrial environment so I have to cost in the time of a skilled person to do complex per-device changes during provisioning and the cost for a technician to visit a failed system. The “substantial increase” to the total price of a much larger microSD card is actually - to me - a substantial cost reduction as I can use regular Raspbian Lite OS with only simple scripted firstboot build config changes. A few minutes of skilled time is needed at provisioning to set credentials, etc. and I get long-term reliability so reduced technician time.

I have worked in the offshore oil industry for 30+ years and if a small capex cost increase at the start can reduce long-term opex cost then it is always done. This capex/opex trading is very normal in commercial/industrial environments. RPi + Raspbian Lite is a great combination in this environment as it is now deployed in huge numbers and therefore very well proven and reliable as well as having the superb systemd and watchdog capabilities to maintain application and OS availability.

Regarding disabling paging/swap (/sbin/dphys-swapfile swapoff in /etc/rc.local) there appear to be no side-effects as long as everything will run within physical memory so system is configured and tuned for embedded / no-graphics use.


(Jens) #40

Hello. :slight_smile:

Check! aaaaaaaaannd Check! Here we go on RasPi with SDcard`s.

  1. Can we speak about of a basic configuration? Means:
  • RasPi 3 / RasPi 3+ or RasPi Zero That are the modells, they are current now 2018.
  • Raspian light (Stretch) for OS
  • a SDcardsize = or > 8GB / CLASS10 from a named producer like SANDISK or eg

I found another description to convert the FS for the RasPi to F2FS, you find it here.

A statetment from there

Basically, F2FS was a file system designed from the ground up for NAND-based devices. Motorola even started using F2FS for their Android smartphones. It’s designed to help reduce wear on the device, and improve performance on this type of storage medium. Since the Raspberry Pi was designed to run off of an SD card, it makes it a perfect candidate to play around with F2FS. It’s currently supported in the 4.4.y kernel so no need to compile your own kernel this time.

So why doesen`t use it??? for our DIY-GWs?

Some question that not clear for me:

  • Swap? yes or no?

Ok, @Amedee give a good input for that. Thank you!

  • Put /temp, /log or eq into RAM?

In some older contribtions, 2015 or so, some posters/bloggers mean to not get the full capacity from the blockdevice for the FS. They should be a “sparearea”? at the end. The size should be ~10% from the capacity of the device to give the controller the chance to outsource buggy/defektive cells.

  • Is this right?

@cultsdotelecomatgmai Is that what you mean about space for wearleveling?

Later in the description the author wrote, about the entry for the fstab (discard):

If your device is does not support TRIM, you can remove the discard option. Typically, most SD cards/eMMC/SSD will support this command, while USB flash drives may not. You’ll know during the mkfs.f2fs process as it will tell you whether your device does or not.

  • Is this right?

Hopfully get the anwers.

Greets and have a nice day.