61

I get an I/O error at boot and then get this 'error': (I am running the latest version of Raspbian)

Entering kdb (current=0xca82ac80, pid 1) due to keyboard entry
kdb> _

And I have seen it before, and was explained that my SD card got corrupted since I did not shutdown correctly. One week later, I didn't deal with this error, until the power flickered on/off thrice and now I can't boot again. It seems like I am re-preparing my SD card ever day!!! I don't like this, not to mention I love my Pi's capabilities. But this 'breakpoint' is driving me crazy! Is there some way I can prevent this or strengthen something so it doesn't happen if the power goes out? Any suggestions? Thanks!

P.S. My zip file was not corrupted and my extraction service is Win32DiskImager

user151324
  • 1,270
  • 4
  • 14
  • 17
  • 3
    Is your card on this list – Butters Jun 15 '13 at 02:05
  • have you ever thought about UPS ? – lenik Jun 15 '13 at 07:29
  • @lenik Are you saying I should have something like a surge protector? – user151324 Jun 15 '13 at 13:53
  • Because you power off the thing without a proper shutdown? – jippie Jun 15 '13 at 15:13
  • @jippie Ha ha! No, I always properly shutdown after that incident. – user151324 Jun 15 '13 at 15:47
  • 2
    @coding_corgi more likely something with a battery inside, that provides power for 5-10 minutes after the power outage, that allows you to ignore short power flicks and gives you time to shutdown (automatically?) your computer when power goes down for a long time. It's called "Uninterruptable Power Supply" or UPS for short. – lenik Jun 15 '13 at 23:52
  • @lenik thanks (i'll look into that), but how can I set up my SD card for something like that to happen, and not have a boot error? What causes that boot error to occur? – user151324 Jun 16 '13 at 01:07
  • I had that problem when I plugged in a USB keyboard that used to much power. I stopped using the keyboard and I did not get that any more.I can pull the power from the PI, start it pull power, start it 10 times and it does not corrupt. – Piotr Kula Jun 17 '13 at 10:17
  • @ppumkin There were lot's of hardware issues that I tracked down, but what I am aiming at is software issues. – user151324 Jun 17 '13 at 15:59
  • Look. You are having too many problems for some reason. Its not that bad. Burn Image, plug in, use. I have 2 early boards and they work just fine like that. NOt sure why you struggling. It must be a crappy SD card. – Piotr Kula Jun 17 '13 at 16:00
  • @ppumkin The hardware issues are not bad, they don't use too much power, and I will but a powered hub, but I have made my Pi work perfectly (before) after tracking down hardware issues, but I don't have hardware issues anymore, I am 99% sure of it – user151324 Jun 17 '13 at 16:02
  • This is an extremely important Pi issue. Sure, I always shut down before powering off, but how about power failures? Adapter unplugs accidentally. Power cord on adapter gets unplugged. General power outage. While a PC or laptop can be powered up again, a raspberry pi runs serious risk of system damage. This is next to unusable. – Roland Jul 13 '21 at 10:12

11 Answers11

55

I'm not going to write about checking your HW and compatible SD card lists, because you most probably have already checked all these. What I'm about to write is the permanent solution, that allows to nip the problem in the bud, and permanently fix the issue.

If you don't want your SD card to get broken when you flip the power switch, you have to use it in a read-only mode. If there's nothing being written to your SD card, it won't get damaged no matter what you do.

Obvious (but non-working) solution would be an attempt to flip "read-only" switch on the side of the SD card, unfortunately this does not work because the schematics shows this switch is routed nowhere and its position is generally ignored.

Another, more subtle (but working) approach would be modifying your /etc/fstab to mount all your partitions read-only. The drawback of this approach is your logs have to be written somewhere else (USB stick, RAM drive?) and in case of RAM drive the logs won't persist during reboot. To do this:

  1. Copy an RPi image to your SD card using any method you like.
  2. Boot from SD and raspi-config will start automatically. Do not "Expand filesystem", just set up your time zone and international settings.

  3. Run sudo fdisk /dev/mmcblk0, press 'p' to print the current partition table. Then enter the following commands:

    n        Create a new partition
[enter]  Make a primary partition
[enter]  Use the default number
[#]      1 greater than the end of mmcblk0p2
[enter]  Expand to the end of the SD card
w        Write the partition table and exit

  4. Edit /etc/fstab. It should look something like the following:

    proc            /proc     proc    defaults 0 0
/dev/mmcblk0p1  /boot     vfat    ro       0 0
/dev/mmcblk0p2  /         ext4    ro       0 0
/dev/mmcblk0p3  /home     ext4    defaults,errors=remount-ro  0  1
none            /var/run  ramfs   size=1M  0 0
none            /var/log  ramfs   size=1M  0 0

  5. Run sudo partprobe to recognize the new partition.

  6. Format your new partition with sudo mkfs --type ext4 /dev/mmcblk0p3.
  7. Reboot.

If for some reason you need to make changes to your system, you can remount the read-only partitions with write access:

sudo mount -o remount,rw /dev/mmcblk0p2
Brad
  • 107
  • 4
lenik
  • 11,541
  • 1
  • 30
  • 37
  • Wait, I can't write to my SD card if I do this? Or only at boot? – user151324 Jun 19 '13 at 02:02
  • 1
    @coding_corgi This should stop writing to the SD card entirely. – apnorton Jun 19 '13 at 02:04
  • 1
    why do you need to write to your SD card? writing to SD + power down = FS corruption, you may use USB stick or create a separate partition on SD card if you need to save some data. in this case at least your system will get up and running, and then you may check your data for possible corruption. – lenik Jun 19 '13 at 02:06
  • @anorton this stops write access to the first two partitions on SD card, but nobody says you cannot create another partition, or use different media to save the data. for example, read-only RasPi writing data over the network -- could be a very reliable solution. – lenik Jun 19 '13 at 02:07
  • @lenik Well, I really don't really think I would like to stop writing to the SD card entirely, could I have a different partion or something?! – user151324 Jun 19 '13 at 02:47
  • @coding_corgi sure, create /dev/mmcblk0p3 and leave it read-write, but it might be a good idea not to mount it automatically, but run some scripts checking if everything is fine before actually accessing it. – lenik Jun 19 '13 at 03:48
  • @lenik But I have /dev/mmcblk0p2 with isn't /dev/mmcblk0p1 which I boot off of, right? – user151324 Jun 19 '13 at 03:53
  • @coding_corgi not sure about your particular layout, but usually p1 is the boot partition, p2 is the root partition (where your system resides) and you need both for a successful system startup. due to reasons specified above, both p1 and p2 are much better off being read-only, and if you want to have some place to write your data to, you may create p3 (use fdisk or something) and keep your data there. – lenik Jun 19 '13 at 04:05
  • 1
    I write to my SD card video data from raspi cam and it died so many times! Not once did I have any issues with booting up, pulled wire out or system crash. Read only is good for production when you plug and play. This problem should not happen during development any way- I suspect even read only will mess up his data because something is happening at a higher level than kernel can handle. – Piotr Kula Jun 19 '13 at 09:26
  • @lenik Well okay, but if I have /dev/mmcblk0p3 will I have my ~ directory? So like I can write/read to it etc. ? And can I write to the / directory too? – user151324 Jun 19 '13 at 15:09
  • @coding_corgi you may mount /dev/mmcblk0p3 to /home and have your ~ directory just fine. Writing to the / defies the purpose, if the power suddenly goes down you'll have a broken system image. – lenik Jun 19 '13 at 19:48
  • @lenik Could we continue this discussion in chat ( The Bakery )? – user151324 Jun 19 '13 at 21:24
  • @lenik okay, this is The Bakery: http://chat.stackexchange.com/rooms/3748/the-bakery – user151324 Jun 19 '13 at 22:02
  • I'm in a similar situation, currently battling it out with help from one of the raspi forum mods - I'll post if anything helps. – James Snell Jun 20 '13 at 16:48
  • @lenik Could you explain to me what I have to do over chat: http://chat.stackexchange.com/rooms/3748/the-bakery – user151324 Jun 21 '13 at 00:21
  • @coding_corgi wait, you often write (i.e. create and delete files) to /??? Don't do that. It means that you will very often write to the directory block of /, the block that contains the crucial directory data to have a working filesystem -- and the memory cells on SD cards have limited write cycles. Use a subdirectory in /tmp or your home directory. In that case, you're more likely to corrupt some part of the SD card that is not used by the boot process. That way, you can still boot up the system when some blocks on the SD card died and recover at least some of the data. – orithena Jun 22 '13 at 22:20
  • @maligree Well... I seem to have put something special in this blend on my SD card for some reason I accidently unplugged my Pi! But it still booted!!! Anyway, how else can I experiment etc. ? – user151324 Jun 22 '13 at 23:37
  • @lenik Could you please show me how I can do this? – user151324 Jun 23 '13 at 01:08
  • @coding_corgi see update – lenik Jun 23 '13 at 03:42
  • @lenik can I do install say mysql and have it installed on an external drive? – user151324 Jun 23 '13 at 03:47
  • @coding_corgi binaries and libraries should go to system partition, but the actual databases where you write and read data may reside on the external drive or, as I wrote, on the extra SD card partition – lenik Jun 23 '13 at 03:49
  • @lenik so i'm supposed to install mysql on the main sd card, etc. ? – user151324 Jun 23 '13 at 03:50
  • @coding_corgi you're supposed to do sudo apt-get update && sudo apt-get upgrade and install everything you need with sudo apt-get install [package_name] before disabling write access – lenik Jun 23 '13 at 04:01
  • @lenik so I need to expand my SD card? – user151324 Jun 23 '13 at 04:09
  • What linux box do you recommend? – user151324 Jun 29 '13 at 00:21
  • linux box meant just any computer running linux. any computer, any flavour of linux. – lenik Jun 29 '13 at 03:41
  • You could also make a partition /srv and put all your server data there, like the databases for MySQL and webbrowser etc. Then you just install that rw and you are ok for servers too. (the purpose of /srv is to store server data in directories under it, liek /srv/mysql and then configure MySQL to use that instead of some directory under /var/. You could put /srv/ on a USB harddisk, USB memorystick or on another partition on SD:card like you did with /home. – Anders Sep 06 '13 at 19:30
  • @lenik what system has mkfs that supports the argument -type instead of --type? – Brad Feb 14 '14 at 21:00
  • @Brad sorry, i'm AFK ATM, but google:"man mkfs" gives this and it says about single "-type". will check later, though. thanks! – lenik Feb 16 '14 at 04:36
  • lenik: @bradjc5's edits were correct. Your link only shows the short form (-t) which is a single dash, but since you used the long form in your command (--type) you need both dashes. See a more complete man page here: http://man7.org/linux/man-pages/man8/mkfs.8.html – Pat Mar 14 '14 at 19:32
  • I have an external hardware clock and having all kind of clock issues when running RPi with read-only drives, so I started looking and found the below site with all kind of suggestions regarding mounting filesystem in read-only including adjtime and others: https://wiki.debian.org/ReadonlyRoot – Zoltan Fedor Dec 25 '14 at 16:01
  • 7
    Modern versions of raspi-config offer a similar read-only overlayFS functionality out of the box! sudo raspi-config, go to Advanced Options, then enable the Overlay FS: Enable/Disable read-only filesystem feature. (I'd add this as an answer, but I do not have enough reputation yet.) – Shane S. Jul 26 '20 at 04:53
  • What about system package updates ? You need to do them manually after setting back to read-write ? – Sébastien Rochette Sep 16 '20 at 12:07
5

Edit

The IPE home page seems to be no more accessible.

It should not be used since it seems not to be maintained anymore.

For the record, here is a link to the old IPE homepage at web.archive.org

Old post

Depending of what you use your RPi for, you may be insterested in IPE, which is a "blackout-proof flavour of Raspbian".

I plan to use it to boot my RPi. If I need data to be writen, I will use an USB drive that I'll mount readonly (I prefer my SD to be safe and corrupt an USB drive than having to repair my SD)

See the IPE homepage

As indicated there, "Also, use “ipe-rw” and “ipe-ro” to switch the root file system to writeable or read-only mode."

lauhub
  • 334
  • 3
  • 8
3

If your Raspberry Pi is frozen then you can not shutdown correctly, then this helps. I was broke my few SD cards before knew this:

Hold down both Alt+PrintScreen, and while holding those keys, hit the following keys in sequence, one at a time, with a few seconds pause between them.

Alt + PrintScreen + R E I S U O

or

Alt + PrintScreen + R E I S U B (reboot)

A handy mnemonic to remember that is, Reboot Even If System Utterly Broken.

Substitute "O" for "B" to shut down the system instead of rebooting (O=off, B=boot).

Source: https://www.raspberrypi.org/forums/viewtopic.php?t=176612#p1126262

Almas Dusal
  • 481
  • 5
  • 5
3

I have overcome this issue by using a USB flash drive for my main partitions.

  1. Restore raspbian image to a USB drive

  2. Format an SD card to fat

  3. Copy contents of fat partition from USB drive onto SD card

  4. Modify config.txt on sd card to boot from /dev/sda

Essentially the rpi will still boot from the SD card, but will use the USB drive for the os and read/writes.

Ace
  • 219
  • 1
  • 2
  • 6
3

I'm using the UPS Pico a specially designed UPS for the Raspberry Pi and never have had such problems.

Ghanima
  • 15,855
  • 15
  • 61
  • 119
Alexander
  • 71
  • 1
1

please ensure your RPi is running on the default speeds (i.e. CPU at 700mhz). personally, i encountered storage corruptions with various SD cards - but only when running over the clock. unfortunately, some distributions (images) come with questionable settings in config.txt.

jitter
  • 258
  • 1
  • 7
1

Just to cover all ground, also check if your power supply is good enough. try other charger or plug the RPI to a PC USB port to test it. Remember that the recommended RPI supply is about 700mA, but some weak chargers might not sustain this at stable levels.

I had a router that start to act crazy for some days, then went ok, then again went crazy and later fine again ... i found that the power supply was outputting changing voltage (3V to 5.5V) on the time the router was crazy. Replaced that supply with a new one all is fine. So try replace the power supply and measure the output voltage.

Finally, turn off the RPI by shutting down the OS and only unplug the RPI when you got only one red led. the RPI is not really shutdown if you have other than one red led. Also, try to avoid disconnect the rpi by removing the power supply from the plug, it's better to remove the power from the RPI and only after remote the power supply from the plug.

higuita
  • 684
  • 5
  • 6
0

I've also experienced SD Failures with my PI, sometimes 5 days in a row, sometimes it works for a month.

The only way I could get it to run reliable for more than a year now is to Boot from a USB stick. This way you don't use your SD card (only at boot) und just need about 500mb sd card (cheap) and a USB Stick.

Gotschi
  • 608
  • 2
  • 7
  • 16
0

I followed a hint that you can do sync before shutdown, and so far I had some success with this command:

sync; sudo shutdown -t 9 -r now
Ghanima
  • 15,855
  • 15
  • 61
  • 119
dirkk0
  • 101
  • 2
  • Are you still having problem with Raspberry using above solution? – CuriousMind Mar 22 '15 at 12:40
  • I don't think this will fix the issue. When you run a controlled shutdown, no damage should happen to your sd card by design. Corruption seems to happen mostly in cases in which the device is simply unplugged (or power outage occurs) which is unrelated to your sync or shutdown commands unless you run sync immediately before the (unexpected!) unplugging which seems unfeasible. – nicolaus-hee Oct 30 '15 at 04:05
0

As found on the elinux site. I only selected what applies to your situation but please visit them for more information...

  • If you have problems, check you have latest firmware version with rpi-update
  • Some SD cards do not work on the R-Pi, so check the list of known SD cards.
  • If you are having problems setting up your SD card you might want to start by erasing it completely - especially if it has been used elsewhere and still contains data / partitions. It is good to do a FULL (low level) format any way as it can find bad sectors sometimes and add them to an ignore list.
    • Windows and Mac users can download a formatting tool from the SD Association: https://www.sdcard.org/downloads/formatter_3/
    • Reformatting cards is also easy to do in a digital camera.
    • If you are manually preparing your SD card on Linux or Mac OS using the dd command, this operation will completely erase any existing data and partitions. Make sure you write to the whole card (e.g. /dev/sdd) and not to an existing partition (e.g. /dev/sdd1).
  • If you have an sdcard that doesn't work with latest firmware, head over here.
  • Make sure you have a good power supply. Try and unplug everything and see how long it lasts. Plug items back in one by one testing the stability.

If you have followed all those steps then the only possible thing to try is use another SD Card carefully selected from the compatible list. Also make sure to buy original card from a trusted place. Not like ebay china or something.

Piotr Kula
  • 17,307
  • 6
  • 65
  • 104
  • I used 3 sd cards in the process (all supported) a SanDisk, a Kingston and a Lexar, I am not manually setting up my SD card either, I used the Win32DiskImager after trying on Ubuntu and Mac OS X. – user151324 Jun 17 '13 at 16:05
  • Its wierd that making your fs read only fixed it. You out of a millions of people has this issue. Maybe you got some rubbish/virus messing up your filesystem. Readonly root is a pain in the ass... wait and see. – Piotr Kula Jun 24 '13 at 07:40
  • thanks, something weird is going on, definetly not a virus though... – user151324 Jun 24 '13 at 13:26
0

I have similar problems on a pi from RS Components. This is one of the pis made on China. Another one made in UK from Element 14 never had corruption with the same sdcard and powersupply. Might be some Hardwareflaw with the ra components units. Not sure of that of course, but everything seems to point in that direction.

mcobit
  • 1