Jolla Tablet broken

My Jolla Tablet seems to have died.

:-(

jolla-tablet kernel: mmc0: Timeout waiting for hardware interrupt.
[...]
jolla-tablet kernel: mmc0: cache flush error -110
jolla-tablet kernel: end_request: I/O error, dev mmcblk0, sector 0
jolla-tablet kernel: mmc0: Timeout waiting for hardware interrupt.

While cleaning up the post Jolla Tablet SailfishOS upgrade from 2.0.0.10 to 4.2.0.21 I did some more flashing on my Jolla Tablet, the old flash memory is sadly now none too happy.

During one of the updates it had trouble with mmc0 and was subsequently not able to boot. Rescue luckily still was functional.

Click to see journal logging a "Timeout waiting for hardware interrupt" while I was in Sailfish OS 4.0.1.48 (Koli) trying to apply an update. :-(
Oct 25 19:58:27 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 729 of 780
Oct 25 19:58:29 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 730 of 780
Oct 25 19:58:29 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 731 of 780
Oct 25 19:58:33 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 732 of 780
Oct 25 19:58:35 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 733 of 780
Oct 25 19:58:36 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 734 of 780
Oct 25 19:58:37 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 735 of 780
Oct 25 19:58:48 jolla-tablet kernel: mmc0: Timeout waiting for hardware interrupt.
Oct 25 19:58:48 jolla-tablet kernel: sdhci: =========== REGISTER DUMP (mmc0)===========
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Sys addr: 0x00000080 | Version:  0x00001002
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Blk size: 0x00007200 | Blk cnt:  0x00000000
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Argument: 0x03200101 | Trn mode: 0x0000002b
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Present:  0x1fef0006 | Host ctl: 0x00000034
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Power:    0x0000000b | Blk gap:  0x00000080
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Wake-up:  0x00000000 | Clock:    0x00000007
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Timeout:  0x0000000e | Int stat: 0x00000000
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Int enab: 0x12ff000b | Sig enab: 0x12ff000b
Oct 25 19:58:48 jolla-tablet kernel: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Caps:     0x446cc801 | Caps_1:   0x00000807
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Cmd:      0x0000061b | Max curr: 0x00000000
Oct 25 19:58:48 jolla-tablet kernel: sdhci: Host ctl2: 0x0000008b
Oct 25 19:58:48 jolla-tablet kernel: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x7698b880
Oct 25 19:58:48 jolla-tablet kernel: sdhci: command pending, Cmdcode: 6
Oct 25 19:58:48 jolla-tablet kernel: sdhci: No data pending
Oct 25 19:58:48 jolla-tablet kernel: sdhci: pwr:     0xa | clock:   200000000
Oct 25 19:58:48 jolla-tablet kernel: sdhci: usage_count 1 | Runtime_status 0
Oct 25 19:58:48 jolla-tablet kernel: sdhci: finish_tasklet NOT start, state 0
Oct 25 19:58:48 jolla-tablet kernel: sdhci: ===========================================
Oct 25 19:58:48 jolla-tablet kernel: mmc0: cache flush error -110
Oct 25 19:58:48 jolla-tablet kernel: end_request: I/O error, dev mmcblk0, sector 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: Timeout waiting for hardware interrupt.
Oct 25 19:58:58 jolla-tablet kernel: sdhci: =========== REGISTER DUMP (mmc0)===========
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Sys addr: 0x00000018 | Version:  0x00001002
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Blk size: 0x00007200 | Blk cnt:  0x00000018
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Argument: 0x00391ce0 | Trn mode: 0x0000002b
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Present:  0x1fff0001 | Host ctl: 0x00000034
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Power:    0x0000000b | Blk gap:  0x00000080
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Wake-up:  0x00000000 | Clock:    0x00000007
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Timeout:  0x00000000 | Int stat: 0x00000000
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Int enab: 0x12ff000b | Sig enab: 0x12ff000b
Oct 25 19:58:58 jolla-tablet kernel: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Caps:     0x446cc801 | Caps_1:   0x00000807
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Cmd:      0x0000193a | Max curr: 0x00000000
Oct 25 19:58:58 jolla-tablet kernel: sdhci: Host ctl2: 0x0000008b
Oct 25 19:58:58 jolla-tablet kernel: sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x7698b800
Oct 25 19:58:58 jolla-tablet kernel: sdhci: command pending, Cmdcode: 25
Oct 25 19:58:58 jolla-tablet kernel: sdhci: data pending
Oct 25 19:58:58 jolla-tablet kernel: sdhci: pwr:     0xa | clock:   200000000
Oct 25 19:58:58 jolla-tablet kernel: sdhci: usage_count 1 | Runtime_status 0
Oct 25 19:58:58 jolla-tablet kernel: sdhci: finish_tasklet NOT start, state 0
Oct 25 19:58:58 jolla-tablet kernel: sdhci: ===========================================
Oct 25 19:58:58 jolla-tablet kernel: ------------[ cut here ]------------
Oct 25 19:58:58 jolla-tablet kernel: WARNING: at /home/abuild/rpmbuild/BUILD/linux/kernel/drivers/mmc/host/sdhci.c:1036 sdhci_send_command+0xba6/0xc70()
Oct 25 19:58:58 jolla-tablet kernel: Modules linked in: bcm_bt_lpm vxd392 gc2235f bcm4330(O) atomisp_css2400b0_v21 videobuf_vmalloc videobuf_core cfg80211 [last unloaded: bcm_bt_lpm]
Oct 25 19:58:58 jolla-tablet kernel: CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O 3.10.20+0.0.51 #1
Oct 25 19:58:58 jolla-tablet kernel: Hardware name: Intel Corporation BayTrail Platform/BayTrail , BIOS MRD8_009 X64 09/15/2015
Oct 25 19:58:58 jolla-tablet kernel:  0000000000000009 ffff880078203cc0 ffffffff82903b0d ffff880078203cf8
Oct 25 19:58:58 jolla-tablet kernel:  ffffffff82096b91 ffff8800768c2958 ffff8800768c2990 0000000000000246
Oct 25 19:58:58 jolla-tablet kernel:  ffff880076985680 ffff880076ad4000 ffff880078203d08 ffffffff82096bca
Oct 25 19:58:58 jolla-tablet kernel: Call Trace:
Oct 25 19:58:58 jolla-tablet kernel:  <IRQ>  [<ffffffff82903b0d>] dump_stack+0x19/0x1b
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff82096b91>] warn_slowpath_common+0x61/0x80
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff82096bca>] warn_slowpath_null+0x1a/0x20
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8268df46>] sdhci_send_command+0xba6/0xc70
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff826894de>] ? sdhci_reset+0xbe/0x190
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8268e132>] sdhci_finish_data+0x122/0x310
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8268e320>] ? sdhci_finish_data+0x310/0x310
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8268e400>] sdhci_timeout_timer+0xe0/0x120
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff820a77e6>] call_timer_fn+0x36/0x180
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8268e320>] ? sdhci_finish_data+0x310/0x310
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff820a7b54>] run_timer_softirq+0x224/0x2d0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8209f367>] __do_softirq+0x107/0x2c0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff829151cc>] call_softirq+0x1c/0x30
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8200455d>] do_softirq+0x6d/0xa0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8209f6cd>] irq_exit+0xcd/0xe0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff829157db>] smp_apic_timer_interrupt+0x6b/0x98
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff82914aaf>] apic_timer_interrupt+0x6f/0x80
Oct 25 19:58:58 jolla-tablet kernel:  <EOI>  [<ffffffff8266ede9>] ? poll_idle+0x49/0x90
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8267094e>] ? menu_select+0x22e/0x5a0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff820e51e3>] ? ktime_get+0x43/0xc0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8266ef39>] cpuidle_enter_state+0x49/0xe0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8266f08e>] cpuidle_idle_call+0xbe/0x290
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff8200b14e>] arch_cpu_idle+0xe/0x30
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff820e2e3e>] cpu_startup_entry+0xce/0x2d0
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff820cb40d>] ? get_parent_ip+0xd/0x50
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff828efc5f>] rest_init+0x7f/0x90
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff83225ea5>] start_kernel+0x402/0x40e
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff832258ac>] ? repair_env_string+0x5e/0x5e
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff832255e5>] x86_64_start_reservations+0x6a/0x6c
Oct 25 19:58:58 jolla-tablet kernel:  [<ffffffff832256ae>] x86_64_start_kernel+0xc7/0xca
Oct 25 19:58:58 jolla-tablet kernel: ---[ end trace f24c2c8465ce988c ]---
Oct 25 19:58:58 jolla-tablet kernel: mmcblk0: error -110 sending stop command, original cmd response 0x0, card status 0x400900
Oct 25 19:58:58 jolla-tablet kernel: mmcblk0: error -110 transferring data, sector 3742944, nr 24, cmd response 0x0, card status 0x0
Oct 25 19:58:58 jolla-tablet kernel: mmcblk0: mmc_blk_reset: md->reset_done (0x0), type (0x2)
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 2 powermode 2 cs 0 Vdd 7 width 3 timing 8
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 0Hz busmode 1 powermode 0 cs 0 Vdd 0 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 0Hz busmode 2 powermode 1 cs 0 Vdd 7 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 2 powermode 2 cs 0 Vdd 7 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 1 powermode 2 cs 0 Vdd 7 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 1 powermode 2 cs 0 Vdd 7 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 1 powermode 2 cs 1 Vdd 7 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 1 powermode 2 cs 0 Vdd 7 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 2 powermode 2 cs 0 Vdd 7 width 0 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 2 powermode 2 cs 0 Vdd 7 width 3 timing 0
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 400000Hz busmode 2 powermode 2 cs 0 Vdd 7 width 3 timing 8
Oct 25 19:58:58 jolla-tablet kernel: mmc0: clock 200000000Hz busmode 2 powermode 2 cs 0 Vdd 7 width 3 timing 8
Oct 25 19:58:58 jolla-tablet packagekitd[4808]: [packagekit-zypp] Download progress update: 736 of 780

For now I’ve put the half redone previous post back in draft mode (that mean you get a 404 at that URL), reflashed the Tablet to factory 2.0 from recovery, powered it off cleanly via refuse license during initial bringup and put it on a shelf.

Click to see log of my reset to factory state.
-----------------------------
     Jolla Recovery v2.0
-----------------------------
Welcome to the recovery tool!
The available options are:
1) Reset device to factory state
2) Reboot device
3) Shell
4) Perform file system check
5) Run sshd
6) Exit
Type the number of the desired action and press [Enter]: 
1

ALL DATA WILL BE ERASED! Clears everything from the device and reverts the
software back to factory state. This means LOSING EVERYTHING you have added to
the device (e.g. updates, apps, accounts, contacts, photos and other media).
Are you really SURE? [y/N] y
  2 logical volume(s) in volume group "sailfish" now active
[OK] No lockcode has been set. Proceeding...
  0 logical volume(s) in volume group "sailfish" now active
sh: /sys/class/block/mmcblk0p10: unknown operand
sh: /sys/class/block/mmcblk0p10: unknown operand
  0 logical volume(s) in volume group "sailfish" now active
  Logical volume "root" successfully removed
  Logical volume "home" successfully removed
  Volume group "sailfish" successfully removed
  Labels on physical volume "/dev/mmcblk0p15" successfully wiped.
  Physical volume "/dev/mmcblk0p15" successfully created.
  Volume group "sailfish" successfully created
  Logical volume "root" created.
  Logical volume "home" created.
resize2fs 1.45.4 (23-Sep-2019)
Resizing the filesystem on /dev/sailfish/root to 640000 (4k) blocks.
The filesystem on /dev/sailfish/root is now 640000 (4k) blocks long.

resize2fs 1.45.4 (23-Sep-2019)
Resizing the filesystem on /dev/sailfish/home to 6620160 (4k) blocks.
The filesystem on /dev/sailfish/home is now 6620160 (4k) blocks long.

/var/lib/platform-updates/flash-bootimg.sh ... 
Flashing to mmcblk0p3
2573+1 records in
2573+1 records out
10541056 bytes (11 MB) copied, 1.17566 s, 9.0 MB/s
OK
/var/lib/platform-updates/flash-esp.sh ... 
Flashing to mmcblk0p1
16384+0 records in
16384+0 records out
67108864 bytes (67 MB) copied, 7.56737 s, 8.9 MB/s
OK
/var/lib/platform-updates/flash-fastboot.sh ... 
Flashing to mmcblk0p5
2929+1 records in
2929+1 records out
11999232 bytes (12 MB) copied, 1.40091 s, 8.6 MB/s
OK
/var/lib/platform-updates/flash-recoveryimg.sh ... 
Flashing to mmcblk0p4
2969+1 records in
2969+1 records out
12163072 bytes (12 MB) copied, 1.34949 s, 9.0 MB/s
OK


[DONE] DEVICE RECOVERED!


[NOTE]: please note that since the device will now reboot, after you press 
[Enter], this connection will be interrupted and you won't be able to interact 
with this screen anymore.
Press [Enter] to reboot the device...
Rebooting...

Steps to Upgrade

These were my steps to upgrade

  1. must read the explanations at https://openrepos.net/content/olf/sfos-upgrade
  2. must read https://jolla.zendesk.com/hc/en-us/articles/201836347, especially the section about Stop Releases
  3. recommended: read https://gitlab.com/Phoen7x/sailfishX/-/blob/master/README.md
  4. factory reset (because I bought mine secondhand)
  5. enable developer mode and ssh access (because I always use that on SFOS)
  6. create .ssh/authorized_keys for nemo and optionally root (because I always disable PasswordAuthentication)
  7. optional: shrink /home and grow / if, like me, you prefer 4GiB / rather than the default 2
  8. determine what latest release is at https://coderus.openrepos.net/whitesoft/sailversion
  9. get latest sfos-upgrade RPM from https://openrepos.net/content/olf/sfos-upgrade
  10. use sfos-upgrade to apply all Stop Releases one by one and finally get to 4.2.0.21 (latest as of 2021-10-20)