75 replies to this topic

[betacentauri]

Hi together,

 

I took gutemines "challenge" and have build a new binary, which can flash images while the box is running. Sources and binary are attached. There's currently no python plugin. Perhaps somebody wants to build one...

 

To use it, copy ofg_bin.zip to the box and unzip it. Usage is quite similar to gutemines binary:

root@et9x00:~# ./ofgwrite 

ofgwrite Utility v0.1
Author: Betacentauri
Based upon: mtd-utils-native-1.4.9
Use at your own risk! Make always a backup before use!

Usage: ofgwrite <parameter> <image_directory>
Options:
   -k --kernel  flash kernel (default)
   -r --rootfs  flash root (default)
   -n --nowrite show only found image and mtd partitions (no write)
   -h --help    show help

I have tested it with my et-9200. I expect that it works with all other et-xx00 boxes and the XP1000. Afaik on VU+ boxes it currently don't run, because of other filenames.

 

WARNING:

• Before use, please make a backup on a usb stick, to be able to flash it, if something goes wrong.
• Don't flash rootfs while a recording is running. It will be interrupted!
• Flashing rootfs stops enigma2 and reboots the box after flashing.

 

Technical things:

• It's a static binary which uses the normal flash_erase, nandwrite, ubiformat sources.
• I had to modify ubiformat a little bit to get it running without ubidetach.
• pivot_root is currently not included. It's quite difficult to test it without using single user mode (init 1). And I can't use single user mode, because it stops remote connection and I have no null modem cable (yet).

Attached Files

•  ofg_bin.zip   303.07KB   45 downloads
•  ofg_src.zip   94.41KB   34 downloads

[Lost in Space]

You are welcome - and it was not a challange, I was just telling you to find out yourself as all the sourcecode pieces are availabe and also the infos how to call them :-)

 

But you should consider using klibc, as the patches for the mtd-utils for this are also available and the resulting static binary will be significantly smaller.

 

And as soon as a static binary is run it is completely in Memory so there is no real need for a pivot root or chroot.

Edited by gutemine, 1 June 2013 - 14:29.

[betacentauri]

Thanks! It was a little bit more than only putting the sources together. I only say using ubiformat without detach...

 

Well, 705k is not quite small, but why make it smaller? If there's really somebody who has problems with flash size, then he can put the binary on the harddisk or somewhere else.

...well, I want to learn new things, so perhaps I'll give it a try. And pivot_root, too.

 

 

And test results are welcome

Edited by betacentauri, 1 June 2013 - 14:51.

[WTE]

Test on XP1000 and works good.

Don't forget to install MTD package "opkg install mtd-utils"

 

 

root@xp1000:/usr/bin# ofgwrite /media/USBSTICK/xp1000/

ofgwrite Utility v0.1
Author: Betacentauri
Based upon: mtd-utils-native-1.4.9
Use at your own risk! Make always a backup before use!

Found kernel file: /media/USBSTICK/xp1000/kernel.bin
Found rootfs file: /media/USBSTICK/xp1000/rootfs.bin

Found /proc/mtd entries:
Device:   Size:     Erasesize:  Name:
/dev/mtd1 00600000  00020000    "kernel"  ->  /media/USBSTICK/xp1000/kernel.bin
/dev/mtd2 07a00000  00020000    "rootfs"  ->  /media/USBSTICK/xp1000/rootfs.bin

Erasing kernel: flash_erase /dev/mtd1 0 0
Erasing 128 Kibyte @ 5e0000 -- 100 % complete
Flashing kernel: nandwrite -pm /dev/mtd1 /media/USBSTICK/xp1000/kernel.bin
Writing data to block 0 at offset 0x0
Writing data to block 1 at offset 0x20000
Writing data to block 2 at offset 0x40000
Writing data to block 3 at offset 0x60000
Writing data to block 4 at offset 0x80000
Writing data to block 5 at offset 0xa0000
Writing data to block 6 at offset 0xc0000
Writing data to block 7 at offset 0xe0000
Writing data to block 8 at offset 0x100000
Writing data to block 9 at offset 0x120000
Writing data to block 10 at offset 0x140000
Writing data to block 11 at offset 0x160000
Writing data to block 12 at offset 0x180000
Writing data to block 13 at offset 0x1a0000
Writing data to block 14 at offset 0x1c0000
Writing data to block 15 at offset 0x1e0000
Writing data to block 16 at offset 0x200000
Writing data to block 17 at offset 0x220000
Writing data to block 18 at offset 0x240000
Writing data to block 19 at offset 0x260000
Writing data to block 20 at offset 0x280000
Writing data to block 21 at offset 0x2a0000
Writing data to block 22 at offset 0x2c0000
Writing data to block 23 at offset 0x2e0000
Writing data to block 24 at offset 0x300000
Writing data to block 25 at offset 0x320000
Writing data to block 26 at offset 0x340000
Writing data to block 27 at offset 0x360000
Writing data to block 28 at offset 0x380000
Writing data to block 29 at offset 0x3a0000
Writing data to block 30 at offset 0x3c0000
Writing data to block 31 at offset 0x3e0000
Switching to user mode 2
Syncing filesystem
Remounting root read-only
Erasing rootfs: flash_erase /dev/mtd2 0 0
Erasing 128 Kibyte @ 7960000 -- 99 % complete flash_erase: Skipping bad block at 07980000
flash_erase: Skipping bad block at 079a0000
flash_erase: Skipping bad block at 079c0000
flash_erase: Skipping bad block at 079e0000
Erasing 128 Kibyte @ 79e0000 -- 100 % complete
Flashing rootfs: ubiformat /dev/mtd2 -f /media/USBSTICK/xp1000/rootfs.bin
ubiformat: mtd2 (nand), size 127926272 bytes (122.0 MiB), 976 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 2048 bytes
libscan: scanning eraseblock 975 -- 100 % complete
ubiformat: 972 eraseblocks are supposedly empty
ubiformat: 4 bad eraseblocks found, numbers: 972, 973, 974, 975
ubiformat: flashing eraseblock 416 -- 100 % complete
ubiformat: formatting eraseblock 975 -- 100 % complete
Successfully flashed rootfs! Rebooting in 5 seconds...

Edited by WTE, 1 June 2013 - 15:28.

[betacentauri]

Hi WTE,

 

thanks for the test!

 

You don't need to install mtd-utils! It's build-in in the binary. I only print the commands to console, to show what I make. I heard you have some coding skills, right? Then look into the sources. I have renamed the original main functions and call them now from ofgwrite.c (e.g. flash_erase_main(argc, argv) )

Edited by betacentauri, 1 June 2013 - 15:48.

[pieterg]

There is one good reason to use pivot_root, which is if you want to replace just the rootfs image in the ubi layer, instead of formatting the entire ubi layer.
If users keep multiple rootfs images in their ubi (for multiboot), and perhaps config backups and such things, they would loose everything if you wipe out their entire ubi layer.

[betacentauri]

If this is possible, then you're right and I should add a warning for that.

 

I try to get pivot_root working, but as said before it's not so easy to use it, because all processes which access old root needs to be stopped. On monday comes my usb->serial adapter and I can hopefully test a little bit more. But I fear without help I don't get it running, because I don't know how multiboot works.

[WTE]

Hi WTE,

 

thanks for the test!

 

You don't need to install mtd-utils! It's build-in in the binary. I only print the commands to console, to show what I make. I heard you have some coding skills, right? Then look into the sources. I have renamed the original main functions and call them now from ofgwrite.c (e.g. flash_erase_main(argc, argv) )

 

I need otherwise I get error on ubiformat

[betacentauri]

Really? Do you know which error you got?

It's a static binary, so no libs are needed...

root@et9x00:/media/hdd# ./ldd ofgwrite 
$       not a dynamic executable
root@et9x00:/media/hdd# 

[pieterg]

If this is possible, then you're right and I should add a warning for that.

I don't think any image makers are using this at the moment, so it would only affect powerusers who are using their ubi in a nonstandard way. So nothing to worry about for now I guess, this type of users will not use a custom binary or plugin to flash their image.

But this is how ubi should be used (and will be used in the future), replace individual images, instead of re-formatting the entire layer.

[pieterg]

btw, good to see the sourcecode, that makes me trust you with my stb's

[betacentauri]

Hi WTE,

 

thanks for the test!

 

You don't need to install mtd-utils! It's build-in in the binary. I only print the commands to console, to show what I make. I heard you have some coding skills, right? Then look into the sources. I have renamed the original main functions and call them now from ofgwrite.c (e.g. flash_erase_main(argc, argv) )

 

I need otherwise I get error on ubiformat

 

 

Hi WTE,

 

strange. I have deinstalled mtd-utils package and my binary still runs without error.

[Lost in Space]

Look at the code of ubirename on how to rename an ubivolume, and all these nice littel tools use libtmd routines, that's why i told you to start reading there.

 

Ubifs even has a smart on the fly update routine (described in their FAQ), but as long as you only have a single volume in the image it doesn't matter if you re-do the image with uniformat.

 

The same functionality with klibc will be less then 100k as you can see from the fgwrite binary.

 

Just another hint - if you are already booted in a chroot (as Barry Allen or MeoBoot can do nicely for you) or have the root filesystem running from USB then your can debug such things easily even without beeing in single user mode.

 

And off course it is not just chicken soup to do it, I said it takes about a day of work and maybe even more dpending on skill and understanding, even I could only do it in shorter time  as I had all these binaries already wrapped as callable routines and more knowledge where to tweak them.

Edited by gutemine, 1 June 2013 - 17:15.

[pieterg]

yes, we all know how good you are

[betacentauri]

I don't think any image makers are using this at the moment

Good to know. But a warning would also not hurt...

, so it would only affect powerusers who are using their ubi in a nonstandard way. So nothing to worry about for now I guess, this type of users will not use a custom binary or plugin to flash their image.

But this is how ubi should be used (and will be used in the future), replace individual images, instead of re-formatting the entire layer.

I just read some articles about ubi. You mean multiple ubi volumes, each containing a rootfs? Then ubiupdatevol should do the work, right?

[pieterg]

I just read some articles about ubi. You mean multiple ubi volumes, each containing a rootfs? Then ubiupdatevol should do the work, right?

indeed.
Using multiple rootfs images would of course need support from the bootloader, but a more common use, what I do for instance is put config backups in my ubi layer.

[Lost in Space]

I is not the question of beeing good or bad, but if I say it is not diffcult then I'm not telling anybody nonsense or send him on an impossible journey.

 

And yes, the upiupdatevol is a possible solution, but if the update fails you stil lost the old image. If you add a new volume let's say newrootfs and unpack the new image to it then you can rename the old one to oldrootfs and rename the new one to rootfs and only if all this succeeds you delete the oldrootfs volume. Then you really have an almost failsave update path on the fly, you only need temporary the space for the rootfs twice, but with modern flashsizes this should not be a real problem. Alternatively you also can use different partitions and have a very simple initramfs deciding which one to mount and boot. So there are lots of nicer ways to upgrade your boxes only somebody needs to try them out and implement them.

 

And the kernel already now can mount the volume you specify on the kernel command line, so there is no need to change the bootloader as this only loads the kernel and starts it with the kernel command line you specify  - Dreambox 7020hd already now has a second data volume and you could also add a second rootfs.

Edited by gutemine, 1 June 2013 - 17:29.

[pieterg]

nobody is saying it is difficult

[Lost in Space]

Then don't tell him that a loder change is needed to boot another rootfs in ubi layer :-)

 

With block2mtd you can even boot the whole ubi layer from an USB or sata device with existing kernel command line options and existing loaders.

Edited by gutemine, 1 June 2013 - 17:32.

[betacentauri]

@gutemine: And I had nothing and also did it in 1-2 days. And I'm not very experienced with Makefiles, bitbake and so on...
The biggest problem was to find the "bug" with the getopt_long calls. It cannot be called multiple times without resetting optind...

And as said before, I'm interested in learning such things.

And I'm (currently) not interested in building a new multiboot mechanism...

And yes, the upiupdatevol is a possible solution, but if the update fails you stil lost the old image.

Yes, and it's now (with ubiformat) exactly the same.