n900 findings

December 11, 2011 by invalidmagic

repairing the n900 camera

i own a nokia n900 and i’m very happy with it. some time ago it fell of my pocket and then the 5mega pixel camera didn’t work anymore as well as the flashlight. i’m a big fan of QR codes and i was often using the flashlight so this was a problem.

the funny thing is that reflashing the device [2] made it work again. now i wonder why the camera is working again. the camera module is plugged to the n900 as can be seen in the videos on youtube, just search for it [3]. maybe while hitting the ground the camera was disconnected from the device while the device was running and that screwed the firmware? i don’t have a clue but i think it is worth mentioning.

0. symptoms

once the camera was not working i had problems:

starting the n900 (coldstart) if still connected to any charger, it simply would not boot
starting the camera application; it would report something like: camera device not ready or something similar and instantly close the widget
flashlight did not work anymore

1. the backup

1.1 create a backup using the backup tool coming with the n900

this is a really great tool, don’t forget to check all the 4 points in the list ‘what to backup’.

also consider to read [4] (i didn’t do that but on this page there are some screenshots of the backup util i’m referring to).

1.2 dpkg and a list of all files in ‘opt’

maybe that is of use later so i note down all files and installed packages:

ls -la opt > opt_fils

dpkg -l > dpkg_l

1.3 now make a backup of all files

as i also develop for the n900 i had already a working ssh root login:

rsync -av –progress –delete n900:/home/user .

2. eMMC/VANILLA

RX-51_2009SE_10.2010.13-2.VANILLA_PR_EMMC_MR0_ARM.bin

./flasher-3.5 -F ../RX-51_2009SE_10.2010.13-2.VANILLA_PR_EMMC_MR0_ARM.bin -f

3. FIASCO/Rootfs

RX-51_2009SE_20.2010.36-2_PR_COMBINED_MR0_ARM.bin

./flasher-3.5 -F ../RX-51_2009SE_20.2010.36-2_PR_COMBINED_MR0_ARM.bin -f -R

4. restoring the backup

once the device was booted after the udpate the camera was working. what a relief! but i didn’t know where to copy the backup as the folder using the ‘mass storage usb’ feature was nearly empty. so i created a new one and connected it again using usb later. after the backup directory was migrated to the device i disconnected the usb cable and restored my 50mb backup. took quite some time and after the final reboot the device also installs the software which was removed in step 2 and 3. i decided to skip this and to install only needed software manually.

note: after the backup is restored and the n900 restarted the wlan starts working again.

5. restoring the applications

a list of software i installed:

gainroot
openssh client and server

one these applications are installed plug in the usb adapter and set the device into ‘pc suite mode’ and use gainroot to type:

ifconfig usb0 10.0.0.1/24 up

on the host computer type:

ip a add 10.0.0.2/24 dev usb0

ip l set usb0 up

ssh root@10.0.0.1

to find applications use the app catalog on the n900 but to find the associated package i did:

apt-cache search adblock | grep plus

(and similar commands)

all other applications can be installed using apt-get via usb or wlan:

rsync
adblock plus
adflashblock-css
agenda timer
battery-eye
countdown timer
datetoday home desktop widget
evopedia
flashlight
convert
gpsjinni
headphone daemon
healthcheck
htop
iptraf
irssi
mad-developer
mappero
mbarcode
mbarcode qr code plugin
mbarcode webrequest plugin
mbarcode wifi plugin
mclock
mstardict
nmap
ogg support
orrery
password safe
personal ip address
quicknote
recaller
tuner
view contact info with bigger font
wifieye
wifi switcher
foreca weather applet

using the ssh shell one can type this on the device as root:

apt-get install adblock-plus-1.0
apt-get install adflashblock-css agenda rsync
apt-get install battery-eye wifi-switcher countdowntimer datetoday-home-widget evopedia flashlight-applet
apt-get install gpsjinni headphoned healthcheck htop network-monitor irssi mad-developer
apt-get install maemo-mapper mbarcode-plugin-qrcode mbarcode-plugin-wifi mbarcode-plugin-webrequest mbarcode mbarcode-core
apt-get install mclock mstardict nmap ogg-support decoders-support orrery pwsafe
apt-get install personal-ip-address quicknote recaller tuner contact-zoom wifieye foreca-weather-applet

other software i newly installed

monav
snuggle
wallet
mdbus2

again – using the ssh shell one can type this on the device as root:

apt-get install monav-routing-daemon snuggle qtwallet mdbus2

6. remove the photo click sound and the intro animation

ssh into the n900 and type:

rm ./opt/usr/share/hildon-welcome/media/Hands-v32-h264.avi ./home/opt/usr/share/hildon-welcome/media/Hands-v32-h264.avi

rm /usr/share/sounds/camera_snd_title_*wav

reboot

random notes

after the reinstallation everything is much faster now. the settings dialog loads twice as fast and the desktop feels much smoother.
application load times are similar to before
for some programs i had to copy files from my backup to the device manually:

my documents
evopedia dumps
monav map data
my camera pictures
osm2go
mstardict
music
quicknote

monav is probably one of the coolest programs for the n900, thanks very much to Christian Vetter and Christoph Eckert

kdevelop on nixos

issue 1

i’ve adapted to a nixos only desktop and i’m facing new problems every day, lately i wanted to use kdevelop but it wasn’t able to import a normal ‘Makefile’based project. next i installed lots of packages somehow mentioned at [1]. still that alone didn’t help so i ran:

kbuildsycoca4

kbuildsycoca4 is a utility which generates a cache of .desktop files and such to be used by programs. PovAddict#kde-devel@irc.freenode.net recommended to set KDEDIRS correctly because when doing so running kbuildsycoca4 manually wouldn’t be neccessary.

not sure how to solve this problem correctly yet but in case i find out i will add it to this post.

issue 2

another point which bugs me is that auto-completion doesn’t work correctly as there is no such thing as /usr/include on nixos as one might expect when coming from a different linux distribution. didn’t find a solution here yet either…

how to integrate a daemon into nixos

in this short article i want to show how services can be added/used in nixos. i personally find the nixos way a quite easy and ‘clean’ approach!

as i integrated cntlm my goals were:

easy to use
password must be stored in a safe place
service should not be run as root but as a special user: cntlm

it helped a lot to look at similar scripts like the sshd integration but i also used “NixOS: A Purely functional Linux Distribution” [1] which is describing most aspects which are needed to get a service up and running.

seen as a developer one has to write two nix expressions:

cntlm/default.nix
the script which describes the software cntlm
nixos/modules/services/networking/cntlm.nix
the script which descirbes the service

seen as a user one has to modify only one nix expression:

/etc/nixos/configuration.nix
the place where all the configuration happens

/etc/nixos/nixpkgs/pkgs/tools/networking/cntlm/default.nix

this nix expression describes the cntlm software and is quite simple as it basically fetches the software/compiles it/installs it into the system. however, a user does not have to install this software using:

nix-env -i cntlm

still this would be possible, and a normal user/root user could use the software in his profile this way.

anyway here is the script:

     1  { stdenv, fetchurl, which}:
     2
     3  stdenv.mkDerivation {
     4    name = "cntlm-0.35.1";
     5
     6    src = fetchurl {
     7      url = mirror://sourceforge/cntlm/cntlm-0.35.1.tar.gz;
     8      sha256 = "7b3fb7184e72cc3f1743bb8e503a5305e96458bc630a7e1ebfc9f3c07ffa6c5e";
     9    };
    10
    11    buildInputs = [ which ];
    12
    13    installPhase = ''
    14      ensureDir $out/bin; cp cntlm $out/bin/;
    15      ensureDir $out/share/; cp COPYRIGHT README VERSION doc/cntlm.conf $out/share/;
    16      ensureDir $out/man/; cp doc/cntlm.1 $out/man/;
    17    '';
    18
    19    meta = {
    20      description = "Cntlm is an NTLM/NTLMv2 authenticating HTTP proxy";
    21      homepage = http://cntlm.sourceforge.net/;
    22      license = stdenv.lib.licenses.gpl2;
    23      maintainers = [ stdenv.lib.maintainers.qknight ];
    24    };
    25  }

the only point of interest might be the buildInputs (line 11) which includes which. in this build script ‘which gcc’ is used to test if gcc is installed.

/etc/nixos/nixos/modules/services/networking/cntlm.nix

this expression is used to integrate cntlm as a system service.

     1  { config, pkgs, ... }:
     2
     3  with pkgs.lib;
     4
     5  let
     6
     7    cfg = config.services.cntlm;
     8    uid = config.ids.uids.cntlm;
     9
    10  in
    11
    12  {
    13
    14    options = {
    15
    16      services.cntlm= {
    17
    18        enable = mkOption {
    19          default = false;
    20          description = ''
    21            Whether to enable the cntlm, which start a local proxy.
    22          '';
    23        };
    24
    25        username = mkOption {
    26          description = ''
    27            Proxy account name, without the possibility to include domain name ('at' sign is interpreted literally).
    28          '';
    29        };
    30
    31        domain = mkOption {
    32          description = ''Proxy account domain/workgroup name.'';
    33        };
    34
    35        password = mkOption {
    36          default = "/etc/cntlm.password";
    37          type = with pkgs.lib.types; string;
    38          description = ''Proxy account password. Note: use chmod 0600 on /etc/cntlm.password for security.'';
    39        };
    40
    41        netbios_hostname = mkOption {
    42          default = config.networking.hostName;
    43          description = ''
    44            The hostname of your workstation.
    45          '';
    46        };
    47
    48        proxy = mkOption {
    49          description = ''
    50            A list of NTLM/NTLMv2 authenticating HTTP proxies.
    51
    52            Parent proxy, which requires authentication. The same as proxy on the command-line, can be used more than  once  to  specify  unlimited
    53            number  of  proxies.  Should  one proxy fail, cntlm automatically moves on to the next one. The connect request fails only if the whole
    54            list of proxies is scanned and (for each request) and found to be invalid. Command-line takes precedence over the configuration file.
    55          '';
    56        };
    57
    58        port = mkOption {
    59          default = [3128];
    60          description = "Specifies on which ports the cntlm daemon listens.";
    61        };
    62
    63       extraConfig = mkOption {
    64          default = "";
    65          description = "Verbatim contents of cntlm.conf.";
    66       };
    67
    68      };
    69
    70    };
    71
    72
    73    ###### implementation
    74
    75    config = mkIf config.services.cntlm.enable {
    76      users.extraUsers = singleton {
    77          name = "cntlm";
    78          description = "cntlm system-wide daemon";
    79          home = "/var/empty";
    80      };
    81
    82      jobs.cntlm = {
    83          description = "cntlm is an NTLM / NTLM Session Response / NTLMv2 authenticating HTTP proxy.";
    84          startOn = "started network-interfaces";
    85          environment = {
    86          };
    87
    88      preStart = '' '';
    89
    90      daemonType = "fork";
    91
    92      exec =
    93        ''
    94          ${pkgs.cntlm}/bin/cntlm -U cntlm \
    95          -c ${pkgs.writeText "cntlm_config" cfg.extraConfig}
    96        '';
    97      };
    98
    99      services.cntlm.extraConfig =
   100        ''
   101          # Cntlm Authentication Proxy Configuration
   102          Username        ${cfg.username}
   103          Domain          ${cfg.domain}
   104          Password        ${cfg.password}
   105          Workstation     ${cfg.netbios_hostname}
   106          ${concatMapStrings (entry: "Proxy ${entry}\n") cfg.proxy}
   107
   108          ${concatMapStrings (port: ''
   109            Listen ${toString port}
   110          '') cfg.port}
   111        '';
   112    };
   113  }
notable parts are:

(line 1-13) if interested in general nix language: read [1] page 6,7 (and 5 might also be interesting)
(line 18-66) where a list of options is declared (some with default arguments; some without default argument which will enforce the user to set them)
(line 76-79) where the service is started as a different user (security measure)
(line 92-97) where the configuration is generated on the fly (yes cntlm.conf is generated everytime the configuration in /etc/nixos/configuration.nix is changed). a user using the cntlm service on nixos does not change the cntlm.conf manually)
(line 99-111) where a minimal configuration (extract of the example cntlm.conf) is parameterized.
(line 106) where a list of items is transformed into a multi line structure where:
cfg.proxy = [ "foo" "bar" "baz" ];
is transformed into:
Proxy foo
Proxy bar
Proxy baz

how to make use of the above expressions

a user has to append this configuration into /etc/nixos/configuration.nix and cntlm will be installed/configured and started

  services.cntlm = {
    enable=true;
    username="myusername";
    domain="mydomain";
    proxy=[ "192.168.3.5:1234" ];
  };

summary

in contrast to most other distributions nixos makes not only packaging subject to a ‘clean’ package management but also configuration management (/etc stuff) and runtime management. this is a very clean design helping to avoid lots of pitfalls.

links

[1] http://www.st.ewi.tudelft.nl/~dolstra/pubs/nixos-jfp-final.pdf

Posted in linux, package manager, security, technology, usability | Leave a Comment »

diploma thesis: Multi-Platform Software Package Management

September 24, 2011 by invalidmagic

finally … the online publication of my diploma thesis (DT) is here, it can be found online [1] including the source at [2].

i hope that the terminology introduced in this DT (chap 9) will be used. this is of course also true for concepts engineered in (chap 7).

candies can be found here:
 - chap 4: components used in a package manager
 - chap 4.6: different integration levels of a package manager
 - chap 5.13: ways to replicate a system
 - chap 5.15 ff
 - chap 6.2: evopedia deployment summary
 - chap 7 (you might want to read this a few times, it is quite complex)
 - chap 9: here i introduce some new terminology in package management
           (probably a _must read_)

see also the README for in [2] for further information.

problem description

when filing a telephone or email ticket often several recipients are contacted via email. dependent on the otrs configuration this is done in sync, so that a user gets success or fail for message sending in realtime. that means if no local mailserver is used it can be quite time consuming.

for example: filing a ticket here for say the ‘inhouse support crew’ would issue 5 emails and using an external mailserver that means:

5x 1,2 seconds ~= 5-8 seconds

where the otrs-GUI is frozen for the user after clicking ‘create’. dependent on how busy the remote email server is this can hinder usability quite fast.

so this post might be interesting for everyone who wants to use otrs but does not also operate a in-house mailserver.

my setup

otrs 3.0.6
windows server 2008 r2
no local mailserver

the solution

windows server 2008 has support for a local smtp server (smart host setup or satellite system setup) which can be used as relay. this setup is very easy is much better than using an external mailserver directly because the microsoft smtp relay server has also a local queue so messages won’t get lost if there is a partial network failure.

so basically i used [1], [2], [3], [4] to create the setup.

important points where:

using smtp.mail.emea.microsoftonline.com
tls encryption
user/pass authentification

after the configuration is final:

test the setup using: ‘telnet localhost 25′ and type the commands as found in [5].

the ms smtp relay queue

one should keep an eye on the queue which is in: c:\inetpub\mailroot\Queue

the otrs configuration

use the webinterface and login as root@localhost:

in otrs one has to login, go to admin->sysconfig->type smtp into the search bar, then ‘Core::Sendmail’, in there configure:

SendMailModule -> SMTP
SendMailModule::Host -> localhost
SendMailModule::Port -> 25
disable AuthUser/AuthPassword

finally you are done and should file a ticket (which is known to produce an email as a side-effect) and check if that mail is sent.

Note: i like that this smtp relay has a local queue, that is what i’ve been missing on linux for ages… (which is quite sad)

the wiki.nixos.org mediawiki theme

for some time i am hosting muse-sequencer.org with a mediawiki but i never was able to design a nice custom layout like it is done on spring RTS [1] for example. however, i often use the ‘clean’ theme which looks much better than the standard mediawiki theme.

anyway, there are times when you need a ‘custom theme’, especially when you have to integrate a custom menu bar (as shown in the top part of the nixos-wiki-theme screenshot below). the advantage is that the wiki page feels ‘integrated’ into the page and especially as the top level menubar helps to navigate the site it can be used to get out of the wiki again.

therefore, not integrating the main navigation menu into the wiki stylesheet causes a huge hit on usability as users have to alter the URL filed in order to get back to the other pages. and the wiki not looking like the rest of the pages also degrades the look and feel.

for quite some time i did not want to learn that much CSS and php/html but lately the integration of wiki.nixos.org into the nixos.org domain made me think otherwise and so i started to hack a new stylesheet [2] based on the ‘clean’ skin [3] i already debugged for my page ‘lastlog.de’.

i needed about 9hours to make the design final and i would like to thank zmlka@irc.freenode.net#nixos for his help and fine tuning! i would also like to thank viric@irc.freenode.net#nixos for his general attributions towards a nice mediawiki.nix modification which makes it easy to handle several third party mediawiki stylesheets. i might upload these modifications upstream (nixpkgs) soon – if i find the time.

i hope nixos.org folks will use the new style soon!

external and internal links

one nice thing i discovered while creating the nixos mediawiki theme was that it is very easy to add external links. i always thought this to be very complicated.

the nixos-mediawiki-theme does have two menues: a top level menu (header) and a bottom menu (footer). i used require_once(..), which is a php command, to include ‘header.php’ and ‘footer.php’.

this way the style can be extended without having to touch the the style. this is especially nice as nixos wants a stateless packages and one can set the ‘links’ from the /etc/nixos/configuration.nix context.

another nice thing is that it is very easy to add ‘internal’ links, that means: the menu generated from ‘header.php’ can be used to add links which will also end in the mediawiki:

this will create an external link:

<li><a href=”http://nixos.org/nixos/”>NixOS</a></li>

this will create an internal link

<li><a href=”demo”>Demo</a></li>

conclusion

although it was very time consuming for me to write this stylesheet (except for my ‘ajax patterns’ [4] project i’ve never done that before) i learned quite much about stylesheets, thus i encourage you to do the same for your page. soon i will write a little guide how to do that in more detail.

it is also worth mentioning that CSS can be used to make the content 100% independent of the position in the page but contrary to all my expectations this might not be a default for most pages.

i would also like to recommend firefox+firebug as this helps to analyze css/html errors and also helps to make experimental modifications on the fly!

motivation

the main purpose is a ‘high availability’ file storage with with two servers: main and failover. the main server has two raid systems:

one with fast disks and moderate disk sizes and
another raid with slow but very big disks

the idea is to find a way to use the fast raid for file-caching similar to ZFS or BTRFS’.

the goal of this experimental setup is to experiment with fscache [1]. so far it seems that fscache is either bound to NFS, AFS or ISOFS (or a similar network fs technology).

experiment using:

virtualbox 3.x on gentoo linux
ubuntu 10.10 (guest)
currently only main is in the experimental setup and DRBD is not used yet
not shown in the diagram is the PV/VG/LV stuff (LVM)

there are two interesting documentations:

[3] how to use fscache
[4] how fscache is implemented (and how to debug it)

setup using nfs

this is a small setup to test if the mount and fscache are working on the target platform. in this case it was working great with ubuntu 10.10 desktop. setup steps:

using [2] we install the needed tools on ubuntu linux:
apt-get install nfs-kernel-server nfs-common
cat /etc/exports
/media/share 127.0.0.1(async,no_subtree_check,rw,insecure,all_squash)
nfs-kernel-server neustarten:
/etc/init.d/nfs-kernel-server restart
change the permissions on /media/share
chmod 0777 /media/share
only to test if the nfs export works we do:
mount -t nfs 127.0.0.1:/media/share /mnt/mounted-Share
after a successful test we unmount it:
umount /mnt/mounted-Share
create the cacheDir disk:
dd if=/dev/zero bs=1M count=512 of=cacheLoopDevice.dd
mkfs.ext4 /mnt/cacheLoopDevice.dd
according to [3]:
tune2fs -o user_xattr /mnt/cacheLoopDevice.dd
mount -o loop,user_xattr /mnt/cacheLoopDevice.dd /mnt/cacheDir
enable the cachefilesd daemon:
vi /etc/default/cachefilesd # remove the # before RUN=yes
change the dir=… in /etc/cachefilesd.conf:
dir /mnt/cacheDir
/etc/init.d/cachefilesd restart
finally let’s mount it again but this time using ‘fscache’:
mount 127.0.0.1:/media/share /mnt/cached-Share/ -o fsc
let’s do a functionality tests:
touch /mnt/cached-Share/a
ls -la /media/share

setup using nfs – no networking filesystem

as shown in the two following examples, it does not work with ext2 or ext4. it seems as if fscache forces the use of NFS/AFS/ISOFS.

ext2

mkfs.ext2 /dev/sdb1
mount /dev/sdb1 /mnt/cached-Share/
umount /mnt/cached-Share/
mount /dev/sdb1 /mnt/cached-Share/ -o fsc

the error message (response to the shell command):

mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
missing codepage or helper program, or other error
In some cases useful info is found in syslog – try
dmesg | tail or so

ext4

mkfs.ext4 /dev/sdb1
mount /dev/sdb1 /mnt/cached-Share/
umount /mnt/cached-Share/
mount /dev/sdb1 /mnt/cached-Share/ -o fsc

the error message (response to the shell command):

mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
missing codepage or helper program, or other error
In some cases useful info is found in syslog – try
dmesg | tail or so

the error message in (/var/log/messages):

[ 846.008443] EXT4-fs (sdb1): Unrecognized mount option “fsc” or missing value

some performance experiments

to answer the question if all this setup is worth the effort, lets’ make some experiments:

time cp /usr/src/linux /mnt/cached-Share/
grep “__” /usr/src/linux -R
time rm -Rf /mnt/cached-Share/usr

please note: i’ve changed the described setup a little bit: the virtual machine guest was given a virtual disk which was stored on a SSD driver.

performance tests inside a virtualbox guests seem to be worthless. i have to repeat similar tests on the target machine. but using a local NFS just to get ‘fscache’ working seems to be a bad design IMHO. we’ll see if it is worth it.

conclusion

seems hot-data can be used with the NFS workaround but not without it. in contrast: zfs/btrfs have direct support for adding ‘hot’ data cache:

btrfs: ‘hot data relocation‘ functionality [5]
zfs: this is referred to as ‘Separate Cache Devices‘ [6] or ‘ssd caching‘ [7]

it would be nice to have a similar feature to fscache but for any filesystem (not limited to NFS/AFS/ISOFS) with the ability to be added dynamically (maybe using remount, so one does not have to unmount/mount the filesystem).

Edit: bcache [8] is what i actually wanted.

links

[1] http://en.wikipedia.org/wiki/CacheFS

[2] http://www.ubuntu-forum.de/artikel/32272/nfs-server-unter-ubuntu-einrichten.htm

[3] /usr/share/doc/cachefilesd/howto.txt.gz (coming with deb-package: cachefilesd)

[4] /usr/src/linux/Documentation/filesystems/caching/fscache.txt (found on my Gentoo system in 2.6.36-gentoo-r5)

[5] http://lwn.net/Articles/397643/ (Btrfs: Add hot data tracking functionality)

[6] http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Separate_Cache_Devices

[7] http://blogs.oracle.com/brendan/entry/test

[8] http://bcache.evilpiepirate.org/

Posted in linux | 1 Comment »

mediawiki::skin::clean relocated

June 2, 2011 by invalidmagic

mediawiki has various built in skins but i usually prefered the ‘clean’ skin from Kevin Hughes (shown in the screenshot).

lately i had massive problems as i was moving from a gentoo server to a nix os server probably caused by different php versions but i’m not absolutely sure. anyway as the clean skin [1] was broken with the more recent php version.

wasn’t it the fault of different mediawiki versions?

i did copy my old mediawiki (and the clean skin) on a 1:1 basis to my nix os based server resulting in the same layout problems compared to using the most recent mediawiki from scratch.

therefore i conclude that issues i fixed are not caused by mediawiki.

the fix

finally i decided to hack the skin to make it work again and i came up with 3 small modifications to the code and the skin was working again! after contacting Kevin Hughes about my changes he decided to release the next version of his skin cc-by-sa/3.0 which is great!

but he also said that he is not hosting the skin anymore, that means: the clean skin has a new home [2] (namely my repository). of course, as it is open source now you can also host it on any other page. but i would welcome other modifications to be done collaboratively.

dear kevin hughes – thank you so much for that great skin!

motivation

i have a gentoo system inside a virtualbox but i wanted to make some ‘long term tests’ so i decided to migrate it to a libvirt machine which is running ‘fedora core 15 beta’.

problems converting the image

first i tried to migrate the ‘Gentoo 64 (portage).vdi’ directly to a libvirt image, using [2]. but anything i tried: afterwards the image was never bootable so i decided to use ssh to copy all the files instead.

boot both virtual machines using the ‘grml64-mediaum_2010.12.iso‘.
assign the ip addresses
while i was using on the virtualbox side using: vboxnet0 in a host only networking schema i used a bridge on the other machine which involved lots of manual configuration as: disable networkmanager (on fedora core, remember?), removing the eth0 configuration (which happens to be called em1); adding a new configuration for the bridge br0 (using eth0).
finally i could ping from the virtualbox image to the libvirt guest system
i used ‘rsync -av /mnt/gentoo -e ssh 192.168.66.20:/mnt/gentoo‘
Note: both local gentoo systems were mounted into /mnt/gentoo
but libvirt used a ide host controller (which was very slow)
therefore i manually removed the ide controller and replaced it by a VirtIO Disk using ‘qcow2′ as storage format and ‘Virtio’ as bus.
after all the copying i installed grub (grub-1.99rc1) but the original system had a grub1 config!
the conversion was not simple!

The grub pitfall

virtualbox image using grub1:

cat /boot/grub/menu.lst

default 0
timeout 30
#splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title Gentoo Linux 2.6.24-r7
root (hd0,0)
kernel /boot/kernel-genkernel-x86_64-2.6.36-gentoo-r5 root=/dev/ram0 real_root=/dev/sda1
initrd /boot/initramfs-genkernel-x86_64-2.6.36-gentoo-r5

in comparison: ‘libvirt guest’ using grub2

cat /boot/grub/grub.cfg

set default=0
set timeout=30

menuentry “Gentoo Linux 2.6.36-gentoo-r5″ {
insmod part_msdos
insmod ext2
set root=(hd0,msdos1)
linux /boot/kernel-genkernel-x86_64-2.6.36-gentoo-r5 root=/dev/ram0 real_root=/dev/vda1
initrd /boot/initramfs-genkernel-x86_64-2.6.36-gentoo-r5
}

Note: i marked the differences.

Note: take care of the different filename as well!

anyway: in the grml shell you can install grub into /dev/vda using:

grub-install –root-directory=/mnt/gentoo /dev/vda

the kernel configuration pitfall

a libvirt guest must be aware of /dev/vda (virtIO) but my genkernel was not. also i lacked ext4 support. so it is a good idea to included this into the kernel (i had it included as modules but it did not work well).

cat /etc/kernels/kernel-config-x86_64-2.6.36-gentoo-r5 | grep -i virt | grep -v “^#”

CONFIG_VIRT_TO_BUS=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_HW_RANDOM_VIRTIO=y
CONFIG_VIRTUALIZATION=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y

just use ‘genkernel’ to build the new kernel (and don’t forget the ext4 support as i did).

fedora core network problems

i basically used [3] to make it work. the benefit is now that em1 is not used directly but the system uses br0 to access the internet.

PRO: the libvirt guests do get their own ‘mac address’, thus are separated from being able to see each others traffic.

fedora core yum problems

i also tried to install virtualbox and followed the instructions found on virtualbox.org but soon i had the problem that the virtualbox kernel modules won’t build and need ‘kernel-devel’ but after installing the kernel-devel package using ‘yum install kernel-devel’ there was a mismatch between ‘used kernel’ and ‘kernel-devel’ headers.

summary

libvirt and the ‘virtual machine manager’ are very nice:

i like that it is so easy to start a virtual machine when the host machine boots.
i also like the ‘virtual machine manager’ as it shows cpu/disk io/network io nicely
(but that is not limited to libvirt virtualizations).
fedora core 15 beta was running quite nicely (except that it crashed while i was writing this article)
so i can at least say: it ran for straight 6hours without crash ;P

motivation

recently i had problems with the deployment of an experimental ‘shinken‘ [1], a monitoring tool, on gentoo. most of the installation was ‘straightforward’. but i had a problem with livestatus in combination to ‘thruk‘ [2].

Note: i basically used the shinken guide [3] for fast and easy testing of shinken

gentoo linux in virtualbox
shinken 0.6
thruk-1.0.3 (using the perl webserver; NOT using apache2)

dependencies

this list contains packages which are not essentially (as the apache and mysql stuff) but it’s basically all packages i’ve installed to make it work, so just pick what you need:

sys-apps/portage-2.1.9.25
app-emulation/emul-linux-x86-baselibs-20100915-r1
sys-kernel/gentoo-sources-2.6.36-r5
sys-kernel/genkernel-3.4.10.907
app-arch/xz-utils-4.999.9_beta
sys-kernel/gentoo-sources-2.6.36-r5
sys-boot/grub-0.97-r10
sys-kernel/genkernel-3.4.10.907
app-portage/eix-0.20.5
net-misc/dhcpcd-5.2.8
app-admin/eselect-vi-1.1.7-r1
sys-libs/gpm-1.20.6
app-admin/eselect-ctags-1.13
app-portage/eix-0.20.5
sys-apps/iproute2-2.6.35-r2
dev-util/ctags-5.7
app-editors/vim-core-7.3.50
app-editors/vim-7.3.50
app-vim/gentoo-syntax-20100215
dev-python/setuptools-0.6.15
dev-python/pyro-3.12
dev-python/pyro-4.3
app-admin/metalog-1
virtual/logger-0
net-libs/libpcap-1.1.1
net-analyzer/nmap-5.21-r1
app-editors/gentoo-editor-2
virtual/pam-0
net-mail/mailbase-1
mail-mta/ssmtp-2.62-r8
virtual/mta-0
app-admin/sudo-1.7.4_p5
dev-libs/oniguruma-5.9.2
dev-db/mysql-init-scripts-1.2
dev-libs/apr-1.4.4
app-admin/eselect-php-0.6.2
dev-libs/libmcrypt-2.5.8-r2
dev-util/pkgconfig-0.25-r2
sys-apps/ed-1.4
dev-perl/Net-Daemon-0.43
perl-core/Storable-2.20
perl-core/File-Spec-3.31
virtual/perl-Sys-Syslog-0.27
dev-libs/libpcre-8.12
dev-libs/apr-util-1.3.11
virtual/perl-Storable-2.20
virtual/perl-File-Spec-3.31
app-admin/apache-tools-2.2.17
dev-perl/PlRPC-0.202.0
www-servers/apache-2.2.17
dev-perl/DBI-1.615
dev-db/mysql-5.0.91
virtual/mysql-5.0
dev-perl/DBD-mysql-4.01.7
dev-lang/php-5.3.6
media-libs/gd-2.0.35-r1
dev-perl/GD-2.44
dev-perl/yaml-0.71
app-misc/screen-4.0.3-r4
dev-perl/JSON-2.27
perl-core/Compress-Raw-Zlib-2.024
app-arch/unzip-6.0-r1
app-vim/genutils-2.5
virtual/perl-MIME-Base64-3.08
dev-perl/URI-1.55
app-misc/beep-1.2.2-r1
dev-libs/eventlog-0.2.12
app-text/build-docbook-catalog-1.4
app-text/sgml-common-0.6.3-r5
dev-libs/libgpg-error-1.10
app-text/docbook-xsl-stylesheets-1.75.2
dev-libs/libgcrypt-1.4.6
dev-libs/libxslt-1.1.26-r1
app-text/docbook-xml-dtd-4.3-r1
dev-util/gtk-doc-am-1.16
dev-libs/glib-2.28.6
app-admin/syslog-ng-3.1.4
net-analyzer/nagios-plugins-1.4.14-r2
app-misc/realpath-1.15-r1
app-portage/gentoolkit-0.2.4.6.1-r1
dev-python/simplejson-2.1.3
dev-db/sqlite-3.7.5
dev-python/pysqlite-2.6.3

problem with livestatus

i’ve missed a line in the README of shinken and therefore the livestatus service on port 50000 was not enabled. but there was no ‘good’ error message mentioning what actually got wrong.

thruk reported (in the webinterface):

No Backend available

shinken reported (/var/lib/shinken/brokerd.log):

[broker-1] Warning : the module type livestatus for Livestatus was not found in modules!

But that was fixed after i installed:

dev-db/sqlite-3.7.5
dev-python/pysqlite-2.6.3

and restarted shinken.

debugging

look into:

/etc/shinken/ (config files)
/var/lib/shinken (log files)

optinally one can restart a single module of the shinken service with debugging:

/etc/init.d/shinken-broker -d restart

solution

links

[1] http://www.shinken-monitoring.org/

[2] http://www.thruk.org/

[3] http://www.shinken-monitoring.org/wiki/shinken_10min_start

Posted in gentoo | 2 Comments »

Older Posts »

technical quirks & misc

repairing the n900 camera

0. symptoms

1. the backup

1.1 create a backup using the backup tool coming with the n900

1.2 dpkg and a list of all files in ‘opt’

1.3 now make a backup of all files

2. eMMC/VANILLA

3. FIASCO/Rootfs

4. restoring the backup

5. restoring the applications

6. remove the photo click sound and the intro animation

random notes

links

kdevelop on nixos

issue 1

issue 2

links

how to integrate a daemon into nixos

/etc/nixos/nixpkgs/pkgs/tools/networking/cntlm/default.nix

/etc/nixos/nixos/modules/services/networking/cntlm.nix

how to make use of the above expressions

summary

links

links

problem description

my setup

the solution

the ms smtp relay queue

the otrs configuration

links

the wiki.nixos.org mediawiki theme

external and internal links

conclusion

links

motivation

setup using nfs

setup using nfs – no networking filesystem

ext2

ext4

some performance experiments

conclusion

links

wasn’t it the fault of different mediawiki versions?

the fix

links

motivation

problems converting the image

The grub pitfall

the kernel configuration pitfall

fedora core network problems

fedora core yum problems

summary

links

motivation

dependencies

problem with livestatus

debugging

solution

links

Recent Posts

Categories

Blogroll

Follow “Invalidmagic's Blog”