Archive for the ‘visualization’ Category

Evopedia Icon

for quite some time i use a wiki at lastlog.de, a mediawiki to be precise, and i wonder why there is no wide adaptation towards the wiki principle. with that i don’t mean collaborative editing but, somehow in contrast, the principle to be quotable.

lately, out of curiosity, i scrolled through my diploma thesis and checked the overall link stability. some were broken. however, all wikipedia links worked. as stated in the document itself, i explicitly link to the wikipedia because of its link stability. if i would have liked i could have even linked to a certain revision. but i decided not to, as the reader always has the option to look at an older revision, based on date and time.

the more interesting aspect, that is why i linked to wikipedia articles, is that i don’t want to waste time describing something when there is a different place doing so already. if someone is smart enough to follow my ideas in my diploma thesis i assume the same when it comes to judging about the quality of wikipedia articles. and before linking a keyword (like ‘package manager’) to a certain wikipedia article, which should describe it, i always read the article. the idea is twofold: first i like to see if my conception or understanding matches with what is in the article. second, if that is the case, i would simply link it and forget about the whole thing. but if my understanding does not match with the article i can evaluate my or their version as being better and pick what fits best.

for some online articles i had to link, i wasn’t even able to provide a direct link and therefore added a google search link into the document.

wiki editing has so many benefits, like being able to rollback to a previous version. do collaborative work. why is there no wiki like support, say when editing libre office/word documents? maybe because back in time that was considered a waste of bits&bytes but using compression that can’t be an argument today.

here is a use-case where that would be great: say you write a document and you pass it to someone else for review and corrections. often i would like the other person doing whatever change he wants to do and later be able to rollback this or that change. with a wiki like document structure this would be very easy.

if you don’t follow, just have a look at this link:


and about link stability: this link might even work when this blog is long gone. 

i see so many benefits by using wikis and wiki like concepts but despite of the wiki-web principle and decentralized VCSs there seems to be no wide use of it.

IMHO i think a webpage, even this wordpress blog, which does not implement a wiki principle, is kind of stupid as one can never be certain what is going on. one could say such a page is schizophrenic to some degree.

hopefully this will change in the future.

update: 11.5.2012 – it would be desirable if the mentioned link stability would be independent of a strict TLD (top level domain). for example: if i move this blog to a different location, say to invalidmagic.de then all the articles here stop working and the links from other pages into this article will fail.

Read Full Post »

kdevelop on nixos

kdevelop on nixos

issue 1

i’ve adapted to a nixos only desktop and i’m facing new problems every day, lately i wanted to use kdevelop but it wasn’t able to import a normal ‘Makefile’based project. next i installed lots of packages somehow mentioned at [1]. still that alone didn’t help so i ran:


kbuildsycoca4 is a utility which generates a cache of .desktop files and such to be used by programs. PovAddict#kde-devel@irc.freenode.net recommended to set KDEDIRS correctly because when doing so running kbuildsycoca4 manually wouldn’t be neccessary.

not sure how to solve this problem correctly yet but in case i find out i will add it to this post.

update: 30.06.2012

in /etc/nixos/configuration.nix write:

environment.systemPackages = with pkgs; [

after nixos-rebuild kdevelop will have makefile and cmake support. however, at least for proper cmake support, needed when kdevelop should build the project, you will need to prepare a decent environment where all the needed programs and dependencies are listed, see [2]

issue 2

another point which bugs me is that auto-completion doesn’t work correctly as there is no such thing as /usr/include on nixos as one might expect when coming from a different linux distribution. didn’t find a solution here yet either…

update: 30.06.2012 – found a nice solution to this problem, just have a look at [2] as well


[1] http://techbase.kde.org/Projects/KDevelop4/HowToCompile

[2] https://invalidmagic.wordpress.com/2012/03/23/developing-software-using-nixos/

Read Full Post »

the wiki.nixos.org mediawiki theme

for some time i am hosting muse-sequencer.org with a mediawiki but i never was able to design a nice custom layout like it is done on spring RTS [1] for example. however, i often use the ‘clean’ theme which looks much better than the standard mediawiki theme.

anyway, there are times when you need a ‘custom theme’, especially when you have to integrate a custom menu bar (as shown in the top part of the nixos-wiki-theme screenshot below). the advantage is that the wiki page feels ‘integrated’ into the page and especially as the top level menubar helps to navigate the site it can be used to get out of the wiki again.

therefore, not integrating the main navigation menu into the wiki stylesheet causes a huge hit on usability as users have to alter the URL filed in order to get back to the other pages. and the wiki not looking like the rest of the pages also degrades the look and feel.

for quite some time i did not want to learn that much CSS and php/html but lately the integration of wiki.nixos.org into the nixos.org domain made me think otherwise and so i started to hack a new stylesheet [2] based on the ‘clean’ skin [3] i already debugged for my page ‘lastlog.de’.

i needed about 9hours to make the design final and i would like to thank zmlka@irc.freenode.net#nixos for his help and fine tuningi would also like to thank viric@irc.freenode.net#nixos for his general attributions towards a nice mediawiki.nix modification which makes it easy to handle several third party mediawiki stylesheets. i might upload these modifications upstream (nixpkgs) soon – if i find the time.

i hope nixos.org folks will use the new style soon!

external and internal links

one nice thing i discovered while creating the nixos mediawiki theme was that it is very easy to add external links. i always thought this to be very complicated.

the nixos-mediawiki-theme does have two menues: a top level menu (header) and a bottom menu (footer). i used require_once(..), which is a php command, to include ‘header.php’ and ‘footer.php’.

this way the style can be extended without having to touch the the style. this is especially nice as nixos wants a stateless packages and one can set the ‘links’ from the /etc/nixos/configuration.nix context.

another nice thing is that it is very easy to add ‘internal’ links, that means: the menu generated from ‘header.php’ can be used to add links which will also end in the mediawiki:

this will create an external link:

this will create an internal link

  • <li><a href=”demo”>Demo</a></li>


although it was very time consuming for me to write this stylesheet (except for my ‘ajax patterns’ [4] project i’ve never done that before) i learned quite much about stylesheets, thus i encourage you to do the same for your page. soon i will write a little guide how to do that in more detail.

it is also worth mentioning that CSS can be used to make the content 100% independent of the position in the page but contrary to all my expectations this might not be a default for most pages.

i would also like to recommend firefox+firebug as this helps to analyze css/html errors and also helps to make experimental modifications on the fly!


[1] https://springrts.com

[2] https://github.com/qknight/nixos

[3] https://github.com/qknight/clean

[4] http://lastlog.de/wiki/index.php/Ajax_patterns

Read Full Post »


this posting is about using a QGraphicsScene/QGraphicsView with a QAbstractItemModel. i’ve blogged about this two times, see [1] and [2]. i’ve made progress and i created a few new concepts which might help _you_ and probably saves a lot of your time.

it is important not to mix the two different Qt concepts: ‘Qt way of ModelViewController’, so let’s revisit the docs:


  1. a QGraphicsScene/QGraphicsView has it’s own model and data storage. one QGraphicsScene (scene) can have several QGraphicsView(s) (views) attached
  2. a QAbstractItemModel is used with QTreeView/QTableView and QListView

IMPORTANT: both concepts (1) and (2) can’t be combined directly – normally (out of the box). what i’m doing here is actually _combining_ them. as you will see, this can be quite complicated.

the basis

the first blog posting refers to the automate project, the second to the ‘spring random map generator’. this posting is mainly about the ‘spring random map generator’. currently the automate codebase is deprecated and broken. i will eventually fix it. this is how the ‘spring random map generator’ looks like:

so what is new since the last blog posting about this?


  1. i’m not inheriting from QAbstractItemView anymore, instead i’m using a QGraphicsScene connected to the QAbstractItemModel using Qt’s ‘signals&slots’
  2. i’ve developed a concept for setting up modules and connections in arbitrary order:
    that means you can add a connection to the QGraphicsScene without having any module!
    this means you can remove QGraphicsItem’s in arbitrary oder as well (no more crashes when shutting down the program)
  3. both concepts are implemented in a messanger class: GraphicsItemRelay and a interface: GraphcisItemRelayInterface
  4. module properties can be edited using a QTreeView (accessing the same QAbstractItemModel as the GraphicsScene), using a QSortFilterProxyModel
  5. how to use graphviz to layout a graph

so let’s have a detailed discussion about every of those points.

1. why not inherit from QAbastractItemView

since most of the code given by the QAbstractItemView wasn’t used anyway i removed it (refactoring). now the GraphicsScene is the new view and functionality is added using direct signal&slot connections.

  • connect( model, SIGNAL(modelReset ()), this, SLOT(reset()));
  • connect( model, SIGNAL(layoutChanged ()), this, SLOT(layoutChanged()));
  • connect( model, SIGNAL(rowsInserted ( const QModelIndex & , int , int  )), this, SLOT(rowsInserted ( const QModelIndex & , int , int  )));
  • connect( model, SIGNAL(rowsAboutToBeRemoved ( const QModelIndex & , int , int  )), this, SLOT(rowsAboutToBeRemoved ( const QModelIndex & , int , int  )));
  • connect( model, SIGNAL(dataChanged ( const QModelIndex & , const QModelIndex &  )), this, SLOT(dataChanged ( const QModelIndex & , const QModelIndex &  )));
that is basically all you need to do, if you want to use your QGraphicsScene as a View on top of a QAbstractItemModel. it’s better to have less code to care about!

2. adding/removing Modules/Connections in arbitrary order

my first implementation of adding nodes and connections between them was very limited. see [4], the automate project. as a arrow in the diagramscene [5] i needed both other QGraphicsItems first in order to know the source and the destination coordinates to draw a line.

  • that meant on the other hand: one would have to insert all nodes first and then all connections later when a QGraphicsScene/QGraphicsView got populated with items
  • removing a a QGraphicsScene/QGraphicsView implies the same in opposite direction: first all connections, later all nodes

not doing so would crash the program as the draw routine of the QGraphicsView, when drawing a connection, would query a QGraphicsItem’s position (the node) which was simply not there anymore. note: this has a cyclic dependency:

  • moving a node (QGraphicsItem) does for a connection (QGraphicsItem) to redraw
  • removing a connection requires both nodes (QGraphicsItem) to be there in order to unregister the connection (see previous point)

see http://github.com/qknight/automate/blob/595770383e34f4bea4beedb99aaa13e6b52b910d/src/GraphicsView/SceneItem_Node.cpp at:

QVariant SceneItem_Node::itemChange ( GraphicsItemChange change, const QVariant &value ) {

to sum up: to reduce code and complexity it is important to make insertion and deletion in arbitrary order, see (3) how i did it.

3. the GraphicsItemRelay and the GraphicsItemRelayInterface

since problem (2) only requires the position of an object i decided to implement a messenger class:

the concept might look quite complex but it works great as one can remove connections or modules (ports) in arbitrary order, just what i discussed in (2).

4. module properties and a QTreeView

in the last posting [2] i wondered how to implement properties without adding yet another hierarchy layer. it seems it can be done by doing so. see the image in this posting: it features two different views using the same QAbstractItemModel.

again, what was the problem?

  • a list of items should be editable just as done in ‘qtdesigner’ where you have a nice list on the right side hosting QObject attributes
  • every element in the list should be accessed via a QModelIndex so that the model is used to set/get the values
  • every time a property changes the view should be updated as the rendered image might (very likely) change as the parameter influence the image rendering process

the problem was with the hierarchy layer:

  • rootitem – module – port – connection

which was extended by:

  • rootitem – module – port – connection
  • rootitem – module – property

as there are two different views:

  • QTreeView: used to make properties editable
  • QGraphicsScene/QGraphicsView to visualize the graph

that means we have to take care for the third layer as there can be a port or a property.

the QGraphicsScene/QGraphicsView ignores properties. as it simply ignores them. the QTreeView would render ports but to avoid that one can use a QFilterProxyModel which filters out all ports (and all childs of them). this way only modules and properties are there.

5. how to use graphiviz to layout a graph

i always wanted to use graphiviz to layout my automate. i’ve found a implementation at [7] which describes how to do so. i did not find the time yet to use that code. maybe i find the time in the future.


[1] https://invalidmagic.wordpress.com/2009/12/10/qgraphicsscene-used-as-a-qabstractitemmodel/

[2] https://invalidmagic.wordpress.com/2010/02/02/qgraphicsscene-used-as-a-qabstractitemview-ii/

[3]  http://github.com/qknight/springrts.com-random-map-generator

[4] http://github.com/qknight/automate

[5] http://doc.trolltech.com/4.5/graphicsview-diagramscene.html

[6] http://labs.trolltech.com/blogs/2008/10/24/itemviews-the-next-generation/

Read Full Post »

lately i started working on a torrent implementation for the evopedia project [1]. that will include the qt torrent example [2] but this posting won’t be about any torrent related stuff.

first i thought about implementing a QAbstractItemModel but reading various examples i discovered the QStandardItemModel.

what i want to do

i have an item, represented via a Model and visualized by a a QTreeView. the model basically hosts the item(s) and several views could display that item(s) in different layouts but using exactly the same data.

when a user clicks an item a torrent client will start downloading it. the torrent client will update the state of the download by using the item’s textfield via the model.

this can be implemented either by using a QAbstractItemModel or a QStandardItemModel, so let’s see which one is easier to be used:

discussion: QAbstractItemModel

i’ve been using this model quite some times now and i really love it but it has one big limitation: every time you want to change an item (which is an item of the model) you have to use the model to do that. that means if an item want’s to change itself, you have to query the representative QModelIndex and then use this with the model:

QModelIndex* m = index(0,0,QModelIndex()); // query the first row and column and a top level item, by using a invalid QModelIndex

m->setData(…); // here we set the data

QModelIndex’es are used by views to access the data by using the model interface. this is all about MVC and abstraction.

CONCEPT: most of the time there is no need for items to be able to change themselves BUT in this case it is a very handy thing.

two ways of implementing this:

  • either the item holds the torrent client object as well but the torrent client object will update the text of the item using the model
  • or one could also have a Model and a list of torrent clients and then the torrent client accesses the item also via the QModelIndex but this would mean having another lists to take care of

i think either case is error prone, so let’s see how this could be done using a QStandardItemModel instead:

discussion: QStandardItemModel

in contrast using this model you don’t use QModelIndex’es at all. instead you use an abstraction: the QStandardItem. the fun part about a QStandardItem is that it is holding data which is accessed by all views BUT it can also be used to change itself.

this code has been taken from the Qt QStandardItemModel manual (but slightly extended):

QStandardItemModel model;
QStandardItem *parentItem = model.invisibleRootItem();
for (int i = 0; i < 4; ++i) {
QStandardItem *item = new QStandardItem(QString(“item %0”).arg(i));
parentItem = item;

therefore it is very easy to extend a QStandardItem by inheriting from it. let’s call this new item a QExtendedStandardItem. within that class one can have a torrent client object which does all the work. if the state of the download changes from download started to download finished, the torrent client can simply use setText(“finished”);

another point worth mentioning is that the torrent object could also expand the QStandardItems with additional childs which could be used to list up/down rates as well as peers/seeders and other valuable parameters (can be seen in the above image).


using a QStandardItem makes it very easy to implement the torrent client.


[1] http://evopedia.info/

[2] http://doc.qt.nokia.com/4.6/network-torrent.html

Read Full Post »

i have a windows xp server on which i use ‘acronis true image home 2009’ for periodical backups.

my problem:

my 1TB backup storage was filled with incremental/full backups although i’ve used incremental backups with consolidation enabled.

to investigate the issue i’ve installed a windows xp into a VirtualBox on my linux machine – to experiment – in the hope to find some easy solution. previously i’d been searching in forums for a fix to the issue but it seems a lot of ppl have the same issues, too, without providing a fix.

the acronis documentation does no explain what ‘acronis’ actually does or it is far to complicated. i’ve read the manual (not the printed version but instead the acronis help when pressing the ‘?’ button on the ‘backups – incremental’ dialog) a few times but i did not understand it as it is quite complex and i don’t like how they explain the single steps.

what i want:

first let’s see what i want:

  1. a incremental backup should be made every day (the first backup is a full backup of course)
  2. the main archive (the first full backup) should be validated on every backup
  3. if more than 6 backups are there, delete the oldest one
  4. the old backup may only be deleted if the new backup is 100% consistent
  5. the backup must always be in a consistent state, no merging of a full+diff when not having another full backup around

i’m not sure if the backup merge (merging a full and a incremental backup) is atomic, which would mean that if the merge fails the old files are not lost. after some experiments i’m not sure but i doubt that it is atomic. i have a bad feeling about this. so i think (4) and (5) can’t be done directly. maybe with two backup jobs, one every ‘odd day’, the other every ‘even day’ would be a solution. but there are so many indications which i can’t take into account – so i currently think the best is to go with (1),(2) and (3) only while checking the backup from time to time manually (which i would do anyway).

so here is the configuration of how i achieve (1),(2) and(3) but NOT (4) and (5) :



automatic consolidation

problem to exceed the harddrive capacity

to recall: my primary problem was that the backups did exceed the harddrive capacity as no old archives were consolidated:

that resulted from the ‘backup method’ where i also checked the last point ‘create a new backup after x incremental or differential backups’ which then would create a new backup. that means: consolidation was never done since the counter for consolidation was 6 (shown in the screenshots from above) but a new backup was created after 3 successive backups (disabled in the first screenshot).

it seems to me that the acronis is executing like this:

if (sum(backups) > consolidation_threshold) -> consolidate backups

where sum(backups) is interpreted: accumulate all different types as full|incremental|differential

example: sum(one full backup and 3 differential backups) = 4

however: if the checkbox ‘create a new backup on every x’th backup’ is checked the algorithm is never executed when x is smaller than the consolidation_threshold leaving old backups undeleted!


currently i disable the ‘create a new backup after x incremental or differential backups’. that means all options are set as done in the 3 screenshots.


this is a very typical flaw in gui design as intuition misleads based on the facts presented. a very clear and intuitive approach would be a schedule visualization for the current setup. i’m not very happy with acronis currently.

i’ve also purchased the more recent version of ‘true image home 2010’ but it still is the same issue. maybe someone understands this better than me, then please give me a hint.

Read Full Post »


i’ve just fixed most bugs in libnoise-viewer [1] and the software is really usable now. it features a QGraphicsScene and a QGraphicsView and to make it faster i’ve used some it::blackmagic called threads. Actually i used QThread and this was my first real use of multi-threading and it works pretty well. When i scroll the planar area small tiles are generated, two at a time are dispatched so that a dual core system is ‘saturated’ cpu load wise.

This viewer uses the ‘noise::module::RidgedMulti‘ module to generate the surface in a black/white fashion. All i did was to apply a color-band according to the relative height. As you might have seen already, there is a black/white vs color toggle. Using this you can visualize the heightmap.

if (job.colorstate) {
renderer.ClearGradient ();

renderer.AddGradientPoint ( 1.000000, utils::Color (0, 90, 0, 255));
renderer.AddGradientPoint ( 0.740000, utils::Color (0, 129, 0, 255));
renderer.AddGradientPoint ( 0.700000, utils::Color (0, 190, 0, 255));
renderer.AddGradientPoint ( 0.650000, utils::Color (0, 250, 0, 255));
renderer.AddGradientPoint ( 0.600000, utils::Color (120, 250, 250, 255));
renderer.AddGradientPoint ( 0.390000, utils::Color (20, 120, 200, 255));
renderer.AddGradientPoint ( 0.340000, utils::Color (20, 50, 250, 255));
renderer.AddGradientPoint ( 0.200000, utils::Color (0, 0, 120, 255));

renderer.SetLightContrast (3.0); // Triple the contrast
renderer.SetLightBrightness (2.0); // Double the brightness

The rendering speed is very good. But see yourself (the upload of the ogv video to wordpress media didn’t work, so i host it somewhere else).


If you want to visualize other ‘libnoise graphs‘ you can edit the sourcode (see renderThread.cpp) , all you need is to add 3 or 5 lines of code:

module::RidgedMulti myModule;
myModule.SetOctaveCount (job.octave);
myModule.SetFrequency (job.frequency);

Again: if you ever have to use threads, use QThreads since signals/slots are threadsafe (coming with qt ~4.2 if i recall correctly) communication between the threads is very easily done using the signals/slot mechanism, which means using the event loop.

I need to say, this is the 3rd project i’ve migrated to cmake and to do that i needed about 30-50 minutes since i also cleaned up the sourcode and did some other things as renaming of files and re factoring of class names.

I’ve also updated to =dev-util/kdevelop-3.9.97 this release fixed all the crashes the previous release had and i absolute love this ide. Currently i can’t think of ever using something else! The cmake integration is very good and makes a lot of fun using it. Renaming of classes or member functions as well as object’s names works like a charm!

Unlike other KDE components kdevelop 3.9.97 works reliably stable for me!


[1] http://github.com/qknight/libnoise-viewer

[2] http://github.com/qknight/libnoise

Read Full Post »

Older Posts »