ogre 3d

i’ve been using ogre 3d lately and here are some issues i had:

• in gentoo there is no way to install ogre WITH the samples in one go, if one wants the samples as well one would have to install the library based from the source (bypassing the package manager) or hack the ebuild
• to update ogre i had to uninstall ‘ogre’ with emerge -C ogre first and then install the new version with ‘emerge ogre‘ in a second go. if not done so ogre would screw up the build for some unknown reason?! if i recall correctly that was a linker issue
• autotools is so ugly, i don’t see any way to compile the samples when not using ./configure from the base library directory which means also compiling all the library as well…
• i think that writing all the ogre documentation must have taken a lot of time – and i honor that – but i prefer how the trolltech guys do  it. the trolls use a ‘incode documetation’ syntax (probably doxygen with some markup) and no external documentation. in contrast the ogre developers use a mixture of  mediawiki/html documetation/library doxygen/internal samples and external samples. if they would for instance only use templates in the wiki. currently not doing so makes it very confusing finding related things.
• most of the ogre examples don’t include any screenshot of what is actually done, this makes searching much more ‘fun’…
• they use autotools for the library and they use autotools for the samples. i used cmake for my project instead. since i don’t know much about autotools i had problems finding out what linker and compiler switches i needed and since my package manager removed all Makefiles after installation of the samples (or probably never build them at all) i had to do ugly things to find out how things work.
  FIX: so if you want to find that out i urge you to download the ogre release by hand (bypassing the package manager) and use ./configure; make, then have a look at the samples directory
• the ‘package manager’ of gentoo (portage) has the latest ogre included but kubuntu didn’t, so we had to use the custom build script there by hand…
• using the package manager of gentoo i set the useflag for double-precision which resulted in linker errors and also ‘our’ codebase which worked on a different machine using the custom build of ogre probably not using ‘double-precision’ at all using mesa for 3d rendering. but it did not work on my laptop after i removed the linker error. it just showed a black screen. took me hours to find out with lots of library recompiles…
• the kubuntu defaults for the ogre library are acceptable but they had only a very old version in contrast gentoo didn’t provide good defaults since cg and devil was disabled but had a very recent version of ogre. funny, isn’t  it?
  this is how i have it working now:
  % equery u ogre
  [ Searching for packages matching ogre... ]
  [ Colour Code : set unset ]
  [ Legend : Left column  (U) - USE flags from make.conf              ]
  [        : Right column (I) - USE flags packages was installed with ]
  [ Found these USE variables for dev-games/ogre-1.6.5 ]
  U I
  + + cg               : NVIDIA toolkit plugin
  + + devil            : image loading support with DevIL
  + + doc              : Adds extra documentation (API, Javadoc, etc)
  - – double-precision : more precise calculations at the expense of speed
  + + examples         : Install examples, usually source code
  + + gtk              : Adds support for x11-libs/gtk+ (The GIMP Toolkit)
  + + threads          : Adds threads support for various packages. Usually pthreads
  you might wonder what that there is a ‘examples‘ use flag. but all it does it to install the example source but not the compiled examples NOR any Makefile, so …. again this autotools issue i don’t know how to compile it then… not in any documentation i could find either.
• documentation how to use custom shaders is really missing in ogre, i managed to compile my own shader (cg file) with cgc (that is the nvidia shader compiler) but the library wasn’t able to use it no matter what i tried…
  it seems that most developers use windows and directX anyway which makes the situation on linux even worse.
• in ogre one has several cfg files which are essential to run a sample application. but one has to create these files individually for every sample. so mixing a ‘package manager ogre library install’ with a ‘bypassing source ogre library build’ makes the situation even worse. the files are:
  • ogre.cfg (can be deleted, since it can be generated at program start with a gui)
  • plugins.cfg (i hate this file since it is required but no sample includes it after build), basically this file contains paths to shared objects .so files which are used by the ogre core backend
  • resources.cfg (this file contains a list of used textures, shaders and other graphics related stuff), this needs to be done per application. but there is a collection of Media files for the samples but the copyright is somehow strange since parts of it may not be redistributed…. so why do they included it at all?

• ogre contains OIS for input handling but i couldn’t figure out how to get joystick support working. this probably might be broken completely. so instead of using OIS for joystick handling i had to include libSDL for that. i wrote a libSDL joystick example some time ago, can be found [1] on my wiki. i wonder what that joystick-support in OIS is all about…?
• in general there are tutorials, about how to use ogre from an ide, in the wiki. there is also tutorials how to use ogre and how to import objects from blender and stuff. finding these resources is very time consuming. they have the tendency to be for ‘pro uses’ only and tend to be incomplete as well.
• there is a ogre documentation which covers the basic concepts. i’ve been reading some stuff and it seems to be very good. however i can’t tell if it’s complete as my current use case is quite limited.

my project

you can find the code at [2]. but i’d like to warn you since:

• we didn’t have much time so guess how the code looks
• has no documentation
• might not compile
• voids many coding standards (especially c++ class/file naming schemes)

but still it might still be useful for some coders to look at, for instance the joystick integration or how i did use cmake with ogre…

so why do i write this post at all?

• i hate the fact that there are multiple distributions – all factoring the same package of ogre – one more incomplete than the other….
  
• the samples are not pre-built – if included at all
  i found a forum thread where developers did not see the samples as part of the ogre core distribution at all – how backward is that? examples and documentation is basically everything if one distributes a ‘library’!
• the buildsystem of ogre and the samples is very confusing and badly documented
• there is no qtdemo like program for ogre included, although there was a summer of code project assigned and completed recently, it’s refered as ’sample interface’ and will be included in ogre 1.7 – i love that!
• despite the fact that there is no ’sample interface’ the samples are hard to build, no documentation about ‘how they can be built’ is included in the release

summary: i don’t like the way the documentation is handled in the ogre project and i don’t like the build system as well. my main criticism however is that the distributions do a very bad job at including the library (with all the samples missing & misc quirks).

possible fix: take a look at how trolltech did it with the qt release. they have assistant for the documentation (no need to open a webbrowser with a distribution specific installation of the doxygen and html documents). they have qtdemo and they build all the examples while also having the source around. they don’t have any external documentation as in a wiki for instance.

links

[1] http://lastlog.de/wiki/index.php/SDL-joystick

[2] http://github.com/qknight/space-game

[3] http://www.ogre3d.org/2009/08/27/google-summer-of-code-2009-projects-coding-time-is-up

QGraphicsScene used as a QAbstractItemView II

Again some thoughts about this complex topic, but this time with content not as last time when i accidentally published an unfinished article – i’m very sorry for that.

updated diagram illustrating the QGraphicsScene used as a QAbstractItemModel

the primary difference to the last one i published is that there is the other view i’ve integrated into the automate project – the ‘TreeView’

extending QGraphicsItem to be used with a QAbstractItemModel

Q: what is this section about (in a few words)?

A: now as QGraphicsItem is used to represent QModelIndex‘es they need a way to query data from the Model, this is how it can be done

in qt4 there are a few different classes using QGraphicsItem as a base class. originally i wanted to extend the items with virtual inheritance [1] but then the ‘dreaded diamond‘ problem showed me that it can’t be done since i could NOT change the way a QGraphicsTextItem would inherit from QGraphicsItem (read: i couldn’t add the keyword virtual between

‘public _here_ QGraphicsItem’

and if that can’t be done the virtual inheritance can’t be used). of course one could now change the qt core library but that would be wrong ‘in concept’ since my application should be able to be used with any version (for example qt 4.4) not just one single version i’ve changed.

so after hours of figuring i came up with the class GraphicsItemModelExtension [2], it makes use of pure virtual functions and since QGraphicsItem(s) do have a data() and setData() function which is exactly the same function name used with the Model i had to rethink the function calls. now data() and setData() for the Model used from a QModelIndex via GraphicsItemModelExtension are named modelData()/setModelData() instead.

but what makes you want to use the GraphicsitemModelExtension class? in contrast to my previous attempt it stores the QPersistentModelIndex and you don’t have to rely on an existing scene() which is then casted from a QGraphicsScene into a GraphicsScene which then would provide a Model* which finally would be used to query data.

the basic concept after adding an item to the QGraphicsScene is that one needs to query data right in the constructor, for instance you need various properties as ‘label’ or ‘color’. since the normal item now has index() it can instantly (even when not added into a scene() yet) query the Model for data. NOTE: right after an item is added to a QGraphicsScene it does NOT have a scene() yet, if you try to query scene() from a constructor of a QGraphicsItem you WILL get NULL which was a big flaw in my previous implmenetaions. but now that problem is gone and because of the nature of pure virtual functions (yes QGraphicsItemModelExtenstion is actually an abstract base class) it helps to design new custom GraphicItems.

again ‘why i used properties instead of another hierarchy layer’

in the automate [3] project i had a TreeView which does have several columns and one can display one property per column, this is done by the data(..) implementation and is quite easy, just see my code. however in the spring random map generator project [4] i need the properties not in one row per column but instead all properties should be top down with:

module name
propertyName propertyValue (displayed by a delegate)
propertyName propertyValue (displayed by a delegate)
propertyName propertyValue (displayed by a delegate)
…

so i can’t use the data(..) function as i did last time. now i need one QModelIndex per row but my model currently does not have properties as QModelIndex’es at all. so what should i do? i’m still not sure about this… but as soon as i found a nice solution i will write about it.

links

[1] http://www.parashift.com/c++-faq-lite/multiple-inheritance.html

[2] http://github.com/qknight/springrts.com-random-map-generator/blob/master/src/frontend/GraphicsItemModelExtension.cpp

[3] http://github.com/qknight/automate

[4] http://github.com/qknight/springrts.com-random-map-generator

the future of the internet

before thinking about ‘the future of the internet’ let’s have a look what the ‘internet’ actually is. the internet consists of two low level protocols, that is: tcp and udp and the most prominent protocol ip. and many other protocols are building on this, as http for instance. you are currently reading this blog page using http, that means you are also connected to the internet.

if you want to be a part of the internet you need a internet address – in short an ‘ip address‘. since most of use are using ipv4 that means often ppl share one ip which is bad since you can’t address some person sharing an ip with someone else directly. Sharing usually means NAT – network address translation – and it also means that the internet does not work correctly. NAT is a failure and was never meant to be in the original design of the internet.

say you call your friend and he has no phone in his room – but one at his mother’s room. usually you can’t call between  [22,8] o’clock. the same thing is true for nat – you usually can’t reach the other person directly.

the only possible solution to this problem is ipv6. however we fail to integrate it. we currently fail at a lot of things (germany 2010):

• apply ipv6 a standard before others do it (and we have to accept their policies & concepts)
• removing 24h disconnects for dsl
• creating balanced upload/download links
• instead of forcing providers to make dsl uptime 99.9%, we still heavily rely on the telephone system which has in comparison far better service guarantees (for instance dialing 112)
• have more applications using osi-layer 5 (session layer) instead of binding a session to ip:tcp states

So ‘what was the internet again’?

Actually the internet is much more than just what you can do with your browser using http. Before looking at http in detail let’s have a look at popular protocols which do NOT use http.

• the first thing which comes into my mind are all the network implementations for games which most often use UDP and in some rare cases use TCP. this is a very big class of applications
• then we have SIP and similar implementations as ’speech over ip’ in short VOIP. most internet providers tend to like SIP. gamers often use TeamSpeak or Mumble but there is skype folks as well. the most notable difference between all sip like implementations and skype is that skype’s implementation bases on a p2p structure but since it is closed source one can only speculate how chat and speech is using this framework.
• then there are the chat clients in various flavors ranging from irc & psyc and to IM protocols as xmpp(jabber), irc, msn, aim & others. these chat clients – more often than not – have also webinterfaces which let’s YOU access the service via http as well. using http is a very interesting approach since it uses http for that and http usually DOES NOT WORK WELL with such an service but we’ll have a look at that later
• p2p filesharing which is (at least bandwidth wise) the most popular internet service. i think about bittorrent (kademlia) or other similar concepts (this does not include services like rapidshare!).
• last but not least there is a concept  known as ‘middle ware‘.
  for a very long time i did not understand the importance of having a ‘middle ware’ when designing internet protocols. corba and it various implementations for what one would call a ‘middle ware’ and from the protocol designer’s point of view it gives you a very high degree of object interoperability.
  • when a game developer has to write network code usually the choice tends to UDP since the laggy nature of TCP flow control can’t be ignored when one uses a internet connection with lots of packet loss. however UDP basically does not care for lost packages at all which kind of ‘overcompensates’ the problem one wants to solve. so most developers extend UDP until all needed features are there. when the network code is done and objects as players interact with other players via the network (read ip here) the game developer basically has created his own ‘middle ware’.
  • the concept using a ‘middle ware’ is very easy to understand: you don’t want to care what the network looks like you just know it’s there but instead of writing parsers for tcp or udp streams with ‘object serialization‘ and ‘object marshaling‘ one can just use an adapter in the programming language of choice which handles all this. of course using custom types here is most often not as easy as using core types as ‘int’, ‘double’, ‘float’ and ’string’ but handled in most cases.

http problems

so after looking at the use case of ip let’s have a look at http and what problems http has and how they are solved. the biggest issue using http is not the performance hit introduced by html/xml and bandwidth loss on the other hand. instead the biggest issue when using http is the fact that http is a ‘polling’ only protocol so a client might download a webpage as: http://www.lastlog.de but then closes the connection. this works well when you only want to look at webpages and if you are only interested in client side events as mouseclicks & similar. but if you want to implement an interactive chat using http it gets more complicated since you would have to query the server every second for new messages from other users (even if there was no new message at all). the server CANNOT connect to your browser and tell you about ‘the new message’ right away as it is done in all the IM protocols.

so there are some new concepts as spdy:// from google [1] but there are other similar ideas as well. all lead to a next generation http protocol with callback capabilities. opera for instance features ‘a socket in the browser‘ which is a interesting concept. both concepts transform the ‘user’ from a pure consumer into a ‘consumer/producer’.

in the first days most internet providers had ‘terms’ in their ‘terms and agreements‘ which forced you to offer no webservices with your private internet connection. the situation since the internet was introduced has changed and providers now sell devices as the fritz-box with DDNS clients for services as dyndns.org preinstalled. this change needs to be extended further as the classic pattern of ‘private’ and ‘commercial’ usage merge more every day. it gets hard to distinguish between the service levels needed for either of the two scenarios. we need to extend:

• service guarantees (uptime/bandwidth)
• extend upload (currently internet connections are pure consumer connections)
• extended QoS guarantees where i think of ‘big bandwidth but high latency‘ vs. ‘small bandwidth with low latency‘ since the normal ‘telephone line’ will vanish soon we need some way of backup line for emergencies of various kinds

So after some history, what makes opera with websockets revolutionary? It’s the ‘ease of use’ to put content online. You don’t need a provider to host your stuff, just host it yourself! But there are pitfals as upload bandwidth is small even with VDSL, i don’t know why… my scale of guesses range from:

• ‘reducing costs since most users are consumers’ to
• forcing users to ’shut up’ and to sell enterprise servers for much money

there are many points for both sides and i don’t have much background to make a good guess.

Some final thoughts:

• Currently I’m waiting for a revolution: the upcomming ‘ipv6′ could make skype ‘vanish’ since traversing NAT is the biggest problem in creating a ‘free clone’ of skype (this is also valid for SIP). also filesharing as p2p will work much better.
• http will be extended with something like spdy:// very soon, which will make ajax programming pattern much more powerful
• the normal browser will not host only some ajax scripts but will also host parts of the database and programs nowadays running on the webservers only and therefore turn into a local webserver. the first step was ajax, the second step needed is spdy:// and the third step to make this work will be adapted software for webservers to run in such a scenario. looking at the success of http is that it is a easy to use protocol with working standards and it can pass nearly every firewall. it basically hosts all kind of services. the next big step will be to make the browser to look like a application. again – the first step is something like google docs, the second step was making it a tv with plugins as flash. the next revolution for example is compiling c/c++ code in a way that it runs on top of the flash plugin [2]. in the end the browser and any normal host application merge…
• flash will vanish – since the biggest use case is streaming videos and commercials, so guess what users will disable once most videos are played using the in-browser player (which has also huge performance improvements)?
  

links

[1] http://www.chromium.org/spdy/spdy-whitepaper

[2] http://www.flashmagazine.com/news/detail/play_doom_online_-_thanks_to_adobe_alchemy/

libnoise-viewer

i’ve just fixed most bugs in libnoise-viewer [1] and the software is really usable now. it features a QGraphicsScene and a QGraphicsView and to make it faster i’ve used some it::blackmagic called threads. Actually i used QThread and this was my first real use of multi-threading and it works pretty well. When i scroll the planar area small tiles are generated, two at a time are dispatched so that a dual core system is ’saturated’ cpu load wise.

This viewer uses the ‘noise::module::RidgedMulti‘ module to generate the surface in a black/white fashion. All i did was to apply a color-band according to the relative height. As you might have seen already, there is a black/white vs color toggle. Using this you can visualize the heightmap.

if (job.colorstate) {
renderer.ClearGradient ();

renderer.AddGradientPoint ( 1.000000, utils::Color (0, 90, 0, 255));
renderer.AddGradientPoint ( 0.740000, utils::Color (0, 129, 0, 255));
renderer.AddGradientPoint ( 0.700000, utils::Color (0, 190, 0, 255));
renderer.AddGradientPoint ( 0.650000, utils::Color (0, 250, 0, 255));
renderer.AddGradientPoint ( 0.600000, utils::Color (120, 250, 250, 255));
renderer.AddGradientPoint ( 0.390000, utils::Color (20, 120, 200, 255));
renderer.AddGradientPoint ( 0.340000, utils::Color (20, 50, 250, 255));
renderer.AddGradientPoint ( 0.200000, utils::Color (0, 0, 120, 255));

renderer.EnableLight();
renderer.SetLightContrast (3.0); // Triple the contrast
renderer.SetLightBrightness (2.0); // Double the brightness
}

The rendering speed is very good. But see yourself (the upload of the ogv video to wordpress media didn’t work, so i host it somewhere else).

http://lastlog.de/misc/wordpress/libnoise-viewer.ogv

If you want to visualize other ‘libnoise graphs‘ you can edit the sourcode (see renderThread.cpp) , all you need is to add 3 or 5 lines of code:

module::RidgedMulti myModule;
myModule.SetOctaveCount (job.octave);
myModule.SetFrequency (job.frequency);

Again: if you ever have to use threads, use QThreads since signals/slots are threadsafe (coming with qt ~4.2 if i recall correctly) communication between the threads is very easily done using the signals/slot mechanism, which means using the event loop.

I need to say, this is the 3rd project i’ve migrated to cmake and to do that i needed about 30-50 minutes since i also cleaned up the sourcode and did some other things as renaming of files and re factoring of class names.

I’ve also updated to =dev-util/kdevelop-3.9.97 this release fixed all the crashes the previous release had and i absolute love this ide. Currently i can’t think of ever using something else! The cmake integration is very good and makes a lot of fun using it. Renaming of classes or member functions as well as object’s names works like a charm!

Unlike other KDE components kdevelop 3.9.97 works reliably stable for me!

links

[1] http://github.com/qknight/libnoise-viewer

[2] http://github.com/qknight/libnoise

linux bashing

i’m a linuxuser for years now and after migrating from debian to gentoo made the linux experience much better since many ABI incompatibilities were solvable by a simple reinstallation of the program. these kind of problems usually come up after library update and the error message usually is something like ‘unresolved symbols’.

right now i face several problems which i can’t ignore:

• after a reboot the two mouse devices on my laptop stop working and on the next reboot they do work. i don’t know why yet?! my current fix is to rely on an external mouse ;P
• i hit the ‘xfs/ext3 journal’ bug with two massive kde file losses which meant a lot of reconfiguration
• i lost all ‘notes’ which were saved in the plasma widget called ‘notes’ and all kmail settings as well as all kaddressbook settings (yes i have a backup but this all costs a lot of time to figure out what exactly is gone). also all bookmarks in konqueror as well as other things i don’t remember are ‘gone’.
• i use thunderbird right now but the selfsigned ssl certificate error pops up every time i start the application. if i don’t hit the ‘ok’ button in say 2minutes the connection timeout forces me to restart the application or i won’t be able to read the mails at all. why isn’t there a simple button to ‘permanently’ trust the selfsigned certificate as there is in kmail?
• my system seems to be very unstable as for instance working on the ‘automate’ project (which is qt4 based) let’s me ‘lock up’ my computer nearly instantly. that means the io load goes high while i get serious OOM problems. it could be the ‘io wait bug’ but using ‘noop’ instead of ‘cfq’ does not make things better here so i doubt that
• my system crashed after a hibernate resume with a blinking cursor using nvidia.ko and now i can’t use the proprietary nvidia.ko driver anymore since it would lock my system right after reboot. currently i’m using ‘nv’ with all it’s slowness
• neither nv nor nvidia.ko seem to be able to work reliably for hibernate-ram. nv does not enable the backlight so my screen stays back after resume so at least i can reboot with pressing the power button. using the nvidia.ko module will crash it no matter which version i try
• i don’t get hdmi working and hot-switching using nvidia.ko does not work well either
• io in general seems to be very slow and many processes have a high ‘io wait’ issue. using htop one can see that both cpus have about 90% gray bars with the rest – 10%  – is valid cpu load (a mixture of user/kernel space)
• booting linux using grub works fine but is very slow (probably caused by slow nvidia console rendering)
• kde4 is very slow as well (might be a consequence of the io-wait issue already mentioned) but using kde3 this issue wasn’t noticeable. the problem is that there are no kde3 ebuilds anymore.
• konqueror seems to crash all 5minutes (i use it to access samba shares using smb:// as well as sftp shares)
• krunner other kde components seem to crash frequently as well (seems to be a random pattern)
• amarok does not work reliably from time to time then i can’t listen to music from magnatune and other webradios but local music works
• smb4k is totally broken here
• kdevelop4 is a great software and i really love it but it is very unstable and crashes very often
• out of curiosity i’ve installed umbrello 4.3.3 and imported the automate project which instantly created a segfault
• since kde3 is ‘obsolete‘ i lose a very cool application: konqueror 3 file size view’er, see http://risto.kurppa.fi/blog/2008/10/disk-space-analyzers/ there seems to be a kde4 plugin for konqueror as well but i can’t find it
• after resume dbus often has crashed and i need to restart it

so now one would think the hardware is bad (which is what i would be thinking as well) but i’ve played some shooter demos and ’sacred 2′ using windows xp for quite a while and this runs rock solid. the only issue windows shares is that after ’standby’ the screen remains black (back-lit off) as well.

to sum up: i’m not sure what to do now but this situation is absolutely depressing since linux can’t be used AT ALL with that many problems. lately i consider installing win7. the ‘macbook pro’ of my wife is doing a great job and it runs stable although the fan makes some serious noise signaling it is damaged. i hate kde4 and the concept of their release policies. i consider kde4 not usable in a productive system and since kde3 has been removed there is no working alternative worth using (maybe xfce4).

i’m not sure what to do now? is the problem ‘open source’? why does nobody care for quality? am i using the wrong distribution again?

happy new year ;P

QGraphicsScene used as a QAbstractItemView

I’ve once written a qt4 program which does use the QGraphicsScene in a very unique way. Since i need the same concept for another project i started to read the code i’ve written quite some time ago. The code [1] is very complex so i got lost very soon but as a solution i came up with this graph, ‘illustrating the concept and the class design‘. There was a feature i wanted to add and it might sound funny but i used this graph ‘as a map’ to navigate through the source.

QGraphicsScene used as a QAbstractItemView

First a few words on the graph:

If you want to do the same thing keep in mind that every layer of abstraction duplicates the data once per instance but the data acquired through the model is the only data which is authoritative. All other copies are for caching purpose and to keep structural integrity for instance a QGraphicsScene needs all items to be placed on the scene this way it is easier for the programmer.

In the early design phase i thought it would be efficient to render only the part of the scene which is actually visible. QGraphicsScene can’t be seen by definition. It is the QGraphicsView which attaches to the scene and which then visualizes the part of the scene (or the whole scene) if it’s viewport is configured to do so. Taken efficiency into account rendering only ‘what is seen” is already given but we have to insert all the nodes and connections into the QGraphisScene. It is important to know that a QGraphisScene is also a model based design that’s why a single QGraphisScene can have several different QGraphisView attached to it. One can be rotated, the other can be zoomed out – do with it whatever you desire. All i did was wrapping the QGraphisScene’s model into another layer.

We currently have 4 layers and we have to wrap something 3 times:

• first we wrap the data to be used by the model, this happens with QModelIndex()es
• second we wrap QModelIndex()es into QPersistentModelIndexes()
• third we wrap QPersistentModelIndexes() into QGraphicsItems()

So one might wonder why did i NOT use the QAbstractItemView in the first place? This is probably the most important question!

A QAbstractItemView gives you a paint() function and you have to draw the scene yourself. There is an excellent example called ‘chart example’, see [2]. It might be a very good solution for many problems which don’t require complex ‘user <-> item’ interaction – think of ‘move a node or you click a node’. (please play with the example, and look at the code).

A QGraphicsScene has it’s own model which handles all the items. Items can have various attributes as moveable/clickable/selectable. You can layer items so that they have individual or group rotation properties. You can use predefined items or you can create your own. A very important point is you have a collision detection and items can be clicked by the mouse and items can even receive keyboard input.  See example [3].

It should be clear now that a ‘graph editor’ needs all the later mentioned capabilities.

I want to discuss several things:

1. inserting/deleting items with the graphical editor
2. how to read and write data seen from a QGraphicsItem
3. why i used properties instead of another hierarchy layer
4. why GraphicsItems access the model NOT being just a delegate

1. inserting/deleting items with the graphical editor

the fundamental concept is: “data is only inserted/deleted through the model”. The model is the most important component in MVC [4] abstraction. If you add a node with the gui you can either place the node by random (which is what i do mainly) or you can query the mouse position (if the mouse is hoovering above the QGraphicsView) and map it to the QGraphicsScene coordinates and finally you ask the model to add a node with addNode(coordinates) using the models public interface which can be accessed from the GraphicsView (NOT QGraphicsView). The model will add a new item at the data layer and the data layer will delegate the ‘added’ item upwards in the hierarchy.

That’s why there are functions with active and passive names as:

• insertNode()
• nodeInserted()

The first is an interface to add nodes. The second will be called when the updated is delegated upwards. You never call a passive named function directly since it is ONLY called by the QAbstractItemModel/QAbstractItemView code.

Removing items is basically the same but now you have an identifier. A item is removed based on it’s identifier which can be a QPersistentModelIndex for example.

2. how to read and write data seen from a QGraphicsItem

A QGraphicsItem (and all derived types) live in the QGraphisScene and don’t have access to the model. It is conceptually easy to generate such an object based on the information given by the model (read simplex communication here) – which can be done in the init() function when the view is initialized and filled with all nodes and node connections.

However it can be  challenging creating a backward communication channel (read duplex, bidirectional). In an ideal world this would be done in a way that a AbstractGraphicsItem inherits from QGraphicsItem. This AbstractGraphicsItem then is specialized in NodeGraphicsItem and into a ConnectionGraphisItem.

The AbstractGraphicsItem would add the communication layer (this is not done in my example). The specialization layer would do the custom drawing stuff as a node is drawn differently than a node_connection.

In my example the QGraphicsScene does setData() and data() wrapping. This is a hack which works nice but i’d prefer the delegate design now since it is a clear interface design.

Most properties are set/queried by: model->data(), model->setData(). One problem for example is when you move one item. Would it make sense to update the position through the model? Probably not since this would slow down the whole operation. Since the QGraphicsScene provides a signal which is emiteed when an moved item is released this signal could be used to update the position of the item through the model. This operation is asynchronous.

For all other properties as ‘node label’ and ‘node_connection color’ and similar it only makes sense to use model->setData() and wait for the model to update the item (which just asked ‘could you please update my color?’). This operation is absolutely synchronous.

3. why i used properties instead of another hierarchy layer

I’m not sure on this yet and i think i have to think about this more time. However i share some thoughts.

my data currently has these 3 types:

• ROOT_NODE
• NODE
• NODE_CONNECTION

and properties for node as for example: nodelabel, startflag, final flag (yes NFA/DFA/… do you remember?) why not add another type called PROPERTY instead of using setProperty() and property() (which is provided by QObject).

i call the type approach (using  PROPERTY) from now on: ‘typeapproach‘  and the QObject::property() approach i call ‘qobjectproperty‘.

In either concept the model code would not be that different. Using QObject::properties one would (as i did) use data() and setData() and query the needed information – a QVariants is returned and automatically converted to a type usable by the view.

The most significant difference when using the ‘typeapproach’ would be the updates for the items in the QGraphisScene. One would have a QPersistentModelIndex() per property (nodelabel/start flag/final flag) and one would instantly know what to change when updateData() want’s the QGraphicsItem to change it’s label. Currently i’m using the ‘qobjectproperty’ and i have to check all those properties every time updateData() is called.

I’m not sure what is the best strategy but i would guess the ‘typeapproach’  could be very interesting.

4. why GraphicsItems access the model NOT being just a delegate

as mentioned earlier this would be the best design. if an item of type NODE is required a respective QGraphicsItem, say a NodeGraphicsItem is synthesized. i probably will use this design for the next project.

some final words

i’ve written this page for myself since i’m currently planning a similar project using a QAbstractItemModel with a QGraphicsScene. but as this subject is complex i’d like to share this with anyone out there.

if you have some thoughts on this, please let me know by email: js adt lastlog dodt de

links

[1] http://lastlog.de/wiki/index.php/AutomateEditor

[2] http://doc.trolltech.com/4.1/itemviews-chart.html

[3] http://doc.trolltech.com/4.3/graphicsview-collidingmice.html

[4] http://de.wikipedia.org/wiki/Model_View_Controller

[5] http://doc.trolltech.com/4.2/itemviews-simpletreemodel.html

updates

(13.12.2009) the title of this document was of course wrong, it must be: QGraphicsScene used as a QAbstractItemView and not ‘used as a QAbstractItemModel’ ;P

the example for the low level ‘data’ structure which is refered as simpletreeexample is actually called ’simple tree model’ example, see [5]

cmake / kdevelop 4

i’ve converted the second project to use ‘cmake‘. this time an qt4 application which usually uses qmake.

i love cmake. it is easy to use (once you get used to the cmake of doing things). i love ‘out of source’ builds. kdevelop4 can handle cmake so i decided to move into the future and use cmake for my kdevelop4 projects as well. so far a very good decision.

i did have problems at first. using cmake with normal cpp/h files is easy and well documented but using it with moc|uic|rcc is different. i searched for help in irc.freenode.net#cmake and ‘Marc Bowes’ gave me a link [2] to his github.com project which uses cmake for a qt4 project – an excellent resource. looking at that code made my work easy.

in case you want to convert your qt4 applications from qmake to cmake see [1] and [2]. in general i recommend reading [3], [4] and [5].

i probably said enought about cmake but one more thing a kde developer told to me:

when using cmake with uic (when using qt4) one has to maintain a list of QObject deriving classes which are then processed by a macro called: ‘QT4_WRAP_UI’. He said when using cmake with kde4 projects cmake does this automagically. This sounds great – maybe someone ports that code over to the qt4 macro – would be nice!

So finally some words about kdevelop4. I loved kdevelop3 but refactoring was a pain with kdevelop3 – that changed with kdevelop4. The language parser and ‘auto completion‘ feature now works so well you don’t think of kdevelop4 as a normal editor with syntax highlighting but more as a fully automated code generator.

Kdevelop4 can now have several projects in one widget (in kdevelop3 i needed two instances – resulting in issues since starting the second instance usually would resume the project which was used at last so one would end up having two IDEs with the same project opened). Kdevelop4 has also support for several worksets – this is one of the best features. Currently i consider it the greatest c++ IDE i’ve ever used and i didn’t even mention ‘compile sets‘ yet…

[1]  http://github.com/qknight/springrts.com-random-map-generator

[2]  http://github.com/marcbowes/mxitc

[3] http://www.linux-magazin.de/Heft-Abo/Ausgaben/2007/02/Mal-ausspannen

[4] http://www-flc.desy.de/ldcoptimization/documents/talks/CMake_Tutorial.pdf

a new workstation

source hp.com

finally i bought a new workstation, a ‘hp elitebook 8530w‘. this is unusual since i prefer ‘ibm’ computers in genreal – but i could not find any ibm computer which was as powerful as this hp laptop featuring a nvidia card. my first impression of the hp was that it’s well designed and the fan wasn’t bad at all. altough the fan is not perfect but i can live with it. another far more important point is that it DOES NOT HAVE ANY ATI (AMD) COMPONENT IN IT which means: NO more suffering. instead it uses a nvidia graphics card:

$ lspci
01:00.0 VGA compatible controller: nVidia Corporation Device 065c (rev a1)

$ cat /var/log/Xorg.0.log | grep Quadro

(II) NVIDIA(0): NVIDIA GPU Quadro FX 770M (G96GL) at PCI:1:0:0 (GPU-0)

I did not install gentoo the usual way but instead i cloned the image from one of my desktop computers (which all happen to be intel c2c duos) with 64bit usage. So the installation was done in about 30minutes (with some help of [1]). i could not install gentoo right away since i considered ‘bouncing the purchase’. the hard drive can be removed easily and the way it is mounted to the laptop looks ‘rock solid’ (it is fixed by 3 screws). so next i attached my desktop hard drive with a usb adapter but grub could not load the kernel and it came up with the mystic error code “error 17″. soon i found out that the kernel needs to be in the first 1gb of disk space (or maybe 4gb? or 10gb?) i really don’t care. it was at about 120gib away from the first sector. so i removed the first partition (which wasn’t in use) and installed a 100mb partition with xfs and i copied the kernel there. now everything worked and i could test the computer without having any problems with that ‘windows vista’ i didn’t want to damage (else i would not be able to return the device). finally i removed windows vista after i found out that everything worked nice – which meant i would not return the device!

The only things which didn’t work then were:

• lan 00:19.0 Ethernet controller: Intel Corporation 82567LM Gigabit Network Connection (rev 03)
• wlan 03:00.0 Network controller: Intel Corporation Device 4236
• webcam (don’t need that)
• second mouse (touchpad – i don’t need that either)

suspend2ram works with the proprietary nvidia driver ‘x11-drivers/nvidia-drivers-185.18.36‘. why should i bother with this amd open  source crap if a nvidia driven pc works out of the box? using spring (springrts.com, game) i have about 280fps peak (in windowed mode) while compositing is running. kde runs at 60fps. one would not expect that – but it’s real!

the power drain using suspend2ram is about 0.6W!  the display 1920 x 1200 Pixels anti-glare and it is very good compared to my ‘thinkpad t43′ and it is excellent compared to my wifes mac book pro’s glare display. overall i’m so happy i didn’t buy a ‘mac book pro’ since i’d be running linux on it and i would expect more problems with a mac book pro running linux than this ‘hp elite book’ – i could be wrong though.

there was one problem. i usually protect my private data with some encryption -> luks and my static 32bit initrd didn’t work anymore since cryptsetup segfaulted on boot. so i had to create a new initrd image which didn’t work at all. i really don’t know why. maybe init couldn’t be loaded because busybox (although linked statically) couldn’t find init. so i thought about giving up and do the installation without luks support – and just in that moment eliasp (a friend of mine) mentioned to me that he usually uses genkernel for doing that.

so i tried that (not expecting genkernel to be able to help me). i simply couldn’t imagine that someone wrote support for a lvm inside a luks-image which means:

• /dev/sda3 is a luks volume which is first opened with cryptsetup luksOpen and
• afterwards the decrypted partition can be interpreted as lvm vg in ‘/dev/vg/’.

but i was wrong with my assumption – the initrd image produced by genkernel worked out of the box, all i needed to do was issuing:

genkernel –lvm2 –luks ramdisk

I have a more detailed installation guide at [2].

To sum up: this device is a dream and if you should consider buying a laptop consider this one!

[1] http://en.gentoo-wiki.com/wiki/HP_Elitebook_8530w

[2] http://lastlog.de/wiki/index.php/Festplatte_verschluesseln (german only)

--[[
When this widget is enabled, air transports guarding factories will
automatically take new units built at the factory to the destination of the factory
--]]

function widget:GetInfo()
  return {
    name      = "Transportation Assister",
    desc      = "Transports guarding factories will automatically move new units",
    author    = "Ray",
    date      = "Feb 18, 2007",
    license   = "GNU GPL, v2 or later",
    layer     = 0,
    enabled   = false
  }
end

--------------------------------------------------------------------------------
--------------------------------------------------------------------------------

include("spring.h.lua")

function widget:Initialize()
	local _, _, spec = Spring.GetPlayerInfo(Spring.GetMyPlayerID())
	if spec then
		widgetHandler:RemoveWidget()
		return false
	end
end

function widget:UnitFromFactory(unitID, unitDefID, unitTeam, factID, factDefID, userOrders)
	if unitTeam ~= Spring.GetMyTeamID() then return end

	local cQueue = Spring.GetCommandQueue(factID)
	local UD = UnitDefs[Spring.GetUnitDefID(unitID)]
	local transID = null
	if not UD.springCategories["notair"] or not UD.springCategories["notship"] then return end
	for _, elem in ipairs(cQueue) do
		if elem.id ~= CMD_MOVE then
			return
		end
	end

	transID = IdleTransport(factID)
	if transID ~= null then
		MoveUnit(unitID, factID)
		TransportUnit(unitID, transID, factID)
	end

end

function TransportUnit(unitID, transID, factID)
	local transCQueue = Spring.GetCommandQueue(transID)
	local params = {}
	local newParams
	local tries = 0
	if table.getn(transCQueue) > 2 then
		Spring.GiveOrderToUnit(transID, CMD_GUARD, {factID}, {"shift"})
		Spring.GiveOrderToUnit(transID, CMD_LOAD_UNITS, {unitID}, {"shift"})
	else
		Spring.GiveOrderToUnit(transID, CMD_LOAD_UNITS, {unitID}, {""})
	end
	local fCQueue = Spring.GetCommandQueue(factID)

	local UD = UnitDefs[Spring.GetUnitDefID(transID)]

	for num, elem in ipairs(fCQueue) do
		params = elem.params
		newParams = params
		while OverOldCmd(transID, newParams) and tries < 20 do
			newParams = {params[1] + math.random(1,100), params[2], params[3] + math.random(1,100)}
			tries = tries + 1
		end
		if num ~= table.getn(fCQueue) then
			Spring.GiveOrderToUnit(transID, CMD_MOVE, newParams, {"shift"})
		else
			Spring.GiveOrderToUnit(transID, CMD_UNLOAD_UNIT, newParams, {"shift"})
		end
	end
	Spring.GiveOrderToUnit(transID, CMD_STOP, {}, {"shift"})

	Spring.GiveOrderToUnit(transID, CMD_GUARD, {factID}, {"shift"})

end

function OverOldCmd(unitID, params)
	local cQueue = Spring.GetCommandQueue(unitID)
	local nParams
	for _, elem in ipairs(cQueue) do

		if elem.id == CMD_MOVE or elem.id == CMD_UNLOAD_UNIT then
			nParams = elem.params
			if nParams[1] > params[1] - 30 and nParams[1] < params[1] + 30 then
				if nParams[3] > params[3] - 30 and nParams[3] < params[3] + 30 then
					return true
				end
			end
		end
	end
	return false
end

function MoveUnit(unitID, factID)
	local fX, fY, fZ = Spring.GetUnitPosition(factID)
	local bF = Spring.GetUnitBuildFacing(factID)
	local xAdd
	local yAdd
	if bF == 0 then
		xAdd = 0
		yAdd = 70
	elseif bF == 1 then
		xAdd = 70
		yAdd = 0
	elseif bF == 2 then
		xAdd = 0
		yAdd = -70
	else
		xAdd = -70
		yAdd = 0
	end
	Spring.GiveOrderToUnit(unitID, CMD_MOVE, {fX + xAdd, fY, fZ + yAdd} , {""})

end

--------------------------------------------------------------------------------
--------------------------------------------------------------------------------

function IdleTransport(factID)
	local cQueue
	local leastQCount
	local idleTrans = null
	local lastCmd
	local uList = Spring.GetTeamUnits(Spring.GetMyTeamID())
	local leastDist = null
	for _, ID in ipairs(uList) do -- cycle through every unit
		UD = UnitDefs[Spring.GetUnitDefID(ID)]
		if UD.isTransport then
			cQueue = Spring.GetCommandQueue(ID)
			lastCmd = table.getn(cQueue)

			if (lastCmd > 0 and cQueue[lastCmd].id == CMD_GUARD and cQueue[lastCmd].params[1] == factID) or (lastCmd > 1 and cQueue[lastCmd-1].id == CMD_GUARD and cQueue[lastCmd-1].params[1] == factID) then

				if idleTrans ~= null then
					if table.getn(Spring.GetCommandQueue(ID)) < table.getn(Spring.GetCommandQueue(idleTrans)) then
						leastQCount = table.getn(Spring.GetCommandQueue(ID))
						idleTrans = ID
					end
				else
					leastQCount = table.getn(Spring.GetCommandQueue(ID))
					idleTrans = ID
				end
			end

		end
	end

	if idleTrans == null then
		return null
	end

	for _, ID in ipairs(uList) do -- cycle through every unit
		UD = UnitDefs[Spring.GetUnitDefID(ID)]
		if UD.isTransport then
			cQueue = Spring.GetCommandQueue(ID)
			lastCmd = table.getn(cQueue)

			if (lastCmd > 0 and cQueue[lastCmd].id == CMD_GUARD and cQueue[lastCmd].params[1] == factID) or (lastCmd > 1 and (cQueue[lastCmd-1].id == CMD_GUARD and cQueue[lastCmd-1].params[1] == factID)) then
				if lastCmd == leastQCount or lastCmd == leastQCount + 1 then
					if leastDist ~= null then
						if DistFromFac(ID, factID) < leastDist then
							leastDist = DistFromFac(ID, factID)
							idleTrans = ID
						end
					else
						leastDist = DistFromFac(ID, factID)
						idleTrans = ID
					end
				end
			end
		end
	end

	return idleTrans

end

function DistFromFac(transID, factID)

	local uX, uY, uZ = Spring.GetUnitPosition(transID)
	local fX, fY, fZ = Spring.GetUnitPosition(factID)
    return math.sqrt((uX - fX)^2 + (uZ - fZ)^2)

end

function echo(msg)
  Spring.SendCommands({"echo " .. msg})
end

cmake and doxygen

i’ve just rewrote a library called libnoise to use cmake (instead of static Makefiles/libtool). i have to admit that cmake is probably the best buildsystem i’ve ever used. cmake does so many things better than others – but you probably know that already so i will focus on different things.

i had lot’s of problems using doxygen with an ‘out of source’ build. but i finally fixed that. i consider my hack an excellent example of how to use doxygen with cmake. the first thing i tried was hacking on [2], a posting from “Philippe Poilbarbe” (i don’t know this developer) which i couldn’t get running at first. exhausted i tried something else (for those who are interested, it was [3] a mail about a library called ‘libLASi’ using doxygen together with cmake (see [4] for libLASi). now happy to get doxygen working the ‘libLASi way’ at least  i decided to branch back to my first attempt which was [2].  finally i got that working, too. so [2] is the basis for this posting – keep that in mind.

i don’t like the way doxygen is integrated in cmake by the ‘libLASi’ developers. why?

because they write to the source directory (called ‘in source build‘) and this is not how cmake developers (that is those who wrote cmake)  consider stuff to be done right.

if you ever have to use doxygen with cmake you should look at my code [2]. it will compile on linux (probably unix in general) and all you need is a g++ compiler and doxygen installed.

in general to adapt the cmake/doxygen from any other build system keep this in mind:

• the “Doxygen” file most often static will be rewritten (then called “Doxygen.in“) and some placeholders are inserted as:
  PROJECT_NUMBER         = @LIBNOISE_VERSION@
• in general one could say “Doxygen.in” needs some adaption to receive cmake variables contents as i already wrote in the libnoise sourcecode (see quote below)
• the cmake script called: CMakeLists.txt dealing with the docu (doxygen) will then replace this placeholders on the fly – that is when executing ‘cmake .‘
• nearly all variables (i can’t think of any which would not work but i didn’t check them either) can be modified this way
• one variable is replaced by a REGEXP (see quote below as well)

# you could also set the version with this, see Doxygen.in
# there you will find a line like this:
#      PROJECT_NUMBER         = @LIBNOISE_VERSION@
# @LIBNOISE_VERSION@ is then replaced by our global LIBNOISE_VERSION
#
# for instance you could uncomment the next 3 lines and change the version for testing
# SET(LIBNOISE_VERSION
#     “1.2.3-foo500″
# )

# doxygen can reference external images with IMAGE_PATH, this is how we set it dynamically
SET( CMAKE_DOXYGEN_IMAGE_PATH
“${CMAKE_CURRENT_SOURCE_DIR}/img”
)

# doxygen searches for source code (defined in FILE_PATTERNS, for example: *.cpp *.h)
# with DOXYGEN_SOURCE_DIR we fill a list of directories and later we write it into
# the Doxyfile with a REGEX REPLACE (see below)
SET( DOXYGEN_SOURCE_DIR
${CMAKE_SOURCE_DIR}
)

SET(DOXYGEN_OUTPUT_DIR html)
STRING(REGEX REPLACE “;” ” ” CMAKE_DOXYGEN_INPUT_LIST “${DOXYGEN_SOURCE_DIR}”)
CONFIGURE_FILE(Doxyfile.in ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile)
SET(HTML_TARGET “html” )
ADD_CUSTOM_TARGET(${HTML_TARGET} ALL
/usr/bin/doxygen ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile)
INSTALL( DIRECTORY “${CMAKE_CURRENT_BINARY_DIR}/html/” DESTINATION “/usr/share/doc/libnoise-${LIBNOISE_VERSION}” )

• it is very important to understand what ${CMAKE_CURRENT_SOURCE_DIR} and ${CMAKE_CURRENT_BINARY_DIR} are used for

i hope you won’t loose as much time as i did figuring this out.

finally: thanks very much for help from straszheim#cmake@irc.freenode.net (general cmake questions, nothing doxygen related) and thanks to the example at [5] but it wasn’t very helpful since i didn’t understand it – nice try, maybe next time!

[1] http://github.com/qknight/libnoise

[2] http://www.cmake.org/pipermail/cmake/2006-August/010794.html

[3] http://marc.info/?l=cmake&m=119868994732284&w=2

[4] http://lasi.svn.sourceforge.net/viewvc/lasi/trunk/

[5] http://tobias.rautenkranz.ch/cmake/doxygen/

UPDATE(30Nov09): replaced static path with ${CMAKE_SOURCE_DIR} in the quoted script above

n0 – netw0rk manager (source release)

I’ve released nNull, please see [1] as official page and find the source at [2].

Some words on the release:

• you can use this client/server to execute any kind of script, not only network scripts (might be handy for other things as well)
• qt4.5+/dbus/GPLv3
• nNull-server is a console qt program (uses no gui)
• nNull-client is a normal QWidget with a systray icon, the icon has a context menu

http://www.lastlog.de/wiki/index.php/NNull

http://github.com/qknight/nNull

maybe someone likes it? if so, please drop me a mail