The SDL forums have moved to discourse.libsdl.org.
This is just a read-only archive of the previous forums, to keep old links working.


SDL Forum Index
SDL
Simple Directmedia Layer Forums
full crashreporter in OpenLieroX
albert


Joined: 22 Oct 2009
Posts: 29
Heya,

I thought that maybe interesting for some other devs here:

We just finished a full crashreporter system, based on Google Breakpad. It doesn't use the same architecture though - the whole report is generated on the client side. I.e., if it crashes, it will generate a nicely readable stacktrace from all threads and some more stuff, add the users configfile (stuff like passwords are removed automatically) and also adds the full stdout+stderr output from the crashed process and sends that to some mail.

Technically:

For stdout+stderr, for Linux/Unix/MacOSX, I coded something similar to the tee-tool but somehow more handy - this is then forked from the process, then dup2 on 0 and 1. For Windows, I am just doing an freopen. This teeStdout-handler forwards the data both to stdout and to a file (or at the very beginning into a buffer before it has a handy logfilename). In all cases, it even writes all the output as long as the application is running, no matter what happens (so absolutly no problems when it crashes).

When it crashes, Breakpads client part will generate a minidump and then start a new instance of our game with that minidumpfile and the logfile as a parameter.

That new instance will generate all needed Breakpad Symbolfiles on the fly and parse the minidumpfile and generates the crashreport. Then it collects the whole environment, the data from logfile and the configfile. It has a small SMTP client builtin and uses this to post everything to our mailinglist (later on perhaps directly into some bugtracker).

Breakpad needs some extensions to work that way. A lot of fixes for Windows were needed to make all the non-client part working (like the processor and that stuff). Also, for Linux, DWARF support was missing, only Stabs was supported (whereby DWARF is the common standard). Some people worked already on that but that was somehow incomplete and didn't worked for me (see http://code.google.com/p/google-breakpad/issues/detail?id=130 for more details). But I fixed that (at least that it works on all systems where I have tested it).

So, we finally have a really simple crash reporting tool which works for Windows, MacOSX and Linux and we don't really have to do any additional work to keep that system running.

I just made the first public release today, so we'll how it behaves in the wild. Smile

If you want to try the end result: It's OpenLieroX 0.58 beta5: http://openlierox.net , https://sourceforge.net/projects/openlierox/

If you want to see the code:
Breakpad stuff:
http://openlierox.git.sourceforge.net/git/gitweb.cgi?p=openlierox/openlierox;a=tree;f=src/breakpad;hb=refs/heads/0.58
TeeStdout stuff: (search for teeStdoutInit)
http://openlierox.git.sourceforge.net/git/gitweb.cgi?p=openlierox/openlierox;a=blob;f=src/main.cpp;hb=refs/heads/0.58

- Albert
albert


Joined: 22 Oct 2009
Posts: 29
Here three examples, always the same crashing dummy function (Cmd_crash::exec):

MacOSX:
http://sourceforge.net/mailarchive/message.php?msg_name=20091028000030.83B5550DDFF%40mail.az2000.de

Windows:
http://sourceforge.net/mailarchive/message.php?msg_name=20091023133141.C202E50DE5A%40mail.az2000.de

Linux:
http://sourceforge.net/mailarchive/message.php?msg_name=20091023131149.EE97E50DE52%40mail.az2000.de

Note also the added console output after the crash report.
full crashreporter in OpenLieroX
Sam Lantinga


Joined: 10 Sep 2009
Posts: 1765
Very cool! Smile

On Thu, Oct 22, 2009 at 12:03 AM, albert wrote:
Quote:
Heya,

I thought that maybe interesting for some other devs here:

We just finished a full crashreporter system, based on Google Breakpad. It doesn't use the same architecture though - the whole report is generated on the client side. I.e., if it crashes, it will generate a nicely readable stacktrace from all threads and some more stuff, add the users configfile (stuff like passwords are removed automatically) and also adds the full stdout+stderr output from the crashed process and sends that to some mail.

Technically:

For stdout+stderr, for Linux/Unix/MacOSX, I coded something similar to the tee-tool but somehow more handy - this is then forked from the process, then dup2 on 0 and 1. For Windows, I am just doing an freopen. This teeStdout-handler forwards the data both to stdout and to a file (or at the very beginning into a buffer before it has a handy logfilename). In all cases, it even writes all the output as long as the application is running, no matter what happens (so absolutly no problems when it crashes).

When it crashes, Breakpads client part will generate a minidump and then start a new instance of our game with that minidumpfile and the logfile as a parameter.

That new instance will generate all needed Breakpad Symbolfiles on the fly and parse the minidumpfile and generates the crashreport. Then it collects the whole environment, the data from logfile and the configfile. It has a small SMTP client builtin and uses this to post everything to our mailinglist (later on perhaps directly into some bugtracker).

Breakpad needs some extensions to work that way. A lot of fixes for Windows were needed to make all the non-client part working (like the processor and that stuff). Also, for Linux, DWARF support was missing, only Stabs was supported (whereby DWARF is the common standard). Some people worked already on that but that was somehow incomplete and didn't worked for me (see http://code.google.com/p/google-breakpad/issues/detail?id=130 for more details). But I fixed that (at least that it works on all systems where I have tested it).

So, we finally have a really simple crash reporting tool which works for Windows, MacOSX and Linux and we don't really have to do any additional work to keep that system running.

I just made the first public release today, so we'll how it behaves in the wild. [img][/img]

If you want to try the end result: It's OpenLieroX 0.58 beta5: http://openlierox.net , https://sourceforge.net/projects/openlierox/

If you want to see the code:
Breakpad stuff:
http://openlierox.git.sourceforge.net/git/gitweb.cgi?p=openlierox/openlierox;a=tree;f=src/breakpad;hb=refs/heads/0.58
TeeStdout stuff: (search for teeStdoutInit)
http://openlierox.git.sourceforge.net/git/gitweb.cgi?p=openlierox/openlierox;a=blob;f=src/main.cpp;hb=refs/heads/0.58

- Albert


_______________________________________________
SDL mailing list

http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org




--
-Sam Lantinga, Founder and President, Galaxy Gameworks LLC