Informatics

Axolotl statistics script… and gource animation!

In a previous post I introduced Axolotl, a documentation system specially suitable for sysadmins, and I stated there that the simple text files generated under this system can be easily parsed to extract statistics.

But in fact these statistics can also be easily converted to Gource animations and videos, so I updated my statistics script just a little, in order to extract a log file suitable as input to Gource.

I’ve published it at my Axolotl’s github project as axolotl.statisctics.gource.pl.

As I was trying to obtain the video from the animation, I obtained this error in a Windows OS, preventing me to create the video on-the-fly throu ffmpeg processing!

$ gource -1920×1080 –key log.txt -o –

gource: stdout PPM mode not supported on Windows

So I dug in the original code and after compiling it in Windows – which was quite a process, btw! – I found that just a few changes would done the work, so I’ve sent a pull request to update the executable with this option enabled.

In the meantime, this zip contains the recompiled Gource for Windows (v0.45 beta) with “-o, –output-ppm-stream” to stdout (“-“) enabled. Decompress over the original Gource installation directory, or just decompress to a new folder and execute gource.exe from there.

After patching this, it was easy to create this example video of a development documented with Axolotl over the years, using the axolotl.statisctics.gource.pl statistics script which creates branches for years and sub-branches for months, and sets different colours for each different tag used in the documentation. The “–key” gource option allows later for a quick peak at the most repeated tasks in the whole period.

axolotl-statistics-gource-mp4

Adjusting “-i, –file-idle-time” gource option to different integer values, the period over which –key shows tags will vary, so the dynamic count for the most used tags in the approx. last n months or whatsoever can be obtained.

The video has been obtained from a “test” directory containing more than a thousand of real Axolotl documents, using the axolotl.statisctics.gource.pl script, and then using the generated axolotl.gource.log file as input to gource.

C:\data> perl axolotl.statisctics.gource.pl test > axolotl.statistics.txt

C:\data> gource -1920×1080 –user-scale 4 –key –hide date,filenames,bloom -s 0.01 axolotl.gource.log -o – | ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i – -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -threads 0 -bf 0 -b 10000000 axolotl.statistics.gource.mp4

Note that in order to correctly identify the tags that represent authors (the people in gource animations!) those strings must be indicated in the script at this line, for example with:

my @AUTHORS=(‘author1’, ‘author2’);

which will match axolotl tags [[author1]] and [[author2]]. Note that axolotl tags are case insensitive, so [[AUTHOR1]] would also match as a recognized author (the same as its lowercase counterpart).

The script axolotl.statisctics.gource.pl also outputs (to stdout) a table with counts of tasks and percentages weighted by the length (in plain text lines) of each task, among other values. An example of statistics output from the script:

    Years comprised:        10
Days comprised:         1503 files, 1616 different days identified
Tasks found:                3591
Total number of lines:  297864
Different tags found:   1839

#tags %weighted %tasks lines  tag
1136  33.9%     30.7%  104858 anonymized_tag_1333
253   10.6%     6.8%    32779   anonymized_tag_234
99    8.2%      2.7%    25200    anonymized_tag_3
260   7.3%      7.0%    22442   anonymized_tag_22
159   6.8%      4.3%    21077    anonymized_tag_139

The table has been formatted with Perl Text::Table, but unfortunately wordpress doesn’t show it perfectly :-(

Both the statistics and the log file for gource has been anonymized previous to publishing.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s