Tue, 29 Apr 2008
GCC Tail-call Bug
Today I spent several hours debugging one of my distributed applications. The bug appeared to be in the sorting routine, which sometimes left the data not completely sorted. The strange thing was that when I added a debugging test at the end of each quick-sort recursive call (thus making sure I catch the unsorted data as early as possible), the problem has disappeared.
So, another Heisenbergish bug, I think. The source code is here: it is a straightforward recursive quick-sort, sorting entries with 32-bit key and 32-bit value (written to work on 64-bit systems only, do not bother to report that it is broken on legacy systems).
Now try to compile it and run with gcc -O2. For me
(gcc 4.1.2 from Fedora 8) it works. However, when using -O3
(or even -O2 -finline-functions) it doesn't. Also when
you uncomment the debug printout at the end of qsort_data(),
it should work even with -O3. I think the tail-call recursion
is not enabled when the debugging code is present.
Does it work for you? Is my code broken in a way I don't see? If this is not only a problem of my system, I will try to report it as a gcc bug.
5 replies for this story:
Spes wrote:
Yeah, I confirm the problem on all my system types -- Fedora 6, 8, CentOS 5, Gentoo 2008.0 beta (all with the gcc 4.1.2). I didn't review the code.
Spes wrote:
Btw. valdgrind is complaining about invalid reads and writes.
Yenya wrote: Re: valgrind
Well, I have never used valgrind, so I am not very familiar with its output. However, (at least some of) these warnings seem to be bogus: it complains about the initialization of data[] array at lines 30 and 31. I don't see anything wrong with it.
Yenya wrote: GCC bug id
OK, reported as http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36082. I hope I am not making fool of myself with some stupid bug in my code.
Yenya wrote:
OK, I have apparently made fool of myself: when I rewrite the bucket_entry_t to be union of one 64-bit part and two 32-bit parts, and use 64-bit part for swapping entries, and 32-bit parts for accessing the key and value, it works. One would think it is equivalent to pointer casting. Maybe I should read the C standard some day.
Reply to this story:
Mon, 28 Apr 2008
OpenSolaris, try 1
Given the recent uproar between the OpenSolaris community and Sun corporate overlords (insightfully commented by Ted Ts'o), I have decided to give OpenSolaris a try.
It was a failure: I did not found what version should I download:
- The Developer preview (aka Indiana) does not boot on my virtual machine: after booting from GRUB, it just prints the three lines of a copyright message, and locks up. Not speaking about their license which prohibits even modifying the software on my own machine.
- The Solaris Express {Developer,Community} Edition requires me to register, and probably to use their proprietary "download manager", whatever it is. Moreover, the license explicitly states that the software is not for production use, and that any evaluation of OpenSolaris one may do is a confidental information of Sun, and as such, it is forbidden to publish it. Talk about an Open Source.
- I do not want to use the Live CD distros (there are several of them available). I want to create a ready-to-use virtual machine, just in case I want to test anything later.
- I do not want to use NexentaOS, as I would like to test also the Solaris userland, not only the kernel.
So, my dear lazyweb, where would you start if you wanted to try OpenSolaris without registering, and with access to all the source in the Open Source(tm) manner, including the rights to modify the software and publish benchmarks? And, more importantly, have you ever kissed a girl?
5 replies for this story:
Vasek Stodulka wrote:
The linked mail is absolutely cool. :-)
Spes wrote:
Interesting, I tried the OpenSolaris last Friday as well and found the LiveCDs sufficient. I couldn't run promising Belenix in Qemu(KVM), but this one should be possible to install on HDD (according to screenshots). The Milax (minimalistic) LiveCD works well in Qemu. If you create any ready to use OpenSolaris image, I would be interested :-).
Kyo wrote:
Vasek Stodulka: AFAIK, Dave Miller *does* have a girlfriend. I find it likely that he has already kissed her, unless she is a nun or something like that. ;-)
Yenya wrote: Kissed a girl?
Well, DaveM's reply is here: http://groups.google.com/group/comp.sys.sun.hardware/msg/1cd06fc491c613d7?dmode=source
Yenya wrote: Re: Spes
Well, if I create an OpenSolaris image, I would most probably not be allowed to give it to you (and maybe even talk to you about how it works :-) Let's ask Sun guys at the EurOpen conference.
Reply to this story:
Fri, 11 Apr 2008
Gentoo Linux - the next try
Since I have got a new workstation which supports hardware virtualization, I have installed several different operating systems to my virtual machines. I have already tried Gentoo Linux, so I gave it another try. Again, it was a bit frustrating - few examples:
- When the Gentoo Handbook writes about the default
grub.conf, why this is not already included in that file itself as a default configuration? For a newbie who wants to learn it is probably good as is, but for me it just costs more unnecessary time. - I needed to test something under GNOME, but "
emerge gnome" did not work - it said something like "required EAPI 1, supported EAPI 0". Fortunately, "emerge gnome-termianl" worked. - Why adding GTK+ and GNOME as the default
USEflags does not imply theXuse flag? In comments to my previously mentioned blog post there is a suggestion to disable theUSE=Xwhen I do not want the X server to be built. Well, it seems GTK+ client side requires X. - What is the equivalent of "
yum update" for Gentoo?
I think Gentoo can be excellent for newbie geeks who want to discover what this Linux thing is about, but even that has its limitations. For example, when I upgrade Fedora on my workstation, often many packages which I would normally refuse to install as a superfluous garbage get installed by the distribution. Which is good, because I have an opportunity to look at them later, in already working preconfigured state (in the past it included D-Bus, HAL, SELinux, and many others).
I also cannot imagine how would
I maintain tens of differently configured machines running Gentoo.
Having fixed release points with security upgrades designed and tested
on them is a big bonus, as are the precompiled packages with automatic
dependencies. yum update and you are done with minimal risk
of breaking something.
11 replies for this story:
Adelton wrote:
emerge world, if I recall correctly.
Yenya wrote: Re: adelton
emerge world can leave you in the state with broken library dependences, can't it? Or at least it has relatively large window when the system is not guaranteed to be in an consistent state.
finn wrote:
revdep-rebuild can fix broken library dependencies
petr_p wrote: EAPI
Have you updated distro before installing gnome? I think you haven't. Otherwise you would get new portage supporting EAPI=1.
Yenya wrote: Re: EAPI
What exactly is meant by "updating the distro"? emerge --sync? And is it described anywhere in the installation handbook?
Yenya wrote: Re: finn, adelton
So in other words - should I run "emerge world; revdep-rebuild" fron the nightly cron (like I run "yum update" on some of my systems)?
Yenya wrote: emerge world
Hmm, emerge world even does not work for me, because of "sys-apps/setarch" package conflict with "sys-apps/util-linux-2.13.1". Are package conflicts to be expected with stock install?
finn wrote:
I run emerge sync && emerge world manually roughly once a week, revdep-rebuild only sometimes -- esp. when I found an application which cannot start because of some missing library :) Sometimes two packages cannot co-exist (because both provide the same functionality, for example). Then you have to unmerge the older one before installing the new one: emerge -C setarch emerge -1 util-linux See these two links: http://forums.gentoo.org/viewtopic-t-578534.html and http://bugs.gentoo.org/show_bug.cgi?id=190476
petr_p wrote: Re: EAPI
Updating distro means two things: synchronising portage (this is a repository) with "emerge --sync" and then updating installed software with "emerge --update --deep --verbose --new world". The problem with changed EAPI you met is a little specific because it's first time when the EAPI (format of ebuilds, something like spec format in RPM world) has changed. In general, you need to understand that fresh installation means you get old software with new portage. So it's highly suggested to do an update. However immediate update is standard thing in gentoo world because user usually changes his preferred profile, compiler options and USE flags on installation and he wants to apply these changes.
Yenya wrote: Re: EAPI
OK, I have tried "merge -C setarch" and then "emerge --update --deep --verbose --new world", which failed with "ERROR: app-text/ghostscript-gpl-8.61-r3 failed.". Which in fact I think reflect the main problem of Gentoo. Without discrete releases, it is much harder to test updates (i.e. new ebuilds) against each configuration.
petr_p wrote: Re: EAPI
You are right. It's impossible to test all configurations. Developers sometimes release ebuild with a bug. Also you could be sometimes first man having such failing configuration. In general, you can ommit last failed ebuild with "emerge --resume --skip-first", or you can mask broken ebuild (/etc/portage/package.mask) to keep current version or unmask by keyword newer version (/etc/portage/package.keyword) (similar to pinning in apt on Debian), or you can search http://bugs.gentoo.org/ for known bugs (or at least reporting the bug), or you can wait few hours/days and update portage to get (usually) fixed ebuild. One note: Adding "--ask" switch to emerge is very reasonable option---you can check all changes you are going to proceed.
Reply to this story:
Fri, 04 Apr 2008
Fedora Bugzilla
Half a year ago, I have ranted about Fedora maintainers ignoring my bug-report #240070 for several months, and then just asking to verify that the problem still exists in the current version, without any attempt to reproduce the bug (do not even think about attempting to solve it). Well, here is the summary of the development since then (you can also read the comments to this bug yourself):
The bug has been originally filled against Fedora Core 6. Since then,
I have reproduced it on Fedora 7, and also Fedora maintainer
Matěj Cepl
has verified it on his workstation. He subsequently changed the
"distribution" attribute to "devel",
so that the bug did not get lost when FC6 reached its end of life.
Well, it did not help.
Without any other maintainer attempting to look at it, today I have got another automatic response:
Based on the date this bug was created, it appears to have been reported against rawhide during the development of a Fedora release that is no longer maintained [...]
If you can reproduce this bug in a maintained Fedora version (7, 8, or rawhide), please change this bug to the respective version and change the status to ASSIGNED.
Just to be cooperative, I have verified that the bug still exists in F7, F8 _and_ the current rawhide. Can anybody tell me how should I report Fedora bugs in so they do not get lost? The bug #201436 is another similar case - read its comments if you need a sad reading for the weekend ...
0 replies for this story:
Reply to this story:
Thu, 03 Apr 2008
EXIF Comment
For an internal project, we need to store comments inside the JPEG images.
I think the EXIF tag UserComment
is suitable for our purpose (we need texts also in the Czech language, and
the alternative tag, ImageDescription, is strictly US-ASCII only).
Nevertheless, the problem still lies in the character set area.
The EXIF standard (PDF warning,
look at 34th page, numbered "page 28" near the bottom) defines the UserComment
data such that the first 8 bytes contain the charset info (strings
"ASCII", "JIS", or "UNICODE" padded to 8 bytes with null bytes), and then
the comment data. The problem is what "UNICODE" means. Is it UTF-8, UTF-16, or what?
I have tried to set the comment using Exiv2 utility, and tried to read it with Image::ExifTool Perl library.
The following code prints the raw UserComment value (i.e.
the string "UNICODE\0my_own_comment_as_utf8_bytes"):
#!/usr/bin/perl -w
use Image::ExifTool
my $info = Image::ExifTool::ImageInfo("exif_comment.jpg",
{ Charset=> "UTF8",PrintConv=>0 });
print $info->{UserComment}, "\n";
However, with PrintConv=>1 it prints garbage, so probably
the UNICODE charset in EXIF means something different than UTF-8.
So, what does your favourite image handling program display as the
EXIF UserComment for the above image? It should read: "Příliš
žluťoučký kůň. こんにちは。".
4 replies for this story:
misch wrote:
Firefox Exif Viewer 1.40 says: User Comment (Hex) = 0x55,0x4e,0x49,0x43,0x4f,0x44,0x45,0x00,0x50,0xc5,0x99,0xc3,0xad,0x6c,0x69,0xc5,0xa1,0x20,0xc5,0xbe,0x6c,0x75,0xc5,0xa5,0x6f,0x75,0xc4,0x8d,0x6b,0xc3,0xbd,0x20,0x6b,0xc5,0xaf,0xc5,0x88,0x2e,0x20,0xe3,0x81,0x93,0xe3,0x82,0x93,0xe3,0x81,0xab,0xe3,0x81,0xa1,0xe3,0x81,0xaf,0xe3,0x80,0x82 User Comment Character Code = Unicode So it recognizes unicode text, but displays it as raw data :-(
Věroš wrote:
We use EXIF heavily at Cestovatel for more than two years. Most of images are commented by Zoner Photo Studio and their UNICODE EXIF is usually is usually saved as UTF-16. BTW: Try XMP ( http://www.adobe.com/products/xmp/ ). It's XML based solution so you don't have to bother with encoding.
Milan Zamazal wrote:
exiv2 displays it correctly, showfoto/digikam displays empty rectangles in place of all characters. Other programs I've tried either don't display user comments at all or they display them as common unknown tags (in hexa).
Yenya wrote: OK, next try
OK, next try - this time in UTF-16. Exiv2 does not display it correctly, Image::ExifTool does. Please reload the above image and retry with it. Thanks!