Yenya's World

Mon, 23 Mar 2015

Backward Compatibility

One of the alleged advantages of certain family of operating systems from Redmond is backward compatibility. They say they support interfaces and applications back to the DOS era, and they sometimes even use this feature as an excuse for some doubtful technical choices they made. Yesterday I have discovered that it is not as good as they often say.

I wanted to install The Neverhood, an old 1996 adventure game. The result was the perfectly working game under WINE and Linux, and partly-working game under Windows 8.1: the gameplay was OK, but the in-game video sequences and their sound were too sluggish, as if it required 5 to 10 times more powerful hardware. According to the discussion forum posts about this topic, it is a common problem in newer versions of Windows. The recommended solution is to run the game under ScummVM, which is a rewrite of many ancient game engines.

Remember this the next time you hear an exaggerated statement about the backward compatibility of Windows.

Section: /computers (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Thu, 19 Mar 2015

Libvirt Dependencies

Welcome to Yenya's rant about software "features". Today we will have look at libvirt in Fedora and its dependencies. But firstly let me show you a funny picture:

systemd-hungry

Anyway. I teach a seminar on Linux administration, where one of the tasks is to compile and use one's own kernel. The task for the following week is to create a virtual machine. One of my students had an interesting problem with the second task - virsh refused to start his KVM-based virtual machine with the "command timeout" message.

Digging into the issue, we discovered that it works with the distribution kernel, but not with his custom kernel. Then we found that virsh tries to do a RPC call over D-Bus, which then times out, because the D-Bus object in question was not present. This object is supposed to be provided by a daemon called systemd-machined, which describes itself with the following headline:

This is a tiny daemon that tracks locally running Virtual Machines and Containers in various ways.

This is in fact an understatement, with the real situation being that this daemon is a core part of the virtualization subsystem, and it is not even possible to start a libvirt-managed guest without it. We have tried to start the daemon from the command line, but it immediately exited without a meaningful message anywhere. The only message in the syslogjournal was that systemd-machined failed to start when the system was booted.

Long story short, my lucky guess was that systemd-machined could have something to do also with containers, and it might have needed a container support in the kernel. After enabling about five namespaces-related kernel config options and booting a recompiled kernel, we were able to start systemd-machined, and only then we managed to start the VM using virsh.

This spaghetti-structured unstraceable mess of interconnected daemons communicating over D-Bus and providing no meaningful error messages, which is masqueraded under a collective name "systemd", makes me sick quite often.

Section: /computers (RSS feed) | Permanent link | 5 writebacks

5 replies for this story:

michich wrote:

If libvirt fails hard when it fails to communicate with systemd-machined and at the same time it reports no useful error message, I'd consider that a bug in libvirt. You did not write what the error about starting systemd-machined said. Was it something like "Failed at step NAMESPACE spawning systemd-machined"? In systemd-machined.service you'll find it declares several security features whose implementation relies on namespaces: PrivateTmp=yes PrivateDevices=yes PrivateNetwork=yes ProtectSystem=full ProtectHome=yes If the kernel does not provide them, the service cannot start. There are many ways to shoot oneself in the foot when configuring a custom kernel. After spending quite a lot of time debugging some issues reported in Red Hat Bugzilla where the reporters failed to disclose they were using custom kernels (and on one occastion, when the fact became apparent, the reporter even refused to do a test under a Fedora kernel), I have developed a dislike for custom kernels. If you got to have one, please at least use the distro's config as the starting point and only trim down what you KNOW you don't need ("make localmodconfig" is nice).

Yenya wrote: Re: michich

My rant is about something other. I do not deny that systemd-machined requires special kernel features. What I find "interesting" is that virsh fails to start a KVM-based virtual machine in a situation when systemd-machined is not running. There is no need for virsh to require systemd-machined. As systemd-machined describes itself, it only tracks the VMs, it should not be a hard requirement for them. Moreover, when I do not use containers, my KVM-based VMs should not fail only because I don't have container-related features in my kernel. Also, the exact "solution" you describe is the reason of why even I don't compile my own kernels anymore - with custom kernel, random pieces of the distribution start failing, even though the features they are missing are not strictly needed for my use cases. This is the reason of recent decline in the number of "voluntary" kernel developers: using custom kernels has become increasingly harder and troublesome.

michich wrote:

I don't know about the goals of libvirt developers, so I cannot say whether the hard requirement on machined is justified or not. As I wrote earlier, at the very least it should report a useful error message. But maybe you're right and it should just continue regardless of the error. Still I don't see this in any way as systemd-machined's fault. I disagree with your point "I'm not knowingly using containers => I should not need container-related features in the kernel". Namespaces have a wider use than just for what people think of as containers. In my view it is perfectly acceptable if programs want to use these features for security hardening or whatever. They don't necessarily have to appear to the user as running in containers. Using localmodconfig it's fairly easy to create a custom kernel that's both quite lean and not lacking essential features. So I disagree with your conclusion.

Yenya wrote:

There are several problems: firstly, I would guess the dependency on systemd-machined has probably been added to libvirt by systemd developers. Even after reading the manpage, I fail to see the benefits systemd-machined brings to the libvirt user. There are probably none. Secondly, I am fully aware of benefits of namespaces besides their use for creating containers (having per-user /tmp is one of the most clever tricks, for example). So the implication you are trying to put into my mouth is in fact different: I would say something like "I don't use containers and I don't depend on namespaces for my applications => the applications should not fail just because the namespaces support is not present in the kernel". It is the same way as (at least for now), the system is still working even when booted with selinux=disabled.

michich wrote:

Looking at libvirt.git, support for systemd-machined was added to libvirt by Daniel P. Berrange, who's not a systemd developer. Though no doubt he communicated with Lennart first. From some of the commit messages, it seems to me the intention was not to have a hard dependency. It may be that the observed hard dependency is simply a bug. This leads me to your corrected implication. Having applications always degrade gracefully when faced with missing kernel features sounds like a nice plan, but may be difficult to achieve in practice. I mean QA most likely only test on the standard kernel. Adding the requirement of testing various kernel configurations would quickly lead to exponential explosion.

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Tue, 30 Dec 2014

PF 2015

I wish a nice year 2015 to all readers of this blog.

PF 2015

Section: /personal (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Sat, 20 Dec 2014

HDMI Sound

Another problem related to getting a new mainboard was sound. The mainboard has an on-board Intel GPU, which I use for the first seat. Unlike my previous graphics card for the Seat0, it is connected by HDMI port to my monitor. So I have decided to give sound over HDMI a try.

The problem was that it did not work: using pavucontrol, I have verified that sound is routed correctly to the HDMI interface, but the interface said that the output is disconnected. And I did not know how to "connect" it, because physically it has obviously been connected.

After some hours of searching I have found the following solution:

$ pactl list cards
...
Card #1
	Name: alsa_card.pci-0000_00_03.0
	Driver: module-alsa-card.c
	Profiles:
		output:hdmi-stereo: Digital Stereo (HDMI) Output (sinks: 1, sources: 0, priority: 5400, available: yes)
		output:hdmi-surround: Digital Surround 5.1 (HDMI) Output (sinks: 1, sources: 0, priority: 300, available: yes)
		output:hdmi-stereo-extra1: Digital Stereo (HDMI 2) Output (sinks: 1, sources: 0, priority: 5200, available: yes)
		output:hdmi-surround-extra1: Digital Surround 5.1 (HDMI 2) Output (sinks: 1, sources: 0, priority: 100, available: yes)
		output:hdmi-stereo-extra2: Digital Stereo (HDMI 3) Output (sinks: 1, sources: 0, priority: 5200, available: yes)
		off: Off (sinks: 0, sources: 0, priority: 0, available: yes)
	Active Profile: output:hdmi-stereo
	Ports:
		hdmi-output-0: HDMI / DisplayPort (priority: 5900, latency
offset: 0 usec, not available)
			Properties:
				device.icon_name = "video-display"
			Part of profile(s): output:hdmi-stereo, output:hdmi-surround
		hdmi-output-1: HDMI / DisplayPort 2 (priority: 5800, latency
offset: 0 usec, not available)
			Properties:
				device.icon_name = "video-display"
			Part of profile(s): output:hdmi-stereo-extra1, output:hdmi-surround-extra1
		hdmi-output-2: HDMI / DisplayPort 3 (priority: 5700, latency
offset: 0 usec, available)
			Properties:
				device.icon_name = "video-display"
				device.product.name = "PLE2607WS"
			Part of profile(s): output:hdmi-stereo-extra2
$ pactl set-card-profile 1 output:hdmi-stereo-extra2

Apparently PulseAudio knows that the hdmi-stereo-extra2 is the only connected output, but remains set up to hdmi-stereo instead. Now that is not very user-friendly, plug&play, etc.

Section: /computers (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Fri, 19 Dec 2014

Multiseat LightDM

After getting a new mainboard, I have upgraded my home computer to Fedora 20, and made my multiseat setup use the udev/logind/loginctl seat tags. About a month ago I have discovered that the seat numbers are not correctly assigned to sessions by xdm(8), so I started to look for solutions. Of course, that piece of crap called gdm was not even been considered for obvious reasons. Apparently the solution does exist, and suprisingly enough, it is really nice: it is called LightDM.

LightDM is the display manager. It has cleanly separated the display manager part (starting up the X servers, listening on XDMCP, etc.), and the user interface part (chooser). The later can be selected from various options - e.g. a KDE/Qt compatible one, and a GTK+ compatible one. The configuration is pretty straigthforward, and it does not try to hide anything from the user, unlike the above mentioned piece of crap.

The multiseat setup in LightDM is pretty straightforward: in /etc/ligthdm/lightdm.conf I have to add the following:

[Seat:0]
xdg-seat=seat0
xserver-command=X -layout Primary -isolateDevice PCI:0:2:0 -seat seat0 vt7

[Seat:1]
xdg-seat=seat1
xserver-command=X -layout Secondary -isolateDevice PCI:1:0:0 -seat seat1
-sharevts vt7

In the udev tags, I had to tag the following device as belonging to Seat1 (using loginctl(8)):

And that's it! The only (minor) nitpick is, that the GTK+ greeter does not remember the last logged-in user per seat, so it preselects the last logged in user on both seats by default. But we usually log in only after the reboot, so it is not a big problem.

Section: /computers/desktops (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Tue, 16 Dec 2014

Systemd: ENOENT

I maintain a small software project (about 4k LOC) which is a part of the university infrastructure. It is versioned in Git and installed on several computers across the university. Today I wanted to deploy it on a Fedora 20 machine, which of course is running systemd.

Firstly about my position on systemd: I think most of the things they are trying to acchieve are pretty cool, but sometimes the implementation and design choices are a bit questionable. Anyway, I have written two unit files for my software, even with the unitname@.service wildcard syntax. The units are OK, systemctl start unitname-instance.service works as expected. The crash landing came when I wanted to enable these units after reboot:

# systemctl enable unitname-instance.service
Failed to issue method call: No such file or directory

What's wrong with it? It can be systemctl start'd anyway, so the unit files should be OK, shouldn't they? After some hair pulling I have discovered that systemd intentionally ingores symlinks in the /usr/lib/systemd/system directory. Moreover, they just set O_NOFOLLOW and print whatever errno they get from the kernel, which is simply misleading. I think my use case - to have my own unit files in my git repository - is valid, and there is no reason for disallowing symlinked unit files.

Related Fedora bug reports: #1014311, #955379.

Section: /computers (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Sat, 13 Dec 2014

CSIRTs Considered Harmful

OK, I am fed up with spam coming from local CSIRTs. Firstly from CSIRT MU, and recently even from the CESNET CSIRT.

"The Guild of Firefighters had been outlawed by the Patrician the previous year after many complaints. The point was that, if you bought a contract from the Guild, your house would be protected against fire. Unfortunately, the general Ankh-Morpork ethos quickly came to the fore and fire fighters would tend to go to prospective clients’ houses in groups, making loud comments like ‘Very inflammable looking place, this’ and ‘Probably go up like a firework with just one carelessly-dropped match, know what I mean?’"
-- Terry Pratchett: Guards! Guards!

This is the problem with Computer security incident response teams (CSIRTs). When they are to actually handle the security incident, they work well. However, security incidents are not very frequent, at least the important ones. So they tend to over-estimate the impact of many so-called security problems, and tend to keep people notified about their own existence by spamming them, or even demanding replies.

For example, CSIRT MU monitors the network traffic and sends notifications about "suspicious" traffic. The report is an e-mail with the URL where the details supposedly can be found. In that page, there is a partial description of the incident, with complete description available through another link. So instead of opening, reading and deleting a single e-mail, one has to read the e-mail, open the included URL, and follow the link in that page. For example, CSIRT MU sends us notifications about some computers in our network "scanning" foreign networks, even though it is clearly visible that the "attack" uses one source and one destination address, and lasts for only a few seconds. Which most probably means that someone ran nmap against his own remote machine. So CSIRT sends us their report using their ticket system, and even demands that we respond in time about the cause (each response gets sent back to us twice - once in Group reply, and the second time through their ticket system). After we explain what is probably going on, their response is not a polite "sorry for bothering you with false-positive, we will refine our detection criteria". The response is "OK, I am closing the ticket", and the next day they send us another false-positive.

A few days ago we've got another "incident report", this time from CESNET CSIRT. They were notifying us about a new HTTPS server in our network with the Poodlebleed vulnerability. OK, we have notified the server owner and got the response "we will eventually look at it, but the same content is available over plain HTTP, and it is only a testing server". Which is a perfectly valid response. But CESNET CSIRT thinks they should spam us every day until this so called "problem" gets fixed.

In my opinion, something like CSIRT with dedicated staff should not exist (except in the largest companies, may be). The security response people should be the regular staff doing their own work, designed to stop their regular work immediately, should the security incident emerge, and work on the security incident instead. But the dedicated staff has too much time in their hands, and tend to look for opportunities to let people know about their existence. The same way as Ankh-Morpork Firefighters Guild did.

Section: /computers (RSS feed) | Permanent link | 5 writebacks

5 replies for this story:

Vašek Stodůlka wrote:

You should create some serious incident for them to keep them busy for a while. :-)

bodik wrote: YES!!

this is very nice feedback and writing about something which I was thinking for some time now ... we'll definitely discuss this topic at the next wg meeting. thank you!

Yenya wrote: Re: bodik

Well, the real feedback (a polite one) has been sent in reply to that spam from CESNET CSIRT. So if you really want to help, look at that mail and think about whether it was OK to send out such a report.

bodik wrote: Re: Re: bodik

I'm not a handler nor regular administrator of that spamming device. I'm just a consultant for the team, so rising a broader discussion (beyond ticketing system) is the only thing I can do for now ...

Yenya wrote: Re: bodik

OK, thanks!

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Wed, 10 Dec 2014

Apache Reload Bug

Yesterday I discovered something that I suspect to be a bug in Apache: we use the same config file for many of our systems, and put the specific parts inside the <IfDefine> blocks.

When the Apache started, it worked as expected. However, after a graceful reload, it seems that some instances of Apache started interpreting some <IfDefine> blocks, even though the particular <IfDefine> string was not present in their command line. I have even verified this by creating a dummy <IfDefine> block with a non-existent directive - the Apache server has started correctly, but died on a syntax error in the config file after a graceful reload.

Long story short, I have upgraded to the latest-greatest version of Apache, and the problem has disappeared. Has anybody seen something similar?

Section: /computers (RSS feed) | Permanent link | 3 writebacks

3 replies for this story:

Atrament wrote:

s/Yesterday I have discovered/I discovered/ Sorry to be a grammar Nazi but I noticed you misuse the present perfect very often. Present perfect (I have discovered) is to be used only when you speak about present, when you are talking about something what happened in the past, use the past simple (I discovered):)

Yenya wrote: Re: Atrament

OK, fixed. Thanks!

Atrament wrote:

s/Yesterday I have discovered/I discovered/ Sorry to be a grammar Nazi but I noticed you misuse the present perfect very often. Present perfect (I have discovered) is to be used only when you speak about present, when you are talking about something what happened in the past, use the past simple (I discovered):)

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Tue, 01 Jul 2014

Static Transfer Switch

Static Transfer Switches (STSs) are amongst the most important parts of power distribution in our datacenter. Some of the datacenters are designed with redundant power paths in mind (as required e.g. by TIER 3 specification). The problem with TIER 3 is, that it requires all the equipment to have two or more power supplies. Some appliances (for example, ethernet switches) are much cheaper with a single PSU. An ethernet switch with two PSUs is usually from the vendor's top line, and is of course priced as such. We have decided to design our datacenter power distribution with single-PSU equipment in mind.

According to our experience, the majority of the power outages in our previous datacenter were either the planned outages, or were caused directly by the failure of the equipment which was supposed to provide higher availability (e.g. the UPSes themselves). So we have planned the datacenter to be able to bridge around the failed part of the equipment, while still providing the uninterrupted power even for the equipment with single power supply.

An STS can be viewed as a box with two incoming power lines and one outgoing line. It monitors the incoming power paths, and can quickly switch to the alternate path, should the currently-used path become faulty, providing uninterrupted service of the outgoing power line even in case of the failure of one of the incoming power lines. The "Static" part in the name means that there are no mechanical parts involved in the switching itself (such as relays), the switching is done by SCRs:

Our STSs are Inform InfoSTS. Their communication protocol and documentation is pretty bad, so I cannot really recommend them. Their proprietary Windows-only management software is even worse. For example, an attempt to set the time fails when the time is before 10:00, because the management software sends the time as H:MM, while the STS itself expects HH:MM even for hours less than 10. I have nevertheless managed to decode the protocol and write my own web-based management application for it (screenshot above).

Probably the most interesting part is that it is the first time I used SVG inside the web page, and Javascript for modifying it when the new data is read. So the schematics can be edited in Inkscape, and provided that the object IDs are unchanged, the application layer can still work with it. I plan to connect it with MRTG or Zabbix, and make all the numbers clickable, leading to the graph of the history of that particular variable.

Section: /computers (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Mon, 30 Jun 2014

Časopis "abc"

Jedno z drsnějších setkání s realitou: koupil jsem dětem časopis "abc", který jsem jako kluk měl dost rád. Dnešní "abc" asi už nemá dovětek "mladých techniků a přírodovědců", zato je tak z poloviny vyplněno reklamami. Vystřihovánky jsou tam značně zjednodušené (kam se hrabou stavební stroje z asi 150 dílů, které byly za mých časů).

Tak nějak mi to připomíná bulvární tisk. Představte si třeba elektrokolo, které filtruje vzduch (aniž by výrazně brzdilo jezdce aerodynamickým odporem), provádí fotosyntézu (i když na většinu plochy kola nesvítí slunko, protože je ve stínu jezdce), a vůbec je celé zelené. A vozit všechny tyhle vymyšlenosti sem a tam na elektrokole se vyplatí daleko víc, než provádět fotosyntézu třeba stacionárními rostlinami. Zaplnit město takovými koly, byl by tam vzduch jako na horách (klikněte na obrázky pro zobrazení ve vyšším rozlišení, podstatný je hlavně text):

Některé texty podle všeho píše polodementní absolvent žurnalistiky odněkud z Bronxu. Ale ty basy, woe, ty tam nejsou, je to příliš malý:

Správný pirát má dálkáč, woe:

No ale nejsou piráti jako piráti. Když nějací podivíni navrhli zbraň vyrobitelnou na 3D tiskárně, americká vláda to samozřejmě zakázala. Ale považte, projekt umístili na _pirátských serverech_(sic!), odkud si jej stáhlo přes sto tisíc lidí!

Zeptejme se, Kdo za tím stojí? Pokud by získání zbraně stálo jen pár stovek, nezbude každému, než aby se taky ozbrojil (podtext: a to je samozřejmě špatné, ááno!). Prostředí Internetu totiž nahrává hlavně pirátům:

A tak podobně. Na kladnou misku vah je třeba přičíst obrázkový kurz Blenderu a koupající se slečnu (byť decentně zakrytou osuškou) v jednom komixu.

Zdá se, že časopis "abc" se od dob mého mládí významně posunul. Nejspíš ale ne k lepšímu.

Section: /czech (RSS feed) | Permanent link | 3 writebacks

3 replies for this story:

Milan Zamazal wrote:

No, ty poslední dvě věty jsou něco, u čeho se také občas přistihuji: Příznaky nadcházejícího stáří projevující se v trousení poznámek o tom, že tráva dříve byla zelenější. Racionálně vzato, že se časopis během dynamického čtvrstoletí významně posune je zcela očekávatelné. A jestli k lepšímu nebo horšímu se nám trochu těžko posuzuje, můžeme kroutit hlavou nad kravinami, na druhou stranu co by asi naše děti řekly seriálu Strážci a na komplikované vystřihovánky dnešní běžné dítě sotva bude mít čas, trpělivost a motivaci (zní to hrozně, ale tímto způsobem fungujeme všichni) a navíc jejich tehdejší legendární tvůrce byl asi i ve své době výjimečný. Já zase kroutím hlavou nad tím, na co moje děti s oblibou čumí v televizi, přitom ale zjišťuji, že některé ty kraviny jsou třeba 20 let staré. Takže ono je to asi svým způsobem pořád stejné, některé věci jsou kvalitní a jiné ne, přitom se občas mění anebo nemění obal, obsah a název.

Yenya wrote: Re: Milan Zamazal

Tak já si ani náhodou nemyslím, že dřív byla tráva obecně zelenější. Spíš je to ve většině věcí naopak. No a ty vystřihovánky - nejkomplikovanější co si pamatuju byl Lotus F1 s řititelnými předními koly, a to už nebylo od arch. Vyškovského. Takže určitě není pravda, že by to záleželo na jednom člověku. No a nakonec: cílem tohoto blogu není ukazovat, že některé věci jsou kvalitní a některé ne.

J.C. wrote:

Podle me vetsina (nejen barevnych) casopisu konverguje k bulvaru.

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Meta: New Section "Czech"

One of the reasons of why I started to write this blog was to improve my English (not sure whether this mission is successful, though :-). Apart from that, I wanted to point out things that I found interesting. Occasionally, there are things that make sense only for people speaking Czech or having the knowledge of the Czech Republic. It does not make much sense to write about them in English, and then point to the resources and documents in Czech.

So I have created a new section of this blog, named simply "Czech". The articles in this section will be in the Czech language. My English-only readers (should there be any :-), please accept my apology, and skip this section entirely.

Section: /czech (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Mon, 09 Jun 2014

Politically Correct Media Players

Hello, welcome to today's issue of your favourite "Bashing the Questionable Fedora Desktop Decisions" series. Today, we will have a look at the politically correct media players.

In a civilized world, there is no place for such insane things like software patents. Unfortunately, there are less free parts of the world, which includes the United States of America. So the companies originating in the U.S. are forced to do absurd decisions like shipping audio players which really cannot play most of the audio files out there (which are, unfortunately, stored in the inferior MP3 format), or video players which cannot play almost any video (which can be encoded in wide variety formats, almost all encumbered by software patents).

For Fedora, the clean solution would be to have a package repository outside the U.S. jurisdiction, and offer it as a part of Fedora by default. Such a repository already exists at rpmfusion.org, and it provides everything needed to play audio and video in free parts of the world. But it is not as promoted as it should be in free parts of the world. However, Fedora does something different: they ship empty shells of audio and video players, such as Pragha or Totem, which in fact cannot play most of the audio and video files. The problem is, that these applications shamelessly register themselves as the handlers of audio/mp3, video/h264, and similar MIME types. Only after the media file is handed to them, they start to complain that they don't have an appropriate plug-in installed.

Hey, Fedora desktop maintainers, stop pretending that the US-based Fedora desktops can handle MP3 and H.264 files, and admit that your inferior but not U.S. software-patent encumbered players cannot handle these files by default. It would be fair to your users. Fedora users: is there anybody who really uses Totem instead of VLC or Mplayer?

Section: /computers/desktops (RSS feed) | Permanent link | 2 writebacks

2 replies for this story:

petr_p wrote: Desktop files

Both named players are based on gstreamer. Thus it's not possible to tell at build-time what media types are supported. This is a story about static desktop files. The only improvement I can imagine is declaring capability of audio/* and video/* instead of specific subtypes.

Vasek Stodůlka wrote:

Yeah, its's weird, but at least it can be fixed by installing VLC. But have you seen iPad? You can't even copy unsupported files there without some hack. :-)

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Tue, 27 May 2014

MPEG Transport Stream

Today I have investigated why some files with the .MTS extension do not have their MIME type detected. The file starts with the following bytes:

$ od -tx1 file.mts | head -n 1
0000000 00 00 00 00 47 40 00 10 00 00 b0 11 00 00 c1 00

According to the current /usr/share/magic from Fedora 20, it is quite similar to the following entry:

0       belong&0xFF5FFF10       0x47400010
>188    byte                    0x47            MPEG transport stream data

Also, the shared-mime-info package contains something similar:

<match type="big32" value="0x47400010" mask="0xff4000df" offset="0"/>

Note that both files expect the 0x47 byte to be at the beginning of the file, not after four NULL bytes as in my example. Yet mplayer(1) can play these files, and ffprobe(1) can detect it as "mpegts" with an audio and video stream. Looking into the ffmpeg source, I have discovered it does horrible things in order to detect a file format. For example, for mpegts, it scans the file for a 0x47 byte at offset divisible by four, and then evaluates some other conditions. The probe function returns score, and a file format with greatest score is returned from the probe function. Ugly as hell, but probably needed for handling real-world data files.

So, what should I do next? Should I submit a patch to file(1) and shared-mime-info to accept also the magic number at offset 4? Are we getting to the point where the already-complicated language of the /usr/share/magic file is not powerful enough?

Section: /computers (RSS feed) | Permanent link | 4 writebacks

4 replies for this story:

petr_p wrote:

file suffers from lack of tests and the parser rules are horrible. Fedora maintainer is pondering complete rewriting becuse of some fundamental insuffuciences in the languge (e.g. early text and binary format fork). There are always funny bug reportes to file (like today's one [https://bugzilla.redhat.com/show_bug.cgi?id=1101404]). I think you can submit your patch and look forward to various regressions :)

Ondřej Caletka wrote:

As MPEG TS is consisted of almost independent 188-bytes long packet, there is no room for good header. AFAIK the only way to detect TS is to seek for 0x47 packet start mark and then check that this mark repeats every 188 bytes. However, for TS stored in file, there is no reason why 0x47 shouldn't be the first byte. Maybe your file is not a raw MPEG TS stream but some MPEG TS stream augumented with packet timestamps.

jm wrote:

This is likely a Blu-Ray video, where the 188-byte pathets are prepended with 4 byte timecodes. http://en.wikipedia.org/wiki/MPEG_transport_stream#Use_in_digital_video_cameras

jm wrote:

packets, of course :)

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Tue, 13 May 2014

A Collapsed Dam

Last weekend we went to Jizerské hory, and we have visited an interesting technological sight there: the collapsed dam on the Desná river. Here are my photos together with a short description, as it apparently does not have its own English Wikipedia page, just a dead link from the Desná disambiguation page. More can be read in the Czech Wikipedia (Přehrada Desná).

The dam collapsed less than year after being built, in 1916. The forest workers spotted a tiny leak, and informed the dam keeper, who in turn informed his supervisor, and was given a permission to open the slide valves. In spite of this, the dam collapsed in about 70 minutes. The resulting flood wave killed 62 people. In the picture, there is the dam body from the side, together with the valve tower, and the river Desná flowing to the left through the gap in the dam.

The view down the river shows the huge gap where the part of the dam collapsed. Behind the new wooden bridge the entry to the water outlet tunnel can be seen.

The outlet chamber and the valve tower. There were two slide valves in the dam: one in the tower, and the second one in the outlet chamber. When the dam started to leak, the first valve was fully open, while the second one was half-open.

The second slide valve. After being informed about the leak, the dam keeper tried to fully open also the second valve, but he was forced to flee from the valve chamber in order to save his life. The valve has remained open at about 75 %.

In times of torrential rains, it was possible to divert part of the incoming water to the neighbouring dam Souš by a tunnel more than one kilometer long. Nowadays, the tunnel is closed, as it is a nature reserve of bat wintering.

Here is a modern analysis of the catastrophe and the reasons of the failure of the dam (in Czech, sorry).

An interesting fact about the political and economical system of that time: this dam and several others in Jizerské hory were built by a non-profit organization (Vodní družstvo, in English something like Water co-operative), formed by local works owners and other local persons and organizations, which were endangered by annual floods from the mountain rivers. The local people paid for the whole system of dams. I wonder why we pay enormous taxes and then build anti-flood infrastructure using the money from Brussels. I would guess the percentage of money lost in this process is huge compared to what our forefathers did.

Section: /world (RSS feed) | Permanent link | 0 writebacks

0 replies for this story:

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

Wed, 07 May 2014

GMail Spam Filter

Apparently, GMail spam filter got too zealous. I have my own domain, and I run my own SMTP server on it. Now it seems Google has decided to reject all mail from my server:

<my.test.gmail.account@gmail.com>: host
    gmail-smtp-in.l.google.com[2a00:1450:4013:c01::1b] said: 550-5.7.1
    [2a01:...my.ipv6.address...] Our system has detected that this
    550-5.7.1 message is likely unsolicited mail. To reduce the amount of spam
    sent 550-5.7.1 to Gmail, this message has been blocked. Please visit
    550-5.7.1 http://support.google.com/mail/bin/answer.py?hl=en&answer=188131
    for 550 5.7.1 more information. o49si12858332eef.38 - gsmtp (in reply to
    end of DATA command)

In the mentioned page, they recommend putting "SPAM" in the subject of forwarded mail :-/ in order to trick GMail to accept it. But then, it is not forwarded mail at all, it is mail originated on the same host from which the SMTP client is trying to send it to GMail.

So, are we getting to the world where only Google and few other big players are allowed to run their own SMTP servers? And after that, they wil "suddenly" decide to stop talking to each other, as we have seen in the XMPP case with Google Talk. The morale of the story is: don't rely on services you cannot control for your private data and communication. They will drop your incoming mail as supposed spam and you will not be able to do anything about it.

UPDATE 2014/05/21: Workaround Available
Apparently, this is indeed IPv6-related, and the workaround is either to use IPv4 for Gmail, or better, make Postfix fall back to IPv4 after trying IPv6 first. This way, Google gets a penalty of two connections, and hopefully will have motivation to fix their problem.

The solution is described here, and more can be read in the postfix-users list archive (another source). The solution is:

Add the following to /etc/postfix/main.cf:

smtp_reply_filter = pcre:/etc/postfix/smtp_reply_filter

Create a file named /etc/postfix/smtp_reply_filter with the following line:

/^5(\d\d )5(.*information. \S+ - gsmtp.*)/ 4${1}4$2

and reload the Postfix configuration using postfix reload command.

Section: /computers (RSS feed) | Permanent link | 4 writebacks

4 replies for this story:

Milan Zamazal wrote:

Do you have both SPF and DKIM set up for your mail domain? If not then you should and it'll probably solve the Gmail problem. (Of course, I wouldn't recommend using Gmail to anyone, but we sometimes need to communicate with their users.)

Yenya wrote: Re: Milan Zamazal

Nope, I don't have SPF nor DKIM set up. Anyway, the outgoing mail in question was sent with envelope sender (MAIL FROM) from another domain than mine (that domain also does not have SPF/DKIM set up). However, I have also tried to send a test mail with MAIL FROM being in my own domain, and Google has rejected it as well. The strange thing is that after several test mails, it has started to pass through the Gmail filter, including the very same messages which have been previously rejected.

misch wrote: IPv6?

I had similar experience ... but ONLY on IPv6 connections. As soon as I switched to IPv4-only for sending mail via google mailservers, every mail was accepted. And when I switched back to IPv6, google started rejecting my mail again. DKIM was enabled (and working) and DNS PTR record was also set-up correctly. And, most important, every other mailserver had no problem accepting mails from me via IPv6 ... except for google. Strange.

Yenya wrote: Re: misch

Interesting. Yes, it is over IPv6.

Reply to this story:

 
Name:
URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Comments:
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)

About:

Yenya's World: Linux and beyond - Yenya's blog.

Links:

RSS feed

Jan "Yenya" Kasprzak

The main page of this blog

Categories:

Archive:

Blog roll:

alphabetically :-)