Archive for the ‘Unix / Linux’ Category

Google creating its own Linux Distribution ?

Tuesday, January 31st, 2006

Apparently, quite a lot of blogs ( BlogORabais , Je Hais le Printemps) and news sites (Slashdot, The Register) are relaying the information…

So, what is Google currently preparing ? Is this information a whole FUD started by Google , just to make this company even more popular ?

In any case, this can only benefit the Ubuntu, Debian  and more generally, the Linux communities. So, Long life Google….

FreeNX is damn crazy !

Thursday, January 26th, 2006

Waoo.. Have you ever tried FreeNX, a free version of NoMachine’s server ? This piece of software is incredible ! In two words, it’s a better, secure, VNC server.

A brief look at NoMachine NX explains some technical details about it… Good luck to understand the gory details ;-)

FUSE is the future, a small HOWTO to FUSE on Ubuntu

Wednesday, January 25th, 2006

Yes, FUSE is the future !!

I personnally think that it does NOT make sense to implement every protocol known to earth inside kernel-space, just to be able to mount remote folders. Implementing stuff in kernel-space implies complexity, and bloat.

Added to that, why would you reinvent the wheel ? If libProtocol already exists, it is somewhat stupid to re-implement it, just for the sake of having something in kernel-space..

That’s why FUSE has been invented. This post describes how to mount, for example, a ssh directory using FUSE.

So now, let’s imagine some great stuff with FUSE… The current /var/log totally sucks. Text files are handy for the system administrator, because he can use his usual UNIX guru commands (grep, awk, perl, whatever). However, they are NOT handy at all for system utilities that have to parse all the different formats of logs, in order to output logs, etc (Awstats, for example does that, for Apache logs).

So, we would live in a better world if, for example, all logs were output’ed to a Database, and we had a virtual /var/log, that reflected the database, just so that people can use grep and perl on it… Not only this would allow stats tools to be more efficient, but we would keep the current compatibility…

Anyways…

Linux = 0.53% of internet users

Wednesday, January 25th, 2006

It is a shame to see that only 0.53% of internet users (see zdnet [fr]) use Linux… It was obvious it didn’t represent many users, but so few ?????

Anyways, Good luck to our lovely OS…

10 things that still suck under Linux

Sunday, January 22nd, 2006

I have recently setup a Linux server, so it was the occasion of pointing out the few things that still suck on this beautiful Operating System. Even though the distribution was Ubuntu Linux Server, the most famous Debian GNU/Linux derivative, all of these remarks apply to most other UNIXes such as FreeBSD.

The article is not meant to conclude that XXX OS is better than Unix. It is just a series of remarks that will, hopefully, contribute to making it better in the future.

  1. Lack of consistency : Anyone having administrated a Linux machine has been faced to the general lack of consistence. I am not referring to the often-criticized lack of consistence in the User Interface, but to the heterogeneity of the miscellaneous components instead.In fact, each component (software, library, daemon) does not integrate to its environment, and no effort is done to ensure a smooth integration. For example, there is no generalized notion of a “virtual host” on the system, whereas it is clear for the system administrator that Apache’s www.bar.com’s VirtualHost, Postfix’s smtp.bar.com’s aliases, ProFTPd’s ftp.bar.com’s virtual Host, and all of their respective logs are somewhat related. Why are all tools so much application-centric, instead of being service-oriented ?Linux would be a better place if all those applications shared, to some extend, a set of configuration parameters, log formats and conventions. When looking at awstats logs, only the web-specific part of the bar.com domain appears, whereas the system administrator would like to have a global vision (HTTP, SMTP, FTP, SQL, etc).Of course, the Virtual Host is only one example of a disparate setting. There are lots of others, such as the lack of generalized identifiers and passwords for people’s accounts. As usual, there is nothing technically impossible here, and the solutions are already existing (LDAP, for example, but not necessarily), but once again, to make it possible, people have to agree on some conventions.If you want to provide FreeNX access to your users, you will have to maintain two sets of user/passwords. So would you if you want to give a MySQL database to each of your users. Additionnally, you will have to define a set of conventions to link a user to its database, since MySQL is a planet, and the system is the rest of the universe : there is no link between the two.Finally, even after all these years of editing and modifying these configuration files in /etc, I still wonder why no single file has the same syntax in /etc. There is no pattern, every single file looks like a different world. History has its part of the responsibility, but sometimes, people should be able to correct their mistakes. I am not speaking about drastically changing the whole /etc, but maybe progressively migrating the unused configuration files (how often have you modified /etc/iniittab by hand ?) to some common scheme. (not necessarily XML, but there should be some consistence in the choice. Consistence is not just about eye candy, it is also, and more importantly, about writing once for all, a generic parser, that can be optimised, and on which would all application rely)
  2. Logging is most probably one of the worst parts of a UNIX system. The current syslog system is old and needs to be replaced by something better, cleaner. People could argue that it still works fine, and that syslog-ng solves part of the problem.
    However, it’s an inconsistent system : why is it that we can say mail.* or uucp.* (that only few people use, actually..), but not jabber.*, http.*, samba.*, etc.
    The answer is simple : the system is way too static, many details have been hardcoded into the system a long time ago, and the only extensible part in it are the localX.* that is limited anyways. The proof ? Any decent application (Apache, Samba, ProFTPd, …) implements its own logging mechanism. This has the consequences of bloating instead of componentizing applications.A solution to this would be to implement a flexible, extensible logging framework, that allows any application to fill a set of user-defined attributed, not static ones. The framework should log to a database (SQL, Native XML, OO, whatever), and indexes should be there to help log analyzers to efficiently perform their job. Text files are not machine-friendly, so any log which is to be analyzed by an application should not be written as a mere text file. Of course, system administrators are used to accessing files, so a possible solution is to use something like FUSE in order to implement a virtual /var/log on which UNIX gurus will be able to tail -f, grep, vi, and less. UNIX not-so-gurus will, on the other side, enjoy seeing better graphical applications focusing on the user experience, search, etc, instead of focusing on parsing and optimizing access the big files.
    Additionnally, FUSE would allow tools such as logrotate to still work.
  3. Everything is based on the polling paradigm. Why would man-db run every week, even though I haven’t touched any man page for years ? why would awstats re-analyze my logs every night even though I haven’t had any query the whole day on several virtual hosts ?
    The problem is both about elegance and performance. The polling paradim gives the impression of a dumb system, that reverts to ugly hacks to minimize the performance hit caused by this inefficient system.
    If my server only uses 1% of its CPU during the day to serve Apache Queries, I do not want to wait until the end of the day for my awstats to be updated. Moreover, if at the end of the day, my Apache still eats 100% of the CPU, I do not want awstats to start analyzing logs.
  4. Permissions. Since sensitive data is disseminated everywhere (passwords all over the configuration files, private keys for some daemons, etc), it is nearly impossible to ensure that a consistent set of permissions are applied.Instead, there should be a central repository where all critical information would be stored, and that could be safely protected and watched by the system administrator. Passwords should not be disseminated to /root/.my.cnf, /etc/freenx, /etc/apache/*, etc..

    Additionnally, no distribution currently takes advantage of ACLs by default. It is always possible to mount the filesystem with acls enabled, but no package would, by default, set ACLs instead of standard permissions. However, this could be useful in some cases, such as setting default ACLs in /usr/local/stow (for those who use this system), to ensure that any file created later in this directory will be readable by the staff member, regardless of the umask of the creator.
    A lot of other files could benefit from ACLs, and more specifically, default ACLs. This could be used to enforce stricter permissions, such as forbidding access to anyone to /var/log, and only authorizing specific users to rotate logs, etc. A lot of things can be thought and re-engineered.

  5. Useless bindings all over the place. There are many languages, it is a fact. Since every language must communicate with libraries written in other languages, everyone creates bindings all over the place. However, it would be a little smarter to take advantage of the current .Net platform, implemented by the Mono project. For example, there are bindings for Gtk and all Gnome libraries for the .Net platform, so why are people developping Gtk / Gnome bindings for Python, since there already is a python compiler targetting the .Net platform.
    Developping less stuff, and concentrating on the already developped architectural blocks would help homogeinizing the system as a whole. I am not against the diversity of languages, but since a platform exists to make all these languages communicate, it should be used.
  6. There should be standard communication patterns between processes. It looks like everybody reinvents the wheel to communicate with other processes. Some applications (pop-before-smtp) watch logs of others (courier-imap, etc), some use IPC, some other prefer UNIX sockets.. It looks like more and more people are adopting dbus these days. Maybe  all applications should take the  same path, to let the system administrator be able to monitor communications (logging, permissions, etc).
  7. Limits and MaxSettings are hard to parameter. The maximum number of Apache threads , for example, is pretty hard to configure, since there is no easy way to calculate it. It is even harder to set a reasonable value when there are other services that may use  the CPU as well…
    So, I believe that there should be global parameters, instead of application-specific parameters. It does not make sense to set the number of Threads/Processes in Apache regardless of the other daemons running.
  8. Applications cannot communicate with users :The only communication mean between applications and users are emails. However, email is a specific communication mean, and not everybody wants to use it. Some system administrators may prefer getting paged when a error outcomes (log, whatever) on the system.
    There is simply no dedicated mean for alerting a user, so people revert to quick and dirty hacks (call a specific shell script that will send a message to the cell phone, setup a email<-> phone bridge, etc..).

    So, there should simply be an abstraction to alert and send messages to the system administrator. The middleware would then use the appropriate plugins to communicate with the user, and such a system would prevent every application to implement specific means of notification.

  9. Too many legacy unsecure systems.Whenever an application ships with SSL/encryption, this encryption is an option. Why wouldn’t things be encypted by default ? Having applications that already implement encryption communicate securly by default does not seem something hard to do, so why would we still stay with all those legacy services, unencrypted just because the system administrator is too lazy to configure the SSL certificates, and stuff ?
    SSH is a good example to follow : keys are generated by default, making the system useable right after installation.
    SSL is a bad example : its limits prevent it from being used easily with Virtual Hosts, so it should be improved..
  10. Running an encrypted / is hackish. It is particularly hackish (init ramdisk, etc..) to run a system where / is encrypted. This should be fixed to allow people with laptops to take their computer without fearing their data might be stolen.

Once again, I am not criticizing any work done by all the volonteers on this planet. I am just pointing out all the items that I think should be enhanced, in case people did not realize the few things that are still problematic today.

I would love to see other people list their wishes and remarks too.

Why is Ubuntu so popular ?

Friday, January 20th, 2006

This article [French] explains it :-)

Horde3 and Imp4 HOWTO under Ubuntu/Debian

Thursday, January 19th, 2006

This post is a simple set of guidelines (a mini-HOWTO) on How to setup Horde3 and Imp4 Webmail. In fact, the official documentation lacks a few important things, so here are a few tricks.

First of all, install the horde3 and Imp4 packages (Ubuntu/Debian)

apt-get install horde3 imp4

It is then necessary to setup an Alias for Apache. If you’re using Apache2, add a file /etc/apache2/conf.d/horde3.conf containing

Alias /horde3 /usr/share/horde3

Also, allow Apache to write horde configuration files :

chown -R www-data:www-data /etc/horde

Or, if you prefer to use ACLs

setfacl -m “g:www-data:rwx” /etc/horde

setfacl -d -m “g:www-data:rwx” /etc/horde

setfacl -m “g:www-data:rwx” /etc/horde/*

setfacl -d -m “g:www-data:rwx” /etc/horde/*

setfacl -m “g:www-data:rw-” /etc/horde/*/*

and restart apache

/etc/init.d/apache2 restart

You can then browse http://server/horde3

Important parameters to change are (in Horde setup) :

  • Horde URL (change it to /horde3)
  • Enable Database Access. Do Not use MySQL Improved (4+), my attempts at using it failed. MySQL Standard runs fine however
  • Enable Authentication. I recommend IMAP authentication (something like {localhost:143/imap/notls}. Do not forget the /notls, not specifying failed on my setup). Also make sure to add your username to the list of Administrators, otherwise, you won’t have access to horde/imp4 parameters. Using anything else than IMAP seeemed to fail on my setup
  • Generate the configuration

Horde should now be setup. You now have to configure imp4

  • generate a configuration using the horde administration panel
  • modify the /etc/horde/imp4/servers.php file. Instruction are given inside the file concerning the syntax. If you run Courier-IMAP, and want Imp to automatically authenticate using horde credentials :

$servers['imap'] = array(
‘name’ => ‘IMAP Server’,
’server’ => ‘localhost’,
‘hordeauth’ => true,
‘protocol’ => ‘imap/notls’,
‘port’ => 143,
‘folders’ => ‘INBOX.’,
‘namespace’ => ”,
‘maildomain’ => ‘domain.com’,
’smtphost’ => ‘localhost’,
’smtpport’ => 25,
‘realm’ => ”,
‘preferred’ => ”,
‘dotfiles’ => false,
‘hierarchies’ => array()
);

You should now have a working setup.. Good luck !

2 things to make Grub work for you

Sunday, January 15th, 2006

No matter how better Grub is compared to Lilo, it still has its flaws, making it sometimes hard to use.
Here are two important things to check :

  • That you have a “boot” symlink pointing to “.” in your /boot directory. Grub sometimes refer to /boot/boot/.., so it is safer to have this symlink
  • That you are having the right “groot” option defined in your menu.lst. It often happens that Grub mappings at boot time are not the same as Linux Grub mappings. For example, you might ask your BIOS to boot on a specific Hard Disk, that is not (hd0). The BIOS will make Grub believe that your (hdX) device is actually (hd0), so your menu.lst will not work. Change your menu.lst accordingly, and re-run update-grub to take the changes into consideration.

Running Ubuntu GNU/Linux on a FakeRAID/1 (mirroring) array

Sunday, January 15th, 2006

Edit: These information work for Ubuntu Breezy. Things may have changed with Dapper Drake
Most cheap hardware RAID controllers such as the VIA VT6421 are not purely hardware RAID systems, but should be seen as semi-soft, or FakeRAID controllers.

In order to install an Operating System on a FakeRAID array, it is thus necessary to setup a few things, since the underlying array is not completly transparent to the Operating System.

This short article, based on the Ubuntu Wiki FakeRaid HOWTO explains how to install Ubuntu Linux on such a FakeRAID array.

First of all, be aware that is it not currently possible (well, it is, actually, but one would have to revert to applying hacking changes in the Initial Ramdisk Image, so it is currently better to forget about it) to setup an LVM Volume on top of a FakeRAID array.

Since the Ubuntu Wiki FakeRaid HOWTO already explains how to install Ubuntu on a FakeRAID/0 array, I am just going to highlight the differences for a FakeRAID/1 array here.

The only difference is the creation of the Initial Ramdisk, which should load the dm-mirror module to allow the OS to read / write from the FakeRAID array.

The /etc/mkinitramfs/scripts/local-top/dmraid should be replaced by

#!/bin/sh

PREREQ=”"

prereqs()
{
echo “$PREREQ”
}

case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac

modprobe -q dm-mod
modprobe -q dm-mirror

/sbin/dmraid -ay

In addition to the dm-mod module, the previous script launches the dm-mirror module. This means that the dm-mirror module should be copied. This is done by adding the line

dm-mirror

to the /etc/mkinitramfs/modules file.

The rest of the Ubuntu FakeRaid Howto gives a good explanation of the tasks that should be done in order to configure the FakeRaid array.

RAID controllers reliability

Sunday, January 15th, 2006

Something that I am currently wondering about is whether people use cheap RAID controllers in Mission-critical environments.

According to my tests with a VIA VT6421 RAID 0/1 controller, there is nothing less reliable than RAID 1…. However, drawing such a conclusion seems inadequate, especially since RAID is so popular.

Using 2 Maxtor 250 GiB Hard Drives, configured as a RAID 1 (mirroring) array using VIA VT6421’s BIOS, and the Ubuntu GNU/Linux Operating System with a 2.6.12-10-686 kernel, RAID is a disaster. In fact, on the 2 installations I did with this setup, both failed at some point :

  • With the following partitions : /boot (100 MiB, ext3), / (20 GiB, ext3), swap (1 GiB), /home (the rest, ext3) , after an installation of Ubuntu server and a reboot, the / partition is mountable, but trying to read any file in / leads to a “Cannot access blocks beyond filesystem limits”
  • With the same partitions, except that / was of type reiserfs, installation is fine, mount is fine, except that trying to copy more than 5 GiB of data crashes the system.

Of course, everything runs fine without RAID, so I am wondering what the real problem is :

  • The VIA VT6421 RAID controller in RAID/1 mode ?
  • Any cheap RAID controller in RAID/1 mode ?
  • The Linux driver for these FakeRAID controllers ?

If anyone has an answer to these questions, do not hesitate to post a comment or send me an email.