CentOS 6 trackpad configuration

I was going to write a blog about how the synaptics trackpad driver on RHEL6 is a bit old and buggy and that I had successfuly back-ported the updated RHEL7 version. It seems however that RedHat also got round to back-porting the driver and you can just get it with a yum update. No blog entry required after all.

PHP Search Speed

I recently had a problem with a PHP command line script I use to sanity check some of our systems.

One particular check compares user accounts across a number of systems. There are ~100,000 user accounts. The script loads data in to memory from various LDAP and SQL interfaces. Once in memory, it iterates through the records and compares them. This process takes a few minutes, mostly spent loading data via LDAP.

I had identified a data problem in one system which uses SQL to get the user data. The SQL in question already joins a few tables and rather than introduce another complex join, I decided to just load the data about the new item into a new array using a separate SQL statement. This added just a few seconds to the script run time. However, once I implemented the record compare, the speed of the script slowed to a halt.

This was unexpected because I regularly process data sets this size and larger in PHP without and problems (once I raise the memory_limit).

This is what I discovered while debugging this problem:

It is well known that the PHP in_array & array_search functions are slow. They perform a linear search on the array resulting in O(n) performance.

Many people have contributed code which can implement a binary search on an array. This in theory reduces the performance to O(log n). My testing shows an improvement in search times but they are still quite slow and not representitive of the perforamce I am use to.
So how was it that I have never had this problem before? A quick inspection of my existing code shows that I always retrieve data by using the key. This results in a hashtable lookup of O(1).
So I changed my code to store my new data in the array key and use isset($array[$key]) and bang! my script is running on time again.

Footnote:

The performance of these functions may not be exactly O(n), O(log n) and O(1) but are close enough to enable comparing the performance of these functions against each other.

Command of the day: ^]

You may have seen it hundreds of times but few know what it means.

Escape character is '^]'.
It is actually quite obvious is you know that the caret ^ is a shorthand notation for 'Control' (or <Ctrl> as would be more familiar to windows users).

So what is it? It is not actually a command but a keystroke for the telnet command. It opens a command prompt from where you can enter a variety of commands.

Possibly the most exciting thing you can do is then press ^Z ( <Ctrl>+z ) and suspend your telnet session, returning you to your local shell. You can then resume later by using your shell job control (ie. 'fg').

Some other useful commands are

• quit (or press ^C)
• send brk (which if supported will send a break)
• speed 115200 (which if supported will switch the serial line speed)
• close (disconnect the session)
• open (establish a new session)
• help (list the available commands)

And there are many more documented in the telnet man page.

In this day and age, any server shell access should be via a secure protocol like ssh. telnet remains however as a handy tool for testing TCP servers and accessing specialised equipment like serial concentrators.

It is also worth noting that telnet is actually a protocol and not just a raw TCP stream. There is a lot of negotiation which is done to ensure that your terminal works correctly. Often this only becomes obvious when your terminal does not work correctly.

XBMC 13.2 on CentOS-6

For a while I have been running XBMC 13.0 on CentOS-6. I turns out that there is a known issue in that version which prevents it from playing .m4a files.
The solution is to upgrade to 13.2 which for me means rebuilding a new RPM.
For you, that means downloading the new version from http://www.chrysocome.net/downloads/xbmc-13.2-1.el6.x86_64.rpm
And don't forget to get the deps from http://rpmfusion.org/ and EPEL.

Command of the day: column

Often I come across a new command which I think Wow! I always wanted to know how to do that.

I decided that I will blog about them and that way I will have a record for next time when I forget how to do it.

So my first command of the day I is one I saw on a Red Hat post. It can show the contents of a file tabulated into easy to read columns.

Example:
cat /etc/passwd | column -s : -t


Now you are probably aware of the <tab> character, often entered as \t or in bash as $'\t'. It will advance the cursor to the next tab stop which is traditionally every 8 characters. Well what if you have a tab delimited file but some data is longer than 8 characters and some it less. Well when you look at it, the output will be all over the place.

Using column you can get all your tab columns to line up like this:
cat data | column -s $'\t' -t
Note: for this to work correctly you must have a value in every column (perhaps a bug?)

PocketWIFI and CentOS-6

Now that I have a laptop I can use on the train I can surf the internet using my Pocket WIFI. This is actually a Vodafone Hauwai E585 with a TPG SIM (Uses the Optus network).
It all works great but I don't get any feedback on the connection status and quality.
I have an app on my phone which does this but I could not find one for Linux so I decided to write one for Linux/Gnome.
Here then is my pocketwifi status applet.
 http://www.chrysocome.net/downloads/pocketwifi-1.0-1.el6.noarch.rpm

1. Install the RPM
2. Right click on your gnome panel
3. Select Add to Panel...
4. Type in PocketWIFI
5. Click Add

The applet will automatically detect your route and find the PocketWIFI. It will then query the connection details and download the same status icons which are used on the PocketWIFI screen.

In the future I might add the option to use Gnome icons for  a more consistent look.I might also add support for some other models. If you want your model supported you can drop me an email.

FIPS integrity verification test failed.

After installing the updates for CentOS-6.6, my box started showing this warning when I run ssh:

FIPS integrity verification test failed.

I did not enable FIPS and don't much want it.
There are two ways to turn FIPS off. First option is to remove /etc/system-fips
rm -f /etc/system-fips
Second is to uninstall dracut-fips (which is the package which owns /etc/system-fips).
rpm --erase dracut-fips
The down side of this is that the package might come back. Something obviously caused it to be installed in the first place though I don't know what. There don't seem to be any rpm dependencies.

Eclipse software install error

I recently had the need to use Eclipse. I am not a fan and when I got this error while trying to install Eclipse plugin:

“Comparison method violates its general contract!”

my opinion was cemented.

The workaround is to use an old version of java like this:

eclipse -vm /usr/java/jdk1.6.0_45/bin/java


And the plugins, well they did not do much for me either.

Arduino IR Blaster

My IR blaster is built on a LeoStick which is an arduino compatible USB stick. I must be a sucker for punishment because I thought it would be easy to make one of the many IR libraries work on here and control my Samsung TV.

There were two problems with my plan. First, the defacto IR libraries don't allow me to use the necessary IO pins and second, Samsung IR codes are not supported.

Rather than detail what when wrong, I will present my solutions.

The first problem arises because the LeoStick has a green LED attached to Digital IO pin 9 which is the default PWM Timer used by Ken Shirriff's IR library. The solution is an updated fork of this library Chris Young which implements more Timer chips and enables selecting Digital IO pin 5. Do to that, edit IRLibTimer.h and find the section with the heading 'it's probably Leonardo' (the LeoStick is apparently compatible with the Leonardo). Comment out the default #define and uncomment the IR_USE_TIMER3. (I wonder why we can't select this at run time?). Based on the IRLib manual, I also installed my IR LED with whatever resistors I had on hand (470 ohm I think).

So, thanks to the hard work of these guys, I have got my LED working. Now, to make it talk Samsung protocol.

There were a few snippits of code in various forums but I had to piece that together into the format for IRLib. I have only added sending as I already have a list of codes from SamyGO and LIRC. The only code I wanted to use is TV_POWER 0xE0E040BF because once the TV is on, I can talk to it via the network.

I have submitted my changes as a GitHub pull request. If they are not accepted in the upstream library you can find them here https://github.com/jnewbigin/IRLib

The final piece to the puzzle is making the IR code send when I want it to. To keep this generic, I made my sketch read a hexadecimal code from the serial port. When it receives a code it just send it, thus allowing control over the time and the code. This could probably be integrated with LIRC but I am just using a simple shell command
echo 0xE0E040BF > /dev/ttyACM0

So here is my sketch. There is not much error detection in the serial reading but what is the worst that can go wrong?

/* Example program for from IRLib – an Arduino library for infrared encoding and decoding
 * Version 1.3   January 2014
 * Copyright 2014 by Chris Young http://cyborg5.com
 * Based on original example sketch for IRremote library
 * Version 0.11 September, 2009
 * Copyright 2009 Ken Shirriff
 * http://www.righto.com/
 *
 * Modified by John Newbigin to support Samsung and USB control May 2014
 */
#include <IRLib.h>

IRsend My_Sender;

void setup()
{
  Serial.begin(9600);
}

void p(char *fmt, ... ){
        char buf[128]; // resulting string limited to 128 chars
        va_list args;
        va_start (args, fmt );
        vsnprintf(buf, 128, fmt, args);
        va_end (args);
        Serial.print(buf);
}

// 0xE0E040BF = Power
void sendCode(unsigned long code)
{
   p("Sending SAMSUNG 0x%08lx\n", code);

  for (int i = 0; i < 3; i++)
  {
    My_Sender.send(SAMSUNG, code, 0);
    delay(30);
  }
}

unsigned long ir_code = 0;
void loop() {
  while(Serial.available() <= 0)
  {
    delay(10);
  }
  int incomingByte = Serial.read();

  int nibble = -1;
  if(incomingByte == 10)
  {
    p("Got newline\n");
    sendCode(ir_code);
    ir_code = 0;
  } 
  else if(incomingByte == 13)
  {
    // ignore
  }
  else if(incomingByte >= '0' && incomingByte <= '9')
  {
    nibble = incomingByte - '0';
    p("Got digit %d\n", nibble);
  }
  else if(incomingByte >= 'a' && incomingByte <= 'f')
  {
    nibble = incomingByte - 'a' + 10;
    p("Got hex digit %d\n", nibble);
  }
  else if(incomingByte >= 'A' && incomingByte <= 'F')
  {
    nibble = incomingByte - 'A' + 10;
    p("Got hex digit %d\n", nibble);
  }
  else if(incomingByte = 'x')
  {
    p("Starting number\n");
    ir_code = 0;
  }
  else
  {
    p("Got %d\n", incomingByte);
  }
 
  if(nibble >= 0)
  {
    ir_code<<= 4;
    ir_code+= nibble;
    p("Code is 0x%08lx\n", ir_code);
  }
}

Arduino on CentOS-6

I decided it was high time that I found a way to turn on my new TV automatically. After much research I decided that an arduino based IR Blaster was the way to go. I happen to have a Freetronics LeoStick which was given out a LCA in Ballarat a few years ago.

I recall at the time I could not get the arduino software to work on Linux (CentOS-5) so I resorted to using a Windows VM with USB passthru.

That was not going to cut it this time. I was going to use my full stubbornness to get it working on CentOS-6.

First of all, I had to get a copy of the IDE in RPM format. There was talk on the internet about just such a thing but I could not find an EL6 version. In the end I selected the Fedora 19 version which is suitable. The files needed are:

• arduino-1.0.5-6.fc19.noarch.rpm
• arduino-doc-1.0.5-6.fc19.noarch.rpm
• arduino-core-1.0.5-6.fc19.noarch.rpm

There are also a number of other dependancies because the IDE does not actually do much. Luckily the other bits are all in CentOS-6 or EPEL so using yum you can get it installed without too much trouble.

Running the IDE however is rigged with rubbish about adding your user account to certain groups. This seems antiquated to me and I refuse to do it. There must be a better way.

An ideal solution would be that the USB devices are assigned to the console owner and it would just work like any other peripheral. The way to implement that changes with every version of RHEL/CentOS but after some sleuthing, I found the answer for version 6 and it is no too difficult.

To get this working there are four jobs:

• Make sure the USB device ID is known to the USB TTY driver
• Set the appropriate permissions with udev
• Remove the restrictions on locking the device
• Configure the IDE

The first job is not immediately obvious and may not be applicable on all versions of Linux. On CentOS-6 (6.5) is was necessary. As if by magic or just infuriating design, the USB device ID of your arduino can change depending on what it is doing. This can be exploited for good, say if you wanted to build a USB keyboard or mouse but if you are using the default serial emulation then it seems unnecessary. For me, the default id used by the boot loader (26ba:0002) is automatically recognised as a serial device and /dev/ttyACM0 is created (by the cdc_acm driver) but when my sketch is running, the id (26ba:8002) is not recognised. This means the device goes away which confuses the arduino tools (and me). This is easily fixed by running this command as root:
echo "0x26ba 0x8002" >> /sys/bus/usb/drivers/cdc_acm/new_id
For your board, the IDs may well be different so you much check with dmesg or lsusb.

This needs to be done after every reboot so it should be added to an init script
modprobe cdc_acm
echo "0x26ba 0x8002" >> /sys/bus/usb/drivers/cdc_acm/new_id
TODO: is there is proper place for this? modprobe.conf? /etc/sysconfig/modules/arduino? For now, /etc/rc.d/rc.local will do.

Device permissions on CentOS-6 are managed by udev. Most devices use the traditional user, group and permissions. However, devices which are shared (for example, with console users) are managed using ACLs. (check with ls -l /dev/ | grep +).

In order to set the permissions, the correct devices must be identified. For me, the device name is /dev/ttyACM0 but this does not hang round for long (remember about the changing ids?). It is only present while the arduino bootloader is running. dmesg can give you a hint but the easiest way to check the device name is the plug in the stick and immediately run ls -l /dev/tty*

Once you know the device name you can interigate it with udev
udevadm info -a -n /dev/ttyACM0
Again you must run this immediatly after plugging in the USB stick.
It will output many properties of the device from which you must find something unique. At first I though that 'ATTRS{manufacturer}=="Freetronics"' was a good value to check for but it turns out that when your sketch is running, that string is not present. Better is to use the USB vendor ID 'ATTRS{idVendor}=="26ba"'.  While I was at it, I also added the Arduino vendor code of 20a0. I could then make a new udev rule by creating the file:
/etc/udev/rules.d/52-arduino.rules
and adding the contents:
ATTRS{idVendor}=="26ba", ENV{ACL_MANAGE}="1"ATTRS{idVendor}=="20a0", ENV{ACL_MANAGE}="1"

Some sources also recommend preventing the Modem Manager from probing the device. You can do that by adding this line too

ATTRS{idVendor}=="26ba", ENV{ID_MM_DEVICE_IGNORE}="1"
ATTRS{idVendor}=="20a0", ENV{ID_MM_DEVICE_IGNORE}="1"
Once the file is in place, you can force the rules to be applies with the command
udevadm trigger

Now check the device with ls -l again (immediately after plugging in the stick) and it should look like this (with a + sign)
crw-rw----+ 1 root dialout 166, 0 Apr 24 23:08 /dev/ttyACM0

If so, check the ACL with  getfacl /dev/ttyACM0
getfacl: Removing leading '/' from absolute path names
# file: dev/ttyACM0
# owner: root
# group: dialout
user::rw-
user:jnewbigin:rw-
group::rw-
mask::rw-
other::---

Note that my username jnewbigin is granted access. You should of course see your username there. If so, job done.

The next headache is device locking. The default rxtx library wants to write to a directory it does not have permissions on. Fixing this was painful. All I wanted was a simple way to skip the locking phase in the rxtx library. I had hoped that there would be an environment variable which could tune the locking. According to the documentation, the only way to turn locking off is by recompiling. Luckily the srpm is easily modified. You can download my lock free version of rxtx from http://www.chrysocome.net/download

Almost done. I would have though that configuring the IDE should not be required but as it happens, you must edit ~/.arduino/preferences.txt and change
serial.port=/dev/ttyACM0

So now we have removed the requirements to add your user into the unnecessary groups. The script which runs the IDE (arduino) however still complains that you are not a member. The best solution would be to rebuild the rpm without that check (and add in the udev rules) but for now you can either select Ignore or edit /usr/bin/arduino and change the line
for group in dialout lock; do
to
for group in ; do

Perhaps I will repackage it with these changes.


And stay tuned for the working IR blaster code!

Windows 8.1 in KVM

As Windows XP comes to its end of life, I have to get with the times and test my windows programs in Windows 8.1.

Naturally I am drawn to KVM which is where I run all my Windows instances.

Despite some initial concerns regarding the lack of the +sep flag (covered in my winpe pxe blog), installing Windows 8.1 turned out to be a breeze. I just used the settings for Windows 7 64bit.

After the install I was prompted to log. I have no interest in using a 'Microsoft Account' so I used the obscure procedure of 'Don't have an account' -> 'Sign in without a Microsoft account' which would be better described as 'Create local account'.

The next confronting issue was the start menu or whatever they call it these days. I worked out that I should not to look at it, just to start typing. That works more or less the same as the 'search' on Windows 7 (but possibly faster. It makes by mind boggle that it can take so long to run cmd.exe on Windows 7).

Once I had the basics under control, I set about installing the VirtIO & Spice drivers for extra performance and GUI features (like screen resizing). Alas, spice-guest-tools-0.74.exe does not support Windows 8 and fails during the install complaining about an unsupported version of Windows.

I know the virtio drivers work because I have used them in WinPE so I set about installing them manually. Although they are compatible, installing them is not straight forward.

The driver files are made available under Program Files (x86)\SPICE Guest Tools

The method I normally use is to use guestfish to install the driver file and configure it in the Critical Driver Data Base (CDDB) using a registry patch. Windows 8 no longer has a CDDB so this method does not work.

In the end I reverted to an old method:

• Add a new (temporary) VirtIO disk to the VM
• Boot into Windows
• Install the drivers for VirtIO
• Shut down windows
• Remove the temporary disk
• Change the main disk from IDE to VirtIO

This method works much like it has since Windows 7 allowed changing of the boot path.

The VirtIO network seems to work fine.

Talking to a Samba sever required adjusting some settings. First there are some updates which are apparently required (according to Microsoft) so I make sure all the OS patches were installed first.

Next I adjusted some policy settings under Administrative Tools -> Local Security Policy -> Local Policies -> Security Options:

• Network security: LAN Manager authentication level = Send LM & NTLM - use NTLMv2 session security if negotiated
• Network security: Minimum session security for NTLM SSP based (including secure RPC) clients = No Minimum

Then restart the workstation service or reboot.

Finally, the graphics drivers. For reasons which don't seem to be explained, the QXL video driver is not compatible with Windows 8.1. The performance of the 'Microsoft' driver is not too bad but it means no resizing which is annoying.

I am hoping that the spice people make their drivers and installer work soon.

One final observation is to do with the windows version number. Windows 8.1 is NT 6.3 but unless your program is manifested for windows 8.1, you will get the version number 6.2. This is by design. Some kind of new design which means the the GetVersion... API has been deprecated. What are they doing over at Microsoft???

OpenSCAD for CentOS 6

I am thinking about getting a 3D Printer but before jumping in, I wanted to test out some software to see how hard it is to drive these things.

OpenSCAD

It was suggested that I check out OpenSCAD, the programmers 3D CAD program.

As usual, for CentOS-6, it did not exist. Building it was tricky and required a bit of finesse. You can download my beta build and dependencies from http://www.chrysocome.net/download

Installation Instructions

Note: gmp 5.0 is named gmp5 so it can co-exist with the system version 4.3
Note: mpfr 3.1 is names mpfr3 so it can co-exist with the system version 2.4
yum install gmp5-5.0.5-3.el6.x86_64.rpm mpfr3-3.1.0-2.el6.x86_64.rpm CGAL-4.1-1.el6.x86_64.rpm opencsg-1.3.2-6.el6.x86_64.rpm openscad-2013.06-8.el6.x86_64.rpm --enablerepo=epel

I also need to make sure that the MCAD libraries are correctly packaged. I'll try and make an updated version soon.

FreeCAD

 

FreeCAD is a more traditional click and draw interface which is apparently very good and has many features. It can also be scripted with python. RPMs are available from RPM Fusion Nonfree

Conclusion

I need a lot of practice before I will be making any 3D objects!

NVIDIA GPU Temp

On my old TV setup I used nvclock to get the NVIDIA GPU temperature and graph it using rrdtool (along with other temperatures and fan speeds).

nvclock was always a bit of a hack and does not work any more (at least, it won't do anything useful for me).

I know the nvidia-settings tool can display the temperature (and much more information) so there must be a modern way to access this data. A quick google showed up the nvidia-smi tool.

Quick and dirty,
nvidia-smi -q | grep Temperature -A 1
will show the current temperature.
    Temperature
        Gpu                     : 72 C

Interestingly, the output can also be in XML format. Combined with some xslt magic, it is possible to get the temperature in a more structured way.
echo '<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method='text'/>
<xsl:template match="/">
<xsl:for-each select="//gpu_temp">
<xsl:value-of select="."/>
<xsl:text>&#xa;</xsl:text>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>
' > nvidia-temp.xml
nvidia-smi --dtd > nvsmi_device_v6.dtd
nvidia-smi -q -x | xsltproc nvidia-temp.xml -

Note: Your driver might be using a different dtd version. Adjust as required. (nvidia-smi --dtd | head -1 | cut -d ' ' -f 2 will give the correct name)

If you have multiple cards and want to know which card has which temperature then you can alter the xsl to format it as you require with something like
<xsl:value-of select="ancestor::gpu/@id"/> or <xsl:value-of select="ancestor::gpu/uuid"/>
eg
0000:04:00.0 Quadro K2000 31 C
0000:05:00.0 Quadro K2000 32 C

The new TV is almost here

Now that summer has hit, it is time to head indoors and tinker with the TV. Some clever planning when I last upgraded the disk in my CentOS-4 machine meant that the required partitions and space for CentOS-6 was available without having to alter anything.

That meant that installing CentOS-6 was relatively easy. (I did it the hard way and PXE booted off the work server via VPN). After a minimal install I configured the machine in puppet and managed to leverage most of my Linux SOE config.

After that was finished, I had a workstation but not a TV. Next step XBMC. That installed without any trouble but would not play any audio and the video kept going choppy. Not what I was expecting.

The choppy video was being caused by high load caused by the sosreport program. I don't know what crashed, and I don't much care. There is no point sending me an email about it.

I remember wrestling with this in the past without any satisfactory result. This time, rpm --erase sos did the trick. (I have to make sure puppet does not put it back.).

The audio seemed to be a problem with PulseAudio. I don't much care for this either. I have never needed it before and it does not appear to support S/PDIF / EC958 which I need so it has to go. Once again, no satisfactory instructions so rpm --erase alsa-plugins-pulseaudio seems required.

After a restart (I hate that!), all traces of pulseaudio are gone and XBMC is offering me S/PDIF again. Audio and video now working.

Next hurdle, Remote Controls. I have lots of experience with LIRC in the past because I had to make some modifications to the old version to get it to do everything I wanted. As hoped, there has been some improvement in this area but alas, also some steps backwards.

The first remote is a DVICO Fusion remote. This is a supported model but requires configuration in /etc/sysconfig/lirc and /etc/lirc/lircd.conf
/etc/sysconfig/lirc wants to know the device but it is a dynamically assigned USB HID device. This means UDEV rules ahead. It would be ideal if LIRC came with these but it seems not. Luckily I have my old EL4 rule but the new new udev uses a new format for rules. This was what I came up with in the end:
/etc/udev/rules.d/20-dvico.rules
SUBSYSTEM!="usb", GOTO="end"
KERNEL!="hiddev*", GOTO="end"
ATTRS{idVendor}=="0fe9", ATTRS{idProduct}=="9010", SYMLINK+="dvico"
LABEL="end"

Now I can configure lirc and it should work reliably (unless I have two receivers, which I do but I only use one!).

Copy the lircd.conf config file from the files supplied in lirc-remotes and that is ready to go.

It seems I still need to build a Lircmap.xml file for XBMC before it will work there. Lets hope I can find a sample on the internet. Writing XML by hand is not what I call fun.

The next challenge with LIRC is my IR Blaster which controls the stereo. This uses the kernel driver called lirc_serial. I understand that this is not an enterprise solution but if lirc_dev can be in the kernel, why no lirc_serial? Well, no worries, perhaps I'll find a package which has the driver. I do require that the driver will work after a kernel update so the atrpms package is no good. And is is only version 0.8 and I am using LIRC 0.9 (may not be an issue but I want them to match).

I guess I'll just build one myself.  Enter a world of 'lirc_serial: disagrees about version of symbol' errors. Briefly, some lirc drivers are now in the EL6 kernel. lirc_serial depends on lirc_dev. The lirc_dev in the kernel is back ported so checking kernel version numbers for the correct API does not work reliably. Also, modversions seem to be cached so once the errors are fixed, the resulting module still does not work.

Finally, I abandoned the LIRC build scripts and went straight to the kernel Makefile. After tweaking a few #defines I had a working build. I then packages in into an elrepo style kmod package and now I have a working ilrc_serial driver (which you can download from http://www.chrysocome.net/download). LIRC does not come with the definition for Panasonic remotes but I have the old one on hand which still works (yay). The LIRC init script does not handle starting multiple instances so I will have to hack that up too. At least I can specify the location of the .pid file which I think was a show stopper in the old version.

The next job is to work out if I can use my IR Blaster to turn the TV on too. For a SMART TV, there are not many options for turning it on...