What happened to java -client?

On our shared server, users are limited to 25 processes. A login script further reduces the soft limit by another 5 processes. This safety margin allows the user to log in an kill something which is using up all their processes. If they want they can raise their soft limit back up to the hard limit.

This works well for most things but java insists on using lots of processes. Confusingly, it prints this message:
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# An error report file with more information is saved as:
# ./hs_err_pid27297.log
The log file is not of much use. It repeats the claim about 'insufficient memory' which is not true and none of the listed solutions will help.

The problem is that the default garbage collector (GC) scales up with the number of CPU cores. Our old server had 8 but the new server has 24. This is easily pushing the users over the 20 thread soft limit.

I don't if the log file lists the number of java threads. If it does I can't work out how to read it. If it could be found it could be compared with the number of available processes:
expr $(ulimit -u) - $(ps x | wc --lines)

In the past I used to solve this problem by adding this to the login script:
alias java="java -client"

but that does not work any more. I don't know when it was removed but it leaves me with the headache of finding a new workaround.

After some research I found this page https://wiki.csiro.au/pages/viewpage.action?pageId=545034311 which suggested using
-XX:ParallelGCThreads=1 -XX:+UseParallelGC
or even just
-XX:+UseSerialGC

This seems to solve the immediate problem but still is not all that useful. To specify the flag to run java you need this syntax:

java -XX:+UseSerialGC

but for javac you have to specify like this:
javac -J-XX:+UseSerialGC


and I don't expect anyone should have to remember that. One simple solution is to alias both those commands but there are many other tools which are part of java which would also need aliases. These other tools may (and do) use their own unique command line parameters which I would rather not have to learn.


Luckily there is also an environment variable you can set like this:
export _JAVA_OPTIONS="-XX:ParallelGCThreads=1 -XX:+UseParallelGC"

however, that makes all the java programs print out this annoying message:
Picked up _JAVA_OPTIONS: -XX:ParallelGCThreads=1 -XX:+UseParallelGC
And the word on the internet is that there is no way to suppress that message.

My solution to use an alias for the most common tools java and javac while keeping the environment variable for all other programs. At first I tried to use this in my alias:
_JAVA_OPTIONS= /usr/bin/java -XX:+UseSerialGC
but an empty variable still shows the message (and best I can tell, bash won't unset a variable like that). I needed to unset it so I had to combine the env command like this:
env -u _JAVA_OPTIONS -XX:+UseSerialGC

My final profile script looks like this:
export _JAVA_OPTIONS="-XX:ParallelGCThreads=1 -XX:+UseParallelGC"
#export _JAVA_OPTIONS="-XX:+UseSerialGC"
alias java="env -u _JAVA_OPTIONS $(which --skip-alias java) $_JAVA_OPTIONS"
alias javac="env -u _JAVA_OPTIONS $(which --skip-alias javac)$(for i in $_JAVA_OPTIONS ; do echo -n " -J$i" ; done)"
You can select which GC you want. I have demonstrated with the more complex example which has two options to be prefixed for javac.

The java users can now happily compile away without java needing the entire machine resources for every invocation.

Can a KVM guest find out who it's host is?

Our puppet configuration performs a number of checks on our asset database to make sure things are recorded correctly.

One of the properties we record for virtual machines is their location (which host they are running on).

By default, KVM does not expose this information even though there are many ways it could technically be done.

Using bits and pieces from around the internet I have come up with a process where I can pass the serial number of the host into the guest.

qemu permits you to specify many DMI values on the command line like this:
qemu-kvm ... -smbios type=1,serial="MY-SERIAL"
The virtual machine will see this value and puppet will automatically create a fact with this value.

Unfortunately, libvirt does not use this mechanism and the serial number is blank in the virtual machine. The libvirt specification permits setting a value in the .xml config file but it is still not used.

I would like to insert a value which is the same as the host serial but with a prefix to indicate that this is indeed a virtual machine and then a suffix to make sure the serial is unique (such as the vm name).

Initially I used dmidecode to get the host serial number
dmidecode -s system-serial-number
but to run that you must be root and using sudo from the qemu user turned out to be a PITA. In the end I settled for
/usr/bin/hal-get-property --udi /org/freedesktop/Hal/devices/computer --key system.hardware.serial
which uses dbus but means I can run it without being root.

The next step was a shim around qemu-kvm which could add the command line parameters. On RHEL/CentOS 6, the binary lives in /usr/libexec so I put my wrapper in /usr/local/libexec (I think that is the first time I have ever used that directory). When a vm is being started, the first parameter is -name and then the machine name is specified (at least in the current EL6 version. This was not the case in a previous release so it could change). I check to see if that has been specified because qemu-kvm is also invoked by libvirt to check it's configuration/capabilities which does not require the dmi serial (although it does not hurt).

/usr/local/libexec/qemu-kvm
#!/bin/bash
# This is a wrapper around qemu which will supply
# DMI information
if [ "$1" = "-name" ] ; then
    SERIAL=$(/usr/bin/hal-get-property --udi /org/freedesktop/Hal/devices/computer --key system.hardware.serial)
    exec /usr/libexec/qemu-kvm "$@" -smbios type=1,serial="KVM-$SERIAL-$2"
else
    exec /usr/libexec/qemu-kvm "$@"
fi

The final step is to tell libvirt to use my new shim rather than the qemu-kvm binary directly. This also does not seem optimal but can be done by editing every guest and setting the <emulator> path to /usr/local/libexec/qemu-kvm
(either using virsh edit or your favourite XML editor).

ProLiant Virtual Serial Port

While working through the configuration of iLO3 for our HP DL380 G7 servers, I found a few pages talking about the Virtual Serial Port (VSP).

This sounded interesting so I though I would try it out. I was using this page as a guide http://www.fatmin.com/2011/06/redirect-linux-console-to-hp-ilo-via-ssh.html but it was a bit out of date.

The first thing to mention was that this does indeed work with iLO3. When configuring your iLO3, remember that in order to ssh to iLO3 you must have a PTY (-t -t), use protocol version 2 (-2) and use DSA keys (ssh-keygen -t dsa). My old configuration for older versions of iLO used exactly the opposite for all of these settings which took some time to figure out (and HP only recently identified/fixed the bugs with ssh keys).

Once you can ssh to your iLO interface, you can issue the command
VSP
and it will start a terminal emulator (actually, it just passes everything through to your terminal program so you can use xterm if you want but safer to assume a vt102). To exit, press ESCape and then ( and you will return to the iLO prompt.

For the OS configuration, I found that I had to make a few alterations. First, RHEL6/CentOS6 uses upstarts and not inittab. If you are not booting with a serial console you must create your own init script:
/etc/init/ttyS1.conf
# ttyS1 - agetty
#
# This service maintains a agetty on ttyS1 for iLO3 VSP.

stop on runlevel [S016]
start on runlevel [235]

respawn
exec agetty -8 -L -w /dev/ttyS1 115200 vt102

You can then enable the console with the command
start ttyS1
Next time you reboot it should start automatically.

In order to log in as root you must add the terminal it to the /etc/securetty file:
echo '/dev/ttyS1' >>  /etc/securetty

So that was all good. Now I want to roll this out to all my servers. To do this I needed some puppet magic. Not surprising, puppet let me down on most of the steps for this.
# Install the init script. This one is easy, just dump in the file 

file { "/etc/init/ttyS1.conf":
...
}

# Add the entry to securetty
augeas { "securetty_ttyS1":
# Ha ha. securetty has a special lens which is different to most other config files
context => "/files/etc/securetty",
changes => [ "ins 0 before /files/etc/securetty/1",
             "set /files/etc/securetty/0 ttyS1",
],
onlyif  => "match *[.='ttyS1'] size == 0",
}

# Enable & start the service. My version of puppet does not support upstarts on CentOS so I can't do this:
service { "ttyS1":
provider => upstart,
ensure => running,
enable => true,
}
# Instead I have created my own type
define upstart($ensure = "running", enable = "true") {
        service { "upstart-$name":
                provider => 'base',
                ensure => $ensure,
                enable => $enable,
                hasstatus => true,
                start  => "/sbin/initctl start $name",
                stop   => "/sbin/initctl stop $name",
                status => "/sbin/initctl status $name | /bin/grep -q '/running'",
        }
}
upstart{"ttyS1": }

And finally, I need to make sure the server BIOS is configured with the VSP.

package { "hp-health": ensure => present } ->
service { "hp-health":
        ensure => running,
        hasstatus => true,
        hasrestart => true,
        enable => true,
} ->
exec { "vsp":
        logoutput => "true",
        path => ["/bin", "/sbin", "/usr/bin", "/usr/sbin"],
        command => "/sbin/hpasmcli -s 'SET SERIAL VIRTUAL COM2'",
        unless => "/sbin/hpasmcli -s 'SHOW SERIAL VIRTUAL' | grep 'The virtual serial port is currently COM2'",
        require => Class['hp_drivers::service'],
} ->

# while we are messing with the serial ports, make COM1 work as the physical device
exec { "com1":
        logoutput => "true",
        path => ["/bin", "/sbin", "/usr/bin", "/usr/sbin"],
        command => "/sbin/hpasmcli -s 'SET SERIAL EMBEDDED PORTA COM1'",
        unless => "/sbin/hpasmcli -s 'SHOW SERIAL EMBEDDED' | grep 'Embedded serial port A: COM1'",
        require => Class['hp_drivers::service'],
}

Visual Studio 2010

I am not a fan of Visual Studio. Unfortunately I must use it for some projects. Recently I was forced to upgrade to Windows 7 and Visual Studio 2010. Not wanting to duplicate all my files, I decided to leave them on a network drive and just access them via the network.

Seems like a good idea, after all, why have a network if you store everything locally? Well, it seems that Visual Studio does not like that.

For some settings it will decide to find a suitable local directory for you. For some other settings it leaves you high and dry.

For example, when I try and build my program I get the error:

Error    1    error C1033: cannot open program database '\\server\share\working\project\debug\vc100.pdb'    \\server\share\working\project\stdafx.cpp    1    1    project

The internet was of little use which is why I thought I would put it in my blog.

This goes a long way to explain my criticism of Visual Studio. After installing several gig of software, is that the best error message it can come up with? Well I will try the help, Oh, that is online only. After jumping through some hoops, the help tells me that:

This error can be caused by disk error.

Well, no disks errors here. Perhaps it means that it does not like saving the .pdb on a network share. What is a .pdb anyway???

In the end, my solution (can I call it that or as Visual Studio hijacked that word?) was to save intermediate files locally:

• Open Project -> Properties...
• Select Configuration Properties\General
• Select Intermediate Directory
• Select <Edit...>
• Expand Macros>>
• Edit the value (by double clicking on the macros or just typing in):

$(TEMP)\$(ProjectName)\Debug\

• Select OK
• Select OK

And that seems to sort it out.

Graphing the inputs of PCF8591

As discussed in previous posts, the PCF8591 provides up to four analog inputs. Displaying these inputs as numbers makes it hard to visualise what is really going on so here I will present some code which can graph the values in real time.

I am still using my demo board from I2C Analog to Digital Converter. (Thanks to Martin X for finding the YL-40 the schematic on the The BrainFyre Blog)

DX pcf8591-8-bit-a-d-d-a-converter-module-150190 YL-40 schematic

It is not clear from the schematic (or looking at the board) but the four inputs are:

• AIN0 - Jumper P5 - Light Dependent Resistor (LDR)
• AIN1 - Jumper P4 - Thermistor
• AIN2 - Not connected
• AIN3 - Jumper P6 - Potentiometer

It also seems that the board has I2C pull-up resistors which are not required because the Raspberry Pi already has them. This does not appear to cause any problems.

So my plan is to graph the four inputs so I can visualise them responding to changes.

Once again, I don't want to set out to teach C programming but I will be introducing a library called curses (actually ncurses) which makes it easy to display a text interface in a terminal window (virtual terminal).

It will also be able to adjust the analog output value using the + and - keys.

I will only add notes where I am doing something new from the example shown in Programming I2C.

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/i2c-dev.h>
We need a new header file
#include <ncurses.h>

int main( int argc, char **argv )
{
        int i;
        int r;
        int fd;
        unsigned char command[2];
        unsigned char value[4];
        useconds_t delay = 2000;

        char *dev = "/dev/i2c-1";
        int addr = 0x48;

        int j;
        int key;
Here we do some ncurses setup. This will allow us to check for a keyboard key press without having to wait for one to be pressed.
        initscr();
        noecho();
        cbreak();
        nodelay(stdscr, true);
        curs_set(0);

This will print out a message on the screen (at the current cursor location which is the top left)
        printw("PCF8591");


This will print out some labels for our graph bars. The text is printed at the specified location (row, column)
        mvaddstr(10, 0, "Brightness");
        mvaddstr(12, 0, "Temperature");
        mvaddstr(14, 0, "?");
        mvaddstr(16, 0, "Resistor");

We must now call refresh which will cause ncurses to update the screen with our changes
        refresh();
        fd = open(dev, O_RDWR );
        if(fd < 0)
        {
                perror("Opening i2c device node\n");
                return 1;
        }

        r = ioctl(fd, I2C_SLAVE, addr);
        if(r < 0)
        {
                perror("Selecting i2c device\n");
        }

        command[1] = 0;
        while(1)
        {
                for(i = 0; i < 4; i++)
                {
                        command[0] = 0x40 | ((i + 1) & 0x03); // output enable | read input i
                        r = write(fd, &command, 2);
                        usleep(delay);
                        // the read is always one step behind the selected input
                        r = read(fd, &value[i], 1);
                        if(r != 1)
                        {
                                perror("reading i2c device\n");
                        }
                        usleep(delay);
 The full range of the analog value 0 - 255 would not fit on most screens so we scale down by a factor of 4. This should fit on a 80x25 terminal nicely.
                        value[i] = value[i] / 4;


Position the cursor at the start of the bar
                        move(10 + i + i, 12);
For each position in the graph, either draw a * to show the value or a space to remove any * that might be there from a previous value
                        for(j = 0; j < 64; j++)
                        {
                                if(j < value[i])
                                {
                                        addch('*');
                                }
                                else
                                {
                                        addch(' ');
                                }
                        }
                }

                refresh();
 Check the keyboard and process the keypress
                key = getch();
                if(key == 43)
                {
                        command[1]++;
                }
                else if(key == 45)
                {
                        command[1]--;
                }
                else if(key > -1)
                {
                        break;
                }
        }

Shutdown ncurses
        endwin();
        close(fd);
        printf("%d\n", key);
        return(0);
}


To compile this program you need to use a new flag -l which says to link with the spcecified library (ncurses)

gcc -Wall -o pcf8591d-graph pcf8591d-graph.c -lncurses

When you run the program you should see something like this:
PCF8591


Brightness  *****************************************************

Temperature *******************************************************

?           *********************

Resistor    *****************************************

While it is running the graphs should move as you change the inputs. Try for example shining a torch on the LDR or adjusting the Pot.

You can adjust the green LED with + and - (hint, use - to go from 0 to 255 for maximum effect). Any other key will cause the program to quit.

The graph shows nicely how the inputs can change but it also shows how the value can fluctuate without any input changes. I can't explain exactly why but I would expect much of the fluctuation is because of the lack of a stable external clock/oscillator and/or instability in the reference voltage. Needless to say this is a low cost demo board and may not be exploiting the full potential of the PCF8591.

Here is the complete source code (pcf8591d-graph.c):
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/i2c-dev.h>
#include <ncurses.h>

int main( int argc, char **argv )
{
        int i;
        int r;
        int fd;
        unsigned char command[2];
        unsigned char value[4];
        useconds_t delay = 2000;

        char *dev = "/dev/i2c-1";
        int addr = 0x48;

        int j;
        int key;

        initscr();
        noecho();
        cbreak();
        nodelay(stdscr, true);
        curs_set(0);
        printw("PCF8591");

        mvaddstr(10, 0, "Brightness");
        mvaddstr(12, 0, "Temperature");
        mvaddstr(14, 0, "?");
        mvaddstr(16, 0, "Resistor");
        refresh();
        fd = open(dev, O_RDWR );
        if(fd < 0)
        {
                perror("Opening i2c device node\n");
                return 1;
        }

        r = ioctl(fd, I2C_SLAVE, addr);
        if(r < 0)
        {
                perror("Selecting i2c device\n");
        }

        command[1] = 0;
        while(1)
        {
                for(i = 0; i < 4; i++)
                {
                        command[0] = 0x40 | ((i + 1) & 0x03); // output enable | read input i
                        r = write(fd, &command, 2);
                        usleep(delay);
                        // the read is always one step behind the selected input
                        r = read(fd, &value[i], 1);
                        if(r != 1)
                        {
                                perror("reading i2c device\n");
                        }
                        usleep(delay);

                        value[i] = value[i] / 4;
                        move(10 + i + i, 12);

                        for(j = 0; j < 64; j++)
                        {
                                if(j < value[i])
                                {
                                        addch('*');
                                }
                                else
                                {
                                        addch(' ');
                                }
                        }
                }
                refresh();

                key = getch();
                if(key == 43)
                {
                        command[1]++;
                }
                else if(key == 45)
                {
                        command[1]--;
                }
                else if(key > -1)
                {
                        break;
                }
        }

        endwin();
        close(fd);
        printf("%d\n", key);
        return(0);
}

scp files with spaces in the filename

scp is a great tool. Built to run over ssh it maintains a good unix design. It does however cause a few problems with it's nonchalant handling of file names.

For example, transferring a file which has a space in the name causes problems because of the amount of escaping required to get the space to persist on the remote side of the connection.

Copying files from local to remote is easy enough, just quote or escape the file name using your shell.
scp Test\ File remote:
scp 'Test File' remote:
scp "Test File" remote:

Copying files from remote to local can be more tricky
scp remtote:Test\ File .
scp: Test: No such file or directory
scp: File: No such file or directory

scp stimulus:"Test File" .
scp: Test: No such file or directory
scp: File: No such file or directory

scp "stimulus:Test File" .
scp: Test: No such file or directory
scp: File: No such file or directory

The escaping is working on your local machine but the remote is still splitting the name at the space.

To solve that, we need an extra level of escape, one for the remote server. Remember that your shell will eat the \ so you need a double \\ to send it to the remote
scp remote:Test\\\ File .
Test File              100%    0     0.0KB/s   00:00

You can also combine the remote escape with local quoting
scp remote:"Test\ File" .
or
scp "remote:Test\ File" .

Best solution of all, avoid spaces in file names. If you can't avoid it, I find the easiest solution is to replace ' ' with '\\\ '.
 

Programming I2C

Although you can perform simple i2c reads and writes using the command line tools i2cget and i2cset, for a more integrated approach you can use a programming language to talk to the bus.

The are dozens of languages which make claims about ease of use and learning etc. and I am sure you can program i2c from them.

What I will demonstrate here is the simple way to do it from c. Although I don't aim to teach how to program in c, I will try and explain what the code is doing so you can follow along even if you are new to c.

This will use some basic i2c read and writes as described at  http://www.kernel.org/doc/Documentation/i2c/dev-interface
We will also need to perform some IO Control (ioctl) which are i2c specific.

First we need some code to get us started. The #include basicly make certain function calls and constants available to the rest of our program.

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/i2c-dev.h>

All of our code will live in the main function for now. main() is where all c programs start.

We need some variables which must be declared at the start of the function.
int main( int argc, char **argv )
{
        int i;
        int r;
        int fd;
        unsigned char command[2];
        unsigned char value[4];
        useconds_t delay = 2000;

        char *dev = "/dev/i2c-1";
        int addr = 0x48;

The [ ] syntax means an array so char command[2] is actually a variable which can hold 2 char values.

Some of our variables have been initialised to specific values so they are ready to use. The ones which have not been initialised will contain random values so we must assign a value to them before they can be used.

0x48 means hexadecimal 48 (which is decimal 72).

Next we print out a banner to show that the program is running
printf("PCF8591 Test\n");

The we get down to business and open the i2c device
        fd = open(dev, O_RDWR );
        if(fd < 0)
        {
                perror("Opening i2c device node\n");
                return 1;
        }

and select our slave device
        r = ioctl(fd, I2C_SLAVE, addr);
        if(r < 0)
        {
                perror("Selecting i2c device\n");
        }

Now we have an infinite loop
         while(1)
        {
There will be no way to end the program except by pressing Control C.

Next we have another loop which will run four times
                for(i = 0; i < 4; i++)
                {
Then we build a command for the pcf8591. The value of this is specified in the data sheet http://doc.chipfind.ru/pdf/philips/pca8591.pdf

In the first 8 bits of the command we will enable the analog output bit (0x40) and select which of the 4 inputs to read ((i + 1) & 0x03). We do a bitwise or to combine these values together with the | symbol.
command[0] = 0x40 | ((i + 1) & 0x03); // output enable | read input i

The // is the start of a comment so you can explain you code to the reader.

In the next 8 bits we increment the value for the analog output
command[1]++;

Now we are ready to send the command to the i2c bus
r = write(fd, &command, 2);

It is not clear why, but we need to wait for the command to be processed
usleep(delay);

Now we are ready to read a value. Remembering that the read is always one value behind the selected input (hence the +1 we used above).
r = read(fd, &value[i], 1);
if(r != 1)
{
        perror("reading i2c device\n");
}
usleep(delay);

Then we end the loop
        }
and now we can print out our results
        printf("0x%02x 0x%02x 0x%02x 0x%02x\n", value[0], value[1], value[2], value[3]);

end our infinite loop
    }

and although we may never reach here, we will clean up and quit.
  close(fd);
  return(0);
}

Now, if you enter all the code into a file called pcf8591d.c (you can copy the complete code as show below) then you are ready to compile it with this command
gcc -Wall -o pcf8591d pcf8591d.c
This says to compile the .c file and write the output (-o) to pcf8591d (if you don't specify an output file the default of a.out will be used which can be a but confusing). -Wall will make sure all warnings are printed out by the compiler.

Assuming the compile (and link) was successful you are ready to run
./pcf8591d
and you should see output like this:
PCF8591 Test
0x5f 0xd3 0xac 0x80
0xc1 0xd3 0xae 0x80
0xc1 0xd3 0xb0 0x80
0xc1 0xd3 0xb2 0x80
0xc1 0xd3 0xb7 0x80
0xc1 0xd3 0xba 0x80
0xc1 0xd3 0xde 0x80
0xc1 0xd3 0xdc 0x80
0xc1 0xd3 0xe0 0x80
To stop the program press ^c (Control + C).

Here is the complete source code:
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/i2c-dev.h>


// Written by John Newbigin jnewbigin@chrysocome.net
// Based on info from http://www.kernel.org/doc/Documentation/i2c/dev-interface
// And http://doc.chipfind.ru/pdf/philips/pca8591.pdf

int main( int argc, char **argv )
{
        int i;
        int r;
        int fd;
        unsigned char command[2];
        unsigned char value[4];
        useconds_t delay = 2000;

        char *dev = "/dev/i2c-1";
        int addr = 0x48;

        printf("PCF8591 Test\n");

        fd = open(dev, O_RDWR );
        if(fd < 0)
        {
                perror("Opening i2c device node\n");
                return 1;
        }

        r = ioctl(fd, I2C_SLAVE, addr);
        if(r < 0)
        {
                perror("Selecting i2c device\n");
        }

        while(1)
        {
                for(i = 0; i < 4; i++)
                {
                        command[0] = 0x40 | ((i + 1) & 0x03); // output enable | read input i
                        command[1]++;
                        r = write(fd, &command, 2);
                        usleep(delay);
                        // the read is always one step behind the selected input
                        r = read(fd, &value[i], 1);
                        if(r != 1)
                        {
                                perror("reading i2c device\n");
                        }
                        usleep(delay);
                }
                printf("0x%02x 0x%02x 0x%02x 0x%02x\n", value[0], value[1], value[2], value[3]);
        }

        close(fd);
        return(0);
}
In the next blog I will improve the output to print a graph of the values so you can see them move up and down.