Launching multiple bash commands in parallel

At $work I’ve found a situation where I needed to run a batch of commands and actions multiple times. At first I did not think it would give us problems, but at the end it did.

The thing is that I have a loop that triggers a lot of actions, using a lot of funtions and using many variables. Using parallel is out of scope because the subprocess will not see the function definitions unless I export them all. Even though all the variables need to be exported: painful and dirty.

On the other hand, there’s another option. I could use a named pipe as a locking mechanism. Imagine that I want to allow N processes to execute inside the loop. At the begining the pipe is empty. I could use an int variable and increase its value in the main thread. As long as the variable is lower than N, no need to read the pipe and so we can run any command as needed. Once the commands are run, we decrease the variable and write something to the pipe. However, when the value arribes to N, we first need to read the pipe, which would block. From this moment on, any process will first need to read the pipe, it won’t continue until one the process running has write anything to the pipe.

 TEMP_FIFO=/tmp/test

 if [ -e ${TEMP_FIFO} ]; then
     rm ${TEMP_FIFO}
 fi

 mkfifo ${TEMP_FIFO}

 let counter=0
 for i in $(seq 1 500); do
     let counter=$counter+1
     if (( $counter < 50 )); then
         let seconds_to_wait=$(( ( RANDOM % 5 )  + 1 ))
         echo "Will wait ${seconds_to_wait}"
         (ls ~ > /tmp/sal$counter; sleep ${seconds_to_wait}; echo done > ${TEMP_FIFO})&
     else
         cat ${TEMP_FIFO} > /dev/null
         (ls ~ > /tmp/sal$counter; sleep ${seconds_to_wait}; echo done > ${TEMP_FIFO})&
     fi
 done

This option is enough for my purpouses. However, I’m thinking of initially writing N characters to the pipe and forget about the counter. Each process will read one character from the pipe, and when the job is done will write one character to the pipe.

Increasing the size of a ZFS pool

A few days ago I noticed my $home nagios was telling me one of my zfs pools was using all the space available. It is a 1TB pool, called multimedia consisting of two WD discs.

So I got myself two 3TB WD discs (right now there are two recicled WD green discs, which I know they are not ready for 24/7 usage, so I plan to get myself a WD RED in the future and replace one of those.

The process is quite easy:

1. Tell the pool one of the discs's gone missing. 
2. Plug in the new disc and use gpart to partition it. In my case I simply used a one large **freebsd-zfs** partition. 
3. Attach the new to pool, and tell the pool to use the remaining disc's data to fill in the new one.

Because the pool was set up with two discs in a mirror layout, when the first 3TB disc got plugged, the pool had two discs with differents capacities, so the total capacity of the pool couldn’t grow to 3TB (one of the discs still was 1TB).

Once the first 3TB disc is up to date, we would be able to replace the remaining 1TB disc that was still there. The process would the same we already described.

Let’s suppose the pool is called multimedia, and that the first disc was /dev/gpt/dades0 and the second was /dev/gpt/dades1. Let’s suppose the new disc is /dev/ada3.

To tell the pool one disc is gone missing:

    # zpool detach multimedia /dev/gpt/dades0

To partition the disc:

    # gpart destroy -F /dev/ada2
    # gpart create -s gpt ada2
    # gpart add -t freebsd-zfs -l dades0 ada2 

Finally, to attach that disc to the pool, using dades1 disc to synchronize the pool’s contents:

    # zpool attach multimedia /dev/gpt/dades1 /dev/gpt/dades0

Using dhcp classes

Today I faced a new problem. There at $work we have a few VLANs with private addresses, in one of those the users plug their Cisco routers and then they do some testing using those Cisco devices as gateways of some other devices (that is, we end up doing double NAT, which works fine btw).

However, I’ve been asked to stop providing IP addresses to just those devices.

My setup is as follows:

1. There are some devices with fixed IP addresses. I map those addresses using their MAC address. 
2. There is a pool of addresses used to configure unknown devices. I apply special policies to those devices.

To keep that setup I can’t simply remove the pool of addresses, it would do the trick with the Cisco routers but I will loose the ability to autoconfigure unknown devices. Instead what I would need would be to somehow deny those Cisco devices, and keep the other policies.

To do so, I could simply list those that bunch of Cisco devices and deny them all. But that’s a real PITA and it is not even cool. Instead, what I will use will be the vendor bytes of the MAC address to group them into one dhcp class and deny them with one dhcpd.conf directive. Because all the routers are Cisco they will probably share the first bytes of the MAC address.

Unfortunately I have two sets of Cisco routers, each set has a different MAC prefix, such as:

    00:26:0b:*  00:15:62:* 

First I need to group them into one dhcp class, this class will then be denied by the DHCP server. Well, in fact they will be simply ignored by the DHCP server but the directive that should be used is called deny.o

I assumed that I will only dealing with ethernet clients. The DHCP server will prepend the 01 byte to every DHCP request it receives. If we assume we will only be dealing with ethernet client we can simply skip that byte and work with the client’s dhcp address.

To group those Cisco address, we need to compare the first bytes of the MAC address against, say, 00:26:0b. If the address matches, then it will be placed into a class, its name can be whatever you want. To do that, we can use the following dhcpd.conf directive:

    class "ignorats" {
            match if
                    (substring(hardware,1,3) = 00:26:0b);
    }

The class gets named ignorats (means ignored in catalan). Let’s see what this does. It takes, skiping one byte there bytes from the MAC address (remember the 01 byte we could safely skip if were to assume there would only be ethernet clients?) and compares that against 00:26:0b. If that matches the host is placed into an special dhcp class called ignorats.

But, as we previously said, we have more than one MAC prefix to check (remember I had two different sets of routers?). We need to, somehow, compare the hardware address against two or more patterns and if the match place that host into the ignored class.

Fortunately we can use logic connectors in the match directive. We can now transform our class definition into something that would work better:

    class "ignorats" {
            match if
                    (substring(hardware,1,3) = 00:26:0b) or
                    (substring(hardware,1,3) = 00:15:62);
    }

Finally we need to, at last, deny (ignore) those clients. Because every VLAN has a pool of dynamic assigned addresses, I would need to deny those Cisco clients in every VLAN (pool) they can be connected to:

    pool{
            range 10.0.10.230 10.0.10.250;
            option routers 10.0.10.1;
            deny members of "ignorats";
    }

That would do the trick.

PS: while checking the documentation and searching around I found some people convert the hardware address to a text string to do match. This is quite error prone because the text conversion will convert the MAC address into its shortest version, removing the leading 0 for every byte. Our 00:26:0b will be converted into 0:26:b.

Why would I need to add a text conversion and then use the shortest MAC version in the match directive when I can use a simplier command and use the full MAC address, which is the MAC that appear in every device after all? Up to you, if you want to do that take a look at the binary-to-ascii dhcpd.conf operator. But I would certainly discourage its usage (at least if you simply want to check the MAC address of every client).