Brain Log - Think it out. Write it down.

Tunneling a connection through a firewall using ssh is pretty straight-forward (although it must be explicitly allowed by the sshd configuration). Here’s my script to do it (using autossh which helps to keep the ssh link up when there’s little activity).

#!/bin/bash

# Start a tunnel to ccom port 3389
echo "Starting a tunnel to ccom"
sudo autossh -M 20001 -L 3389:sandlance.ccom.nh:3389  -N vschmidt@ccom.unh.edu 

echo "Now remote desktop to localhost."

As a test, I installed webmin on my Linux server today. It’s pretty slick!

Here’s the recipe for Fedora 7:

Set up yum by creating /etc/yum.repos.d/webmin.repo containing:

[Webmin]
name=Webmin Distribution Neutral
baseurl=http://download.webmin.com/download/yum
enabled=1

Get the GPG key for webmin:

sudo rpm --import http://www.webmin.com/jcameron-key.asc

Install webmin with yum

sudo yum install webmin

The installation, by default, allows access only from the local machine by root (your machine’s root). However you can allow access from other machines by adding the following line to /etc/webmin/miniserv.conf

allow=xxx.xxx.xxx.xxx

where xxx.xxx.xxx.xxx can be an IP address. There’s a way to specify an network here as well but the syntax isn’t clear to me just yet. (actually there’s an example in the webmin configuration web page for this)

During the installation, webmin did not recognize the Fedora syslog install properly. So I had to set up the syslog module configuration manually. This consists only of specifying the correct path for syslod, the syslod pid file and a few other things that will be obvious on the webmin configuration page for it.

I also set up bandwidth monitoring for eth0. In the bandwidth monitoring module page you simply select the network interface you'd like to monitor and then select the “setup now” button. The only tricky part here is that the setup adds entries to your iptables firewall. Since I roll my own firewall from a script and then save the results (with service iptables save) so it is loaded automatically on boot, these additional lines are not captured in my script. On the list of things to do is to decipher exactly how to specify these lines manually so if I make manual changes to the firewall I don’t loose them. (and them of course remove them if I decided I no longer want bandwidth monitoring)

I noticed there is a module for webalizer logfile analysis. Webalizer is a very handy tool for monitoring all kinds of system status on a computer with lots of nice graphical plots. But I'm not sure this is the same thing (but I wish it was). I have yet to figure this part out exactly.

I've finally fixed my code for plotting beam patterns in MATLAB. (An update to a post from a few days ago.)

John Kelly of NOAA sent me a screen shot of the coolest page showing NOAA forecast model output in graphical form. (we're in for a load of snow tomorrow)

After a little sleuthing around on the NOAA weather site, I found it. Here’s a link for Dover.

I didn’t know about any of these super handy screen capture shortcuts on MBP. SO cool.

Often you want to plot up the beam pattern of a sonar (or RF antenna) array. Here is some simple code to handle the task for a rectangular array with the given dimensions and wavelength

Ok, here’s the test of using base_url (surrounded by “%” as in the title) in nanoblogger to ensure relative links are resolved correctly.:

It works!

The problem was this: relative links for images posted to my blog were broken in the archives of the the published version. This method of specifying links allows me to continue to use relative links and ensure the url in the archives remains correct where-ever my blog happens to love. Special thanks to Terry, Kevin and Haul on the nanoblogger list for convincing me that this was the right thing to do. I stubbornly was not sure.

There’s an interesting recent publication by Luciano Fonseca and a small crowd of other scientists here at CCOM in November’s Hydro-International.

It is all about the Law of the Sea data collected over the past several years and gives some insight into how surveys are conducted by some of the best in the industry. Worth a read.

It’s come to my attention that, in the nanoblogger archives, the images don’t show up because I've inserted them with relative links. Ugh. So from here on our I'm going to use absolute ones so everything will work. I don’t like it. I take my blog with me – it’s what I like about nanoblogger and now none of the images will resolve on my laptop when I'm not connected to the network There must be a better way.

To make things easier, I've modified my blog/templates/entry.metadata to include a commented out line of html to help me remember how to add an image. If all works right, you should be able to ‘view source’ and see it in this page somewhere.

This week I've been doing lab testing of the wifi link between our base-station:

and our buoy:

because we froze in the cold trying to figure out why we couldn’t get a link a few weeks ago.

There is a nice link budget calculator here.

Here are our specs:

Base Station

• Tx Pwr: 200mW (10 x log10( 200 mW / 1mW) = 23dBm spec sheet says 20 dBm???
• Cable Losses: 4 dB (estimated)
• Antenna Gain: 15 dBi
• Rx Sensitivity: -86 dBm (11 Mbps) -92 dBm (1 Mbps)

Buoy

• Tx Pwr: 100mW (20 dBm)
• Cable Losses: 4 dB (estimated)
• Antenna Gain: 5 dBi
• Rx Sensitiity: -95 dBm (estimated)

There are several values I'm not sure about. For example, on the buoy the device is a BnB Electronics 4-port serial server which has an Atheros mini-pci 802.11b card. The specs on the web page say Tx power of 16 dBm, but the configuration page allows one to select 100mW (20 dBm). Also receiver sensitivity for the card appears to be -95 dBm (at least that’s what the driver says if you, uh, hack the server and look. ahem.) But the web page says -82 to -92, and another page with a similar card gives -95 dB for 1 Mbps and -90 dB for 11 Mbps.

Our base stations, which is a Metrix Mark II kit, doesn’t support wireless statistics collection (/proc/net/wireless is all 0’s for link levels). So to monitor the link I'm using the statistics from the one side only – the buoy.

On the buoy, I've added a web page cgi script to capture the wifi statistics. I'd reproduce it here, but embedding html in html is tricky. Here’s the beginning and the rest is all printing html.:

#!/usr/bin/perl

# Wireless monitor script
# Val Schmidt
# CCOM/JHC
# JAN 2009

$stats = `/bin/cat /proc/net/wireless`;
if ($stats){
    ($line1, $line2,$line3) = split('\n',$stats);

    ($interface,$status,$snr,$level,$noise,$nwid,$crypt,$frag,$retry,$misc,$beaconmissed) = split(/\s+/,$line3);
}

There is a trick to getting this to work that’s worth noting. In perl, when printing the html, you should use this method:

print <<HTMLEND;

htmlstuff

HTMLEND

I can’t explain why, but if I print the html stuff in separate print statements, apache refuses to serve it up, giving a mal-formed header error on the first line. To my eye, the result is exactly the same.

To long the wifi link levels as seen by the buoy, I'm querying the buoy cgi script above, parsing and plotting the results. All this is in python, of course using matplotlib (my not quite satisfactory attempt in an earlier post). Here’s the code:

#!/usr/bin/python
#
# snrplotter.py
#
# Val Schmidt
# CCOM/JHC
# 2009
#
# A program to capture and plot snr levels from our slightly hacked
# serial wifi server. One can login to the serial-to-wifi server (BnB
# electronics) via telnet (login:root, password:not required> ). I then
# installed a new cgi script in the web pages which captures the wifi
# stats from /proc/net/wireless. Without this in place, none of this is
# possible. 
#
# This script will poll the web page at 1-second intervals and display
# the results.

import sys
import os
import matplotlib.pyplot as plt
import numpy as np
import datetime
import time
import urllib2
import re

# This doesn't work yet. Must be 60.
history = 60

sec = range(history)
snr = np.zeros((1,history))


filename = datetime.datetime.now().isoformat().replace(':','-') + '_snr.log'
F = file(filename,'w',0)
#cnt = datetime.datetime.now().second
#secondstodisplay = history/60.0

plt.draw()
plt.show()
time.sleep(1)
while 1:

    data = urllib2.urlopen('http://10.10.0.2/cgi-bin/stats.cgi')
    for line in data:
        if line.find('ath0') != -1:
            fields = line.split(',')

    dts = datetime.datetime.now()
    nowsec = dts.second
    value = int(fields[2].replace('.',''))
    snr.put(nowsec,value)
    print dts.isoformat() + '   '+ str(snr.item(nowsec))

    plt.cla()
    plt.plot(sec,snr.tolist()[0],'ro')
    plt.plot([nowsec],[value],'bo')
    plt.xlabel('CURRENT:  ' + str(value),fontsize='large')
    plt.axis([0, history, 0, 100])
    plt.show()
    plt.draw()

    F.write(dts.strftime('%Y\t%M\t%d\t%H\t%M\t%S').expandtabs() +'\t'+str(snr.tolist()[0][nowsec]))
    time.sleep(1)
    

This produces a plot that looks something like this:

For this data collection the buoy antenna was removed (bar coax) and the base station was pole mounted abut 6 feet higher than the buoy. Hence the relatively low snr levels.

Now that I had this set up I wanted to compare my own results with the theory and see if we'll be able to get the 2-3 km range we desire. The link budget calculation looks something like this:

Fade Margin = TxPwr - CableLoss + Tx Antenna Gain - FreeSpaceLoss
+ Rx Antenna Gain - CableLoss - Receive Sensitivity

The rule of thumb is, all else being equal, a Fade Margin of 10dB minimum is required for a link. Free Space Loss we haven’t talked about yet. Free Space Loss are the losses due to absorption spreading of the signal. Wikipedia has a nice entry for Free Space Loss calculations. In a nutshell:

FSL = (4 pi d / lambda)^2

where d is the distance from the transmitter to receiver and lambda is the wavelength. It is more commonly written in dB like this:

FSL = 20*log10(d) + 20*log10(f) + 32.44

where f is the frequency in MHz and 32.44 combines the constants and unit conversions. Here is a plot of Free Space Loss for 2.4 GHz wifi:

So next turned to the link budget equation and calculated the Fade Margin (here called Link Excess) vs Range from the theoretical specs above.:

Of course, since I was monitoring link level I thought to check to see that the theory works. In my setup the antennas were 3.8 m apart in the same vertical plane. I took the remaining values from the specs above. I found that my received signal level was about 15 dB below what theory would predict. There are lots of reasons this might occur, but for now, I plotted a second curve of Fade Margin vs Range with this 15dB reduction. It shows that for a 10 dB Fade Margin our max range would be about 1600 m. I think a 15-20 dB Fade Margin might be required however reducing our range to about 1000m.

All of this assumes a clear line of sight however. And it’s not just the line of sight but the entire fresnel zone that must be relatively clear. Most documentation I've seen recommends a minimum 20% obstruction of the fresnel zone for reasonable link. The higher the power, the more the fresnel zone can be obstructed (to a point) without loss of signal. I made a plot using the equations (from the link above) for Fresnel Zone Radius:

Fn = sqrt( (n lambda d1 d2) / (d1 + d2) )

where Fn is the fresnel zone radius of the n'th zone, lambda is the wavelength, d1 is the distance from one antenna to a point between them and d2 is the distance from the second antenna to the point between them.

Here the antennas are separated by 1 km. The antenna on the right is the buoy, 1m above the water, and the antenna on the left is the base station, 5.6 m, above the water. The blue line is the line-of-sight between them. The green curves denote the 1st fresnel zone. Disregard the red curve – it doesn’t do what I thought.

In this plot at 300 meters the fresnel zone is approximately 10 m in diameter and about 3 m or 30% is occluded by the water. This brings up (yet) another question for which I don’t know the answer. Over seawater, do RF signals bounce at low grazing angles, as one might expect for other waves, or is salt water sufficiently absorptive that these signals are truly occluded? My guess, I suppose, is that they are scattered by the water’s surface, and absorbed when the wave shape is such that the angle of incidence is not too shallow. Although some signal might be received by such a scattering surface, probably not much and might not be usable by the receiving system.

So what’s the upshot? There’s some 15dB of signal we're loosing that we might be able to restore with careful checking of connectors and antennas. And with a 1 m high antenna on the buoy we're going to need all the signal we can get.

I am always forgetting the rule of thumb for determining when one is in the near-field of a transducer, transducer array or antenna (the math is the same). There is a very nice description of the issue here..

From the text above:

The distances where the planer, parallel ray approximation breaks down is known as the near field.  The crossover distance between near and far fields (R) is taken to be where the phase error is 1/16 of a wavelength, or about 22.5 degrees.

R =  2 D^2/ L

where L is the wavelength and D is the largest dimension of the transmit antenna.

So for example, in a project I'm currently working on, we are building buoys to be used in coastal waters with a WiFi link to shore. The shore base station antenna is 1 m long. The WiFi link is 2.6 GHz which is a wave length of 2.00702458e8 / 2.6e9 = .12 (12 cm). So the far-field for this antenna starts at:

2 * 1^2 / .12 = 16 m

I'm troubleshooting our WiFi link in the lab and I'm definitely in the near field. So my results have sometimes been hard to interpret.

My experience in writing a real-time plotting routine using matplotlib has been less than satisfactory. In my first go, unless I created my figure window manually before running the plot it never got created (even though there are draw() commands in the plot loop). Then, when it does work, if I move my figure window on the screen it stops updating (goes white) and I have to restart the application.

So when I found this page describing the combination of matplotlib with wxpython I was very excited. I've tried the real-time plot available there and it works like a champ as best I can tell.

My only beef, is that I do NOT want to have to muddle through wxpython code for every application I write. So when I have the time, my hope is to borrow the technique above and create a plotting container template that will allow me to focus on the matplotlib and data parsing code rather than than the wxpython code. Hmm. How to best do this…

I made a plot of the multibeam data collected by the USCGC HEALY during their 2008 Law of the Sea cruise using MB-System.

This link has a nice collection of global raster data sets. Something to remember.

I have just endured a BRUTAL several months of revision to a paper on a LBL-type whale tracking system. A portion of the revision involved how one estimates the uncertainty in the resulting whale track. It hurts my head just to think about, but I wanted to make some notes for future reference.

• My first recommendation is to first read the Guide to Uncertainty In Measurement and understand the difference between Type A errors (those you can measure with statistics) and Type B errors (those you can only estimate) and their recommendations regarding how to estimate them. If you do this before you even begin a measurement project it will help you think about what you need to measure later on. (The GUM is a NIST/ISO document that isn’t free so you won’t usually find it floating around the web willy-nilly.)
• When ever possible, try to make a series of ground-truth measurements (i.e. try to position something for which you know the position already). Using real measurements to characterize your uncertainty is far far easier than trying to estimate it by other means.
• When you have to measure the uncertainty in various parts of a measurement and then propagate those uncertainties through a subsequent calculation (using the law of propagation of variance), it is worth while to list every measurement and how you will assess it’s uncertainty. For example, in our whale tracking, how do you know the GPS uncertainty, the detection algorithm uncertainty, the sound speed and ray-tracking uncertainty, the uncertainty associated with false detections or multipath, etc. Understanding this up front may help you design measurements to address them before you get to writing your paper.
• When attempting to assess the uncertainty in some measurement, don’t be tempted to look at an O-scope, and roughly characterize the uncertainty by the maximum variation you see. It might be more conservative, but it is too hard to describe later. Make actual measurements (preferably with a data acquisition system) save the results, calculate a real statistic (RMS, std deviation, etc.)
• Finally, if it were all possible, it would be worth while to get buy-in from all involved on the strategies for assessing each of these uncertainties early on in the process.

There’s more to say and more to think about, but this is enough for now.

Because I always forget, here’s how to convert difference in lat and lon to heading:

# First calculate the difference in x and y coordinates.
# This requires correction of the longitude difference for the mean latitude (a rough approximation).
dx = lat2 - lat1
dy = (lon2 - lon1) / cos( mean( lat1, lat2 ) * pi/180 )

# Then calculate the heading. Note that atan2 normally takes arguments (dy,dx). These are swapped intentionally here.
mod(atan2(dx,dy)*180/pi + 360,360)