Recently in Darien-WiFi (AnywhereTechnology) Category

FiOS/Fiber to Curb - Why not Me?

| | Comments (1)

Owning a small broadband ISP has produced its own series of experiences I never imagined having - and thoughts I never imagined coming from my own person. (For example, I never thought the day would come when I would say 3 Mbit/s is good enough because the cost of increasing said bandwidth is prohibitively expensive.) Reading Verizon hits the gas on fiber campaign (from CNet) started me thinking again about how technology will affect where people live just as the industrial revolution did. I have my doubts it will be as severe as what we have learned about the effects of industrialization - e.g. people moving to the city to specialize in careers instead of being general handymen with no real talent - but many of the choices and opportunities people in my generation have relates to the technology to which they have relatively cheap access. With Verizon's move to bring fiber (enabling a blinding amount of media and data to be efficiently transported to a home or business) to small areas of large communities, it makes me wonder how I can work to bring that type of advanced technology to the neighborhoods I serve. From a business and practical standpoint, delivering the kind of reliability fiber offers without the fiber is a daunting task. Already with our 900MHz and 5.7GHz services, it is difficult to tell what type of RF interference the day is going to bring. Licensed technologies are almost as daunting as trying to use wired technologies. Both require working with bureaucrats and getting licensing to do something that I am doing for more altruistic reasons than the money. Sure, there is some money in providing these types of technologies to people, but nobody wants to pay the real price and you are required - as an operator - to give your business customers the harder end of the bargain. I have a hard time with this type of marketing. I am often inclined at changing midstream and simply marketing to businesses again. I say to myself, however, that the children of our community deserve better. The opportunities that technology brings is staggering if nurtured and used in the correct manner - i.e. learning about your surrounding world instead of looking up porn or playing the latest mindless video game. (Didn't you hear that IM'ing makes you 10 IQ points dumber? [hint sarcasm])
What it boils down to is this: the cost of true broadband services (i.e. DS3) is still prohibitively expensive. I as a second/third tier ISP would love to have that type of connectivity to relay it to our customers (of which, there are very few who actually use much bandwidth). The aggregate cost of supplying ISP's like myself bandwidth is lower than one could imagine due to the fact that so much remains as available excess. Verizon wouldn't be forced to spend billions on ripping up streets. They could spend tens of thousands to provide access to customers like myself who can in turn bring the broadband dream to others.
Hopefully the continued advances in technology will make this possible. If not, we may see another type of industrialization to population movement just because of the internet.

Anti-Virus Gateways? Lack of reviews

| | Comments (0)

Is it possible that the sheer lack of reviews out there means that no one is using these devices or does it simply mean that no one that has implemented one has had the time to review it? I am looking into installing a virus, spam and spyware scanning gateway for a number of clients on our network. I have looked into products by SonicWall, TrendMicro, Symantec and McAfee, but am having trouble finding reviews on any of the products. It is great to see the claim that they can do all of these things and more (firewalling, nat, vpn, etc), but I want to know how effective their products are. In the process of researching the different devices offered by all of these companies, I have come across SafeSquid. It is a proxy based upon Squid that also has the advertised ability to scan for viruses. Through a bit of research, though, I was unable to find whether it would also help with detecting spyware. In my opinion and based upon experience, I'm seeing spyware as just as great or greater of a problem than viruses. (Perhaps because anti-virus software seems to be fairly mature whereas anti-spyware software really has a lot of difficult obstacles to overcome before it reaches the quality or reliability of anti-virus software.) If I am filtering viruses at the point of origin for the network, it would be no better than what I currently have (nothing other than the firewall) if everyone were to become overrun with spyware. My buddy Jeff and I spent four hours at a customer's home back in October "fixing" a computer completely bombed with spyware. If we could reduce the number of calls people make regarding problems with their computers (which will inevitably happen with the expansion of our residential service), we would alleviate a large future headache that I am very concerned about.
Given my intention to find a device that will help me protect 40-75 customers at < 6 mbps (all future, not current), I think it would be intelligent to see what other people are using and learn from their experience instead of being the one to conquer the world. I have neither the time nor energy to be a leader in this area, as there are plenty of other issues I want to deal with at this point (like getting towers up and running and increasing bandwidth availability to my customers).
So, if anyone out there has had experience with any of the above listed companies/devices, please drop me a line. I would love to hear from you (yeah - the two of you who look at this website once a year). In the mean time, I am working on finding vendors who would be willing to let me try their products to see if they even work the way they claim they should. Wouldn't that be ducky?

...need I continue. Anywhere Technology has a good number of new projects on the horizon that I'm hoping to document with this weblog. One of my last projects was to get bacula up and running to finally take care of those nagging late night rememberings that I forgot to run my weekly backups. I had intended to document that project but found so many resources on the web that described how to install and configure bacula that I gave up trying to document it. One word to any aspiring administrators: never forget to document what you have done to create or install a new process that you depend on. What I have fooled myself into thinking (more times than I can count) is that if the documents exist now, they will exist when I need them again. The problem is that every new server/daemon I have put into service is tweaked to fit the needs of the system on which I am installing it. The fact is that if you are good enough and are capable of following directions (obscure as they may be), it is likely that you won't have to go back to do any work on what you implemented for some time. By the time your return to your project for maintenance or major upgrades, you have forgotten much of what makes your installation work for you. So, one of my new New Year's resolutions is to actually document what I do.
With the growth in the ISP portion of Anywhere Technology, I have a number of very difficult decisions to make. Now I better understand many of the difficult choices my Dad had to make all the years nurturing the business at Tankcraft. While it is relatively easy or straightforward to run a small business, the difficulty is twofold. One, if you are small, it is difficult to make things work financially - to support the ongoing costs of doing business and try to cut a paycheck for yourself. Two, if the opportunity presents itself to grow the business, you have the tough choice to give less attention to your current customers and go after new ones. In my situation, it seems like the timing is good to go after new territory because current customers are relatively quiet. I won't forget what my Dad told me early on that it is easy to get new customers, but it is difficult to keep them. That phrase rings loud in my head when I contemplate moving ahead. Once the decision is made, we find ourself with yet more decisions. See the routine? Decisons, decisions, decisions...
What does the future hold? Right now I am working on our new tower design. I am trying to decide whether to dump my current manufacturer, Trango and go with Motorola's Canopy system. I'm not totally unhappy with Trango, but there are nagging issues of which I don't see any move towards resolution (ahem - fixing the SNMP monitoring of bandwidth usage on the M900S SU's that we were told would come some months ago - to name one). I also like the outlook that Motorola paints for its Canopy product. I've been burned by Moto before (not directly), but I think that they have resolved some of their management issues that created the problems in the past. Hopefully I will hear from some current Canopy users tomorrow or at least by Thursday to help make my decision.
Hopefully I'll have time to talk more about my experiences with my current equipment... In the mean time, I'll go to bed so I can get some more work done tomorrow.

Written by Steven N. Fettig
Rev. 1.1 - 8 May 2004
(See Creative Commons Copyright in right pane for copy restrictions)
Materials:
IBM Small Form Factor Workstation (Overkill)
Integrated Intel 10/100 NIC
Sangoma Dual CSU/DSU T1 PCI Card

Preface: As part of the Darien-WiFi project, I spent the past two weeks trying to install a router for our two T1's from Global Crossing. I had originally intended to use FreeBSD as the base system. What I found was that there was no (documentation showing a) way to do native load balancing across both T1's. The Sangoma Wanpipe drivers for Linux were able to do load balancing using the TEQL module (used in firewall QoS). Although an open supporter of what works - which can often be Linux - there was no way that I was going to use Linux on my router considering that I have had much, much more experience with FreeBSD and am familiar with locking it down for security uses. With quite a bit of Google digging, I found that OpenBSD'spf was not only capable of doing load balancing, but a whole array of items that I had spent much time learning how to do with ipfw on FreeBSD. After working until 10:30 pm on a Wednesday, I decided not to try OpenBSD, but instead opted for the pf port for FreeBSD. At about 1 am the next morning, I found out that not only would pf not load on a pre-5.0 machine (I know, I know - it says so in the docs, but I'm not enough of a kernel hacker to understand that it really wouldn't work), but I also couldn't get the Wanpipe drivers to load on the FreeBSD 5.2.1 system I just installed. By this time, I had been in contact with the Wanpipe BSD developer, Alex Feldman, and found that he hadn't thoroughly tested the driver install packages on 5.1 + and he asked me to wait for him to do his work. (He was extremely helpful - if you continue reading, you will find that I found more than one helpful soul through this whole process.) Instead of waiting, though - I had already spent enough wasted time on this project (which might have been completely avoided by using a Cisco Router with dual CSU/DSU's)* - I decided to dive into OpenBSD for the first time. I thought, with all of the structural similarities, it shouldn't be that difficult to acclimate myself to OpenBSD's peculiarities. So, here are basic instructions as to how I was able to get load balancing to work using the Sangoma card and pf on OpenBSD.

Instructions:
It happens that the University of Wisconsin - Madison runs an OpenBSD mirror. I downloaded the 3.4 boot iso from http://mirror.cs.wisc.edu/pub/mirrors/OpenBSD/3.4/i386/ on my OS X workstation:


wget http://mirror.cs.wisc.edu/pub/mirrors/OpenBSD/3.4/i386/cd34.iso

Using DiskTools, I burned the bootable iso image to CD. There are a slew of tools out there to burn ISO's to CD. I assume that if you are reading how to configure a custom workstation/server to be your router, that you know how to burn a CD. I put the CD in my future router and booted the machine.
I was careful to follow the install instructions at:

Some people have complained that FreeBSD's install procedure was difficult... well, they obviously have not tried to install OpenBSD. This is not to say that I found it all too difficult - I'm neither afraid of a command line, nor am I worried if I hose the system - I can rebuild it, right? I learn best through trial-and-error anyway. This was one time, however, that I was able to get the base system installed without a hitch. Well, that is not entirely true - I tried to do a base install on a system with a Broadcom Gigabit NIC and could never get the NIC to activate during the install. Perhaps had I read the supported hardware information on the main site, I would have found that support for the NIC needs to be added via a custom kernel. I had another machine waiting idly by to get used - with an onboard Intel 10/100 NIC (supported by the generic kernel) - and went through the same procedure again.
Because I had not yet received the CD's I purchased (c'mon - support the open source projects you use!), I installed from the UW-Madison http server.
After choosing the install source, I selected the following filesets (in FreeBSD lingo, I suppose this is base packages):

The following sets are available. Enter a filename, 'all' to select all the sets, or 'done'. You may de-select a set by prepending a '-' to its name.
      [X] bsd
      [   ] bsd.rd
      [X] base34.tgz
      [X] etc34.tgz
      [   ] misc34.tgz
      [X] comp34.tgz
      [X] man34.tgz
      [   ] game34.tgz
      [   ] xbase34.tgz
      [   ] xshare34.tgz
      [   ] xfont34.tgz
      [   ] xserv34.tgz

There are a few more items that need to be configured after the appropriate distributions have been downloaded. Again, follow the install FAQ for assistance. I would take more time to outline how to install your OpenBSD system, but the FAQ provides an amazingly detailed explanation of the installation procedure. It would be wrong of me to provide another explanation when it is laid out so well in the OpenBSD documentation.
Since I had already physically installed the Sangoma card, I rebooted the system and started to install the drivers and configure the T1's (according to the settings that Global Crossing is using). I downloaded the Wanpipe drivers and software:

# wget ftp://ftp.sangoma.com/OpenBSD/current_wanpipe/wanpipe-obsd-1.5.2-b2.tgz

I took care to follow the installation instructions given to us by Sangoma and simply ran:

# pkg_add ./wanpipe-obsd-1.5.2-b2.tgz

I had the install script do all the defaults - install drivers, add support to kernel, install startup scripts, etc. Everything went off without a hitch. When I had originally tried the installation on the pre 5.x FreeBSD system, the compilation of the drivers failed. I have yet to test the Wanpipe installation on OpenBSD 3.5, so I can't say whether it works or not. Due to the stability of OpenBSD, I imagine the packages will install without any problems as they did on my 3.4 system.
Again, following instructions that come with the Sangoma card, I configured the T1's.

# /usr/sbin/wancfg

This part requires that you have pertinent protocol and IP information from your provider.
Sidenote: I must say that Global Crossing is amazing. Their Circuit Turn-up staff was amazingly helpful. Through this whole ordeal, a gentleman named John Brox helped me out and was patient with my insistence on doing a custom install - I say patient because I initially was not able to even bring up the circuits with a box I had intended to use (a Eden 500 VIA mini-itx machine), as the ethernet interface caused the circuits to cycle up and down at weird intervals. I don't know if this was a problem with the motherboard itself or with the VIA ethernet interface. I have yet to do the same install on another VIA mini-itx based system, so I don't know. He was more than willing to help me make sure the circuit was up for good before turning it over to the monitoring group.
I haven't worked on routers in years and considering that, not only does Sangoma make the circuit configuration easy, but Global Crossing made the circuit turn-up a breeze.
On to the actual configuration of the firewall and load balancing. The Wanpipe drivers give you two interfaces to work with (once the T1's have been enabled). The last line in wanrouter.rc (in /etc/wanpipe) should read:

WAN_DEVICES="wanpipe1 wanpipe2"

so that both lines are enabled when the machine is rebooted. If you have not rebooted the machine, you can start the wanpipes by (assuming you configured wanpipe1 and wanpipe2):

# wanrouter start wanpipe1
# wanrouter start wanpipe2

After starting the circuits, running

# ifconfig -A

gives you:

wpachdlc0: flags=51 mtu 1500
     inet 22.109.101.234 --> 22.109.101.233 netmask 0xfffffffc
wpbchdlc0: flags=51 mtu 1500
     inet 22.222.9.162 --> 22.222.9.161 netmask 0xfffffffc

at the end.
You also need to make sure that your machine is set up to act as a gateway. Straight from the FAQ: http://www.openbsd.org/faq/faq6.html

The GENERIC kernel already has the ability to allow IP Forwarding, but needs to be turned on. You should do this using the sysctl(8) utility. To change this permanently you should edit the file /etc/sysctl.conf to allow for IP Forwarding. To do so add this line in that configuration file.

net.inet.ip.forwarding=1

My internal ethernet interface (fxp0) was set as 24.192.154.1 - the beginning of my address block available for use. At first - just to try the load balancing - I modified the rules at http://openbsd.org/faq/pf/pools.html#outgoing for my interface:

lan_net = "24.192.154.1/24"
int_if  = "fxp0"
ext_if1 = "wpachdlc0"
ext_if2 = "wpbchdlc0"
ext_gw1 = "22.109.101.233"
ext_gw2 = "22.222.9.161"

so that the whole pf.conf looked like:

# define the interfaces
lan_net = "24.192.154.1/24"
int_if  = "fxp0"
ext_if1 = "wpachdlc0"
ext_if2 = "wpbchdlc0"
ext_gw1 = "22.109.101.233"
ext_gw2 = "22.222.9.161"

# nat outgoing connections on each internet interface
nat on $ext_if1 from $lan_net to any -> ($ext_if1)
nat on $ext_if2 from $lan_net to any -> ($ext_if2)

# default deny
# I disabled these rules because we are in the testing phase - i.e. I am not yet using this as a real firewall - do not forget to re-enable these rules!
#block in from any to any
#block out from any to any

# pass all outgoing packets on internal interface
pass out on $int_if from any to $lan_net
# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if
# load balance outgoing tcp traffic from internal network.
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto tcp from $lan_net to any flags S/SA modulate state
# load balance outgoing udp and icmp traffic from internal network
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto { udp, icmp } from $lan_net to any keep state

# general "pass out" rules for external interfaces
pass out on $ext_if1 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if1 proto { udp, icmp } from any to any keep state
pass out on $ext_if2 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if2 proto { udp, icmp } from any to any keep state

# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
pass out on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 to any
pass out on $ext_if2 route-to ($ext_if1 $ext_gw1) from $ext_if1 to any


At this point, I was able to ping out of both interfaces, ping beyond the gateway, ping internally, etc. So, I set up a client behind the firewall and ran speed tests. Lo and behold, I was getting load balancing on the inbound (vis a vis Global Crossing's router configuration - it equally passes packets down both pipes), but it did not look like it was working on the outbound. It was about 12 am by this time and I was simply happy to see the OpenBSD machine working with the T1's and everything being pingable. Plus, John was gone by this time. I decided to call it quits and start again in the morning.
I woke up still puzzled as to why I couldn't get load balancing on the outbound route to work - theoretically, I had everything configured correctly! I called John and asked him if he could take a look at packet counts while I was running the bandwidth tests. After running a few tests, we found that the outbound were indeed being passed across both interfaces, but for some reason the bandwidth tests were not showing that. So, I decided to try to run more than one bandwidth test at once to see if I could fill up the pipes with more data. After getting my timing right (I kept starting one test well before the other), we were able to show that both pipes were indeed being load balanced, but for whatever reason, the speed tests would "sense" a limit at the 1 T1 barrier. Running multiple speed tests showed that just as much bandwidth was available down as it was up - i.e. just under 3 mbit/s. John was able to confirm that data was being sent down both pipes - one being favored by about 2% over the other. We chalked this up to "imperfect" round-robin'ing and called it a day. To this day, I still have not been able to figure out why the speed tests show outbound limited by one T1, but I don't really care as long as my available outbound is really the two T1's. I'm sure a ttcp test would be a better measure of throughput, but I haven't had the time to run such tests and am content that things are working as advertised.
After closing the firewall by enabling the default deny rules, I started working on allowing inbound connections. I struggled quite a bit with ssh. So much so that I finally emailed the OpenBSD misc list. I never did get a response to my question - I never really expected to seeing as I doubted many people were trying to do what I was and I further figured that they would pass my query off as being that of someone who failed to read the manual. They were wrong... I read the manual over and over but was never able to wrap my head around setting up the rules such that I could pass ssh through after default denying everything (I did confirm that ssh worked without the firewall...) So, I set out to test this through trial-and-error. Finally, after 5 hrs of testing, I came up with:

pass log quick on { $int_if, $ext_if1, $ext_if2 } inet proto tcp from any to \
     any port 22 flags S/SA keep state

This was the culmination of so many combinations of pass etc. I feign to write them all down out of sheer embarrassment. How simple and elegant it ended up being...
There are a number of other ports I ended up opening. There are also a few clients behind the T1 who want their IP's completely open. I was able to accomplish all of this with the following ruleset:

# define the interfaces
lan_net = "24.192.154.1/24"
int_if  = "fxp0"
ext_if1 = "wpachdlc0"
ext_if2 = "wpbchdlc0"
ext_gw1 = "22.109.101.233"
ext_gw2 = "22.222.9.161"

# a few macros
icmp_types = "echoreq"

priv_nets = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }"

## define all of the open hosts
open_hosts = "{ 24.192.154.15, 24.192.154.9 }"

## define some servers
dns_servers = "{ 24.192.154.3, 24.192.154.6 }"
email_servers = "{ 24.192.154.21 }"
http_servers = "{ 24.192.154.11 }"
# scrub all
scrub in all

# default deny
block log all

# pass all on internal loop
pass quick on lo0 all

# block spoofing from external interfaces
block drop in log quick on { $ext_if1, $ext_if2 } from $priv_nets to any
block drop out log quick on { $ext_if1, $ext_if2 } from any to $priv_nets

# allow ssh
pass log quick on { $int_if, $ext_if1, $ext_if2 } inet proto tcp from any to \
    any port 22 flags S/SA keep state

# allow dns
pass quick on { $int_if, $ext_if1, $ext_if2 } inet proto udp from any to \
    $dns_servers port 53

# allow mail
pass quick on { $int_if, $ext_if1, $ext_if2 } inet proto tcp from any to \
    $email_servers port 25 flags S/SA keep state

# allow traffic to httpd servers
pass quick on { $int_if, $ext_if1, $ext_if2 } inet proto tcp from any to \
    $http_servers port 80 flags S/SA keep state

# define the non-firewalled hosts
pass quick on { $int_if, $ext_if1, $ext_if2 } inet proto tcp from any to \
    $open_hosts flags S/SA keep state
pass quick on { $int_if, $ext_if1, $ext_if2 } inet proto udp from any to \
    $open_hosts

# allow icmp
pass inet proto icmp all icmp-type $icmp_types keep state

# pass all outgoing packets on internal interface
pass out on $int_if from any to $lan_net

# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if

# load balance tcp
pass in on $int_if route-to \
    { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
    proto tcp from $lan_net to any flags S/SA modulate state

# load balance udp and icmp
pass in on $int_if route-to \
    { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
    proto { udp, icmp } from $lan_net to any keep state

# general "pass out" rules for external interfaces
pass out on $ext_if1 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if1 proto { udp, icmp } from any to any keep state
pass out on $ext_if2 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if2 proto { udp, icmp } from any to any keep state

# route packets on external interfaces through the appropriate
# gateway
pass out on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 \
     to any
pass out on $ext_if2 route-to ($ext_if1 $ext_gw1) from $ext_if1 \
    to any


I realize that these rules can be redundant and must be rewritten in a much more efficient manner. That is what I am in the middle of working on. I hope, though, that these will help guide you to set the firewall and routing rules for your needs.
Remaining Problems:
One other item that needs to be cleaned up is having the rules loaded after the T1 interfaces are activated at boot time. This process usually is finished 45-60 seconds after the machine is fully booted. This means that if there is a power failure or problem with the machine and it reboots, one has to physically re-set the default gateway and load the rules. I don't understand enough of OpenBSD to make this happen, but when I do, I'll tell you how it is done.

HTH,
Steve

* In between all of this, I had given up on using pf for load balancing because of what I had seen as problems with pf - I was mistaken at the time, but I was convinced the problem was with pf and not with what I was seeing. I unwrapped a brand new Cisco 1721 and two T1 interfaces from a box - my just in case box - and spent eight hours trying to get it working. Fortunately, I was never able to bring up the circuits with the Cisco. I say fortunately because I ended up back with the BSD box and got it working - what I wanted in the beginning.

NLOS Connections on 802.11b a No-Go, Center-Node Change

|

     In the continuing saga of building out the Darien WiFi project, I have spent the past two weeks learning a very, very important lesson. 802.11b (aka 2.4-2.5 GHz ISM band communications) does not like trees. While surveying for good antenna locations on some of the end-nodes, I did not do anything to take into account natural foliage. We erected the center-node almost 3 weeks ago and at that time, most of the trees were completely bare. So, when we went around to do signal testing, we were able to visually ignore the fact that a tree obstructed a given site or location at the end-node. Over the past few weeks, however, the weather has gotten warmer and the trees are starting to do their spring dance. I thought that some leaves wouldn't be much of a problem considering the strength of the signal and the distance from the center-node. Unfortunately, I was proven wrong. Over the past two weeks, I have seen the signal degrade 25% a week until this week it went well below acceptable long-term levels - I can't imagine what would have happened to the signal when the trees were in full bloom. With the generous help of a member of the BAWUG mailing list, I have learned quite a few things about signal propagation/attenuation and natural interference. I am thankful that I am running into these problems now and not later. It could have been completely disastrous for a few of the end-nodes, as I had considered trees to be small problems and not big ones. So, this weekend will be used for moving the antenna for a third time on the end-node in question. This time, we will be mounting the antenna to a tripod on the roof. I must say that I hate roofs, but this roof installation will be a necessary evil.
     On top of that small lesson learned, I also found how harmful antenna movement can be to the system in general. The original center-node antenna installation was less than adequate. It consisted of a 1 1/4" OD 10 ft/3 m antenna mast attached to a railing on the top of a silo. Well, I learned how flexible a 10 ft/3 m 1 1/4" mast could be. Movement (depending on the wind conditions) were on the magnitude of 2-8" (5-20 cm)! Considering the wavelength of 2.4GHz being 12 cm, this made reception/transmissions horrible when it was windy. With the continuing generous help of my friend, we went up on the tower on Sunday and replaced the mast with a larger OD mast supported with guy-wires. What a difference! Signal levels on the feed and one of the end-nodes increased by almost 25%. I'm sure that with better aiming of the end-node and feed antennas, we could increase the signal effectiveness even further. We still have to replace the feed antenna mount, but for now, we have seen significant improvements in network performance.
Here are some other short observations:
1) While Proxim's Tsunami gear is great, they would do themselves some good by designing a mobile receiver for signal testing. It is a severe pain to try to work out of the back of my truck and stretch out LMR400 cable in every which direction to see where I can get the best signal. Something that I could plug into my laptop would be helpful, thank you very much...
2) When planning the network build-out, plan, plan, test, plan and test some more. Natural obstructions are just as bad as buildings themselves.
3) When budgeting what you are going to spend, multiply it by 3 unless your time is abundant. Mine isn't and many of the work-arounds that I have had to implement have cost a lot of money.
4) And as my father has always said, do it right the first time...

     After the adventures of Saturday, I have been able to take some time to learn more about tweaking the Feed and the Center. Located at the Feed, I have a Proxim MP.11 Subscriber Unit and at the center, I have a MP.11 Base Station (also known as an SU and BSU). Most of the clients will end up using a Residential Subscriber Unit (RSU) because of a) cost and b) lack of need for the one feature that differentiates it from the SU - unlimited MAC address association. The Center is broadcasting using a 15.5 dBi omni directional antenna about 55 ft in the air and the Feed is transmitting using a 15 dBi directional patch antenna (with a beam width of about 30 deg.) located about 40 ft. off the ground. One of the potential clients of the system asked about network latency... I hadn't actually put much thought into it until the question was asked! How will the environmental and setup factors effect network latency? This is especially important as the system grows. If I have drastic latency issues right now, I can only imagine them getting worse as more users come online.
     To begin simple latency tests, I took a workstation behind the feed-node and pinged the center-node. The results look a lot like:

bash-2.05b$ ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=157.648 ms
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=72.399 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=24.191 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=5.961 ms
64 bytes from 10.0.0.1: icmp_seq=4 ttl=64 time=6.612 ms
64 bytes from 10.0.0.1: icmp_seq=5 ttl=64 time=6.525 ms
64 bytes from 10.0.0.1: icmp_seq=6 ttl=64 time=6.662 ms
64 bytes from 10.0.0.1: icmp_seq=7 ttl=64 time=14.702 ms
64 bytes from 10.0.0.1: icmp_seq=8 ttl=64 time=6.383 ms
64 bytes from 10.0.0.1: icmp_seq=9 ttl=64 time=6.441 ms
64 bytes from 10.0.0.1: icmp_seq=10 ttl=64 time=64.546 ms
64 bytes from 10.0.0.1: icmp_seq=11 ttl=64 time=16.346 ms
64 bytes from 10.0.0.1: icmp_seq=12 ttl=64 time=6.614 ms
64 bytes from 10.0.0.1: icmp_seq=13 ttl=64 time=6.834 ms
64 bytes from 10.0.0.1: icmp_seq=14 ttl=64 time=6.490 ms
64 bytes from 10.0.0.1: icmp_seq=15 ttl=64 time=110.844 ms
64 bytes from 10.0.0.1: icmp_seq=16 ttl=64 time=27.913 ms
64 bytes from 10.0.0.1: icmp_seq=17 ttl=64 time=6.147 ms
64 bytes from 10.0.0.1: icmp_seq=18 ttl=64 time=167.393 ms
64 bytes from 10.0.0.1: icmp_seq=19 ttl=64 time=84.476 ms
64 bytes from 10.0.0.1: icmp_seq=20 ttl=64 time=162.557 ms
64 bytes from 10.0.0.1: icmp_seq=21 ttl=64 time=79.655 ms
64 bytes from 10.0.0.1: icmp_seq=22 ttl=64 time=31.363 ms
64 bytes from 10.0.0.1: icmp_seq=23 ttl=64 time=69.290 ms
64 bytes from 10.0.0.1: icmp_seq=24 ttl=64 time=37.645 ms
64 bytes from 10.0.0.1: icmp_seq=25 ttl=64 time=6.147 ms
64 bytes from 10.0.0.1: icmp_seq=26 ttl=64 time=6.734 ms
64 bytes from 10.0.0.1: icmp_seq=27 ttl=64 time=6.072 ms
64 bytes from 10.0.0.1: icmp_seq=28 ttl=64 time=6.550 ms
64 bytes from 10.0.0.1: icmp_seq=29 ttl=64 time=15.885 ms
^C
--- 10.0.0.1 ping statistics ---
30 packets transmitted, 30 packets received, 0% packet loss
round-trip min/avg/max/stddev = 5.961/40.901/167.393/49.723 ms

As you can see, there may be wild variances in the round-trip time, but for the most part, the feed to node round-trip time is around 6ms. At first I was quite a bit concerned about the fluctuations - in fact, I still am concerned, but it is a reality of the OS on the equipment and the environment. Another thing that one should notice is that it will maintain a round-trip time of 6ms for a few pings and then jump to some double or triple digit number. In my estimation, this is partly due to flex of both of the antennas in the wind and reflection issues. One thing I have been reading about is background noise when using an omni. Eventually, if we got over 20 subscribers (which would mean that we would be setting up a ResNet - and is not the current goal of the project), I would consider sectorizing the center-node (i.e. breaking up the omnidirectional center-node and puting in directional patch antennas that cover a 60-120 deg. area instead of 360 deg.). On top of the environmental issues, a feature of the MP.11 series is SU/RSU registration polling periods. It appears that every 5 seconds, the SU/RSU re-registers with the BSU - which would be good reason for latency spiking at what appears to be 5 second intervals. I ran a ping test over night on Saturday and found that the cycle of jumps from 6ms to >6ms was extremely regular - again, every 3rd-7th ping.
     Another setting I toyed with was overall throughput settings - i.e. 11 Mbits/s vs. 5 Mbits/s, etc. I have found on other wireless systems that although a higher bitrate connection can be achieved, the signal strength is low enough that throughput at that bitrate is worse than if we were to simply go to a lower bitrate setting. I played with different combinations and found that the signal between feed and center is good enough that leaving it at 11 Mbits/s doesn't give me worse round-trip ping times than setting the BSU or SU/RSU at a lower bitrate. This may change, however, with the introduction of more SU/BSU's. Luckily, though, as long as the signal is decent, changing these settings won't impact the latency to any great extent.
     The literature is very vague concerning how to reduce latency issues. I think that the first objective of any project is to provide the access and then move on to improving network performance. I am surprised, however, that not much effort seems to be put into this issue - I don't know whether it is because of the apparent connection between signal strength ad latency (it appears that as the signal gets worse, the latency goes up - no surprise there - however, what is the threshold?) or whether it is simply a forgotten issue. I will continue to tackle the problem, however, as I add more end-nodes and see what system parameter changes I can make that will help reduce over-all system related latency.

Anywhere Technology: Center-Node & Feed Construction

| | Comments (0)

    Well, the project has begun and I feel good. We have a link up between the supply facility and the center-point. I had the fantastic help of an old, old friend. I plan on taking a few pictures to post tomorrow. One thing I debated was whether I would set up a detailed map of the node locations, but due to the concerns about privacy of the end-users and worries about node hijacking (something extremely remote, but possible), I am going to simply take pictures of the antenna installations. The center-point antenna was the craziest installation I have ever done - at one point, my buddy was riding the silo like the guy from Dr. Strangelove who rides the bomb to the end yelling yee-haa the whole way... And that was one of the more tame moments of us trying to get the antenna up while battling the 25-30 mi/hr winds (plus my insane fear of heights).
    I had been watching the weather all week to see whether it would be possible to do the work today or not. Yesterday would have been much more ideal, but at least it didn't rain today (other than 10 minutes around 7:30 after we had just gotten on top of the center-point). The center-point is at the top of a silo that supplies one of our facilities with raw materials for our product. I had the choice of five locations and debated the pros and cons of each by some objective and subjective guidelines:
1) Which center-point gives the greatest range of line-of-site to potential end-nodes.
2) Which center-point will have the most reliable source of power and be the easiest to maintain if something happens.
3) Which center-point will have the potential to be a bridge to other center-points - i.e. which location offers the best potential for continued growth. If the system were to grow to multiple center-points/nodes, which one would provide the most useful center-point?
4) Trying to minimize latency on a wireless network can be difficult due to reflection from natural and man-made objects. I tried to look at the surrounding buildings as reflectors - like multiple stones thrown into a pond and the ripple effects that collide. If my understanding of rf reflection is correct, then reflection can also be to your advantage. The problem is that predicting reflection versus absorption is practically impossible - at least I don't have the training to do so - nor do I have the understanding to do the calculations quickly. Plus, due to the varying heights of the buildings and different roof pitches, it would be even more difficult to figure out where the signal can and cannot go without simply relying on a line of site model.
    The silo we ended up mounting to has turned out to be better than I thought. We were able to reach some sites that I thought would be obstructed by other buildings. After finalizing the mount of the center-node antenna, we started work on the feed. The original plan was to use a tripod mount on a portion of the roof that had a 1-2 deg. pitch. Ironically, the wind that caused terror up until then, ended up becoming our saving grace - at least in the future we would have learned this to be a saving grace. Trying to mount a 10 ft. mast to an relatively small tripod base (without being able to screw it to the roof - holes in roofs eventually leak...) turned out to be all but impossible - the wind whipped around the base even with 25 lbs. weights on each of the feet. We ended up having to use a side mount method whereby mounts are attached to the side of the building. We were able to mount a 20 ft. mast using a 5 ft. standoff between the two mounts. The antenna mast ended up flexing quite a bit in the wind, but the flex proved to be a means by which neither the antenna nor mast were damaged by the wind. I will eventually replace the directional patch at the top of the mast with a mini grid antenna so that wind is less of a factor. The beam width and antenna sensitivity differences are negligible, so weather is really the issue.
    After testing the link between feed and center-node, we cleaned up and put power/surge protection in place (on the base stations and between the center-node antenna and the base station). So, the link is up and running... 8 Hrs of work and we have a functioning core. Tomorrow I'll do so more signal tests from different points and see if we need to make any further adjustments to the center-node. From here, we will start installing end-nodes and testing over-all network performance.

The Latest Challenge: Darien Wireless Co-Op

| | Comments (0)

    It started out as an idea that had been haunting me over the past few years, but now it just might become a reality. You see, we live out in the sticks. Out in the country. Out in the middle of nowhere. After working for Hillsdale College and having gotten used to the raw power of a T1 and a host of servers to play with, I had a somewhat difficult time adjusting myself back to country life (a little over three years ago, my wife and I moved back to my home town so that I could start working with our family businesses). After playing with three phone lines and multiple dial-up accounts, then testing the fledgling two-way satellite internet service, I came across (a now defunct company), Dakota DSL. They ended up changing their name, but its been so long and the service was so short lived that I can't even remember anymore. I polished up some old workstations I had accumulated over the years and others gotten off of eBay and put Windows NT Server on two machines and started my own little webhosting operation. It was costing me a fortune, but still better than anything I had seen since moving back to Wisconsin. At the time, Aaron and I did a little webhosting off of the servers for both of our domains. It didn't become anything more than two pokey websites because about three months into it, we got hacked... We were hacked pretty badly, too. So, I researched around and around and around and realized that upgrading to newer versions of Windows NT was going to be too costly, so I decided to learn what I needed to do to get our services running on some form of Linux or Unix. By that time, Aaron had split - found a better and more reliable host - and I was convinced I could do webhosting better and cheaper than anyone else. I took about three more months and learned the basics of FreeBSD and got apache, qmail, php4, mysql and a host of other services running on my first FreeBSD box. I also learned that I couldn't do webhosting better and cheaper than anyone else because 1) I had a real job and 2) the box that was hanging off the net was 15 minutes from work - so any time there were troubles, there were troubles. It suited my needs, but I needed stronger, faster and more of everything in order to make the venture work. Then Dakota went bankrupt and I lost my super fast connection. Then, we also got kicked out of the house we were living in (not really, but the owners wanted to move back in and we had to look for a new place to rent or buy). So, I shut my website and hosting service (of one) down and took a 4 month hiatus.
    Between then and now, I have been on Charter in one form or another and have amassed a server farm that makes my head swim (when I realize the money I have sunk into my addiction). I had, at one time, a neighborhood wireless network running - which is what got me started on this topic in the first place - whereby I supplied some of my neighbor/friends with access to my mega-pipe to the net and they helped defray the cost of my service. My wife and I ended up moving, yet again, to a place where I think we'll stay put for some time to come. We also moved our offices and are closer to where I live and are in a business park - whereas before, we were literally in the middle of a very large corn/soy bean field.
    If you are still reading by now, you are either entranced by my writing, you are on the loo with your wireless laptop and having nothing better to read or you are trying to sleep - BUT, I'm actually getting to a point. I'm starting up another wireless project. This time, however, my project has no grand goals other than covering my @ss on bandwidth expenses. I don't want to be the worst and if you graded what I want to do solely on price, I can't be the best - but I can give you the best experience for your dollar and help us help each other. If you are somewhere near this area (click on the link and you will see an amazing little satellite photo), then there is a good chance I can get you online. Sure, there is competition (GenevaOnline and ElkNet - and a few other dial-up entities), but I think we can beat them in every way - price, service and accessibility. But, we can only do it by pooling our resources. I found some great equipment by Proxim and some people who seem genuinely interested in the project. I hope to use this site to document my progress. I have a few goals in mind and would like to put up a chronology of what happens:
1) Get the center base station up and running this weekend.
2) Get final commitment from other users (hopefully 10 of us will sign up).
3) Based upon final commitment, I will put in my T1 orders.
4) Get a few subscribers up and running next week.
5) Keep those subscribers up and running...
6) Get the rest of the subscribers up and running.
7) Keep the rest of the subscribers up and running...
8) Monitor bandwidth to make sure we are utilizing it to the best of our abilities.
9) Put in redundant nodes in case the main node goes down.
10) ...I'm thinking residential on a different scale than approached by ISP's before. Fully private addressing scheme, SPAM filters on all email addy's and nodes supplied by private DSL lines all over the city and countryside. P2P sharing limited to 256kbps of overall bandwidth... period. A ResNet for Darien, built in much the same fashion that the Co-Op is designed to run - a few volunteers and a lot of users.

We'll see how it goes...

About this Archive

This page is a archive of recent entries in the Darien-WiFi (AnywhereTechnology) category.

Find recent content on the main index or look in the archives to find all content.

Contact

Steven N. Fettig
Delavan, WI - somewhere between Delavan & Darien: map link
Phone: +1 262 432 1704
Email: snfettig AT gmail.com
AIM/Yahoo/MSN/GoogleTalk-
Skype/twitter:
snfettig

Technorati

Technorati search

» Blogs that link here

Powered by Movable Type 4.21-en