Uptown Services report finally appears
Posted by Russell Senior on 26 May 2007On Friday, May 25th, sometime after I sent out a press release which included a note that the Uptown Services report had not yet been published, the City published the Uptown Services report. You can download it here: http://www.portlandonline.com/shared/cfm/image.cfm?id=157216.
Caleb Phillips and I have been awaiting its release to try to understand how they reached such different conclusions from ours on the question of outdoor coverage.
Recall that we found, in a random sample, that we succeeded in achieving a connection in 31 of 53 tested locations, a rate of 58%. Further, we calculated the probability that that rate of failures would occur by chance if MetroFi was meeting the 90% coverage standard at about 2 in a billion. In contrast, Uptown Services reported in its Executive Summary that MetroFi provided outdoor coverage in 95% of the POC area.
Having had a chance to look over the Uptown Services report, it looks like there is a plausible model to explain the discrepancy. Here it is:
- MetroFi tells the City of Portland that they should expect a connection if the received signal strength is -79 dBm or greater based on some testing MetroFi did (City Project Manager Logan Kleier: “MetroFi has found that -79 dBm is the minimum necessary signal strength to achieve a 1 Mbps modulation rate based on the utilization of a 30 mW wireless client, such as an Intel 2915ABG, in non-line of sight conditions at 300 feet from an access point.”)
- The City of Portland tells Uptown Services that they should use the -79 dBm received signal strength as a surrogate for a “connection”.
- Uptown Services does their drive testing with a Proxim Orinoco Gold 60 mW radio and a 5.5 dBi roof-mounted omni-directional antenna.
- Uptown Services performs their proprietary analysis, which involves creating a grid of 100-foot squares, averaging the received signal strength values found within the squares, and then counting the proportion of squares within 500 feet of an access point with an average signal strength of -79 dBm or greater, and finds that about 95% of them do. Since the contractual standard is 90%, Uptown Services finds that MetroFi passes that standard.
There are several problems here:
- The ability to get a connection is largely determined by the transmit power and antenna gain of the client device. This is because network communication requires two-way communication, both that the client hears the access point and the access point hears the client. The access point is loud (a 400 mW or more transmit power and a 7.5 dBi antenna), very near the maximum allowed by the FCC. Client devices can hear the access point from potentially great distances. The limiting factor is whether the access point can hear the client device from those great distances.
- The original Unwire Portland RFP (Sections 2.2.1.2 and 2.2.1.5) indicates that the network should support coverage for common client devices, builtin as well as plugin PCMCIA and USB radios. These typically have 30 mW radios and low-gain antennas (2.2 dBi or less).
- It is not at all clear that the MetroFi-supplied -79 dBm received signal level is relevant to these common client devices.
- Logan Kleier claims that the Intel 2915ABG radio is a 30 mW device, but it is not clear that the FCC disclosures agree (see page 6 of 91). There are values reported there than imply that the peak transmit power is as high as 250 mW. If true, that would amount to a discrepancy of 9 dB in MetroFi’s favor.
- The City of Portland relies on the -79 dBm received signal level without, apparently, verifying it in any way. There is no indication in the Uptown Services report that they tried to verify it. Logan has not indicated that the City has verified it.
- Uptown Services performs their drive testing using a radio with a 5.5 dBi antenna gain. Most client devices do not have such a high gain antenna. Compared to a typical client device, it is at least 3 dB optimistic. Therefore, even if the MetroFi -79 dBm standard is accurate, the cut point that Uptown Services should have used to compensate for their antenna should have been no lower than -76 dBm. Unfortunately, the level of detail necessary for a reader to compensate for this is not available in the Uptown Services report.
- It is not clear from the report (at least as of my initial reading) how the 100-foot square grid cells were aligned, and how much of the cell needed to be within 500 feet in order to “count” in the calculation.
Another item of note, page 13 of the Uptown Services report indicates that the MetroFi network failed the extended availability test, providing a connection in a fixed location for only 91% of the time over the course of a month-long test, less than the 99% required.
As Caleb and I, and perhaps you dear reader, are able to examine the Uptown Services report more closely in the coming days, additional notable issues may be discovered.
Results Complete, Report Finished
Posted by Caleb Phillips on 23 May 2007We are proud to announce the final version and full public release of our report on the Unwire-Portland Proof-of-Concept network. You can find it on the results page along with all of our data, source code, and configuration. In addition to being a revealing analysis of the network deployment in Portland, we hope that our approach and experiences testing the network can be a useful example for other burgeoning municipal wireless networks.
Of course, we’re very interested to hear any questions, comments or well-founded criticism.
Report Draft goes to Review
Posted by Caleb Phillips on 3 May 2007Just a quick update. Russell and I have completed a full draft of the report and have sent it to several people for review. We are lucky to have had several very qualified individuals offer to review the report. Once the review process is complete, and we have responded to their criticisms, we will publish the report publicly here.
Incidentally the full report from Uptown Services is still either unfinished, or has not been made public. We are very interested to see the report in full, with hopes that it justifies some of the questionable methodologies mentioned in the Executive Summary.