Printerr-friendly PDF file

Flaws-in-the-Glass Snake Oil:  SFMTA’s October 5, 2015 Analysis
SFMTA’s Premature Evaluation Report of Its
“Commuter Shuttle Pilot Program”

by Patrick Monette-Shaw
November 16, 2015; and Updated November 27, 2015


Why SFMTA released its Evaluation Report nearly four months prematurely analyzing its Commuter Shuttle Pilot Program on October 5, 2015 — when the trial period is set to run through the end of January 2016 — isn’t known, but it’s thought the premature analysis was rushed into print so SFMTA’s Board could vote on November 17 to make the program permanent, even before the end of the 18-month pilot trial period.

Sadly, SFMTA’s October 5 report is riddled with flaws.

Flawed Data Collection Methodology

On page 5 of the report, Muni (“Muni” is used interchangeably in this secondary analysis to refer to the SFMTA) claims that the Pilot Program allowed it to collect data regarding the movement and usage of commuter shuttle buses; further claimed that based on the data collected, the October 5 report evaluated how the Pilot Program had performed on its objectives; and also claimed the Evaluation Report would be used as a basis to make recommendations as to whether the program should be continued, making it permanent.

One of the key objectives of the Pilot Program was to gather data regarding shuttle activity in the City. Given the paucity of data collected from shuttle bus operators, the objective to adequately collect reliable data appears to have failed, miserably.

And given that the data that was collected was wholly incomplete, the premature Evaluation Report should not be used as the basis for making the shuttle bus program permanent.

It is not until page 34 of the Evaluation Report that Muni admits the severity of gaps in the data collected from shuttle operators.

The report notes on page 34 that it had seemed to be a straightforward requirement at the outset of the program to require shuttle operators to provide real-time data on shuttle stop-events and vehicle movements, but it proved to be “more complicated that originally contemplated.” Jeesh! Collecting GPS data is now “complicated”? In 2015? Really?

Maybe GPS tech whizzes at Google could volunteer their skills and jump in to help the Google-bus shuttle operators uncomplicate the collection and submission of GPS data.

For that matter, the City has seventy (70) 1042 “IS Engineer-Journey” employees on the City’s payroll, with combined total pay of $7.8 million, many of whom are GPS experts. Why can’t they be pressed into service to get the commuter shuttle bus GPS flowing accurately to SFMTA?

Shuttle Operators Fail to Provide Data

Despite a requirement to provide GPS data regarding real-time location and stop-events of each shuttle bus in order to receive a permit, damning admissions in the report noted on page 34 indicate the failure of the shuttle operators to provide the required GPS data, even though shuttle providers were required to reapply for all permits by February 2015 six months into the project, but may have received renewed permits, despite being GPS non-compliant:

Muni appears to have allowed shuttle operators to continue in the Pilot Program even though they weren’t complying with the GPS requirement to obtain a permit, and when the operators had to reapply six months into the program, Muni appears to have allowed operators to obtain renewed permits, despite not complying with the GPS requirement.

Muni claims on page 34 that it is now undertaking processes to bring the data collection and reporting in-house to eliminate vendor issues, but didn’t indicate how long this will take, or when ti might be fully implemented.

But Muni offered no information on why Muni buses — the majority of which are thought to already have GPS capabilities — hadn’t been submitting reports of conflicts in Muni bus zones themselves through Muni’s on-board camera systems that are capable of recording video of traffic in front of them violating double-parking regulations.

Brazenly, public records show that on August 8, 2014, LinkedIn Corporation’s Global Transportation Program Manager, Michael Alba, e-mailed SFMTA Shuttle Program manager Carli Paine indicating LinkedIn wanted to implement a new Caltrain shuttle between the Financial District and a new site in SoMa within a week, but LinkedIn’s GPS provider had a two-month backlog.

When Alba asked Paine if he could submit “a formal request for leniency on the GPS requirement” to obtain a permit, Paine clearly and thoughtfully replied saying the GPS requirement “is not a negotiable element of the program,” and suggested LinkedIn “look into other GPS providers so that you can meet the permit terms.” But it’s not known whether LinkedIn quickly found a new GPS provider, or whether LinkedIn simply began using the shuttle without GPS capability, or without a permit.

Flawed Data Collection Methodology:  Multiple Sampling Biases

The Evaluation Report appears to have several problems with “sampling bias.” First, on page 18 of the report, Muni admits it conducted field data collection only in June 2014 (before the pilot Program began in August 2014), and June 2015 to examine the impact of the program on traffic conflicts and potential safety issues.

Nowhere in the report does it indicate on how many days in June 2014 and June 2015 field data collected actually occurred on a daily basis. And the report fails to mention why field data collection occurred in just one (June 2015) of the first 11 months (between August 2014 and June 2015) of the 18-month study. Why wasn’t data collected between July and October 2015, or was it collected and not reported?

First, of the 124 bus zones in the Pilot Program, Muni surveyed just 20 locations (16.1%), 10 each during the morning and evening commute periods, which drops to just 8.1% of locations during each commute period. This may not have been a very representative sampling of the 124 bus zones.

Second, of the 20 stops used for data collection, 7 of them — 35% — were in “shuttle-only” zones that Muni buses weren’t allowed to use, skewing how many blocked Muni buses there were per hour. Theoretically, you shouldn’t expect any blocked Muni buses in shuttle-only zones (although that happened at one of the 7 shuttle-only zones), but instead of sampling 20 Muni zones, SFMTA allowed the co-mingling of data from 35% (7) of the 20 zones reserved for shuttles only.

Third, Muni noted on page 28 that the Pilot Program included funding for a l0-person morning and evening enforcement team known as the “shuttle detail,” and then acknowledged on the next page that there was limited staffing for the shuttle detail.

This is ludicrous: Extrapolating from the 2,978 daily stop-events made by the shuttle buses times $3.67 per bus per stop-event, Muni received $2.94 million on an annualized basis from fees for stop-events alone (based on about 269 week days annually). And it could only fund a 10-person enforcement team from the stop-event fees?

And what about the 1,200 citations issued to shuttle buses by all of Muni’s enforcement officers (not just by the 10-person shuttle detail assigned to enforcement) between August 2014 and the end of May 2015 — ostensibly at only the 20 zones monitored during the pilot period for the 124 zones? Didn’t those citations generate any revenue in fines from the citations to hire additional shuttle detail enforcement staff? For that matter, how many additional citations — and resulting increase in revenue from fines — would a much larger shuttle-detail enforcement team have issued had the other 104 zones not even surveyed been routinely monitored for enforcement?

Footnote 13 on page 30 of the Evaluation Report noted that SFMTA can shift enforcement staffing “by using shuttle bus GPS data to determine where enforcement is needed most.” An obvious, but unanswered, question is why Muni can’t issue more citations just from the GPS data it is collecting. Since FasTrak mails citations to drivers crossing Bay Area Bridges when transponders fail to collect the bridge tolls, why can’t MTA issue citations based on GPS data?

For that matter, Muni mentioned not one word in its Evaluation Report about why Muni Transit Operators (bus drivers) weren’t enlisted to collect and report bus-zone conflicts using their on-board cameras, or merely using a clipboard to record the number of times drivers had to lower wheelchair mechanical lifts directly into the roadway for disabled passengers when shuttle buses blocked Muni bus access to red zones daily between August 2014 and May 2015.

Almost comically, Muni reported on page 29 that the primary goal of the shuttle detail was not to issue citations, but to keep zones safe and traffic flowing smoothly. Then the report says the number of citations issued “is not necessarily instructive of whether the Pilot Program’s goals were met through enforcement actions.”

The goals of the pilot program included reducing conflicts with Muni and other vehicles, and improving safety interactions with other users. It sounds to this observer that an average of 103 citations monthly points to a glaring problem with both safety issues, Muni bus conflicts, and other traffic issues.

Did Muni think readers wouldn’t spot skewing of data from its data collection sampling biases?

Garbage In, Garbage Out

Given the biases Muni introduced gathering data to assess the effectiveness of the Commuter Shuttle Pilot Project, is it any wonder that much of the data presented in the report is suspect, when not apparent lies?

For instance:

Apparently Muni may never have heard the phrase “Garbage In, Garbage Out,” which seems to be the most glaring problem with its premature Evaluation Report.

There’s plenty of flaws-in-the-glass snake oil throughout this Evaluation Report, but no explanation or presentation of any data that may have been collected between June 2015 and November 2015. Are we to believe no data was collected during the intervening four months since June 2015 because Muni has been on hiatus?

Lawsuit Reached Trial in Superior Court

The lawsuit filed by the Coalition for Fair, Legal, and Environmental Transit on May 1, 2014 was finally heard by Superior Court Judge Garrett Wong on Friday, November 13, 2015.

Courtroom observers, including this author, were rather startled when Judge Wong asked at the outset whether if he ruled on the so-called “pre-emption” argument in the case, whether he then needed to rule on the CEQA environmental review aspect of the lawsuit. His question was seen by some observers to be a good sign that he may have concerns about the “pre-emption” issue, which involves whether local ordinances can “trump” State law, which of course,
they can’t.

As plaintiff’s in the lawsuit noted, California Vehicle Code §22500(i) stipulates:

“No person shall stop, park, or leave standing any vehicle in” … or “alongside curb space authorized for the loading and unloading of passengers of a bus engaged as a common carrier in local transportation when indicated by a sign or red paint on the curb erected or painted by local authorities pursuant to an ordinance.”

Common carriers are individuals or businesses that advertise to the public that they are available for hire to transport people in exchange for a fee. Obviously, the tech commuter shuttle buses are not common carriers, and have no business stopping in bus zones deemed by local ordinance.

So it’s quite clear that California’s vehicle code pre-empts the City ordinance that authorized the shuttle pilot program to have begun in August 2014, and it appears Judge Wong understands this.

For that matter, public records show that as far back as October 4, 2011, SFMTA program manager Carli Paine had e-mailed SFPD Captain Al Casciato and two other SFPD staff providing the citation and full text of Vehicle Code §22500 to Casciato, and using yellow highlighting on §22500(i).

In other words, SFMTA knew three years before rolling out the shuttle project in August 2014 that not only were the commuter shuttles not common carriers, SFMTA knew State law preempts Muni from allowing the commuter shuttle buses to use Muni red zones. More brazenly, when San Francisco’s Board of Supervisors denied a CEQA review of the Shuttle Pilot Program, they also had to have been aware a local ordinance cannot preempt or “trump” a State law.

Shockingly, at the conclusion of the Superior Court trial on November 13, a Mr. Carr, a lawyer for Genentech††, stupidly asserted after the period for taking all evidence in the case had officially ended, that the plaintiffs have “no beneficial interest” and “no public interest exception” — no standing — for bringing their lawsuit to Superior Court, in another brazen attempt to seek summary judgment, asking Judge Wong dismiss the lawsuit altogether.

Plaintiff’s attorney Richard Drury rebutted Carr’s nonsense to Judge Wong and did so very competently, asserting that the plaintiff’s First Amended Verified petition had provided more than sufficient justification of “standing” to have filed the case, so hopefully Judge Wong will accept Drury’s line of rebuttal.

Carr’s tactic of waiting to announce the “standing” issue until after the period for submitting evidence in the trial had ended amounted to a legal trick called “sandbagging,” which involves trial lawyers noticing, but not mentioning during a trial, potential errors in hopes of using it during a subsequent appeal. “Sandbagging” usually doesn’t keep an issue alive, and Trial Judges typically don’t like it when counsel on either side attempts to sandbag the process by not pointing out potential errors in a timely manner. Carr should have brought this issue up before the period for submitting evidence in the trial had ended.

Leave it to Genentech and the City of San Francisco to pull such a brazen stunt! Hopefully, Judge Wong will not only uphold the plaintiff’s standing, he’ll rule that the pre-emption provision has merit separately from the CEQA aspect of the lawsuit.

Fast-Tracking Permanent Shuttle Program Using Flawed Data

As I noted in my extended article, “Tech Buses” Drive Displacement located at, in response to public records requests Sue Vaughan had placed, e-mail records received reveal City planners repeatedly referred to making the program permanent on a “tight schedule,” and that in an effort of adhere to the tight schedule, some data of the program were made on an “educated guess” basis by planners, rather than relying on real data. SFMTA claimed it disagreed that the program was being “fast tracked.”

Indeed, during SFMTA’s CAC (Citizens’ Advisory Council) meeting on November 5, Ms. Vaughan introduced a motion bravely recommending that the commuter shuttle buses be removed from Muni stops. Unfortunately, her motion failed to pass — with just one vote shy of passage. One vote!

Sounds to me like there’s a Citizen on this Advisory Council who may not have my — and other Muni passenger’s — best interests at heart.

Neither SFMTA’s Evaluation Report nor the Planning Department’s October 22 new EIR exemption letter for making the shuttle program permanent mention there is a distinct probability the shuttle program will face no upward limits, and there may be unlimited expansion of the shuttle program in the future.

Given the many flaws and gaps in reliable data in the premature Evaluation Report, SFMTA’s Board of Directors should reject making the shuttle program permanent.

Monette-Shaw is an open-government accountability advocate, a patient advocate, and a member of California’s First Amendment Coalition. He received the Society of Professional Journalists-Northern California Chapter’s James Madison Freedom of Information Award in the Advocacy category in March 2012.   Feedback: monette-shaw@westsideobserver.

  Memo - Potential Increase in Commuter Shuttle Activity, from Hank Willson (SFMTA) to Melinda Hue (SF Planning Department), dated October 8, 2015.
†† Genentech and several other tech companies, including Google and Apple, had initially been named as defendants in the lawsuit, but fought bitterly to be removed, and were. Later, the tech companies reapplied to become "intervenors" in the lawsuit; intervenors are third parties who were not originally a party to a lawsuit who become a party if approved by the Court. Why Genentech first fought to get out, and then later rejoined, isn't known. Perhaps Genentech concluded San Francisco Deputy City Attorney Audrey Pearson was handling the defense badly.