Load a month of Bluebikes (the Boston metro bikeshare) trip data and flag stations that probably had a broken dock. A station looks broken when no bikes were returned during an hour where the same station typically receives several. The flag is a Bayesian probability combining a prior on dock failures with a Poisson model of expected returns.
YYYYMM-bluebikes-tripdata.zip files here, or
hubway-data S3 bucket). The S3 bucket blocks browser
fetches, so files have to be downloaded manually and then dropped back in here.
Each (station, day-of-week, hour-of-day) cell gets its own
baseline rate λ. To compute it for a query date, take returns at
the same station and same hour-of-day on every other loaded date that fell on the
same day-of-week, then average. A month of trip data typically contributes 4 or 5
dates per day-of-week; the queried date is excluded (leave-one-out) so a long outage can't drag its own baseline toward zero. If only one date of
that day-of-week is loaded there's no out-of-sample evidence and the cell is
ineligible. Returns within a single hour are modeled as a
Poisson
process, so the probability of zero returns when the dock is working is
e−λ.
Let π be the prior probability that any one hour at any one station is
"broken" (default 1%, slide to taste). When the dock is broken, returns are forced
to zero. Bayes then gives:
Increasing the time window pools observed counts and expected counts across ±N
hours; the same formula applies to the summed λ. Near hour 0 or 23 the
window is clipped (no wrap), so edge hours pool fewer hours and need a
proportionally higher baseline to clear the cutoff. Cells where the summed λ is
below the cutoff are marked
ineligible
because you can't distinguish a broken dock from a routinely-quiet hour.
Known gaps. A station broken for the entire loaded window won't appear at all (no trip records means no coordinate and no baseline to compare against). Trip timestamps are local time and daylight-saving isn't accounted for: spring-forward leaves one DOW-hour cell with no observations and fall-back double-counts another, each by one date per year.