From bd61cd042c1be59e59d9af2d5f6fb2d1f61aa423 Mon Sep 17 00:00:00 2001 From: zemion Date: Mon, 6 Jul 2026 14:41:40 +0200 Subject: [PATCH] Sync Repo-docs-04-data-sources from project files --- Repo-docs-04-data-sources.-.md | 131 +++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 Repo-docs-04-data-sources.-.md diff --git a/Repo-docs-04-data-sources.-.md b/Repo-docs-04-data-sources.-.md new file mode 100644 index 0000000..91d1e94 --- /dev/null +++ b/Repo-docs-04-data-sources.-.md @@ -0,0 +1,131 @@ + + +> Mirrored from `/mnt/DATA/git/pikebacker/docs/04_data_sources.md`. +> Origin: `repository`. +> Active tasks and changing state belong in Gitea issues; this wiki page is durable project context. + +--- +# Data Sources + +Verified source links are listed in `docs/source_references.md`. + +## Core principle + +The raw route network can be built from OpenStreetMap. The competitive advantage comes from derived intelligence: + +- route segment suitability, +- loaded-bike rideability, +- route-corridor logistics, +- water/food/sleep reliability, +- local rules confidence, +- trusted local reports, +- actual ride feedback. + +## Data-source matrix + +| Domain | Source candidates | MVP use | Production use | Caveats | +|---|---|---|---|---| +| Base map/network | OpenStreetMap extracts via Geofabrik or planet.osm | Road/path graph, surfaces, access, POIs | Full regional/global ingestion | ODbL obligations; data quality varies | +| Routing engine | openrouteservice, GraphHopper, Valhalla, BRouter | Hosted API or local engine adapter | Self-hosted customized profiles | Requires tuning; public APIs have limits | +| Elevation | Copernicus DEM GLO-30/GLO-90, NASA SRTM | Elevation profile and grade warnings | Segment-level grade/hike-a-bike modeling | DEM artifacts; bridges/tunnels need handling | +| Vector maps | OpenMapTiles, Protomaps/PMTiles, MapTiler, custom tiles | Web/mobile map display | Offline vector packs | Attribution and tile generation costs | +| POIs | OSM POIs, Overture Places, partner APIs | Water/food/sleep/repair corridor | Reliability scoring and deduplication | POIs stale/incomplete; licensing varies | +| Protected areas | Protected Planet WDPA/WDPCA, national/regional open data | Rule warnings | Rule confidence engine | Rules are jurisdiction-specific | +| Weather | Open-Meteo, MET Norway/Yr, national weather services, commercial APIs | Forecast along route | Weather-aware rerouting | Forecast uncertainty; API licensing | +| Accommodation | OSM campsites/shelters, Booking Demand API, direct campsite/hut partners, Warmshowers-style partnerships | Deep links and search cards | Reservation/check availability where allowed | API access/partner terms | +| Bike devices | GPX/TCX/FIT export, Garmin Courses API, Wahoo Cloud API | GPX export | Direct sync | API approval required | + +## OSM tags to prioritize + +### Route network + +- `highway=*` +- `bicycle=*` +- `access=*` +- `surface=*` +- `smoothness=*` +- `tracktype=*` +- `sac_scale=*` +- `mtb:scale=*` +- `incline=*` +- `ford=*` +- `bridge=*` +- `tunnel=*` +- `barrier=*` +- `route=bicycle` +- `network=*` + +### Water + +- `amenity=drinking_water` +- `drinking_water=yes/no` +- `natural=spring` +- `man_made=water_well` +- `amenity=fountain` + +### Food and resupply + +- `shop=supermarket` +- `shop=convenience` +- `shop=bakery` +- `amenity=cafe` +- `amenity=restaurant` +- `amenity=fast_food` +- `amenity=fuel` + +### Sleep + +- `tourism=camp_site` +- `tourism=caravan_site` +- `tourism=alpine_hut` +- `tourism=wilderness_hut` +- `tourism=hostel` +- `tourism=hotel` +- `amenity=shelter` + +### Bike repair + +- `shop=bicycle` +- `amenity=bicycle_repair_station` +- `service:bicycle:repair=yes` +- `service:bicycle:pump=yes` + +### Bailout + +- `railway=station` +- `public_transport=station` +- `amenity=bus_station` +- `route=ferry` + +## Data ingestion notes + +### OSM + +Use regional PBF extracts for development. Do not query public Overpass for production-scale route-corridor POI needs. Import regional extracts into PostGIS or a search index. + +### POI deduplication + +POIs may appear in multiple sources. Use a deduplication key based on: + +- geohash proximity, +- normalized name, +- category, +- phone/website if available, +- source reliability. + +### Reliability scoring + +Each POI should have: + +- `source`, +- `last_seen_at`, +- `last_confirmed_by_user_at`, +- `seasonality`, +- `opening_hours`, +- `confidence`, +- `bikepacker_notes_count`, +- `negative_reports_count`. + +### Rules + +Rules must be jurisdiction-specific and confidence-scored. Never present legal/camping rules as absolute unless sourced from official/local authority data and current.