diff --git a/examples/osm-taxiways/README.md b/examples/osm-taxiways/README.md index 8b1abd1..30bbb52 100644 --- a/examples/osm-taxiways/README.md +++ b/examples/osm-taxiways/README.md @@ -9,11 +9,19 @@ become a shell operator or a path). ``` ssh agent@ └─ bastion-broker (key auth; matches request vs allowed-commands.list) - ├─ taxiways-list → ls *-taxiways.geojson in /osm-maps - ├─ taxiways-export → export-icao-geojson.sh (osmium/ogr2ogr/jq) - └─ taxiways-cat → base64 -taxiways.geojson (ssh-safe tunnel) + ├─ taxiways-list → ls *-taxiways.geojson in /osm-maps + ├─ taxiways-export [ ] → export-icao-geojson.sh [ ] + └─ taxiways-cat → base64 -taxiways.geojson (ssh-safe tunnel) ``` +**Pass the aerodrome's coordinates when you have them** (`taxiways-export SJXN -22.710000 +-46.500000`, six decimals). OSM reuses ICAO tags: a helipad or a namesake field elsewhere can +carry the same code, and an export by tag alone then centres on the wrong object. With +coordinates the export keeps the tagged object nearest them (aerodrome/airstrip before +helipads, `select-aerodrome.jq`); when no tagged object lies within 15 km it looks for an +aerodrome/airstrip around the coordinates instead. A backend should fall back to the ICAO-only +form when an older bastion refuses the coordinate form (broker exit 126). + The OSM toolchain (`osmium`, `ogr2ogr`, `jq`) is baked into the image (see `Dockerfile`), so the export runs **inside** this container against the mounted `osm-maps` directory (which holds a region PBF + `export-icao-geojson.sh`). No docker socket, no second @@ -28,6 +36,8 @@ container — the only host data reachable is that one directory. | `entrypoint` | broker-only sshd: key-only, single non-root user, every session forced through the broker | | `bin/taxiways-{list,export,cat}` | the only executables the allowlist lets a client run | | `allowed-commands.list` | the regex allowlist (`taxiways-export [A-Z]{4}`, …) — the security boundary | +| `export-icao-geojson.sh` + `select-aerodrome.jq` | the export itself; copy BOTH into the osm-maps dir (not into the image). The PBF is `$OSM_PBF`, default `/osm-maps/aerodromes-world.osm.pbf` (a symlink there may point at a PBF mounted elsewhere) | +| `tests/*.test.sh` | `bash tests/select-aerodrome.test.sh` (jq only) and `bash tests/export-anchor.test.sh` (osmium/ogr2ogr stubbed) | | `docker-compose.yml` | broker-mode service, osm-maps + allowlist + keys mounts, port 6770 | > This flavor is **Debian standalone** (not `FROM blaxsoftware/bastion`) because the export needs diff --git a/examples/osm-taxiways/allowed-commands.list b/examples/osm-taxiways/allowed-commands.list index 39857c1..6506daf 100644 --- a/examples/osm-taxiways/allowed-commands.list +++ b/examples/osm-taxiways/allowed-commands.list @@ -13,4 +13,7 @@ taxiways-list taxiways-export [A-Z]{4} +# With the caller's coordinates of the aerodrome (decimal degrees, exactly six decimals): +# the export then picks the airfield there instead of any object carrying the ICAO tag. +taxiways-export [A-Z]{4} -?[0-9]{1,2}\.[0-9]{6} -?[0-9]{1,3}\.[0-9]{6} taxiways-cat [A-Z]{4} diff --git a/examples/osm-taxiways/bin/taxiways-export b/examples/osm-taxiways/bin/taxiways-export index 391840d..73b22e4 100755 --- a/examples/osm-taxiways/bin/taxiways-export +++ b/examples/osm-taxiways/bin/taxiways-export @@ -1,10 +1,12 @@ #!/bin/sh -# Run the OSM taxiway export for one ICAO (osmium/ogr2ogr/jq over the Europe PBF). -# Broker rule: `taxiways-export [A-Z]{4}`. Writes -taxiways.geojson into the -# osm-maps dir. This takes MINUTES on a cold cache. +# Run the OSM taxiway export for one ICAO (osmium/ogr2ogr/jq over the aeroway PBF). +# Broker rules: `taxiways-export [A-Z]{4}` and `taxiways-export [A-Z]{4} ` (six +# decimals each). With coordinates the export picks the airfield there instead of whatever +# carries the ICAO tag worldwide. Writes -taxiways.geojson into the osm-maps dir. This +# takes MINUTES on a cold cache. # -# The ICAO is re-validated here (defence in depth — the allowlist already pins -# it to [A-Z]{4}, and the broker exec's us shell-free so $1 can't be an operator). +# Everything is re-validated here (defence in depth — the allowlist already pins the shape, +# and the broker exec's us shell-free so no argument can be an operator). set -eu OSM_DIR=/osm-maps @@ -14,7 +16,22 @@ case "$icao" in *) echo "invalid ICAO: '$icao'" >&2; exit 2 ;; esac +lat="${2:-}" +lon="${3:-}" +if [ -n "$lat$lon" ]; then + for v in "$lat" "$lon"; do + case "$v" in + ''|*[!0-9.-]*) echo "invalid coordinate: '$v'" >&2; exit 2 ;; + esac + done + awk -v a="$lat" -v o="$lon" 'BEGIN { exit !(a >= -90 && a <= 90 && o >= -180 && o <= 180) }' \ + || { echo "coordinates out of range: '$lat' '$lon'" >&2; exit 2; } +fi + cd "$OSM_DIR" 2>/dev/null || { echo "osm-maps dir not mounted at $OSM_DIR" >&2; exit 4; } [ -x ./export-icao-geojson.sh ] || { echo "export-icao-geojson.sh missing in $OSM_DIR" >&2; exit 5; } +if [ -n "$lat$lon" ]; then + exec ./export-icao-geojson.sh "$icao" "$lat" "$lon" +fi exec ./export-icao-geojson.sh "$icao" diff --git a/examples/osm-taxiways/export-icao-geojson.sh b/examples/osm-taxiways/export-icao-geojson.sh new file mode 100755 index 0000000..fabb101 --- /dev/null +++ b/examples/osm-taxiways/export-icao-geojson.sh @@ -0,0 +1,525 @@ +#!/usr/bin/env bash + +set -e + +AIRPORT="${1:-}" +# Optional: the caller's coordinates of this aerodrome (decimal degrees). With them the export +# picks the airfield there instead of whatever carries the ICAO tag worldwide (see 3b below). +ANCHOR_LAT="${2:-}" +ANCHOR_LON="${3:-}" + +if [ -z "$AIRPORT" ]; then + echo "Usage: $0 ICAO_CODE [LAT LON]" + exit 1 +fi + +# The aeroway PBF (all aerodromes/runways/taxiways/aprons). Default: inside the osm-maps dir; +# a symlink there may point at a PBF mounted elsewhere in the container. +PBF="${OSM_PBF:-/osm-maps/aerodromes-world.osm.pbf}" +WORKDIR="work" +# The selection program next to this script (one airfield out of every match, see 3b). +SELECT_JQ="$(cd "$(dirname "$0")" && pwd)/select-aerodrome.jq" +# A match farther than this from the caller's coordinates is not "at our coordinates". +NEAR_KM=15 +# Companions of the chosen object (its boundary polygon and its point) kept within this radius. +KEEP_KM=5 + +valid_coord() { # valid_coord + [[ "$1" =~ ^-?[0-9]{1,3}(\.[0-9]+)?$ ]] && awk -v v="$1" -v m="$2" 'BEGIN { exit !(v >= -m && v <= m) }' +} +if [ -n "$ANCHOR_LAT$ANCHOR_LON" ]; then + if valid_coord "$ANCHOR_LAT" 90 && valid_coord "$ANCHOR_LON" 180; then + echo "Anchor: $ANCHOR_LAT, $ANCHOR_LON" + else + echo "⚠️ Ignoring invalid coordinates '$ANCHOR_LAT' '$ANCHOR_LON' (exporting by ICAO only)." + ANCHOR_LAT="" + ANCHOR_LON="" + fi +fi + +# FAILSAFE: never export more than this area around the aerodrome (km^2). A broken/oversized +# aerodrome match — or a bbox-extract miss — must NEVER fall back to the whole (world) dataset, +# which is what produced the "entire Europe" blow-up. Everything gets clamped to a box centered +# on the aerodrome of at most this area. ~30 km^2 comfortably holds even large international +# airports while stopping a runaway export. +MAX_AREA_KM2=30 + +echo "Using PBF: $PBF" +echo "Airport: $AIRPORT" +echo "Work dir: $WORKDIR" +echo + +mkdir -p "$WORKDIR" + +# Clean up any stale intermediate files for this airport +rm -f "$WORKDIR"/*"$AIRPORT"* "$WORKDIR"/aerodrome-"$AIRPORT".geojson 2>/dev/null +rm -f "${AIRPORT}-taxiways.geojson" 2>/dev/null + +# ------------------------- +# 1-3) Extract THIS aerodrome directly by ICAO from the PBF. +# ------------------------- +# Matches both Europe-style tagging (ref=ICAO, e.g. EDDF) AND ICAO-tag style (icao=ICAO +# with ref empty, e.g. US airports like KJFK), then converts its polygon(s) + point(s) to +# one GeoJSON. This replaces the old "build a whole-world aerodrome index, then jq-select +# by ICAO" approach, which was slow, and matched only top-level ref/iata/icao (so airports +# whose ICAO lives in other_tags — e.g. KJFK — were never found). GDAL's OSM driver needs +# the layer named explicitly (multipolygons/points) + interleaved reading. +AERO_GJ="$WORKDIR/aerodrome-$AIRPORT.geojson" + +if [ ! -f "$AERO_GJ" ]; then + echo "1) Extracting aerodrome '$AIRPORT' by ICAO from $PBF ..." + osmium tags-filter "$PBF" nwr/icao="$AIRPORT" nwr/ref="$AIRPORT" \ + -o "$WORKDIR/aerodrome-$AIRPORT.osm.pbf" --overwrite + + echo "2) Converting aerodrome geometry (polygons + points) to GeoJSON ..." + ogr2ogr -f GeoJSON "$WORKDIR/aerodrome-$AIRPORT-mp.geojson" "$WORKDIR/aerodrome-$AIRPORT.osm.pbf" \ + --config OGR_INTERLEAVED_READING YES multipolygons 2>/dev/null || true + ogr2ogr -f GeoJSON "$WORKDIR/aerodrome-$AIRPORT-pt.geojson" "$WORKDIR/aerodrome-$AIRPORT.osm.pbf" \ + --config OGR_INTERLEAVED_READING YES points 2>/dev/null || true + + for f in "$WORKDIR/aerodrome-$AIRPORT-mp.geojson" "$WORKDIR/aerodrome-$AIRPORT-pt.geojson"; do + [ -s "$f" ] || echo '{"type":"FeatureCollection","features":[]}' > "$f" + done + + echo "3) Merging aerodrome polygon + point into $AERO_GJ ..." + jq -s '{ type:"FeatureCollection", features: ((.[0].features // []) + (.[1].features // [])) }' \ + "$WORKDIR/aerodrome-$AIRPORT-mp.geojson" "$WORKDIR/aerodrome-$AIRPORT-pt.geojson" \ + > "$AERO_GJ" 2>/dev/null || echo '{"type":"FeatureCollection","features":[]}' > "$AERO_GJ" +else + echo "1-3) Reusing existing $AERO_GJ" +fi + +# ------------------------- +# 3b) ONE airfield, not every object carrying this ICAO. +# ------------------------- +# OSM reuses ICAO tags: a helipad or a namesake field on another continent can carry the same +# one. Merged together they centre the export box between them, or on the wrong object. Keep +# the match nearest the caller's coordinates (aerodrome/airstrip before helipads); without +# coordinates, aerodrome/airstrip polygons first. If no tagged object lies near the caller's +# coordinates, look for an aerodrome/airstrip around those coordinates instead, tag or not. +# The whole bbox logic below then centres on that one airfield. +select_aerodrome() { # select_aerodrome + local lat="${ANCHOR_LAT:-null}" lon="${ANCHOR_LON:-null}" + jq --argjson lat "$lat" --argjson lon "$lon" --argjson nearkm "$NEAR_KM" --argjson keepkm "$KEEP_KM" --arg icao "$AIRPORT" \ + -f "$SELECT_JQ" "$1" > "$2" +} +match_km() { jq -r '.match.distance_km // "null"' "$1"; } + +if [ ! -f "$SELECT_JQ" ]; then + echo "⚠️ $SELECT_JQ missing: keeping every object tagged $AIRPORT (old behaviour)." +else + SELECTED_GJ="$WORKDIR/aerodrome-$AIRPORT-selected.geojson" + select_aerodrome "$AERO_GJ" "$SELECTED_GJ" + echo "3b) Chosen: $(jq -c '.match' "$SELECTED_GJ") of $(jq '.features | length' "$AERO_GJ") tagged feature(s)" + + KM=$(match_km "$SELECTED_GJ") + FAR=0 + if [ -n "$ANCHOR_LAT" ]; then + if [ "$KM" = "null" ] || awk -v k="$KM" -v n="$NEAR_KM" 'BEGIN { exit !(k > n) }'; then FAR=1; fi + fi + + if [ "$FAR" -eq 1 ]; then + echo "3c) No $AIRPORT-tagged object within ${NEAR_KM} km of $ANCHOR_LAT,$ANCHOR_LON: searching aerodromes there ..." + read -r MINLON MINLAT MAXLON MAXLAT <<< "$(awk -v la="$ANCHOR_LAT" -v lo="$ANCHOR_LON" -v km="$NEAR_KM" 'BEGIN { + c = cos(la * 3.14159265358979 / 180); if (c < 0.05) c = 0.05; + dla = km / 111.32; dlo = km / (111.32 * c); + printf "%.6f %.6f %.6f %.6f", lo - dlo, la - dla, lo + dlo, la + dla }')" + NEAR="$WORKDIR/near-$AIRPORT" + NEAR_GJ="$NEAR.geojson" + echo '{"type":"FeatureCollection","features":[]}' > "$NEAR_GJ" + if osmium extract --bbox "$MINLON,$MINLAT,$MAXLON,$MAXLAT" "$PBF" -o "$NEAR-area.osm.pbf" --overwrite 2>/dev/null \ + && osmium tags-filter "$NEAR-area.osm.pbf" nwr/aeroway=aerodrome,airstrip -o "$NEAR.osm.pbf" --overwrite 2>/dev/null; then + ogr2ogr -f GeoJSON "$NEAR-mp.geojson" "$NEAR.osm.pbf" --config OGR_INTERLEAVED_READING YES multipolygons 2>/dev/null || true + ogr2ogr -f GeoJSON "$NEAR-pt.geojson" "$NEAR.osm.pbf" --config OGR_INTERLEAVED_READING YES points 2>/dev/null || true + for f in "$NEAR-mp.geojson" "$NEAR-pt.geojson"; do + [ -s "$f" ] || echo '{"type":"FeatureCollection","features":[]}' > "$f" + done + jq -s '{ type:"FeatureCollection", features: ((.[0].features // []) + (.[1].features // [])) }' \ + "$NEAR-mp.geojson" "$NEAR-pt.geojson" > "$NEAR_GJ" 2>/dev/null || true + fi + NEAR_SELECTED="$NEAR-selected.geojson" + select_aerodrome "$NEAR_GJ" "$NEAR_SELECTED" + NEAR_KM_FOUND=$(match_km "$NEAR_SELECTED") + if [ "$NEAR_KM_FOUND" != "null" ] && awk -v k="$NEAR_KM_FOUND" -v n="$NEAR_KM" 'BEGIN { exit !(k <= n) }'; then + echo " → Using the aerodrome at our coordinates: $(jq -c '.match' "$NEAR_SELECTED")" + SELECTED_GJ="$NEAR_SELECTED" + else + echo " → No aerodrome within ${NEAR_KM} km of our coordinates either; keeping the tagged match." + fi + fi + + cp "$SELECTED_GJ" "$AERO_GJ" +fi + +# ------------------------- +# Validate geometry (avoid jq crashes later) +# ------------------------- +HAS_GEOMETRY=$(jq '.features | length' "$AERO_GJ") + +if [ "$HAS_GEOMETRY" -eq 0 ]; then + # No node/way/relation tagged icao=/ref= in the source PBF: this aerodrome simply + # isn't in OSM under this ICAO. That is EXPECTED, not an infra failure — signal it + # DISTINCTLY (exit 44) and on STDERR (broker mode only forwards the wrapper's stderr as the + # error) so the backend classifies it as `unavailable` (no admin notice) instead of a hard + # failure. The "No aerodrome geometry found" marker is also matched by the backend as a + # fallback for boxes still on the old exit-1 script. + msg="No aerodrome geometry found for $AIRPORT (not in OSM under this ICAO)" + echo "❌ $msg" + echo "$msg" >&2 + exit 44 +fi + +# Check if polygon-like geometry exists +HAS_POLY=$(jq ' + .features + | map(select(.geometry.type == "Polygon" or .geometry.type == "MultiPolygon")) + | length +' "$AERO_GJ") + +if [ "$HAS_POLY" -eq 0 ]; then + echo "⚠️ Aerodrome '$AIRPORT' has NO polygon geometry (tiny or point aerodrome)." + echo " → Using 5km radius box around center point instead." + echo +fi + +# ------------------------- +# 4) Extract all taxiways, runways, and aprons once +# ------------------------- +if [ ! -f "$WORKDIR/taxiways.osm.pbf" ]; then + echo "4) Extracting all taxiways..." + osmium tags-filter "$PBF" nwr/aeroway=taxiway -o "$WORKDIR/taxiways.osm.pbf" +else + echo "4) Reusing existing $WORKDIR/taxiways.osm.pbf" +fi + +if [ ! -f "$WORKDIR/runways.osm.pbf" ]; then + echo "4b) Extracting all runways..." + osmium tags-filter "$PBF" nwr/aeroway=runway -o "$WORKDIR/runways.osm.pbf" +else + echo "4b) Reusing existing $WORKDIR/runways.osm.pbf" +fi + +# If runways extract exists but is empty, regenerate (quick check; uses counts via fileinfo -e) +RUNWAY_WAYS=$(osmium fileinfo -e -g data.count.ways "$WORKDIR/runways.osm.pbf" 2>/dev/null || echo 0) +if [ "$RUNWAY_WAYS" -eq 0 ]; then + echo " ⚠️ runways extract is empty, regenerating from source PBF..." + osmium tags-filter "$PBF" nwr/aeroway=runway -o "$WORKDIR/runways.osm.pbf" --overwrite + RUNWAY_WAYS=$(osmium fileinfo -e -g data.count.ways "$WORKDIR/runways.osm.pbf" 2>/dev/null || echo 0) + echo " Runway ways after regen: $RUNWAY_WAYS" + if [ "$RUNWAY_WAYS" -eq 0 ]; then + echo " ⚠️ Still zero runways. Likely no aeroway=runway data in source PBF or wrong PBF path." + fi +fi + +if [ ! -f "$WORKDIR/aprons.osm.pbf" ]; then + echo "4c) Extracting all aprons..." + osmium tags-filter "$PBF" nwr/aeroway=apron -o "$WORKDIR/aprons.osm.pbf" +else + echo "4c) Reusing existing $WORKDIR/aprons.osm.pbf" +fi + +APRON_WAYS=$(osmium fileinfo -e -g data.count.ways "$WORKDIR/aprons.osm.pbf" 2>/dev/null || echo 0) +if [ "$APRON_WAYS" -eq 0 ]; then + echo " ⚠️ aprons extract is empty, regenerating from source PBF..." + osmium tags-filter "$PBF" nwr/aeroway=apron -o "$WORKDIR/aprons.osm.pbf" --overwrite + APRON_WAYS=$(osmium fileinfo -e -g data.count.ways "$WORKDIR/aprons.osm.pbf" 2>/dev/null || echo 0) + echo " Apron ways after regen: $APRON_WAYS" + if [ "$APRON_WAYS" -eq 0 ]; then + echo " ⚠️ Still zero aprons. Likely no aeroway=apron data in source PBF or wrong PBF path." + fi +fi + +# ------------------------- +# 5) Extract taxiways, runways, and aprons for the specific ICAO +# ------------------------- +TAXI_OSM="$WORKDIR/taxiways_${AIRPORT}.osm.pbf" +RUNWAY_OSM="$WORKDIR/runways_${AIRPORT}.osm.pbf" +APRON_OSM="$WORKDIR/aprons_${AIRPORT}.osm.pbf" +BUILDING_OSM="$WORKDIR/buildings_${AIRPORT}.osm.pbf" + +# Compute a bbox that is ALWAYS centered on the aerodrome and NEVER larger than MAX_AREA_KM2. +# Gathers every coordinate of the matched aerodrome feature(s) (any geometry type), takes the +# centroid, and clamps each half-extent to a centered box of <= MAX_AREA_KM2 — with a small +# margin so runways just outside the polygon are still caught, and a floor so point-only +# aerodromes still get a usable box. Prints "MINLON MINLAT MAXLON MAXLAT", or "" if no coords. +calculate_capped_bbox() { + local aero_geojson="$1" + jq -r \ + --argjson maxarea "$MAX_AREA_KM2" \ + --argjson margin 0.6 \ + --argjson minhalf 1.2 ' + def coords: [ .features[].geometry.coordinates | .. | arrays + | select(length == 2 and (.[0]|type) == "number" and (.[1]|type) == "number") ]; + coords as $p + | if ($p | length) == 0 then "" else + ($p | map(.[0])) as $lo | ($p | map(.[1])) as $la + | ($lo | min) as $mnlo | ($lo | max) as $mxlo | ($la | min) as $mnla | ($la | max) as $mxla + | (($mnlo + $mxlo) / 2) as $clon + | (($mnla + $mxla) / 2) as $clat + | (($maxarea | sqrt) / 2) as $maxhalf # max half-side (km) + | (($clat * 3.14159265358979 / 180) | cos) as $coslat + | ((($mxla - $mnla) / 2) * 111.32) as $rawhalflat # raw half-extent (km) + | ((($mxlo - $mnlo) / 2) * 111.32 * $coslat) as $rawhalflon + | ([ [ ($rawhalflat + $margin), $maxhalf ] | min, $minhalf ] | max) as $hlat + | ([ [ ($rawhalflon + $margin), $maxhalf ] | min, $minhalf ] | max) as $hlon + | ($hlat / 111.32) as $dlat + | ($hlon / (111.32 * $coslat)) as $dlon + | "\($clon - $dlon) \($clat - $dlat) \($clon + $dlon) \($clat + $dlat)" + end + ' "$aero_geojson" 2>/dev/null +} + +# A tiny out-of-data bbox → a valid, EMPTY .osm.pbf. Used instead of copying the whole +# (world) dataset on any miss — an empty layer is correct; the whole planet is not. +extract_empty() { + osmium extract --bbox "0,0,0.001,0.001" "$1" -o "$2" --overwrite 2>/dev/null || true +} + +# Extract features within the capped, aerodrome-centered bbox. NEVER copies the whole dataset. +extract_with_bbox() { + local input_pbf="$1" + local output_pbf="$2" + local aero_geojson="$3" + local feature_type="$4" + + BBOX_STR=$(calculate_capped_bbox "$aero_geojson") + + if [ -z "$BBOX_STR" ] || [ "$BBOX_STR" = "null" ]; then + echo " ⚠️ No usable coordinates for $feature_type; emitting EMPTY extract (refusing whole-dataset copy)." + extract_empty "$input_pbf" "$output_pbf" + return + fi + + read MINLON MINLAT MAXLON MAXLAT <<< "$BBOX_STR" + echo " → Capped bbox (<= ${MAX_AREA_KM2} km2, centered on aerodrome): $MINLON,$MINLAT,$MAXLON,$MAXLAT" + osmium extract --bbox "$MINLON","$MINLAT","$MAXLON","$MAXLAT" \ + "$input_pbf" -o "$output_pbf" --overwrite 2>/dev/null || { + echo " ⚠️ bbox extraction failed for $feature_type; emitting EMPTY (not the whole dataset)." + extract_empty "$input_pbf" "$output_pbf" + } +} + +if [ ! -f "$TAXI_OSM" ]; then + echo "5) Extracting taxiways for '$AIRPORT' to $TAXI_OSM ..." + extract_with_bbox "$WORKDIR/taxiways.osm.pbf" "$TAXI_OSM" "$AERO_GJ" "taxiways" +else + echo "5) Reusing existing $TAXI_OSM" +fi + +if [ ! -f "$RUNWAY_OSM" ]; then + echo "5b) Extracting runways for '$AIRPORT' to $RUNWAY_OSM ..." + extract_with_bbox "$WORKDIR/runways.osm.pbf" "$RUNWAY_OSM" "$AERO_GJ" "runways" + + # Check result — 0 within the capped box means this airport simply has no mapped runways + # (or they use different tagging). Leave it EMPTY; never fall back to the whole dataset. + RUNWAY_EXTRACTED_WAYS=$(osmium fileinfo -e -g data.count.ways "$RUNWAY_OSM" 2>/dev/null || echo 0) + if [ "$RUNWAY_EXTRACTED_WAYS" -eq 0 ]; then + echo " ⚠️ No runways within the capped bbox — leaving empty (NOT copying the whole dataset)." + fi +else + echo "5b) Reusing existing $RUNWAY_OSM" +fi + +if [ ! -f "$APRON_OSM" ]; then + echo "5c) Extracting aprons for '$AIRPORT' to $APRON_OSM ..." + extract_with_bbox "$WORKDIR/aprons.osm.pbf" "$APRON_OSM" "$AERO_GJ" "aprons" + + # Check result — 0 within the capped box means no mapped aprons for this airport. Leave it + # EMPTY; never fall back to the whole dataset. + APRON_EXTRACTED_WAYS=$(osmium fileinfo -e -g data.count.ways "$APRON_OSM" 2>/dev/null || echo 0) + if [ "$APRON_EXTRACTED_WAYS" -eq 0 ]; then + echo " ⚠️ No aprons within the capped bbox — leaving empty (NOT copying the whole dataset)." + fi +else + echo "5c) Reusing existing $APRON_OSM" +fi + +# 5d) Buildings (terminals / hangars / generic footprints) for immersion. Unlike the aeroway +# layers we CANNOT tags-filter the whole PBF for nwr/building first (buildings are a huge global +# set) — so we slice the capped aerodrome bbox out of the PBF, then filter buildings from that +# small slice. Empty is fine (e.g. if the source PBF was filtered to aeroway only). +if [ ! -f "$BUILDING_OSM" ]; then + echo "5d) Extracting buildings for '$AIRPORT' to $BUILDING_OSM ..." + BBOX_STR=$(calculate_capped_bbox "$AERO_GJ") + if [ -z "$BBOX_STR" ] || [ "$BBOX_STR" = "null" ]; then + echo " ⚠️ No usable bbox for buildings; emitting EMPTY." + extract_empty "$PBF" "$BUILDING_OSM" + else + read MINLON MINLAT MAXLON MAXLAT <<< "$BBOX_STR" + AREA_OSM="$WORKDIR/area_${AIRPORT}.osm.pbf" + osmium extract --bbox "$MINLON","$MINLAT","$MAXLON","$MAXLAT" "$PBF" -o "$AREA_OSM" --overwrite 2>/dev/null || extract_empty "$PBF" "$AREA_OSM" + osmium tags-filter "$AREA_OSM" nwr/building nwr/aeroway=terminal nwr/aeroway=hangar nwr/aeroway=gate -o "$BUILDING_OSM" --overwrite 2>/dev/null || extract_empty "$PBF" "$BUILDING_OSM" + fi + BUILDING_WAYS=$(osmium fileinfo -e -g data.count.ways "$BUILDING_OSM" 2>/dev/null || echo 0) + echo " Building ways: $BUILDING_WAYS" +else + echo "5d) Reusing existing $BUILDING_OSM" +fi + + + +# ------------------------- +# 6) Convert taxiways, runways, and aprons to GeoJSON and merge into one +# ------------------------- +OUT_GJ="${AIRPORT}-taxiways.geojson" + +# Temp files for individual conversions +TAXI_GJ="$WORKDIR/taxiways_${AIRPORT}.geojson" +RUNWAY_GJ="$WORKDIR/runways_${AIRPORT}.geojson" +APRON_GJ="$WORKDIR/aprons_${AIRPORT}.geojson" +BUILDING_GJ="$WORKDIR/buildings_${AIRPORT}.geojson" + +# Helper function to check if GeoJSON has features +has_features() { + local file="$1" + if [ -f "$file" ]; then + jq '.features | length' "$file" 2>/dev/null || echo 0 + else + echo 0 + fi +} + +if [ ! -f "$TAXI_GJ" ] || [ ! -s "$TAXI_GJ" ]; then + echo "6) Generating taxiways GeoJSON ..." + ogr2ogr \ + -f GeoJSON "$TAXI_GJ" \ + "$TAXI_OSM" \ + --config OGR_INTERLEAVED_READING YES \ + -nln "taxiways" + TAXIWAY_FEATURE_COUNT=$(has_features "$TAXI_GJ") + echo " → Found $TAXIWAY_FEATURE_COUNT taxiway features" +fi + +if [ ! -f "$RUNWAY_GJ" ] || [ ! -s "$RUNWAY_GJ" ]; then + echo "6b) Generating runways GeoJSON ..." + ogr2ogr \ + -f GeoJSON "$RUNWAY_GJ" \ + "$RUNWAY_OSM" \ + --config OGR_INTERLEAVED_READING YES \ + -nln "runways" + + # Check if runways GeoJSON is empty after export + RUNWAY_FEATURE_COUNT=$(has_features "$RUNWAY_GJ") + echo " → Found $RUNWAY_FEATURE_COUNT runway features" + if [ "$RUNWAY_FEATURE_COUNT" -eq 0 ]; then + echo "⚠️ WARNING: runways_${AIRPORT}.geojson is empty (no features found)" + echo " This may indicate runways aren't available for this airport or use different tagging." + fi +fi + +if [ ! -f "$APRON_GJ" ] || [ ! -s "$APRON_GJ" ]; then + echo "6c) Generating aprons GeoJSON ..." + ogr2ogr \ + -f GeoJSON "$APRON_GJ" \ + "$APRON_OSM" \ + --config OGR_INTERLEAVED_READING YES \ + -nln "aprons" + + # Check if aprons GeoJSON is empty after export + APRON_FEATURE_COUNT=$(has_features "$APRON_GJ") + echo " → Found $APRON_FEATURE_COUNT apron features" + if [ "$APRON_FEATURE_COUNT" -eq 0 ]; then + echo "⚠️ WARNING: aprons_${AIRPORT}.geojson is empty (no features found)" + echo " This may indicate aprons aren't available for this airport or use different tagging." + fi +fi + +if [ ! -f "$BUILDING_GJ" ] || [ ! -s "$BUILDING_GJ" ]; then + echo "6e) Generating buildings GeoJSON ..." + # Buildings are closed ways → the GDAL OSM driver emits them on the multipolygons layer. + ogr2ogr \ + -f GeoJSON "$BUILDING_GJ" \ + "$BUILDING_OSM" \ + --config OGR_INTERLEAVED_READING YES \ + multipolygons \ + -nln "buildings" 2>/dev/null || echo '{"type":"FeatureCollection","features":[]}' > "$BUILDING_GJ" + echo " → Found $(has_features "$BUILDING_GJ") building features" +fi + +# Always regenerate the merged output with feature_type properties +TAXI_COUNT=$(has_features "$TAXI_GJ") +RUNWAY_COUNT=$(has_features "$RUNWAY_GJ") +APRON_COUNT=$(has_features "$APRON_GJ") + +echo "6d) Merging all layers into $OUT_GJ ..." + + # Ensure all files exist and have valid GeoJSON; create empty ones if missing + for gj_file in "$TAXI_GJ" "$RUNWAY_GJ" "$APRON_GJ" "$BUILDING_GJ"; do + if [ ! -f "$gj_file" ] || [ ! -s "$gj_file" ]; then + echo '{"type":"FeatureCollection","features":[]}' > "$gj_file" + fi + done + + jq -s ' + { + type: "FeatureCollection", + features: ( + [ + (.[0].features // [])[] + | .properties.layer = "taxiway" + | if (.properties.other_tags // "" | contains("holding_position")) then + .properties.feature_type = "holding_point" + else + .properties.feature_type = "taxiway" + end + | if (.properties.ref // "") == "" and (.properties.other_tags // "" | test("\"ref\"\\s*=>\\s*\"([^\"]+)\"")) then + .properties.ref = (.properties.other_tags | match("\"ref\"\\s*=>\\s*\"([^\"]+)\"") | .captures[0].string) + else + . + end + | if (.properties.name // "") == "" and (.properties.other_tags // "" | test("\"name\"\\s*=>\\s*\"([^\"]+)\"")) then + .properties.name = (.properties.other_tags | match("\"name\"\\s*=>\\s*\"([^\"]+)\"") | .captures[0].string) + else + . + end + ] + + [ + (.[1].features // [])[] + | .properties.layer = "runway" + | .properties.feature_type = "runway" + | if (.properties.ref // "") == "" and (.properties.other_tags // "" | test("\"ref\"\\s*=>\\s*\"([^\"]+)\"")) then + .properties.ref = (.properties.other_tags | match("\"ref\"\\s*=>\\s*\"([^\"]+)\"") | .captures[0].string) + else + . + end + ] + + [ + (.[2].features // [])[] + | .properties.layer = "apron" + | .properties.feature_type = "apron" + ] + + [ + (.[3].features // [])[] + | select(.geometry.type == "Polygon" or .geometry.type == "MultiPolygon") + | .properties.layer = "aerodrome" + | .properties.feature_type = "aerodrome" + ] + + [ + (.[4].features // [])[] + | select(.geometry.type == "Polygon" or .geometry.type == "MultiPolygon") + | .properties.layer = "building" + | .properties.feature_type = "building" + ] + ) + } + ' "$TAXI_GJ" "$RUNWAY_GJ" "$APRON_GJ" "$AERO_GJ" "$BUILDING_GJ" > "$OUT_GJ" + + # Validate merged output + TOTAL_FEATURES=$(jq '.features | length' "$OUT_GJ") + TAXIWAY_FINAL=$(jq '[.features[] | select(.properties.feature_type == "taxiway")] | length' "$OUT_GJ") + HOLDING_FINAL=$(jq '[.features[] | select(.properties.feature_type == "holding_point")] | length' "$OUT_GJ") + RUNWAY_FINAL=$(jq '[.features[] | select(.properties.feature_type == "runway")] | length' "$OUT_GJ") + APRON_FINAL=$(jq '[.features[] | select(.properties.feature_type == "apron")] | length' "$OUT_GJ") + BUILDING_FINAL=$(jq '[.features[] | select(.properties.feature_type == "building")] | length' "$OUT_GJ") + BOUNDARY_FINAL=$(jq '[.features[] | select(.properties.feature_type == "aerodrome")] | length' "$OUT_GJ") + + echo "✓ Merged successfully: $TOTAL_FEATURES total features" + echo " - Taxiways: $TAXIWAY_FINAL" + echo " - Holding Points: $HOLDING_FINAL" + echo " - Runways: $RUNWAY_FINAL" + echo " - Aprons: $APRON_FINAL" + echo " - Buildings: $BUILDING_FINAL" + echo " - Boundary: $BOUNDARY_FINAL" + +echo "Done. Complete airport data for '$AIRPORT' is in:" +echo " - $OUT_GJ" diff --git a/examples/osm-taxiways/select-aerodrome.jq b/examples/osm-taxiways/select-aerodrome.jq new file mode 100644 index 0000000..05f1157 --- /dev/null +++ b/examples/osm-taxiways/select-aerodrome.jq @@ -0,0 +1,75 @@ +# One airfield out of every OSM object that carries the requested ICAO (or out of the +# aerodromes found around the caller's coordinates). OSM reuses tags: a helipad or a +# namesake field on another continent can carry the same ICAO, and merging them all +# centres the export box somewhere between them. +# +# Input: a GeoJSON FeatureCollection (polygons + points, as ogr2ogr writes them). +# Args: --argjson lat --argjson lon the caller's coordinates +# --argjson nearkm radius in which a match counts as "at our coordinates" +# --argjson keepkm companions of the chosen object kept with it (its +# boundary polygon and its point are separate features) +# --arg icao the requested code: an object tagged icao= +# is another airfield and never chosen ("" = no check) +# Output: the chosen object (+ companions) as a FeatureCollection, plus `match`: +# {distance_km, aeroway, anchored, candidates}. Empty input → empty collection. +# +# Choice: with coordinates, the objects within `nearkm` of them, aerodrome/airstrip before +# other kinds before heliport/helipad, then the nearest; if none is that close, simply the +# nearest. Without coordinates: aerodrome/airstrip first, polygons before points. + +def aeroway: + (.properties.aeroway + // ((.properties.other_tags // "") | (capture("\"aeroway\"=>\"(?[^\"]*)\"") // {}) | .v) + // ""); + +def rank: + aeroway as $a + | if ($a == "aerodrome" or $a == "airstrip") then 0 + elif ($a == "heliport" or $a == "helipad") then 2 + else 1 end; + +def points: + [ .geometry.coordinates? | .. | arrays + | select(length >= 2 and (.[0] | type) == "number" and (.[1] | type) == "number") ]; + +def centre: + points as $p + | if ($p | length) == 0 then null + else [ ($p | map(.[0]) | add / length), ($p | map(.[1]) | add / length) ] end; + +# Equirectangular distance in km between two [lon, lat] points: plenty at airfield scale. +def km($a; $b): + ((($a[1] + $b[1]) / 2) * 3.14159265358979 / 180 | cos) as $c + | (((($a[0] - $b[0]) * $c) | . * .) + (($a[1] - $b[1]) | . * .)) | sqrt * 111.32; + +def ispoly: (.geometry.type == "Polygon" or .geometry.type == "MultiPolygon"); + +# The object's own icao tag (top level or in other_tags), "" when it has none. `ref` is left +# out on purpose: it often holds a national code, not the ICAO. +def icaotag: + (.properties.icao + // ((.properties.other_tags // "") | (capture("\"icao\"=>\"(?[^\"]*)\"") // {}) | .v) + // "") | ascii_upcase; + +(if $lat != null and $lon != null then [$lon, $lat] else null end) as $anchor +| [ (.features // [])[] + | select(.geometry != null) + | select($icao == "" or icaotag == "" or icaotag == ($icao | ascii_upcase)) + | . + {_c: centre, _r: rank} + | select(._c != null) + | . + {_d: (if $anchor != null then km(._c; $anchor) else 0 end)} ] as $all +| ($all | map(select($anchor == null or ._d <= $nearkm))) as $near +| (if ($near | length) > 0 + then $near | sort_by(._r, ._d, (if ispoly then 0 else 1 end)) + else $all | sort_by(._d, ._r) end) as $ranked +| if ($ranked | length) == 0 + then { type: "FeatureCollection", features: [], match: null } + else + $ranked[0] as $best + | { type: "FeatureCollection", + features: [ $ranked[] | select(._r <= $best._r and km(._c; $best._c) <= $keepkm) | del(._c, ._r, ._d) ], + match: { distance_km: (if $anchor != null then ($best._d * 10 | round / 10) else null end), + aeroway: ($best | aeroway), + anchored: ($anchor != null), + candidates: ($all | length) } } + end diff --git a/examples/osm-taxiways/tests/export-anchor.test.sh b/examples/osm-taxiways/tests/export-anchor.test.sh new file mode 100755 index 0000000..8ebb655 --- /dev/null +++ b/examples/osm-taxiways/tests/export-anchor.test.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# The anchoring glue of export-icao-geojson.sh (steps 3b/3c), with osmium and ogr2ogr stubbed: +# the tagged match is a helipad far away, the aerodrome sits at the caller's coordinates. +# Needs bash + jq + awk only: bash examples/osm-taxiways/tests/export-anchor.test.sh +set -euo pipefail +DIR="$(cd "$(dirname "$0")/.." && pwd)" +T="$(mktemp -d)" +trap 'rm -rf "$T"' EXIT +mkdir -p "$T/bin" "$T/run" +cp "$DIR/export-icao-geojson.sh" "$DIR/select-aerodrome.jq" "$T/run/" +touch "$T/run/world.osm.pbf" + +# osmium: every write just creates its -o file; counts are 0. +cat > "$T/bin/osmium" <<'EOF' +#!/usr/bin/env bash +[ "$1" = "fileinfo" ] && { echo 0; exit 0; } +while [ $# -gt 0 ]; do [ "$1" = "-o" ] && { touch "$2"; exit 0; }; shift; done +EOF +# ogr2ogr: by output name, the tagged far helipad, the aerodrome near the anchor, or nothing. +cat > "$T/bin/ogr2ogr" <<'EOF' +#!/usr/bin/env bash +out="$3" +case "$out" in + *aerodrome-*-pt.geojson) + echo '{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"other_tags":"\"aeroway\"=>\"helipad\""},"geometry":{"type":"Point","coordinates":[-35.0,-15.0]}}]}' > "$out" ;; + *near-*-mp.geojson) + echo '{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"aeroway":"aerodrome","name":"Field at our coordinates"},"geometry":{"type":"MultiPolygon","coordinates":[[[[-46.51,-22.71],[-46.49,-22.71],[-46.49,-22.69],[-46.51,-22.69],[-46.51,-22.71]]]]}}]}' > "$out" ;; + *) echo '{"type":"FeatureCollection","features":[]}' > "$out" ;; +esac +EOF +chmod +x "$T/bin/osmium" "$T/bin/ogr2ogr" + +fail=0 +check() { if [ "$2" = "$3" ]; then echo "ok $1"; else echo "FAIL $1: expected '$2', got '$3'"; fail=1; fi; } + +cd "$T/run" +log=$(PATH="$T/bin:$PATH" OSM_PBF="$T/run/world.osm.pbf" bash ./export-icao-geojson.sh ABCD -22.700000 -46.500000 2>&1 || true) +check "falls back to the aerodrome at our coordinates" 1 "$(grep -c 'Using the aerodrome at our coordinates' <<< "$log")" +check "the export is centred on that field" 'Field at our coordinates' "$(jq -r '.features[0].properties.name' work/aerodrome-ABCD.geojson)" + +rm -rf work ABCD-taxiways.geojson +log=$(PATH="$T/bin:$PATH" OSM_PBF="$T/run/world.osm.pbf" bash ./export-icao-geojson.sh ABCD 2>&1 || true) +check "without coordinates: no search, the tagged object stays" 'helipad' "$(jq -r '.match.aeroway' work/aerodrome-ABCD.geojson)" +check "without coordinates: no fallback line" 0 "$(grep -c 'searching aerodromes there' <<< "$log" || true)" + +rm -rf work +log=$(PATH="$T/bin:$PATH" OSM_PBF="$T/run/world.osm.pbf" bash ./export-icao-geojson.sh ABCD '1;rm' -46.5 2>&1 || true) +check "invalid coordinates are ignored" 1 "$(grep -c 'Ignoring invalid coordinates' <<< "$log")" + +exit $fail diff --git a/examples/osm-taxiways/tests/select-aerodrome.test.sh b/examples/osm-taxiways/tests/select-aerodrome.test.sh new file mode 100755 index 0000000..92398c5 --- /dev/null +++ b/examples/osm-taxiways/tests/select-aerodrome.test.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# Tests for select-aerodrome.jq (needs only jq): bash examples/osm-taxiways/tests/select-aerodrome.test.sh +set -euo pipefail +DIR="$(cd "$(dirname "$0")/.." && pwd)" +JQ="$DIR/select-aerodrome.jq" +fail=0 + +select_() { # select_ <<< geojson + jq -c --argjson lat "$1" --argjson lon "$2" --argjson nearkm 15 --argjson keepkm 5 --arg icao "${ICAO:-ABCD}" -f "$JQ" +} + +check() { # check + if [ "$2" = "$3" ]; then echo "ok $1"; else echo "FAIL $1: expected $2, got $3"; fail=1; fi +} + +poly() { # poly : a ~1 km square + jq -nc --argjson x "$1" --argjson y "$2" --argjson p "$3" \ + '{type:"Feature",properties:$p,geometry:{type:"MultiPolygon",coordinates:[[[[$x,$y],[$x+0.01,$y],[$x+0.01,$y+0.01],[$x,$y+0.01],[$x,$y]]]]}}' +} +point() { # point + jq -nc --argjson x "$1" --argjson y "$2" --argjson p "$3" '{type:"Feature",properties:$p,geometry:{type:"Point",coordinates:[$x,$y]}}' +} +fc() { jq -sc '{type:"FeatureCollection",features:.}'; } + +# 1) Our field: an aerodrome polygon + its point; the same ICAO on a helipad ~1400 km away. +field=$(poly -46.50 -22.71 '{"aeroway":"aerodrome","name":"Our field"}') +fieldpt=$(point -46.495 -22.705 '{"other_tags":"\"aeroway\"=>\"aerodrome\",\"icao\"=>\"ABCD\""}') +helipad=$(point -35.0 -15.0 '{"other_tags":"\"aeroway\"=>\"helipad\",\"icao\"=>\"ABCD\""}') +out=$(printf '%s\n%s\n%s\n' "$helipad" "$field" "$fieldpt" | fc | select_ -22.70 -46.50) +check "anchored: the field near our coordinates, not the far helipad" '2 aerodrome' "$(jq -r '"\(.features|length) \(.match.aeroway)"' <<< "$out")" +check "anchored: distance is reported" 'true' "$(jq -r '.match.distance_km < 2' <<< "$out")" + +# 2) Only the far helipad carries the ICAO: it is returned with its distance, so the caller +# can see the match is not at our coordinates and search around them instead. +out=$(printf '%s\n' "$helipad" | fc | select_ -22.70 -46.50) +check "far only: nearest returned, distance shows it is far" 'helipad true' "$(jq -r '"\(.match.aeroway) \(.match.distance_km > 1000)"' <<< "$out")" + +# 3) Near our coordinates both an airfield and a helipad: the airfield wins even if a bit farther. +nearheli=$(point -46.501 -22.701 '{"aeroway":"helipad"}') +out=$(printf '%s\n%s\n' "$nearheli" "$field" | fc | select_ -22.70 -46.50) +check "near: aerodrome before helipad" 'aerodrome' "$(jq -r '.match.aeroway' <<< "$out")" + +# 4) Two namesake aerodromes: the one at our coordinates. +other=$(poly 10.0 50.0 '{"aeroway":"aerodrome","name":"Namesake"}') +out=$(printf '%s\n%s\n' "$other" "$field" | fc | select_ -22.70 -46.50) +check "namesakes: the one at our coordinates" 'Our field' "$(jq -r '.features[0].properties.name' <<< "$out")" + +# 5) No coordinates: aerodrome before helipad, polygon before point. +out=$(printf '%s\n%s\n%s\n' "$helipad" "$fieldpt" "$field" | fc | select_ null null) +check "no anchor: aerodrome polygon first" 'MultiPolygon aerodrome false' "$(jq -r '"\(.features[0].geometry.type) \(.match.aeroway) \(.match.anchored)"' <<< "$out")" + +# 6) Searching around our coordinates (wrong coordinates on our side, say): an aerodrome there +# that carries ANOTHER ICAO is another airfield and is never chosen; an untagged one may be. +otherfield=$(poly -46.50 -22.71 '{"aeroway":"aerodrome","name":"Someone else","other_tags":"\"icao\"=>\"WXYZ\""}') +out=$(printf '%s\n' "$otherfield" | fc | select_ -22.70 -46.50) +check "another ICAO: not chosen" '0' "$(jq -r '.features | length' <<< "$out")" +untagged=$(poly -46.50 -22.71 '{"aeroway":"airstrip","name":"Untagged strip"}') +out=$(printf '%s\n%s\n' "$otherfield" "$untagged" | fc | select_ -22.70 -46.50) +check "untagged strip: chosen" 'Untagged strip' "$(jq -r '.features[0].properties.name' <<< "$out")" +ours=$(poly -46.50 -22.71 '{"aeroway":"aerodrome","name":"Ours","icao":"abcd"}') +out=$(printf '%s\n' "$ours" | fc | select_ -22.70 -46.50) +check "same ICAO in any case: chosen" 'Ours' "$(jq -r '.features[0].properties.name' <<< "$out")" + +# 7) Empty and geometry-less input stays empty. +out=$(echo '{"type":"FeatureCollection","features":[{"type":"Feature","properties":{},"geometry":null}]}' | select_ 1 1) +check "empty: no features, no match" '0 null' "$(jq -r '"\(.features|length) \(.match)"' <<< "$out")" + +exit $fail