Skip to content

Commit b8c5b20

Browse files
committed
Merge API system refactor
2 parents f12c528 + a683fa6 commit b8c5b20

26 files changed

+631
-280
lines changed

.flake8

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,4 +20,4 @@ extend-ignore=
2020
extend-exclude=
2121
.venv,
2222
per-file-ignores =
23-
tests/test_*.py:D103 E501,
23+
skyportal/networklib.py:E731,

CHANGELOG.md

+11
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,17 @@
11
# Changelog
22
Versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html) (`<major>`.`<minor>`.`<patch>`)
33

4+
## [vNext]
5+
### Added
6+
* #14 Add support for the ADSB.lol API
7+
* #14 Add support for a generic flight data API
8+
9+
### Changed
10+
* (Internal) Refactor API handlers to share a common base class
11+
12+
### Removed
13+
* #11 Remove screenshot UI feature
14+
415
## [v1.1.0]
516
### Added
617
* #3 Add optional screenshot UI target, enabled using the `SHOW_SCREENSHOT_BUTTON` config var

README.md

+51-14
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# skyportal
2-
A PyPortal based flight tracker powered by [Adafruit](https://io.adafruit.com/), [Geoapify](https://www.geoapify.com/), and [The OpenSky Network](https://opensky-network.org/).
2+
A PyPortal based flight tracker powered by [Adafruit](https://io.adafruit.com/), [Geoapify](https://www.geoapify.com/), [ADSB.lol](https://adsb.lol), and [The OpenSky Network](https://opensky-network.org/).
33

44
Heavily inspired by Bob Hammell"s PyPortal Flight Tracker ([GH](https://github.com/rhammell/pyportal-flight-tracker), [Tutorial](https://www.hackster.io/rhammell/pyportal-flight-tracker-0be6b0#story)).
55

@@ -50,24 +50,61 @@ secrets = {
5050
"aio_username" : "YOUR_AIO_USERNAME",
5151
"aio_key" : "YOUR_AIO_KEY",
5252
# Open Sky Network credentials, for getting flight information
53+
# Can be omitted if not using OpenSky
5354
"opensky_username": "YOUR_OPENSKY_USERNAME",
54-
"opensky_password": "YOUR_OPENSKY_PASSWORD"
55+
"opensky_password": "YOUR_OPENSKY_PASSWORD",
56+
# Proxy API Gateway credentials
57+
# Can be omitted if not using a proxy server
58+
"proxy_api_url": "YOUR_PROXY_API_URL",
59+
"proxy_api_key": "YOUR_PROXY_API_KEY",
5560
}
5661
```
5762

5863
#### Skyportal Configuration
5964
A collection of functionality-related constants is specified in `skyportal_config.py`, which can be adjusted to suit your needs:
6065

61-
| Variable Name | Description | Default |
62-
|----------------------------|-------------------------------------------------------|----------|
63-
| `SHOW_SCREENSHOT_BUTTON` | Provide a UI button for taking screenshots | `False` |
64-
| `KEEP_N_SCREENSHOTS` | Keep the `n` most recent screenshots in SD storage | `5` |
65-
| `USE_DEFAULT_MAP` | Use the default map image rather than query Geoapify | `False` |
66-
| `MAP_CENTER_LAT` | Map center latitude, decimal degrees | `42.41` |
67-
| `MAP_CENTER_LON` | Map center longitude, deimal degrees | `-71.17` |
68-
| `GRID_WIDTH_MI` | Map grid width, miles | `15` |
69-
| `SKIP_GROUND` | Skip drawing aircraft on the ground | `True` |
70-
| `GEO_ALTITUDE_THRESHOLD_M` | Skip drawing aircraft below this GPS altitude, meters | `20` |
66+
| Variable Name | Description | Default |
67+
|----------------------------|-------------------------------------------------------|-----------|
68+
| `USE_DEFAULT_MAP` | Use the default map image rather than query Geoapify | `False` |
69+
| `MAP_CENTER_LAT` | Map center latitude, decimal degrees | `42.41` |
70+
| `MAP_CENTER_LON` | Map center longitude, deimal degrees | `-71.17` |
71+
| `GRID_WIDTH_MI` | Map grid width, miles | `15` |
72+
| `AIRCRAFT_DATA_SOURCE` | Aircraft State API to utilize<sup>1</sup> | `opensky` |
73+
| `SKIP_GROUND` | Skip drawing aircraft on the ground | `True` |
74+
| `GEO_ALTITUDE_THRESHOLD_M` | Skip drawing aircraft below this GPS altitude, meters | `20` |
75+
76+
**Notes:**
77+
1. See [Data Sources](#data-sources) for valid options
78+
79+
## Data Sources
80+
### OpenSky-Network - `"opensky"`
81+
Query the [OpenSky Network](https://opensky-network.org/) API. This requires a user account to be created & credentials added to `secrets.py`.
82+
83+
Information on their REST API can be found [here](https://openskynetwork.github.io/opensky-api/rest.html).
84+
85+
### ADSB.lol - `"adsblol"`
86+
Query the [ADSB.lol](https://adsb.lol/). This currently does not require user authentication.
87+
88+
Information on their REST API can be found [here](https://api.adsb.lol/docs).
89+
90+
**NOTE:** This API provides a lot of interesting information in the state vector provided for each aircraft. Depending on the level of congestion in your query area, may be more data than can fit into RAM (See: [Known Limitations](#known-limitations)).
91+
92+
### Proxy API - `"proxy"`
93+
Query a user-specified proxy server using the URL and API key provided in `secrets.py`.
94+
95+
For authentication, the API is assumed to expect an key provided in the `"x-api-key"` header.
96+
97+
The proxy API is assumed to expect three parameters:
98+
* `lat`, center latitude, decimal degrees
99+
* `lon`, denter longitude, decimal degrees
100+
* `radius`, search radius, miles
101+
102+
The proxy API is expected to return two parameters:
103+
* `"ac"` - A list of state vectors, as dictionaries, whose kv pairs map directly to `skyportal.aircraftlib.AircraftState`
104+
* `"api_time"` - UTC epoch time, seconds, may be a float
105+
106+
An example using ADSB.lol and AWS Lambda is provided by this repository in [`./adsblol-proxy`](./adsblol-proxy/README.md)
107+
71108

72109
## Touchscreen Functionality
73110
**NOTE:** Touchscreen input is mostly limited to one touch event per screen tap, rather than continuously firing while the screen is being touched.
@@ -77,5 +114,5 @@ A collection of functionality-related constants is specified in `skyportal_confi
77114
### Aircraft Information
78115
Tapping on an aircraft icon will display state information for the aircraft closest to the registered touch point.
79116

80-
### Screenshot
81-
If enabled in the SkyPortal configuration file, a screenshot button is created in the lower left, allowing the user to take a screenshot to SD card storage. The device utilizes a rolling storage, keeping the `n` most recent screenshots and discarding the oldest screenshot if above this threshold.
117+
## Known Limitations
118+
The PyPortal is a highly memory constrained environment, which presents challenges when aiming to create a highly expressive UI. While every attempt is being made to minimize memory usage to keep the Skyportal functioning, the device may occasionally run out of memory. The most likely point for this to happen is when receiving the web request with aircraft state information from your API of choice. Depending on how congested your selected airspace is at query time, there may simply be too much information provided by the API for the device to handle & I've intentionally left the exception unhandled so it will crash the device. Should you find this ocurring often, you may be interested in setting up a proxy server to return only the information needed for the device to function, which can significantly alleviate the amount of RAM needed. See [Proxy API](#proxy-api---proxy) for more information.

adsblol-proxy/README.md

+98
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# ADSB.lol Proxy API
2+
The enclosed `adsblol_proxy.py` file was developed to be deployed using an [AWS Lambda](https://aws.amazon.com/lambda/) behind an [AWS API Gateway](https://aws.amazon.com/api-gateway/). For the default Skyportal configuration this should fall well within the monthly limits of the AWS Free Tier.
3+
4+
If, like me, you've never used AWS before this point, the following steps should help you get up and running. If you know what you're doing then you can probably figure all this out without my help.
5+
6+
## AWS Lambda
7+
### Create Function
8+
From the Lambda console, create a new function with the following configuration:
9+
* Author from scratch
10+
* Whatever function name you want
11+
* Python 3.11 Runtime
12+
* x86_64 architecture
13+
14+
Once created, edit your Runtime Settings and change the Handler to `adsblol_proxy.lambda_handler`.
15+
16+
### Create a `.zip` deployment
17+
Our Lambda depends on [`httpx`](https://github.com/encode/httpx/) to make its web request, so it will need to be installed along with its dependencies before we can deploy. One way to achieve this with Lambda is to upload a [`.zip` deployment package](https://docs.aws.amazon.com/lambda/latest/dg/python-package.html#python-package-create-dependencies); the documentation can be a little obtuse but ultimately the goal is to end up with a zip file whose contents look something like this:
18+
19+
```
20+
anyio/
21+
anyio-4.0.0.dist-info/
22+
certifi/
23+
certifi-2023.7.22.dist-info/
24+
h11/
25+
h11-0.14.0.dist-info/
26+
httpcore/
27+
httpcore-1.0.2.dist-info/
28+
httpx/
29+
httpx-0.25.1.dist-info/
30+
idna/
31+
idna-3.4.dist-info/
32+
sniffio/
33+
sniffio-1.3.0.dist-info/
34+
adsblol_proxy.py
35+
```
36+
37+
I accomplished this using a virtual environment, e.g.:
38+
39+
```
40+
$ python -m venv ./.venv
41+
$ source ./.venv/Scripts/activate
42+
$ python -m pip install -U pip httpx
43+
```
44+
45+
Move or copy the everything from `./.venv/Lib/site-packages` **EXCEPT** `pip` (blows up the file size unnecessarily & we don't need it) into the directory with `adsblol_proxy.py` and zip everything together so you get the layout above. You can then upload this zip file to Lambda & then deploy the code.
46+
47+
## AWS API Gateway
48+
### Create API
49+
From the API Gateway console, create a new API:
50+
* REST API
51+
* New API
52+
* Whatever name you'd like
53+
* Optional description
54+
* Regional endpoint type
55+
56+
### Create Method
57+
Under resources, create a new method:
58+
* `GET` method type
59+
* Lambda function integration type
60+
* Enable "Lambda proxy integration"
61+
* If you've already created your Lambda function above, you should be able to select it
62+
* Default timeout should be fine
63+
64+
### Edit Method
65+
Edit your method request settings:
66+
* Authorization - None
67+
* Request validator - None
68+
* API key required - CHECK
69+
* URL query string parameters
70+
* `lat`, required
71+
* `lon`, required
72+
* `radius`, required
73+
74+
Once this is done, Deploy your API. You will need to specify a stage name if you haven't previously. Since you're just doing a hobby API you can name it whatever you want; I called mine `live`.
75+
76+
Make a note of your Invoke URL, which can be found under Stages. It will be something like `https://abcd123.execute-api.us-east-69.amazonaws.com/live/`.
77+
78+
### Create an API key
79+
Under API Keys create a new API key & store in a secure location that you can access later.
80+
81+
### Create a Usage Plan
82+
This must be created in order for the API key to work. Fill out the options however you'd like.
83+
84+
Once this is created you'll need to add a stage, this is what you targeted when you deployed your API.
85+
86+
Finally, you'll need to add your API key to the Associated API keys.
87+
88+
## Testing
89+
You can check that your API is functional using `curl`:
90+
91+
```
92+
$ curl --location "https://abcd123.execute-api.us-east-69.amazonaws.com/live/?lat=42.41&lon=-71.17&radius=30" --header "x-api-key:<key>"
93+
```
94+
95+
Which should give back some aircraft data.
96+
97+
## Configuring Skyportal
98+
To utilize the proxy server, copy your Invoke URL and API key into `secrets.py`, and set `AIRCRAFT_DATA_SOURCE = "proxy"` in your `skyportal_config.py`.

adsblol-proxy/adsblol_proxy.py

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
import json
2+
3+
import httpx
4+
5+
URL_BASE = "https://api.adsb.lol/v2"
6+
7+
8+
def query_data(lat: float, lon: float, radius: float) -> dict:
9+
"""Execute the desired ADSB.lol query & return the JSON response."""
10+
query_url = f"{URL_BASE}/lat/{lat}/lon/{lon}/dist/{radius}"
11+
r = httpx.get(query_url)
12+
r.raise_for_status()
13+
14+
return r.json()
15+
16+
17+
def simplify_aircraft(flight_data: list[dict]) -> list[dict]:
18+
"""
19+
Simplify the ADSB.lol API response into something more Skyportal memory friendly.
20+
21+
The following cleanup operations are made on the provided aircraft state vectors:
22+
* All non-airborne flights are discarded
23+
* Keys are reorganized into a layout that directly matches `skyportal.AircraftState`
24+
* Value units are converted, where necessary
25+
* Missing keys are set to `None` if expected to be present
26+
"""
27+
airborne_flights = []
28+
for state_vector in flight_data:
29+
if (baro_alt := state_vector["alt_baro"]) == "ground":
30+
# Skip airborne aircraft
31+
continue
32+
33+
if (callsign := state_vector.get("flight", None)) is None:
34+
callsign = state_vector.get("r", None)
35+
if callsign is not None:
36+
callsign = callsign.strip()
37+
38+
# Ground track is likely not transmitted on the ground
39+
# If an aircraft is on the ground it may be transmitting true_heading
40+
if (track := state_vector.get("track", None)) is None:
41+
track = state_vector.get("true_heading", None)
42+
43+
if (alt_geo := state_vector.get("alt_geom", None)) is not None:
44+
alt_geo *= 0.3048 # Provided in ft
45+
46+
if (baro_rate := state_vector.get("baro_rate", None)) is not None:
47+
baro_rate *= 0.3048 # Provided in ft
48+
49+
simplified_vector = {
50+
"icao": state_vector["hex"],
51+
"callsign": callsign,
52+
"lat": state_vector["lat"],
53+
"lon": state_vector["lon"],
54+
"track": track,
55+
"velocity_mps": state_vector["gs"] * 0.5144, # Provided in kts
56+
"on_ground": False,
57+
"baro_altitude_m": baro_alt,
58+
"geo_altitude_m": alt_geo,
59+
"vertical_rate_mps": baro_rate,
60+
"aircraft_category": state_vector.get("category", "NO_INFO"),
61+
}
62+
63+
airborne_flights.append(simplified_vector)
64+
65+
return airborne_flights
66+
67+
68+
def lambda_handler(event, context) -> dict: # noqa: ANN001 D103
69+
try:
70+
params = event["queryStringParameters"]
71+
full_data = query_data(lat=params["lat"], lon=params["lon"], radius=params["radius"])
72+
except httpx.HTTPError as e:
73+
return {
74+
"statusCode": 400,
75+
"body": json.dumps(f"HTTP Exception for {e.request.url} - {e}"),
76+
}
77+
except KeyError as e:
78+
return {
79+
"statusCode": 400,
80+
"body": json.dumps(str(e)),
81+
}
82+
83+
return {
84+
"statusCode": 200,
85+
"body": json.dumps(
86+
{
87+
"ac": simplify_aircraft(full_data["ac"]),
88+
"api_time": full_data["now"] / 1000, # Server time given in milliseconds
89+
}
90+
),
91+
}

assets/camera_green.bmp

-4.74 KB
Binary file not shown.

assets/camera_red.bmp

-4.74 KB
Binary file not shown.

assets/splash.bmp

-225 KB
Binary file not shown.

code.py

+38-13
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,7 @@
88

99
from skyportal.displaylib import SkyPortalUI
1010
from skyportal.maplib import build_bounding_box
11-
from skyportal.networklib import APIException, APITimeoutError
12-
from skyportal.opensky import OpenSky
11+
from skyportal.networklib import APIException, APIHandlerBase, APITimeoutError
1312

1413
try:
1514
from secrets import secrets
@@ -22,7 +21,7 @@
2221
raise Exception("Could not locate configuration file.") from e
2322

2423

25-
def _utc_to_local(utc_timestamp: int, utc_offset: str = "-0000") -> datetime:
24+
def _utc_to_local(utc_timestamp: float, utc_offset: str = "-0000") -> datetime:
2625
"""
2726
Convert the given timestamp into local time with the provided UTC offset.
2827
@@ -38,7 +37,7 @@ def _utc_to_local(utc_timestamp: int, utc_offset: str = "-0000") -> datetime:
3837

3938
# Device Initialization
4039
PYPORTAL = PyPortal() # This also takes care of mounting the SD to /sd
41-
skyportal_ui = SkyPortalUI(enable_screenshot=skyportal_config.SHOW_SCREENSHOT_BUTTON)
40+
skyportal_ui = SkyPortalUI()
4241

4342
PYPORTAL.network.connect()
4443
print("Wifi connected")
@@ -51,32 +50,58 @@ def _utc_to_local(utc_timestamp: int, utc_offset: str = "-0000") -> datetime:
5150
grid_bounds = build_bounding_box()
5251
skyportal_ui.post_connect_init(grid_bounds)
5352

54-
opensky_handler = OpenSky(grid_bounds=grid_bounds)
55-
53+
api_handler: APIHandlerBase
54+
if skyportal_config.AIRCRAFT_DATA_SOURCE == "adsblol":
55+
from skyportal.networklib import ADSBLol
56+
57+
api_handler = ADSBLol(
58+
lat=skyportal_config.MAP_CENTER_LAT,
59+
lon=skyportal_config.MAP_CENTER_LON,
60+
radius=skyportal_config.GRID_WIDTH_MI * 2,
61+
)
62+
print("Using ADSB.lol as aircraft data source")
63+
elif skyportal_config.AIRCRAFT_DATA_SOURCE == "opensky":
64+
from skyportal.networklib import OpenSky
65+
66+
api_handler = OpenSky(grid_bounds=grid_bounds)
67+
print("Using OpenSky as aircraft data source")
68+
elif skyportal_config.AIRCRAFT_DATA_SOURCE == "proxy":
69+
from skyportal.networklib import ProxyAPI
70+
71+
api_handler = ProxyAPI(
72+
lat=skyportal_config.MAP_CENTER_LAT,
73+
lon=skyportal_config.MAP_CENTER_LON,
74+
radius=skyportal_config.GRID_WIDTH_MI * 2,
75+
)
76+
print("Using proxy API as aircraft data source")
77+
else:
78+
raise ValueError(f"Unknown API specified: '{skyportal_config.AIRCRAFT_DATA_SOURCE}'")
79+
80+
gc.collect()
5681
print(f"\n{'='*40}\nInitialization complete\n{'='*40}\n")
5782

5883
# Main loop
5984
skyportal_ui.touch_on()
60-
loop_start_time = datetime.now() - opensky_handler.refresh_interval # Force first API call
85+
loop_start_time = datetime.now() - api_handler.refresh_interval # Force first API call
6186
while True:
62-
if (datetime.now() - loop_start_time) >= opensky_handler.refresh_interval:
87+
if (datetime.now() - loop_start_time) >= api_handler.refresh_interval:
6388
skyportal_ui.touch_off()
6489
try:
65-
opensky_handler.update()
90+
api_handler.update()
6691
except (APITimeoutError, APIException) as e:
6792
print(e)
6893

6994
gc.collect()
7095

71-
if opensky_handler.can_draw():
96+
if api_handler.can_draw:
7297
print("Updating aircraft locations")
73-
skyportal_ui.draw_aircraft(opensky_handler.aircraft)
74-
skyportal_ui.time_label.text = f"{_utc_to_local(opensky_handler.api_time, utc_offset)}"
98+
skyportal_ui.draw_aircraft(api_handler.aircraft)
99+
skyportal_ui.time_label.text = f"{_utc_to_local(api_handler.api_time, utc_offset)}"
75100
else:
76101
print("No aircraft to draw, skipping redraw")
77102

78103
loop_start_time = datetime.now()
79-
next_request_at = loop_start_time + opensky_handler.refresh_interval
104+
next_request_at = loop_start_time + api_handler.refresh_interval
80105
print(f"Sleeping... next refresh at {next_request_at} local")
81106
skyportal_ui.touch_on()
82107

lib/adafruit_ticks.mpy

-626 Bytes
Binary file not shown.

lib/asyncio/__init__.mpy

-439 Bytes
Binary file not shown.

lib/asyncio/core.mpy

-3.43 KB
Binary file not shown.

lib/asyncio/event.mpy

-664 Bytes
Binary file not shown.

lib/asyncio/funcs.mpy

-1.07 KB
Binary file not shown.

lib/asyncio/lock.mpy

-603 Bytes
Binary file not shown.

lib/asyncio/manifest.mpy

-197 Bytes
Binary file not shown.

lib/asyncio/stream.mpy

-2.03 KB
Binary file not shown.

lib/asyncio/task.mpy

-1.48 KB
Binary file not shown.

0 commit comments

Comments
 (0)