MacOS amd 64 release error

[hellkite500]

Running hfsubset from the macos release fails with the follow error

./hfsubset -l "all" -o test.gpkg -t nldi_feature "nwis:USGS-02300700"
hfsubset ==> 12:03:13 [0/4] sending http request
hfsubset ==> 12:03:13 [1/4] waiting for response
hfsubset ==> 12:03:59 [2/4] reading hydrofabric subset
hfsubset ==> 12:03:59 [3/4] parsing base64 response
panic: illegal base64 data at input byte 0

goroutine 1 [running]:
main.makeRequest({0x12b67d8?, 0x11?}, 0x5?, 0x12ac700?)
	/home/runner/work/hfsubset/hfsubset/cli/main.go:196 +0x5ca
main.main()
	/home/runner/work/hfsubset/hfsubset/cli/main.go:266 +0x3d1

System details:
macOS 14.1.1 (23B81)
Apple M2 Max

[program--]

@mikejohnson51 This is related to subset_network when using argument nldi_feature -- any ideas?

Error log output on endpoint:

[jsinghm-hfsubset] [2023-11-29 19:03:13] START RequestId: 28a133cb-4879-4a8b-91ce-36354f576490 Version: $LATEST
[jsinghm-hfsubset] [2023-11-29 19:03:13] INFO [2023-11-29 19:03:13] [subset] Received request:
[jsinghm-hfsubset] [2023-11-29 19:03:13] {
[jsinghm-hfsubset] [2023-11-29 19:03:13]   s3: s3://lynker-spatial/v20/, 
[jsinghm-hfsubset] [2023-11-29 19:03:13]   id: NULL, 
[jsinghm-hfsubset] [2023-11-29 19:03:13]   comid: NULL, 
[jsinghm-hfsubset] [2023-11-29 19:03:13]   hl_uri: NULL, 
[jsinghm-hfsubset] [2023-11-29 19:03:13]   nldi_feature: {"featureSource":"nwis","featureId":"USGS-02300700"}, 
[jsinghm-hfsubset] [2023-11-29 19:03:13]   xy: NULL, 
[jsinghm-hfsubset] [2023-11-29 19:03:13]   layers: divides,nexus,flowpaths,network,hydrolocations,lakes,reference_flowline,reference_catchment,reference_flowpaths,reference_divides, 
[jsinghm-hfsubset] [2023-11-29 19:03:13]   version: v20
[jsinghm-hfsubset] [2023-11-29 19:03:13] }
[jsinghm-hfsubset] [2023-11-29 19:03:59] ERROR [2023-11-29 19:03:59] object 'origin' not found
[jsinghm-hfsubset] [2023-11-29 19:03:59] END RequestId: c7762de4-eeab-43e3-a860-1a0b6364712f
[jsinghm-hfsubset] [2023-11-29 19:03:59] REPORT RequestId: c7762de4-eeab-43e3-a860-1a0b6364712f Duration: 46210.95 ms   Billed Dur

Note

Need to double check, but the nldi_feature argument may be passed as a JSON string instead of a list in the runtime, so I might need to double check that

[mikejohnson51]

A lot of requests (non NLDI too) are getting that

hfsubset ==> 12:03:59 [3/4] parsing base64 response
panic: illegal base64 data at input byte 

response

For example from Jordan

[mikejohnson51]

With respect to the NLDI call, it needs to be nwissite

get_nldi_feature(list("featureSource" = "nwissite","featureId" = "USGS-02300700"))

[program--]

A lot of requests (non NLDI too) are getting that

hfsubset ==> 12:03:59 [3/4] parsing base64 response
panic: illegal base64 data at input byte 

response

For example from Jordan
...

That one went through, but I do see for comids 1 and 10:

[jsinghm-hfsubset] [2023-11-28 14:19:37] START RequestId: ca14a187-37bc-4074-b773-62e6e23fd3cb Version: $LATEST
[jsinghm-hfsubset] [2023-11-28 14:19:37] INFO [2023-11-28 14:19:37] [subset] Received request:
[jsinghm-hfsubset] [2023-11-28 14:19:37] {
[jsinghm-hfsubset] [2023-11-28 14:19:37]   s3: s3://lynker-spatial/v20/, 
[jsinghm-hfsubset] [2023-11-28 14:19:37]   id: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:19:37]   comid: 1, 
[jsinghm-hfsubset] [2023-11-28 14:19:37]   hl_uri: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:19:37]   nldi_feature: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:19:37]   xy: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:19:37]   layers: divides,nexus,flowpaths,network,hydrolocations, 
[jsinghm-hfsubset] [2023-11-28 14:19:37]   version: v20
[jsinghm-hfsubset] [2023-11-28 14:19:37] }
- [jsinghm-hfsubset] [2023-11-28 14:20:27] ERROR [2023-11-28 14:20:27] In argument: `id == origin | toid == origin`.
[jsinghm-hfsubset] [2023-11-28 14:20:27] END RequestId: fabeadde-03fc-421e-b053-982198023d6d
[jsinghm-hfsubset] [2023-11-28 14:20:27] REPORT RequestId: fabeadde-03fc-421e-b053-982198023d6d Duration: 50087.14 ms   Billed Duration: 50088 ms       Memory Size: 3008 MB    Max Memory Used: 3008 MB
[jsinghm-hfsubset] [2023-11-28 14:20:56] START RequestId: 1f325e4e-e148-49a3-ba8a-4bdd69514a2a Version: $LATEST
[jsinghm-hfsubset] [2023-11-28 14:20:56] INFO [2023-11-28 14:20:56] [subset] Received request:
[jsinghm-hfsubset] [2023-11-28 14:20:56] {
[jsinghm-hfsubset] [2023-11-28 14:20:56]   s3: s3://lynker-spatial/v20/, 
[jsinghm-hfsubset] [2023-11-28 14:20:56]   id: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:20:56]   comid: 10, 
[jsinghm-hfsubset] [2023-11-28 14:20:56]   hl_uri: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:20:56]   nldi_feature: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:20:56]   xy: NULL, 
[jsinghm-hfsubset] [2023-11-28 14:20:56]   layers: divides,nexus,flowpaths,network,hydrolocations, 
[jsinghm-hfsubset] [2023-11-28 14:20:56]   version: v20
[jsinghm-hfsubset] [2023-11-28 14:20:56] }
- [jsinghm-hfsubset] [2023-11-28 14:21:46] ERROR [2023-11-28 14:21:46] In argument: `id == origin | toid == origin`.
[jsinghm-hfsubset] [2023-11-28 14:21:46] END RequestId: 6bcb87af-6c17-4f87-b576-1425671ad01a
[jsinghm-hfsubset] [2023-11-28 14:21:46] REPORT RequestId: 6bcb87af-6c17-4f87-b576-1425671ad01a Duration: 50362.74 ms   Billed Duration

With respect to the NLDI call, it needs to be nwissite

get_nldi_feature(list("featureSource" = "nwissite","featureId" = "USGS-02300700"))

I'll get this fixed sometime tomorrow

[mikejohnson51]

Might be obvious but just to check, I get the same using the correct call to NLDI:

~ » ./hfsubset -l "all" -o test.gpkg -t nldi_feature "nwissite:USGS-02300700"                                                                                                                                            137 ↵ mjohnson@mikes-mbp-2
hfsubset ==> 18:51:20 [0/4] sending http request
hfsubset ==> 18:51:20 [1/4] waiting for response
hfsubset ==> 18:52:06 [2/4] reading hydrofabric subset
hfsubset ==> 18:52:06 [3/4] parsing base64 response
panic: illegal base64 data at input byte 0

goroutine 1 [running]:
main.makeRequest({0x12b67d8?, 0x11?}, 0x5?, 0x12ac700?)
	/home/runner/work/hfsubset/hfsubset/cli/main.go:196 +0x5ca
main.main()
	/home/runner/work/hfsubset/hfsubset/cli/main.go:266 +0x3d1