API results do not match manual run #2090
Comments
|
I am attempting to take a look pending Nikhil's return. I'm currently blocked because my host is not beefy enough to run the Python (upgrading my ec2 now) |
|
I can now run the code, but I'm not clear how to repro comparing the code results to the API results (it looks like the code computes a vector of values). Asking how others repro while looking into it myself. |
|
The actual data viewed by the app is retrieved by https://api.policyengine.org/us/economy/72622/over/2?region=ny&time_period=2025&version=1.168.1 and returns the following data (as of writing) {
"average_time": 82.356605,
"message": null,
"result": {
"budget": {
"baseline_net_income": 805230658091.7635,
"benefit_spending_impact": 0.0,
"budgetary_impact": -649513940.8227539,
"households": 7987399.28894043,
"state_tax_revenue_impact": -649513947.2561035,
"tax_revenue_impact": -649513940.8227539
},
"decile": {
"average": {
"1": 52.16235774130196,
"10": 7.792905488804229,
"2": 97.32359137463641,
"3": 134.58936338829545,
"4": 123.96727039271677,
"5": 143.72944427374023,
"6": 95.08983368351011,
"7": 74.16569077508164,
"8": 43.455887870355,
"9": 24.14385891021873
},
"relative": {
"1": 0.003146526840724313,
"10": 0.000021141517489404118,
"2": 0.0026796351672095584,
"3": 0.0025844203449431817,
"4": 0.0018878227425934077,
"5": 0.0017733413658272553,
"6": 0.0009503647728876599,
"7": 0.0005983514780297754,
"8": 0.00028269536879631593,
"9": 0.00012236136765460105
}
},
"detailed_budget": {},
"inequality": {
"gini": {
"baseline": 0.4141076423182025,
"reform": 0.4132007488484535
},
"top_10_pct_share": {
"baseline": 0.3091987280364086,
"reform": 0.3090170971728428
},
"top_1_pct_share": {
"baseline": 0.06558326861181414,
"reform": 0.0655446522753806
}
},
"intra_decile": {
"all": {
"Gain less than 5%": 0.2665745707396575,
"Gain more than 5%": 0.032295111422464426,
"Lose less than 5%": 0.0,
"Lose more than 5%": 0.0,
"No change": 0.701130317837878
},
"deciles": {
"Gain less than 5%": [
0.06699577922970977,
0.1976327003429331,
0.42107032579864384,
0.4340493832625858,
0.4763546460639225,
0.39319804819716864,
0.33292379819973955,
0.19509287680833562,
0.10637138886544358,
0.042056760628092674
],
"Gain more than 5%": [
0.12878664677125207,
0.09497665132165789,
0.061004674337578765,
0.020954394508522683,
0.01722874728563277,
0.0,
0.0,
0.0,
0.0,
0.0
],
"Lose less than 5%": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"Lose more than 5%": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"No change": [
0.8042175739990381,
0.707390648335409,
0.5179249998637774,
0.5449962222288915,
0.5064166066504446,
0.6068019518028314,
0.6670762018002604,
0.8049071231916644,
0.8936286111345564,
0.9579432393719073
]
}
},
"intra_wealth_decile": {},
"labor_supply_response": {
"decile": {
"average": {
"income": {
"-1": 0.0,
"1": 0.0,
"10": 0.0,
"2": 0.0,
"3": 0.0,
"4": 0.0,
"5": 0.0,
"6": 0.0,
"7": 0.0,
"8": 0.0,
"9": 0.0
},
"substitution": {
"-1": 0.0,
"1": 0.0,
"10": 0.0,
"2": 0.0,
"3": 0.0,
"4": 0.0,
"5": 0.0,
"6": 0.0,
"7": 0.0,
"8": 0.0,
"9": 0.0
}
},
"relative": {
"income": {
"1": 0.0,
"10": 0.0,
"2": 0.0,
"3": 0.0,
"4": 0.0,
"5": 0.0,
"6": 0.0,
"7": 0.0,
"8": 0.0,
"9": 0.0
},
"substitution": {
"1": 0.0,
"10": 0.0,
"2": 0.0,
"3": 0.0,
"4": 0.0,
"5": 0.0,
"6": 0.0,
"7": 0.0,
"8": 0.0,
"9": 0.0
}
}
},
"hours": {
"baseline": 346573692.3224205,
"change": 0.0,
"income_effect": 0.0,
"reform": 346573692.3224205,
"substitution_effect": 0.0
},
"income_lsr": 0,
"relative_lsr": {
"income": 0.0,
"substitution": 0.0
},
"revenue_change": 0,
"substitution_lsr": 0,
"total_change": 0
},
"poverty": {
"deep_poverty": {
"adult": {
"baseline": 0.04495636748999899,
"reform": 0.04399380068422636
},
"all": {
"baseline": 0.040491580260793546,
"reform": 0.039377491024581684
},
"child": {
"baseline": 0.029280349863935046,
"reform": 0.02660400455921055
},
"senior": {
"baseline": 0.03778525719155958,
"reform": 0.03778525719155958
}
},
"poverty": {
"adult": {
"baseline": 0.1338882429487337,
"reform": 0.132665730428481
},
"all": {
"baseline": 0.13727840761484678,
"reform": 0.13570536913664089
},
"child": {
"baseline": 0.1274494269511646,
"reform": 0.12324646248259706
},
"senior": {
"baseline": 0.15805006325910761,
"reform": 0.15805006325910761
}
}
},
"poverty_by_gender": {
"deep_poverty": {
"female": {
"baseline": 0.04285992616648795,
"reform": 0.041924303586102676
},
"male": {
"baseline": 0.03792803881536468,
"reform": 0.03662077396754197
}
},
"poverty": {
"female": {
"baseline": 0.14694765442000127,
"reform": 0.14536709721174249
},
"male": {
"baseline": 0.12681223589917992,
"reform": 0.12524733583355185
}
}
},
"poverty_by_race": {
"poverty": {
"black": {
"baseline": 0.2081929621201948,
"reform": 0.20621439132991548
},
"hispanic": {
"baseline": 0.22047993428778878,
"reform": 0.21958178567700826
},
"other": {
"baseline": 0.17637145492548176,
"reform": 0.17637145492548176
},
"white": {
"baseline": 0.08334872823769275,
"reform": 0.081286891396107
}
}
},
"wealth_decile": {}
},
"status": "ok"
} |
|
Looking at the API to see if I can understand where those values come from in the data in the code sample and how to compare... |
|
Before updating dependencies to match the API, I got 570m locally in both the reproduce in python segment and the policyengine.py package. After running I think this means that either core or policyengine-us updates caused this change. |
|
Here's the segment that now gets me to 649m locally: 
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
https://policyengine.org/us/policy?focus=policyOutput.codeReproducibility&reform=72622®ion=ny&timePeriod=2025&baseline=2&household=49717
PAvel and David report that the outputs produced by the API here do not match the outputs produced by the code in the example.
The text was updated successfully, but these errors were encountered: