Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Enzyme on demo models #813

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Test Enzyme on demo models #813

wants to merge 3 commits into from

Conversation

penelopeysm
Copy link
Member

@penelopeysm penelopeysm commented Feb 16, 2025

This PR is mostly a speculative attempt to see how Enzyme fares on our demo models.

Enzyme AD takes very long to run (technically, the preparation step takes a long time) and so this PR comments out all other tests (including AD tests on other backends).

tldr so far: All tests pass (both forward- and reverse-mode of Enzyme) with all demo models when using TypedVarInfo, but UntypedVarInfo segfaults

Copy link

codecov bot commented Feb 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 4.31%. Comparing base (e4fa7f2) to head (d6bf74a).

❗ There is a different number of reports uploaded between BASE (e4fa7f2) and HEAD (d6bf74a). Click for more details.

HEAD has 6 uploads less than BASE
Flag BASE (e4fa7f2) HEAD (d6bf74a)
12 6
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #813       +/-   ##
==========================================
- Coverage   84.43%   4.31%   -80.13%     
==========================================
  Files          34      34               
  Lines        3849    3804       -45     
==========================================
- Hits         3250     164     -3086     
- Misses        599    3640     +3041     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TuringLang TuringLang deleted a comment from github-actions bot Feb 16, 2025
@penelopeysm penelopeysm force-pushed the py/enzyme branch 2 times, most recently from f42a6e1 to 409fc3b Compare February 16, 2025 21:47
@penelopeysm penelopeysm force-pushed the py/no-ldp-ad branch 2 times, most recently from 8f8018a to 566257e Compare February 18, 2025 22:23
Base automatically changed from py/no-ldp-ad to release-0.35 February 19, 2025 18:36
@yebai yebai mentioned this pull request Feb 20, 2025
Base automatically changed from release-0.35 to master February 28, 2025 15:52
@coveralls
Copy link

coveralls commented Mar 19, 2025

Pull Request Test Coverage Report for Build 13958820972

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2913 unchanged lines in 27 files lost coverage.
  • Overall coverage decreased (-80.1%) to 4.405%

Files with Coverage Reduction New Missed Lines %
src/varname.jl 6 0.0%
ext/DynamicPPLForwardDiffExt.jl 7 0.0%
src/test_utils/model_interface.jl 7 0.0%
src/model_utils.jl 11 0.0%
src/distribution_wrappers.jl 16 0.0%
src/transforming.jl 17 0.0%
src/extract_priors.jl 19 0.0%
src/test_utils/varinfo.jl 23 0.0%
src/values_as_in_model.jl 23 0.0%
src/logdensityfunction.jl 25 0.0%
Totals Coverage Status
Change from base Build 13957720054: -80.1%
Covered Lines: 164
Relevant Lines: 3723

💛 - Coveralls

@penelopeysm penelopeysm changed the title Test Enzyme on demo models / varinfos Test Enzyme on demo models Mar 19, 2025
Copy link
Contributor

github-actions bot commented Mar 19, 2025

Benchmark Report for Commit d6bf74a7846be978e13625907d2e19db86e713f3

Computer Information

Julia Version 1.11.4
Commit 8561cc3d68d (2025-03-10 11:36 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Benchmark Results

|                 Model | Dimension |  AD Backend |      VarInfo Type | Linked | Eval Time / Ref Time | AD Time / Eval Time |
|-----------------------|-----------|-------------|-------------------|--------|----------------------|---------------------|
| Simple assume observe |         1 | forwarddiff |             typed |  false |                  9.6 |                 1.6 |
|           Smorgasbord |       201 | forwarddiff |             typed |  false |                621.3 |                42.1 |
|           Smorgasbord |       201 | forwarddiff | simple_namedtuple |   true |                369.1 |                55.9 |
|           Smorgasbord |       201 | forwarddiff |           untyped |   true |               1228.8 |                28.4 |
|           Smorgasbord |       201 | forwarddiff |       simple_dict |   true |               4178.7 |                20.9 |
|           Smorgasbord |       201 | reversediff |             typed |   true |               1469.6 |                29.6 |
|           Smorgasbord |       201 |    mooncake |             typed |   true |                948.5 |                 5.5 |
|    Loop univariate 1k |      1000 |    mooncake |             typed |   true |               5636.4 |                 4.1 |
|       Multivariate 1k |      1000 |    mooncake |             typed |   true |               1098.2 |                 8.4 |
|   Loop univariate 10k |     10000 |    mooncake |             typed |   true |              62319.2 |                 3.8 |
|      Multivariate 10k |     10000 |    mooncake |             typed |   true |               9089.9 |                 9.6 |
|               Dynamic |        10 |    mooncake |             typed |   true |                126.0 |                12.4 |
|              Submodel |         1 |    mooncake |             typed |   true |                 27.1 |                 6.9 |
|                   LDA |        12 | reversediff |             typed |   true |                469.8 |                 4.7 |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants