migrate/improve auction functionality tests in `z:acceptance` #10229

anilhelvaci · 2024-10-05T03:10:24Z

closes: https://github.com/Agoric/BytePitchPartnerEng/issues/8
closes: #10111

Description

Currently acceptance tests do not cover auction functionality. When we look at the proposals in https://github.com/Agoric/agoric-3-proposals, there are no tests that check for depositors or bidders payouts from a given auction. There is code for changing auction params here and there, including z:acceptance.

In this PR, we test an auction from start to end. Here's the test case implemented;

In USE phase of n:upgrade-next, we place a bid
Start executing TEST phase of z:acceptance
We push a price for ATOM
gov1 deposits 100 IST to ATOM auction book
user1 places a bid
gov2 places a bid
Auction round ends
Depositor gets correct payouts
All bidders including the one placed in an earlier proposal get correct payouts

Why is this PR a draft?

This pr depends on tools in #10171, as soon that's approved I'll rebase this one on top of that one.

Security Considerations

None.

Scaling Considerations

Currently this test takes from 4-5 minutes to 12-13 minutes. It can take up to 20 minutes. This is because how the auction params are set. Why not just change the auction params? See Testing Considerations.

Documentation Considerations

None.

Testing Considerations

Testing auctions in a3p have a huge disadvantage since we don't have any tools for controlling how the time advances when running an actual node. I believe Agoric/agoric-3-proposals#179 should be a priority as it will unlock very comprehensive test coverage in a3p testing.

Since time is a variable that we cannot control but it also plays a huge role when running our auction tests, we need to wait for it. Another complication is coming from how a3p images are built. Imagine this scenario;

chain stopped at t - 3
when the chain stopped, nextStartTime of the auction schedule was t - 1
auction tests started the chain at t
when chain started at t the waker set for the round t - 1 is triggered before our run-test.sh is run
the callback of the waker captured the prices then tried to start the round at t - 1 when it is actually at t
scheduler realized this and rescheduled the round at t - 1 to start at t + 1
round at t + 1 is now the activeStartTime but there's a +1 delay that we have to wait for
our test starts depositing and bidding after the price we set is captured by the auctioneer
price for activeStartTime is already captured when the t - 1 waker run during the fast forward when chain restarted
we need the auction to run with the price we set so we can make sure the long living bid is ended with correct payouts
auctioneer will capture new prices at newStartTime
so the soonest we can start running out test is at newStartTime
we have to wait for +1 (assuming we are still at t) PLUS a whole auction round

This scenario is common when yarn test -m acceptance uses caching as the latest cached layer will have it's state at when it was built. So if you built your latest cached layer an hour ago and auction start frequency is 10 minutes, you will face this issue.

If the latest cached layer is built less than 15 mins ago (10 mins round time and 5 mins maxLate, auctioneer is okay if round starts late less than 5 mins), it is very unlikely that you will face this issue("unlikely" because my math there might be wrong.) So I assume we will not face this issue when running in CI as I think all build layers are built from scratch in CI.

Why not update auction params?

Changed params take effect AFTER the next scheduled round. We are only interested in the next scheduled round since it is the soonest we can start depositing and bidding. One way to make the changed params effective for the next scheduled round in our test-acceptance build might be to change them in an earlier proposal USE state which means we have to depend on the global state to carry out our tests which is something I'm not a fan of.

Who places the long living bid?

Again, in order not to depend on global state we create a new user called long-living-bidder. Because other tests might mess with the bidder's wallet history and make querying payouts more complex.

Upgrade Considerations

None.

a3p-integration/proposals/n:upgrade-next/performAction.js

a3p-integration/proposals/z:acceptance/auction.test.js

a3p-integration/proposals/z:acceptance/test-lib/sync-tools.js

turadg

I'll fully review when it's out of draft

a3p-integration/proposals/n:upgrade-next/package.json

cloudflare-workers-and-pages · 2024-10-08T20:39:45Z

Deploying agoric-sdk with Cloudflare Pages

Latest commit:	`9901261`
Status:	✅ Deploy successful!
Preview URL:	https://8f527361.agoric-sdk.pages.dev
Branch Preview URL:	https://anil-auction-tests.agoric-sdk.pages.dev

View logs

turadg

I'll fully review when it's out of draft

a3p-integration/proposals/z:acceptance/test-lib/auction-lib.js

a3p-integration/proposals/z:acceptance/auction.test.js

Refs: Agoric/BytePitchPartnerEng#8

…st time related complexities fix: apply change requests from PR review, resolve rebase conflicts, style fixes * change `performAction.js` name to `submitBid.js` * remove `Math.round` from `scale6` * update indexing of bids and other constants in `config` object to improve readability (`auction.test.js`) * move helper functions in `auction.test.js` to `test-lib/auction-lib.js` * move `lib/vaults.mts` to `test-lib/vaults.mts` and remove empty `lib` directory * let it be known `sync-tools.js` is a stand-in code for #10171 Refs: Agoric/BytePitchPartnerEng#8 fix: style fixes fix(acceptance-auction): lint fixes

…llision Refs: Agoric/BytePitchPartnerEng#31 fix(auction-acceptance): formatting fixes fix(auction-acceptance): formatting fixes

a3p-integration/proposals/n:upgrade-next/submitBid.js

a3p-integration/proposals/z:acceptance/auction.test.js

turadg · 2024-10-30T14:41:50Z

a3p-integration/proposals/z:acceptance/auction.test.js

+  const [gov1Results, longLivingBidResults, user1Results, gov3Results, brands] =
+    await Promise.all([
+      agoric
+        .follow(


I lament that we still have to go out through the CLI to get this functionality that is in JS:

agoric-sdk/packages/agoric-cli/src/follow.js

Line 104 in 33309e0

case 'text': {

Agoric/agoric-3-proposals#98 is related.

Filed #10369

Definitely agree

Refs: Agoric/BytePitchPartnerEng#8 Refs: #10111

anilhelvaci requested a review from a team as a code owner October 5, 2024 03:10

anilhelvaci requested a review from AgoricTriage October 5, 2024 03:10

anilhelvaci marked this pull request as draft October 5, 2024 03:10

anilhelvaci requested review from turadg and Chris-Hibbert October 5, 2024 03:10

Chris-Hibbert added the force:integration Force integration tests to run on PR label Oct 7, 2024

Chris-Hibbert requested changes Oct 7, 2024

View reviewed changes

turadg reviewed Oct 8, 2024

View reviewed changes

a3p-integration/proposals/n:upgrade-next/package.json Outdated Show resolved Hide resolved

anilhelvaci mentioned this pull request Oct 8, 2024

a method for getting latest round id from a given price feed in synthetic-chain Agoric/agoric-3-proposals#182

Closed

anilhelvaci force-pushed the anil/auction-tests branch from b29d36e to 50a0e17 Compare October 8, 2024 20:39

anilhelvaci force-pushed the anil/auction-tests branch from f1dad49 to a89e8e2 Compare October 8, 2024 20:58

anilhelvaci requested review from Chris-Hibbert and turadg October 8, 2024 21:01

turadg reviewed Oct 8, 2024

View reviewed changes

Chris-Hibbert reviewed Oct 9, 2024

View reviewed changes

a3p-integration/proposals/z:acceptance/test-lib/auction-lib.js Outdated Show resolved Hide resolved

a3p-integration/proposals/z:acceptance/auction.test.js Show resolved Hide resolved

This was referenced Oct 24, 2024

Anil/price feed tests bytepitch/agoric-sdk#4

Closed

chore(acceptance-price-feeds): make sure old vats are quiescent and vaults receive quotes #10334

Open

October 18 Retro bytepitch/agoric-sdk#5

Open

chore: run auction, WIP

dedf1f3

Refs: Agoric/BytePitchPartnerEng#8

anilhelvaci force-pushed the anil/auction-tests branch from a89e8e2 to c32bcb5 Compare October 29, 2024 12:49

anilhelvaci force-pushed the anil/auction-tests branch from b714d15 to 20ca789 Compare October 30, 2024 13:15

chore(auction-acceptance): workaround for auctioneer vats vstorage co…

bf05414

…llision Refs: Agoric/BytePitchPartnerEng#31 fix(auction-acceptance): formatting fixes fix(auction-acceptance): formatting fixes

anilhelvaci force-pushed the anil/auction-tests branch from 20ca789 to bf05414 Compare October 30, 2024 13:18

anilhelvaci marked this pull request as ready for review October 30, 2024 13:22

anilhelvaci requested review from turadg and Chris-Hibbert October 30, 2024 13:55

turadg reviewed Oct 30, 2024

View reviewed changes

chore(auction-acceptance): address pr suggestions

9901261

Refs: Agoric/BytePitchPartnerEng#8 Refs: #10111

turadg mentioned this pull request Oct 31, 2024

chore: have bankSend accept from argument set to default as validator address Agoric/agoric-3-proposals#189

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

migrate/improve auction functionality tests in `z:acceptance` #10229

migrate/improve auction functionality tests in `z:acceptance` #10229

anilhelvaci commented Oct 5, 2024

turadg left a comment

cloudflare-workers-and-pages bot commented Oct 8, 2024 •

edited

Loading

turadg left a comment

turadg Oct 30, 2024

anilhelvaci Oct 31, 2024

migrate/improve auction functionality tests in z:acceptance #10229

Are you sure you want to change the base?

migrate/improve auction functionality tests in z:acceptance #10229

Conversation

anilhelvaci commented Oct 5, 2024

Description

Why is this PR a draft?

Security Considerations

Scaling Considerations

Documentation Considerations

Testing Considerations

Why not update auction params?

Who places the long living bid?

Upgrade Considerations

turadg left a comment

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Oct 8, 2024 • edited Loading

Deploying agoric-sdk with Cloudflare Pages

turadg left a comment

Choose a reason for hiding this comment

turadg Oct 30, 2024

Choose a reason for hiding this comment

anilhelvaci Oct 31, 2024

Choose a reason for hiding this comment

migrate/improve auction functionality tests in `z:acceptance` #10229

migrate/improve auction functionality tests in `z:acceptance` #10229

cloudflare-workers-and-pages bot commented Oct 8, 2024 •

edited

Loading