v0.5.13-rc1

What's Changed

• ci: Fix cache path for update-trivy-cache workflow by @SanjayVas in #1892
• feat: Add new view for retrieving additional log entries by @tristanvuong2021 in #1889
• ci: Set TCP keepalive for Bazel test workflows by @SanjayVas in #1894
• fix: check updated and created fields before assignment by @roaminggypsy in #1895
• fix: Remove if check for update time filters when adding shard ID clause in StreamMeasurements by @tristanvuong2021 in #1901
• feat: Log ComputationToken details for ALREADY_EXISITS computation. by @renjiezh in #1897
• test: prober integration test by @roaminggypsy in #1887
• feat: Add new Bigquery Table for Computation Participant Stages by @tristanvuong2021 in #1805
• feat: Add new filter for checking externalComputationId, and checking externalComputationId if view is COMPUTATION or COMPUTATION_STATS by @tristanvuong2021 in #1753
• fix: Ensure population is persisted in ModelRelease upon creation by @jojijac0b in #1914

Full Changelog: v0.5.12...v0.5.13-rc1

v0.5.12

Highlights

• The EDP simulator allows custom EventGroup metadata to be specified. See #1880

Potentially Requiring Action

• EDP Simulator EventGroup metadata is now supplied by the required --event-group-metadata option. See #1886 and the updated deployment guide.
  • The previous behavior was to set the metadata to the SyntheticEventGroupSpec specified in the --event-group-spec option.
  • The --event-group-metadata-type-url option is used to indicate the message type of the EventGroup metadata, which must be a known metadata type.
  • The simulator now has the same --known-event-group-metadata-type option as the Kingdom and Reporting API servers.

What's Changed

• feat(edp-simulator)!: Specify EventGroup metadata via command-line option by @SanjayVas in #1886

Full Changelog: v0.5.11...v0.5.12

v0.5.11

Issues Addressed

• Variance calculation throws an error when reach is 0. See #1883

Potentially Requiring Action

• The BigQuery variant of the EDP simulator has been deleted. Only the synthetic generator is supported. See #1881

What's Changed

• refactor: Prevent Get Report failure when result or variance calculation fails by @tristanvuong2021 in #1859
• fix: ensure kplus reach ratio not exceeding 1.0 by @ple13 in #1868
• ci: Use BuildBuddy for Bazel results, remote caching, and remote execution by @SanjayVas in #1840
• refactor: Graduate Report post-processing from experimental by @ple13 in #1853
• test: implement measurement system prober unit test by @roaminggypsy in #1848
• fix: Ensure update-cmms workflow can run with ci Bazel configuration by @SanjayVas in #1876
• ci: Keep Trivy cache up-to-date on default branch by @SanjayVas in #1878
• refactor!: Delete BigQuery and CSV EDP simulator variants by @SanjayVas in #1884
• fix: Allow zero reach in relative frequency variance computation. by @ple13 in #1883

Full Changelog: v0.5.10...v0.5.11

v0.5.10

Highlights

• Last Report time interval extended to end of Report. See #1869
• Liquid Legions sketch params are validated when calculating variance. See #1860
  • This should provide a clearer error message for the case hit in #1858.

Potentially Requiring Action

• During the update process, there may be a temporary performance drop on Kingdom service methods as the MeasurementsByContinuationToken index is dropped and recreated. See #1852 and #1863

What's Changed

• feat: Call noise correction python binary from kotlin. by @ple13 in #1803
• docs: Update Kingdom Deployment doc for HMSS config file. by @renjiezh in #1850
• feat: Add create_time to internal Requisition parent_measurement by @tristanvuong2021 in #1851
• refactor: Use separate queries for each bigquery metrics table by @tristanvuong2021 in #1823
• Add updatetime to Requisition by @roaminggypsy in #1847
• feat: Implement measurement prober metric creation and publication by @roaminggypsy in #1842
• ci: Read Spanner processing units from GitHub environment by @SanjayVas in #1856
• fix: Shard MeasurementsByContinuationToken index to avoid hotspotting by @SanjayVas in #1852
• feat: Throw Transient exceptions in HMSS GetCertificate call. by @renjiezh in #1857
• refactor: verify liquid legion sketch params when computing variances. by @ple13 in #1860
• fix!: Ensure that MeasurementIndexShardId is non-negative by @SanjayVas in #1863
• ControlPlane initial proto definitions. by @Marco-Premier in #1843
• test: Temporarily disable SpannerModelRolloutsServiceTest by @SanjayVas in #1872
• build: Update common-jvm dep for Spanner emulator JUnit rule by @SanjayVas in #1866
• refactor: Remove unneeded measurement view by @tristanvuong2021 in #1854
• ci: Update aquasecurity/trivy-action to 0.28.0 by @SanjayVas in #1875
• build: Update common-jvm for Spanner AsyncDatabaseClient by @SanjayVas in #1839
• feat: Extend last report time interval to end of report by @tristanvuong2021 in #1869

Full Changelog: v0.5.9...v0.5.10

v0.5.10-rc2

What's Changed

• feat: Call noise correction python binary from kotlin. by @ple13 in #1803
• docs: Update Kingdom Deployment doc for HMSS config file. by @renjiezh in #1850
• feat: Add create_time to internal Requisition parent_measurement by @tristanvuong2021 in #1851
• refactor: Use separate queries for each bigquery metrics table by @tristanvuong2021 in #1823
• Add updatetime to Requisition by @roaminggypsy in #1847
• feat: Implement measurement prober metric creation and publication by @roaminggypsy in #1842
• ci: Read Spanner processing units from GitHub environment by @SanjayVas in #1856
• fix: Shard MeasurementsByContinuationToken index to avoid hotspotting by @SanjayVas in #1852
• feat: Throw Transient exceptions in HMSS GetCertificate call. by @renjiezh in #1857
• refactor: verify liquid legion sketch params when computing variances. by @ple13 in #1860
• fix!: Ensure that MeasurementIndexShardId is non-negative by @SanjayVas in #1863

Full Changelog: v0.5.9...v0.5.10-rc2

v0.5.10-rc1

What's Changed

• feat: Call noise correction python binary from kotlin. by @ple13 in #1803
• docs: Update Kingdom Deployment doc for HMSS config file. by @renjiezh in #1850
• feat: Add create_time to internal Requisition parent_measurement by @tristanvuong2021 in #1851
• refactor: Use separate queries for each bigquery metrics table by @tristanvuong2021 in #1823
• Add updatetime to Requisition by @roaminggypsy in #1847
• feat: Implement measurement prober metric creation and publication by @roaminggypsy in #1842
• ci: Read Spanner processing units from GitHub environment by @SanjayVas in #1856
• fix: Shard MeasurementsByContinuationToken index to avoid hotspotting by @SanjayVas in #1852

Full Changelog: v0.5.9...v0.5.10-rc1

v0.5.9

Highlights

• Honest Majority Share Shuffle (HMSS) protocol is ready for use. See previously shared guide on enablement.
  • The synthetic generator EDP simulator has a new --support-hmss option that can be used to enable support for the HMSS protocol.
• Reporting ListEventGroups method is more likely to return results in first page. See #1762
• There is a Maven artifact for noise post-processing of Reports. See #1841 and #1844
  • This is currently a no-op, but provides the interface for development.
• The --event-message-descriptor-set option for the EDP simulator should no longer error when provided a valid serialized FileDescriptorSet. See #1838

Potentially Requiring Action

• There's a new WITHDRAWN Requisition state. See #1746
  • This should not affect most EDP clients as they tend to only list fulfillable Requisitions.
• The Duchy computation control server now needs access to the Duchy internal API server. See #1755
  • NetworkPolicy configurations may need to be updated. See #1824.
    • In the Halo dev configuration, this means updating the <duchy-id>-internal-api-server NetworkPolicy to allow ingress from the <duchy-id>-computation-control-server Deployment.
  • The computation control server requires the --computations-service-target and --computations-service-cert-host options.
• The wfa.measurement.internal.kingdom.HmssProtocolConfigConfig message definition has changed. See #1737

What's Changed

• fix[edp-simulator]: Refuse Requisition with invalid EventGroup name by @SanjayVas in #1758
• refactor[edp-simulator]: Consolidate logging of Requisition refusal by @SanjayVas in #1759
• feat!: Update cross-media-measurement-api dep for Requisition.State.WITHDRAWN by @SanjayVas in #1746
• feat: Enable CloudWatch Observability EKS add-on by @SanjayVas in #1763
• feat: Increase chance of getting Event Groups from Reporting Server ListEventGroups by @tristanvuong2021 in #1762
• fix: Update Kingdom's duchy id config to select fulfillingDuchyId for HMSS by @renjiezh in #1737
• refactor: Move dp consistency to cmm. by @ple13 in #1771
• feat: Fulfill hmss requisition via the share shuffle library. by @ple13 in #1748
• build!: Update JVM build target to 11 by @tristanvuong2021 in #1766
• fix: Rollback JVM target upgrade by @tristanvuong2021 in #1776
• fix: Resolve k8s version mismatch in correctness test by @renjiezh in #1775
• fix: Fix type_url for EncryptionPublicKey. by @renjiezh in #1774
• fix: Fix Mill image name on aws. by @renjiezh in #1773
• fix!: Ensure default network policies are created by @SanjayVas in #1768
• fix!: Add missing NetworkPolicy for Mill job scheduler by @SanjayVas in #1769
• feat!: Update JVM build target to 11 by @tristanvuong2021 in #1778
• chore: Update EKS clusters to 1.29 in Terraform configs by @SanjayVas in #1789
• Update skecth_params in ro_llv2_protocol_config_config by @renjiezh in #1784
• build: Upgrade Kubernetes client to 21.0.1 by @tristanvuong2021 in #1779
• Fix mismatch vid range of synthetic population specs. by @renjiezh in #1783
• feat!:Add rpc GetStage in duchy control service. by @renjiezh in #1755
• style: Upgrade api-linter to v1.67.2 by @SanjayVas in #1782
• feat: Build and test dp consistency with bazel. by @ple13 in #1787
• feat: set up measurement prober flags by @roaminggypsy in #1791
• build: Update common-jvm dep for Maven BOMs by @SanjayVas in #1790
• feat: Mill skips advancing stage when the next stage is in a future stage by @renjiezh in #1793
• refactor: Extract Details message type definitions in Kingdom internal API by @SanjayVas in #1798
• feat: Expose panel exchange flags via CUE tags by @robinsons in #1795
• refactor: Use protobuf types in Kingdom Spanner DB by @SanjayVas in #1799
• feat: Write Metrics to BigQuery by @tristanvuong2021 in #1696
• refactor: Drop JSON debugging columns from Kingdom Spanner DB by @SanjayVas in #1800
• fix: Disable insecure kubelet readonly port on GKE node pools by @SanjayVas in #1809
• refactor: Use built-in Spanner client library functions for protobufs by @SanjayVas in #1808
• feat: Regard some DEADLINE_EXCEEDED errors as transient in MillBase by @renjiezh in #1704
• Update Duchy control service to tolerate certain stage mismathes. by @renjiezh in #1801
• fix: Switch to project id variable from hard coded project id by @tristanvuong2021 in #1806
• Add timestamp filter to ListMeasurementsRequest and ListMeasurementsPageToken by @roaminggypsy in #1807
• feat: Add incrementBy to FrequencyVectorBuilder by @kungfucraig in #1817
• fix: Add Duchy ControlService to InternalApiServer's NetworkPolicy allow list. by @renjiezh in #1824
• build: Update common-jvm dep to 0.92.0 by @SanjayVas in #1816
• fix: Address compile warnings on custom Truth subjects by @SanjayVas in #1818
• build: Update RBE configs for Bazel 7.3.1 by @SanjayVas in #1825
• test: Add explicit unit test and integration test for end of Report time interval by @tristanvuong2021 in #1822
• Enable HMSS in kingdom with k8s test for both MPC protocols by @renjiezh in #1813
• Add 'created after' filter to ListMeasurements and StreamMeasurements by @roaminggypsy in #1828
• build: Update common-jvm dep to update rules_oci dep by @SanjayVas in #1819
• fix: Fix FrequencyVectorBuilder performance issue by range count. by @renjiezh in #1832
• build: Update common-jvm dep for thread pool instrumentation by @SanjayVas in #1831
• feat: Implement prober's new measurement creation by @roaminggypsy in #1796
• refactor!: Improve performance of StreamRequisitions by @tristanvuong2021 in #1837
• fix: Parse --event-message-descriptor-set as binary protobuf by @SanjayVas in #1838
• feat: Export no-op report processor maven artifact. by @ple13 in #1841
• docs: correct postprocessing-v2alpha maven artifact name. by @ple13 in https://github.com/wo...

v0.5.9-rc4

What's Changed

• fix[edp-simulator]: Refuse Requisition with invalid EventGroup name by @SanjayVas in #1758
• refactor[edp-simulator]: Consolidate logging of Requisition refusal by @SanjayVas in #1759
• feat!: Update cross-media-measurement-api dep for Requisition.State.WITHDRAWN by @SanjayVas in #1746
• feat: Enable CloudWatch Observability EKS add-on by @SanjayVas in #1763
• feat: Increase chance of getting Event Groups from Reporting Server ListEventGroups by @tristanvuong2021 in #1762
• fix: Update Kingdom's duchy id config to select fulfillingDuchyId for HMSS by @renjiezh in #1737
• refactor: Move dp consistency to cmm. by @ple13 in #1771
• feat: Fulfill hmss requisition via the share shuffle library. by @ple13 in #1748
• build!: Update JVM build target to 11 by @tristanvuong2021 in #1766
• fix: Rollback JVM target upgrade by @tristanvuong2021 in #1776
• fix: Resolve k8s version mismatch in correctness test by @renjiezh in #1775
• fix: Fix type_url for EncryptionPublicKey. by @renjiezh in #1774
• fix: Fix Mill image name on aws. by @renjiezh in #1773
• fix!: Ensure default network policies are created by @SanjayVas in #1768
• fix!: Add missing NetworkPolicy for Mill job scheduler by @SanjayVas in #1769
• feat!: Update JVM build target to 11 by @tristanvuong2021 in #1778
• chore: Update EKS clusters to 1.29 in Terraform configs by @SanjayVas in #1789
• Update skecth_params in ro_llv2_protocol_config_config by @renjiezh in #1784
• build: Upgrade Kubernetes client to 21.0.1 by @tristanvuong2021 in #1779
• Fix mismatch vid range of synthetic population specs. by @renjiezh in #1783
• feat!:Add rpc GetStage in duchy control service. by @renjiezh in #1755
• style: Upgrade api-linter to v1.67.2 by @SanjayVas in #1782
• feat: Build and test dp consistency with bazel. by @ple13 in #1787
• feat: set up measurement prober flags by @roaminggypsy in #1791
• build: Update common-jvm dep for Maven BOMs by @SanjayVas in #1790
• feat: Mill skips advancing stage when the next stage is in a future stage by @renjiezh in #1793
• refactor: Extract Details message type definitions in Kingdom internal API by @SanjayVas in #1798
• feat: Expose panel exchange flags via CUE tags by @robinsons in #1795
• refactor: Use protobuf types in Kingdom Spanner DB by @SanjayVas in #1799
• feat: Write Metrics to BigQuery by @tristanvuong2021 in #1696
• refactor: Drop JSON debugging columns from Kingdom Spanner DB by @SanjayVas in #1800
• fix: Disable insecure kubelet readonly port on GKE node pools by @SanjayVas in #1809
• refactor: Use built-in Spanner client library functions for protobufs by @SanjayVas in #1808
• feat: Regard some DEADLINE_EXCEEDED errors as transient in MillBase by @renjiezh in #1704
• Update Duchy control service to tolerate certain stage mismathes. by @renjiezh in #1801
• fix: Switch to project id variable from hard coded project id by @tristanvuong2021 in #1806
• Add timestamp filter to ListMeasurementsRequest and ListMeasurementsPageToken by @roaminggypsy in #1807
• feat: Add incrementBy to FrequencyVectorBuilder by @kungfucraig in #1817
• fix: Add Duchy ControlService to InternalApiServer's NetworkPolicy allow list. by @renjiezh in #1824
• build: Update common-jvm dep to 0.92.0 by @SanjayVas in #1816
• fix: Address compile warnings on custom Truth subjects by @SanjayVas in #1818
• build: Update RBE configs for Bazel 7.3.1 by @SanjayVas in #1825
• test: Add explicit unit test and integration test for end of Report time interval by @tristanvuong2021 in #1822
• Enable HMSS in kingdom with k8s test for both MPC protocols by @renjiezh in #1813
• Add 'created after' filter to ListMeasurements and StreamMeasurements by @roaminggypsy in #1828
• build: Update common-jvm dep to update rules_oci dep by @SanjayVas in #1819
• fix: Fix FrequencyVectorBuilder performance issue by range count. by @renjiezh in #1832
• build: Update common-jvm dep for thread pool instrumentation by @SanjayVas in #1831
• feat: Implement prober's new measurement creation by @roaminggypsy in #1796
• refactor!: Improve performance of StreamRequisitions by @tristanvuong2021 in #1837
• fix: Parse --event-message-descriptor-set as binary protobuf by @SanjayVas in #1838
• feat: Export no-op report processor maven artifact. by @ple13 in #1841

New Contributors

• @roaminggypsy made their first contribution in #1791

Full Changelog: v0.5.8...v0.5.9-rc4

v0.5.9-rc3

Highlight

TODO

What's Changed

TODO

What's Changed

• fix[edp-simulator]: Refuse Requisition with invalid EventGroup name by @SanjayVas in #1758
• refactor[edp-simulator]: Consolidate logging of Requisition refusal by @SanjayVas in #1759
• feat!: Update cross-media-measurement-api dep for Requisition.State.WITHDRAWN by @SanjayVas in #1746
• feat: Enable CloudWatch Observability EKS add-on by @SanjayVas in #1763
• feat: Increase chance of getting Event Groups from Reporting Server ListEventGroups by @tristanvuong2021 in #1762
• fix: Update Kingdom's duchy id config to select fulfillingDuchyId for HMSS by @renjiezh in #1737
• refactor: Move dp consistency to cmm. by @ple13 in #1771
• feat: Fulfill hmss requisition via the share shuffle library. by @ple13 in #1748
• build!: Update JVM build target to 11 by @tristanvuong2021 in #1766
• fix: Rollback JVM target upgrade by @tristanvuong2021 in #1776
• fix: Resolve k8s version mismatch in correctness test by @renjiezh in #1775
• fix: Fix type_url for EncryptionPublicKey. by @renjiezh in #1774
• fix: Fix Mill image name on aws. by @renjiezh in #1773
• fix!: Ensure default network policies are created by @SanjayVas in #1768
• fix!: Add missing NetworkPolicy for Mill job scheduler by @SanjayVas in #1769
• feat!: Update JVM build target to 11 by @tristanvuong2021 in #1778
• chore: Update EKS clusters to 1.29 in Terraform configs by @SanjayVas in #1789
• Update skecth_params in ro_llv2_protocol_config_config by @renjiezh in #1784
• build: Upgrade Kubernetes client to 21.0.1 by @tristanvuong2021 in #1779
• Fix mismatch vid range of synthetic population specs. by @renjiezh in #1783
• feat!:Add rpc GetStage in duchy control service. by @renjiezh in #1755
• style: Upgrade api-linter to v1.67.2 by @SanjayVas in #1782
• feat: Build and test dp consistency with bazel. by @ple13 in #1787
• feat: set up measurement prober flags by @roaminggypsy in #1791
• build: Update common-jvm dep for Maven BOMs by @SanjayVas in #1790
• feat: Mill skips advancing stage when the next stage is in a future stage by @renjiezh in #1793
• refactor: Extract Details message type definitions in Kingdom internal API by @SanjayVas in #1798
• feat: Expose panel exchange flags via CUE tags by @robinsons in #1795
• refactor: Use protobuf types in Kingdom Spanner DB by @SanjayVas in #1799
• feat: Write Metrics to BigQuery by @tristanvuong2021 in #1696
• refactor: Drop JSON debugging columns from Kingdom Spanner DB by @SanjayVas in #1800
• fix: Disable insecure kubelet readonly port on GKE node pools by @SanjayVas in #1809
• refactor: Use built-in Spanner client library functions for protobufs by @SanjayVas in #1808
• feat: Regard some DEADLINE_EXCEEDED errors as transient in MillBase by @renjiezh in #1704
• Update Duchy control service to tolerate certain stage mismathes. by @renjiezh in #1801
• fix: Switch to project id variable from hard coded project id by @tristanvuong2021 in #1806
• Add timestamp filter to ListMeasurementsRequest and ListMeasurementsPageToken by @roaminggypsy in #1807
• feat: Add incrementBy to FrequencyVectorBuilder by @kungfucraig in #1817
• fix: Add Duchy ControlService to InternalApiServer's NetworkPolicy allow list. by @renjiezh in #1824
• build: Update common-jvm dep to 0.92.0 by @SanjayVas in #1816
• fix: Address compile warnings on custom Truth subjects by @SanjayVas in #1818
• build: Update RBE configs for Bazel 7.3.1 by @SanjayVas in #1825
• test: Add explicit unit test and integration test for end of Report time interval by @tristanvuong2021 in #1822
• Enable HMSS in kingdom with k8s test for both MPC protocols by @renjiezh in #1813

New Contributors

• @roaminggypsy made their first contribution in #1791

Full Changelog: v0.5.8...v0.5.9-rc3

v0.5.9-rc2

Highlight

TODO

What's Changed

TODO

What's Changed

• fix[edp-simulator]: Refuse Requisition with invalid EventGroup name by @SanjayVas in #1758
• refactor[edp-simulator]: Consolidate logging of Requisition refusal by @SanjayVas in #1759
• feat!: Update cross-media-measurement-api dep for Requisition.State.WITHDRAWN by @SanjayVas in #1746
• feat: Enable CloudWatch Observability EKS add-on by @SanjayVas in #1763
• feat: Increase chance of getting Event Groups from Reporting Server ListEventGroups by @tristanvuong2021 in #1762
• fix: Update Kingdom's duchy id config to select fulfillingDuchyId for HMSS by @renjiezh in #1737
• refactor: Move dp consistency to cmm. by @ple13 in #1771
• feat: Fulfill hmss requisition via the share shuffle library. by @ple13 in #1748
• build!: Update JVM build target to 11 by @tristanvuong2021 in #1766
• fix: Rollback JVM target upgrade by @tristanvuong2021 in #1776
• fix: Resolve k8s version mismatch in correctness test by @renjiezh in #1775
• fix: Fix type_url for EncryptionPublicKey. by @renjiezh in #1774
• fix: Fix Mill image name on aws. by @renjiezh in #1773

Full Changelog: v0.5.8...v0.5.9-rc2