Feature enhancements

What's Changed

• Fixed a race condition and client privacy.json issue. (#1164) by @yhwen in #1172

Full Changelog: 2.2.2...2.2.3

2.2.2

What's Changed

• Add list of publication and talks [skip ci] by @holgerroth in #1090
• fix paper order [skip ci] by @holgerroth in #1092
• update model selection main by @holgerroth in #1134
• Cherry pick commits for documentation updates by @nvkevlu in #1153

Full Changelog: 2.2.1...2.2.2

NVFlare 2.2.1 Release

Feature Highlights

FL Simulator -- A lightweight simulator of a running NVFLARE FL deployment. It allows researchers to test and debug their application without provisioning a real project. The FL jobs run on a server and multiple clients in the same process but in a similar way to how it would run in a real deployment. Researchers can quickly build out new components and jobs that can then be directly used in a real production deployment.

FLARE Dashboard NVFLARE's web UI. In its initial incarnation, the Flare Dashboard is used to help project setup, user registration, startup kits distribution and dynamic provisions. Dashboard setup and apis can be found here

Site-policy management -- Prior to NVFLARE 2.2, all policies (resource management, authorization and privacy protection, logging configurations) can only be defined by the Project Admin during provision time; and authorization policies are centrally enforced by the FL Server. NVFLARE 2.2 makes it possible for each site to define its own policies in the following areas:

• Resource Management: the configuration of system resources that are solely the decisions of local IT.
• Authorization Policy: local authorization policy that determines what a user can or cannot do on the local site. see related Federated Authorization
• Privacy Policy: local policy that specifies what types of studies are allowed and how to add privacy protection to the learning results produced by the FL client on the local site.
• Logging Configuration: each site can now define its own logging configuration for system generated log messages.

Federated XGBoost -- We developed federated XGBoost for data scientists to perform machine learning on tabular data with popular tree-based method. In this release, we provide several approaches for the horizontal federated XGBoost algorithms.

• Histogram-based Collaboration -- leverages recently released (XGBoost 1.7.0) federated versions of open-source XGBoost histogram-based distributed training algorithms, achieving identical results as centralized training (trees trained on global data information).
• Tree-based Collaboration -- individual trees are independently trained on each client's local data without aggregating the global sample gradient histogram information. Trained trees are collected and passed to the server / other clients for aggregation and further boosting rounds.

Federated Statistics -- built-in federated statistics operators that can generate global statistics based on local client side statistics. The results, for all features of all datasets at all sites as well as global aggregates, can be visualized via the visualization utility in the notebook.

MONAI Integration In 2.2 release, we provided two implementations by leveraging MONAI Bundle.

MONAI ClientAlgo Integration -- enable running MONAI bundles directly in a federated setting using NVFLARE
MONAI ClientAlgoStats Integration -- through NVFLARE Federated Statistics we can generate, compare and visualize all clients' data statistics generated from MONAI summary statistics

Tools and Production Support

Improved POC command
Dynamic Provision
Docker Compose
Preflight Check

Migrations tips
To migrate from releases prior to 2.2.1, here are few notes that might help migrate to 2.2.1.

What's Changed

• Pick up main changes by @YuanTingHsieh in #692
• Change np model persistor log level by @YuanTingHsieh in #693
• Remove env command in fl admin api by @YuanTingHsieh in #696
• Change print to logger.info in client executor by @YuanTingHsieh in #699
• Add list_files and download_job for single file by @nvkevlu in #698
• Add provisioned setting to integration tests and re-factor by @YuanTingHsieh in #697
• Add dns entries into ci scripts by @YuanTingHsieh in #700
• Add citation file by @YuanTingHsieh in #702
• Add doi to citation file by @YuanTingHsieh in #703
• Fix blossom CI/CD tests by @YuanTingHsieh in #704
• Remove self-hosted runners and use only blossom ci by @YuanTingHsieh in #707
• Fix setup.py by @YuanTingHsieh in #709
• Clean up workflow implementations, use self._engine by @YuanTingHsieh in #712
• Fix typos in tf related codes in examples and tests by @YuanTingHsieh in #713
• Docker compose by @IsaacYangSLA in #714
• Fix the shared volume settings among servers in compose.yaml by @IsaacYangSLA in #716
• Add cov.xml to unit test script by @YuanTingHsieh in #715
• Wait for cleanup process before start next test case by @YuanTingHsieh in #726
• Update example structure from app to job by @YuanTingHsieh in #724
• 625 Add monai bundle example by @yiheng-wang-nv in #721
• Fix overseer code style and issues by @YuanTingHsieh in #729
• Add research and integration folder by @YuanTingHsieh in #732
• Clean up ClientRunner and Communicator by @YuanTingHsieh in #720
• Add two command options to add user/client by @IsaacYangSLA in #723
• Helm chart by @IsaacYangSLA in #725
• Add nvflare poc command by @chesterxgchen in #739
• Clean up overseer by @YuanTingHsieh in #741
• Add org to cert by @IsaacYangSLA in #743
• Add two functions to retrieve roles and org from cert by @IsaacYangSLA in #744
• rename README.MD -> README.md by @holgerroth in #748
• Add Collective communication and hello-mpi example by @YuanTingHsieh in #719
• nvflare poc syntax update and add NVFLARE_POC_WORKSPACE env variable by @chesterxgchen in #751
• 1. add fed_sm and auto_fed_rl research directories with index.html re… by @chesterxgchen in #754
• Remove doc build in github by @YuanTingHsieh in #758
• add client GPU assignment for POC command by @chesterxgchen in #759
• Add preflight check scripts by @YuanTingHsieh in #742
• Fix package checker utils test by @YuanTingHsieh in #762
• Move GPU utils out of poc_commands for later reuse by @YuanTingHsieh in #765
• Update integration test validator log level by @YuanTingHsieh in #768
• handle no gpu or no nvidia-smi installed cases by @chesterxgchen in #773
• consolidate NVFLARE CLI by @chesterxgchen in #774
• Federated Statistics Operator with Data Frame Statistics example. by @chesterxgchen in #750
• Add report resources command and fix resource manage issues by @YuanTingHsieh in #766
• Fix License check issue by @chesterxgchen in #779
• Added options to disable old TLS versions by @nvidianz in #775
• XGBoost with cyclic and bagging training by @ZiyueXu77 in #740
• Only restore the FLcomponent from the snapshot. by @yhwen in #796
• Add dashboard cli by @IsaacYangSLA in #801
• Clean up fuel utils and client executor by @YuanTingHsieh in #761
• Include org info in sub_start.sh, compose.yaml, helm chart and other … by @IsaacYangSLA in #808
• Local folder from provisioning by @IsaacYangSLA in #800
• link to temporary Auto-FedRL repo by @holgerroth in https://github.com/N...

2.2.0

What's Changed

• Fix the protobuf version to solve incompatibility by @IsaacYangSLA in #738
• Update 2.1 ci by @YuanTingHsieh in #747
• Replacing Pickle with FOBS by @nvidianz in #746
• Added msgpack to setup by @nvidianz in #756
• Cherry-Pick changes to 2.2. for 1) Fed Stats 2) nvflare poc by @chesterxgchen in #780
• MONAI-FL integration into 2.2 by @holgerroth in #788
• update submit model to follow monai convention by @holgerroth in #795
• Simulator for 2.2.0 release by @yhwen in #792
• MONAI-FL add 3rd party licenses by @holgerroth in #797
• Fed Statistic Updates for 2.2 branch by @chesterxgchen in #804
• Fixed typos and return run status for simulator by @yhwen in #803
• simulator nvflare CLI command by @chesterxgchen in #805
• Monai_nvflare setup update [skip ci] by @chesterxgchen in #820
• setup the 2.2.0 nightly build version different from Dev branch [skip ci] by @chesterxgchen in #821
• add missing init.py file [skip ci] by @chesterxgchen in #826
• Fed stats 3 [skip ci] by @chesterxgchen in #815
• add nvflare command to 2.2 [skip ci] by @chesterxgchen in #832
• Addressed several QA issues. by @yhwen in #813
• Optimize the SimulatorRunner config. by @yhwen in #816
• add MONAI bundle persistor; update readme to include simulator commands by @holgerroth in #819
• 2.2 branch 1) move Fed stats example to store data outside POC workspace 2) rename nflare.py to cli.py [skip ci] by @chesterxgchen in #845
• Fix simulator bug [skip ci] by @chesterxgchen in #846
• add missing Init files [skip ci] by @chesterxgchen in #847
• add make sure it can include log.config file in 2.2 wheel [skip ci] by @chesterxgchen in #852
• Fix issue of error in fl_addmin_api_runner [skip ci] by @nvkevlu in #841
• 2.2 branch Fix pytorch-ignite and monai version [skip ci] by @chesterxgchen in #854
• 2.2 fed stats simulator [skip ci] by @chesterxgchen in #855
• Ported decomposer for OrderedDict from dev to 2.2 [skip ci] by @nvidianz in #853
• update monai_nvflare dependecies by @holgerroth in #857
• Update sim arguments in monai-fl example [skip ci] by @holgerroth in #860
• Fixed several issues in simulator by @yhwen in #848
• Fixed the FLCommunicationError log error. by @yhwen in #858
• Fixed some log issues and a timing issue for CTR-C kill the simulator. by @yhwen in #866
• [2.2 brach] nvflare simulator won't exit after run [skip ci] by @chesterxgchen in #870
• 2.2_Branch: fix POC command bug [skip ci] by @chesterxgchen in #879
• Break the recursion in task result by @nvidianz in #880
• Monai : Update monai to 1.0.0.rc2 and NVFLARE==2.2.0rc1 [skip ci] by @chesterxgchen in #884
• remove pytorch-ignite license [skip ci] by @holgerroth in #883

Full Changelog: 2.1.2...2.2.0

2.2.0rc1

What's Changed

• Fix the protobuf version to solve incompatibility by @IsaacYangSLA in #738
• Update 2.1 ci by @YuanTingHsieh in #747
• Replacing Pickle with FOBS by @nvidianz in #746
• Added msgpack to setup by @nvidianz in #756
• Cherry-Pick changes to 2.2. for 1) Fed Stats 2) nvflare poc by @chesterxgchen in #780
• MONAI-FL integration into 2.2 by @holgerroth in #788
• update submit model to follow monai convention by @holgerroth in #795
• Simulator for 2.2.0 release by @yhwen in #792
• MONAI-FL add 3rd party licenses by @holgerroth in #797
• Fed Statistic Updates for 2.2 branch by @chesterxgchen in #804
• Fixed typos and return run status for simulator by @yhwen in #803
• simulator nvflare CLI command by @chesterxgchen in #805
• Monai_nvflare setup update [skip ci] by @chesterxgchen in #820
• setup the 2.2.0 nightly build version different from Dev branch [skip ci] by @chesterxgchen in #821
• add missing init.py file [skip ci] by @chesterxgchen in #826
• Fed stats 3 [skip ci] by @chesterxgchen in #815
• add nvflare command to 2.2 [skip ci] by @chesterxgchen in #832
• Addressed several QA issues. by @yhwen in #813
• Optimize the SimulatorRunner config. by @yhwen in #816
• add MONAI bundle persistor; update readme to include simulator commands by @holgerroth in #819
• 2.2 branch 1) move Fed stats example to store data outside POC workspace 2) rename nflare.py to cli.py [skip ci] by @chesterxgchen in #845
• Fix simulator bug [skip ci] by @chesterxgchen in #846
• add missing Init files [skip ci] by @chesterxgchen in #847
• add make sure it can include log.config file in 2.2 wheel [skip ci] by @chesterxgchen in #852
• Fix issue of error in fl_addmin_api_runner [skip ci] by @nvkevlu in #841
• 2.2 branch Fix pytorch-ignite and monai version [skip ci] by @chesterxgchen in #854
• 2.2 fed stats simulator [skip ci] by @chesterxgchen in #855
• Ported decomposer for OrderedDict from dev to 2.2 [skip ci] by @nvidianz in #853
• update monai_nvflare dependecies by @holgerroth in #857
• Update sim arguments in monai-fl example [skip ci] by @holgerroth in #860
• Fixed several issues in simulator by @yhwen in #848
• Fixed the FLCommunicationError log error. by @yhwen in #858
• Fixed some log issues and a timing issue for CTR-C kill the simulator. by @yhwen in #866
• [2.2 brach] nvflare simulator won't exit after run [skip ci] by @chesterxgchen in #870
• 2.2_Branch: fix POC command bug [skip ci] by @chesterxgchen in #879
• Break the recursion in task result by @nvidianz in #880

Full Changelog: 2.1.2...2.2.0rc1

Update dependency to fix protobuf version incompatibility

2.1.3

Fix the protobuf version to solve incompatibility

Minor fixes

Please check this document for more information.

What's Changed

• Remove last number of version in doc content by @YuanTingHsieh in #687
• Merged security fix to admin conn by @nvidianz in #688

Full Changelog: 2.1.1...2.1.2

NVFlare 2.1 Release

Please see the document for more details.

What's Changed

• Fix create_analytic_dxo method calling by @YuanTingHsieh in #141
• Fix poc permission by @IsaacYangSLA in #143
• Add protection to prevent loop stream logging. by @yhwen in #146
• Remove the log streaming codes. by @yhwen in #159
• Enhanced the client_runner to handle the abort command, return Return… by @yhwen in #161
• Enhanced the DXO data and meta default to empty dict. by @yhwen in #169
• added hello-pt-tb example with Learner API and tb streaming by @SYangster in #166
• Fix issue 156 on numpy default casting by @IsaacYangSLA in #172
• Coding style by @IsaacYangSLA in #171
• add aggregation helper class by @holgerroth in #165
• Add to the documentation to clarify about specific cases by @nvkevlu in #175
• Fix private/fed/server docstrings by @YuanTingHsieh in #174
• Improve docstrings and coding style for fuel by @nvkevlu in #176
• Private docstring fixes by @yhwen in #181
• Standardize error messages by @SYangster in #178
• Update admin client command line interface to based on AdminAPI by @YuanTingHsieh in #162
• Fixe cross_validation locate_model should ony return the model weight… by @yhwen in #186
• Fix CredentialType by @YuanTingHsieh in #188
• improve docstrings for apis and widgets by @SYangster in #187
• Fix show_stats command by @YuanTingHsieh in #191
• Fix all issues of isort/black after including nvflare/private folder by @IsaacYangSLA in #154
• Add default FL app validator by @YuanTingHsieh in #179
• Fix admin shutdown client issue by @YuanTingHsieh in #194
• Add information in documentation to help explain fed events by @nvkevlu in #195
• Fix full model shareable generator by @YuanTingHsieh in #196
• Quit admin after timeout dev by @SYangster in #244
• HA: overseer/overseer_agent/overseer_agent_app/overseer_agent_gui by @IsaacYangSLA in #246
• Ha support by @yhwen in #261
• add dummy overseer agent by @SYangster in #262
• adding filesystem and s3 storage and persistor implementations by @SYangster in #255
• updated cli.py to enable HA. by @yhwen in #263
• remove old nfs_storage (filesystem_storage works for nfs) by @SYangster in #264
• Dev 2.1 cli change by @yhwen in #265
• Support HA in POC. by @yhwen in #270
• Add initialize by @IsaacYangSLA in #267
• Add json files and update templates by @IsaacYangSLA in #271
• Fix poc client start by @IsaacYangSLA in #272
• get open port for client_executor. by @yhwen in #275
• Minor changes to master-template.yml to support older versions of bash by @kkersten in #278
• Update with fed_admin config by @nvkevlu in #274
• Change fl_admin launch script to use fed_admin.json config for poc mode by @nvkevlu in #283
• update fed_admin_HA.json in poc to not use SSL by @nvkevlu in #284
• Update dev by @YuanTingHsieh in #285
• Fix intime accumulated aggregators and add tests by @YuanTingHsieh in #288
• Change log levels for some logs in private by @YuanTingHsieh in #289
• Fix license issue by @YuanTingHsieh in #287
• support SAG HA. by @yhwen in #290
• Bugfix HE aggregator by @holgerroth in #292
• Fix isort/black/flake8 issues and CI/CD by @YuanTingHsieh in #296
• Add props to SP dataclass by @IsaacYangSLA in #304
• Persist wf_index in server_runner by @YuanTingHsieh in #305
• make task name configurable by @holgerroth in #254
• Refactor overseer_agent to AdminAPI from cli, add ha commands by @nvkevlu in #302
• Enable poc command in 'pip install -e .' environment. by @IsaacYangSLA in #318
• support data kind WEIGHTS in HE by @holgerroth in #326
• Fix set_run_number for snapshot restore. by @yhwen in #312
• Clean up communicator by @YuanTingHsieh in #315
• Re-organise widgets and handlers by @YuanTingHsieh in #309
• Fix FLComponent docstring typo by @YuanTingHsieh in #329
• Add event_type to AnalyticsSender by @YuanTingHsieh in #330
• disable byoc in provisioning by @holgerroth in #327
• Clean up codes in nvflare/private/fed/app by @YuanTingHsieh in #286
• add storage tests by @SYangster in #325
• Add persist/restore to cyclic workflow by @IsaacYangSLA in #331
• Initial checkin of code with job management and admin commands by @nvkevlu in #332
• 343 fixes svt_privacy by @wyli in #348
• Improve fed_(server|client).json readability by @IsaacYangSLA in #347
• Storage improvements by @SYangster in #341
• move job def related into apis, and add get_apps by @nvkevlu in #350
• Fix persistable docstring by @YuanTingHsieh in #351
• Remove pickle dump/load calls from handling signature file by @IsaacYangSLA in #361
• Show warning after running poc command by @IsaacYangSLA in #363
• Rename study to project in all files in lighter by @IsaacYangSLA in #365
• Runner process by @yhwen in #369
• Make study spec consistent with study tool by @IsaacYangSLA in #359
• study manager by @IsaacYangSLA in #368
• Add study config user app by @IsaacYangSLA in #345
• Requires implementation of fire_event method for ServerEngineSpec by @YuanTingHsieh in #352
• Fix import issues by @IsaacYangSLA in #373
• Yaml loader known to unsafe. Switch to yaml's safe_loader to reduce … by @IsaacYangSLA in #380
• Fix some uncaught yaml loader codes. Replace them with safe_load by @IsaacYangSLA in #383
• Remove refresh api as it wipes out all records. by @IsaacYangSLA in #385
• Fix a few issues on study manager by @IsaacYangSLA in #375
• removed pickle from storage by @SYangster in #381
• Remove pickle from job commands by @nvkevlu in #382
• Update docstrings by @YuanTingHsieh in #390
• Multi run support by @yhwen in #388
• add to docstrings and add method to get app content from Job by @nvkevlu in #394
• Fix job def and manager specs by @YuanTingHsieh in #396
• fix upload and download job commands, add clone_job by @nvkevlu in #397
• fixes the issue with keys in meta not being str by @nvkevlu in #399
• Add Job scheduler by @YuanTingHsieh in #400
• update the commands in the FLAdminAPI by @nvkevlu in #401
• Fix gunicorn cert issues by @IsaacYangSLA in #402
• Fix docstrings by @YuanTingHsieh in #408
• The privilege yaml file must be signed and loaded by secure content s… by @IsaacYangSLA in #403
• Add system state into overseer reply. by @IsaacYangSLA in #410
• Re-factor unit test's fo...

2.0.16

Fixes #301 by lowering max connections to 16 so max mem usage will be…

2.0.15

Fix isort issue (#297)