Local simulation instructions before deploying to the real world and questions thereof`

[AkshayBhardwajMN]

Hi all

We need to deploy a federated learning server (microservice) to collaborate with other institutions. However, our own compute environment is airtight (no access to the internet) due to regulatory requirements. Therefore, we need to develop a prototype solution outside of this environment, however, the prototype has to be more thorough than a simple simulation as listed on the OpenFL site.

Please let me know if the following workflow is possible:

1. Go to WSL2 on a local machine
2. Create a docker container to host the OpenFL server
3. Create either two docker containers or virtual environments (whichever works better) to simulate clients (2 external sites, 1 main site that is a participant but also the controller of this server)
4. The goal is to execute provisioning as you would in a real world setting (login, PKI etc), and to capture exactly what type of data is leaving the sites and entering the server, and vice versa e.g. contents of protobuff files
5. Is OpenFL capable of running on scikit learn too? On the documentation site, I can only see TF and Pytorch
6. From an end user perspective (the data scientists who will be using the OpenFL server from their respective computes); what will the control site need to create and share with them so they can convert their python codes to FL codes by simple availing the openFL API (python) e.g. connect to the server using username/password/cryptographic key etc
7. How can we manage the server i.e. activity profiling, managing different projects (and making sure different projects are not able to look into another site's other projects that they're not a part of), determining model convergence, and retire/decommision client sites once they're no longer a part of an active project?

We're trying to decide between NFLARE & OpenFL, and would like a lean low overhead solution that meets our security architecture.

Thank you

[mansishr]

Hi @AkshayBhardwajMN, thanks for your interest in OpenFL!

1-4: The described workflow is possible through the use of OpenFL-aggregator(server) can be hosted in one docker container and collaborators(clients) can be hosted in different containers. The simulation happens through GRPC by establishing a secure connection using mTLS after PKI exchange. The contents that are transferred are actually numpy tensors ( TensorKeys holding tensor name, origin, fl_round, report, tags, numpy array). Let us know if you'd like more information/help in setting up the system.
5: Yes, OpenFL is framework agnostic and can be run with Scikit learn too.
6: Data scientists can directly plug in different models/datasets and change hyperparameters for experimentation for Director-envoy API and Workflow API. For Taskrunner API (which you might want to use to run participants within containers), a federated learning plan (FL plan) is created and workspace is formed which is then distributed manually. So, you'll need to modify the FL plan along with the python code that defines the model and data. We have preexisting templates to work from where FL plan and source code can be changed easily.
7. Some of these features are supported for Director-envoy flow but not for Taskrunner API yet.