Bayer’s Science@Scale Platform and the Role For NVIDIA Federated Learning

By Allison Proffitt 

April 15, 2021 | AI is developing in healthcare, explained Kimberly Powell, VP of healthcare at NVIDIA in a press briefing for this week’s NVIDIA GTC event. And while Clara Discovery, NVIDIA’s computational platform for healthcare, has more than 40 pre-trained models, Powell does not expect those to be sufficient.

“We know that healthcare environments are continuously changing, and healthcare data is spread all over the world. In order to build robust AI applications, we need to adapt to the changing environments and learn from diverse data. We need an AI training framework that can live on the edge and learn from local data,” Powell said.

At the GTC event earlier this week, NVIDIA announced the next generation of federated learning: Clara 4.0 with global orchestration of edge clients, and an addition to privacy-preserving homomorphic encryption, Powell explained.

Yesterday, David Ruau, VP, head of Global Data Assets and Decision Science at Bayer, outlined Bayer’s digital transformation and their work with NVIDIA to incorporate Clara Federated Learning.

Science@Scale at Bayer

Getting to federated learning began with Bayer’s earliest efforts at digital transformation, Ruau said. Companies’ digital transformations happen in phases, Ruau said, starting with unstructured data scattered across the organization, moving to a centralized model, bringing data and data scientists together. This centralization eventually becomes a bottleneck, prompting organizations to develop a more balanced hub-and-spoke model. “You achieve a mature environment so the spokes are more impactful,” he explained, while acknowledging that the structure will always need to be adapted to the size, maturity, and mission of the organization. 

Bayer, Ruau said, is past phase three—the hub-and-spoke model—and closest to phase four, which he called ROI. Data and analytics are fully embedded within the whole organization and serve as the basis for business decisions. “We have very strong capacities into the different functions and we have a central hub where we are able to not only connect, but provide overarching workforce,” he said.

Along this digital transformation journey, Ruau said Bayer prioritized applicability in creating digital solutions across the whole organization. “We did not fall into the trap of developing applications and AI without having a relevant business strategy,” he said.

Bayer also prioritized a development model that would move quickly from ideation, experimentation, and industrialization to operation. “We are time-boxing the experiments,” he said. “The experiments need to deliver a minimal, viable product in the space of three months. Why we are doing that is to keep the idea moving through the pipeline and not get stuck into the research phase.” 

Bayer’s investment in its Science@Scale platform was an effort to keep data and analytics solutions moving through the data science lab and into industrialization and operation across the Bayer pharmaceuticals, Ruau said. Its mission is to deliver smart, simple, secure and scalable analytics in the cloud and includes Bayer’s data science platform, auto-machine learning platform, and other platforms-as-a-service. 

“We got much inspiration from our crop science colleagues, which were much more advanced than us in the domain, and we built something that was much more adapted for our pharma division,” he said. 

The Science@Scale platform takes an open-source approach within the Bayer ecosystem. Data assets are the heart of the platform. Ruau described a federated single source of truth, including global assets made available for all functions under robust governance. Additionally, all data are meant to be FAIR—findable, accessible, interoperable, and reusable. “I find this FAIR system to be a bit like karate or judo belt system,” Ruau said. “The minimal level would be the white belt; we need to make the data findable. All the way to [reusability], which is a black belt.”

Data are used in the lab environment, a sandboxed environment for exploration and development of new features; in common, always-on services like Tableu servers and RStudio Connect; and in “factories”, which are environments for running AI jobs—where deployment happens—such as the R&D factory, and the RAD factory. 

A Flexible Platform

“So we had this platform, we have a mission to do digital transformation,” Ruau said. “We faced, still, the same problem. We needed this platform to be able to standardize a bit our work, nevertheless, we needed to operate with a bit of a mindset shift toward having a platform that is excellent at the exploratory type of work, but we needed to have also a platform that is able to handle the deployment of solutions.” 

The solution is the collaboration with NVIDIA on Clara Federated Learning. For the earliest work, Bayer is focusing on its Radiology Data Lake, or RDL. The proof-of-concept work successfully trained an AI model using federated learning and a globally distributed dataset: ten compute clients on five continents and multiple cloud providers. 

Bayer used pre-curated data, the Medical Decathlon benchmark dataset for spleen segmentation, and trained a segmentation algorithm. It wasn’t a perfect segmentation algorithm, Ruau pointed out—that wasn’t the point (this time). Instead the goal was to collect data at the edge from globally-distributed partners and use that data in an anonymized way to train a data model. 

“The beauty of it is that it was feasible, first of all, and it was feasible within our ecosystem,” Ruau said. Next steps will address hospital firewalls, the accuracy of the model, etc. 

NVIDIA’s Powell highlighted the strengths of Clara FL. The orchestration feature allows federated learning programs to be deployed, managed, and monitored from a single user interface, and GPUs aren’t required for the orchestration server. This is crucial, she says, so that federated learning can scale to very diverse edge locations at client sites. From there, the homomorphic encryption preserves the privacy of the client data, while still allowing compute to happen on client data. “This is making sure that the clients that are participating in these programs feel completely protected,” she added.

In Bayer’s experience, Clara FL’s orchestration “hides a huge amount of complexity,” Ruau said. “For the data scientist coming on board, it’s relatively easy.”

Limitations and Open Questions 

As federated learning is still new, there is much to be learned and not yet a great deal of experience. Setting up a federation and gathering data isn’t particularly easy, Ruau admitted. It requires a lot of relational work.

In a real-world scenario, for example, a hospital would put in their own data. In the case of radiology data, perhaps the data should be all images of a certain size, shape, and resolution. 

“As a researcher, you have no view on these data. You cannot look at these data and see if the images [have the appropriate] contrast level, what is the shape of the data,” he explained. “You’ll have a difficult way to debug in the real-world scenario.” We are also still learning, he said, about the best way to protect against security risks such as model inversion, adversarial attacks, and data poisoning. 

Ruau called federated learning one more tool in the Bayer data toolbox.

“We’ll still prefer to have all of the data provided on a central server and all centralized. That is still basically the preferred way of doing any type of activity. Data would need to be centralized into one mega dataset where you can look at the data, expose the data, run your algorithm locally and compare. That is still the gold standard,” he said.

But federated learning offers the best tool to overcome the difficulties of very large datasets, he added, and said Bayer is integrating Clara FL into its entire software stack. “It’s an excellent tool to have! Without this tool, we wouldn’t be able achieve some of our ambitions.”