Distributed machine learning: scaling up with coarse-grained parallelism

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Machine learning is increasingly used in biological data analysis.
• Scaling machine learning to large biological and medical datasets presents computational and memory challenges.
• Existing methods struggle with the scale of modern biological data.

Purpose of the Study:

• To address the challenges of scaling machine learning algorithms for large biological datasets.
• To introduce a novel distributed learning approach utilizing workstation networks.
• To present a system capable of handling extensive biological and medical data.

Main Methods:

• Introduction of the invariant-partitioning property for data subset analysis.
• Development of a distributed learning system named CorPRL.
• Utilizing interprocess communication for cooperative learning across multiple processors.
• Leveraging ubiquitous workstation networks for distributed computation.

Main Results:

• CorPRL effectively scales machine learning to very large datasets.
• Demonstrated effectiveness in analyzing data from two distinct biological databases.
• The invariant-partitioning property enables efficient distributed learning.
• Cooperative learning ensures rules are satisfactory across the entire dataset.

Conclusions:

• Distributed learning systems like CorPRL can overcome computational limitations for large-scale biological data.
• The invariant-partitioning property is a key enabler for efficient distributed machine learning in biology.
• CorPRL offers a viable solution for analyzing massive biological and medical datasets.