The Modern Research Data Portal: a design pattern for networked, data-intensive science

Area of Science:

• Computer Science
• Data Management
• Scientific Research Infrastructure

Background:

• Traditional monolithic web-based data portals face challenges in providing convenient, high-speed, and secure access to large datasets.
• Increasing data volumes in scientific research necessitate more efficient data management and transfer solutions.

Purpose of the Study:

• To introduce a new design pattern, the Modern Research Data Portal, for improved research data access.
• To outline best practices for building and deploying high-performance, secure research data portals.

Main Methods:

• Disaggregation of traditional monolithic portal architecture.
• Leveraging high-performance data enclaves and cloud-based data management services.
• Utilizing Python APIs for authentication, authorization, data transfer, and data sharing.

Main Results:

• Achieved orders-of-magnitude increases in data transfer performance.
• Enabled new deployment architectures decoupling control logic from data storage.
• Reduced development and operations costs for research data portals.

Conclusions:

• The Modern Research Data Portal design pattern offers a scalable and cost-effective solution for research data access.
• Implementation guidance and sample code are provided to facilitate adoption.
• This approach supports diverse research environments, including experimental facilities and supercomputing sites.