Analysis of big data job requirements based on K-means text clustering in China

Area of Science:

• Data Science
• Computational Linguistics
• Human-Computer Interaction

Background:

• The rapid growth of big data necessitates a clear understanding of associated job market demands.
• Identifying key characteristics of big data roles is crucial for both employers and job seekers.

Purpose of the Study:

• To characterize domestic big data job requirements using text clustering.
• To aid enterprises and employees in identifying big data talent.
• To foster advancements in big data research.

Main Methods:

• Web crawling of "big data" recruitment information from zhaopin.com.
• Jieba word segmentation and K-means text clustering for job position categorization.
• Determining optimal cluster count via within-group sum of squares.

Main Results:

• Big data job demands are predominantly located in first-tier and new first-tier cities.
• Employers prefer candidates with college or bachelor's degrees and at least one year of experience.
• Significant variations in salary exist across different job categories, with higher positions demanding greater education and experience.

Conclusions:

• Big data job market exhibits distinct geographical and qualification-based patterns.
• Clustering analysis provides valuable insights into the evolving landscape of big data employment.
• The findings offer a data-driven perspective for career development and talent acquisition in the big data field.