职位描述

We are hiring on behalf of our client who is a technical infrastructure firm specializing in the delivery of massive-scale web data to organizations developing advanced artificial intelligence models. The organization supports high-capacity bandwidth-sharing networks and operates a distributed crawler capable of accessing high-quality public web data at a global scale. Additionally, the team has engineered sophisticated pipelines for the ingestion, segmentation, and annotation of billions of multimedia files, facilitating dataset creation for frontier research labs.

任职要求

Extensive programming experience in one or more of the following: Go, Rust, Python, Java, or C++ Proven experience in building web crawlers or large-scale data pipelines Solid understanding of HTTP, networking protocols, and browser behavior Familiarity with distributed systems and parallel processing techniques Experience handling large datasets, ideally at the terabyte to petabyte scale Demonstrated ability to debug and maintain systems within unstable or adversarial environments

福利待遇

Impactful Opportunity: Contribute to the development of a web-scale crawler and knowledge graph at the forefront of AI data accessibility High-Performance Culture: Join a lean, low-ego team that prioritizes high output and professional growth Remote Work: This position is part of a fully remote team, offering flexibility and autonomy Competitive Compensation: A package including a competitive salary, comprehensive benefits, and equity, commensurate with experience and the ability to operate at scale