Senior Software Engineer - Data Platform Architecture
Cloud and Platform Engineering
At Netflix, we are building a big data analytics infrastructure in
the cloud by leveraging open source technologies like Hadoop, Hive, Pig
and more. We provide a reliable platform that powers search and
recommendations and surfaces product and consumer insights from a
petabyte scale data-warehouse. Tens of billions of events are processed
daily and the volume is climbing rapidly as we grow our global
subscriber base. This represents a tremendous opportunity to build a
world-class data pipeline and analytics platform in the cloud for our
global and expanding services.
Here is a tech blog that describes the core of our infrastructure: http://techblog.netflix.com/2013/01/hadoop-platform-as-service-in-cloud.html
In this role...
You will:
- Design new services that make analyzing massive datasets easier and more efficient.
- Architect more efficient strategies for acquiring and processing
data, integrate new technologies into the pipeline, and create
abstractions around these strategies and technologies to make them
easily accessible to other engineers.
- Create tooling to monitor, optimize, and make our data transformation pipelines more resilient in the AWS cloud.
- Contribute to open source tooling that makes the Hadoop ecosystem better integrated and more user friendly.
You should:
- Be excited to work with very little guidance and loosely defined
goals. While you shouldn't expect any handholding, we are a close-knit
team that works together closely to solve problems.
- Be confident in proposing and creating disruptive new
technologies and solutions that change the way others approach problems
or interact with data, ultimately adding value to Netflix.
- Value simplicity and clarity in design. Our engineering strongly reflects this value.
- Appreciate when “good enough is good enough”.
- Iterate quickly. We like to push solutions out in hours to days.
- Think no problem is too difficult, big, or complex.
- Have a strong background in Computer Science and distributed systems. MS/PhD in CS is preferred.
Technologies we use:
- Python, Groovy, and Java // Pig, Hive, and Hadoop // Git
- The right technologies for the problem at hand. We're flexible and open to rapid change.
No comments:
Post a Comment