Senior Software Engineer - Data | AppNexus Inc. | San Francisco, United States
Senior Software Engineer - Data (San Francisco)Engineering | San Francisco, United States |
Team Description:
The AppNexus Data Platform team plays a key role in a technology company experiencing explosive growth. Our data pipeline processes 74 billion log records a day (more than 60TB of data) generated by our real-time bidding engines and ad serving platform. Data volume has rocketed over the past year and is continuing to increase. This data is used to generate aggregated analytics reports, update budgets, and drive optimization engines. Clients rely on analytics data to run their businesses, so it’s our job to provide excellent reporting to ensure they don’t lose money. We do all this while running against extremely tight SLAs. A budget is aggregated at 10-minute intervals, no matter how many more billions of transactions come in. To that end, we aim to provide real-time stats and budgeting to freeze campaign spending as close to real time as possible.
Consider some of the problems we tackle every day:
- Conversion attribution (or finding the needle in the haystack): In online advertising, we have to determine which events caused a particular conversion – or which ad impression caused a user to make a purchase. We hunt for the needle in a 150 billion-row “haystack,” and we do this millions of times every hour.
- Getting insight: Not only are we dealing with a growing body of data, but we also need to be able to process it rapidly, get insights and share it all with clients at whatever level of granularity they need.
- Processing data: We have built a platform that allows us to process data but many of our clients struggle to build tools that allow them to process our data directly. A key goal is to abstract out the functionality we have so our clients can benefit.
- New tools: We are always seeking new and better tools to help us meet challenges. For example, our data is so distributed that keeping it consistent globally is a major concern. To solve this problem, our team built Starfish—a monitoring tool that can self-heal any data discrepancies.
Role description:
As a senior software engineer on the data team, you would be working on and leading projects that are important to AppNexus’s success. Projects will involve processing big data streamed from thousands of ad-servers. The goal is to collect, process and deliver results to consumers in a most efficient and reliable way. Running big data systems also involves production support. In this role, the candidate will be working with operations and data teams to help provide production support for our data systems.
Responsibilities:
- Ensure that jobs are being completed within our SLAs
- Take ownership of reducing the complexity of our data model
- Optimize our jobs to utilize Hadoop, HBase, Vertica resources in the most efficient way
- Ensure data consistency and the accuracy of our data
- Take ownership of adding new jobs to support client facing features
- Prioritize bug fixes with the goal of keeping the unreliability index close to 0
- Monitor and manage nagios services related to the ETL jobs schedule in our system. (reporting, billing, budget, optimization…)
- 5+ years of software engineering experience, developing software for distributed production systems
- Fluency in Java, along with some experience in PHP or Python
- Familiarity with large-scale distributed data processing (i.e. Hadoop)
- Experience in start-to-finish deployment of large scale multi-TB data processing systems is a PLUS
- Strong understanding of RDBMS, SQL, no SQL alternatives Experience supporting production systems
- Passion for engineering and computer science around data
- Looks for novel and creative solutions to complex problems. Is a good collaborator and team player, with strong communication skills
- You are an independent thinker who is self-motivated and thrives in a fast-paced work environment.
- You are passionate about and thrive in a culture of learning and teaching – you are both open minded to others’ opinions and take opportunities to help and teach colleagues.
- You find new, creative solutions to complex problems and are passionate about high-performance computing and ultra-low latency.
- You love to develop and have a thorough understanding of server-side application development, scalable web services, query optimization, and database design.
- You are hands-on when designing applications and are comfortable coding off of not only a formal spec, but also a problem statement.
- You want to be part of an organization with a transparent and agile environment and a clear communication path between your team and the CEO where you can communicate freely and openly.
- Industry-changing work – build bold solutions to some of today’s most exciting technical challenges.
- Significant impact from your very first day thanks to our rapidly growing scale and agile engineering teams.
- Global presence – AppNexus has offices around the world: Manhattan, San Francisco, Seattle, Portland, London, Paris, Hamburg, Toronto, Sydney, and Singapore. Engineers are primarily based in Manhattan and Portland.
- Regular access to industry events - AppNexus hosts many technical meet-ups and speakers on site.
- We are a technology company at heart. Our founders can hold their own in architecture discussions with the team. Occasionally, our CEO even checks-in bug fixes to the ad server codebase.
- Great Technology: Any computer of your choosing. Most of our team develop using retina displayed MacBook Pros, with multiple monitors for additional screen real estate.
- Free Food: Every office has fully-stocked kitchens featuring bagels each morning, as well as healthy and tasty snacks, drinks, and coffee throughout the day. Family Lunch Fridays – AppNexus provides free lunch for the company every Friday.
- Career Development: Annual budget for conferences and continuing education of your choosing.
- An Active Development Community: Participate in company sponsored Hackathons, Innovation days and events.
- Speak, Publish, and Patent: Your groundbreaking work will help enhance the engineering brand at an emerging powerhouse technology company.
No comments:
Post a Comment