Treasure Data Vancouver, British Columbia
10 janv., 2020Permanent - Temps plein
Arm Treasure Data began by offering data warehousing and processing services; since then we've moved further up the value chain with our Customer Data Platform application (CDP), which is seeing a lot of traction with customers new and old. CDP is built on top of our existing data infrastructure and Query Engines (which currently includes Hive/Hadoop and Presto), it sees more than two million events every second and services nearly seven million data queries monthly. We are seeking an engineer that eagerly wants to take on the challenge of multi-tenant large scale data processing, including job and resource scheduling and resilience engineering. Together with the team you'll be working towards the continuous advancement of the state of the art of data processing at Arm Treasure Data. Along with your team, you will work with other Engineering Teams, Product Managers, QA Engineers, and Customer Support Staff to understand what to build next and how to do it. You will collectively build, maintain, and operate the Presto Query Engine, while continuously monitoring, evaluating, and course correcting based on key metrics. The ideal candidate will be able to help our team improve product related outcomes, such as ensuring that the Presto Query Engine continues to scale with customer growth, and handles new types of workloads as we expand the CDP feature set. Things You Will Do: Manage deployment of Distributed Query Engines (Presto and Hive). Troubleshoot production issues and coordinate with the team to correct the problem. Automate solutions to frequent production problems and reduce PagerDuty alerts. Have infrastructure as code mentality and leverage cloud tools (Kubernetes, Terraform, CodeDeploy, etc) to simplify the deployment pipeline. Reduce the time for code changes to get into production. Automate building and testing in development, staging and production environments. Improve operational visibility by establishing and maintaining monitoring and notification systems. Identify affected customers in a production issue and compile incident response report. Develop and maintain runbooks for production environment. Participate throughout the software development life cycle to ensure the software components are designed for ease of deployment and high availability in the cloud. Your experience and skills will include: BE or MS in Computer Science or a related field. 3+ years of hands-on experience in a cloud deployment environment. Experience with and knowledge of Linux command line tools. Working knowledge of Databases, SQL, Java, Scala and Github. We would be thrilled if you had: Basic understanding of SQL and statistics as you will be analyzing massive amount of application logs. Experience of building data analysis pipeline. You can expect a work environment where the team is collaborative and open to your ideas, while we keep our collective eye on supporting our customers' needs. Our team is committed to technical innovation in our product and in the world through customer collaboration, open-source projects, and by continuing to make our product an integral part of our customers' growth and success. We are an equal opportunity employer dedicated to building an inclusive and diverse workforce. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. About Us Arm Treasure Data provides an end-to-end, fully managed cloud service (data acquisition, storage and analysis capability) for Big Data that is trusted and simple. As the original developers of Fluentd, an advanced open-source log collector specifically designed to solve the big data log collection problem, Arm Treasure Data solves the problems for companies wanting the ability to manage their big data needs. Agencies and recruiters, we cannot consider your candidate(s) without a contract in place. Any resumes received without having an active agreement will be considered gratis referrals to us. Thank you for your understanding and cooperation!