Senior Data Engineer
To see similar active jobs please follow this link: Remote Development jobs
Note: Contractors (C2C, C2H) that directly apply will not be considered. Individual applicants only
Spokeo is a people search engine and identity platform that enlightens and empowers our customers. With nearly 15 billion records and 18 million monthly visitors, we reconnect friends, reunite families, prevent fraud, and more.
As a Senior Data Engineer at Spokeo, you will develop, optimize, and maintain the ETL data pipeline. This involves working with infrastructure built in AWS, including Airflow, PySpark, EMR, S3, DynamoDB, and more. This role will help build and improve automation platform features, analytical software packages, and data pipeline orchestration tools.
What You'll Do: This includes an estimate of where time will be spent. This is subject to change:
40% - Build infrastructure and data automation pipeline for extracting, preparing, and loading data from various sources. Automate and integrate new components into the data pipeline.
30% - Implement robust ETL processes to efficiently execute product vision and strategy in alignment with organizational goals and priorities.
10% - Create unit and stress test components to monitor technical performance and ensure identified issues are resolved.
10% - Develop data analysis tools to provide data insights and capture key metrics.
10% - Research solutions and maintain technical documentation.
Follow best practices for data governance, quality, cleansing, and ETL-related activities.
Requirements:
You must be authorized to work for any employer in the U.S. Spokeo is unable to sponsor or take over sponsorship of one’s employment visa at this time.
7+ years of development experience in data engineering.
5+ years of hands-on programming experience with Python.
5+ years of professional experience working in big data ecosystems, preferably with Spark
3+ years experience with SQL, schema design, and dimensional data modeling.
2+ years of professional experience working with dataflow management tools, such as Airflow
2+ years of development experience in highly scalable, distributed systems and cluster architectures using AWS.
2+ years experience with non-relational databases (e.g., DynamoDB, Elasticsearch, etc.)
Prior experience working with large data sets (>100M+ records)
B.S. in Computer Science, Information Systems, or related fields
Named Best Company for 2023 by Comparably in the areas of Perks & Benefits, Happiness, Compensation, and Work-Life Balance.
Spokeo offers a bonus program, equity plans, and 401K matching for qualified roles. Twice a year, we do discretionary, merit-based salary increases. Additional benefits include 100% medical/dental/vision coverage for all employees and unlimited PTO.
Spokeo extends written offers to candidates who successfully complete their selection process. Spokeo’s offers include a base salary, participation in a company bonus program, stock options, and comprehensive benefits. A final offer will depend on several factors, including, but not limited to, marketplace competition, job leveling, the candidate’s experience, skills, etc.
Privacy Notice for Candidates: https://www.spokeo.com/recruiting-policy
Spokeo is an equal opportunity employer. Applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, or protected veteran status. Spokeo fosters a business culture where ideas and decisions from all people help us grow, innovate, create the best products, and be relevant in a rapidly changing world.
Recruiters or staffing agencies: Spokeo is not obligated to compensate any external recruiter or search firm who presents a candidate or their resume or profile to a Spokeo employee without 1) a current, fully-executed agreement on file, and 2) being assigned to the open position (as a search) via our applicant tracking solution.
#LI-Remote
About the job
Senior Data Engineer
To see similar active jobs please follow this link: Remote Development jobs
Note: Contractors (C2C, C2H) that directly apply will not be considered. Individual applicants only
Spokeo is a people search engine and identity platform that enlightens and empowers our customers. With nearly 15 billion records and 18 million monthly visitors, we reconnect friends, reunite families, prevent fraud, and more.
As a Senior Data Engineer at Spokeo, you will develop, optimize, and maintain the ETL data pipeline. This involves working with infrastructure built in AWS, including Airflow, PySpark, EMR, S3, DynamoDB, and more. This role will help build and improve automation platform features, analytical software packages, and data pipeline orchestration tools.
What You'll Do: This includes an estimate of where time will be spent. This is subject to change:
40% - Build infrastructure and data automation pipeline for extracting, preparing, and loading data from various sources. Automate and integrate new components into the data pipeline.
30% - Implement robust ETL processes to efficiently execute product vision and strategy in alignment with organizational goals and priorities.
10% - Create unit and stress test components to monitor technical performance and ensure identified issues are resolved.
10% - Develop data analysis tools to provide data insights and capture key metrics.
10% - Research solutions and maintain technical documentation.
Follow best practices for data governance, quality, cleansing, and ETL-related activities.
Requirements:
You must be authorized to work for any employer in the U.S. Spokeo is unable to sponsor or take over sponsorship of one’s employment visa at this time.
7+ years of development experience in data engineering.
5+ years of hands-on programming experience with Python.
5+ years of professional experience working in big data ecosystems, preferably with Spark
3+ years experience with SQL, schema design, and dimensional data modeling.
2+ years of professional experience working with dataflow management tools, such as Airflow
2+ years of development experience in highly scalable, distributed systems and cluster architectures using AWS.
2+ years experience with non-relational databases (e.g., DynamoDB, Elasticsearch, etc.)
Prior experience working with large data sets (>100M+ records)
B.S. in Computer Science, Information Systems, or related fields
Named Best Company for 2023 by Comparably in the areas of Perks & Benefits, Happiness, Compensation, and Work-Life Balance.
Spokeo offers a bonus program, equity plans, and 401K matching for qualified roles. Twice a year, we do discretionary, merit-based salary increases. Additional benefits include 100% medical/dental/vision coverage for all employees and unlimited PTO.
Spokeo extends written offers to candidates who successfully complete their selection process. Spokeo’s offers include a base salary, participation in a company bonus program, stock options, and comprehensive benefits. A final offer will depend on several factors, including, but not limited to, marketplace competition, job leveling, the candidate’s experience, skills, etc.
Privacy Notice for Candidates: https://www.spokeo.com/recruiting-policy
Spokeo is an equal opportunity employer. Applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, or protected veteran status. Spokeo fosters a business culture where ideas and decisions from all people help us grow, innovate, create the best products, and be relevant in a rapidly changing world.
Recruiters or staffing agencies: Spokeo is not obligated to compensate any external recruiter or search firm who presents a candidate or their resume or profile to a Spokeo employee without 1) a current, fully-executed agreement on file, and 2) being assigned to the open position (as a search) via our applicant tracking solution.
#LI-Remote