Senior Data Engineer - Consulting Services
To see similar active jobs please follow this link: Remote Development jobs
Truelogic is a leading provider of nearshore staff augmentation services, located in New York. Our team of 500 tech talents is driving digital disruption from Latin America to the top projects in U.S. companies. Truelogic has been helping companies of all sizes to achieve their digital transformation goals.
Would you like to make innovation happen? Have you ever dreamed of building Products that impact millions of users? Nice! Then we have a seat for you on our team!
What are you going to do?
You will have the opportunity to work in a forward-thinking and growth-oriented environment, at the only network with a balance of 50% creative and 50% engineers at scale. The technology-first group is built to partner with businesses to navigate the complexity of changing consumer behaviors, emerging technologies and AI. With a global footprint and the capabilities to work across the entire consumer journey, we crave the hardest problems to solve.
Occupy a unique position in the market, since this candidate will analyze and categorize inbound data, establish data pipelines, and ensure seamless connectivity to move data into Snowflake or a similar Datalake. This role is crucial in laying the foundation for our new system by ensuring accurate, well-organized, and accessible data
Data Analysis and Categorization: - Analyze inbound data sources to understand structure, quality, and formats. - Categorize data based on relevance, sensitivity, and usage requirements. - Collaborate with stakeholders to define data schemas and metadata.
Data Pipeline Development: - Establish connectivity to various data sources, including APIs, databases, and file systems. - Develop and configure data ingestion pipelines using tools like Airbyte or Fivetran to move data into Snowflake. - Ensure data is loaded, then transformed, cleaned, and stored appropriately for downstream use.
Data Governance and Security: - Implement data governance policies, including data ownership and access controls. - Ensure compliance with data protection regulations (e.g., GDPR, CCPA). - Implement encryption, authentication, and authorization mechanisms to secure data.
Collaboration and Communication: - Work closely with cross-functional teams, including data scientists, developers, and project managers. - Create and maintain documentation, including data dictionaries and pipeline designs. - Communicate progress and issues to relevant stakeholders.
Performance Optimization and Maintenance: - Monitor and optimize data pipeline performance. - Troubleshoot and resolve data-related issues. - Plan for scalability to handle increased data volumes and additional data sources.
What will help you succeed
Experience: - 3+ years of experience as a Data Engineer or in a similar role. - Proven experience with ETL/ELT processes and data pipeline development. - Hands-on experience with data integration tools like Airbyte, Fivetran, or similar.
Technical Skills: - Proficiency in SQL and experience with relational databases. - Experience working with cloud data warehouses, particularly Snowflake. - Familiarity with workflow orchestration tools such as Apache Airflow. - Programming skills in Python or another relevant language. - Knowledge of data modeling, data warehousing concepts, and data architecture.
Senior Data Engineer - Consulting Services
To see similar active jobs please follow this link: Remote Development jobs
Truelogic is a leading provider of nearshore staff augmentation services, located in New York. Our team of 500 tech talents is driving digital disruption from Latin America to the top projects in U.S. companies. Truelogic has been helping companies of all sizes to achieve their digital transformation goals.
Would you like to make innovation happen? Have you ever dreamed of building Products that impact millions of users? Nice! Then we have a seat for you on our team!
What are you going to do?
You will have the opportunity to work in a forward-thinking and growth-oriented environment, at the only network with a balance of 50% creative and 50% engineers at scale. The technology-first group is built to partner with businesses to navigate the complexity of changing consumer behaviors, emerging technologies and AI. With a global footprint and the capabilities to work across the entire consumer journey, we crave the hardest problems to solve.
Occupy a unique position in the market, since this candidate will analyze and categorize inbound data, establish data pipelines, and ensure seamless connectivity to move data into Snowflake or a similar Datalake. This role is crucial in laying the foundation for our new system by ensuring accurate, well-organized, and accessible data
Data Analysis and Categorization: - Analyze inbound data sources to understand structure, quality, and formats. - Categorize data based on relevance, sensitivity, and usage requirements. - Collaborate with stakeholders to define data schemas and metadata.
Data Pipeline Development: - Establish connectivity to various data sources, including APIs, databases, and file systems. - Develop and configure data ingestion pipelines using tools like Airbyte or Fivetran to move data into Snowflake. - Ensure data is loaded, then transformed, cleaned, and stored appropriately for downstream use.
Data Governance and Security: - Implement data governance policies, including data ownership and access controls. - Ensure compliance with data protection regulations (e.g., GDPR, CCPA). - Implement encryption, authentication, and authorization mechanisms to secure data.
Collaboration and Communication: - Work closely with cross-functional teams, including data scientists, developers, and project managers. - Create and maintain documentation, including data dictionaries and pipeline designs. - Communicate progress and issues to relevant stakeholders.
Performance Optimization and Maintenance: - Monitor and optimize data pipeline performance. - Troubleshoot and resolve data-related issues. - Plan for scalability to handle increased data volumes and additional data sources.
What will help you succeed
Experience: - 3+ years of experience as a Data Engineer or in a similar role. - Proven experience with ETL/ELT processes and data pipeline development. - Hands-on experience with data integration tools like Airbyte, Fivetran, or similar.
Technical Skills: - Proficiency in SQL and experience with relational databases. - Experience working with cloud data warehouses, particularly Snowflake. - Familiarity with workflow orchestration tools such as Apache Airflow. - Programming skills in Python or another relevant language. - Knowledge of data modeling, data warehousing concepts, and data architecture.