We are seeking a Data Engineer to join our client, a Growing Tech Firm, in the San Francisco area for a full-time perm engagement. This position will be hybrid onsite/remote when it is safe to be in person.
What you will have the chance to learn:
- Maintain and improve our data warehouse (Redshift) and data lake.
- Design and build batch and streaming pipelines using Airflow, Spark, Kinesis/Kafka, S3, Redshift, ElasticSearch, etc.
- Assist our Data Science and Analytics teams with DAGs, data modeling, and data validation.
- Assist our product teams with designing and building out our Analytics Platform, ML Platform, and Discovery Platform.
- 6+ years experience in a Data Engineering role
- 4+ years of experience developing in Python and/or Scala.
- 4+ years working with databases and writing SQL.
- Experience with databases such as Redshift, Snowflake, BigQuery, ElasticSearch. Click house etc.
- Experience building data pipelines using such tools as Airflow, Databricks/EMR, Kafka/Kinesis, Hadoop/Hive, S3, etc.
- Knowledge of infrastructure as code and configuration management systems, such as Terraform, Ansible, etc.
- Revamp our data warehouse with new access controls and monitoring for improved performance.
- Work with data science to build new analytics pipelines and improve existing pipelines.
- Work with product engineers to build a product-facing analytics platform. This will involve such projects as real-time traffic and click aggregation and maintenance of an ElasticSearch cluster.
- Work with ML Engineers to build model training and deployment pipelines.
- Work with search and discovery engineers building out a discovery platform.
- Data Warehouse and Pipelining best practices
- On a team of 5-10 determining the long-term data architecture
- AWS Infrastructure, Machine Learning, Databases, Databricks
Does this sound like you? Apply today!
- Onward Search is an equal opportunity employer, dedicated to a policy of non-discrimination in an employment with regard to race, color, religion, gender, sexual orientation, national origin, age, disability, genetic information, veteran status or any other classification protected by federal, state and local laws and ordinances. Candidates must be at least 18 years old to be considered for employment. EOE/M/F/D/V/SO.
- In compliance with federal law, all persons hired will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification document form upon hire.
- Onward Search offers medical, dental, vision benefits, as well as a 401k program, for all eligible full time temporary assignments.
- Onward Search is unable to support 1099 / Corp-to-Corp or Independent Contractor arrangements at this time. All freelancers will be paid as W2 employees.
- Onward Search is a drug-free workplace.
SHARE THIS JOB