Data Engineer
ReliaQuest
Software Engineering, Data Science
Pune, Maharashtra, India
Posted on Oct 22, 2024
Why it’s worth it:
Qualified candidates will work closely with cross-functional teams that turn data into critical information and knowledge that can be used to make sound business decisions and business processes. They will gain an in-depth understanding of the cybersecurity business environment and will be responsible for the maintenance and health of tools that ingest data generated by a diverse set of platforms. This position will support our software developers, database architects, analytics engineers, business analysts and data scientists on data initiatives and will ensure optimal data delivery is consistent throughout ongoing projects. This position will also be responsible for data integration between in-house and commercial applications within the environment.
The everyday hustle:
Ability to work with multiple developer teams
Complex SQL Query development including performance tuning in Snowflake, Postgres, and SQL Server Environments
Develop in Python using object-oriented approach to efficiently create jobs to ingest raw data, transform, and create application integrations
Ingest raw data using Python with Snowflake, Postgres, MySQL, Elasticsearch, SQL Server, API, and various other sources and destinations
Create models in dbt (Data Build Tools)
Creation and optimization of tables, indexes, stored procedures, views, and functions in Snowflake, Postgres, and SQL Server
Creation of Argo CRON Workflows for job orchestration
Contribute to multiple GitLab Repositories
Basic GitLab CI/CD changes and modifications
Work with docker containers for development and modify existing Dockerfiles
Work with internal tools that are deployed on Kubernetes (EKS) Pods
Responsible for the design, development, implementation and maintenance of Snowflake and Postgres databases
Develop and interpret database models
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
May work after hours to complete business objectives if required
Do you have what it takes?
2-4 years of Snowflake, Postgres, MSSQL or related Database Development and/or DBA experience
Bachelor's degree in Computer Science or related discipline preferred
Experience using Snowflake and/or other cloud-based databases / data stores preferred
Excellent written and oral communication skills including an ability to communicate with internal and external cross functional teams (technical and business)
Great Attitude, Energy, and Effort
