Senior Data Engineer

What you will Do

Software Engineering

Architects, deploys, and maintains production-grade Python microservices, serverless components (AWS Lambda), batch workers, web services, and shared core libraries across the analytics ecosystem

Engineers RESTful interfaces and application integrations to expose dataset capabilities to platform applications, encompassing secure object storage exchange, query orchestration, automated report delivery, and embedded analytics infrastructure

Constructs backend infrastructure powering AI analytics solutions, incorporating LLM and natural language query translation engines like Snowflake Cortex, controlling dialog state, guaranteeing multi-tenant data isolation before execution, and optimizing throughput and resource utilization

Authors clean, resilient codebase backed by comprehensive test suites while directing GitHub Actions continuous integration and continuous deployment pipelines for automated compilation, verification, security inspection, and production releases

Safeguards system configuration and secrets using key-pair authentication, parameter management, and service credentials while proactively updating software dependencies based on vulnerability scanning metrics

Data Engineering

Constructs and manages ingestion and transformation workflows utilizing Airflow, AWS Glue, Spark, and Snowflake dynamic tables to feed multi-tier lakehouse environments from diverse sources

Engineers automated data cleansing and enrichment routines integrating machine learning models and record reconciliation methods, while safely executing massive historical data re-processing

Architects robust multi-tenant security frameworks in Snowflake employing row-level policies, secure functions, and isolated access controls across global cloud infrastructure regions

Implements metadata-driven automation to dynamically generate client-specific semantic layers from templates, incorporating state tracking, automated recovery, and version rollbacks

Executes data governance protocols and privacy compliance workflows, including systematic tenant data purging across all analytical storage tiers

Develops internal platform analytics modules to track, aggregate, and report infrastructure consumption and operational expenditures across cloud vendor ecosystems

Establishes proactive telemetry and alerting, drives resolution for system incidents, conducts thorough root-cause investigations, and deploys long-term technical remediation

Process

Follows SDLC processes, including adopting agile-based processes, peer code-reviews, and technical preparations required for scheduled releases and demos.

Partners with product, analytics and data science team to drive requirements that take into account all parties’ needs

Establishes monitoring, responds to alerts and resolves issues within data pipelines

Uses and helps shape AI-assisted development practices and tooling for the team.

Impact

Executes intricate, multi-faceted capabilities spanning microservices, automated workflows, and analytical storage infrastructure with high autonomy while providing technical leadership across mid-scale initiatives

Assumes direct accountability for the reliability, security architecture, and long-term viability of deployed systems while systematically uncovering technical remediation opportunities

Pinpoints architectural risks across peer technical deliverables and formulates pragmatic, alternative implementation strategies

Streamlines operational processes, resolves critical system defects, performs advanced codebase refactoring, and exhibits expert troubleshooting capabilities across application code, cloud infrastructure, and analytical queries

Authors comprehensive technical documentation, operational runbooks, and detailed data dictionaries to support platform capabilities

Drives system architecture discussions and fulfills expanded peer code review obligations across the engineering organization

Spearheads onboarding initiatives and provides technical coaching and mentorship to junior staff members

What you will bring

Experience:

Demonstrated background of at least six years architecting enterprise software solutions and big data platforms in operational environments.

Required: Bachelor’s degree in Computer Science or related field, or equivalent years’ experience

Advanced proficiency in Python engineering, spanning distributed microservices, internal modules, RESTful interface design, unit testing frameworks, and GitHub Actions release pipelines rather than ad-hoc scripting.

Expert SQL query optimization, data modeling techniques, and extensive background delivering cloud warehouse architectures using Snowflake.

Proven mastery deploying and orchestrating data processing flows in live production through Apache Airflow.

Hands-on expertise utilizing Amazon Web Services ecosystems, including AWS Lambda, S3 storage, Glue jobs, IAM policies, Key Management/Secrets Manager, and EventBridge integration.

Deep comprehension of multi-tenant security architecture—incorporating row-level access policies, fine-grained access governance, strict tenant isolation, and compliant handling of restricted financial assets across international cloud regions.

Track record of effective cross-functional partnership in product-driven organizations alongside engineering peers and product managers.

Preferred: 

Snowflake advanced features: row access policies, secure views, dynamic tables, stored procedures, Cortex AI / Cortex Analyst, semantic views.

Experience building LLM-powered or text-to-SQL features, or other GenAI applications in production.

Spark (AWS Glue or Databricks).

Fivetran or an equivalent ELT tool; Looker or an equivalent BI platform, including embedded analytics.

Infrastructure as code and GitHub Actions.

Experience with ML-assisted data matching or entity resolution.

Financial, expense or travel industry experience.