-
View all jobs
- Monitor and provide ongoing support/maintenance of the data pipelines to ingest and transform data using Scala and SQL on Spark.
- Investigate and remediate data pipeline errors and performance issues. Identify and resolve data discrepancies, ambiguities, and inconsistencies.
- Provide technical support for the data analysis Source and version control code/configuration artifacts using GitHub.
- Deploy code artifacts using GitHub Workflow/actions.
- Technical Oversight: Provide technical leadership and hands-on oversight in developing data processing applications on Spark using Scala programming, focusing on Microsoft Azure Synapse Spark Runtime.
- Data Pipeline Optimization: Design and optimize data pipelines processing through various zones in Medallion architecture using Azure Synapse pipelines.
- Data Ingestion and Quality: Manage data ingestion, ensure data quality checks with tools like DQ, and handle data validation and error management.
- Configuration Management: Develop and manage configuration settings using JSON files read by classes such as ApplicationConfig and TableConfig for various zones.
- Cross-Functional Collaboration: Collaborate with data scientists, analysts, and cross-functional teams to ensure seamless integration and alignment of data engineering practices with marketing strategies.
- Logging and Auditing: Oversee logging, auditing, and error handling processes to track and ensure data processing integrity. Knowledge of Azure Log Analytics and KQL queries a plus.
- Testing and Validation: Implement unit testing with tools like Scala Test and maintain data quality checks for reliable data processing outcomes.
Key Skills
Ranked by relevance
c
scala
spark
ai
ha
git
configuration management
data analysis
sql
spi
pan
ui
kf
Related Jobs
3 roles aligned with this opportunity
View Job Details
Related
Junior Data Analyst (Gen AI)
2026-05-23
Full-time
Entry
Poland
IT Services
Information Technology
View Job Details
Related
Senior Solidity & Ethereum Engineer
2026-05-23
Full-time
Mid-Senior
Poland
IT Services
Information Technology
View Job Details
Related
Data Engineer
2026-06-03
Full-time
Associate
Poland
IT Services
Information Technology
Login to Apply
- Posted
- Nov 29, 2024
- Type
- Contract
- Level
- Entry
- Location
- Cracow
- Company
- Hirexa Solutions
Industries
IT Services
IT Consulting
Categories
Information Technology
Related Jobs
3 roles aligned with this opportunity
View Job Details
Related
Junior Data Analyst (Gen AI)
2026-05-23
Full-time
Entry
Poland
IT Services
Information Technology
View Job Details
Related
Senior Solidity & Ethereum Engineer
2026-05-23
Full-time
Mid-Senior
Poland
IT Services
Information Technology
View Job Details
Related
Data Engineer
2026-06-03
Full-time
Associate
Poland
IT Services
Information Technology