Senior Data Engineer
- Hyderabad
- Analytics
- Permanent
- Hybrid
- Full Time
- Senior Data Engineer is responsible for design, develop and support ETL data pipelines solutions primary in AWS environment
- Design, develop, and maintain scaled ETL process to deliver meaningful insights from large and complicated data sets.
- Work as part of a team to build out and support data warehouse, implement solutions using PySpark to process structured and unstructured data.
- Play key role in building out a semantic layer through development of ETLs and virtualized views.
- Collaborate with Engineering teams to discovery and leverage new data being introduced into the environment
- Support existing ETL processes written in SQL, or leveraging third party APIs with Python, troubleshoot and resolve production issues.
- Strong SQL and data to understand and troubleshoot existing complex SQL.
- Hands-on experience with Apache Airflow or equivalent tools (AWS MWAA) for orchestration of data pipelines
- Create and maintain report specifications and process documentations as part of the required data deliverables.
- Serve as liaison with business and technical teams to achieve project objectives, delivering cross functional reporting solutions.
- Troubleshoot and resolve data, system, and performance issues
- Communicating with business partners, other technical teams and management to collect requirements, articulate data deliverables, and provide technical designs.
Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them save time and money.
We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments.
We invest in people and new advanced technologies to unlock the power of data. As a FTSE 100 Index company listed on the London Stock Exchange (EXPN), we have a team of 22,500 people across 32 countries. Our corporate headquarters are in Dublin, Ireland. Learn more at experianplc.com.
- you have completed graduation from BE/Btech
- 6 to 9 years of experience in Data Engineering development
- 5 years of experience in Python scripting
- You should have 8 years experience in SQL, 5+years in Datawarehouse, 5yrs in Agile and 3yrs with Cloud
- 3 years of experience with AWS ecosystem (Redshift, EMR, S3, MWAA)
- 5 years of experience in Agile development methodology
- You will work with the team to create solutions
- Proficiency in CI/CD tools (Jenkins, GitLab, etc.)
Our uniqueness is that we celebrate yours. Experian's culture and people are important differentiators. We take our people agenda very seriously and focus on what matters; DEI, work/life balance, development, authenticity, collaboration, wellness, reward & recognition, volunteering... the list goes on. Experian's people first approach is award-winning; World's Best Workplaces™ 2024 (Fortune Top 25), Great Place To Work™ in 24 countries, and Glassdoor Best Places to Work 2024 to name a few. Check out Experian Life on social or our Careers Site to understand why.
Experian is proud to be an Equal Opportunity and Affirmative Action employer. Innovation is an important part of Experian's DNA and practices, and our diverse workforce drives our success. Everyone can succeed at Experian and bring their whole self to work, irrespective of their gender, ethnicity, religion, colour, sexuality, physical ability or age. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.
#LI-Onsite
Benefits
Experian care for employee's work life balance, health, safety and wellbeing.
1) In support of this endeavor, we offer the best family well-being benefits,
2) Enhanced medical benefits and paid time off.
Experian Careers - Creating a better tomorrow together
Find out what its like to work for Experian by clicking here
Explore Experian's award winning 'people first' culture.
Being part of Experian is exciting. Every day brings new challenges, new opportunities, new ways for us to do what we do best: using our innovative technologies and data to work in new and surprising ways. All brought together in a single thought: Discover the Unexpected.
Learn moreRecommended Jobs
Salary
Location
United States
Experience Level
Not Applicable
Employment
Permanent
Location
United States
Role Type
Home
Salary Range
$33,624 - $54,993
Schedule
Full Time
Department
Data Management
Brand
Experian
Remote working available
Yes
Description
The OCR Assistant will support document processing by operating Abbyy Flexicapture and FlexiLayout applications to extract data from batches of PDF documents. You will focus on accurate data extractio
Reference
5e679fca-ced5-4ced-9bd7-2809e6c5f487
Expiry Date
01/01/0001
Salary
Location
São Paulo, Brazil
Experience Level
Not Applicable
Employment
Permanent
Location
Sao Paulo
Role Type
Hybrid
Schedule
Full Time
Department
Marketing and Communications
Brand
Serasa Experian
Remote working available
No
Description
Estamos em busca de uma pessoa apaixonada por marketing, com forte experiência em eventos B2B, para atuar de forma estratégica e operacional na criação, planejamento e execução de eventos que gerem im
Reference
714f69d6-30d0-46ee-8ac3-1a019e4b241f
Expiry Date
01/01/0001
Salary
Location
Arizona, United States
Experience Level
Not Applicable
Employment
Permanent
Location
Arizona
Role Type
Hybrid
Salary Range
$176,036 - $316,865
Schedule
Full Time
Department
Analytics
Brand
Experian
Remote working available
Yes
Description
The Director of Analytics and Regulatory Technology will lead the strategic and operational efforts of the analytics and regulatory technology departments. You will oversee the development of advanced
Reference
ae8390b7-c0f3-47a7-a6c8-b0e18b6f160f
Expiry Date
01/01/0001
Salary
Location
Heredia, Costa Rica
Experience Level
Not Applicable
Employment
Permanent
Location
Heredia
Role Type
Hybrid
Schedule
Full Time
Department
Project Management
Brand
Experian
Remote working available
Yes
Description
Reporting to the Senior Manager Continuous Assurance, the Controls Testing Manager ("Manager") will create the testing methodology, including the risk assessment and planning assurance activities ove
Reference
0e559529-f507-48c1-b467-15bd5b884ac1
Expiry Date
01/01/0001
Salary
Location
United States
Experience Level
Associate
Employment
Permanent
Location
United States
Role Type
Home
Salary Range
See Pay Range
Schedule
Full Time
Department
Sales & Business Development
Brand
Experian
Remote working available
Yes
Description
Financial Services Software Solution Specialist As a Financial Services Software Sales Specialist, you will be an important player on our sales team, responsible for driving revenue from both existing
Reference
78a3decb-9e90-44c8-b849-beaa7a591c25
Expiry Date
01/01/0001
Salary
Location
Heredia , Costa Rica
Experience Level
Associate
Employment
Permanent
Location
Heredia
Role Type
Hybrid
Schedule
Full Time
Department
Product Development
Brand
Experian
Remote working available
Yes
Description
The Agile Developer II is an entry-level position ideal for individuals early in their careers who are eager to gain hands-on experience in a collaborative Agile environment.You will report to Experia
Reference
f124ef1b-dc43-4e67-82e0-90598489fdfa
Expiry Date
01/01/0001
Salary
Location
Nottingham, England
Experience Level
Not Applicable
Employment
Permanent
Location
Nottingham
Role Type
Hybrid
Schedule
Full Time
Department
Sales & Business Development
Brand
Experian
Remote working available
No
Description
We are looking for a GTM Sales Operations Business Partner, to join our UK Operations Team within Experian UK&I GTM to report into one of the senior leaders.As a Sales Operations Business Partner with
Reference
822678c0-f009-47df-bd6b-70c83b8584d0
Expiry Date
01/01/0001
Salary
Location
United States
Experience Level
Associate
Employment
Permanent
Location
United States
Role Type
Home
Salary Range
$115,747 - $208,344
Schedule
Full Time
Department
Information Technology & Systems
Brand
Experian
Remote working available
Yes
Description
As the Application Owner, you will report to Experian Automotive's Senior Director of IT. This role is considered "Hybrid", whereby you will work in our Franklin, TN office 1-2 days/week with our mana
Reference
d68b7922-3c6b-4294-a8c7-89e6b3e9cd69
Expiry Date
01/01/0001
Salary
Location
United States
Experience Level
Executive
Employment
Permanent
Location
United States
Role Type
Hybrid
Salary Range
See Executive Salary Range
Schedule
Full Time
Department
Product Management
Brand
Experian
Remote working available
Yes
Description
The Head of Product Management is a key member of the Automotive Senior Leadership Team reporting directly to the President of Experian Automotive. Experian Automotive provides businesses with automot
Reference
e2beca7b-2818-41ef-a9ca-9fd68ac1abb9
Expiry Date
01/01/0001
Salary
Location
United States
Experience Level
Not Applicable
Employment
Permanent
Location
United States
Role Type
Home
Salary Range
$36,986 - $60,492
Schedule
Full Time
Department
Verification
Brand
Experian
Remote working available
Yes
Description
We would love you to join us at Experian in helping us to shape the future of employer services for Human Resources compliance and outsourcing. We are looking for an Unemployment (UI) Claims Analyst t
Reference
101d7ee4-b749-4d4d-8418-5114235a2834
Expiry Date
01/01/0001
Salary
Location
Costa Mesa, United States
Experience Level
Mid-Senior Level
Employment
Permanent
Location
Costa Mesa
Role Type
Home
Salary Range
$133,109 - $239,596
Schedule
Full Time
Department
Information Technology & Systems
Brand
Experian
Remote working available
Yes
Description
As a Lead Product Designer for new commercial product development and our developer platform teams, you will help with design strategy for net-new products and growth plans. This is a strategic, hands
Reference
abd09609-3102-4d3f-a3f1-0668fcbebcbb
Expiry Date
01/01/0001
Salary
Location
California, United States
Experience Level
Not Applicable
Employment
Permanent
Location
California
Role Type
Hybrid
Salary Range
$115,747 - $208,344
Schedule
Full Time
Department
Product Management
Brand
Experian
Remote working available
Yes
Description
Experian's Direct-to-Consumer (D2C) business is looking for a Senior Product Manager to join our growing Insurance team. In this strategic role, you'll translate needs into scalable, compliant, and da
Reference
192960f4-3ed4-4ae0-8582-f891757de027
Expiry Date
01/01/0001
Salary
Location
Heredia, Costa Rica
Experience Level
Mid-Senior Level
Employment
Permanent
Location
Heredia
Role Type
Home
Schedule
Full Time
Department
Information Technology & Systems
Brand
Experian
Remote working available
Yes
Description
You will be responsible for the design, perform operations, engineering, automation, and daily management of Crypto infrastructure, which includes Windows and Linux servers, COTS appliances, and Hardw
Reference
7ee023b6-8e0c-41b5-9e41-40c9a80f11cc
Expiry Date
01/01/0001
Salary
Location
São Paulo, Brazil
Experience Level
Not Applicable
Employment
Permanent
Location
Sao Paulo
Role Type
Hub
Schedule
Full Time
Department
Sales & Business Development
Brand
Serasa Experian
Remote working available
No
Description
Área: PME Subárea: Gestão de Clientes Quais serão suas principais entregas?Atendimento e suporte ao cliente com o objetivo de negociação, fidelização e retenção de contratos pós-pago;Os canais par
Reference
7a6737a8-63e6-4d37-b08f-23ea8506c1cf
Expiry Date
01/01/0001
Salary
Location
São Paulo, Brazil
Experience Level
Not Applicable
Employment
Permanent
Location
Sao Paulo
Role Type
Hybrid
Schedule
Full Time
Department
Product Development
Brand
Serasa Experian
Remote working available
No
Description
Estamos à procura de um(a) Engenheiro(a) de Software Especialista com vasta e comprovada experiência para liderar e inspirar equipes de desenvolvimento de software. O profissional será responsável por
Reference
5d4a8a7d-5477-4f0a-a357-ecfaf386e37e
Expiry Date
01/01/0001
Salary
Location
Nottingham, England
Experience Level
Not Applicable
Employment
Permanent
Location
Nottingham
Role Type
Home
Schedule
Full Time
Department
Legal & Compliance
Brand
Experian
Remote working available
No
Description
We are looking for a dynamic and strategic Business Operations Lead to join our Cyber Fusion Center (CFC). This leadership role is important to managing programmes, and helping the CFC defend against
Reference
51950595-dc9c-4b9a-a6c2-4e724e0ca8b7
Expiry Date
01/01/0001