Big Data Hadoop Developer Senior
Job Number: 188411
Experian is the world’s leading global information services company, unlocking the power of data to create opportunities for consumers, businesses and society. For more than 125 years, we’ve helped consumers and clients prosper, and economies and communities flourish – and we’re not done. Experian has operated at the center of Big Data since before the phrase was coined. Few companies can offer information services on an equivalent scale. Experian has been named to Forbes Magazine’s Top 100 list of the “World’s Most Innovative Companies” for 4 years in a row. We have 16,000 people operating across 37 countries, and every day we’re investing in new technologies, talented people and innovation to help create a better tomorrow.
- Full Time
- Level: All
- Travel: Limited
- Glassdoor Rating: 3.7 out of 5
Experian Technology is central to the future success of Experian. Whether developing a new product, enhancing a SaaS solution or securing our networks from external threats, you have a critical role to play. When looking for new talent to join our technical teams, we look for individuals with the following traits:
- Innovation 9
- Problem-Solver 9
- Presenter 8
- Technologically savvy 10
- Confident 6
- Independent 4
- Relationship expertise
- Team player
- Technologically savvy
We are looking for Cloudera Hadoop System Specialists who will focus on Big Data development within an existing development team supporting Business Analytics applications. The role requires experience working with Hadoop ecosystems and related technologies, implementing, troubleshooting and optimizing distributed solutions based on modern big data architecture.
- Responsible for design, development and delivery of data-sets from operational systems and files and ingestion into ODSs (operational data stores), Data Marts and files.
- Troubleshoot and develop on Hadoop technologies including HDFS, Kafka, Hive, Pig, Flume, HBase, Spark, Impala and Hadoop ETL development via tools such as ODI for Big Data and API’s to extract data from source.
- Translate, load and present disparate data-sets in multiple formats and from multiple sources including JSON, Avro, text files, Kafka queues, and log data. Data will range in type from structured through semi-structured to unstructured.
- Responsible for building solutions involving large data sets using SQL methodologies, Data Integration Tools like ODI in any database.
- Perform all technical aspects of software development (ARCHITECT I WRITE, I TEST, I SUPPORT) and automation. Deployment considerations will include scheduling and automation of routine processing activities.
- Will implement quality logical and physical ETL designs that have been optimized to meet the operational performance requirements for our multiple solutions and products, this will include the implementation of sound architecture, design, and the use of agreed development standards.
- Perform unit, component and integration testing of software components including the design, implementation, evaluation and execution of unit and assembly test scripts.
- Conduct code reviews and tests of automated build scripts.
- Debug software components, identify, fix and verify remediation of code defects (own work and the work of others).
- Work with Business Analysts, end users and architects to define requirements, agree end process, build code efficiently and work in collaboration with the rest of the team for effective solutions
Knowledge, Experience & Qualifications
- 3 or more years of experience with various tools and frameworks that enable capabilities within the big data ecosystem (Hadoop, Kafka, NIFI, Hive, YARN, HBASE, NoSQL, Cassandra and MongoDB).
- 4 or more verifiable years of software development experience in a professional environment.
- Experience of Big Data Query tools like Pig, Spark SQL and phoenix.
- Experience of data ingestion tools like Flume and Sqoop.
- Experience with AD and Kerberos - need to troubleshoot issues with service/user accounts on kerberized cluster
- Experience with Spark Streaming, Apache NIFI and Kafka for real time data processing
- Experience in DevOps tools like Jenkins
- Experience with source code controlled environments like TFS, GIT or SVN
- Hands on Experience with application design, software development and automated testing
- Solid experience with Java/J2EE, XML, XPath, Web Services, REST services
- Bachelor’s Degree in Computer Science, Computer Science Engineering, or related field required; Higher role Degree preferred.
Experian offers comprehensive coverage with flexible options for you as well as your spouse/domestic partner and eligible dependents. This includes medical, dental and vision insurance, health care savings and spending accounts, dependent care FSA, health & wellness program, and telemedicine.
Personal Finance Protection
We offer several benefits that strengthen your financial protection such as a 401k plan with an employer match, an employee stock purchase plan, life & additional insurance coverage.
We invest in your professional future through our education assistance program and many professional development programs. We also grant 20 scholarships to high school seniors each year to children of Experian employees.
We offer flexible time off in addition to corporate holidays, sick days and paid volunteer time off.
Experian supports your life outside the office with benefits such as our new parental leave policy covering time off for birth mothers as well as birth fathers and mothers/fathers of adoptive or foster children, adoption assistance, and pet insurance.
Experian employees enjoy many other benefits through our employee discounts, company sponsored activities, and flexible total rewards packages. Many of our office locations provide onsite amenities such as fitness centers, game rooms, cafeterias and free snacks in break rooms. We support work/life balance with convenient offices in several metropolitan locations and flexible work arrangements when possible.
“I’m a problem solver, strategist, and security engineer in the Cyber Threat Management realm. The best part of my job is creativity and thinking outside the box is encouraged. I work in a team that enables me to innovate and collaborate with other teams to help drive our organization to a better tomorrow.”Lauren Alvarez, Security Engineer
“The best part of my job is the people I get to work with and for. I am surrounded with some of the most innovative, smart and down to earth characters in the field.”Rafael Molina, Sr. Information Security Specialist, EITS
“At Experian, the technology environment is always changing to keep up with the latest technology out there. It’s innovative, yet what’s motivative is that when there is a change in technology, our employees host training sessions because not everyone fully understand the new technology, so we all grow together to find our best way to every solution.”Wes Hoang, Software Dev Intern
“If you are looking for a workplace to Innovate, grow in professional career and Work with great leaders, then Experian is the best place to be at!”Dhaval Dagli, Software Developer
“I personally enjoy working with a variety of application and technical experts in trouble-shooting and figuring out why something failed, and/or working with the same caliber of colleagues to implement real-world solutions to ensure that we have a solid environment that is sustainable and meets the needs of the business.”Kenn Johnson, Interim Manager of Cyber Security Engineering
Our strategy is centred on delivering world-class expertise to organisations, transforming our relationship with consumers and delivering growth consistently, underpinned by foundations...Read More
Machine Learning Applications for Payment Transaction Data
"Experian DataLabs provide a safe and secure environment to partner with our clients to enable breakthrough data experimentation and innovation." Eric Haller(Experian North America)
Experian Hackathon 2017
Collaborating in the Office
IT Clinic in Schaumburg