Hadoop Developer

Full Time
West Palm Beach - FL
Posted 7 months ago
Job Duties:

  1. Design and implement big data ETL (extract, transform and load) pipelines for large scale (TB/PB) applications which involve moving data between data sources that include Hadoop Distributed File System, Hive, HBase, Kafka and various Relation Database Management systems (Microsoft SQL Server, Oracle and MySQL).
  2. Import and export large scale data using Sqoop from Hadoop Distributed File System to/from Relational Database Management Systems.
  3. Develop Hive Query Language code using built in Hive functions and extended Hive functionality by implementing custom Hive user defined functions (UDF) to generate aggregation reports.
  4. Design and implement ETL pipelines with both Map Reduce and Spark Big Data execution/processing engines.
  5. Design and develop Big Data applications using the CASK open source application development framework.
  6. Write Big Data ETL applications using programming languages like Java, Scala and Python.
  7. Orchestrate and manage resource allocation across Hadoop applications with Cron, Bash scripts, Oozie and YARN.
  8. Write Impala queries to generate ad-hoc reports on hot data.
  9. Generate deep insights from data and provide valuable business intelligence by joining multiple datasets, implementing Big Data Analytics and machine learning techniques across financial and information security related data.
  10. Pre-process Big Datasets using Pig Latin and build full text searches on ETL metrics data using SOLR.
  11. Consume SOAP/RESTful web services, parse and extract XML and JSON data between different systems hosting data.
  12. Develop software products using the Scrum agile process framework.
  13. Implement object-oriented programming concepts, design principles and patterns.
  14. Build and deploy Java/Scala Big Data applications using the Maven build tool and the continuous integration/deployment tool Jenkins.
  15. Application Development, monitoring/troubleshooting of Cloudera distributed Hadoop (CDH) clusters in different production and non-production environments.
  16. Develop, re-factor and enhance existing products/software artifacts and resolve product defects to meet customer issues.
  17. Alert and monitor production deployed Hadoop ETL applications and provide maintenance and support.
  18. Provide technical expertise and code reviews to team members.
  19. Write technical specifications and software documentation for ETL applications.
  20. Work both individually and as a part of a team as required and possess strong analytical, communication and interpersonal skills.
  21. Cross disciplinary knowledge in Big Data engineering, data security and the financial domain.

Travel:

  • The only travel required is when the Applicant relocates to unanticipated client locations anywhere in the U.S. for long-term assignments.
  • Relocation:

  • Must be able to relocate to unanticipated client locations anywhere in the U.S. for long-term assignments.
  • Work Schedule:

  • 40 hours per week; Mon to Fri 9 a.m. to 6.00 p.m.
  • Education:

  • U.S. Master of Science Degree in Computer Science, Software Engineering or a closely related field.
  • Experience:

    18 months of experience in the job offered or as a Big Data Engineer. Experience to include, 12 months of experience in the following:

    • Design and implement big data ETL pipelines for large scale applications which involve moving data between data sources that include Hadoop Distributed File System, Hive, HBase, Kafka and various Relation Database Management Systems such as MS SQL Server, Oracle and MySQL.
    • Develop advanced Hive Query Language code using built in Hive functions and extended Hive functionality by implementing custom Hive user defined functions (UDF) to generate aggregation reports.

    Employer Name:
    Naztec International Group LLC

    Work Location:
    263 N Jog Road, West Palm Beach, FL 33413

    Mail Resume:
    Jesus M. Velarde – Operations Manager
    Naztec International Group LLC
    263 N Jog Road,
    West Palm Beach, FL 33413

    Job Features

    Job CategoryDeveloper, Information Technology, Programmer
    Job TitleHadoop Developer
    LocationWest Palm Beach - FL
    PositionFull Time
    Job IDNZG31021

    Apply Online

    A valid email address is required.
    A valid phone number is required.