10025 - Big Data Engineer
Company: Hyundai Autoever America
Location: Fountain Valley
Posted on: May 26, 2023
Job Description:
10025 - Big Data EngineerJob SummaryDesign, build, and maintain
the platform that enables large-scale data processing and analysis.
Responsible for developing and maintaining data pipelines, data
lakes, and other data-related platform. Work closely with data
scientists, analysts, and other stakeholders to ensure that data is
properly collected, stored, and processed for analysis and
reporting purposes. Implement and maintain data security and access
controls to ensure that data is protected. Responsible for
troubleshooting and resolving technical issues related to data
infrastructure and ensuring that data systems are scalable and
efficient. Play a critical role in enabling organization to derive
insights and value from their data assets.Essential Functions
This job requires experience in
building and maintaining scalable data pipelines and robust data
models from structured and unstructured sources for AI/ML. The
ideal candidate should have advanced SQL skills and be able to
query and transform large structured/unstructured datasets using
Spark/PySpark, Spark SQL/Hive and Hive/NoSQL. They should also have
experience in developing Big Data pipelines in orchestration tools
such as Airflow and Oozie, designing tooling for access management,
monitoring, data controls, and self-service ETL/Analytics
pipelines.Other requirements include hands-on experience with
On-Prem Big Data Platform, sound knowledge of Distributed Data
Processing frameworks like YARN, and proficiency in writing data
pipelines using Spark, Python and Scala.The ideal candidate should
also have experience in developing frameworks/utilities in Python,
working in a Dev/Ops environment, and following development best
practices such as code reviews and unit testing.Additionally, the
candidate should be able to diagnose software issues and
engineering workarounds, have a good understanding of BI tools such
as Tableau/Power BI and MicroStrategy for Big Data, and be able to
lead, guide and assist team members with project development and
problem solving.The candidate should also be flexible and able to
learn and use new technologies, work well in a team environment as
well as independently to achieve goals.Please note this job
description is not designed to cover or contain a comprehensive
listing of activities, duties or responsibilities that are required
of the employee for this job. Duties, responsibilities, and
activities may change at any time with or without notice.Required
Skills, Attributes & Education
Bachelor's Degree or equivalent (with
major course work in computer science or a related field).8+ years
of experience working with and maintaining IT Application
support.5+ years of experience with Big Data Engineer Big Data
Technologies: Knowledge of Big Data technologies such as Hadoop,
Spark, Hive, Pig, Kafka, and NoSQL databases such as MongoDB,
Cassandra, and HBase.Distributed Systems: Understanding of
distributed systems and distributed computing
principles.Programming Languages: Proficiency in programming
languages such as Java, Python, Scala, and SQL.Data Modeling:
Knowledge of data modeling techniques and tools to design efficient
data structures for Big Data systems.Data Processing: Experience
with data processing and ETL (Extract, Transform, Load) tools and
techniques.Cloud Computing: Familiarity with cloud computing
platforms such as AWS, Azure, and Google Cloud.Data Security:
Knowledge of data security principles and experience implementing
security measures for Big Data systems.Data Warehousing:
Understanding of data warehousing concepts and experience designing
and maintaining data warehouses.Analytics and Machine Learning:
Familiarity with analytics and machine learning tools and
techniques and their implementation in Big Data systems.Performance
Tuning: Experience with performance tuning and optimization
techniques for Big Data systems to ensure scalability, reliability,
and high availability.CertificationsCloudera/Hortonworks
Spark/Hadoop certification -Salary Range - $91,810 to $131,285
-Powered by JazzHR
Keywords: Hyundai Autoever America, Fountain Valley , 10025 - Big Data Engineer, Engineering , Fountain Valley, California
Didn't find what you're looking for? Search again!
Loading more jobs...
