Person, Clothing, Face, Female, Sleeve, Woman, People, Long Sleeve, Portrait, Smile

Lead Big Data Platform Engineer

 

Lead Big Data Platform Engineer

  • JR-122645
  • Singapore
  • Information Technology
  • Full time
Lead Big Data Platform Engineer

Who are we?

Equinix is the world’s digital infrastructure company, operating 210 data centers across the globe and providing interconnections to all the key clouds and networks. Businesses need one place to simplify and bring together fragmented, complex infrastructure that spans private and public cloud environments. Our global platform allows customers to place infrastructure wherever they need it and connect it to everything they need to succeed.

 

At Equinix, we help the world’s digital leaders scale with agility, speed the launch of digital services, deliver world-class experiences, and transform people’s lives. Our culture is based on collaboration and the growth and development of our teams. 

 

We hire hardworking people who thrive on solving challenging problems and give them opportunities to hone new skills, and try new approaches, as we grow our product portfolio with new software and network architecture solutions. We embrace diversity in thought and contribution and are committed to providing an equitable work environment. that is foundational to our core values as a company and is vital to our success.

Are you passionate about incubating technologies and driving the delivery of new products and services? Are you the person who can drive innovation and wants to make a bigger impact? If you answered yes, then come join the Emerging Technologies Organization at Equinix. You will collaborate with some of the brightest minds in the industry and enjoy a competitive package, great benefits, and a creative and diverse work environment. 

Lead Big Data Platform Engineer

Job Description:

In the role of Sr. Data Platform Engineer, you will be responsible for end-to-end development of Data Analytics Platform and AI/ML Ops use cases to take our data engineering and analytics team and system into the future. The role must be hands on.

The role is for a self-motivated individual with software engineering skills and expertise with Big Data technologies. The candidate will be extensively involved in hands-on activities including POCs, design, documentation, development, and test of new functionality. Candidate must be agile and flexible with changing priorities based on team’s needs.

Responsibilities

  • Design & Architecture: You help to design and implement an optimal data pipeline architecture for the Equinix analytics system. Identify, design, and implement improvements to automate manual processes, optimize speed and efficiency of data delivery, architecting and designing for high availability, scaling, and reliability. Architect and design infrastructure to facilitate data extraction, transformation, export, and query of data from a variety of data sources, both internal and external, both small and large.
  • Massive data: You will source / examine, analyze, engineer data pipelines for gigabytes/terabytes of structured and unstructured data with our platform to create value for customers. You will also be working with Enterprise data.
  • Pushing the limits: This role will be on the cutting edge of our Data / Machine Learning platform. As we push to solve more of our customer challenges, you will be prototyping new features, tools, and ideas. Innovate at a very fast pace to maintain our competitive edge.
  • Distributed Processing Engine: You will be masterfully working on the data platform which involves processing of both unbounded and bounded data sources, perform im-memory computation & transformation.
  • Production deployment: You will be responsible for integration and deployment of the Data Ingestion & machine learning pipelines into production where your ideas can come to life.
  • Work with stakeholders including the Data Science teams, Business Systems Analysts, and Architecture teams to assist with data platform technical and organizational issues and support the company’s data and analytics needs.
  • Educate, train, and mentor members of the Data Engineering and Analytics teams in the design, implementation, and usage of modern data systems.

Qualifications for Data Platform Engineer

  • IT / CS fundamentals: You have earned at least a Bachelor’s / Master’s in Information Technology / Computer Science, or related degree AND you have a strong ethos of continuous learning.
  • Commercial software engineering: You have 11+ years of professional software development experience in multiple programming languages, including modern virtual-machine languages such as Java, as well as common scripting and glue languages such as Python and version control (git), with good analytical & debugging skills. Experience performing root-cause analysis on bugs and performance problems in distributed systems, including network and source-level debugging.
  • Big data: You have extensive experience with data analytics and working knowledge of big data infrastructure such as Google Cloud, Big Query, Data Flow, Hadoop Eco System, HDFS, Apache Storm, Apache Spark. You've routinely built data pipelines with gigabytes/terabytes of data and understand the challenges of manipulating such large datasets.
  • Cloud Exposure: Strong experience implementing systems and applications using distributed and cloud infrastructure. GCP preferred, but AWS or Azure are also okay.
  • Data Modeling: Flair for data, schema, data model, PL/SQL, Star & snowflake schema, how to bring efficiency in data modeling for efficient querying data for analysis, understands criticality TDD and develops data validation techniques. Working knowledge of distributed relational and tabular data stores, message queues, stream processing facilities, and other scalable big-data platform technologies.
  • Real Time Systems: Understands evolution of databases for in-memory, NoSQL & indexing technologies along with experience on real-time & stream processing systems like Google pub/sub, GCP technologies, Kafka, Storm, Spark Streaming.
  • Strong design skills: with a proven track record of success on large/highly complex projects preferably in Enterprise Apps and Integration. Advanced, hands-on knowledge of design, implementation, and optimization of big data architectures, pipelines, and data sets.
  • Project management: Professional experience in a modern software development life cycle, including requirements gathering, system design, unit and integration testing, continuous integration, and deployment. You demonstrate excellent project and time management skills, exposure to scrum or other agile practices in JIRA.
  • Excellent verbal and written communication skills: Must be able to effectively communicate & work with fellow team members and other functional team members to coordinate & meet deliverables.

Must Have:

-Commercial software engineering 

Proficient in Java, Hands-on Coding / development experience, SQL, REST API
 

-Big Data Systems 

Google Cloud Platform technologies, 

Pub/Sub, Kafka,

Big Query, 

Dataflow,

Apache Beam, Apache Spark, Apache Storm, 

Realtime and Batch processing

(Data volume in TBs / Millions of events per day for real time)
 

-Database 

SQL and NoSQL Database (Cassandra, HBase), Google Cloud Bigtable, Redis, Oracle 

Good to have:
-Commercial software engineering 
Python, PySpark
 

-Big Data Systems 
AWS/Azure, Hadoop Eco System, HDFS, MLOps

-Database 

Exasol
 

-Tools

Tableau, PowerBI

Apply Now

Sign in with LinkedIn
Autofill my information with LinkedIn
Alert me about jobs like this

Not You?

You're now being redirected to the application website

Know someone who would be great for this job? Let them know here!

Fill in your details 

Not You?

Thank you

People Also Viewed

Sales Support Associate

Regular JR-123625 Singapore Singapore Singapore Sales Strategy and Operations Sales S4
Sales Support AssociateWho are we?Equinix is the world’s digital infrastructure company, operating 200+ data centers across the globe and providing interconnections to all the key clouds and networks. Businesses need one place to simplify and brin...

HR Shared Services Intern

Intern JR-123007 Singapore Singapore Singapore Intern Interns and Apprentices S1
HR Shared Services InternWho we areEquinix is the world’s digital infrastructure company. Digital leaders harness our trusted platform to bring together and interconnect the foundational infrastructure that powers their success.A career at Equinix...

Sales Support Associate

Regular JR-122691 Singapore Singapore Singapore Sales Strategy and Operations Sales S4
Sales Support AssociateWho are we?Equinix is the world’s digital infrastructure company, operating 200+ data centers across the globe and providing interconnections to all the key clouds and networks. Businesses need one place to simplify and brin...

Technicien Support IT

Regular JR-123582 Saint Denis France Saint Denis IBX Engineering Operations O5
Technicien Support ITEquinix, Inc. (Nasdaq: EQIX) exploite les centres de données qui offrent aux entreprises un lieu de fonctionnement fiable et d'échange sécurisé d'informations essentielles en Amérique du Nord, en Europe et en Asie-Pacifique. C...

Technicien Datacenter

Regular JR-123604 Saint Denis France Saint Denis IBX Technician Operations O3
Technicien DatacenterEquinix, Inc. (Nasdaq: EQIX) exploite les centres de données qui offrent aux entreprises un lieu de fonctionnement fiable et d'échange sécurisé d'informations essentielles en Amérique du Nord, en Europe et en Asie-Pacifique. C...

Techniker Facility Management (m/w)

Regular JR-123500 Frankfurt am Main Germany Frankfurt am Main IBX Engineering Operations O4
Techniker Facility Management (m/w)Arbeiten, wo das Internet zu Hause istEquinix verbindet weltweit mehr als 10.000 Unternehmen mit Partnern und Kunden über seine Hochleistungs-Rechenzentren mit dem weltweit größten Netzwerkangebot.Als globaler Ma...

Apply Now

Sign in with LinkedIn
Autofill my information with LinkedIn
Alert me about jobs like this

Not You?

You're now being redirected to the application website

Equal Employment Opportunity: 
Equinix is an Equal Employment Opportunity and Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, sex, sexual orientation, gender identity, age, status as a protected veteran, or status as a qualified individual with disability.

(US Applicants)

Please click here to see the “EEO is the Law” poster and supplement.

Please click here to see our EEO Policy Statement.

Please click here to see our Pay Transparency Policy Statement.