Senior Software Engineer (Data Platform Engineer, Online Analytical Processing)

• Sitting at the intersection of Data Engineering, Backend Engineering, and Systems Engineering, Data Platform Engineers at Adyen build the foundational layer of tooling and processes for our on-premise Big Data Platforms.
• These tools support 10s of products, 100s of developers, and 1000s of daily jobs that add to Adyen’s strong portfolio of capabilities
• You’ll collaborate with data engineers and ML scientists and engineers to build and roll-out these tools.
• Along with DataOps Engineers, you’ll develop and operate multiple big data platforms, of which the biggest is 500+ nodes, with 25k+ CPU cores, 240+ TBs of RAM, and 23+ PBs of storage capacity, running tens of thousands of spark jobs a day
• We are currently expanding our platform capabilities. Depending on your expertise, your immediate focus will be driving one of these two key ecosystems:
• The OLAP Ecosystem: Scaling and optimizing our real-time analytical powerhouses (Druid, FastAPI, Java, Kafka)
• The Data Engineering Ecosystem: Designing and building data contracts, validation frameworks, optimization tooling, and robust Kafka ingestion pipelines
• You will balance building innovative new tools from scratch with relentlessly improving our existing infrastructure
• Scale: Developing and maintaining high-performance, distributed big data platforms
• Distributed Systems: Working deeply with various flavors of databases, filesystems, and compute engines
• Data Quality & Discoverability: Building tools that enhance metadata collection, data lineage, and platform-wide data quality
• CI/CD & Infrastructure: Optimizing continuous release/deployment environments within a large-scale private cloud system
• Streaming: Developing stream processing applications and frameworks to reduce time-to-insight for analysts and data scientists

Benefits

• Global exchange program
• Weekly happy hour
• Delicious healthy lunches
• Phantom share package
• Yearly trip to Amsterdam
• Paid holidays
• Work from home opportunities- Python or Java Expert: 6+ years of experience with object-oriented programming
• Distributed compute/data systems (e.g., Spark, Trino, Druid, ClickHouse)
• Collaborative Enabler: A strong communicator who thrives on building bridges between the teams you enable (analysts, data scientists) and the teams you depend on (infrastructure, security)
• Real-time and batch pipelines (Kafka, Spark Streaming) with a focus on user-friendliness and reliability
• Distributed Systems Architect: Proven experience developing and maintaining:- Ensuring a smooth and enjoyable candidate experience is critical for us. We aim to get back to you regarding your application within 5 business days. Our interview process tends to take about 4 weeks to complete, but may fluctuate depending on the role. Learn more about our hiring process here. Don’t be afraid to let us know if you need more flexibility