Lead/Principal Data Engineer

We're a Belgrade-based company that uses software to solve real business challenges. We take the time to understand our clients' businesses properly, and that's what lets us find and build solutions that actually fit their market. We're proud of what we create and we love doing it, and the trust our clients give us is something we've earned that way.

About the project

Tens of thousands of clinics. Tens of millions of pet owners. One data platform that doesn’t exist yet.

Our client is a US leader in veterinary technology, and Deversity has been their engineering partner for ten years. Their platform connects tens of thousands of veterinary clinics with an online pharmacy serving tens of millions of pet owners. Every prescription, refill and repeat order is a signal about pet health, clinic performance, and owner behavior. Today, most of that signal sits in separate silos.

We’re changing that. The goal is to turn the organization into one where data is a first-class citizen: streamed in near real time, governed by clear contracts, owned by the domains that produce it, and ready to power data-driven products for clinics and pet owners.

You’ll be shaping architecture at real scale, inside a long-term partnership, in a domain where the products you enable help clinics keep pets healthier.

What you'll do

• Define the target architecture for an event-driven data platform on Kafka and Flink, from ingestion to serving.
• Design near real-time pipelines that join clinic data with pharmacy order streams at scale.
• Own the schema registry strategy: formats, compatibility rules, versioning, and how schemas evolve without breaking consumers.
• Introduce data contracts between producing and consuming teams, covering ownership, quality expectations and SLAs.
• Model how clinics, pets, owners and orders connect across systems that were built separately.
• Connect the streaming layer with the existing analytics stack (Snowflake, dbt, Hex on GCP) so products and analytics work from the same trusted data.
• Set engineering standards for delivery semantics, late and out-of-order events, replay and backfill, and observability.
• Decide where streaming is worth its complexity and where batch is enough.
• Lead design reviews, write architecture decision records, and mentor other engineers.
• Work with engineering and product leadership to help domain teams take real ownership of their data.

What we expect

• 10+ years in data engineering, with significant time at senior, staff or principal level designing enterprise-scale data platforms.
• Experience across several companies, so you’ve seen these patterns work and fail in different organizations.
• Production experience with Kafka: topic and partition design, retention, consumer groups, Kafka Connect, and operating it at scale.
• Change data capture experience, preferably with Kafka Streams.
• Hands-on data mesh or domain-ownership implementation.
• Production experience with Flink: stateful processing, windowing, watermarks, checkpointing and state management.
• Hands-on schema registry experience, including schema evolution in live systems.
• Data contracts implemented across real teams, not just read about.
• Battle scars. You can walk us through a streaming or data platform initiative that went wrong, why it happened, and what you’d do differently now.
• Strong data modeling across both streaming and analytical use cases.
• The ability to explain trade-offs clearly to engineers and executives, in writing and in person.
• Bachelor’s degree in computer science, data science, engineering, or a related field, or equivalent experience
• Fluent English for daily work with a US-based client.

Nice to have

• Experience with Snowflake, dbt, Hex, AWS, and GCP.
• Data quality, lineage and observability tooling (for example Great Expectations, Monte Carlo, OpenLineage).
• Lakehouse table formats such as Apache Iceberg.
• Background in healthcare, pharmacy, e-commerce or other domains with sensitive or regulated data.
• Strong Java or Groovy alongside Python.
• Public talks, writing or open source contributions in the data engineering space.

Izvor: https://www.linkedin.com/jobs/view/4480674250/

Nivo: senior