Backend Engineer Collection Platform

About the Role

At Vannevar we build mission software for the national security defense and intelligence communities.

The Collection Platform team builds the systems that acquire mission-critical data from websites APIs feeds and third-party platforms.

In this role you’ll own web scrapers third-party integrations and data pipelines that produce complete correct and timely data despite unreliable upstream systems.

You’ll partner with mission teams product managers and engineers to define requirements and make tradeoffs among data quality reliability scale and cost.

What you’ll do

• Build and operate production web scrapers third-party integrations and data pipelines.
• Design for schema changes rate limits authentication requirements and partial failures.
• Build reliable retry backfill deduplication idempotency and replay mechanisms.
• Define and monitor data completeness correctness freshness duplication and provenance.
• Develop APIs databases and shared services for collection workflows.
• Partner with mission teams platform teams and product managers to evaluate sources and support new data requirements.
• Influence architecture technology selection and engineering standards for collection systems.

What we’re looking for

• You have 4+ years of backend engineering experience including production web scraping data ingestion ETL or third-party integrations.
• You write production software in Python and are familiar with relational databases and API design.
• You know how to maintain data quality across large changing datasets.
• You have designed systems that tolerate upstream failures and changing external interfaces.
• You can make thoughtful tradeoffs among reliability scalability performance maintainability and cost.
• You communicate technical decisions clearly and work well across engineering product and mission teams.
• U.S.

  Person status is required as this role requires access to U.S.-only data systems and the ability to obtain a security clearance.

Nice to Have

• Experience with distributed task processing workflow orchestration or large-scale data pipelines.
• Experience monitoring data quality and investigating production regressions.

What we offer

• Health dental and vision insurance
• Remote friendly with WeWork access
• Unlimited PTO shared downtime during the federal holiday calendar and company-wide off time at the end of each year
• 401(k) match
• Lifestyle & wellbeing stipends
• Salary top-up during military reserve duty
• Fully paid parental leave
• Child and pet care reimbursement during travel