CertivityPosted 12h ago
Full-Stack Engineer | Web Crawling (m/f/d)
München - HQ
Primary technologies, domain competencies, and required qualifications
Verified specifications, primary responsibilities, and qualifications
We build and operate large-scale web crawling and extraction systems that turn unstructured, dynamic websites into clean, structured, and correct data. You'll own crawlers end-to-end: discovery, resilient fetching against real anti-bot defenses, and turning HTML/PDF into trustworthy structured output.
What you will do• Build and scale crawlers that handle dynamic, large-scale sites - and keep them running as those sites change and fight back.• Stay ahead of anti-bot measures: TLS/browser fingerprinting, proxies, session and rate-limit strategy, change detection.• Turn raw HTML and PDF into clean, correct structured data - including LLMassisted extraction - where silently-wrong output is worse than a crash.• Design and operate pipelines for ingestion, deduplication, versioning, and enrichment.• Ship and run containerized apps in the cloud, with CI/CD.• Shape our technical architecture and engineering culture.
Our Tech StackPython, Docker, GCP (Cloud Run jobs) and Azure Blob, MongoDB. Go services and a React frontend on the periphery. You don't need all of it on day one, but you do need to ramp up fast
Your profile• 3+ years of professional software development (5+ for the senior track).• Strong Python - real production code, async.• Hands-on web scraping at scale, including firsthand experience with anti-bot defenses and the failure modes of large-scale collection.• A data-correctness mindset - you care that extracted data is right, not just that the job finished.• Docker + at least one major cloud (GCP, Azure, or AWS), Git, comfortable on the CLI.• Clear communicator who works well in a small, fast-moving team.Nice to have• LLM-assisted extraction / working with LLMs, ML pipelines, or data labeling workflows.• Go or Java/Spring Boot.• Frontend (React) — or eagerness to pick it up.• Terraform, Kubernetes, modern DevOps tooling.
Why us?Certivity is an equal opportunities employer. We are committed to equal employment opportunities regardless of race, religion, sexual orientation, age, marital status, disability, or gender identity. Please do not submit personal data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, data concerning your health, or data concerning your sexual orientation.
Find Jobs in Germany on Arbeitnow
Ready to apply?
Submit your application directly through the employer portal.
We are developing the digital twin of commercial aviation, empowering our clients with fully integrated 360° industry data to drive greater efficiency and sustainability. ✈ The Cha
En bref Chez Picnic, nous ne faisons pas que réinventer les courses en ligne : nous grandissons vite et continuons à nous développer dans de nouvelles zones de livraison chaque moi
Application Performance Monitoring (APM) is one of Datadog’s premier products that helps developers see inside complex distributed systems. Our SDKs are both the delivery vehicle f
WHY JOIN ALO? Mindful movement. It’s at the core of why we do what we do at ALO—it’s our calling. Because mindful movement in the studio leads to better living. It changes who yogi