Back to all jobs
Data & AI
Full-time
October 26, 2026

Senior Software Engineer, AI Gateway (Machine Learning)

๐ŸŒ Milan, Italyโฐ Full-time๐Ÿ“… Apply by May 18, 2027
๐Ÿค– Data & AIFull-time
Kong ยท Direct from employer
Milan, Italy
NEW Join our Discord for daily job alerts! โ†’

Kong is hiring a

Senior Software Engineer, AI Gateway (Machine Learning)

Description

Are you ready to unlock intelligence?

If you donโ€™t think you meet all of the criteria below but are still interested in the job, please apply. Nobody checks every box - weโ€™re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others.

The Mission

The AI Gateway team is on a mission to build the central nervous system for the agentic era, enabling developers to securely and reliably connect, observe, and manage their AI and ML services at scale. Some of the gatewayโ€™s more advanced capabilities, such as intelligent model routing, semantic caching, request classification, and content guardrails, work best when backed by small, purpose-built models. In this role you will build those features end to end: the gateway code that runs them in the request path, and the models behind them.

What Youโ€™ll Do

  • Own the technical direction for model-backed AI Gateway features, from design to production rollout.

  • Build and maintain high-performance components for Kongโ€™s AI Gateway in Rust and Lua.

  • Train, fine-tune, and evaluate models for specific gateway use cases, such as classifiers for intelligent routing, embeddings for semantic caching, and guardrail models.

  • Curate and maintain the datasets used for training and evaluation, and define the metrics and thresholds that decide whether a model is good enough to ship.

  • Integrate models into the gateway request path, optimizing inference for latency, throughput, and memory footprint.

  • Troubleshoot bugs, tune performance, and do low-level optimizations across both the gateway and the model serving stack.

  • Work with product management and other engineering teams to turn model capabilities into reliable, documented product features.

  • Mentor other engineers on the team on ML practices and model integration.

Requirements

The Toolkit

  • 3+ years of experience in server-side software development: building and debugging applications that are highly scalable, distributed, reliable and resilient.

  • 2+ years of experience in machine learning: building, training and evaluating models for production or research.

  • Proficiency with tensor-based deep learning frameworks, in Python and beyond.

  • Experience with dataset curation and evaluation methodology.

  • Experience running models in production outside of Python, optimizing inference for latency and memory (e.g. ONNX Runtime, quantization, batching).

  • Working knowledge of embeddings and vector databases.

  • Hands-on expertise with Linux systems.

The Kong DNA

  • Ownership: You take full ownership of your work from concept to production, including the documentation: engineers at Kong write the documentation for their products.

  • Pragmatism: You can tell when a model is the right tool and when a simpler solution is, and you measure before you decide.

  • Collaboration: You communicate clearly in writing and work effectively with a remotely distributed team.

  • Leadership: You make and explain technical trade-offs where there is no established playbook.

Bonus Points

  • Contributions to open-source software projects.

  • Proficiency in Rust, Lua or C.

  • Familiarity with the Rust ML ecosystem (e.g. Candle, Burn, tokenizers).

  • Experience with MLOps tooling and pipelines for training, versioning, and deploying models.

  • Experience with experiment tracking tools (e.g. MLflow, Weights & Biases).

  • Experience with LLM post-training techniques such as LoRA or other parameter-efficient fine-tuning methods.

  • Knowledge of networking fundamentals and protocols on both L4 and L7 (TCP/IP, HTTP/2, gRPC).

  • Hands-on experience with NGINX and OpenResty.

#LI-EA1

About Kong:

Kong Inc., the AI Connectivity Company, is building the connectivity layer of AI. Trusted by the Fortune 500ยฎ and AI-native startups alike, Kongโ€™s unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic โ€” on any model, any cloud. For more information, visit www.konghq.com.

Apply for this job

You will be sent to the employer's application page.

Questions about this role

Where can I work from for this Senior Software Engineer, AI Gateway (Machine Learning) role?

Kong lists the location as "Milan, Italy". If your country is not named, check the job description for eligibility rules before applying.

Is this a fully remote job?

Yes. Every role on Euro Remote Talent is remote. This one is full-time, for candidates based in Milan, Italy.

What does this role pay?

Kong has not published a salary for this role. Ask about the range in your first conversation; our salary calculator helps you set expectations.

How do I apply to Kong?

Click Apply now. Create an account if you don't have one, or log in to your account, then apply to the job. Applications close on May 18, 2027.

Kong

Fintech, Software, Technology

Apply now

About Kong

Kong provides API and AI connectivity technologies, including API management, AI management, and event management, for platform teams, executives, AI teams, and developers.

They offer various products and services, such as API Gateway, AI Gateway, and Kong Mesh, to support businesses in industries like financial servic

๐Ÿญ Industry
Fintech, Software, Technology
All jobs at Kong โ†’
Job summary
  • TypeFull-time
  • LocationMilan, Italy
  • CategoryData & AI
  • PostedOctober 26, 2026
  • DeadlineMay 18, 2027
About the company

Kong provides API and AI connectivity technologies, including API management, AI management, and event management, for platform teams, executives, AI teams, and developers. They offer various products and services, such as API Gateway, AI Gateway, and Kong Mesh, to support businesses in industries like financial servic

Kong

Senior Software Engineer, AI Gateway (Machine Learning)

Apply now
Share this job
00
October 26, 2026
May 18, 2027

Section heading style, do not delete

Sub heading style, do not delete

apply
Julien Sicard
Book A Call
Get started today

For Employers

For Job Seekers

Remote Jobs by Category

Remote Jobs by Country

Resources

Collaboration

Legal

Compare

About Euro Remote Talent

Connect

Address

Euro Remote Talent LLC
30 N Gould St Ste N
Sheridan, WY 82801, USA