Solutions Architect

Description

The Solutions Architect will lead the design and deployment of AI-powered applications directly on the Azure Databricks platform. This individual will architect solutions that run natively on the Databricks serverless environment—eliminating the need for separate hosting infrastructure—while integrating tightly with core platform services for data governance, querying, authentication, and model serving. The role blends hands-on technical architecture with cross-functional leadership across data, security, and application teams.

CampusWorks consultants engage with our clients in higher education to assist them in fully leveraging their people, processes, policies, and technologies. We work collaboratively with clients' functional and technical teams and our CampusWorks colleagues to evaluate current processes, design future-state optimal processes, and to architect, develop, test, train, and rollout process and technical solutions that delight our clients. As a 1099 project-based consultant, you will enjoy the flexibility of remote work while contributing to the success of leading intstitutions.

Responsibilities:

• Lead end-to-end deployment of AI applications on the Databricks Apps platform, from architecture design through production rollout and ongoing maintenance.
• Architect serverless-first solutions that leverage Databricks Apps to eliminate external hosting infrastructure, inheriting the platform’s built-in security, compliance, and resource management capabilities.
• Design and integrate AI agents (e.g., Retrieval-Augmented Generation / RAG patterns) using Databricks Agent tooling, and manage the full agent lifecycle—creation, deployment, testing, and monitoring.
• Establish data governance through Unity Catalog for centralized access control, lineage, and security across application data assets.
• Enable efficient data access by integrating applications with Databricks SQL for querying large datasets at scale.
• Implement secure authentication using OAuth and platform-native identity and authorization patterns.
• Configure and optimize Model Serving and Serving Endpoints for deploying and querying ML models and LLM agents at scale.
• Collaborate with data engineering, security, and business stakeholders to translate requirements into scalable, governed, production-ready solutions.
• Define architecture standards, reference patterns, and best practices for AI application development on Databricks.

Requirements

Databricks SQL	Designing and optimizing queries against large datasets.
OAuth / IAM	Implementing secure authentication and authorization flows.
Model Serving & Endpoints	Deploying and scaling ML models and LLM agents via serving endpoints.
AI Agent Frameworks	RAG architectures and frameworks such as LangChain and/or LlamaIndex.
MLflow	Logging, testing, debugging, and monitoring agents and models.
Application Frameworks	FastAPI, Flask, Streamlit, Dash, or Gradio; front-end experience with React or a comparable modern JavaScript framework.
Programming	Strong Python proficiency for application, agent, and pipeline development.
Data Pipelines / ETL	Experience with Databricks Jobs and workflow orchestration.
Cloud Architecture	Microsoft Azure networking, security, and compliance fundamentals.

Preferred:

• Prior experience delivering enterprise chatbot or conversational AI solutions.
• Familiarity with token streaming, request logging, and review-app patterns for production AI agents.
• Industry-specific solution experience (e.g., operations management, analytics-heavy domains).
• Relevant certifications (e.g., Databricks Certified Data Engineer / ML, Azure Solutions Architect)

Core Competencies:

• Strong architectural judgment balancing scalability, security, governance, and cost.
• Ability to lead technical delivery and mentor engineering teams.
• Clear communication with both technical and executive/business stakeholders.
• Bias toward practical, outcome-driven solutions.