How Upside Unlocked Faster Model Innovation and Impact by Partnering with Qwak

Upside was faced with the decision of either continuing to enhance their own ML platform, or adopting a third-party ML platform. They conducted a comprehensive Proof of Concept with Qwak, evaluating product functionality, architecture, cost, vendor health, and its impact on the data science and ML engineering teams.

Ultimately, they were looking for an ML platform that would address the following challenges:

• Enabling self-service building for data scientists; removing roadblocks that stall development
• Rapid iteration - the ability to build, deploy and update models with ease
• Experimentation - having a clear, simple interface that allows different variations of a given model to be run, compared, and analyzed in parallel
• Production grade - In addition to general platform concerns like resource scaling, performance, and reliability, they also needed a system that could easily keep track of historical inference data and feature sets for auditing, rollback, and ad-hoc analysis 
• Comprehensive - The team was looking for an MLOPS platform that didn’t require stitching together many different point solutions. They needed a solution for model registry, inference, feature store, monitoring and alerting, CI/CD, and they wanted it all in one place. 

As an organization, Upside also maintains a strong commitment to source control, test driven development, and CI/CD. They wanted an MLOPS platform that would enforce and encourage good development practices that matched the standards already put in place by the rest of their engineering team.

Upside implemented Qwak's MLOps platform after assessing it could accelerate model deployment, improve quality, and simplify operations. Key implementation steps included:

• Migrating models into Qwak for full lifecycle management
• Building CI/CD pipelines with Qwak APIs for automated ML deployments
• Configuring Qwak's feature store to serve model data for training and real time inference 
• Leveraging Qwak monitoring and alerts for model performance  and quality 
• Setting up multi-environment architecture for dev, test, and prod

Model Build and Deploy: The Qwak build registry standardizes the ML project structure allowing every team member to adopt the same code structure, data versioning and testing capabilities for every model and version. When it’s time for deployment, with the click of a button they can turn a model experiment or variation into a production grade, real-time REST endpoint, with monitoring, alerting, and auto-scaling immediately available.  

Shadow Deployments: Qwak allows teams to deploy models in shadow mode. When testing out a new model variation or preparing the release of a new model, Qwak will send a copy of your production traffic to the shadow variant, allowing the model to run silently on real production data. When confident with the model’s performance, you can seamlessly move the model from shadow to production so that it can replace the existing variation and start receiving real traffic. 

CI/CD: When it’s time to make an update to an existing model, Upside team members tie new model variations to Github pull requests, dynamically creating and training new models multiple times a day without any changes to configuration or infrastructure management. For production releases, Upside also utilizes Qwak’s protected variation feature, ensuring that production inference deployments can only be removed or edited by an elevated permission user or authentication token. 

Model Monitoring: Qwak offers extensive insight into the performance of inference services. Upside heavily utilizes the prediction timers feature of Qwak, adding additional tracing layers that breakdown the latency and performance of each step of their prediction logic. 

Feature Store: Upside Data Scientists have the ability to freely create and manage their feature engineering, removing the dependency on DevOps teams. Upside heavily utilizes Snowflake as their underlying data storage layer, and the ability to create advanced feature pipelines directly on top of their Snowflake tables significantly increases the efficiency and reduces the complexity of managing features for machine learning models. Moreover, Upside now have visibility of their feature freshness, lineage and distribution all in one view. 

With Qwak, Upside was able to:

• Reduce model deployment cycles from quarters to 1-2 weeks
• Speed up model development by ~30% for data scientists
• Cut engineering delivery time by 20% with simplified platform ops
• Improve model accuracy and business impact with faster iteration
• Enable self-service access for data scientists to deploy models
• Establish robust CI/CD pipelines for automated deployments
• Gain integrated visibility into model metrics and alerts

By implementing Qwak's MLOps platform, Upside accelerated their machine learning efforts through faster model development, deployment, and collaboration between teams. This helped them enhance their core algorithms to uncover even more savings for businesses.