AI in Enterprise

Start with business requirements: false negative rate <0.1% (catch fraud), false positive rate <2% (user friction). Then explain architecture: 1) Real-time data pipeline ingesting transactions into Kafka, 2) Feature store serving pre-computed customer behavior features, 3) Ensemble of models (gradient boosted trees for structured features, RNNs for time-series patterns, isolation forests for anomalies), 4) Sub-100ms serving latency requirement using edge caching, 5) Continuous monitoring for model drift and performance degradation, 6) A/B testing framework for model improvements with statistical significance testing, 7) Explainability component (SHAP values) to explain why transaction flagged for customer support, 8) Automated retraining triggered by PSI > 0.1 indicating data drift, 9) Cost optimization using quantization and batch inference where possible.

Discuss technical and business challenges: 1) Cost - inference on 70B+ parameter models is $0.001-$0.01 per request; mitigate with LoRA fine-tuning instead of full fine-tuning (reduces VRAM by 100x), retrieval-augmented generation (RAG) to avoid fine-tuning, prompt caching, and model quantization; 2) Latency - models take 5-10s per response in naive implementation; mitigate with speculative decoding, token streaming (show results as they're generated), and caching; 3) Hallucination - models confidently make up false information; mitigate with RAG using authoritative sources, grounding with external tools, and confidence scoring; 4) Safety & Alignment - harmful outputs possible; implement input/output filtering, jailbreak-resistant prompting, and red-team testing; 5) Data privacy - customer data sent to external APIs is risky; use self-hosted open-source models (Llama, Mistral) for sensitive data; 6) Monitoring - traditional ML metrics don't work; monitor output relevance, toxicity, informativeness using separate evaluator models.

Explain complete lifecycle with specific tools and practices: 1) Version control: Git for code, DVC for large artifacts (data, models), feature versioning in feature store; 2) Experiment tracking: MLflow to track 100+ experiments per week, compare model performance, manage hyperparameters; 3) Automated training pipeline: Apache Airflow orchestrating daily retraining triggered by data drift detection, automated hyperparameter tuning (Ray Tune); 4) Model registry: MLflow Model Registry storing model artifacts, performance metrics, fairness audits, deployment history for audit trail; 5) Safe deployment: Canary rollout starting at 5% traffic, monitoring for latency increase, accuracy drop; gradually scale to 100% if healthy; 6) Real-time serving: KServe handling 100K+ predictions/sec with auto-scaling, model versioning, A/B test routing; 7) Comprehensive monitoring: 100+ metrics tracked (accuracy, latency, fairness), data drift detection (KL divergence), model drift detection (performance degradation), automated alerts; 8) Automated rollback: If model accuracy drops >1%, latency spikes >50%, or error rate >1%, automatically rollback to previous version; 9) Infrastructure: Kubernetes for orchestration, GPUs for training, CPUs for serving, Spark for batch, Kafka for streaming; 10) Cost optimization: Use spot instances for training, batch inference for non-urgent predictions, model compression.

Explain that ROI requires tying ML to business metrics, not just model accuracy: 1) Define baseline: Current state before ML (conversion rate, churn rate, cost, revenue). Example: Current churn prediction accuracy is 60%, costs $100K annually to contact low-value customers. 2) Measure impact: Post-deployment, measure actual business improvement. Isolate using A/B testing. Example: Improved churn model increases precision to 85%, reduces wasted outreach costs by $60K annually, while recovering $500K in retention value. 3) Calculate ROI = (benefit - cost) / cost. Example: Total benefit $560K, ML costs $200K (compute, salaries, tools), ROI = 180%. 4) Payback period: 200K / (560K/12) = 4.3 months. 5) Track leading indicators (model accuracy) and lagging indicators (business impact separately). Don't confuse the two. 6) Communicate simply: 'We invested $200K in ML, it generated $560K in value, that's 2.8x return'. Report monthly. 7) Attribute correctly: Use control groups and A/B testing to prove your model caused the improvement, not other factors.

Explain these critical concepts: Data drift occurs when input feature distribution changes over time, causing model performance degradation. Model drift occurs when relationship between inputs and outputs changes. Example: Fraud model trained on pre-pandemic data doesn't work well during recession when spending patterns change dramatically. Detection methods: 1) Statistical tests: Kolmogorov-Smirnov test, Population Stability Index (PSI > 0.1 signals drift), Chi-square test for categorical features; 2) Visualization: Compare distributions with histograms, quantile plots, box plots; 3) Performance monitoring: Track accuracy, precision, recall; if degrading without clear cause, likely drift; 4) Automated triggers: Set up daily checks, alert if drift detected. Handling: 1) Automated retraining: Retrain immediately when drift detected, A/B test new model, deploy if performance better; 2) Online learning: Update model parameters continuously with new data instead of retraining from scratch; 3) Adaptive thresholds: Adjust decision thresholds based on new data distribution; 4) Monitoring frequency depends on use case: Real-time systems (fraud) monitor every hour; batch systems (churn) monitor daily. Typical enterprise retrains models monthly.

Feature stores solve a critical problem: feature engineering is repeated across teams, causing inconsistency and wasted compute. Design considerations: 1) What is a feature store: Centralized system that computes features once, stores them, serves to multiple models for training and serving. Eliminates training/serving skew. 2) Architecture: Online store (Redis for <100ms latency) serves features for real-time predictions; offline store (S3, BigQuery) serves for batch training. 3) Key features: Version control (know which features used to train which model), monitoring (track feature quality and staleness), lineage (trace feature back to source data), discovery (data scientists can browse available features). 4) Example workflow: Data engineer writes feature definition (SQL), feature store computes daily and stores both online and offline; ML engineer requests features for model training; data scientist uses same features for inference; no skew between train and serve. 5) Benefits: 30-50% reduction in ML development time, prevents bugs from feature discrepancy, cost savings from deduplication. 6) Tools: Feast (open-source), Tecton (enterprise), Chip (data-centric). 7) Challenges: Data quality, staleness, cost of maintaining online and offline stores.

Bias and fairness are critical for ethics and legal compliance: 1) Define fairness for your use case: Group fairness (no disparate impact on protected groups), individual fairness (similar individuals treated similarly), calibration (predicted probability matches actual rate); 2) Audit data: Check for historical discrimination in labels (hiring models trained on biased hiring patterns perpetuate bias), demographic imbalance (does training data represent all populations); 3) Measure fairness: Compute metrics like demographic parity (same approval rate across groups), equalized odds (same false positive rate across groups), calibration; 4) Mitigate bias: Rebalancing training data, fair representation learning, threshold adjustment; 5) Example: Credit lending model that rejects minorities at higher rate must be detected (fairness audit), understood (why is this happening?), and fixed (retrain with balanced data or adjust thresholds); 6) Production monitoring: Continuously audit fairness, especially as data distribution changes; 7) Transparency: Document model limitations, get informed consent, provide explanation when decisions made, offer appeals process; 8) Tools: Fairlearn, AI Fairness Toolkit, What-If Tool.

This is a critical decision affecting infrastructure cost and reliability: 1) Simple models (logistic regression, decision trees) are fast, interpretable, easy to monitor, but may underfit and have lower accuracy. 2) Complex models (deep learning, large ensembles) can be very accurate but require: 10x more compute (GPU training + serving), longer training time, harder to debug and explain, higher latency (model inference time), more careful monitoring (more ways to fail). 3) Real question: Is 2% accuracy improvement worth 10x compute cost? Almost never. 4) Enterprise best practice: Start with simple baseline (logistic regression). Only move to complex model if: simple model clearly insufficient AND business value justifies cost AND team has expertise to maintain it. 5) Example: Netflix doesn't use single best model; uses ensemble of 10-100 simple models because simpler to manage at scale. 6) Strategy: Use transfer learning to reduce complexity (fine-tune pre-trained model instead of training from scratch). Use model compression (quantization, pruning) to reduce inference cost. 7) Monitoring: Complex models need 10x more monitoring because more failure modes. 8) Rule of thumb: 60% of time/effort goes to data and features, 20% to modeling, 20% to infrastructure and operations. If you're spending 50% on modeling, your architecture is wrong.