Posts on Gabes Gulch

Coordinating Agents With Mechanism Design

Fri, 31 Jul 2026 00:00:00 +0000

Imagine the large scale deployment of AI agents that make decisions on behalf of individuals or firms. For example, in online advertising auctions, an advertiser gives their high-level goal and constraints to a personal bidding agent. The responsibility of the bidding agent is to convert the advertiser’s goal and constraints into per-auction bids without human review. How do we ensure: (1) the advertiser’s goal is maximized, and (2) the outcome (who wins the auction) is efficient?

Mechanism Design - A Primer

Fri, 29 May 2026 00:00:00 +0000

This post is an ongoing investigation of my alternative view of AI and the role of algorithms for multi-agent systems in markets.

Game theory is concerned with predicting the behavior of agents participating in an strategic interaction. We can also ask the inverse. Given a desired behavior of the agents, what strategic interaction would give rise to this behavior? For example, there are two firms $A$ and $B$. Each firm can choose to cooperate $C$ or defect $D$ which results in the following payoff matrix

The Construction Versus Finance Worker

Sun, 12 Apr 2026 00:00:00 +0000

Being able to build something, or at least having some knowledge about how something is built is sort of beautiful. It’s a kind of hidden superpower. Whenever I see a new apartment building or street being built, it always crosses my mind how the people in this line of work are paid less, on average, than those in offices. It appears that the work of those building the houses and streets is more valuable than those of the management class, so why are they paid less?

Distributionally Robust Optimization - Part 1

Fri, 06 Mar 2026 00:00:00 +0000

I recently read the paper Carbon-Aware Computing for Data Centers with Probabilistic Performance Guarantees which demonstrated the benefits and power of a Distributionally Robust Optimization method and I wanted to dive deeper into the method. This blog post is a result of the deep dive.

Introduction

In Stochastic Optimization (SO), we are interested in solutions that account for different possible outcomes of uncertain data. Accounting for this uncertainty requires us to define a probability distribution over the uncertain quantities—a forecast error, a demand signal, a compute load—and then optimize an objective defined with respect to that distribution. The choice of how to use the distribution depends on the application: one might minimize expected cost, guard against a high-percentile loss using Conditional Value-at-Risk (CVaR), or enforce a chance constraint that a capacity limit is violated with probability no greater than some threshold. What all of these approaches share is the assumption that we know, or can accurately estimate, the distribution itself. Distributionally Robust Optimization is the framework we reach for when that assumption breaks down.

Stochastic Model Predictive Control

Tue, 18 Nov 2025 00:00:00 +0000

Many applications of optimization and control are performed in a deterministic setting. That is, the quantities of the problem such as the state variables, control variables, and or parameters of a model are treated as fixed known values. This assumption may be reasonable in applications such as robotics, but in others such as resource allocation, we may need to incorporate the uncertainty of various quantities in order to compute the expectation of the objective.

Reproducing Uber's Marketplace Optimization

Mon, 15 Sep 2025 00:00:00 +0000

Uber allocates money across different regions and programs to incentivize riders and drivers to use Uber products. This incentive structure ultimately influences the market. This leads to the natural question of “how much to allocate to each city and which program” to maximize business objectives? Uber has a finite amount of money that must be allocated accordingly. Given a total budget of say, $1,000,000, how should it be divied up amongst the cities and programs?

Alternative View of AI

Tue, 02 Sep 2025 00:00:00 +0000

Nowadays, AI is primarily associated with LLMs, where the goal is to develop autonomous agents with human-level intelligence (whatever that means). I have always found the goal of creating something with “intelligence” boring for some reason. Maybe it is because the objective is too diffuse—create something at least as smart as humans and hope it figures out <insert your problem here>. Regardless, these LLM-based agents are useful tools and should be used, but these systems are a narrow view of intelligence.

[WIP] Database Systems - Query Execution and Processing

Wed, 01 Jan 2025 00:00:00 +0000

Operator execution

In OLAP systems, sequential scans are the primary method for query execution. The goal is two-fold: (1) minimize the amount of data fetched from the disk or a remote object store, and (2) maximize the use of hardware resources for efficient query execution.

Andy’s (unscientific) top three execution optimization techniques:

Data parallelization (vectorization). Breaking down a query into smaller tasks and running them in parallel on different cores, threads, or nodes.
Task parallelization (multi-threading). Breaking down a query into smaller independent tasks and executing them concurrently. This allows the DBMS to take full advantage of hardware capabilities and or multiple machines to improve query execution time.
Code specialization (pre-compiled / JIT). Code generation for specific queries, e.g. JIT or pre-compiled parameters.

which fall into three primary ways for speeding up queries:

[WIP] Database Systems - Storage

Sun, 01 Dec 2024 00:00:00 +0000

Introduction

As the business landscape embraces data-driven approaches for analysis and decision-making, there is a rapid surge in the volume of data requiring storage and processing. This surge has led to the growing popularity of OLAP database systems.

An OLAP system workload is characterized by complex queries that require scanning over large portions of the database. In OLAP workloads, the database system is often analyzing and deriving new data from existing data collected on the OLTP side. In contrast, OLTP workloads are characterized by fast, relatively simple and repetitive queries that operate on a single entity at a time (usually involving an update or insert).

Creating C Callbacks with Numba and Calling Them From Rust

Sat, 30 Nov 2024 00:00:00 +0000

When interfacing with libraries written in C/C++ from Rust, it may require writing native callbacks to provide functionality or logic to the library. A C Callback is a function pointer that is passed as an argument to another function, allowing that function to “call back” and execute the passed function at runtime.

When interfacing with Python from Rust, there may be scenarios where the Rust code also needs to be able to call a Python function. Rust’s foreign function interface (FFI) and pyo3 crate in fact lets you do this. However, calling Python from Rust involves invoking the Python interpreter, which can reduce performance. If one of the goals for using Rust is to improve the performance of your application or library, this overhead might be undesirable. To avoid invoking the Python interpreter, you can use Numba. Numba allows you to create a C callback, pass this function pointer to Rust, and perform the callback without incurring the overhead associated with Python.

Database Systems - Series Overview

Tue, 22 Oct 2024 00:00:00 +0000

A blog series consisting of my notes on the Carnegie Mellon University (CMU) Introduction and Advanced Database Systems Lectures by Andy Pavlo and Jignesh Patel. The primary goal of this series is to: (1) consolidate my notes, and (2) act as a reference guide for my future self. Perhaps some readers may extract some value, but I would highly recommend watching the lectures for yourself.

The series will cover:

Database storage
Indexes
Join algorithms
Query execution and processing
Query optimization
Query scheduling and coordination
Concurrency control

OLAP database management system components

The series will primarily focus on the components of OLAP database management systems (DBMS). A recent trend of the last decade is the breakout of OLAP DBMS components into standalone services and libraries for:

Hierarchical Regression With Missing Data

Tue, 10 Sep 2024 00:00:00 +0000

Hierarchical regression, also known as multilevel modeling, is a powerful modeling technique that allows one to analyze data with a nested structure. This approach is particularly useful when dealing with data that has natural groupings, such as students within schools, patients within hospitals, or in the example below, product configurations within manufacturing processes. One of the key advantages of hierarchical regression lies in its ability to handle missing data in groups, i.e., when one group may not share the same covariates as another group or some groups may contain missong observations.

Stateful Joins in SQL

Thu, 22 Aug 2024 00:00:00 +0000

Introduction

In some scenarios, one needs to enrich an event stream with data from another source that holds “state”. This state provides additional context to the event stream.

For example, in manufacturing, a machine may use a set of machine process parameters (pressure, speed, force, etc.) when producing an item. The process parameters represent the “state” of the machine at production time $t$. However, the software services that publishes messages on what is being produced and the machine process parameters currently used are separate. Furthermore, to avoid the duplication of data, the service that publishes process parameters only publishes a message when there is a change in state, e.g when an operator changes one of process parameters.

Alternative Samplers to NUTS in Bambi

Fri, 29 Mar 2024 00:00:00 +0000

Alternative sampling backends

This blog post is a copy of the alternative samplers documentation I wrote for Bambi. The original post can be found here.

In Bambi, the sampler used is automatically selected given the type of variables used in the model. For inference, Bambi supports both MCMC and variational inference. By default, Bambi uses PyMC’s implementation of the adaptive Hamiltonian Monte Carlo (HMC) algorithm for sampling. Also known as the No-U-Turn Sampler (NUTS). This sampler is a good choice for many models. However, it is not the only sampling method, nor is PyMC the only library implementing NUTS.

Advanced Interpret Usage in Bambi

Sat, 09 Dec 2023 00:00:00 +0000

Interpret Advanced Usage

The interpret module is inspired by the R package marginaleffects and ports the core functionality of {marginaleffects} to Bambi. To close the gap of non-supported functionality in Bambi, interpret now provides a set of helper functions to aid the user in more advanced and complex analysis not covered within the comparisons, predictions, and slopes functions.

These helper functions are data_grid and select_draws. The data_grid can be used to create a pairwise grid of data points for the user to pass to model.predict. Subsequently, select_draws is used to select the draws from the posterior (or posterior predictive) group of the InferenceData object returned by the predict method that correspond to the data points that “produced” that draw.

Outcome Constraints in Bayesian Optimization

Tue, 28 Nov 2023 00:00:00 +0000

 python
 
 
#| code-fold: true
import matplotlib.pyplot as plt
import torch
import numpy as np

from botorch.acquisition import qLogExpectedImprovement
from botorch.fit import fit_gpytorch_model
from botorch.models import SingleTaskGP
from botorch.optim import optimize_acqf
from gpytorch.mlls import ExactMarginalLogLikelihood
from torch.distributions import Normal

plt.style.use("https://raw.githubusercontent.com/GStechschulte/filterjax/main/docs/styles.mplstyle")

Outcome constraints

In optimization, it is often the goal that we need to optimize an objective function while satisfying some constraints. For example, we may want to minimize the scrap rate by finding the optimal process parameters of an manufacturing machine. However, we know the scrap rate cannot be below 0. In another setting, we may want to maximize the throughput of a machine, but we know that the throughput cannot exceed the maximum belt speed of the machine. Thus, we need to find regions in the search space that both yield high objective values and satisfy these constraints. In this blog, we will focus on inequality outcome constraints. That is, the domain of the objective function is

Survival Models in Bambi

Wed, 25 Oct 2023 00:00:00 +0000

Survival Models

Survival models, also known as time-to-event models, are specialized statistical methods designed to analyze the time until the occurrence of an event of interest. In this notebook, a review of survival analysis (using non-parametric and parametric methods) and censored data is provided, followed by a survival model implementation in Bambi.

This blog post is a copy of the survival models documentation I wrote for Bambi. The original post can be found here.

Predict New Groups with Hierarchical Models in Bambi

Tue, 10 Oct 2023 00:00:00 +0000

Predict New Groups

In Bambi, it is possible to perform predictions on new, unseen, groups of data that were not in the observed data used to fit the model with the argument sample_new_groups in the model.predict() method. This is useful in the context of hierarchical modeling, where groups are assumed to be a sample from a larger group.

This blog post is a copy of the zero inflated models documentation I wrote for Bambi. The original post can be found here.

Ordinal Models in Bambi

Fri, 29 Sep 2023 00:00:00 +0000

 python
 
 
#| code-fold: true
import arviz as az
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
import numpy as np
import pandas as pd
import warnings

import bambi as bmb

warnings.filterwarnings("ignore", category=FutureWarning)

WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

Ordinal Regression

This blog post is a copy of the ordinal models documentation I wrote for Bambi. The original post can be found here.

Zero Inflated Models in Bambi

Fri, 29 Sep 2023 00:00:00 +0000

 python
 
 
#| code-fold: true
import arviz as az
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
import numpy as np
import pandas as pd
import scipy.stats as stats
import seaborn as sns
import warnings

import bambi as bmb

warnings.simplefilter(action='ignore', category=FutureWarning)

WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

Zero inflated models

This blog post is a copy of the zero inflated models documentation I wrote for Bambi. The original post can be found here.

Google Summer of Code - Final Report

Thu, 10 Aug 2023 18:43:46 +0200

My project “Better tools to interpret complex Bambi regression models” was completed under the organization of NumFOCUS, and mentors Tomás Capretto and Osvaldo Martin. Before I describe the project, objectives, and work completed, I would like to thank my mentors Tomás and Osvaldo for their precious time and support throughout the summer. They were always available and timely in communicating over Slack and GitHub, and provided valuable feedback during code reviews. Additionally, I would like to thank NumFOCUS and the Google Summer of Code (GSoC) program for providing the opportunity to work on such an open source project over the summer. It has been an invaluable experience, and I look forward to contributing to open source projects in the future.

Google Summer of Code - Average Predictive Slopes

Tue, 01 Aug 2023 00:00:00 +0000

It is currently the beginning of week ten of Google Summer of Code 2023. According to the original deliverables table outlined in my proposal, the goal was to have opened a draft PR for the basic functionality of the plot_slopes. Subsequently, week 11 was reserved to further develop the plot_slopes function, and to write tests and a notebook for the documentation, respectively.

However, at the beginning of week ten, I have a PR open with the majority of the functionality that marginaleffects has for slopes. In addition, I also exposed the slopes function, added tests, and have a PR open for the documentation.

Google Summer of Code - Average Predictive Comparisons

Fri, 30 Jun 2023 00:00:00 +0000

It is currently the end of week five of Google Summer of Code 2023. According to the original deliverables table outlined in my proposal, the goal was to have opened a draft PR for the core functionality of the plot_comparisons. Subsequently, week six and seven were to be spent further developing the plot_comparisons function, and writing tests and a demo notebook for the documentation, respectively. However, at the end of week five, I have a PR open with the majority of the functionality that marginaleffects has. In addition, I also exposed the comparisons function, added tests (which can and will be improved), and have started on documentation.

Gibbs Sampler From Scratch

Wed, 12 Oct 2022 00:00:00 +0000

A variant of the Metropolis-Hastings (MH) algorithm that uses clever proposals and is therefore more efficient (you can get a good approximate of the posterior with far fewer samples) is Gibbs sampling. A problem with MH is the need to choose the proposal distribution, and the fact that the acceptance rate may be low.

The improvement arises from adaptive proposals in which the distribution of proposed parameter values adjusts itself intelligently, depending upon the parameter values at the moment. This dependence upon the parameters at that moment is an exploitation of conditional independence properties of a graphical model to automatically create a good proposal, with acceptance probability equal to one.

Metropolis Hastings Sampler From Scratch

Sat, 08 Oct 2022 00:00:00 +0000

Main Idea

Metropolis-Hastings (MH) is one of the simplest kinds of MCMC algorithms. The idea with MH is that at each step, we propose to move from the current state $x$ to a new state $x’$ with probability $q(x’|x)$, where $q$ is the proposal distribution. The user is free to choose the proposal distribution and the choice of the proposal is dependent on the form of the target distribution. Once a proposal has been made to move to $x’$, we then decide whether to accept or reject the proposal according to some rule. If the proposal is accepted, the new state is $x’$, else the new state is the same as the current state $x$.

Monte Carlo Approximation

Fri, 07 Oct 2022 00:00:00 +0000

Inference

In the probabilistic approach to machine learning, all unknown quantities—predictions about the future, hidden states of a system, or parameters of a model—are treated as random variables, and endowed with probability distributions. The process of inference corresponds to computing the posterior distribution over these quantities, conditioning on whatever data is available. Given that the posterior is a probability distribution, we can draw samples from it. The samples in this case are parameter values. The Bayesian formalism treats parameter distributions as the degrees of relative plausibility, i.e., if this parameter is chosen, how likely is the data to have arisen? We use Bayes’ rule for this process of inference. Let $h$ represent the uknown variables and $D$ the known variables, i.e., the data. Given a likelihood $p(D|h)$ and a prior $p(h)$, we can compute the posterior $p(h|D)$ using Bayes’ rule:

Variational Inference - Evidence Lower Bound

Fri, 03 Jun 2022 18:43:46 +0200

We don’t know the real posterior so we are going to choose a distribution $Q(\theta)$ from a family of distributions $Q^*$ that are easy to work with and parameterized by $\theta$. The approximate distribution should be as close as possible to the true posterior. This closeness is measured using KL-Divergence. If we have the joint $p(x, z)$ where $x$ is some observed data, the goal is to perform inference: given what we have observed, what can we infer about the latent states?, i.e , we want the posterior.

No Code, Dependency, and Building Technology

Tue, 10 Aug 2021 00:00:00 +0000

Modernity and Abstraction

‘Programmers’, loosely speaking, in some form or another have always been developing software to automate tedious and repetitive tasks. Rightly so, as this is one of the tasks computers are designed to perform. As science and technology progresses, and gets more technological, there is a growing seperation between the maker and the user. This is one of the negative externalities of modernism - we enjoy the benefits of a more advanced and technologically adept society, but fewer and fewer people understand the inner workings. Andrej Karpathy has a jokingly short paragraph in his blog on the matter, “A courageous developer has taken the burden of understanding query strings, urls, GET/POST requests, HTTP connections, and so on from you and largely hidden the complexity behind a few lines of code. This is what we are now familiar with and expect”.