Howto’s – Pedro Leitão

A Primer on Spark for Data Processing

a five minute introduction to Spark and PySpark

Data Processing

A quick introduction to Apache Spark, its components, and how to use PySpark for data processing tasks.

Reasoning Models for Fun and Profit

deepseek r1 and the chinese room

Language Models

Since the advent of GPT-3, foundation models have rapidly progressed from single pass transformer models, to multi-step models that can reason over…

Model Fine-tuning with the Hugging Face `transformers` Library

the basics of traditional fine-tuning

Language Models

Previously, we learned how to use Apple’s MLX framework to fine-tune a language model. This is an Apple specific framework and is not available to…

Fine-tuning an LLM with Apple’s MLX Framework

fine-tuning pre-trained language models in apple silicon

Language Models

Modern GPU’s come with inbuilt memory, which is separate from the CPU’s memory. This means that when training large models, the data has to be…

Model Management with MLflow

how to log, compare, and deploy machine learning models consistently with mlflow

Machine Learning

Model Management

As you develop machine learning models, you will find that you need to manage many different versions and variations as you move towards the desired…

Caching long running jobs

how to cache the results of long running computations in python

In any data science or machine learning pipeline, one often has to re-try and experiment with long running computations. For example, maybe you will…