16 Sep 2019
https://www.oranlooney.com/post/ml-from-scratch-part-6-pca/
In the previous article in this series we distinguished between two kinds of unsupervised learning (cluster analysis and dimensionality reduction) and discussed the former in some detail. In this installment we turn our attention to the later.
In dimensionality reduction we seek a function $f : \mathbb{R}^n \mapsto \mathbb{R}^m$ where $n$ is the dimension of the original data $\mathbf{X}$ and $m$ is less than or equal to $n$.
05 Jun 2019
https://www.oranlooney.com/post/ml-from-scratch-part-5-gmm/
Consider the following motivating dataset:
It is apparent that these data have some kind of structure; which is to say, they certainly are not drawn from a uniform or other simple distribution. In particular, there is at least one cluster of data in the lower right which is clearly separate from the rest.
19 Feb 2019
https://www.oranlooney.com/post/fibonacci/
A common example of recursion is the function to calculate the $n$-th Fibonacci number:
def naive_fib(n): if n < 2: return n else: return naive_fib(n-1) + naive_fib(n-2) This follows the mathematical definition very closely but it’s performance is terrible: roughly $\mathcal{O}(2^n)$. This is commonly patched up with dynamic programming.
03 Feb 2019
https://www.oranlooney.com/post/ml-from-scratch-part-3-backpropagation/
In today’s installment of Machine Learning From Scratch we’ll build on the logistic regression from last time to create a classifier which is able to automatically represent non-linear relationships and interactions between features: the neural network. In particular I want to focus on one central algorithm which allows us to apply gradient descent to deep neural networks: the backpropagation algorithm.
27 Dec 2018
https://www.oranlooney.com/post/ml-from-scratch-part-2-logistic-regression/
In this second installment of the machine learning from scratch we switch the point of view from regression to classification: instead of estimating a number, we will be trying to guess which of 2 possible classes a given input belongs to. A modern example is looking at a photo and deciding if its a cat or a dog.
29 Nov 2018
https://www.oranlooney.com/post/ml-from-scratch-part-1-linear-regression/
To kick off this series, will start with something simple yet foundational: linear regression via ordinary least squares. While not particularly exciting, linear regression finds widespread use both as a standalone learning algorithm and as a building block in more advanced learning algorithms.
11 Jul 2018
https://www.oranlooney.com/post/craps-game-variants/
Craps is a suprisingly fair game. I remember calculating the probability of winning craps for the first time in an undergraduate discrete math class: I went back through my calculations several times, certain there was a mistake somewhere. How could it be closer than $\frac{1}{36}$?
(Spoiler Warning If you haven’t calculated these odds for yourself then you may want to do so before reading further.
12 May 2024
https://www.oranlooney.com/post/jeopardy/
How good are LLMs at trivia? I used the Jeopardy! dataset from Kaggle to benchmark ChatGPT and the new Llama 3 models. Here are the results:
There you go. You’ve already gotten 90% of what you’re going to get out of this article. Some guy on the internet ran a half-baked benchmark on a handful of LLM models, and the results were largely in line with popular benchmarks and received wisdom on fine-tuning and RAG.