Elijah Rippeth

Elijah Rippeth

Bad at computering

Blog posts

2022

MTMA22 Day 5: Continuous Integration and Finalizing Packaging

less than 1 minute read

Published:

Day 5 is the final day of MTMA and we were prepping for our end-of-day demo. We largely had everything in place that we wanted, but are looking forwards to standardizing the build, packaging, and distribution of pymarian. I spent a lot of day fixing CI by making sure the right cmake flags were set and setting up python environments as appropriate. Meanwhile I learned that twine cannot upload arbitrary .whl files built for linux. We need to repair with auditwheel which is no big deal. Our CI matrix will eventually need to be something like python_versions X {cpu,gpu} X cuda_versions, but it will be worth it for consumers who no longer need to compile marian from source and can use it on all major platforms.

MTMA22 Day 4: Benchmarking and Applications

4 minute read

Published:

Now that the package is mostly solid, I spent the day benchmarking the code. I previously developed a small microbenchmark to test native vs. pybind speeds to ensure there are no major regressions, but I wanted to check against other existing libraries. Because we’re using Helsinki NLP’s OPUS MT models to test the Marian inference, we thought we would use the HuggingFace Transformer’s exported checkpoints of the same models. I’m not exactly sure how the checkpoints were made compatible (likely model surgery like how HF exports fairseq checkpoints), but it mostly doesn’t matter.

MTMA22 Day 3: Packaging

less than 1 minute read

Published:

With the code looking better and the build system improved, day 3 is primarily spent working to prepare pymarian for packaging and automated builds as a first class citizen in Marian CI. I was unavailable for most of the day due to internship obligations and some wedding planning, so I don’t have much to report. Marcin seems to have gotten pymarian building on Windows with some minor cmake magic. Meanwhile Alex checked in some of the initial Huggingface-like API’s and Liling has been fighting with CI. That’s really it for today. :-)

MTMA22 Day 2: Hardening the API

3 minute read

Published:

After yesterday’s productive day of fighting with setuptools and cmake we were able to get a kind-of-awkward-to-use pymarian translator working. Our outstanding TODO’s were to test the speed (to ensure we aren’t incurring terrible latency by adding bindings) and to clean up some of the API’s.

NMT with xFormers: Part 2

7 minute read

Published:

In the last post I explored subword tokenization, data prep, and the training loop for a transformer-based NMT model. While I had promised the next post would begin implementing decoding strategies, I wanted to explore efficiency gains to be had by digging into batch samplers. I’ll start by describing the problem we’re trying to solve.

NMT with xFormers: Part 1

8 minute read

Published:

In this post I explore the APIs of the wonderful new xFormers library while revisiting torchtext which has undergone a great deal of maturation since I last used it. This serves as an exploration and hopefully some motivation to develop tooling around the ideas explored here in hopes of developing a principled neural machine translation library that leverages optimized tooling under the hood. Let’s jump in.

2021

Kantorovich’s Inequality

2 minute read

Published:

Kantorovich’s inequality is an inequality used in convergence analysis of the method of steepest descent. Specifically, we concern ourselves with the optimization problem