Zigler: Zig NIFs for Elixir with Isaac Yonemoto

Elixir Wizards

In the Season 14 premiere, hosts Dan Ivovich and Sundi Myint chat with Isaac Yonemoto, creator of the Zigler library, to explore how Zigler brings Zig’s performance and safety to Elixir through Native Implemented Functions (NIFs). Isaac walks through the core design of Zigler and how it auto-generates the Elixir-to-Zig bridge, enforces type safety, and exposes multiple execution modes (normal, dirty, threaded). The conversation covers real-world applications, from SIMD-powered token selection for LLM hardware acceleration to OTP-style fault tolerance in low-level code.

Isaac shares his own journey: stepping back from professional software work to launch a biotech startup focused on reducing drug manufacturing costs while continuing to maintain Zigler and even leveraging Elixir for bioinformatics pipelines.

Topics discussed in this episode:

• What is the Zigler library and what does it do?
• What does it mean to run a "dirty NIF"?
• Async mode is temporarily removed from Zig (therefore, yielding NIFs is temporarily deprecated in Zigler)
• Zigler’s three execution modes (normal, dirty, and threaded) and how you switch modes with a single config change
• Isaac’s journey from professional software work to launching a biotech startup
• How Isaac leverages Elixir in bioinformatics pipelines at his startup
• LLM hardware acceleration using Zigler NIFs and SIMD-powered token picking
• Fault-tolerant load balancing of NIF workloads via OTP principles
• Transparent handling and recovery from hardware failures through monitoring
• Potential future memory-safety features in Zig and their implications
• The Elixir-based borrow-checker prototype: purpose and design
• Unit-checking for scientific computations to enforce correctness
• New OS support in Zigler 0.14: macOS, Windows, and FreeBSD
• Inline Zig code authoring directly within Elixir modules
• Isaac's commitment to maintain Zigler through its 1.0 release (...and beyond?)

Links mentioned:

https://github.com/E-xyza/zigler
https://github.com/ziglang/zig
https://vidalalabs.com/
Zig Programming Language: https://ziglang.org/
https://obsidian.md/
https://hexdocs.pm/elixir/macros.html
https://erlang.org/documentation/doc-4.7.3/doc/extensions/macros.html
A Deep Dive Into the Elixir AST: https://dorgan.ar/posts/2021/04/the_elixir_ast/
https://www.erlang.org/doc/system/nif.html
https://nodejs.org/en
Llama Open-Source LLM: https://www.llama.com/
Mixtral Open-Source LLM: https://mistral.ai/news/mixtral-of-experts
https://Fly.io
SIMD: https://en.wikipedia.org/wiki/Single_instruction,_multiple_data
https://opentrons.com/
CI/CD: https://en.wikipedia.org/wiki/CI/CD
https://hexdocs.pm/zigler/Zig.html
http://www.x.com/DNAutics
https://bsky.app/profile/dnautics.bsky.social