A collection of Deep Learning Koans written for Flux.jl and Julia Programming language.
A Koan is a short puzzle designed to elucidate understanding upon completion. The plural, "koans" are also a format of interactive programming puzzles designed to be run locally and to teach the user programming, through a series of challenges in order of increasing difficultly. This is exactly what this project is!
Thus, a koan critically consists of three components:
- A text introduction to a concept and problem
- Section of code that applies the concept
- A test that is not passed untill the concept is applied correctly
This format borrows a lot from the pedalogical approach of Daniel P. Friedman, and his book, The Little Schemer.
A list of koans projects for a variety of platforms and languages can be found here on github.
This project attempts to take the Koans approach to teaching programming, and apply that to the subject matter of deep learning in Julia.
To do this, we will focus first on giving a skippable overview of Julia, then move on to demonstrating the library Flux.jl.
Technically, this project is implemented as a series of IJulia Notebooks that you generate from source code, host locally with IJulia, the extension to run Julia in a Jupyter Notebook, then interactively modify the code until "it works".
Links and additional resources are scattered throughout the material.
First, Julia offer a couple advantages compared to Python and R that make it superior for some projects, namely, its gradual typing improves the developer experience and code reliability, and compilation to LLVM makes Julia code faster than R/Matlab/Python.
However, Julia is young, and does not have the depth and breadth of machine learning, and web development packages that make Python a safe production choice, or the depth of statistical packages and academic work that makes R so revered.
Nonetheless, Julia's LLVM compilcation offers huge advantage for Deep Learning in research and practice: we can actually program using a single language, instead of having to call an underlying C++ library, like Tensorflow or PyTorch. This requires a programming language, differential programming, which is currently being studied and integrated into Julia through the Zygote project
Flux is a Julia only, neural network library, with basic differential programing capability, with a library of optimizers, layers, and helper functions that can facilitate deep learning, and has first-class support for GPU allocation. Thus, solutions like Julia/Flux, that use a single language to compile to fast LLVM bytecode and leverage GPUs, are a good future for for Deep Learning, as they are conceptually simpler to understand than current solutions like PyTorch/TensorFlow with the same performance. However, and you'll see this very soon throgh the koans, the API maturity is not quite there!
The chapter outline is as follows:
- 1: Introduction to Julia
- 2: Working With Data In Julia
- 3: Intro to Flux
- 4: Convolutional Neural Networks and Layers in Flux
- 5: Recurrent Neural Networks and Layers in Flux
- 6: Flux optimization
- 7: Putting in all together, and more examples!
Many examples of these koans are borrowed and modified from sources in the Flux source code, or examples in the Flux Documentation. The Flux source code can be found here: Flux.jl.
To run this project, you will need Julia installed locally, which is available here. Additionally, you will need NVIDIA drivers installed on your machine, for the GPU based examples in Chapter 7.
Clone the Repository, or download directly from Github.
git clone https://github.com/adamwespiser/deep-learning-koans.git
Run julia in the deep-learning-koans directory
$ cd deep-learning-koans/
$ julia --project='.'
Instantiate the environment, which will download all the needed packages
julia> using Pkg; Pkg.instantiate();
Build the notebooks
julia> include("deps/build.jl")
Open the Julia Notebooks
julia> using IJulia
julia> notebook(dir=".")
The projects is built using the script found in deps/build.jl.
For pkg X, we can get the UUID with the following:
julia> Pkg.METADATA_compatible_uuid("X")
Then, add an entry to [deps] in Project.toml.
Finally, we will want to re-populate the Manifest.toml, which we can do as followings:
$ julia --project="."
julia> Pkg.resolve()
julia> Pkg.instantiate()
Run all the notebooks at once.
$ julia --project="." test/run_koan_source.jl