0.32.0: Reverse-mode AD is here!

This release has been a long time coming, and introduces some substantial improvements to Emmy. The top-line items are:

Reverse-mode automatic differentiation

Forward-mode AD is now much simpler to understand; the implementation lives in emmy.dual. emmy.tape and emmy.autodiff round out our AD implementations.

Our reverse-mode AD implementation is faster for everything in the library, but substantially faster for functions with many inputs and few outputs (like Lagrangians). See the new D-forward, D-reverse, partial-forward and partial-reverse for more control over which you like.

Native Complex and Fraction

@littleredcomputer implemented our Fraction.js and Complex.js dependencies in pure CLJS, letting us remove these JS dependencies. This opens the door for symbolic complex numbers and more fun down the road.

Value protocol => multimethods

The emmy.value/Value protocol is now gone, in favor of a "multimethods everywhere" approach. If you want speed you can compile your functions... otherwise simple is better, and this keeps us closer to the scmutils style.

Excellent docs on https://2.maria.cloud

Thanks to @mhuebert, SCI now properly grabs all docstrings and arglists from all Emmy functions, making for an excellent user experience.

Faster compilation

@littleredcomputer (in #143) replaced thee the implementation of common subexpression elimination with the algorithm described in Flajolet's 1990 paper (cited within). This speeds up (and makes possible at all) compilation of some of the big, nasty functions we ran into generating manifolds in examples like https://sritchie.github.io/strange-loop-2023/notebooks/stl/pq_knot

All CHANGELOG entries

• #170:

  • changes D and partial to use reverse mode automatic differentiation by default, and fixes all associated tests

  • adds emmy.generic/{zero?,one?,identity?} implementations (all false) to emmy.tape/Completed, in case some collection type tries to simplify these during reverse-mode AD

• #185:

  • adds a dynamic variable emmy.calculus.derivative/*mode* that allows the user to switch between forward and reverse mode automatic differentiation

  • adds a new emmy.calculus.derivative/gradient that acts like emmy.tape/gradient but is capable of taking multiple variables

  • adds new operators emmy.calculus.derivative/{D-forward, D-reverse} and operator-returning-functions emmy.calculus.derivative/{partial-forward, partial-reverse} that allow the user to explicitly invoke forward-mode or reverse-mode automatic differentiation. D and partial still default to forward-mode

  • modifies emmy.tape/gradient to correctly error when passed invalid selectors, just like emmy.dual/derivative.

• #183:

  • adds emmy.{autodiff, tape} to emmy.sci's exported namespace set

  • adds emmy.dual/extract-id implementations for all supported output types (every type that already implements emmy.dual/IPerturbed)

  • moves emmy.tape/Completed to emmy.dual/Completed; it doesn't really make sense there, but this is the only current way to remove the circular dependency between emmy.dual and emmy.tape. (tape needs a dual import to gain emmy.dual/IPerturbed.)

  • simplifies the emmy.tape/gradient implementation down to only handle single real-or-structural arguments, just like emmy.dual/derivative. We'll share the "handle-multiple-input" implementation between the two in a follow-up PR

  • makes the tests in emmy.calculus.derivative generic on the derivative implementation, so we can run all tests in forward and reverse mode.

• #182:

  • moves the generic implementations for TapeCell and Dual to emmy.autodiff

  • moves emmy.calculus.derivative to emmy.dual/derivative

  • removes emmy.dual/perturbed? from IPerturbed, as this is no longer used.

• #180 renames emmy.differential to emmy.dual, since the file now contains a proper dual number implementation, not a truncated multivariate power series.

• #179:

  • Moves the IPerturbed implementation for functions to emmy.function, out of emmy.calculus.derivative

  • Adds a new mode parameter to emmy.differential/extract-tangent, in preparation for allowing reverse and forward mode across all output types

• #175:

  • Adds emmy.env/{tau,-tau} constants for the $\tau$ fans out there

  • Adds ^:const metadata to all constants, reaping small performance wins

  • Updates emmy.numerical.unimin.brent/{brent-min,brent-max,brent-min-commons,brent-max-commons} to:

    • take a new :initial-guess argument, useful if you have some idea of where the minimum might lie

    • swaps the relative and absolute threshold defaults to match those from scmutils

    • adjusts the initial guess from the midpoint between a and b to a golden section cut (closer to a), to match scmutils

• #156:

  • Makes forward- and reverse-mode automatic differentiation compatible with each other, allowing for proper mixed-mode AD

  • Adds support for derivatives of literal functions in reverse-mode

• #165:

  • Fixes Alexey's Amazing Bug for our tape implementation

  • Adds the id field back into TapeCell, required for the tag replacement machinery for fixing Alexey's Bug

  • Adds an emmy.differential/IPerturbed implementation to TapeCell

  • Fixes some old documentation referencing :in->partials, plural

  • Updates emmy.tape/tag-of to return nil in case of a non-tape argument vs ##-Inf, again preparation for forward/reverse interactions

  • Adds support for partial derivatives via a selectors argument to gradient

  • Fixes a bug in the g/abs implementation for TapeCell, where (g/abs <tape>) would return a positive primal

  • Adds a simplify implementation for TapeCell

• #163:

  • replaces the emmy.differential.Differential generalized dual and its term list algebra with a simplified emmy.differential.Dual number type

    This new approach works because the emmy.differential/*active-tags* stack allows us to make sure that lifted binary operations always wrap their output in a new Dual with the tag assigned by the inner-most derivative call.

  • deletes emmy.util.vector-set and tests, as these are no longer used

  • adds a nil implementation for extract-tangent, meaning that nil-valued functions now work with D

• #159:

  • Fixes Differential's implementation of emmy.value/numerical? to always return false. The reason is that numerical? is used by g/* and friends to decide on simplifications like (* <dx-with-1> x) => x, which would lose the structure of dx-with-1. By returning false we avoid these simplifications.

  • Converts a number of emmy.value/numerical? calls to emmy.value/scalar?. The numerical? protocol method is used only in generic functions like g/* for deciding whether or not to apply numerical simplifications, like (* x 1) => x.

    Guarding on v/scalar? instead allows us to let in numbers, tapes and differentials, since these latter two are meant to WRAP numbers, but should not be subject to numerical simplifications.

  • Adds emmy.structure/fold-chain for performing a tree-like fold on structures, saving us work over the alternate pattern of s/mapr, flatten and reduce.

• #155

  • Replace the implementation of arbitrary-precision rational arithmetic provided by fraction.js with Clojure code, allowing us to remove a JavaScript dependency.

  • We note here a subtlety with fraction reader syntax. When the CLJS compiler runs and encounters input like #emmy/ratio 1/2, since the compilation occurs in a JVM Clojure environment the 1/2 will deserialize as a Ratio, which is then transformed by the reader into code which will generate a ClojureScript Fraction. The ClojureScript reader, however, evaluates the input as q JS expression resulting in 0.5, which is then rationalized with an expensive algorithm which may lose exactness because of the unwanted floating-point conversion. For this reason, when we emit a ratio in string form, we quote the arguments: #emmy/ratio "1/2" to prevent the conversion. Consider this a deprecation notice for the unquoted form. We also now allow the form #emmy/ratio [1 2], like Complex, and now no longer allow an initial + on the numerator.

• #154:

  • Adds emmy.tape with an implementation of reverse-mode automatic differentiation. The implementation is based on Alexey Radul's implementation in dvl, and seems to be higher-performance by quite a bit and capable of replacing our forward-mode implementation.

    • The centerpiece of the implementation is emmy.tape/gradient, which can handle $R^n \to R^m$ functions, as well as nested derivatives.

    • All operations supported by [[emmy.differential/Differential]] are supported by the backing [[emmy.tape/TapeCell]] instance.

• #151

  • By porting complex.js to Clojure, we can remove the dependency on this library on the JavaScript side as well as the Apache Commons complex implementation on the JVM side.

    The JavaScript implementation is followed fairly closely and done with generic Emmy arithmetic at the component level (except when that is clearly unnecessary). The (complex) constructor has been made equivalent to the reader parser.

    The former implementation made a special case of i^r, raising the complex unit to a real power, but it only worked for integral r, and threw an exception in other cases; this special case is removed.

• #149

  • Retires the Value protocol in favor of MultiFns in the generic scope. Doing this carefully, by revoking the existing implementation and then restoring it step by step, revealed some interesting corner cases:

    • (zero? [0]) but (not (zero? (lazy-seq [0])))
      ...

0.31.0: emmy.js support, TeX improvements

• #139:

  • Fixes matrix and simplification-related test failures introduced in the last couple of PRs.

  • Removes the custom ->Quaternion constructor in emmy.quaternion; this was causing build warnings in cljs. Any place you might have used ->Quaternion, please use emmy.quaternion/make.

  • Backs off the various protocol extensions in emmy.collection from IPersistentMap and IPersistentSet to the concrete types shipped with Clojure. This prevents these implementations from attaching, on the JVM side, to new defrecord instances or other types that extend those protocols.

  • Fixes a documentation error with emmy.generic/atan.

• #135:

  • Adds some hidden features for use by emmy.js to support use of Emmy from JavaScript. These features are not used by default, but are meant to be opted-in by JS libraries that need them.

  • The function object now uses the field f-name instead of name to hold the name of the function. The name property of Function objects in JS is readonly, so we need a different property that we can modify in order to create a JS object which is both an Emmy and an ES6 function.

  • Adds the function make-es6-callable to promote an IFn object to a native JS function by birthing a new function object which delegates to the original object's apply method, and copying over the rest of the object's identity.

  • Adds the function make-es6-indexable to promote an Emmy Structure to something that behaves more like an ES6 array.

  • Structure objects are declared es6-iterable using a standard CLJS feature.

  • Using with-meta on a function creates a new object which is not a native JS function, though still invokable in Clojure. This is a steep price to pay for the privilege of adding metadata to a function, so for function objects we allow metadata to be directly applied.

    The IMeta interface can be used to retrieve this property-based metadata, but IWithMeta cannot be used to attach it (this interface guarantees the production of a new object). See emmy.function/with-meta.

• #134:

  • Adds ->TeX handlers for matrix-by-rows and column-matrix.

  • matrices now freeze into (matrix-by-rows [...] [...]) instead of (matrix-by-rows (up ...) (up ...)).

  • functions in emmy.numerical.ode now set :compile? to true by default.

  • Fixes some TeX typos in a few of the literate notebooks.

• #131 fixes accidental simplification of keywords to nil.

0.30.0: first Emmy release!

This release includes big performance upgrades to the emmy.numerical.ode and emmy.expression.compile namespaces. These features anticipate lots of lovely visualization features that'll be coming soon.

To port from sicmutils, simply rename sicmutils to emmy everywhere. (If you were using pattern.*, you'll need to add an emmy prefix and use emmy.pattern.match etc instead.)

We also have a new website at https://emmy.mentat.org, and a new project template for getting started with Emmy and Clerk.

Changelog entries:

• #126:

  • Adds a deps-new template that sets up a basic Clerk project with Emmy installed as a dependency. The template lives here.

  • Adds dev/emmy/notebook.clj as the new homepage for https://emmy.mentat.org.

• #98 upgrades same/ish to 0.1.6, renames all same requires to same.core and removes final :include-macros true from the codebase.

• #124:

  • Upgrades babashka/sci to 0.7.39.

  • Adds a working bb repl command to the repository.

  • Renames emmy.env.sci => emmy.sci, and adds an emmy.sci/install! command to make it easier to install all namespaces into a shared SCI context.

  • Renames context-opts in the new emmy.sci to emmy.sci/config, to match the sci-configs style.

  The full library is now published to https://emmy.mentat.org as a series of Clerk notebooks. This required a dependency on https://clerk-utils.mentat.org, but no actual Clerk dependency in the library.

• #119:

  • Removes support for :flat compilation mode (this was a step toward what is now called :primitive). Flat has no further use case.

  • You can now "bring your own array" to the function provided by stream-integrator, avoiding an allocation in performance-sensitive contexts.

• #118:

  • Fixes problems with the resolve calls in emmy.util.def triggered by vanilla (non-shadow) cljs compilation.

  • Updates Complex.js and Fraction.js dependencies to their latest NPM versions.

  • Updates odex to the latest release candidate on NPM.

  • The cljs side of emmy.numerical.ode/stream-integrator gains a :js? option that defaults to true. If false, the returned function will return native JS arrays vs converting the return value to Clojure.

  • In emmy.expression.compile, compile-fn and compile-state-fn gain a :simplify? argument that defaults to true. If false, compilation will not attempt to simplify the function body before proceeding.

• #115:

  This PR introduces significant upgrades to the functions in emmy.expression.compile. compile-state-fn and compile-fn now share the same code. Expect some more shifts here as we work on animations.

  Specifically, this update:

  • Removes timers from the code in emmy.numerical.ode. If you want timing data you can generate an derivative yourself that performs timing. Including this by default introduced a significant performance cost.

  • Renames emmy.numerical.ode/integration-opts to make-integrator*; it now returns only a call to stream-integrator instead of the old map of this result and timers.

  • In emmy.expression.compile:

    • compile-state-fn* and compile-fn* are now removed, in favor of the non-starred versions with option :cache? false supplied.

    • compile-fn now calls compile-state-fn with params set to false and an :arity option supplied, vs using its own mostly-duplicated implementation.

    • :flatten? option removed in favor of the more granular :calling-convention. This option supports values of :structure, :flat and :primitive. See the docstring for details.

    • :mode supports :js, :source, :clj, :native or :sci.

• #110:

  • Moves all docstrings that existed as metadata on defs (i.e., (def ^{:doc "..."} sym ...)) down below the symbol. I hadn't realized that this was a valid way to attach a docstring!

  • Upgrades GitHub Actions clj-kondo invocation to version 2013.01.20 and saves some work in the actions setup. Fix all linting errors that resulted.

• #109:

  • ->JavaScript now produces expressions, and not function bodies. This change makes ->JavaScript do the same job as ->infix and ->TeX. Initially, the JS rendering emitted a function in order to facilitate experiments with embedding equations of motion in dynamic web pages. This can still be done: compile-state-fn has been extended to allow the compilation of a state function into either Clojure or Javascript notation. The test directory contains many worked examples.

  • numerous local gensym replacements found useful as part of test fixtures have been gathered together into monotonic-symbol-generator.

  • generation of sums and differences in all of the infix generators has been improved to more closely approach standard mathematical notation (e.g., instead of -2 * x + -2 * y you will see - 2 * x - 2 * y).

• #107:

  • move CSE to its own namespace to avoid the circular dependency compile->render->compile

  • refactor JS rendering to allow compiler to use it

  • adjust meaning of :native and :source compilation modes: now you get what's compatible with your execution environment. You can also ask for a specific language, allowing tests to be bilingual.

• #100:

  • Implements predicate support for segment, entire-segment and reverse-segment in emmy.pattern.match. This support bubbles up to forms in rules like (?? x pred1 pred2).

  • Removes the :emmy.pattern/ignored-restriction linter keyword, and all clj-kondo code warning that restrictions aren't supported on segment binding forms.

• #96 renames #sicm/{bigint, quaternion, complex, ratio} to #emmy/{bigint, quaternion, complex, ratio}.

• #1, #2:

  • Renames the sicmutils portion of all namespaces to emmy.

  • Moves the pattern.* namespaces into emmy.pattern.*

  • Ran cljfmt on all code to tidy up hidden tabs, extra whitespace, etc

  • Attempted to switch over all badges and links to the proper new locations... we'll see how this went, and I'll fix anything I missed as it comes up.